object counting in high resolution remote sensing images with otb

IGARSS 2009, Cape Town

Motivation Original Simplified Conclusion

Object counting in high resolution remotesensing images with OTB

E. Christophe1, J. Inglada2

1CENTRE FOR REMOTE IMAGING, SENSING AND PROCESSING,NATIONAL UNIVERSITY OF SINGAPORE

2CENTRE NATIONAL D’ÉTUDES SPATIALES, TOULOUSE, FRANCE

http://www.cnes.fr

http://www.crisp.nus.edu.sg/

http://www.orfeo-toolbox.org



Outline

Motivation

Original solutionWorkflowPan SharpeningClassificationSegmentation: Mean shiftVector data

Simplified versionsTrade-offDescriptionResults

Conclusion

http://www.cnes.fr





Motivation

Object counting

I Correspond to a wide range of problems for remotesensing data users

I Often have to be performed on large areaI Time consuming

Examples

I Houses in a city: particularly for country where urbanplanning data is not available

I Tents in refugee campI Tree stands in a field

http://www.cnes.fr





PRRS 2008 algorithm performance contest

I Time constraints: can’t spend months refining thealgorithm

I Deliver a result: whole processing chain requiredI Each step can be improvedI Goal: illustrate the on the shelf approach

ContestI Count the building from a Quickbird scene over Legaspi,

PhilippinesI XS and Pan images were provided

Aksoy et al., ”Performance evaluation of building detection and digital surface model extraction algorithms:Outcomes of the PRRS 2008 algorithm performance contest,” in 5th IAPR Workshop on Pattern Recognitionin Remote Sensing, Tampa, Florida, Dec. 2008.

http://www.cnes.fr




Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data

Outline

Motivation



Conclusion

http://www.cnes.fr





Algorithm description

Pan

Mul

Pan sharpening

Classification

Segmentation Vectorization

Edge dectect.

Refinement Obj

http://www.cnes.fr






Pan

Mul

Pan sharpening

Classification

Segmentation

Vectorization

Edge dectect.

Refinement Obj

http://www.cnes.fr






Pan

Mul

Pan sharpening

Classification


Edge dectect.

Refinement Obj

http://www.cnes.fr






Pan

Mul

Pan sharpening

Classification


Edge dectect.

Refinement

Obj

http://www.cnes.fr






Pan

Mul

Pan sharpening

Classification


Edge dectect.

Refinement Obj

http://www.cnes.fr





Preprocessing

Pansharpening

I The pansharpening is the first step to perform to takeadvantage of the high resolution of the Panchromatic band(61 cm) with the four spectral bands of the multispectral.

Pan Mul Pan Shapening

http://www.cnes.fr





Classification

Obvious sources of errorsI There is some obvious sources of error: boat in the middle of the water which

look like houses, cars in the middle of the streetI Classification (used as a mask) can help remove these sources of error

ClassificationI Here we used a simple classification by SVMI Non classification specialists just provided a few samples per class (water,

vegetation, road, shadows, 4 colors of buildings)I Only a pixel classification: no use of texture here (that was before all the textures

were introduces in OTB)

http://www.cnes.fr





QB scene Land cover classification

http://www.cnes.fr





Segmentation: Mean shift

Too much detailsI Higher resolution is better but. . . sometimes, you would like

less details (roof superstructures, cars)I What details to remove?I Mean shift algorithm

D. Commaniciu,“Mean shift: A robust approach toward feature space analysis,”IEEE Transactions on PatternAnalysis and Machine Intelligence, vol. 24, no. 5, pp. 603–619, May 2002.

http://www.cnes.fr





Pan Sharpened Mean shift clustering

http://www.cnes.fr





Refining the boundaries

Simplification

I Easier to handle vector data than raster: vectorizationI The vectorization led to too many contour point:

simplification of the points which are roughly aligned

Fine adjustment

I Using an image contour as an input, an energy iscomputed along the polygon contour

I Introducing a random perturbation in the position of eachpoint, the energy is maximized

I Only a very basic optimization used here (ground forimprovement)

http://www.cnes.fr





Filtering

Filtering on compacity

I Buildings are usually compactI C = 4π A

L2

http://www.cnes.fr





Performances on the contest

ResultsI Two results were submitted with a difference mainly in the classificationI One result was very close with 3600 building detected for 3065 in the ground

truthI Interesting to see that most other algorithms tend to underdetection while the

proposed algorithm tends to overdetect.

Evaluation criteriaI {Correct, over, under, missed} detection and false alarm rates based on

Overlapping Area MatrixI Maximum-weight bipartite graph matchingI Normalized Hamming distanceI Clustering indices (Fowlkes-Mallows and Jaccard index)

http://www.cnes.fr





Performances on the contest

Conclusion on these resultsI Particularly hard to conclude given the wide variety of

criteria: organizer of the contest have been careful not todeclare an overall winner

I However, the proposed methods provided goodperformances (particularly on the clusterings indicescriteria) with a bias towards over segmentation.

http://www.cnes.fr




Motivation Original Simplified Conclusion Trade-off Description Results

Outline

Motivation



Conclusion

http://www.cnes.fr





Trade-off of the previous method

DrawbacksThe previous method relies on the classification of the image

I require a good understanding of the algorithm that followI influence significantly the output

Simplified version

I different trade-offs on complexity-performanceI remove the classification stepI just require the operator to click on several (2 to 5)

examples of objects

http://www.cnes.fr





Simplified version

Algorithm

I Produce a likelihood map of the region containing theobjects of interest

I Followed by the same step as the previous algorithm:segmentation, vectorization,. . .

Likelihood map: 2 choices

I Spectral angleI One class SVM

http://www.cnes.fr






Pan

Mul

Pan sharpening

Spectral angle

One class SVM


Edge dectect.

Refinement Obj

http://www.cnes.fr





Results of the simplified version

Poorer preformances

I Not as good as the original one (expected)⇒ required tounderstand the algorithm

AdviceI Spectral angle: spectral characteristics of the objects are

stableI SVM: better when object have radiometric differences but

more samples are required

http://www.cnes.fr





Outline

Motivation



Conclusion

http://www.cnes.fr





Conclusion

Modular processing chain

I An application with GUI isavailable

I It can be used for processingremote sensing images (noconstraint on the size)

I Can be easily modified andimproved as the steps aremodular and follow the pipelinephilosophy.

http://www.cnes.fr



object counting in high resolution remote sensing images with otb

Technology