Dec. 2006 J. China Univ. of Mining & Tech. (English Edition) Vo1.16 No.4

1

Automatic Road Extraction Using Particle Filters from High Resolution Images

YE Fa-mao"2, SU Lin2, TANG Jiang-long3 'Department of Computer Science and Engineering, Nanchang University, Nanchang, Jiangxi 330029, China

'Institute of Remote Sensing Applications, Chinese Academy of Sciences, Beijing 100101, China 3East China Institute of Technology, Fuzhou, Jiangxi 344000, China

Abstract: Road extraction plays an important role in many applications such as car navigation, but the manual extraction of roads is a laborious, tedious task. To speed the extraction of roads, an approach based on particle filtering to extract automatically roads from high resolution imagery is proposed. Particle filtering provides a statistical framework for propagating sample-based approximations of posterior distributions and has almost no restriction on the ingredients of the model. We integrate the similarity of grey value and the edge point distribution of roads into particle filtering to deal with complex scenes. To handle road appearance changes the tracking algorithm is allowed to update the road model during temporally stable image observations. A fully automatic initialization strategy is used. Experimental results show that the proposed approach is a promising and fully automatic method for extracting roads from images, even in the presence of occlusions. Key words: road extraction; particle filters; high resolution images CLC number: TW53

Introduction

Automatic road extraction from digital imagery has been an active research area in computer vision, photogrammetry, remote sensing and geographic information system (GIS) communities for over two decades. A number of semiautomatic and automatic methods and algorithms for road extraction have been developed. Those approaches ma involve template matching"-21, heuristic reasoningz4], dynamic pro- gram~nming'~~] or stochastic

Our approach to road extraction using particle filtering belongs to the latter family. Particle filtering, based on sequential importance samplinghesampling, provides a statistical framework for propagating sample-based approximations of posterior distribu- tions, with almost no restriction on the ingredients of the modelr8]. We integrate the similarity of grey value and the edge point distribution of roads into particle filtering. The grey value of road surfaces is a very characteristic property of a road but it is sensitive to the disturbing influence of cars, trees, or other objects. However the edge point distribution is robust to partial occlusions. The road looks vary, caused by different brightness within the image or by changes in the surface material. To handle these appearance changes, the tracking algorithm is allowed to update the road model during temporally stable image observations. Road extraction using particle filtering needs to be initialized for each road extracted. In

order to solve the initialization problem, our method uses a simple segmentation method, together with the Hough Transform and gradient operators.

Our paper is organized as follows. In Section 2, a road extraction framework using particle filtering is given, while the model update is described in Section 3. Automatic extraction of roads and experimental results are given in Section 4 and in Section 5 conclusions are drawn.

Received 01 March 2006; accepted 04 April 2006 Corresponding author. Tel: +86-13870874919; E-mail address: yfm-email@ IZ!6.com

2 Roads Extraction Framework by Partic- le Filtering

Particle filtering is a recursive procedure to estimate the parameters of a dynamic system and has found many applications in object tracking"'. Road extraction using particle filtering can be understood by an analogy with manual tracing. It starts from a point xo on the centerline of the road and then moves along the centerline. Give the current position at time t denoted by the statex,, the growing contour is represented by an ordered sequence X,, = ( X, , . , X, ) . The observed image is represented by the vector Y,., =q, ... , y ) . To grow such a sequence, particle filtering manipulates the filtering density, p(x ,+ , I Y ~ ) , to estimate the most probable next position X,,, according to different decision criteria. The filtering density is represented by a set of random samples with associated weights. Let { x:, i = 0, ..- , N, }

YE Fa-ma0 et al Automatic Road Extraction Using Particle Filters from High Resolution Images 49 1

denote a set of samples or particles that characterizes the filtering density and let { 4, i = 0, -.. , N , } denote the associated weights. The weights are normalized such t h a t 2 . N = I . Then, the filtering density at t can be apprbximated as

I d

where d i s the delta function. We therefore have a discretely weighted approximation to the true filtering density p ( X , I Y,:,) . The weights are chosen using the principle of importance sampling [’, 31 to reduce the variance of the weights; so, if the samples,xi were drawn from an importance sampling density, q ( X , I Yk.), then the weights in (1) are

The term p ( X , I X,-l)corTesponds to the prior on the road and p(Y, I X,) to the likelihood. The prior retains expected propetties of the roads to be extracted and the likelihood provides evidence about whether a measurement is, or is not, in the vicinity of the “true” road. It is often convenient to choose the importance sampling density to be the prior[

(3)

(4) This would Seem to be the most common choice

of importance sampling density since it is intuitive and simple to implement. To avoid a skewed weight distribution, which leads to degeneracy of the particles, a selection step is added. The basic idea of selection is to eliminate particles which have small weights and to concentrate on particles with large weights. The final outline of particle filtering is summarized in Fig. 1.

10-1 11 . 4 ( X f I X,,,Y,) = P ( X , I X,-l)

Substitution of (3) into (2) then yields

q OC P(Y, I x, 1

1. Initialization ( t = 1 )

4 = x 0 i=l;.. 2JS b= ...... =@ = I / N ,

2. Sampling step

a) For i = l , . . . , N , Sample x: from the importance

sampling density q(x; I x,-, , y, )

@-p(y,Ix;), i = l , . . . , N ,

4 =41CZ,ciy, i = 1 , . . . , ~ ,

b) Evaluate the new weights

c) Normalize the weights

3. Output step

4. Selectionstep

- x, =.?y(r)=argmp(x; ly,)

b:,@ I = R E s A M P ~ ~ x : , @ l I 5. t = t + l , g o t o s t e p 2

Fig. 1 Outline of particle filter

2.1 Prior

In the case of road tracing, the parameters to be estimated are the parameters that describe the position and shape of the road. In each position along the road, the row and column coordinates of the road centerline position (q,c,) , the road direction (0,) and the road width(w,) are estimated. These four parameters constitute the state vector (X,) .

In the time update the state vector at time t is predicted by the following equation using the state vector at time ( t - 1).

where d i s the fixed step length, t$-l the moving direction estimated at time (r- 1) and U, the direction change.

The direction of roads will not change sharply except at the junctions. So we specify the prior probability distribution on the smoothness of the road. The smoothness can be simply controlled by choosing the distribution of U, as a Gaussian distribution with vahiancecr’ . In order to dlaw for abrupt direction changes when a junction has been detected, we chose this distribution as a uniform

distribution over [-- -1 at the junction. x x 2’ 2

The prior finally reads:

2.2 Likelihood We designed our observation model based on the

following considerations. In high-resolution images, roads appear as elongated regions with parallel borders and have distinctive brightness patterns compared to their surroundings. The width and the brightness of roads will not change quickly. And hence, our observation model Yt consists of two components. If we let T denote the rectangular road template and S denote a sample rectangular window, then the first component is the gray similarity model, G, which is intended to measure the degree of gray similarity between T and S. The second component is an edge point distribution model, E, which characteri- zes the similarity between the edge point distribution of Tand S.

Our gray similarity measure between T and S is a normalized cross correlation coefficient defined as:

J. China Univ. of Mining & Tech. (English Edition) Vo1.16 No.4 492

We take the measure

(8) as the distance between T and S. The G observation probability of each sample

d2 1-c

(9)

is specified by a Gaussian distribution with variance

1 -- 1 e-2": pc = q - e '4 =- &;a,

o* *

The edge point distribution pa ={p~'}Y=l,...,m of a rectangular window R at location Q is calculated as

c f ( P 9 c ) m W - U l (10) *p'=- 1

fr,ckR

where

f (r, 4 =

h(r) assigns one of the rn -bins of the histogram to a given coordinate at location (P, c) . k is the total number of the pixels in R . Fig. 2d shows a histogram of edge points of the selected rectangular window in Fig. 2a.

1 if the pixel at location (r,c) is an edge point i 0 otherwise

(a) Original image (b) Zoomed part of marked a m (c) Canny edge detected image (d) Histogram of edge points

Fig. 2 Histogram of edge points of the selected rectangular window

We use the Bhattacharyya coefficient [12], which is a popular measure between two distributions, to measure the similarity between the edge point or too noisy. So we use the update rule distributions of T and S . Considering discrete densi-

outliers, the observation model can be protected against updating the model when the road is blocked

c, >c, (15) Xf ties PT = (P;' )u+..,m and PS = { ~?)Iu=l,....m the co-

efficient is defined as where C2, is the cross correlation coefficient in terms

of the state it and C, is athreshold. (11)

As distance between two distributions we define the measure

d=J- (12) Which is Called the BhamChVYa diStance- The E observation probability of each sample

The update of the template T is implemented by u=l the equation

I; = (1 - ~ ) l ; - , + Q T ~ , (16)

where a weighs the contribution of Tf , . Thus, we evoke a forgetting process in the sense that the contribution of a specific template decreases expon- entially the further it lies in the past.

dz 1-P( P I . Ps ) 1 -- 4 Automatic Extraction of Roads 1 e*=- e 'a: (13)

--

PE=2zE &o- Road extraction using particle filtering needs to

be initialized for each roBd extracted. In other words, is specified by a Gaussian distribution with variance 0,. it needs a starting point. To solve the initialization

problem, we use a method based on the Hough transform and on some road properties to find the initial seeds. The main idea is the following: the Hough transform could not be used in the whole image, due to the fact that roads are not perfectly straight lines. But it should be noted that streets are

3 Modelupdate composed by straight segments that could be extracted by the Hough transform by analyzing only

To Overcome these aPparance changes Of the small neighbourhoods of pixels t41. So we can use the r ' a d 9 we update the Observation Hough transform to look for parallel straight lines in changing image o b ~ ~ a t i o n s . BY discarding image s d l subimages and then some road properties to

Finally we obtain the likelihood - (1-f : 1-PI )

24 24 (14) e 1

P<y, I X , ) = PGPE = 21sa,ae

during

YE Fa-ma0 et al Automatic Road Extraction Using Particle Filters from High Resolution Images 493

verify whether the parallel straight lines are the road sides or not. The road properties are as follow:

1) The distance of the parallel straight lines must not exceed a certain threshold corresponding to the maximum width of the roads.

2) These lines have a high gradient and uniform colour.

The parallel straight lines that have proved to be the road sides are then used for initializing the particle filtering algorithm. The initial point can be the points in the middle of the two lines and the initial direction can be the direction of the lines. At the same time, we take the road width according to the distance of the parallel straight lines.

We detect the junctions of the road by the following criteria:

(17)

where Ci, and pi, are the cross correlation coefficients and the coefficient of the edge point distribution of the state it , C, and pT are thresholds,

An example of the results obtained from the road extraction process is illustrated in Fig. 3, in

ci, ' 'T and P$, ' PT

which one can notice that the task is difficult, due to shadows and car occlusions.

Fig. 3 Extracted roads

5 Conclusion

In this paper a fully automatic approach to road extraction from high resolution images has been proposed. Through integrating the similarity of grey vdues and the edge point distribution of roads into particle filtering and updating the road model during temporally stable image observations, the proposed approach can extract roads from images, even in the presence of occlusions.

References

Shukla V, Chandrakanth R, Ramachandran R. Semi-automatic road extraction algorithm for high resolution images using path following approach. In: ICVGIPO2. Ahmadabad, 2002: 23 1-236. Park S R, Kim T. Semi-automatic road extraction algorithm from IKONOS images using template matching. In: 22nd Asian Conference on Remote Sensing. Singapore, 2001: 1209-1213. Steger C. Extracting curvilinear structures: a differential geometric approach. In: Proc of European Conference on Computer Vision. New York Springer-Verlag, 1996 630441. zlotnick A, Camine P D. Finding road seeds in aerial images. CVGZP: Image Understanding, 1993,57(2): 243-260. Gruen A, Li H. Road extraction from aerial and satellite images by dynamic programming. ISPRS Journal of Photogrammctry and Remote Sensing, 1995,50(4): 11-12. Gruen A, Li H. Semi-automatic linear feature extraction by dynamic programming and LSB-snake. PhotogMnune?ric Engineering & Remote Sensing, 1997,63(8): 985-995. Vosselman G, De K J. Road tracing by profile matching and Kalman filtering. In: Proceedings Workshop on Automatic Extraction of Man-Made Objectsfiom Aerial and Space Images, Switzerland: Birkhlluser Verlag, 1995: 265-274. Perez P, Blake A, Gangnet M. Jetstream: probabilistic contour extraction with particles. In: Pro of Int Con on Compitfer Vision. Vancouver, 2001: 524-531. Isard M, Blake A. Condensation-conditional density propagation for visual tracking. International Journal on Computer Vision, 1998, 1: 5-28. Arulampalam M S, Maskell S, Gordon N, et al. A tutorial on particle filters for online nonlinearhon-Gaussian Bayesian tracking. IEEE Transactions Signal Processing, 2002,50(2): 174-188. Doucet A, De F N, Gordon N. Sequential Monte Carlo Methods in Practice. New Yo& Springer-Verlag, 2001, Aheme F, "hacker N, Rockett P. The Bhattacharyya metric as an absolute similarity measure for frequency coded data. Kybenzetika. 1997,334): 1-7.