estimating surface orientation from directional textures

Pattern Recognition Letters 14 (1993) 431-439

May 1993North-Holland

PATREC 1056

Estimating surface orientation from directionaltextures

Richard BuseCentre for Signal Proecssing Research, Queensland University - Technology, Queensland, Ausiralut

Hung-Tat TsuiDeportment of Electronic Engineering . The Chinese University of Hong Kong, Shafin . Hong Kong

Received 22 May 1992

Al)stract

Buse . R . and H.-T . Tsui. Estimating surface orientation from directional textures, Pattern Recognition Letters [4(1993)431 - 439

Witkin (1981) described a maximum likelihood method of shape from texture using tangent vectors of textu-e line segments,and assumed directional isotropy for these vectors . However, natural textures are often non-isotropic with a detectabledirection . In this paper we propose a maximum likelihood method for recovering surface orientation from surfaces containingmm-isotropic textures . Our method is an extension of Witkin's ML method and includes a priori information on the texturebias . Experiments using artificial and real data show that our method is far better than Witkin's ML method for thesenon-isotropic textures .

Keri'ords Shape from texture, non-isotropic textures .

1. Introduction

Using texture to perceive surface shape was firstproposed by Gibson [10] . The idea was to assumethe texture density on a plane to be uniform anduse the density gradient as a means for surface per-ception by human beings . Since then, this line orvariations of it have been followed in a number ofpapers on computational vision [1-5,7,12-14,16] .Wit kin [19] proposed another concept of recoveringsurface shape and orientation from texture basedon the directional distortion of the line segmentson a surface . As opposed to the assumption of spa-

Correspondence to: Dr. H .T . Tsui, Dept . of Electronic Engineer-ing, The Chinese University of Hong Kong . Shatin, New Terri-tories . Hong Kong.

0167-8655/93/$06 .(10 : 1993 - Elsevier Science Publishers B .V . All rights reserved

tial homogeneity of Gibson, directional isotropyfor the line segments was assumed . Witkin alsopointed out that his assumption is more practicalfor textures of natural scenes . Any distortion in thetexture shape (composed of line segments) was as-sumed solely due to the projection of the texture .It is obvious that this method will fail badly if theassumption of directional isotropy is not valid . Anumber of papers have extended Witkin's MI, meth-od . Davis et al . [8] improved the accuracy of thealgorithm and its computational time, GottesfeldBrown and Ibrehim [11] suggested a parallel im-plementation, and Garding [9] developed a lessrestrictive closed form solution . But no one has ad-dressed the case when the texture is highly direc-tional .

As opposed the texture patterns on man-made

431

Volume 14, Number 5

objects, many natural texture patterns have a di-rectional bias or a non-uniform texture density .The texture of the bark of a tree, the scene of abamboo forest, straw, grassland, cloth are someexamples of texture with directional bias . Euro-pean marble, beach pebbles, houses of city suburbsviewed from the air are some examples of non-uniform texture density . This points to two issuesfor solving the problem of orientation estimatingof surfaces with non-uniform texture gradient and/orwith texture having directional bias . The first iswhether we can detect the non-homogeneity of atexture scene . The second is whether we can makeuse of the detected non-homogeneity to improveour orientation estimation . For the first issue, itseems reasonable to assume that non-homogeneityin texture can be detected easily in most cases .However, it is not obvious how we can make useof the information provided by the detected non-homogeneity. The contribution of this paper is toprovide a method to tackle this problem for a wideclass of texture patterns .

For texture patterns which can be characterisedby line segments, our method will make use of thea priori knowledge or computed information ofthe directional bias of the tangent angles derivedfrom the line segments of the texture . Like Witkin'sML method, orthographic projection is assumedas the target application is for natural scenes at adistance . In Section 2, a brief description of theproblem will be given followed by a detailed ac-count of the method we propose . Section 3 showsthe results of experiments using both our methodand Witkin's ML method as a comparison . Bothreal data and synthetic data with random noise

432

ImagePlane

X'

Figure I . Representation of slant and tilt angles .

Y

PATTERN RECOGNITION LETTERS

Direction of Tilt

May 1993

(a)

(b)

(ciFigure 2 . Effect of foreshortening on a circle, tilt = 45°, (a)

slant = 0°, (b) slant = 60°, (c) slant = 75° .

added were used . Some discussions and concludingremarks are given in Section 4 .

2. Estimating surface orientation

The surface orientation can be described by twoangles-slant (a) and tilt (r) as shown in Figure 1 .At a given viewpoint, only two degrees of freedomare required to uniquely describe the surface orien-tation [10,17-18] . The slant angle is defined as theangle between the surface and image planes, whilethe tilt angle is defined as the direction of the slantand is measured with respect to the x-axis in theimage plane .

The process of projecting a scene plane, in 3-Dspace, onto an image plane introduces distortionsinto the texture pattern . Using orthographic pro-jection, the only distortion introduced is due to theforeshortening effect. As the slant angle is increased,the amount of foreshortening will increase alongthe angle tilt . This can be visualised by slanting acircle oriented at a certain tilt angle, as shown inFigure 2 . A curve or a texture pattern (edge detectedtexture) is composed of a number of these tangentvectors, and these will naturally be distorted by theprojection . It is this distortion of the texture's edgesthat is used to infer the surface orientation .

For texture elements with a directional bias, themethod of Witkin can no longer be applied as thesurface orientation will be interpreted incorrectly(e.g ., an ellipse would be interpreted as a circlewith a certain slant and tilt, even if the original tex-ture shape was an ellipse) . In the sequel, we outlinea method of maximum likelihood estimate usingthe a priori information on the directional distribu-tion of the line segments . Great improvements onperformance are obtained for cases where the direc-tional distribution is not isotropic .

Volume 14, Number 5

z


Figure 3 . Relationship between surface and image vectors . a isthe tangent angle in the image plane, fi is the tangent angle in

the scene plane .

2.1. Projection of surface plane

The slant and tilt angles can be alternatively de-scribed using homogeneous coordinates. The slantangle can be considered as the rotation about they-axis, and the tilt considered as the rotation aboutthe z-axis . To obtain the relationship between atangent angle /3 in the surface plane to the angle ameasured in the image plane, a vector in the imageplane is first oriented into the surface plane by anamount of slant and tilt (Figure 3) . The vector isthen projected back into the image plane, and theangle a between the projected vector and the x-axisdetermined . This is the tangent angle measuredfrom the image (see Figure 3) . The position of thescene plane is uniquely defined when there are twoor more vectors in the image .

Starting with

~=rot(z,r)rot(y,a)[cosf,sinf,0) t

(1)

cos r cos a cos /3-sin r sin /3

sin r cos a cos /i+ cos r sin J3

Y

.

(2)

~rJ

sin a cos IiThus the angle a can be determined from

a = tan '(~,/~ z.) .

(3)

Simplifying (3) yields the required relationship

a = tan I I/V

tan3\+ r-

cos a

a can be measured from the image ; but /3, a, r areall unknown . The values for a and r are those re-quired to be estimated .

2.2. Effects of slant and tilt on a distribution

The distribution of the tangent vectors projectedonto the image plane will be altered from that inthe scene plane due to the projection, as describedin the previous section . This change of distributionis used as the clue to infer the surface orientation .The effect can be demonstrated using the distribu-tion of a circle, which has a flat distribution, as anexample . The projected circle will be viewed as anellipse on the image plane, whose minor axis is inthe direction of the tilt angle, and the ratio oflengths of the major and minor axes is determinedby the slant angle (Figure 2) .

Tangent mrection (deg)(a)

4Tangent Dlrecbon (dog)

GO

Figure 4 . Effect of slant and tilt on the distribution of it circle .(a) Changing the slant angle (tilt = 0`) (h) Changing the tilt

angle (slant = 70°) .

May 1993

a

(4)

43 3

Volume 14, Number 5

As the slant angle is increased, the distributionwill concentrate around a point at right angles tothe tilt direction, as shown in Figure 4a. When theslant angle is increased to 90°, the distribution willbe concentrated at a single point (viewing the circleedge on). On the other hand, as the tilt is varied(keeping the slant constant), the shape of the distri-bution remains practically the same, but is trans-lated horizontally. The peak of the distributionwill correspond to a point at 90° to the tilt direc-tion within the range 0° to 180°, as shown in Fig-ure 4b . W e now have a means of detecting a changein the orientation of the scene plane .

2.3. Most likely surface orientation

When the texture is non-isotropic (and thus itstangent angle distribution is non-isotropic), theassumptions used by Witkin can no longer be ap-plied, and other information about the texture dis-tribution has to be sought . If the distribution oftangent angles in the scene (Dg) is known a priori,this distribution can be compared to the distribu-tion of tangent angles (Da ) measured from the im-age, and a comparison made to select the mostlikely orientation .

To determine the most likely surface orientationfrom all possible orientations, the distributions arefirst formed into cumulative distribution functions(cdf) . The cdf has the advantage that it is a con-tinuous function and thus there is no resolutionlost due to binning of the data as there is withhistograms . The cdf is defined as

n=0,

C(n)_

E9I

4t, I<n<nmax,t=1

i=1

n inmax •

(5)

The minimum absolute area difference between thetwo cdf's is used as the selection criterion and isdefined as

min E CQ -Ch l

(6)o<a<,;z0<r<,

where CQ is the cdf formed from the distributionDp using (4) for each a and r over their ranges, and

434


Figure 5 . Random shape texture pattern .

C6 is the cdf formed from Da . The problem is thusone of selecting the best match between the Dadistribution and one from an ensemble of possibledistributions using (4), or the orientation of the apriori distribution that best matches the image .The p most likely orientations are then selectedand used as candidates for selecting the maximumlikelihood estimate .

2.4. Maximum likelihood estimate

For each of the p most likely orientations ob-tained from (6), a new distribution of /i can beobtained. From (4)

/i; = tan -'(cos a tan(ai - r)),

(7)

i=1, . . .,n . A discrete pdf(f I a, r) is then formedfrom /i;, noting that pdf(/i I a, r)=pdf(f) .

Witkin used a pdf(f I a, r)=1/n through the as-sumption of an isotropic texture . As this no longerholds, the calculated distribution of pdf(f), ob-

May 1993

Figure 6 . Random rectangle texture pattern .

Volume 14, Number 5

Figure 7 . Estimated slant error for the random texture patternwith 2° of added noise using Witkin's ML method .

tamed from (7), can be used in its place . The likeli-hood function used by Witkin can now be writtenin the form

sin aL (a, r A) =

n

pdf(fl 1 (7,r)cosax i

-, cos 2 (a;-r)+sin 2 (a;-r)cos 2 a

(K)

where A={a1 , . . . . a„} .The maximum likelihood estimate can be found

from

maxL(a1,r, IA), j=1, . . .,p

where a, and T, arc obtained from (6) .

0

Figure 8 . Estimated tilt error for the random texture patternwith 2° of added noise using Witkin's ML method .


(9)

Mav 19

Figure 9 . Estimated slant error for the random rectangle pat-tern with 2° of added noise using Witkin'a ML method .

3. Results

The algorithm was tested with both artificiallygenerated texture shapes and with natural texturedimages derived from the Brodatz album [6] . Thesetextures all possess a directional bias . The orien-tation estimates produced from our method arecompared to those estimated using Wilkin's MLmethod .

3.1 . Artificial textures

Two texture shapes were used to test the estima-tion method : a random shape texture pattern (Fig-

6'

dl

Figure 10. Estimated tilt error for the random rectangle patternwith 2° of added noise using Wirkin's MI . method .

4 3 5

Volume 14, Number 5

436

Figure 11 . Estimated slant error for the random texture patternwith 2' of added noise using our method .

ure 5), and a random rectangular texture pattern(Figure 6) . Each shape consisted of 100 tangentangles . To simulate random errors associated withthe edge extraction process, gaussian noise wasadded to the tangent vectors (if no random varia-tion was included, the method would always selectthe correct orientation) .

To evaluate the method over the range of possi-ble orientations, the texture shape was artificiallyoriented, noise added, and was then processed bythe algorithm . The results are shown in Figures 7to 14 (Witkin's ML method in Figures 7 to 10, and

Figure 12 . Estimated tilt error for the random texture patternwith 2° of added noise using our method .

PATTERN RECOGNITION LETTERS May 1993

Figure 13. Estimated slant error for the random rectangle pat-tern with 2' of added noise using our method .

our method in Figures 11 to 14) with 2° of addedgaussian noise, and the orientations in five degreeincrements . The graphs show the absolute errorbetween the estimated orientation and the trueorientation . The texture examples showed an im-provement on Witkin's ML method in the orienta-tion estimate .

The largest amount of error is associated withthe small slant angles . This is caused by the factthat the tangent distribution is relatively flat andany variation of the tangent angles has a greater ef-fect in changing the shape of the distribution . For

a

Figure 14- Estimated tilt error for the random rectangle patternwith 2' of added noise using our method .

Volume 14, Number 5

large slant, these small variations will have a lessoreffect as the distribution is concentrated aroundone point . Alternatively, it is obvious from (7) thatfor small a, any noise in a will have a greater effecton the value of Q . The orientation error resultingfrom a given slant angle as the tilt is varied isrelatively constant . In general, we can estimate thetilt angle more accurately than the slant angle .

.3 .2 . Natural textures

The sample textures were obtained from Brodatz[6] . The plates used were D18 Raffia Weave (Fig-ure 15) and D15 Straw (Figure 16), which are bothexamples of non-isotropic non-homogeneous tex-tures . These images were first edge detected, thinned,and the tangent angles extracted at each pixel alongthe edge detected curves .

The tangent angles were determined from anedge detected image by first using a curve follow-ing algorithm to extract the edge detected curve . Aspline was then fitted to this curve and the tangentangles extracted for each of the pixels along thecurve . This method was used as opposed to usinggradient operators as the gradient operators arenot accurate for large changes in the edge direc-tion, e .g . corners, which regularly appear in naturaltextures . The use of the spline also increases the

Figure 15 . Raffia weave texture .


Figure lb . Straw texture .

number of different tangent angles thus producinga better distribution . This also reduces the effect ofthe rectangular grid having only four possible linedirections (0°, 45°, 90°, 135°) . The curve follow-ing algorithm also thresholds the length of thecurve, thus filtering out any spurious pixels re-suiting from the edge detection process .

The image was divided into a number of non-overlapping blocks, and the orientation of eachblock was estimated . If the whole image was cor-rectly processed as one block, a cdf of 5000+ vec-tors would have to be formed for each of thepossible orientations, at considerable computa-tional expense. The average orientation of theblocks was then calculated using directional distri-bution statistics [15] to obtain the estimate for thesurface orientation .The trigonometric mean was used as directionaldata and should be treated as a distribution on aunit circle rather than as a distribution on a line(arithmetic mean) [15] . This can be seen by con-sidering two angles e and it c where e is somesmall value . Over the interval [0, n] the arithmeticmean will be n/2 with a large variance . If the inter-val was [-7t/2,7z/21 the arithmetic mean would hezero with small variance (the correct result) . On theother hand, consider another two angles 7T/2+eand m/2-e where c is again some small value .Over the interval [0, n] the arithmetic mean will be

May 1993

4 3 7

Volume 14, Number 5

it/2 with a small variance (the correct result) . Overthe interval [-n/2, n/2] the arithmetic mean willbe zero with a large variance . This inconstancydoes not occur for the trigonometric means . Thetrigonometric mean is defined by

new direction=

En sin a

=tan'=tif O<-a<,Eni 1 cos a1

2

I

sin 2r1=-tan

if O<r<7r . (10)2

cos 2r1

The reason for the inclusion of the factor of twofor the interval [0,7r] is to make the distributioncover the whole circle . This is not required whenthe distribution is over the interval [0, n/2] .

The results of orientation estimation for the twotextures are given in Table 1 for a square patch sizeof 64 pixels . The results show that the greatest er-ror is for the small slant angles, with the estimateimproving with increasing slant angle . The resultsshow a large improvement on the results obtainedby Witkin's ML method in general for these non-isotropic textures . The results obtained for the raf-fia texture were better than those obtained for thestraw texture as the straw texture was more non-homogeneous .

Table IComparison of orientation estimates

Texture

Actual (deg)

Estimated (deg)

4 3 8

Our method

Wilkin'sML method

* 4° and 5° are effectively 184° and 185°


May 1993

4. Conclusions

A method is proposed for estimating the orienta-tion of a planar textured surface where a priori in-formation is available on the distribution of thetexture's tangent directions . The method is an ex-tension of the concept proposed by Witkin to caterfor non-isotropic textures . It compares the distri-bution of tangents obtained from a textured imageto the model distribution using a cumulative distri-bution function . A number of good candidateorientations was obtained . The best orientation isthen selected from these using a maximum likeli-hood method . The results show that our methodhas a great potential for accurate surface orienta-tion estimation where a priori knowledge existsabout the surface .

References

[1] Aloimonos, J . (1986) . Detection of surface orientationfrom texture I : the case of planes . Proc. IEEE Conf. onComputer Vision and Pattern Recognition, 584-593 .

[2] Aloimonos, J . and M- Swain (1985) . Shape from texture .Proc. IJCAI, 926-931 .

[31 Bajcsy, R . and L . Lieberman (1976) . Texture gradient asa depth cue . Computer Graphics and Image Processing 5,52-67 .

[4] Blake, A . and C. Marinos (1990). Shape from texture :estimation, isotropy and moments . Artificial Intelligence45, 323-380 .

[5] Blostein, D . and N . Ahuja (1989) . Shape from texture : in-tegrating texture element extraction and surface estima-tion . IEEE Trans. Pattern Anal . Machine Intell. 13,1233-1251 .

[6] Brodatz, P . (1966) . Textures . Dover, New York .[7] Brown, 1. and H . Shvaytser (1988) . Surface orientation

from projective foreshortening of isotropic texture auto-correlation . Proc. CVPR, 584-588 .

[8] Davis, L.S ., L . Janos and S.M. Dunn (1983) . Efficientrecovery of shape from texture . IEEE Trans . PatternAnal. Machine Intel( . 5, 485-492 .

[91 Garding, J . (1990) . Shape from texture and contour byweak isotropy . loth Internat . Coot : on Pattern Recogni-tion, 324-330 .

[10] Gibson, J .J . (1950) . The perception of visual surfaces .Amer_ J. Psychology 63, 367-384 .

[Ill Gottesfeld Brown, L . and H .A . Ibrehim (1987) . A parallelimplementation and exploration of Wilkin's shape fromtexture method . Proc. Image Understanding Workshop,DARPA, 927-932 .

Slant Tilt Slant Tilt Slant Tilt

Raffia 10 160 40 4* 70 39Raffia 20 60 50 86 74 114Raffia 30 45 44 65 86 91Raffia 40 90 38 92 68 136Raffia 50 70 59 75 84 87

Straw 10 20 30 51 84 100Straw 20 160 38 5* 85 56Straw 30 45 47 175 83 39Straw 40 90 51 62 89 89Straw 50 160 44 153 87 14

Volume 14, Numbers

PATTERN RECOGNITION LEI iERS

May 1993

1121 lkcuchi . K . (1984) . Shape from regular patterns . ArtificialIntelligence 22, 49-75 .

[13 ] Jan, J . and R . Chin (1990) . Shape from texture using theWiener distribution . Computer Vision, Graphics, and Im-age Processing 52, 248-263 .

[14] Kanatani, K. and T . Chou (1989) . Shape from texture :general principle . Artificial Intelligence 38, 1-48 .

[15] Mardia, K .V . (t972) . Statistics ofDirectional Data . Aca-demic Press, London .

[16] Pentland, A . (1986) . Shading into texture . Artificial In-telligence 29, 147-170 .

[17] Stevens, K .A. (1983) . Slant-tilt : the visual encoding of sur-face orientation . Biol. Cybern . 46, 183-195 .

[18] Stevens, K .A. (1983) . Surface tilt (the direction of slant) :a neglected psychophysical variable . Perception and Psy-chophysics 33, 241-250.

[191 Witkin, A .P . (1981) . Recovering surface shape and orien-tation from texture . Artificial Intelligence 17, 17-45 .

4 3 9

estimating surface orientation from directional textures

Documents