Proc. IEEE Workshop on Variational, Geometric, and Level Set Methods in Computer Vision (VLSM’03), Nice, France, October 2003

A Differential Geometrical Model for Contour-Based Stereo Correspondence

Gang Li and Steven W. ZuckerDepartment of Computer Science

Yale University, New Haven, CT 06520�gang.li, steven.zucker � @yale.edu

Abstract

Standard approaches to the computation of stereo corre-spondence have difficulty when scene structure does not liein or near the frontal-parallel plane, in part because an ori-entation disparity as well as a positional disparity is intro-duced. We propose a correspondence algorithm based ondifferential and projective geometry, and inspired by neu-robiology, that takes explicit advantage of both disparities.Based on curves, the algorithm relates the (2D Frenet) dif-ferential structures (position, tangent, and curvature) in theleft and right images with the Frenet geometry of the (3D)space curve. A compatibility function is defined via trans-port of the Frenet descriptors, and they are matched by re-laxing this compatibility function on overlapping neighbor-hoods along the curve. False correspondences are concur-rently eliminated by a model of ”near” and ”far” neuronsderived from neurobiology. Examples of our algorithm andstandard approaches are compared.

1. IntroductionA glance at Fig. 1 reveals many of the problems fac-ing stereo correspondence algorithms [18]: structure isnot fronto-parallel, as in many man-made scenes; order-ing constraints [16] are violated accordingly; uniquenessconstraints [12, 16, 23] are inconsistent with self-occlusion,branching and discontinuities are inconsistent with smooth-ness constraints [12, 13]; and feature density is not uniform.Nevertheless, there is a structural relationship between fea-tures in the (left, right) image pair and the 3D scene, andour goal in this paper is to develop this relationship into anew stereo-correspondence algorithm.

There are two sources of motivation for our algorithm.The first is biological: tree-dwelling primates are readilyable to solve for correspondence in images such as Fig. 1using binocularly-selective neurons that are also orienta-tion selective. This suggests an approach based on differ-ential geometry, with orientation identified with the tangentto spatial structure. Our second motivation develops thismathematical interpretation, and we use projective differen-tial geometry to derive the underlying structural relationship

between image features and 3D spatial structure. But evenwith the epipolar constraint there still exist ambiguities: dif-ferent pairings of image features give rise to different 3Dstructures, thereby creating multiple ghost matches as wellas correct ones. So finally we return to the biological moti-vation, and introduce an extended model for binocular neu-rons that implements their “near” and “far” tuning proper-ties. The result is a relaxation network that eliminates ghostmatches and explains mechanistically the disparity gradientlimit [16].

Our geometric calculations follow the literature on curvematching. Cipolla and Zisserman [3] determined imagetangent and geodesic curvature from a single view of aspace curve under perspective projection. Faugeras andRobert [17, 6] used tangent and curvature constraints in atrinocular system, and predicted the curvature at a point inthe third image based on measurements in the first two. As-suming knowledge of the fundamental matrix, Schmid andZisserman [19] described how to compute the normal at one3D space curve point, thus determining the osculating planeat that point from corresponding tangents and curvatures attwo perspective image points. We extend this previous workby showing how to compute both the normal and the curva-ture at one 3D space curve point from the perspective pro-jections of the space curve in two views.

These normals and curvatures provide the basis for ouralgorithmic framework, which follows the relaxation or be-lief propagation model. Compatibility fields are definedfrom a local (Frenet) approximation to a space curve, andthese are used to determine how compatible two pairs ofcandidate matches are. Shan and Zhang [20] used com-bined binary measurements for line segments and curvesas the compatibility function. [1] approximated the curveheuristically with a z-helix, and they used a disparity bias(to the plane of fixation) to eliminate false matches. Weindividually compute the space curve local approximationfor each candidate match pair using curvature, to eliminatethe need for heuristics. A minimum torsion approximationguides the matching, and we integrate the near and far neu-ron mechanism to address the remaining ambiguity. To ourknowledge this is the first time such mechanisms have beenused in stereo matching, and they eliminate the need for a

disparity bias.

Figure 1: (top) Natural scenes are dense in physical structure withdepth discontinuities, occlusions, and junctions, thereby posing aproblem to traditional stereo algorithms. For the region aroundthe center of the left image, when compared to its correspondingpart in the right image, note that the ordering constraint fails be-cause of the depth discontinuity. Partial occlusion at the junctionwould cause a problem for the uniquess constraint. Our algorithmis designed to function on image pairs such as this. (bottom) The(left,right) tangent (edge) map pair for the (top) images. Such tan-gent maps form the basis for our correspondence algorithm, andsuggest a curve-based approach.

2. Geometry of Space CurvesLet � ����� be a regular curve parametrized by arc lengthin ��� . If the Frenet (tangent, normal, binormal) frame� ���������������

, the curvature � � , and torsion � � are knownat � ����� , we can obtain a local Frenet approximation of thecurve by taking the third-order Taylor expansion of � at��� �

and keeping only the dominant terms:

!� �"���#� � �����%$&�' � $�'() � � � � $

� �* � � � � � � (1)

Figure 2: Frenet approximation of 3D curve around +-,/.�0 .The projection operator 1 maps this local frame to the

left and right image planes:

132�� � 465 � � � � 467 � � � � 4�8 � � � � 4 � � � � 4 � � � �:9;� ( 465 � � ( � 4 � � � � 4 � ( 4<5 � � ( � 4 � � � �

That is, the Frenet trihedron in � � projects to two Frenet(tangent, normal)-dihedra, one in the left image and onein the right image, each augmented by curvature. We re-fer to the space � ( 4 � ( 4&5 � � (=� 4>5 � � ('� 4 � � � � 4� � � � as the stereo tangent space; a point in this space is�/?A@B�DC�@E��?AF��DC�F��DG�@E�DGHF�� � @E� � FI� , where

?KJandCLJ

are the imageprojection coordinates of MONP� � , G J the orientation of pro-jected tangent in the image planes, and � J the image curva-tures, with Q �R-SB�DTU� representing left and right images.

2.1. 3D Space Curve Structure From TwoViews

In this section our technical goal is to determine the inversemapping 1WVYX , the space curve structure around the 3D pointM from its projection in the stereo tangent space. Unfortu-nately, as we will show, 1�VYX is not one-to-one: given a nodein the stereo tangent space Q �Z�/?K@B�DC�@E�D?AF��DC�F��DG�@E�DGHF�� � @E� � F=� ,we can only determine the position [ , the Frenet frame�\���]���^�

, and the curvature � at the space curve pointM . The torsion � can not be determined. To prove this weassume the following image measurements are given: po-sition, tangent, curvature, i.e.

��?%��CK��G_� � � at every imagecurve point in both images. We further suppose (for spacereasons)that the 3D position [ and tangent

are computed

by standard methods [5] or [7]. We now describe how tocompute the normal and curvature at a 3D space curve pointfrom two views.

We begin with a standard construction [2]. Let � be asmooth space curve with non-vanishing curvature � . De-note its position by the vector ` ����� in the world coordinatesystem, and its spherical projection to the unit radius imagesphere with center a by a vector b in the camera coordinatesystem (see Fig. 3). Assume the world and the camera co-ordinate systems have the same orientation, that is, there isonly translation, no rotations between them. Then the spacecurve can be described as: ` �"�'�P� a $dc������ b ����� , wherea is the vector pointing to the camera center in the worldcoordinate system,

cis the Euclidean distance of ` �"��� to

the camera coordinate system origin, and b is the vectorpointing to its spherical projection in the camera coordinatesystem.

Previous work [3, 2] shows that the relationship of imagegeodesic curvature and the 3D space curve geometry underperspective projection is:

�_e �c � � b 4 ��Wf=��Dgihj� b fI�� ( ��kl (2)

Figure 3: 3D space curve, its spherical projection to unit sphere,and its perspective projection to image plane.

where � e is the image geodesic curvature at the imagespherical projection b of the space point being studied,

and�

are the space tangent and normal at that point, � isthe curvature at the space point ` �"�'� . We now calculate therelationship between 3D space curve properties and their(measurable) persective projection image plane properties.

Consider the 2D image plane curve formed by ` �����through perspective projection, with � the vector pointingto the projection in the image plane ( b � ��������� ), and �the vector pointing to the image center, both in the cameracoordinate system. � and are the tangent and normal vec-tors in the image coordinate system, respectively. Since wechoose the camera and image coordinate systems such thatthey are aligned, we can add a third component zero to thetwo dimensional vector � and to express the tangent andnormal vectors in the camera coordinate system, which is3D. Note that this construction will form the same spher-ical projection as ` �"�'� . Furthermore, although for a planecurve the curvature is defined as the signed curvature �� ,and the normal is defined by rotating tangent 1�� ) counter-clockwise, the quantity �� is still the same if we study thiscurve as a 3D curve, and that the binormal at any point inthis curve is � � � 4 � � ������� . The relationship betweenthe perspective projection image curvature and the geodesiccurvature is thus given by:

� e � �����=�� � b 4 � �Wf �Bgih � b f � � ( ��k l� �����I��Hb f�� � 4 ��Dgihj� b f � � ( ��kl

� �����=���b f��� � ��Bg h � b f � � ( � kl� X� � � �� � f ��Bgih ���� � � f � � ( � kl

� �����=���Bg h � �� � � f � � ( ��kl (3)

Using the geodesic curvature as the bridge, we can now

connect the above two equations to formulate the relation-ship of the 3D space curve curvature and its perspective pro-jection image curvature:

�� � � � � 4 �WfI��Bgih � �� � � fI � ( ��kl �� ������Bgih � �� � � f � � ( ��kl � (4)

Remark By assuming the osculating plane (spanned by

,�) at the space curve point coincides with the M�� plane of

the world coordinate system (as in [22]), and using similartechniques as [19], we could also derive the relationship ofthe 3D space curvature and image curvatures in a projectivegeometry framework. The world coordinate system and thecamera coordinate system could have different orientations,with a homography used to relate the image plane and theosculating plane. We omit the formulas due to space limita-tion.

Now we can compute the normal�

and curvature� at a 3D space curve point from image measure-ments

��? @D�DC�@B��?AF��DC�FH��G�@E�DGHF�� � @E� � F'� of one correspondingpair, where � J is obtained from calibration, � J , � J can beeasily computed from the above image measurements, withQ � �SB��T#� .

Proposition: Given two perspective views of a 3D spacecurve with full calibration, the normal

�and curvature �

at a space curve point M are uniquely determined from thepositions, tangents, and curvatures of its projections in twoimages. Thus the Frenet frame

-\���]���^�and curvature �

at the space curve point M can be uniquely determined.

Figure 4: The geometry for Proposition. Given,����! #"$�% &��'( #")'( &*��+ &*�'( &,��+ #,�' 0 , we can compute normal -and curvature , at the 3D space curve point. Thus we candetermine the position, the Frenet frame .0/1 #-2 &354 , and thecurvature , . But the torsion 6 can not be determined.

Rewriting eq. (4) to keep the unknown normal�

andcurvature � on the left side, and using it for the left and right

images, the relationship of 3D position, tangent, normal,and curvature at a space curve point and its image positions,tangents, and curvatures under perspective projection is:

� � @ 4 �WfI� � � ��� @ � ��� @ � �Dgihj� � �� � � � f=��B(�� klc @D�Bgih ��� �� � � � f � @�� ( � kl � @

� � F 4 �WfI� � � ��� F � ��� F � �Bgih � ���� � � � f= �B(�� klcAF��Dg hj������ ��� � f � FI� ( � k l � F (5)

dfI� � � �

where subscriptS

andT

represent measurements from leftand right images, respectively. The last equation specifies�

has to be orthogonal to tangent

. This system can besolved for

� � , from which we can compute the curvature �and normal

�.

Figure 4 illustrates that from two views we can computethe 3D space curve normal, thus determine the Frenet frame�\���]���^�

and curvature � at a 3D space curve point, butnot the torsion � .3. Stereo Correspondence AlgorithmIn the previous section, we described how to compute thespace curve local approximation given image measurementsin two perspective views. Now we describe how this geo-metric relationship can be used to solve the stereo corre-spondence problem. The central idea is as follows.

At each point � ��� � � along the space curve, a Frenet ap-proximation can be constructed to characterize the localbehaviour of the curve around that point based solely onthe information at that point. Since measured informationis also available for nearby points, this can be transportedalong the approximation to the neighborhood of � ���H��� andthen compared with the frame at � ������� . That transportedinformation which is ”close to” the measured informationis supported, and that which is ”far” is not supported. Bothposition and orientation information are used in determin-ing these distances.

The situation is analagous to co-circularity [14] for im-age curves. A unit speed planar curve

� �����, with posi-

tive curvature, has a unique osculating circle that approx-imates

�around

�. This approximation is used to define

co-circularity constraints so that nearby (estimated) edge el-ements can be transported along the curve locally using itsosculating circle, and a relaxation labeling network extrem-izes a global functional of how well the transported edgesmatch. Compatibility functions encode these transport op-erations.

Three distances are used in building compatibility func-tions for co-circularity: the transport distance along the os-culating circle, the distance from this point to the edge, and

the angular difference between the measured and the trans-ported edges. We now seek to do this for space curves, butthe situation is a little more complicated.

3.1. The Minimal Torsion Constraint

To infer the 3D space curve, we first get its local approxima-tion at each point, and then transport (estimated) tangents tonearby positions. A relaxation labeling network will inferthe space curve geometry.

The heart of this process is cartooned in Fig. 5, wheretwo space tangents along the 3D Frenet approximation areshown. Observe that each of the space tangents projects to apair of image tangents, so the nodes Q and � in the relaxationnetwork consist in pairs of tangents, one in the left imageand one in the right, and compatibilities

T J��are defined over

these pairs. Observe further that, while the expected posi-tional disparity between these tangents is introduced, thereare also orientation disparities; such higher-order disparitiescontribute to the matching process and also have biologicalcounterparts [8].

Figure 5: Cartoon of the stereo relaxation process. (top) showsa pair of space tangents associated with the Frenet approximationaround the point � . Each of these tangents projects to a (left,right)image tangent pair; compatibility between the space tangents thuscorresponds to compabilitity over (left,right) image tangent pairs.The projected tangents are shown as thick lines. One left imagetangent is redrawn in the right image (as thin lines) to illustratepositional disparity ( � � ) and orientation disparity ( �)* ).

A stereo tangent pair, or point in the stereo tangent space,consists in the 8-tuple:

��?A@B��C�@E�D?KF���C�F���G'@E��GHF�� � @E� � F=� . The setof all stereo tangent pairs � in the neighborhood of Q , suchthat Q and � are compatible, is called the compatibility fieldaround Q .

To build the compatibility fields for stereo, recall thattorsion remains a free parameter. Varying this term yields afamily of local approximations; Fig. 6 .

The Frenet frame does specify the osculating plane in��� , however, so the transport distance that we use to define

unit:mm

-4-2

02

4

X

-5

-2.5

0

2.5

5

Y

1598

1600

1602Z-5

-2.5

0

2.5

5

Y

1598

1600

1602Z

-40 -20 0 20 40 60

-4

-2

0

2

4

unit:pixel

-40 -20 0 20 40 60

-4

-2

0

2

4

unit:pixel

Figure 6: (TOP) Family of local approximations in 3D ob-tained by varying torsion. (BOTTOM) Discrete CompatibilityField projected to the left and right image planes. Both for��� ,��=. D. ����I. D. �� � ����� D.�� ������. �.�� ����� ��0 , focal length �I.'.pixels, baseline �=.'. ��� .

the compatibility field is piecewise co-circularity in the os-culating plane at each point (see Fig. 7). Since the changeof the pose of the osculating plane in � � is related to the tor-sion, the result amounts to a minimal torsion constraint.Weconjecture that � varies smoothly along the curve.

3.2. Stereo Relaxation Labeling

We now specify the stereo algorithm in more detail. Thetask is to select pairs of nodes in the stereo tangent spacethat are most consistent with one another by stereo relax-ation labeling using the discrete compatibility fields justsketched. (They will be refined below.)

Relaxation labeling processes assign labels to nodes ina graph based on the parallel use of local constraints. Sup-pose a set of nodes are given, and a set of labels are definedfor each node. � J��"cA� is the probability that label

cis correct

for node Q , with � � � J ��cK�U�Zg for every node Q . Labels areselected at each node by an iterative gradient ascent that ex-tremizes the functional � � � � � � � J �"cA�BT J�� ��c%��c �/� � � ��c � � inparallel for all nodes Q and labels

cin the network. The com-

patibilitiesT J�� �"cY� c!�/�

specify the local constraint betweennodes Q and � , with labels

candc �

, respectively [9].For the stereo correspondence problem, a simplified ver-

sion of the above algorithm is used. Let " be the set ofnodes Q � �/? @ �DC @ ��? F ��C F ��G @ ��G F � � @ � � F � . Each node is as-signed two labels

c N 5 �$#&% �(' �*),+-% � . Since thereare only two possible labels at each node and � J � 5 �.#.% � $� JD�/' �*),+-% � � g

, we only store � J � � JD� 5 �.#.% � .(� JD��' ��)�+-% �#�dgih � J ). The update rule is:

Figure 7: Illustration of transport distance. The nearby tangent at�10 is projected onto the Frenet approximation computed at � . Ac-cording to the minimum torsion constraint, this is approximated byprojection onto the osculating circle at � , which lies in the oscu-lating plane given by the ,32 �4 0 frame. Thus three componentscontribute to the compatibility: transport distance along the os-culating circle; projection distance onto the osculating circle; andangular rotation to tangency.

�!576 XJ �98 � 5 J $;:�T J�� � 5�1< X� (6)

where8>= < X� denotes projecton onto [0,1].

The relaxation labeling graph is built from all possiblestereo corresponding pairs, as described below.

3.3. The Discrete Tangent Map

Edges are detected in each image to provide�/?��DC �DG � � � ,

with��?%�DC_�

the position coordinates,G

the orientation of the(edge) tangent, and � the curvature. Again following bi-ology, the edge measurements are taken redundantly, oneat each orientation, so that multiple tangents can be repre-sented at each image point. This is important for the prob-lem of partial occlusion at junction points. To illustrate: atthe image junction in Fig. 1(left), the tangent map has twodifferent tangents, one for the branch in front and one forthe branch behind. These two tangents can match with thetangents in the right image along the epipolar lines. As aresult, both the branch in front and the branch behind havea valid correspondence pair at the junction point.

To get the discrete tangent map(Fig. 1), we use the log-ical/linear operators [10] on the original images, followedby a trace inference stage [14] and an interpolation func-tion [4]. The initial edge detection is performed at 16 ori-entations, and interpolated to ? � quantized orientations and) g

discrete curvatures. The additional accuracy in orienta-tion is required for the orientation disparity component ofthe compatibility distance function.

3.4. Building Compatibility Fields withNear/Far Inhibition

The (left,right) tangent maps provide the candidatematches to build the relaxation network. For each item�/? @ �DC @ ��G @ � � @ � in the left tangent map, a search alongthe epipolar line in the right tangent map reveals pairsof possible correspondences. Each stereo tangent pair�/? @ �DC @ ��? F ��C F ��G @ �DG F � � @ � � F � is then a node in the stereo re-laxation labeling graph. The edge relationship in the graphis given by spatial neighborhoods.

For each stereo tangent pair, we compute its local spacecurve structure in the osculating plane, as described pre-viously. Fig. 8 illustrates this idea for two correspondingpairs.

Figure 8: Are two neighboring candidate correspondence pairscompatible? The relationship has to be derived from the geometryof space curve and its image projection curves. Shown are twoFrenet frames in ��� at nearby points along a space curve, whichproject to two frames in the (left,right) image pair.

Even with the local geometric relationship derivedabove, however, there are still ambiguities along the epipo-lar line. To illustrate, consider the highlighted upper rightregion in Fig. 1. (Fig. 9) shows the reconstruction basedsolely on the geometric relationships. In the front view ofthe reconstruction (Fig. 9(c)), the left and right parts havesimilar - correct - depth, while the middle parts are falsematches. This is even more clear in the rotated reconstruc-tion view (Fig. 9(d)). where the correct matches correspondto the middle two branches with similar depth, and the outertwo branches are the false matches. Without global con-straint, such false matches will generically arise becauseone point in the left image could possibly match to multiplepoints in the right image.

This correspondence ambiguity is classical [12], but can-not be dealt with uniformly by the standard heuristics thathave been introduced for physical surfaces. Since many

(a) (b)

(c) (d)

Figure 9: False matches from attempting correspondence on apair of twigs, because the geometric compatibilities can apply toall possible matches. (E.g., the left branch in (a) can match boththe left and right brances in (b), and so on.) (a)(b) Left and righttangent maps of the highlighted upper right region in Fig. 1. (c)Front view of reconstruction showing false matches (the middletwo). (d) Rotated view showing false matches plus ghosts fromnoise in the tangent map. Depth scale shown at right.

of these false matches tend to have large disparity gradi-ents, the Disparity Gradient constraint from the PMF al-gorithm [16] might be appropriate. Neurobiology suggestsanother implementation of this, which is natural because itfits directly into our compatibility functions.

We remarked earlier that binocular neurons were orien-tation selective, and exploited that property. We now expoitthe fact that they can be classified into three groups that re-late to disparity offsets in receptive field structure [15]. (1)Tuned excitatory neurons form a group that are disparity se-lective over a limited (and often narrow) range. (2)Far neu-rons exhibit a selectivity for uncrossed disparities; and (3)Near neurons are selective for crossed disparities. Far andnear neurons are complementary: One set gives excitatoryresponses to objects farther than the point of fixation and in-hibitory responses to nearer objects; while other set has theopposite behavior, excitation for nearer objects and inhibi-tion for farther ones [11]. It is precisely this property thatwe include in our compatibility functions, with inhibitionfor each possible match in proportion to the probability ofmatch (� J ); see Fig. 10. The result successfully eliminatesthe great majority of false matches, as we now show.

4. Results and ConclusionFig. 11 shows the 3D reconstruction of image pair (Fig. 1)by our algorithm, together with an enlargement of the prob-lem area discussed previously. The color depth map indi-cates both the front and the back branches are correctly re-constructed. Furthermore, note how the false matches were

Figure 10: Near and Far neuron tuning function. In biologicalterms depth can be encoded by neurons tuned to specific dispari-ties; this is cartooned here as an excitatory neuron tuned to 0 dis-parity. Also shown are tuning curves for near and far neurons,which are excited (resp. inhibited) by stimuli closer (resp. fur-ther) than the point of fixation. Our compatibility functions arethe product of these tuning curves with the geometric ones (Fig.5, bottom). Thus potential matches are supported by geometricconsistency but inhibited by better matches in the neighborhood.

removed by the near/far inhibition from the upper-right por-tion (compare Fig. 9). The relaxation process converged af-ter 5 iterations in 60 sec. on a Pentium III PC.

Lacking ground truth on the twig example, we assessedthe accuracy of our algorithm with synthetic images. Tenalgebraic space curves were generated with parameters cho-sen randomly. The curves were all started at distanceof 1800mm, and ranged in depth from about 1640mm to1960mm. These were then projected through the cameramodel to create an image pair, and our algorithm was runon this pair. The results were then compared with groundtruth. In Fig. 12(LEFT) we show one example (an ellipse)in which the actual and reconstructed depth are plotted asa function of arc length. The standard deviation is about1.665mm from the true value, and the maximal differenceis 8.04 mm. The quantization in the system is such that onepixel corresponds to about 9.33 mm (at 1.8m), which showsthat our edge detection and interpolation are functioning tosub-pixel accuracy.

Finally, to provide a comparison with other algorithms,in Fig. 13 we show our reconstruction for another, morecomplex twig scene with disparity maps from traditionalalgorithms. Neither the refined correlation method [21](Fig. 13(f)) nor the cooperative algorithm [23](Fig. 13(g))could be made to perform adequately by varying their in-ternal parameters, although the cooperative algorithm per-forms much better than the correlation method.

By relating the differential structures at image points(position, tangent, and curvature) in the left and right im-ages with the geometry of the space curve point (position,Frenet frame, and curvature), we show how nearby imagecorrespondence pairs should be consistent with the Frenetapproximation of the space curve, and how this compati-

Figure 11: 3D reconstruction of the opening Fig. 1: (TOP) Frontview. Note how the false matches were removed by the near/farinhibition (compare Fig. 9). (BOTTOM) A detailed view of the re-construction of the problem region in which both the ordering con-straint and the uniqueness constraint break down. Colored depthscale is shown at right (units: meters).

bility relation can be put into a stereo relaxation labelingnetwork where positional disparity and orientation disparityare combined naturally. Examples demonstrate how our al-gorithm outperform traditional stereo algorithms, especiallyat places where heuristic constraints break down.

References[1] S. Alibhai and S. W. Zucker. Contour-based correspondence

for stereo. In Proc. 6th ECCV, 2000.

[2] R. Cipolla and P. Giblin. Visual Motion of Curves and Sur-faces. Cambridge University Press, 2000.

[3] R. Cipolla and A. Zisserman. Qualitative surface shape fromdeformation of image curves. IJCV, 8:53–69, 1992.

[4] C. David and S. W. Zucker. Potentials, valleys, and dynamicglobal coverings. IJCV, 5:219–238, 1990.

[5] O. Faugeras. Three-Dimensional Computer Vision. The MITPress, 1993.

[6] O. Faugeras and L. Robert. What can two images tell usabout a third one? IJCV, 18:5–19, 1996.

[7] R. Hartley and A. Zisserman. Multiple View Geometry inComputer Vision. Cambridge University Press, 2000.

[8] I. P. Howard and B. J. Rogers. Binocular Vision and Stere-opsis. Oxford University Press, 1995.

[9] R. A. Hummel and S. W. Zucker. On the foundations of re-laxation labeling processes. IEEE Trans. on PAMI, 5(3):267–287, 1983.

0 200 400 600 800 1000 12001700

1720

1740

1760

1780

1800

1820

1840

1860

1880

1900

s (arc length)

Z (

dept

h)

Std: 1.665 mm

unit:mm

True dpethRecons. depth

1650 1700 1750 1800 1850 1900 1950−10

−8

−6

−4

−2

0

2

4

6

8

10unit:mm

Z (depth)

Ave

rage

Rec

ons.

Err

or

Std:2.163 mm

Figure 12: Error analysis from a series of 10 algebraic curveswith random parameters (see text). (LEFT) Plot of reconstructeddepth and true depth vs. arc length for an ellipse; standard devia-tion: 1.665mm. (RIGHT) Average absolute value of the differencebetween the reconstructed depth and the true depth, vs. depth.Standard deviation: 2.163mm.

[10] L. A. Iverson and S. W. Zucker. Logical/linear operators forimage curves. IEEE Trans. on PAMI, 17(10):982–996, 1995.

[11] S. Lehky and T. Sejnowski. Neural model of stereoacuityand depth interpolation based on a distributed representationof stereo disparity. J. Neurosci., 10:2281–99, 1990.

[12] D. Marr. Vision. W.H. Freeman and Company, 1982.

[13] G. Medioni and R. Nevatia. Segment-based stereo matching.CVGIP, 31(1):2–18, July 1985.

[14] P. Parent and S. W. Zucker. Trace inference, curvatureconsistency, and curve detection. IEEE Trans. on PAMI,11(8):823–839, 1989.

[15] G. F. Poggio and B. Fischer. Binocular interaction and depthsensitivity in striate and prestriate cortex of behaving rhesusmonkey. J. Neurophys., 40:1392–1405, 1977.

[16] S. B. Pollard, J. E. W. Mayhew, and J. P. Frisby. Pmf: Astereo correspondence algorithm using a disparity gradientlimit. Perception, 14:449–470, 1985.

[17] L. Robert and O. Faugeras. Curve-based stereo: Figural con-tinuity and curvature. In Proc. IEEE conf. on CVPR, 1991.

[18] D. Scharstein and R. Szeliski. A taxonomy and evaluation ofdense two-frame stereo correspondence algorithms. IJCV,47(1/2/3):7–42, 2002.

[19] C. Schmid and A. Zisserman. The geometry and matching oflines and curves over multiple views. IJCV, 40(3):199–233,2000.

[20] Y. Shan and Z. Zhang. New measurements and corner-guidance for curve matching with probabilistic relaxation.IJCV, 46(2):157–171, 2002.

[21] C. Sun. Fast stereo matching using rectangular subregioningand 3-d maximum-surface techniques. IJCV, 47(1/2/3):99–117, 2002.

[22] Z. Zhang. A flexible new technique for camera calibration.IEEE Trans. on PAMI, 22(11):1330–1334, 2000.

[23] C. Zitnick and T. Kanade. A cooperative algorithm for stereomathching and occlusion detection. IEEE Trans. on PAMI,22(7):675–684, 2000.

(a) (b)

(c) (d)

(e)

(f) (g)

Figure 13: 3D Reconstruction: (a)(b) Image pair. (c)(d) Discretetangent maps. (e) Front view of recontruction, colored depth scaleshown at right. (f)(g) Disparity maps of traditional algorithms [21,23].