shape from silhouette: image pixels for marching cubes

7
Shape from Silhouette: Image Pixels for Marching Cubes Bruno Mercier SIC Lab, Bât. SP2MI, téléport 2 Bd Marie et Pierre Curie 86962 Futuroscope Chasseneuil, France [email protected] Daniel Meneveaux SIC Lab, Bât. SP2MI, téléport 2 Bd Marie et Pierre Curie 86962 Futuroscope Chasseneuil, France [email protected] ABSTRACT In this paper, we propose to use image pixels for geometry reconstruction with a shape from silhouette approach. We aim at estimating shape and normal for the surface of a single object seen through calibrated images. From the voxel-based shape obtained with the algorithm proposed by R. Szeliski in [18], our main contribution concerns the use of image pixels together with marching cubes for constructing a triangular mesh. We also provide a mean for estimating a normal inside each voxel with two different methods: (i) using marching cubes triangles and (ii) using only voxels. As seen in the results, our method proves accurate even for real objects acquired with a usual camera and an inexpensive acquisition system. Geometry reconstruction from images, shape from silhouette, marching cubes. 1. INTRODUCTION Since the early years of computer vision, much effort has been dedicated to automatically digitizing shape and reflectance of real objects. For instance, Stan- ford Digital Michelangelo [11] project aims at digitiz- ing large statues, Callet. et al. [1] digitize statuettes and reconstruct plaster models covered with bronze. Hasenfratz et al. [7] use a digitize shape for placing a real actor in a virtual environment so that shadows and lighting be properly computed. In most cases, acquisition hardware play a major role for reconstructing objects shape and research efforts have increased a lot during the last decade. Our concern is about objects only described with a set of calibrated photographs. Our final goal is to insert real objects (corresponding to lightfields/lumigraphs or described by a series of images) into virtual environments with proper lighting and global illumination. The whole problem is thus not only geometry estimation but also initial light sources position, reflectance properties for the real object as well as rendering process. This paper only addresses a small part of the whole work: geom- etry and normal estimation. The basis of our work is the voxel-based shape from silhouette technique presented by Szeliski in [18]. We propose a new method for combining the marching cubes algorithm with image pixels for precisely re- covering a triangular mesh corresponding to the ob- ject shape. We also propose two methods for esti- mating surface normal. As shown in the results, our method has a consequent impact on geometry. Based on this method, we have successfully recovered light sources geometry and object surface reflectance prop- erties [14]. This paper is organized as follows. We firstly describe work most related to our concerns. Section 3 presents the acquisition system we use and work overview. We then detail our reconstruction and normal estimation method. Finally, we provide a series of results before we conclude. 2. RELATED WORK The literature concerning geometry reconstruction is vast and this section only presents a quick run through the area for most closely related works.

Upload: others

Post on 16-Feb-2022

11 views

Category:

Documents


0 download

TRANSCRIPT

Shape from Silhouette:Image Pixels for Marching Cubes

Bruno MercierSIC Lab, Bât. SP2MI, téléport 2

Bd Marie et Pierre Curie86962 Futuroscope Chasseneuil, France

[email protected]

Daniel MeneveauxSIC Lab, Bât. SP2MI, téléport 2

Bd Marie et Pierre Curie86962 Futuroscope Chasseneuil, France

[email protected]

ABSTRACT

In this paper, we propose to use image pixels for geometry reconstruction with a shape from silhouette approach.We aim at estimating shape and normal for the surface of a single object seen through calibrated images. From thevoxel-based shape obtained with the algorithm proposed by R. Szeliski in [18], our main contribution concerns theuse of image pixels together with marching cubes for constructing a triangular mesh. We also provide a mean forestimating a normal inside each voxel with two different methods: (i) using marching cubes triangles and (ii) usingonly voxels. As seen in the results, our method proves accurate even for real objects acquired with a usual cameraand an inexpensive acquisition system.������������

Geometry reconstruction from images, shape from silhouette, marching cubes.

1. INTRODUCTIONSince the early years of computer vision, much efforthas been dedicated to automatically digitizing shapeand reflectance of real objects. For instance, Stan-ford Digital Michelangelo [11] project aims at digitiz-ing large statues, Callet. et al. [1] digitize statuettesand reconstruct plaster models covered with bronze.Hasenfratz et al. [7] use a digitize shape for placing areal actor in a virtual environment so that shadows andlighting be properly computed.

In most cases, acquisition hardware play a major rolefor reconstructing objects shape and research effortshave increased a lot during the last decade. Our concernis about objects only described with a set of calibratedphotographs. Our final goal is to insert real objects(corresponding to lightfields/lumigraphs or describedby a series of images) into virtual environments with����������������������� �"!$#��&%'��(��)�*!$+,�$�.-/!0�*%21���3/���4�5�$67!$+�+�$�839!:���8�$6;��-'�<�>=?�$��#56@�$�83A�������$�9!$+B�$�814+<!$���C���D�$�FE'������G($�*!0�H���4%I=�)��-'��E'��6J�4�K3'���:LH�<%'�M%N��-9!:�G14�$3/���4� !:����/�0�O�"!�%'�5�$�P%Q�<�C������R'E'���M%S6J�$�P3Q���$T/�U�$�P14�������V��14�<!$+!�%QL�!0�H�*!0(��8!$�/%7��-9!:�W1���3/���4�XRA�M!0�Y��-/���Y�/�0���<1��Z!0�9%���-'�6JE'+<+�14�)�*!0�����$�.���[��-'�,T/���C�\3/!$($��]_^`�U14�$3Da5�0��-/����=������b���P����3/E/R'+��<��-cb����53A�$�C�d����������Le�����,�$�f���P���M%'���C������R/EQ�������+����C���4bD���MgDE/�)���4�h3'�����$�i��3A�414�)T/1j3A�V�����<�������$��!$�/%9k0�0�l!m6@�4��]

n�o/p�qHr0sQt*uAv$w�rCxzyY{`|~}l���'r�w<�J�$�$���*{'{Q���:�Q�$�e�����D���yX{�|~}�� �Q�����$��qHv$u/s'v$t*�Y�H���J��p4��t*s'v$t*�;�'�h�Q������w �:pVu'�j|��:p��*oW�p��9s'�4w ���:�� ��3DaH����(�-D��m�¡ �¢m�5£¡(����/1�a"¤?¥Q14���4�'14�������4���4]

proper lighting and global illumination. The wholeproblem is thus not only geometry estimation but alsoinitial light sources position, reflectance properties forthe real object as well as rendering process. This paperonly addresses a small part of the whole work: geom-etry and normal estimation.

The basis of our work is the voxel-based shape fromsilhouette technique presented by Szeliski in [18]. Wepropose a new method for combining the marchingcubes algorithm with image pixels for precisely re-covering a triangular mesh corresponding to the ob-ject shape. We also propose two methods for esti-mating surface normal. As shown in the results, ourmethod has a consequent impact on geometry. Basedon this method, we have successfully recovered lightsources geometry and object surface reflectance prop-erties [14].

This paper is organized as follows. We firstly describework most related to our concerns. Section 3 presentsthe acquisition system we use and work overview. Wethen detail our reconstruction and normal estimationmethod. Finally, we provide a series of results beforewe conclude.

2. RELATED WORKThe literature concerning geometry reconstruction isvast and this section only presents a quick run throughthe area for most closely related works.

{���pVt�p�r��M������r$u[5, 2] uses two cameras located close one

to another so that images of the object be slightly dif-ferent. Internal and external camera parameters knowl-edge help to determine corresponding points on imagesand deduce depth with the help of textures, laser gridsor structured light.

{QoQvV�'p¡xVtCr����Co'v0�Ju��methods aim

at recovering objects shape with the assumption thatsurfaces are lambertian (or almost lambertian) [16, 8].

Shape from silhouette approaches [13, 3] are moreadapted to our problem since we do not want to makeany assumption about images. Objects can have glossysurfaces, with or without textures and we cannot useactive laser grids or test patterns to reconstruct the ob-ject geometry. Most shape from silhouette methodsrely on a voxel-based datastructure and the approachdescribed by R. Szeliski in 1993 [18] is often used as abasis. For such methods, a well-known drawback con-cerns cavities. For example a coffee cup handle willbe recovered since the hole can be deduced from thevisual hull on the image, but the inside will be filled-upwith material (unless there is no bottom). Improve-ments have been proposed for solving this problemwith voxel coloring [15], space carving [9] or gradientvoxel flux [4] with the assumption that objects surfaceis mostly lambertian.

From voxels, the marching-cubes algorithm can easilygenerate a triangular mesh corresponding to the objectsurface [17]. For example Hasenfratz et al. use such areconstruction method for interactively integrate a realperson into a virtual environment [7]. Our method fo-cuses on such approaches and aims at improving thetriangular shape accuracy. To achieve this goal, wepropose to use image pixels for guiding the marchingcubes algorithm and estimating accurate surface nor-mal.

3. WORK OVERVIEW

Acquisition SystemFor this work, we used images of both virtual objectsand real objects. Virtual objects are convenient for val-idating the method and providing result quality sincecamera parameters and object geometry are known.For real objects, we devised the acquisition system de-scribed in figure 1. Object and light sources are fixedon a turntable: a camera is located on a tripod withfixed aperture and shutter speed. During acquisition,camera position does not change. Every 5 degrees,two images are acquired with the same viewpoint. Thefirst one is overexposed with an additional light sourcefor separating object from background (a lambertianblack cloth) while the second one is used for acquiringthe actual object radiance. After one turn, the camerais raised of several centimeters. In practice, only 1turn (76 viewpoints) is necessary for precisely recov-ering the object shape, but we also used this system for

acquiring complete image-based objects.

� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ����������������������������������������������������������������������������������������������

Turntable

Diode

CameraLight sources

��������������� �"!#$����%&�('&�()�*+%-,$%-'-�/.10

Image ProcessingFor separating background from the object, we use apowerful light source for overexposing images. Theobject is so bright that it is easy to determine the blackbackground with a seed-fill algorithm even with darkregions on the object. Except background, only 2 con-nected regions remain: the object and the red diodeused for estimating camera orientation. Backgroundpixels are then set as perfectly black on images: (0,0,0)for R,G,B values.

The diode is used to determine the rotation axis of theturntable seen on photographs. Focal length is knowna priori (fixed camera parameters) and 3D position ismanually estimated. Orientation can thus be deducedfrom the red diode. We did not use any test pattern forestimating camera parameters.

Reconstruction ProcessOur reconstruction method is composed of 3 mainsteps:

1. the shape from silhouette approach proposed byR. Szeliski in [18] provides a set of voxels;

2. a triangular mesh is generated from marchingcubes and image pixels;

3. a normal is estimated for each voxel either withthe marching cubes triangles or with a methodusing only voxels.

4. SHAPE FROM SILHOUETTEOctree ConstructionAs a first (rough) approximation of the object geometry,we used the shape from silhouette approach proposedin [18]. With this approach, all the images are used it-eratively to hierarchically sculpt the object. The shapeinitially corresponds to a voxel, recursively refined dur-ing the reconstruction process. For each view of theobject, the octree voxels obtained so far are projectedonto the image plane and compared to image pixels.When a voxel is seen outside the object for one image,it is actually marked as

r$s��(figure 2(a)); when a voxel

is seen inside the object for all the images, it is marked

as�Ju

(figure 2(b)); all the others are often seen insidethe object and sometimes on the object boundary, theyare marked as

v��;��� �$s'r$s��(figure 2(c)) and subdivided

into 8 sub-voxels. This process is repeated until no am-biguous voxels exist or a minimum size criterion hasbeen reached. For our method, the algorithm should

� !��.��E'��¤L����Q�4+

� R��?���'¤ Le���Q�4+ � 1��P!0� RQ¤Le���'��+

� �(�������� �� ) � ��� !�����%&%&��� !��/'&�()�* 0

stop when ambiguous voxels correspond to a series of4 (or 9) pixels for more reliable results. Figure 3 showssome results for a clown (real object). Note that for a256x256 image, pixel resolution corresponds to a depthof 8 in the octree.

� �(��������� ��� � ! )�*�%-'-� � ! '&�()�* � � %&����'&%�� )����(����� ��%��! �"��*$#&% 0

Obviously, all ambiguous voxels have the same size,according to the reconstruction process. At the oppo-site, voxels marked as

�Juorr$s��

are not further subdi-vided and have various sizes.

Practical AspectsFor our method, voxel projection on the image planeis performed with raytracing. This will be further usedwith marching cubes in the following section. Eachpixel corresponds to a ray originating at the imagecenter-of-projection and going through the pixel (figure4). These rays are called

�9�('Dp�w)��t�v$� �from now on. Pixel-

rays corresponding to the object silhouette are called�Ju/� tCv$���since they hit the object and rays corresponding

to background are calledr$s���� tCv$���

.

Surface ThicknessThe reconstruction process results in a set of ambigu-ous voxels called

$�(����t�p ��p���sHtJx4v��Vp. For marching cubes,

this surface needs to be 6-connected which is not en-sured by the previous algorithm. To achieve this goal,we propose to modify the discrete surface with an ad-

Background(out−ray)

(in−ray)

Object

� �(��������)��+* ��� � ��, ����,�% 0

ditional process applied every time ambiguous voxelsare subdivided. Voxels classified as in or out can bereclassified as ambiguous when the discrete surface isnot 6-connected (as illustrated in figure 5): when twoadjacent in and out voxels do not correspond to thesame hierarchy level in the octree, we choose to re-classify the smallest one, -/.1032�454 , as ambiguous (since- .10326454 parents had been longer classified as ambigu-ous). In the case of two voxels with similar size, weconsider that out-voxels should remain outside the ob-ject according to the sculpture method seen above; thein-voxel will consequently be reclassified as ambigu-ous. A final operation reduces the surface thicknesswhile keeping the 6-connection.

� !�� � R/� � 1��

out−voxelamb−voxelin−voxel

discontinuity problemvoxel to be modified

� �(�������7� �8 ) �$� ��% ��� !�����%&%&�(� !���'-�()�*:9;�$07<�=�)/���/%�> ��*'�=�� %&�����?��! �/9A@ 0B��)�� � ��%�'-)7@ � . )!# ��� ��#C9 !/0�. )!# ��,� � #D#���%&! ����'-� %&�$�E�?��!���FHG , ! )�* *�� ! '-� # ��*&�JILK�0

5. PIXELS FOR MARCHING CUBESOriginal Marching CubesFor reconstructing a triangular mesh from a set of am-biguous voxels, marching cubes [12] are obviouslywell-suited to the problem. Originally, the algorithmis dedicated to medical images and uses some densityvalues (weights) associated with each voxel vertex; asa rough simplification, positive weights correspond topoints inside the object and negative weights are out-side the object. A linear interpolation provides the(estimated) intersection between each voxel edge andthe actual object surface. According to these intersec-tions, a triangular mesh can be defined for each voxelwith a limited number of configurations (see figure 6).

� �(������� G ��� �/��!E= ��*$� !� @ � % ! )�* � ��������'&��)�* % F ���$,� ������# '-) ��.�@ ������)�� % ��)�� � ��% )�*$�(, K� @ ����!���#�)�'-% ��* ,# ��! ��'-��������'&��! � %8�() !��/'-� # )���'&%&� #$� '6=�� %&����� ��! ��0

Since weighting values cannot be computed directlyfrom the discrete surface, edges centers are often usedfor generating triangles. However, as explained in thefollowing, it is possible to use pixel-rays with march-ing cubes so that triangles fit the model shape moreprecisely.

Refining the MethodAs a first estimation, voxel vertices are classified ac-cording to neighborhood. Each voxel vertex of the6-connected discrete surface has (at least) either onein-voxel or one out-voxel neighbor. For placing trian-gles in

v������ �$s/r0s � �0r�'Dp�w �, we also use pixel-rays (figure

7).

� �(������� " ��� ��' , ��� ,$%"%�=�)���� #+*�)�' '-)�� ! = '�=$� %&����,�?��! � ��* %&� #�� ��* ��.�@ �(����)���% ��) �$� � 9$��* , �E��,$% ����� *�)�'� %-� # 0

Our algorithm firstly computes the intersection pointsbetween out-rays and voxel faces (figure 8). Then, aline corresponding to the object surface on each face isestimated (called

��sHt@x4ve�*pBw �JuApfrom now on). This line

is placed as close as possible to all intersection pointsand out vertices. Let us consider the 2D convex hullassociated to all these points � ��� . It can be shownthat for the general case, such a line corresponds to aconvex hull line segment ��� . The surface line is thuschosen among ������ so that the following function beminimized: ��� ��� ����� �

��� � � � �"! �#� �%$

Unfortunately, in some cases, a surface line cannot becomputed using the convex hull (see figure 8(d) forexample). For such cases, the surface line is defineddirectly using in-vertices.

Finally, the intersection between surface lines and voxeledge defines the point (

���4� �$pVt ��pH') used for marching

cubes. Note that when the surface line contains a facevertex, our algorithm slightly shift the mc-vertex foravoiding degenerated triangles.

� !��

&'&'&'&'&'&'&&'&'&'&'&'&'&&'&'&'&'&'&'&&'&'&'&'&'&'&&'&'&'&'&'&'&&'&'&'&'&'&'&('('('('('('(('('('('('('(('('('('('('(('('('('('('(('('('('('('(('('('('('('(

� R�� � 1��

surface linemc−vertexout−ray intersectionout−vertexin−vertex

� %J�� �(�������*) �,+ * '-��� %-� ! '&�()�* @ ��'.- ��� * )���' , ����,�%&��*�#��) �$� � �?��! � %�, %-�������E�J� !���%-�/%�9 �$0&� )�� � %&�$�J� ! ��%-� %�9@ 00/ !/01- �('6= � # ��%&! )�* *�� ! '-� # %&�$�E�?��!�� 9;# 0���)�� � �������'&��! � % � %-� #D��% � @ )�� *�#$�/� ,�0

The defined surface line is bounded by voxel verticesaccording to classification so that surface continuity bemaintained (figure 8(d)). For two adjacent faces, thecorresponding surface lines have to be connected (seefigure 9(a)). The only adequate choice consists in usingthe mc-vertex closer to the

�Ju/� �$pVt ��pH'since the other one

would re-introduce out-rays intersection points insidethe object surface. Finally, the same principle appliesto adjacent voxels on the discrete surface (figure 9(b)).

6. SURFACE NORMALFor some application, a normal can be needed for sur-face voxels (for instance, for estimating light sourcesproperties and BRDF of the object [14]). We proposetwo methods: the first one uses the triangles gener-ated by our adapted marching cubes algorithm whilethe second one relies on the discrete surface and itsneighborhood.

Normal from TrianglesInside each ambiguous voxel, several triangles definethe object surface. We propose to compute the averagetriangle normal weighted by area.

Using triangles located in only one voxel leads to a

� !�� � R/�� �(���$� �% ���$����� ��! � ! )�* '&��* � �(' , ��*�# . !E,H����� '-��! � % 0

� !��Object

� R��� �(������� ���$��� )�� . ���A� ��) . .��� !E= ��*�� !�$@ � % ' ������* ,�/�(� %�9 '6=�� %&����� ��! � ��%������� �0

bumpy normal, introducing artifacts for light sourcesestimation or during the rendering process (figure 10(b)and 11). This is why we propose to smooth normalsaccording to neighbor voxels triangles. In our applica-tion, a parameter called

� �;rMr��@oH�Ju � $��� � v$u9�*pis fixed by

the user. Practically, with an octree depth equal to 7,our best results have been obtained with a smoothingdistance set to 3 or 4 voxels (depending on the objectgeometry).

� �(������� ����� ��)�� !������� # %&����� ��! � %� �#���� '-)# ��%&!�� ��'-���� � ���/%-� * '6��'-�()�*: '6=�� .�/� ! = �(*�� ! �$@ �/% ���(��)�� �('�= .#�) � % *�)�''��� ) �$� #�� %�. ) )�'�= %&�$�E�?��!��/% 0Discrete NormalIntuitively, a normal estimated from marching cubesshould be quite representative of the object surface.However, it is also possible to define a normal ineach voxel directly with the discrete surface. Par-ticularly, if a surface mesh is not needed, acceptableresults can be obtained. Normal is estimated accordingto out-voxels in the given neighborhood (figure 12):�� � � ����� �����

��- � ����� , where

��- � corresponds to the unit

vector going from the current voxel center to the

��� �

neighbor voxel center;�

is the number of voxels used.With this method, normals are defined by a fixed num-ber of directions which could be useful for compressionalgorithms.

out−voxel

amb−voxel

in−voxel

current voxel

� �(������� ��� �!� )���.�J� � %-'&��.�/'&�()�* � ��) . ��) �$� ��% 0

7. RESULTSObject ShapeBefore actually using our method with real objects,validation has been done with known geometric objects(a sphere and a cube). As shown in table 1, using imagepixels with marching cubes (MC in the figure) improvesconsequently shape precision. For the experiments wemade, the error is noticeably reduced compared to amarching cubes algorithm with edges centers. Notethat when hierarchy depth increases, the two methodstend to provide the same results because the number ofpixel-rays becomes lower. Our method will obviouslybe more accurate with a low-depth hierarchy.

"$#$%�&('*)�+-,�./,0%�1�23&4+5,*.6,%�798�:;,%6,<>=�?A@�BCBEDFBCG0?�H I J K

LNM6O*P/QERSPUT�V P/W>X(Y Z�[*\3]_^`^ a\3b_^c^ ]�\da!^`^egfih*P/j klW�monoQ>XpY qo\ r-^`^ q*\�s!^c^ qo\tsu^`^

";#4%�&v'*)S+5,�./,%`w!2�1;.x:vyuz9{|,<>=�?A@�BCBEDFBCG0?�H I J K

LNM6O*P/QERSPUT�V P/W>X(Y }0qo\ r-^`^ Z*a\ r-^c^ Zqo\tsu^`^egfih*P/j klW�monoQ>XpY }�Z�\i}~^`^ Z*a\3Z_^c^ Z*]�\tqu^`^

� �J@��(� ��� � ����������� # ��%-'6��* ! � @ � '.- � � * ���/!�)�* ,%-'-� � ! ' ��# ' ������*��/���/%8��*�#D��!�'&�$��� )�@�� �/!�' %&����� ��! ��0

Note that a cube is the worst example. A polygon isdifficult to reconstruct with a shape from silhouette ap-proach (as any flat surface) since the camera viewpointis never perfectly located on the polygon plane.

Normal EstimationThis paper describes two different methods for estimat-ing normal inside each voxel. The first one is basedon marching cubes triangles while the second one onlyrelies on voxels. For each case, it is possible to smoothnormal values according to a user-specified smoothingdistance. For estimating normal quality, we comparedthe estimated normal with the actual (known) surfacenormal (table 2)

As for surface accuracy, a surface normal obtained withthe help of pixel-rays is sensibly more precise (20-25%) than with using edges centers marching cubes or

"$#$%�&('*)�+-,�./,0%�1�23&4+5,*.6,%�798�:;,%6,<>=C?A@�BCB DFBCG0?AH I K

�������x?AH����uD����?� �0=UB � � ��EfdQ�R/W�P/V PcT��oW�^umoj Z*Z�\dZ�� Zo}0\3Z�� }\db��L M/O�PUQERSP/T�V P/W>XpY �\�s � q*\3b � }\dZ �eNf h�PUj klW�m*noQ>X(Y b\3b�� r4\3a�� Z�\d]��

� �J@��(� � � � ����������� ��*$�/�(� % # ��� �����/*�! � F1#���������� %EK@ ��' - ���/* � %-'&��.�/'-� # *�)���.�J�L��*�# ��! '&� ��� )�@�� � ! '*�)���.�J� 0

the discrete surface. Our method provides a precisionwith an error less than 5

�even without any smoothing.

When smoothing, normal precision is about one degree(with a smoothing distance of 5 voxels). Note that withthe discrete surface, the smoothing distance should notbe less than 3 voxels.

Rendering using NormalsFrom geometry and normal we have generated newviews. Triangles can be directly with Graphics Hard-ware (OpenGL). For example, figures 13, 15 and 14show new images.

� !�� � R/�� �(������� � � ��� �/*$#������(*�� � %&��*$� '-� ����*��/�(� %�9B� 0 - �('�=������� ��!�"���#%$&��'�� .�/��!E= ��*$� ! �$@ �/%�9 @ 0 - �('�= � ��� � ��,����,�% . ��� ! =���*���!� @ � % 0

� !�� � R�� � 1��� �(������� � )�� � � *�#���� ��*��1� %&��*�� ��) � ��� *�)���.����- �('�=%�. ) )�'�=���*��$9 �$0 - �('�= �(�)��� ��!�"���#*$&��' � . ��� ! =���*��!�$@ � % *�)�'-� '�=��8@ � . �$, %&�����?��! � )�* )�@�� � ! ' %&� ��=�)�� ,��'-'-�/9�@ 0�- �('�= � ��� � ��, ����,�%� . ��� ! =���*��+!�$@ � % !�)�* ,'-)��$� ��% %�. ) )�'6=�����9$!/0 - �('6=��!�'&�$���$%�� =������ *�)�� . ��� 0Rendering using Voxel RadiancesIt is also possible with pixel-rays (in-rays) to estimatean average radiance emitted by the voxel (figures 16).

� !�� � R��

� 1��� �(��������� � ��� � *�#$� � ��*$��� %&��*�� ��)�� � � *�)�� . ��� � ��* ���) �$� � !����� '6=�� '-� ����*��/�(� % *$)���.����%+����� � � � ����! � #�@$,'�=�� ��)�� � � *$)���.��� 0 �$0 - �('6= ������� ��!�"���#%$&��'�� � ��� !E= ,��*�� !�$@ � %�9 @ 0 - �('6=0� ��� � ��, ����,�%�9�!/0 - �('�= %�. ) )�'�=�� #*�)���.�J�BF1# �(%-'6��* !�� )�� ���) �$� ��%EK&0

8. CONCLUSIONThis paper presents a method for using image pixels to-gether with marching cubes for a shape from silhouetteapplication. Our work relies on a 6-connected discretesurface obtained with the method proposed by Szeliski[18]. We also propose two methods for estimating thenormal inside voxels, using either triangular mesh ordiscrete surface. As seen in the results, our methodproves robust even for a cheap acquisition system withusual camera and turntable.

In the future, we aim at combining the reconstructedinformation with image-based techniques such as ligh-fields/lumigraphs [10, 6] for integrating (real) objectsinto virtual environments with realistic relighting andglobal illumination. This method has already been usedfor estimating light sources positions from images [14].

9. ACKNOWLEDGMENTSWe would like to pay a tribute to Alain Fournier (ImagerLab, University of British Columbia, Vancouver) withwhom the overall framework around this paper hasbeen initiated in 1999. The quad has been designed byStewart Cowley who kindly accepted to let us use thismodel for our work.

10. REFERENCES[1] P. Callet. Rendering of binary alloys. In

�0|~| ��}�Q�e���

, sep 2004.[2] Q. Chen and G. Medioni. A volumetric stereo

matching method: Application to image-based

� !�� � R/�

� 1�� � %�� � ���� �(������� � G � � ������*��/��� ! )/��)�� ! )�� ��� %�� )�*�# %1'-) '�=���������E�/���1�E�J# ����* ! �&� )�� ����!E= ��)�� � � 9+�$0 � )�� � �$�(��,'&�$�J� #$� ��#:9 @ 0 � )�� � ������� !���)�- * F '-)/,�K�9 !/0 �"! ,'&�$�J� � =�)�'-)������ ��=:9�# 0� )�� � ������� - ) )!# ,�� ) - � � 9���0- ) )!# ,�� )�- ��� - �('�= . )!# �(� � # ���(�/= '&��*$��0

modeling. In| �A�i�

, pages 29–34. IEEEComputer Society, 1999.

[3] C H Chien and J K Aggarwal. Volume/surfaceoctrees for the representation ofthree-dimensional objects.

|�r��m�/s ���m�c������r$u}lt�v*�'o/��� ��v&�Dp¡�~tCrM�Vp � ���

, 36(1):100–113, 1986.[4] C. Hernández Esteban and F. Schmitt. Silhouette

and stereo fusion for 3d object modeling. In���m��� �Q���$�, pages 46–53, 2003.

[5] Olivier Faugeras and Renaud Keriven. Completedense stereovision using level set methods.��p�� � sHt�p � r���p � �Jud|�r��¡�9s���pVt�{A��� pVu9�Vp

, 1406:379+,1998.

[6] Steven J. Gortler, Radek Grzeszczuk, RichardSzeliski, and Michael F. Cohen. The lumigraph.� |� |�r��m�9s���pVt"}ltCvV�'oH��� �

, 30(AnnualConference Series):43–54, August 1996.

[7] Jean-Marc Hasenfratz, Marc Lapierre,Jean-Dominique Gascuel, and Edmond Boyer.Real-time capture, reconstruction and insertioninto virtual world of human actors. In

�c������r$u'�����Dp�rYv0u�>}lt�vV�QoH��� �

, pages 49–56. Eurographics,Elsevier, 2003.

[8] D. R. Hougen and N. Ahuja. Adaptivepolynomial modelling of the reflectance map forshape estimation from stereo and shading. In| �A�i�

, pages 991–994, 1994.[9] Kiriakos N. Kutulakos and Steven M. Seitz. A

theory of shape by space carving. TechnicalReport TR692, , 1998.

[10] Marc Levoy and Pat Hanrahan. Lightfieldrendering.

|�r��m�9s���pVt"}ltCvV�'oH��� �, 30(Annual

Conference Series):31–42, August 1996.[11] Marc Levoy, Kari Pulli, Brian Curless, Szymon

Rusinkiewicz, David Koller, Lucas Pereira, MattGinzton, Sean Anderson, James Davis, JeremyGinsberg, Jonathan Shade, and Duane Fulk. Thedigital michelangelo project: 3D scanning oflarge statues. In Kurt Akeley, editor,

{/� �&�$tCvV�'o�Q�e�e�:�j|�r �m�9s���pVt"}lt�v*�'oH��� �¡�lt�rM�*p�p $�Ju����

, pages131–144. ACM Press / ACM SIGGRAPH /Addison Wesley Longman, 2000.

[12] William E. Lorensen and Harvey E. Cline.Marching cubes: A high resolution 3d surfaceconstruction algorithm.

� |� |�r��m�/s ��pVt}lt�v*�'oH��� �

, 21(Annual ConferenceSeries):163–169, July 1987.

[13] W. N. Martin and J. K. Aggarwal. Volumetricdescriptions of objects from multiple views.�����n/tCv$u��Vve� � � r$u��Br$uX�v�����pVt*u � uAv$w ��������v0u��Zve�*oH�@u p��*u/��p�wJw � �DpVu9�Vp

, 5(2):150–158, March1983.

[14] B. Mercier and D. Meneveaux. Joint estimationof multiple light sources and reflectance fromimages. In

�0|~| ��}O�Q�e�V�, sep 2004.

[15] S. Seitz and C. Dyer. Photorealistic scenereconstruction by voxel coloring, 1997.

[16] Hemant Singh and Rama Chellappa. Animproved shape from shading algorithm.Technical Report CS-TR-3218, Department ofComputer Science, University of MarylandCenter for Automation Research, College Park,MD, February 1994.

[17] G. Slabaugh, B. Culbertson, T. Malzbender, andR. Schafer. A survey of methods for volumetricscene reconstruction from photographs. In��}j�e�

, pages 81–100, 2001.[18] Richard Szeliski. Rapid octree construction from

image sequences. In| ��}i�C�� �� �;v&�Dp

��u�DpVt � � v0u�$�Ju �, volume 1, pages 23–32, July

1993.