scattering tomography with path integral

Scattering tomography with path integral

Toru%Tamaki%(Hiroshima%University)%%%

Joint%work%with%Bingzhi%Yuan,%Yasuhiro%Mukaigawa,%

and%many%others%

Computed Tomography

Mul`plexing%Radiography%for%UltraVFast%Computed%Tomography%J.%Zhang,%G.%Yang,%Y.%Lee,%S.%Chang,%J.%Lu,%O.%Zhou%University%of%North%Carolina%at%Chapel%Hill%h6ps://www.aapm.org/mee`ngs/07AM/VirtualPressRoom/LayLanguage/IIMul`plexing.asp%

h6p://www.sciencekids.co.nz/pictures/humanbody/braintomography.html

Different ModalitiesCT%:%h6p://www.hiroshimaVu.ac.jp/news/show/id/13475/dir_id/19%PET%:%h6p://www.hiroshimaVu.ac.jp/hosp/hitokuchi/p_vdo8u6.html%%MRI%:%h6p://www.hiroshimaVu.ac.jp/news/show/id/14427/dir_id/19%

CT%(X%Ray)

MRI%(Magne`c%field)

PET%(Positron)

%(infrared)

Diffuse Optical Tomography (DOT)

,% − −% 19 9 5 ,% ,% 3 ,%3% ,%

%h6p://www.mext.go.jp/b_menu/shingi/gijyutu/gijyutu3/toushin/07091111/008.htm

Input%Light%pulse%

Detector%

Detector%Time%

Time%

Solving%PDE%by%FEM%under%diffuse%approxima`on%with%boundary%condi`on%

Lee,%Kijoon.%“Op`cal%Mammography:%Diffuse%Op`cal%Imaging%of%Breast%Cancer.”%World%Journal%of%Clinical%Oncology%2.1%(2011):%64–72.%PMC.%Web.%8%June%2015.%h6p://dx.doi.org/10.5306/wjco.v2.i1.64%

Our research targetX"ray&CT Our&target DOT

No%Radia`on

Sharpness%%%%%

( )

Sca6ering No Yes So%much

Method Radon%trans. Ours PDE%by%FEM

h6p://www.sciencekids.co.nz/pictures/humanbody/braintomography.htmlLee,%Kijoon.%“Op`cal%Mammography:%Diffuse%Op`cal%Imaging%of%Breast%Cancer.”%World%Journal%of%Clinical%Oncology%2.1%(2011):%64–72.%PMC.%Web.%8%June%2015.%h6p://dx.doi.org/10.5306/wjco.v2.i1.64%

h6p://photosozaiVdatabase.com/modules/photo/photo.php?lid=664%

Sca6ering

Volumetric%sca6ering%in%CG

1987

1993

1994

Tomoyuki%Nishita,%Yasuhiro%Miyawaki,%and%Eihachiro%Nakamae.%1987.%A%shading%model%for%atmospheric%sca6ering%considering%luminous%intensity%distribuòn%of%light%sources.%SIGGRAPH(Comput.(Graph.%21,%4%(August%1987),%303V310.%DOI=10.1145/37402.37437%h6p://doi.acm.org/10.1145/37402.37437%

Tomoyuki%Nishita,%Takao%Sirai,%Katsumi%Tadamura,%and%Eihachiro%Nakamae.%1993.%Display%of%the%earth%taking%into%account%atmospheric%sca6ering.%In%Proceedings(of(the(20th(annual(conference(on(Computer(graphics(and(interac>ve(techniques%(SIGGRAPH%'93).%ACM,%New%York,%NY,%USA,%175V182.%DOI=10.1145/166117.166140%h6p://doi.acm.org/10.1145/166117.166140%

Tomoyuki%Nishita%and%Eihachiro%Nakamae.%1994.%Method%of%displaying%op`cal%effects%within%water%using%accumulaòn%buffer.%In%Proceedings(of(the(21st(annual(conference(on(Computer(graphics(and(interac>ve(techniques%(SIGGRAPH%'94).%ACM,%New%York,%NY,%USA,%373V379.%DOI=10.1145/192161.192261%h6p://doi.acm.org/10.1145/192161.192261%

Copyright%©%2011%by%the%Associaòn%for%Compu`ng%Machinery,%Inc.%(ACM).%

Tomoyuki%Nishita,%Yoshinori%Dobashi,%and%Eihachiro%Nakamae.%1996.%Display%of%clouds%taking%into%account%mul`ple%anisotropic%sca6ering%and%sky%light.%In%Proceedings(of(the(23rd(annual(conference(on(Computer(graphics(and(interac>ve(techniques%(SIGGRAPH%'96).%ACM,%New%York,%NY,%USA,%379V386.%DOI=10.1145/237170.237277%h6p://doi.acm.org/10.1145/237170.237277%

Yonghao%Yue,%Kei%Iwasaki,%BingVYu%Chen,%Yoshinori%Dobashi,%and%Tomoyuki%Nishita.%2010.%Unbiased,%adap`ve%stochas`c%sampling%for%rendering%inhomogeneous%par`cipa`ng%media.%ACM(Trans.(Graph.%29,%6,%Ar`cle%177%(December%2010),%8%pages.%In%ACM(SIGGRAPH(Asia(2010(papers%(SIGGRAPH%ASIA%'10).%DOI=10.1145/1882261.1866199%h6p://doi.acm.org/10.1145/1882261.1866199%

Volumetric%sca6ering%in%CG

1996

2010

Copyright%©%2011%by%the%Associa`on%for%Compu`ng%Machinery,%Inc.%(ACM).%

Computer vision approach to the inverse problem

Observed%image

Forward%model%Generated%by%Computer%Graphics%

Image Computer%Graphics

Generation Analysis

Computer%Vision

Model%parameters%

2

minimizing%observa`ons%and%model%predic`on

Vision / CG approaches

Stochastic Tomographyand its Applications in 3D Imaging of Mixing Fluids

James Gregson⇤ Michael Krimerman Matthias B. Hullin Wolfgang Heidrich

University of British Columbia

Figure 1: Left: Photo of an acquisition rig for fluid phenomena, consisting of 5–16 strobe-synchronized consumer cameras. Middle:Example capture of one of the cameras. Right: Reconstruction of an unsteady two-phase flow at different points in time. Note how the noveltomographic technique introduced in this paper manages to capture the fine-scale features in this challenging dataset. SAD regularizer, 100Msample mutations/frame, 6 min/frame.

Abstract

We present a novel approach for highly detailed 3D imaging of tur-bulent fluid mixing behaviors. The method is based on visible lightcomputed tomography, and is made possible by a new stochastictomographic reconstruction algorithm based on random walks. Weshow that this new stochastic algorithm is competitive with special-ized tomography solvers such as SART, but can also easily includearbitrary convex regularizers that make it possible to obtain high-quality reconstructions with a very small number of views. Finally,we demonstrate that the same stochastic tomography approach canalso be used to directly re-render arbitrary 2D projections withoutthe need to ever store a 3D volume grid.

CR Categories: I.3.3 [COMPUTER GRAPHICS]: Picture/ImageGeneration—Digitizing and scanning;

Keywords: Stochastic sampling, Tomography, Fluid Imaging

Links: DL PDF WEB VIDEO DATA

⇤{jgregson|krim|hullin|heidrich}@cs.ubc.ca

1 Introduction

The capture of dynamic 3D phenomena has been the subject of con-siderable research in computer graphics, extending to both the scan-ning of deformable shapes, such as human bodies (e.g. [de Aguiaret al. 2007; de Aguiar et al. 2008]), faces (e.g. [Bickel et al. 2007;Alexander et al. 2009]), and garments (e.g. [White et al. 2007;Bradley et al. 2008]), as well as natural phenomena including liquidsurfaces [Ihrke and Magnor 2004; Wang et al. 2009], gases [Atch-eson et al. 2008], and flames [Ihrke and Magnor 2004]. Accessto this kind of data is not only useful for direct re-rendering, butalso for deepening our understanding of a specific phenomenon.The data can be used to derive heuristic or data-driven models, orsimply to gain a qualitative understanding of what a phenomenonshould look like before simulating it.

In this paper we focus on the capture of mixing processes betweentwo liquids, as well as the dissolving of powdered dye into liq-uid. Like several other recent works [Ihrke and Magnor 2004; Ihrkeet al. 2005; Atcheson et al. 2008], our capture process is based onvisible light computed tomography (CT), which allows us to useinexpensive, off-the-shelf camera arrays. However, in addition todevising an effective capture setup for this specific problem, wealso make significant algorithmic improvements to tomographic re-construction techniques in general. In particular, we develop a newstochastic tomography algorithm that is especially well suited forthe types of application scenarios encountered in graphics. Such ap-plications typically have a relatively sparse set of views comparedto more traditional settings for CT, such as medical imaging.

To our knowledge, Stochastic Tomography is the first CT algorithmthat is both matrix-free and, by default, also gridless. Neither the to-mography matrix nor the final volume grid need to be stored, whichenables the reconstruction of larger, more detailed volumes thanwould be feasible with traditional methods. Moreover, we show

Stochastic Tomographyand its Applications in 3D Imaging of Mixing Fluids

James Gregson⇤ Michael Krimerman Matthias B. Hullin Wolfgang Heidrich

University of British Columbia

Figure 1: Left: Photo of an acquisition rig for fluid phenomena, consisting of 5–16 strobe-synchronized consumer cameras. Middle:Example capture of one of the cameras. Right: Reconstruction of an unsteady two-phase flow at different points in time. Note how the noveltomographic technique introduced in this paper manages to capture the fine-scale features in this challenging dataset. SAD regularizer, 100Msample mutations/frame, 6 min/frame.

Abstract

We present a novel approach for highly detailed 3D imaging of tur-bulent fluid mixing behaviors. The method is based on visible lightcomputed tomography, and is made possible by a new stochastictomographic reconstruction algorithm based on random walks. Weshow that this new stochastic algorithm is competitive with special-ized tomography solvers such as SART, but can also easily includearbitrary convex regularizers that make it possible to obtain high-quality reconstructions with a very small number of views. Finally,we demonstrate that the same stochastic tomography approach canalso be used to directly re-render arbitrary 2D projections withoutthe need to ever store a 3D volume grid.

CR Categories: I.3.3 [COMPUTER GRAPHICS]: Picture/ImageGeneration—Digitizing and scanning;

Keywords: Stochastic sampling, Tomography, Fluid Imaging

Links: DL PDF WEB VIDEO DATA

⇤{jgregson|krim|hullin|heidrich}@cs.ubc.ca

1 Introduction

The capture of dynamic 3D phenomena has been the subject of con-siderable research in computer graphics, extending to both the scan-ning of deformable shapes, such as human bodies (e.g. [de Aguiaret al. 2007; de Aguiar et al. 2008]), faces (e.g. [Bickel et al. 2007;Alexander et al. 2009]), and garments (e.g. [White et al. 2007;Bradley et al. 2008]), as well as natural phenomena including liquidsurfaces [Ihrke and Magnor 2004; Wang et al. 2009], gases [Atch-eson et al. 2008], and flames [Ihrke and Magnor 2004]. Accessto this kind of data is not only useful for direct re-rendering, butalso for deepening our understanding of a specific phenomenon.The data can be used to derive heuristic or data-driven models, orsimply to gain a qualitative understanding of what a phenomenonshould look like before simulating it.

In this paper we focus on the capture of mixing processes betweentwo liquids, as well as the dissolving of powdered dye into liq-uid. Like several other recent works [Ihrke and Magnor 2004; Ihrkeet al. 2005; Atcheson et al. 2008], our capture process is based onvisible light computed tomography (CT), which allows us to useinexpensive, off-the-shelf camera arrays. However, in addition todevising an effective capture setup for this specific problem, wealso make significant algorithmic improvements to tomographic re-construction techniques in general. In particular, we develop a newstochastic tomography algorithm that is especially well suited forthe types of application scenarios encountered in graphics. Such ap-plications typically have a relatively sparse set of views comparedto more traditional settings for CT, such as medical imaging.

To our knowledge, Stochastic Tomography is the first CT algorithmthat is both matrix-free and, by default, also gridless. Neither the to-mography matrix nor the final volume grid need to be stored, whichenables the reconstruction of larger, more detailed volumes thanwould be feasible with traditional methods. Moreover, we show

0◦ reconstructed view (30◦) reconstructed view (60◦) 90◦input multiplication flame sheet blobs multiplication flame sheet blobs input

Figure 7. Views of three different reconstructions of the “torch” dataset. The same two input images were used for all methods.Since all methods reconstruct the input views, only novel views of the reconstructed flame are shown.

reconstruction method (views) RMS errorinput all

flame sheet solution (2) 0 26.6multiplication solution (2) 0 21.5blob-based method (3) 13.3 19.0flame sheet decomposition (3) 10.1 18.4flame sheet decomposition (3/7) 11.5 15.8

Table 1. Per-pixel RMS image reconstruction error for dif-ferent algorithms applied to the “jet” dataset. For the lastentry, three input views were used for basis generation andseven for optimizing photo-consistency. Note that the firsttwo reconstructions reproduce the input views exactly.

0 100 200 300 400 50015

20

25

30

35

number of decomposed flame sheets

RM

S er

ror p

er p

ixel

input viewsall views

Figure 8. Dependence of image error on the number B ofgenerated basis fields (Section 5.2, Step 1).

7. Concluding Remarks

A current limitation of our multi-view technique is theplanar configuration of the input viewpoints. While pla-narity is used for efficiency reasons, it may be possibleto derive an efficient reconstruction method that combinesFlame Sheets from non-planar views.More generally, the question of how best to capture the

global 3D structure and dynamics of fire remains open. To-ward this goal, we are investigating new spatio-temporal co-herence constraints and are studying ways to integrate ourapproach with traditional fire simulation methods.Finally, while our limited experiments suggest that

Flame Sheets and the Flame-Sheet Decomposition Algo-rithm are useful for fire reconstruction, these tools may also

prove useful in more general contexts. This includes sparse-view tomography problems in medical diagnostics [17] andaccelerated image-based methods for volume rendering.

References

[1] D. Nguyen, R. Fedkiw, and H. Jensen, “Physically based modelingand animation of fire,” in Proc. SIGGRAPH, pp. 721–728, 2002.

[2] A. Lamorlette and N. Foster, “Structural modeling of flames for aproduction environment,” in Proc. SIGGRAPH, pp. 729–735, 2002.

[3] J. Stam and E. Fiume, “Depicting fire and other gaseous phenomenausing diffusion processes,” in Proc. SIGGRAPH, pp. 129–136, 1995.

[4] N. Chiba, K. Muraoka, H. Takahashi, and M. Miura, “Two-dimensional visual simulation of flames, smoke and the spread offire,” J. Visualiz. and Comp. Anim., vol. 5, no. 1, pp. 37–54, 1994.

[5] R. Fedkiw, J. Stam, and H. W. Jensen, “Visual simulation of smoke,”in Proc. SIGGRAPH, pp. 15–22, 2001.

[6] N. Foster and D. N. Metaxas, “Modeling the motion of a hot, turbu-lent gas,” in Proc. SIGGRAPH, pp. 181–188, 1997.

[7] R. T. Baum, K. B. McGratten, and M. Nyden, “An examinationof the applicability of computed tomography for the measurementof component concentrations in fire-generated plumes,” Combustionand Flame, vol. 113, pp. 358–372, 1998.

[8] A. Schwarz, “Multi-tomographic flame analysis with a schlieren ap-paratus,” Measurement Science & Tech., vol. 7, pp. 406–413, 1996.

[9] D. P. Correia, P. Ferrao, and A. Caldeira-Pires, “Advanced 3D emis-sion tomography flame temperature sensor,” Combustion Science andTechnology, vol. 163, pp. 1–24, 2001.

[10] W. Xue, W. Donglou, and P. Gongpei, “Use of Moire tomography tomeasure the temperature field of the flame of a pyrotechnical compo-sition from its infrared radiation,” Combustion, Explosion, and ShockWaves, vol. 37, no. 4, pp. 440–442, 2001.

[11] G. W. Faris and R. L. Byer, “Beam-deflection optical tomography,”Optics Letters, vol. 12, no. 3, pp. 72–74, 1987.

[12] T. Otsuka and P. Wolanski, “Particle image velocimetry (PIV) anal-ysis of flame structure,” Journal of Loss Prevention in the ProcessIndustries, vol. 14, no. 6, pp. 503–507, 2001.

[13] A. K. Agrawal, N. Butuk, S. R. Gollahalli, and D. Griffin, “Three-dimensional rainbow schlieren tomography of a temperature field ingas flows,” Applied Optics, vol. 37, no. 3, pp. 479–485, 1998.

[14] A. Schodl, R. Szeliski, D. H. Salesin, and I. Essa, “Video textures,”in Proc. SIGGRAPH, pp. 489–498, 2000.

[15] D. Drysdale, An Introduction to Fire Dynamics. Chichester: JohnWiley and Sons, Second ed., 1998.

[16] P. J. Narayanan, P. W. Rander, and T. Kanade, “Constructing virtualworlds using dense stereo,” in Proc. ICCV, pp. 3–10, June 1998.

[17] A. C. Kak and M. Slaney, Principles of Computerized TomographicImaging. New York: IEEE Press, 1988.

[18] T. Frese, C. A. Bouman, and K. Sauer, “Adaptive wavelet graphmodel for Bayesian tomographic reconstruction,” IEEE Transactionson Image Processing, vol. 11, pp. 756–770, July 2002.

[19] G. T. Herman and A. Kuba, eds.,Discrete Tomography: Foundations,Algorithms, and Applications, (Boston), Birkhauser, 1999.






0 100 200 300 400 50015

20

25

30

35


RM

S er

ror p

er p

ixel








References

























0 100 200 300 400 50015

20

25

30

35


RM

S er

ror p

er p

ixel








References

























0 100 200 300 400 50015

20

25

30

35


RM

S er

ror p

er p

ixel








References




















90°0° 30° 60°

input inputreconstructed

Gregson+%2012TOG%

Hasinoff%and%Kutulakos%2007PAMI%

No%sca6ering

Figure 2: Eight images captured during a single sweep of the laser plane.

off the mirror, the laser is spread into a vertical sheet of light by thecylindrical lens. This sheet of light creates an illuminated plane ofsmoke, which the camera views from a perpendicular direction.

During scanning, the galvanometer sweeps the laser sheet fromthe front to the back of the smoke volume, while the camera cap-tures images of the scattered light. The galvanometer then rapidlyjumps back to the front of the smoke volume and the process is re-peated. In this work we capture 200 images during each volumescan. At 5000 frames per second, this allows the volume to bescanned at 25 Hz. We placed both the camera and galvanometer 3meters from the working volume, and chose the camera lens, cylin-drical lens, and galvanometer scan angle so that the camera fieldof view and laser frustum were both 6�. This provided a workingvolume of approximately 30x30x30 cm.

The galvanometer control software also triggers the camera, al-lowing the frame capture to be synchronized with the position ofthe laser slice. This assures that corresponding slices in successivecaptured volumes are sampled at the same position.

The high frame rate and the small fraction of the total light scat-tered to the camera require a relatively powerful laser. We use anargon ion laser that produces 3 watts of light at a set of wavelengthsclosely clustered around 500 nm.

3.1 Calibration

Interpreting the image data as a density field requires knowing thegeometry and radiometry of the camera and lighting setup. We usea planar checkerboard pattern to calibrate the pose and intrinsic pa-rameters of the camera using the technique of [Zhang 2000].

(a) (b)

Figure 3: (a) Filter with horizontal lines used for characterizinglight volume. (b) Captured image of laser through filter.

We calibrate the position and intensity of the laser, as well as theposition of the sequential laser planes Ps, by placing a pattern ofhorizontal lines between the cylindrical lens and the working vol-ume. This changes the vertical slice of light into a series of dots.We then place a checkerboard at an angle such that it is illuminatedby the laser and also visible to the camera. Because the positionof the checkerboard is determined by the camera calibration pro-cedure, we can then determine the position of a given dot of lightin a given frame corresponding to one position of the laser slice.Moving the checkerboard to another position allows the same dotof light to be observed at a different 3D position. Connecting thesedots then gives a line which passes through the nodal point Lc of

the laser scanner. Repeating this for another dot gives another suchline, and intersecting the two lines determines Lc.

Similarly, any two identified points on the laser line for a givenslice s, together with Lc, determines the plane Ps of illumination.

We observed that the intensity of illumination varied as a func-tion of slice s and of elevation angle f within each laser slice. Tocorrect for this, we placed a uniform diffuse plane at an angle whereit was visible to both the laser and the camera and captured a se-quence of frames. As the laser scans this reflectance standard, theobserved intensities are directly proportional to the intensity of laserlight for each laser plane and pixel position. Having calibrated thelaser planes Ps and the camera position, we can find the 3D pointv corresponding to each laser plane and pixel position, and applythe appropriate intensity compensation when computing densitiesalong the ray from the laser position Lc through the point v. Al-though this corrects for the relative intensity differences across thelaser frustum, we should note that we do not measure the absoluteintensity of the laser. The resulting density fields are therefore onlyrecovered up to an unknown scale factor.

We have assumed that image intensity values can be directly in-terpreted as density values. This assumes that the effects of extinc-tion and multiple scattering within the volume are negligible. Al-though we found that this assumption held for the smoke volumeswe captured, this assumption will fail when the densities becomevery high such as for thick smoke or dyes in water, or when thescattering volume becomes large.

4 Measuring scattering parameters

To render a captured density volume accurately, we must also knowthe basic scattering characteristics of the medium. In particular wemust know the phase function, which describes the angular distribu-tion of scattered light, and the albedo, which describes the tendencyof the medium to scatter light rather than absorb it.

To measure these quantities, we use the setup shown in Figure 4.Smoke is confined within a transparent tank. A fan circulates the airto maintain homogeneity. A spherical glass observation chamber isadded at the top.

4.1 Phase Function

The phase function of a scattering medium describes the distribu-tion of outgoing scattered light per scattering event. It is assumedto be a function only of the angle between the direction of the in-coming light and the direction of scattered light, or phase angle.

We developed a novel apparatus to measure the phase functionfrom a single photograph. We first fill the tank and observationchamber with smoke. Relatively thin smoke is used to minimize theeffect of multiple scattering. We then direct the laser beam throughthe spherical observation chamber. A strip of a 45 degree conicalmirror encircles the observation chamber, as seen in Figure 5(a).Two small holes are drilled through the conical mirror to allow thelaser to pass through it into the observation chamber, and to exitwithout creating unwanted reflections or stray light.

Smoke%onlyHawkins+%2005TOG%An Inverse Problem Approach for Automatically Adjusting the Parameters for

Rendering Clouds Using Photographs

Yoshinori Dobashi1,2 Wataru Iwasaki1 Ayumi Ono1 Tsuyoshi Yamamoto1 Yonghao Yue3 Tomoyuki Nishita31Hokkaido University 2JST, CREST 3The University of Tokyo

(a) (b) (c)

(d) (e) (f)

Figure 1: Examples of our method. The parameters for rendering clouds are estimated from the real photograph shown in the small inset atthe top left corner of each image. The synthetic cumulonimbus clouds are rendered using the estimated parameters.

Abstract

Clouds play an important role in creating realistic images of out-door scenes. Many methods have therefore been proposed for dis-playing realistic clouds. However, the realism of the resulting im-ages depends on many parameters used to render them and it isoften difficult to adjust those parameters manually. This paper pro-poses a method for addressing this problem by solving an inverserendering problem: given a non-uniform synthetic cloud densitydistribution, the parameters for rendering the synthetic clouds areestimated using photographs of real clouds. The objective func-tion is defined as the difference between the color histograms ofthe photograph and the synthetic image. Our method searches forthe optimal parameters using genetic algorithms. During the searchprocess, we take into account the multiple scattering of light insidethe clouds. The search process is accelerated by precomputing a setof intermediate images. After ten to twenty minutes of precomputa-tion, our method estimates the optimal parameters within a minute.

CR Categories: I.3.7 [Computer Graphics]: Three-DimensionalGraphics and Realism; I.3.3 [Computer Graphics]: Picture/ImageGeneration;

Keywords: clouds, rendering parameters, inverse problem

Links: DL PDF

1 Introduction

Clouds are important elements when synthesizing images of out-door scenes to enhance realism. A volumetric representation is of-ten employed and the intensities of the clouds are computed takinginto account the scattering and absorption of light in order to displayrealistic clouds. However, one of the problems is that the quality ofthe rendered image depends on many parameters, which need to beadjusted manually by rendering the clouds repeatedly with differentparameter settings. This is not an easy task since the relationshipbetween the resulting appearance of the clouds and the parametersis highly nonlinear. The expensive computational cost for the ren-dering process makes this more difficult. This paper focuses onautomatic adjustment of the parameters to address this task.

Recently, many real-time methods have been proposed for editingthe parameters used in rendering images [Harris and Lastra 2001;Bouthors et al. 2008; Zhou et al. 2008]. These methods are fast so-

Dobashi+%2012TOG%

Narasimhan+%2006SIGGRAPH

Gkioulekasj+%2013SIGGRAPHAsia%

Appears in the SIGGRAPH Asia 2013 Proceedings.

Inverse Volume Rendering with Material Dictionaries

Ioannis GkioulekasHarvard University

Shuang ZhaoCornell University

Kavita BalaCornell University

Todd ZicklerHarvard University

Anat LevinWeizmann Institute

Figure 1: Acquiring scattering parameters. Left: Samples of two materials (milk, blue curacao) in glass cells used for acquisition. Middle:Samples illuminated by a trichromatic laser beam. The observed scattering pattern is used as input for our optimization. Right: Renderingof materials in natural illumination using our acquired material parameter values.

AbstractTranslucent materials are ubiquitous, and simulating their appear-ance requires accurate physical parameters. However, physically-accurate parameters for scattering materials are difficult to acquire.We introduce an optimization framework for measuring bulk scat-tering properties of homogeneous materials (phase function, scat-tering coefficient, and absorption coefficient) that is more accurate,and more applicable to a broad range of materials. The optimizationcombines stochastic gradient descent with Monte Carlo renderingand a material dictionary to invert the radiative transfer equation. Itoffers several advantages: (1) it does not require isolating single-scattering events; (2) it allows measuring solids and liquids that arehard to dilute; (3) it returns parameters in physically-meaningfulunits; and (4) it does not restrict the shape of the phase functionusing Henyey-Greenstein or any other low-parameter model. Weevaluate our approach by creating an acquisition setup that col-lects images of a material slab under narrow-beam RGB illumina-tion. We validate results by measuring prescribed nano-dispersionsand showing that recovered parameters match those predicted byLorenz-Mie theory. We also provide a table of RGB scattering pa-rameters for some common liquids and solids, which are validatedby simulating color images in novel geometric configurations thatmatch the corresponding photographs with less than 5% error.

CR Categories: I.3.7 [Computer Graphics]: Three-DimensionalGraphics and Realism—Raytracing;

Keywords: scattering, inverse rendering, material dictionaries

Links: DL PDF WEB

1 Introduction

Scattering plays a critical role in the appearance of most materials.Much effort has been devoted to modeling and simulating its visualeffects, giving us precise and efficient scattering simulation algo-rithms. However, these algorithms produce images that are only asaccurate as the material parameters given as input. This creates aneed for acquisition systems that can faithfully measure the scatter-ing parameters of real-world materials.

Collecting accurate and repeatable measurements of scattering isa significant challenge. For homogeneous materials—which is theprimary topic of this paper—scattering at any particular wavelengthis described by two scalar values and one angular function. Thescattering coefficient �

s

and absorption coefficient �a

represent thefractions of light that are scattered and absorbed, and the phasefunction p(✓) describes the angular distribution of scattering. Mea-surement is difficult because a sensor almost always observes thecombined effects of many scattering and absorption events, andthese three factors cannot be easily separated. Indeed, for deeply-scattering geometries, similarity theory [Wyman et al. 1989] provesthat one can analytically derive distinct parameter-sets that nonethe-less produce indistinguishable images.

Most existing acquisition systems address the measurement chal-lenge using a combination of two strategies (e.g., [Hawkins et al.2005; Narasimhan et al. 2006; Mukaigawa et al. 2010]). First, theymanipulate lighting and/or materials to isolate single-scattering ef-fects; and second, they “regularize” the recovered scattering param-eters by relying on a low-parameter phase function model, such asthe Henyey-Greenstein (HG) model. These approaches can provideaccurate results, but both of the employed strategies have severelimitations. The single-parameter HG model limits applicability tomaterials that it represents well; and this excludes some commonnatural materials [Gkioulekas et al. 2013]. Meanwhile, isolatingsingle scattering relies on either: (a) diluting the sample [Hawkinset al. 2005; Narasimhan et al. 2006], which cannot be easily ap-plied to solids or to liquids that have unknown dispersing media;or, (b) using structured lighting patterns [Mukaigawa et al. 2010],which provide only approximate isolation [Holroyd and Lawrence2011; Gupta et al. 2011] and therefore induce errors in measuredscattering parameters that are difficult to characterize.

1

Uniform%parameter%es`ma`on

INTRODUCTION TO SCATTERING

Scattering model

•  Volumetric%sca6ering

Par`cipa`ng%medium%(fog,%cloud,%smoke,%etc)%

Types of Phase function

Forward%sca6ering

Backward%sca6ering

Phase%func`on

Isotropic%sca6ering%

input

Output%distribu`on

Scattering model

Phase%func`on

Rendering for CG

the%line%of%sight

screen

Numerical%integra`on%along

Light in participating medium

A6enua`on

Absorp`on

OutVsca6ering

InVsca6ering

Mul`ple%sca6ering

Emission

Usually%ignored

Absorption

(Microscopic)%BeerVLambert%Law

Absorpòn%coefficient% Infinitesimal%path%length%

(Macroscopic)%BeerVLambert%Law

Integraòn%from%%0%to%d

or

Exponenàl%a6enuaòn When%σa%is%not%constant

op`cal%depth%

h6p://www.sciencekids.co.nz/pictures/humanbody/braintomography.html

CT:%es`ma`ng%%XVRay%absorpòn

Out-scattering

(Microscopic)%BeerVLambert%Law

Sca6ering%coefficient% Infinitesimal%path%length%

(Macroscopic)%BeerVLambert%Law

Integraòn%from%%0%to%d

or

Exponenàl%a6enuaòn

Attenuation = absorption + out-scattering

Absorpòn

OutVsca6ering

A6enuaòn

Ex`ncòn%coefficient%Exponenàl%a6enuaòn

Emission

Emission

Absorp`on%coefficient%(Emission%of%absorbed%energy)%

Emi6ed%light%

In-scattering

Phase%func`on%(from%ω’%to%ω%at%x)%

(usually%common%for%all%x)

Incident%light%(from%ω’%at%x%)

Integra`ng%all%incoming%lights%over%the%sphere

or

Rendering equation

Light%transport%equaòn%Volume%rendering%equaòn%(differenàl%form)

InVsca6ering%termA6enuaòn%term

DOT with Diffuse Approximation

InVsca6ering%termA6enuaòn%termTimeVresolved%observaòn

PDE%solved%by%FEM

No%forward%sca6ering

Blurred%results

Problem

Diffuse%Approximaòn%(first%order%spherical%harmonics%approximaòn)

h6p://commons.wikimedia.org/wiki/File:Harmoniki.png

Main assumption

Rendering equation

Light%transport%equaòn%Volume%rendering%equaòn%(integral%form)

InVsca6ering%term

A6enuaòn%term

A6enuaòn%of%sca6ered%light

Light Paths: from light sources to cameras

Light%source

Camera%/%eye

Path (Ray) tracing: from cameras to light source

Light%source

Camera%/%eye

Paths

Path integral approachintroduced%by%Eric%Veach%(1997%Ph.D%thesis)

h6p://graphics.stanford.edu/papers/veach_thesis/

Pixel%value%of%pixel%j

Paths%of%length%kAll%possible%paths%of%length%k 2

Path%measureProduct%of%measures%at%each%vertex%

All%possible%paths%of%any%length% (path%space)

Monte%Carlo%Integra`on

Path integral for participating mediaintroduced%by%Mark%Pauly%(1999%Master%thesis)

h6p://graphics.stanford.edu/~mapauly/publica`ons.html

Pixel%value%of%pixel%j

measureArea%measure%if%surface

Voxel%measure%if%volume

Metropolis%Light%Transport%for%Par`cipa`ng%Media%Mark%Pauly,%Thomas%Kollig,%Alexander%Keller%Eurographics%Workshop%on%Rendering,%2000%

Details of paths in participating mediaAnders%Wang%Kristensen,%Efficient%Unbiased%Rendering%using%Enlightened%Local%Path%Sampling,%Ph.D%thesis,%2010%URL%

A6enuaòn

Geometric%termBRDF%/%sca6ering%coeff,%phase%funcòn

surface

inside

Cos%term

All%paths%of%length%k%

measure

Camera%response%funcòn

To%be%es`mated!

PROPOSED METHOD TO THE INVERSE PROBLEM

Discretization. How?Ex`nc`on%coefficient%at%voxel%b

Length%that%the%path%xiV>xi+1%passes%voxel%b%Z 1

0�

t

((1� s)xi

+ sx

i+1)ds ⇡X

b

�

t

[b] dxi,xi+1 [b] = �

t

· dxi,xi+1

Z 1

0�

t

((1� s)xi

+ sx

i+1)ds ⇡X

b

�

t

[b] dxi,xi+1 [b] = �

t

· dxi,xi+1

Z 1

0�

t

((1� s)xi

+ sx

i+1)ds ⇡X

b

�

t

[b] dxi,xi+1 [b] = �

t

· dxi,xi+1

%vectorize

Sparse%vector%of%coefficients

Sparse%vector%of%path%length

Inner%product%of%sparse%vectors

Discretization. How?Ex`nc`on%coefficient%at%voxel%b

Length%that%the%path%xiV>xi+1%passes%voxel%b%Z 1

0�

t

((1� s)xi

+ sx

i+1)ds ⇡X

b

�

t

[b] dxi,xi+1 [b] = �

t

· dxi,xi+1

I =1X

k=1

X

⌦k

Le(x0, x1)e

��t

·dx0,x1

kx0 � x1k2"k�1Y

i=1

�s(xi)fp(xi�1, xi, xi+1)e

��t

·dx

i

,x

i+1

kxi � xi+1k2

#

We(xk�1, xk)dµk

I =1X

k=1

X

⌦k

L

e

(x0, x1)e

�R 10 �t((1�s)x0+sx1)ds

kx0 � x1k2"k�1Y

i=1

�

s

(xi

)fp

(xi�1, xi

, x

i+1)e

�R 10 �t((1�s)xi+sxi+1)ds

kxi

� x

i+1k2

#

W

e

(xk�1, xk

)dµk

Z 1

0�

t

((1� s)xi

+ sx

i+1)ds ⇡X

b

�

t

[b] dxi,xi+1 [b] = �

t

· dxi,xi+1

Z 1

0�

t

((1� s)xi

+ sx

i+1)ds ⇡X

b

�

t

[b] dxi,xi+1 [b] = �

t

· dxi,xi+1

before

a|er

Addition of Vectorized paths

I =1X

k=1

Le(x�1, x0)X

⌦k"

k�1Y

i=1

�s(xi)fp(xi�1, xi, xi+1)

#"e

��t

·(P

k�1i=0 d

x

i

,x

i+1 )

Qk�1i=0 kxi � xi+1k2/dVi

#dA�1 dVk dAk+1

I =1X

k=1

Le(x�1, x0)X

⌦k

Hk e��t·Dk

I =Le(x�1, x0)X

⌦

Hk e��t·Dk

I =1X

k=1

X

⌦k

Le(x0, x1)e

��t

·dx0,x1

kx0 � x1k2"k�1Y

i=1

�s(xi)fp(xi�1, xi, xi+1)e

��t

·dx

i

,x

i+1

kxi � xi+1k2

#

We(xk�1, xk)dµk

Assumed%to%be%known

Everything%else

The forward model

x_kV1

x_k

x1

x0

Source%locaòn%s%

Is,t =Le

X

⌦

Hk(s,t) e��t·Dk(s,t)

Observaòn%locaòn%t%

Observaòn%at%s,t

x_kV1

x_k

x1

x0

Source%locaòn%s%


x_kV1

x_k

x1

x0

Source%locaòn%s%


The inverse problem

�t � 0subject%to

Coefficient%must%be%posi`vemodelobservaòn4.2.2 Constrained problem: Interior point

Here we introduce a constrained optimization with inequality constraints of the form;

min

�t

f0(�t

) subject to fi

(x) 0, i = 1, · · · , m, (37)

where �t

2 <N⇥M is a real vector and f0, · · · , fm

: <N⇥M ! < are twice continuously differen-tiable.

The idea is to approximate it as an unconstrained problem. Using Lagrange multipliers, we canfirst rewrite problem (37) as

min

�t

f0(�t

) +

mX

i=1

I(fi

(�t

)), (38)

where I : < ! < is an indicator function which keeps the solution inside the feasible region;

I(f) =

(0, f 0

1, f > 0.(39)

The problem (38) now has no inequality constraints, while it is not differentiable due to I .The barrier method26 is an interior point method which introduces a logarithmic barrier function

to approximate the indicator function I as follows;

Î(f) = �(1/t) log(�f), (40)

where t > 0 is a parameter to adjust the accuracy of approximation. The log barrier function goesto infinity rapidly as f goes close to 0 while it is close to 0 when f are far away from 0. Since Î(f)

is differentiable, we have

min

�t

f0(�t

) +

mX

i=1

�(1/t) log(�fi

(�t

)), (41)

or equivalently,

min

�t

tf0(�t

) �mX

i=1

log(�fi

(�t

)). (42)

The barrier method solves (42) iteratively by increasing the parameter t. At the limit of t ! 1,the above problem coincides with the original problem (38).

4.3 Algorithm for solving the inverse problem

Algorithm 3 shows the our algorithm which uses a barrier method with Quasi-Newton for solvingthe inverse problem. We should mention the following parts where we have modified the originalalgorithm.26

Warm start For each inner loop, the Quasi-Newton method needs initial guess of the inverseHessian B0. Instead of fixing B0 for every inner loop, we reuse B

k

of the last inner loop toaccelerate the convergence (shown in Lines 4 and 19 in Algorithm 3).

14

Solved%by%the%logVbarrier%interior%point%method

Cost%funcòn%with%constraints

4.2.2 Constrained problem: Interior point

Here we introduce a constrained optimization with inequality constraints of the form;

min

�t

f0(�t

) subject to fi

(x) 0, i = 1, · · · , m, (37)

where �t

2 <N⇥M is a real vector and f0, · · · , fm

: <N⇥M ! < are twice continuously differen-tiable.

The idea is to approximate it as an unconstrained problem. Using Lagrange multipliers, we canfirst rewrite problem (37) as

min

�t

f0(�t

) +

mX

i=1

I(fi

(�t

)), (38)

where I : < ! < is an indicator function which keeps the solution inside the feasible region;

I(f) =

(0, f 0

1, f > 0.(39)

The problem (38) now has no inequality constraints, while it is not differentiable due to I .The barrier method26 is an interior point method which introduces a logarithmic barrier function

to approximate the indicator function I as follows;

Î(f) = �(1/t) log(�f), (40)

where t > 0 is a parameter to adjust the accuracy of approximation. The log barrier function goesto infinity rapidly as f goes close to 0 while it is close to 0 when f are far away from 0. Since Î(f)

is differentiable, we have

min

�t

f0(�t

) +

mX

i=1

�(1/t) log(�fi

(�t

)), (41)

or equivalently,

min

�t

tf0(�t

) �mX

i=1

log(�fi

(�t

)). (42)

The barrier method solves (42) iteratively by increasing the parameter t. At the limit of t ! 1,the above problem coincides with the original problem (38).

4.3 Algorithm for solving the inverse problem

Algorithm 3 shows the our algorithm which uses a barrier method with Quasi-Newton for solvingthe inverse problem. We should mention the following parts where we have modified the originalalgorithm.26

Warm start For each inner loop, the Quasi-Newton method needs initial guess of the inverseHessian B0. Instead of fixing B0 for every inner loop, we reuse B

k

of the last inner loop toaccelerate the convergence (shown in Lines 4 and 19 in Algorithm 3).

14

Update%t%(larger)

Iniàl%value�t � 0

repe

at Solve

LogVbarrier%funcòns

by%QuasiVNewton

Layer

2D3D

Target model

2D

Smallest%cost%Rough%approxima`on%

Ideal%Costly%in%space%and%`me

Second%op`on%Less%expensive%but%s`ll%large%cost

Proposed 2D layered model

•  Light%sca6ers:%– Layer%by%layer%– At%voxel%centers%– Forward%only%

•  Subject%to%a%simplified%phase%func`on%

– With%uniform/known%sca6ering%coefficients

Enumerating paths

Source%locaòn%s%


Source%locaòn%s%


Exhaus`ve%(with%thresholding) (BiVdirecònal)%Monte%CarloVlike

Different directions

IV. NUMERICAL SIMULATIONS

A. Simple materialWe describe numerical simulation results of the pro-

posed method for evaluation. A two-dimensional materialis divided into 10⇥10 voxels as shown in Figure 4(a). Thematerial has almost homogeneous extinction coefficientsof value 0.05 except few voxels with much higher coeffi-cients of 0.2, which means those voxels absorb light muchmore than other voxels. A light source is assumed to lit alight at each of m positions to the top layer, and a detectorobserves the outgoing light at each of m positions fromthe bottom layer. The variance �

2 of Gaussian function(1) for scattering from one layer to the next layer is set to2. The upper bound u in Eq. (8) is set to 1.

Estimated extinction coefficients are shown in Figure4(b). The homogeneous part of the material with extinc-tion coefficients of 0.05 (voxels shaded in light gray) isreasonably estimated because estimated values fall in therange between 0.04 to 0.06. In contrast, the dense partof the material with extinction coefficients of 0.2 (voxelsshaded in dark gray) is not estimated well but looks blurredvertically. This blur effect is due to the experimentalsetting that the light paths go vertically from top to bottominside the material.

This effect can be verified by simply rotating the settingby 90 degrees (now the configuration is a horizontal one)and applying the proposed method. Figure 4(c) showsthe estimation result when the light source is on the leftside of the material, and the detector is on the rightside. The estimated extinction coefficients are blurred nowhorizontally because light paths go from left to right.

A simple trick to integrate these two results is to takethe average of them, which is shown in Figure 4(d). Theblur effect is then reduced by averaging and the estimationof extinction coefficients is improved. Root means squareerrors (RMSEs) for Fig. 4(b) and Fig. 4(c) are 2.92⇥10

�4

and 4.40 ⇥ 10

�4, respectively. By averaging, RMSE forFig. 4(d) decreases to 2.60⇥ 10

�4.Values of cost functions are shown in Figure 5. The

original cost function (6) decreases steadily and is lessthan 10

�5 after 24 iterations, while the cost function of thebarrier method seems not to change over iterations. Theseplot shows that the barrier method effectively minimizesthe original cost function while the constraints on theextinction coefficients are satisfied. This can be verifiedalso as the observed and estimated light intensities as thedifference of which is minimized. Figures 6(a) and 6(b)shows observations Iij for ground truth and estimation(of vertical configuration). Each row i in this matrixrepresentation is a set of observations at j. For example,the third and fourth rows represent the observations whereincident light position is i = 3, 4, where it is right abovethe dense voxels. Hence the observed light at j = 3, 4

below the dense voxels is much weaker then other diagonalvalues. Because our formulation of constraint optimizationminimizes the difference between them, Figures 6(a) and6(b) are quite similar to each other as the value ofdifference in Fig. 5 indicates.

(a) (b)

(c) (d)

Figure 4: Simulation results of a material of size 10⇥ 10.(a) Ground truth of the distribution of extinction coeffi-cients shown as values in each voxel as well as by voxelcolors. (b) Estimated extinction coefficients. A light sourceand detector are above and blow the media. (c) Estimatedextinction coefficients. A light source and detector are leftand right sides of the media. (d) Average of (b) and (c).

Cost function values over iterations are shown in Figure8 for each material. These curves show that the proposedmethod effectively minimizes the objective function andseems to work well for any kind of materials.

B. Complex materials

Simulation results with more complex materials areshown in Figure 7. All materials are of size 10 ⇥ 10 buthave more complex distribution of extinction coefficients.Estimated results are shown in the right column, made ofthe averaging results of vertical and horizontal configura-tions as in Fig. 4(d)

First material has four dense voxels apart from eachother (Fig. 7(c)). Because vertically (or horizontally)aligned voxels affect to light paths, the estimated resulthas a significant blur effect that makes the appearance ofthe distribution a blurred faint rectangle instead of fourcorners (7(b)). This is a limitation of the proposed methodcurrently, which must be improved in future.

Second material has dense voxels arranged in a U-shape(Figure 7(c)). Nevertheless this has also a lot of verticallyor horizontally aligned voxels, the estimated result looksreasonable (Figure 7(d)) in which we can see clearly theU-shape is reconstructed. This result is quite promising,while the values of four corners are much thinner than theground truth compared to voxels in between.








�4

and 4.40 ⇥ 10






(a) (b)

(c) (d)














�4

and 4.40 ⇥ 10






(a) (b)

(c) (d)














�4

and 4.40 ⇥ 10






(a) (b)

(c) (d)







Averaging

Ver`cal%es`mates%

Horizontal%Es`mates%Toru%Tamaki,%Bingzhi%Yuan,%Bisser%Raytchev,%Kazufumi%Kaneda,%Yasuhiro%

Mukaigawa,%"Mul`pleVsca6ering%Op`cal%Tomography%with%Layered%Material,"%The%9th%Interna`onal%Conference%on%SignalVImage%Technology%and%InternetVBased%Systems%(SITIS%2013),%December%2V5,%2013,%Kyoto,%Japan.%(2013%12)%

Joint optimization








�4

and 4.40 ⇥ 10






(a) (b)

(c) (d)







Bingzhi%Yuan,%Toru%Tamaki,%Takahiro%Kushida,%Bisser%Raytchev,%Kazufumi%Kaneda,%Yasuhiro%Mukaigawa%and%Hiroyuki%Kubo,%"Layered%op`cal%tomography%of%mul`ple%sca6ering%media%with%combined%constraint%op`miza`on",%in%Proc.%of%FCV%2015%:%21st%KoreaVJapan%joint%Workshop%on%Fron`ers%of%Computer%Vision,%January%28VV30,%2015,%Shinan%Beach%Hotel,%Mokpo,%KOREA.%

results of numerical simulation

Ground%truth

GT

�

2 = 0.2

�

2 = 0.4(a) (b) (c) (d) (e)

Fig 7 Numerical simulation results for a grid of size 20 ⇥ 20. Darker shades of gray represent larger values (morelight is absorbed at the voxel). The graybar shows the preojetcion from gray scale to value. The first row shows thegroundtruth of the five different media (a)–(e) used for the simulation. The second row shows the estimated resultswhen the �

2 in the Gaussian phase function equals to 0.2. The last row shows the estimated results when the �

2 = 0.4.

in the middle and bottom rows, and the root-mean-squared error (RMSE) shown in Table 5 is smallenough with 0.0075/1.05 = 0.7% of coefficient values. The other media, shown in columns (b)–(e), have more complex distributions of the extinction coefficients. In the top part of Table 5, weuse the RMSEs to show the quality of the estimated results. The ratio in the brackets shows therelative magnitude of RMSE compared with the majority extinction value or the background in thegrountruth. Here, according to the groundtruth of material (a)–(e) in Fig. 7, we set this majorityextinction value to be 1.05.

In the bottom part of Table 5, we show the computation time for the numerical simulation withdifferent media and different Gaussian phase function. As we implement our proposed method onMatlab, the further reduction of computation time can be expected by implementing on C++.

The values of the cost function f0 over iterations of the outer loop in Algorithm 3 are shown inFigure 9 for each medium. These curves show that the proposed method effectively minimizes theoriginal objective function (31) and maintain the decsending tendency for many different kinds ofmedia. Figure 10 demonstrates how the log-barriered cost function f in Eq. (43) evolves over alliterations of the inner loop; the number of iterations in the horizontal axis accumulates all inneriterations of the Quasi-Newton method. We can see that each inner loop successively minimizesthe log-barriered function and the warm start (reusing the Hessian from the previous outer loop)may help the gap of values between inner loops.

18

Fig 7 Numerical simulation results for a grid of size 20⇥20. Darker shades of gray represent larger values (more lightis absorbed at the voxel). The bars on the side shows extinction coefficient values in greyscale. The first row showsthe ground truth of the five different media (a)–(e) used for the simulation. The second and third rows show estimatedresults for �2 = 0.2 and �2 = 0.4, respectively, of the Gaussian phase function.

much higher coefficients of 1.2 (voxels shaded in dark gray), which means that those voxels ab-sorb much more light than other voxels. The coefficients are estimated reasonably well as shownin the middle and bottom rows, and the root-mean-squared error (RMSE) shown in Table 1 is smallenough with the relative error of 0.0075/1.05 = 0.7% to the background coefficient value. Theother media, shown in columns (b)–(e), have more complex distributions of the extinction coeffi-cients. We summarize the quality of the estimated results in terms of RMSE in Table 1. Numbersin the brackets are relative errors of RMSE to the background extinction coefficient values (i.e.,1.05). Computation time is also shown in Table 1. Note that the our proposed method has beencurrently implemented with Matlab, which can be accelerated further by using C++.

The values of the cost function f0 over iterations of the outer loop in Algorithm 3 are shown inFigure 9 for each medium. These curves show that the proposed method effectively minimizes theoriginal objective function (31) for the five media shown here and probably for other media. Figure10 demonstrates how the log-barriered cost function f in Eq. (43) evolves over all iterations of theinner loop; the number of iterations in the horizontal axis accumulates all inner iterations of theQuasi-Newton method. We can see that each inner loop successively minimizes the log-barrieredfunction and the warm start (reusing the Hessian from the previous outer loop) may help the gapof values between inner loops.

18

GT

�

2 = 0.2

�

2 = 0.4(a) (b) (c) (d) (e)



2 = 0.4.




18




18

•  20x20%grid%of%20[mm]x20[mm]%•  Ex`ncòn%coefficients:%between%1.05%and%1.55%[mmV1]%•  Sca6ering%coefficients%(known):%1%[mmV1]%•  Phase%funcòn%(known):%a%simplified%approximaòn

•  Fixed%742%paths%for%each%s,t%(exhaus`ve%with%th.)%

•  Joint%op`mizaòn%with%interior%point%method%

Es`mated%results

Shepp-Logan phantom

GT

�

2 = 0.2

�

2 = 0.4(a) (b) (c) (d) (e)



2 = 0.4.




18




18

GT

�

2 = 0.2

�

2 = 0.4(a) (b) (c) (d) (e)



2 = 0.4.




18




18

GT

�

2 = 0.2

�

2 = 0.4(a) (b) (c) (d) (e)



2 = 0.4.




18




18

20%x%20

Comparison with DOT

GT

Ours

DOT (GN)

DOT (PD)(a) (b) (c) (d) (e)

Fig 11 Numerical simulation results for a grid of size 24 ⇥ 24 with different inverse solver. Darker shades of grayrepresent larger values (more light is absorbed at the voxel). The graybar shows the preojetcion from gray scale tovalue. First row shows the groundtruth of five different media (a)–(e) used for the simulation. Second row shows theestimated results of our proposed. Third row shows the estimated results of DOT using an optimization method ofGauss-Newton method with NOSER prior. The last row presents the simulation results of DOT using an inverse solverof Primal-Dual Interior Point Method.

5.1 Comparison results

we compare with a standard DOT with FEM64, 65with different optimization methods implementedin EIDORS.64, 65

The groundtruth used in the our proposed method and DOT are shown in the first row of Fig.11.They are 24 [mm] ⇥ 24 [mm] mediums.

To fitting the change of the size of the medium, the parameters in Algorithm 3 were slightlymodified. For this special 2D layered medium, the grid size was set to N = M = 24 with squarevoxels of size 1 [mm].

For DOT, we chose Gauss-Newton(GN) method coorperated with NOSER prior and the Primal-Dual Interior Point Method(PDIPM) as the optimization methods.

Following description in section 3.5, we set the parameter of the Gaussian phase function tobe �2

= 0.4, as the shape of Gaussian phase function when �2= 0.4 is relatively smooth. The

estimated result of Algorithm 3 is shown in the second row of Fig.11. And the estimated resultby Gauss-Newton methed is shown in the third row of Fig.11. The last row of Fig.11 shows thesimulation results of primal-dual interior point method. Comparing the three sets of results, it’sobvious that Algorithm 3 correctly estimate the high extinction area. DOT obtained the resultswhich the high extinction area are larger than it in Algorithm 3 and groundtruth.

21

Fig 11 Numerical simulation results for a grid of size 24 ⇥ 24 with our method and DOT with two solvers. Darkershades of gray represent larger values (more light is absorbed at the voxel). The bars on the side shows extinctioncoefficient values in greyscale. First row shows the ground truth of five different media (a)–(e) used for the simulation.Second row shows the estimated results of our proposed. Third and fourth rows show estimated results of DOT byusing Guass-Newton (GN) and Primal-Dual internal point method (PD) solvers.

6 Conclusion with discussion

In this paper, we have proposed a path integral based approach to optical tomography for multi-ple scattering in discretized participating media. Assuming the scattering coefficients and phasefunction are known and uniform, the extinction coefficients at each voxel in a 2D layered mediumare estimated by using an interior point method. Numerical simulation examples are shown todemonstrate that the proposed framework works better than DOT in the simplified experimentalsetup while its computation cost needs to be reduced.

There are many directions for further research including: relaxing the assumption of 2D layeredscattering model to more realistic scattering with other phase functions, using paths generated byMonte-Carlo based statistical methods, extending the formulation to a full 3D scattering model,and solving the issues mentioned below.

Limitations: stability and uniqueness The current formulation presented in this paper esti-mates only the extinction coefficients; the scattering coefficients and phase function parameters areassume to be known and uniform. This is one of the limitations of the proposed method, howevera common limitation of optical tomography. It is known that the scattering and absorption coeffi-cients can not be separated from stationary measurements of light intensity,34 and the solutions arenot unique. Also, given stationary measurements without angle information the problem becomes

22

Ground%Truth

Ours

DOT%(GaussVNewton)

DOT%(PrimalVDual%interior%point)

•  24x24%grid%of%24[mm]x24[mm]%•  24x24x2%=%1152%triangle%meshes%for%FEM%•  16%light%sources%and%16%detectors%around%the%square%

•  With%EIDORS%(Electrical%Impedance%Tomography%and%Diffuse%Op`cal%Tomography%Reconstruc`on%So|ware)%eidors3d.sourceforge.net%

Accuracy and cost

(a) (b) (c) (d) (e)

RMSE

Ours 0.007662 0.01244 0.026602 0.021442 0.051152�2

= 0.4 (0.730%) (1.18%) (2.53%) (2.04%) (4.87%)DOT (GN) 0.053037 0.060597 0.7605 0.059534 0.0855

(5.05%) (5.77%) (7.53%) (5.67%) (8.14%)DOT (PD) 0.052466 0.0626 0.081081 0.066042 0.080798

(5.25%) (5.97%) (8.11%) (6.60%) (8.08%)

Computationtime [s]

Ours 257 217 382 306 504�2

= 0.4

DOT (GN) 0.397 0.390 0.407 0.404 0.453

DOT (PD) 1.11 1.09 1.14 1.08 1.15

Table 2 RMSEs and computation time of the numerical simulations for five different media (a)–(e) with grid size of24 ⇥ 24 with our proposed method and DOT with two solvers. Numbers in the brackets are relative errors of RMSEto the background extinction coefficient values (i.e., 1.05).

5.1 Comparison results

We compare our method to a standard DOT with Finite Element Methods (FEM)64, 65 with differentoptimization methods implemented in EIDORS.64, 65 The ground truth used in this comparison isshown in the top row of Figure 11 (a) – (e); N = M = 24 medium of the size 24 [mm] 24 [mm]with extinction coefficient distributions almost the same with Fig.7 (a) – (e).

For solving DOT by EIDORS, we used 24 x 24 x 2 = 1152 triangle meshes (i.e., each voxelis divided into two triangle meshes), and for the boundary condition we placed 16 light sourcesand detectors at the same interval around the material. We chose two solvers: primal-dual interiorpoint method (PD) and Gauss-Newton method (GN). We used �

t

0= 0. as the initial guess for

both our method and EIDORS.The estimated results by our method (�2

= 0.4) and DOT with GN and PD are shown inFig.11. Results of our proposed method are shown in the second row, which are similar to those inthe third row of Fig.7. The third row shows results of DOT with GN. This kind of blurred resultsare typical DOT estimation due to its diffusion approximation. The last row shows results of DOTwith PD, which look better than DOT with GN, but still have a tendency of overestimation of highcoefficient value areas.

We summarize RMSE values and computation time of each method in Table .2 in the sameformat with Table .1. RMSE values of our method are 2 to 5 times smaller than those of DOT, andthis demonstrates that our proposed method can achieve much more accurate results.

The current disadvantage is its large computation cost, an out method takes up to 1000 timeslonger than DOT. Our future research therefore includes to reducing the computation cost by opti-mizing the code with C++ and adopting other solvers.

21

GT

�

2 = 0.2

�

2 = 0.4(a) (b) (c) (d) (e)



2 = 0.4.




18




18

Our%implementa`on:%MATLAB

Summary

•  Op`cal%tomography%for%sca6ering%media%•  Observa`on%model:%path%integral%•  LeastVsquare%approach%with%constraints%

•  Solver:%interior%point%method%•  Experiments:%numerical%simula`on%

•  Future%work:%a%lotGT

�

2 = 0.2

�

2 = 0.4(a) (b) (c) (d) (e)



2 = 0.4.




18




18

GT

�

2 = 0.2

�

2 = 0.4(a) (b) (c) (d) (e)



2 = 0.4.




18




18

scattering tomography with path integral

Engineering