scattering-aware texture reproduction for 3d printing · motivation: color printing in 3d...
TRANSCRIPT
![Page 1: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/1.jpg)
Scattering-aware Texture Reproduction for 3D Printing
Oskar Elek* Denis Sumin* Ran Zhang Tim Weyrich
Karol Myszkowski Bernd Bickel Alexander Wilkie Jaroslav Křivánek
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 642841.
![Page 2: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/2.jpg)
Motivation: Color Printing in 3D
computational fabrication of highly detailed textures (slabs above are 1 cm thick)
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 2
![Page 3: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/3.jpg)
Enabler: Multi-material Printing
Stratasys J750 (poly-jetting printer)
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 3
‘Vero Opaque’ materials (not actually opaque!)
Cyan
Magenta
Yellow
blacK
White
![Page 4: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/4.jpg)
Color in the Wild
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 4
[Stratasys]
![Page 5: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/5.jpg)
State of the Art
we can fabricate translucent appearance rather well…
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 5
[Hašan et al. @ SIGGRAPH 2010] [Dong et al. @ SIGGRAPH 2010] [Brunton et al. @ ToG 2015]
![Page 6: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/6.jpg)
State of the Art
…however, fine details are problematic
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 6
target → ← print
[Babaei et al. @ SIGGRAPH 2017]
![Page 7: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/7.jpg)
“The Dream”
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 8
achieving quality and reproducibility of 2D prints
![Page 8: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/8.jpg)
Our Achievement
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 9
target standard print our print
520 px (~5 cm)
550 px
![Page 9: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/9.jpg)
Our Inverse Pipeline
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 14
optimization
intrinsic material parameters
target appearance
printing
physical printout
3D material distribution
prediction
? ==
solution refinement
RGB → CMYKW conversion
current solution (proxy RGB)
![Page 10: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/10.jpg)
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 15
Technical Contributions
![Page 11: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/11.jpg)
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 16
Material Calibration
![Page 12: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/12.jpg)
Material Calibration
affordable optical calibration setup based on transmissive measurement
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 17
0.5 mm 1 mm W
C M Y K
![Page 13: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/13.jpg)
Material Calibration
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 18
optical density scattering albedo scattering anisotropy
![Page 14: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/14.jpg)
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 19
Material Mapping
![Page 15: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/15.jpg)
Material Mapping
data-driven approaches: impractical for multi-material, translucent printing
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 20
Task: mapping RGB ←→ CMYKW
C × M × Y × W
[Brunton et al. @ ToG 2015]
C
M Y [Stratasys, Object Connex materials]
![Page 16: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/16.jpg)
Material Mapping
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 21
RGB ←→ CMYKW Our solution: RGB ←→ optical parameters ← CMYKW
scattering albedo → RGB
![Page 17: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/17.jpg)
Material Mapping
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 22
Our solution: RGB ←→ optical parameters ← CMYKW
![Page 18: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/18.jpg)
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 23
Optimization
![Page 19: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/19.jpg)
Optimization
despite the non-linearity of the appearance, it changes monotonically → simple residual energy minimization
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 24
![Page 20: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/20.jpg)
Optimization
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 25
despite the non-linearity of the appearance, it changes monotonically → simple residual energy minimization
![Page 21: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/21.jpg)
Optimization
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 26
despite the non-linearity of the appearance, it changes monotonically → simple residual energy minimization
∆
![Page 22: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/22.jpg)
Optimization
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 27
despite the non-linearity of the appearance, it changes monotonically → simple residual energy minimization
∆
![Page 23: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/23.jpg)
Optimization
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 28
despite the non-linearity of the appearance, it changes monotonically → simple residual energy minimization
∆
![Page 24: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/24.jpg)
Solution Refinement
• difficult: we have 2D appearance gradient → 3D material distribution
• two key heuristics to achieve balanced color and sharp structure
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 29
adaptive ‘vertical’ color placement ‘horizontal’ edge erosion
![Page 25: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/25.jpg)
Results
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 30
optimization
intrinsic material parameters
target appearance
printing
physical printout
3D material distribution
![Page 26: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/26.jpg)
Alternatives?
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 31
[Cignoni et al. @ VAST 2008]
image enhancement (e.g. unsharp masking)
[Babaei et al. @ SIGGRAPH 2017]
approximate deconvolution
optimization
![Page 27: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/27.jpg)
Results
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 32
“Timmy”
“marine”
“sunset”
← ← input (gamut-mapped) ←
![Page 28: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/28.jpg)
Results
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 33
“Timmy”
“marine”
“sunset”
← ← standard print ←
![Page 29: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/29.jpg)
Results
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 34
“Timmy”
“marine”
“sunset”
← ← our un-corrected print ←
![Page 30: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/30.jpg)
Results
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 35
“Timmy”
“marine”
“sunset”
← ← sharpened print ←
![Page 31: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/31.jpg)
Results
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 36
“Timmy”
“marine”
“sunset”
← ← ←
our optimized print
![Page 32: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/32.jpg)
Results
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 37
“Timmy”
“marine”
“sunset”
our optimized print standard print
![Page 33: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/33.jpg)
Results: Non-standard Composition
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 38
our reproduction ‘random’ target
‘random’ structured target
our reproduction
![Page 34: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/34.jpg)
Open Questions
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 39
[Babaei et al. @ SIGGRAPH 2017]
• general 3D geometry – (near-)convex
– arbitrary
• perceptual considerations – local contrast manipulation
– “similar appearance”?
• efficient prediction – VPT currently takes
~3 minutes on a small CPU cluster
![Page 35: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/35.jpg)
Take-home Message
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 40
a de-scattering solution must consider full 3D material composition
→ inverse, constraint-based design is the key
![Page 36: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/36.jpg)
Scattering-aware Texture Reproduction for 3D Printing
Oskar Elek* Denis Sumin* Ran Zhang Tim Weyrich
Karol Myszkowski Bernd Bickel Alexander Wilkie Jaroslav Křivánek
Thanks: Stratasys Ltd. and Alan Brunton, Filip Šroubek, Per H. Christensen, Michal Šorel and Rhaleb Zayer, Piotr Didyk.
Primary funding: European Union’s Horizon 2020 research and innovation programme, under the Marie Skłodowska-Curie grant agreement No 642841 (DISTRO).
Further support: European Research Council grant agreement No 715767 (MATERIALIZABLE); Czech Science Foundation grants 16-18964S and 16-08111S; Charles University grant SVV-2017-260452; Engineering and Physical Sciences Research Council grant EP/K023578/1.
tinyurl.com/TexFab
![Page 37: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/37.jpg)
Extra: Non-standard Illumination
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 42
“tree”
“cork”
“tree” “cork” pure white
sharpened sharpened ours ours
![Page 38: Scattering-aware Texture Reproduction for 3D Printing · Motivation: Color Printing in 3D computational fabrication of highly detailed textures (slabs above are 1 cm thick) Elek*,](https://reader030.vdocuments.us/reader030/viewer/2022040814/5e5a0a70831d0267f555f6fc/html5/thumbnails/38.jpg)
Extra: SGA Logo
Elek*, Sumin*, Zhang, Weyrich, Myszkowski, Bickel, Wilkie, Křivánek → Scattering-aware Texture Reproduction for 3D Printing 43
standard print our optimized print target