![Page 1: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/1.jpg)
Towards Real-Time Texture Synthesis With
the Jump Map
Steve Zelinka Michael GarlandUniversity of Illinois at Urbana-Champaign
Thirteenth Eurographics Workshop on Rendering (2002)
![Page 2: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/2.jpg)
Texture Synthesis
![Page 3: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/3.jpg)
Neighbourhood-based
• Compare local causal neighbourhoods
Efros and Leung (ICCV ’99)Wei and Levoy (SIGGRAPH 2000)Ashikhmin (I3D 2001)
Input Output
![Page 4: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/4.jpg)
Fast Texture Synthesis
• Goal: Interactivity• Want synthesis that is:
• Fast• Simple• High quality
![Page 5: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/5.jpg)
Patch-based Methods
• Copy patches of pixels rather than single pixels
![Page 6: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/6.jpg)
Patch-based Methods
• Copy patches of pixels rather than single pixels
• Chaos Mosaic, Xu et al, 1997
![Page 7: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/7.jpg)
Patch-based Methods
• Copy patches of pixels rather than single pixels
• Chaos Mosaic, Xu et al, 1997• Patch-Based Sampling, Liang et al, 2002
![Page 8: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/8.jpg)
Patch-based Methods
• Copy patches of pixels rather than single pixels
• Chaos Mosaic, Xu et al, 1997• Patch-Based Sampling, Liang et al, 2002• Image Quilting, Efros and Williams, 2001
![Page 9: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/9.jpg)
• Video Textures• Schodl et al, SIGGRAPH 2000
• Given a sample video, generate endless video without looping
• Generate links between similar frames
• Play video, randomly following links
Our Inspiration
..
![Page 10: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/10.jpg)
Our Approach
• Divide task into two phases:• Analysis
• Once per input texture (need not be fast)• Generates jump map
• Synthesis• Uses jump map• Fast enough for interactive applications
![Page 11: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/11.jpg)
What is a Jump Map?
• Same size as input
![Page 12: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/12.jpg)
What is a Jump Map?
• Same size as input• Set of jumps per
pixel
![Page 13: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/13.jpg)
What is a Jump Map?
• Same size as input• Set of jumps per
pixel• Jumps are weighted
according to similarity• Need not sum to 1
0.4
0.3
0.2
![Page 14: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/14.jpg)
Output
Synthesis with Jump Maps
Input
![Page 15: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/15.jpg)
Output
Synthesis With Jump Maps
Input
![Page 16: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/16.jpg)
Synthesis Order
• Synthesis order influences patch shapes
• Not likely to extend in directions where there aren’t already-synthesized neighbours
![Page 17: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/17.jpg)
Synthesis Orders
• Serpentine• Reverse direction at end of scan-line• Better than scan-line, just as fast
• Hilbert curve• Maximizes locality
• Much higher quality
• Adds some overhead
![Page 18: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/18.jpg)
Synthesis Issues
• Artifacts may occur if a patch hits an input image boundary• Modify probability of taking a jump
• Increase for jumps from input boundary• Decrease for jumps to input boundary
• Blend patch boundaries
![Page 19: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/19.jpg)
Texture Analysis
• Need best matches for each input pixel
• Pose as high-dimensional ANN problem
Input
…
NeighbourhoodVectors
…
ANN Vectors
PCA
![Page 20: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/20.jpg)
Multi-resolution Analysis
• Use image pyramid and multi-resolution neighbourhood vectors
• Smaller neighbourhood required
• Improves PCA reduction
![Page 21: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/21.jpg)
Jump Map Diversity
• Undesirable repitition may occur if jumps cluster spatially
• L2 norm is particularly susceptible
![Page 22: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/22.jpg)
1
5
6
4
3
2
Poisson Disc Sampling
• Find extra matches• Iteratively accept
matches satisfying Poisson disc criterion
• Include a Poisson disc at the source
1
4
3
![Page 23: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/23.jpg)
Analysis Summary
• Need best matches for each input pixel• Use multi-resolution neighbourhoods• Pose as high-dimensional ANN problem
• Reduce dimension with PCA
• Filter matches with Poisson discs• Normalize similarity values across
the jump map
![Page 24: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/24.jpg)
Results
• Current implementation: 2.1 million pixels/second scan-line0.8 million pixels/second Hilbert
• Good quality on stochastic textures• Not so good on structured textures• TBD: demo
![Page 25: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/25.jpg)
Future Work
• Analysis phase:• Use perceptual metrics• Clustering instead of ANN
• Synthesis phase• Multi-resolution synthesis• Output control mechanisms
![Page 26: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/26.jpg)
Future Work
• Generalization to patches• Reduce storage used
• Sample size required?
![Page 27: Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics](https://reader036.vdocuments.us/reader036/viewer/2022081603/5697bf881a28abf838c8978f/html5/thumbnails/27.jpg)
Contact Information
Steve [email protected]
Michael [email protected]
http://graphics.cs.uiuc.edu/~zelinka/jumpmaps