towards real-time texture synthesis with the jump map steve zelinka michael garland university of...
TRANSCRIPT
Towards Real-Time Texture Synthesis With
the Jump Map
Steve Zelinka Michael GarlandUniversity of Illinois at Urbana-Champaign
Thirteenth Eurographics Workshop on Rendering (2002)
Texture Synthesis
Neighbourhood-based
• Compare local causal neighbourhoods
Efros and Leung (ICCV ’99)Wei and Levoy (SIGGRAPH 2000)Ashikhmin (I3D 2001)
Input Output
Fast Texture Synthesis
• Goal: Interactivity• Want synthesis that is:
• Fast• Simple• High quality
Patch-based Methods
• Copy patches of pixels rather than single pixels
Patch-based Methods
• Copy patches of pixels rather than single pixels
• Chaos Mosaic, Xu et al, 1997
Patch-based Methods
• Copy patches of pixels rather than single pixels
• Chaos Mosaic, Xu et al, 1997• Patch-Based Sampling, Liang et al, 2002
Patch-based Methods
• Copy patches of pixels rather than single pixels
• Chaos Mosaic, Xu et al, 1997• Patch-Based Sampling, Liang et al, 2002• Image Quilting, Efros and Williams, 2001
• Video Textures• Schodl et al, SIGGRAPH 2000
• Given a sample video, generate endless video without looping
• Generate links between similar frames
• Play video, randomly following links
Our Inspiration
..
Our Approach
• Divide task into two phases:• Analysis
• Once per input texture (need not be fast)• Generates jump map
• Synthesis• Uses jump map• Fast enough for interactive applications
What is a Jump Map?
• Same size as input
What is a Jump Map?
• Same size as input• Set of jumps per
pixel
What is a Jump Map?
• Same size as input• Set of jumps per
pixel• Jumps are weighted
according to similarity• Need not sum to 1
0.4
0.3
0.2
Output
Synthesis with Jump Maps
Input
Output
Synthesis With Jump Maps
Input
Synthesis Order
• Synthesis order influences patch shapes
• Not likely to extend in directions where there aren’t already-synthesized neighbours
Synthesis Orders
• Serpentine• Reverse direction at end of scan-line• Better than scan-line, just as fast
• Hilbert curve• Maximizes locality
• Much higher quality
• Adds some overhead
Synthesis Issues
• Artifacts may occur if a patch hits an input image boundary• Modify probability of taking a jump
• Increase for jumps from input boundary• Decrease for jumps to input boundary
• Blend patch boundaries
Texture Analysis
• Need best matches for each input pixel
• Pose as high-dimensional ANN problem
Input
…
NeighbourhoodVectors
…
ANN Vectors
PCA
Multi-resolution Analysis
• Use image pyramid and multi-resolution neighbourhood vectors
• Smaller neighbourhood required
• Improves PCA reduction
Jump Map Diversity
• Undesirable repitition may occur if jumps cluster spatially
• L2 norm is particularly susceptible
1
5
6
4
3
2
Poisson Disc Sampling
• Find extra matches• Iteratively accept
matches satisfying Poisson disc criterion
• Include a Poisson disc at the source
1
4
3
Analysis Summary
• Need best matches for each input pixel• Use multi-resolution neighbourhoods• Pose as high-dimensional ANN problem
• Reduce dimension with PCA
• Filter matches with Poisson discs• Normalize similarity values across
the jump map
Results
• Current implementation: 2.1 million pixels/second scan-line0.8 million pixels/second Hilbert
• Good quality on stochastic textures• Not so good on structured textures• TBD: demo
Future Work
• Analysis phase:• Use perceptual metrics• Clustering instead of ANN
• Synthesis phase• Multi-resolution synthesis• Output control mechanisms
Future Work
• Generalization to patches• Reduce storage used
• Sample size required?
Contact Information
Steve [email protected]
Michael [email protected]
http://graphics.cs.uiuc.edu/~zelinka/jumpmaps