Download - Video Textures
Video TexturesVideo Textures
Arno SchödlRichard Szeliski
David SalesinIrfan Essa
Microsoft Research, Georgia Tech
Still PhotosStill Photos
Video ClipsVideo Clips
Video TexturesVideo Textures
Problem StatementProblem Statement
video clip video texture
ApproachApproach
How do we find good transitions?
Finding Good Transitions Finding Good Transitions
Compute L2 distance Di, j between all frames
Similar frames make good transitions
frame ivs.
frame j
Markov Chain RepresentationMarkov Chain Representation
2 3 41
Similar frames make good transitions
Transition Costs Transition Costs
Transition from i to j if successor of i is similar to j
Cost function: Cij = Di+1, j
i
j
i+ 1
j-1
i j D i+ 1, j
Transition ProbabilitiesTransition Probabilities
Probability for transition Pij inversely related
to cost:
high low
2exp
ji
ji
CP
Preserving DynamicsPreserving Dynamics
Preserving Dynamics Preserving Dynamics
Preserving Dynamics Preserving Dynamics
Cost for transition ij
Cij = wk Di+k+1, j+kk = -N
N -1
i
j j+ 1
i+ 1 i+ 2
j-1j-2
i jD i, j - 1 D Di+ 1, j i+ 2, j+ 1
i-1
D i-1, j-2
Preserving Dynamics – Effect Preserving Dynamics – Effect
Cost for transition ij
Cij = wk Di+k+1, j+kk = -N
N -1
2 3 41
DeadendsDeadends
No good transition at the end of sequence
2 3 41
Future CostFuture Cost
• Propagate future transition costs backward
• Iteratively compute new cost
Fij = Cij + mink Fjk
2 3 41
Future CostFuture Cost
• Propagate future transition costs backward
• Iteratively compute new cost
Fij = Cij + mink Fjk
2 3 41
Future CostFuture Cost
• Propagate future transition costs backward
• Iteratively compute new cost
Fij = Cij + mink Fjk
2 3 41
Future CostFuture Cost
• Propagate future transition costs backward
• Iteratively compute new cost
Fij = Cij + mink Fjk
2 3 41
• Propagate future transition costs backward
• Iteratively compute new cost
Fij = Cij + mink Fjk
• Q-learning
Future CostFuture Cost
Future Cost – EffectFuture Cost – Effect
Finding Good LoopsFinding Good Loops
• Alternative to random transitions
• Precompute a good set of loops up front (using dynamic programming)
Visual DiscontinuitiesVisual Discontinuities
• Problem: Visible “Jumps”
Crossfading Crossfading
• Solution: Crossfade from one sequence to the other.
A i-2
A i-2
B j-2
15
…
…
3
1 2
2 1
3
4
4 4
4 4
4+ + +
A i-1
A i-1 / B j-2 A i-1 / B j-2 A i-1 / B j-2
B j-1
A i
B j
A i+1
B j+1
B j+1
Morphing Morphing
• Interpolation task:
A25 B2
5 C15+ +
Morphing Morphing
• Interpolation task:
• Compute correspondencebetween pixels of all frames
A25 B2
5 C15+ +
Morphing Morphing
• Interpolation task:
• Compute correspondence between pixels of all frames
• Interpolate pixel position andcolor in morphed frame
• based on [Shum 2000]
A25 B2
5 C15+ +
Results – Crossfading / MorphingResults – Crossfading / Morphing
Results – Crossfading / MorphingResults – Crossfading / Morphing
Jump Cut Crossfade Morph
CrossfadingCrossfading
Frequent Jump & CrossfadingFrequent Jump & Crossfading
Video PortraitVideo Portrait
Useful for web pages
Combine with IBR techniques
Video Portrait – 3DVideo Portrait – 3D
Region-based AnalysisRegion-based Analysis
• Divide video up into regions
• Generate a video texture for each region
Automatic Region AnalysisAutomatic Region Analysis
User selects target frame range
User-controlled Video TexturesUser-controlled Video Textures
slow variable fast
Lengthen / shorten video without affecting speed
Time WarpingTime Warping
shorter original longer
Summary Summary
• Video clips video textures
• define Markov process
• preserve dynamics
• avoid dead-ends
• disguise visual discontinuities
Summary Summary
• Extensions
• regions
• external constraints
• video-based animation
Discussion Discussion
• Some things are relatively easy
Discussion Discussion
• Some are hard