compression and gradient domain€¦ · livingstone, vision and art: the biology of seeing....
TRANSCRIPT
![Page 1: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/1.jpg)
Compression and Gradient Domain
CS194: Image Manipulation, Comp. Vision, and Comp. PhotoAlexei Efros, UC Berkeley, Spring 2020
![Page 2: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/2.jpg)
Da Vinci and Peripheral Vision
https://en.wikipedia.org/wiki/Speculations_about_Mona_Lisa#Smile
![Page 3: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/3.jpg)
Leonardo playing with peripheral vision
Livingstone, Vision and Art: The Biology of Seeing
![Page 4: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/4.jpg)
Frequency Domain and Perception
Campbell-Robson contrast sensitivity curve
![Page 5: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/5.jpg)
Lossy Image Compression (JPEG)
Block-based Discrete Cosine Transform (DCT)
![Page 6: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/6.jpg)
Using DCT in JPEG The first coefficient B(0,0) is the DC component,
the average intensityThe top-left coeffs represent low frequencies,
the bottom right – high frequencies
![Page 7: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/7.jpg)
Image compression using DCTQuantize
• More coarsely for high frequencies (which also tend to have smaller values)
• Many quantized high frequency values will be zero
Encode• Can decode with inverse dct
Quantization table
Filter responses
Quantized values
![Page 8: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/8.jpg)
JPEG Compression SummarySubsample color by factor of 2
• People have bad resolution for color
Split into blocks (8x8, typically), subtract 128For each block
a. Compute DCT coefficientsb. Coarsely quantize
– Many high frequency components will become zeroc. Encode (e.g., with Huffman coding)
http://en.wikipedia.org/wiki/YCbCrhttp://en.wikipedia.org/wiki/JPEG
![Page 9: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/9.jpg)
Block size in JPEG
Block size• small block
– faster – correlation exists between neighboring pixels
• large block– better compression in smooth regions
• It’s 8x8 in standard JPEG
![Page 10: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/10.jpg)
JPEG compression comparison
89k 12k
![Page 11: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/11.jpg)
Pyramid Blending
![Page 12: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/12.jpg)
Gradient Domain vs. Frequency DomainIn Pyramid Blending, we decomposed our
images into several frequency bands, and transferred them separately• But boundaries appear across multiple bands
But what about representation based on derivatives (gradients) of the image?:• Derivatives represent local changes (across all frequencies) • No need for low-res image
– captures everything (up to a constant)• Blending/Editing in Gradient Domain:
– Differentiate– Copy / Blend / edit / whatever– Reintegrate
![Page 13: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/13.jpg)
Gradients vs. Pixels
![Page 14: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/14.jpg)
Gilchrist Illusion (c.f. Exploratorium)
![Page 15: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/15.jpg)
![Page 16: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/16.jpg)
White?
![Page 17: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/17.jpg)
White?
![Page 18: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/18.jpg)
White?
![Page 19: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/19.jpg)
![Page 20: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/20.jpg)
Drawing in Gradient Domain
James McCann & Nancy PollardReal-Time Gradient-Domain Painting, SIGGRAPH 2009
(paper came out of this class!)
http://www.youtube.com/watch?v=RvhkAfrA0-w&feature=youtu.be
![Page 21: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/21.jpg)
Gradient Domain blending (1D)
Twosignals
Regularblending
Blendingderivatives
bright
dark
![Page 22: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/22.jpg)
Gradient hole-filling (1D)
sourcetarget
![Page 23: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/23.jpg)
It is impossible to faithfully preserve the gradients
sourcetarget
![Page 24: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/24.jpg)
Gradient Domain Blending (2D)
Trickier in 2D:• Take partial derivatives dx and dy (the gradient field)• Fiddle around with them (copy, blend, smooth, feather, etc)• Reintegrate
– But now integral(dx) might not equal integral(dy)• Find the most agreeable solution
– Equivalent to solving Poisson equation– Can be done using least-squares (\ in Matlab)
![Page 25: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/25.jpg)
Example
Source: Evan Wallace
Gradient Visualization
![Page 26: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/26.jpg)
Source: Evan Wallace
+Specify object region
![Page 27: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/27.jpg)
Poisson Blending AlgorithmA good blend should preserve gradients of source region without changing the background
Treat pixels as variables to be solved– Minimize squared difference between gradients of
foreground region and gradients of target region– Keep background pixels constant
Perez et al. 2003
![Page 28: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/28.jpg)
Perez et al., 2003
![Page 29: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/29.jpg)
sourcetarget mask
no blending gradient domain blending
Slide by Mr. Hays
![Page 30: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/30.jpg)
What’s the difference?
no blendinggradient domain blending
- =
Slide by Mr. Hays
![Page 31: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/31.jpg)
Perez et al, 2003
Limitations:• Can’t do contrast reversal (gray on black -> gray on white)• Colored backgrounds “bleed through”• Images need to be very well aligned
editing
![Page 32: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/32.jpg)
Gradient Domain as Image Representation
See GradientShop paper as good example:
http://www.gradientshop.com/
![Page 33: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/33.jpg)
Can be used to exert high-level control over images
![Page 34: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/34.jpg)
Can be used to exert high-level control over images gradients – low level image-features
![Page 35: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/35.jpg)
Can be used to exert high-level control over images gradients – low level image-features
+100pixelgradient
![Page 36: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/36.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features
+100pixelgradient
![Page 37: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/37.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features
+100
+100
+100
+100
+100
+100pixelgradient
+100
+100
+100
+100
+100
+100pixelgradient
![Page 38: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/38.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features
+100
+100
+100
+100
+100
+100pixelgradient
+100
+100
+100
+100
+100
+100pixelgradient
image edge image edge
![Page 39: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/39.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features manipulate local gradients to
manipulate global image interpretation
+100
+100
+100
+100
+100
+100pixelgradient
+100
+100
+100
+100
+100
+100pixelgradient
![Page 40: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/40.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features manipulate local gradients to
manipulate global image interpretation
+255
+255
+255
+255
+255
pixelgradient
![Page 41: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/41.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features manipulate local gradients to
manipulate global image interpretation
+255
+255
+255
+255
+255
pixelgradient
![Page 42: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/42.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features manipulate local gradients to
manipulate global image interpretation
+0
+0
+0
+0
+0
pixelgradient
![Page 43: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/43.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features manipulate local gradients to
manipulate global image interpretation
+0
+0
+0
+0
+0
pixelgradient
![Page 44: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/44.jpg)
Can be used to exert high-level control over images
![Page 45: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/45.jpg)
Optimization framework
Pravin Bhat et al
![Page 46: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/46.jpg)
Optimization framework Input unfiltered image – u
![Page 47: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/47.jpg)
Optimization framework Input unfiltered image – u Output filtered image – f
![Page 48: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/48.jpg)
Optimization framework Input unfiltered image – u Output filtered image – f Specify desired pixel-differences – (gx, gy)
min (fx – gx)2 + (fy – gy)2
f
Energy function
![Page 49: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/49.jpg)
Optimization framework Input unfiltered image – u Output filtered image – f Specify desired pixel-differences – (gx, gy) Specify desired pixel-values – d
min (fx – gx)2 + (fy – gy)2 + (f – d)2
f
Energy function
![Page 50: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/50.jpg)
Optimization framework Input unfiltered image – u Output filtered image – f Specify desired pixel-differences – (gx, gy) Specify desired pixel-values – d Specify constraints weights – (wx, wy, wd)
min wx(fx – gx)2 + wy(fy – gy)2 + wd(f – d)2
f
Energy function
![Page 51: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/51.jpg)
![Page 52: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/52.jpg)
![Page 53: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/53.jpg)
![Page 54: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/54.jpg)
Pseudo image relighting change scene illumination
in post-production example
input
![Page 55: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/55.jpg)
Pseudo image relighting change scene illumination
in post-production example
manual relight
![Page 56: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/56.jpg)
Pseudo image relighting change scene illumination
in post-production example
input
![Page 57: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/57.jpg)
Pseudo image relighting change scene illumination
in post-production example
GradientShop relight
![Page 58: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/58.jpg)
Pseudo image relighting change scene illumination
in post-production example
GradientShop relight
![Page 59: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/59.jpg)
Pseudo image relighting change scene illumination
in post-production example
GradientShop relight
![Page 60: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/60.jpg)
Pseudo image relighting change scene illumination
in post-production example
GradientShop relight
![Page 61: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/61.jpg)
Pseudo image relighting
u
f
o
![Page 62: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/62.jpg)
Pseudo image relighting
min wx(fx – gx)2 + f wy(fy – gy)2 +
wd(f – d)2
Energy function
u
f
o
![Page 63: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/63.jpg)
Pseudo image relighting
min wx(fx – gx)2 + f wy(fy – gy)2 +
wd(f – d)2
Energy function
Definition: d = u
u
f
o
![Page 64: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/64.jpg)
Pseudo image relighting
min wx(fx – gx)2 + f wy(fy – gy)2 +
wd(f – d)2
Energy function
u
f
o Definition: d = u gx(p) = ux(p) * (1 + a(p)) a(p) = max(0, - u(p).o(p))
![Page 65: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/65.jpg)
Pseudo image relighting
min wx(fx – gx)2 + f wy(fy – gy)2 +
wd(f – d)2
Energy function
u
f
o Definition: d = u gx(p) = ux(p) * (1 + a(p)) a(p) = max(0, - u(p).o(p))
![Page 66: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/66.jpg)
Sparse data interpolation Interpolate scattered data
over images/video
![Page 67: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/67.jpg)
Sparse data interpolation Interpolate scattered data
over images/video Example app: Colorization*
input output*Levin et al. – SIGRAPH 2004
![Page 68: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/68.jpg)
u
f
user data
Sparse data interpolation
![Page 69: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/69.jpg)
u
f
user data
Sparse data interpolation
min wx(fx – gx)2 + f wy(fy – gy)2 +
wd(f – d)2
Energy function
![Page 70: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/70.jpg)
u
f
user data
Sparse data interpolation
min wx(fx – gx)2 + f wy(fy – gy)2 +
wd(f – d)2
Energy function
Definition: d = user_data
![Page 71: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/71.jpg)
u
f
user data
Sparse data interpolation
min wx(fx – gx)2 + f wy(fy – gy)2 +
wd(f – d)2
Energy function
Definition: d = user_data if user_data(p) defined
wd(p) = 1else
wd(p) = 0
![Page 72: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/72.jpg)
u
f
user data
Sparse data interpolation
min wx(fx – gx)2 + f wy(fy – gy)2 +
wd(f – d)2
Energy function
Definition: d = user_data if user_data(p) defined
wd(p) = 1else
wd(p) = 0 gx(p) = 0; gy(p) = 0
![Page 73: Compression and Gradient Domain€¦ · Livingstone, Vision and Art: The Biology of Seeing. Frequency Domain and Perception Campbell-Robson contrast sensitivity curve. Lossy Image](https://reader035.vdocuments.us/reader035/viewer/2022070111/604d6ab72ecc885f67602178/html5/thumbnails/73.jpg)
u
f
user data
Sparse data interpolation
min wx(fx – gx)2 + f wy(fy – gy)2 +
wd(f – d)2
Energy function
Definition: d = user_data if user_data(p) defined
wd(p) = 1else
wd(p) = 0 gx(p) = 0; gy(p) = 0 wx(p) = 1/(1 + c*|ux(p)|)
wy(p) = 1/(1 + c*|uy(p)|)