a gentle introduction to bilateral filtering and its applications limitation? pierre kornprobst...
TRANSCRIPT
![Page 1: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/1.jpg)
![Page 2: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/2.jpg)
A Gentle Introductionto Bilateral Filteringand its Applications
A Gentle Introductionto Bilateral Filteringand its Applications
Limitation?
Pierre Kornprobst (INRIA)
0:20
![Page 3: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/3.jpg)
ExamplesExamples
Input Bilateral filter
Soft texture is removed
![Page 4: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/4.jpg)
ExamplesExamples
Input Bilateral filter
[Buades, Coll, Morel, 2005]
Constant regions appear
![Page 5: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/5.jpg)
Staircase effectStaircase effect
• Bilateral filter tends to remove texture, create flat intensity regions and new contours
• Questions
– Why does it occur?
– Can this be an advantage?
– Otherwise, can we solve this problem?
input
output
![Page 6: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/6.jpg)
Why?Why?
• Bilateral filter is a weighted average of intensities and…
space
range
![Page 7: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/7.jpg)
Why?Why?
• The number of points q satisfying Ip-h<Iq<Ip is larger than the number satisfying Ip<Iq<Ip+h.
space
range
![Page 8: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/8.jpg)
Why?Why?
• Thus the average value is smaller than Ip,
enhancing that part of the signal.
Note: Of course, opposite reasoning the the concave case
space
range
![Page 9: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/9.jpg)
And Gaussians don’t change anythingAnd Gaussians don’t change anything
space
range
![Page 10: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/10.jpg)
And Gaussians don’t change anythingAnd Gaussians don’t change anything
space
range
![Page 11: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/11.jpg)
And Gaussians don’t change anythingAnd Gaussians don’t change anything
space
range
![Page 12: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/12.jpg)
So… Can this be an advantage?So… Can this be an advantage?
• Yes! Since we obtain cartoon-like pictures, let us do cartoons!...
[Winnemöller, Olsen, Gooch, 2006]
Input Output
![Page 13: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/13.jpg)
I said cartoons?I said cartoons?
[Winnemöller, Olsen, Gooch, 2006]
![Page 14: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/14.jpg)
Few words about the approachFew words about the approach[Winnemoller, Olsen, Gooch, 2006]
![Page 15: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/15.jpg)
And you can do more!And you can do more!
• Real-time video abstraction
• To know more
[Winnemöller, Olsen, Gooch, 2006]
http://www.cs.northwestern.edu/~holger/Research/VideoAbstraction/
You want to see some example?
![Page 16: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/16.jpg)
![Page 17: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/17.jpg)
![Page 18: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/18.jpg)
But…But…
• We don’t always want to have this kind of rendering
• When bilateral filter is used some side effects car appear
![Page 19: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/19.jpg)
HDR input
![Page 20: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/20.jpg)
Result without correcting the BF output
Tone mapping with look transfer [Bae, Paris and Durand, 2006]
Not acceptable for a photographer!
![Page 21: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/21.jpg)
Can we avoid this defect?Can we avoid this defect?
![Page 22: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/22.jpg)
“Gradient manipulation”“Gradient manipulation”
1. In the gradient domain:
– Compare gradient amplitudes of input and current
– Prevent increase
2. Solve the Poisson equation
Goal of the paper was to control photographic look and transfer a “look” from a model photo
[Bae, Paris and Durand, 2006]
See [Perez etal, 2003] on Poisson image editingSee [Agarwala, 2007] on solving Poisson equation for large images
![Page 23: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/23.jpg)
Result without correcting the BF output
Tone mapping with look transfer [Bae, Paris and Durand, 2006]
![Page 24: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/24.jpg)
Result with corrected BF output
Tone mapping with look transfer [Bae, Paris and Durand, 2006]
Note that problems are essentially visible near strong contours
![Page 25: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/25.jpg)
Edge BlendingEdge Blending
• With a single iteration, staircase effects is visible only at edges.
• Edges detected with normalization factor (see also [Smith and Brady, 1997])
• Blend edges with smoothed version of input to counteract staircase effect
Goal of the paper was the display of high-dynamic-range images
[Durand and Dorsey, 2002]
(Combination between BF and Gaussian results at strong contours locations)
![Page 26: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/26.jpg)
Result without correctionResult without correction Result with correctionResult with correctionTone Mapping [Durand 02]Tone Mapping [Durand 02]
![Page 27: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/27.jpg)
“Linear interpolation”“Linear interpolation”
• We saw that bilateral filter behaves like Perona-Malik and thus creates flat zones
• They proposed to replace the simple average by a linear regression
• How?
Goal of the paper was to establish the link between integral formulations and differential operators
[Buades, Coll, Morel, 2005]
![Page 28: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/28.jpg)
• Bilateral filter can be expressed by
• If you derive, you obtain
“Linear interpolation”“Linear interpolation”
![Page 29: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/29.jpg)
• Bilateral filter can be expressed by
• [Buades, Coll, Morel, 2005] changed the constant model by an affine model
• New value at p will be
“Linear interpolation”“Linear interpolation”
![Page 30: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/30.jpg)
“Linear interpolation”“Linear interpolation”
• Remember, the problem was that lower values were more taken into consideration
Geometrical interpretation
space
range
![Page 31: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/31.jpg)
• Now, left and right-hand side parts have the same influence
“Linear interpolation”“Linear interpolation”Geometrical interpretation
space
range
![Page 32: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/32.jpg)
Staircase effectStaircase effect
Input Bilateral filter
![Page 33: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/33.jpg)
With linear interpolation…With linear interpolation…
Input Bilateral filter modified
[Buades, Coll, Morel, 2005]
![Page 34: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/34.jpg)
Also…Also…
• This new operator is also related to differential operators, i.e., PDEs!
• In this paper, you will also find extensions of bilateral filter, called non local filter.
[Buades, Coll, Morel, 2005]
Average when similar intensities
Average when similar patch around
(correlation of neighborhood)
![Page 35: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/35.jpg)
How to choose?How to choose?
• Two methods which correct afterward defects of bilateral filter, mainly visible on boundaries.
Efficient
Correction of an existing problem
• One method which solves the problem by adapting the bilateral filter.
Directly address the problem
Computationally expensive
![Page 36: A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20](https://reader035.vdocuments.us/reader035/viewer/2022081520/56649dbe5503460f94ab221c/html5/thumbnails/36.jpg)
SummarySummary
• Bilateral filter produces staircase effect
• It has been used as a tool for many applications such as texture extraction
• By itself, it has some interest too!
• Staircase effect can be controlled
• The link with PDEs is again appearing