fast colour2grey ali alsam and mark s. drew the scientific department school of computing science...
Post on 18-Dec-2015
215 views
TRANSCRIPT
![Page 1: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/1.jpg)
Fast Colour2GreyFast Colour2Grey
Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science
The National Gallery London Simon Fraser [email protected] [email protected]
![Page 2: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/2.jpg)
2
What’s the colour2grey problem?• A black and white image has one channel.
• Ok, so make from RGB: use some form of Luminance – e.g., L*, Y(IQ), Y(UV), Y(CbCr),… or use PCA to find main direction
• What about a multi-spectral satellite image: 200 channels?
• Well, start with RGB colour images – how about averaging, say:
3.0
yx,B+yx,G+yx,R=yx,g
![Page 3: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/3.jpg)
3
Oops – equi-luminous images:
Very common situation, from graphics, so need more sophistication.
![Page 4: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/4.jpg)
4
The problem geometrically: say in L*a*b*:
Obviously, very different colours project to identical grey values.
![Page 5: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/5.jpg)
5
Many approaches:
• [Bala & Eschbach, CIC2004] -- a component related to edges of the chromatic 2-vector is added to the brightness: spatially adaptive.
• [Gooch et al., SIGGRAPH05] – Color2Gray: iterative optimization (with multiple local minima!) to map colours to grey maintaining chromatic plus luminance difference as well as possible.
• [Grundland & Dodgson, Patt.Rec. 2007] – a single achromatic channel to optimize chromatic contrast: a combination of luminance and a global “predominant” chromatic change
coordinate must set several parameters, need user input.
E.g.,
![Page 6: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/6.jpg)
6
Color2Gray is likely the most well-known. Again, needs several parameters to be set by the user:
-- so-called “aesthetic decisions”:• choose a colour direction for “cooler” to “hotter” colours• choose a window size for the optimization• choose a clamping value to clip the effect of a*,b* on L*
Algorithm is very slow (unconstrained optimization over thousands of variables)… Also,
![Page 7: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/7.jpg)
Parameters make trouble:
This paper
Input
Gooch
Lum.
Uh-oh! For this image (© Jay Neitz) only colorblind people should see the “45”!
![Page 8: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/8.jpg)
8
Another example:Lum.
This paper
Gooch
![Page 9: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/9.jpg)
9
We could of course improve the Gooch et al. output by fixing the parameters manually…[and, some aspects of these algorithms are indeed compelling and could be added to the method presented here!]
But, can we devise a parameter-free method? (and also, let’s not forget higher-D colour than RGB)
Basic tenet of such algorithms is that we wish to map colour contrast into greyscale contrast.
But what is contrast ? --
![Page 10: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/10.jpg)
10
[Socolinsky & Wolff, IEEE Trans. Image Proc., 2002] – use math definition of what is gradient of an n-dimensional image:
the colour-channel gradient 2-vectors ▽R, ▽G, ▽B, combine to induce a metric for the underlying geometry. The resulting greyscale gradient is in the direction of maximal change.
Contrast:
Details:
[Di Zenzo, 1986]: Define a 2x2 structure tensor (first fundamental form, in math) --- main eigenvector points in the direction of maximal rate of change.
![Page 11: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/11.jpg)
11
What does this actually look like:
* ▽g based on strong mathematical guarantee on results *
![Page 12: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/12.jpg)
12
Details:
In the case of a 3-band color image = {R, G, B}, we consider the 2x3 array of color-channel gradient 2-vectors
▽ = {▽R, ▽G, ▽B} .
Now form the 2x2 array Structure Tensor Z:
200
200
Let’s not forget satellite images…
200
200
![Page 13: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/13.jpg)
Back to RGB:Z is the matrix of outer products of the 2D color-channel gradient components:
(2x3) * (3x2)
Z is real symmetric, so its eigenvectors form an orthogonal matrix V,
The (normalized) eigenvectors of Z point in the direction of minimum- and maximum-contrast, for an underlying grayscale image with metric induced by the structure tensor.
![Page 14: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/14.jpg)
14
Colour to Grey-Gradient:
Let V={u, v} with 2-vectors u v. Then the maximum-eigenvalue direction v is associated with maximum contrast in the grey image, with norm
Grey-Gradient to Grey:
Re-integrating gray-gradient -- take another derivative and solve Poisson’s equation:
![Page 15: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/15.jpg)
15
Done ?
Problem: eigenvectors are defined only up to sign:
±
If we get sign wrong, we have integrability error:
![Page 16: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/16.jpg)
If we make a ± mistake,leads to bends, folds,halos!
Wolff solution: use sign of gradient of Luminance.
Problem: what if image is iso-luminant (common issue).
![Page 17: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/17.jpg)
If we make a ± mistake,leads to bends, folds,halos!
Problem: Iso-luminant:
WolffRGB This paper
![Page 18: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/18.jpg)
18
Our solution: use max’s of ▽R, ▽G, ▽B, in each compass direction:
No
need
to c
onsi
der
± si
gns!
![Page 19: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/19.jpg)
19
Learn the weights ={,,,} by optimizing closeness of colourZ to resulting grey Z:
…
![Page 20: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/20.jpg)
20
Median weights ={0.530, 0.430, 0.537, 0.443};
Can we make faster?
Use Frankot-Chellappa algorithm in Fourier space to solve for grey g from gradient ▽g:
Takes a 2nd derivative, by multiplying by the transform of thederivative operator in the frequency domain, and solvesthe resulting Poisson equation by going back to the spatial domain. Uses ={1,0,1,0}.
Can use FFTW – fast!
and then solving for g:
![Page 21: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/21.jpg)
21
Results:Input Lum. Fast Colour2Grey
![Page 22: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/22.jpg)
22
…Results
Input Lum. This paper
![Page 23: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/23.jpg)
23
…Results
Input
Lum.
This paper
![Page 24: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/24.jpg)
24
…Results
Input
Lum.
This paper
Lum.
![Page 25: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/25.jpg)
25
…Results
Input Lum. Wolff This paper
![Page 26: Fast Colour2Grey Ali Alsam and Mark S. Drew The Scientific Department School of Computing Science The National Gallery London Simon Fraser University ali.alsam@gmail.com](https://reader034.vdocuments.us/reader034/viewer/2022042717/56649d265503460f949fcfed/html5/thumbnails/26.jpg)
26
Thanks!To Natural Sciences and
Engineering Research Council of Canada