style-aware robust color transfer
TRANSCRIPT
![Page 1: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/1.jpg)
1/51
Introduction Our method Summary
Style-aware robust color transfer
H. Hristova O. Le Meur R. Cozot K. Bouatouch
IRISA - University of Rennes 1
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 2: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/2.jpg)
2/51
Introduction Our method Summary
Outline
1 IntroductionContextObjectiveContributions
2 Our methodAlgorithmEvaluationResults
3 Summary
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 3: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/3.jpg)
3/51
Introduction Our method Summary
Context
Outline
1 IntroductionContextObjectiveContributions
2 Our methodAlgorithmEvaluationResults
3 Summary
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 4: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/4.jpg)
4/51
Introduction Our method Summary
Context
Color transfer domain
Objective of color transfer
Color transfer aims at modifying the look of an input imageconsidering the illumination and the color palette of a referenceimage (e.g., considering statistical characteristics of the light andcolor distributions, histograms, gradients, etc.).
Input Our result Reference
Transformationf(µ, Σ, φ, ...)
light, colors light, colors
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 5: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/5.jpg)
5/51
Introduction Our method Summary
Context
State-of-the-art global methods
� [Reinhard et al., 2001]� Linear mapping� L��� Diagonal covariance matrix� Gaussian distribution
� [Piti�e et al., 2007]� Linear mapping as a solution to Monge-Kantorovich
optimization problem� CIE Lab� Non-diagonal covariance matrix� Gaussian distribution
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 6: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/6.jpg)
6/51
Introduction Our method Summary
Context
State-of-the-art local methods
� [Tai et al., 2005]� Linear mapping: [Reinhard et al., 2001]� L��� 3D image clustering� Cluster mapping: maps the closest clusters in terms of their
luminance channel values� Gaussian distribution (cluster-wise)
� [Bonneel et al., 2013]� Linear mapping: [Piti�e et al., 2007]� CIE Lab� Image clustering in terms of the luminance channel� Cluster mapping: shadows to shadows, midtones to midtones,
highlights to highlights� Gaussian distribution (cluster-wise)
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 7: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/7.jpg)
7/51
Introduction Our method Summary
Context
Limitations
Style inconsistency
Inp
ut
imag
eR
efe
rence
im
age
[Rein
hard
et
al. 2
00
1]
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 8: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/8.jpg)
7/51
Introduction Our method Summary
Context
Limitations
Style inconsistency
Inp
ut
imag
eR
efe
rence
im
age
[Rein
hard
et
al. 2
00
1]
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 9: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/9.jpg)
7/51
Introduction Our method Summary
Context
Limitations
Style inconsistency
Inp
ut
imag
eR
efe
rence
im
age
[Rein
hard
et
al. 2
00
1]
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 10: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/10.jpg)
8/51
Introduction Our method Summary
Context
Limitations
Style inconsistency
Gaussian distribution
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 11: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/11.jpg)
8/51
Introduction Our method Summary
Context
Limitations
Style inconsistency
Gaussian distribution
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 12: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/12.jpg)
9/51
Introduction Our method Summary
Context
Limitations
Style inconsistency
Gaussian distribution
Clustering on the samecolor space components
for input/reference images
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 13: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/13.jpg)
9/51
Introduction Our method Summary
Context
Limitations
Style inconsistency
Gaussian distribution
Clustering on the samecolor space components
for input/reference images
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 14: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/14.jpg)
10/51
Introduction Our method Summary
Context
Limitations
Style inconsistency
Gaussian distribution
Clustering on the samecolor space components
for input/reference images
Over-saturation orlack of contrast
[Pitie et al., 2007]
[Reinhard et al., 2001]
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 15: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/15.jpg)
10/51
Introduction Our method Summary
Context
Limitations
Style inconsistency
Gaussian distribution
Clustering on the samecolor space components
for input/reference images
Over-saturation orlack of contrast
[Pitie et al., 2007]
[Reinhard et al., 2001]
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 16: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/16.jpg)
11/51
Introduction Our method Summary
Context
Limitations
Style inconsistency
Gaussian distribution
Clustering on the samecolor space components
for input/reference images
Over-saturation orlack of contrast
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 17: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/17.jpg)
12/51
Introduction Our method Summary
Objective
Outline
1 IntroductionContextObjectiveContributions
2 Our methodAlgorithmEvaluationResults
3 Summary
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 18: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/18.jpg)
13/51
Introduction Our method Summary
Objective
Our objective
Transferring the light and color features of a reference image to aninput image with respect to the reference style features. Obtaininga naturally looking image which style features are as close aspossible to those of the reference style.
Style notion in our context
In our method, style in images is characterized by two features:color and light.
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 19: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/19.jpg)
14/51
Introduction Our method Summary
Objective
Example
Input Reference
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 20: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/20.jpg)
15/51
Introduction Our method Summary
Objective
Example
Input Reference
[Pit
ie e
t al.
, 2007]
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 21: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/21.jpg)
16/51
Introduction Our method Summary
Objective
Example
Input Reference
[Pit
ie e
t al.
, 2007]
[Bon
neel et
al.
, 2013]
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 22: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/22.jpg)
17/51
Introduction Our method Summary
Objective
Example
Input Reference
[Pit
ie e
t al.
, 2007]
[Bon
neel et
al.
, 2013]
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 23: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/23.jpg)
18/51
Introduction Our method Summary
Contributions
Outline
1 IntroductionContextObjectiveContributions
2 Our methodAlgorithmEvaluationResults
3 Summary
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 24: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/24.jpg)
19/51
Introduction Our method Summary
Contributions
� Classi�cation of the input and reference images� Light-based style images� Colors-based style images
� Two ways of clustering:� Luminance channel� Luminance-Hue distributions
� Four mapping policies: Light to Light, Light to Colors, Colorsto Light, Colors to Colors
� Evaluation of the results� Subjective user study� Objective metrics
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 25: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/25.jpg)
19/51
Introduction Our method Summary
Contributions
� Classi�cation of the input and reference images� Light-based style images� Colors-based style images
� Two ways of clustering:� Luminance channel� Luminance-Hue distributions
� Four mapping policies: Light to Light, Light to Colors, Colorsto Light, Colors to Colors
� Evaluation of the results� Subjective user study� Objective metrics
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 26: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/26.jpg)
19/51
Introduction Our method Summary
Contributions
� Classi�cation of the input and reference images� Light-based style images� Colors-based style images
� Two ways of clustering:� Luminance channel� Luminance-Hue distributions
� Four mapping policies: Light to Light, Light to Colors, Colorsto Light, Colors to Colors
� Evaluation of the results� Subjective user study� Objective metrics
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 27: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/27.jpg)
19/51
Introduction Our method Summary
Contributions
� Classi�cation of the input and reference images� Light-based style images� Colors-based style images
� Two ways of clustering:� Luminance channel� Luminance-Hue distributions
� Four mapping policies: Light to Light, Light to Colors, Colorsto Light, Colors to Colors
� Evaluation of the results� Subjective user study� Objective metrics
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 28: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/28.jpg)
20/51
Introduction Our method Summary
Algorithm
Outline
1 IntroductionContextObjectiveContributions
2 Our methodAlgorithmEvaluationResults
3 Summary
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 29: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/29.jpg)
21/51
Introduction Our method Summary
Algorithm
Framework
light, colors
light, colors
Inputimage
Referenceimage
Classification
Clustering
Clustering
Chromaticadaptation
Colortransfer
Mer
ging
pro
cess
Mappingpolicies
Inputclusters
Set of correspondingclusters
Finalimage
Referenceclusters
Classification
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 30: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/30.jpg)
22/51
Introduction Our method Summary
Algorithm
Automatic image classi�cation system
Colors-based style images
Images which color information is su�cient enough to well-de�neat least two di�erent and signi�cant colors.
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 31: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/31.jpg)
23/51
Introduction Our method Summary
Algorithm
Automatic image classi�cation system
Light-based style images
Images which are not classi�ed as colors-based style images areclassi�ed as light-based style images. Their light features are moremeaningful than their color features.
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 32: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/32.jpg)
24/51
Introduction Our method Summary
Algorithm
Framework
light, colors
light, colors
Inputimage
Referenceimage
Classification
Clustering
Clustering
Chromaticadaptation
Colortransfer
Mer
ging
pro
cess
Mappingpolicies
Inputclusters
Set of correspondingclusters
Finalimage
Referenceclusters
Classification
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 33: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/33.jpg)
25/51
Introduction Our method Summary
Algorithm
Clustering: Gaussian mixture models
Light-based style image Colors-based style image
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 34: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/34.jpg)
26/51
Introduction Our method Summary
Algorithm
Framework
light, colors
light, colors
Inputimage
Referenceimage
Classification
Clustering
Clustering
Chromaticadaptation
Colortransfer
Mer
ging
pro
cess
Mappingpolicies
Inputclusters
Set of correspondingclusters
Finalimage
Referenceclusters
Classification
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 35: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/35.jpg)
27/51
Introduction Our method Summary
Algorithm
Pseudo code algorithm
for k = 1; : : : ; N do
if Light to Colors then
ILab := FindDarkestCluster()JLab := FindColdestCluster()
end if
if Colors to Light then
ILab := FindColdestCluster()JLab := FindDarkestCluster()
end if
if Light to Light then
ILab := FindDarkestCluster()JLab := FindDarkestCluster()
end if
if Colors to Colors then
[ILab , JLab ] := FindMinDistPair()end if
OLab := PerformTranspOnAB(ILab , JLab)Exclude ILchExclude JLch
end for
Ofinalrgb := CATLocal(Orgb; Jrgb)
� Darkest cluster - the clusterwhich centroid has the minimumluminance value
� Coldest cluster - the clusterwhich centroid has themaximum hue value
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 36: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/36.jpg)
28/51
Introduction Our method Summary
Algorithm
Mapping policies
for k = 1; : : : ; N do
if Light to Colors then
ILab := FindDarkestCluster()JLab := FindColdestCluster()
end if
if Colors to Light then
ILab := FindColdestCluster()JLab := FindDarkestCluster()
end if
if Light to Light then
ILab := FindDarkestCluster()JLab := FindDarkestCluster()
end if
if Colors to Colors then
[ILab , JLab ] := FindMinDistPair()end if
OLab := PerformTranspOnAB(ILab , JLab)Exclude ILchExclude JLch
end for
Ofinalrgb := CATLocal(Orgb; Jrgb)
� Darkest cluster - the clusterwhich centroid has the minimumluminance value
� Coldest cluster - the clusterwhich centroid has themaximum hue value
� Light to Colors
� Input light-based styleimage
� Reference colors-basedstyle image
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 37: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/37.jpg)
29/51
Introduction Our method Summary
Algorithm
Mapping policies
for k = 1; : : : ; N do
if Light to Colors then
ILab := FindDarkestCluster()JLab := FindColdestCluster()
end if
if Colors to Light then
ILab := FindColdestCluster()JLab := FindDarkestCluster()
end if
if Light to Light then
ILab := FindDarkestCluster()JLab := FindDarkestCluster()
end if
if Colors to Colors then
[ILab , JLab ] := FindMinDistPair()end if
OLab := PerformTranspOnAB(ILab , JLab)Exclude ILchExclude JLch
end for
Ofinalrgb := CATLocal(Orgb; Jrgb)
� Darkest cluster - the clusterwhich centroid has the minimumluminance value
� Coldest cluster - the clusterwhich centroid has themaximum hue value
� Colors to Light
� Input colors-based styleimage
� Reference light-based styleimage
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 38: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/38.jpg)
30/51
Introduction Our method Summary
Algorithm
Mapping policies
for k = 1; : : : ; N do
if Light to Colors then
ILab := FindDarkestCluster()JLab := FindColdestCluster()
end if
if Colors to Light then
ILab := FindColdestCluster()JLab := FindDarkestCluster()
end if
if Light to Light then
ILab := FindDarkestCluster()JLab := FindDarkestCluster()
end if
if Colors to Colors then
[ILab , JLab ] := FindMinDistPair()end if
OLab := PerformTranspOnAB(ILab , JLab)Exclude ILchExclude JLch
end for
Ofinalrgb := CATLocal(Orgb; Jrgb)
� Darkest cluster - the clusterwhich centroid has the minimumluminance value
� Coldest cluster - the clusterwhich centroid has themaximum hue value
� Light to Light
� Input light-based styleimage
� Reference light-based styleimage
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 39: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/39.jpg)
31/51
Introduction Our method Summary
Algorithm
Mapping policies
for k = 1; : : : ; N do
if Light to Colors then
ILab := FindDarkestCluster()JLab := FindColdestCluster()
end if
if Colors to Light then
ILab := FindColdestCluster()JLab := FindDarkestCluster()
end if
if Light to Light then
ILab := FindDarkestCluster()JLab := FindDarkestCluster()
end if
if Colors to Colors then
[ILab , JLab ] := FindMinDistPair()end if
OLab = PerformTranspOnAB(ILab , JLab)Exclude ILchExclude JLch
end for
Ofinalrgb := CATLocal(Orgb; Jrgb)
� Darkest cluster - the clusterwhich centroid has the minimumluminance value
� Coldest cluster - the clusterwhich centroid has themaximum hue value
� Colors to Colors
� Input colors-based styleimage
� Reference colors-basedstyle image
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 40: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/40.jpg)
32/51
Introduction Our method Summary
Algorithm
Color transformation
for k = 1; : : : ; N do
if Light to Colors then
ILab := FindDarkestCluster()JLab := FindColdestCluster()
end if
if Colors to Light then
ILab := FindColdestCluster()JLab := FindDarkestCluster()
end if
if Light to Light then
ILab := FindDarkestCluster()JLab := FindDarkestCluster()
end if
if Colors to Colors then
[ILab , JLab ] := FindMinDistPair()end if
OLab := PerformTranspOnAB(ILab , JLab)Exclude ILchExclude JLch
end for
Ofinalrgb := CATLocal(Orgb; Jrgb)
� Color transformation betweeneach two corresponding clusters
� Carried out on the a and b
channels of CIE Lab
� Closed-form solution: solution toMonge-Kantorovich'soptimization problem
� Non-diagonal covariance matrixbetween the channels of thecolor space
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 41: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/41.jpg)
33/51
Introduction Our method Summary
Algorithm
Color transformation
for k = 1; : : : ; N do
if Light to Colors then
ILab := FindDarkestCluster()JLab := FindColdestCluster()
end if
if Colors to Light then
ILab := FindColdestCluster()JLab := FindDarkestCluster()
end if
if Light to Light then
ILab := FindDarkestCluster()JLab := FindDarkestCluster()
end if
if Colors to Colors then
[ILab , JLab ] := FindMinDistPair()end if
OLab := PerformTranspOnAB(ILab , JLab)Exclude ILchExclude JLch
end for
Ofinalrgb := CATLocal(Orgb; Jrgb)
� Color transformation betweeneach two corresponding clusters
� Carried out on the a and b
channels of CIE Lab
� Closed-form solution: solution toMonge-Kantorovich optimizationproblem
� Non-diagonal covariance matrixbetween the channels of thecolor space
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 42: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/42.jpg)
34/51
Introduction Our method Summary
Algorithm
Chromatic adaptation
for k = 1; : : : ; N do
if Light to Colors then
ILab := FindDarkestCluster()JLab := FindColdestCluster()
end if
if Colors to Light then
ILab := FindColdestCluster()JLab := FindDarkestCluster()
end if
if Light to Light then
ILab := FindDarkestCluster()JLab := FindDarkestCluster()
end if
if Colors to Colors then
[ILab , JLab ] := FindMinDistPair()end if
OLab := PerformTranspOnAB(ILab , JLab)Exclude ILchExclude JLch
end for
Ofinalrgb := CATLocal(Orgb; Jrgb)
� Adapting pixel-wisely the colorsof the output to the referenceilluminant
� Input illuminant: \white image"obtained by performing Gaussianlow-pass �lter
� Reference illuminant estimation:Gray World assumption[FSDP14, HCWxW06]
� Local CAT algorithm [KJF07]
� Avoids undesired colorsaturation, prevents the resultfrom becoming at and reducesfalse colors in the result
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 43: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/43.jpg)
35/51
Introduction Our method Summary
Evaluation
Outline
1 IntroductionContextObjectiveContributions
2 Our methodAlgorithmEvaluationResults
3 Summary
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 44: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/44.jpg)
36/51
Introduction Our method Summary
Evaluation
Subjective evaluation
� 15 users
� Results from 4
state-of-the-arts
� Results from our
method
� Style match
evaluation
� Aesthetic
pleasingness
evaluation
� 5-point scale
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 45: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/45.jpg)
37/51
Introduction Our method Summary
Evaluation
Subjective evaluation: the analysis
Ours Pitie Reinhard Bonneel Tai
23
45
Ours Pitie Reinhard Bonneel Tai
Sty
le s
core
**
******
Ours Pitie Reinhard Bonneel Tai
12
34
5
Ours Pitie Reinhard Bonneel TaiA
esth
etic
ssc
ore
*****
*
� Box-and-Whisker plots
� Paired T-tests
� Correlation between the two scores
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 46: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/46.jpg)
38/51
Introduction Our method Summary
Evaluation
Objective evaluation
SSIM
It measures the degree of artifacts in the result. It is appliedbetween the input image and the result.
S(x ; y) = f (c(x ; y); s(x ; y)) (1)
The luminance component is removed from the computation of themetrics leaving the structural and contrast components.
Bhattacharya coe�cient
Measures the distance between the result/reference histograms[ATR98] of the components of CIE Lab color space.
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 47: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/47.jpg)
39/51
Introduction Our method Summary
Evaluation
Objective evaluation: the analysis
(a) Our method (b) Pitié et al. (c) Bonneel at al.(d) Reinhard et al. (e) Tai et al.
SSIM
Bhattachary
a
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 10.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1
2
3
4
5
6
7
8
9x 10
−3
SSIM
Bhattachary
a
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 10.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1
2
3
4
5
6
7x 10
−3
SSIM
Bhattachary
a
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 10.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1
2
3
4
5
6
7x 10
−3
SSIM
Bhattachary
a
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 10.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1
2
3
4
5
6
7x 10
−3
SSIM
Bhattachary
a
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 10.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
5.5x 10
−3
� Joint distribution of SSIM and Bhattacharya coe�cient
� Paired T-tests
� (SSIM, Bhattacharya) = (1, 1): optimal for the pair
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 48: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/48.jpg)
40/51
Introduction Our method Summary
Results
Outline
1 IntroductionContextObjectiveContributions
2 Our methodAlgorithmEvaluationResults
3 Summary
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 49: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/49.jpg)
41/51
Introduction Our method Summary
Results
Results: Light to Colors
Result
Input image
Reference image
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 50: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/50.jpg)
42/51
Introduction Our method Summary
Results
Results: Colors to Light
Input image
Reference image Result
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 51: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/51.jpg)
43/51
Introduction Our method Summary
Results
Results: Light to Light
Result
Input image
Reference image
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 52: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/52.jpg)
44/51
Introduction Our method Summary
Results
Results: Colors to Colors
Result
Input image
Reference image
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 53: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/53.jpg)
45/51
Introduction Our method Summary
Results
Results: classical example
Result
Input image
Reference image
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 54: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/54.jpg)
46/51
Introduction Our method Summary
Summary
� New method for style transfer for a wide class of image pairs
� Automatic image classi�cation
� Four mappings policies
� Subjective and objective evaluations
� Future work� Enriching the image feature space� Finding a connection between the subjective user study and
the objective evaluation
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 55: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/55.jpg)
47/51
Introduction Our method Summary
Thank you for your attention!
More results and information on
http: // people. irisa. fr/ Hristina. Hristova
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 56: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/56.jpg)
48/51
Appendix
References
References I
Aherne F. J., Thacker N. A., Rockett P. I.:The bhattacharyya metric as an absolute similarity measure forfrequency coded data.Kybernetika 34, 4 (1998), 363{368.
Bonneel N., Sunkavalli K., Paris S., Pfister H.:Example-based video color grading.ACM Transactions on Graphics (Proceedings of SIGGRAPH
2013) 32, 4 (2013), 2.
Frigo O., Sabater N., Demoulin V., Pierre H.:Optimal transportation for example-guided color transfer.12th Asian Conference on Computer Vision (ACCV) (2014).
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 57: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/57.jpg)
49/51
Appendix
References
References II
Huo J., Chang Y., Wang J., xia Wei X.:Robust automatic white balance algorithm using gray colorpoints in images.IEEE Trans. Consumer Electronics 52, 2 (2006), 541{546.URL: http://dblp.uni-trier.de/db/journals/tce/tce52.html#HuoCWW06.
Kuang J., Johnson G. M., Fairchild M. D.:icam06: A re�ned image appearance model for hdr imagerendering.Journal of Visual Communication and Image Representation
18, 5 (2007), 406{414.
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 58: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/58.jpg)
50/51
Appendix
References
References III
Piti�e F., Kokaram A. C., Dahyot R.:Automated colour grading using colour distribution transfer.Computer Vision and Image Understanding 107, 1 (2007),123{137.
Reinhard E., Ashikhmin M., Gooch B., Shirley P.:Color transfer between images.Computer Graphics and Applications, IEEE 21, 5 (2001),34{41.
Shih Y., Paris S., Durand F., Freeman W. T.:Data-driven hallucination of di�erent times of day from asingle outdoor photo.ACM Transactions on Graphics (TOG) 32, 6 (2013), 200.
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch
![Page 59: Style-aware robust color transfer](https://reader031.vdocuments.us/reader031/viewer/2022020208/55c3fb7ebb61eb7b198b4605/html5/thumbnails/59.jpg)
51/51
Appendix
References
References IV
Tai Y.-W., Jia J., Tang C.-K.:Local color transfer via probabilistic segmentation byexpectation-maximization.In Computer Vision and Pattern Recognition, 2005. CVPR
2005. IEEE Computer Society Conference on (2005), vol. 1,IEEE, pp. 747{754.
Style-aware robust color transfer H. Hristova, O. Le Meur, R. Cozot, K. Bouatouch