alessandro rizzi john j. mccann · 1 spatial models of color alessandro rizzi john j. mccann...
TRANSCRIPT
-
1
Spatial Modelsof Color
Alessandro RizziJohn J. McCann
International Symposium "Vision by Brains and Machines"November 13th - 17th Montevideo, Uruguay
Thanks to:
Carlo Gatta, Silvia Zuffi, Daniele Marini, Edoardo Provenzi, Angelo Moretti
color is the product of light at certain wavelenghts
color is the product of light at certain wavelenghts
NewtonNewton
color is the product of observer’s mindcolor is the product of observer’s mind
GoetheGoethe
Vision process
Newton
Goethe
Physical Perceptual
OUTLINE• The reality of color• The input• The appearance• Colorimetry vs
appearance• The models• The behavior• Applications
-
2
The physical reality of color
380 nm to 780 nm
Visible light
Solar SPD before and after passing atmosphere
a - zenith sunny sky b - northern sunny sky c – cloudy skyd – direct sunlight (sunny)
(normalized at 556nm)
SPD in nature
Fluorescent bulb
6500°K compensated fluorescent bulb
Low pressure sodium
Fluorescent bulb
Artificial light SPD
Macbeth Color Checker Spectral Reflectances
Dark skin
Light skin
Bluesky Foliage
Blueflower
Bluishgreen
Orange PurplishblueModerate
red PurpleYellowgreen
Orangeyellow
Blue Green Red Yellow Magenta Cyan
White Neutral 8Neutral
6.5Neutral
5Neutral
3.5 Black
ρ(λ)E(λL(λ )) =
The basic ambiguity
-
3
Color does not exist by itself
• Same SPD different color perception
• Same color perception different SPD
The input
Two very different devices Weber-Fechner law
cost=∆II
Stimulus difference
Stimulus magnitude
∆I
I
Light and dark adaptation
Light
Retina
-
4
Retinal cone and rod distribution
Retinal cone and rod distribution No (or bad) color without 3 separate channels
http://www.toledo-bend.com/colorblind/Ishihara.html
0 : Spectrum (45) gm©
400 500 600 700
1.00
0.00
Sensitivity curves of the human eye
Maximum red peak falls on green
Cone spectral sensitivity(normalized)
Cone spectral sensitivity
Max signal ratio
L/M=2
-
5
1.1:11.1:152.7%52.7%
16.5:116.5:194.3%94.3%
L:ML:M
The appearance
Mach bands Hermann illusion
-
6
Lightness perception
Simultaneous contrast
Color does not exist by itself
• Same SPD different color perception
• Same color perception different SPD
(J. Albers)
Assimilation(White effect)
Assimilation High vs low level
-
7
Sensation vs perception• Sensation: ”mode of mental functioning that
is directly associated with stimulation of the organism”;
• Perception: ”mode of mental functioning that include the combination of different sensations and the utilization of past experience in recognizing the objects and facts from which the present stimulation arises”.
committee on Colorimetry of the Optical Society of America in “The Science of Color”
Levels of appearance
Which radiance comes from the two sides ?STIMULUS = VERY DIFFERENT
which paint should a painter use to depict an image of the float ?
SENSATION = SLIGHTLY DIFFERENT
the float is painted with the same paint ?PERCEPTION = THE SAME
sensation: before recognition of shadows, junctions, geometries, belongingness, etc..
SCAs deal with sensation
Importance of the context
-
8
High level ?
Courtesy of John McCann
Colorimetryvs
appearance
Wright (1928) e Guild (1931) experiment
435,8 nm
700 nm
546,1 nm
monochromatic light
Wright(1928)
Guild(1931)
CIE 1931
-
9
(monitor Hitachi)
Phosphors spectral emissions
)(λeL)( )( )( )( λλλλ bBgGrRC ++=
∫∫∫
=
=
=
λλλ
λλλ
λλλ
dzLZ
dyLY
dxLX
e
e
e
)()(
)()(
)()(
1
1
1
∫∫∫
=
=
=
λλλ
λλλ
λλλ
dzCZ
dyCY
dxCX
)()(
)()(
)()(
2
2
2
∫∫∫∫∫∫
===
===
===
λλλλλλ
λλλλλλ
λλλλλλ
dzCdzLZZ
dyCdyLYY
dxCdxLXX
e
e
e
)()()()(
)()()()(
)()()()(
21
21
21
Metameric colors
Color does not exist by itself
• Same SPD different color perception
• Same color perception different SPD
Spectral data CMF triplets reproduced colors
CMF
CMF
CMF goal: preserve metamerism
C.I.E. 1931 C.I.E. 1964
Stiles and Burch (1955)
Demarco, Smith and Pokorny (1992) Stockman, MacLeod and Johnson (1993)
Vos and Walraven (1970)
Thornton(2002)
-
10
CMF spatial normalization
before after
overall
single colors
Bipartite fieldexperiment
CIE 1931
Color spaces generations1 - Physical color spaces (XYZ, RGB ..)2 - Psychophysical color spaces (CIELab,
Munsell ..)3 - Appearance (spatial) color
models/algorithms (CAM, Retinex, ACE ..)
?)(λeL
Vision Process
ρ(λ)
Physical Perceptual
-
11
Color depends on context
color is subject to several changes that depend on
contextSpatial computation
Lightnessconstancy
HVS adaptational mechanisms
HVS
HVS
Not completely
Hipparchus line Gray World
100
95 190
50 100
5 10
Perceived intensity
EstimateIllum. Int.
Illuminantintensity
Different illuminants
HVS
Colorconstancy
HVS
HVS adaptational mechanisms
Not completely
-
12
Which reference ?
D65, D50 ?
which is the “real color” ?
color constancy is controlled by channel maxima
Maximov shoeboxes
White Patch
HVS normalizes towards an hypothetical white reference area.
Von Kries
Sun / Shade
119119
InputGlobal vs local Global vs local
-
13
Edwin Land
Retinex Theory
John McCann
SCIENTIFIC AMERICAN 1959
Edwin Land experimentB/W film
Red filter Green filter
projection
Red filter
Slide projectors
Edwin Land projection
Edwin Land projection Second Land experiment
-
14
Land’s experiment
LS=255 LL=255LM=115
Projector Colorimeter
ES=100 EL=100EM=100
Observer
PINK PINKGRAY
Projector Colorimeter
ES=50 EL=50EM=111
Observer
LS=128 LL=128LM=128
Land’s experiment
The models
Computing the appearance
Retinex
ACE(Automatic Color Equalization)
Color comes from the comparison of the three lightness scales
“Color is the correlation number for several rank orders of lightness” (Land, 1977)
Retinex theoryThe term RETINEX comes from retina-and-cortex since is not clear
where the passage from radiant flux to lightness takes place
RadianceRadiant light power through a unitary solid angle (for a certain direction)measured in [W/m2/sr]
LightnessVisual sensation quality related to a surface that appear lighter or darker in relation to a “local reference white”
Reflectanceratio between radiant flux and reflected radiant flux (spectral)
Physical Perceptual
-
15
Let’s bust a myth
Retinex DOES NOT separate
illumination from reflectance(like human visual system)
ρ(λ)E(λL(λ )) =
Sequential products(Land and McCann 1971)
Random pathsSequential products
serching the global/local neighborooh to compute the ratios
The Retinex algorithm
• Relative lightness along a path is the sum of the log ratios of intensities:
• Reset mechanism (WP)
∑∈
+=pathx x
xjisml I
Ir 1, ,, log.δ
δ = 1, if logI x +1I x
> threshold
0 else
j
i
IkIk+1
The Retinex algorithm
• Relative lightness (R) at a point is the mean value of the relative lightnesses (r) computed along N random paths to the point, separately for each band long, medium, short:
N
rR
N
k
jisml
isml
k∑== 1
,,,
,,
-
16
A key point:how to realize locality
[S. Zeki]
Cortical receptive field distribution
Brownian paths Retinex MLV
Random Spray Retinex Center/surround Retinex (Jobson, Rahman, Woodel)
[ ]),(),(log),(log),( yxIyxFyxIyxR iii ∗−=
22 /),( crKeyxF −=
R(x,y) is Retinex output, I(x,y) is the image, F(x,y) is a neighbor function
• In 1986 Land propose a version without paths;• lightness is computed in a circular neighborood defined by 1/r2
Basic idea: receptive fields
-
17
Why it is not a spatial color algorithm
Filter shape does not depend on the image
Implementation coming from the latter work of Land
Multiresolution Retinex (McCann 99)No pathsMultiresolutionStarting from the smaller:
ratio-product-reset-averageon each levelAt the end oversampling and go to the upper level
Digital flash (HP)
-
18
Computing the appearance
Retinex
ACE(Automatic Color Equalization)
Chromatic/SpatialAdaptation
Dynamic ToneReproduction ScalingIc Rc Oc
subset selection
s(·) function
GW WP scaling referencelocal/globalbalancing
ACE structure Pixel recomputation
) (s=)(iRc ∑≠∈ ijsubsetj ,
)()( jIiI cc −
),( jid
For each channel C
∑≠∈
−=
ijsubsetj
ccc jid
jIiIsiR, ),(
))()(()(
>−
=−
−
otherwise1
thresholdLLifLL
δ1ki,ki,
1ki,
ki,jc
jcj
c
jc
ki,c
Ic(i)-Ic(j) lateral inhibition mechanism
Chromatic/Spatial recomputation (1)
ACERetinex
ratio product1ki,
ki,
jc
jc
LL
−
∑≠∈
−=
ijsubsetj
ccc jid
jIiIsiR, ),(
))()(()(
Reset non-linearity, WP
s(·) non-linearity, WP, GW
ACERetinex
x
j1,1j 2,1
ji,1j i,k+1
ji,k
... ...
...
...
Resetmechanism
Chromatic/Spatial recomputation (2)
-
19
Contrast tuningACE
Linear Saturation Signumd1
x
j1,1j 2,1
ji,1j i,k+1
ji,k
... ...
...
...
ACERetinex
d(·) local/global balancingEuclidean distance
∑≠∈
−=
ijsubsetj
ccc jid
jIiIsiR, ),(
))()(()(
Random paths local/global balancing
Chromatic/Spatial recomputation (3)
Chromatic/SpatialAdaptation
Dynamic ToneReproduction ScalingIc Rc Oc
subset selection
s(·) function
GW WP scaling referencelocal/globalbalancing
ACE structureWP/GW scaling:
Estimated “white” White PatchEstimated “gray” Gray World
)](5.127
5.127[ )( max iRRroundiO cc +=
Dynamic tone reproduction scaling
medium gray point Rmax
R Histogram O Histogram
127 2550
Spatial color models common structure
• First phase: pixel recomputationaccording to the visual (spatial) image characteristics
• Second phase:rescaling according to– available dynamic – Global principles (WP, GW)– rendering intent
The behavior
-
20
Filtering goals
• Qualitative
• Quantitative (parameters, I/O calibration,
dynamic, etc...)
Color Constancy
Original Filtered(255,255,255)
No constraint and no a priori information required
Color constancy by spatial comparison
equal
different
Unwanted color removal
Alternative solution: apply SCA only on Lightness channel
Local filtering effectOriginal Filtered
RGB differences filtered-Original around “128” gray
Contrast correction
Original Filtered Original Filtered
global
local local
global
-
21
Shadow removal
Rearrange contrast locallyDo not separate illuminant and reflectance
Orig Filt O-F
Image dynamic adjustment
Tone remapping (LC)
Original
Filtered ACE
Original
ALL
GW required
Spatial dequantization
WP WP + GW
Equal radiance, different appearance Filtering visual Illusion
211
43
128
128
Original Filtered
Original
Filtered
-
22
HVS spatial computationnever adapts completely
does not estimate illuminantchange according to image content
STABLE UNSTABLE
Colour ConstancyLightness ConstancyContrast constancy ?
Simoultaneous ContrastAssimilationEdge effects
ROBUSTNESSApplications
“Incendio di Borgo” (Raffaello 1514)
Application: image DB
Retinex Prefiltering
Matching ?
Matching !
Application: image DB
Prefilteringfor computer vision or medical imaging Prefiltering for edge detection
-
23
InterfacesUser preferences
0
10
20
30
40
50
60
70
80
luminosità colori leggibilità preferenzaglobale
originaleACE
~8000 cd/m2
~1000 cd/m2
L=205
L=105
8:1 → 2:1
[Gatta]
HDR images remapping Automatic digital movie restoration
Another example
Hue histogram original image
Original image After filtering
Hue histogram result image
Thank you
International Symposium "Vision by Brains and Machines"November 13th - 17th Montevideo, Uruguay