shape-from-polarimetry: recovering sea surface topography
DESCRIPTION
Shape-from-Polarimetry: Recovering Sea Surface Topography. Howard Schultz Department of Computer Science University of Massachusetts 140 governors Dr Amherst, MA 01003 hschultz @cs.umass.edu >. October 2011. Outline. Why recover the spatial -temporal structure of ocean waves? - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/1.jpg)
Shape-from-Polarimetry:Recovering Sea Surface Topography
Howard Schultz Department of Computer Science
University of Massachusetts140 governors Dr
Amherst, MA [email protected]>
October 2011
![Page 2: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/2.jpg)
Outline
• Why recover the spatial-temporal structure of ocean waves?• Requirements• What is polarimetry?• What is the Shape-from-Polarimetry?• Build and Test an Imaging Polarimeter for Ocean Apps. • Recent Experiment and Results• Optical Flattening• Seeing Through Waves
![Page 3: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/3.jpg)
• Why recover the structure of the ocean surface?– Characterize small small-scale wave dynamics and microscale breaking– Air-sea interactions occur at short wavelengths– Non-linear interaction studies require phase-resolved surface topography– Enable through-the-wave imaging– Detect anomalies in surface slope statistics
• Why use a passive optical technique– Probes disturb the air-sea interaction– Radar do not produce phase-resolved surfaces– Active techniques are complex and expensive
• Requirements– Spatial resolution (resolve capillary waves) ~ 1mm– Temporal resolution ~60Hz sampling rate– Shutter speed < 1 msec
![Page 4: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/4.jpg)
What is polarimetry?
• Light has 3 basic qualities• Color, intensity and polarization• Humans do not see polarization
![Page 5: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/5.jpg)
Linear Polarization
http://www.enzim.hu/~szia/cddemo/edemo0.htm
![Page 6: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/6.jpg)
Circular Polarization
![Page 7: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/7.jpg)
• A bundle of light rays is characterized by intensity, a frequency distribution (color), and a polarization distribution
• Polarization distribution is characterized by Stokes parametersS = (S0, S1, S2, S3)
• The change in polarization on scattering is described by Muller Calculus
SOUT = M SIN
• Where M contains information about the shape and material properties of the scattering media
• The goal: Measure SOUT and SIN and infer the parameters of M
What is polarimetry?
Amount of circular polarizationOrientation and degree of linear polarizationIntensity
Incident LightMuller MatrixScattered Light
![Page 8: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/8.jpg)
What is Shape-from-Polarimetry (SFP)?
• Use the change in polarization of reflected skylight to infer the 2D surface slope, , for every pixel in the imaging polarimeter’s field-of-view
€
∂z /∂x and ∂z /∂y
![Page 9: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/9.jpg)
What is Shape-from-Polarimetry (SFP)?
![Page 10: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/10.jpg)
What is Shape-from-Polarimetry (SFP)?
€
RAW =
α +η α −η 0 0α −η α +η 0 0
0 0 γ Re 00 0 0 γ Re
⎡
⎣
⎢ ⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥ ⎥
and TWA =
′ α + ′ η ′ α − ′ η 0 0′ α − ′ η ′ α + ′ η 0 00 0 ′ γ Re 00 0 0 ′ γ Re
⎡
⎣
⎢ ⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥ ⎥
€
α =12
tan θ i −θ t( )tan θ i +θ t( )
⎡
⎣ ⎢
⎤
⎦ ⎥2
η =12
sin θ i −θ t( )sin θ i +θ t( )
⎡
⎣ ⎢
⎤
⎦ ⎥2
γRe =tan θ i −θ t( ) sin θ i −θ t( )tan θ i +θ t( ) sin θ i +θ t( )
€
′ α =12
2sin ′ θ i( ) sin ′ θ t( )sin ′ θ i + ′ θ t( ) cos ′ θ i + ′ θ t( )
⎡
⎣ ⎢
⎤
⎦ ⎥2
′ η =12
2sin ′ θ i( ) sin ′ θ t( )sin ′ θ i + ′ θ t( )
⎡
⎣ ⎢
⎤
⎦ ⎥2
′ γ Re =4 sin2 ′ θ i( ) sin2 ′ θ t( )
sin2 ′ θ i + ′ θ t( ) cos2 ′ θ i + ′ θ t( )
SAW = RAWSSKY and SWA = TAWSUP
€
sin θ i( ) = n sin θ t( ) and sin ′ θ i( ) =1n
sin ′ θ t( )
![Page 11: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/11.jpg)
What is Shape-from-Polarimetry (SFP)?
• For RaDyO we incorporated 3 simplifying assumptions– Skylight is unpolarized SSKY = SSKY(1,0,0,0) good for overcast days– In deep, clear water upwelling light can be neglected
SWA = (0,0,0,0). – The surface is smooth within the pixel field-of-view
€
DOLP θ( ) =S1
2 + S22
S02 and φ =
12
tan−1 S2
S1
⎛ ⎝ ⎜
⎞ ⎠ ⎟+ 90°
![Page 12: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/12.jpg)
What is Shape-from-Polarimetry (SFP)?
![Page 13: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/13.jpg)
How well does the SFP technique work?
• Conduct a feasibility study– Rented a linear imaging polarimeter– Laboratory experiment
• setup a small 1m x 1m wavetank• Used unpolarized light• Used wire gauge to simultaneously measure wave profile
– Field experiment• Collected data from a boat dock• Overcast sky (unpolarized)• Used a laser slope gauge
![Page 14: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/14.jpg)
![Page 15: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/15.jpg)
Looking at 90 to the wavesLooking at 45 to the wavesLooking at 0 to the waves
![Page 16: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/16.jpg)
![Page 17: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/17.jpg)
Slope in Degrees
X-Component
Y-Component
![Page 18: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/18.jpg)
X-Component Y-Component
Slope in Degrees
![Page 19: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/19.jpg)
![Page 20: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/20.jpg)
Build and Test an Imaging Polarimeter for Oceanographic Applications
–Funded by an ONR DURIP–Frame rate 60 Hz–Shutter speed as short as 10 μsec–Measure all Stokes parameters–Rugged and light weight–Deploy in the Radiance in a Dynamic
Ocean (RaDyO) research initiativehttp://www.opl.ucsb.edu/radyo/
![Page 21: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/21.jpg)
Motorized Stage12mm travel5mm/sec max speed
ObjectiveAssembly
Polarizing beamsplitterassembly
Camera 1(fixed)
Camera 2
Camera 3Camera 4
![Page 22: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/22.jpg)
![Page 23: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/23.jpg)
FLIP INSTRUMENTATION SETUPScanning Altimeters
Infrared Camera
Air-Sea Flux Package
Polarimeter
Visible Camera
![Page 24: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/24.jpg)
Sample Results
• A sample dataset from the Santa Barbara Channel experiment was analyzed
• Video 1 shows the x- and y-slope arrays for 1100 frames• Video 2 shows the recovered surface (made by
integrating the slopes) for the first 500 frames
![Page 25: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/25.jpg)
Sample Results
![Page 26: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/26.jpg)
X and Y slope field
![Page 27: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/27.jpg)
Convert slope arrays to a height array
Use the Fourier derivative theorem
€
sX =∂h∂x
, sY =∂h∂y
ˆ s X = F sX( ), ˆ s Y = F sY( )
ikXˆ h = ˆ s X , iky
ˆ h = ˆ s Y
ˆ h =−ikX ˆ s X − ikY ˆ s Y
k 2
h = F −1 ˆ h ( )
![Page 28: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/28.jpg)
Reconstructed Surface Video
![Page 29: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/29.jpg)
Seeing Through Waves
• Sub-surface to surface imaging• Surface to sub-surface imaging
![Page 30: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/30.jpg)
Optical Flattening
![Page 31: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/31.jpg)
Optical Flattening
• Remove the optic distortion caused by surface waves to make it appear as if the ocean surface was flat– Use the 2D surface slope field to find the
refracted direction for each image pixel– Refraction provides sufficient information to
compensate for surface wave distortion– Real-time processing
![Page 32: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/32.jpg)
Image FormationSubsurface-to-surface
Imaging Array
Exposure Center
Observation RaysAir
Water
![Page 33: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/33.jpg)
Image Formationsurface-to-subsurface
Imaging Array
Exposure Center
Air
Water
Imaging Array
Exposure Center
![Page 34: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/34.jpg)
Seeing Through Waves
![Page 35: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/35.jpg)
0 20 40 60 80 0 10 20 30 40
Seeing Through Waves
![Page 36: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/36.jpg)
Optical Flattening
• Remove the optic distortion caused by surface waves to make it appear as if the ocean surface was flat– Use the 2D surface slope field to find the
refracted direction for each image pixel– Refraction provides sufficient information to
compensate for surface wave distortion– Real-time processing
![Page 37: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/37.jpg)
Un-distortionA lens maps incidence angle θ to image position X
Lens
Imaging Array
X
θ
![Page 38: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/38.jpg)
X
θ
Lens
Imaging Array
Un-distortionA lens maps incidence angle θ to image position X
![Page 39: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/39.jpg)
X
Lens
Imaging Array
Un-distortionA lens maps incidence angle θ to image position X
![Page 40: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/40.jpg)
X
θ
Lens
Imaging Array
Un-distortionA lens maps incidence angle θ to image position X
![Page 41: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/41.jpg)
X
θ
Lens
Imaging Array
Un-distortionA lens maps incidence angle θ to image position X
![Page 42: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/42.jpg)
Distorted Image Point
Image array
Un-distortionUse the refraction angle to “straighten out” light
rays
Air
Water
![Page 43: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/43.jpg)
Un-distorted Image Point
Image array
Un-distortionUse the refraction angle to “straighten out” light
rays
Air
Water
![Page 44: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/44.jpg)
Real-time Un-Distortion• The following steps are taken Real-time
Capable– Collect Polarimetric Images ✔– Convert to Stokes Parameters ✔– Compute Slopes (Muller Calculus) ✔– Refract Rays (Lookup Table) ✔– Remap Rays to Correct Pixel ✔
![Page 45: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/45.jpg)
Image Formationsurface-to-subsurface
Imaging Array
Exposure Center
Air
Water
Imaging Array
Exposure Center
![Page 46: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/46.jpg)
Detecting Submerged Objects“Lucky Imaging”
• Use refraction information to keep track of where each pixel (in each video frame) was looking in the water column
• Build up a unified view of the underwater environment over several video frames
• Save rays that refract toward the target area• Reject rays that refract away from the target
area
![Page 47: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/47.jpg)
Questions?
![Page 48: Shape-from-Polarimetry: Recovering Sea Surface Topography](https://reader036.vdocuments.us/reader036/viewer/2022062410/56815f4a550346895dce2684/html5/thumbnails/48.jpg)
For more information contactHoward SchultzUniversity of MassachusettsDepartment of Computer Science140 Governors DriveAmherst, MA 01003Phone: 413-545-3482Email: [email protected]