lidar r s p c cu-b lecture 30. lidar data inversion...

LIDAR REMOTE SENSING PROF. XINZHAO CHU CU-BOULDER, SPRING 2016

Lecture 30. Lidar Data Inversion (1) q  Introduction of data inversionq  Resonance-Doppler lidar and basic ideas (clues) for lidar data inversionq  Data retrieval procedure Ø  Procedure overviewØ  Preprocess Ø  Profile process (next lecture)Ø  Main process (next lecture)Ø  Error analysis (next lecture)q  Summary

1


From Raw Data to Physical Parameters

Raw DataNS (R)

T (R)VR (R)nc (R)β (R)α (R)δ (R)… …

PMC, PSC, AerosolsCloudsConstituent Density Sporadic Layers MeteorsClimatologyGWs, Tides, PWs Momentum Flux Heat FluxConstituent FluxInstability … … …

Data Inversion& Error AnalysisData Retrieval

Data Analysis& InterpretationScience Study 2

Use resonance Doppler lidar as an example


Introduction: Lidar Data Inversion q  Lidar data inversion deals with the problems of how to derive meaningful physical parameters from raw data. q  Raw data are usually a column or a row of photon counts, where the positions of photon counts in the column or row mark their time bins, thus the ranges or heights.q  Data inversion is basically a reverse procedure to the development of lidar equation.q  It is necessary to understand the detailed physical procedure from light transmitting, to light propagation, to light interaction with objects, and to light detection, in order to conduct data inversion correctly.q  In this lecture we discuss the data inversion for Na Doppler lidar (K and Fe lidar would be similar) as an example.

3


Resonance Fluorescence Doppler Lidar

4

fa f+f-

0

200

400

600

800

1000

1200

1400

1600

0 1000 2000 3000 4000 5000 6000

fa Channelf+ Channelf- Channel

Phot

on C

ount

s

Bin Number

Raw Data Profiles for 3-Frequency Na Doppler Lidar

Example Lidar Raw Signals


Doppler-Limited Na Spectroscopy

€

σeff (ν) =12πσe

e2 f4ε0mec

An exp −[νn − ν(1−VR /c)]

2

2σe2

'

( )

*

+ ,

n=1

6∑

€

σabs(ν) =12πσD

e2 f4ε0mec

An exp −[νn − ν(1−VR /c)]

2

2σD2

'

( )

*

+ ,

n=1

6∑

q  Doppler-broadened Na absorption cross-section is approximated as a Gaussian with rms width σD

q  Assume the laser lineshape is a Gaussian with rms width σL q  The effective cross-section is the convolution of the atomic absorption cross-section and the laser lineshape

€

σe = σD2 + σL

2

€

σD =kBTMλ0

2where and

The effective cross-section depends on both T and VR. How to infer both variables from the lidar-measured σeff? 5


Basic Clue (1): Lidar Equation & Solution q  From lidar equation and its solution to derive preprocess procedure of lidar data inversion

€

NS (λ,z) =PL (λ)Δthc λ

$

% &

'

( ) σeff (λ,z)nc(z)RB (λ) + 4πσR (π ,λ)nR (z)[ ]Δz A

4πz2$

% &

'

( )

× Ta2(λ)Tc

2(λ,z)( ) η(λ)G(z)( ) + NB

€

NS (λ ,zR ) =PL (λ)Δthc λ

$

% &

'

( ) σR (π ,λ)nR (zR )[ ]Δz A

zR2

$

% & &

'

( ) ) Ta

2(λ ,zR ) η(λ)G(zR )( )+NB

€

NNorm (λ,z) =NNa(λ,z)

NR (λ,zR )Tc2(λ,z)

z2

zR2 =

σeff (λ,z)nc(z)σR (π ,λ)nR (zR )

14π

=NS (λ,z) − NBNS (λ,zR ) − NB

z2

zR2

1Tc2(λ,z)

−nR (z)nR (zR )

+

ò

6

Physics

Photon Counts


Principle of Doppler Ratio Technique q  Lidar equation for resonance fluorescence (Na, K, or Fe)

€

NS (λ ,z) =PL (λ)Δthc λ

$

% &

'

( ) σ eff (λ ,z)nc(z)RB (λ) +σR (π ,λ)nR (z)[ ]Δz A

4πz2$

% &

'

( )

× Ta2(λ)Tc

2(λ ,z)( ) η(λ)G(z)( )+NB

RB = 1 for current Na Doppler lidar since return photons at all wavelengths are received by the broadband receiver, so no fluorescence is filtered off.

€

NNa(λ ,z) =PL (λ)Δthc λ

$

% &

'

( ) σ eff (λ ,z)nc(z)[ ]Δz A

4πz2$

% &

'

( ) Ta

2(λ)Tc2(λ ,z)( ) η(λ)G(z)( )

€

NR (λ ,z) =PL (λ)Δthc λ

$

% &

'

( ) σR (π ,λ)nR (z)[ ]Δz A

z2$

% &

'

( ) Ta

2(λ)Tc2(λ ,z)( ) η(λ)G(z)( )

q  Pure Na signal and pure Rayleigh signal in Na region are

q  So we have

€

NS (λ,z) = NNa (λ,z) + NR (λ,z) + NB 7


Principle of Doppler Ratio Technique q  Lidar equation at pure molecular scattering region (35-55km)

€

NS (λ ,zR ) =PL (λ)Δthc λ

$

% &

'

( ) σR (π ,λ)nR (zR )[ ]Δz A

zR2

$

% & &

'

( ) ) Ta

2(λ ,zR ) η(λ)G(zR )( )+NB

q  Pure Rayleigh signal in molecular scattering region is

q  So we have

€

NS (λ,zR ) = NR (λ,zR ) + NB

€

NR (λ,zR ) =PL (λ)Δthc λ

$

% &

'

( ) σR (π ,λ)nR (zR )[ ]Δz A

zR2

$

% &

'

( ) Ta

2(λ,zR ) η(λ)G(zR )( )

€

NR (λ ,z)NR (λ ,zR )

=σR (π ,λ)nR (z)[ ]Ta2(λ ,z)Tc2(λ ,z)G(z)σR (π ,λ)nR (zR )[ ]Ta2(λ ,zR )G(zR )

zR2

z2=nR (z)nR (zR )

zR2

z2Tc2(λ ,z)

q  The ratio between Rayleigh signals at z and zR is given by

Where nR is the (total) atmospheric number density, usually obtained from atmospheric models like MSIS00. 8


Principle of Doppler Ratio Technique q  From above equations, the pure Na and Rayleigh signals are

€

NNa (λ,z) = NS (λ,z) − NB − NR (λ,z)

€

NR (λ,zR ) = NS (λ,zR ) − NB

q  Normalized Na photon count is defined as

€

NNorm (λ ,z) =NNa(λ ,z)

NR (λ ,zR )Tc2(λ ,z)

z2

zR2

€

NNorm (λ,z) =NNa(λ,z)

NR (λ,zR )Tc2(λ,z)

z2

zR2 =

σeff (λ,z)nc(z)σR (π ,λ)nR (zR )

14π

q  From physics point of view, the normalized Na count is

q  From actual photon counts, the normalized Na count is

€

NNorm (λ ,z) =NNa(λ ,z)

NR (λ ,zR )Tc2(λ ,z)

z2

zR2 =

NS (λ ,z) −NB −NR (λ ,z)NR (λ ,zR )Tc

2(λ ,z)z2

zR2

=NS (λ ,z) −NBNS (λ ,zR ) −NB

z2

zR2

1Tc2(λ ,z)

−nR (z)nR (zR ) 9


Basic Clue (2): Ratio Computation q  From physics, we calculate the ratios of RT and RW as

€

RT =σeff ( f+ ,z) +σ eff ( f− ,z)

σ eff ( fa ,z)

€

RW =σeff ( f+ ,z) −σ eff ( f− ,z)

σ eff ( fa ,z)

q  From actual photon counts, we calculate the ratios as

RT =NNorm( f+, z)+ NNorm( f−, z)

NNorm( fa, z)RW =

NNorm( f+, z)− NNorm( f−, z)NNorm( fa, z)

10

Note: There are other formats of temperature and wind ratios RT and RW that are much more complicated than these two simple ratios. The purpose is to reduce the cross-talk between temperature and LOS wind.


Principle of Doppler Ratio Technique q  From physics, the ratios of RT and RW are then given by

Here, Rayleigh backscatter cross-section is regarded as the same for three frequencies, since the frequency difference is so small. Na number density is also the same for three frequency channels, and so is the atmosphere number density at Rayleigh normalization altitude.

€

RT =NNorm ( f+ ,z) +NNorm ( f− ,z)

NNorm ( fa ,z)=

σ eff ( f+ ,z)nc(z)σR (π , f+)nR (zR )

+σ eff ( f− ,z)nc(z)σR (π , f−)nR (zR )

σ eff ( fa ,z)nc(z)σR (π , fa)nR (zR )

=σ eff ( f+ ,z) +σeff ( f− ,z)

σ eff ( fa ,z)

€

RW =NNorm ( f+ ,z) −NNorm ( f− ,z)

NNorm ( fa ,z)=

σ eff ( f+ ,z)nc(z)σR (π , f+)nR (zR )

−σ eff ( f− ,z)nc(z)σR (π , f−)nR (zR )

σ eff ( fa ,z)nc(z)σR (π , fa)nR (zR )

=σ eff ( f+ ,z) −σeff ( f− ,z)

σ eff ( fa ,z)

11


Principle of Doppler Ratio Technique q  From actual photon counts, the ratios RT and RW are

€

RT =NNorm ( f+ ,z) +NNorm ( f− ,z)

NNorm ( fa ,z)

=

NS ( f+ ,z) −NBNS ( f+ ,zR ) −NB

z2

zR2

1Tc2( f+ ,z)

−nR (z)nR (zR )

#

$ % %

&

' ( ( +

NS ( f− ,z) −NBNS ( f− ,zR ) −NB

z2

zR2

1Tc2( f− ,z)

−nR (z)nR (zR )

#

$ % %

&

' ( (

NS ( fa ,z) −NBNS ( fa ,zR ) −NB

z2

zR2

1Tc2( fa ,z)

−nR (z)nR (zR )

€

RW =NNorm ( f+ ,z) −NNorm ( f− ,z)

NNorm ( fa ,z)

=

NS ( f+ ,z) −NBNS ( f+ ,zR ) −NB

z2

zR2

1Tc2( f+ ,z)

−nR (z)nR (zR )

#

$ % %

&

' ( ( −

NS ( f− ,z) −NBNS ( f− ,zR ) −NB

z2

zR2

1Tc2( f− ,z)

−nR (z)nR (zR )

#

$ % %

&

' ( (

NS ( fa ,z) −NBNS ( fa ,zR ) −NB

z2

zR2

1Tc2( fa ,z)

−nR (z)nR (zR )

12


Basic Clue (3): T, VR, nC, or β Derivation

€

βPMC (z) =NS (z) − NB[ ]⋅ z2

NS (zRN ) − NB[ ]⋅ zRN2−

nR (z)nR (zRN )

%

& ' '

(

) * * ⋅ βR (zRN )

€

βR (zRN ,π ) =β4π

P(π) = 2.938 ×10−32 P(zRN )T (zRN)

⋅1

λ4.0117

q  Constituent (e.g., Na) density can be inferred from the peak freq signal

€

nNa(z) =Nnorm ( fa ,z)

σa4πnR (zR )σR =

Nnorm ( fa ,z)σa

4π ×2.938×10−32 P(zR )T (zR )

⋅1

λ4.0117

q  Compute actual ratios RT and RW from photon counts, and then look up these two ratios on the calibration curves to infer the corresponding Temperature (T) and Wind (VR) from isoline/isogram.

q  Volume backscatter coefficient can be derived as

where

q  If there is only RT ratio, then infer the temperature from the calibration curve, like the Fe Boltzmann lidar case.

13


How Does Ratio Technique Work?

14

Isoline / Isogram

180 K

-40 m/s

q  Compute Doppler calibration curves from physicsq  Look up these two ratios on the calibration curves to infer the corresponding Temperature and Wind from isoline/isogram.


Considerations in Data Inversion q  How to obtain related information like date, time, location, base altitude, operation conditions?-- from data header and other info sourcesq  How to obtain range or altitude information?-- from bin number, data header and other source

€

R= nbin ⋅ tbin ⋅c /2

R is range, nbin is bin number, tbin is bin width in time, c is light speed, z is absolute altitude, θ is off-zenith angle, and zbase is the base altitude relative to sea-level.

€

z = R ⋅cosθ + zbase

15


Data Retrieval Procedure

Preprocess Procedure

16

Discriminator

Data inversion is a reverse procedure to lidar equation development.


17

Preprocess Procedure and Profile-Process Procedure for Na/Fe/K Doppler Lidar

Read Data File

PMT/Discriminator Saturation Correction

Chopper/Filter Correction

Subtract Background

Remove RangeDependence ( x R2 )

Add Base Altitude

Estimate Rayleigh Signal@ zR km

Normalize ProfileBy Rayleigh Signal @ zR km

q  Read data: for each set, and calculate T, W, and n for each setq  PMT/Discriminator saturation correctionq  Chopper/Filter correction

q  Background estimate and subtractionq  Range-dependence removal (xR2, not z2)q  Base altitude adjustment q  Take Rayleigh signal @ zR (Rayleigh fit or Rayleigh mean)q  Rayleigh normalization

q  Subtract Rayleigh signals from Na/Fe/K region after counting in the factor of TC

Integration

Main Process

€

NN (λ,z) =NS (λ,z) − NB

NS (λ,zR ) − NB

z2

zR2


Main Process Load Atmosphere nR, T, PProfiles from MSIS00

Start from Na layer bottomTC (z=zb) = 1

Calculate Nnorm (z=zb) from photon counts and MSIS

number density for each freq

Calculate RT and RW from NNorm

Calculate Na density nc(z)

Create look-up table or calibration curves From physics

€

NNorm (λ,z) =NS (λ,z) − NBNS (λ,zR ) − NB

z2

zR2

1Tc2(λ,z)

−nR (z)nR (zR )

€

RT =σeff ( f+ ,z) +σ eff ( f− ,z)

σ eff ( fa ,z)

RW =σeff ( f+ ,z) −σ eff ( f− ,z)

σ eff ( fa ,z)

Are ratios reasonable?

Look-up TableCalibration

No

Yes

Set to nominal values or MSIST = 200 K, W = 0 m/s

Find T and W from the Table

18


Main Process (Continued) Calculate NNorm (z+Δz) from

photon counts and MSIS number density for each freq

Calculate RT(z+Δz) & RW(z+Δz) from NNorm

Calculate Na density nc(z)

Are ratios reasonable?

Look-up TableCalibration

No

Yes

Set to nominal values or MSIST = 200 K, W = 0 m/s

Find T and W from the Table

Calculate TC(z+Δz) Using nc(z) and σeff (z)

Reach Layer Top

No

Yes Save T, W, nc with altitude

19


20

Considerations behind Profile-Process and Preprocess

q  Indicated from the lidar equation and its solution, the profile process for Na Doppler lidar data is Ø  Background estimation and subtraction (- NB)Ø  Range-dependence removal (x R2)Ø  Base altitude adjustment (+ zBase)Ø  Rayleigh normalization [1/(NS(zR)-NB)]

q  More considerations on lidar hardware and detection - preprocess procedureØ  PMT and discriminator saturation correctionØ  Chopper or electronic gain correctionØ  Integration in time and/or range bins to obtain sufficient signal-to-noise ratio (SNR)


21

Step1. Read Raw Data

10240 1 3 10 1 1500 4 11 2 7.060833 0 7 0 1 3.05 20.708 -156.258 12 1543 0 0 1574 4694 … … …

TotalBin #

LowBin #

# ofFreq

SetNo.

ProfileNo.

# ofShots

Month Day

Year-2000

Time (UT)

PhotonCounts

q  Headers + One Column Photon Counts (ASCII or Binary)

AzimuthAngle

BinResolution

Off-ZenithAngle

BaseAltitude(km)

Lat Longi

Frequency Number


22

Another Example of Lidar Raw Data

10240 1 3 11 2 1500 4 11 2 7.090278 180 7 30 2 3.05 20.708 -156.258 12 1532 0 0 2400 3771 … … …

TotalBin #

LowBin #

# ofFreq

SetNo.

ProfileNo.

# ofShots

Month Day

Year-2000

Time (UT)

PhotonCounts

AzimuthAngle

BinResolution

Off-ZenithAngle

BaseAltitude(km)

Lat Longi

Frequency number


23

Step 2. Nonlinearity of PMT + Discriminator For small input photon flux, PMT output photon counts are proportional to the input photon counts:

€

λoP = λS = λiηQE

When the input photon flux is considerably large, the output photon counts are no longer linear with input photons. Nonlinearity of PMT occurs:

€

λoP = λS e−λSτ p

A discriminator is used to judge real photon signals and also has a saturation effect, i.e., its output photon counts are smaller than input photon counts when input count rate is large:

€

λo =λiD

1+ λiDτd


24

Nonlinearity of PMT + Discriminator Since PMT output is the input of discriminator

€

λiD = λoPwe obtain

€

λo =λS e

−λSτ p

1+ λSτd e−λSτ p

=λS e

−λSτ p

1+ λSτd e−λSτ p

where

€

λS = λiηQE ηQE is the quantum efficiency of cathode

Maximum output count rate is reached when

€

λS =1/τ p

€

λomax =1

τpe+τd

€

τp =1

λomax−τde


25

PMT+Discriminator Saturation Correction

0

10

20

30

40

50

60

0 100 200 300 400 500 600

Outp

ut C

ount

Rat

e [M

Hz]

Input Count Rate [MHz]

λo =λ ie

−λ iτp

1+ λ iτde−λ i τp

τP = 3.2 nsτd = 10 ns

€

λS

Na lidar PMT + discriminator€

λo =λSe

−λSτ p

1+ λSτ de−λSτ p

€

λS = λiηQE


26

Step 3. Chopper Correction q  Chopper function is measured and then used to do chopper correction for lower atmosphere signals

020406080

100120140160

0 200 400 600 800 1000 1200 1400

Raw

Pho

ton

Cou

nts

Time (us)


27

Chopper Correction

Smoothed Chopper Function

Chopper Curve from Raw Data


28

[ ] ( ))()(),()(),()(),( 22 RRaR

RRRL

BRS zGzTzAzzn

hctPNzN ληλλπσ

λλ

λ %%&

'(()

*Δ%%

&

'(()

* Δ=−

[ ] ( ) BRRaR

RRRL

RS NzGzTzAzzn

hctPzN +!

!"

#$$%

&Δ!!

"

#$$%

& Δ= )()(),()(),()(),( 2

2 ληλλπσλ

λλ

Step 4. Subtract Background NB

€

NS (λ ,z) =PL (λ)Δthc λ

$

% &

'

( ) σ eff (λ ,z)nc(z)RB (λ) +σR (π ,λ)nR (z)[ ]Δz A

4πz2$

% &

'

( )

× Ta2(λ)Tc

2(λ ,z)( ) η(λ)G(z)( )+NB

ò


29

Background Estimate q  Background is estimated from high altitude signal

0

200

400

600

800

1000

1200

1400

1600

0 1000 2000 3000 4000 5000 6000

fa Channelf+ Channelf- Channel

Phot

on C

ount

s

Bin Number

Raw Data Profiles for 3-Frequency Na Doppler Lidar

BackgroundEstimate

There could be titled background due to PMT saturation


Step 5. Remove Range Dependence

€

NS (λ,zR ) − NB[ ]R2 =PL (λ)Δthc λ

%

& '

(

) * σR (π,λ)nR (zR )[ ]ΔR A( )Ta2(λ,zR ) η(λ)G(zR )( )

θ

€

h = Rcosθ

h R

30


Step 6. Add Base Altitude Altitude (relative to mean sea level)

= Observed Height + Base Altitude

€

z = h + zBase = Rcosθ + zBase

Observed HeightAltitude

Base AltitudeMean sea Level

31


32

Step 7. Rayleigh Normalization q  Estimate of Rayleigh Normalization Signal -

Rayleigh Fit or Sum


33

Rayleigh Normalization

2

2

),(),(),(

RBRS

BSionNormalizat z

zNzNNzN

zN−

−=

λλ

λ

Photon Counts at Rayleigh Normalization Altitude

(30-55 km)

Normalization Altitude (40 km)


34

Summary q  Lidar data inversion is to convert raw photon counts to meaningful physical parameters like temperature, wind, number density, and volume backscatter coefficient. It is a key step in the process of using lidar to study science. q  The basic procedure of data inversion originates from solutions of lidar equations, in combination with detailed considerations of hardware properties and limitations as well as detailed considerations of light propagation and interaction processes.q  Output of the preprocess and profile process is Normalized Photon Count, which is a preparation for the main process to derive temperature, wind, density, or backscatter coefficient, etc.

lidar r s p c cu-b lecture 30. lidar data inversion...

Documents