stennis space center ikonos v&v 1 nasa scientific data purchase ikonos v&v overview mary...

IKONOS V&V

1

Stennis Space Center

NASA Scientific Data Purchase

IKONOS V&V Overview

Mary Pagnutti

IKONOS V&V

2

Stennis Space CenterStennis Space Center

NASA Scientific Data Purchase (SDP)

• Began with $50 Million appropriation for NASA purchase of scientific data sets for commercial sources

• Goal: Test new way of doing business; tap new sources of data

• Managed by NASA Commercial Remote Sensing Program, Stennis Space Center, MS

• Phased Approach– Phase I - simulated / prototypical data sets

received and reviewed– Phase II - selection of products and data purchase

over 3 years

• Firm fixed price contracts - cash on delivery of data

IKONOS V&V

3


• Perspective: SDP is an experiment– Can the commercial sector provide useful, cost effective data to the

science community?

– Only one satellite (no backup), buy data while we can

• Assumptions – Space Imaging will attempt to provide high quality data

– NASA Stennis will trust but verify

– Releasing data in a timely fashion is important

– Feedback from users is important

• Competition is developing– Earth Watch, Orbital, Russian Spin Data, West Indian Aerospace

– Methodologies developed will be used again.

IKONOS SDP V&V Philosophy

IKONOS V&V

4


IKONOS V&V Activities

Performance Product V&V Characterization

(Sample V&V data sets)

Shipment Verification

(All acquired data sets)

• Data Format• Meta File• Location / Coverage• Cloud Cover• General Statistics

– mean– standard deviation– pixel histogram– fraction of saturated pixels

• Auto Report generation • Thumbnail generation• Band-to-band registration• MODTRAN / radiance intercomparison • NIIRS evaluation when appropriate

Product Characterization


• Image Quality

–Spatial–SNR

• Geometric Accuracy–Positional

Accuracy–DEM evaluation

• Radiometric Performance

–SpectralCharacterization

Applications Utility


• University of Maryland assessment• Obtain feedback from ESE Researchers

IKONOS V&V

5


Joint Agency Commercial Imagery Evaluation (JACIE) Team Formed

IKONOS one meter panchromatic image of Jefferson Memorial in Washington, D.C.

• Joint NASA / NIMA / USGS team focused on characterizing commercial remote sensing system performances

• Overall performance assessment to meet each agency’s mission requirements

– NASA - System Characterization– NIMA- Photogrammetry– USGS - Cartographic Assessment

• Improved communications with industry

– More productive interchanges between industry and government

– Minimize duplication of effort– Reduction in costs incurred by industry and

government

IKONOS V&V

6


SCIENCE USERS

JACIE TEAM

EOS Validation

Teams U of Maryland U of Arizona

S.Dakota State U

SDP IKONOS Characterization Team

IKONOS V&V

7


IKONOS Validation Sites

SDSU

SSC

BeltsvilleKonza Prairie

Sevilleta

WSMR

Maricopa

U of A

Morrison Quad.

Ivanpah

Railroad Valley

Lunar Lake

Lake Tahoe

Stillman Creek

USGS-EDC

Kaintuck Hollow Quad

Big Springs

IKONOS V&V

8


IKONOS Validation Sites Overview

IKONOS V&V

9


NASA Stennis Space Center, Mississippi

Site: Scattered buildings within heavily wooded area, man-made reservoirs and canal.Elevation 5.5m - 10m.30.388 degrees N, 89.61 degrees W

Targets: • 2 orthogonal concrete edge targets (70x80m oriented E-W, 80x80m oriented N-S) 130m concrete radial target (96.4 deg arc, 0.08-5.0m range), 4 step concrete grayscale target (35x40m each)• over 40 known GPS markers known to within 2 cm• 4 step gray tarps (20mx20m each)

In-Situ Instrumentation: Cimel Sunphotometer, Shadowband Radiometer, ASD and GER Spectro- radiometers, SSC Weather Station, Radiosondeballoon launch,Pyronometer, Goniometer.

Coincident Collects: Landsat 7

Purpose: Spatial, geometric and radiometric characterization, BRDF evaluation.

Concrete edge, radial and 4 step grayscale targets

IKONOS Image Area

ATLAS image and showing known GPS points at SSC

Radiometric Tarp Coupons

IKONOS V&V

10

Stennis Space CenterImage Quality Characterization

VERIFICATION &VALIDATION DATA SETSPECIFICATION VALIDATION

APPROACH SIOn-

Orbit

Funct.Collect

Std.Collect

TYPE

Panchromatic MTF = 0.1 atNyquist frequency

Review on-boardcalibrations provided by SI

Optimize and prepare SSCedge targets

Procure tarps Analyze images over

various targets to obtainMTF

Validate periodicallythroughout the data buy

X X Std OrigPrec Orig

SI data

Multispectral MTF = 0.24 atNyquist frequency Same as above X X

Std OrigPrec OrigSI data

Panchromatic GSD = 1.0 mMultispectral GSD = 4.0 m

Review on-boardmeasurements provided bySI

Analyze images overvarious targets to obtainGSD

Validate prior to datapurchase


SI data

IKONOS V&V

11


Big Springs, TX Edge Target

• Three 15 m x 15 m squares

• Northing and easting edges allow MTF analyses in two directions

• Edges are aligned approx 8 deg off true north and true east

IKONOS Panchromatic Band

IKONOS V&V

12

Stennis Space CenterImage Quality Characterization, cont’d.


APPROACH SIOn-

Orbit

Funct.Collect

Std.Collect

TYPE

SNR = 10 at Nyquistfrequency (Based on: 10%target reflectance 2:1 target-to-background ratio, 30 degsolar elevation)

Review on-boardcalibrations provided by SI

Optimize and prepare SSCedge targets

Analyze images overvarious targets to obtainSNR



SI data

SNR = 94 at zero spatialfrequency Same as above X X

Std OrigPrec OrigSI data

IKONOS V&V

13


Uniform Sites Selection for SNR

• Utilize NASA Scientific Data Purchase EarthSat Databuy

– TM data for Africa– 30 meter GSD data– Develop algorithms to find most

uniform scenes for Radiometry & SNR estimates

RGB 321

50 100 150 200 250

100

200

300

400

500

600

700

0.5 1 1.50

2

4

6

8

10x 10

4 TM1 std=3.3125%

0.5 1 1.50

2

4

6

8

10x 10

4 TM2 std=4.7671%

0.5 1 1.50

2

4

6

8

10x 10

4 TM3 std=5.4742%

0.5 1 1.50

2

4

6

8

10x 10

4 TM4 std=6.2394%

Libya TM Scene

Tight Gaussian Histogram Indicate Uniform Scenes

African Map with Sites

IKONOS V&V

14

Stennis Space CenterGeometric Accuracy Characterization


APPROACH Lab Funct.Collect

Std.Collect

TYPE

Absolute Horizontal GeometricAccuracy* ± 250 m

Analyze images overground control points toobtain accuracy


XStd Orig

Absolute Horizontal GeometricAccuracy* ± 12.2 m Same as above X Std Master

Absolute Horizontal GeometricAccuracy* ± 3 m Same as above X Pan Prec Orig

Absolute Horizontal GeometricAccuracy* ± 2 m Same as above X Pan Prec Master

Absolute Horizontal GeometricAccuracy* ± 5 m Same as above X

MS Prec OrigMS Prec Master

* 90% circular error

IKONOS V&V

15


Geometric Accuracy Assessment

• USGS EROS Data Center– Brookings, South Dakota (SDSU)– Sioux Falls, South Dakota (EDC)– Each site surveyed 50+ points throughout 11x11 km scene

• USGS– Morrison Quad– Kaintuck Hollow Quad– Utilize USGS database for both sites

• NASA CRSP– Stennis Space Center– Utilize site survey & Geodetic Targets

Brookings South Dakota

Fast Static SurveyChecked to less than five cm. horizontal using two Trimble 4000ssi units over B Order monuments.

IKONOS V&V

16

Stennis Space CenterRadiometric Performance Characterization


APPROACH SIOn-

Orbit

Funct.Collect

Std.Collect

TYPE

Absolute radiometric accuracyto within ± 10% over the entireimaging exposure dynamicrange (temporal stability)

Review on-board calibrationsprovided by SI

Analyze images overradiometric targets to obtainaccuracy


X X All

SI data

Relative radiometric accuracyto within ± 5% (pixel to pixel)

Same as above X X All

SI data

IKONOS V&V

17

Stennis Space CenterRadiometric Performance Characterization, cont’d.


APPROACH SIOn-

Orbit&

Lab

Funct.Collect

Std.Collect

TYPE

97% of all detectors should be within5% of the mean quantum efficiency anddark current

Review on-board calibrationsprovided by SI X SI data

Less than 1% inoperable detectorsprior to launch and less than 3% after 5years MMD. No more than 3 adjacentdetectors inoperable.

Review laboratorymeasurements provided by SI

Perform general statisticalanalysis of data

X SI data

IKONOS V&V

18

Stennis Space CenterRadiometric Performance Characterization, contd.



Std.Collect

TYPE

Linearity to within ± 5% over the entireimaging exposure dynamic range

Review laboratorymeasurements provided by SI

Analyze images overradiometric targets to obtainaccuracy


X X All

Lab data

Dynamic range of either 8 or 11 bits Validate periodicallythroughout the data buy X X All

IKONOS V&V

19


Radiometric Vicarious Calibration Approaches

• Reflectance Method– Measurement of ground reflectance and atmospheric properties

allows prediction of Top-of-the-Atmosphere (TOA) radiance estimates

– Typically require ground teams and a great deal of coordination

• Radiance Method– Coincident collect with well characterized airborne or satellite

system

IKONOS V&V

20


Reflectance Method

Method: Utilize ground reflectance measurements (Spectroradiometer) and atmospheric measurements (Sun Photometer & Radiosonde) to determine radiometric accuracy of new commercial satellite systems (IKONOS)

Sun Photometer

Spectroradiometer

Commercial Satellite

Radiosonde Balloon

IKONOS V&V

21


Radiance Method

Method: Utilize coincident airborne or spaceborne calibrated imagery (e.g. Landsat 7) to determine radiometric accuracy of new commercial satellite systems (IKONOS)


Well Characterized System (Aircraft or Satellite)

IKONOS V&V

22


IKONOS Characterization Status

• Spatial Characterization – Three acquisitions have been made (received std original imagery only)– Four SI archive images have been acquired– Two acquisitions are pending collection– Awaiting two precision original images– All but one image has been evaluated

• Geometric Characterization– Three acquisitions have been made (received std original imagery only)– Two acquisitions are pending collection (same as above)– Two DEM acquisitions scheduled for spring 2001– Two have been evaluated

• Radiometric Characterization– Six acquisitions have been made and evaluated (received std original imagery

only)– Two acquisitions are pending collection (same as above)

IKONOS V&V

23


IKONOS Characterization Summary

Preliminary Findings:• Geometric Characterizations show imagery is

well within specification• Spatial Characterizations indicate imagery meets

MTF and SNR specifications• Radiometric Characterizations indicate potential

concerns with the red and NIR bands• No serious issues associated with on-board

compression have been found

IKONOS V&V

24


Backup

IKONOS V&V

25

Stennis Space CenterIKONOS Sensor Facts from SI Web Site

IKONOS V&V

26


Spectral Bands

Swath Widths & Scene Sizes

Nominal swath width:13 km at nadir

Areas of interest:A nominal single image at 13 km x 13 km Strips of 11 km x 100 km up to 11 km x 1000 km Image mosaics of up to 12,000 square km. Up to two 10,000 square kilometer contiguous

areas in a single pass within a region

Metric Accuracy

12 meter horizontol and 10 meter vertical accuracywith no ground control

2 meter horizontal and 3 meter vertical accuracywith ground control

These are specified as 90% CE (circular error) forthe horizontal and 90% LE (linear error) for thevertical

IKONOS Sensor Facts from SI Web Site, cont’d.

IKONOS V&V

27


Orbital Information

Altitude 423 miles / 681 kilometers

Inclination 98.1 degrees

Speed 4 miles per second / 7 kilometers per second

Descending Nodal Crossing Time 10:30 am

Revisit Frequency2.9 days at 1 meter resolution;1.5 days at 1.5 meter resolution

These values are for targets at 40 degreeslatitude. The revisit times will be more frequentfor higher latitudes and less frequent forlatitudes closer to the equator.

Orbit Time 98 minutes

Orbit Type Sun – synchronous

Viewing Angle Agile spacecraft – in-track and cross-trackpointing

Weight 1600 pounds


IKONOS V&V

28


CameraSystem

Designed and built by Eastman Kodak Total system weight: 376 lbs. (171 kg.) Total system power consumption: 350 watts

OpticalTelescopeAssembly

Assembly size: 61 in. long x 31 in. wide Assembly weight w/out focal plane unit: 240 lbs. Optical design: Three mirror anastigmat Focal length/ focal ratio: 10m / f14.3 Image resolution (at nadir): 1m panchromatic, 4m multispectral Primary mirror: 27.5in. diameter x 4in. thick, 29.5 lbs.

ImagingSensors &Electronics

Focal plane unit Unit size: 10in. x 9in. x 9in. Panchromatic sensor: 12 micron pixel pitch, 13,500 pixels Multispectral sensor: 48 micron pixel pitch, 3375 pixels

DigitalProcessingUnit

Unit size 18in x 7.5in. x 12.4in. Compression rate:11 bits per pixel compressed to2.6 bpp Compression speed: 4 million pixels per second per

processing channel.

PowerSupply Unit Unit size: 7in. x 8in. x 16in.

Launch Date September 24, 1999 (11:21:08 am PDT)

LaunchVehicle

Athena II

LaunchVehicleManufacterer

Lockheed Martin


IKONOS V&V

29

Stennis Space CenterStennis Target Set

Positive Systems ImageExpanded View of Target

EdgeTarget: 70x80 meters and 70x70 meters 8:1 contrast ratio

IKONOS V&V

30


Edge Response

x

:response edge the get

to x vs. value pixel plot

tanGSDx

edge and line scan between angle :

Foot Print

0 2 4 6 8 10 12 1460

80

100

120

140

160

180

200

220

240

260Edge Response ADAR 5500 -- Blue band 1m

Meters

Edg

e R

espo

nse

¬ Undersampled

Focal plane array undersamples edge with classical cross section samplingSolution: Sample tilted edge to improve sampling

IKONOS V&V

31


Edge Response Relationship to Line Spread Function

dx

d)x(E

)x(l

x x

)x(lFT)0,(MTF

)y,x(PSFFT),(MTF

IKONOS V&V

32

Stennis Space CenterModulation Transfer Function

• MTF is a measure of an imaging system’s ability to recreate the spatial frequency content of scene

MTF is the magnitude of the Fourier Transform ofthe Point Spread Function / LineSpread Function.

1.0

Cut-off

Spatial frequency

MTF

IKONOS V&V

33


SNR Estimate Example Results

0 50 1000

2000

4000

6000

8000

10000TM1

Pseudo SNR0 50 100

0

2000

4000

6000

8000

10000TM2

Pseudo SNR

0 50 1000

2000

4000

6000

8000

10000TM3

Pseudo SNR0 50 100

0

2000

4000

6000

8000

10000TM4

Pseudo SNR

Histograms of Mean/Std of 5x5 Pixel Blocks

TM data

Location of Histogram Peak is a measure of the SNR

IKONOS V&V

34

Stennis Space CenterGPS Accuracies

• Trimble Pathfinder or Geo Explorer – Raw - 10 meters

– Coarse mode corrected - 5 meters per measurement reading

– Coarse mode corrected - averaging data 2 meter

– Fine mode corrected - averaging 10 minutes of differentially corrected data gives sub-meter

Geo Explorer

Pathfinder

IKONOS V&V

35


Radiometric Error Associated with Finite Size Tarp

Insert Measured Reflectances Figures

5 10 15 20 25

5

10

15

20

25

-6 -4 -2 0 2 4 60

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

GSD

Tar

p R

espo

nse

Tarp/GSD=5 MTF@Nyquist 0.1

For typical background/tarp reflectances, the error at the center pixels will be less than a few percent

Cross Sectional Plot of Response

Finite Size of Tarp will limit Radiometric Accuracy

Simulated 20 m Tarp Image with 4 m GSD with MTF @ Nyquist of 0.1

IKONOS V&V

36


CRSP Tarp Sample Data

50% Tarp 30% Tarp 20% Tarp <5% Tarp

Vendorsupplied mean:420-1050nm

52.78% 38.27% 23.60% 2.46%

Measuredmean:420-1050nm

45.56% 32.69% 20.90% 2.37%

Tarp Sample DataReflectance Data

300 400 500 600 700 800 900 1000 11000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Wavelength (nm)

Ref

lect

ance

(%

)

Reflectivity of tarp samples

50% tarp30% tarp20% tarp<5% tarp

Tarp Samples are Spectrally Flat Over Region of Interest

IKONOS V&V

37

Stennis Space CenterGPS Accuracies

• Trimble 4000ssi Survey grade GPS units– Static survey- 5mm + 1ppm in base line distance.

– Fast Static - Based on conditions and Satellite geometry

– Real Time Kinematic (RTK) - less than five centimeters horizontal and vertical

Brookings South Dakota

Fast Static SurveyChecked to less than five cm. horizontal using two Trimble 4000ssi units over B Order monuments.

IKONOS V&V

38


Radiometric Vicarious Calibration Background

• Radiometric stability of remote sensing data is critical for long term studies

• On-orbit calibration sources are not always stable or reliable

• Methods which independently verify the radiometric calibration of remote sensing data are essential

• Vicarious calibrations use natural well characterized scenes

IKONOS V&V

39


CRSP Radiometric Tarp Sample Examination

Samples were obtained of the 4 grayscale radiometric tarps procured by CRSP. These samples were tested in the laboratory under conditions similar to those encountered in the field. These tests were performed to evaluate specified reflectivity values.

4” tarp Samples50%

<5% 20%

30%

IKONOS V&V

40


Desired Calibration Site Characteristics

• Bright and Uniform– High reflectance minimizes atmospheric uncertainties

– Uniformity allows statistically meaningful results

• Stable– Long term studies are facilitated with stability

• Low Precipitation and Cloud Cover– Maintains stability and increases chance of acquisition

• Large– Several swath widths in extent or more to avoid adjacency effects

IKONOS V&V

41


Tarp Deployment at SSC

IKONOS V&V

42


SDP Shipment Verification

Shipment Ver ification ReportContract No: Tracking ID No: Ve ndor: Task No: Mission No:

Transmittal Date: V&V Date Received Product Descriptor: Location:

Ve rify Shipment Contents. Product/Task # Images Volume (GB)

Observed Shipment Content and Count:

Are any of the materials listed in the Transmittal Letter missing?

Are any of the deliverables missing? (Calibration Report? Mission Report? Index Maps? Ground Truth Data?)

Ve rify Shipment Quality.

Is there any indication in the mission documents, that data with the wrong specifications was delivered?

Does geographic coverage match Delivery Order? Does acquisition date match Delivery Order?

List File Readability Problems or NA: Ingest Date: % Sampled:

List Image Quality Problems or NA: Method: % Sampled:

Discrepancy Report No.: Disposition of NC Products:

I have verified this shipment according to CRSP-WI-22 procedures.Primary Reviewer: Date:

Secondary Reviewer: Date:

CRSP Form 021 (2/2000)

• Shipment content and media count

• Mission document review

• Approximate geographic coverage

• Comparison with original task order

• File readability

• General image quality

• 100% visual inspection of JPEG quicklooks

• Discrepancy report (As Needed)

• Shipment tracking database

• Shipment verification report

IKONOS V&V

43


NASA IKONOS V&V FocusIKONOS Data Product Characterization

• MTF and spatial resolution

• SNR estimates

• Geometric accuracies of imagery

• Radiometric accuracies of products

• Band-to-band registration of multispectral products

• Intercomparison of IKONOS, Landsat 7 and other satellite imagery

Quantitative System Characterizations:

1m GSD PAN 4m GSD MS

IKONOS V&V

44


IKONOS Spectral Characterization



Std.Collect

TYPE

Band edge points at 50% peakresponse shall be within ± 0.01microns

Review laboratorymeasurements providedby SI

X Lab data

Slope through the 50% pointshall be at least 20% / 0.02microns (NIR long edge slope <20% / .04 microns)


X Lab data

Out of band filter response <5.5% of total integratedtransmittance within 5%transmission points of that band


X Lab data

The response for 70% of thedata centered on the peakresponse shall be within 20% ofthe maximum value. (NIRresponse for 35% of the datacentered on the peak responseshall be within 20% of themaximum value.


X Lab data

IKONOS V&V

45


IKONOS Spectral Bandpasses

IKONOS2 Relative Spectral Response

0

0.2

0.4

0.6

0.8

1

350 550 750 950

Wavelength (nm)

Rel

ativ

e S

pec

tral

R

esp

on

sivi

ty

Pan

Blue

Green

Red

NIR

Provided by Space Imaging

IKONOS V&V

46


Well Characterized Site Method

Method: Utilize well characterized satellite systems (Landsat-7), not necessarily coincident, and well characterized sites (African deserts) to determine the radiometric accuracy of new commercial satellite systems (IKONOS)


Calibrated Satellite

IKONOS V&V

47


NASA CRSP Tarps

• Portable radiometric characterization of 4 m class GSD systems and smaller

• MTF characterization of 1 m class GSD systems and smaller

• SNR estimation on 1 m class GSD systems• Fiducial markers for remote uniform sites

IKONOS V&V

48


CRSP Tarp Specifications

• Size and Shape– 20 m x 20 m

• Reflectivity– Spectral Range 400-1000 nm

– Tarp 1 < 5 %

– Tarp 2 20-25 %

– Tarp 3 30-40 %

– Tarp 4 50-55 %

• Spatial Uniformity – Standard deviation of 9 equally

space reflectivity measurements <1% of absolute reflectance

• Spectral Uniformity– <10% variation/100 nm band

• Deployability– Less than 2 hours with 2 people

in less than 3m/s wind

• Maintainability– Meet Specifications for 60

Deployments or two years which ever comes first

stennis space center ikonos v&v 1 nasa scientific data purchase ikonos v&v overview mary...

Documents