![Page 1: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/1.jpg)
High Altitude JavaThe software behind an ESA-sponsored airborne imaging spectrometer
Jason Brazile, Andreas Karrer, and Peter KohlerNetcetera Zurich7640
![Page 2: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/2.jpg)
2
Airborne Prism EXperiment
> The Mission– Airborne earth-observing “camera”
(pushbroom imaging spectrometer)~500 bands instead of 3 (red, green, blue)
> Selected Software– Data Acquisition/Control/Monitoring
Flight plan triggered camera controlImagery/Positional/Calibration/Environmental dataNear real-time “waterfall image” monitoring
– Offline processingInstrument calibration support (Seasonal)Radiometric/spatial correction (Per campaign)Parametric ortho-rectification (Per campaign)Atmospheric correction (Per campaign)
APEX installed in DLR Dornier Do228
“Quick Look” from first APEX imaging flight
![Page 3: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/3.jpg)
3
![Page 4: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/4.jpg)
4
Scientific AdvisorKlaus Itten
Principal InvestigatorMichael Schaepman
Who: the APEX team
ESA Team
Instrument Manager &Detector Development
Gerd Ulbrich &Roland Meynart
PRODEXTechnical Officer
Hilde Schroeven-Deuceuniinck
Contracts OfficerUlrich Sterzl
Science Team
Project ManagerEdoardo Alberti
Calibration ScientistFrancesco Dell'Endice
Processing ScientistDaniel Schläpfer
PAF ScientistAndreas Hüni
Operations Team
Co-InvestigatorKoen Meuleman
Flight OperationsBart Bomans
Processing OperationsJan Biesemans
OperatorJohan Mijnendonckx
Industrial Team
Industrial PrimeIntegration and systems
RUAG AG
Electronic SubsystemNetcetera AG
SWIR DetectorsSofradir
Calibration Home BaseDLR
Optical SubsystemOIP Sensor Systems
Scientific AdvisorJens Nieke
![Page 5: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/5.jpg)
5
What: imaging spectroscopy
Red
Green
Blue
Digital Camera: 3 bands (red,green,blue) Imaging spectrometer: 100s bands
simultaneous acquisition of spatially coregistered images, in many narrow, spectrally contiguous bands, measured in calibrated radiance units
![Page 6: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/6.jpg)
6
What: visualizing “non-visible” wavelengths…
ultraviolet visible (blue, green, red)
Photos: Austin Richards, Alien Vision, ISBN: 0819441422
infrared
![Page 7: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/7.jpg)
7
What: “data cube” w/100’s of bands
![Page 8: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/8.jpg)
8Why: imaging spectroscopy applications
100.0%
0.0%
Precision Farming
Land cover studies
Forest Fire Modeling
Water Quality
Air Pollution
Natural Hazards
Security relevant analysis
From the spectral cube
![Page 9: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/9.jpg)
9
Why: 100s of bands? Isn’t ~10 enough?
7 bands (Landsat) not detailed enough!
1) basalt; 2) hematite coating and montmorillonite;3) hematite coating and alunite; 4) kaolinite; 5) dolomite; 6) vegetation. (Courtesy of NASA/JPL)
Goetz, A. F. H., Three decades of hyperspectral remote sensing of the Earth,Remote Sensing of Environment (2009), doi:10.1016/j.rse.2007.12.014
![Page 10: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/10.jpg)
10How: airborne data acquisition
![Page 11: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/11.jpg)
11How: the optical sub-unit
VNIR Detector(380 - 1000 nm)
SWIR Detector(940 - 2500 nm)
SWIR Cryo-Cooler
Beam Splitter Ground imager,
slit and sphericalmirror
Main Collimator
SWIR Optics
VNIROptics
folding mirror
72kg w/o COG counter-weight
![Page 12: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/12.jpg)
12How: the in-flight characterization facility
StabilisedIFC QTH lamp
Spectrometer optical path
Filter wheel
IFC light path
(glass fiber)
Shutter for standardimaging operations
EntranceBaffle
Available Filters on the filter wheel:• 3 Bandpass (color) Filters• 1 Rare Earth Material (NIST) filter• 1 Neutral Density (gray) filter
The IFC is a tool designed to investigate the overall instrument (radiometric, spectral, geometric) stability during the flight
![Page 13: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/13.jpg)
13How: instrument deployment (DO-228)
Thermal Control Unit (TCU)
Optical SubsystemUnit (OSU)
Stabilising Platform(Leica Pav30)
Environmental Thermal Control (ETC) Box
![Page 14: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/14.jpg)
14
Operator rack workstationfor flight operations
How: electronics deployment (DO-228)
Navigation sub-system (Applanix POS/AV)
Main control computer (CSU)
Storage Unit (Solid State Disks and tape)
On laptop:- Operator’s interface- Flight Management System (FMS) (Track’Air)
Power Distribution
![Page 15: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/15.jpg)
15
Aside: Originally, array of sealed hard drives
Consumer Hard-drives don’t work at high altitude
We’ve since moved on to SSD tech…
![Page 16: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/16.jpg)
16How: remotely + ground-sensed spectroscopy
Remote Sensing Tutorial, Nicholas M. Short, Sr.
![Page 17: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/17.jpg)
17How remote?
April 2008: DLR Do228 Air Worthiness Certification
October 2008: 1st test flight (DLR Do228)
End 2009: DLR HALO Air Worthiness Certification
[Other platforms under evaluation]
![Page 18: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/18.jpg)
18
![Page 19: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/19.jpg)
19
Aside: Real-Time Java (JSR-001, JSR-282)
> Java RTS is Sun’s implementation of JSR-001 (Final Release 3 – Jul 2006)– Real-Time Threads, Scheduling, and Synchronization – Avoiding Garbage Collection via Immortal and/or Scoped memory– Asynchronous handling of jvm-external events– ATC (one thread can “throw exception” in another)– Time and Timers– Direct access to physical memory
> Also: JSR-282 “to fill in some minor gaps” (Early Draft Review – May 2009)
"the ability to reliably and predictably respond to a real-world event"
![Page 20: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/20.jpg)
20
Aside: Real-Time Java (not used in APEX)
> Standard Java
while (true) do {acquire_data();now = System.currentTimeMillis()Thread.sleep(next_period – now);send_data();next_period += period;
}
> Real-Time Java
setPriority(my_RTPriority);setReleaseParameters(myPeriodParam);while (true) do {
acquire_data();RealtimeThread.waitForNextPeriod();send_data();
}
e.g. Real-Time Java’s RealtimeThread + RawMemoryAccess could be used to implement APEX’s current producer/consumer acquisition logic
Code adapted from Bertrand Delsart, Real-Time Java for Latency Critical Banking Applications
![Page 21: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/21.jpg)
21
Selected APEX Software> Data Acquisition/Control/Monitoring (some examples)
– Flight plan triggered camera control– Multi-sensor data (w/temporal synchronization)
ImageryPositionalCalibrationEnvironmental
– Near real-time “waterfall image” monitoring> Offline Processing (some examples)
– Instrument calibration support (Seasonal)– Radiometric/spatial correction (Per campaign)– Parametric ortho-rectification (Per campaign)– Atmospheric correction (Per campaign)
![Page 22: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/22.jpg)
22Acq: Flight-plan triggered camera control
14 km long Flight Line plus turnings
![Page 23: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/23.jpg)
23
Aside: options for 3-D Data Arrangement
Band interleaved by Line (BIL) - compromise
Band interleaved by Pixel (BIP) - spectral Band Sequential (BSQ) - spatialFigures, Phil Hurvitz, U. Washington
![Page 24: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/24.jpg)
24Acq: waterfall image viewer
![Page 25: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/25.jpg)
25
Acq: waterfall image viewer> What?
– A “false color” line-by-line preview of incoming data…
> Why?– Verify proper “operation
mode” – calibration vs acquisition
– Verify “frame-rate/integration time settings”…
– Verify “band settings/groupings” for this application…
– Verify pilot’s flight path is as expected…
“did we remove the lens cap?”“do we have the right ‘shutter’ speed?”“is the subject within the frame?”
![Page 26: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/26.jpg)
26
Aside: queuing theory issues and near-real time
DatagramPacket dgram = new DatagramPacket(new byte[msgsize], msgsize);while (isrunning && udpsock != null) {try {
udpsock.receive(dgram);if (dgram.getLength() == msgsize) {dataProcessor.processData(dgram.getData());
} else {LOGGER.error("Bad packet, len = " + (dgram.getData()));
}} catch (java.net.SocketTimeoutException e) {
LOGGER.info("No lines received for “ + timeoutms + " ms");dataProcessor.repaint();// expect to sometime receive nothing…
} catch (java.io.IOException e) {LOGGER.error("udp receive error!");
}}
UDP – expect dropped packets and data underflow or overflow.
Server logic tries to keep waterfall buffer 20-80% full for handling bursts and variable frame rates.
![Page 27: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/27.jpg)
27Offline: Seasonal Calibration (CHB@DLR Munich)*
* Under ESA-EOP ContractStatus: Acceptance review successful Jan 2007
6 ton granite optical bench
APEX
MonochromatorCollimator
Slit and folding mirrorFor Relative radiometric
calibration
1.6 m Integrating Sphere
0.5 m Integrating Sphere
Seismic Platform(max displacement 0.3 um)
For Absolute radiometric calibration
![Page 28: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/28.jpg)
28
Offline: Image feature-based calibration
Well-known Fraunhofer line “A” (due to atmospheric oxygen) used to calibrate potential instrument band center shifts
![Page 29: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/29.jpg)
29
Offline: Radiometric calibration
Illustration, CRIM - Centre de recherche informatique de Montréal
Converting recorded digital numbers to physical parameters in calibrated radiance units
![Page 30: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/30.jpg)
30
Offline: Orthorectification
attitude
GPS
![Page 31: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/31.jpg)
31
Offline: ATCOR - Atmospheric Correction
Image: Satellite Imaging Corporation, satimagingcorp.com
Using radiative transfer algorithms based on atmospheric optical properties to derive surface reflectance by inversion
![Page 32: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/32.jpg)
32
Offline: BRDF Correction
BRDF effects can be seen as stripes in freshly mowed grass
![Page 33: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/33.jpg)
33
Offline: Product Generation
InstrumentData
APEXLevel 0 - 1CProcessor
Calibrated, uniformImageCubes
Level 2A-2C Products
Level 3A-3B Products
Geographically,Atmospherically
and BRDFcorrected data
Level 1C Product
APEXLevel 2A – 2C
Processor
Quantitative andqualitative
Thematic Mapsand Variables
APEXLevel 3A – 3BProcessors
• Vegetation
• Limnology
• Snow
• Gas & Aerosols
HW hosted @ VITO (Mol, B)
![Page 34: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/34.jpg)
34Offline: Do-it-yourself Module/Algorithm dev?
PAF Research ProductsOn Level 2/Level 3
Algorithm Developer(e.g., scientists, students)responsible for module
and GUI
User community
APEX Science Center (RSL - ZH) APEX Service Center (VITO – B)
interface
experts module..
GUIParameter,Settings, I/O
help & advice
module XZ
module XY
module XX
feedback
GUIproduct & ATPD*
x
GUIproduct & ATPD*
1
…
module ..
Delivery ofmodule ....
Integrationinto the
work flow
Can integrate customized processing module at any point of the processing chain.
![Page 35: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/35.jpg)
35
Test Flight over Wittenberge (D), 31/10/2008
1st Radiometrically Calibrated APEX Cube
![Page 36: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/36.jpg)
Jason Brazile http://netcetera.ch/Netcetera Zurich [email protected]
Andreas Karrer http://netcetera.ch/Netcetera Zurich [email protected]
Peter Kohler http://netcetera.ch/Netcetera Zurich [email protected]
![Page 37: High Altitude Java - UZHjbrazile/jazoon09-7640-apex-jb-final.pdf · High Altitude Java The software behind an ESA-sponsored airborne imaging spectrometer Jason Brazile, Andreas Karrer,](https://reader034.vdocuments.us/reader034/viewer/2022042318/5f076b2b7e708231d41ce357/html5/thumbnails/37.jpg)
37
Mission Status/Timeline
Institutes Phase C/D
Industry Phase C/D
Delivery
Chracterization & Testing activities
Integration
AcceptanceDo 228
AWRR1
Calibration& Test flight
PAF End to End Test
PAF Data Format Testing
VersionTests
System Tests
First Testof OSU on
CHB
Today
Test Flight
2008 2009
CTM TestingCTM TRR Phase E
(5+5 years)
2010
RR2 RR3