smos sag, november 2-3, 2006 smos data processing ground segment (dpgs)
Post on 19-Dec-2015
217 views
TRANSCRIPT
SMOS SAG, November 2-3, 2006
SMOS
Data Processing Ground Segment (DPGS)
SMOS SAG, November 2-3, 2006
Function ESAC (Villafranca)
ESRANGE (Kiruna)
Payload planning, TC generation, and monitoring
SPGF,PLPC-PXMF -
X-band acquisition and Level 0 generation
XBAS -Processing infrastructure: inventory, storage, data distribution, monitoring and control
PDPC-Core MMFI
Processing and monitoring L1OP,L2OP,SPQC,
MF
Reprocessing Centre
Analysis Tools IATs -Archiving - MMFI
Product Ordering via User Services - MMFI
CEC
New
SMOS Ground Segment
SMOS SAG, November 2-3, 2006
SMOS Ground Segment
S-band X-bandAuxiliary Data
Providers
SOGS
PLPC
SPGF
XBAS
PXMF
Fast Processing Centre
Long Term Archive
CEC
IAT
MF
SPQC
Refer.Processor
Level 0
PDPC Core
Level 1
Level 2
Reprocessing Centre
Level 1
Level 2 MF
SPQC
Local Archive
Data Dissem.
Multi-mission Facility
Infrastructure
DPGS
Multi-mission User Services
USERS
ReportsData
Subscriptions
Data On-request
SMOS SAG, November 2-3, 2006
SMOS Ground Segment
S-band X-bandAuxiliary Data
Providers
SOGS
PLPC
SPGF
XBAS
PXMF
Fast Processing Centre
Long Term Archive
CEC
IAT
MF
SPQC
Refer.Processor
Level 0
PDPC Core
Level 1
Level 2
Reprocessing Centre
Level 1
Level 2 MF
SPQC
Local Archive
Data Dissem.
Multi-mission Facility
Infrastructure
DPGS
Multi-mission User Services
USERS
ReportsData
Subscriptions
Data On-request
Indra Espacio responsibility
SMOS SAG, November 2-3, 2006
SMOS Ground Segment
S-band X-bandAuxiliary Data
Providers
SOGS
PLPC
SPGF
XBAS
PXMF
Fast Processing Centre
Long Term Archive
CEC
IAT
MF
SPQC
Refer.Processor
Level 0
PDPC Core
Level 1
Level 2
Reprocessing Centre
Level 1
Level 2 MF
SPQC
Local Archive
Data Dissem.
Multi-mission Facility
Infrastructure
DPGS
Multi-mission User Services
USERS
ReportsData
Subscriptions
Data On-request
Datamat responsibility
SMOS SAG, November 2-3, 2006
SMOS Ground Segment
S-band X-bandAuxiliary Data
Providers
SOGS
PLPC
SPGF
XBAS
PXMF
Fast Processing Centre
Long Term Archive
CEC
IAT
MF
SPQC
Refer.Processor
Level 0
PDPC Core
Level 1
Level 2
Reprocessing Centre
Level 1
Level 2 MF
SPQC
Local Archive
Data Dissem.
Multi-mission Facility
Infrastructure
DPGS
Multi-mission User Services
USERS
ReportsData
Subscriptions
Data On-request
CNES responsibility
SMOS SAG, November 2-3, 2006
SMOS Ground Segment
S-band X-bandAuxiliary Data
Providers
SOGS
PLPC
SPGF
XBAS
PXMF
Fast Processing Centre
Long Term Archive
CEC
IAT
MF
SPQC
Refer.Processor
Level 0
PDPC Core
Level 1
Level 2
Reprocessing Centre
Level 1
Level 2 MF
SPQC
Local Archive
Data Dissem.
Multi-mission Facility
Infrastructure
DPGS
Multi-mission User Services
USERS
ReportsData
Subscriptions
Data On-request
ESA responsibility
Also:
Site
External network
GSOV
Operations
SMOS SAG, November 2-3, 2006
SMOS Data Flow
SPGF PLPC SOGS
XBASPDPC Core
L1OP
L2OP
SPQC
Instrument Planner CNES
Operations
Post-launch Support
Aux Data Handling
Aux Data Providers
Instrument plan
On-board SW updates
Telecommands
S Band uplink
X Band downlink
PXMF
Level 0 products
Auxiliary Data
SMOS Auxiliary
Data Files
Products and Aux Data
Quality Stamp
ESAC Operations
Expert Team
Reports
SMOS SAG, November 2-3, 2006
SMOS Data Flow
PDPC Core
L1OP
L2OP
SMOS Auxiliary
Data Files
ESAC Operations
Time Critical Users
Other Users
LTA
MF
IATs
Aux Data Generation
Expert Team
ESRIN Help Desk
Products, Reports and Aux Data via media
Products, Reports and Aux Data via
FTP
Reports
Products, Reports and
Aux Data
Distribution configuration
Products and Aux Data
Reports
Maintenance Team
Problems identified
and corrections recommend
ed
Updated algorithms
and/or formats
Evolution Team
Improvements identified and
evolutions recommended
SMOS SAG, November 2-3, 2006
Reprocessing Campaign Data Flow
MMFI
SMOS Auxiliary
Data Files
ESRANGE Operations
Time Critical Users
Other Users
LTA
MFExpert Team
Products and Aux Data
Reprocessed Products,
Reports and Aux Data via
FTP
Distribution configuration
Reprocessed Products,
Reports and Aux Data
Reports
Maintenance Team
Problems identified
and corrections recommend
ed
Updated algorithms
and/or formats
Evolution Team
Improvements identified and
evolutions recommended
SPQC
Quality Stamp
Level 1 Processing
Level 2 Processing
Reprocessing Centre
Reprocessed Products
Aux Data Handling
Aux Data Providers
Auxiliary Data
Reprocessed Products,
Reports and Aux Data via
media
User Services
Users
Product Orders
ESRIN Help Desk
Catalogue
Updates
Distribution Requests
Mission Manager
Reprocessing Campaign Authorisatio
n
SMOS SAG, November 2-3, 2006
DPGS
KO
LTA
CDR DPGS V1 DPGS V2
KO CDR OSAT
GSOV
Aug. 05 Oct. 06 June 07 Dec. 07
June 06 Feb. 07 Sept. 07
Dec. 06
Accepted SPGF and
PLPC
Accepted MF and SPQC
March 07 May 07
Accepted L1 (V1) and L2
(V1) Processors
Complete DPGS except: • L1OP = DPM of mid-2006• L2OP = prototype of late-2006• MF and SPQC = specific functionality of spring 2007
• L1OP = DPM of spring 2007• L2OP = prototype of mid-2007• MF and SPQC = specific functionality of late 2007
SMOS High Level Schedule
SMOS SAG, November 2-3, 2006
ESAC
Calibration Tower
You are here
SMOS SAG, November 2-3, 2006
ESAC
SMOS SAG, November 2-3, 2006
ESAC
North
SMOS SAG, November 2-3, 2006
ESAC
Calibration Tower
Future SMOS Antenna
SMOS SAG, November 2-3, 2006
SMOS Mask 2006-10-17
0.000
2.000
4.000
6.000
8.000
10.000
12.000
14.000
16.000
18.000
0.000 50.000 100.000 150.000 200.000 250.000 300.000 350.000
AZIMUTH (deg)
ELEV
ATIO
N (d
eg)
Assumes removal of the TS-1 antenna.
Castle
Trees
Calibration Tower (93.8 degrees)
Vil-1 antenn
a
Vil-2 antenn
a
Trees
Street light
SMOS SAG, November 2-3, 2006
ESAC Site
• OPS-1 ready; subsystem installation underway; furniture for early 2007
• Network installation underway; external access at end of 2006
PLPC-PXMF/SPGF
• Accepted at ESAC in October; ready for GSOV
XBAS
• Front end processing system from SMP very late; acceptance expected in November
• Antenna ready at Indra; Demeter tracking before year’s end; acceptance at ESAC expected in February
• FEP-AIT to be delivered to CASA in November for first SMOS space-to-ground compatibility testing
• Analysis shows that for full polarisation descending (evening) passes at ESAC are not sufficient for X-band download; some ascending (morning) passes will need to be used
Status
SMOS SAG, November 2-3, 2006
L1OP (Operational Processor)
CDR
L1PP (Prototype Processor)
V1 V2
Sept. 06 Jan. 07 Sept. 07
Oct. 06
DPM V1
V1 cross-verification
Level 1 Processor Status
V2 cross-verification
Dec. 06
TDS V1
TDD V1
DPM V2
TDS V2
TDD V2
April 07 Aug. 07June 07
Delta-CDR
April 07
• 8 dual processor dual core CPU machines required to meet 2.5 NRT requirement for dual polarisation• G/J matrix software from L1PP re-used• aux data handling in the CEC• consolidation of file formats
• regular updates available• on-going work on calibration issues, NIR processing,…
SMOS SAG, November 2-3, 2006
SM ESLs
KO DPGS V2 delivery
Dec. 06
Level 2 Processor Status
Algo validation: phase 1
• 8 dual processor dual core CPU machines required to meet 2.5 NRT requirement for dual polarisation• aux data handling in the CEC• consolidation of file formats versus the prototypes
• regular updates of prototype available• operational processor is largely re-use of the prototype• operational service from ECMWF for the end of 2006
OS ESLs
SM L2PP
OS L2PP
CDRAlgo validation: phase 2
Algo validation: phase 2
Algo validation: phase 1
July 06
LTA delivery
SM L2OP and open prototype
OS L2OP and open prototype
Apr. 07 Nov. 07
SMOS SAG, November 2-3, 2006
Description ESAC LTA Portable SMOS Product Quality Control (SPQC)
See slide X X
Monitoring Facility (MF) See slide X X
Interactive Analysis Tools (IAT) SMOS Data Viewer
(SDV) See slide X
Reference Processing Facility
Off-line processing environment for testing new processor releases before they go into operation
X
Level 1 Prototype Processor (L1PP)
Output of the development lead by Deimos Portugal
X
Soil Moisture Open Prototype (SM L2PP)
Output of the development lead by Array
X
Ocean Salinity Open Prototype (OS L2PP)
Output of the development lead by ACRI
X
SEPS SMOS end-to-end performance simulator
X
MIRASIM MIRAS instrument simulator for command and control testing
X
Land Cover Tool Overlays SMOS values onto Google Earth
X
Auxiliary Data Handling Generates aux data for the DPGS X IDL Tool-Box Standalone version of report generation
functions integrated into the MF X
Others TBD
CEC
SMOS SAG, November 2-3, 2006
SPQC
SW system that operationally controls the quality of all SMOS products and auxiliary data before they are released to their final users
KO CDR FAT V1 FAT V2
April 06 July 06 Mar. 07Nov. 06
ESA detailed specification of checks
Example checks resulting in no wide distribution to users:
• File internal consistency: size, structure, coherence
• Product confidence data generated by processor is bad
• Too many unrealistic values in product
• Product cannot be processed to next level
LTA and DPGS V2
DPGS V1
SMOS SAG, November 2-3, 2006
MF
SW system that 1) monitors performance of instrument, platform, and ground segment, and 2) performs automated data synthesis to help with calibration and product quality assessment
KO CDR FAT V0 FAT V3
April 06 July 06 Dec. 07Nov. 06
ESA detailed specifications
Example reports:• Daily monitoring of on-board temperatures• Daily monitoring of DPGS production versus planning• Monitor aux data usage• Monitor temporal and spatial distribution of anomalous values• Continuous collection of statistics per pixel• Monitoring of SMOS product values versus external sources• Monitor archiving and distribution completeness
DPGS V2 DPGS V1 and LTA
FAT V1
Mar. 07
FAT V2
July 07
SMOS SAG, November 2-3, 2006
SMOS Data Viewer
For all SMOS products and auxiliary data, the SDV 1) aids in the viewing of the contents, and 2) provides specific visualisation functions to help in the detailed analysis of the contents
KO CDR FAT V1
Mar. 06 June 06 Oct. 06
The SDV V1 version will be made available to all, along with a set of sample products, in early December.
FAT V2
Feb. 07
FAT V3
June 07
L1A, L1B only
Generic visualisation (scatter plot, temporal plot, spectrum plots, histograms)
Specific visualisation:• L1A: Correlator matrix (I, Q, amplitude and phase) for science and calibration products• L1B: Visibilities in the star domain• L1B: Fourier transformation of BT components in the star domain
L1C, L2 added
Specific visualisation:
• L1C: Projection on to a map (selection of incidence angle range)
• L2: TBD
Specific visualisation:
• TBD
SMOS SAG, November 2-3, 2006
SMOS Data Viewer
SMOS SAG, November 2-3, 2006
XML Read/Write API
XML R/W API
A first coherent set (XML R/W API, schemas, test data) will be available to all in December up to L1C. Level 2 should be available by March. All of these tools will evolve with the processor developments throughout 2007.
L0 Processor
L1 Processor
L2 Processor
MF
SPQC L2 Open Prototype
Aux. Data Handling
Test Data• subsystem/system acceptance
• algorithm verification/validation
File Formats• XML schema (BinX language for binary data sets)
• documents
SMOS SAG, November 2-3, 2006
Data Distribution
User Services
Data Dissemination
Eoli Interface (eoli.esa.int)• internet browser based ordering (L1B, L1C, L2 user, SMOS ECMWF only) of small volumes (<50 products)• intended for users new to SMOS
Help Desk• subscription configuration• on-call support during working hours
ESAC• all fast processing files• time critical users via network, everyone else via media• subscriptions: time range, geographic area, ascending/descending• likely >> 10 Mbps for SMOS
ESRANGE• all fast processing and reprocessing files• time critical users via network, everyone else via media• subscriptions: time range, geographic area, ascending/descending• likely >> 50 Mbps for SMOS
New UserTime Critical
UserNominal
User
SMOS SAG, November 2-3, 2006
Coming Attractions
ECMWF Data Assimilation Study• Prepare for the operational assimilation of SMOS data with a goal to reduce the start-up time
• Provide inputs to aid in SMOS validation activities
• KO by January with a 27 month duration
Surface TB Processor• On-going project with CLS will deliver an ATBD for spring 2007
• This project takes the ATBD and produces an operational processor that will go in the CEC at launch and in the operational fast processing chain by the end of commissioning
• Open ITT to go out in early 2007 with a KO by May
SMOS SAG, November 2-3, 2006
Coming Attractions
Reprocessing Study• Current processor implementation addresses only the fast processing scenario and not a reprocessing campaign: no allowance for improved aux. data or calibration processing
• Take current L1 and L2 DPMs and modify these to apply to the case of a reprocessing campaign
• Open ITT to go out in early 2007 with a KO by May and a completion at the end of 2007
• There will be a follow-on activity to implement this DPM in the Reprocessing Centre of the LTA for the first reprocessing campaign (expected to follow shortly after the end of the commissioning phase)
SMOS SAG, November 2-3, 2006
Coming Attractions
NRT Service• Decision will be made at the next PB-EO (November 23-24)
• Proposal is to acquire during each of the 10 non-ESAC orbits and deliver the data (one file per APID) to ESAC for processing using a separate NRT processing chain and then distribution to NRT users via the baseline ESAC data distribution infrastructure
• Procurement will be by two open ITTs: 1) acquisition of the 10 non-ESAC orbits and delivery to ESAC, and 2) implementation of a NRT processing chain largely independent of the existing DPGS infrastructure
• Not the optimal solution financially or technically but meets all the political constraints for acceptance
SMOS SAG, November 2-3, 2006
Summary
• It is subsystem acceptance and integration season and this will likely continue for the next four-six months
• The Fast Processing and LTA developments are both experiencing delays but this is no longer launch critical; a decision will have to be made in summer regarding which system is more likely to be ready and stable at launch
• Level 2 operational processor non-start is the biggest fire today
• CEC is starting to take shape; although granted it is yet to be visible outside ESA
• By next SAG there should be more to show you then these slides and some empty offices at ESAC