reprocessing and product distribution: collection 6 and future plans
DESCRIPTION
Reprocessing and Product Distribution: Collection 6 and future plans. Edward Masuoka Code 619 [email protected]. Total Volume of MODIS Terra/Aqua Products Distributed by Fiscal Year. Note: 1) only half way through FY 12 (expect >4PB in 2012) - PowerPoint PPT PresentationTRANSCRIPT
Reprocessing and Product Distribution: Collection 6 and future plans
Edward MasuokaCode 619
MODIS Science Team Meeting May 7, 2012 1
2
Total Volume of MODIS Terra/Aqua Products Distributed by Fiscal Year
MODIS Science Team Meeting May 7, 2012
FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120.0
500.0
1,000.0
1,500.0
2,000.0
2,500.0
3,000.0
3,500.0
4,000.0
TERRA AQUA
Fiscal Year
Dist
ributi
on V
olum
e (T
B)
Note: 1) only half way through FY 12 (expect >4PB in 2012) 2) numbers exclude distribution via disk drives
3
Number of Terra/Aqua Products files
MODIS Science Team Meeting May 7, 2012
FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120
50,000
100,000
150,000
200,000
250,000
TERRA AQUA
Fiscal Year
Num
ber o
f Pro
duct
s Dist
ribut
ed (1
000s
)
Expect >250M product files distributed in 2012
4
Number of distinct MODIS users
MODIS Science Team Meeting May 7, 2012
FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,000
180,000
TERRA AQUA
Fiscal Year
Num
ber o
f Use
rs
Note: only half way through FY 12
MODIS Science Team Meeting May 7, 2012
5
FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120
500
1,000
1,500
2,000
2,500
Distribution Volume of Terra MODIS ProductsGESDISC LPDAAC MODAPS MODAPSNRT NSIDC OBPG ORNL PODAAC
Fiscal Year
Dist
ributi
on V
olum
e (T
B)
FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120
200400600800
1,0001,2001,4001,600
Distribution Volume of Aqua MODIS ProductsGESDISC LPDAAC MODAPS MODAPSNRT NSIDC OBPG ORNL PODAAC
Fiscal YearDist
ributi
on V
olum
e (T
B)
MODIS Science Team Meeting May 7, 2012
6
FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,000
Number of Terra MODIS UsersGESDISC LPDAAC MODAPS MODAPSNRT NSIDC ORNL PODAAC
Fiscal Year
Num
ber o
f Use
rs
FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY120
5,000
10,000
15,000
20,000
25,000
Number of Aqua MODIS UsersGESDISC LPDAAC MODAPS MODAPSNRT NSIDC ORNL PODAAC
Fiscal Year
Num
ber o
f Use
rs
MODIS Reprocessing
C4 2003 C5 2007 C6 20120
20
40
60
80
100
120
515
100
MODAPS – Level 1, Land and Atmosphere OBPG – Ocean products
2003 2007 20120
100200300400500600700800
20x forward processing 50x reprocessing string 80x test strings100x reprocessing string waiting on disk
Complete reprocessing of all Ocean missions to produce uniform suite of products underway at 700x
MODIS Science Team Meeting May 7, 2012
8
Test Systems – Atmosphere/Land• Integration and Test system at Wisconsin
identically configured to our software transfer group’s I&T system
• Science test string for Atmospheres (mtvs6) used to test individual changes to PGEs, 40x processing rate, tests involve processing multiple months
• Science test string for Land (mtvs5), 40x processing rate, tests involve processing multiple years up to the entire data record
MODIS Science Team Meeting May 7, 2012
9
C-6 Lessons Learned
• Separate strings for each discipline are essential to avoid delays in testing which impact development and integration of PGEs.
• Atmosphere’s science tests involving single changes to a PGE have proven an effective approach to algorithm development.
• Developing individual test plans for science testing and communicating them to operations is an area that still needs improvement.
MODIS Science Team Meeting May 7, 2012
10
Future Vision
• Collection 7 defined with Science Team• Long-term data records AVHRR-MODIS-VIIRS • Standards for metadata to support provenance• Documentation improved for long-term archive• Continued expansion of product formats • More cross DAAC orders filled• Common web services across DAACs• Virtualization to support 32bit PGEs and older OS