11
Unidata Policy Committee Unidata Policy Committee 5 February5 February
Level II Data Access & DistributionLevel II Data Access & DistributionLinda MillerLinda Miller
RootsRoots Why Level II Data?Why Level II Data? CRAFTCRAFT ApplicationsApplications NWS distribution NWS distribution UPC involvementUPC involvement Next StepsNext Steps
22
RootsRoots 1998 discussions with reps from USWRP, 1998 discussions with reps from USWRP,
OU, Unidata, NCAR, FSL, et alOU, Unidata, NCAR, FSL, et al• Use of LDM to distribute Level II data for Use of LDM to distribute Level II data for
universities and to NCDC for archivaluniversities and to NCDC for archival Unanimous endorsement for UPC to Unanimous endorsement for UPC to
proceed with collaboration at 1998 Policy proceed with collaboration at 1998 Policy Committee meeting in R.I. Committee meeting in R.I.
33
Roots-cont.Roots-cont. Collaborative Radar Acquisition Field Collaborative Radar Acquisition Field
Test (CRAFT) was bornTest (CRAFT) was born Main goal: “..to demonstrate real-Main goal: “..to demonstrate real-
time compression and data time compression and data distribution of multiple radars distribution of multiple radars delivering Level II data with a view delivering Level II data with a view toward nationwide implementation” – toward nationwide implementation” – technology transfer!technology transfer!
44
Why Level II Data?Why Level II Data? Level III (NIDS-type) are digital products Level III (NIDS-type) are digital products
(pictures)(pictures) Level II provides full resolution digital data; Level II provides full resolution digital data;
all elevation angles; full data precisionall elevation angles; full data precision Level I is too voluminous-includes I&Q in Level I is too voluminous-includes I&Q in
time series from reflectivity, velocity, time series from reflectivity, velocity, spectrum width (nuts and bolts of data)spectrum width (nuts and bolts of data)
Level II is research quality dataLevel II is research quality data
55
CRAFTCRAFT Successful collaboration with stakeholders Successful collaboration with stakeholders
from universities, private sector, governmentfrom universities, private sector, government Testing and refinements with LDM and Testing and refinements with LDM and
NSSL’s compression (Bzip2) with initial 6 NSSL’s compression (Bzip2) with initial 6 radarsradars
Use of LDM to move data from radars to Use of LDM to move data from radars to NCDC for archiving requirementNCDC for archiving requirement• Allowed transition from 8mm Exabyte Allowed transition from 8mm Exabyte
tapes tapes • Huge cost savings and reliability realizedHuge cost savings and reliability realized
66
CRAFT-contCRAFT-cont Currently 102 radars Currently 102 radars 35 sites receiving data from OU-CAPS35 sites receiving data from OU-CAPS
• 17 universities17 universities• 12 government 12 government • 6 private industry6 private industry
1 March- NWS will have 121 NWS and 3 1 March- NWS will have 121 NWS and 3 DOD sites on “CRAFT“/NWS NetworkDOD sites on “CRAFT“/NWS Network
NWS will announce options for distributing NWS will announce options for distributing Level II data by end of Feb 2004Level II data by end of Feb 2004
77
ApplicationsApplications Use in courses to expose students to full suite of radar Use in courses to expose students to full suite of radar
information-future plans include mesoscale modelinginformation-future plans include mesoscale modeling Cloud and wind analysis & mesoscale data assimilationCloud and wind analysis & mesoscale data assimilation Initializing forecastsInitializing forecasts Support NASA’s Tropical Rainfall measuring Mission (TRMM) Support NASA’s Tropical Rainfall measuring Mission (TRMM)
Ground Validation ProgramGround Validation Program Real time storm trackingReal time storm tracking Spotter operationsSpotter operations Experimental correction of echo tops researchExperimental correction of echo tops research Storm-scale data assimilation and numerical predictionStorm-scale data assimilation and numerical prediction Visualization with IDV and IRASVisualization with IDV and IRAS Storm analysis and radar data quality improvementsStorm analysis and radar data quality improvements Creating special products for customers Creating special products for customers Data archivalData archival
88
NWS DistributionNWS Distribution
ROC/NWS active CRAFT stakeholders CRAFT deemed an appropriate project for technology
transfer to NWS operations• NOAA/NWS became Abilene collaborator• Use of Internet2 High Speed Network to support distribution
of Level II data to NCEP, NCDC, and academic community• Private sector has 10 sec latency requirement• NCEP has 60 sec latency requirement• NWS held meeting in June, 2003, to discuss proposed method
of data distribution using Internet• Private sector responded by recommending FOS option
99
• Family of Services Proposed Radar Products Service II Estimated Costs
- One Time Cost (OTC) for processors (servers), routers, router I/O cards, cables, matrix switch I/O cards, etc. ($68K)
- OTC will be spread out over a period of 5 years to minimize the cost to external users ($13.6K)
- Other Direct Cost (Includes OTC) ($15K)
- Direct Labor Cost ($42K)
- Distributed (Indirect) Costs ($25K)
- Total Product/Service Cost ($83K/YR)
- Estimated Quantity (4), (5), (6), (7), (8)
- Final Product/Service Price ($21K), ($17K), ($14K), ($12K), ($10K)
1010
NWS Distribution StatusNWS Distribution Status Solicitation sent 12/05Solicitation sent 12/05 Evaluations conducted by:Evaluations conducted by:
• 3 Unidata technical staff3 Unidata technical staff• 3 NWS technical staff3 NWS technical staff
Point system to each proposal-Miller collected all Point system to each proposal-Miller collected all evaluations and tallied outcome – sent to NWSevaluations and tallied outcome – sent to NWS
Awards will be announced after MOAs have been Awards will be announced after MOAs have been signed by selecteessigned by selectees
1111
NWS Distribution & UPC Involvement NWS Distribution & UPC Involvement
UPC involvement UPC involvement • LDM 6LDM 6• Coordination with NWS Hdqtrs for a “Call for Coordination with NWS Hdqtrs for a “Call for
Participation” for top level sites to provide Level II data Participation” for top level sites to provide Level II data distribution to universities, government, private industry distribution to universities, government, private industry
Terms of “Call” – freely available to univ, gov, and on “cost Terms of “Call” – freely available to univ, gov, and on “cost recovery” basis for private industryrecovery” basis for private industry
Sign an MOA with NOAA/NWS with terms associated with Sign an MOA with NOAA/NWS with terms associated with the solicitation.the solicitation.
Solicitation-not FAR (Federal Acquisition Regulation) Solicitation-not FAR (Federal Acquisition Regulation) • UPC to provide LDM training to NWS regional hdqtrs staff UPC to provide LDM training to NWS regional hdqtrs staff
1212
NWS NWS 4 sites will provide LDM data 4 sites will provide LDM data
distribution of Level II datadistribution of Level II data 2 sites were pre-selected by the NWS;2 sites were pre-selected by the NWS;
• Mid-Atlantic Crossroads (Max) GigaPop Mid-Atlantic Crossroads (Max) GigaPop (collocated with Univ of Maryland)(collocated with Univ of Maryland)
• ERC GigaPop (collocated with NCDC-ERC GigaPop (collocated with NCDC-Asheville)Asheville)
2 remaining sites to be selected2 remaining sites to be selected
1313
INTERNET2(Abilene)
NEX
RA
DS
NWSNet
NWSNet
NWSNet
NWSNet
Eastern RegionLDM Server
Central RegionLDM Server
Southern RegionLDM Server
Western RegionLDM Server
WFO
WFO
WFO
WFO
WFO
WFO
WFO
WFO
WFO
WFO
WFO
WFO
Utah EducationNetwork GigapoP
NOAA NWSSilver Spring
DS3 Gigabit
DS3
DS3
OC3
Mid Atlantic Crossroads(MAX) GigaPoP
North Texas GigaPoP
Great PlainsNetwork GigaPoP
Ethernet
ONENetGigaPoP
(Oklahoma City)
OU LDM Server
Private SectorClients
I2 GigaPoP
Unidata ClientUniversity LDM
Server
I2 GigaPoP
Unidata ClientUniversity LDM
Server
TOC LDMServer
ERC GigaPoP(NCDC Asheville, NC)
ERC LDMServer
Backup Capability
LDM Feeds to Univ ersities
LDM Feeds to TOC Serv er
LDM Feeds to ERCWC Serv er
LDM Feeds to OU Serv er
Distribution of NEXRAD Level II Archive DataTo Research/Private Sector Users
(11/13/2003)
NCEP LDMServer
NCDC LDMServer
Private SectorClients
GigabitEthernet
OC3 ATM
1414
Next StepsNext Steps Unidata to train regional hdqtr sitesUnidata to train regional hdqtr sites
• Salaries, benefits, & travel covered by Salaries, benefits, & travel covered by NWSNWS
Assist selected top level sites with Assist selected top level sites with IDD topology, if desiredIDD topology, if desired• Provide support to top level <.edu> Provide support to top level <.edu>
sitessites• Provide support for <.edu> recipients of Provide support for <.edu> recipients of
datadata
1515
More Information?More Information?
Contact:Contact:[email protected]@unidata.ucar.edu
ororLinda Miller Linda Miller
External Liaison, UnidataExternal Liaison, [email protected]@unidata.ucar.edu
303-497-8646 303-497-8646