GOES-R Data Distribution

Timothy J. Schmit

NOAA/NESDIS – Office of Research and Applications (ORA)

Advanced Satellite Products Team (ASPT), Madison, WI

James Gurka and Roger Heymann (NOAA/NESDIS/OSD)

and the whole GRB working group, etc

May 2004GOES User Conference

IntroductionGOES Re-Broadcast (GRB)Data compressionOther optionsSummary

What we know• Data distribution is very important• There are a large number of current GVAR reception

sites, more sites are being added• These are a combination of fixed and mobile sites. • There is a wide range of current users needs• GOES-R instrument data rates increase by

approximately two orders of magnitude over current instruments

• Data compression can help to reduce data rates (and hence distribute more data) while preserving information

What we think we know

• Data (raw) downlink most likely will be in the X-band• Data re-broadcast will most likely be in the L-band• GOES Re-Broadcast (GRB) format will be different than

todays GVAR• There will be some form of satellite re-broadcast• A tunable range of data compression options, depending

on data, is ideal.• Data compression techniques will continue to improve

What we do not know

• Data formats

• Data rate of re-broadcast data

• Contents of re-broadcast data

• Type or Amount of data compression

• Relationship between amount of data “pushed” versus “pulled”

• Etc.

• Working Group (WG) assembled to examine the nature of GOES-R high rate data to be distributed

Led by James Gurka (OSD) & Tim Schmit (ORA)

Includes senior scientists, senior engineers, and policy officials

James Gurka - WG Co-Lead NESDIS OSD

Tim Schmit - WG Co-Lead NESDIS ORA

Bill Brockman Short & Associates

Ken Carey Mitretek

Dennis Chesters NASA

Monica Coakley MIT Lincoln Lab

Roger Heymann NESDIS OSD

Eric Madsen NESDIS International Affairs

John Pereira NESDIS OSD

Thomas Renkevens NESDIS OSDPD

Dick Reynolds Short & Associates

Steve Short Short & Associates

GOES Re-Broadcast (GRB)

While no final decisions have been made, several data distribution options are being investigated for the GOES-R era.

- Various amounts of bandwidth rebroadcast via GOES

- Various amounts of data rebroadcast via other than GOES (commercial)

- Other methods (Internet, push/pull etc)

- Combination, etc.

Future GOES Rebroadcast• GRB system is extension of current GVAR system for GOES-R era

– Serve users in Near RealTime throughout western hemisphere– Current GVAR mainly only rebroadcasts radiance data; GRB may/could

rebroadcast both radiances and some products

• GRB system is payload service, separate from any direct service (LRIT, DCPI/R, EMWIN, and SAR)– GRB is needed to make a large amount of data available to a wide range of

users (both geographically and in terms of data use) in a cost efficient manner

• Main difference between GVAR and GRB:– Due to new, large data rates for the instruments currently planned for

GOES-R, the GRB system will not realistically be able to transmit all level 1b data without data compression

Possible GOES ReBroadcast System

“Data Pool”

Data compression can have applications in a number of areas:- downlink –rebroadcast - distribution - archive

“Downlink”2:1?

“Re-broadcast”6:1?

Users

Satellite Satellite

Wallops CDAS/Backup

Any GRB site

“Archive” 2:1 10+:1

“Data pool” of full GOES (the compete level 1b data) would not be compressed, nor rebroadcast. Full GOES would be available for ground-based transfers and archive.

General GRB Assumptions• User needs

– All users and applications not known – Forecasting all future users and applications uncertain

– If the data is available, user will work to gain access to it

• Users (DoD, WMO nations, academia, etc) will expect a similar (or higher) level of service

• Communications capabilities will continue to evolve – bringing improved capability, technologies, and lower cost

• Future data compression techniques will continue to improve (both lossless and lossy)

• Send out as much information (as opposed to just data) to as many users as possible while balancing cost of dissemination with the goal of maximizing the usefulness of the information

How might we move from 100 to 24 Mbps for a rebroadcast?

• Data compression is the key • Maintain a "data pool" concept for land access and archive of uncompressed sensor data

Users could then "test what is not being sent“ for their own application(s)

• One method going from ~100 Mbps to 24 Mbps: 10X for the 0.5 km visible band 6X for the "1km” bands 2X for the IR bands of the ABI 6X for the HES-IR

• Means approximately half of the band width would be used by the imager and half by the sounder; each image could be sent out

0

2

4

6

8

10

12

14

16

18

Dat

a R

ate

(Mb

ps)

"0.5 km" "1 km" "2 km"

Spatial Resolution

ABI Data Rates (Uncompressed)

Total (uncompressed) 43.5 MbpsAssuming 13-bit data and a 5-minute full disk scan mode

0.5 km visible data dominates the data rate of ABI

Total (uncompressed) 43.5 MbpsTotal (compressed) 10.4 Mbps

IR dominates the data rate of ABI after compression

0

2

4

6

8

10

12

14

16

18

Dat

a R

ate

(Mb

ps)

"0.5 km" "1 km" "2 km"

Spatial Resolution

ABI Data Rates (Compressed)

10:1 6:1 2:1

Original

MODIS 0.64 μm (0.5 km on ABI)

Compression Ratio ~9:1

Original

MODIS 0.86 μm (1.0 km on ABI)

Compression Ratio ~6:1

Simulation study shows that reconstructed brightness temperature is nearly noise-free

AIRS instrument noise

Reconstructed minus noise-free

Poster…Goldberg

Lossless Compression Comparison of Methodologies

Without pre-processing, JPEG-LS gives the best compression ratioPoster…B. Huang

Very Fast Lossy Compression JPEG-LS RMS error

A compression ratio of 5 is less than the instrument noise.A pre-processing step (right side graph) improves the performance.

Poster…B. Huang

NOAA NESDIS has investigated Alternative Dissemination Methods (ADM) for distribution of weather/environmental data by means of Internet, Commercial Space Communications, or Dedicated Landline. The ADM methods of communication are separate from communication methods utilized in Direct Readout (DR), which is a broadcast from government satellites.

12

ADM Systems View (SV–1)

May 2003 v1.0

CommercialSatellite

ADM - Alternative Dissemination Methods Center Hub

UplinkCEMSCS - Central Environmental

Satellite Computer System

SpaceDomain

Ground-stationDomain

NOAA Satellite SegmentGOES-R

N/O/PNPOESSNASA

POES

InternetFTP …

World Weather Building

DirectReadout

Users

Ground-DistributionDomain

Archival Domain

GOESI - M

Land Lines

APT/HRPT

for POES

LRD/HRDNPOESS

UserData

UserData

UserData

Processing Domain

System Interface Description (Diagram)

ADMUsers

CDA SPS - Command and Data Acquisition Sensor Processing System

UserData

GOESData

NOAAUserADMUser

Commercial

Acquisition Authority Legend

NASA

Leased by ADM

Acquired by ADM

SOCC

From Marlin Perkins

A Broad Agency Announcement (BAA) for future geostationary satellite architecture study is currently under-way.

Task Area 3: Production Generation and Distribution, Archive and Access, and User Interface Segments

http://www.osd.noaa.gov/goesr_arch_study/index.htm

Breakout Session Quesiton

• Question for each group during the break-out session on Thursday:

“Please discuss your needs for [realtime] data and product distribution, archiving and access. For example, timeliness, registration, etc. ”

end