1

ASTER DATA Transfer System

Earth Remote Sensing Data Analysis Center

Kunjuro Omagari

ERSDAC

23th APAN Meeting

2

Agenda

Research Purposes Network Structure Data Transfer / Archive Data Transfer / Distribution Future Challenges Summary

ERSDAC

3

Research Purposes － International cooperation －

NASA Goddard Space Flight Center (GSFC/US)

Jet Propulsion Laboratory (JPL/US) USGS / National Center for Earth

Resources Observation & Science (EROS/US)

ERSDAC/JAPAN

ERSDAC

4

AIST/JAPAN ERSDAC/JAPAN

Transfer all ASTER L0 data for research and development of AIST GEO Grid supercomputer System.

ERSDAC

Research Purposes － Domestic cooperation －

5

Carry out research and development of a system which aims at reducing the time between observation and data acquisition by researchers, and to provide an infrastructure

・ Time between observation and data acquisition ・ Old system: about 14 days (by sending a tape ）

・ Target: about 4 days (by network transfer)

ERSDAC

Research Purposes － Reduced distribution time －

6

Amount of received data（ NASA -> ERSDAC ） : 500 observed scenes / day

about 60GB Level 0 data / day Amount of send data

(ERSDAC -> EROS):

500 scenes / day

about 60GB processed Level 1 data / day

ERSDAC

Research Purposes － Large-volume data transfer －

7

Transfer data for research and development of the supercomputer technology (AIST GEO Grid System ）

800 GB / day

ASTER L0 data transfer in operation Transfer newly-arrived ASTER L0 data

Research Purposes － Large-volume data transfer －

8

Send and receive observation schedule etc. (ftp)

Receive satellite telemetry data （ GRE tunnel + multicast ）

Voice hotline (VoIP)

ERSDAC

Research Purposes － Support for mission management －

9ERSDAC

JGN2/TrnsPAC2

JGN2/TrnsPAC2

ERSDAC

EROS/JPLGoddard Space Flight Center

Tokyo XPAPAN

StarlightChicago

MAX/Abilene

MAX/Abilene

VBNS+VBNS+

Router

Router-1 Router-2

EBnet LAN

Router

1G Ether

Network Structure － Network overview －

Using 1Gbps Ether as the access line (changed from 100M bps to 1Gbps in 2006/Nov)

Connecting from JGN2 to Starlight Chicago (Internet2), via Tokyo XP APAN, with transPAC2 as the backup line

Connecting from Starlight Chicago to NASA ： Goddard Space Flight Center, USGS EROS and JPL, via MAX/VBNS+ etc

11

Tracking and Data Relay Satellite

日本

TERRA（ 5 sensors onboard)

ASTER sensor

Data distribution

Control command

Tracking and Data Relay Satellite System

U.S.A.

JPL GSFC

Data Acquisition RequestData Processing Request

ERSDAC Telemetry data

Level 0 data/VoIP

Control command

observationobservation dataTelemetry data

Data processing to Level 0Level 1 dataProcessing/Archive

ASTER Ground data system

EOSDIS／ NASA

User

EROS

Observation schedule

Level 1 data

JGN2/transPAC2

Tokyo XPAPAN

Star LightChicago

ERSDAC

Network Structure － Observation system overview －

Observation dataTelemetry data

User

Data Acquisition RequestData Processing Request

Data distribution

Japan

After observation by ASTER sensor, receive the data from satellite TERRA in the U.S

Process the data to Level 0 at NASA, and transfer it to ERSDAC via networkProcess the data to Level 1 at ERSDAC

Process the data to Level 1 at ERSDAC and transfer it to EROSArchive the data both at EROS and ERSDAC

13

IPV4 TCP/UDP network Installed the hotline between NASA

and ERSDAC by two VoIP lines Receiving telemetry data with GRE

tunnel + multicast packet Confirmed the data transfer by multi-

session ftp and by e-mail

ERSDAC

Network Structure － Communication protocol －

17

Receiving the GRE tunnel, which is used for multicast packet transfer, by WAN connecting router

The firewall that ERSDAC currently has can’t let multicast packet go through. => unable to use the firewall

Change of the GRE tunnel － Before change －

ERSDAC

18

To extend the end of the GRE tunnel to the connecting router (Cisco8201) of a required segment, as well as to duplicate it.

=> able to make good use of the existing firewall

ERSDAC

Change of the GRE tunnel end － After change －

19

Network Structure － GRE tunnel 　 before －

Multicast-Server

Router　:　Multicast-Routing

PIM-Interface(sparse-dense-mode)

Multicast-Client

GRE Tunnel

OSPF

129.165.221.28

Cisco2801

210.138.100.32/27

Cisco2801

Multicast-client

PIM-Interface(dense-mode)

Version 11.2(15a)

Version 11.1(8)Version 11.1(8)Cisco4500MCisco4500M

NASA

ERSDAC

Tunnel

Tunnel

210.138.100. 64/27

Cisco4500M

???.???.???.???/ ??

Cisco7206VXR

Multicast-Server

Router　:　Multicast-Routing

PIM-Interface(sparse-dense-mode)

Multicast-Client

GRE Tunnel

OSPF

129.165.221.28

Cisco2801

210.138.100.32/27

Cisco2801

Multicast-client

PIM-Interface(dense-mode)

Version 11.2(15a)

Version 11.1(8)Version 11.1(8)Cisco4500MCisco4500M

NASA

ERSDAC

Tunnel

Tunnel

210.138.100. 64/27

Cisco4500M

???.???.???.???/ ??

Cisco7206VXR

ERSDAC

20

Multicast-ServerRouter　:　non　Multicast-Routing

Router　:　Multicast-Routing

PIM-Interface(sparse-dense-mode)

Multicast-Client

GRE Tunnel

GRE Tunnel

ip mroute 129.165.223.6 255.255.255.255 Tunnel0ip mroute 129.165.223.14 255.255.255.255 Tunnel0ip mroute 198.118.223.27 255.255.255.255 Tunnel0ip mroute 198.118.223.28 255.255.255.255 Tunnel0

OSPF

129.165.221.28

Cisco2801

ip mroute 129.165.223.6 255.255.255.255 Tunnel0ip mroute 129.165.223.14 255.255.255.255 Tunnel0ip mroute 198.118.223.27 255.255.255.255 Tunnel0ip mroute 198.118.223.28 255.255.255.255 Tunnel0

210.138.100.32/27

Cisco2801

Multicast-client

PIM-Interface(dense-mode)

Version 11.2(15a)

Version 11.1(8)Version 11.1(8)Cisco4500MCisco4500M

NASA

ERSDAC

Tunnel0

Tunnel1

210.138.100. 64/27

interface Tunnel0ip unnumbered Loopback0ip pim sparse-dense-modetunnel source Loopback0tunnel destination 129.165.221.28!interface LoopBack0203.181.248.222/32

interface Tunnel0ip unnumbered Loopback0ip pim sparse-dense-modetunnel source Loopback0tunnel destination 129.165.221.28!interface LoopBack0203.181.248.221/32

Cisco4500M

???.???.???.???/ ??

interface Tunnel0ip unnumbered Loopback0ip pim sparse-dense-modetunnel source Loopback0tunnel destination 203.181.248.221! interface Tunnel1ip unnumbered Loopback0ip pim sparse-dense-modetunnel source Loopback0tunnel destination 203.181.248.222!interface LoopBack0129.165.221.28/32

Existing setting

Existing setting

There must be reachavility between Source Internet Protocol address and Destination Internet Protocol address of Tunnel.

ERSDAC

Change of the GRE tunnel － After change －

25

Tokyo XP APAN

JGN2 　

Data transfer server

Tape storage

Data processing server

Data distribution control server

Distribution server

ERSDAC

InterNET

Receive L0 dataFrom NASA

Transfer the received L0 data to AIST

Data Transfer / Archive － ERSDAC internal system －

Put the data on DTF-2 tapes into storage

Generate primary data by data processor

Archive the primary data

27

Tokyo XP APAN

JGN2 　

ERSDAC

InterNET

Send the primary data to EROS

Distribute the primary data to Users

Data Transfer / Distribution － ERSDAC internal system －

Distribution server

Data transfer server

Data distribution control server

Tape storage

Data processing server

Retrieve products for high-level processing

Distribute the high-level data to Users

Archive the high-level data

28

Future Challenges

To apply the results of research and development of supercomputer technology （ GEO Grid System ） by AIST

To assess the transfer of not only ASTER-data, but also PALSAR-data （ TBD ）

ERSDAC

30

Summary

As of now, the interval between observation and Level 1 data distribution has been reduced to about 4 days, while it used to be around 12 days. (The interval between observation and data generation has been reduced to about 2 days.)

We plan to review and change the archive method, so that we can process the data immediately after receiving it. (Currently, we wait for two-day’s data to be stored and then process it.) (TBD)

We started the data transfer for development of the AIST GEO Grid system, with 800GB/day (2007/Nov)

We plan to use the GEO Grid system.

ERSDAC