astronomical grid applications at esac [email protected] science archives and computer...

Astronomical GRID Applicationsat ESAC

[email protected]

Science Archives and Computer Engineering Unit

Science Operations Department

ESA/ESAC – Madrid, Spain

Christophe ARVISETEGEE 09, ESAC GRID activities, 24/09/2009, page 2ESA Science Operations

Department

European Space Astronomy Centre

ESAC default location for: Science operations Centres (SOC),

long history with astronomical missions

expanding with solar system missions

Science archives, Astronomy Planetary Solar Systems

ESA Virtual Observatory (VObs) activities, ESAC is the European VObs

node for space-based astronomy

http://www.esa.int/SPECIALS/ESAC/Located near Madrid, Spain


Department

ESA’s sentinels in the Solar SystemFrom the Sun to the planets

SOCsArchivesat ESAC


Department

The wonders of the UniverseAstronomy and fundamental physics at ESA

SOCsArchivesAt ESAC


Department

ESAC GRID Objectives

ESAC to be a Data Centre and a Data Processing Centre Natural link between Science Archives, GRID and VObs

ESAC Scientists to use the GRID for their daily science tasks Must be easy to use (“my homedir on the GRID”, “gridlogin”) User support to port their software to the GRID New massive computing facilities to substitute user workstation

GRID in support to the projects Project Standard Data Processing On-the-fly reprocessing from the Archives Instruments calibration monitoring and trend analysis HPC in the GRID

Collaboration with other GRID partners / organizations EGEE, Planck, UCM, IFCA, IVOA


Department

GRIDs at ESAC

ESACGRID On ESAC DMZ Part of EGEE Collaboration with other

EGEE sites Project Data Processing Close link to the ESAC

Archives

SCIGRID Intranet Globus based Scientist daily use

DMZ

INTRANET

FW

Computing Element (CE)

LCG CE(GT-preWS)

PBS

User Workstation

GW client commandsNFS

… SE, LFC, MON, VOMS ...

Worker Nodes(Blades PE1955)

SCIGRID PRIVATE LAN

node node...

SGE

GRIDWAY

GT4

SCIGRID


Department

ESACGRID Hardware Details

In production since the beginning of 2007

40 Blade Servers for around 280 cpus and 860GB RAM

Hardware features: DELL PowerEdge 1855, 1955 and M60. 2 Dual/Quad core Intel CPU 3.20GHz and 1.86GHz

OS: Scientific Linux CERN 4 32 bits or 64 bits

Based on EGEE middleware gLite 3.1 Part of EGEE

Planck and eurovo-dca Virtual Organizations


Department

Herschel Pipeline Data Processing

Herschel Science Centreat ESAC

MissionOperationsCentreESOC

Raw DataDB

Herschel PipelineRun on the GRID

Herschel Science Archive

Raw data

Calibrationdata

Scientific Products

Search and Download Herschel data


Department

Traditional XMM-NewtonScience Analysis Workflow

Vizualize data

Run SAS tasks andStore results

On Local computer

Download and Install SAS,Calibration DB,

3rd party SW, libraries, …

Search and download datafrom XMM-Newton Archive


Department

XMM-Newton RISA – Remote Interface for Science Analysis

Vizualize Results withVObs tools (VOSpec, Aladin)

VObs Access data

VObs DisplayRequest

Save Results onVOSpace

VObs Search data

XMM-Newton Archive

Define SAS Tasks

Search data with RISA Web client

Run SAS tasks on ESAC GRID

VObs Download data

Send SAS TasksTo the GRID


Department

XMM-Newton Mosaic Construction

Send Processingto the GRID

VObs Downloaddata

XMM-Newton Archive

Final Mosaics

Basic processing to get the pipeline

products (PPS) run in parallel for every

observation

4 maps x 5 bands

x 3 instrum.

Total mosaic (group all previously

generated mosaics)

3 finalmosaic

Build mosaics for every observation per band

3 combinedimagesper obs10 observations in 5 energy bands:

10 obs x 3 instrument x 5 bands x 4 maps (cts, exp, var, bkg) 600 maps before mosaicking+60 x 3 (cts, exp, var) = 180 summed maps per obs

10 obs = 780 maps

Construct all images and spectra (per band and per instrument)

18 mapsper obs


Department

XMM-Newton Mosaic Construction

Send Processingto the GRID

VO Downloaddata

XMM-Newton Archive

Final Mosaic

Basic processing to get the pipeline

products (PPS) run in parallel for every

observation

4 maps x 5 bands

x 3 instrum.

Total mosaic (group all previously

generated mosaics)

1 finalmosaic

Build mosaics for every observation per band

1 mosaicper obs

10 observations in 5 energy bands:10 obs x 3 instrument x 5 bands x 4 maps (cts, exp, var, bkg) 600 maps before mosaicking+60 x 3 (cts, exp, var) = 180 summed maps per obs

10 obs = 780 maps

Construct all images and spectra (per band and per instrument)

18 mapsper obs


Department

Acknowledgments

ESAC Computer Support Group ESAC Science Archives and VObs Team ESAC Herschel Data Processing Team ESAC XMM-Newton SAS Team Guillaume Belanger for the XMM-Newton Mosaicing work

astronomical grid applications at esac [email protected] science archives and computer...

Documents