some grid science
DESCRIPTION
Some Grid Science. Roy Williams Paul Messina. California Institute of Technology. Grids and Virtual Observatory Grids and and LIGO. Virtual Sky: Image Federation. http://virtualsky.org/ from Caltech CACR Caltech Astronomy Microsoft Research. Virtual Sky has 140,000,000 tiles - PowerPoint PPT PresentationTRANSCRIPT
Some Grid Science
California Institute of Technology
Roy WilliamsPaul Messina
Grids and Virtual Observatory
Grids andand LIGO
Virtual Sky: Image Federation
Xray (ROSAT) theme
Change scale
Change theme
http://virtualsky.org/fromCaltech CACRCaltech AstronomyMicrosoft Research
Optical (DPOSS)
Coma cluster
Virtual Sky has140,000,000 tiles
140 Gbyte
VO and Grid ComputingSupercomputer center support for
Data Pipelines (not just MPI)
12 hr 250 Gb
HPSS
HP Superdome
VS1 Science productHPSS system
VS2 Web productWin2000 database
Raw data
CPU Utilization
VirtualSky.org Ingestion
Copy raw data and filter (background subtraction)
Resampling
Write VS1
Build VS2
Write VS2
Today ~ 3 TB
1 TB
140 GBComputation also takes about 12 hrs.
VO and database JoinR
AD
ec
mag
Jm
ag
Hm
ag
K
glo
ng
lat
flu
x I
flu
x Q
flu
x U
Infraredcatalog
(indexed by RA)
Radiocatalog
(indexed by glon)
Crossmatchengine
SQL select
SQL select
Need same sorting for crossmatchWho does it?Or can we impose a standard?
order by glon?
order by RA?
sort here?
Sorting 10^9 objects is expensive
VO Interoperability• Directory white/yellow pages
– VO Schema repository
• How to publish, how to define content• action> Document schema for
– archive content– web service capability– table/image/spectrum
• How to publish algorithms• Plug and play data services
– SOAP, UDDI, WSDL, Jini?
• Semantic web– “Question and answer in the language of the client” – Topic maps
VO and Interoperability
Client application (e.g., OASIS)
asks for available catalogs
then for attributes
makes a query
then displays the result
What is the standard interactionwith a catalog service?
Client Service
Want to be dynamic, be able to
add catalogs and the applications still work
VO and International
•Of course there are many astronomy data collections in the UK, Europe, Japan, etc.
•We intend to collaborate/coordinate with international efforts
LaserInterferometricGravitational waveObservatory
Listening to Collisions ofBlack Holes andNeutron Stars
LIGO
LIGO Pictures
Grid LIGO Architecture
Clientseg Web, Script, Agent
Clientseg Web, Script, Agent
Text requestText request Request Manager
Request Manager
GriPhyN LDAS
Gatekeeper(GRAM)
Gatekeeper(GRAM)
Science AlgorithmsSoftware Collaboratory
Parallel ComputingGridFTPGridFTP
Local Disk
Data
HPSS
GridFTPGridFTP
Replica CatalogReplica ManagementTransformation CatalogVirtual Data Catalog
Virtual Data Request
Data Movement
Globus RPC
other LDASCondor jobs
Objective: Add security & wide-area data servicesLDAS is Ligo Data Analysis System
Grid LIGO Logic• VD Request Request Manager
– XML, Key-value, script?– Action> Build document schema for
this• Do we have it already in data caches
– if yes, get from caches, use simple transformations
– else build LDAS script•Move relevant data to LDAS local
disk•Queue job to LDAS, get jobID•Pass updates, then result pointer
back to user•Perhaps cache result
International GW Grid
50 kbyte/sec
GEO
International GW Grid• Coincidence in Ligo-Virgo data
– Not astrophysics data yet….– Try seismic, electromagnetic data for now
Caltech
VirgoElectromagneticSeismic
Coincidence events!
Currently using rsync
LLO
LHO
Merging 3 streams to 1
Correlate signals(LDAS)
~12 kbyte/sec each way