virtual observatory interfaces reused in the virtual atomic and molecular data centre
Post on 08-Jan-2016
22 Views
Preview:
DESCRIPTION
TRANSCRIPT
Virtual Observatory Interfaces reused in the
Virtual Atomic and Molecular Data Centre
Guy RixonInstitute of AstronomyUniversity of CambridgeSeptember 2009
Rixon: VObs interfaces reused in VAMDC 2
VAMDC reusing VObs IT
Virtual Observatory
VAMDC
Paradigms,Paradigms,Standards,Standards,
ImplementationImplementationss
Toocomplicated?
Start fromscratch?
Toosimplistic?
WHY?
Rixon: VObs interfaces reused in VAMDC 3
Past, present and future
2009 2010 2011 2012 2013 Time
Use
fuln
es
s
Era of Separateweb-sites
VAMDCproject ???
IT support for use of A & M data does not start and end with the VAMDC project.
Rixon: VObs interfaces reused in VAMDC 4
Goals Open
Easy to add users (and user communities) Extensible
Easy to add data and applications Composable
Easy to combine with other software Sustainable
Easy (cheap) to maintain Finished in < 3 years
Rixon: VObs interfaces reused in VAMDC 5
Goal conflicts
Extensible, composable, cheap:
pick any two...
“Good Software Takes Ten Years. Get Used To it” – Joel Spolsky
Rixon: VObs interfaces reused in VAMDC 6
How does the Virtual Observatory do?
Tries to be open, extensible, composable, etc.
Web services Standard protocols Resource discovery via registry Workflow support via data grids
Rixon: VObs interfaces reused in VAMDC 7
Open and Composable
GUI Script Workflow Etc.
Vobs service Vobs service Vobs service Vobs service
HTTP callsacross WAN
Web (HTTP) services
Rixon: VObs interfaces reused in VAMDC 8
Extensible: adding data
VObs clients
Site A Site B Site C
Pu
bli
c
Private
Rixon: VObs interfaces reused in VAMDC 9
My community's
apps
My community's
apps
The IVOA “API library”
Open & extensible: adding applications
Pu
bli
c
Private“The Virtual Observatory”
My personal
apps
My personal
apps
My personal
apps
My community's
apps
Standard Apps &utilities
Standard Apps &utilities
Generic apps &utilities
Rixon: VObs interfaces reused in VAMDC 10
Extensible and sustainable: registry
Publish, describe
Discover,choose
Resources registered in on-line directory Scientists can discover meaning of resources Software can discover interface details No need to update software to use new
resources
Rixon: VObs interfaces reused in VAMDC 11
What can be registered?
Hypothetical extension by VAMDC
Rixon: VObs interfaces reused in VAMDC 12
Structure of registration document
Resource identity
Curation
Content synopsis
Capability(TAP)
Capability(VAMDC)
Tableset
“Dublin core”: meaningful to science usersUsed to select services.
Each Capability describes a major interfaceLists access URL(s) and support for optional parts of interfaces. Machine readable detail;drives UIs.
Describes the tables available for query.Machine-readable detail; drives querybuilding UIs.
Rixon: VObs interfaces reused in VAMDC 13
Relational vs. XML storage
Relational: Robust, mature Choice of RDBMS SQL query Hard to add new
schemata
Native XML DB New(ish)
technology Few
implementations XQuery Trivial to add
new schemata
Registries have been built on both relationaland native-XML databases.
XQuery + schema flexibility wins every time.
Rixon: VObs interfaces reused in VAMDC 14
Publishing, harvesting, searching
“Full” registry
NVO (STSci)
Publishingregistry
VizierPublishing
registry
CASUPublishing
registry
VAMDCregistry
Etc.
All registrations“harvested” to fullregistry
VAMDC providers registerServices here
Virtual Observatory VAMDC
VAMDC users search here
Rixon: VObs interfaces reused in VAMDC 15
Extensible and composable: data grid
Service A Service B Service C
VOSpace VOSpace
Fast network
My app.
Slower network
Rixon: VObs interfaces reused in VAMDC 16
UIs: VOSpec Multi-wavelength spectral
analysis tool developed at ESAC by ESA-VO
Create a SED with spectra from: a wide range of
different providers registered in the VO Registry
local data (FITS, ASCII, VOT)
Many fitting/ plotting functions
ISO LWSHST
ISO PHT
EUVE
HUT
HST FOSIUE
INES
http://esavo.esac.esa.int/vospec/
Rixon: VObs interfaces reused in VAMDC 17
VOSpec cont. VOSpec provides access
to atomic and molecular line Databases (SLAP): NIST Atomic Spectra CHIANTI (X-rays) CIELO (XMM-Newton) IASD (ISO) LERMA (molecular) STSCI (UV)
on-the-fly identification of transitions associated to a given observed feature.
Rixon: VObs interfaces reused in VAMDC 18
UIs: Taverna
Rixon: VObs interfaces reused in VAMDC 19
UIs: VODesktop
Rixon: VObs interfaces reused in VAMDC 20
VODesktop: actions on resources
Rixon: VObs interfaces reused in VAMDC 21
VODesktop: query builder
Rixon: VObs interfaces reused in VAMDC 22
VODesktop/SAMP: UI integration
Rixon: VObs interfaces reused in VAMDC 23
VODesktop: data-grid access
Rixon: VObs interfaces reused in VAMDC 24
Looter's checklist
Web-service paradigm Registry paradigm Data-grid paradigm VODesktop Other UIs/desktop applications (as
applicable) Service standards (as applicable) Service implementations (as applicable)
Rixon: VObs interfaces reused in VAMDC 25
IVOA standard services
Standard Purpose Data model Relevance?
Registry Discovery of services Extensible High
Cone search Data access: sky search Astronomical None
SIAP Data access: images Astronomical None
SSAP Data access: spectra Astronomical Low/medium
TAP Data access: tables Generic High
SimDB/SimDAP Data access: simulations Generic ???
SLAP Data access: line lists Astronomical Medium/high
VOSpace Grid-accessible storage Generic Medium
UWS-PA/CEA Grid execution Generic Medium
Rixon: VObs interfaces reused in VAMDC 26
The data-model issue
Web-service shell
Data-model
Resource adaptor
Requests
Results
What the client needs
Elaborate data-modelmakes it hard to repurpose the service,especially if domain-specific
For reuse, prefer serviceswith little or no data model
Rixon: VObs interfaces reused in VAMDC 27
TAP: Table Access Protocol
Web-service shell
Data-model
Resource adaptor: async. queries on JDBC
Requests: SQL Results: VOTable/HTML/CSV/???
Relational database
TAP data-model isvery generic:“relational data”
Works with any DB schema
Rixon: VObs interfaces reused in VAMDC 28
TAP: possible use in VAMDC year 1
TAPservice
TAP
DB
VAMDCApp (desktop)
Describe DB schematain registry
Reuse same implementation on each VAMDC site
Sites provide these
Deal with schemataVariations here
One per VAMDC site
Rixon: VObs interfaces reused in VAMDC 29
TAP: possible evolution
Web-service shell
Data-model
Resource adaptor: async. queries on JDBC
Requests: XQuery
Results: XSAMS
Relational database
Replace the TAP data-model with something VAMDC-specific
Works with any DB schema
Extend this for VAMDC
Rixon: VObs interfaces reused in VAMDC 30
VOSpace
Storage accessible to users and services Useful in workflows, grid jobs Can use simple storage or data grid
Site A Site B
iRODS
Site C
iRODS
Site C
iRODS
VOSpace VOSpace VOSpace
Rixon: VObs interfaces reused in VAMDC 31
SLAP: Simple Line Access Model
Web-service shell
Data-model
Resource adaptor
Requests Results: VOTable
Relational database
SLAP data-model isvery simple: “just enough for astronomy”
Needs a particular schema
Rixon: VObs interfaces reused in VAMDC 32
CEA & grid
CEA = Common Execution Architecture Describe, register, discover, run server-side apps:
grid “A standard for everything else” Powerful:
CEA UI runs any registered app Allows long-running jobs (asynchronous
execution) Can be connected to the Grids, e.g. EGEE Integrates with VOSpace, Taverna workflow
Rixon: VObs interfaces reused in VAMDC 33
If all else fails...
...design a new service type! Need to specify:
Service semantics Web-service protocol (e.g. REST) Query language (e.g. XQuery) Representation of results (e.g. XSAMS) XML schema for registration
Consider following IVOA “DAL” architecture
Rixon: VObs interfaces reused in VAMDC 34
VAMDCstrategy
Adapting to new service types
VODesktop UI
Registry parser
AstroRuntime
SLAPstrategy
SSAPstrategy
TAPstrategy
CEAstrategy VAMDC
strategy
Rixon: VObs interfaces reused in VAMDC 35
Summary
VAMDC needs adaptable, future-proof software on the cheap
We think the VObs model will work We will copy the VObs ideas We will use their service protocols to
start We will use the ready-made software We will exploit the flexibility of the
system progressively to fit it to VAMDC.
top related