Download - DAL (Data Access Layer) protocols
Workshop on How to Publish Data in VO
ESAC, June 25-June 29 2007
DAL (Data Access Layer) DAL (Data Access Layer) protocolsprotocols
Jesus SalgadoJesus SalgadoESAVO-ESACESAVO-ESAC
[email protected]@sciops.esa.int
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 22
Why do we need protocols?Why do we need protocols?
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 33
Why do we need protocols? (II)Why do we need protocols? (II)
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 44
Simple ProtocolsSimple Protocols
SIAP (Images), SSAP (Spectra), SIAP (Images), SSAP (Spectra), SLAP(Spectral Lines)SLAP(Spectral Lines)
HTTP basedHTTP based VOTable responseVOTable response Similar interfaces; different protocols for Similar interfaces; different protocols for
different types of products (images, spectra, different types of products (images, spectra, lines,…) to converge in a latter statelines,…) to converge in a latter state
Not difficult to be developedNot difficult to be developed
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 55
SIAPSIAPSimple Image Access ProtocolSimple Image Access Protocol
(Tody et al.)(Tody et al.)
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 66
SIAP: IVOA standardSIAP: IVOA standard […] […] standard for retrieving image data from a variety of standard for retrieving image data from a variety of
astronomical image repositories through a uniform astronomical image repositories through a uniform interfaceinterface […][…]
[…] […] based primarily on two documentsbased primarily on two documents […] […]"Simple Image Retrieval: Interface Concepts and Issues""Simple Image Retrieval: Interface Concepts and Issues",, […] […] "Simple Cone Search specification""Simple Cone Search specification" [...] [...]
Originally [...] primarily as an "image on demand" service, with Originally [...] primarily as an "image on demand" service, with images created on-the-fly [...]images created on-the-fly [...]
[…][…]Required query parameters: POS, SIZE, FORMATRequired query parameters: POS, SIZE, FORMAT[…][…] […][…]Required response columns: Image_TitleRequired response columns: Image_Title, POS_EQ_RA_MAIN, , POS_EQ_RA_MAIN,
POS_EQ_DEC_MAIN , Image_Format […]POS_EQ_DEC_MAIN , Image_Format […]
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 77
SIAP BasicsSIAP Basics
SIAP Servers service URLs
Query Response:VOTable with list of
matches
Data Response:Image
Coordinates/Target matches?
SIAP Servers available?
Give ACCREF data (real Data)
Client application REGISTRY
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 88
SIAP: factsSIAP: facts
More than 80 SIAP services registeredMore than 80 SIAP services registered The most successful IVOA protocol up to The most successful IVOA protocol up to
nownow Easy access to images from different Easy access to images from different
spectral ranges in the same VO application spectral ranges in the same VO application (e.g. Aladin)(e.g. Aladin)
Some examples:Some examples:– HST, 2MASS, XMM, SDSS, ISO, Integral, HST, 2MASS, XMM, SDSS, ISO, Integral,
ROSAT, FIRST, IRAS,… and many moreROSAT, FIRST, IRAS,… and many more
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 99
SSAPSSAPSimple Spectral Access ProtocolSimple Spectral Access Protocol
(Tody et al.) (Tody et al.)
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 1010
SSAP: A summarySSAP: A summary […][…]To define a uniform interface to spectral dataTo define a uniform interface to spectral data[…][…]
[…][…]The term “Simple” in Simple Spectral Access Protocol The term “Simple” in Simple Spectral Access Protocol refers to the design goal of simplicity in both implementing refers to the design goal of simplicity in both implementing spectral data services and in retrieving spectroscopic data spectral data services and in retrieving spectroscopic data from distributed data collectionsfrom distributed data collections[…][…]
[…][…]Required query parameters: POS, SIZE, FORMATRequired query parameters: POS, SIZE, FORMAT[…][…]
[…][…]Required response columns: FORMAT, ACREF, SED Required response columns: FORMAT, ACREF, SED Object, Data Source, Coverage MetadataObject, Data Source, Coverage Metadata…]…]
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 1111
SSAP BasicsSSAP Basics
SSA Servers service URLs
Query Response:VOTable with list of
matches
Data Response:Spectrum
( fits, VOTable, …)
Coordinates/Target matches?
SSA Servers available?
Give ACCREF data (real Data)
Client application REGISTRY
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 1212
SSAP 1.01 - Basic UsageSSAP 1.01 - Basic Usage Simple querySimple query
– POS, SIZE - like cone searchPOS, SIZE - like cone search– Possibly refined by spectral or time bandpass, etc.Possibly refined by spectral or time bandpass, etc.– Most metadata in query response is optionalMost metadata in query response is optional
Data retrievalData retrieval– URL-basedURL-based– Get back a dataset (normally VOTable or FITS)Get back a dataset (normally VOTable or FITS)– In simplest case could be wavelength, flux as text (for In simplest case could be wavelength, flux as text (for
Spectrum)Spectrum)– Or external data pass-throughOr external data pass-through
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 1313
SSAP 1.01 - Query InterfaceSSAP 1.01 - Query Interface Mandatory query parametersMandatory query parameters
– POSPOS RA, DEC (ICRS)RA, DEC (ICRS)– SIZESIZE diameter (decimal degrees)diameter (decimal degrees)– FORMATFORMAT VOTable, fits, xml, text, graphics, VOTable, fits, xml, text, graphics,
html, external html, external
Should other parameters be mandatory?Should other parameters be mandatory?– e.g., BANDPASS, TIME, SPECRES, APERTUREe.g., BANDPASS, TIME, SPECRES, APERTURE
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 1414
SSAP 1.01 - Query ResponseSSAP 1.01 - Query Response Dataset MetadataDataset Metadata
– Dataset.Type Dataset.Type Spectrum, TimeSeries, SED, etc.Spectrum, TimeSeries, SED, etc.– Dataset.DataModel Dataset.DataModel DM name, e.g., "SSA-V1.0"DM name, e.g., "SSA-V1.0"– DataID.Title DataID.Title Brief descriptive title of datasetBrief descriptive title of dataset– Dataset..Length Dataset..Length Number of points in Number of points in
datasetdataset– Dataset.SSA.SpectralAxis Dataset.SSA.SpectralAxis SpectralCoord axis (external data)SpectralCoord axis (external data)– Dataset.SSA.FluxAxis Dataset.SSA.FluxAxis Flux axis (external data)Flux axis (external data)– Dataset.SpectralSIDataset.SpectralSI SI factor and dimensionsSI factor and dimensions– Dataset.FluxSIDataset.FluxSI SI factor and dimensionsSI factor and dimensions– Dataset.CreationType Dataset.CreationType atlas, pointed, cutout, resampledatlas, pointed, cutout, resampled– DataID.DataSourceDataID.DataSource Original source of data Original source of data
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 1515
SSAP 1.1– SSAP 1.1– Some specialized response metadataSome specialized response metadata
Characterization - Reference FramesCharacterization - Reference Frames– CoordSys.SpaceFrame.NameCoordSys.SpaceFrame.Name Spatial Coordinate frame (default ICRS)Spatial Coordinate frame (default ICRS)– CoordSys.SpaceFrame.Equinox CoordSys.SpaceFrame.Equinox Equinox (2000)Equinox (2000)– CoordSys.TimeFrame.Name CoordSys.TimeFrame.Name Timescale (TT)Timescale (TT)
Characterization - CoverageCharacterization - Coverage– Char.SpatialAxis.Coverage.Location.ValueChar.SpatialAxis.Coverage.Location.Value
Observed position, e.g., RA DECObserved position, e.g., RA DEC– Char.SpatialAxis.Coverage.Bounds.Extent Char.SpatialAxis.Coverage.Bounds.Extent
Aperture angular diameterAperture angular diameter– Char.SpectralAxis.Coverage.Location.Value Char.SpectralAxis.Coverage.Location.Value
Midpoint of Spectral coord. rangeMidpoint of Spectral coord. range– Char.SpectralAxis.Coverage.Bound.Extent Char.SpectralAxis.Coverage.Bound.Extent
Width of spectrumWidth of spectrum
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 1616
SSAP implementationsSSAP implementations Implement service able to provide SSAP-like accessImplement service able to provide SSAP-like access
– ESA’s Infrared Space Observatory (ISO) spectra available in ESA’s Infrared Space Observatory (ISO) spectra available in SSAP-like form since December 2003. First ever SSAP-like server.SSAP-like form since December 2003. First ever SSAP-like server.
– Other SSAP-like services that followed suite (in chronological Other SSAP-like services that followed suite (in chronological order): IUE, HST, SDSS, FUSE, HYPERLEDA….up to 18 services order): IUE, HST, SDSS, FUSE, HYPERLEDA….up to 18 services
(all available from (all available from VOSpecVOSpec tool) tool) Implementation of latest protocol specification not ready for Implementation of latest protocol specification not ready for
most of themmost of them Design and implement client able to consume SSAP Design and implement client able to consume SSAP
spectra servicesspectra services– ESA’s initiative response called VOSpec: a tool for handling SSAP-ESA’s initiative response called VOSpec: a tool for handling SSAP-
like spectra in VO contextlike spectra in VO context
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 1717
TSAP (Theoretical spectra)TSAP (Theoretical spectra)
Same protocol (input/output) re-used to give Same protocol (input/output) re-used to give support theoretical spectral serverssupport theoretical spectral servers
Both coverage input parameters and output Both coverage input parameters and output characterization coverage metadata non-characterization coverage metadata non-compulsorycompulsory
As it will be seen in next section, discovery As it will be seen in next section, discovery of server capabilities is crucialof server capabilities is crucial
9 services already in place9 services already in place
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 1818
SLAPSLAPSimple Line Access ProtocolSimple Line Access Protocol
(Salgado et al.) (Salgado et al.)
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 1919
SLAP (Simple Line Access Protocol)SLAP (Simple Line Access Protocol)
VOTable, VOTable, one spectral line per rowone spectral line per row
A SIAP-like URL service, would allow A SIAP-like URL service, would allow an easy implementation for all spectral an easy implementation for all spectral line providersline providers
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 2020
SLAP input parametersSLAP input parameters If the only compulsory parameters are the minimum and maximum If the only compulsory parameters are the minimum and maximum
wavelength, the number of lines in the output result, in particular for wavelength, the number of lines in the output result, in particular for theoretical spectral line databases, could be HUGE.theoretical spectral line databases, could be HUGE.– Some non-compulsory parameters added to the protocol to filter or Some non-compulsory parameters added to the protocol to filter or
score the output linesscore the output lines– CHEMICAL_ELEMENT:CHEMICAL_ELEMENT: This parameter would constraint the This parameter would constraint the
search to the chemical element selected (syntax problems, but search to the chemical element selected (syntax problems, but some standards found)some standards found)
– LEVEL_ENERGY_START & LEVEL_ENERGY_END: LEVEL_ENERGY_START & LEVEL_ENERGY_END: Parameters Parameters to specify the minimum and maximum energy for the START & to specify the minimum and maximum energy for the START & FINAL level of the transition. Filter at CLIENT sideFINAL level of the transition. Filter at CLIENT side
– TEMPERATURE:TEMPERATURE: This parameter would be used (in particular for This parameter would be used (in particular for theoretical spectral line databases) to score the lines in the output theoretical spectral line databases) to score the lines in the output using physical models. Score at SERVER sideusing physical models. Score at SERVER side
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 2121
SLAP extra parametersSLAP extra parameters The servers could have extra parameters that cannot be identified The servers could have extra parameters that cannot be identified
easily and included in the document.easily and included in the document. For Theoretical Spectral Line services, the parameters could be used For Theoretical Spectral Line services, the parameters could be used
to filter/score the result due to physical modelsto filter/score the result due to physical models For Observational Spectral Line services, the parameters could be For Observational Spectral Line services, the parameters could be
project dependentproject dependent Solution adopted:Solution adopted:
– Use same approach than proposed for TSAP. All the input parameters will Use same approach than proposed for TSAP. All the input parameters will be DESCRIBED in a FORMAT=METADATA query.be DESCRIBED in a FORMAT=METADATA query.
– The description should include not only the parameter names but a The description should include not only the parameter names but a description, UCDs (to identify the physical meaning of the parameters), description, UCDs (to identify the physical meaning of the parameters), utypes whenever possible (to Line Data Model or another IVOA data utypes whenever possible (to Line Data Model or another IVOA data model)model)
– This FORMAT=METADATA could be requested by the registry services or This FORMAT=METADATA could be requested by the registry services or by the VO clientsby the VO clients
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 2222
FORMAT=METADATAFORMAT=METADATA <RESOURCE type="results"> <DESCRIPTION>IASD Simple Line Access Service</DESCRIPTION> <INFO name="QUERY_STATUS" value="OK"/> <PARAM name="INPUT:WAVELENGTH" ucd="em.wl;stat.min" utype=”ldm:Line.wavelength” value="30"> <DESCRIPTION> Specify the wavelength spectral range. To be specified in micrometers. This wavelength will be interpreted as the wavelength in the vacuum of the transition originating the line </DESCRIPTION> </PARAM> <PARAM name="INPUT:CHEMICAL_ELEMENT" ucd=“phys.atmol.element" utype=”ldm:Line.initialElement.name” value="190"> <DESCRIPTION> Specify the chemical element that originates the line </DESCRIPTION> </PARAM> <PARAM name="INPUT:OBSNO" ucd="obs.id"> <DESCRIPTION> Specify the ISO observation number where this line has been identified </DESCRIPTION> </PARAM> ……………………………………..
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 2323
Output “MUST” Compulsory fields– wavelength (ucd="em.wl”; utype=”ldm:Line.wavelength”)
wavelength in the vacuum of the transition originating the line in micrometers
– title (ucd=“em.line”;utype=”ldm:Line.title”) contains a small description identifying the line.
Output “SHOULD” non-compulsory field. – chemicalelement_name
(ucd=“phys.atmol.element”;utype=”ldm:ChemicalElement.name”) contains a name of the chemical element source of this line
– initial_level & final_level (ucd=“phys.atmol.[initial/final];phys.atmol.level”; utype=”ldm:Line.[initial/final]Level) contain a full description of the initial & final levels of the transition originating the line
SLAP output parameters (I)SLAP output parameters (I)
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 2424
SLAP output parameters (II)SLAP output parameters (II) Summary of output COULD optional fields
– observed_wavelength (ucd="em.wl“; utype=”ldm:Line.observedWavelength”) contains the observed wavelength in the vacuum of the transition originating the line in micrometers, as it was observed. This could be used by observational spectral line databases
– score (utype=”Query.Score”) similar concept to SSAP 0.9. Easier to be implemented for physical models
– level_energy_start & level_energy_end (ucd="phys.energy; phys.atmol.[initial/final];phys.atmol.level”; utype=”ldm:Level.energy.[start/end]”) describe the energy for the starting & final levels of the transition which originates this line.
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 2525
URL-like fields– publication_link (ucd="meta.bib”) specifies a http link that contains
a bibliographic reference related to the spectral line – Other fields could have ucd=“meta.ref.url" specifying URLs that
contains extra information related to the spectral line Non-Standard output fields
– In many occasions, extra scientifically interesting parameters could be added to the output. Implementors are encouraged to add descriptions and UCDs to the return fields to clarify the meaning of this information and utypes to the Line data Model or other existing Data Model, whenever possible.
4 services already in place; IASD, Cielo (observational), NIST, Lerma (laboratory). Plans to include ALMA in a near future
SLAP output parameters (III)SLAP output parameters (III)
Jesus SalgadoJesus Salgado Plenary Session, Page Plenary Session, Page 2626