votable agenda current votable status news from applications referring stc (as a data model example)...

VOTable agenda

Current VOTable status News from Applications Referring STC (as a data model example) Relations between <TABLE>s Questions about VOTable schema Late topics Actions & Future

Where are we ?

VOTable1.1 is almost 3 years old (quite stable!)

quite generally used, many tools What’s missing:

general referencing to external data models, especially STC

Final schema which satisfies the applications ... and some (bogus?) tools

News from Applications

VOConvert v1.0 (successor of ConVOT) VOStat (beta version) (on Friday) VOMegaPlot (many rows, density plots),

VOPlot v1.4 TOPCAT v2.3 (supports all formats:

TABLEDATA / FITS / BINARY). Problem of GROUP not solved ?

Referencing Models in VOTable The tabular material may contain in its fields

(columns) any kind of data it is impossible to add into a VOTable

document all the various XML codes related to all data models developed by the VO

a VOTable document therefore refers to data models without including them

Utypes: what is it ?

in VOTable schema: utype is a non-mandatory attribute of any RESOURCE TABLE FIELD PARAM GROUP originally created for DAL needs is an acceptable attribute wherever the ucd accepted contrary to the ucd, gives a fully detailed meaning of the

field, parameter or group ucd = broad semantics, typically used for data mining utype = detailed semantics, refers to a data model

Utype: its usage in VOTable

can supply an exact description of the column contents immediate application for referencing quantities

(parameters and/or fields) which exact meaning is crucial systems of coordinates: celestial, terrestrial, solar, ...

(connection with STC) time definitions (connection with STC) photometric systems & filters more generally any parameter part of a model, simulation...

STC Connection

STC is an essential component to precise the conventions of dates, locations, coordinate systems (WhenWhere in VOEvent)

used in most VO components should replace (and deprecate) the COOSYS

ad-hoc convention

Scenarios for STC

5 scenarios are proposed on an example of a table having 5 columns: ObservingTime, RightAscension, Declination, ErrorRA, ErrorDec

examples proposed on the VOTable forum. Very few answers up to now – but we have to take a decision this week !

-----------------------------------------------------------------ObservingTime RightAscension Declination RA_Err Dec_Err (ICRS, deg) (ICRS, deg) (arcsec) (arcsec)-----------------------------------------------------------------2007-05-13T23:00:05 358.1234 +05.1234 0.19 0.192007-05-13T23:00:06 358.4567 -05.4321 0.21 0.222007-05-13T23:00:07 358.5678 +04.1234 0.23 0.222007-05-13T23:00:08 358.6789 -04.4321 0.22 0.25-----------------------------------------------------------------

A very simple example

<!-- Proposal #1 for an accurate definition of time and position: o Time and space kept apart, o the GROUP describes the parameters of the coordinate system, as well as its members ("double" referencing) o No description of the ObservingTime column, i.e. default (UTC) assumed.--><GROUP utype="stc:AstroCoords.Position2D" ID="ICRS_TOPO_SPH"> <PARAM name="CooFrame" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.CoordRefFrame.Type" value="ICRS" /> <PARAM name="CooFrameOrigin" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.ReferencePosition.Type" value="TOPOCENTER" /> <PARAM name="CooType" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.CoordFlavor.Type" value="SPHERICAL" /> <!-- We reference here the columns which are part of this group --> <FIELDref ref="RA" /> <FIELDref ref="DEC" /> <FIELDref ref="Err1" /> <FIELDref ref="Err2" /></GROUP>

Double referencing

<FIELD ID="OT" name="ObservingTime" datatype="char" arraysize="19" utype="stc:AstroCoords.Time.TimeInstant.ISOTime" ucd="time.start;obs.exposure" /><FIELD ID="RA" name="Right Ascension" datatype="double" utype="stc:AstroCoords.Position2D.Value.C1" ref="ICRS_TOPO_SPH" unit="deg" ucd="pos.eq.ra" /><FIELD ID="DEC" name="DEClination" datatype="double" utype="stc:AstroCoords.Position2D.Value.C2" ref="ICRS_TOPO_SPH" unit="deg" ucd="pos.eq.dec" /><FIELD ID= "Err1" name="RA_Error" datatype="float" utype="stc:AstroCoords.Position2D.Error.Value.C1" ref="ICRS_TOPO_SPH" unit="arcsec" ucd="error;pos.eq.ra" /><FIELD ID= "Err2" name="DEC_Error" datatype="float" utype="stc:AstroCoords.Position2D.Error.Value.C2" ref="ICRS_TOPO_SPH" unit="arcsec" ucd=”error;pos.eq.dec" />

Default UTC Time

<?xml version="1.0" encoding="UTF-8"?><!-- Proposal #2: o one GROUP for each of the space / time definitions o the GROUP does not describe its members (no "double referencing")--><GROUP utype="stc:AstroCoords.Time" ID="TT_TOPO"> <PARAM name="TimeScale" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.TimeFrame.TimeScale" value="TT" /> <PARAM name="TimePos" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.TimeFrame.ReferencePosition.Type" value="TOPOCENTER" /></GROUP> <GROUP utype="stc:AstroCoords.Position2D" ID="ICRS_TOPO_SPH"> <PARAM name="CooFrame" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.CoordRefFrame.Type" value="ICRS" /> <PARAM name="CooFrameOrigin" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.ReferencePosition.Type" value="TOPOCENTER" /> <PARAM name="CooType" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.CoordFlavor.Type" value="SPHERICAL" /></GROUP>

Time defined

No double referencing

<FIELD ID="OT" name="ObservingTime" datatype="char" arraysize="19" ucd="time.start;obs.exposure" utype="stc:AstroCoords.Time.TimeInstant.ISOTime" ref="TT_TOPO"/><FIELD ID="RA" name="Right Ascension" datatype="double" ucd="pos.eq.ra" unit="deg" utype="stc:AstroCoords.Coord.Value.C1" ref="ICRS_TOPO_SPH"/><FIELD ID="DEC" name="DEClination" datatype="double" ucd="pos.eq.dec" unit="deg" utype="stc:AstroCoords.Coord.Value.C2" ref="ICRS_TOPO_SPH"/><FIELD ID= "Err1" name="RA_Error" datatype="float" ucd="error;pos.eq.ra" unit="arcsec" utype="stc:AstroCoords.Coord.Error.Value.C1" ref="ICRS_TOPO_SPH"/><FIELD ID= "Err2" name="DEC_Error" datatype="float" ucd="error;pos.eq.dec" unit="arcsec" utype="stc:AstroCoords.Coord.Error.Value.C2" ref="ICRS_TOPO_SPH"/></TABLE>

<?xml version="1.0" encoding="UTF-8"?><!-- Proposal #3: o Hierarchy of GROUPs to mime the STC requirements for a hierarchy o Use the direct value in utype (e.g. .TOPOCENTER) , and then value is void. o the GROUP does not describe its members (no "double referencing")--><GROUP ID="TT-ICRS-WAVELENGTH-TOPO" utype="stc:AstroCoords" > <GROUP utype="stc:AstroCoordSystem.TimeFrame" > <PARAM name="TimeScale" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.TimeFrame.TimeScale" value="TT" /> <PARAM name="TimePos" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.TimeFrame.TOPOCENTER" value="" /> </GROUP> <GROUP utype="stc:AstroCoordSystem.SpaceFrame" > <PARAM name="CooFrame" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.ICRS" value="" /> <PARAM name="CooFrameOrigin" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.TOPOCENTER" value="" /> <GROUP name="CooType" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.SPHERICAL" > <PARAM utype="stc:AstroCoordSystem.SpaceFrame.SPHERICAL.coord_naxes" value="2" /> </GROUP> </GROUP> <GROUP utype="stc:AstroCoordSystem.SpectralFrame" > <PARAM name="SpectralPos" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpectralFrame.TOPOCENTER" value="" /> </GROUP></GROUP>

<FIELD ID="OT" name="ObservingTime" datatype="char" arraysize="19" ucd="time.start;obs.exposure" utype="stc:AstroCoords.Time.TimeInstant.ISOTime" ref="TT-ICRS-WAVELENGTH-TOPO" /><FIELD ID="RA" name="Right Ascension" datatype="double" ucd="pos.eq.ra" unit="deg" utype="stc:AstroCoords.Position2D.Value2.C1" ref="TT-ICRS-WAVELENGTH-TOPO" /><FIELD ID="DEC" name="DEClination" datatype="double" ucd="pos.eq.dec" unit="deg" utype="stc:AstroCoords.Position2D.Value2.C2" ref="TT-ICRS-WAVELENGTH-TOPO" /><FIELD ID= "Err1" name="RA_Error" datatype="float" ucd="error;pos.eq.ra" unit="arcsec" utype="stc:AstroCoords.Position2D.Error2.C1" ref="TT-ICRS-WAVELENGTH-TOPO" /><FIELD ID= "Err2" name="DEC_Error" datatype="float" ucd="error;pos.eq.dec" unit="arcsec" utype="stc:AstroCoords.Position2D.Error2.C2" ref="TT-ICRS-WAVELENGTH-TOPO" />

<?xml version="1.0" encoding="UTF-8"?><!-- Proposal #4 (suggested by Roy) follows STC definitions o 2 GROUPs: one for the definition of the system, another for the coordinates o the GROUP does not describe its members (no "double referencing")--><GROUP utype="stc:AstroCoordSystem"> <PARAM name="STC_ID" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.id" value="TT_ICRS_TOPO"/> <GROUP utype="stc:AstroCoordSystem.TimeFrame" > <PARAM name="TimeScale" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.TimeFrame.TimeScale" value="TT" /> <PARAM name="TimePos" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.TimeFrame.ReferencePosition.Type" value="TOPOCENTER" /> </GROUP> <GROUP utype="stc:AstroCoordSystem.SpaceFrame" > <PARAM name="CooFrame" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.CoordRefFrame.Type" value="ICRS" /> <PARAM name="CooFrameOrigin" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.ReferencePosition.Type" value="TOPOCENTER" /> <PARAM name="CooType" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.CoordFlavor.Type" value="SPHERICAL" /> </GROUP></GROUP><GROUP ID="Coords1" utype="stc:AstroCoords"> <PARAM name="AstroCoo" datatype="char" arraysize="*" utype="stc:AstroCoords.coord_sys_id" value="TT_ICRS_TOPO"/></GROUP>

duplicationof themeaning of TT_ICRS_TOPO

<FIELD ID="OT" name="ObservingTime" datatype="char" arraysize="19" utype="stc:AstroCoords.Time.TimeInstant.ISOTime" ucd="time.start;obs.exposure" ref="Coords1" /><FIELD ID="RA" name="Right Ascension" datatype="double" unit="deg" utype="stc:AstroCoords.Position2D.Value2.C1" ucd="pos.eq.ra" ref="Coords1" /><FIELD ID="DEC" name="DEClination" unit="deg" datatype="double" utype="stc:AstroCoords.Coord.Value2.C2" ucd="pos.eq.dec" ref="Coords1" /><FIELD ID= "Err1" name="RA_Error" datatype="float" unit="arcsec" utype="stc:AstroCoords.Coord.Error.Value.C1" ucd="error;pos.eq.ra" ref="Coords1" /><FIELD ID= "Err2" name="DEC_Error" datatype="float" unit="arcsec" utype="stc:AstroCoords.Coord.Error.Value.C2" ucd="error;pos.eq.dec" ref="Coords1" /></TABLE>

<!-- Proposal #5 (expanded from 4) o 2 GROUPs: one for the definition of the system, another for the coordinates --><!-- Definition of one AstroCoordSystem --><GROUP utype="stc:AstroCoordSystem"> <PARAM name="STC_ID" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.id" value="TT_ICRS_TOPO"/> <GROUP utype="stc:AstroCoordSystem.TimeFrame" > <PARAM name="TimeScale" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.TimeFrame.TimeScale" value="TT" /> <PARAM name="TimePos" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.TimeFrame.ReferencePosition.Type" value="TOPOCENTER" /> </GROUP> <GROUP utype="stc:AstroCoordSystem.SpaceFrame" > <PARAM name="CooFrame" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.CoordRefFrame.Type" value="ICRS" /> <PARAM name="CooFrameOrigin" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.ReferencePosition.Type" value="TOPOCENTER" /> <PARAM name="CooType" datatype="char" arraysize="*" utype="stc:AstroCoordSystem.SpaceFrame.CoordFlavor.Type" value="SPHERICAL" /> </GROUP></GROUP>

<!-- Description of the coordinates used (referencing AstroCoordSystem) --><GROUP "ID=Coords1" utype="stc:AstroCoords"> <PARAM name="AstroCoo" datatype="char" arraysize="*" utype="stc:AstroCoords.coord_sys_id" value="TT_ICRS_TOPO"/> <GROUP utype="stc:AstroCoords.Time.TimeInstant.ISOTime"> <FIELDref ref="OT" /> </GROUP> <GROUP utype="stc:AstroCoords.Position2D"> <GROUP utype="stc:AstroCoords.Position2D.Value2"> <FIELDref ref="RA" /> <FIELDref ref="DEC" /> </GROUP> <GROUP utype="stc:AstroCoords.Position2D.Error2"> <FIELDref ref="Err1" /> <FIELDref ref="Err2" /> </GROUP> </GROUP></GROUP>

Double referencing

<FIELD ID="OT" name="ObservingTime" datatype="char" arraysize="19" utype="stc:AstroCoords.Time.TimeInstant.ISOTime" ucd="time.start;obs.exposure" ref="Coords1" /><FIELD ID="RA" name="Right Ascension" datatype="double" unit="deg" utype="stc:AstroCoords.Position2D.Value2.C1" ucd="pos.eq.ra" ref="Coords1" /><FIELD ID="DEC" name="DEClination" unit="deg" datatype="double" utype="stc:AstroCoords.Coord.Value2.C2" ucd="pos.eq.dec" ref="Coords1" /><FIELD ID= "Err1" name="RA_Error" datatype="float" unit="arcsec" utype="stc:AstroCoords.Coord.Error.Value.C1" ucd="error;pos.eq.ra" ref="Coords1" /><FIELD ID= "Err2" name="DEC_Error" datatype="float" unit="arcsec" utype="stc:AstroCoords.Coord.Error.Value.C2" ucd="error;pos.eq.dec" ref="Coords1" /></TABLE>

…conclusion ? definition of AstroCoordSystem – necessary or just

reference some component of STClib (not yet fully described in STC) ?

Wouldn't it be better to have distinct references for time and coordinates ? (most frequently time specification is completely disconnected from coordinate system)

double referencing (from GROUP and from FIELD) – useful or bad ?

GROUP hierarchy – can it be problematic ?

Why not choose Simplicity?

AstroCoordSystem definition only for non-standard system – use otherwise a reference <PARAM ... utype="stc:AstroCoords.coord_sys_id" value="TT_ICRS_TOPO"/>

if agreed with other groups, have separate Space, time, wavelength, etc... axes

“double referencing” optional

Relations between tables (1)

One VOTable document may contain several tables, meaning tables logically grouped.

In the relational model, relations between tables are specified via the concept of keys

basic key definitions: primary key (non-null, unique) and foreign key (must exist as primary in the related table)

A simple solution: GROUPs

Relations between tables (2)

<GROUP name=”primaryKey”> <FIELDref ref=”ClusterName”><FIELDref ref=”GalaxyName”>

</GROUP>...<FIELD ID=”ClusterName” .../><FIELD ID=”GalaxyName” .../>

Relations between tables (3)

<GROUP name=”foreignKey” ref=”mainTable”> <FIELDref ref=”ClusterName”><FIELDref ref=”GalaxyName”>

</GROUP> ...<FIELD ID=”ClusterName” .../><FIELD ID=”GalaxyName” .../>

Relations between tables (4)

Other possible interesting details: order of the data, e.g. with

<GROUP name=”order” ><PARAM name=”sequence” value=”increasing”><FIELDref ref=”ClusterName”><FIELDref ref=”GalaxyName”>

</GROUP>

Relations between tables (5)

What is better: just a <GROUP> with a specific name ? prefer the definition of some “relational” data

model and refer to these groups with utypes ?

ref attribute to TABLE (1) Current definition:

A TABLE may have a ref attribute referencing the ID ofanother table previously described, which is interpretedas defining a table having a structure identical to the one referenced: this facility avoids a repetition of the definition of tables which may be present many times in a VOTabledocument.

ref attribute to TABLE (2) the ref attribute refers to the table structure,

what about the following:

<TABLE ID="template"> <FIELD name="Col1" … /> <FIELD name="Col2" … /> <DATA><TABLEDATA> <TR> <TD>1</TD><TD>2</TD> </TR> </TABLEDATA></DATA></TABLE>

<TABLE ref="template"> <DATA><TABLEDATA> <TR> <TD>1.1</TD><TD>2.1</TD> </TR> </TABLEDATA></DATA></TABLE>

ref attribute to TABLE (3)

from Moscow meeting: allow a reference to an empty table only (i.e. a table without <DATA> part)

is useful for many chunks of the same data structure (e.g. excerpts around a set of positions) but would it better to allow several <DATA> in a

<TABLE> ? prefer a supplementary column ?

VOTable schema changes (1)

Problem of code generators (xsd) where the class generated by TABLEDATA consists of TD[][] instead of a class TR[] where TR consists in TD[] Easy to solve if an attribute is added to <TR>, e.g.

<TR ID="…"> discussed in Moscow – strong opposition ! could be in the schema without being allowed

VOTable schema changes (2) <INFO> problem – again a bug from code generator

<INFO> has just 3 attributes: name value ID, plus simple content (i.e. any non-tagged text between <INFO> and </INFO>). This content generates a conflict with the “value” attribute

would be wise to make <INFO> similar to <PARAM> where datatype is a string (character string of any length), which means:

accept utype ucd ID ref unit attributes accept <DESCRIPTION> <VALUES> <LINK> sub-

elements

VOTable schema changes (3)

Streaming processing delivering VOTable output: there must be a possibility of providing a diagnostic about possible failure in the output stream

allow to insert <PARAM> and <INFO> tags after a </TABLE> ?

VOTable schema changes (4) discussed in Moscow: RESOURCE recursivity

can be a problem, accept <RESOURCE type="notRecursive"> ?

Deprecate <COOSYS>, replaced by <GROUP> with its parameters and utypes to the STC model.

Accept several <DATA> in a <TABLE> ? VOTable1.2 : volunteers to test the new

schema ?

VOTable document changes

Write down how to reference the STC with actual examples

Explain the table relations (primary / foreign key conventions) with examples; other conventions ?

Detail the null/NaN conventions