astrodas : sharing assertions across astronomy catalogues through distributed annotation

AstroDAS: Sharing Assertionsacross Astronomy Cataloguesthrough Distributed Annotation

Rajendra Bose, Robert G. Mann, Diego Prina-Ricotti

Digital Curation Centre

4 May 2006

International Provenance and Annotation Workshop (IPAW’06)

Outline

1. Astronomy catalogues and existing OpenSkyQuery system

2. Custom cross-matching algorithms: AstroDAS

3. How AstroDAS compares to other annotation systems

SDSS (Visual)

TWOMASS (Infrared)

SDSS (Visual)

TWOMASS (Infrared)

AstroDAS: Astronomy Distributed Annotation SystemExample astronomy catalogue schema

SDSS:Sky node

TWOMASS:Sky node

USNOB:Sky node

:OpenSky Query client

OpenSkyQuery Portal

OpenSkyQuery

SELECTs.objid, t.objid, u.objid,s.ra, s.dec, s.type,t.ra, t.dec,u.ra, u.decFROMSDSS:photoprimary s, TWOMASS:photoprimary t, USNOB:photoprimary uWHEREXMATCH(s,t,u)<3.5 ANDRegion(’CircleJ2000 200 -1 0.02’) ANDs.type=3 AND

ADQL query

SDSS: Sloan Digital Sky Survey

TWOMASS: the Two Micron All Sky Survey

USNOB: U.S. Naval Observatory USNO-B1.0 catalogue

X-Match cross-matching algorithm built into OpenSkyQuery

based on spatial proximity; user specifies parameter sigmawhich encodes tolerance of match

AstroDAS: Astronomy Distributed Annotation SystemExisting OpenSkyQuery system for astronomy catalogue access

National Virtual Observatory. (2006). Open SkyQuery Help: The XMatch Algorithm <http://openskyquery.net/Sky/SkySite/help/algo.asp>

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

AstroDAS: Astronomy Distributed Annotation SystemExisting OpenSkyQuery system for astronomy catalogue access

http://openskyquery.net/

SDSS:Sky node

TWOMASS:Sky node

USNOB:Sky node

:OpenSky Query client

OpenSkyQuery Portal

OpenSkyQuery

SELECTs.objid, t.objid, u.objid,s.ra, s.dec, s.type,t.ra, t.dec,u.ra, u.decFROMSDSS:photoprimary s, TWOMASS:photoprimary t, USNOB:photoprimary uWHEREXMATCH(s,t,u)<3.5 ANDRegion(’CircleJ2000 200 -1 0.02’) ANDs.type=3 AND

ADQL query

AstroDAS: Astronomy Distributed Annotation System

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Existing OpenSkyQuery system provides X-Match results based on proximity

SDSS:Sky node

TWOMASS:Sky node

USNOB:Sky node

:OpenSky Query client

OpenSkyQuery Portal

OpenSkyQuery

SELECTs.objid, t.objid, u.objid,s.ra, s.dec, s.type,t.ra, t.dec,u.ra, u.decFROMSDSS:photoprimary s, TWOMASS:photoprimary t, USNOB:photoprimary uWHEREXMATCH(s,t,u)<3.5 ANDRegion(’CircleJ2000 200 -1 0.02’) ANDs.type=3 AND

ADQL query

AstroDAS: Astronomy Distributed Annotation System

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

But X-Match results based on proximity not always adequate

Catalogue1

Catalogue2

SDSS:Sky node

TWOMASS:Sky node

USNOB:Sky node

:OpenSky Query client

OpenSkyQuery Portal

OpenSkyQuery

SELECTs.objid, t.objid, u.objid,s.ra, s.dec, s.type,t.ra, t.dec,u.ra, u.decFROMSDSS:photoprimary s, TWOMASS:photoprimary t, USNOB:photoprimary uWHEREXMATCH(s,t,u)<3.5 ANDRegion(’CircleJ2000 200 -1 0.02’) ANDs.type=3 AND

ADQL query

AstroDAS: Astronomy Distributed Annotation System

UEdinburgh:<SDSS, 58778470> ↔ <TWOMASS, 35773478><SDSS, 58778470> ↔ <USNOB, 13317280><SDSS, 58777039> ↔ <USNOB, 13317184>

URome:<SDSS, 58778583> ↔ <TWOMASS, 35777512>

So group produces its own cross-match results

URome:AstroDAS

Server

SDSS:Sky node

UEdinburgh:AstroDAS

Server

TWOMASS:Sky node

USNOB:Sky node

:OpenSky Query client

:AstroDAS client

AstroDAS Portal

OpenSkyQuery Portal

AstroDAS

OpenSkyQuery

AstroDAS: Astronomy Distributed Annotation System

UEdinburgh:<SDSS, 58778470> ↔ <TWOMASS, 35773478><SDSS, 58778470> ↔ <USNOB, 13317280><SDSS, 58777039> ↔ <USNOB, 13317184>

URome:<SDSS, 58778583> ↔ <TWOMASS, 35777512>

SELECTs.objid, t.objid, u.objid,s.ra, s.dec, s.type,t.ra, t.dec,u.ra, u.decFROMSDSS:photoprimary s, TWOMASS:photoprimary t, USNOB:photoprimary uWHEREXMATCH(s,t,u)<3.5 ANDRegion(’CircleJ2000 200 -1 0.02’) ANDs.type=3 AND

ADQL query

Storing annotations to map database objects

AstroDAS: Astronomy Distributed Annotation SystemStoring annotations to map database objects

db_object db_object

SDSS_58778470 TWOMASS_35773478

annote1 author annote_sourceSAME OBJECT

(algorithm1) researcher1

(algorithm1) researcher1

… … … … ……

id

1

2

3

SDSS_58778470 USNOB_13317280

UEdinburgh:<SDSS, 58778470> ↔ <TWOMASS, 35773478><SDSS, 58778470> ↔ <USNOB, 13317280><SDSS, 58777039> ↔ <USNOB, 13317184>

URome:<SDSS, 58778583> ↔ <TWOMASS, 35777512>

SAME OBJECT

NOT SAME OBJECT

(algorithm2) researcher2SDSS_58778470 TWOMASS_35773478

URome:AstroDAS

Server

SDSS:Sky node

UEdinburgh:AstroDAS

Server

TWOMASS:Sky node

USNOB:Sky node

:OpenSky Query client

:AstroDAS client

AstroDAS Portal

OpenSkyQuery Portal

null3577751258778583

13317184null58777039

133172803577347858778470

USNOBTWOMASSSDSSAstroDAS

OpenSkyQuery

mapping table created dynamically from annotations

12

3

4

AstroDAS: Astronomy Distributed Annotation SystemQuerying annotations on astronomy catalogues

SELECTs.objid, s.ra, s.dec, s.type,t.objid, t.ra, t.decu.objid, u.ra, u.decFROMSDSS:photoprimary s, TWOMASS:photoprimary t, USNOB:photoprimary uAS:UEdinburgh e, AS:URome rWHERERegion(’CircleJ2000 200 -1 0.02’) ANDs.type=3 ANDe.author=’algorithm1’ ANDr.author=’algorithm2’

DSQL query

URome:AstroDAS

Server

SDSS:Sky node

UEdinburgh:AstroDAS

Server

TWOMASS:Sky node

USNOB:Sky node

:OpenSky Query client

:AstroDAS client

AstroDAS Portal

OpenSkyQuery Portal

null3577751258778583

13317184null58777039

133172803577347858778470

USNOBTWOMASSSDSSAstroDAS

OpenSkyQuery

mapping table created dynamically from annotations

12

3

4

AstroDAS: Astronomy Distributed Annotation SystemCreating a mapping table from stored annotations: inference

SELECTs.objid, s.ra, s.dec, s.type,t.objid, t.ra, t.decu.objid, u.ra, u.decFROMSDSS:photoprimary s, TWOMASS:photoprimary t, USNOB:photoprimary uAS:UEdinburgh e, AS:URome rWHERERegion(’CircleJ2000 200 -1 0.02’) ANDs.type=3 ANDe.author=’algorithm1’ ANDr.author=’algorithm2’

DSQL query

UEdinburgh:<SDSS, 58778470> ↔ <TWOMASS, 35773478><SDSS, 58778470> ↔ <USNOB, 13317280><SDSS, 58777039> ↔ <USNOB, 13317184>

URome:<SDSS, 58778583> ↔ <TWOMASS, 35777512>

Outline

1. Astronomy catalogues and existing OpenSkyQuery system

2. Custom cross-matching algorithms: AstroDAS

3. How AstroDAS compares to other annotation systems

http://www.wormbase.org/db/das/elegans/features?segment=Ihttp:// <server> /das/<source>/<command>?<args>

BioDAS: Biology Distributed Annotation System (Dowell 2001)Example 1: Genome annotation and BioDAS

Dowell, R., Jokerst, R., Day, A., Eddy, S., & Stein, L. (2001). The Distributed Annotation System. BMC Bioinformatics, 2(7).

Ensembl system which includes BioDAS functionalityExample 1: Genome annotation and BioDAS

http://www.ensembl.org

Ensembl system which includes BioDAS functionalityExample 1: Genome annotation and BioDAS

http://www.ensembl.org

Outline

1. Astronomy catalogues and existing OpenSkyQuery system

2. Custom cross-matching algorithms: AstroDAS

3. How AstroDAS compares to other annotation systems

AstroDAS: Sharing Assertionsacross Astronomy Cataloguesthrough Distributed Annotation

Rajendra Bose, Robert G. Mann, Diego Prina-Ricotti

Digital Curation Centre

4 May 2006

International Provenance and Annotation Workshop (IPAW’06)

Annotation of the Malaria Mosquito Anopheles gambiae genome sequenceExample 1: Genome annotation and BioDAS

The Genome Sequence of the Malaria Mosquito Anopheles gambiae, Robert A. Holt, et al., Science 4 October 2002: Vol. 298. no. 5591, pp. 129 - 149 DOI: 10.1126/science.1076181; http://www.sciencemag.org/cgi/data/298/5591/129/DC2/1

Annotation of the Malaria Mosquito Anopheles gambiae genome sequenceExample 1: Genome annotation and BioDAS

The Genome Sequence of the Malaria Mosquito Anopheles gambiae, Robert A. Holt, et al., Science 4 October 2002: Vol. 298. no. 5591, pp. 129 - 149 DOI: 10.1126/science.1076181; http://www.sciencemag.org/cgi/data/298/5591/129/DC2/1

Annotation of the Malaria Mosquito Anopheles gambiae genome sequenceExample 1: Genome annotation and BioDAS

The Genome Sequence of the Malaria Mosquito Anopheles gambiae, Robert A. Holt, et al., Science 4 October 2002: Vol. 298. no. 5591, pp. 129 - 149 DOI: 10.1126/science.1076181; http://www.sciencemag.org/cgi/data/298/5591/129/DC2/1

Lauer, Kim P., Llorente, Isabel, Blair, Eric, Seto, Jason, Krasnov, Vladimir, Purkayastha, Anjan, Ditty, Susan E., Hadfield, Ted L., Buck, Charles, Tibbetts, Clark, Seto, DonaldNatural variation among human adenoviruses: genome sequence and annotation of human adenovirus serotype 1J Gen Virol 2004 85: 2615-2625http://intl-vir.sgmjournals.org/cgi/citmgr?gca=vir;85/9/2615

Example of genome annotation from the biological literatureExample 1: Genome annotation and BioDAS

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Human Brain Project (HBP) image annotation (Gertz 2002, 2003)Example 2: Medical image annotation

http://neuroscience.ucdavis.edu/hbp/viewer.html

Human Brain Project (HBP) image annotation (Gertz 2002, 2003)Example 2: Medical image annotation

Gertz, M., Sattler, K.-U., Gorin, F., Hogarth, M., & Stone, J. (2002). Annotating Scientific Images: A Concept-based Approach. Proceedings of the 14th International Conference on Scientific and Statistical Database Management (SSDBM 2002), Edinburgh, Scotland. IEEE Computer Society.Gertz, M., & Sattler, K. U. (2003). Integrating scientific data through external, concept-based annotations. In Efficiency and Effectiveness of Xml Tools and Techniques and Data Integration over the Web (Vol. 2590, pp. 220-240).

Edinburgh Mouse Atlas Project (EMAP) (Baldock 1999)Example 2: Medical image annotation

Baldock, R. A., Dubreuil, C., Hill, W., & Davidson, D. (1999). The Edinburgh Mouse Atlas: Basic Structure and Informatics. In S. I. Letovsky (Ed.), Bioinformatics: Databases and Systems (pp. 129-140). Kluwer Academic Publishers. (See http://genex.hgu.mrc.ac.uk/)

AstroDAS: Astronomy Distributed Annotation SystemStoring annotations to map database objects

db_object db_object

SDSS_112233 TWOMASS_445566

annote1 annote2 annote_sourceSAME OBJECT

(algorithm1) GROUP1

NOT SAME OBJECT

(algorithm2) GROUP2

NOT SAME OBJECT

(algorithm1) GROUP1

… … … … ……

id

1

2

3

SDSS_112233

SDSS_112233 TWOMASS_778899

TWOMASS_445566