cwic data partner’s guide (opensearch)ceos.org/.../cwic_opensearch_data-partner-guide.pdf ·...
TRANSCRIPT
CEOS CWIC Project
CWIC Data Partner’s Guide (OpenSearch) ApprovalDate:2017-05-09
PublicationDate:2017-05-10
ReferencenumberofthisDocument:CWIC-DOC-14-001r010
Documentversion:V1.0
Category:CWICTechnicalDocument
Editors: Eugene G. Yu, Lingjun Kang, Archie Warnock, Li Lin
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page I
CWICImplementationTeam
EugeneG.Yu,GeorgeMasonUniversity([email protected])ArchieWarnock,A/WWWEnterprises([email protected])LiLin,GeorgeMasonUniversity([email protected])
Approvals ApprovedBy Signature Date
Yonsook K. Enloe
DocumentControl
Name CWICDataPartner’sGuide(OpenSearch)Doc. Ref. No. CWIC-DOC-14-001r010Document status Under reviewing Date of release 2017-04-26
RevisionHistory
Date Version Brief Description Author
2014-04-26 0.9 CWIC Data Partner’s Guide (OpenSearch) drafting
Lingjun Kang Archie Warnock
2017-04-26 1.0 Revised release Eugene G. Yu, Archie
Warnock, Lingjun Kang, Li Lin
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 2
TableofContentsCWICImplementationTeam........................................................................................................I
DocumentControl........................................................................................................................I
RevisionHistory............................................................................................................................I
ExecutiveSummary.........................................................................................................................4
RecommendationsforCWICDataPartners................................................................................4
1.BeforeYouBegin.........................................................................................................................5
1.1CWICBackground..................................................................................................................5
1.2CWICConceptandDesign.....................................................................................................5
1.3CWICArchitecture.................................................................................................................7
1.4CWICTermsandDefinitions..................................................................................................8
1.5CWICSystems........................................................................................................................9
1.6.ContactInformation.............................................................................................................9
2.CWICOpenSearchQueryInterface...........................................................................................10
2.1Introduction.........................................................................................................................10
2.2ObtainingOpenSearchDescriptionDocument(OSDD).......................................................10
2.3Searchrequest.....................................................................................................................11
3.CWICOpenSearchresponse......................................................................................................12
4.PartnerGuidelines.....................................................................................................................13
4.1Metadata&SemanticMapping..........................................................................................13
4.1.1 HTTPaccess...............................................................................................................13
4.1.2 Spatialsearch............................................................................................................13
4.1.3 Temporalsearch.......................................................................................................13
4.1.4 RequestfordatasetbyID.........................................................................................14
4.1.5 RequestforgranulebyID.........................................................................................14
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 3
4.1.6 UniquegranuleIDs...................................................................................................14
4.1.7 Recordcounts...........................................................................................................14
4.1.8 Pagination.................................................................................................................14
4.1.9 Othersearchparameters..........................................................................................14
4.1.10 Searchstatus&Errorresponses..........................................................................15
4.2Interaction&ServicesModel..............................................................................................15
4.2.1 UniquegranuleID.....................................................................................................15
4.2.2 Contactinformation..................................................................................................15
4.2.3 BrowseURL...............................................................................................................15
4.2.4 OrderURL.................................................................................................................15
4.2.5 Paginationsupport....................................................................................................16
4.2.6 Spatialsearch............................................................................................................16
4.2.7 Temporalsearch.......................................................................................................16
4.3ErrorHandling.....................................................................................................................16
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 4
ExecutiveSummary
RecommendationsforCWICDataPartnersWhileCWICcanserveasanOpenSearchproxytosearchalmostanyInternet-accessibleinventorysystemforDataPartners,thereareasmallnumberofrecommendationsforDataPartnersthatwillmakethejobvastlyeasier.
1. Registereachdistinct,searchabledatasetintheIDN2. ProvideasearchinterfaceaccessibleviaasimpleURL(i.e.HTTPGET),ideallyincluding
parametersforstartingrecordnumberandnumberofrecordsdesiredintheresponse3. Supportsearchingonspatialboundingbox4. Supportsearchingontemporalextent,atleastobservationstartandenddates5. Providesearchresponsesinwell-structuredtext(XML,JSON,etc.)returningmatching
datagranules6. Identifyeachreturneddatagranulebyanidentifierthatisuniquewithintheinventory
system7. Provideacapabilityforusingthegranuleidentifiertoretrievemetadataaboutthe
granule8. ReturnURLsforbrowsedataanddirectaccesstogranule-leveldata(ortoadata
orderingsystem)inthesearchresponse
Ingeneral,thesearecommonandwidelyimplementedcapabilitiesinalmostanygranulesearchsystemandshouldnotrepresentanimpedimenttojoiningCWICasaDataPartner.Ifanyofthesecapabilitiesarenotimplemented,itisstillpossibletobecomeaCWICDataPartner–contactanyoftheCWICteamfordetails.
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 5
1.BeforeYouBegin
1.1CWICBackgroundForscientistswhoconductmulti-disciplinaryresearch,theremaybeaneedtosearchmultiplecatalogsinordertofindthedatatheyneed.Suchworkcanbeverytime-consumingandtedious,especiallywhendifferentcatalogsmayusedifferentmetadatamodelsandcataloginterfaceprotocols.Itwouldbedesirable,therefore,forthosecatalogstobeintegratedintoacatalogfederationwhichwillpresentawell-knownanddocumentedmetadatamodelandinterfaceprotocoltousersandhidethecomplexityanddiversityoftheaffiliatedcatalogsbehindtheinterface.Withsuchafederation,usersonlyneedtoworkwiththefederatedcatalogthroughthepublicinterfaceorAPItofindthedatatheyneedinsteadofworkingwithvariouscatalogsindividually.
CommitteeonEarthObservationSatellite(CEOS)addressescoordinationofthesatelliteEarthObservation(EO)programsoftheworld'sgovernmentagencies,alongwithagenciesthatreceiveandprocessdataacquiredremotelyfromspace.WorkingGrouponInformationSystemsandServices(WGISS)isasubgroupofCEOS,whichaimstopromotecollaborationinthedevelopmentofsystemsandservicesthatmanageandsupplyEOdatatousersworld-wide.TorealizeafederatedcataloguefordatadiscoveryfrommultipleEOdatacenters,theCEOSWGISSIntegratedCatalog(CWIC)systemwasimplemented.CWICwasexpectedtoprovideinventorysearchtoWGISSagencycatalogsystemsforEOdata.
1.2CWICConceptandDesignCWICusesamediator-wrapperarchitecturethathasbeenwidelyadoptedtorealizetheintegratedaccesstoheterogeneous,autonomousdatasources.AsdepictedinFig.1,thedatasourcearchivesdataanddisseminatesitthroughtheInternet.Thewrapperontopofthedatasourceprovidesauniversalqueryinterfacebyencapsulatingheterogeneousdatamodels,queryprotocols,andaccessmethods.Themediatorinteractswiththewrapperandprovidestheuserwithanintegratedaccessthroughtheglobalinformationschema.
Wrappersofferqueryinterfaceshidingtheunderlyingdatamodel,accesspath,andinterfacetechnologyofthepartnercatalogsystems.Wrappersareaccessedbyamediator,whichoffersusersafront-endintegratedaccessthroughitsglobalschema.Theuserposesqueriesagainsttheglobalschemaofthemediator;themediatorthendistributesthequerytotheindividualsystemsusingtheappropriatewrappers.Thewrapperstransformthequeriessotheyareunderstandableandexecutablebythepartnercatalogsystemstheywrap,collecttheresults,transformthemappropriatelyandreturnthemtothemediator.Finally,themediatorintegratestheresultsasauserresponse.
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 6
Fig. 1 The Mediator-Wrapper Architecture
Basedonthemediator-wrapperarchitecture,currentversionofCWIChasbeendevelopedandoperationalwithfollowingdatapartnercatalogsystems:theCommonMetadataRepository(CMR)ofNASA,theNationalCentersforEnvironmentalInformation(NCEI)ofNOAA,theGroupforHighResolutionSeaSurfaceTemperature(GHRSST)ofNOAA,theUSGSLandsatSurfaceImaging(LSI)Explorer,theNationalInstituteforSpaceResearch(INPE)CatalogSystemofBrazil,theEuropeanOrganisationfortheExploitationofMeteorologicalSatellites(EUMETSAT),theCanadaCentreforMappingandEarthObservation(CCMEO),theMeteorologicalandOceanographicSatelliteDataArchivalCentre(MOSDAC)oftheIndianSpaceResearchOrganisation(ISRO),andtheNationalRemoteSensingCenter(NRSC)ofISRO.
Differentqueryinterfaceswereusedtoaccessthedatapartnercatalogsystems:
Data partner OpenSearch OGC CSW Native query interfaceNASACMR Yes No YesNOAAGHRSST Yes Yes NoNOAANCEI Yes Yes NoUSGSLSI Yes No NoBrazilINPE No No YesCanadaCCMEO Yes Yes NoISROMOSDAC No Yes YesISRONRSC Yes No YesEUMETSAT Yes No No
Inordertoimplementaone-stopfederatedcatalogsystem,wrappershavebeendevelopedtoimplementCWICOpenSearchforindividualmembercatalogsthatdonotcurrentlyofferthatcapability.
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 7
Fig. 2 The System Architecture of CWIC
Fig.2illustratesthesystemarchitectureofCWIC.Wrapperswereimplementedfordifferentdatapartnercatalogsystems(i.e.,NASACMR,NOAAGHRSST,NOAANCEI,USGSLSI,INPE,CCMEO,ISROMOSDAC,andISRONRSC).Thewrapperisresponsiblefortranslatinganddispatchingrequesttodifferentdatainventories.Themediatorisinchargeofdispatchingthequeryrequesttothedatapartnerinventorysystemandreturnstheresponsetodatauser.
1.3CWICArchitectureAtitscore,CWICpresentstoEndUsersandClientsanOpenSearchserver.ToDataPartners,itappearstobeaweb-basedclient.Itconnectsthetwo(EndUsersandDataPartners)throughtheMediatoronthefrontendservingastheOpenSearchservertoendusersandOpenSearchclienttoConnectors.TheConnectorsarecustom-writtenproxiesforthedatagranuleinventorysearchsystemsattheindividualDataPartners,acceptingOpenSearchsearchrequestsfromtheMediator,translatingthemintovalidsearchrequestsforthetargetdataset,thenparsingtheresultsfromtheinventorysearchsystemandtranslatingthoseintoOpenSearchsearchresponseswhicharepassedbacktotheMediator.
Inthisway,outsideclientsand,forthemostpart,theMediatoritselfneedtohavenospecificknowledgeoftheparticularpartnerdatasystemsandcommunicateonlyviaOpenSearch.EachDataPartnerwillgenerallybeaccessedbyadedicatedConnectorcalledbytheMediator.TheConnectorhandlesallofthedetailsuniquetoindividualdatapartnerinventorysystemandallof
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 8
thecommunicationswiththepartner’sinventorysystemismanagedexclusivelybytheconnector.
1.4CWICTermsandDefinitionsForthepurposesofthisdocument,thefollowingtermsanddefinitionsapply:
(1) Client
Asoftwarecomponentthatcaninvokeanoperationfromaserver
(2) DataClearinghouse
Thecollectionofinstitutionsprovidingdigitaldata,whichcanbesearchedthroughasingleinterfaceusingacommonmetadatastandard
(3) Identifier
Acharacterstringthatmaybecomposedofnumbersandcharactersthatisexchangedbetweentheclientandtheserverwithrespecttoaspecificidentityofaresource
(4) IDNDatasetID
UniquedatasetidentifierinIDN,returnedfromtheIDNinresponsetotheOSDDrequest.ThisidentifierisassignedbytheIDNCMRdatabase.
(5) NativeID
DatasetidentifierusedbyCWICtoretrievegranulemetadatathroughdataproviderAPI.Thisidentifierisassignedbythedataprovider.
(6) CatalogID
Identifiersofdataprovidercatalogsorconnectionsservinggranulemetadata.
(7) Operation
Thespecificationofatransformationorquerythatanobjectmaybecalledtoexecute
(8) Profile
Asetofoneormorebasestandardsand-whereapplicable-theidentificationofchosenclauses,classes,subsets,optionsandparametersofthosebasestandardsthatarenecessaryforaccomplishingaparticularfunction
(9) Request
TheinvocationofanoperationbyaCWICclient
(10) Response
Theresultofanoperation,returnedfromCWICservertoCWICclient
(11) Collection
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 9
Agroupingofgranulesthatallcomefromthesamesource,suchasamodelinggrouporinstitution.Collectionshaveinformationthatiscommonacrossallthegranulesthey"own"andatemplatefordescribingadditionalattributesnotalreadypartofthemetadatamodel.
(12) Dataset
Samemeaningascollection,see(8)
(13) Granule
Thesmallestaggregationofdatathatcanbeindependentlymanaged(described,inventoried,andretrieved).Granuleshavetheirownmetadatamodelandsupportvaluesassociatedwiththeadditionalattributesdefinedbytheowningcollection.
(14) IDN
TheCEOSInternationalDirectoryNetwork(IDN)isaGatewaytotheworldofEarthSciencedata.
1.5CWICSystemsTherearetwooperationalCWICsystemstowhichend-usershaveaccess.
§ CWICPROD–thisisCWICproductioninstanceandisavailabletoallusers.
Location:http://cwic.wgiss.ceos.org/
§ CWIC TEST – this is CWIC testing instance used by data partners and CWIC clients toperformtestingbeforechangesaremadetotheCWICproductioninstance.
Location:http://cwictest.wgiss.ceos.org/
TheproductioninstancewillprovideaccesstoonlydatasetswhichhavebeenregisteredwiththeIDN.Thetestinginstancemayprovideaccesstoadditionaldatasets(e.g.,newdatasetsundergoingtestingandnotyetregisteredintheIDN),andcapabilitieswhichhavenotyetbeentestedsufficientlytomovetotheproductionsystem.
1.6.ContactInformationAllthedocumentsandinformationaboutCWICareavailableatWGISSCWICpageat
http://wgiss.ceos.org/cwic
AnyquestionsregardingtoCWIC,pleasesendtheemailto
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 10
2.CWICOpenSearchQueryInterface
2.1IntroductionOpenSearchisalight-weightsearchspecification1usedbyCWICtosearchandreturnmetadatarelatedtogranule-levelinventorydata.CWICOpenSearchisnotdesigned,norisitusedforreturningobservationaldatafromtheinventorysystems,althoughthemetadatareturnedshouldincludelinksdirectlytodatagranulesortoadataorderingsystem.CWICOpenSearchisintendedtotaketheenduserasclosetoactualdataaspossiblewithintheconstraintsofthedatapartnerinventorysystemsandthelimitsoftheOpenSearchprotocolitself.
TheCWICConnectorshavethetaskofreturningvalidresponsesinAtom2formatasperCWICOpenSearchrequeststotheMediator.Thesearegeneratedon-the-flybysubmittingsearchrequeststotheDataPartner’sinventorysystemfortherequesteddataset,retrievingtheresultsandtranslatingthemintosyntacticallyvalidandsemanticallymeaningfulCWICOpenSearchresponses.TheConnectorimplementerwillworkwiththeDataPartner’ssupportteamtodefinethemappingsbetweenquantitiescontainedintheinventorysystemresponseandtheassociatedelementsintheCWICOpenSearchresponses.
2.2ObtainingOpenSearchDescriptionDocument(OSDD)OpenSearchDescriptionDocuments(OSDDs)providenecessaryinformationforclientstoprogrammaticallyconstructvalidsearchrequests.Specifically,clientsareabletoacquirebothcardinalityanddomainofrequestparametersbasedonquerytemplate.Datasetvalids(i.e.spatialfootprintandtemporalextent)arealsoprovidedthroughtheOSDDinbothmachineparsableandhumanreadableformats.Withdatasetvalids,clientsareabletoconstructrequestsyieldingmoreaccurateresults.
CWICprovidesbothgenericanddatasetspecificOSDDs.ClientscanfetchagenericOSDDthroughtheCWICOSDDendpoint.TheOSDDrequestmustalsoincludeaclientidentifierstring,asrecommendedbytheCWICOpenSearchBestPractices.Clientscanalsoretrieveadataset-specificOSDDthroughtheOSDDendpointbysendingbothclientIDanddatasetidentifier(i.e.DIFEntryID).Inadataset-specificOSDD,domainisalsoprovidedforsomeparameters(i.e.timeStartandtimeEnd)inadditiontotherequestparametersyntax.
GenericOSDDrequestURLexample:
http://cwic.wgiss.ceos.org/opensearch/datasets/osdd.xml?clientId=foo
1OpenSearchspecificationverison1.1draft5(http://www.opensearch.org/Specifications/OpenSearch/1.1)
2Atomsyndicationformat(http://tools.ietf.org/search/rfc4287)
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 11
DatasetspecificOSDDrequestURLexample:
http://cwic.wgiss.ceos.org/opensearch/datasets/C1235542031-USGS_LTA/osdd.xml?clientId=foo
2.3SearchrequestTheCWICOpenSearchrequestoperationcanbeusedtoconductgranule-levelsearchesonthetargetsystemforarangeofparameters.Italsodefinesthestartpageoftheresultsetandmaximumnumberofresultsperpagetobereturned.CWICOpenSearchcurrentlysupportsfollowingrequestparameters:uniqueidentifier(dataset/granuleID),spatial(geo:box)andtemporalsearch(time:start/time:end)
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 12
3.CWICOpenSearchresponseAnOpenSearchresponseisanATOMfeedwithzeroormoreATOMentries.Eachentryrepresentsthemetadataofasinglegranulepertainingtothequerysubmittedwithinasetofresultsalsodefinedbythequery.
AnATOMresponseisasingleATOMfeedelementcontainingthefollowing,1. Informationaboutthesearchconductedintermsoftitle,authorandID.2. Informationaboutthenatureoftheresultsetintermsoftotalnumberofresults,numberof
resultsreturnedandhowmanytheclientaskedfor.3. Navigationinformationfortraversingthatresultsetincludinglinkstotheprevious,next,
firstandlastresultsintheset.4. Zeroormoreentriespertainingtogranulemetadatamatchingtheclientquery
AnentryisanATOMentryelementcontainingthefollowing,1. Basicinformationaboutreturnedgranuleintermsoftitle,author,summaryandID.2. Informationaboutthespatialfootprintofreturnedgranule.3. Informationaboutthetemporalextentofreturnedgranule.4. CollectionofATOMlinkelementscontainingendpointsofgranulebrowse,orderand
metadataforthereturnedgranule.
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 13
4.PartnerGuidelines
4.1Metadata&SemanticMapping
4.1.1 HTTPaccessAlthoughCWICwillattempttouseanymechanismavailableforconnectingtoDataPartners’datamanagementsystemsinordertoaccesstheavailableinventorysearch,thereareafewspecificswhichmaketheprocesssimplerandmorerobust.
TheuseofHTTPforaccessingtheinventorysearchengineisstronglypreferred.Thisiswidelyusedalready,aswebbrowsersarenearlyuniversalandprovideaneffectiveuserinterfaceforbothhumanandautomatedaccess.
Similarlyforresults,CWICwillattempttoextracttherelevantresultsfromanyresponsesthepartnerdatasystemreturns.However,structuredtextofsomesort–XML,forexample–isstronglypreferred.Theabilitytoeasilyanddefinitivelyparseresultsmakestheprocessofmappingthemetadatareturnedinthesearchresponsesimplerandlesserror-prone.Otherstructuredformatslikecomma-delimitedtablesorJSONareacceptable.
4.1.2 SpatialsearchAllCWICdatapartnersareexpectedtosupportsomelevelofspatialsearchsincealloftheinventorydataareanticipatedtohaveaspatialcomponent.Foreachgranule,asimpleboundingbox,withtheboundingcoordinatesindividuallyidentified,istheminimumrequired,althoughmorecomplexspatialfootprintgeometriesarepossibleinthefuture.Thenumberofspatialfootprintsindatapartner’sresponseisnotlimitedtoone.
ItisdesirabletohavetheAPIalsosupportadynamiccalltoreturnthelimitsofthespatialsearch,althoughnotnecessary.ThepresenceofsuchaservicecanhelpCWICavoidinvalidorinappropriatesearchrequests,suchasthoseoutsidethespatialboundariesforspecificdatacollections.
4.1.3 TemporalsearchSimilartospatialsearchdescribedintheprevioussection,allCWICdatapartnersareexpectedtosupportsomeleveloftemporalsearchsincealloftheinventorydataisanticipatedtohaveatemporalcomponent.Simpletemporalextent,withthestartandendtimesindividuallyidentifiedistheminimumrequired,althoughmorecomplextemporalrelationsareanticipatedinthefuture.ItisbesttosupportsomeminimalsubsetoftheISO8601timespecificationforsyntax–YYYY-MM-DD,atleast.
ItisdesirabletohavetheAPIalsosupportadynamiccalltoreturnthelimitsofthetemporalextentsearch,althoughnotnecessary.ThepresenceofsuchaservicecanhelpCWICavoid
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 14
invalidorinappropriatesearchrequests,suchasthoseoutsidetheexistingtemporalextentforspecificdatacollections.
4.1.4 RequestfordatasetbyIDDatasetID(cwic:datasetId)servesasamandatoryparameterintheCWICOpenSearchqueryinterface.CWICwillqueryagainstdatapartner’ssearchsystem,usingthedatasetID,toretrievethematchinggranulelevelmetadata.Fromthispoint,alldatapartnersareexpectedtosupportthequerybasedondatasetidentifier.Inparticular,agloballyuniquedatasetidentifier(i.e.DIFEntryID)isrecommendedtobeimplementedasqueryparameterondatapartnerside.However,otheruniquedatasetidentifierswithindatapartnersystemisalsoacceptable.
4.1.5 RequestforgranulebyIDCWICOpenSearchsupportssearchingforasinglegranulebygranuleID.CWICwillqueryagainstdatapartner’ssearchsystemwiththegranuleIDinordertoretrievethematchinggranulelevelmetadata.Fromthispoint,alldatapartnersarerecommendedtosupportthequerybasedongranuleidentifier.
4.1.6 UniquegranuleIDsEachdatagranulereturnedinasearchresponseshouldhaveanidentifierassociatedwithitwhichisuniquewithinthedataset.Itisimportantthatthesearchresponseincludetheuniqueidentifierforeachgranulesothatthefulldataonindividualgranulesmayberetrievedwithoutre-executinga(potentiallytime-consuming)search.
4.1.7 RecordcountsAspartofthesearchresponsefromtheinventorysystem,itishighlydesirabletohavethetotalcountofmatchinggranulesreturned,evenifthemetadataforthegranulesisnotcontainedinthesearchresponse.Thisparameter,coupledwiththeabilitytospecifythestartingrecordnumberandnumberofdesiredrecordsfromtheinventorysystem,willallowclientstoimplementresultspagingandreducingtheloadonboththeCWICsystemandonthedatapartners.
4.1.8 PaginationPaginationissupported(i.e.searchbyspecifyingstartpageandcountsperpage)intheCWICOpenSearchqueryinterface.Thesearchbasedonpaginationparameter(e.g.startpageanditemperpage)orcursorparameter(e.g.startindexanditemperpage)isexpectedtobeimplementedbytheDataPartner.
4.1.9 OthersearchparametersTheDataPartnerinventorysystemsmaysupportsearchonadditionalparameterslikeLANDSATpath/row.ThesearenotsupportedthroughCWICOpenSearchatthecurrenttimebutposenoparticularissuesfortheCWICconnectors.
CWICisalsoconsideringsupportforresultssorting–bystarttime,byduration,bygranuleID,etc.–anditwouldbeuseful,althoughnotmandatory,tohaveasortoptionavailablefromthePartnerinventorysystem.
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 15
4.1.10 Searchstatus&ErrorresponsesUsefulstatusanderrormessageshelptheConnectormanageclientsessionseffectively.Anylimitationsonsubmittedsearchrequeststotheinventorysystemsshouldbenotedintheresponse(e.g.,“toomanyrecordsrequested”,“searchtimedout”)sothatpredictableerror-handlingcanbemanagedbytheConnector.
4.2Interaction&ServicesModel
4.2.1 UniquegranuleIDAsdescribedabove,eachdatagranuleshouldhaveauniqueidentifierwhicha)ispassedbacktotheclientaspartofthesearchresponseandb)canbeusedasakeywithwhichtoretrievethatspecificgranule.TheCWICcomponentswillmanagethetaskofassociatingtheidentifierwiththecorrectdatasetanddatacenter.
4.2.2 ContactinformationTheCWICOpenSearchresponsesincludecontactinformationforeachgranule.Theseareusuallythesameforalldatagranules,andfrequentlythesameforalldatasetswithinasingledatacenter.
Thereisnoneedforthisinformationtobereturnedwitheachsearchresponseoreachdatagranule,althoughitmightbe.TheCWICConnectorcancachethisinformationintheCWICruntimeenvironment,butthismechanismisnotrecommendedbecauseitremovescontroloverthecontactinformationfromtheDataPartnerandchangesmayrequiremodificationstotheConnectorsourcecode.Thiscanleadtodelaysinincludingthecorrectcontactinformationwhenchangesareimplementedbythedatacenter.
4.2.3 BrowseURLIfbrowseimagesofthedatagranuleareavailable,avalidURLtodisplaythebrowseimageshouldbeincludedinthesearchresponseforeachgranulesothattheclientcandisplayitasalink.WhileitispossiblefortheCWICconnectortobuildtheURLbasedonsomepre-defined,fixedpattern,thismechanismisnotrecommendedbecauseitremovescontrolovertheformoftheURLfromtheDataPartnerandchangesmayrequiremodificationstotheConnectorsourcecode.ThiscanleadtodelaysinthedeploymentofthecorrectURLwhenchangesareimplementedbythedatacenter.
4.2.4 OrderURLTheCWICteamrecommendsthat,whenthegranuledatacanbedownloadedfromthedatacenterdirectly,avalidURLtoretrievethedatabeincludedinthesearchresponseforthegranule.
Alternatively,thesearchresponsemaycontainaURLdirectingtheusertoawebsitefororderingthedataifthisistheonlyoptionpermittedbythedatacenter.Thisisoftennecessaryevenforfreelyavailabledataif,forexample,datacenterpoliciesrequireuserregistrationbeforedownloadsaremadeavailable.Insuchcases,theCWICteamstronglyrecommendsthatthegranuleIDrequestedbecachedattheorderingsystemsothatwheneverthedatacenter
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 16
requirementsfordownloadingdataaremet,theuserwillbeabletoretrievethedatawithoutre-enteringthegranuleID.
4.2.5 PaginationsupportAsstatedabove,thereisnostrictrequirementintermsofpaginationmechanismorpaginationparametersimposedbyCWIC.Variouspaginationmechanismandheterogeneousparametersareexpectedacrossdatapartners.Toprovideuniversalpagination,theCWICconnectortranslatesstartPageandcount(stipulatedbyCWICOpenSearchBestPractices)tonativepaginationparametersforeachDataPartner.
4.2.6 SpatialsearchAsstatedabove,aDataPartnerisexpectedtosupportspatialsearchandincludeproperspatialcomponentineachgranuleresponse.Nativespatialcomponents(oneormany)fromdatapartnerareconvertedsyntacticallyandsemanticallytoproperspatialcomponents(e.g.georss:boxandgeorss:polygon)whicharestipulatedbyCWICOpenSearchBestPractices.Besidesthat,aminimumboundingrectangle(MBR)isalsocalculatedbasedonnativespatialcomponentsforeachgranuleifitisnotreturnedexplicitlyintheresponseandincludedinCWICOpenSearchresponse.
WiththeknowledgeofDataPartnerspecificlimitationsonspatialsearch,CWICwillimposevalidationonthespatialrequestparameterandreturnboththeHTTPstatuscodeandanerrormessageifnecessary.
4.2.7 TemporalsearchAsstatedabove,aDataPartnerisexpectedtosupporttemporalsearchandincludestandard(ISO8601compliant)temporalelementsineachgranuleresponse.Temporalelementswithnativeformatfromdatapartnerareconvertedsyntacticallyandsemanticallytosingletemporalelement(i.e.dc:date)whicharestipulatedbyCWICOpenSearchBestPractices.
WiththeknowledgeofDataPartnerspecificlimitationsontemporalsearch,CWICwillimposevalidationontemporalrequestparameterandreturnboththeHTTPstatuscodeandanerrormessageifnecessary.
4.3ErrorHandlingErrorhandlinginCWICOpenSearchisbasedonstandardHTTPstatuscodes.TheCWICdevelopmentteamisinvestigatingwaystoenhancetheerrorlogginganddocumenting.EfforthasalsobeendedicatedtoprovidesensibleerrormessagesinadditiontothegenericHTTPstatuscodetotheenduserorclient.Inordertosupportthiseventuality,itwouldbeusefulfortheinventorysearchsystemtoattempttoreturnsensibleandrelevantHTTPstatuscodes(whereapplicable)ifsomethinggoeswrongwiththesearchor,perhapsevenbetter,asmall,descriptiveresponsedocument(inXMLorJSONorwhateverthedefaultformatmightbe)providingerrorcodesanderrortext.Inthisway,theCWICconnectorscandistinguishthetypeoferrorarisingattheinventorysystemfromthosearisingelsewhereandtakeappropriate
CWIC Data Partner’s Guide (OpenSearch) Version: 1.0
Page 17
action.TherearenospecificrecommendationsatthistimebutthisshouldbepartofongoingdiscussionsbetweentheConnectordevelopersandtheDataPartner’ssupportstaff.