g02 walter koch_micro_services

XIIth Annual International Conference for Professionals in Cultural HeritageLoCloud Micro Services andthe Digitisation WorkflowWalter Koch ([email protected])AIT‐Angewandte InformationstechnikForschungs GmbH, Graz ‐ Austria

LoCloud is funded by the European Commission's ICT Policy Support Programme

2015‐11‐08 Walter Koch (AIT) 2

Content

• LoCloud Project (relevant activities)• Vocabulary Microservice• Microservice used in the OpenUp! Project• Digitisation Workflow (Books)• Digital Curation (Long Term Preservation)

LoCloud Project(relevant activities)



locoud.eu

About:– LoCloud is a Best Practice Network co‐funded under the CIP ICT‐PSP programme of the European Commission.

– LoCloud is supporting small and medium‐sized institutions in making their content and metadata available to Europeana.

– A cloud‐based technology infrastructure will enable the aggregation of local content, and

– a number of micro‐services will help to reduce technical, semantic and skills barriers and to render the content more discoverable and interoperable.


Microservices

Six development teams:AIT, Athena RC, AVINET, IPCHS, UPV/EHU, VUKFC

All the services are implemented on virtual machines in a cloud testlab (using the OpenNebula cloud computing platform)

API documentation is available for each serviceEnd‐user interfaces are provided for three microservicesEach service has been integrated in MORe


LoCloud Microservices

http://support.locloud.eu/

Cloud‐based micro‐services for metadata enrichment and capture:

• Geo‐location API• Vocabulary service• Historic Place Names service• Geo‐coding application• Vocabulary matching service • Background linking service• Wikimedia application


LoGeo API

• Uses NER-Name Entity Recognition• Recognizes and returns one or more place names

candidates accompanied by geographical coordinates.

GPLIPCHS


Vocabulary service• Based on “TemaTres”• Incorporates 29 standard

thesauri • allows users to add and

create vocabularies, and add terms or translations

• add thesauri terms to all items of a metadata package

• Integrated in MORE• Exploited by the LoCloud

vocabulary matching microservice via its API

GPLAIT


Historic Place Names service

• A semi‐automatic historical geo‐information management function and web service.

• Enables local cultural institutions to collaborate in developing the Thesaurus of HPN and enriching ingested metadata.

• Enables export of selected LoCloud HPN Thesaurus data sets.

GPLVUKF


Geo‐coding application• End-user oriented application• allows institutions to add geographical locations to existing content in

a controlled crowd-sourcing environment and then • load the augmented data back into their original production

databases.

GPLAVINET


Vocabulary matching service

• Builds on work in the PATHS project (FP7)• Automatically assigns relevant concepts and terms to item records • Retrieves terms from the Vocabulary serviceUPV/EHU Apache License 2.0


Background linking service

• Based on work in PATHS• Follows the Named Entity

Disambiguation approach• Enriches items with links to

external contextual information in DBpedia and Wikipedia

Apache License 2.0UPV/EHU


Wikimedia application

• A web service that uses a REST interface to communicate with Wikimedia commons

• The main functionalities are harvesting content form Wikimedia, parsing the harvested content, and identifying useful entities that can be mapped to the ESE or EDM metadata schemas.

• The mapped ESE / EDM records are then sent to MORe to be delivered to Europeana.

Wikimedia Commons LicensingAthena RC


The LoCloudAggregation Structure

EnrichmentServices

Europe

ana

Europe

ana

Clou

dTestlab


About: Microservice

Wikipedia: https://en.wikipedia.org/wiki/Microservices (2015‐09‐15)

[...] microservices is a • software architecture style in which • complex applications are composed

of small, independent processes• communicating with each other using language‐agnostic APIs.

• These services are small, highly decoupled and focus on doing a small task,facilitating a modular approach to system‐building.


Evolution of Architecturespwc: Technology Forecast: Rethinking Integration, Issue 1, 2014, page 3

Vocabulary Microservice



http://vocabulary.locloud.eu/

• Vocabulary Microservices • This is the entry page for the LoCloud vocabulary application.

• LoCloud uses a variety of standard vocabularies during the metadata enrichment process.These vocabularies have been established with the TemaTres tool and are available in the LoCloud cloud testlab.

• In addition, it is possible to feed the tool with your own vocabularies, establish new vocabularies or add languages to existing vocabularies.


Vocabulary WebService20+ Functions, API

Parameters

Example


Example: SearchUNESCO‐Thesaurus

http://test113.ait.co.at/tematres/unesco/services.php?task=search&arg=peace


SPARQL‐ExampleDISMARC‐Genres

• Query: "Find all Broader Term for 'Acid Jazz‘“• SPARQL End Point: http://test113.ait.co.at/tematres/vocab/sparql.php

QueryResult

ThesaurusEntry


Service Call („fetchTopTerms“)embedded in a test application

Micro-Service

Vocabulary Web Service


Test Application DevelopmentDesigner View

Development of a Drop Down List

Development of a Drop Down List


The Test Application:Part of Data Entry

Path to the Thesaurus Item

„Candidate Term“ Crowdsourcing activity

Next Slide(Vocabulary Management)

Microservice used in theOpenUp! Project (open‐up.eu)„Common Name Service“



OpenUp! ‐ Natural HistoryAggregator for Europeana

Micro Service:Common Name

Service

Micro Service:Common Name

Service


OpenUp!Transformation Workflow

Invoke Micro Service:

Common Name Service

Next Slide


OpenUp!Transformation Parameters

Micro Service:Common Name Service

Service End Point (Cache)


Result: OpenUp!Enriched Meta Data Set

Latin Name

Common Names

Digitisation Workflow(Books)



Bibliotheca Alexandrinahttp://www.bibalex.org

Points for MicroService Integration


Digital Curation(Long Term Preservation)


Digital CurationState‐of‐the‐Art Examples (2010)

• Archivematica– Linux VM based toolset allows producers and archivists to implement a small scale workflow

• UC3 / CDL Merritt– Micro‐Services specifications – implement building blocks for the Digital Curation Lifecycle


ArchivmaticaMicro Services


California Digital LibraryUC3 Merritt

• California Digital Library UC Curation Center• Independant, Interoperable Micro‐Services• Complex curation function through combination of atomistic services

Knowledge Enhancement

Annotation


Micro‐Services(Merritt vs. Archivematica)

CDL/UC3-MERRITT ARCHIVEMATICA


High‐Level Workflow

• OAIS activities providea sound basis

• Archivematica Use Cases

• Merritt specifications• Micro‐Services are thebuilding blocks


CDLCuration Micro Services

1) „An Emergent Micro‐Services Approach to Digital Curation Infrastructure“, IPRES 2009https://escholarship.org/uc/item/5313h6k9

2) UC Curation Center / California Digital Library, merritt: An Emergent Micro‐services Approach to Digital Curation Infrastructure; Rev. 0.6, 2010‐03‐25

1) Annotation service 2)

The Annotation service provides a mechanism by which the consumers of managed digital content can enrich that content through additional description, association, or related content. The service supports the following methods:

• Annotate

• Submit.


Contact

Walter Koch, [email protected]

http://www.ait.co.at

g02 walter koch_micro_services

Internet