ics-forth september 21, 2010 crm digital a digital provenance ontology center for cultural...

ICS-FORTH September 21, 2010

CRM Digital

A Digital Provenance Ontology

Center for Cultural Informatics, Institute of Computer Science

Foundation for Research and Technology - Hellas

Martin Doerr

Berlin, GermanySeptember 25, 2011

TPDL 2011


CRM Digital

Requirements Competitors CRM and Provenance Data example About Provenance-based reasoning Conclusions

Outline


CRM Digital

Scientific data are empirical or synthetic.

Scientific data cannot be understood without knowledge about the meaning of the data and the ways and circumstances of their creation

We use Metadata to assess meaning (view, experimental setup, instrument settings), relevance (depicted things, their status, their conditions), quality (calibration, tolerances, errors, “artifacts”), possibilities of Improvement and Reprocessing.

From generation to use, permanent storage, reuse (life-cycle)

Digital Provenance Metadata Requirements


CRM Digital

Acquisition: Reliable registration of the process and context conditions

The experimental setup and environment (geometry, light sources, tools, obstacles, sources of noise/reflections etc.)

Capture device type, identity (individual behavior!) Hierarchical model: Inherit metadata common to series of “shots” The identity of the measured or depicted object

import identifiers, metadata identity of location – GPS data import?

Requirements


CRM Digital

Processing: Reliable registration of parameters Workflow logs, reliable identification of outputs with inputs

input files (URIs!) output files (URIs!), formats, warning and error reports. S/W identifiers and parameters, manual adjustments! process types for reasoning

Reliable linking with captured data

Use and Reuse: parts, wholes and annotation: Composition of final products, information packages (SIP, DIP, AIP)

composition of aggregates, selection of versions or parts for permanent storage, reuse or transfer between labs, to and from Digital Libraries.

Migration to other formats (compatibility and obsoletion) Authenticity, rights

Requirements


CRM Digital

There is no provenance data standard format. Too many application-oriented, partial, overspecialized solutions. Several stand-alone models, overgeneralizations No integrated ontology of activity context

Competitors: “Open Provenance Model”,”Provenance Vocabulary”,”Provenir”,”Premise”

no notion of acquisition (measurement, observation), place

Confuse agentive role with substance of actors, machines, S/W, context

No notion of temporal indeterminacy

W3C Provenance WG

precondition: No use of a larger reference ontology => a dogmatic reinvention of the wheel….”antimodularity of ontologies?”

Competitors


CRM Digital Competitors “Provenance Vocabulary”


CRM Digital Competitors: “Provenir”


CRM Digital

The Idea: First conceived by Stephen Stead for CHI, San Francisco 2007 Scientific data and metadata are historical records! Scientific observation and machine-supported processing is initiated,

on behalf of and controlled by human activity in physical space-time, not in cyber-space!

Things, data, people, times and places are causally related by events. Other relations are either deductions from events or found by observation events.

CRM Digital: Specialize the CIDOC CRM (ISO21127)! Will allow for rich integrated reasoning (Christ – Ascension – Ivory panel)

Innovations: The Digital Measurement Event transfers from physical to digital world.

Machines “act” due to human initiative and responsibility. Humans use machines. No non-human actors!

CRM and Provenance


CRM Digital

SS

tt

operatoperatoror

11stst ComputerComputer

coherence volume coherence volume of mesh-creationof mesh-creation

coherence coherence volume of volume of acquisitionacquisition

MuseumMuseum

museumuseummobjectobject

It-LabIt-Lab

22ndnd ComputerComputer

coherence coherence volume of volume of renderingrendering

scan-scan-datadata

3D 3D modelmodel

3D Model Creation as Meetings

scannscannerer

mesh-mesh-datadata


CRM Digital

CRM Digital 2.5

http://www.ics.forth.gr/isl/rdfs/3D-COFORM_CRMdig_v2.5.rdfs


CRM Digital

D2 Digitization Process

E11 Modification

E7 Activity

E65 Creation

E16 Measurement

D11 Digital Measurement Event

D7 Digital Machine Event

D10 Software Execution

D3 Formal Derivation

D12 Data Transfer Event

D27 Calibration Process

CRM Digital 2.5 : Digital Events http://www.ics.forth.gr/isl/rdfs/3D-COFORM_CRMdig_v2.5.rdfs


CRM Digital

D1 Digital Object

E54 Dimension

D9 Data Object

E73 Information Object

E70 Thing

D8 Digital Device

E22 Man-Made Object

E84 Information Carrier

D13 Digital Information Carrier

D35 AreaD14 Software

CRM Digital 2.5: Digital Things


CRM Digital

D1 Digital Object

E24 Physical Man-Made Thing

P31 has modified(was modified by)

P94 has created (was created by)

E54 Dimension

P40 observed dimension(was observed in)

E18 Physical Thing

E1 CRM Entity

E16 Measurement E65 Creation

P39 measured(was measured by)

E11 Modification

E28 Conceptual Object

L1 digitized (was digitized by) L20 has created

(was created by)

D2 Digitization Process

D11 Digital Measurement Event

D9 Data Object


L19 stores(is stored on)

Digitization = feature transfer from physical to digital

CRM Digital 2.5: Digitization


CRM Digital


D8 Digital Device

D1 Digital ObjectL10 had input(was input of)

D1 Digital Object L11 had output(was output of)

L12 happened on device(was device for)

D10 Software Execution

D3 Formal Derivation

L2 used as source (was source for)

D1 Digital Object

L13 used parameters (parameters for)

D1 Digital Object


L18 has modified(was modified by)

D1 Digital Object

L21 used as derivation source (was derivation source for)

D1 Digital Object

L22 created derivative (was derivative created by)

CRM Digital 2.5: Software Execution

Formal Derivation = feature transfer from digital to digital

E55 Type

P2 has type (is type of)


CRM Digital


D8 Digital Device

D1 Digital ObjectL11 had output(was output of)

L12 happened on device(was device for)

D12 Data Transfer Event

L10 had input(was input of)

D1 Digital Object

D1 Digital ObjectL14 transferred

(was transferred by)

D8 Digital Device D8 Digital Device

L16 has receiver(was receiver for)

L15 has sender(was sender for)


L18 has modified(was modified by)

CRM Digital 2.5: Data Transfer Event

Unreliable transfer


CRM DigitalApplications

European IP CASPAR European Space Agency: satellite data IRCAM: Digital media performances FORTH: Art Object Digitization FORTH/ Metaware: Integrating Digital Rights with Provenance model.

European IP 3D-COFORM 3D model acquisition by camera, manual or by camera array. Up to

20.000 files per object. 3D model acquisition by laser scan. Mesh processing, rendering Synthetic models and scene compositions. Provenance-based reasoning Scalable repositories, representative amounts of data.


CRM Digital3D Acquisition

Example: 3D Reconstruction from Photographs – The Gipsmuseum Campaign Sven Havemann, CGV, TU Graz June 30, 2009

worst case for metadata capture: a complex manual process


CRM Digital

Object Acquisition Event OAE1

Sequence EventSE1

Capturing EventCapE1

Data Acquisition Event DAE1

Digital Documentation Event DDE2

Calibration EventCE2

Digital Documentation Event DDE1

Calibration EventCE3

Calibration Event CE1

D2 Digitization Process instantiation example

has part

has part

has part

has parthas part

has part

has part

Sequence EventSE1

Object Acquisition Event OAE1

Capturing EventCapE1Capturing Event

CapE1Capturing EventCapE1Capturing Event

CapE1

Sequence EventSE1

Acquisition Workflow Hierarchy


CRM DigitalModelling the Acquisition Process (AP)

Register: Who, when, where. equipment identifiers, equipment models, firmware Setup geometry and conditions.

Assumptions: worst case, a completely manual process! Set of objects captured under common conditions. Each object captured by a sequence of “shots” Metadata are stored by “historical order” (like workflow logs)

step-by-step as executed, not as planned! concatenated by referring to identifiers of previously existing or created entities

and initialized events. = robust against exceptions in the planned workflow

Avoid redundancy of information Hold common information as high as possible in a hierarchy of nested activities


CRM Digital3D Acquisition

Workflow 3D scanning – NextEngine 3D model creation – MeshlabRapid prototyping Testing glue, stabilizers, coloursPrint final replica

Colour final replica

Example:The Kazafani Boat

Found in 1963, during a salvage excavation in the now Turkish occupied part of Cyprus (inaccessible and destroyed site).Tomb from the 12th century B.C. Unique object, hand made pottery 40x20.5x23 cm – canoe boat shape Permanently exhibited at the Nicosia

Museum


CRM DigitalData Acquisition Event - Schema

Person: uuid:aeac5200-0138-11e0-a976-0800200c9a66 (E21 Person)

P131 is identified by : D21 Person Name

L51 has first name: Martin (Literal E62 String)

L52 has last name: Doerr (Literal E62 String)

P107 is current or former member: http://www.ics.forth.gr/ (E40 Legal Body)

L62 in the role of: http://www.3d-coform.eu/RoleType/researcher (E55 Type)

Persons (“operators”)


CRM DigitalData Acquisition Event - Schema

Legal Body: http://starc.cyi.ac.cy/ (E40 Legal Body)L4 has preferred label: STARC-The Cyprus Institute, Nicosia, Cyprus (Literal E62 String)

no address

P74 has current or former residence: http://www.geonames.org/146268/ (E53 Place)

L4 has preferred label: Nicosia (Literal E62 String)

P3 has note: Cyprus (Literal E62 String)

exact address and the city where it is located

P74 has current or former residence: uuid: dbae7cd0-e371-11e0-9572-0800200c9a66 (E53 Place)

L4 has preferred label: 15 Kypranoros Street (Literal E62 String)

P89 falls within: http://www.geonames.org/146268/ (E53 Place)

P3 has note: 15 Kypranoros Street, Nicosia 1061, Cyprus (Literal E62 String)

just the address without details for city

P74 has current or former residence: uuid: dbae7cd0-e371-11e0-9572-0800200c9a66 (E53 Place)

L4 has preferred label: 15 Kypranoros Street (Literal E62 String)

P3 has note: 15 Kypranoros Street, Nicosia 1061, Cyprus (Literal E62 String)

Legal Bodies & Places


CRM Digital

Data Acquisition Event: uuid:354c91e0-b3fa-11de-98c6-0002a5d5c30a (D2 Digitization Process)

L4 has preferred label: 2010 Laser scanning in Arch. Museum of Nicosia (Literal E62 String)P2 has type: http://www.3d-coform.eu/EventType/laser_scanning (E55 Type)P2 has type: http://www.3d-coform.eu/EventType/data_acquisition (E55 Type)P3 has note: “evening sun shines through the west window” (Literal E62 String)

SUPER-EVENTS:

P9 forms part of: uuid:07f05f40-b415-11de-9d48-0002a5d5c30c (E7 Activity) (Project)

WHEN:

L31 has starting date-time: 2010-05-28T08:00:00Z (xs:dateTime E61 Time Primitive)

L32 has ending date-time: 2010-06-02T18:00:00Z (xs:dateTime E61 Time Primitive)

WHERE:

P7 took place at: http://starc.cyi.ac.cy/#Place/ArchMuseumNicosia/ConservationLab (E53 Place)

WHO:

L29 has responsible organisation: http://starc.cyi.ac.cy/ (E40 Legal Body)

L30 has operator: uuid:aeac5200-0138-11e0-a976-0800200c9a66 (E21 Person)

Data Acquisition Event


CRM Digital

Data Acquisition Event: uuid:354c91e0-b3fa-11de-98c6-0002a5d5c30a (D2 Digitization Process)

WITH WHAT (camera):

L12 happened on device: http://www.nextengine.com/.../E4035623490 (D8 Digital Device) L59 has serial number: E4035623490 (Literal E62 String)L4 has preferred label: “Next Engine Desktop 3D scanner” (Literal E62 String)(=Model)P2 has type: http://www.3d-coform.eu/DeviceType/laser_scanner (E55 Type)L33 has maker: http://www.nextengine.com/ (E39 Actor)P3 has note: Next Engine Desktop 3D scanner, Multi stripe laser (Literal E62 String)L23 used software or firmware: http://www.nextengine.com/.../Scan_Studio (D14 Software)

WITH WHAT (additional devices):

P16 used specific object: http://b2b.sony.com/.../SONVPLFE4035623490 (E22 Man Made Object)

L59 has serial number: 32526158 (Literal E62 String)

L4 has preferred label: SONY PLFE 40 Projector (Literal E62 String)(= Model)

P2 has type: http://www.getty.edu/research/tools/vocabularies/aat/300022665 (E55 Type)

L33 has maker: http://www.nikon.com/ (E39 Actor)

P16 used specific object: http://www.cgv.tugraz.at/structure_slide_T45a (E22 Man Made Object)

Data Acquisition Event


CRM Digital

Object Acquisition Event: uuid:07f05f40-b415-11de-9d48-0002a5d5c30b (D2 Digitization Process)

L4 has preferred label: 2010 Laser scanning of Kazafani Boat 249.377 in Archaeological Museum of Nicosia (Literal E62

String)P2 has type: http://www.3d-coform.eu/EventType/object_acquisition (E55 Type)

L10 had input: uuid:3d066a90-9cb1-11e0-aa82-0800200c9a66 (D9 Data Object) (calibration file)

L10 had input: uuid:46963500-9cce-11e0-aa82-0800200c9a66 (D9 Data Object) (configuration file)

SUPER-EVENTS:

P9 forms part of: uuid:354c91e0-b3fa-11de-98c6-0002a5d5c30a (D2 Digitization Process) (Data Acquisition Event)

Object Acquisition Event


CRM Digital

Object Acquisition Event: uuid:07f05f40-b415-11de-9d48-0002a5d5c30b (D2 Digitization Process)

WHAT (acquired object):

L1 digitized: uuid:e4761f00-0ce7-11e0-81e0-0800200c9a66 (E22 Man-Made Object)

P1 is identified by: http://www.mcw.gov.cy/mcw/DA/DA.nsf/Objects/249.377 (E42 Identifier)

(all “known” URIs)

L4 has preferred label: Kazafani Boat, vase, 249.377 (Literal E62 String)

L53 is not uniquely identified by: Kazafani Boat (Literal E62 String)

L53 is not uniquely identified by: Bronze Age model of a boat (Literal E62 String)

L55 has inventory no: 249.377 (Literal E62 String)

P2 has type: http://www.getty.edu/research/tools/vocabularies/aat/300132254 (E55 Type) (vase)

P3 has note: “Deep hollow hull with in-curving flat-topped gunwale ….” (Literal E62 String)

P50 has current keeper: uuid:6f2972e6-ad9e-4a72-930d-263f01e75d8c (E40 Legal Body)

(Archaeological Museum of Nicosia)

Object Acquisition Event


CRM Digital

Calibration Event: uuid:07f05f40-b415-11de-9d48-0002a5d5c30c (D2 Digitization Process)

P2 has type: http://www.3d-coform.eu/EventType/calibration (E55 Type)

L1 digitized: http://cg.cs.uni-bonn.de/#Calibration/Bariumsulfate/Block10 (E18 Physical Thing)

L4 has preferred label: block of bariumsulfate (10x10x1cm) (Literal E62 String)

P2 has type: http://www.3d-coform.eu/InformationObjectType/MultiviewdomeCalibrationData

(E55 Type) (color chart, ruler, greyscale)

WHEN:



OUTPUT:

L20 has created: uuid:07f05f40-b415-11de-9d48-0002a5d5c31c (D9 Data Object)

Calibration Event


CRM Digital

Capturing Event: uuid:07f05f40-b415-11de-9d48-0002a5d5c30n (D2 Digitization Process)

L4 has preferred label: Capture 1_0 for Boat (Literal E62 String)

P2 has type: http://www.3d-coform.eu/EventType/capture (E55 Type)

SUPER-EVENTS:

P9 forms part of: uuid:07f05f40-b415-11de-9d48-0002a5d5c30b (D2 Digitization Process)

(Object Acquisition Event)

WHEN: L31 has starting date-time: 2009-07-02T16:07:54Z (xs:dateTime E61 Time Primitive)L32 has ending date-time: 2009-07-02T16:07:54Z (xs:dateTime E61 Time Primitive)

Capturing Event


CRM Digital

Capturing Event: uuid:07f05f40-b415-11de-9d48-0002a5d5c30n (D2 Digitization Process) (cont’d)

OUTPUT:

L20 has created: uuid:07f05f40-b415-11de-9d48-0002a5d5c31g (D9 Data Object)

(image file, zip file ...)

L4 has preferred label: 1_0.ply (Literal E62 String)

P2 has type: http://www.3d-coform.eu/ObjectType/mesh (E55 Type)

P2 has type: http://www.3d-coform.eu/MimeType/ply (E55 Type)

P43 has dimension: (E54 Dimension)

P2 has type: http://www.3d-coform.eu/DimensionType/mesh_vertices (E55 Type)

P90 has value: 394302 (xs:integer E60 Number)

P91 has unit: http://www.3d-coform.eu/UnitType/vertices (E58 Measurement Unit)

P43 has dimension: (E54 Dimension)

P2 has type: http://www.3d-coform.eu/DimensionType/mesh_faces (E55 Type)

P90 has value: 782543 (xs:integer E60 Number)

P91 has unit: http://www.3d-coform.eu/UnitType/faces (E58 Measurement Unit)

Capturing Event


CRM Digital

ARC 3Dcomponent

ARC 3Dcomponent

ImagesUsed images +

+ Depth map for each used image+ metadata

Perform the 3D reconstruction of an artefact from images retrieved from the RI

For an input sequence of images, ARC 3D produces a calibration matrix and a depth map for each image identified as usable for the reconstruction.

This output data is then ingested into the RI so that it can be retrieved and loaded into MeshLab to perform the final reconstruction step (integration of the depth maps).

ARC 3D web service component


CRM Digital

Process Event: uuid:2f7d22db-1d89-11e0-ac64-0800200c9a66 (D3 Formal Derivation)

L4 has preferred label: Processing of Ivory Panel raw data with Arc3D (Literal E62 String)

P2 has type: http://www.3d-coform.eu/EventType/process_event (E55 Type)

P2 has type: http://www.3d-coform.eu/EventType/modeling_process (E55 Type)

WHO:

L29 has responsible organisation: http://www.vam.ac.uk/ (E40 Legal Body)

L30 has operator: uuid:2f7d22d3-1d89-11e0-ac64-0800200c9a66 (E21 Person)

WHEN:



WHERE:

P7 took place at: http://www.vam.ac.uk/#Place/ConservationLaboratory (D23 Room)

ARC 3D Process Event


CRM Digital

Process Event: uuid:354c91e0-b3fa-11de-98c6-0002a5d5c50c (D3 Formal Derivation) (cont’d)

WITH WHAT (Software):

L2 used as source: http://www.esat.kuleuven.be/psi/visics/ARC3D (D14 Software)

L4 has preferred label: ARC3D (Literal E62 String)

P2 has type: http://www.esat.kuleuven.be/psi/visics/ARC3D/Version_1.0.0 (E55 Type)

P2 has type: http://www.3d-coform.eu/SoftwareType/processing_software (E55 Type)

L33 has maker: http://www.esat.kuleuven.be/psi/visics (E39 Actor)

L4 has preferred label: KULeuven PSI VISICS (Literal E62 String)



CRM Digital

Process Event: uuid:354c91e0-b3fa-11de-98c6-0002a5d5c50c (D3 Formal Derivation) (cont’d)

WHAT (Input):

L21 used as derivation source:uuid:2f7d22d2-1d89-11e0-ac64-0800200c9a66 (D9 Data Object)

L4 has preferred label: A.15-1955-dome-out.zip (Literal E62 String)

WHAT (Derivative output):

L22 created derivative: uuid:2f7d22dc-1d89-11e0-ac64-0800200c9a66 (D9 Data Object)

L4 has preferred label: Arc3D-A.15-1955_dmy.v3d (Literal E62 String)

P2 has type: http://www.3d-coform.eu/#ObjectType/mesh (E55 Type)

P2 has type: http://www.3d-coform.eu/#mimetype/v3d (E55 Type)

(calibration files, depth map files, CUN file and respective images)



CRM Digital

params

devices,device models

meshsmeshs

params

Acquistion metadata

subevents

Acquistion metadata

subevents

params

raw data objects

2nd Acquistion who, when, where, what , using what

Reasoning: A Coherent Semantic Net

object metadata

features, history

Acquistion metadata

subevents

Acquistion metadata

subevents

Acquistion metadata

subevents

Acquistion metadata

subevents

Processing metadata

who, when, how, what , using what

Processing metadata


Processing metadata


models

meshs

meshs

models

Acquistion metadata

who, when, where, what , using what

raw data objectsraw data

objectsraw data objectsraw data

objectsraw data objectsraw data

objects

software,algorithms

software,algorithms

…+ who - when - where


CRM DigitalGood Reasons for Reasoning (3D-COFORM)

The integrated semantic network of provenance metadata allows for supporting data consistency, interpretation, reuse, preservation

Management: garbage collection of all reproducible intermediate results (classify software!) export packages: collect all acquisition data and parameters used for one model.

Preservation: monitor obsoletion of all processing tools, format viewers necessary to interprete or

reprocess certain data.

Property propagation to subevents and derivatives (economy & consistency): For instance, “Which object represents my mesh?” result in long query paths:

“Jesus Christ”. forms part of “Ascension” is carried by: “Ivory Panel”. was digitized by: “MiniDomeEvent2011934”.has created: “A.15-1955model v1.zip”.used as derivation source:…….


CRM Digital3D-COFORM: Concatenated Metadata

1IvoryPanel_ObjAcqEvent.rdf1IvoryPanel_ObjAcqEvent.rdf

2IvoryPanel_DocEvent.rdf2IvoryPanel_DocEvent.rdf

forms_part_of

forms_part_of

A.15-1955-dome-out.rdfA.15-1955-dome-out.rdf

A.15-1955-dome-out.zip

has_created

2009CA5306_0.rdf2009CA5306_0.rdf

2009CR4851_0.rdf2009CR4851_0.rdf

…

2009CA5306_0.tif

…

2009CR4851_0.tif

has_created

4Ivory_Arc3DProcEvent.rdf4Ivory_Arc3DProcEvent.rdf

used_as_derivation_source

Arc3D-A.15-1955_dmy.v3d

created_derivative

5Ivory_MeshLabProcEvent.rdf5Ivory_MeshLabProcEvent.rdf

used_as_derivation_source2009CA5307vColoured.ply

created_derivativedigitized

3IvoPan_LegacyData.rdf3IvoPan_LegacyData.rdf

Digitization_Process

Formal_Derivation

Sub-events

Data_Object

Legend

Man_Made_Object

3IvoryPanel_NE_ObjAcqEvent.rdf

3IvoryPanel_NE_ObjAcqEvent.rdf

digitized

4IvoryPanel_NE_DetSeqEvent.rdf4IvoryPanel_NE_DetSeqEvent.rdf

A.15-1955 corner scans.rdfA.15-1955 corner scans.rdf

A.15-1955 NXTENG whole model v1.rdfA.15-1955 NXTENG whole model v1.rdf

A.15-1955 NXTENG whole model v1.zip

A.15-1955 corner scans.zip

forms_part_of

7Ivory_NE_MeshLabProcEvent.rdf7Ivory_NE_MeshLabProcEvent.rdf

A.15-1955 Degree scans.zip

A15-1955 Retouched.zip

created_derivative




A.15-1955_nxtng_5_degrees_complete_bjbrown.ply

created_derivative

created_derivative


A.15-1955 Master.zip


forms_part_of

has_created

has_created


created_derivative

forms_part_of

forms_part_of

has_created

forms_part_of


CRM DigitalConclusions

CRMdig provides a good high-level model for empirical provenance of digital data, open for further specialization, integrated with arbitrary context representations

CRM-CRMDig outperforms competitors in expressive power and integration potential.

Future work: Theory of property propagation to subevents and derivatives

Needed: Theories of feature conservation by kinds of derivation processes.

Links: http://www.ics.forth.gr/isl/rdfs/3D-COFORM_CRMdig_v2.5.rdfs

ics-forth september 21, 2010 crm digital a digital provenance ontology center for cultural...

Documents