metadata harvesting and validationv2
DESCRIPTION
TRANSCRIPT
![Page 1: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/1.jpg)
Metadata Harvesting and Validation
Bram VandeputteK.U.Leuven
1
![Page 2: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/2.jpg)
slideshare
• http://www.slideshare.net/bramvandeputte
2
![Page 3: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/3.jpg)
• Validation Service• Integration• Online Validation Service• Harvesting Component
3
Overview
![Page 4: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/4.jpg)
• Interoperability : Application Profile (AP)
• Manual check : very time consuming
• Need a tool for enforcing an AP => validation scheme
• A set of validation rules
• Reusable & extendable
4
Validation Service
Best practices derived from previous projects such as MELT and MACE
Reusable : modular + inheritance possible
![Page 5: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/5.jpg)
• Components :
• XML schema : structure• schematron :
• mandatory/conditional elements
• empty fields
• vocabularies (auto generated)
• ...
• Vcard component
5
Validation Service
![Page 6: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/6.jpg)
• Terminology :
• Validation Component
• Validation Scheme
• Validation Scheme URI :• http://aspect-project.org/validation/ASPECTv1.0/core
6
Validation Service
component : atomic block which does specific validation checking
scheme : collection of components that ensures validity against a whole AP
URI : unique identifier of a scheme
![Page 7: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/7.jpg)
7
Validation Service
![Page 8: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/8.jpg)
8
Validation Service
LOM loose
lomloose.xsd
vcard validator
empty attribute fields
ASPECTv1.0/core
vocabulary bank
Legend
uses
extends
ASPECT
vcard validator
validationScheme
validation component
recommended schematron rules
core schematron rules
ASPECTv1.0/recommended
IMS ILOX
![Page 9: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/9.jpg)
9
Validation Service
!
![Page 10: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/10.jpg)
Integration
• When do we validate ?• Possible integration points :
• Mapping• Harvesting• Enriching
10
![Page 11: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/11.jpg)
11
Online Validation Service demo
![Page 12: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/12.jpg)
12
![Page 13: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/13.jpg)
validation to lre APrefer to lre ap document
13
![Page 14: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/14.jpg)
14
invalid : discarded or identifier recorded for next harvesting
![Page 15: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/15.jpg)
14
The Harvest component
invalid : discarded or identifier recorded for next harvesting
![Page 16: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/16.jpg)
ARIADNE Harvester
harvester log
14
invalid : discarded or identifier recorded for next harvesting
![Page 17: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/17.jpg)
ARIADNE Harvester
validation serviceharvester log
ASPECT Repository
14
SQISPI
invalid : discarded or identifier recorded for next harvesting
![Page 18: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/18.jpg)
OAI
External Repository
LOMLOM
OAI-PMH
LOM
LOMLOM
OAI-PMH
ARIADNE Harvester
validation serviceharvester log
ASPECT Repository
14
LOM
SQISPI
invalid : discarded or identifier recorded for next harvesting
![Page 19: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/19.jpg)
OAI
External Repository
LOMLOM
OAI-PMH
LOM
LOMLOM
OAI-PMH
ARIADNE Harvester
validation serviceharvester log
ASPECT Repository
14
LOM
SQISPI
invalid : discarded or identifier recorded for next harvesting
![Page 20: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/20.jpg)
OAI
External Repository
LOMLOM
OAI-PMH
LOM
LOMLOM
OAI-PMH
ARIADNE Harvester
validation serviceharvester log
ASPECT Repository
14
LOMLOM
SQISPI
invalid : discarded or identifier recorded for next harvesting
![Page 21: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/21.jpg)
OAI
External Repository
LOMLOM
OAI-PMH
LOM
LOMLOM
OAI-PMH
ARIADNE Harvester
validation serviceharvester log
ASPECT Repository
14
LOM
LOM
SQISPI
invalid : discarded or identifier recorded for next harvesting
![Page 22: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/22.jpg)
OAI
External Repository
LOMLOM
OAI-PMH
LOM
LOMLOM
OAI-PMH
ARIADNE Harvester
validation serviceharvester log
ASPECT Repository
Validation Msg
14
LOM
SQISPI
invalid : discarded or identifier recorded for next harvesting
![Page 23: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/23.jpg)
OAI
External Repository
LOMLOM
OAI-PMH
LOM
LOMLOM
OAI-PMH
ARIADNE Harvester
validation serviceharvester log
ASPECT Repository
Validation Msg
14
LOM
SQISPI
invalid : discarded or identifier recorded for next harvesting
![Page 24: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/24.jpg)
OAI
External Repository
LOMLOM
OAI-PMH
LOM
LOMLOM
OAI-PMH
ARIADNE Harvester
validation serviceharvester log
ASPECT Repository
Validation Msg
14
LOM
SQISPI
invalid : discarded or identifier recorded for next harvesting
![Page 25: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/25.jpg)
OAI
External Repository
LOMLOM
OAI-PMH
LOM
LOMLOM
OAI-PMH
ARIADNE Harvester
validation serviceharvester log
ASPECT Repository
Validation Msg
14
LOM
SQISPI
invalid : discarded or identifier recorded for next harvesting
![Page 26: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/26.jpg)
OAI
External Repository
LOMLOM
OAI-PMH
LOM
LOMLOM
OAI-PMH
ARIADNE Harvester
validation serviceharvester log
ASPECT Repository
Validation Msg
15
LOM
LOM
SQISPI
LOMLOM
OAI-PMH
LOM
Validation Msg
LOM
LOM
Validation Msg
1 2
34
5
6
![Page 27: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/27.jpg)
Validation Reports
• After harvesting -> report generated and put online
• report has 4 “levels” :
• full log (incl. metadata)
• reporting log
• Grouped Errors
• Error Summary
16
![Page 28: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/28.jpg)
• Questions ?
17
![Page 29: Metadata Harvesting And Validationv2](https://reader034.vdocuments.us/reader034/viewer/2022051818/549f4a18ac79592e768b496a/html5/thumbnails/29.jpg)
• SPI : http://ariadne.cs.kuleuven.be/lomi/index.php/SimplePublishingInterface
• IEEE LOM : http://ltsc.ieee.org/wg12/
• OAI-PMH : http://www.openarchives.org/
• Online Validation : http://ariadne.cs.kuleuven.be/validationService/
References
18