ist- 2001-320015 open archives initiative in europe and germany uwe müller humboldt university...
Post on 26-Mar-2015
218 Views
Preview:
TRANSCRIPT
Open Archives Initiative in Europe and Germany
Uwe MüllerHumboldt University Berlin, Germany
Electronic Publishing GroupUniversity Library / Computer and Media Service
u.mueller@cms.hu-berlin.de
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Agenda
1. Open Archives
2. The Open Archives Initiative
3. OAForum: European Activities
4. DINI: German Activities
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Open Archives
Archive “repository” of digital information
text documents, images, revords, audio/video sequences, ...
Open Archive
provides open machine interface for making content externally available
provides open machine interface for making content externally available
not necessarily: open (= free) usage of metadata (and digital objects)
not necessarily: open (= free) usage of metadata (and digital objects)
mostly: usage of open standards as exchange protocols
mostly: usage of open standards as exchange protocols
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Value of Open Archives
“open” archive A“open” archive A “open” archive B“open” archive B “open” archive C“open” archive C
external serviceexternal service
InterfaceInterface InterfaceInterface InterfaceInterface
InterfaceInterface
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Open Archives
Different types of “archives”scholarly publication server (pre-prints, e-prints)libraries (OPAC, e-journals)museum databases (object metadata)archives (historical documents) and cultural heritageeducation
Origin self archivingunclosing existing databasesestablishing new databases
“open archives” approach gains popularitycross archive accesslow cost dissemination of previously “hidden” resourcesbuilding of new service provision
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Open Archives: Problems
lacking interoperabilitydifferent metadata standards
formats (DC, MAB, Marc ...)
interpretations (Creator: Author vs. Artist vs. Photographer)
different terminology
different languages
different access strategies
different interfaces / transfer protocols
different copyright regulations
Difficulty to establish joint services based on open archives
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
General Search Methods
Cross Search Approach (e.g. Z39.50)
Harvest Approach (e.g. DIENST protocol)
serviceserviceRequest
Answerarchivearchive
1
4
2
3
serviceserviceRequest
Answerarchivearchive
4
6
1
databasedatabase
35
2
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Agenda
1. Open Archives
2. The Open Archives Initiative
3. OAForum: European Activities
4. DINI: German Activities
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
The Open Archives Initiative (OAI)
Main ideasworld-wide consolidation of scholarly archives
free access on the archives (at least: metadata)
consistent interfaces for archives and service provider
effortless implementation
based on existing standards (e.g. HTTP, XML, DC)
Basic functioning
Harvester Repository
Requests (based on HTTP)
Metadata (encoded in XML)
Metadata(Documents)
Metadata
Service Provider Data Provider
„Service”
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAI: General Assumptions
exchange metadata, not digital objects themselves based on harvest approach (asynchronous) two groups of participants Data Providers (Open Archives, Repositories)
free access of metadatanot necessarily: free access and usage of resourceseasy to implement, low barriers
(useable for small institutions) Service Providers
use OAI interfaces of the Data Providers harvest and store metadatamay select certain subsets from Data Providers
(set hierarchy, date stamp)may enrich metadataoffer (value-added) service on the basis of the metadata
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAI: Technical Model
Se
rvic
e P
rovi
der
e-print
Da
ta
Pro
vid
er e-prints
e-print
Da
ta
Pro
vid
er Images
e-print
Da
ta
Pro
vid
er
OPAC
e-print
Da
ta
Pro
vid
er Museum
e-print
Da
ta
Pro
vid
er Archive
Requests:
Identify
ListMetadataformats
ListSets
ListIdentifiers
ListRecords
GetRecord
Responses:
General information
Metadata formats
Set structure
Record identifier
Metadata
Da
ta
Pro
vid
er Harvester
Repository
Repository
Repository
Repository
Repository
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAI-Protocol for Metadata Harvesting
Basics of OAI-PMHprotocol based on HTTP
request arguments as GET or POST parameters
six request types
e.g. http://archive.org?verb=ListRecords&from=2003-08-01
responses are encoded in XML syntax
supports any metadata format (at least: Dublin Core)
Details of OAI-PMHlogical set hierarchy (definition: data providers)
date stamps (last change of metadata set)
error messages
flow control
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAI-Protocol for Metadata Harvesting
Metadata sets (Records)1. header
unique identifier (key for further archive requests),e.g. oai:HUBerlin.de:30000231
datestamp, e.g. 2003-08-11
logical sets in which the record is contained
2. metadata
metadata prefix (identifier for metadata format)
metadata set (at least: Dublin Core, but arbitrary other metadata formats can be transmitted)
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Example: http://edoc.hu-berlin.de/OAI-2.0?verb=ListIdentifiers&from=2002-01-03&until=2002-01-08&metadataPrefix=oai_dc&set=doctypes:dissertations
<?xml version="1.0" encoding="UTF-8"?> <OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"> <responseDate>2002-10-22T17:49:49+01:00</responseDate> <request verb="ListIdentifiers" from="2002-01-03" until="2002-01-08" metadataPrefix="oai_dc" set="doctypes:dissertations">http://edoc.hu-berlin.de/OAI-2.0</request> <ListIdentifiers> <header> <identifier>oai:HUBerlin.de:3000819</identifier> <datestamp>2002-01-08</datestamp> <setSpec>doctypes</setSpec> <setSpec>doctypes:dissertations</setSpec> <setSpec>dnb</setSpec> <setSpec>dnb:dnb33</setSpec> </header> <header> <identifier>oai:HUBerlin.de:3000831</identifier> <datestamp>2002-01-07</datestamp> <setSpec>doctypes</setSpec> <setSpec>doctypes:dissertations</setSpec> <setSpec>dnb</setSpec> <setSpec>dnb:dnb27</setSpec> </header> </ListIdentifiers> </OAI-PMH>
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAI: Data Provider Architecture
SQL-Database
OAI Data Provider
Web server (e.g. Apache, IIS)
OAI request(HTTP request) Programming
extension (e.g. PHP, Perl)
SQL request
DB response
OAI response(XML instance)
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAI: Service Provider Architecture
Data Provider Data Provider Data Provider
Scheduler
Flow controlXML Parser
Normaliser
Database
Service module
User Harvester User
OAI Service Provider
Dublication checker
Update mechanism
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Problems beyond the OAI-PMH
agreement on metadata usage (except DC)semantics
XML schema
agreement on set definitionsselective harvesting
e.g. subject gateways
definition of rights statementsagreement on different right states
machine readable information
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAI: Examples / Participants
Data Provider / Repositoriessee http://www.openarchives.org/Register/BrowseSites.pl
Service ProviderRepository Explorer: http://oai.dlib.vt.edu/cgi-bin/Explorer/oai2.0/testoai/
Cross Archive Searching Service: http://arc.cs.odu.edu/
MyOAI: http://www.myoai.org/
DINI: http://edoc.hu-berlin.de/e_suche/oai.php
Physnet: http://physnet.uni-oldenburg.de/oai/query.php
…
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Agenda
1. Open Archives
2. The Open Archives Initiative
3. OAForum: European Activities
4. DINI: German Activities
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAForum: Background
Project Open Archives ForumEuropean Union Information Society Technologies (IST) Programmeaccompanying measureproject start: October 2001 (duration: 2 years)project partners:
UKOLN, University of Bath, United Kingdom I.E.I.-CNR, Pisa, Italy Humboldt University Berlin, Germany
http://www.oaforum.org/ Motivation
increasing discussion about open archives approachsetting up a framework for the approach in generalpromotion of the open archives approachEuropean view ...
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAForum: Project Partners
UKOLN, Bathvarious projects contexts of metadata and interoperability, cross searching
Renardus project, Schemas, DESIRE
I.E.I.-CNR, PisaCYCLADES project, DELOS
development of services on top of OAI specification
Humboldt University, BerlinDissertation Online, NDLTD
DINI workshops on OAI
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAForum: Objectives
raise and sharpen awareness on the open archives approach
promote “low barrier” interoperability
opening cultural resources
detecting potentials for new services
encourage collaborative development of solutionsdiscussion of problems
exchange of experiences and information
support European liaison with OAI
… build community of interest
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAForum: Objectives (2)
Establish an information repository activities related to the open archives approachexperiences, developed services (e.g. document delivery, searching, browsing, summarisation, linking)share the database with other organisations (e.g. OAI)
Validation of European experience concerningimplementing and using the OAI-PMH and other similar approachesrequirements from implementers and users
Organisational review and analysis possible business modelsIntellectual Property Rights
Dissemination of the open archives approachworkshops
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAForum: Ask and Answer Questions
Not: yet another OAI implementation project … supporting activities
European initiatives with open archives based approach clustering activities
existing and new communitiesIST projectsnational initiatives
dissemination activitiesshare experiences on Open Archivesinvestigation of usage: different paradigmsglobal availabilityshare developments
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAForum: Benefits
tool to reach communities bring together what is happening in Europe raise awareness on and discuss main issues
common terminology on digital repositories
metadata / full text harvesting models
needs of users and communities
advanced services
make European projects ready for actiondevelop possible solutions
establish business models
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Communitiesinstitutions of cultural heritage
museums
European digitising projects
scholarly institutions
public libraries
special user groups
publishers
commercial sector
education
Service Providere-print archives
subject gateways with aggregating functions
value-added services
Data Providerexisting metadata repositories
new metadata collections
OAForum: Participants
Carriers and Users of Open Archives
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAForum: Topics
1. Workshops, Distribution distribute information and experiences on technologies, etc.produce interest for topics connected with open archivesarticles, talks, etc.
2. Organisational Evaluation analyse business modelstackle issues of copyrights (IPR …)
3. Technical Evaluationapply the technical framework of OAIdevelop an information portal with
information on projects, repositories, service providermetadata schemas software and implementations
discuss problems of interoperability
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAForum: Workshops, Distribution
Contact: I.E.I.-CNR, Pisa – Donatella Castelli May 2002, Pisa, Italy
experiences from the European e-prints community
establish the forum
December 2002, Lisbon, Portugalarchives and libraries
open access to hidden archives
March 2003, Berlin, Germanymetadata schemas
networking multimedia resources
September 2003, Bath, United Kingdom“In Practice – Best Practice”
final workshop, EU review
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAForum: Organisational Evaluation
Contact: UKOLN, Bath – Philip Hunter Business models
co-operations between data providers to establish service networks
metadata exchange between archives and services
provision of value-added services (e.g. enrichment of metadata, automatic classification, OpenURL)
Copyright issuesIPR and copyright (influence on producers and distributors)
property rights on metadata (collective use of metadata, metadata exchange, agreements with publishers etc.)
long-term availability of digital resources
Discussion group Dennis Nicholson (University Glasgow)
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAForum: Technical Evaluation
Contact: Humboldt University, Berlin – Susanne Dobratz Interoperability
integration of new (open archive) approaches with existing technologiesIs unqualified Dublin Core sufficient?
Issues on database management concurrency and update problemsscalabilityde-dublication
Software and toolscollect and share experiences and existing solutionsestimation of necessary expenditure
establish data provider / service …skills, manpower, time
Tutorials on OAI technologiesonline tutorial – will be presented at Bath Workshop
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
OAI Activities in Europe
6053
4 2
0
10
20
30
40
50
60
Overview of OAI activity (continents)
Europe
America
Australia
Asia
15
12
7 76
32 2
1 1 1 1 1 1 1 1
0
2
4
6
8
10
12
14
16
Overview of european countries engaged in OAI implementationUK
Germany
France
Sweden
Italy
Netherlands
Austria
Finland
Belorussia
Belgium
Denmark
Ireland
Norway
Portugal
Russia
Switzerland
(numbers from November 2002)
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
http://www.oaforum.org/resources/tecvalq2.php
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
http://www.oaforum.org/resources/glossary.php
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
http://www.oaforum.org/oaf_db/register/
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
www.oaforum.org/oaf_db/list_db/list_services.php
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Agenda
1. Open Archives
2. The Open Archives Initiative
3. OAForum: European Activities
4. DINI: German Activities
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
DINI recommendations for usage of OAI-PMH
created by DINI-OAI working group http://www.dini.de/ target: agreement on syntax and semantics of OAI set
definitions for German data and service providers enhance retrieval quality and support subject gateways (e.g.
Physnet, Dissertation search engine, ...) definition of three classification types
subjects (according to DNB)
formal publication types (e.g. dissertation)
formal document types (e.g. text, audio)
example service provider based on recommended sets: http://edoc.hu-berlin.de/e_suche/oai.php
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Classification according to subjects
SetSpec SetName
dnb:01 Knowledge and Culture in Generaldnb:02 Books and Libraries, Information and Documentationdnb:03 Reference Books, Bibliographiesdnb:04 Directories and Phone Booksdnb:05 Calendarsdnb:06 Journalismdnb:07 Children's and Youth Literaturednb:08 Comics, Cartoons, Caricatures Miscellaneadnb:09 Esoterica Manuscripts, Book Artdnb:10 Philosophydnb:11 Psychologydnb:12 Christianitydnb:13 General and Comparative Theology, Non-Christian
Religiondnb:14 Sociology, Sociography... ...dnb:65 Economic History
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Classification according to formal publication types
SetSpec SetName
pub-type:monograph Books, Monographspub-type:article Journal Articlespub-type:dissertation Dissertations and Professional Dissertationspub-type:masterthesis Diploma Thesespub-type:report Reportpub-type:paper Paperpub-type:conf-proceeding Conference Proceedingspub-type:lecture Lecturespub-type:music Musicpub-type:program ProgramsPub-type:play PlayPub-type:news NewsPub-type:standards Standards
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Classification according to formal document types
SetSpec SetName
doc-type:text Textdoc-type:notes Notesdoc-type:image Imagedoc-type:audio Audiodoc-type:video Videodoc-type:multimedia Multimediadoc-type:data Datadoc-type-binary Binary data, (executable) program
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Multiple Data and Service Providers
Data providers
Service providers
Harvestingbased onOAI-PMH
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Aggregators – Example: HBZ Köln
Data providers
Service providers
Aggregator
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Hybrid Search – Example: Metalib at HU
Data providers
Service providers
Harvestingbased onOAI-PMH
Searchingbased onZ39.50 orSRW
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
A German OAI Example: ProPrint
project of SUB Göttingen and CMS of HU Berlin duration: 2000 – 2003 target: integrate heterogenous document servers in order to
provide PoD service components:
search engine for documents
PDF documents preview
generation of compound PDF file with front page and table of contents
production and delivery by selected print service provider
underlying technology: extension of Dublin Core and OAI-PMH (with an extension for document exchange)
http://www.proprint-service.de/
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Thank you …
Questions?
Uwe Müller
Humboldt University Berlin, Germany
u.mueller@cms.hu-berlin.de
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
Additional Information
Open Archives Initiativehttp://www.openarchives.org/ http://www.openarchives.org/OAI/openarchivesprotocol.html (OAI-PMH)http://www.openarchives.org/service/listproviders.html (Service Providers)http://www.openarchives.org/Register/BrowseSites.pl (Data Providers)http://www.openarchives.org/tools/index.html (Tools, ...)
Open Archives Forumhttp://www.oaforum.org/ http://www.oaforum.org/workshops/bath_invitation.php (workshop, Bath, September 2003)http://www.oaforum.org/resources/tecvalq2.php (Technical Validation Questionnaire)
DINIhttp://www.dini.de/http://edoc.hu-berlin.de/e_suche/oai.php (OAI search engine)
top related