mirroring unified metadata as solution for the digital library network at developing countries case...

Mirroring Unified Metadata as Solution for the Digital

Library Network at Developing Countries

Case Study:The Indonesian Digital Library NetworkIsmail Fahmi < ismail@itb.ac.id >

Knowledge Management Research Group

Institut Teknologi BandungIndonesia

2

Students

Researcher

Businessmen

Farmer

Etc…

documents

audios

videos

images

Knowledge is everywhere…

3

The needs…

Manage the knowledge Reuse and utilize the knowledge Disseminate the knowledge

O ACPTUNSYIAH

O JBPTITBPP

O IJPTUNCEN

O JBPTIPBMMA O JIPTUMMO JKPKBPPKO JKPKELNUSAO JKPKFORLINKO JKPNPNRIO JKPTBINUSO JKUNUAJ

O SAPTUNSRAT

O SGPTUNHALU

O YOPTIAIN

Mission

Unlock !

Unlock ACCESS to the Indonesian knowledge

Problems

Internet connection availability Bandwidth

Inhabitants: 210 million

Basic Requirements

We need SOLUTION. Not state of the art of the technology, but a Regional Technology.

Running on narrow bandwidth and unreliable connections.

Common-sense User Interface. Attract individual and institution

(knowledge producers). Ease of implementation and

integration.

Strategies (Technical)

Networked: through a national networked digital library

Distributed: every institutions can manage their own digital library system

Unified: all knowledge will be unified Disseminated: all knowledge will be

disseminated (mirroring), bring information near to the users.

Standardized: using international acknowledged metadata standard (Dublin Core)

Strategies (Society)

Type of partnership: Institution, Individual, and Internet Café.

Promote sub-networks: I.e. health, agriculture, human rights, etc.

License: GPL it !

Knowledge Map

Information: abstract and full content of the LOCAL CONTENT.

Including: Research report Final Project, Theses, Dissertation Article, paper, proceeding, journal E-book, e-learning material, course

material clipping, Heritage, Multimedia, expertise

directory, etc.

Architecture of GDL Network

Key:-Standard Metadata-Protocol

There are a hub and many digital library servers.

*GDL = GaneshaDigital Library

IndonesiaDLN Interoperability Metadata Std. Objective: to manage electronic resources

(documents, audio, video files) on the Internet among the digital library networks.

Dublin Core Metadata Element Sets Has the same objective Simple, multipurpose Widely used

IndonesiaDLN Interoperability Metadata Std was derived from NDLTD Metadata Std (also based on Dublin Core).

Elements of the Metadata

15 Core elements:

TitleCreatorPublisherSubjectDescriptionDateType

FormatIdentifierSourceLanguageRelationCoverageRights

Qualifiers of The Elements

Some elements have qualifiers. IndonesiaDLN allows

implementation of specific qualifiers within the digital library networks,

But when the networks will share into IndonesiaDLN, they must use the IndonesiaDLN Interoperability Metadata Std. (see http://idln.itb.ac.id).

Example of the Qualifiers

Element: TitleQualifiers:

Alternative dc.title.alternative Series dc.title.series

Element: CreatorQualifiers:

Orgname dc.creator.orgname Email dc.creator.email …

Example of the XML format

<?xml version=“1.0” ?><IndonesiaDLN><dc.title> Title goes here (native language)

<alternative>Title in English is here </alternative><series> series, number,… </series>

</dc.title><dc.creator> creator full name

<orgname> organization of the creator </orgname><email> creator’s email </email>

</dc.creator>……</IndonesiaDLN>

Generated by servers during exchange process.

Example: Metadata in XMLMetadata is formatted into XML before exchanged among the digital library servers.

All digital library servers in IndonesiaDLN should use the IndonesiaDLN Interoperability Metadata Standard. Currently is version 1.1.

Key: Identifier

Every metadata will be recognized from its IDENTIFIER.

Identifier is unique within IndonesiaDLN. It contains the following information:

Publisher ID Digital Library Network ID Type of the electronic resources Year of creation Creator Record number, and Content theme

Example of Identifier

jbpeismail-gdl-res-2001-ismail-11-idrc

Publisher IDDigital Library Network IDType of the electronic resourcesYear of creationCreatorRecord number, andContent theme

Benefit: Entering URL containing the identifier to any digital library servers will show the same metadata content.Example:

http://digilib.xxx.xxx/go.php?id= jbpeismail-gdl-res-2001-ismail-11-idrc

File Naming

The IDENTIFIER also be used to assign name to files that are uploaded together with their metadata.

Example: jbpeismail-gdl-res-2001-ismail-11-idrc-report-1.zip jbpeismail-gdl-res-2001-ismail-11-idrc-report-2.pdf jbpeismail-gdl-res-2001-ismail-11-idrc-software-3.zip

20

File storing

Directory: /files/[1,2,3…n]/file-names

n = incremental number for every 50 metadata.

...<dc.relation>

<haspart>jbpeismail-gdl-res-2001-ismail-11-idrc-report-1.gif</haspart><hasformat>image/gif</hasformat><hassize>34322</hassize><hasuri>/download.php? f=/file/1/jbpeismail-gdl-res-2001-ismail-11-idrc-report-1.zip </hasuri><hasdescription></hasdescription>

</dc.relation>...

21

Synchronization

The synchronized data: Metadata Files (optional) Publisher information Membership information

The Synchronization Services (1/2)

Using the services of: HTTP (operational)

Real time transaction Good for reliable connection

SMTP (in progress) Not real time Good for unreliable connection

Post office :-) Burned into CD-ROM, takes days or weeks Good for no internet connection at all

The Synchronization Services (2/2)

Data ProviderLOCAL

Service ProviderHUB

metadata, file

metadata

PUT protocolrequests:- PutMetadata- ListMyIdentifiers- DeleteMetadata- PutFileFragment- MergeFileFragment- PutMembers

GET protocolrequests:- ListIdentifiers- GetMetadata- GetPublishers

AUTHENTICATIONprotocol requests:- Connect- Disconnect- GetMemberInfo

authentication

Metadata are:- sent from LOCAL to HUB server and,- downloaded from HUB to LOCAL server,

through the synchronization services.

Optionally, the files also can be sent from LOCAL to HUB.

There are 3 groups of requests: Authentication, PUT, and GET.

Authentication

Required for: Data synchronization Member access (roaming login)

Publisher &Membership data

Hub Server

Metadata &Local Members

Partner Server

Sync

Roaming Login

PUT

Send PUBLIC information to HUB server

Including: Metadata and Files (optional) Membership Registries

Publisher &Membership data

Hub Server

Metadata &Local Members

Partner Server

Metadata

Files

PUT The Files

Optionally (especially for temporary connection), the metadata related files can be PUT into HUB server.

Files will be sliced (by 10 KB, or customized) before they are sent.

Using HTTP PUT Method.

GET

Download: Metadata Publisher registries

Publisher &Membership data

Hub Server

Metadata &Local Members

Partner Server

Metadata (unified)

Publishers

Publisher Registry

<?xml version="1.0"?><DC.PUBLISHER>

GaneshaDL Central Hub<ID>GDLHUB</ID><TYPE>INSTITUTION</TYPE><APPS>GDL</APPS><ORGNAME>Knowledge Management Research Group ITB</ORGNAME><HOSTNAME>gdlhub.indonesiaDLN.org</HOSTNAME><IPADDRESS>167.205.23.27</IPADDRESS><PROTOCOL>HTTP</PROTOCOL><ADMIN>donfau@kmrg.lib.itb.ac.id</ADMIN><CKO>cko@kmrg.lib.itb.ac.id</CKO><POSTMASTER>pm@digilib.itb.ac.id</POSTMASTER><CONNECTION>DEDICATED</CONNECTION>

</DC.PUBLISHER>

File: /util/publisher.ini

User InterfaceMost users are familiar with interface like Yahoo, Google, Altavista, etc.

The information is organized by categories. Easy to Navigate and Search.

GDL-NetworkGDL-Network

IndonesiaDLN Topology

IndonesiaDLNHUB

institusiwarnet

personal

Other DLNetwork

OtherGDL-HUB

GDL-Network

GDL-NetworkGDL-Network

institusiwarnet

personal

GDL-HUB

ITB

RSHS

Unsyiah

UMM

Binus

UAJ

IPB

Salman

Depkes

Scenario of Sharing & Replication

GDL-NetworkGDL-Network

institusiwarnet

personal

GDL-HUB

ITB

RSHS

Unsyiah

UMM

Binus

UAJ

IPB

Salman

Depkes

Share

Replication

Replication

Replication

Scenario of Access

GDL-NetworkGDL-Network

institusiwarnet

personal

GDL-HUB

ITB

WarnetWarnet

Server GDL Warnet

Server GDL ITB (Source)

Search &^%?

Result = url

Download Full resources

GDL Sub-HUB

GDL-NetworkGDL-Network

GDL-HUB

GDL Sub HUBInstitusi X

Dept A

Dept B

personal

personal

GDL-NetworkHUB

Institution XGDL Sub HUB

GDL Server Dept

Benefits of Promoting Sub-Hub

Encourage institutions with distributed offices to develop DL network.

Boost the number of IndonesiaDLN partners.

Managing subject-specific knowledge and information (ie. Agriculture, healthy, human rights, etc)

Implementation Status

GDL (Ganesha Digital Library) version 1 = End of 1999.

Idea of developing the IndonesiaDLN = October 2000.

Launching of IndonesiaDLN = June 2001 (it just 3 months ago).

The Technology

Operating system: Unix/Linux Windows 98/NT/2000 Windows 95 – needs winsock32

Web server: Apache Scripting Lang: PHP 4 Database Server: MySQL Infrastructure: Dedicated or Dial-Up

Benefits

Easy to install (especially for Windows)

Able run on PC, Laptop promote personal digital library, small institution, SME

Allow dial-up and behind-proxy installation (temporary connection) good for small institution, personal, and internet café.

Problems & Limitations

Updating information at resource server will not automatically update the mirror sites.

The Networks

Currently only GDL-Network that has been fully operational.

Future: DLN for Human Rights, Agribusiness, Distance Learning, Bibliographic, Health Research, and Heritage.

The Network of Networks

IndonesiaDLN

GDL-Engine

GDL-Network

GDL-Engine

Human Rights-Network

GDL-Engine

Agribusiness

Network

GDL-Engine

Heritage-Network

GDL-EngineBiblio-

Network

GDL-Engine

NewSpektraDistance Learning-

Network

GDL-Engine

GDL-Network PartnersBy August 2001

The Distribution Map of GDL-Network PartnersBy August 2001

Members of GDL-Network

Member = users that have registered at GDL servers.

Currently the total members registered at GDL Hub is about 3028 users.

Members of GDL-Network (by partners)By August 2001

Members’ Country

By August 2001

Members’ JobBy August 2001

The ContentsBy August 2001

Important URLs

IndonesiaDLN web site; Mailing list; technical, scenario, society, announcement; standard, http://idln.itb.ac.id

About GDL-Network, registration, join, software, patch, download, etc, http://gdlhub.indonesiaDLN.org/faqgdl.php

Download GDL source code for free, http://gdl.itb.ac.id/download/

GDL-HUB, http://gdlhub.indonesiaDLN.org Digital Library ITB, http://digilib.itb.ac.id Contact KMRG: kmrg@kmrg.lib.itb.ac.id

Thank youThank you