Stewarding Research Data with Fedora and

IslandoraMark Leggott, University of PEI/Islandora Foundation/

discoverygardenDuraSpace Webinar - September 10, 2013

Evolution•Developed @ UPEI (2007)•Core team at DGI/UPEI• For diverse needs of digital

asset management in all areas•Assumes will need to change

all or some components over time a key requirement

• Integration with other systems also a critical design aspect

Conceptualizing

Initializing

Creating/Analyzing

Reporting

Formalizing

Popularizing

Research Institutes

Libraries & Archives

Museums

Media Organizations

Health Centres

Government Agencies

Private Companies

Universities & Colleges

NGOs & Non-ProfitsOther

Access Collaboration Preservation

E-Mail, Letters, Published Research,

Requirements

Meeting Minutes, Grants, Data Collection,

Acquisitions

Forms, Data, Cataloguing,

Findings, Discussion

Reports, Theses, Datasets, Visualizations

Articles, Curricular Content, Policies,

Exhibits

Blogs, Twitter, Newspapers, iTunesU, Flickr

Information Life Cycle

Object Space

User Space

Individual

Group

Department

Museum

University

External

Private Shared Open

Co

lla

bo

rate

Pu

bli

sh

Re

-Use

Cre

ate

Preservation, Migration, Transformation

Basics

• Drupal+Fedora+Other OS = ecosystem

• Flexible UI on top of Fedora + other apps

• Support for 180+ languages via Drupal

• Focus on robust preservation features and services + flexibility in data models and UI

• VM/code, documentation, lists, Camps

imagined:208361 (PID)

Object Properties

Relations (RELS-EXT)

Dublin Core (DC)

Audit Trail (AUDIT)

JP2K Web (JP2)

JP2K Archival (LOSSLES_JP2)

Low Res JPEG (JPG)

Thumbnail (TN)

Descriptive Metadata (MODS)

Object Model - IslandImagined/Large Image

Digital Object Identifier

System PropertiesManage & Track Object

Reserved DatastreamsKey Object Metadata

DatastreamsAggregates Content Items

Content Models

• Flexibility supports any data model

• Atomistic and compound objects

• Support for RDF allows integration of specific ontologies

Solution Packs• Image, Large Image, Audio,

Video, Book, PDF, Newspaper, WARC

• Includes MODS form, DC mapping, sample data, viewer(s), TechMD extraction, etc.

• Solution Pack module makes it easier to create new ones, modify existing

Form Builder• Create a rich form for any

XML schema• Multiple forms for specific

schemas - present different forms to different users

• Multiple schemas in a single form

• Add advanced functions, such as look-up fields using indexes for other objects

• Apply security policies

Editorial Workflows

• Simple approach to Editorial Workflow

• Provides “human” nodes in the services framework

• Upcoming version support more granular controls and workflow states/actions

Preservation Services

• Fedora provides robust service framework

• TechDS+DescDS+RightsDS+AuditDSs to a Dynamic PREMIS record coming soon

• Adding DuraCloud support via “Vault”

• Archivematica integration

Tools Modules• FITS Extractor, creates

technical metadata• Batch Import (RIS, EndNote,

PubMed, DOI, OAI)• OCR, Tesseract with OCR/

HOCR• MARCXML, ingest and view

MARC data• Others: OAI, Bookmark

Taverna Integration

Taverna Integration

Community

• Estimate 150+ Islandora sites worldwide in production or development

• 550+ people on Google Groups List

• Some projects starting to contribute back

• Libraries bulk of use now, but includes museums, archives, private companies

• Non-profit Islandora Foundation: help maintain code, documentation, training, community participation and more

• Membership model (12+ members as of Sep)

• Partner - $10K, Board, Resources, Camps

• Collaborator - $4K, Roadmap, Resources

• Member - $2K, Discounts

• Melissa Anez Project & Community Manager

discoverygarden• Commercial UPEI spin-off -

full service• Installation, Configuration,

Customization• Support, System Audit,

Consulting• Hosting, Platforms, Vendor

partnerships• Primary codebase contributor

Research Data

Physical Data Model

• UPEI/DGI developing a generic data tool to work with systems researchers use now

• Provide a range of filesystem sync tools

• Minimal service - store data in repository

• Enhance with metadata, transform services

• Project metadata CASRAI/VIVO/CERIF +

Fedora Repository

DescMDTechMD

AdminMDAssets

Local File System

DropBox

Box.net

DataStage

Google Drive

Private Cloud

Storage

Generic Research Data SP

(+ Standard SPs, Viewers)

Sync

Extract

Transform

Enrich

Check

MintTaverna DataCite

FITS + Authority

Islandora Generic Research Data Architecture

Islandora Framework

Islandora VRE (Virtual

Research Environment)

Islandora IR (Research Articles)

BackupsRegional &

National TDRs

DropBox

• Alpha module provides sync between DropBox and Islandora

• Creates Collection objects for each folder and a separate file object for each contained file with all relationships

• Provides basic DC record for metadata

• Upcoming for Google Drive, DataFlow ++

Intellectual Data Model

• Smithsonian/DGI developing Sidora system to respond to specific research data needs

• Custom interface, Content Models and Forms, adding Taverna/R integration

• Camera trap images, archaeological data, carbon sequestration data

• File browse interface for all operations

Fedora Repository

DescMDTechMD

AdminMDAssets

Image SP + FGDC,

DwC

Numeric Data SP +

FGDC, DDI

Panama Dig Data +

LIDO

Research Articles

Sidora Application

Taverna R

FITS + Authority

The Smithsonian Data Architecture

Islandora Framework

Sidora

Intellectual Data Model

Physical Data Model

Examples

Institutional Repository

UPEI VRE

• Rich implementation of Islandora

• Used for digital stewardship of research, administrative and learning assets of UPEI

• Over 150 VREs with wide range of features

• VRE Management Team with 4 librarians

• Standard no cost, extra features charged

Links• General: islandora.ca, discoverygarden.ca, islandora.ca/if, sandbox.islandora.ca,

wiki.duraspace.org/display/FF/Fedora+Futures+Home, duracloud.org

• Code: github.com/Islandora, jenkins.discoverygarden.ca, travis-ci.org/Islandora/islandora/pull_requests, wiki.duraspace.org/display/ISLANDORA/Islandora, jira.duraspace.org/browse/ISLANDORA

• Institutional Repositories: islandscholar.ca, digital.march.es/ceacs-ir, digital.grinnell.edu/drupal/, digitalunc.coalliance.org/

• Digital Library Collections: peildo.ca, digital.march.es/clamor, digital.march.es/merce, newspapers.vre.upei.ca, mirc.sc.edu, islandimagined.ca, vre2.upei.ca/pwc/, atmintis.mb.vu.lt/en, unbound.williams.edu

• Research Data: library.upei.ca/vre, www.taverna.org.uk/, vdp.vre3.upei.ca/, modernistcommons.ca, vre2.upei.ca/herbarium/, discoveryspace.upei.ca/parca, discoveryspace.upei.ca/quantumchem/, upeikerrlab.ca

• Consortia: cairnrepo.ca, adrresources.coalliance.org

Note: some of these sites require authentication access - contact Mark for more information.

Questions?Mark Leggott - University of PEI

mleggott@upei.ca