presentation slides, “creating access to audio & video digital media: the variations on video...

June 6, 2012 Hot Topics: DuraSpace Community Webinar Series

Hot Topics: The DuraSpace Community Webinar Series

Series Two: Managing and preserving audio and

video in your digital repository

Curated by Karen Cariani


Webinar 2: Creating Access to Audio & Digital Media:

The Variations on Video Project & the Rock and Roll Hall of Fame

Presented by: Karen Cariani,

Jon Dunn & Adam Wead


About our presenters

• Karen Cariani, Director, Media Library & Archives, WGBH Boston

• Jon Dunn, Director, Library Technologies & Digital Libraries, Indiana University

• Adam Wead, Systems & Digital Collections Librarian, Rock and Roll Hall of Fame


Rock and Roll Hall of Fame

Adam Weed, Systems & Digital Collection Librarian

Rock and Roll Hall Of Fame Library and Archives

• Hall of Fame was founded in 1985• Original library was in the Museum, but was

overrun due to space limitations• Archival materials were moved to off-site

storage• In 2000, the Rockhall Foundation (NYC) began

a capital campaign• Raised $10 million• New facility built at Cuyahoga Community

College Metro campus in 2010

Hot Topics: DuraSpace Community Webinar Series

Library and Archives (cont.)

• Began archival processing and cataloging in 2010

• Grand opening in April 2012• 256 processed archival collections• Over 8,000 cataloged library items• Free and open to the public 9–5, M-F• Serves researchers, Museum staff and

anyone who wants to come in


Library and Archives Staff

• 8 full time staff (permanent)• Librarians: Director, Public Services

Librarian, Metadata Librarian, Library Assistant, Systems Librarian

• Archivists: Head Archivist, A/V Archivist, Assistant Archivist

• 3 project archivists and 3 project catalogers (2010-2012)

• Interns and volunteersHot Topics: DuraSpace Community Webinar Series

Digital Collections: Museum

• No pre-existing asset management system• No digitization• No archival processing• Needed to have something in place by April

2012• Decided to focus on institutional content first• Other archival collections from storage were

unprocessed and their contents unknown• Institutional content had the greatest need for

preservation and access requirementsHot Topics: DuraSpace Community Webinar Series

Digital Collections: Museum (cont.)

• Approx. 1,500 hours of video in various tape formats: Betacam, MiniDV, DigiBeta

• Educational department lectures and presentations

• Songwriters to Soundmen TV series• Some induction footage• No born-digital content (yet)• Outsourced to vendor for digitization


Digital Collections: Foundation

• Induction ceremony footage• Over 800 additional tapes• Selected 150 for priority digitization • Remaining tapes can be processed “as

needed” – mostly ISOs and miscellaneous footage

• Formats include BetacamSP, DigiBeta and HDCam

• 90 mins to 2 hours eachHot Topics: DuraSpace Community Webinar Series

Video Content Challenges

• Archival-quality: uncompressed, 10-bit, 4:2:2 chroma video yields very large files

• 125 GB/hr for SD, the majority of our content, and 500 GB/hr for HD

• Currently, only 60 hours of priority HD content from HDCam tapes but 400 total tapes exist

• Future induction footage will be HDHot Topics: DuraSpace Community Webinar Series

Access Issues

• Access limited to onsite only• Surrogate files are primary means of

access: compressed H.264 video• Uncompressed files only needed for

future transcoding or for third-party distribution if H.264 video is unsuitable

• Have not had any requests for preservation video yet


Storage Solutions

• Tailored our storage system to fit our access needs: high data volume, low access rate

• Small, surrogate video files are kept on disk, in a SAN for immediate access

• Uncompressed video files are kept on LTO-5 tape using hierarchical storage so that they can be moved back to disk when needed

• For archiving, all files are backed up to two LTO-5 tapes: one stored at the Museum (3 mi. away), and a second at Iron Mountain in Boyers, PA


Storage Pros and Cons

• Start small, grow as needed• Archiving data to tape is time-consuming• Ex. 8 TB ingest takes about a week• Retrieving uncompressed data from tape is

time-consuming• Requests for large files must be scheduled:

limited hardware resources for multiple retrievals

• Modular design of our storage system offers easy expansion should requirements change


Tools

• Hydra software stack• Fedora repository• Large video files are stored as external

datastreams accessible via HTTP• PBCore for descriptive metadata • Records are exported to Blacklight for

public access• Video streamed using open source

software: Wowza with FlowplayerHot Topics: DuraSpace Community Webinar Series

System Overview


Fedora Model


Costs and Limitations

• Storage systems represent the bulk of the monetary costs

• Digitization vendor expenses come second: George Blood Audio/Video digitizes our video

• Staff hours spent developing software (me)

• Staff hours spent cataloging items: one librarian, one archivist, plus volunteers


Pros and Cons

• Costs saved through in-house development vs. ability to produce working software

• Atomistic fedora model can work with any kind of A/V content

• Staffing determines cataloging rate• Lack of organization make processing

collections time-consuming• We have a digital “backlog” for content that

we can’t yet ingest


Future Work/Challenges

• Images and documents: institutional and from archival collections

• Audio from archival collections• Models in Hydra to deal with new

content types• Mapping archival description to Fedora

objects to create coherent finding aids• Merging physical archival content with

digital Hot Topics: DuraSpace Community Webinar Series

For More Info

• About Us: http://library.rockhall.com• Our Catalog: http://catalog.rockhall.com• About Hydra: http://hydraproject.org/• Our Hydra “head”:

https://github.com/awead/Hydra-Rock


Variations on Video: Building the Next Generation Library Media Management System

DuraSpace Hot Topics Webinar

Jon DunnIndiana University

June 6, 2012

Presentation Outline

Background and motivations Technical architecture Development roadmap and process

Background: Variations

Open source digital music library system developed at Indiana University

In use at IU since 1996, in various forms Used at about 20 institutions, mainly for

streaming audio course reserves variations.sourceforge.net

Variations on Video

Educational institutions collaborating with open source software communities

Led by Indiana University and Northwestern University Libraries

Goal: Create an open source system to enable libraries and archives to provide online access to video and audio collections

Variations on Video

Planning grant from IMLS August 2010 – June 2011

Implementation grant from IMLS October 2011 – September 2014

Variations on Video:Project Objectives

Develop a digital audio/video management and delivery system, focused on needs of libraries and archives

Follow an agile, open source development model Leverage existing technologies, where feasible Communicate and market the project broadly to

increase awareness and grow the community of users and developers

Partners and Collaborators

Variations on Video:Motivators

Demand from Variations implementers and other institutions

Increased video digitization and creation at IU IU Media Preservation Initiative IU IT strategic plan: Empowering People

No existing system serves needs of libraries History of involvement in open and community source

software Desire to create a sustainable foundation for Variations

development and maintenance

Indiana University BloomingtonMedia Preservation Plan

www.indiana.edu/~medpres/

Serving the needs of libraries and archives

Wide variety of audio/video collections, uses, and access needs Licensed educational video collections Video/audio e-reserves Archival collections Open access collections

Research and teaching & learning use Long-term management and access Tie-in with preservation

Variations on Video: Content

Video digitized

from library

collections

Files with purchased or licensed streaming

rights

University produced

video

Archival collections

Faculty-produced

video

Feature Films

Documentaries

TV shows

Live Performances

LectureSeries

Field Recordings

Research-relatedVideo

Existing Products Institutional and digital library repositories

DSpace, Fedora, Digital Commons, ContentDM Web video services

YouTube, Vimeo Streaming servers

Flash Media Server, Real Helix, Wowza, Red5 Classroom lecture capture

Echo360, Mediasite, Opencast Matterhorn Digital asset management systems

OpenText, NetXposure Online video environments

Kaltura, Brightcove

Variations on Video

Designed to meet needs of libraries and archives Storage requirements Streaming Transcoding Access control Media players Structural metadata / navigation Ease of use and administration

Variations on Video:Basic Functionality

Video file upload/ingestion Transcoding Descriptive metadata entry / bulk upload Basic structural metadata Discovery Delivery and navigation Reuse

Key Architectural Components

Hydra framework Fedora repository Opencast Matterhorn Streaming Server

Hydra Framework

http://projecthydra.org

Opencast Matterhorn

http://opencast.org/matterhorn/

Architecture Revisited Hydra Framework

Search for free Rich toolset for quick development “One body, many heads”

Opencast Matterhorn Flexible processing pipeline; highly modular

Streaming Server Flash/RTMP, HTML5/HTTP, and Apple Live HTTP Streaming all

required Red5 – open source and support in Matterhorn Flash Media Server – popular and iOS support

Variations on Video: Roadmap Release 0 (July 2012)

Manual video and basic metadata ingest Transcoding Search and basic playback in desktop browsers

Release 1 (December 2012) Desktop and mobile audio/video playback Clip and playlist creation Authentication and group-based authorization Manual and batch ingest More complete discovery interface

Release 2+ (2013-) …

Development Process

Scrum agile approach, based at IU and NU Single virtual distributed team

Other partners will install, test, and provide feedback

Transparent planning and development process Open wiki, Jira, mailing lists Regular public demos

Engage additional community involvement over time

For more information

http://variationsonvideo.org/

Variations on Video e-mail list E-mail [email protected] to subscribe

twitter.com/varvideo

facebook.com/varvideo


Questions?

Jon DunnIndiana [email protected]

Adam WeadRock and Roll Hall of [email protected]


• DuraCloud is simple, flexible and cost effective

• DuraCloud offers expandable storage space as needed for your media files

• Streaming media services available

Solutions for Digital Media Preservation


• Sign up for a free 2 month trial account today www.duracloud.org

• Priced for every budget http://duracloud.org/pricing

Solutions for Digital Media Preservation