a new jersey statewide video portal based on open source ...€¦ · based on open source...
TRANSCRIPT
![Page 1: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/1.jpg)
A New Jersey Statewide Video Portal
Based on Open Source Technologies
Isaiah BeardDigital Standards & Workflow Manager - SCC
Repository Architects: Ron C. Jantz and Grace Agnew
Rutgers, the State University of New Jersey
Internet2 Spring Member MeetingArlington, VA - April 23, 2008
Tuesday, April 15, 2008
![Page 2: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/2.jpg)
Concepts and History
Architecture and
Usage Scenarios
Challenges unique to
NJVid
Demos/
Proof of concept
Outline & Overview
2
Tuesday, April 15, 2008
![Page 3: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/3.jpg)
Concepts & History
Beginnings
Tuesday, April 15, 2008
![Page 4: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/4.jpg)
Predecessor Technologies
New Jersey Digital Highway (njdigitalhighway.org)
Statewide repository - IMLS grant
Established Foundation Platform
RUcore - Rutgers Community Repository
Expansion of NJDH technology
Customization, portals
Born Digital Archiving
4
Tuesday, April 15, 2008
![Page 5: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/5.jpg)
Predecessor Technologies
New Jersey Digital Highway (njdigitalhighway.org)
Statewide repository - IMLS grant
Established Foundation Platform
RUcore - Rutgers Community Repository
Expansion of NJDH technology
Customization, portals
Born Digital Archiving
4
Tuesday, April 15, 2008
![Page 6: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/6.jpg)
Predecessor Technologies
Existing Video Collections
Partner Institutions with video content
New Jersey Digital Highway
Local Collections - WPUNJ, Montclair, etc.
5
Tuesday, April 15, 2008
![Page 7: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/7.jpg)
Collection/Video Types
Three Major Video Collection Types
Video Commons
Locally Owned/Locally produced
Commercial Videos - Consortia
Lectures on Demand
6
Tuesday, April 15, 2008
![Page 8: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/8.jpg)
Architecture & Workflow
Nuts & Bolts
Tuesday, April 15, 2008
![Page 9: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/9.jpg)
A Trusted Repository
Reliable content delivery
Standards-baseddigitization
Preservation Platform
Rights Enforcement
Authentication/Authorization
Reference Architecture
8
Tuesday, April 15, 2008
![Page 10: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/10.jpg)
A Trusted Repository
Reliable content delivery
Standards-baseddigitization
Preservation Platform
Rights Enforcement
Authentication/Authorization
Reference Architecture
8
Tuesday, April 15, 2008
![Page 11: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/11.jpg)
Reference Architecture:
Technologies in Use
9
Area Software Used
Applications/Web Interface PHP / MySQL
Journals Open Journal System (PKP)
Search Engine Amberfish
Workflow Mangement PHP / MySQL (developed in-house)
Collection Services PHP / MySQL (developed in-house)
Handles/Persistent IDs CNRI Handle Server
Video Streaming Server Darwin (Apple)
Digital Library Framework FEDORA
OS/Web Server Linux/ApacheTuesday, April 15, 2008
![Page 12: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/12.jpg)
MatroxAxio
Encoder
Reference Architecture
10
User Input(Metadata)
Workflow ManagementSystem
Video
<XML>Digital Object
Repository(FEDORA)
DigitizedVideo Master
FEDORA Repository Service
Collection/Preservation Services
NJVid Commons
Partner Portals
Tuesday, April 15, 2008
![Page 13: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/13.jpg)
MatroxAxio
Encoder
Reference Architecture
10
User Input(Metadata)
Workflow ManagementSystem
Video
<XML>Digital Object
Repository(FEDORA)
DigitizedVideo Master
FEDORA Repository Service
Collection/Preservation Services
NJVid Commons
Partner Portals
Tuesday, April 15, 2008
![Page 14: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/14.jpg)
Large Master Video Files
Master DV/AVI files @ 20GB+ per hour
Ingest, presentation, preservation
Lectures on Demand - Annotate Video
Authentication/Authorization Facility
Architectural Challenges
11
Tuesday, April 15, 2008
![Page 15: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/15.jpg)
Architectural Challenges
12
Object Architecture
Repository ID
Metadata
Behaviors(dissemintaors)
Data StreamsSMAP1 - StrMap (TOC)
FLV1-Download
MOV1-Quicktime Stream
PDF1 - Transcript
XACML Policy
ARCH1 - Archival Master
Tuesday, April 15, 2008
![Page 16: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/16.jpg)
Architectural Challenges
12
Object Architecture
Repository ID
Metadata
Behaviors(dissemintaors)
Data StreamsSMAP1 - StrMap (TOC)
FLV1-Download
MOV1-Quicktime Stream
PDF1 - Transcript
XACML Policy
ARCH1 - Archival Master
Descriptive
Technical
Source
Rights
Digital Provenance
MD Structure
Tuesday, April 15, 2008
![Page 17: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/17.jpg)
Demo
Workflow Management System (WMS)
Reference Architecture
13
Tuesday, April 15, 2008
![Page 18: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/18.jpg)
Large Files:
The Digital Video Object
14
Archival Masters
Film transfer/VHS/Betacam/Analog Sources
Uncompressed AVI as Master
Digital (DVCAM, miniDV, HDV, etc.)
DV file with native format/encoding
DVDs
Disk Image (ISO) kept as archive Masters
Tuesday, April 15, 2008
![Page 19: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/19.jpg)
Large Files:
The Digital Video Object
15
Presentation StreamsStreaming Protocol: MPEG-4 H.264 (Quicktime)
15-20 frames/sec, 640x480
Data rate: 384-512 kbps
Progressive Download
Flash Video (FLV with SWF wrapper)
15 fps, 640 x 480 - 256-384 kbps
Intended as an alternate presentation mode
Tuesday, April 15, 2008
![Page 20: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/20.jpg)
Large Files:
The Digital Video Object
15
Presentation StreamsStreaming Protocol: MPEG-4 H.264 (Quicktime)
15-20 frames/sec, 640x480
Data rate: 384-512 kbps
Progressive Download
Flash Video (FLV with SWF wrapper)
15 fps, 640 x 480 - 256-384 kbps
Intended as an alternate presentation mode
Tuesday, April 15, 2008
![Page 21: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/21.jpg)
Large Files:
The Digital Video Object
15
Presentation StreamsStreaming Protocol: MPEG-4 H.264 (Quicktime)
15-20 frames/sec, 640x480
Data rate: 384-512 kbps
Progressive Download
Flash Video (FLV with SWF wrapper)
15 fps, 640 x 480 - 256-384 kbps
Intended as an alternate presentation mode
Tuesday, April 15, 2008
![Page 22: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/22.jpg)
Large Files:
The Digital Video Object
16
Presentation Streams
Codec Selection Criteria
Easy to Use/Install
Supports Multiple Platforms
Built on established industry standards
Excellent quality/bandwidth ratio
Widely Used (Akamai, YouTube, etc)
Tuesday, April 15, 2008
![Page 23: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/23.jpg)
Large Files:
The Digital Video Object
17
Content Demo
Demo Video Clip from Existing Collections
Tuesday, April 15, 2008
![Page 24: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/24.jpg)
Large Files:
The Digital Video Object
18
Presentation Streams
Open Source-able architecture - Darwin Streaming Server (DSS)
Can run on all major OS platforms
Free, open source license (APSL)
Tuesday, April 15, 2008
![Page 25: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/25.jpg)
Large Files:
The Digital Video Object
18
Presentation Streams
Open Source-able architecture - Darwin Streaming Server (DSS)
Can run on all major OS platforms
Free, open source license (APSL)
Tuesday, April 15, 2008
![Page 26: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/26.jpg)
Large Files:
The Digital Video Object
19
Presentation Streams
Transcripts
When available
Searchable & IndexableText/XML
Will require labor (transcribing & data entry)
Tuesday, April 15, 2008
![Page 27: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/27.jpg)
Application Architecture
20
Backend StorageSubsystem
Digital ObjectRepository(FEDORA)
Partner/Custom Portals
Middleware(Host Server/Shibboleth)
Archival MasterFiles
Darwin Streaming Server(Video Content Delivery)
- External Storage- Checksum Integrity Validation
Tuesday, April 15, 2008
![Page 28: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/28.jpg)
Application Architecture
20
Backend StorageSubsystem
Digital ObjectRepository(FEDORA)
Partner/Custom Portals
Middleware(Host Server/Shibboleth)
Archival MasterFiles
Darwin Streaming Server(Video Content Delivery)
Progressive Download(Flash Video)
Retrieve Streams in FEDORA via NFS
- External Storage- Checksum Integrity Validation
Quicktime/H.264 Stream
Tuesday, April 15, 2008
![Page 29: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/29.jpg)
The Front-End Interface
Usage Scenarios
Tuesday, April 15, 2008
![Page 30: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/30.jpg)
Faculty Annotation for Lectures on Demand
Specific clips, segments for dissemination
General Public Access - NJVid Commons
Student Access to Licensed Videos
A/A required for selective access
End User Scenarios
22
Tuesday, April 15, 2008
![Page 31: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/31.jpg)
Faculty Annotation for Lectures on Demand
Specific clips, segments for dissemination
General Public Access - NJVid Commons
Student Access to Licensed Videos
A/A required for selective access
End User Scenarios
22
Tuesday, April 15, 2008
![Page 32: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/32.jpg)
Workflow
Instructor previews video and selects segments for students
Lectures on Demand:
Annotated Video Clip
23
Tuesday, April 15, 2008
![Page 33: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/33.jpg)
Workflow
Instructor previews video and selects segments for students
Lectures on Demand:
Annotated Video Clip
23
Tuesday, April 15, 2008
![Page 34: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/34.jpg)
Workflow
Instructor previews video and selects segments for students
Lectures on Demand:
Annotated Video Clip
24
Instructor authenticates, and creates annotation objects for the video, specifying in and out points
Tuesday, April 15, 2008
![Page 35: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/35.jpg)
Workflow
Instructor previews video and selects segments for students
Lectures on Demand:
Annotated Video Clip
24
Instructor authenticates, and creates annotation objects for the video, specifying in and out points
Tuesday, April 15, 2008
![Page 36: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/36.jpg)
Workflow
A structure map with appropriate time segments and persistent ID of the source video is created. All pieces fall into place.
Lectures on Demand:
Annotated Video Clip
25
Tuesday, April 15, 2008
![Page 37: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/37.jpg)
Workflow
A structure map with appropriate time segments and persistent ID of the source video is created. All pieces fall into place.
Lectures on Demand:
Annotated Video Clip
25
Object ID
Metadata
Behaviors
Data Stream
TOC
Video for Class 101:01Structure Map
Intro: 0:00.00 - 11:01.29
History: 20:00.09 to 45:08.10
Wrap-Up: 65:22.24 to 70:01.18 DarwinStreaming
Server
Presented Video Object
Tuesday, April 15, 2008
![Page 38: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/38.jpg)
Finalization of Video Architecture (Summer 2008)
Implement Shibboleth for Authentication and Authorization (Fall 2008)
Fine-Grained object access via XACML (Fall 2008)
Video Table of Contents (early to mid 2009)
Multiple Structure Map Annotations for Complex Segments (mid 2009)
NJVID Custom Installation Configuration
Development Needs
for NJVID
Tuesday, April 15, 2008
![Page 39: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/39.jpg)
Questions?
Isaiah BeardDigital Standards & Workflow Manager
Scholarly Communication Center
Rutgers University Libraries
Internet2 Spring Member MeetingArlington, VA - April 23, 2008
Tuesday, April 15, 2008
![Page 40: A New Jersey Statewide Video Portal Based on Open Source ...€¦ · Based on Open Source Technologies Isaiah Beard Digital Standards & Workflow Manager - SCC Repository Architects:](https://reader034.vdocuments.us/reader034/viewer/2022050205/5f5840edd0f0d12e0337a6f9/html5/thumbnails/40.jpg)
Watch This Space!
View our blog, sample videos and progress reports at:
www.njvid.org
Internet2 Spring Member MeetingArlington, VA - April 23, 2008
Tuesday, April 15, 2008