my media at university of toronto libraries
Upload: university-of-toronto-libraries-information-technology-services
Post on 20-May-2015
1.572 views
DESCRIPTION
TechKnowFile Conference 2010 http://mymedia.library.utoronto.caTRANSCRIPT
![Page 2: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/2.jpg)
Sian MeikleDigital Services LibrarianInformation Technology ServicesUniversity of Toronto [email protected]
![Page 3: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/3.jpg)
Overview• How the media server came to be
• How to use the media server
• How the media server was built
• Rapid application deployment: media server as a case study
• What’s next
![Page 4: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/4.jpg)
Why a media server?• Give university a home for media
storage
• Provide tools for publishing media to web pages easily
• Integrate media resources into discovery tools such as library’s discovery layer, university search engine
![Page 5: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/5.jpg)
Why a media server? File format Creators can make video in many formats
13 very common formats .3g2 .3gp .asf .asx .avi .flv .mov .mp4 .mpg .rm .swf .vob .wmv23 common formats .diva .dream .dvr-ms .f4v .fbr .hdmov .m4v .mkv .moi .mpeg .mts .mxf .
ogm .rcproject .rmvb .scm .smil .srt .stx .ts .vro .wtv .xvid 172 average or less common file formats .3gp2 .3gpp .3mm .aep .ajp .amv .amx .arf .avb .avs .avs .axm .bik .bin .bix .box .bsf .byu .camrec .cvc .d2v .d3v
.dat .dce .dir .dmb .dpg .dv .dvx .dxr .evo .eye .fbz .fcp .flc .flh .fli .flx .gl .grasp .hkm .ifo .imovieproj .imovieproject .ivf .ivr .ivs .izz .izzy .lsf .m1pg .m21 .m21 .m2t .m2ts .m2v .m4u .mgv .mj2 .mjp .mnv .moov .mp21 .mp21 .mpv .mqv .msdvd .msh .mswmm .mtv .mvb .mvc .mvd .ncor .nsv .nvc .ogv .par .pds .piv .playlist .pmf .prel .pro .prproj .pxv .qtch .qtl .qtz .rdb .rec .roq .rp .rts .rts .rum .rv .sbk .scn .sfvidcap .smi .smk .spl .ssm .svi .swi .swt .tda3mt .tivo .tod .tp .tpr .trp .vc1 .vcv .vdo .veg .vf .vgz .vid .viewlet .viv .vivo .vlab .vp6 .vp7 .w32 .wcp .wm .wmd .wmmp .wmx .wp3 .wvx .yuv .zm1 .zm2 .zm3 .zmv .dsy .gvi .m4e .mmv .mod .mpf .mpg2 .mpv2 .scm .60d .dav .ddat .dif .gvp .iva .lsx .m1v .m2a .meta .mjpg .modd .movie .mp2v .mp4v .mpe .pva .qt .sbt .smv .str .vem .vfw
MyMedia converts media to just 3 formats for users
flashmp4mp3
![Page 6: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/6.jpg)
Why a media server? File size.
Text: 5 MB / 1000 pgs (5 kb / pg)
Sound: 460 MB / hour (128 kb / s)
Video: 5 GB / hour ( 1.4 Mb / s)
![Page 7: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/7.jpg)
Why a media server NOW?
• Exploratory work since early 2008• Commercial options megalithic, expensive
• August 2009: pandemic preparedness• Media server built and deployed in 4 weeks
![Page 8: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/8.jpg)
Media creator
1. Upload mediaa) Got media? Upload it to MyMedia b) Need to make media? Use a tool like Camtasia Relay to capture video. Camtasia Relay saves your media to the media server.
2. Describe media Fill in a web form: i) rights: UofT / world ii) play type: stream / downloadiii) metadata:keywords title description
3. Publish mediai) Embed it in your web pageii) Link to it on media server
2 Describemedia
3Publish media
UofT Portal (Blackboard)University web sitesAny web page
Upload media1
LibraryMedia Server
(eg Camtasia Relay)
Capture media
![Page 9: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/9.jpg)
How people use the media server• Instruction
– Lecture capture– 3rd party content– How-to:
Library, Blackboard, …• Scholarship
– Conferences, invited lectures • University life
– Video contests
• NOT live web casting Stream
only64%
Download / stream
36%
UofT only62%
Public access
38%
![Page 10: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/10.jpg)
Gordon BelraySystems and Interface DesignerInformation Technology ServicesUniversity of Toronto [email protected]
![Page 11: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/11.jpg)
• Pandemic planning required rapid integration with Camtasia Relay server
• https://eclasscam1.erin.utoronto.ca/Relay
![Page 12: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/12.jpg)
• What is streaming?• Checks the connection speed of the user and buffers a small
portion of the video using a Flash player• Uses RTMP (real time messaging protocol) on a Flash Media
Server • File formats: Flash (.flv) MPEG4 (.mp4) encoded with H.264
codec, and MP3s• Progressive Downloads• Media must be downloaded to the end user’s browser• Cannot scrub beyond point of download
![Page 13: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/13.jpg)
• Demo
![Page 14: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/14.jpg)
• Login using UTORid
![Page 15: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/15.jpg)
• Home page showing processed and unprocessed media
![Page 16: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/16.jpg)
• Media management page
![Page 17: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/17.jpg)
• Example of published media using the myMedia links
![Page 18: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/18.jpg)
• Permanent link to play media
![Page 19: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/19.jpg)
Bilal KhalidSenior Application Developer,Information Technology ServicesUniversity of Toronto [email protected]
![Page 20: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/20.jpg)
Software Components
• Database Schema (MySQL 5)• Monitoring script for Camtasia (perl)• MyMedia administrative interface
(Java/WebObjects)• Video encoding using ffmpeg• Media viewing interface (PHP)
![Page 21: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/21.jpg)
filedump
t_user
t_media
Database Schema• Temporary staging area for files uploaded through Camtasia• Populated by perl script
• Stores user information• Populated using LDAP feed, after WebLogin
• Maintains media information• Populated by user input/video file analysis
Note: A transition to a more comprehensive schema is planned for the next phase of development.
![Page 22: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/22.jpg)
Camtasia Monitor• Camtasia has been setup to
transfer uploaded user videos to our server
• Perl script detects any newly transferred files, creates entry in filedump table
• Media file is renamed using randomized string token
• Accompanying xml file contains file information (e.g. duration, resolution etc.)
• Email sent to user, pointing them to MyMedia application
Comic Source: http://www.agileali.com/myblog/2009/10/02/script-to-see-if-perl-is-working-e/
![Page 23: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/23.jpg)
MyMedia Application
• http://mymedia.library.utoronto.ca• Media administration• Developed in Java/WebObjects• Source can be Camtasia OR direct user
uploads• WebLogin-based authentication, followed by
user profile creation• Media metadata management (e.g. title,
description, keywords etc.)
![Page 24: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/24.jpg)
Media Access Options
• Four profiles offered, analogous to server-side directories with permissions– uoft + stream-only– uoft + download/stream– public + stream-only– public + download/stream
![Page 25: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/25.jpg)
User Uploads/Media Encoding• Alternative to Camtasia• Incoming files encoded using ffmpeg (http://www.
ffmpeg.org/)
• Many supported file formats (e.g. .avi, .mov, .m4v, .m4a .wmv)
"/opt/local/bin/ffmpeg -i /upload-dir/incoming_file -acodec libfaac -ab 128k -ar 44100 -r 20 -vcodec libx264 -b 256000 -s 800x600 -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -me_method umh -subq 5 -trellis 1 -refs 2 -bf 1 -coder 1 -me_range 16 -g 300 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 256000 -maxrate 4M -bufsize 4M -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 21 /encoded-output-dir/output_file.mp4";
![Page 26: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/26.jpg)
Media Viewing• http://media.library.utoronto.ca• Written in PHP• WebLogin layer activated
depending on access permissions of media
• Basic metadata displayed on page below media
• LongTail Video - open source player used for media display (http://www.longtailvideo.com/)
![Page 27: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/27.jpg)
Scenario 1: Camtasia Recording
1. User generates video via local Camtasia Relay Client
2. Media uploaded to Camtasia Server, encoded into mp4
3. Media transferred to stream.library Server via SMB mount
4. Perl script detects file, creates entry in ‘filedumpᾼ
5. Email with link to MyMedia sent to user
7. User edits Media info (stored in ‘t_mediaᾼ)
6. User logs onto MyMedia (via WebLogin)
6b. Profile creation for first-time login (stored in ‘t_userᾼ)
8. Media made viewable at media.library
![Page 28: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/28.jpg)
Scenario 2: Direct File Upload
1. User obtains Media by own means
4. User edits Media info (stored in ‘t_mediaᾼ)
2. User logs onto MyMedia (via WebLogin)
2b. Profile creation for first-time login (stored in ‘t_userᾼ)
3. User uploads Media to Server
5. Media encoded using ffmpeg
6. Media made viewable at media.library
![Page 29: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/29.jpg)
Graham StewartNetwork and Storage Services Manager, Information Technology ServicesUniversity of Toronto [email protected]
![Page 30: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/30.jpg)
MyMedia hardware
• Servers: Apple XServe / Dell PowerEdge
• OS: Mac OS X / Redhat Linux• Storage: Pillar Data Systems SAN• Network: gigabit Ethernet from MyMedia
servers to campus backbone and gateway.
![Page 31: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/31.jpg)
Modular service design• 6 services: front end, upload, conversion, stream, download,
database.• Currently running on 3 servers: can expand to 6 or more as
demand increases.• Target for Fall 2010: file conversion on separate server.• Target for January 2011: full virtualization of most services
allowing for quicker resource allocation. We do virtualization on Ubuntu Linux using KVM/QEMU.
![Page 32: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/32.jpg)
![Page 33: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/33.jpg)
![Page 34: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/34.jpg)
![Page 35: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/35.jpg)
![Page 36: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/36.jpg)
![Page 37: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/37.jpg)
![Page 38: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/38.jpg)
![Page 39: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/39.jpg)
![Page 40: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/40.jpg)
![Page 41: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/41.jpg)
![Page 42: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/42.jpg)
![Page 43: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/43.jpg)
![Page 44: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/44.jpg)
![Page 45: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/45.jpg)
![Page 46: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/46.jpg)
![Page 47: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/47.jpg)
![Page 48: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/48.jpg)
![Page 49: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/49.jpg)
![Page 50: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/50.jpg)
![Page 51: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/51.jpg)
MyMedia components
stream.library
MySQL
Storage Media Conversion
UTORauth
MyMedia
media.library
User: HTTP, HTTPS
Inter-server: iSCSI, NFS, MySQL, SMB
![Page 52: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/52.jpg)
Open Source vs. Proprietary
![Page 53: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/53.jpg)
Rapid application deployment• Sudden requirement• Staff pulled away from other projects• Intensive 4 week schedule• Several versions / iterations• Modular application development and testing• Staff flexibility• Management support
![Page 54: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/54.jpg)
What’s next? • Deepen metadata:
– Preservation metadata (PREMIS)– Rights metadata: more granular– Descriptive metadata (CanCore, IEEE LOM)
• Integrate content:– Build feeds for public content to university search
engines– Build embed tools for selected targets
(FADIS, Blackboard)
• More tools for content creators:– Batch upload– File conversion tailored to purpose
![Page 55: My Media at University of Toronto Libraries](https://reader033.vdocuments.us/reader033/viewer/2022052321/555bf437d8b42a56448b48b9/html5/thumbnails/55.jpg)
Questions?Web address:
http://mymedia.library.utoronto.ca
Media server support:[email protected]
Camtasia video production support:[email protected]
We welcome your input, suggestions and videos!