wlcd: a web lecture capture system with robotic speaker tracking
DESCRIPTION
WLCD: A Web Lecture Capture System with Robotic Speaker Tracking. Steven Goldfarb for Jeremy Herr CHEP Mumbai, India - 15 Feb 2006. Web Lecture Capture Device. The Web Lecture Archive Project. Partnership University of Michigan UM ATLAS Collaboratory Project Department of Physics - PowerPoint PPT PresentationTRANSCRIPT
Steven Goldfarb for Jeremy HerrCHEP
Mumbai, India - 15 Feb 2006
WLCD: A Web Lecture Capture System with Robotic
Speaker Tracking
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 2
Partnership University of Michigan
• UM ATLAS Collaboratory Project• Department of Physics• Media Union
CERN• Technical & Acedemic Training• Summer Student Program• IT
American Physical Society
Primary Goals implement an electronic archival system for slide-based
presentations on the Internet; assess the ability of the system to improve upon and complement
existing archival methods; determine the resources required to install and maintain the
system as a standard archival service.
The Web Lecture Archive Project
Web Lecture Capture Device
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 3
Web Lecture Recording ATLAS & LHC In General
• Physics & Software Tutorials• ATLAS Week plenary sessions• CERN Training Seminars, Tutorials
American Physical Society• Opportunities for Physics in Biology 2002, 2004• April Meeting 2004
Fermilab• Software Tutorials
University of Michigan• Saturday Morning Physics• Medical School Symposium on Clinical Research 2004
Harvard• International Conference for Systems Biology 2005
Current Recording Activities
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 4
Motivations Driving Automation Simultaneous Recording of Multiple Lectures Limited Manpower Need for Instant Publication
Components of Lecture Archiving Requiring Automation Video Capture and Compression Audio Recording Slide Capture and Synchronization Construction, Publication of Electronic Lecture
Development of Automated Recording Technology
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 5
Brief History of Lecture Recording Automation 1999: Used Sync-O-Matic 3000 Software to Record CERN Summer School Lectures
• Sync-O-Matic completely automatic if Installed on presentation computer Slides in Powerpoint
• But Everybody wrote plastic transparencies back then! And access to presentation computers not always possible
• So Had to record timing by hand, process lectures after the talk.
2000: Extra video camera was pointed at the screen• timing information was collected manually afterward
2001: Giosue Vitaglione wrote CarpePPT• Same idea as Sync-O-Matic, but producing Lecture Objects (non-proorietary)
2002: Jake Bourjaily wrote CarpeLecture• Extension, allowing PDF and integrated with CDS Agenda
Now: Web Lecture Capture Device• VGA splitter connected to signal coming from presentation computer• high-resolution capture card is used with change-detection software
captures every image displayed on the speaker’s computer No limit to format (PowerPoint, PDF, Web pages, command line prompts,…)
Development of Automated Recording Technology (2)
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 6
Lecture Object Standardize Storage of Web Lecture Ingredients, Recipe
• Archive media in standard formats (mpeg-4, jpeg, etc.)• XML extension to SMIL (Synchronized Multimedia Integration Language)
Describes slide timing, media location, other information SMIL = W3C Standard
• Add metadata with RDF (Resource Description Framework) RDF = W3C Standard
Standard Procedure for WLAP
Recall the Lecture Object (Talk before Coffee)
mpeg-4
jpeg
XMLDescription
meta data
Lecture Object
Transform
ations
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 7
Lecture Object - Definition
Organized Directory Structure and Naming System
Contents
Audio/Video media: MPEG-4 video file
Slides: JPEG images of everything displayed on lecturer’s screen
Supporting Documents: original PowerPoint file, animations, etc.
Log Files: describing technical history of and changes to media files
XML file: a simple format similar to SMIL, following commonly used international
standards and customized to effectively describe a typical lecture. contains metadata fields using subsets of Dublin Core and IEEE Learning
Object Metadata, describing media data timing and synchronization information
Future Directions Allow more media streams (animations, data displays, chalkboards) Allow annotations, notes, links to other resources Transformations to more viewing objects: Quicktime, Media Player, iPod, etc.
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 8
Lecture Object - Example
Example of Lecture Object using current standard
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 9
Robotic Camera Tracking
Human camera operator Intelligent tracking that generates pleasing video
Very expensive, gets tired after several hours
UltrasonicDeveloped by AT&T labs
Probably accurate enough for tracking a lecturer
Requires extensive permanent installation
Radio Frequency (RF)BlueSoft
Almost reasonably priced (6,000 USD)
Slow and insufficient accuracy, especially indoors
Ultra Wide Band (UWB)UbiSense
Almost reasonably priced (10,000 USD), more accurate than RF
Requires extensive set-up and calibration
“passive” IRUsed in Boeing factories
Inexpensive (4000 USD), simple and very accurate
Very easily distracted by any reflective objects in the room
System Requirements Portable Robust Affordable Runs without expert intervention Little setup or calibration
Survey of Existing Technologies
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 10
Current solution: “active” IR system using dual-cameras and infrared necklace Target wears necklace made of IR LEDs Invisible to human eye, but very bright in IR Two cameras mounted together
• IR camera tracks target• visible-light camera provides color video
This system satisfies our criteria Portable: sits on a cart Robust: simple design makes it very robust Affordable: currently under 4,000 USD No expert intervention: start it and it just works Little setup: almost no calibration required Accurate to within centimeters
Robotic Camera Tracking – Current System
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 11
Remaining Challenges Rooms with windows: on bright days, excessive sunlight can
increase background noise and illuminate reflective objects Other IR sources
• Tablet PC’s use bright IR pens• incandescent light fixtures give off a lot of IR• transparency projectors give off a lot of IR
Improvements - Next system: machine-vision camera Stationary, wide-angle camera Frame rate, exposure time, timing are all controllable The necklace strobes
• identifies necklace uniquely to the machine vision camera• saves battery life
We are building and testing this system now This should greatly improve reliability in adverse conditions
Robotic Camera Tracking – Future Development
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 12
March 2005 Provisional patent filed, describing the current dual-camera with
active IR necklace.October 2005
University of Michigan researchers filed 287 invention disclosures in 2005
Eight of these were selected to be featured at the 2005 Tech Transfer Inventor’s Reception, as the most commercially promising and world-changing.
Our tracking system was one of these eight.
March 2006 Permanent patent will be filed.
Technology Transfer
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 13
Plans for Using the WLCD Enable automated large-scale recording, streaming and archiving
of university courses Record and disseminate multiple parallel sessions at large
professional society meetings Facilitate the preparation of materials for distance learning Generate supplemental instructional content for outside-of-class
review Provide services for recording of sample lectures given by faculty
candidates Recording of ATLAS Plenary Sessions Many more ...
Applications
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 14
Now - Physics 140 test Big impact: over 500 students each semester, including pre-
medical, engineers, science majors. Recording with current tracking system will begin March 2005 Two 1-hour lectures per week, each available through U-M
CourseTools within hours, and also podcast to student’s iPods. Chalkboard work will be captured. Entire system fits on one cart.
Fall 2006 – Pilot Project begins Project has support of the Provost and many academic units Three different classes will be recorded Student usage will be analyzed through surveys and log
monitoring by education experts in the Center for Research on Learning and Teaching (CRLT).
Project will continue through Summer 2006
Hot Item: U-M Campus-Wide Classroom Recording
Mumbai, India - 15 Feb 2006 J. Herr - University of Michigan WLCD - Slide 15
Demo video made with tracking system in May 2005
Tracking camera video of Jen McCormick made in December 2005
Web Lecture for iPod - David Gerdes, UM Physics Professor
Example Videos