smart meeting systems josh reilly. why are smart meeting systems worth studying?
TRANSCRIPT
![Page 1: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/1.jpg)
Smart Meeting Systems
Josh Reilly
![Page 2: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/2.jpg)
Why are Smart Meeting Systems worth studying?
![Page 3: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/3.jpg)
Objectives of a Smart Meeting System
Improves the productivity of a team by automating the:
– Capture of the meeting
– Processing of the meeting for valuable information
– Displaying of that information accurately and effectively to the end user through a client application
![Page 4: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/4.jpg)
Organization of Smart Meeting System Processes
A smart meeting system can be decomposed into three sets of processes
Meeting Capture Meeting Recognition Semantic Processing
![Page 5: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/5.jpg)
Organization of Smart Meeting System Processes
![Page 6: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/6.jpg)
Meeting Capture
Gathering raw inputs from the meeting Video Capture Audio Capture Other Context
![Page 7: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/7.jpg)
Video Capture
Video feeds from: Cameras for the attendees
Could use a single static camera Could use a single camera with pan, tilt, zoom (PTZ)
capabilities Recommend camera view of every contributor's face
Visual Aids Separate camera Digital feed from device
![Page 8: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/8.jpg)
Microsoft Distributed Meetings Project Camera Placement
![Page 9: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/9.jpg)
Microsoft Distributed Meetings ProjectVideo Capture
RingCam Array of 90º Cameras 360º Panoramic view
![Page 10: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/10.jpg)
Audio Capture
Use an array of microphones Placed on the table Placed on the ceiling Worn on the person
Levels need to be controlled so that they are similar levels for each contributor
![Page 11: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/11.jpg)
Microsoft Distributed Meetings Project Audio Capture
RingCam Has an array of
microphones on its base.
![Page 12: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/12.jpg)
Other Context Capture
RFID to track attendees Attendees swipe their RFID cards when they enter
the meeting to add their ID to the list of people attending this meeting
Motion Detectors to track the locations of attendees within the room
![Page 13: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/13.jpg)
Organization of Smart Meeting System Processes
![Page 14: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/14.jpg)
Meeting Recognition
The processing of the raw capture before it is organized into something useful
Steps: Person Identification Attention Detection Activity Recognition Hot Spot Recognition Summarization
![Page 15: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/15.jpg)
Person Identification
Person Identification
is associating sections of video, audio, and the visual aids that were captured from the meeting with the attendee(s) that they belong to
Face Recognition Face Tracking Speech Recognition SSL Beamforming
![Page 16: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/16.jpg)
Person IdentificationFace Recognition
Facial Recognition Identify the person speaking from a list of attendees Eigenface Approach Challenges
Poor Quality Images Poor Room Lighting Continuously changing facial expressions Occlusion
![Page 17: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/17.jpg)
Face RecognitionThe Eigenface Approach
All faces are assumed to be made up of different percentages of different eigenfaces
A set of eigenfaces is a set of very generalized pictures of faces that were generated so that each has a basic ingredient that can be used to make a face Eigenfaces from AT&T
Laboratories Cambridge
![Page 18: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/18.jpg)
Person IdentificationSpeech Recognition
• Speech Recognition
• Match the voice of the person speaking to someone on the list of attendees
• Using Voice recognition in conjunction with face recognition allows for an accurate identification of the speaker
• Sound Source Localization (SSL)
• Used to determine which camera is pointed at the speaker
• Could be used to point PTZ camera
• Beamforming
![Page 19: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/19.jpg)
Person IdentificationWriter Recognition
Writer Recognition When someone writes on the whiteboard, they may
not be in clear view of the cameras Writing recognition algorithms can be used to
identify who wrote what during a meeting
![Page 20: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/20.jpg)
Attention Detection
Attention Detection Attempt to determine who is looking at whom during
a meeting. Provides information used for activity recognition
and hot spot recognition Done using:
Hidden Markov Models (HMM) Sound Source Localization (SSL) Known layout of room
![Page 21: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/21.jpg)
Activity Recognition
Determine what is happening during the meeting Step 1:
Determine what each individual is doing at each point during the meeting
Person Identification, Attention Detection, SSL, Gesture Recognition
Step 2: Take that information to determine what activity the
entire group is engaging in at each point during the meeting
![Page 22: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/22.jpg)
Hot Spot Recognition
Find the important parts of the meeting Using sound queues
Ex: Changes in pitch Using activity recognition
When people are nodding When their focus changes
![Page 23: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/23.jpg)
Summarization
Takes all of the information that the smart meeting system has learned about the meeting and creates a quick overview of the events that took place during that meeting.
This information will be used in the semantic processing stage
![Page 24: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/24.jpg)
Organization of Smart Meeting System Processes
![Page 25: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/25.jpg)
Semantic Processing
Takes the information from the meeting recognition step and makes it usable by the end user.
Meeting Annotation Meeting Indexing Meeting Browsing
![Page 26: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/26.jpg)
Meeting Annotation
Describe the raw data from the meeting from each viewpoint
Attempt to label all meeting segments Implicitly
Automatically Explicitly
By Hand
![Page 27: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/27.jpg)
Meeting AnnotationImplicit
Automated Annotation Assumes that the meeting recognition processes
performed with relatively high efficiency Tags every person in the video Narrates what was happening during the meeting Has not been achieved
![Page 28: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/28.jpg)
Meeting AnnotationExplicit
Annotation By Hand When the recognition processes fail to gather
sufficient correct information about the raw data Users will have to go through the meeting and tag
the people attending as well as indicate what events are happening all through the meeting
![Page 29: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/29.jpg)
Meeting Indexing
Indexing is done at all levels of data from a raw audio feed to the annotations
The best form of indexing to use is the event-based indexing
An index is created every time an event occurs This is the best way for users to find a specific spot
in the meeting when performing a query
![Page 30: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/30.jpg)
Meeting Browsing
The interface that the end user uses to retrieve information from the meetings
Functions: Can browse/search a list of all meetings for a
specific meeting Can browse/search the contents of the chosen
meeting Aided by tools like bookmarks, a meeting outline, and
queries (content, people, camera angles, visual aids, etc...)
![Page 31: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/31.jpg)
Meeting BrowsingMicrosoft Distributed Meetings
![Page 32: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/32.jpg)
Remote Attendee
Use the smart meeting system as the attendee's eyes and ears
Microsoft's PING project Uses a monitor and speaker to display the remote
attendee's voice and audio during the meeting However, the remote attendee is often ignored
![Page 33: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/33.jpg)
Carnegie Mellon University’sMeeting System Architecture
Lacks•Activity Recognition•Hot Spot Recognition•Annotations
![Page 34: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/34.jpg)
University of California, San DiegoAVIARY System Architecture
• 2 PCs• 4 Static Cameras• 4 PTZ Cameras• No SSL
![Page 35: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/35.jpg)
RicohPortable Meeting Recorder
![Page 36: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/36.jpg)
RicohPortable Meeting Recorder
Doughnut Camera
![Page 37: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/37.jpg)
RicohPortable Meeting Recorder
Meeting Browser
![Page 38: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/38.jpg)
Technology Limitations
Speech recognition and facial recognition algorithms are not yet as efficient as they should be in order for a smart meeting system to perform accurately
![Page 39: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/39.jpg)
Workspace Limitations
Cameras and microphones can block view, distract, or intimidate attendees during the meeting
Security and Privacy needs to be addressed
![Page 40: Smart Meeting Systems Josh Reilly. Why are Smart Meeting Systems worth studying?](https://reader038.vdocuments.us/reader038/viewer/2022110401/56649de95503460f94ae4585/html5/thumbnails/40.jpg)
References
[1] Zhiwen Yu and Yuichi Nakamura. 2010. Smart meeting systems: A survey of state-of-the-art and open issues. ACM Comput. Surv. 42, 2, Article 8 (March 2010), 20 pages. DOI=10.1145/1667062.1667065 http://doi.acm.org/10.1145/1667062.1667065
[2] Ross Cutler , Yong Rui , Anoop Gupta , Jj Cadiz , Ivan Tashev , Li-wei He , Alex Colburn , Zhengyou Zhang , Zicheng Liu , Steve Silverberg. (2002). Distributed Meetings. A Meeting Capture and Broadcasting System. 10 pages. http://research.microsoft.com/en-us/um/people/yongrui/ps/mm02.pdf
[3] Harold Fox. 2004. The eFacilitator: A Meeting Capture Application and Infrastructure. 89 pages. http://hdl.handle.net/1721.1/17672
[4] Yong Rui, Eric Rudolph, Li-wei He, Rico Malvar, Michael Cohen, Ivan Tashev. 2006. Ping: A Group-To-Individual Distributed meeting System. 4 pages. http://research.microsoft.com/apps/pubs/default.aspx?id=76779
[5] Dar-Shyang Lee, Berna Erol, Jamey Graham, Jonathan Hull, Norihiko Murata. 2011. Portable Meeting Recorder. 10 pages. http://rii.ricoh.com/sites/default/files/Portable_Meeting_Recorder.pdf