Video in CSCW

Part I—Michael Boyle

Part II—Michael Rounding

Part I—Video Mediated Communication

Role of visual channel in collaboration Video-mediated communication

technologies Opportunities and applications of VMC

Role of Visual Channel

Significance Function and operation of visual

behaviours Technological requirements Consequences of inadequate support

Significance of Visual Channel

Principle means for navigation and orientation in our 3D physical world Hand-eye coordination and motor skills

Principle means for experiencing our 3D physical world Other senses are limited Greatly inconvenienced if disabled

Characteristics of Visual Channel

Humans can only focus on small area Large peripheral visual cone

Humans are adept at handling vast quantities of visual information Scanning, searching, reacting to changes Recognizing, classifying

Predominantly diurnal creatures Large eyes; eyebrows; stereoscopic vision

Uses of Visual Channel

Infer much about people from visual information Gender, age, status Identity, background, personality

Bulk of human knowledge encoded visually

Arts, aesthetics, entertainment Communication

Visible Behaviours

Gaze Facial expressions Gestures Posture

Gaze—Function and Operation

Indicates locus of interest, activity One generally looks at something when one

talks about it or is about to use it Uses: grounding, workspace awareness

Signals affective state Interest, disposition, attitude, emotion Uses: pre-interaction informal awareness,

conversational awareness

Gaze—Function and Operation

Conveys extremely subtle yet significant content Infer big meaning from even small physical

changes Although important, not always observed

Observed <7% of conversation time Mutual gaze (eye contact) <5% of time

Gaze—Technological Requirements

Careful camera position Eyes must be fully visible, aperture at eye-level

Large field of view See point of interest

High image resolution Clearly see orientation, percieve minor variations

High frame rate, low latency Smoothly convey rapid changes in real-time

Gaze—Consequences of Inadequate Support

Systemic miscommunication, misinterpretation if undiagnosed

Requires extra effort to explicitly communicate signals, or disambiguate/repair miscommunication

De-referencing focal points is tough!

Facial Expressions—Function and Operation

Involves whole of the face Eyes, eyebrows, forehead, mouth (especial

lips and corners), cheeks, ears, nose Conveys affective information

Blinks, winks, smiles, frowns, furrowed brows, wrinkled up nose, etc.

Uses: conversational awareness, disambiguate meaning of sensitive or easily-misinterpreted content

Facial Expressions—Technological Requirements

Tight audio/video synchronization Face must be in full view with high

resolution Fast frame rate to capture winks, etc.

Facial Expressions—Consequences of Inadequate Support

If latencies are too high, then signals may come at inappropriate times

If resolution too low, may not be able to clearly discern expression, adding to ambiguity

Could lead to break-down of conversation if important expression miscommunicated

Gestures—Function and Operation

Uses mostly hands, fingers, and arms Position, orientation, movement Position relative to other parts of the body, or

other objects in environment Rapid, generally small movements

Context crucial for interpretation Silent, subtle, implicit long-distance

communication

Gestures—Classes

Pointing gestures Grounding: de-reference deictic references

Emblematic or iconic Convey content Useful when words don’t come to mind Sometimes redundant

“Beats” Convey emphasis

Gestures—Technological Requirements

Large field of view Full body must be visible

High image fidelity Resolution, frame rate high enough to capture

small, rapid motions Likelihood that there will be obstructions

Fixed, mono-camera configuration insufficient

Gestures—Consequences of Inadequate Support

Miscommunication, resulting in explicit communication

Misinterpretation of gestures could lead to conversation breakdown

Missed deictic references impairs workspace awareness

Loss of personable content

Posture—Function and Operation

Consequence of work Involves whole body

Particularly head, neck back, hips Position, relative orientation are cues to

look for Communicates affective information

Particularly availability: informal awareness

Posture—Technological Requirements

Field of view Must capture whole body yet provide

background for reference Fidelity doesn’t need to be high

Posture—Consequences of Inadequate Support

Incorrect assessment of availability during pre-interaction periods Unnecessary interruptions, missed

opportunities Erroneous assessment of affective

information Impact on smoothness, but likely not so

terrible

Visible Behaviours—Recap

Subtle yet significant, sometimes redundant

Signal attitude, disambiguate meaning Valuable personable aspect Conflicting technological constraints People can get by without them, but

interaction dynamics are fundamentally altered

Part I—Video Mediated Communication

Role of visual channel in collaboration VMC technologies Opportunities and applications of VMC

A/V Infrastructure

Cameras and displays Digital vs. analog

Microphones and speakers Digital vs. analog Half-duplex vs. full-duplex Microphone arrays, environmental audio Noise, feedback, attenuation

Network Infrastructure

Medium Digital vs. analog Dial-up, ADSL, T1, wireless

Quality of service (QoS) factors: Bandwidth, latency, jitter, delivery guarantees

Packet-switched vs. circuit-switched Broadcast/multicast vs. unicast

Audio Software

Properties: sample size, rate, number of channels

Compression: PCM, µ-Law, ADPCM, GSM/CELP, MP3

Buffer size: transmission overhead vs. latency

Filtering, mixing

Video Software

Lossy vs. lossless compression RLE (run-length encoding): lossless

replace runs of same-coloured pixels with count JPEG: lossy

uses signal-processing techniques to change representation of image to bring out redudant aspects

quantization throws away resolution in chroma channel not visible to human eye

entropy (Huffman) coding on result

Video Software

Intraframe vs. interframe compression “M-JPEG:” intraframe MPEG: intraframe/interframe hybrid

JPEG compress key frames Encode back and forward differences of non-key

frames Decreases bandwidth, increases latency

Quality: bits-per-pixel

Video Software

Frame rate Frame size

General Impact of Inadequacies

Frame size, resolution, fidelity Subtle facial expressions, gaze not

discernable Lost of critical eye contact

Field of view Locus of attention not visible Parts of face, body used in expressions not

visible

General Impact of Inadequacies

Frame rate Subtle changes lost

Latency Cues come at inappropriate time

A/V synchronization Reduces value of redundant encoding

VMC Technology—Recap

Technology never perfect Bandwidth/quality trade-off Results:

Content communicated subtly and implicitly via visual channel must be communicated explicitly via some other channel, or lost altogether

People adapt, but personability of FTF is lost

Part I—Video Mediated Communication

Role of visual channel in collaboration VMC technologies Opportunities and applications of

VMC

Opportunities and Applications

Tele-presentation/distance learning Augmenting shared workspaces Supporting informal interaction Tele-presence and mediating contact

Function, operation, example systems, and lessons learned

Tele-presentation

Presenter/audience with presentation matter

Numerous configurations: with/without local audience multiple participants at each remote site multiple presenters at same/different sites

Formal Information dissemination applications

Tele-presentation

Example systems: Forum Telep TV call-in show

Tele-presentation—Lessons Learned

Audience likes it Enables participation Integrates well with other work tasks

Presenters not so favourable Value of back-channels Reduced local attendance

Sub-conversations

Augmented Shared Workspaces

Coordinating activities in shared workspace requires communication channels E.g., text-chat or telephone

Integrate shared workspace and shared communication channels to create a shared tasks environment

Traditional meeting tasks

Augmented Shared Workspaces

Example systems: Montage ClearBoard TeamWorkstation NetMeeting

Augmented Shared Workspaces—Lessons Learned

Diminished priority of video relative to audio

Call setup: minimize time, provide sense of approach

Requirements vary with task Critical mass Know your users

Casual Interaction Support

Promote social interactions of serendipitous nature Based on value of social interactions to

productivity “Seamless” integration of distant sites into

a signal interaction space Hope to overcome effect of distance on

social interactions and collaboration

Casual Interaction Support

Example systems: VideoWindow Vkitchen CRUISER

Casual Interaction Support—Lessons Learned

Public reaction mixed Not 100% transparent: technology still a

barrier to forging new relationships Difficult to diagnose failures

Requires extended installation/observation periods

Tele-presence and mediating contact

Video makes for a subtle, unobtrusive means of delivering tele-presence for accruing distributed informal awareness

Easy, natural for people to capitalize on visual information to coordinate interactions

Summary

Role of visual channel in collaboration Technological foundations Opportunities for video in CSCW Challenges

Summary—Role of Video

Communication medium Coordination medium

Regulation and mediation of interactivity Subtle, implicit Important, but not in the obvious ways

Importance varies with task

Summary—Video Technologies

Present technology not very robust Bandwidth/fidelity trade-off Issues: latency, field of view Consequences of inadequate technology

Vary from minor inconvenience to conversation breakdown

Must understand when video is crucial Design should reflect this

Summary—Opportunities

Tele-presentation Meeting support Communication channel for shared work

activities Casual interaction Mediating interaction

Summary—Challenges

Understanding where video succeeds and fails Success is not always obvious Some failures are inevitable

Basic technological issues Deployment, ubiquity, acceptance,

incorporation Social issues: privacy, accountability