Immersive Telepresence and how user

experience becomes a key part of the

overall picture

Patrick Luthi

Manager, Technical Standards

TANDBERG, now part of Cisco

ITU-T Rapporteur for Question 1/16 (multimedia terminals, systems and dataconferencing)

Telepresence systems

2

Telepresence systems (2)

3

Current TANDBERG solution basics

� Immersive: See eye-to-eye with crystal clear, natural 1080p video shared on 3 large 65" screens with inverted cameras for better eye-contact.

� Three codecs� Multiple H323 calls (H.264 video, AAC-LD audio)� External control unit which controls all three codecs� Proprietary solution for signaling left and right codecs� Proprietary solution for signaling left and right codecs

Site ASite A

Site BSite B4

ProsProsProsPros

Pros and Cons for current solution

Easy to implementEasy to implement

Easily extensibleEasily extensible

Few components needed to Few components needed to changechange

ConsConsConsCons

ProprietaryProprietary

Long call setupLong call setup

Multiple calls = multiple licensesMultiple calls = multiple licenses

•• CallsCalls

••changechange

Good solution for firewall Good solution for firewall traversaltraversal

General interoperability with General interoperability with nonnon--telepresencetelepresence systemssystems

Works very well together with Works very well together with TelepresenceTelepresence--specific MCUspecific MCU

•• Firewall traversalFirewall traversal

•• RegistrationsRegistrations

Hard to do bandwidth controlHard to do bandwidth control

Goes against normal call setupGoes against normal call setup

Each camera maps to 1 and only Each camera maps to 1 and only 1 output screen.1 output screen.

5

Cisco’s current solution: video flow between triple screen

systems

VideoVideoRTP SessionRTP Session

CTS 2CTS 2CTS 1CTS 1

� Up to 4 Video Streams� Center, Left and Right Camera = 3 Video streams� Data Video = 1 Video stream

� All Video Streams share 1 common RTP Connection� Each Camera stream is sent to the corresponding Display

oror

6

First step – Make sure telepresence works

7

Telepresence standardization efforts

� Dangers and challenges ahead� Lack of telepresence interoperability� Different and incompatible media codecs� Different signaling mechanisms� How to deal with multiple video streams� What about multiple multichannel audio streams

� IETF� New work item in Dispatch WG or creation of a new WG� Multiple streams control mechanism� Specify the information about media streams from one endpoint

to another endpoint� Information between sources and sinks about media stream

capabilities

8

Telepresence Standardization Efforts (2)

� ITU-T New Question 5/16 “Telepresence Systems”� Full interworking between telepresence systems� Standardizing the means for interworking between current

telepresence systems and other systems through additions to H.246 and other Recommendations

� IMTC� Telepresence Activity Group (AG)� Telepresence Activity Group (AG)

� Specify requirements and use cases for multiple media streaming as input to SDOs

� Telepresence Interoperability Protocol (TIP) AG� Management of changes and enhancements to TIP� Improving interoperability of telepresence and videoconferencing

systems that implement TIP� Conferencing and Telepresence Interoperability AG (to be created)

� Definition of test cases and scenarios that focus on telepresence and existing systems interoperability

9

Telepresence User Experience

� Beyond ”HD” technologies: the feeling of being in that other location� Large monitors� Identical furniture at each site� Neutral background and

special lighting

10

Telepresence system, presenting content from PC

Telepresence User Experience

� Collaboration

Telepresence User Experience

Works well for scheduled callsWorks well for scheduled calls

But Telepresence is becoming personal, usage is But Telepresence is becoming personal, usage is

becoming adbecoming ad--hochoc

Telepresence system, presenting content from PC

12

� Collaboration

Telepresence User Experience

� Collaboration:� More than presentations� The ability to manipulate a

remote object or environment

13

What’s Next?

� Multiple media streams� Collaboration and interaction� User interfaces -� Video quality beyond HD� Personal and mobile telepresence

14

What’s Next?

� Multiple media streams� Collaboration and interaction� User interfaces� Video quality beyond HD� Personal and mobile telepresence

15

What’s Next?

� Multiple media streams� Collaboration and interaction� User interfaces� Video quality beyond HD� Personal and mobile telepresence

16

Video Coding Technologies – H.264 and beyond

� Earlier:

� 1989: H.261� 1993: H.262 (MPEG-2)� 1996-2000: H.263 (+, ++)� 1996-2000: H.263 (+, ++)� 2003: H.264 (MPEG-4 Part 10)� 2006: H.241 Annex B, Reduced Complexity Decoding for H.264 (RCDO)

� Today:

� H.264 720p60� H.264 1080p30� H.264 1080p60 – the next logical step

17

What’s next? Video coding technologies of the future?

� Very high resolutions – H.264 up to 4096 x 2048� High Efficiency Video Coding (HEVC) – H.265

� ITU/MPEG’s JCT-VC (Joint Collaborative Team on Video Coding)

� 2nd meeting in July 2010 with over 100 contributions and 220 participants

� 2x coding efficiency, parallelism, high error resilience, low complexity profile, etc.complexity profile, etc.

� Approval is targeted for 2012-2014

� 4k video resolution (4096 x 2160)� Digital cinema technologies: Ultra HD is four times as wide and four

times as high as HD, producing a 7680 × 4320 pixel resolution.� Ultra HD is 16 times the pixel resolution of standard HD

� 8k video resolution (8192 x 4320)

18

Video coding technologies – What’s really next?

� H.265 goal: 50% of H.264 Bandwidth

19

Conclusion

� Telepresence is changing the way people communicate, but this is still not a very big market!

� Without interoperability, telepresence will fail.

� We need even more efficient video codecs – telepresence studios of the future will be even more bandwidth hungry than todaythe future will be even more bandwidth hungry than today

� The design of the room and furniture, the eye-to-eye connection, the clarity of video and the crisp audio are integral parts of the user experience.

� It easy to forget that the other participants aren’t actually in the same room.

20

Thank you!

21