advances in remoting protocol technology for 3d graphics

© 2016 Citrix | Confidential

Advances in Remoting Protocol

Technology for 3D GraphicsHuman UX Protocol Design Concepts and Experiences

Derek Thorslund

Director of Product Management

NVIDIA GPU Technology Conference, April 2016

Stephen Vilke

Founder of Framehawk


And the innovation continues…Recent product news

• Support for latest NVIDIA cards for GRID 2.x– XenServer 6.5 SP1 HCL; GPU Passthrough and vGPU

• PVS (provisioning) enhancements for vGPU

• 3D Linux desktops with NVENC H.264 encoding

• WAN optimizations for specialty USB devices


Generic USB

Skype

Mobility SDK

ICA

ICA Protocol: Virtual Channels

• HDX is built on the multi-

stream ICA protocol

• Multiple virtual channels

are available for display

remoting (screen graphics)

• Tremendous ongoing

innovation with Thinwire and

FramehawkKB & Mouse

Clipboard

Printing

Audio

Mobile sensorsSmartcard

Flash

CDM

Multimedia

Framehawk

DCR

Thinwire

Display

Remoting


HDX FramehawkWhen user experience is paramount

• Users of desktop virtualization often experienced sluggish

performance on wireless, Internet and long-haul connections

• Global organizations want to consolidate data centers, and

users want to work from anywhere

• Spectral interference is a common problem for mobile users

Framehawk technologies speed up interactivity

over challenging broadband connections


XenDesktop/XenApp HDX 3D Pro over Framehawk

Built into these VDA* versions:XenDesktop 7.8 (current)

XenApp 7.6.3 and above

Citrix Receiver support: Windows Receiver 4.3 minimum

iOS Receiver 6.0 minimum

Remote access support:NetScaler Gateway 11.0-62.10 minimum

Unified Gateway 11.0F-64.34 minimum

* VDA = Virtual Delivery Agent


Case study: HDX 3D Pro over Framehawk at WBCM

• AEC firm headquartered in Baltimore MD

• Architecture & engineering services, plus

construction management and facility construction

• Prestigious accounts including Lockheed Martin,

National Institutes of Health, Ritz-Carlton, US Army

Corps of Engineers

• Continually expanding geographic reach

• Growing complexity of projects



• For several years, Citrix’s industry-leading ICA

protocol with Thinwire display remoting and other

HDX technologies met their needs very well

• But the use cases have become more demanding…– More graphics-intensive applications

– Larger 3D models / huge files

– Longer distances (high latency)

– Wireless connectivity to job sites (packet loss)

– Architects and engineers want flexibility in work location;

“necessary to retain or recruit” and cope with winter storms


Case study: Meeting user needs at WBCM

• NVIDIA GRID vGPU hardware acceleration– Autodesk AutoCAD incl. Plant 3D, Civil 3D

– Autodesk Revit

– Bentley MicroStation

– etc. (even the browser is faster!)

• Goliath Technologies Performance Monitor

• XenDesktop with HDX 3D Pro and the new

Framehawk virtual channel for display remoting


Case study: Identifying the need for Framehawk at WBCM

• Goliath Technologies

Performance Monitor

• Users in California on

3G/4G cards were

experiencing high

network latency (peaks

of over 300ms)

• Performance Monitor

enables near real-time

troubleshooting



• Big improvement in user experience (“it made a big

difference”, “like night and day”)

• WBCM were able to set up two remote offices “with

little extra capital or skill needed”; no local

infrastructure, just terminals, printers and a firewall

• Easy provisioning of new employees

• Large data transfers as apps load happen in

seconds within the data center, and IP is secure

• Employees are work effectively from anywhere


More feedback on Framehawk

• Real estate investment trust– “Working in XenDesktop over WAN and working at 35% packet loss! Welcome to the future!”

– “I love how @citrix #framehawk handles scrolling and redraws under adverse conditions.”

• Automotive manufacturer with HQ in Detroit– 3D apps over WAN connection from North America to Asia-Pacific at 500 ms roundtrip latency

– “While the difference between LAN and WAN was perceptible, it was barely so.”

• Aerospace manufacturer– “We’re using Framehawk with Siemens Teamcenter NX on XenApp between India and the

United States at 450ms roundtrip latency!”

• Architecture, Engineering & Construction firm in Texas– “Very impressive results… Our Spain users are happy... In fact, it is the only way they can be

productive at this point. Hands down, Framehawk is a game changer.”


Cost reduction

(replace something expensive with

something less expensive)

Vs

Productivity

(replace something productive with

something more productive)

Business Cases – Different Drivers

Guaranteed access

Prioritize: thrifty

Real-time Productivity

Prioritize: fluidity and click response

Bottom line management

Increase revenue

Framehawk


Productivity UX – Visual Requirements

1. Fluidity / FPS

2. Linearity

3. High Fidelity

4. Click ResponseResilient in ALL networks


What are the root causes of most UX issues when you want

Real Time Response?

? ?

Sloooow

Network


History: Lossy & Latency Networks

Question: What was the traditional source of network issues?

Congestion


Industry Response to Congestion


Result: Diminishing Returns


Today = New “weakest link”


Spectral Loss

Due to Interference


Common Compression Strategies:

Formulaic AND Sequentially Dependent

Raw Data

Native App/HTML5 - 00100101001001010011110001

x.264 streaming, etc

Graphics commands

Render

CompressRender |

Content & Layout

Content & Layout

g(x) & Content & Layout (reprioritized)

Clie

nt/

Serv

er

Sp

litS

tream

Raw Data Or Content

Broadcast

jpeg streaming, commands etc

Compress, evalRender |Content & Layout

Bitm

ap

Bitmap Caching

Network compression

Server-side

compression

Client-Server

compression


Click…....click.....click.......clickclickclickclick

Loss + Latency = “Rubberband effect”

i.e. Was that my intent?

a b c e f g i j k abcefgijk….d…h = abc – waiiiiit – defg – waiiiiiit - hijk


How to fix the “Rubberband Man” and get Real-Time

1. Break the dependency on

sequential delivery.

2. Solve the disconnect between

the user and the server = “Intent”


Real-time Rule #1:

Don’t use compression as

delivery mechanism.

Use

self-healing

transforms


Real-time Rule #2:

Don’t retransmit if

you don’t need to!


"Aoccdrnig to a rscheearch at Cmabrigde

Uinervtisy, it deosn't mttaer in waht oredr the

ltteers in a wrod are, the olny iprmoatnt tihng is

taht the frist and lsat ltteers be at the rghit

pclae. The rset can be a toatl mses and you

can sitll raed it wouthit porbelm. Tihs is bcuseae

the huamn mnid deos not raed ervey lteter by

istlef, but the wrod as a wlohe.”

Real-time Rule #3: Leverage Human Tolerance!


Learning: human eyeballs are amazing!

…..so we created one on the server side


Identify “scoll-y” vs “video-y” vs “read-y” regions


Gear mechanism


Always Be Real-Time Video


2) Where is the “Intent”?

Raw Data

Native App/HTML5 - 00100101001001010011110001

x.264 streaming, etc

Graphics commands

g(x) & Content & Layout (reprioritized)

Raw Data Or Content

Broadcast

jpeg streaming, commands etc

Bitmap Caching

Clie

nt/

Serv

er

Split

Str

ea

mB

itm

ap

Native apps


Who is the target of the experience?

✔✗

(What is more important – Right or Relevant?)


Intent Engine: Create a shared mind between client and server


Focus on What’s Important


Command and Control Video


The Future of Productivity-focused Remoting Protocols

1) Be Real-time, always

2) Let humans own control!!


Work better. Live better.Work better. Live better.

advances in remoting protocol technology for 3d graphics

Documents