![Page 1: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/1.jpg)
Application and Desktop Sharing
Omer BoyaciNovember 1, 2007
![Page 2: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/2.jpg)
Overview
• Introduction• Demo• Architecture• Challenges• Features• Conclusion
![Page 3: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/3.jpg)
Application Sharing Models
• Application specific+ Efficient- Participants need application- Application has to be modified
• Generic- Inefficient (sometimes)
+ Participants don't need application
+ All applications are supported
![Page 4: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/4.jpg)
Application Sharing Models
• Application specific+ Efficient- Participants need application- Application has to be modified
• Generic- Inefficient (sometimes)
+ Participants don't need application
+ All applications are supported
![Page 5: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/5.jpg)
Overview
• Introduction• Demo• Architecture• Challenges• Features• Conclusion
![Page 6: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/6.jpg)
Application Sharing
Sharing an application with multiple users
There is only one copy of the application
Participants do not need application itself
Briefly, participantsreceive screen updatessend keyboard and mouse events
Desktop sharing is also supported.
![Page 7: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/7.jpg)
Screenshot
![Page 8: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/8.jpg)
Screenshot (2)
![Page 9: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/9.jpg)
Screenshot (Overlapped Windows)
1
23
4
![Page 10: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/10.jpg)
Screenshot (Multicast App. Tool)
1
23
4
4
![Page 11: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/11.jpg)
Screenshot (Ultra VNC)
1
2 3
4
4
![Page 12: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/12.jpg)
Screenshot (App & Desk Sharing)
1
2
4
![Page 13: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/13.jpg)
Supported Platforms/OS
Server Client*Windows + +*nix +Mac OS X - +
-+
*Client is Java based.
![Page 14: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/14.jpg)
Overview
• Introduction• Demo• Architecture• Challenges• Features• Conclusion
![Page 15: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/15.jpg)
Demo
Demo
![Page 16: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/16.jpg)
Overview
• Introduction• Demo• Architecture• Challenges• Features• Conclusion
![Page 17: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/17.jpg)
System Architecture
• Client/Server Software Architecture
![Page 18: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/18.jpg)
System Architecture
• Client/Server Software Architecture
Screen Updates
![Page 19: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/19.jpg)
System Architecture
• Client/Server Software Architecture
Keyboard Mouse Events
![Page 20: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/20.jpg)
Client (Viewer) Architecture
• Client can– Connect to server– Wait for incoming connections
• Client supports– TCP– UDP (+Multicast)
![Page 21: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/21.jpg)
Client (Viewer) Architecture
• Client receives these commands – Open new window– Window size changed– Pixel update– Close window
• Client sends– BFCP (Binary Floor Control Protocol) commands– Keyboard and mouse events
![Page 22: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/22.jpg)
Windows Server Architecture
Kernel space
User space
Mirror Driver
0X 300 ,500Y 200 ,600
Server Main Thread
Window coordinates
![Page 23: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/23.jpg)
Windows Server Architecture
X 320 ,340Y 100 ,200Kernel space
User space X 320 ,340Y 100 ,200
X 320 ,340Y 100 ,200
X 320 ,340Y 100 ,200
Mirror Driver
2X 300 ,500Y 200 ,600
Server Main Thread
Window coordinates
![Page 24: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/24.jpg)
Windows Server Architecture
Kernel space
User space X 320 ,340Y 100 ,200
X 320 ,340Y 100 ,200
X 320 ,340Y 100 ,200
Mirror Driver
2X 300 ,500Y 200 ,600
Server Main Thread
Window coordinates
![Page 25: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/25.jpg)
Windows Server Architecture
K e r n e l s p a c e
U s e r s p a c e X 3 2 0 , 3 4 0Y 4 5 0 , 5 5 0
X 4 2 0 ,4 4 0Y 2 0 0 ,6 0 0
X 3 2 0 , 3 4 0Y 4 0 0 , 5 0 0 2
S e r v e r M a i n T h r e a d
W i n d o w c o o r d i n a t e s
C l i e n t T h r e a d - 1
C l i e n t T h r e a d - 2
C l i e n t T h r e a d - 3
X 320,340
Y 400,500
timestam
pP
NG
X 320,340
Y 400,500
timestam
pP
NG
X 320,340
Y 400,500
timestam
pP
NG
![Page 26: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/26.jpg)
Overview
• Introduction• Demo• Architecture• Challenges• Features• Conclusion
![Page 27: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/27.jpg)
Challenges
• Different client bandwidths/speeds• Late Joiner• The effects of packet loss• Reliable multicast
![Page 28: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/28.jpg)
Multimedia Support (Movies)
![Page 29: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/29.jpg)
Multimedia Support (Movies)
• Our system uses PNG to compress and transmit the region updates
• PNG is lossless and effective for computer generated images but ineffective for real world captures like pictures or movies
• JPG is more suitable for photographic images• However, JPG is lossy and not effective for
computer generated images (text, line, shapes,...)
• Our system should use both
![Page 30: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/30.jpg)
Multimedia Support (Movies)
• Composite image comparing JPEG and PNG: notice artifacts in JPEG versus solid PNG background.
![Page 31: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/31.jpg)
Multimedia Support (PNG vs JPG)
Size x Size 4x0
10
20
30
40
50
60
70
80
90
100
110
120
130
140
150
Image Size
PNG
JPG
Size x360x150162K
Size 4x720x300648K
1:301:20
1:51:4
![Page 32: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/32.jpg)
Multimedia Support (PNG vs JPG)
Ethernet (60Mb/s)
Size x Size 4x0
2.5
5
7.5
10
12.5
15
17.5
20
22.5
FPS
PNG
JPG
Size x Size 4x0
100
200
300
400
500
600
700
800
900
1000
1100
KB/s
PNG
JPG
![Page 33: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/33.jpg)
Multimedia Support (PNG vs JPG)
Size x Size 4x0
2.5
5
7.5
10
12.5
15
17.5
20
22.5
FPS
PNG
JPG
Size x Size 4x0
50
100
150
200
250
300
350
400
KB/s
PNG
JPG
Wireless (4Mb/s)
![Page 34: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/34.jpg)
PNG/JPG Detection Algorithm
Region>40,000px
?
YES
New Region
?
NO
Use DetectedFormat
YES
-1,0,1
coordinates
PNG Size
Time Stamp
counter
Region recordCreate a record&
Start Checking
Detected?
Continue Checking
NO YES
![Page 35: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/35.jpg)
Sharing a Movie (Media Player)
![Page 36: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/36.jpg)
Sharing a Movie File
• Capturing from the Frame Buffer is expensive.• Instead
– Transcode the movie on the fly to JPGs
• FFmpeg (subproject of mplayer)● FFmpeg is a complete solution to record, convert
and stream audio and video. It includes libavcodec, the leading audio/video codec library. FFmpeg is developed under Linux, but it can compiled under most operating systems, including Windows.
• FFmpeg-Java (subproject of FMJ)– Java wrapper (JNA: Java Native Access)
![Page 37: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/37.jpg)
Sharing a Movie (Our Method)
![Page 38: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/38.jpg)
Sharing a Movie File
Movie File (wmv,mpg,divx)Movie File (wmv,mpg,divx)
JPG images
FFmpeg-Java
Java Streaming
Server
Java Client
Java Client
UDP/Multicast
decodeNextFrame()encodeFrameToJPG()
![Page 39: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/39.jpg)
Demo
DemoMultimedia Support
![Page 40: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/40.jpg)
Challenges
• Different client bandwidths/speeds• Late Joiner• The effects of packet loss• Reliable multicast
![Page 41: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/41.jpg)
Different Client Bandwidths/Speeds
S e r v e r M a i n T h r e a d
C l i e n t T h r e a d - 1
C l i e n t T h r e a d - 2
C l i e n t T h r e a d - 3
X 320,340
Y 400,500
timestam
pP
NG
X 320,340
Y 400,500
timestam
pP
NG
X 320,340
Y 400,500
timestam
pP
NG
X 320,340
Y 400,500
timestam
pP
NG
X 320,340
Y 400,500
timestam
pP
NG
X 320,340
Y 400,500
timestam
pP
NG
1 M b p s
2 M b p s
3 M b p s
![Page 42: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/42.jpg)
Different Client Bandwidths/Speeds
• Possible Solutions– Slowest one
– Average speed
– Fastest one
![Page 43: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/43.jpg)
Different Client Bandwidths/Speeds
• Possible Solutions– Slowest one
• Problem: Penalize everybody except the slowest
– Average speed
– Fastest one
![Page 44: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/44.jpg)
Different Client Bandwidths/Speeds
• Possible Solutions– Slowest one
• Problem: Penalize everybody except the slowest
– Average speed• Possible solution
– Fastest one
![Page 45: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/45.jpg)
Different Client Bandwidths/Speeds
• Possible Solutions– Slowest one
• Problem: Penalize everybody except the slowest
– Average speed• Possible solution (Can we do better?)
– Fastest one
![Page 46: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/46.jpg)
Different Client Bandwidths/Speeds
• Possible Solutions– Slowest one
• Problem: Penalize everybody except the slowest
– Average speed• Possible solution (Can we do better?)
– Fastest one• The best solution• Client bandwidths are fully utilized
![Page 47: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/47.jpg)
S e r v e r M a i n T h r e a d
C l i e n t T h r e a d - 1
C l i e n t T h r e a d - 2
C l i e n t T h r e a d - 3
X 320,340
Y 400,500
timestam
pP
NG
X 320,340
Y 400,500
timestam
pP
NG
X 320,340
Y 400,500
timestam
pP
NG
X 320,340
Y 400,500
timestam
pP
NG
X 320,340
Y 400,500
timestam
pP
NG
X 320,340
Y 400,500
timestam
pP
NG
1 M b p s
2 M b p s
3 M b p s
Different Client Bandwidths/Speeds
![Page 48: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/48.jpg)
Different Client Bandwidths/Speeds
Server Main Thread
Client Thread -1
Client Thread -2
Client Thread -3
X 3
20,340
Y 4
00,500
timesta
mp
PN
G
X 32
0,340
Y 40
0,500
timestam
pP
NG
X 32
0,34
0Y
400,5
00
time
stam
pP
NG
X 3
20,34
0Y
400
,500
timesta
mp
PN
G
X 3
20,340
Y 4
00,500
time
stamp
PN
G
X 32
0,34
0Y
400,5
00
timestam
pP
NG
1Mbps
2Mbps
3Mbps
![Page 49: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/49.jpg)
Challenges
• Different client bandwidths/speeds• Late Joiner• The effects of packet loss• Reliable multicast
![Page 50: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/50.jpg)
Late Joiner
• Force server to generate full screen update
![Page 51: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/51.jpg)
Late Joiner
• Force server to generate full screen update– Problems
• Misbehaving clients can degrade performance• If Join/Leave rate is high, too much burden on server
– Solution• Generate full screen updates if really necessary• Otherwise start the new client from last full screen update
![Page 52: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/52.jpg)
Different Client Bandwidths/Speeds
Server Main Thread
Client Thread -1
Client Thread -2
Client Thread -3
X 3
20,340
Y 4
00,500
time
stamp
PN
G
X 32
0,34
0Y
400,5
00
timestam
pP
NG
1Mbps
2Mbps
3Mbps
Client Thread -4
8Mbps
![Page 53: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/53.jpg)
Different Client Bandwidths/Speeds
Server Main Thread
Client Thread -1
Client Thread -2
Client Thread -3
X 3
20,340
Y 4
00,500
time
stamp
PN
G
X 32
0,34
0Y
400,5
00
timestam
pP
NG
1Mbps
2Mbps
3Mbps
Client Thread -4
8Mbps
FIR
![Page 54: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/54.jpg)
Different Client Bandwidths/Speeds
Server Main Thread
Client Thread -1
Client Thread -2
Client Thread -3
X 3
20,34
0Y
400
,500
timesta
mp
PN
G
X 3
20,340
Y 4
00,500
timesta
mp
PN
G
X 32
0,340
Y 40
0,500
timestam
pP
NG
X 32
0,34
0Y
400,5
00
time
stam
pP
NG
X 3
20,34
0Y
400
,500
timesta
mp
PN
G
X 3
20,340
Y 4
00,500
time
stamp
PN
G
X 32
0,34
0Y
400,5
00
timestam
pP
NG
1Mbps
2Mbps
3Mbps
Client Thread -4
8Mbps
![Page 55: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/55.jpg)
Different Client Bandwidths/Speeds
Server Main Thread
Client Thread -1
Client Thread -2
Client Thread -3
X 3
20,34
0Y
400
,500
timesta
mp
PN
G
X 3
20,340
Y 4
00,500
timesta
mp
PN
G
X 32
0,340
Y 40
0,500
timestam
pP
NG
X 32
0,34
0Y
400,5
00
time
stam
pP
NG
X 3
20,34
0Y
400
,500
timesta
mp
PN
G
X 3
20,340
Y 4
00,500
time
stamp
PN
G
X 32
0,34
0Y
400,5
00
timestam
pP
NG
1Mbps
2Mbps
3Mbps
Client Thread -4
8Mbps
![Page 56: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/56.jpg)
Different Client Bandwidths/Speeds
Server Main Thread
Client Thread -1
Client Thread -2
Client Thread -3
X 32
0,34
0Y
400,5
00
time
stam
pP
NG
X 3
20,34
0Y
400
,500
timesta
mp
PN
G
X 3
20,340
Y 4
00,500
timesta
mp
PN
G
X 32
0,340
Y 40
0,500
timestam
pP
NG
X 32
0,34
0Y
400,5
00
time
stam
pP
NG
X 3
20,34
0Y
400
,500
timesta
mp
PN
G
X 3
20,340
Y 4
00,500
time
stamp
PN
G
X 32
0,34
0Y
400,5
00
timestam
pP
NG
1Mbps
2Mbps
3Mbps
Client Thread -4
8Mbps
![Page 57: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/57.jpg)
Different Client Bandwidths/Speeds
Server Main Thread
Client Thread -1
Client Thread -2
Client Thread -3
X 32
0,34
0Y
400,5
00
time
stam
pP
NG
X 3
20,34
0Y
400
,500
timesta
mp
PN
G
X 3
20,340
Y 4
00,500
timesta
mp
PN
G
X 32
0,340
Y 40
0,500
timestam
pP
NG
X 32
0,34
0Y
400,5
00
time
stam
pP
NG
X 3
20,34
0Y
400
,500
timesta
mp
PN
G
X 3
20,340
Y 4
00,500
time
stamp
PN
G
X 32
0,34
0Y
400,5
00
timestam
pP
NG
1Mbps
2Mbps
3Mbps
Client Thread -4
8Mbps
![Page 58: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/58.jpg)
Different Client Bandwidths/Speeds
Server Main Thread
Client Thread -1
Client Thread -2
Client Thread -3
X 32
0,34
0Y
400,5
00
time
stam
pP
NG
X 3
20,34
0Y
400
,500
timesta
mp
PN
G
X 3
20,340
Y 4
00,500
timesta
mp
PN
G
X 32
0,340
Y 40
0,500
timestam
pP
NG
X 32
0,34
0Y
400,5
00
time
stam
pP
NG
X 3
20,34
0Y
400
,500
timesta
mp
PN
G
X 3
20,340
Y 4
00,500
time
stamp
PN
G
X 32
0,34
0Y
400,5
00
timestam
pP
NG1Mbps
2Mbps
3Mbps
Client Thread -4
8Mbps
Client Thread -5
2Mbps
![Page 59: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/59.jpg)
Different Client Bandwidths/Speeds
Server Main Thread
Client Thread -1
Client Thread -2
Client Thread -3
X 32
0,34
0Y
400,5
00
time
stam
pP
NG
X 3
20,34
0Y
400
,500
timesta
mp
PN
G
X 3
20,340
Y 4
00,500
timesta
mp
PN
G
X 32
0,340
Y 40
0,500
timestam
pP
NG
X 32
0,34
0Y
400,5
00
time
stam
pP
NG
X 3
20,34
0Y
400
,500
timesta
mp
PN
G
X 3
20,340
Y 4
00,500
time
stamp
PN
G
X 32
0,34
0Y
400,5
00
timestam
pP
NG1Mbps
2Mbps
3Mbps
Client Thread -4
8Mbps
Client Thread -5
2Mbps
![Page 60: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/60.jpg)
Challenges
• Different client bandwidths/speeds• Late Joiner• The effects of packet loss• Reliable multicast
![Page 61: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/61.jpg)
The effects of Packet Loss
• This problem applies to – Multicast– UDP
• The PNG images can be large– Regular desktop can be ~900KB– ~600 Ethernet packets– One packet loss wastes all PNG image
![Page 62: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/62.jpg)
The effects of Packet Loss
• Solution– Small PNG images
• Around ~1500 bytes• Consist of a few scanlines
– Disadvantages• Increased CPU usage (client&server)• Lower compression ratio (%20 lower)
– Advantages• One packet loss = no update for a few scanlines
![Page 63: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/63.jpg)
The effects of Packet Loss
![Page 64: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/64.jpg)
Challenges
• Different client bandwidths/speeds• Late Joiner• The effects of packet loss• Reliable multicast
![Page 65: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/65.jpg)
Reliable Multicast
• RTP Library stores last N rtp packets
• Clients send NACK for lost packets
• RTP Library resend the requested packets
![Page 66: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/66.jpg)
The effects of Packet Loss
![Page 67: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/67.jpg)
Overview
• Introduction• Demo• Architecture• Challenges• Features• Conclusion
![Page 68: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/68.jpg)
Recording
• Clients can record the whole/part session• Anybody can play these files locally• These files can be streamed to receivers via
streaming server• Streaming server supports multiple receivers
– Also late joiners
![Page 69: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/69.jpg)
Listening Client
• Client waits for incoming connections• It can display windows from multiple user• Can be used for RGB cable replacement
![Page 70: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/70.jpg)
Overview
• Introduction• Demo• Architecture• Challenges• Features• Conclusion
![Page 71: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/71.jpg)
Conclusion
• Application sharing allows users to share a single application with multiple participants.
• Participants don't need the application.
• It is not specific to a single application.
• Extra features like recording is added.
![Page 72: Application and Desktop Sharing - Columbia Universityboyaci/appshare/other/irt_presentation_nov_1... · Y 450 ,550 X 420 ,440 Y 200 ,600 ... • Our system uses PNG to compress and](https://reader031.vdocuments.us/reader031/viewer/2022022520/5b1d89e27f8b9ab06b8bb91c/html5/thumbnails/72.jpg)
Thanks
• Questions?