measuring interaction qoe in internet videoconferencingreal-time or online monitoring of end-user...
TRANSCRIPT
![Page 1: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/1.jpg)
Measuring Interaction QoE in Internet Videoconferencing
Prasad Calyam (Presenter)Ohio Supercomputer Center, The Ohio State University
Mark Haffner, Prof. Eylem Ekici Prof. Chang-Gun LeeThe Ohio State University Seoul National University
MMNS, November 1st 2007
![Page 2: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/2.jpg)
2
Outline• Background
• Voice and Video over IP (VVoIP) Overview• Network QoS and End-user QoE in VVoIP• Streaming QoE versus Interaction QoE• Network Fault Events
• Multi-Activity Packet Trains (MAPTs) methodology• Participant Interaction Patterns• Traffic Model for MAPTs emulation
• Vperf tool implementation of MAPTs• Performance Evaluation• Concluding Remarks and Future Work
![Page 3: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/3.jpg)
3
Outline• Background
• Voice and Video over IP (VVoIP) Overview• Network QoS and End-user QoE in VVoIP• Streaming QoE versus Interaction QoE• Network Fault Events
• Multi-Activity Packet Trains (MAPTs) methodology• Participant Interaction Patterns• Traffic Model for MAPTs emulation
• Vperf tool implementation of MAPTs• Performance Evaluation• Concluding Remarks and Future Work
![Page 4: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/4.jpg)
4
Voice and Video over IP (VVoIP) Overview
Large-scale deployments of VVoIP are on the riseVideo streaming (one-way voice and video)
MySpace, Google Video, YouTube, IPTV, …Video conferencing (two-way voice and video)
Polycom, MSN Messenger, WebEx, Acrobat Connect, …
Challenges for large-scale VVoIP deploymentReal-time or online monitoring of end-user Quality of Experience (QoE)
Traditional network Quality of Service (QoS) monitoring not adequateNetwork QoS metrics: bandwidth, delay, jitter, loss
Need objective techniques for automated network-wide monitoringCannot rely on end-users to provide subjective rankings – expensive and time consuming
![Page 5: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/5.jpg)
5
Network QoS and End-user QoE
End-user QoE is mainly dependent on the combined impact of network factorsDevice factors such as voice/video codecs, peak video bit rate (a.k.a. dialing speed) also matter
Our study maps the network QoS to end-user QoE for a given set of commonly used device factors
H.263 video codec, G.711 voice codec, 256/384/768 Kbps dialing speeds
End-user QoENetwork QoS
![Page 6: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/6.jpg)
6
Voice and Video Packet Streams
Total packet size (tps) – sum of payload (ps), IP/UDP/RTP header (40 bytes), and Ethernet header (14 bytes)Dialing speed is ; = 64 Kbps fixed for G.711 voice codec
Voice has fixed packet sizes (tpsvoice ≤ 534 bytes)Video packet sizes are dependent on alev in the content
![Page 7: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/7.jpg)
7
Video alevLow alev
Slow body movements and constant background; E.g. Claire video sequence
High alevRapid body movements and/or quick scene changes; E.g. Foreman video sequence
‘Listening’ versus ‘Talking’Talking video alev(i.e., High) consumes more bandwidth than Listening video alev (i.e., Low)
Claire Foreman
![Page 8: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/8.jpg)
8
End-user QoE TypesStreaming QoE
End-user QoE affected just by voice and video impairments Video frame freezingVoice drop-outsLack of lip sync between voice and video
Interaction QoEEnd-user QoE also affected by additional interaction effort in a conversation
“Can you repeat what you just said?”“This line is noisy, lets hang-up and reconnect…”
QoE is measured using “Mean Opinion Score” (MOS) rankings
![Page 9: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/9.jpg)
9
Network Fault Events“Best-effort service” of Internet causes network fault events that impact application performance
Cross traffic congestion, routing instabilities, physical link failures, DDoS attacksOur definition of network fault events is based on the “Good”, “Acceptable”and “Poor” (GAP) performance levels for QoS metrics causing GAP QoE
Type-I: Performance of any network factor changes from Good grade to Acceptable grade over a 5 second durationType-II: Performance of any network factor changes from Good grade to Poor grade over a 10 second duration
![Page 10: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/10.jpg)
10
Related WorkCharacteristics of network fault events well understood
Bursts, spikes, complex patterns – lasting few seconds to a few minutes (Markopoulou et. al., Ciavattone et. al.)
Measuring Streaming QoE impact due to network fault events has been well studied
ITU-T E-Model is a success story for VoIP QoE estimationDesigned for CBR voice traffic and handles only voice related impairments
ITU-T J.144 developed for VVoIP QoE measurement“PSNR-based MOS” – Requires original and reconstructed video frames for frame-by-frame comparisonsOffline method - PSNR calculation is a time consuming and computationally intensive process
Online VVoIP QoE measurement proposalsPSQA (G. Rubino, et. al.), rPSNR (S. Tao, et. al.)
Measuring Interaction QoE impact due to network fault events has NOT received due attention
Need for schemes to measure interaction difficulties in voice and video conferences presented by A. Rix, et. al.
![Page 11: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/11.jpg)
11
Problem Summary
Given:Voice/video codecs used in a videoconferenceDialing speed of the videoconferenceNetwork fault event types to monitor
Develop:An objective technique that can measure Interactive VVoIP QoEReal-time measurement without involving actual end-users, video sequences and VVoIP appliancesAn active measurement tool that can: (a) emulate VVoIP traffic on a network path, and (b) use the objective technique to produce Interaction QoE measurements
Vperf Tool
Multi-Activity Packet Trains Methodology
![Page 12: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/12.jpg)
12
Outline• Background
• Voice and Video over IP (VVoIP) Overview• Network QoS and End-user QoE in VVoIP• Streaming QoE versus Interaction QoE• Network Fault Events
• Multi-Activity Packet Trains (MAPTs) methodology• Participant Interaction Patterns• Traffic Model for MAPTs emulation
• Vperf tool implementation of MAPTs• Performance Evaluation• Concluding Remarks and Future Work
![Page 13: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/13.jpg)
13
Proposed Solution Methodology
“Multi-Activity Packet Trains” (MAPTs) measure Interaction QoE in an automated manner
They mimic participant interaction patterns and video activity levels as affected by network fault events Given a session-agenda, excessive talking than normal due to unwanted participant interaction patterns impacts Interaction QoE“Unwanted Agenda-bandwidth” measurement and compare with baseline (consumption during normal conditions)
Higher values indicate poor interaction QoE and caution about potential increase in Internet traffic congestion levelsMeasurements serve as an input for ISPs to improve network performance using suitable traffic engineering techniques
![Page 14: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/14.jpg)
14
Proposed Solution Methodology (2)
‘repeat’‘disconnect’‘reconnect’‘reorient’
Type-I and Type-II fault detection
![Page 15: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/15.jpg)
15
Participant Interaction PatternsAssumption: Question (Request) and Answer (Response) items in a session agenda
Side-A listening when Side-B talking, and vice versa
Normal – PIP1
![Page 16: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/16.jpg)
16
Participant Interaction Patterns (2)
Participant Interaction Patterns (PIPs) using MAPTs for a “Type-I”network fault event
Repeat – PIP2
![Page 17: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/17.jpg)
17
Participant Interaction Patterns (3)
Participant Interaction Patterns (PIPs) using MAPTs for a “Type-II”network fault event
Disconnect/Reconnect/Reorient – PIP3
![Page 18: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/18.jpg)
18
Participant Interaction Patterns (4)
Our goal is to measure the Unwanted Agenda-bandwidth and Unwanted Agenda-timemeasurements after MAPTs emulation of the Q & A session agenda
![Page 19: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/19.jpg)
19
Traffic Model for MAPTs Emulation
Traffic Model for probing packet trains obtained from trace-analysisCombine popularly used low and high alev video sequences and model them at 256/384/768 Kbps dialing speeds for H.263 video codecLow – Grandma, Kelly, Claire, Mother/Daughter, SalesmanHigh – Foreman, Car Phone, Tempete, Mobile, Park Run
ModelingVideo Encoding Rates (bsnd) time seriesPacket Size (tps) distribution Derived instantaneous inter-packet times (tps) by dividing instantaneous packet sizes by video encoding rates
![Page 20: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/20.jpg)
20
Video Encoding Rates (bsnd) ModelingTime-series modeling of the bsnd data using the classical decomposition method We find a Second order moving average [MA(2)] process model fitθ1 – MA(1) parameterθ2 – MA(2) parameter
Low alevHigh alev
![Page 21: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/21.jpg)
21
Video Packet Size (tps) Distribution ModelingDistribution-fit analysis on the tps data We find a Gamma distribution fit
α – shape parameterβ – scale parameter
For High alevat 256 Kbps
![Page 22: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/22.jpg)
22
Traffic Model Parameters for MAPTs Emulation
![Page 23: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/23.jpg)
23
Outline• Background
• Voice and Video over IP (VVoIP) Overview• Network QoS and End-user QoE in VVoIP• Streaming QoE versus Interaction QoE• Network Fault Events
• Multi-Activity Packet Trains (MAPTs) methodology• Participant Interaction Patterns• Traffic Model for MAPTs emulation
• Vperf tool implementation of MAPTs• Performance Evaluation• Concluding Remarks and Future Work
![Page 24: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/24.jpg)
24
Vperf Tool Implementation of MAPTs
Per-second frequency of “Interim Test Report” generationInteraction QoE reported by Vperf tool - based on the progress of the session-agenda
![Page 25: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/25.jpg)
25
Example – Session Agenda and Network Factor Limits File
![Page 26: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/26.jpg)
26
Outline• Background
• Voice and Video over IP (VVoIP) Overview• Network QoS and End-user QoE in VVoIP• Streaming QoE versus Interaction QoE• Network Fault Events
• Multi-Activity Packet Trains (MAPTs) methodology• Participant Interaction Patterns• Traffic Model for MAPTs emulation
• Vperf tool implementation of MAPTs• Performance Evaluation• Concluding Remarks and Future Work
![Page 27: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/27.jpg)
27
MAPTs Emulation at different Dialing Speeds
256 Kbps
768 Kbps384 Kbps
![Page 28: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/28.jpg)
28
MAPTs Measurements EvaluationIncreased the number of Type-I and Type-II network fault events in a controlled LAN testbed for a fixed session-agenda
NISTnet network emulator for network fault generationRecorded Unwanted Agenda-Bandwidth and Unwanted Agenda-Timemeasured by Vperf tool
(a) Impact of Type-I Network Fault Events on Unwanted Agenda-Bandwidth
(b) Impact of Type-II Network Fault Events on Unwanted Agenda-Bandwidth
![Page 29: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/29.jpg)
29
MAPTs Measurements Evaluation (2)
(c) Impact of Type-I and Type-II Network Fault Events on Unwanted Agenda-Time
![Page 30: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/30.jpg)
30
Outline• Background
• Voice and Video over IP (VVoIP) Overview• Network QoS and End-user QoE in VVoIP• Streaming QoE versus Interaction QoE• Network Fault Events
• Multi-Activity Packet Trains (MAPTs) methodology• Participant Interaction Patterns• Traffic Model for MAPTs emulation
• Vperf tool implementation of MAPTs• Performance Evaluation• Concluding Remarks and Future Work
![Page 31: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/31.jpg)
31
Conclusion
We proposed a Multi-Activity Packet Trains methodology Mimic participant interaction patterns and video activity levels as affected by network fault events
MAPTs provide real-time objective measurements of Interaction QoE in a large-scale videoconferencing system
Without requiring end-users, actual video sequences, VVoIP appliances
Defined new Interaction QoE MetricsUnwanted Agenda-Bandwidth, Unwanted Agenda-Time
Implemented MAPTs in an active measurement tool called Vperfand evaluated Interaction QoE measurements on a network testbed
![Page 32: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/32.jpg)
32
Future Work
Our work is a first-step towards measuring how network fault events impact Interaction QoE in videoconferencing sessionsWe considered basic participant interaction patterns and network fault event types
Future scope could include several other participant interactionpatterns and network fault event types
E.g. MAPTs for network fault events that cause lack of lip-sync
Human subject testing to more accurately map and validate network fault event types and participant interaction patterns
![Page 33: Measuring Interaction QoE in Internet VideoconferencingReal-time or online monitoring of end-user Quality of Experience (QoE) ... DDoS attacks Our definition of network fault events](https://reader033.vdocuments.us/reader033/viewer/2022060405/5f0f206a7e708231d4429ed9/html5/thumbnails/33.jpg)
33
Thank you for your attention!☺
Any Questions?