voip testing with tems investigation pc-based clients
TRANSCRIPT
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
1/18
Prepared by: Date: Document:Ascom Network Testing 5/6/2013 NT11-12850
Ascom (2013)TEMS is a trademark of Ascom. All other trademarks are the property of their respective holders.
No part of this document may be reproduced in any form without the written permission of the copyright holder.
The contents of this document are subject to revision without notice due to continued progress in methodology, design andmanufacturing. Ascom shall have no liability for any error or damage of any kind resulting from the use of this document.
VoIP Testing with TEMS InvestigationPC-based Clients
Technical Paper
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
2/18
Ascom (2013) Document:NT11-12850
Contents
1
Introduction ................................................................ 1
2 How VoIP Works: A Brief TechnologyOverview ..................................................................... 1
3 Testing VoIP with TEMS Investigation ..................... 2
3.1
Physical Configuration .............................................................2
3.2 Scripting ....................................................................................3
3.2.1 Tips on Scripting .........................................................................5
3.3
Voice Quality Measurement .....................................................7
3.4
Output ........................................................................................7
3.4.1
VoIP-specific Information Elements ............................................7
3.4.2 Other Information Elements of Interest ..................................... 103.4.3 VoIP Events .............................................................................. 103.4.4 VoIP KPIs (Key Performance Indicators) .................................. 10
3.5 Presentation in TEMS Investigation Windows ...................... 11
3.6 Ascom Test Setup .................................................................. 11
4 Troubleshooting....................................................... 11
4.1
Problem: Script Activity Fails ................................................ 11
4.2 Problem: Bad Audio Quality (PESQ/POLQA Score Low) ..... 12
5 Limitations ................................................................ 12
6 Appendices ............................................................... 13
6.1 SIP Response Codes .............................................................. 13
6.1.1 Informational Responses .......................................................... 136.1.2 Successful Responses.............................................................. 136.1.3 Redirection Responses ............................................................. 136.1.4
Client Failure Responses .......................................................... 13
6.1.5 Server Failure Responses ........................................................ 156.1.6 Global Failure Responses ......................................................... 156.1.7 Extended Codes ....................................................................... 15
6.2 Abbreviations .......................................................................... 16
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
3/18
Ascom (2013) Document:NT11-12850 1(16)
1 Introduction
VoIP, Voice over IP, is a technology for delivering voice communications
over IP networks such as the Internet. Being a voice-over-data service,VoIP has characteristics from both realms. On one hand it is very delay-sensitive (like circuit-switched voice); on the other it is subject to all of thevarious challenges associated with packet-switched services, such askeeping down packet loss and jitter. This makes VoIP a very complicatedservice to optimize.
The present paper describes VoIP in general terms and tells how to testand measure VoIP performance using TEMS Investigation PC-basedclients. The document does not cover VoIP testing with on-device clients,which is another option available in TEMS Investigation.
2 How VoIP Works: A Brief TechnologyOverview
VoIP (Voice over IP) is an umbrella term for technologies that enabledelivery of voice calls and multimedia sessions over IP networks, such asthe Internet, rather than the public switched telephone network (PSTN).
The voice signal is digitized and encoded using audio codecs, just as incircuit-switched cellular telephony, and then divided into IP packets fortransmission over the packet-switched network. On the receiving sidesimilar steps are applied in the reverse order to reproduce the original voice
stream: reception and decoding of IP packets followed by digital-to-analogconversion.
The range of audio codecs used differs between VoIP implementations;some implementations rely on narrowband and compressed speech, whileothers support high fidelity stereo codecs.
VoIP systems employ session control protocols to control the setup andteardown of calls. Examples of such protocols are:
H.323
Media Gateway Control Protocol (MGCP)
Session Initiation Protocol (SIP)
Real-time Transport Protocol (RTP)
Session Description Protocol (SDP)
Of these, SIP and RTP have gained particularly widespread use, and theseprotocols also figure in the present document.
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
4/18
Ascom (2013) Document:NT11-12850 2(16)
3 Testing VoIP with TEMS Investigation
3.1 Physical ConfigurationPC-based VoIP testing is conducted using two instances of TEMSInvestigation, installed on two different PCs, each of which have a mobilephone connected.1This setup is necessary to enable end-to-end speechquality measurement for VoIP.
The calling device (caller) is connected to one PC and the called device(callee) to the other. Audio is sent in semi-duplex fashion between theparties, that is, in both directions but only in one direction at a time.
TEMS Investigation has a built-in PJSIP VoIP client; the VoIP clients thusreside in the PCs and not in the mobile devices.
It should be noted from the outset that no further devices running dataservices can be connected to the PCs during VoIP testing. See alsochapter5.
Schematic diagram of physical configuration for VoIP testing.
1
Two TEMS Investigation licenses are thus also required, as well as a speciallicense option for VoIP.
VoIPserver
Caller
side
Callee
side
PCs running TEMS Investigation
Mobile phones VoIP
client
VoIP
client
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
5/18
Ascom (2013) Document:NT11-12850 3(16)
3.2 Scripting
Two scripts are needed in TEMS Investigation, one for the caller and one
for the callee. Predefined snippets, VoIP PC Dialand VoIP PC Answer,are supplied with TEMS Investigation for this purpose.
It is worth underlining that the timing between caller and callee is essential.The callee must be registeredwith the SIP server and finish itspreparations for answering before the caller dials the call. See step below.
VoIP PC Dial snippet VoIP PC Answer snippet
Screenshots of VoIP PC Dial snippet (left)and VoIP PC Answer snippet (right)asdisplayed in the TEMS Investigation Service Control Designer. The numbering refersto the step-by-step description that follows below.
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
6/18
Ascom (2013) Document:NT11-12850 4(16)
Network Connect
First, both parties need to have an active data session. This is done in
TEMS Investigation through the Network Connectactivity.
SIP Registration
Before a VoIP call can begin, both caller and callee must register with theSIP server to be used for VoIP. The SIP Registeractivity is used for thispurpose. Here you indicate the IP address or host name of the server touse. If no special domain needs to be chosen, enter the server addressunder Domain, and leave Proxyempty. If on the other hand you need tospecify a domain within the server, enter the server address in the Proxyfield and the domain in the Domainfield. You also specify the user andpassword the client should use when registering.
Answer
The callee must be ready to answer before the caller can initiate a call. Tothis end the callee executes the script activity Answerwith Call Typeset toVoIP PC. In this activity you select the audio codec and encoding rate thecallee should use. The callee will communicate these settings to the caller,so that the parties agree on the same codec and rate.
To ensure that the callee has reached Answer before the caller dials thecall, you should insert a wait period in the callers script. See section3.2.1.This detail has been left out of the above diagram to keep things
straightforward.
Dial
The caller initiates the call by running the activity Dial, again with Call Typeset to VoIP PC.In this activity you indicate the codec the caller shoulduse, which must be the same as the callees designated codec (Answeractivity, see step ). You also specify the codec rate and the phonenumber to call.
Voice Quality Measurement
Once the call has been connected, voice quality can be measured on bothsides using one out of several algorithms supported (see section3.3 fordetails on this matter). This is done with the Voice Qualityactivity, whereAudio Sourceis set to VoIP (PC).The call durations should preferablydiffer between caller and callee, so that it can be controlled which sidehangs up the call. Compare step.
It is possible to store audio files containing the received audio. All audiosentences having a MOS score lower than or equal to the MOS limitwill be
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
7/18
Ascom (2013) Document:NT11-12850 5(16)
stored on the PC.2 If three consecutive voice quality measurement reportswith indication of silence are received, the VoIP call is terminated.
If for some reason you do not wish to measure voice quality, simply use a
Wait activity instead of Voice Quality. However, be aware that in this caseyou will not obtain any other VoIP quality measurements either, such asFER and jitter buffer metrics (see the list in section3.4.1), nor will MTSISession Completion Failure events (dropped calls) be generated.
Hangup
One of the parties (the one with the duration of the Voice Qualityactivityset lower) hangs up the call. This is to ensure that the party hanging up hasthe time to do so before the other party unregisters; otherwise the hangupwill fail. This is done through the Hang Upactivity (Call Type= VoIP PC).In the above diagram, the caller performs the hangup.
SIP Unregister
Both sides unregister from the SIP server. This is done using the SIPUnregisteractivity.
Network Disconnect
Finally the data session is terminated as each party performs a NetworkDisconnect. If the snippet is executed in a loop, this activity is necessary toforce a disconnect from the network after each VoIP call (desirable for thepurpose of KPI calculation).
3.2.1 Tips on Scripting
The above description covered the key steps in conducting a VoIP call. Inpractice, the scripts should be made slightly more sophisticated.
Suggested setup:
On both sides, run all activities within a while loop. Add an extra Waitactivity as the last item in each loop, with the callers wait period longerthan the callees, to make sure the callee is ready and waiting for thecall when the caller dials. Suggested wait durations are 30 s for the
caller and 10 s for the callee. Compare step in section3.2 above. To ensure that the timing becomes right for the first VoIP call, start the
script on the callee side first, then the caller script.
If the parties are not in sync, that is, if the callee is not registered when thecaller places the call, then the caller will generate an MTSI Session SetupFailure event (see section3.4.3).
2
Storage location: C:\Users\\Documents\TEMS Product Files\TEMSInvestigation \PESQ.
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
8/18
Ascom (2013) Document:NT11-12850 6(16)
Caller Callee
Use of while loops with VoIP scripting: caller (left) and callee (right).
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
9/18
Ascom (2013) Document:NT11-12850 7(16)
3.3 Voice Quality Measurement
The Voice Qualityscript activity has an Algorithmparameter where the
following choices can be made: PESQ P862.1
POLQA P863 Narrowband
POLQA P863 Super Wideband
Non-intrusive
PESQ and POLQA are industry standard algorithms for assessing voicequality as perceived by a human listener, POLQA being a refinement of theolder PESQ algorithm. Their output is a value on the MOS scale rangingfrom 1 (worst) to 5 (best). The score obtained is a function of the radioenvironment, of the speech codec and codec rate used, and of otherfactors. Both algorithms are dealt with at length in the document AQM inTEMS Products (Including PESQ and POLQA).
Each of the two algorithms requires the purchase of a special license optionto be enabled in TEMS Investigation. Even without PESQ and POLQA,however, a more basic estimate of voice quality is obtained in the form ofthe quantity VoIP FER Combined Packet Loss (see section3.4.1.3), whichis always computed. This is the meaning of the term Non-intrusive.
Note that both uplink and downlink must use the samevoice qualityalgorithm. It is not possible to use PESQ on one link and POLQA on theother, even if you possess license options for both.
3.4 Output
3.4.1 VoIP-specific Information Elements
All of these are found in the Data information element category in TEMSInvestigation unless otherwise noted.
3.4.1.1 Jitter
VoIP RFC 1889 Jitter (ms)
Packet jitter or delay variation as defined in IETF RFC 1889, section 6.3.1:
An estimate of the statistical variance of the RTP data packet interarrivaltime [...] The interarrival jitteris defined to be the mean deviation(smoothed absolute value) of the difference in packet spacing at thereceiver compared to the sender for a pair of packets. As shown in theequation below, this is equivalent to the difference in the relative transittime for the two packets [...].
If is the RTP timestamp from packet , and is the time of arrival inRTP timestamp units for packet , then for two packets and, may beexpressed as
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
10/18
Ascom (2013) Document:NT11-12850 8(16)
The interarrival jitter is calculated continuously as each data packet is
received [...] using this difference for that packet and the previouspacket in order of arrival (not necessarily in sequence), accordingto the formula
The quantityis what is output in the VoIP RFC 1889 Jitterinformationelement. The latter is updated once every second.
3.4.1.2 Jitter Buffer
A jitter buffer is used to mitigate the effects of packet jitter. The jitter bufferholds the received voice packets briefly, reorders them if necessary, andthen plays them out at evenly spaced intervals to the decoder.
These elements are updated once every second.
VoIP Decoding Errors (%) Percentage of audio frames that could not bedecoded by the speech codec.
VoIP Jitter Buffer LostPackets (%)
Percentage of packets that were missing from theaudio reproduction because they were notdelivered from the jitter buffer to the decoder intimely fashion.
Note that the packet need not have been lost onthe way to the receiving party; it may just havebeen delayed too long, so that it was discarded bythe jitter buffer.
VoIP Jitter Buffer PlayoutDelay Average (ms)
Average playout delay in ms: that is, the averagetime the voice packets were held by the jitterbuffer.
VoIP Jitter Buffer PlayoutDelay Maximum (ms)
Maximum playout delay in ms.
VoIP Jitter Buffer Playout
Delay Minimum (ms)
Minimum playout delay in ms.
VoIP Jitter Buffer SizeIncrease (%)
Percentage of audio frames where the VoIP clientdecided to increase the jitter buffer size (becausethe jitter was found to be too high). Thisprocedure results in a period of silence in theaudio reproduction as the jitter buffer accumulatespackets without releasing any.
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
11/18
Ascom (2013) Document:NT11-12850 9(16)
VoIP Jitter BufferOverruns (%)
Percentage of audio frames with overruns.
The VoIP client tries to keep the delays caused bythe jitter buffer reasonably low. When the buffer
becomes too long, the VoIP client will throw awayreceived packets to decrease the buffer size. Thisis referred to as overruns, and it affects the audioreproduction.
Usually occurs after underruns (see below).
VoIP Jitter BufferUnderruns (%)
Percentage of audio frames where the jitter bufferwas empty and had no packets to deliver to thespeech decoder.
3.4.1.3 Audio Quality Related
Data category
VoIP FER CombinedPacket Loss (%)
Total percentage of packet loss that affects thereproduction of the audio. Encompasses decodingerrors, underruns, overruns, and jitter buffer sizeincreases: compare the information elements insection3.4.1.2.Should in general correlateclosely to PESQ and POLQA.
VoIP Speech Codec Speech codec selected for the VoIP client in the
governing script (Dialand Answeractivities: see
section3.2,stepsand).
Media Quality category (see also section3.3):
PESQ Score Downlink PESQ (ITU P.862.1) voice quality score.
For VoIP measurements the speech sentencesare 5.5 s in length. This means that a MOS scorewill be calculated every 11 s (since transmissionsare done in semi-duplex). Notethat aperformance degradation that occurs while themeasurement is done at the other end will notregister in the PESQ score.
POLQA NB Score Downlink POLQA (ITU P.863.1) voice quality score for
narrowband.
POLQA SWB ScoreDownlink
POLQA voice quality score for super-wideband.
In the real time presentation, the PESQ and POLQA scores appear themoment they have been computed. When loading a logfile for analysis, onthe other hand, the PESQ and POLQA scores are moved backward in timeto the point when the corresponding speech sentence was received by theVoIP client. That is, sentences are aligned in time with their quality scores.This is not much of an issue for PESQ, which takes only a fraction of asecond to compute, but it can be for POLQA, whose computation may
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
12/18
Ascom (2013) Document:NT11-12850 10(16)
require several seconds (the worse the degradation of the signal, the morecomplex POLQA is to evaluate).
3.4.2 Other Information Elements of Interest
Application throughput IEs (Data category).
RAN throughput at various protocol levels (details being dependent onthe cellular technology used; the IEs are found in the relevant category,such as LTE, WCDMA).
3.4.3 VoIP Events
These events underlie the KPIs in section3.4.4:
MTSI Registration Failure One of the parties failed to register with the SIPserver.
MTSI Registration Time Time required for the terminal to register with theSIP server.Also functions as a success event.
MTSI Session CompletionFailure
A VoIP session that was successfully set up failedto complete. Similar to dropped call for CS voice.
MTSI Session CompletionTime
Duration of the VoIP session. Also functions as asuccess event.
Note:This event does not have an associated
KPI, since the VoIP session duration is not a
relevant performance measure.
MTSI Session SetupFailure
The terminal failed in setting up a VoIP session.Similar to blocked call for CS voice.
MTSI Session Setup Time Time required to set up the VoIP session. Alsofunctions as a success event.
TEMS Investigation also generates the following VoIP events, which areunrelated to KPI computation:
VoIP Start A VoIP session was started.
VoIP End A VoIP session ended normally.
VoIP Error A VoIP session was aborted because of an error.
3.4.4 VoIP KPIs (Key Performance Indicators)
TEMS Investigation provides data for computation of the following KPIs.The actual computation is done in TEMS Discovery or TEMS Automatic.
MTSI Registration FailureRatio (%)
Denotes the probability that the terminal cannotregister towards IMS when requested.
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
13/18
Ascom (2013) Document:NT11-12850 11(16)
MTSI Registration Time (s) Denotes the time elapsing from the IMSregistration request until the terminal is registeredto IMS.
MTSI Session SetupFailure Ratio (%)
Denotes the probability that the terminal cannotset up an MTSI session. An MTSI session setup isinitiated when the user presses the call buttonand concludes when the user receives, within apredetermined time, a notification that the calleehas answered.
MTSI Session Setup Time(s)
Denotes the time elapsing from initiation of anMTSI session until a notification is received thatthe session has been set up.
MTSI Session CompletionFailure Ratio (%)
Denotes the probability that a successfully set upMTSI call is ended by a cause other than
intentional termination by either party.
3.5 Presentation in TEMS Investigation Windows
Suitable presentation windows for VoIP data:
VoIP Qualitystatus window containing the information elementsdescribed in section3.4.1
VoIP Quality Line Charttracking VoIP PESQ/POLQA scores and VoIPFER Combined Packet Loss, also indicating MTSI events
VoIP AMR Codecs Usagestatus window Data Reportsmessage window
IP Protocol Reportsmessage window.
3.6 Ascom Test Setup
The VoIP function in TEMS Investigation has been tested with TekSIP, aSIP Registrar and SIP Proxy for Windows (www.teksip.com), as well aswith an Ericsson IMS server.
4 Troubleshooting
4.1 Problem: Script Activity Fails
Check that caller and callee are synchronized, that is, that the calleereaches Answer before the caller begins Dial. See section3.2,step 3,and section3.2.1.
In the Eventswindow, look for MTSI failure events.
In the Data Reportsmessage window, look into the VoIP Error
Message category.
http://www.teksip.com/http://www.teksip.com/http://www.teksip.com/http://www.teksip.com/ -
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
14/18
Ascom (2013) Document:NT11-12850 12(16)
In the IP Protocol Reports message window, study the SIP messages.
If any other ports than SIP port 5060 and RTP port 4000 are used onthe SIP server, the corresponding settings have to be changed in the
file \Application\Configuration\Investigation.Voip.config.
If SIP response code 422 (Session interval too small) is received, setDisableTimers="false"in the same file.
4.2 Problem: Poor Audio Quality (PESQ/POLQAScore Low)
Investigate throughput and BLER values at different levels. Example:For LTE, this includes the application layer, PDSCH, RLC, PDCP, and
MAC. Remember to look at both uplink and downlink. Check channel quality indicators and serving/neighbor signal strength.
Example: In an LTE network, study CQI, Serving Cell RSRP, andNeighbor Cell RSRP.
Check for excessively frequent handovers.
In the IP Protocol Reports message window, look into the RTPmessages.
5 Limitations
You cannot have any other internet connections in parallel while runningVoIP measurements. That is, the PCs cannot be connected to anyfurther IP addresses, whether through other external devices, throughan Ethernet cable, or by other means. All network interfaces exceptthe testing devices, both fixed and wireless, must be disabled. It ishowever possible to make CS voice calls with devices connected to thePCs.
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
15/18
Ascom (2013) Document:NT11-12850 13(16)
6 Appendices
6.1 SIP Response Codes
6.1.1 Informational Responses
100 Trying
180 Ringing
181 Call is being forwarded
182 Queued
183 Session progress
6.1.2 Successful Responses
200 OK
202 Accepted
Indicates that the request has been understood but actually cannotbe processed
6.1.3 Redirection Responses
300 Multiple choices
301 Moved permanently
302 Moved temporarily
305 Use proxy
380 Alternative service
6.1.4 Client Failure Responses
400 Bad request
401 Unauthorized
Used only by registrars or user agents. Proxies should use proxyauthorization 407
402 Payment required
Reserved for future use
403 Forbidden
404 Not found
User not found
405 Method not allowed
406 Not acceptable
407 Proxy authentication required
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
16/18
Ascom (2013) Document:NT11-12850 14(16)
408 Request timeout
Could not find the user in time
409 Conflict
410 Gone
The user existed once, but is no longer available here
412 Conditional request failed
413 Request entity too large
414 Request URI too long
415 Unsupported media type
416 Unsupported URI scheme
417 Unknown resource priority
420 Bad extensionBad SIP protocol extension used, not understood by the server
421 Extension required
422 Session interval too small
423 Interval too brief
424 Bad location information
428 Use identity header
429 Provide referrer identity
433 Anonymity disallowed
436 Bad identity info
437 Unsupported certificate
438 Invalid identity header
480 Temporarily unavailable
481 Call/transaction does not exist
482 Loop detected
483 Too many hops
484 Address incomplete
485 Ambiguous
486 Busy here
487 Request terminated
488 Not acceptable here
489 Bad event
491 Request pending
493 Undecipherable
Could not decrypt S/MIME body part
494 Security agreement required
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
17/18
Ascom (2013) Document:NT11-12850 15(16)
6.1.5 Server Failure Responses
500 Server internal error
501 Not implementedThe SIP request method is not implemented here
502 Bad gateway
503 Service unavailable
504 Server timeout
505 Version not supported
The server does not support this version of the SIP protocol
513 Message too large
580 Precondition failure
6.1.6 Global Failure Responses
600 Busy everywhere
603 Decline
604 Does not exist anywhere
606 Not acceptable
6.1.7 Extended Codes
701 The called party has hung up
702 VoIP socket error
703 Connection cancelled because of timeout
704 Connection interrupted because of a SIP error
705 SIP memory error
706 SIP transaction memory error
751 Busy tone: No codec match between the calling and calledparty
810 General socket layer error
811 General socket layer error: Wrong socket number
812 General socket layer error: Socket is not connected
813 General socket layer error: Memory error
814 General socket layer error: Socket not availablecheck IPsettings/connection problem/VoIP setting incorrect
815 General socket layer error: Illegal application on the socketinterface
922 No DNS server known
923 DNS name resolution failed
-
7/24/2019 VoIP Testing With TEMS Investigation PC-based Clients
18/18
Ascom (2013) Document:NT11-12850 16(16)
924 Insufficient resources for DNS name resolution
925 URL error
6.2 Abbreviations
AMR-NB Adaptive Multi Rate Narrowband
AMR-WB Adaptive Multi Rate Wideband
BLER Block Error Rate
CQI Channel Quality Indicator
FER Frame Erasure Rate
IMS IP Multimedia Subsystem
IP Internet Protocol
KPI Key Performance Indicator
LTE Long Term Evolution
MAC Medium Access Control
MOS Mean Opinion Score
MTSI Multimedia Telephony Service for IMS
PDSCH Physical Downlink Shared Channel
PESQ Perceptual Evaluation of Speech Quality
POLQA Perceptual Objective Listening Quality Assessment
PSTN Public Switched Telephone Network
RAN Radio Access Network
RSRP Reference Signal Received Power
RTP Real-time Transport Protocol
SIP Session Initiation Protocol
VoIP Voice over IP