key performance indicators v1 6

20
Key Performance Indicators Nemo CEM Solution is a unique QoS/QoE solution provided by Anite, which extracts real life information directly from a smartphone (or datacard) to deliver end-to-end measurement of your services in any network scenario and on any bearer.

Upload: hoainamcomit

Post on 15-Jan-2016

22 views

Category:

Documents


8 download

DESCRIPTION

Key Performance Indicators V1 6

TRANSCRIPT

Page 1: Key Performance Indicators V1 6

Key Performance Indicators

Nemo CEM Solution is a unique QoS/QoE solution provided by Anite, which extracts real life information directly from a smartphone (or datacard) to deliver end-to-end measurement of your services in any network scenario and on any bearer.

Page 2: Key Performance Indicators V1 6

SERVICES  IN  A  NUTSHELL

MODE   SERVICE ANDROID iOS BLACKBERRY SYMBIAN WINDOWS  PHONE WINDOWS  DONGLE

Data  HTTP A A A AData  FTP AVideo  Streaming A AWeb  Browsing A A A AData  HTTP A A ✖ PData  FTP A ✖Web  Browsing A AVideo  Streaming AVoice ASMS A AMMS APING ASurvey A PMail ACoverage AApplication A A PData  Session A A PData  Connection A A A PVoice A A A ASMS A AMMS A AMail AWeb  Browsing A PCoverage A

OCM

SSM

SLM

Page 3: Key Performance Indicators V1 6

READ  ME

FILE  VERSION PORTAL  VERSION DATE COMMENT

1.0 1.5.X 130110Revamped  version  of  the  KPI's  list

1.1 1.5.X 130111

1.2 1.5.X 130111

DESCRIPTION

A AvailablePS

ü MMS  :  9ü PING  :  4ü Mail  :  9ü Coverage  :  2

ü Voice  :  21 ü Application  :  16ü SMS  :  10 ü Data  Session  :  8

ü Web  Browsing  :  29

Changed  name  for  throughput  and  data  exchangedAdded  Device  related  KPIsAdded  iOS  Store  mode  for  Common  KPIs

ü Data  HTTP  :  31ü Data  FTP  :  24ü Video  Streaming  :  38

Changed  KPI  Android  SSM  Voice  from  S  to  P  status

Short  term  Study  (<  6  months)Planed  for  next  release

MORE  THAN  225  KPIs  !

Page 4: Key Performance Indicators V1 6

COMMON  KPIs

SERVICES KPIs  /  REF DEFINITIONS UNITS  /  CODE ANDROID iOS   iOS  STORE RIM SYMBIAN WIN  PHONE WIN  DONGLE

Network  Status Give  the  status  of  the  phone  on  the  network:  if  it  is  connected,  in  roaming,  in  emergency  only…

Integer A A A A A

Data  Status Integer A A AMCC Mobile  Country  Code String A A A AMNC Mobile  National  Code String A A A ACID Cell  ID Integer A A A ALAC Local  Area  Code Integer A A A ADetailed  Technology Used  Technology Integer A A A AAggregated  Technology Technology  such  as  2G,  3G,  4G,  Wi-­‐Fi Integer A A A A ASignal  Level Radio  Signal  Strength Integer  -­‐  dBm A A A A

Wi-­‐Fi  status Wi-­‐Fi  connection  description Integer A A A A A AWi-­‐Fi  type Public  or  private  Wi-­‐Fi Integer A ASSID SSID  of  the  Wi-­‐Fi  hotspot String A A A A ASignal  Level Wi-­‐Fi  Radio  Signal  Strength integer  -­‐  dBm A A A A A

Battery  Level Current  level  of  the  battery Integer  -­‐  % A A A AOS Name  of  the  Operating  System String A A A A A A PDevice  Model Name  of  the  Device  Model String A A A A A A POS  version Version  of  the  Operating  System String A A A A A A AManufacturer Name  of  the  Manufacturer  of  the  Device String A A A A A A AIMEI   IMEI  of  the  device  (only  if  anonymous  mode  is  disabled) String A A A A A A

Provider Indicate  where  the  coordinates  come  from String A A A A

Accuracy Indicates  the  accuracy  radius  in  meters Integer  -­‐  meters A A AA A A

SIM  State Status  of  the  SIM  card Integer A A A ASIM  MCC A A ASIM  MNC A A AMSISDN MSISDN  available  on  Non  Anonymous  mode  only String A A A

Wi-­‐Fi

SIM

Radio  Inform

ation

Phon

eLocatio

n

Page 5: Key Performance Indicators V1 6

OCM  1/2SERVICES KPIs  /  REF DEFINITION  UNITS  /  CODE ANDROID iOS RIM SYMBIAN

WIN  PHONE

WIN  DONGLE

Termination  code Code    gving  the  information  of  how  the  test  ended:  in  a  failure,  a  timeout  or  a  drop.

Integer A A A A

Direction Were  the  test  an  upload  or  a  download Integer  (1=  upload;  2=  Download) A A A AData  Volume Volume  total  exchanged  during  the  test kBytes A A A P

Minimum  Throughput Minimum  throughput  speed  (peak)  achieved  during  the  test Kbit/s

A A A A

Average  Throughput Average  throughput  speed   Kbit/s A A A A

Maximum  Throughput Maximum  throughput  speed  (peak)  achieved  during  the  test Milliseconds

A A A A

Minimum  Latency Minimum  Round  Trip  Time  (peak)  achieved  during  the  test Milliseconds

A A S A

Average  Latency Average  Round  Trip  Time  during  the  test Milliseconds A A S A

Maximum  Latency Maximum  Round  Trip  Time  (peak)  achieved  during  the  test Milliseconds

A A S A

URL  server Server  URL  from/to  where  you  are  uploading  or  downloading  the  files String

A A A A

Timeout Time  after  which  you  consider  the  test  has  failed Seconds A A A APDP  Setup  time Time  needed  to  set  up  the  PDP  context Milliseconds A

IP  Setup  timeTime  needed  to  get  an  IP  address  from  DNS  server Milliseconds

A A A

Transfer  time Time  total  to  transfer  the  file  completely Milliseconds A A A AIP  Service  access  (A) Milliseconds A A ASession  time Time  total  of  the  test Milliseconds A A A A

Termination  code Code    gving  the  information  of  how  the  test  ended:  in  a  failure,  a  timeout  or  a  drop.

Integer A

Direction Were  the  test  an  upload  or  a  download Integer  (1=  upload;  2=  Download) AData  Volume Volume  total  exchanged  during  the  test kBytes AMinimum  Throughput Minimum  throughput  speed  (peak)  achieved   Kbit/s AAverage  Throughput Average  throughput  speed   Kbit/s A

Maximum  ThroughputMaximum  throughput  speed  (peak)  achieved  during  the  test Milliseconds A

Minimum  Latency Minimum  Round  Trip  Time  (peak)  achieved  during  the  test

Milliseconds A

Average  Latency Average  Round  Trip  Time  during  the  test Milliseconds A

Maximum  Latency Maximum  Round  Trip  Time  (peak)  achieved  during  the  test

Milliseconds A

URL  server Server  URL  from/to  where  you  are  uploading  or  downloading  the  files

String A

PDP  Setup  time Time  needed  to  set  up  the  PDP  context MillisecondsIP  Service  access  (A) Milliseconds ASession  time Time  total  of  the  test Milliseconds A

Data  HTTP

Data  FTP

Page 6: Key Performance Indicators V1 6

LIMITATIONSOS SERVICE KPI >=  iOS4.1 >=  iOS  s5

iPhone  3GS iPhone  4 iPad  1 iPad  2 Store iPhone  3GS iPhone  4 iPhone  4S iPad  1 iPad  2 iPad  3 Store

Phone  IMEI  number A A A A U A A U A A U U

SIM  MSISDN U U U U U A A A A A U UNetwork  status  begin U U U U U A A A A A UMCC  begin A A A A U A A A A A U UMNC  begin A A A A U A A A A A U UNetwork  type  begin U U U U U A A A A A URSSI  begin A A A A U A A A A A UTechnology  used  at  the  beginning A A A A U A A A A A UWiFi  RSSI  begin A A A A U A A A A A USLM  MODE A A A A A A A A A UCELL  ID    and  LAC  ONLY  ON  SLM  MODE

OS SERVICE KPI OS  2.X <  OS  4.X OS  4.X   OS  4.1 OS  4.2

HTC  Magic Samsung  Note

Samsung  Galaxy  S  2

ALL HTC  Sense ALL ALL

Dialing  Time U UEstablishing  Time U URinging  time U USetup  time  A  (outgoing) U UConnected  time  (outgoing) U USession  time  (outgoing)   U URemote  alert  time  (outgoing) U URemote  released  activity  time  (outgoing) U UConnected  time  (VO005  +  VO006) U UTermination  code U U UExtended  code U U UExtended  code  messages  (details) U U U

MMS SSM  :  Device  can't  sent  MMS  when  device  is  connected  to  the  WiFi UService  unavailable UAccess  delay U UParts U UTransferred  parts U UNetwork  retry U U

Radio RSSI U UData Data  throughput  doest  not  work  on  IDLE  mode UVideo  

Streamingvideo  streaming  screen  not  supported  but  all  kpis  availables U

Qmon  detect  browsing  only  with  the  default  android  browserContent  size U U

U Means  that  the  KPI  is  unavailable

iOS Radio  &  

General

ANDR

OID

Web All

Random

Voice

SMS

Page 7: Key Performance Indicators V1 6

SSM  1/5SERVICES KPIs  /  REF DEFINITIONS  UNITS  /  CODE ANDROID iOS   RIM SYMBIAN WIN  

PHONEWIN  DONGLE

Termination  codeCode    giving  the  information  of  how  the  test  ended:  in  a  failure,  a  timeout  or  a  drop

int A A P

Direction Were  the  test  an  upload  or  a  download

Upload  /  Download A A P

Data  Volume Volume  total  exchanged  during  the  test

kBytes A A P

Minimum  Throughput Minimum  throughput  speed  (peak)  achieved  during  the  test

Kbit/s A P P

Average  Throughput Average  throughput  speed   Kbit/s A A P

Maximum  Throughput Maximum  throughput  speed  (peak)  achieved  during  the  test

Milliseconds A P P

Minimum  Latency Minimum  Round  Trip  Time  (peak)  achieved  during  the  test

Milliseconds A P P

Average  Latency Average  Round  Trip  Time  during  the  test

Milliseconds A P P

Maximum  Latency Maximum  Round  Trip  Time  (peak)  achieved  during  the  test

Milliseconds A P P

URL  serverServer  URL  from/to  where  you  are  uploading  or  downloading  the  files

String A A P

Timeout Time  after  which  you  consider  the  test  has  failed

Seconds A A P

IP  Setup  time Time  needed  to  get  an  IP  address  from  DNS  server

Milliseconds A P

Transfer  time Time  total  to  transfer  the  file  completely

Milliseconds A A P

IP  Service  access  (A) Milliseconds A A PSession  time Time  total  of  the  test Milliseconds A A P

Data  HTTP

Page 8: Key Performance Indicators V1 6

SSM  2/5SERVICES KPIs  /  REF DEFINITIONS  UNITS  /  CODE ANDROID iOS   RIM SYMBIAN WIN  

PHONEWIN  DONGLE

Termination  codeCode    gving  the  information  of  how  the  test  ended:  in  a  failure,  a  timeout  or  a  drop.

Integer A

Direction Were  the  test  an  upload  or  a  download

Upload  /  Download A

Data  Volume Volume  total  exchanged  during  the  test

kBytes A

Minimum  Throughput Minimum  throughput  speed  (peak)  achieved  during  the  test

Kbit/s A

Average  Throughput Average  throughput  speed   Kbit/s A

Maximum  Throughput Maximum  throughput  speed  (peak)  achieved  during  the  test

Milliseconds A

Minimum  Latency Minimum  Round  Trip  Time  (peak)  achieved  during  the  test

Milliseconds A

Average  Latency Average  Round  Trip  Time  during  the  test

Milliseconds A

Maximum  Latency Maximum  Round  Trip  Time  (peak)  achieved  during  the  test

Milliseconds A

URL  serverServer  URL  from/to  where  you  are  uploading  or  downloading  the  files

String A

IP  Service  access  (A) Milliseconds ASession  time Time  total  of  the  test Milliseconds A

Termination  code Code    gving  the  information  of   Integer A A

Content  volume Size  of  the  whole  content  that  needs  to  be  loaded

Bytes A A

Downloaded  volume Total  size  of  downloaded  data Bytes AUploaded  volume Total  size  of  uploaded  data Bytes A

Mean  data  rate Average  speed  reached  during  the  transfer

Kbit/s A

Element  number Number  of  element  downloaded

A A

Element  number  in  error Number  of  elements  that  failed A A

Timeout Timeout  in  seconds seconds A ASession  time Time  total  of  the  test ms A A

FTP  HT

TPWeb

Page 9: Key Performance Indicators V1 6

SSM  3/5SERVICES KPIs  /  REF DEFINITIONS  UNITS  /  CODE ANDROID iOS   RIM SYMBIAN WIN  

PHONEWIN  DONGLE

Termination  codeCode    gving  the  information  of  how  the  test  ended:  in  a  failure,  a  timeout  or  a  drop.

Integer A

Data  Volume Volume  of  data  exchanged  with  the  network

kBytes A

Buffering  count Buffering  count integer AMinimum  buffer  Throughput

Minimum  speed  (peak)  during  buffering

Kbit/s A

Average  buffer  Throughput Average  speed  during  buffering Kbit/s A

Maximum  buffer  Throughput

Maximum  speed  (peak)  during  buffering

Kbit/s A

Minimum  Throughput  during  streaming

Minimum    speed  (peak)  during    streaming

Kbit/s A

Average  Throughput  during  streaming

Average  speed  during  streaming

Kbit/s A

Maximum  Throughput  during  streaming

Maximum    speed  (peak)  during  streaming

Kbit/s A

Minimum  buffer  latency Minimum  RTT  latency  (peak)  during  buffering

Milliseconds A

Average  buffer  latency Average  RTT  latency  during  buffering

Milliseconds A

Maximum  buffer  latency Max  RTT  latency  (peak)  during  buffering

Milliseconds A

Minimum  latency  during  streaming

Min  RTT  latency  (peak)  during  streaming

Milliseconds A

Average  latency  during  streaming

Average  RTT  latency  during  streaming

Milliseconds A

Maximum  latency  during  streaming

Maximum  RTT  latency  (peak)  during  streaming

Milliseconds A

Initial  Buffering  Time Duration  of  the  first  buffer  (before  video  launch)

Milliseconds A

Rebuffering  Time Total  duration  of  rebuffering Milliseconds ASession  Time Milliseconds A

Vide

o  Stream

ing

Page 10: Key Performance Indicators V1 6

SSM  4/5SERVICES KPIs  /  REF DEFINITIONS  UNITS  /  CODE ANDROID iOS   RIM SYMBIAN WIN  

PHONEWIN  DONGLE

Termination  codeCode    gving  the  information  of  how  the  test  ended:  in  a  failure,  a  timeout  or  a  drop.

int A

Call  Direction Incoming  or  outgoing  call Incoming  /  outgoing ACalling  /  Called  Party  Number

Phone  number  from  /to  which  the  call  comes/goes String A

Dialing  Time Time  of  number  dialing  in  the  network

Milliseconds A

Establishing  Time Time  between  dialing  and  Ringing  time

Milliseconds A

Ringing  time Time  during  which  the  phone  rang

Milliseconds A

Setup  Time  (outgoing) Dialing  time  +  Establishing  time Milliseconds A

Connected  time  (outgoing) Dialing  time  +  establishing  time  +  ringing  time

Milliseconds A

Session  time  (outgoing) Time  total  from  dialing  to  termination

Seconds A

Connected  time  (incoming) Dialing  time  =  establishing  time  =  ringing  time

Seconds A

Session  time  (incoming) Time  total  from  dialing  to  termination

Seconds A

Termination  codeCode    gving  the  information  of  how  the  test  ended:  in  a  failure,  a  timeout  or  a  drop.

Integer A P

Direction Reception  or  emission  of  a  SMSInteger  :  1  incoming  SMS;  2:  outgoing  SMS,  0  :  UNKNOWN

A P

SMS  Size Size  of  the  SMS Integer A PCalling  /  Called  Party  Number

Phone  number  from  /to  which  the  message  comes/goes

String A P

E2E  time Time  between  emission  and  reception Milliseconds A P

SMS

Voice

Page 11: Key Performance Indicators V1 6

SSM  5/5SERVICES KPIs  /  REF DEFINITIONS  UNITS  /  CODE ANDROID iOS   RIM SYMBIAN WIN  

PHONEWIN  DONGLE

Termination  codeCode    gving  the  information  of  how  the  test  ended:  in  a  failure,  a  timeout  or  a  drop.

Integer A

Direction Reception  or  emission  of  a  MMSInteger  :  1  incoming  MMS;  2:  outgoing  MMS,  0  :  UNKNOWN

A

MMS  Size Size  of  the  MMS Integer ACalling  /  Called  Party  Number

Phone  number  from  /to  which  the  message  comes/goes

String A

E2E  time Time  between  emission  and  reception Milliseconds S

Termination  codeCode    gving  the  information  of  how  the  test  ended:  in  a  failure,  a  timeout  or  a  drop.

Integer A

URL URL  used  to  test  ping  value String AIP  address IP  address  of  the  UE String ATimeout Seconds ATime RTT  time   Milliseconds A

Termination  Code End  code  for  the  survey A P

Termination  codeCode    gving  the  information  of  how  the  service  ended:  in  a  failure,  a  timeout  or  a  drop.

Integer A

DirectionThe  mail  was  incoming  or  outgoing

Integer  :  1=outgoing,  2=incoming A

Mail  Size Size  of  the  mail kBytes AData  Transfer  Time Time  to  transfer  the  mail Milliseconds AServer  E2E  time Milliseconds

Service  Availability A

Survey

Mail

Coverage

Ping

MMS

Page 12: Key Performance Indicators V1 6

SLM  1/4SERVICES KPIs  /  REF DEFINITIONS UNITS  /  CODE ANDROID iOS RIM SYMBIAN WIN  

PHONEWIN  DONGLE

Termination  code Code  when  the  app  stopped  being  used/active IntegerDownloaded  data Volume  downloaded  by  the  app kBytes AUploaded  data Volume  of  data  uploaded  by  the  app kBytes AMinimum  download  speed Minimum  value  of  download  speed Kbit/s AAvearge  download  speed Average  Download  speed Kbit/s AMax  download  speed Maximum  value  in  dl  speed  reached Kbit/s AMinimum  Upload  Speed Minimum  upload  speed Kbit/s AAverage  Upload  Speed Average  Upload  Speed Kbit/s AMaximum  Upload  Speed Maximum  upload  speed  reached A

Application  Name Name  of  the  application  dependant  of  the  phone's  language

String A A P

Application  package Exact  package  name,  independant  of  the  phone's  language

String A A P

Application  version Version  of  the  application String A A P

Foreground The  application  was  on  the  foreground  (used  willingly  by  the  End  User)

A P

Time  in  foreground Time  spent  with  the  application  in  foreground A A P

Total  Download  Time Total  time  while  the  application  downloaded  data Seconds A

Total  Upload  Time Time  while  the  application  uploaded  data Seconds A

Termination  code Code  showing  how  data  session  has  been  terminated Integer A ADirection Upload  or  download A AData  Volume Totalsize  ofdata  exchanged  during  the  session kBytes A AMinimum    Throughput Minimum  speed  value  during  the  session Kbit/s A AAverage    Throughput Average    data  speed  of  the  session Kbit/s A AMaximum  Throughput Maximum  data  speed  reached  during  the  session Kbit/s A AActivity  time Time  bwhen  data  has  been  exchanged Milliseconds A ASession  time Total  time  of  the  whole  data  session Milliseconds A A

Data  Session

Applications

Page 13: Key Performance Indicators V1 6

SLM  2/4SERVICES KPIs  /  REF DEFINITIONS

 UNITS  /  CODE  /  DATABSE ANDROID iOS RIM SYMBIAN

WIN  PHONE

WIN  DONGLE

Termination  code Code  showing  how  connection  (PDP,  Ethernet)  has  been  terminated

Integer A A

Downloaded  Volume Volume  of  downloaded  data kBytes A A A PUploaded  Volume Volume  of  uploaded  data kBytes A A A PAPN Access  Point  Name  used  for  the  connection String A A AIP  Adress IP  adress  of  the  UE String A A A P

PDP  Context  Activation  Time Time  to  mount  a  PDP Milliseconds A A

Time  of  the  session Total  duration  of  the  connection Seconds A A A P

Termination  code Code    gving  the  information  of  how  theservice  ended:  in  a  failure,  a  timeout  or  a  drop.

Integer A A A A

Call  Direction Incoming  or  outgoing  call Incoming  /  outgoing

A A A A

Calling  /  Called  Party  Number

Phone  number  from  /to  which  the  call  comes/goes String A A A

Dialing  Time Time  of  number  dialing  in  the  network Milliseconds A AEstablishing  Time Time  between  dialing  and  Ringing  time Milliseconds ARinging  time Time  during  which  the  phone  rang Milliseconds A ASetup  Time  (outgoing) Dialing  time  +  Establishing  time Milliseconds A AConnected  time  (outgoing) Dialing  time  =  establishing  time  =  ringing  time Milliseconds A ASession  time  (outgoing) Time  total  from  dialing  to  termination Seconds A AConnected  time  (incoming) Dialing  time  =  establishing  time  =  ringing  time Seconds A ASession  time  (incoming) Time  total  from  dialing  to  termination Seconds A

Voice

Data  Con

nections

Page 14: Key Performance Indicators V1 6

SLM  3/4SERVICES KPIs  /  REF DEFINITIONS  UNITS  /  CODE  /  

DATABSEANDROID iOS RIM SYMBIAN WIN  

PHONEWIN  

DONGLE

Termination  code Code    gving  the  information  of  how  the  service  ended:  in  a  failure,  a  timeout  or  a  drop.

Integer A P A

Direction Reception  or  emission  of  a  SMS

Integer  :  1  incoming  SMS;  2:  outgoing  SMS,  0  :  UNKNOWN

A P A

SMS  SIZE Size  of  the  MMS Integer A P ACalling  /  Called  Party  Number

Phone  number  from  /to  which  the  message  comes/goes

String A P A

Transfer  Time Time  needed  to  send  the  message Milliseconds A P AE2E  time Time  between  emission  and  reception Milliseconds

Termination  code Code    gving  the  information  of  how  the  service  ended:  in  a  failure,  a  timeout  or  a  drop.

Integer A A

Direction Reception  or  emission  of  a  MMS

Integer  :  1  incoming  MMS;  2:  outgoing  MMS,  0  :  UNKNOWN

A A

MMS  SIZE Size  of  the  MMS Integer A ACalling  /  Called  Party  Number

Phone  number  from  /to  which  the  message  comes/goes

String A A

E2E  time Time  between  emission  and  reception Milliseconds S

Termination  code Code    gving  the  information  of  how  the  service  ended:  in  a  failure,  a  timeout  or  a  drop.

Integer A

Direction The  mail  was  incoming  or  outgoingInteger  :  1=outgoing,  2=incoming

A

Mail  Size Size  of  the  mail kBytes AData  Transfer  Time Time  to  transfer  the  mail Milliseconds AServer  E2E  time Milliseconds A

SMS

MMS

Mail

Page 15: Key Performance Indicators V1 6

SLM  4/4SERVICES KPIs  /  REF DEFINITIONS  UNITS  /  CODE  /  

DATABSEANDROID iOS RIM SYMBIAN WIN  

PHONEWIN  

DONGLE

Termination  code Code    gving  the  information  of  how  the  service  ended:  in  a  failure,  a  timeout  or  a  drop.

Integer A P

Content  volume Size  of  the  whole  content  that  needs  to  be  loaded Bytes PDownloaded  volume Total  size  of  downloaded  data Bytes A PUploaded  volume Total  size  of  uploaded  data Bytes A PMean  data  rate Average  speed  reached  during  the  transfer Kbit/s A PElement  number Number  of  element  downloaded Integer PElement  number  in  error Number  of  elements  that  failed Integer PURL Visited  URL A PTimeout Timeout  in  seconds Seconds PTransfer  time   Time  during  which  data  has  been  exchanged Milliseconds PSession  time Time  total  of  the  test Milliseconds A P

Service  Availability A

Web

Coverage

Page 16: Key Performance Indicators V1 6

LIMLTAT IONSOS SERVICE KPI >=  iOS4.1 >=  iOS  s5

iPhone  3GS iPhone  4 iPad  1 iPad  2 Store iPhone  3GS iPhone  4 iPhone  4S iPad  1 iPad  2 iPad  3 Store

Phone  IMEI  number A A A A U A A U A A U U

SIM  MSISDN U U U U U A A A A A U UNetwork  status  begin U U U U U A A A A A UMCC  begin A A A A U A A A A A U UMNC  begin A A A A U A A A A A U UNetwork  type  begin U U U U U A A A A A URSSI  begin A A A A U A A A A A UTechnology  used  at  the  beginning A A A A U A A A A A UWiFi  RSSI  begin A A A A U A A A A A USLM  MODE A A A A A A A A A UCELL  ID    and  LAC  ONLY  ON  SLM  MODE

OS SERVICE KPI OS  2.X <  OS  4.X OS  4.X   OS  4.1 OS  4.2

HTC  Magic Samsung  Note

Samsung  Galaxy  S  2

ALL HTC  Sense ALL ALL

Dialing  Time U UEstablishing  Time U URinging  time U USetup  time  A  (outgoing) U UConnected  time  (outgoing) U USession  time  (outgoing)   U URemote  alert  time  (outgoing) U URemote  released  activity  time  (outgoing) U UConnected  time  (VO005  +  VO006) U UTermination  code U U UExtended  code U U UExtended  code  messages  (details) U U U

MMS SSM  :  Device  can't  sent  MMS  when  device  is  connected  to  the  WiFi UService  unavailable UAccess  delay U UParts U UTransferred  parts U UNetwork  retry U U

Radio RSSI U UData Data  throughput  doest  not  work  on  IDLE  mode UVideo  

Streamingvideo  streaming  screen  not  supported  but  all  kpis  availables U

Qmon  detect  browsing  only  with  the  default  Android  browserContent  size U U

U Means  that  the  KPI  is  unavailable

iOS Radio  &  

General

ANDR

OID

Web All

Random

Voice

SMS

Page 17: Key Performance Indicators V1 6

MODE DEFINITIONS

OCM

SLM

SSM

OCM | On-Click MonitoringThe end user launches the service stress test himself provisioned by SDI server. A user-friendly interface is provided with the OCM mode. It performs service tests, displays information in real time and records the KPIs. The KPIs are then uploaded through HTTP/HTTPS to SDI server for post processing analysis.In terms of user interface, a navigation monitor is available and based on Google maps where the test measurements by technologies are also displayed. Activity logs are displayed within the application for a quick summary on every measure results. All test case scenario such as, list of URLs, throughput download and upload scenarios by technologies can be customised remotely by the server.

SLM | Silent Live MonitoringThis application is always running in background and monitors the customer experience. For instance, when an end-user receives a call, all network and services KPIs such as call duration, setup time, and call event (drop, fail) are sent to SDI server. In addition a customer experience questionnaire can be displayed either or not at the end of a service session (voice, web browsing...). Thus, it will provide an overview of the real customer experience on how your subscribers perceive the quality of your network services.Moreover, information such as the inventory of device features (e.g. GPS enable or not, storage memory size, number of applications running in background, battery life information…) can be also monitored.

SSM | Silent Script MonitoringThis application is only running in the background when the back office manager triggers a performance test campaign.This mode provides a pro-active and always-on automated monitoring solution. For instance, when a problem occurs within the mobile network on a specific location, the back office manager is able to design a stress test ready to be spread out to any registered Nemo Qmon. Elected agents are ready to provide related KPIs.Thanks to SDI, a mobile operator can isolate faults from a central site and diagnose customer problems more precisely when linked either or not to probe systems.

Page 18: Key Performance Indicators V1 6

SERVICE DEFINITIONS

APPLICATIONS

DATA CONNECTION

DATA FTP

DATA HTTP

DATA SESSION

It refers to active data tests (OCM and SSM modes) using the FTP protocols. These tests are done by downloading or uploading a file of a specific size to or from a defined URL. They depend on the technology used to do the test because parameters can change from a technology to another. Parameters that can be configures are for each technology: the file size, the direction (upload or download), the URL, the timeout and the scenario that will be played during the test (number of downloads and uploads).

Service that uses as the main information the name of an application. An application is a software installed on the User Equipment to execute a specific task or answer a specific need. Nemo CEM Solution only monitors applications that use the broadband between the User Equipment and the Mobile Operator Network.

This service refers to all connections that have been set up to transfer data between User Equipment and a Network, wireless or not. Wired Networks are only available for Windows Phone. It monitors if the connection have been set up and closed well or if an anomaly has occurred.

It refers to active data tests (OCM and SSM modes) using the HTTP protocols. These tests are done by downloading or uploading a file of a specific size to or from a defined URL. They depend on the technology used to do the test because parameters can change from a technology to another. Parameters that can be configures are for each technology: the file size, the direction (upload or download), the URL, the timeout and the scenario that will be played during the test (number of downloads and uploads).

A Data Session is a passive monitoring Session in SLM during which a threshold of packets has been exchanged in downlink and uplink between the User Equipment and the Network. Thresholds can be configured in the Monitoring Engine under the throughput service for the following parameters: inactive period, active period, min DL size to be OK, min UL size to be OK.

Page 19: Key Performance Indicators V1 6

SERVICE DEFINITIONS

EMAIL

ISHO

MMS

PING

SMS

VIDEO STREAMING

VOICE

WEB BROWSING

This service is monitoring if a SMS has been properly sent or received, either on passive mode or by using an active mode.

This service is monitoring different pieces of information that are used to check the customer experience when a video is being streamed on the mobile, using a wireless connection. Today, this service is available on the active mode and use the Youtube protocol.

Voice service refers to the telephony function. It monitors if a voice call has been connected to the network or not and if possible the time it took to do so. This service is available either on passive mode SLM or on active mode SSM according to the OS.

Filled by either active test or passive surfing on the net, this service is mainly represented by the time needed to load an URL. We are focusing our attention on the time needed to load a URL and the number of errors on the total of requests.

This service is monitoring if an email has been properly sent or received.

This service is gathering information on technology and location when an ISHO is occurring only while the phone was on idle mode. This way, we avoid gathering ISHO on specific parameters: for instance the MNO could have decided to switch all voice calls in 2G no matter the technology available before the call. By gathering the ISHO on idle mode only, we are sure to collect information that were not meant to change on specific network parameters.

This service is monitoring if a MMS has been properly sent or received, either on passive mode or by using an active mode.

Ping Service that focus on the idle round-trip delay time value.

Page 20: Key Performance Indicators V1 6

SERVICES  IN  A  NUTSHELL

MODE   SERVICE ANDROID iOS BLACKBERRY SYMBIAN WINDOWS  PHONE WINDOWS  DONGLE

Data  HTTP A A A AData  FTP AVideo  Streaming A AWeb  Browsing A A A AData  HTTP A A ✖ PData  FTP A ✖Web  Browsing A AVideo  Streaming AVoice ASMS A AMMS APING ASurvey A PMail ACoverage AApplication A A PData  Session A A PData  Connection A A A PVoice A A A ASMS A AMMS A AMail AWeb  Browsing A PCoverage A

OCM

SSM

SLM