evaluación de calidad para servicios de voz móviles, incluyendo interfaces de usuario en...

Colombia, 23-24 September 2013

QoS assessment for mobile speech services including user interfaces in vehicles and

hands-free communication

Dr.-Ing. H. W. Gierlich

Head of Telecom Division

CITEL (PCC.I)/ ITU Forum on Information and

Communication Technology Service:

Quality, Control and Surveillance

(Cartagena de Indias, Colombia, 23-24 September 2013)

Colombia, 23-24 September 2013 2

Outline

Introduction

The impact of terminal design on speech quality

Assessment methods and standards for mobile terminals

Driver distraction and speech services in cars

Summary

Users Expectation (QoE)

Colombia, 23-24 September 2013 3

SpeechQuality

... ...

Sound Quality &Naturalness

Intelligibility

TalkingEffort

ConversationalEffort

Double TalkPerformance

Expectation

Transmission ofBackground Noise

SpeechCharacteristics

Individual Listening

NetworkConditions

EnvironmentalConditions

ListeningEffort

Signal Processing in Modern Terminals

Colombia, 23-24 September 2013 4

Modulation/

Demodulation

H(f )

-NLP

CNEC

Decoder

Coder

Loudspeaker

DecoderComfort Noise

Mikrofon –(Array)

EC

AGC/ companding

… Coder

Noise cancellatio

n

+

NR

z.B. GSM/UMTS

SE

Speech-“enhancement”speech recognition/

control

EQ

Important: All signal processing isnon-linear, time variant and

targeted to speech enhamcement

Speech Quality

Colombia, 23-24 September 2013 5

TESTING…in general should be based on users perception…may be based on simple signals and procedures modelling aspectsof users perception (e.g. loudness)…may be based on complex signals and advanced proceduresmodelling aspects of users perception (e. g. switching and double talk tests)…may be based on perceptual models using psychoacousticallymotivated procedures (models of the human hearing)

USERS PERCEPTION

…is subjective and generally different between subjects…is based on experience…is cultural dependant

Colombia, 23-24 September 2013 6

Outline

Introduction

The impact of terminal design on speech quality

Assessment methods and standards for mobile terminals

Driver distraction and speech services in cars

Summary

Speech Quality Evaluation

Colombia, 23-24 September 2013 7

Speech-

Quality

Echo

Performance

Receiving

Double Talk

Quality of Background

Noise Transm.

Sending

Terminal Testing

Bring the world to the lab:

Simulate user

Simulate user behaviour

Simulate user environments

Simulate network and access points

Simulate network conditions

Valdidate correct NTP access

Colombia, 23-24 September 2013 8

The Typical Lab Setup

Colombia, 23-24 September 2013 9

Sub-woofer

Testsystem

Network Simulation

Background

Noise

Simulation

Relevant Standards

GPP TS 51.010 (GSM 11.10)

3GPP TS 26.131 & TS 26.132

3GPP2 C.S0056

TIA 928 (CDMA/TDMA)

CTIA

GSMA HDvoice standard

For more advanced testing:

ETSI TS 103 737 – TS 103740 standard series

Colombia, 23-24 September 2013 10

Moste Important Test Parameters Enabling a Good QoS Performance

Basic acoustic tests e.g. Loudness Ratings (ITU-T P.79), Frequency response tests…

Delay and echo tests

Speech quality tests – with and without background noise

Conversational test including double talk, switching…

Important: Test signals are mostly speech based – ITU-T P.501

Colombia, 23-24 September 2013 11

The Impcat of Delay on Conversational Dynamics

Colombia, 23-24 September 2013 12

Best case between fixed & mobile network (no satelite):

~150 ms

Worst case across networks:

>> 500 ms

For superior conversational quality: delay < ~150 ms

(from the users perspective)

Network

Terminal A Terminal B0 ms (.wav)150 ms

(.wav)500 ms (.wav)

Conversational/Double Talk Test Signal

Colombia, 23-24 September 2013 13

Word fully masked, one speaker continuosly talking

Word in partially masked

Word conicident with the other channel speaker, Simulating delay and unwanted double talk

Double talk sentences

Receive

Send

Male Female Male Female

Short isolated words

Word in a speech pause

Summary of Quality Parameters Using „Quality Pies“ according to ITU-T P.505

Colombia, 23-24 September 2013 14

TMOS SND

23

45

SLR

2/14

4/12

6/10

8

D-Value

-10

0

-5

5

BGNT (far)

10

0DT Type

2c

2b

2a1

Subj. Exp. Assessmentpoorfairgood

TCLw max

30

4050

70

TCLw nom

30

40

50

70

Echo vs. time

not Ok

OK

TMOS RCV

2

3

1,5

4

RLR nom

8/-46/-2

4/02

G-MOS CAR

23,5

5

N-MOS CAR

2

3,5

5

S-MOS CAR

2

3,5

5

G-MOS Mensa

2

3,4

5

N-MOS Mensa

2

3,4

5

S-MOS Mensa

23,4

5

G-MOS ROAD

1,52,3

3,2

N-MOS ROAD

1,5

2,3

3,2

S-MOS ROAD

1,5

2,3

3,2

G-MOS TRAIN

2

3

4,3

N-MOS TRAIN

2

3

4,3

S-MOS TRAIN

23

4,3

Phone 4

Conv. Quality

TMOS SND

23

45

SLR

2/14

4/12

6/10

8

D-Value

-10

0

-5

5

BGNT (far)

10

0DT Type

2c

2b2a

1

Subj. Exp. Assessmentpoorfairgood

TCLw max

30

4050

70

TCLw nom

30

40

50

70

Echo vs. time

not Ok

OK

TMOS RCV

2

3

1,5

4

RLR nom

8/-46/-2

4/02

G-MOS CAR

23,5

5

N-MOS CAR

2

3,5

5

S-MOS CAR

2

3,5

5

G-MOS Mensa

2

3,4

5

N-MOS Mensa

2

3,4

5

S-MOS Mensa

23,4

5

G-MOS ROAD

1,52,3

3,2

N-MOS ROAD

1,5

2,3

3,2

S-MOS ROAD

1,5

2,3

3,2

G-MOS TRAIN

2

3

4,3

N-MOS TRAIN

2

3

4,3

S-MOS TRAIN

23

4,3

Phone 3

Conv. Quality

TMOS SND

23

45

SLR

2/14

4/12

6/10

8

D-Value

-10

0

-5

5

BGNT (far)

10

0DT Type

2c

2b

2a1

Subj. Exp. Assessmentpoorfairgood

TCLw max

30

4050

70

TCLw nom

30

40

50

70

Echo vs. time

not Ok

OK

TMOS RCV

2

3

1,5

4

RLR nom

8/-46/-2

4/02

G-MOS CAR

23,5

5

N-MOS CAR

2

3,5

5

S-MOS CAR

2

3,5

5

G-MOS Mensa

2

3,4

5

N-MOS Mensa

2

3,4

5

S-MOS Mensa

23,4

5

G-MOS ROAD

1,52,3

3,2

N-MOS ROAD

1,5

2,3

3,2

S-MOS ROAD

1,5

2,3

3,2

G-MOS TRAIN

2

3

4,3

N-MOS TRAIN

2

3

4,3

S-MOS TRAIN

23

4,3

Phone 2

Conv. Quality

TMOS SND

23

45

SLR

2/14

4/12

6/10

8

D-Value

-10

0

-5

5

BGNT (far)

10

0DT Type

2c

2b2a

1

Subj. Exp. Assessmentpoorfairgood

TCLw max

30

4050

70

TCLw nom

30

40

50

70

Echo vs. time

not Ok

OK

TMOS RCV

2

3

1,5

4

RLR nom

8/-46/-2

4/02

G-MOS CAR

23,5

5

N-MOS CAR

2

3,5

5

S-MOS CAR

2

3,5

5

G-MOS Mensa

2

3,4

5

N-MOS Mensa

2

3,4

5

S-MOS Mensa

2

3,45

G-MOS ROAD

1,52,3

3,2

N-MOS ROAD

1,5

2,3

3,2

S-MOS ROAD

1,5

2,3

3,2

G-MOS TRAIN

2

3

4,3

N-MOS TRAIN

2

3

4,3

S-MOS TRAIN

23

4,3

Phone 1

Conv. Quality

Background Noise Performance

Colombia, 23-24 September 2013 15

Outline

Introduction

The impact of terminal design on speech quality

Assessment methods and standards for mobile terminals

Driver distraction and speech services in cars

Summary

Driver Distraction

Colombia, 23-24 September 2013 16

Physical/manual distractionManual operation of devices

Visual distractionWatching information other than road

Auditory distractionFocusing on auditory events not related tothe driving task

Cognitive distractionOccupied by non driving related tasks

Technology Based Distraction in Cars

Colombia, 23-24 September 2013 17

Car entertainment systems

Navigation systems

Phones for speech communication

Car information systems

Text messanging systems

„Texting while driving“

:

To be Avoided

Colombia, 23-24 September 2013 18

In general all activities distractingfrom the driving task, esp.:

Visual cues which require different focusthan road

Non intuitive manual operation of the car

Loading the human auditory system bydistracting cues

Unnatural and low quality dialogs and communications

Driving Task

Colombia, 23-24 September 2013 19

The driving task

=> Auditory Channel and Speech “available”

mostly occupied:

visual system

mainly involved:

hands and legs

involved:

auditory systemnot involved:

talking

Speech Based Services in Cars

Colombia, 23-24 September 2013 20

o The main speech applications:

• Speech recognition systems

• Speech dialog systems

• Text to speech systems

• Speech enhancement for communication systems

• Hands-free communication

• Enhanced in-car communication systems between passengers

Communicational Quality –Drivers Distraction

Colombia, 23-24 September 2013 21

Superior Communication

Quality -> ReducedDrivers Distraction

Seamless Interaction -> Low Delay,

Perfekt Double Talk Performance

Perfekt Intelligibility, High Speech

Naturalness at the Far End

Perfekt Intelligibility, High Speech

Naturalness in the Car

Hands-Free in Cars

Colombia, 23-24 September 2013 22

Why hands-free in cars?

Reduce phsyical distraction (dialing, holding the phone…)

Reduce visual distraction (watching display, keyboard …)

Superior Sound Quality & Intelligibility in the Car

Colombia, 23-24 September 2013 23

o Wideband services in mobile networks available soon->

o Enabling wideband telephony (100 Hz- 8 kHz) in cars• Fullband• Narrow band (car)• Wideband (car)

o Efficient use of the high quality audio systems in cars:• Getting superior sound quality• Increasing speech intelligibility• Increasing naturalness of a conversation• Reduce drivers distraction due to poor speech sound quality

Colombia, 23-24 September 2013 24

Bluetooth Link

Phoneadapter &antennaamplifier

Radio Link

Hands-free signal

processing

Caraudio

system

Car Hands-Free Testing

Colombia, 23-24 September 2013 25

analysisanalysis

Mobiles Sound Playback System (can be installed in any

vehicle)

Subwoofer

HFTACQUA

BackgroundNoise

Generatorand Analyzer

MFE VIUSBFrontend

PEQ IVProgrammableEqualizer

PEQ IVProgrammableEqualizer

PEQ IVProgrammableEqualizer

Radio Tester

(Analog)

(Digital)AES / EBU

(Digital)AES / EBU

(Digital)USB

(Digital)AES / EBU

Subwoofer

HFT

Subwoofer

HFTACQUA

BackgroundNoise

Generatorand Analyzer

MFE VIUSBFrontend

PEQ IVProgrammableEqualizer

PEQ IVProgrammableEqualizer

PEQ IVProgrammableEqualizer

Radio Tester

(Analog)

(Digital)AES / EBU

(Digital)AES / EBU

(Digital)USB

(Digital)AES / EBU

Test System

ITU-T Standards Contributing to Reduced Drivers Distraction

Colombia, 23-24 September 2013 26

ITU-T standards for Hands-Free Communication:

• ITU-T P.1100 for narrowband hands-free

• ITU-T P.1110 for wideband hands-free

ITU-T standards work in FG CarCOM:• New work on subsystem requirements for Hands-Free

Systems in Cars

ITU-T standards for Dialog Systems:

• P.851: Subj. evaluation of dialog systems

• Suppl. 24 to P. Rec.: Parameters describing the interaction with spoken dialog systems

The Main Section in ITU-T P.1100 & P.1110

Colombia, 23-24 September 2013 27

Definition of the test arrangement in a car

Description of a digital interface concept for development and debugging

Microphone test specification for separate microphones in a car

Measurement parameters and requirements for hands-free terminals

Bluetooth test interface for validation of telephone performance

Subjective test strategy for car to car communication

Conclusions and Recommendations

Mobile terminal testing is essential to provide sufficient or even superior QoE to the user – many operator have specific terminal standards for their suppliers

A variety of parameters contribute to the overall (conversational) speech quality

Network conditions have to be considered in the testing

Low driver distraction is essential to offer mobile services

Speech based operation is the key to the success of mobile services in cars

Both, terminals and networks have to be thoroughly engineered and adapted to each other

Colombia, 23-24 September 2013 28

Tandeming of Signal Processing in Terminals & Networks

Colombia, 23-24 September 2013 29

BaseStation

EC

ES

NR

Mobile Phone

EC

ES

NR

gain gain

Mobile Network

Netw. 1

Netw. 2