speech enhancement methods for vehicle applications
TRANSCRIPT
![Page 1: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/1.jpg)
International Telecommunication Union
“The Fully Networked Car, A Workshop on ICT in Vehicles”ITU-T Geneva, 2-4 March 2005
Speech Enhancement Methods for Speech Enhancement Methods for Vehicle ApplicationsVehicle Applications
Tim HaulickTEMIC Speech Dialog Systems
![Page 2: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/2.jpg)
2dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
Contents
o Overviewo Beamforming for vehicle applications
• Principle• Examples• Adaptive self calibration• Wind-noise suppression• Compact dual-microphone array
o Bandwidth Extensiono In-car communication
• Concept• Results of speech quality and intelligibility tests
![Page 3: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/3.jpg)
3dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
Integrated Hands-Free System
Microphone Array
phonephone
speech recognizerspeech
recognizer
speechoutputspeechoutput
bandwidthextension
bandwidthextension
ECEC
ECEC
ECEC
ECEC adap
tive
bea
mfo
rmin
gad
apti
ve b
eam
form
ing
noisereduction
noisereduction
+
![Page 4: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/4.jpg)
4dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
Beamforming
Structure of a generalized beamformerInt
erfere
nce
Target Signal
Steering Delay(Alignment)
T Filter
T
T
0
1 +
MicrophoneArray
m0
m1
mM-1
Filter
Filter
Filtering (Beampattern)
Output Signal
M-1
![Page 5: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/5.jpg)
5dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
Beamforming
speech signal
0
-10
-20- adaptive beamformer
- fixed beamformer
microphone arrayinterference
![Page 6: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/6.jpg)
6dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
Beamforming
Beampattern at 1500Hz for a rotating noise source
4 Microphones, d = 5cm
Att
enua
tion
[dB
]
blue – fixed beamformer
red – adaptive beamformer
0
-10
-20
speech signal
![Page 7: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/7.jpg)
7dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
Beamforming Microphone Array Integration
o Cost-efficient integration due to integrated microphone module
o Fixed steered beamformercould be used as driver direction of arrival (DOA) varies only within a small range (62°-75°)
o Microphone array could be used by driver and co-driver
4 Microphone Array Integrated in Interior Mirror
Mirror
Microphone Module
5 cm
![Page 8: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/8.jpg)
8dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
Driving Situation 120 km/h
Significantly higher noise suppression in the low frequency range with the adaptive beamformer compared to the delay & sum beamformer
Beamforming Examples
Att
enua
tion
[dB
]
Frequency [Hz]Time [s]
![Page 9: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/9.jpg)
9dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
Interfering Co-Driver
Beamforming Examples
0 1 2 3 4 5 6time [s]
microphonefixed beamformeradaptive beamformer
Co-Driver Driver/ Co-Driver
Suppression of interferer >15dB by adaptive beamformer
Single Microphone
Fixed Beamformer
Adaptive Beamformer
![Page 10: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/10.jpg)
10dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
Beamforming Examples
0
10
20
30
40
50
60
70%
130km/h 160km/h 100km/h,defroster on
reduction of word error rate referring to a single beamformer microphone
Speech recognition tests: 50 speakers, 1000 digits strings with in sum 9000 digits per situation
Reduction of word error rate by adaptive beamformer is significantly higher than 50%!
![Page 11: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/11.jpg)
11dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
BeamformingAdaptive Self-CalibrationBeamformers are very sensitive to a mismatch of the microphones. Mutual deviations of the individual microphones may provoke a significant distortion of the beamformer output signal. Deviations inevitably occur due to fabrication tolerances and aging of the microphones.
Problem:
Solution: The mutual deviations of the microphones are compensated in a preprocessing unit which adjusts itself adaptively without being noticed by the driver.
Benefits: o A costly calibration of the beamformer can be saved o Aging effects are tracked
Self-Calibration Beamformer
![Page 12: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/12.jpg)
12dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
BeamformingAdaptive Self-Calibration
0 2 4 6 8 10 12 14 16time [s]
microphone signal
adaptive beamformer
adaptive beamformerwith self-calibration
![Page 13: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/13.jpg)
13dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
0 2 4 6 8 1 0-3
-2
-1
0
1
2
3x 1 0
4
t i m e [ s ]0 2 4 6 8 1 0
-3
-2
-1
0
1
2
3x 1 0
4
t i m e [ s ]
Microphone Signal Beamformer Output Signal
Problem: Wind noise can provoke strong pulse-like disturbances of the microphone signals. In cars, this problem is mainly caused by the fan or an open top of a convertible. Due to design reasons or lack of space the standard wind shield of the microphones is often insufficient.
Solution: Suppression of wind buffets by a (multi-channel) wind-noise suppression algorithm
BeamformingWind Noise Suppression
Mic
![Page 14: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/14.jpg)
14dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
Compact Dual-Microphone-Array
housing with optimized wind-protection
microphone directed to driver
microphone directed to co-driver
-2
0
2x 10
4 Microphone Signal (Driver)
-2
0
2x 10
4 Processed: Hands-free Mode
0 2 4 6 8-2
0
2x 10
4
time [s]
Processed: Recognizer Mode
Mic
RecDriver Co-Driver
HF
![Page 15: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/15.jpg)
15dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
Bandwidth Extension
Problem:
Solution: Extrapolation of missing frequency components from the received speech signal
Degradation of speech quality due to the bandwidth limitation of the telephone network
Telephone Network
Bandwidth Extension
![Page 16: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/16.jpg)
16dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
In-Car Communication
Current Situation:o Communication between passengers is difficult,
because of the acoustic loss (especially front to back
o Front passengers have to speak louder than normal – longer conversations will be tiring
o Driver turns around – road safety is reducedSolution:o Improve the speech quality and intelligibility by
means of an intercom system
Application:o Mid and high class automobiles, which are
already equipped with the necessary audio and signal processing
o Vans, etc. g systems with reduced quality
Passenger compartment
*Acoustic loss (referred to the ear
of the driver)
-5…-15dB*
![Page 17: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/17.jpg)
17dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
In-Car Communication Implementation
One-Way Systemo 2-4 microphoneso 2-4 loudspeakers
Two-Way Systemo 4-8 microphoneso 6-8 loudspeakers
Intercomsystem
Intercomsystem
![Page 18: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/18.jpg)
18dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
In-Car CommunicationSignal Processing Components
Problems and Challenges:o Stability o System delayo Correlation of excitation and distortion
Algorithmic Structure for One Direction (Front g Rear):
Frontmicro-phones
Frontloud-speakers
Echocancellation
Beam-forming
Feedbackcancellation
Feedbacksuppression
Prepro-cessing
Losscontrol
Postpro-cessing
Rearloud-
speakers+ +
![Page 19: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/19.jpg)
19dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
In-Car CommunicationDemo System
4 microphones withinthe front top control unit
2x2 microphones (integratedwithin the rear grab handles)
![Page 20: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/20.jpg)
20dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
In-Car CommunicationSubjective Tests
o Driving-Situations• 0km/h beside motorway• 130km/h on motorway
o Prerecorded speech examples with different Lombard levels were played back via an artificial mouth
o Binaural recordings were made by means of a HEADacoustics NoiseBookon the seat behind the driver
![Page 21: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/21.jpg)
21dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
In-Car CommunicationResults of Speech Quality Test (CMOS-
Test)
o 0 km/h, vehicle parked close to a motorway• 19,7% prefer the system to be
switched off• 29,7% have no preference• 50,7% prefer the system to be
switched on
o 130 km/h, motorway• 4,3% prefer the system to be
switched off• 7,1% have no preference• 88,6% prefer the system to be
switched on
25 signal pairs per driving situation (intercom on/off) / 15 listeners per scenario
![Page 22: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/22.jpg)
22dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
In-Car CommunicationResults of Speech Intelligibility Test
(MRT)
o 0 km/h, vehicle parked close to a motorway• No significant difference (95.2% correct answers for system on
versus 95.0% for system off) • Due to the automatic gain adjustment the intercom system
operates with only very small gain at these noise levels
48 utterances were presented to each listener per driving situation
o 130 km/h, motorway• Significant
improvement of speech intelligibility by the intercom
• Nearly 50% error reduction
![Page 23: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/23.jpg)
23dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
Conference Calls/ In-Car Communication
System Functionality:o Multi-channel hands-free system for driver
and co-driver or passengers on the back seatso Conference calls with up to 4 partners with
intercom functionality from the front to the backo Intercom functionality between passengers in the front and in the
backo Speech recognition capabilities available for all seats
"Hello...
"Hello... "Hello..
"Hello.. "Hello..
GSM
![Page 24: Speech Enhancement Methods for Vehicle Applications](https://reader030.vdocuments.us/reader030/viewer/2022012409/616a4eb511a7b741a3511201/html5/thumbnails/24.jpg)
24dates
ITU-T
The Fully Networked Car, A Workshop on ICT in VehiclesITU-T Geneva, 2-4 March 2005
Thank you for your Thank you for your attention!attention!