foreground speech segmentation and enhancement using ... · 12 db as opposed to existing spectral...

2329-9290 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TASLP.2016.2549699, IEEE/ACMTransactions on Audio, Speech, and Language Processing

FOREGROUND SPEECH SEGMENTATION AND ENHANCEMENT USING GLOTTAL CLOSURE INSTANTS AND MEL CEPSTRAL COEFFICIENTS 1

Foreground Speech Segmentation and Enhancementusing Glottal Closure Instants and Mel Cepstral

CoefficientsK. T. Deepak and S. R. M. Prasanna,Senior Member, IEEE

Abstract—In this work, the speech signal recorded from thedesired speaker close to microphone in natural environmentis regarded as foreground speechand rest of the interferingsources asbackground noise. The proposed work exploits speechproduction features like Glottal Closure Instants (GCIs) intime domain and vocal tract information in spectral domain tosegment the desired speaker’s speech and to further enhanceit. The foreground speech is perceptually enhanced using theauditory perception feature in mel frequency domain using melcepstral coefficients (MCC) and its inversion using mel logspectrum approximation (MLSA) filter. The focus is on enhancingthe production and perceptual features of foreground speechrather than relying on modeling the interfering sources. Thespeech data is collected in different natural environments fromdifferent speakers in order to evaluate the proposed method. Theenhanced speech signals derived at three different stages of theproposed method are evaluated with state-of-the-art methods interms of subjective and objective measures. The proposed methodprovides improved performance compared to the consideredstate-of-the-art methods. In terms of the proposed objectivemeasureForeground to Background Ratio (FBR), the enhancementapproach presented in this work gives an average improvement of12 dB as opposed to existing spectral subtraction based methodwhich provides 3 dB. Moreover, subjective evaluation using 24different subjects corroborates the objective test results.

Index Terms—Foreground segmentation, zero band filter(ZBF), glottal closure instants (GCI), formant peaks, MCC,MLSA, speech enhancement.

I. I NTRODUCTION

I N most of the cases, the speech signal recorded in natu-ral environment is degraded by other interfering acoustic

sources. The natural environment typically refers to an of-fice or laboratory environment with relatively high acousticbackground noise like call centers. The degradation can be ofdifferent levels and consists of one or more interfering sources.Speech enhancement is still a challenging task when signal isrecorded using a single sensor in natural environment [1]. Thetask is complicated if the interfering sources are from otherbackground speakers. The perceptual quality improvement interms of intelligibility and reduction in the background noiseto enable comfortable speech communication is of paramountimportance [2]. In such cases, it is of interest to segmentthe desired speaker’s speech from rest of the interferingbackground noise first and then enhance the desired speaker’sspeech.

K. T. Deepak and S. R. M. Prasanna are with the Department of Elec-tronics and Electrical Engineering, Indian Institute of Technology Guwahati,Guwahati 781039, India e-mail: (deepakkt, [email protected]).

The general approach of most methods is to model theadditive noise only components from noisy speech signaland suppress them to obtain the enhanced speech [3], [4].The noise modeling depends on efficient segmentation ofadditive background noise from rest of the speech regions.The suppression of noise usually takes place in spectral do-main through subtraction. However, such spectral subtractionmethods introduce distortions in enhanced speech because ofoverestimating the noise spectrum in the form of undesiredmusical tones. Many methods were proposed in order toovercome this problem [5], [6]. The human auditory perceptualcues were considered to improvise the spectral subtractionmethods [7], [8]. The objective is not to completely suppressthe additive noise. Rather, the purpose is to utilize the maskingproperties of human auditory system and live with certainamount of residual noise that are below the masking threshold.The advantage of such algorithms over conventional methodsis that, they do not introduce spectral distortions in the formof audible musical noise due to over subtraction. However,utilizing complex auditory phenomena like psychoacousticmodels for the sake of speech enhancement can make suchmethods to be more complex.

There is some evidence that humans perceive speech bycapturing the features from high signal to noise ratio (SNR)regions and extrapolate those features to low SNR regions,both in temporal and spectral domains [9]. The idea is toenhance the high SNR regions further relative to low SNRregions for enhancing the noisy speech signal. This is equiv-alent to the phenomenon of Lombard effect, where, speakeremphasizes the production apparatus to increase SNR of theproduced speech when background noise in feedback pathincreases [10]. There are some methods proposed in literaturethat basically utilize the high SNR regions like instants ofsignificant excitation of speech signal in the temporal domainto enhance speech [11]. The idea is to emphasize instantsof significant excitation which are predominantly glottal clo-sure instants (GCIs) relative to other regions of LP residualsignal. The GCIs are located using Hilbert envelope of LPresidual (HELP), using which, a temporal weighting functionis derived so to enhance GCI positions relative to otherregions. The synthesized speech signal from such temporalenhancement may not completely eliminate the backgroundnoise, nevertheless, the distortion caused by such enhancementmethods are minimal. The recently proposed method in [12]utilizes temporal enhancement as the preliminary stage ofenhancement and subsequently uses spectral enhancement to




eliminate the remaining residual noise. The advantage of thismethod is that it causes lesser spectral distortion.

There is evidence that when speech signal is recorded innatural environment, the speech production characteristics tendto vary depending on the levels of interfering sources [13].Therefore, speech enhancement techniques may have to focuson exploiting such unique characteristics of speech signal. Thepresent work focuses on practical scenario when the speechsignal is recorded in different natural environments. In a typ-ical recording scenario (head mounted microphone or mobilephones held close to ears), the desired speaker is closer tomicrophone relative to other interfering sources. In this workit is assumed that the proximity of desired speaker is closestto microphone compared to the distance between other soundsources and the microphone. The speech signal so recordedfrom desired speaker is termed asforeground speechand restof the interfering sources are termed asbackground noise.Due to close proximity of speaker to microphone and alsothe modified speech production characteristics due to acousticfeedback, there are significant differences in the nature of sig-nal for foreground speech and the background noise [14], [15].The core idea being that not all foreground speech regions areaffected equally by interfering background noise. In particular,the instants of significant excitation in temporal domain andformant locations in the spectral domain remain robust tobackground noise. Hence, such regions form high SNR regionsof speech signal that can be utilized for speech enhancement.Furthermore, the proposed method utilizes the subtle aspectsof human auditory feature to enhance the foreground speech.The merits of proposed method lies in exploiting the speechproduction features such as excitation source and vocal tractinformation along with auditory perception feature and studythe benefits of each in terms of speech enhancement.

The rest of the paper is organized as follows: Section IIillustrates the overall block diagram of the proposed methodand explains foreground segmentation in detail. The SectionsII-A and II-B describe excitation source based features andvocal tract articulatory feature, while, Section II-C explainsthe combination of features for foreground speech segmenta-tion. Section III-A describes the details of excitation sourcebased enhancement, while, Section III-B explains the detailsof formant based enhancement using a block diagram. TheSection III-C explains the details of analysis and synthesissub modules using mel cepstral coefficients (MCC) and mellog spectral approximation (MLSA) filter, respectively. SectionIV describes the details of the evaluation procedure and resultsobtained in terms of subjective and objective evaluation scores.The summary and conclusions of the present work, and thescope for future work are mentioned in Section V.

II. FOREGROUNDSPEECHSEGMENTATION

The proposed work mainly consists of foreground speechsegmentation and multistage foreground speech enhancementmodules. Ifs(n) is the speech signal recorded in foregroundscenario from a natural environment, then there can be otherinterfering sources along with foreground speech signal. Theobjective of the current work is to first temporally segment

Foreground Speech

Segmentation

Filtering

Zero Band

Temporal

Weight

Function

LP Analysis

Order

LP Synthesis

Formant

Enhancement

Mel Cepstral

Coefficients

Epoch Extraction

Inverse

Filtering

Speech

Applications

Mel Log Spectral

ApproximationFilter

Foreground Speech Segmentation

Excitation Source based Enhancement

Perceptual based Enhancement

s(n)

wg(n)

w(n)

l(n)

rw(n)

st(n)

sf (n)

sp(n)

pth

st(n)

sf (n)

Fig. 1: The overall block diagram of the proposed foreground speech segmentationand enhancement method, where,s(n) is the input speech signal recorded in foregroundscenario,wg(n) is the gross weight function that mainly segments the foreground speechregions from rest of the background noise,w(n) is the final temporal weight function,l(n) is the LP residual signal,rw(n) is the temporally weighted LP residual signal,st(n) is excitation based enhanced output,sf (n) is formant based enhanced outputandsp(n) is perceptually enhanced output.

foreground speech regions from rest of the background noiseand then enhance foreground speech regions. The overall blockdiagram of the proposed work is shown in Fig. 1. The detailsof foreground speech segmentation module is explained inthis Section. The acoustic background noise picked up bythe recording microphone can be speech like. It is thereforedifficult to distinguish the desired foreground speech fromrest of the background noise. However, it is important totemporally segment foreground speech regions from rest ofthe background noise for subsequent processing to enhanceforeground regions.

A. Excitation Source based Features

Due to the close proximity of foreground speaker to micro-phone relative to other acoustic sources there are significantdifferences in the signal characteristics of foreground speechand background noise [15]. Most of the voice activity detectionmethods developed either in temporal or spectral domainshave to deal with all frequency components of speech signal.Alternatively, a method is developed in [15] that analysesforeground speech and background noise at zero frequencyusing zero frequency filter (ZFF) [16]. The ZFF allows signalcomponents around zero frequency and significantly attenuates




rest of the frequency components. The output of ZFF is calledas zero frequency filtered signal (ZFFS) using which twotemporal domain featuresviz., normalized first order auto-correlation coefficient (NACC) and ZFFS energy are derivedto segment foreground speech from rest of the backgroundcontent. However, ZFF is a marginally stable filter and theoutput of filter grows or decays exponentially and such a filterrequires remove trend procedure to obtain ZFFS. The trend isremoved by subtracting the average over 1 to 2 pitch periodfrom each sample of ZFF output. In a method proposed [17]recently, the stable realization of resonator filter called zeroband filter (ZBF) is used to obtain zero band filtered signal(ZBFS), similar to ZFFS. The advantage of using ZBF is that,the output of the filter is converging type and does not requireF0 estimation.

Let s(n) be the foreground speech signal recorded in naturalenvironment that can have other interfering sources. The firstorder difference of this signal is given by

d(n) = s(n)− s(n− 1) (1)

The difference signald(n) is to deemphasize low frequencyfluctuations and emphasize high frequency components presentin signal and is passed twice through the transfer functions asgiven in [17]

H1(z) =Y1(z)

D(z)=

1

1− 2rz−1 + r2z−2(2)

H2(z) =Y2(z)

Y1(z)=

1

1− 2rz−1 + r2z−2(3)

where,r represents the value of radius on unit circle in z-plane at which the poles are placed, andr value should satisfy0 < r < 1 for stability and its value is taken as0.99 in ZBF.In time domain, the filter outputs can be represented in theform of summation series given by

y1(n) =

∞∑

k=0

(k + 1)rkd(n− k) (4)

y2(n) =∞∑

k=0

(k + 1)rky1(n− k) (5)

The outputy2(n) is filtered using a Butterworth4th orderhigh pass filter having a cutoff frequency 80 Hz in orderto attenuate low frequency fluctuations. The nature of theresulting ZBFS is different for background acoustic sourcescompared to foreground speech. It can be noted from [15] that,the nature of ZBFS is nearly periodic and has higher amplitudelevels for foreground speech, while, ZBFS appears to bedispersed in case of background speech and other acousticsources. Also, the amplitude levels of ZBFS is significantlylow in background noise regions compared to foregroundspeech regions. This can be attributed to the fact that, theproximity of foreground source is closer to microphone thanother acoustic sources. This is illustrated in Fig. 2, where,Figs. 2(a) and (b) show 50 ms segments of foregroundspeech and background speech chosen from same recording,respectively. The signal is recorded at 16 kHz sampling rateusing headphone microphone connected to laptop in a living

room. The speaker is closer to microphone while talkingand a television is playing speech at the background. Since,the signal is recorded in natural environment using singlemicrophone sensor there is no reference clean speech signalavailable to measurea priori signal to noise ratio (SNR).Hence, a similar measurement can be made using foregroundto background ratio (FBR) that essentially computes the ratioof normalized foreground speech to background power and it isexpressed in decibels (dB). The detailed description of FBR isgiven in Section IV-B3. The segments shown in Figs. 2(a) and(b) are taken from signal having FBR 6.27 dB. The Figs. 2(e)and (f) illustrate the corresponding ZBFS, while, Figs. 2(i) and(j) show normalized autocorrelation sequence computed fromZBFS, respectively. The slope value measured at positive zerocrossings of ZBFS indicates the strength of excitation (SoE)[18]. The Figs. 2(m) and (n) show SoE computed from ZBFSsegments shown in Figs. 2(e) and (f), respectively. The Figs.2(q) and (r) illustrate ZBFS normalized energy computed usingthe segments shown in Figs. 2(e) and (f), respectively. TheZBFS normalized energy is computed using a frame size of 5ms with one sample shift. However, for illustration purpose theZBFS energy is sampled at glottal closure instants to comparethem with SoE. It can be noticed that, the signal contours ofSoE and ZBFS energy follows similar trend and hence offersthe same discriminative abilities to distinguish foreground andbackground speech regions. The similar trend of these twosignals can be attributed to the fact that ZBFS energy dependson SoE and hence they are related. The Fig. 2(c) shows 50 mssegment of background music with vocals chosen from signalhaving FBR 5.21 dB. The signal is recorded from a foregroundspeaker when television is playing background music withvocals at the background. The corresponding ZBFS, nor-malized autocorrelation sequence computed from ZBFS, SoEderived using ZBFS and ZBFS normalized energy plots areshown in Figs. 2(g), (k), (o) and (s), respectively. Further, Fig.2(d) shows 50 ms segment of background noise chosen fromanother recording. The speech signal is recorded in foregroundscenario from a speaker in office environment when mosaicpolishing machine is operating at the background. The segmentis chosen from signal having FBR 7.38 dB. The correspondingZBFS, normalized autocorrelation sequence computed fromZBFS, SoE derived using ZBFS and ZBFS normalized energyplots are shown in Figs. 2(h), (l), (p) and (t), respectively.

It can be observed that, the ZBFS derived from foregroundspeech region appears to be nearly periodic, while, it appearsdistorted for background speech, music, and noise regions.This fact is further illustrated using normalized autocorrelationsequence plots. It can be noticed that the value of first largestpeak (excluding center) normalized with respect to the centervalue in autocorrelation sequence defined as normalized auto-correlation coefficient (NACC) is larger in case of foregroundspeech regions compared to background regions. The reasonfor such a change in ZBFS characteristics can be ascribedto the fact that, foreground source is closer to microphonesensor and hence the excitation source is least affected byinterfering sources. The amplitude level of ZBFS is directlyrelated to epoch strength of excitation source and hence energyat foreground speech region is relatively larger compared




0 10 20 30 40 50

−0.2

−0.1

0

0.1

0.2

0 10 20 30 40 50

−0.2

−0.1

0

0.1

0.2

0 10 20 30 40 50

−0.2

−0.1

0

0.1

0.2

0 10 20 30 40 50

−0.2

−0.1

0

0.1

0.2

0 10 20 30 40 50−1

−0.5

0

0.5

1

0 10 20 30 40 50−1

−0.5

0

0.5

1

0 10 20 30 40 50−1

−0.5

0

0.5

1

0 10 20 30 40 50−1

−0.5

0

0.5

1

0 10 20 30 40 50−1

−0.5

0

0.5

1

0 10 20 30 40 50−1

−0.5

0

0.5

1

0 10 20 30 40 50−1

−0.5

0

0.5

1

0 10 20 30 40 50−1

−0.5

0

0.5

1

Time (ms)

0 10 20 30 40 50

0

0.02

0.04

0.06

0.08

0.1

0 10 20 30 40 50

0

0.05

0.1

0 10 20 30 40 50

0

0.05

0.1

0 10 20 30 40 50

0

0.05

0.1

0 10 20 30 40 50

0

0.2

0.4

0.6

0.8

0 10 20 30 40 50

0

0.2

0.4

0.6

0.8

0 10 20 30 40 50

0

0.2

0.4

0.6

0.8

0 10 20 30 40 50

0

0.2

0.4

0.6

0.8

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

(k)

(l)

(q)

(r)

(s)

(t)

(m)

(p)

(o)

(n)

Fig. 2: 50 milliseconds of (a) foreground speech, (b) background speech, (c)background music with vocals, and (d) background noise. Respective, ZBFS ((e) -(h)), normalized autocorrelation sequence using ZBFS ((i) - (l)), strength of excitationcomputed using ZBFS ((m) - (p)), and ZBFS energy sampled at glottal closure instants((q) - (t)).

to background regions. Hence, such discriminative featuresderived from ZBFS can be used to segment foreground speechfrom rest of the background noise that may include speech likesources. This is further illustrated by choosing a full sentencespoken by foreground speaker in Fig. 3. The Fig. 3(a) showsspeech signal recorded from a male speaker in office, while,mosaic polishing machine is operating in background, the FBRvalue is 7.38 dB. The signal is recorded using a headphonemicrophone connected to a laptop for recording. It can beobserved that there is significant amount of noise present inbackground. The NACC and ZBFS energy computed fromsignal is shown in Figs. 3(b) and (c), respectively. The twofeatures derived from ZBFS offers discriminative informationbetween foreground and background regions. The NACC andZBFS energy are relatively higher in case of foreground speechregions relative to background noise regions. Hence, thesestwo features can be used to segment foreground speech fromrecorded signal.

B. Vocal Tract Articulatory Feature

The features discussed so far are derived using excitationsource information and does not make use of vocal tractarticulatory gestures that are unique to foreground speech. Oneway of capturing vocal tract information is by measuring thespectral envelope. However, it is suggested that, the temporaldynamics of spectral envelopes as more reliable means forcarrying the linguistic context of speech message and canbe obtained through modulation spectrum energy of speechsignal. The temporal envelope of speech is dominated bylow frequency components that are in similar range to thedynamics of speech production, in which, the articulatorsmove at such slow rate [19]. It is studied that, the linguisticinformation of speech signal lies in the range of 2 to 16Hz and centered at 4 Hz. Subsequently, filtering slow andfast varying trajectories of spectral envelopes can be usefulin alleviating the effects of interfering background sources.Hence, modulation spectrum energy can be used as a feature tosegment foreground speech from rest of the background noise.In this work, the modulation spectrum energy is extractedfrom signal as given in [20]. However, the signal is divided

into 18 subbands using compressive gammachirp auditoryfilter (cGC) [21], [22]. The auditory filter designed is leveldependent and nonlinear, this emulates psychophysical dataon masking and two tone suppression. The cGC consistsof two filters viz., a passive gammachirp filter (pGC) anda dynamic filter which is asymmetric function that shifts infrequency with stimulus level. The cGC filter is realized usingthe following relationship

gc(t) = atn1−1exp(−2πb1ERBN(fr1)t)

× exp(j2πfr1t+ jc1 ln t+ jφ1)(6)

where, a is amplitude;n1 and b1 are parameters definingthe envelope of gamma distribution;c1 is chirp factor;fr1is frequency referred to as asymptotic frequency, since theinstantaneous frequency of carrier converses to it whent isinfinity; ERBN(fr1) is equivalent rectangular bandwidth ofaverage normal hearing subjects;φ1 is initial phase; andln tis natural logarithm of time. The Fourier magnitude spectrumof cGC is given by

|Gc(f)| = aΓ. |GT (f)| . exp(c1θ1(f)) (7)

|θ1(f)| = arctan

(

f − fr1b1ERBNfr1

)

(8)

where,|GT (f)| is the Fourier magnitude spectrum of gamma-tone filter; exp(c1θ1(f)) is an asymmetric function becauseθ1 is an antisymmetric function centered at asymptotic fre-quencyfr1 andaΓ is constant. Further the asymmetric func-tion exp(c1θ1(f)) is decomposed into lowpass and highpassasymmetric filter functions to represent passive and dynamiccomponents separately. The resulting compressive cGC filter|Gcc(f)| is

|Gcc(f)| = [aΓ. |GT (f)| . exp(c1θ1(f))]. exp(c2θ2(f)) (9)

|Gcc(f)| = |Gcp(f)| . exp(c2θ2(f)) (10)

The compressive gammachirp is composed of a level in-dependent passive gammachirp filter (pGC)Gcp(f) whichrepresents passive basilar membrane and a level dependenthighpass asymmetric function which simulates active compo-nent in the cochlea. The Amplitude envelope is computed fromeach individual subband outputs obtained from filter bank.Each of the filter’s output is first halfwave rectified and thensubjected through a lowpass filter having a cutoff frequency28 Hz. The amplitude envelope obtained is downsampled bya factor of 100 and normalized by average value obtainedfrom the respective filter bank output. The modulations ofthe normalized envelope signals obtained are further analyzedusing Discrete Fourier Transform (DFT). The DFT is com-puted using 250 ms Hamming window with shift of 12.5ms, this essentially captures dynamic properties of the signal.However, 2-16 Hz components from each such channel aresummed together to obtain the modulation spectrum energy.The modulation spectrum energy can be computed using thefollowing relationship




0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2−0.5

0

0.5

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.5

1

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.5

1

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.5

1

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.5

1

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.5

1

time (s)

(a)

(b)

(f)

(e)

(d)

(c)

Fig. 3:The figure illustrates the foreground segmentation using the combined evidencefrom different features (a) speech signal recorded in foreground scenario, (b) normalizedfirst order autocorrelation coefficients derived from ZBFS, (c) short term energy derivedfrom ZBFS, (d) modulation spectrum energy derived from speech signal, (e) combinedevidence, and (f) combined evidence passed through sigmoidal function to segment theforeground speech regions from rest of the background noise.

m(i) =

18∑

p=1

k=k2∑

k=k1

| Sp(k, i) |2 (11)

where,i is frame index,p represents critical band filter andk1,k2 represents frequency index of 4 Hz and 16 Hz, respectively.Sp(k, i) is obtained using the following relationship

Sp(k, i) =

N−1∑

n=0

sp(n+ i× F )w(n)e−j2πnk/N (12)

where, sp(n) represents normalized envelope ofpth filteroutput,F is frame shift,w(n) is Hamming window, andNis number of points used for computing DFT. The modulationspectrum energy computed for each frame are upsampled to16000 samples/s. The modulation spectrum energy computedfrom the signal shown in Fig. 3(a) is illustrated in Fig. 3(d).It can be noticed that, the modulation spectrum energy valuesare significantly higher in foreground speech regions comparedto background noise regions. This can be attributed to thefact that vocal tract articulatory gestural movements operatein an exclusive frequency range of 2-16 Hz compared to otheracoustic sources. Hence, the modulation spectrum energy canbe used to distinguish foreground speech regions from rest ofthe background content.

C. Combined Evidence

The excitation source based features such as NACC andZBFS energy are combined with vocal tract based modulationspectrum energy in order to identify the foreground speechregions. In order to illustrate the concepts of NACC, ZBFS en-ergy, and modulation spectrum energy, features are computedwith 1 sample shift and each such signal is amplitude normal-ized. Let us denote such normalized sequence of NACC, ZBFSenergy, and modulation spectrum energy asNc(n), ZE(n),andME(n), respectively, where,n represents the number ofsamples in a given signal. The features are temporally addedand normalized with respect to maximum value of the addedsequence to obtain the combined evidence given byE(n) =

(Nc(n)+ZE(n)+ME(n))/max(Nc(n)+ZE(n)+ME(n)).The combined evidence of all three features is shown inFig. 3(e). However, it is difficult to set the threshold directlyon such signal for foreground segmentation. Alternatively,the gross level feature is obtained by passingE(n) throughsigmoidal function given by

wg(n) = (1− wgm)1

1 + exp (−λ(E(n) − Th))+ wgm (13)

where, wg(n) is sigmoidal function ofE(n), λ is slopeparameter set to 20,Th is threshold derived from meanvalue of the signalE(n) and wgm is minimum value ofsigmoidal function which is set to 0 in this case. Thewg(n)function forms gross level feature that mainly helps to segmentforeground speech in the presence of background noise. Thisis illustrated using Fig. 3(f), where, the foreground regionsare further enhanced relative to other background regions. Theweight functionwg(n) derived from three features can be usedas gross level feature to temporally enhance the foregroundspeech.

III. F OREGROUNDSPEECHENHANCEMENT

The foreground speech enhancement is carried using mul-tiple stages. The proposed enhancement scheme is explainedusing the block diagram shown in Fig. 1, classified into fourmajor modulesviz., foreground speech segmentation, excita-tion source based enhancement, formant based enhancementand perceptual based enhancement. The foreground speechsegmentation is discussed in Section II. The enhanced speechsignal obtained from different modules are named as excitationsource based enhancement (st(n)), formant based enhance-ment (sf(n)), and perceptual based enhancement (sp(n)).It can be observed that all three modules are connectedsequentially.

A. Excitation Source based Foreground Speech Enhancement

The excitation source information, especially instants ofsignificant excitation are high SNR regions which can beused as anchor points to enhance the foreground speechregions. A robust method is required to locate instants ofsignificant excitation even when speech signal is recorded innoisy environments. It is shown in [17] that, ZBF is robust inidentifying the locations of GCIs in foreground scenarios andthere is no requirement for estimation ofF0. The positive zerocrossings of ZBFS derived from Eqn. (5) precisely match withthe locations of GCIs. In order to modify excitation sourcesignal using GCI locations, it is beneficial to resolve speechsignal in terms of source and filter components.

In linear prediction (LP) analysis, the vocal tract systemcan be modeled as a time varying all pole filter using framebased analysis. The LP analysis works on the principle that, thecurrent speech sample can be predicted from pastp samples,where, p is called linear prediction order and is chosen asFs/1000+ 4 (Fs is sampling frequency). In order to computeLP coefficients, the frame size is selected to be 20 ms with aframe shift of 10 ms. Ifs(n) denotes speech signal recorded




in foreground scenario, then the predicted sample at the timeinstantn is given by

s(n) = −

p∑

k=1

aks(n− k) (14)

where,ak is the set of LP coefficients predicted. The residualerror is the difference between actual sample sequences(n)and predicted sample sequences(n) and it is given by thefollowing relationship

l(n) = s(n)− s(n) (15)

From Eqns. (14) and (15), the residual signall(n) can bewritten in z-domain transfer function as

L(z) = S(z) +

p∑

k=1

akS(z)z−k (16)

i.e.,

A(z) =L(z)

S(z)= 1 +

p∑

k=1

akz−k (17)

where, the LP residual signal can be obtained by filteringspeech signalS(z) through the filterA(z), which is generallycalled as inverse filtering. The prediction error is relativelyhigh at GCI locations compared to other regions of speechsignal. Hence, the amplitude level of LP residual signal ishigher at GCI locations compared to other regions of LPresidual signal. The LP residual signal is uncorrelated andtherefore any modification of such residual signal introducesleast distortion for later synthesis of speech signal. It is shownin [11] that, the enhanced speech can be synthesized bymodifying LP residual signal without much audible distortion.The LP residual signal is derived from noisy speech signal andmodified by retaining 2 ms regions around GCIs. The methodproposed in [12] uses similar approach by utilizing GCIs asanchor points obtained from HELP. The identification rate(IDR) and accuracy (IDA) of locating GCIs using HE of LPresidual is inferior in case of noisy speech signal compared torecently proposed methods [23]. Alternatively, in the proposedmethod, ZBF is used to locate GCIs directly from speechsignal recorded in natural environment.

This is illustrated in Fig. 4, where, Fig. 4(a) shows fore-ground speech signal recorded when music is playing in thebackground. It can be noticed that there is significant amountof background music present along with foreground speechsignal. The foreground speech regions are shown in Fig. 4(a)as dotted lines and the corresponding ZBFS can be obtainedby passing speech signal through Eqns. (2) and (3) as shownin Fig. 4(b). The positive zero crossings of ZBFS correspondsto GCI locations of speech signal. The region around GCIlocations can be used as anchor points to modify the LPresidual signal. The LP residual signal derived from speechsignal is shown in Fig. 4(e). In order to emphasize the regionsaround GCI locations, a fine weight function is obtainedsimilar to [12]. The GCI location is convolved with Hammingwindow functionhw(n) having a temporal duration of 3 msthat closely corresponds to closed phase interval of a glottal

0 0.2 0.4 0.6 0.8 1 1.2−1

0

1

0 0.2 0.4 0.6 0.8 1 1.2−1

0

1

0 0.2 0.4 0.6 0.8 1 1.20

0.5

1

0 0.2 0.4 0.6 0.8 1 1.20

0.5

1

0 0.2 0.4 0.6 0.8 1 1.2−0.2

00.2

0 0.2 0.4 0.6 0.8 1 1.2−0.1

00.1

0 0.2 0.4 0.6 0.8 1 1.2−1

0

1

time (sec)

(f)

(e)

(g)

(d)

(c)

(b)

(a)

Fig. 4: Illustration of excitation based enhancement. (a) speech signal recorded inforeground scenario (dotted lines and arrows indicate the foreground region), (b) thepositive zero crossings of ZBFS indicate the epoch locations, (c) the foreground weightfunction wg(n), (d) the temporal weight functionw(n) by combining the evidenceof epoch locations with foreground segmentation, (e) LP residual signal derived fromspeech signal, (f) modified LP residual signalrw(n) (g) excitation based enhancedspeech signal synthesized.

cycle. If GCIs are considered as shifted train of impulses, thenthe fine weight functionwf (n) is given by

wf (n) =

(

Nk∑

k=1

δ(n− ik)

)

∗ hw(n) (18)

where,Nk is total number of GCIs located,ik is estimatedlocation of GCI. The minimum value ofwf (n) is set toa threshold value ofT in order to keep the distortion lowbecause of overemphasizing GCI locations in LP residual andthe relationship is expressed as

wf (n) =

T, if wf (n) < Twf (n), otherwise

(19)

where,T is set to 0.5 in this work. It can be noted that, tempo-ral processing is not sensitive to a range ofT values [12]. Thefinal weight functionw(n) is obtained by multiplying grossweight functionwg(n) as shown in Fig. 4(d) with fine weightfunctionwf (n) and it is expressed as

w(n) = wg(n)× wf (n) (20)

The normalized final weight functionw(n) is multiplied toresidual signall(n) to obtain the weighted LP residual signal(WLPR)rw(n) as shown in Fig. 4(f). The temporally enhancedspeech signalst(n) can be synthesized by the transfer functiongiven in z-domain as

St(z) =Rw(z)

1 +∑p

k=1 akz−k

(21)

where,St(z) is temporally enhanced speech signal andRw(z)is WLPR in z-domain, while,ak are the LP filter coefficients.The temporally enhanced speech signal is shown in Fig.4(g). Temporally there is overall reduction in the backgroundnoise after temporal enhancement. This is further illustratedusing Fig. 5, where, Fig. 5(a) shows foreground speech signalrecorded while music is being played at the background andFig. 5(e) shows the corresponding narrowband spectrogram.There is significant amount of background noise presentthroughout the temporal duration of speech recording. The




0 0.5 1 1.5−0.4

−0.2

0

0.2

0 0.5 1 1.5012345678

0 0.5 1 1.5−1

0

1

0 0.5 1 1.5012345678

0 0.5 1 1.5−1

0

1

Am

plit

ud

e

Fre

qu

en

cy (

kH

z)

0 0.5 1 1.5012345678

0 0.5 1 1.5−1

0

1

Time (s) Time (s)0 0.5 1 1.5

012345678

(b)

(a)

(c)

(d)

(e)

(f)

(g)

(h)

Fig. 5: Illustration of enhancement outputs obtained at different stages and theirnarrowband spectrogram plots. (a) speech signal recorded in foreground scenario, (b)excitation based enhanced output, (c) formant based enhanced output, (d) perceptualbased enhanced output, ((e) - (h)) are corresponding narrowband spectrograms.

narrowband spectrogram shown in Fig. 5(f) obtained fromtemporally enhanced foreground speech signal and this illus-trates that there is significant reduction of background noise.Although, there is reduction of background noise in foregroundregions, still there is audible background noise present in theforeground regions and it is evident from the spectrogramplots. Hence, excitation source based enhancement alone maynot be sufficient to suppress background noise present inforeground speech regions.

B. Formant based Foreground Speech Enhancement

The Sections II and III-A helped to temporally segmentforeground speech regions from rest of the background noiseand further enhance the foreground speech using excitationsource information. However, the foreground speech enhance-ment st(n) using excitation source information is still leftwith some residual background noise. Considering the factthat, instants of significant excitation are high SNR regionsin temporal domain, similarly, the formant peak locationsare high SNR regions in spectral domain. The vocal tractinformation is intact in most of the foreground recordingscenarios and hence such information can be exploited toenhance foreground speech regions further. The formant peakenhancement relative to spectral valleys have been exploitedin many methods for speech enhancement [24]. In proposedmethod, the1st stage of excitation based enhanced outputst(n) is further subjected through formant based enhancement(FBE). The formant enhancement is carried on LP spectrumwhich helps to enhance formant locations relative to adjacentvalleys. The formant enhancement is shown in the form ofblock diagram in Fig. 6.At(z) is the 1st order LP inversefilter expressed as

At(z) = 1 + bz−1 (22)

andHvt(z) is LP filter predicted frompth order LP analysis,where,p is chosen asFs/1000 + 4 [25]

Hvt(z) =1

1 +∑p

k=1 ckz−k

(23)

In order to estimate spectral tilt fromst(n), a 1st order LPanalysis is carried using Eqn. (22) as shown in the block

Order

FilteringInverse

LP Analysis

LP AnalysisOrder

LP Filter

1stst(n)

pth

Hvt(z)

sf(n)

At(z)

Fig. 6: The formant enhancement block diagram, where,st(n) is excitation basedenhanced foreground speech signal,At(z) is a1st order LP filter,Hvt(z) is thepth

order LP filter, andsf (n) is formant enhanced foreground speech signal.

0 1000 2000 3000 4000 5000 6000 7000 8000

−20

0

20

0 1000 2000 3000 4000 5000 6000 7000 8000

−20

0

20

Ma

gn

itu

de

in

De

cib

els

0 1000 2000 3000 4000 5000 6000 7000 8000

−20

0

20

40

Frequency(Hz)

(a)

(c)

(b)

Fig. 7: Illustration of formant enhancement. (a) LP magnitude spectrum obtained froma 20 ms voiced segment before formant enhancement, (b) LP magnitude spectrum of theintermediate stageHvt(z), and (c) LP magnitude spectrum after formant enhancement.

diagram in Fig. 6. The residual signal obtained from1st orderLP analysis is the excitation based enhanced speech signalst(n) minus the spectral tilt. Hence, the LP filter estimationusing Eqn. (23) would model the vocal tract informationwithout spectral tilt. Therefore, the foreground speech signalst(n) passed through Eqn. (23) will further enhance theformant peaks relative to adjacent valleys while maintainingthe spectral tilt. The output speech signalsf (n) is formantenhanced foreground speech signal. The formant enhancementis illustrated using LP magnitude spectrum plots obtained fromvoiced frame before and after formant enhancement as shownin Fig. 7. The Fig. 7(a) shows the LP magnitude spectrum ofcoefficients derived from speech signalst(n). The Fig. 7(b)shows LP magnitude spectrum derived fromHvt(z). It canbe noticed that, the LP magnitude spectrum is similar to Fig.7(b) except that, the magnitude response is relatively flat. Itcan be noticed that, the LP spectrum derived from speechsignalsf (n) as shown in Fig. 7(c) has sharper formant peaksrelative to Fig. 7(a). Also, the peak to adjacent valley ratio isincreased.

This is further illustrated using Figs. 5(c) and (g), whichshows the foreground speech signalsf (n) after formant en-hancement and its corresponding narrow band spectrogram,respectively. It can be noticed that there is reduction ofbackground noise and formant tracks are enhanced in theforeground regions. The effect of passing foreground speechsignal st(n) through LP filter makes formant peaks much




sharper than original signal by moving poles of the filtercloser to unit circle. Hence, Sections III-A and III-B uses theproduction aspects to enhance foreground speech signal. Theenhancement of foreground speech signal is achieved withoutexclusive modeling of noise and such enhancement does notintroduce unwanted musical noise to enhanced speech signal.Since, the poles of all pole LP filter moves close to unit circlein case of such formant enhancement, this makes speech soundunnatural [26]. Consequently, further processing is necessaryto perceptually make speech more natural and enhance theforeground speech of any left over residual noise.

C. Perceptual based Foreground Speech Enhancement

The formant enhanced foreground speech signal obtainedin Section III-B is further subjected to enhancement us-ing cepstral analysis and synthesis on mel frequency scale.However, the cepstral analysis involves deriving mel cepstralcoefficients (MCCs) using which, the enhanced speech sig-nal is synthesized through mel log spectral approximation(MLSA) filter [27]. The advantage of using MCCs is twofold, where, the coefficients can be directly used in speechand speaker recognition applications apart from foregroundenhancement. The MCCsCα(m) are the Fourier cosine coef-ficients of spectral envelope derived from mel log spectrum.The spectrum represented by MCCs closely resemble humanauditory spectral resolution having higher resolution at lowerfrequencies and lower resolution at higher frequencies [28].The MLSA filter is applied to get approximate vocal tractresponse from MCCs using the adaptive algorithm. The truespectrum of MLSA filter formth order MCCsc(m) is givenby

Hα(z) =z−1 − α

1− αz−1(24)

is an all-pass function, which represents the mel-warpedfrequency characteristics andα is a coefficient correspondingto the mel-scale (for exampleα = 0.35 for 10 kHz samplingrate)

βα(Ω) = tan−1 1− α2sin(Ω)

(1 + α2)cos(Ω)− 2α(25)

where, α depends on sampling frequency andβα(Ω) isthe phase of all-pass function, the smooth spectral envelopeGα(Ω) of mel log spectrum is expressed as polynomialfunction of orderM given by

Gα(Ω) =

M∑

m=0

Cα(m)cos(mΩ) (26)

where,Ω is mel frequency scale given byβα(Ω) andCα(m)are the cepstral coefficients of orderM .

In order to compute 34 dimensional MCCs, a Hammingwindowed frame of size 20 ms with a frame shift of 10 msis considered. The smoothed spectral envelope is computedfrom the MCCs using MLSA filter. The MLSA filter providesthe best mean square approximation of log spectrum enve-lope on linear frequency scale and further used to directlysynthesize the best quality speech signal. The MLSA filter

0 1000 2000 3000 4000 5000 6000 7000 8000−80−60−40−20

02040

0 1000 2000 3000 4000 5000 6000 7000 8000−80−60−40−20

02040

0 1000 2000 3000 4000 5000 6000 7000 8000−80−60−40−20

02040

Ma

gn

itu

de

in

De

cib

els

0 1000 2000 3000 4000 5000 6000 7000 8000−80−60−40−20

02040

0 1000 2000 3000 4000 5000 6000 7000 8000−80−60−40−20

02040

Frequency(Hz)

(a)

(b)

(c)

(d)

(e)

Fig. 8: Illustration of different outputs obtained from 20 ms voiced frame usinglog magnitude spectrum of (a) original speech recording in foreground scenario, (b)excitation based enhanced output, (c) formant based enhanced foreground speech, (d)smoothed envelope obtained from MCCs using MLSA filter, and (e) perceptuallyenhanced foreground speech signal using MLSA filter.

needs excitation signal along with MCCs in order to synthe-size the speech signal. The excitation signal is synthesizedusing F0 information along with voiced/unvoiced decision.Alternatively, in the current work, WLPRrw(n) is used asexcitation signal along with MCCs to synthesize perceptuallyenhanced foreground speech signalsp(n). This is illustratedin Fig. 8, where, Fig. 8(a) shows log magnitude spectrumof 20 ms voiced frame taken from the original recording.The Fig. 8(b) shows log magnitude spectrum from excitationbased enhancement (EBE) method, while Fig. 8(c) illustratesthe log magnitude spectrum of FBE method. The Fig. 8(d)shows smoothed log magnitude spectrum derived from MLSAfilter. It can be observed that, the smoothed log magnitudespectrum forms the envelope of spectrum shown in Fig. 8(c).Consequently, the log magnitude spectrum of speech signal issynthesized through MLSA filter by using WLPR signalrw(n)and MCCs and it is shown in Fig. 8(e). It can be noticed that,the sharpness of formant peaks are relatively reduced. Thishelps to get rid of unnaturalness that is introduced in SectionIII-B due to formant enhancement. Also, the current blockfurther attenuates remaining background noise. The Figs. 5(d)and (h) show perceptually enhanced speech signal and itsnarrowband spectrogram, respectively. It can be observed, thatthe sharpness of formant locations are reduced and furtherthere is attenuation of background noise.

IV. EXPERIMENTAL RESULTS AND DISCUSSIONS

The proposed scheme of foreground speech enhancementis evaluated with some of the well known speech qualitymeasures along with 3 state-of-the-art methods considered.In order to evaluate the proposed work both subjective andobjective measures are used. The mean opinion scores (MoS)and preference test scores are used for subjective analysiswhich are discussed in Sections IV-B1 and IV-B2, respectively.The clean speech signal is seldom available while recordingin noisy environments. Therefore, two objective measuresviz.,foreground to background ratio (FBR) similar toa posterioriSNR and epoch to non-epochal ratio (ENR) measurementsare introduced in Sections IV-B3 and IV-B4, respectively. Allmethods considered for evaluation are compared using FBR




and ENR measurements. The data set consisting of naturalrecordings in noisy environments are used to evaluate differentmethods using MoS, preference test, FBR, and ENR mea-sures. Subjective analysis of enhancement methods are timeconsuming task and often it is difficult to get right subjectsfor subjective evaluations. Therefore, perceptual evaluation ofspeech quality (PESQ) is one of the important objective mea-sures that closely resemble subjective analysis [29]. However,PESQ measurement requires reference clean speech signalfor evaluations. Hence, TIMIT speech database is used tobenchmark the proposed work [30].

A. Performance Evaluation of Foreground Speech Segmenta-tion

The foreground speech segmentation can be compared tovoice activity detection (VAD) as they have similar objective.Hence, the performance is compared with considered twostate-of-the-art VADs. One of the VAD considered for per-formance evaluation is the latest G.729 ITU-T VAD standarddeveloped for fixed telephony and multimedia communica-tions [31]. The G.729 VAD uses multiple boundaries forvoice activity decision. More recently a variable frame rateapproach is proposed on the basis that, the speech signalis not stationary in short period of fixed frame rate [32].Therefore, a variable frame rate (VFR) approach is followedbased ona posterioriSNR weighted energy distance. In orderto compare the performance of all three different methods,10 different TIMIT speech files consisting of 5 female and5 male speakers are considered [30]. The speech files aremodified by appending 50 ms silence on either sides of speechsignal to closely simulate the natural recording scenario. Also,it can be noted that the ground truth of VAD is manuallymarked in TIMIT files and they form the reference boundariesfor evaluation. The noisy speech recordings are simulatedby additively combining clean speech signal with 4 differ-ent types of noise at different levels. The 4 different typesof noise considered are mosaic machine noise, hostel messrecording (babble noise), background music with vocals andbackground speech. The background noise files are recordedin 4 different natural environments as explained in Section IIusing headphone microphone connected to laptop. However,the background noise sources are far away from microphone.In order to compare the performance of all three methods, thefollowing parameters are used

• Correct VAD decision (CD): Correct decisions made bythe VAD.

• Front End Clipping (FEC): Clipping introduced whilepassing from noise to speech activity.

• Missed Speech Clipping (MSC): Clipping due to speechmisclassified as noise.

• Noise Detected as Speech (NDS): Noise interpreted asspeech within silence period.

• OVER: Noise interpreted as speech due to VAD flagremaining active in passing from speech activity to noise.

All parameters are expressed in terms of percentage. The Fig.9 illustrates the performance of three different methods in

the form of graph plots. The Figs. 9(a), (e), (i), (m) and(q) show the performance of different methods in terms ofCD, FEC, MSC, NDS, and OVER for additive backgroundmosaic machine noise, respectively. It can be noted that, thenoise is added at 20, 15, 10, 5, 0, and -5 decibels. It isobserved that, the performance of foreground segmentationand VFR VAD are equally robust for background machinenoise even at low SNR levels. However, the performance ofG.729 VAD degrades as the noise level increases. The reasonfor degradation in case of G.729 is due to increase in FEC,MSC and OVER. Similarly, Figs. 9(b), (f), (j), (n) and (r) showthe performance of three methods for additive backgroundnoise recorded in a large hostel mess environment when itis crowded (babble noise). It can be observed from the plotsthat the performance of foreground segmentation and VFRVAD remains robust upto additive noise level of 5 dB andfollowed by degradation in performances. However, it can benoticed that the performance of VFR VAD is better comparedto foreground segmentation and G.729 VAD at lower SNRlevels. The deterioration in the performance of foregroundsegmentation is mainly due to the increased OVER rate at lowSNR levels, particularly at 0 and -5 dB levels. The Figs. 9(c),(g), (k), (o) and (s) show the performance curves for additivebackground music with vocals at different levels. It can beobserved from the plots that the foreground segmentationperformance is better than other two methods at low SNRlevels. Similarly, Figs. 9(d), (h), (l), (p) and (t) show theperformance of 3 different methods for additive backgroundspeech at different levels. It can be noticed that there isdegradation in performance of all 3 methods at low SNRlevels. However, the performance of foreground segmentationis marginally better compared to VFR VAD at low SNRlevels. The signal characteristics of background music withvocals and background speech are similar to foreground speechsignal. Hence, analyzing the signal at zero frequency ratherthan entire set of frequencies helps in better discrimination offoreground speech from rest of the background noise as in caseof foreground speech segmentation. As explained in SectionII, the features derived from ZBFS such as ZBFS energy andNACC are used as features for foreground segmentation. Also,modulation spectrum energy centered at 4 Hz is used in com-bination with features derived from ZBFS. The performanceevaluation shows that, the foreground segmentation is robustto interfering background noise.

B. Performance Evaluation of Foreground Speech Enhance-ment using Natural Recordings

In order to evaluate proposed method, the speech filesrecorded in natural environment from ten different speakersare considered, and this include 3 female and 7 male speakers.Typically each speaker has spoken two to three differentsentences in each such recordings. The spoken sentences arechosen from English radio broadcast and it has the com-position of 74.10% of voiced sounds and25.90% unvoicedsounds. All speakers were native Indians and were wellversed with English as their second language. The speechis recorded in 7 different natural environments that includes




Clean 20 15 10 5 0 −50

50

100

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

50

100

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

50

100

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

50

100

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

10

20

30

40

Clean 20 15 10 5 0 −50

10

20

30

40

Noise in Decibels

Pe

rce

nta

ge

VFRFGSG729CD FEC MSC NDS OVER

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

(k)

(l)

(m)

(n)

(o)

(q)

(r)

(s)

(t)(p)

Fig. 9: Illustration of performance evaluation of VAD using foreground segmentation, VFR VAD, and G.729 VAD for 4 different noise types, ((a) - (d)) is CD, ((e) - (h)) isFEC, ((i)-(l)) is MSC, ((m) - (p)) is NDS, ((q) - (t)) is OVER plots with different additive noises of machine noise, babble noise, background music with vocals and backgroundspeech, respectively.

busy city road traffic, office room when mosaic machine isoperating at the background, air condition machine room,home environment when television is on, room environmentwhen background music is playing along with vocals, crowdedhostel mess (babble noise), and building construction site whenconcrete mixing machine is on. All speech files are recorded inforeground scenario, where, the foreground speaker is closerto microphone sensor relative to other interfering sources. Therecording setup includes headphone connected to laptop. Thespeech files are recorded using WAVESURFER [33] tool atthe sampling rate of 16 kHz. Three different headphone setsof different make and prices were used for the recordings inorder to maintain the variability of sensors. It can be notedthat, the headphones used for recording had no special frontend preprocessing circuits to enhance the speech signal. Inall such recordings, the speaker is wearing headphone andtypically microphone is closer to mouth of the speaker (within1 to 2 inches). The background acoustic sources are far awayfrom microphone sensor compared to foreground speaker.

The proposed method is compared with three other methodschosen from literature. The spectral domain subtraction basedmethod is one of the earliest methods for enhancing speechsignal distorted by additive noise. In spectral subtraction basedmethods the noise components are modeled by average magni-tude spectrum using several frames of noise only regions. Thenoise spectrum is subtracted from signal spectrum to obtainthe enhanced output [3]. The method proposed in [4] assumesthat the Fourier coefficients of speech and noise can be inde-pendently modeled as zero mean Gaussian random variables.The method aims to minimize the mean square error betweenthe clean speech and enhanced speech signal and hence it iscalled as minimum mean square error (MMSE). There areseveral modifications suggested to these two basic methods inliterature to improve the quality of enhancement by reducing

the musical noise. The matlab implementations available inVOICEBOX [34] are used to compare with the proposedmethod in this paper. Recently, a method is proposed thatmakes use of temporal and spectral processing to enhance thedegraded speech signal [12]. The temporal processing relies onHELP signal to extract the epoch locations and enhance thoselocations using LP residual signal, similar to excitation basedenhancement module in this work. The signal is enhancedin spectral domain by a comb like function that emphasizesthe fundamental component and its harmonics. Both temporaland temporal-spectral enhancement are compared with theproposed method.

FGS BAK OVL0

0.5

1

1.5

2

2.5

3

3.5

Me

an

Op

inio

n S

co

re

OrgSSMMSETETSPEBEFBEPBE

Fig. 10: Bar graph representing the mean opinion scores obtained from differentsubjects, where, FGS is foreground speech, BAK is background noise and OVL is overallratings. The graph also depicts the error computed using95% confidence interval fromthe subjective scores.

1) Subjective Evaluation using MoS Score:The subjectiveevaluation is carried using three parameters as foregroundspeech (FGS), background noise (BKG) and the overall quality(OVL) of enhanced speech signal. The original and enhancedspeech files were provided to 24 different subjects in random




order for subjective evaluation scores similar to [35]. Thesubjects include 16 male and 8 female adult listeners havingan average age of26 ± 5 years. All listeners are well versedwith English language and English was their medium ofinstruction in their academic studies. The listeners are mainlyworking in the area of speech processing, speech synthesis,speaker verification and speech recognition topics and theytypically have 1 to 3 years of experience in this domain.All listeners had normal hearing abilities without any kind ofhearing impairment. The files were listened to in a typical labenvironment when air condition is switched on. The files werelistened without visualizing the waveform using similar typesof headphones connected to personal computer. The subjectswere provided with the following instructions for each of suchparameters.

• Focus listening on FGS regions alone in terms of reducedbackground noise, intelligibility and lesser distortion withscales suggesting [1 - Very Unnatural, 2 - Fairly Unnat-ural, 3 - Somewhat Natural, 4 - Fairly Natural, 5 - VeryNatural].

• Focus listening on BAK regions alone in terms of reducedbackground noise, and lesser distortion with scales sug-gesting [1 - Very Intrusive, 2 - Somewhat Intrusive, 3 -Noticeable but not Intrusive, 4 - Somewhat Noticeable, 5- Not Noticeable].

• Focus listening on the OVL in both foreground and back-ground regions in terms of reduced background noise,intelligibility and lesser distortion with scales suggesting[1 - Bad, 2 - Poor, 3 - Fair, 4 - Good, 5 - Excellent].

The subjects were provided with three sets of same filesarranged in random order for evaluation. The Fig. 10 showsthe results of subjective evaluation in terms of MoS using theoriginal and enhanced speech files generated using differentmethods. The Fig. 10 shows the bar graph plots along withmargin of error with 95% confidence interval. It can beobserved from Fig. 10 that all three scores i.e., FGS, BAKand OVL is lower in case of original signal as expected andthis forms the reference score to evaluate other methods. Thesubjective scores obtained for MMSE is better compared toSS in all three parameters. Also, MMSE based method isbetter than temporal enhancement in terms of FGS and OVL,while, TE remains better in terms of BAK compared to SSand MMSE. Though, conceptually TE and EBE are similarin approach, it can be noticed that EBE performs better thanTE in all three parameters. The reason of EBE performingbetter than TE is because of better identification rate (IDR),identification accuracy (IDA), and lower miss rate (MR) toextract epoch locations using ZBF compared to HELP [17].Also, ZBF remains robust to noisy conditions and henceleads to lower distortion while temporally processing the LPresidual signal. This helps to set reasonably lower thresholdvalue to attenuate the background noise in case of EBE.Consequently, further processing modules can benefit fromsuch higher attenuation of background noise in the proposedwork.

It can be noticed that TSP, FBE, and PBE methods performbetter than SS, MMSE, TE, and EBE methods. The TSP and

TABLE I: Subjective evaluation of different methods using preference test score interms of percentage, where, SS - spectral subtraction, MMSE - minimum mean squareerror approximation, TE - temporal enhancement, TSP - temporal and spectral processing,EBE - excitation based enhancement, FBE - formant based enhancement, and PBE -perceptual based enhancement.

Method SS MMSE TE TSP

EBE 51% 44% 53% 39%

FBE 58% 54% 61% 49%

PBE 61% 58% 63% 54%

FBE based methods outperform other methods in terms ofFGS, while, FBE and PBE based methods are better in termsof BAK and OVL. It can be noted that spectral processing incase of TSP is useful in attenuating the valleys of spectralmagnitude and formant peaks remain unmodified. In caseof FBE, the formant peaks are boosted further relative tospectral valleys. The formant locations are high SNR regionsin spectral domain and hence relative enhancement of formantpeak locations help to reduce the background noise andenhance the foreground speech regions. This is evident fromsubjective evaluation using FGS and OVL parameters. Anyresidual background noise left over can be reduced further byPBE method and this can be observed from Fig. 10 by theincreased BAK and OVL. Also, it can be observed that MoSof TSP, FBE and PBE methods in terms of FGS, BAK, andOVL are better than TE and EBE methods.

2) Subjective Evaluation using Preference Test:Preferencetest is one of the simpler tests to asses the speech quality.A subset of speech files that were used for MoS evaluationsin Section IV-B1 are used in preference test. The speech filesrecorded from 5 different speakers in 5 different environmentsare used. The speakers include 2 female and 3 male speakersand each has spoken a English sentence chosen from EnglishBroadcast. The speech files are assessed by 10 different sub-jects that include 5 female and 5 male listeners. All listenerswere adults and not having any hearing impairments andthe average age of listeners is28 ± 5 years. The listeningenvironment was in a typical computer laboratory with rela-tively less background noise. The listeners used high qualityheadphones of similar make to listen speech files. The filesare enhanced using SS, MMSE, TE, TSP, EBE, FBE and PBEmethods. A pairwise listening test was conducted, where, oneof the speech files were either EBE, FBE, or PBE, while, theother is SS, MMSE, TE, or TSP. The listeners were asked tolisten to reference file before listening to pair. Here, referencefile is the original recording in foreground scenario. The 5different noisy environments include mosaic machine noise,building construction noise, traffic noise, background musicwith vocals, and background speech. The subjects were givenwith following instructions to asses files

• Listen to reference file before listening to individual pairof files.

• Asses the speech files in terms of reduced backgroundnoise, lesser distortion, and speech intelligibility by giv-ing equal weightage to all 3 features.

Table I shows the preference test scores averaged for 5different types of noisy files assessed by 10 different subjects.




The percentage score refers to which subjects have preferredEBE, FBE, and PBE over other methods. It can be observedthat, the subjects have preferred EBE over SS and TE, while,MMSE and TSP scores better compared to EBE. The EBEdoes not cause distortion in the form of musical noise. Also,due to better epoch extraction method, the background noise issuppressed relatively more compared to TE method. However,the temporal enhancement alone is not sufficient to reduce thebackground noise and hence MMSE and TSP scores betterthan EBE. The FBE is preferred relatively higher comparedto SS, MMSE, and TE. The formant enhancement helps toincrease the spectral peaks which are essentially high SNR re-gions in spectral domain compared to spectral valleys. Hence,there is reduction of background noise in spectral domain andthis may be the reason for preference over SS, MMSE, and TE.The sharpening of formant peaks leads to audible distortionand this can be the reason for low preference score comparedto TSP. However, the spectral envelope is smoothened byPBE module to reduce the distortion and further eliminationof any residual noise left in previous stages. Hence, overallPBE performs better in terms of preference score. In case ofpreference test it is difficult to asses the specific reason forthe choice made by subject as the preference depends on all 3factors. However, comparing preference test score with MoStest reveals that, the overall trend of preference test score issimilar to OVL of MoS from Section IV-B1.

3) Foreground to Background Ratio (FBR):When speechsignal is recorded in natural environment, seldom speech andnoise signals are available separately toa priori estimationof speech and noise signal powers, respectively. Since, thespeech signal is recorded in natural environments and therecan be many types of interfering noises, and that may includebackground speakers. As explained in Section II foregroundspeech segmentation is one of the reliable ways to tempo-rally segment foreground speech from rest of the backgroundregions. Hence, gross weight functionwg(n) obtained fromEqn. (13) is used to segment the foreground and backgroundregions using the following relation

f(n) =

1, if wg(n) > µwg(n)

0, otherwise(27)

where, f(n) is the binary signal indicating foreground andbackground regions andµwg(n) is the mean value derivedfrom wg(n). Hence, the SNR computed using foregroundsegmentation is called foreground to background ratio (FBR).The FBR can be computed using the following relationships

σ2f =

1

Lf

l−1∑

n=0

s2(n).f(n) (28)

σ2b =

1

Lb

l−1∑

n=0

s2(n).(1 − f(n)) (29)

FBR = 10 log10σ2f

σ2b

(30)

where,σ2f is foreground speech power estimation,Lf is the

number of samples in foreground speech region,s(n) is the

naturally recorded speech signal,f(n) is foreground back-ground binary signal,σ2

b is the background power estimation,and Lb is the number of samples in background region. In

20 15 10 5 00

10

20

Machine Noise

20 15 10 5 00

10

20

Babble Noise

20 15 10 5 00

10

20

Decibels

FBR

Background Music with Vocals

20 15 10 5 0

10

15

20

Decibels

Background Speech

(a)

(b)

(c)

(d)

Fig. 11: Average FBR obtained from 10 different TIMIT speech files by addingnoise at different levels of 20, 15, 10, 5 and 0 dB, with different types of noise like (a)Mosaic Machine Noise, (b) Hostel Mess Noise (Babble Noise), (c) Background Musicwith Vocals, and (d) Background Speech.

order to study the characteristics of FBR, 10 different speechfiles from TIMIT database consisting 5 female and 5 malespeakers are considered. The clean speech files are added with4 different types of noise at different levels as shown in Fig.11. The x-axis represents the added noise levels to speechfiles at 20, 15, 10, 5 and 0 dB, whereas, y-axis representsthe FBR calculated using the relationship given in Eqn. (30)expressed in decibels. It can be observed that there is linearrelationship between the noise added to clean speech filesand the foreground to background ratio estimated in all 4noisy casesviz., machine noise, babble noise, backgroundmusic with vocals and background speech. However, it can benoticed that there is degradation in estimating FBR in case ofbackground speech case at 0 dB noise level. Overall the FBRestimate is reliable and robust to different types of additivenoise at different levels. Hence, FBR can be used to measurea posteriori SNR that can help indicate the suppression ofbackground noise by different methods.

Org SS MMSE TE TSP EBE FBE PBE0

5

10

15

20

25

Ave

rag

e F

ore

gro

un

d t

o B

ackg

rou

nd

Ra

tio

(F

BR

) in

dB

Fig. 12: Bar graph representing objective scores in terms of average FBR obtainedfrom original and processed outputs. The graph also depicts the error computed using95% confidence interval from the objective scores.

The same set of files used to evaluate subjective measuresas described in Secion IV are used to evaluate the performanceof different methods using FBR. Totally 27 different speech




files collected naturally in different noise environments areconsidered for FBR measurement. The FBR is computed forall the enhanced outputs using different methods. The objectivescores are illustrated using bar graph in Fig. 12 in termsof average FBR expressed in decibels (dB). Also, the plotindicates the margin of error with95% confidence interval. Itcan be noticed that the performance of TSP, EBE, FBE, andPBE methods are superior compared to SS, MMSE and TEmethods. The objective scores obtained further corroboratesthe subjective analysis as discussed above. However, theperformance of FBE, and PBE remains best in terms of FBR.The difference between the average scores of original signaland the enhanced outputs from FBE and PBE shows that thereis an improvement of 12 dB. This shows that, the proposedmethod performs best in attenuating the background noisecompared to other methods and still maintains the overallquality of enhanced foreground speech. The similar trend canbe observed from subjective analysis using BAK parameter asshown in Section IV-B1.

4) Epoch to Non-Epochal Ratio (ENR):The effect ofinterfering sources are not uniform throughout the foregroundregions. There are certain regions of foreground speech thatare relatively more robust to interfering sources, especiallythe regions around instants of significant excitation are highSNR regions compared to other regions within a glottal cycle.Hence, the ratio between energy around epochal region to non-epochal region within a glottal cycle of foreground speechcan be an important objective measure to evaluate differentmethods. In order to compute such a ratio, Hilbert Envelopeof LP residual (HELP) signal is considered similar to [36],where, Hilbert Envelope of LP residuale(n) is given by thefollowing relationship

h(n) =√

e2(n) + eh2(n) (31)

where,h(n) is Hilbert envelope andeh(n) is the Hilbert trans-form of e(n). This measurement, when compared between theoriginal recording and enhanced output will help to assess theperformance of different methods in terms of enhancing highSNR regions further relative to other regions. The quantityessentially measures the energy between epochal and non-epochal regions and hence termed asEpoch to Non-EpochalRatio (ENR).

The epochal energy is calculated by considering the sum-mation of normalized energy around 3 ms of epoch locations,where, 3 ms closely corresponds to glottal closure interval.The non-epochal energy is computed by the summation ofnormalized energy excluding 3 ms region around the epochlocations with reference to each glottal cycle. The ENR canbe computed using the following relationships

E =

Nk∑

k=1

12M+1

∑ik+Mp=ik−M h2

e(p)

1L1

∑ik−M−1q=ik−1+M+1 h

2e(q) +

1L2

∑ik+M+1s=ik+1−M−1 h

2e(s)(32)

and

O =

Nk∑

k=1

12M+1

∑ik+Mp=ik−M h2

o(p)

1L1

∑ik−M−1q=ik−1+M+1 h

2o(q) +

1L2

∑ik+M+1s=ik+1−M−1 h

2o(s)(33)

where, he(p) is HELP derived from enhanced foregroundspeech,E is the estimation of ENR using enhanced foregroundspeech,Nk is total number of epochs in foreground speechregions,M corresponds to samples of 1.5 ms,ik is the epochlocation atkth epoch,ik−1 is the previous epoch location toik, ik+1 is the epoch location afterik, L1 = ik−ik−1−2M−2,L2 = ik+1 − ik + 2M + 2, and O is the estimation of ENRusing HELP derived from original recordingho(p). The ratiobetween10 log 10 E

Orepresents the improvement achieved in

terms of enhancing high SNR regions further relative to lowSNR regions. A similar approach is followed to characterizeENR as in case of FBR explained in Section IV-B3. The 10speech files are from different speakers chosen from TIMITdatabase is additively corrupted using 4 types of noise atdifferent levels. The Fig. 13 shows the ENR evaluation forclean and additive noise cases. It can be observed that there isa linear relationship between additive noise and ENR for all4 different noise types. It can be noticed that ENR increaseslinearly as additive noise increases. The ENR is computed

20 15 10 5 02

4

6

8

Machine Noise

20 15 10 5 02

4

6

8Babble Noise

20 15 10 5 02

4

6

8

Decibels

Cle

an to

Noi

se R

atio

of E

NR

Background Music with Vocals

20 15 10 5 00

2

4

Decibels

Background Speech

(a)

(b)

(c)

(d)

Fig. 13: Average ENR obtained from 10 different TIMIT speech files by addingnoise at different levels of 20, 15, 10, 5, and 0 dB, with different types of noise like (a)Mosaic Machine Noise, (b) Hostel Mess Noise (Babble Noise), (c) Background Musicwith Vocals, and (d) Background Speech.

for all the enhanced outputs using different methods. Theobjective scores are as shown in Table II that is computedby averaging the scores from all files. It can be noted thatthe ratio is computed between enhanced output to originalrecording while computing ENR. It can be observed that EBEand PBE methods are best amongst all other methods in termsof ENR. The improvement achieved in case of EBE is due tobetter identification of epochs using ZBF, as a result of whichthe noise components are suppressed in LP residual to obtainWLPR. The speech signal synthesized using WLPR resultsin enhanced output in which high SNR regions are enhancedfurther compared to low SNR regions. However, in case ofPBE the spectral envelope is smoothened and this leads toreduction of background noise, mainly in low SNR regions.Hence, EBE and PBE methods have high ENR compared toother methods. In case of FBE, due to LP filter enhancement,the formant peaks are sharpened and thereby increasing the




TABLE II: Objective evaluation of different methods using Epoch to Non-EpochalRatio (ENR) that is computed as a ratio between ENR of enhanced foreground speechto original recordings. The table represents the average ratio expressed in decibels (dB)computed across all the enhanced speech files obtained from different methods.

SS MMSE TE TSP EBE FBE PBE

1.28 0.83 3.68 2.31 4.23 1.26 5.46

TABLE III: Perceptual Evaluation of Speech Quality Scores, where, MMN - MosaicMachine Noise, MN - Hostel Mess Noise, TN - Traffic Noise, BM - Background Musicwith Vocals, BS - Background Speech SS - spectral subtraction, MMSE - minimum meansquare error approximation, TE - temporal enhancement, TSP - temporal and spectralprocessing, EBE - excitation based enhancement, FBE - formant based enhancement,and PBE - perceptual based enhancement.

Noise Decibels Original SS MMSE TE TSP EBE FBE PBE

15 2.5 3.2 3.1 2.9 3.0 2.9 3.0 3.1

MMN 10 2.3 3.1 3.0 2.8 2.9 2.8 2.9 3.0

5 1.8 2.7 2.6 2.4 2.5 2.5 2.5 2.6

15 2.6 3.0 2.9 2.8 3.1 2.8 2.9 3.0

MN 10 2.2 2.7 2.6 2.7 3.0 2.7 2.8 2.9

5 1.9 2.3 2.2 2.5 2.6 2.4 2.5 2.6

15 2.4 2.6 2.5 2.8 2.9 2.7 2.8 2.9

TN 10 2.2 2.3 2.2 2.7 2.8 2.6 2.7 2.8

5 1.8 2.0 2.0 2.2 2.3 2.0 2.1 2.2

15 2.4 2.8 2.7 2.7 3.0 2.8 2.9 3.0

BM 10 2.2 2.4 2.3 2.5 2.7 2.6 2.7 2.8

5 1.8 2.1 2.0 2.2 2.4 2.2 2.3 2.4

15 2.8 2.8 2.7 2.8 3.0 2.8 3.0 3.1

BS 10 2.7 2.7 2.6 2.7 2.9 2.8 2.9 3.0

5 1.9 1.8 1.8 2.2 2.3 2.3 2.3 2.4

gain of the filter transfer function at formant locations. Theenhanced LP filter convolves with excitation signal resultingin relatively higher amplitude at non-epochal regions andhence the ENR is lower in case of FBE. The enhancementof foreground speech signal using FBE is achieved mainlybecause of enhancement of high SNR regions in spectraldomain.

C. Perceptual Evaluation of Speech Quality (PESQ)

The PESQ is carried using 10 TIMIT speech files takenfrom 5 female and 5 male speakers. The clean speech files arecorrupted by adding 5 different types of noise at 3 differentlevels. The Table III shows the average PESQ scores obtainedfrom 10 different speech files. It can be observed that in case ofmosaic machine noise (MMN) the spectral subtraction methodperforms well. Since, MMN is nearly stationary noise andhence SS is able to better model the background noise in suchcases. However, in case of hostel mess noise (babble noise) andtraffic noise which are relatively non stationary in nature, TSPand PBE methods performs better than other enhancements.The background music with vocals and background speechcases are the most challenging, as the background noisecharacteristics is similar to foreground speech regions. Dueto robustness of foreground speech segmentation and better

estimation of GCI locations using ZBF, the performance ofthe proposed work is relatively better compared to othermethods. Overall, the performance of TSP and PBE are betterand comparable. Though, it is difficult to correlate all theparameters of subjective MoS with PESQ score, the trendremains consistent with subjective analysis.

V. SUMMARY AND CONCLUSIONS

In this paper, a new way to approach the problem ofspeech enhancement is suggested, where, the distance be-tween the foreground speaker to microphone and rest of thebackground sources is utilized. The proposed work relies onknown production and perceptual features to enhance theforeground speech. The advantage of proposed method is thatthe distortion is significantly lower and does not introducemusical noise unlike other methods as spectral subtractionand MMSE. The method exploits reliable ZBF for foregroundsegmentation and extraction of glottal closure instants, dueto which, higher attenuation of background noise is possiblewith least distortion. The advantage of using ZBF is that,there is no necessity of findingF0 of foreground speaker.The performance of proposed work is compared with 3 otherexisting state-of-the-art methods in terms of subjective andobjective evaluations. It is found that, the proposed method cansignificantly attenuate the background noise and still maintainthe better quality of enhanced foreground speech signal. Futurework should focus on using proposed framework of productionand perception features for enhancement in other types ofdegradation like reverberation and multi-speaker environment.

ACKNOWLEDGMENT

This work is part of the projects on the development of”Prosodically Guided Phonetic Engine for Assamese Lan-guage” funded by the Technology Development for IndianLanguages (TDIL) Programme initiated by the Departmentof Electronics & Information Technology (DeitY), Govt. ofIndia and the development of Speech Based Multi-level PersonAuthentication System funded by the e-security division ofDepartment of Electronics & Information Technology (DeitY),Govt. of India.

REFERENCES

[1] P. C. Loizou, Speech Enhancement: Theory and Practice, Taylor andF. Group, Eds. CRC Press, 2013.

[2] K. Parlak and O. G. Moreno,Applied Speech Enhancement in MobileCommunication Acoustics: Background Noise Elimination with FilteringAlgorithms. LAP LAMBERT Academic Publishing, 2012.

[3] S. Boll, “Suppression of acoustic noise in speech using spectral sub-traction,” IEEE Trans. Acoustics, Speech and Signal Processing, vol.ASSP-27, no. 2, pp. 113–120, April 1979.

[4] Y. Ephraim and D. Malah, “Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator,”IEEE Trans.Acoust., Speech and Signal Process., vol. ASSP-32, no. 6, pp. 1109–1121, Dec. 1984.

[5] R. Martin, “Noise power spectral density estimation based on optimalsmoothing and minimum statistics,”IEEE Trans. Audio, Speech andLanuage Processing, vol. 9, pp. 504–512, 2001.

[6] T. Gerkmann and R. C. Hendriks, “Unbiased mmse-based noise powerestimation with low complexity and low tracking delay,”IEEE Trans.Audio, Speech and Lanuage Processing, vol. 20, pp. 1383–1393, 2011.




[7] N. Virag, “Single channel speech enhancement based on maskingproperties of the human auditory system,”IEEE Trans. Audio, Speechand Lanuage Processing, vol. 7, pp. 126–137, 1999.

[8] Y. Hu and P. C. Loizou, “A perceptually motivated approach for speechenhancement,”IEEE Trans. Audio, Speech and Lanuage Processing,vol. 11, pp. 457–465, 2003.

[9] F. S. Cooper, “Acoustics in human communication: Evolving ideas aboutthe nature of speech,”Journal of the Acoustical Society of America,vol. 68, pp. 18–21, 1980.

[10] J. C. Junqua, “The lombard reflex and its role on human listenersand automatic speech recognizers,”Journal of the Acoustical Societyof America, vol. 93, pp. 510–524, 1993.

[11] B. Yegnanarayana, C. Avendano, H. Hermansky, and P. S. Murthy,“Speech enhancement using linear prediction residual,”Speech Com-munication, vol. 28, pp. 25–42, May 1999.

[12] P. Krishnamoorthy and S. R. M. Prasanna, “Enhancement of noisyspeech by temporal and spectral processing,”Speech Communication,vol. 53, pp. 154–174, February 2011.

[13] W. V. Summers, D. B. Pisoni, R. H. Bernacki, R. I. Pedlow, and M. A.Stokes, “Effects of noise on speech production: Acoustic and perceptualanalyses,”Journal of the Acoustical Society of America, vol. 84, pp.917–928, 1988.

[14] H. Hermansky and N. Morgan, “RASTA processing of speech,”IEEETrans. Speech and Audio Processing, vol. 2, no. 4, pp. 578–589, October1994.

[15] K. T. Deepak, B. D. Sarma, and S. R. M. Prasanna, “Foreground speechsegmentation using zero frequency filtered signal,” inInterspeech,September 2012.

[16] K. S. R. Murthy and B. Yegnanarayana, “Epoch extraction from speechsignals,” IEEE Trans. Audio, Speech and Lanuage Processing, vol. 16,pp. 1602–1613, November 2008.

[17] K. T. Deepak and S. R. M. Prasanna, “Epoch extraction using zero bandfiltering from speech signal,”Circuits, Systems, and Signal Processing,p. [Online], December 2014.

[18] K. S. R. Murty, B. Yegnanarayana, and M. A. Joseph, “Characterizationof glottal activity from speech signals,”IEEE Signal Processing Lett.,vol. 16, no. 6, pp. 469–472, June 2009.

[19] C. L. Smith, C. P. Browman, R. S. McGowan, and B. Kaytt, “Extractingdynamic parameters from speech movement data,”Haskins LaboratoriesStatus Report on Speech Research, vol. SR-105/106, pp. 107–140, 1991.

[20] S. Greenberg and B. E. D. Kingsbury, “The modulation spectrogram:In pursuit of an invariant representation of speech,” inICASSP, vol. 3,April 1997, pp. 1647–1650.

[21] T. Irino and R. D. Patterson, “A compressive gammachirp auditory filterfor both physiological and psychophysical data,”J. Acoust. Soc. Am,vol. 109 (5), pp. 2008–2022, May 2001.

[22] M. Unoki, T. Irino, B. Glasberg, B. C. J. Moore, and R. D. Patterson,“Comparison of the roex and gammachirp filters as representations ofthe auditory filter,”J. Acout. Soc. Am., vol. 120(3), pp. 1474–1492, 2006.

[23] T. Drugman, M. Thomas, J. Gudnason, P. Naylor, and T. Dutoit,“Detection of glottal closure instants from speech signals: A quantitativereview,” IEEE Trans. Audio, Speech and Lanuage Processing, vol. 20,no. 3, pp. 994–1006, March 2012.

[24] J. Yang, F. Luo, and A. Nehorai, “Spectral contrast enhancement:Algorithms and comparisons,”Speech Commun., vol. 39, pp. 33–46,2002.

[25] L. R. Rabiner and R. W. Scafer,An Introduction to Digital SpeechProcessing (Foundations and Trends in Signal Processing). NowPublishers Inc, 2007.

[26] V. Ramamoorthy, N. S. Jayant, R. V. Cox, and M. M. Sondhi, “En-hancement of adpcm speech coding with backward-adaptive algorithmsfor postfiltering and noise feedback,”IEEE Journal On Selected Areasin Communication, vol. 6, pp. 364–382, 1988.

[27] K. Tokuda, T. Kobayashi, T. Masuko, and S. Imai, “Mel-generalizedcepstral analysis – a unified approach to speech spectral estimation,” inICSLP, September 1994, pp. 1043–1046.

[28] G. Fant,Speech sound and features. MIT Press, Cambridge, 1973.[29] A. W. Rix, J. G. Beerends, D. S. Kim, P. Kroon, and O. Ghitza,

“Objective assesment of speech and audio quality - technology andapplications,”IEEE Tran. Audio, Speech and Signal Processing, vol. 14,pp. 1890–1901, 2006.

[30] TIMIT, “Timit Acoustic-Phonetic Continuous Speech Corpus”, NISTOrder PB91-505065, National Institute of Standards and Technology,Gaithersburg, MD, USA, 1990, Speech Disc 1-1.1., 1990.

[31] A. Benyassine, E. Shlomot, and H. Y. Su, “ITU-T recommendationG.729 annex B: A silence compression scheme for use with G.729

optimized for V.70 digital simultaneous voice and data application,”IEEE Commun. Mag, vol. 35, pp. 64–73, 1997.

[32] Z.-H. Tan and B. Lindberg, “Low-complexity variable frame rate anal-ysis for speech recognition and voice activity detection,”IEEE Journalof Selected Topics in Signal Processing, vol. 4, pp. 798–807, 2010.

[33] K. Sjlander and J. Beskow, “Wavesurfer an open source speech tool,” inSixth International Conference on Spoken Language Processing, 2000.

[34] VOICEBOX: Speech Processing Toolbox for MATLAB.[35] Y. Hu and P. C. Loizou, “Subjective comparison of speech enhancement

algorithms,” in ICASSP, May 2006.[36] V. C. Raykar, B. Yegnanarayana, S. R. M. Prasanna, and R. Duraiswami,

“Speaker localization using excitation source information in speech,”IEEE Tran. Audio, Speech and Signal Processing, vol. 13, pp. 751–761,2005.

K. T. Deepak was born in India on 1976. Hereceived the B.E. degree in Instrumentation Technol-ogy from Malnad College of Engineering, Universityof Mysore, India, in the year 1999 and M.Techdegree in Bio Medical Instrumentation from SriJayachamarajendra College of Engineering, Visves-varaya Technological University, Mysore, India inthe year 2002. He worked as Senior Project Engineerin ST Microelectronics, Bangalore from the year2005 to 2010, mainly focusing on audio post pro-cessing algorithms for the Digital Television group.

He is currently working as Technical Leader at NXP Semiconductors IndiaPvt. Ltd. and pursuing Ph.D. degree in Electronics and Electrical Engineeringat the Indian Institute of Technology Guwahati, India. His research interestsinclude speech/audio processing, analysis and recognition.

S. R. M. Prasanna was born in India on 1971.He received the B.E. degree in Electronics Engi-neering from Sri Siddartha Institute of Technology(then with Bangalore University), India, in 1994. Hereceived the M.Tech. degree in Industrial Electronicsfrom the National Institute of Technology, Surathkal(then KREC Surathkal), India, in 1997, and thePh.D. degree in Computer Science and Engineeringfrom the Indian Institute of Technology Madras,Chennai, India, in 2004. He is currently the Profes-sor in the Department of Electronics and Electrical

Engineering, Indian Institute of Technology, Guwahati. His research interestsare in speech and signal processing and published research articles in bothnational and international journals and conferences.

foreground speech segmentation and enhancement using ... · 12 db as opposed to existing spectral...

Documents