Cognitive control and intoxicated speech

varianceThomas Purnell

University of Wisconsin-Madison

tcpurnell at wisc.edu

Challenge

• How do we model intoxication to know the effect of ethyl alcohol on behavior?– Toxin-induced syndrome– Phonetic control and naturalness in

language variation and change

• Using intoxication as a tool– Allows for control of behavior– Intoxicate vernacular speech disrupts or

severs the speech chain

Programmatic Goals

• Examine global and low level measures– Known prosodic or temporal information– Known spectral frequency information

• Unknown effects– On dialect – On forensics

• Research Question– Can acoustic features reliably correlate to

low-level (2 drinks) alcohol intoxication?

Areas of Inquiry

1. Variation from the mean: Prosody and pacing

2. Global effects: Long-term average spectra

3. Lower level effects: Vernacular vowel changes

Intoxicated speech generally

• Intoxicated speech affects motor control of speaking leading to coarticulatory, precision and timing differences from sober speech (Klingholz et al. 1988, Pisoni & Martin 1989, Behne & Rivera 1990, Hollien et al. 2001)

• In past, primary focus on prosody and

consonants (e.g., Hollien et al. 2001)

Gross effects

• Dysfluencies are common at the sentence, word, and segmental levels

• Duration is increased, speaking rate decreases

• Sound substitutions/segmental changes– Lengthening of individual segments– Replacing [s] with [ʃ] is particularly common– Devoicing of final obstruents– Deaffrication– Spirantizing stops

Recordings

• Subjects recorded as participated in a larger study (Moberg & Curtin 2009)

• As such, four recordings 1. Prior to intoxication (two drinks achieving a

BAC of 0.08% in 30 minutes; Curtin & Fairbanks 2003)

2. In the ascending arm of intoxication (15 minutes after the first drink)

3. At peak (30 minutes after first drink)4. In the descending arm (15 minutes, post-

peak intoxication).

BAC @ T1, T4

Effects at 0.08 BAC

• Euphoric condition– Increase in talkativeness– Shortened attention span– Impaired judgment, relaxation– Impaired fine muscle coordination, but

not quite ataxia (begins soon after and by 0.12 BAC)

PROSODY AND PACING

Acoustics of the actors’ speech simulation of intoxication

• Hollien, Liljegren, Martin & DeJong (2001) investigated the acoustics of the actors’ speech

• F0– Mean F0 higher under intoxication and simulated

intoxication for the males– Mean F0 higher under intoxication and lower under

simulated intoxication for the females

• Duration: statistically significantly longer for both real and feigned intoxication

• Intensity: showed no systematic change

Pitch and Intoxication

• Guiding idea is that intoxication is a family of variances from the mean that exceed the standard variance– E.g., observing someone driving a car

involves multiple cues

• Hanke & Purnell (2006) – Pitch variation widely variant across the

four conditions– Placebo effect: belief and behavior

T1 Pitch

Z R IY1 CH HHIH1 Z F REH1 N Z S EY1 HH IY1 IH1 Z L

his reach his friends say he is looking

50

150

60

80

100

120

140

Time (s)28.75 30.46

28.7527194 30.4601862Rainbow4_aligner

DHHIH1 Z R IY1 CH HHIH1 Z F REH1N Z S EY1 HHIY1IH1 Z L

beyond his reach his friends say he islooking

50

150

60

80

100

120

140

Time (s)33.3 35.02

33.2959658 35.0200507Rainbow1_aligner

T4 Pitch

Z R IY1 CH HHIH1 Z F REH1 N Z S EY1 HH IY1 IH1 Z L

his reach his friends say he is looking

50

150

60

80

100

120

140

Time (s)28.75 30.46

28.7527194 30.4601862Rainbow4_aligner

DHHIH1 Z R IY1 CH HHIH1 Z F REH1N Z S EY1 HHIY1IH1 Z L

beyond his reach his friends say he islooking

50

150

60

80

100

120

140

Time (s)33.3 35.02

33.2959658 35.0200507Rainbow1_aligner

Comparison of Prosody, S01 M

• T1 (137 seconds)• N = 66 phrases• xS Dur =1.5 sec• xS Phrase F0=117.6

Hz• xS Phrase Std=8.1

Hz• xS Min F0=102.7 Hz• xS Max F0=133.4 Hz

• T4 (123 seconds)• N = 48 phrases• xS Dur =2.2 sec• xS Phrase F0=119.0

Hz• xS Phrase Std=15.0

Hz• xS Min F0=95.6 Hz• xS Max F0=153.1 Hz

Rainbow, Para 1

• When the sunlight strikes raindrops in the air, they act as a prism and form a rainbow. The rainbow is a division of white light into many beautiful colors. These take the shape of a long round arch, with its path high above, and its two ends apparently beyond the horizon. There is, according to legend, a boiling pot of gold at one end. People look, but no one ever finds it. When a man looks for something beyond his reach, his friends say he is looking for the pot of gold at the end of the rainbow.

Where do the breaks go?

Rainbow, Para 1• When the sunlight strikes raindrops in the air, (8 words)• they act as a prism and form a rainbow. (9)• The rainbow is a division of white light into many beautiful

colors. (12)• These take the shape of a long round arch, (9)• with its path high above, (5)• and its two ends apparently beyond the horizon. (8)• There is, according to legend, (5)• a boiling pot of gold at one end. (8)• People look, but no one ever finds it. (8)• When a man looks for something beyond his reach, (9)• his friends say he is looking for the pot of gold (11)• at the end of the rainbow. (6)

Rainbow, Para 1, T1

• When the sunlight strikes raindrops in the air, [0.309] they act as a prism [0.170] and [0.140] form a rainbow. [0.898] The rainbow is a division of white light into many beautiful colors. [0.329] These take the shape [0.439] of a long round arch, [0.678] with [0.858] its path high above, [0.289] and its two ends apparently [0.180] beyond the horizon. [0.509] There is, according to legend, a boiling pot of gold at [0.200] one end. [0.758] People look, [0.050] but no one [0.100] ever finds it. [1.177] When a man looks for something beyond his reach, his friends say he is looking for the pot [0.040] of [0.220] gold [0.090] at the end of the [0.020] rainbow [1.207].

Rainbow, Para 1, Time 4

• When the sunlight strikes [0.359] raindrops in the air, they act as a prism and form a rainbow. [0.259] The rainbow is a division of white light into many beautiful colors. [0.748] These take the shape of a long round [0.110] arch, [0.050] with [0.190] its path high above, [0.529] and its two ends apparently beyond the horizon. [0.788] There is, according to legend, a boiling pot of [0.040] gold [0.040] at one end. [0.629] People look, [0.050] but no one ever finds it. [0.778] When a man looks for something beyond his reach, his friends say he is looking for the pot of gold at the end of the rainbow. [1.856]

(large) Phrase Boundaries

Word T1 T4

Rainbow 0.898 ----

Colors ----- 0.748

Arch/above 0.678 0.529

Horizon 0.509 0.788

End 0.758 0.629

It 1.177 0.777

Rainbow 1.207 1.856

Work to do

• Verbal stumbling– Need to apply a more thorough family of

measures – Better statistical modeling: differences

in distribution

LONG TERM AVERAGE SPECTRA

Long Term Average Spectra (LTAS)

• Measure overall energy envelope• Use longer passages• Gives a better picture of habitual vocal

tract behavior of an individual speaker• With enough speech, smooth out

situational variation

Spectral envelope

“The rainbow is a division of white light into many beautiful colors.”

LTAS & bandwidth

500 Hz500 Hz 100 Hz100 Hz

1,000 Hz1,000 Hz

Previous work

• Previous results have largely found just changes with no pattern besides simply variation (e.g., Schiel and Heinrich, 2009)

• Used for a wide variety of tasks to discriminate between different categories of speech (e.g., Boersma and Kovacic, 2006; Pauk, 2006)

• Our question was asked before, using LTAS to find acoustic cues to intoxication, but the full data are not available, and some experimental balances not in place (Klingholz et al. 1988)  

Importance for variationists

• Seems awkward because not fine-tuned measure

• May show gross effects of nasalization, pharyngealization, etc.

• All detail is not lost

• Unknown uses such as informing spectral tilt analyses

Hypotheses

• Null • No difference in spectral energy between

sober and intoxicated states

• Test• Difference in spectral energy with

intoxicated speech being more variable in energy

• Difference in spectral energy with intoxicated speech being lower in energy

Speakers & Task

• 10 native American English speakers from Wisconsin- Divided by gender- Average subject age: 26 (range, 21-36)

• 5 placebo subjects• 5 controls who came back after 2 weeks

Acoustic Analysis

• Rainbow passage only• Recordings normalized • LTAS, 3 bandwidths

• 100 Hz• 500 Hz• 1,000 Hz

• Begin with contrastive states• T1 sober• T3 peak intoxication

• Paired t-tests

1kHz Bandwidth

Significant 2.8 dB average reduction under intoxication in 0-2kHz and 4-5kHz regions

500Hz Bandwidth

Significant 2.0 dB average reduction under intoxication in 500Hz-1.5kHz and 4-5kHz

100Hz Bandwidth

Significant 2.2 dB average reduction under intoxication in specific areas

Fit of Parabolic Functions

 

Normal Variation

• Five WI subjects’ normal variation• Small decrease in amplitude (0.8dB, cf 2.2 dB intox)

Placebo Variation

• Five placebo subjects’ averaged variation• Small increase in amplitude (0.7dB)

Subject consistency

Probability of reduction in amplitude for significant frequency ranges

Placebo consistency

Discussion

• Rejects the null hypothesis (“no systematic differences”) and tests hypothesis 1 (“intox. more variable”)

• Supports test hypothesis 2 (“intox. lowers envelope”)• Found statistically-significant variation across

the sober and intoxicated conditions reduced by roughly 2 dB

• Two general regions• 300 to 1300 Hz• 4 to 5 kHz

• Why?

Low frequencies

• Physiological possibility• Alcohol inhibits tongue muscle (Krol, 1984)• Jaw lowering movement for low back

vowels (300 - 1300 Hz) is suppressed

800

400

F1: Vowel Height

F2: Vowel Backness

50015002500

a Ɔ

ʌo

High frequencies

• Potentially known “lush” characteristic

• Apical [s] laminal [ʃ]

• Fine motor control substituted by less accurate gross gesture

Alternative: Overall nasality

If velopharyngeal port is left open, then expect overall lowering of spectral envelope

rainbow rainbow

VOWELS

Intoxicated vowels

• Vowel formants in intoxicated speech may reduce (e.g., diminished F1/F2 ratio; Klingholz et al. 1988) and may be variable across speakers (Behne & Rivera 1990, Hollien et al. 2001)

• Other studies argue that a low blood alcohol content (BAC) affects vowels less than prosody and consonants (Pisoni & Martin 1989), perhaps because of greater aperture in vowels controlled by jaw movement (Perkell 1969, Stevens 1989)

Sociophonetics of vowels

• But, we are more sophisticated in our understanding of vowels– E.g., /aj/, /aw/, /æ/ etc. should be examined by

following consonant; /o/ and /u/ with light of initial consonant, ...

– Vowels can be style shifted (Labov 2001), that is, vowels are under some conscious motor planning

• Need to take into account a speaker’s dialect, esp relation to contemporary vowel shifts and mergers

Goal

• Describe and contextualize– Vowel changes within specific speakers– Between a speaker-internal control state

(not intoxicated) to speaker-internal test state (intoxicated)

• How does ethyl alcohol-induced motor control modify vowel qualities and interact with dialectal variation?

Null Hypothesis

• No significant difference across intoxication level for subjects– Vowels in the vowel space should be the

same for the two times (Pisoni & Martin 1989)

– Perhaps because of the gross motor movement of the jaw involved in vowel articulation rather than fine aperture movements of consonants (Perkell 1969)

Test Hypotheses1. Control

– Production of vowels becomes less accurate with intoxication, as would be the case if motor control simply breaks down under alcohol (Eckardt et al. 1998)

Reduction– Vowels become more centralized and reduced (e.g.,

smaller F1/F2 ratio for intoxication; Klingholz et al. 1988)

2. Standardization– If speakers’ shifting is an ‘active’ gesture for identity we

might expect that regional shifted vowels appear as unshifted towards a northern standard position

Vernacularization– Unshifted vowels become shifted if subjects who

produced more ‘rule-governed’ (in the Labov 2001 sense of female speech) or careful speech generally should be effected by alcohol (Moberg & Curtin 2009)

1. Control & Reduction

• Test of Control > more variability in all directions

• Test of Reduction > centralization

• Robust regression of overall data• Residuals from regression line

– Compare distribution to ‘own’ line– Compare residual distribution of intox to sober

2. Standardization & Vernacularization

• For /æ/, – Test Standardization > move down

continuum– Test Vernacularization > move up

continuum

• For /ɛ/, – Reverse direction

• For /ɔ/ and /a/, – Fronting and backing

Selection of tokens

• Rainbow passage (Fairbanks 1960) • Two times examined: sober & post-

peak speech • Word classes important for upper

Midwestern Am English (Labov et al. 2006) – Ash raising: BET, BAT, BAD– Low back merger: BOT, BOUGHT– Canadian Raising: BITE along with

reference vowel classes (BEET, BAIT, BIDE)

Subjects• Five female speakers• Average age 24.2 yrs at recording• From upper Midwest

Measures

• Describe vowel space– NORM plots (Thomas & Kendall 2007)– Telsur G values (Labov et al. 2006)

• Vowel head and tail (Nearey & Assmann 1986; Andruski & Nearey 1992)– F1, F2, F3 (Hz), Euclidean distance– B1 (Hz)– F1/F2 ratio (Klingholz et al. 1988)

• T-tests on each word at two times

1. Control & Reduction

• Per speaker, regression line for each set of data (red = sober; blue = intox)– Front vowel continuum, approx BOT to BEET

• Examine sober and intox residuals from regression lines

• Histogram of sober residual distribution (red) and intox-from-sober distribution (blue)– Kertosis (K-3) suggests Control– Positive skewness suggests Reduction, negative

indicates peripherality

Subj 2

K = 0.1s = -0.05

Subj 42

K = 1.5s = -0.77

Subj 31

K = 1.8s = -0.65

Subj 50

K = 5.1s = -1.8

Subj 44

K = -0.7s = -0.49

Slope

• Very similar between sober and intox• Three subjects close to normal

distribution– S2, intox inside sober (reduced??)– S42, intox outside sober (peripheral??)– S31, lines are almost identical

• Two subjects show some rotation– S50, high kurtosis– S44, negative kurtosis but rotation at

mid-vowel space

Summary

• Control– Compact (leptokurtic), 4 of 5 speakers– Diffuse (platykurtic), 1 of 5 speakers– S50 only sharply compact– S44, diffusion suggests control, but rotation of

continuum more suggestive of something else

• Reduction– All speakers have a negative skewness of intox

to sober regression, i.e., more data points peripherally

– S44 reduction of high vowels; peripheralization of low vowels > inconsistent

2. Standardization & Vernacularization

• Overall measures are inconsistent and what looks like reduction is vowel continuum rotation (S50, S44)

• Therefore, examine individual vowels – Stable: BEET, BAIT, BOT– Variable: BET, BAN, BAT, BITE, BIDE

Subj 2

Subj 42

Subj 31

Subj 50

Subj 44

Significant results, paired t-tests (p<0.05)

• F1– BAT, BAN, BITE

• F2– BET

• F1/F2 ratio– BAT, BAN

• Euclidean distance (Δ trajectory)– BAN, BITE

Summary of Reference Vowels

• BEET– Relatively close– Intoxicated variant can be inside (S2, S44), at

(S31, S42) or outside (S50) the control– Changes relative to overall slope of front

continuum• BAIT

– Most consistent vowel for three speakers– Some lowering/centering (S31, S42)

• BET– Significant sober-intox F2 difference in t-test– Reduced vowel

BET

Summary of Ash Raising• BAN

– Centering (2), lowering (2, 44, 31, 50), peripheral (42, 50??)– Significant differences for F1, F1/F2 ratio and Euclidean

distance

• BAD– Generally centering, but due to weak selection of BAD tokens– Wrt BAN, equalish (S31), lower (S2, S42, S44, S50) but the

lowering of BAN and raising of BAD moves them closer– Some raising (S42, S50) and some lowering (S2)

• BAT– Expect to be stable, but tends to rise when intoxicated, not

centered– Intoxicated variants are raised for all subjects except S44– Significant differences for F1 and F1/F2 ratio

BAT

Summary of Low Back Vowels

• BOT– F2 generally just above 1,400 Hz– S2, S42, S44 show intox lowering and fronting

while S31 and S50 show raising of intox

• BOUGHT– F2 generally below 1,200 Hz– General pattern is for intox to be higher and

more central, although lower and peripheral for S31 and S44 (rotation?)

– S31 might be fronting

BOUGHT

Summary of Canadian Raising vowels

• BIDE– Central to continuum line (S2)– Slight rise (S2, S31, S50), slight lowering

(S42)• BITE

– Central to continuum line (S2, S42, S44)– Same height (S2), lowering to line (S50),

lowering (S31, S42)– Significant differences for length of

trajectory, intox are shorter than sober > reduction?

Vowel Trajectory Length

BAN, BITE: significance on t-test

Consistent Variation?

• Cross-speaker variation (Behne & Rivera 1990, Hollien et al. 2001)– Surprisingly some consistency in holding

to continuum– BAT raising, not centering or

low/fronting– Need to solve issue of normalizing to

slope before explaining reduction

Standardization & Vernacularization

• Standardization has to go on, but perhaps situation is not right to test

• Vernacularization looks promising with BAT

BROAD CONCLUSIONS

Towards an understanding of cognitive control

• Prosody– Control > sychronization of events

• Global speech– Control > velum

• Vowel variation– Control > jaw

Towards an understanding of cognitive control

• Rotation problem with dispersion theory and reduction story– Need to look at entire vowel space to see if

back continuum also is more upright

• More likely slope rotation reflects emphasis on (privative) vowel height– More studies with articulation and intoxication

• Alcohol blocks/removes/… sensitivity to social/linguistic/formality rules by females– Analysis of males

Thank You.

Special thanks to John Curtin, Ryan Hanke, Eric Raimy, Blake Rodgers, Joe Salmons, and Nicholas Williams for assistance with recordings, data and comments. Partial funding by generous support of the Graduate School, University of Wisconsin—Madison. Errors are mine, all mine.

References• ANDRUSKI, JEAN E. and NEAREY, TERRANCE M. 1992. On the sufficiency of compound target

specification of isolated vowels and vowels in/bVb/syllables. Journal of the Acoustical Society of America, 91.390-410.

• BEHNE, D.M. and RIVERA, S.M. 1990. Effects of alcohol on speech: Acoustic analysis of spondees. Research on Speech Perception, 16.263-91. BEHNE, D.M. and RIVERA, S.M. 1990. Effects of alcohol on speech: Acoustic analysis of spondees. Research on Speech Perception, 16.263-91.

• ECKARDT, M.J., CAMPBELL, G.A., MARIETTA, CH.A., MAJCHRONWICZ, E. and WEIGHT, F. 1988. Acute ethanol administration selectively alters localized cerebral glucose metabolism. Brain Research, 444.53-58.

• FAIRBANKS, G. 1960. The rainbow passage. New York: Harper & Row.• HOLLIEN, H., DEJONG, G., MARTIN, C. A., SCHWARTZ, R. and LILJEGREN, K. 2001. Effects of

ethanol intoxication on speech suprasegmentals [Dec]. Journal of the Acoustical Society of America, 110.3198-206.

• KLINGHOLZ, F., PENNING, R. and LIEBHARDT, E. 1988. Recognition of Low-Level Alcohol-Intoxication from Speech Signal [Sep]. Journal of the Acoustical Society of America, 84.929-35.

• LABOV, WILLIAM. 2001. Principles of Linguistic Change: Social Factors.vol. 2. Oxford: Blackwell.

• LABOV, WILLIAM, ASH, SHARON and BOBERG, CHARLES. 2006. Atlas of North American English: Phonetics, phonology, and sound change. Berlin: Mouton de Gruyter.

• MOBERG, C. A. and CURTIN, J. J. 2009. Alcohol Selectively Reduces Anxiety but Not Fear: Startle Response During Unpredictable Versus Predictable Threat [May]. Journal of Abnormal Psychology, 118.335-47.

• NEAREY, TERRANCE M. and ASSMANN, PETER F. 1986. Modeling the role of inherent spectral change in vowel identification. Journal of the Acoustical Society of America, 80.1297-308.

• PERKELL, JOSEPH S. 1969. Physiology of speech production: Results and implications of a quantitative cineradiographic study. Cambridge, MA: MIT Press.

• PISONI, D. B. and MARTIN, C. S. 1989. Effects of Alcohol on the Acoustic-Phonetic Properties of Speech - Perceptual and Acoustic Analyses [Aug]. Alcoholism-Clinical and Experimental Research, 13.577-87.

• STEVENS, KENNETH N. 1989. On the quantal nature of speech. Journal of Phonetics, 17.3-45.• THOMAS, ERIK R. and KENDALL, TYLER. 2007. NORM: The vowel normalization and plotting

suite. [ Online Resource: http://ncslaap.lib.ncsu.edu/tools/norm/ ]