prosody as design, outcome and confounding variable in...

Prosody as Design, Outcome and

Confounding variable in

Speech & Language Pathology

Toni Rietveld

Dept. of Linguistics, Radboud University Nijmegen (NL) and

Sint Maartenskliniek, Nijmegen

[email protected]

Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010 2

OUTLINE of the talk

1. Definition of prosody, the concepts of Design, Confounding

and Outcome variable and short description of three

pathologies which will be discussed in relation with prosody:

aphasia, dysarthria and hearing loss (CI).

2. Common misunderstandings on prosody.

3. Examples 1: Prosody as design variable: Melodic Intonation

Therapy for aphasia and Intonation as design variable for

perception research with CI-users

4. Examples 2: Prosody as confounding and outcome variables

in dysarthria.

5. Conclusions.

1. Some definitions:

Prosody: encompasses all characteristics of speech

which are not relevant for the recognition of

individual speech segments:

Pitch (intonation), Intensity, Temporal structure.

Speech and Language Pathology (3 pathologies relevant in this talk):

Dysarthria: a neurogenetic disturbance in speech, which results in

articulatory/phonatory deficiencies, like problems with

the realization of stop consonants, reduction of the

vowel space, monotony, nasality. Dysarthria can be the

result of a CVA, a trauma or M. Parkinson.

Aphasia: a neurogenetic language disturbance, resulting in

a variety of language problems: agrammatism,

wordfinding, language comprehension etc.

3Rietveld: Prosody and Speech & Language Pathology; Lecture at the

University of Cambridge, October 19, 2010

Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of Cambridge, October 19, 2010

4

Hearing loss: severe hearing loss is often

compensated by a Cochlear Implant

(CI), a device that functions as a

„bypass‟ of the middle and inner ear.

2 = send coil

3 = receiver

4 = electrodes array

5 = auditory nerve

uitwendig oor = outer ear

middenoor = middle ear

binnenoor = inner ear

Variables: Design, Outcome and Confounding

1) Design variable:

a variable which is under the control of the experimenter;

for instance „singing‟ in the MIT (Melodic Intonation

Therapy) for Aphasia.

2) Outcome variable:

a variable which is the result of a pathology (for instance

M. Parkinson: monotony) or a therapy (for instance

restoration of „normal‟ intonation).

5Rietveld: Prosody and Speech & Language Pathology;

Lecture at the University of Cambridge, October 19, 2010

3) Confounding variable:

a variable which may obscure results of a therapy;

example: goal of therapy is pitch lowering and

increasing Intensity: a variant of the Silverman Therapy

for dysarthria).

If, for instance, before therapy words are spoken as if

in one sentence, and after therapy as a list of short

utterances, quite strong pitch decreasing effects can

be expected (see later).

6Rietveld: Prosody and Speech & Language Pathology; Lecture at

the University of Cambridge, October 19, 2010


7

Prosody can be measured in different ways:

1) On the basis of stochastic measures:

mean F0, SDF0, mean duration, SDdur, a Variability Index

(VI), mean intensity, SDintensity, ranges of measures,

maximum values of measures etc.

2) Qualitatively:

For intonation: in terms of symbols used in autosegmental

phonology: H, H*, L, L* etc.

For intensity and tempo: in terms of subjective scale

values assigned to utterances.

2. Common misunderstandings on prosody:

Confusion between stress and (sentence) accent

„permit vs. per‟mit (N vs. V)

- in sentences (I HAven‟t got a „permit; I DON‟t per‟mit..)

permit shows (word) stress, either on the 1st or 2nd

syllable; NO accent here; duration strongest cue.

- as single-word sentences they are accented (by

means of pitch movements).

Role of intensity in stress and sentence accent; perceptually

quasi irrelevant.

Funny thoughts on relation between high

pitch and sentence accent: nobody thinks of L*H:



F0-contour of the utterance: „Is Noorwegen een DUUR

vakantieland?‟ (Is Norway an exPENsive („duur‟) holiday

country?). Pitch accent: % L L* HH%



3. Examples of prosody as design variable:

3a) The Melodic Intonation Therapy (MIT)

for aphasia

Purpose of the MIT: to activate the right hemisphere in

aphasics. This is the “non-speech” hemisphere, which is

active in melody generation (and singing).

It is known that contralateral brain structures can be

activated in performing tasks for brain structures which are

damaged (in aphasia: very often the left hemisphere:

Broca and Wernicke regions).

Prosody as Design variable (Task): singing utterances.





Prosody is a clear design variable in experiments on

the MIT, as far as brain images are concerned. Are the

contralateral brain structures activated?

Prosody is NOT only a design variable if we look at

other outcome variables: intelligibility, precision of

articulation, word finding.

In the latter case intonation (pitch) may function as

confounding variable: singing elicits slower speech,

which may lead to:

• more time for word finding,

• more precise articulation, thus

• better intelligibility.


12

Findings:

• In MIT-task also higher activation in left

hemisphere. Why? Developmental studies suggest

that exaggerating speech prosody may facilitate

recruitment of language related brain areas

(mothers speaking with children, learning poetry).

• Indeed, intelligibility increases, but also on the

basis of rhythmic cues.

3 Examples of prosody as design variable (cont):

3b) Perception of intonation by CI users

13

Citroen (lemon)!

Citroen (lemon)?

Processing F0-information (pitch) is limited in CI-users.

Questions (Straatman, Rietveld et al., JASA (2010):

a) Is performance of CI-users in discriminating between

affirmative utterances and questions lower than that of

„normal‟ listeners? The same question for a same/different

task for speech stimuli.

b) Is bimodal listening (Hearing Aid + CI) of any help?

Ad a) Questions and Affirmations (in Dutch)


14

Stimuli used to elicit perception of Questions (Q) or Affirmations


15

She doesn‟t know

He does

Responses: pointing to illustrations (sorry for non-political

correctness)




Results: Performance of Normal Hearing (NH) >

Cochlear Implants + Hearing Aid (CI+HA) > Cochlear

Implant (CI) alone.

17

The utterance [ba: ba:] with different pitch heights (ranging from 100 to

358 Hz (male voice)

Ad b) Same/Different task; sequences of bisyllabic

utterances only differing in pitch range. Note: pitch

accent on one-word utterance.


18

“equal”

“different”

Responses:



19

Results: here too: Performance of Normal Hearing

(NH) > Cochlear Implants + Hearing Aid (CI+HA) >

Cochlear Implant (CI) alone.

Reference:

Straatman, L.V, Rietveld, A.C.M., Beijen,J., Mylanus, E.A.M. & Mens, L.H.M. (2010).

Advantages of bimodal fitting in prosody perception for children using a cochlear implant

and a hearing aid. Journal of the Acoustical Society of America,128(4), 1884-1895.

20

Clearest effects, of course, in tone languages:

mother hemp horse to curse

„hoog‟ = „high‟, „stijgend‟ = „rising‟, „duikend‟ = „high-low-high‟, „dalend‟ = „falling‟


21

4) Intonation as confounding (and outcome)

variable

Example in which intonation plays a role both as an

outcome variable and a confounding variable.

The PLVT: Pitch Limiting Voice Therapy for speakers

with M. Parkinson (derived from Lee Silverman V.T.)

Goal: increase loudness (in order to increase articulatory

precision), but limit pitch.

Outcome variable: extent to which pitch stays within

predetermined limits, measured numerically

Confounding variable: prosodic make-up determines

pitch values; qualitative measurement (H*, L, etc.)


Time (s)0 2.278

Pitch

(Hz)

0

300

Time (s)0 2.278

Pitch

(Hz)

0

300

Time (s)0 2.278

Pitch

(Hz)

0

300

je kunt van liMOEnen ook limoNAde maken

“you can of LEmons also LEmonade make”

%L L H* L H*L L%

%L L H* H*L L%

%L L H*L H*L L%



Linked contour

Median pitch: 122 Hz

Mean pitch: 127 Hz

Standard deviation: 16 Hz

Minimum pitch: 100 Hz

Maximum pitch: 179 Hz

Voice Reports given by PRAAT:

Flat hat


Mean pitch: 146 Hz



Maximum pitch: 179 Hz

Two pointed hats


Mean pitch: 127 Hz



Maximum pitch: 179Hz




24

Experiment:

Speakers with M. Parkinson were submitted to the PLVT

therapy (Pitch Limiting Voice Therapy).

Hypothesis: After therapy (T2), the intensity will have

increased, but the average pitch (F0) decreased.

maandag dinsdag woensdag donderdag vrijdag zaterdag zondag

(days of the week in Dutch)

Time (s)

0 7.687

Pitch

(Hz)

75

200

Time (s)

0 7.687

Pitch

(Hz)

75

200

Time (s)

0 7.687-0.7932

0.5783

0

Time (s)

0 7.687-0.7932

0.5783

0

Time (s)

0 16.03

Pitch

(Hz)

75

300

Time (s)

0 16.03

Pitch

(Hz)

75

300

Time (s)

0 16.03-1

0.9677

0

T1

T2

200 Hz

300 Hz



Med = 161 Hz

Mean = 154 Hz

SD = 21 Hz

Med = 138 Hz

Mean = 138 Hz

SD = 34 Hz


26

However: on the basis of qualitative measurements:

T1 %L H* H*L L% („flat hat‟ )

Prediction: Higher mean F0, lower SD.

T2: %LH*LL% %LH*LL% %LH*LL% %LH*LL% etc.

Prediction: Lower mean F0, higher SD.

Predictions OK, see preceding slide, but not necessarily

due to therapy!


27

5) Prosody as confounding variable; degree

of prominence as confounder: M.Parkinson

Relative Intensity of noise in SI of /t/ as outcome

variable: Δ Int (= Int. Vowel – Int. SI, in dB)

Time (s)

0 0.835-0.3299

0.3758

0

0.096316905 0.755674754

ata1

[ a t a ]

Time (s)

0 0.9145-0.3376

0.3541

0

0.457244898

atha1

59 dB 77 dB Δ = 18 dB

120 Hz 159 Hz Δ = 39 Hz

Difference between intensity of noise in silent interval and the

intensity of the following vowel is often seen as an index of

correct realization of /t/. However, promincene may have a

differential effect on noise and the vowel amplitude:

a) Weak pitch accent



Time (s)

0.004918 0.8153-0.6099

0.6706

0

atha2

60 dB 82 dB Δ = 22 dB

120 Hz 178 Hz Δ = 58 Hz

b) Strong pitch accent


Rietveld: Prosody and Speech & Language Pathology; Lecture at the University of

Cambridge, October 19, 201030

The preceding pictures show the following:

In the „weak accent‟ condition, the difference

between the maximum amplitude of the noise in the

SI and that of the amplitude was 18 dB.

In the „strong accent‟ condition, the difference

between the maximum amplitude of the noise in the

SI and that of the amplitude was 22 dB., an increase

of 4 dB, which might be seen as an improvement.

More plausible, however, is that the increase in

amplitude after a therapy resulted in a larger

increase in the amplitude of the vowel than of the

noise in the SI.


31

Conclusions

• Prosody, both in the temporal and the pitch domain,

can play a role as design, confounding or outcome

variable.

• A thorough knowledge of the effects of tune choice on

voice parameters (mean F0, SD of F0 etc) is a

prerequisite for valid research (ex.: dysarthria)

• The same holds for prosodic effects on segmental

characteristics and processing (ex.: aphasia).

prosody as design, outcome and confounding variable in...

Documents