Schizophrenia and Depression – Evidence in Speech Prosody

Student: Yonatan VaizmanAdvisor: Prof. Daphna WeinshallJoint work withRoie Kliper and Dr. Shirley Portuguese

Agenda

• Presenting the approach– Speech prosody– Mental states, mental pathologies, Schizophrenia

• Our work– Data– Methods– Results

• Future directions

Information in speech prosody

• Syntactic disambiguation from intonation

• Prosodic content as signal for mental state

Large variabilitywave form spectrogram

Estimating mental states

Previous works

• Voice analysis to detect cancer of the larynx(Murry, T. and E. Doherty 1980)

• Differences in speech between Schizophrenians and normals(e.g. Stassen, H., et al 1995)

Our work - Data

Strict expression Free expression

Reading a list of words

Reading a passage

Free interview

Dr. (M.D.) Shirley Portuguese

Time (sec)Time (sec)

Our work - Methods

• Auditory signal processing• Collecting acoustic features

– Duration features– Variability features– Speech density features

Time (sec)

Acoustic feature extraction

Duration features:

utterance gap

•Mean utterance duration•Mean log utterance duration•Mean gap duration•Mean log gap duration

Acoustic feature extractionVariability features:

Meso scale

Pitch (f0) / period

Variability

Power variability

Shimmer

Jitter

Micro scale

Results – acoustic featuresList of wordsFluent textDuring task

mean jitter score

Results - classification

C vs. SC vs. DS vs. D

List of words0.73810.700.7143

Passage0.80950.690.5200

Interview0.750.87100.7667

ControlSchizophreniaDepression

List of words202220

Passage111015

Interview333129

Sample sizes:

Correct classification rates (LOU):

Linear classification with SVM

Results – classificationseparated by gender

C vs. SC vs. DS vs. D

Both genders0.750.87100.7667

Male only0.68570.73080.4516

Female only0.69230.93330.9231

C vs. SC vs. DS vs. D

Both genders0.73810.700.7143

Male only0.54170.63160.4783

Female only0.88240.75000.6316

List of words:

Interview:

Results – Correlation

Acoustic featureSANS total (cc, p-val)

Utterance duration(-0.5953, 0.0004)

Gap duration(0.4223, 0.0180)

Spoken ratio(-0.5134, 0.0031)

Fragmented speech(-0.4831, 0.0059)

Emphasis(-0.4782, 0.0065)

Inflection(-0.3475, 0.0554)

Acoustic featureHAM-D

Utterance duration(0.4661, 0.0164)

Inflection(-0.5007, 0.0092)

Jitter(-0.4555, 0.0194)

Schizophrenia:

Depression:

Results - summary

• Possible to find signal to mental state in prosodic features

• Local and micro-scale features are discriminative

• Type of speech test influences discrimination• Possibly signal is more prominent in females

Future directions

• Spectral features• Using linguistic elements: phonemes, words• Forced alignment and functional data analysis• Language and speaker identification methods• Personal directions:

– Other signal domains: general sounds, music…

Thank you!