ultrasound research in linguistics

Imaging and ultrasound basicsMethodology

Speech applicationsPractical concerns

Demonstration

Ultrasound Research in Linguistics

Diana Archangeli and Jeff MielkeUniversity of Arizona

*Supported by

College of Social and Behavioral Sciences, University of Arizona,and James S. McDonnell Foundation grant #220020045 BBMB

October 8, 2005

Diana Archangeli and Jeff Mielke University of Arizona * Supported by College of Social and Behavioral Sciences, University of Arizona, and James S. McDonnell Foundation grant #220020045 BBMBUltrasound Research in Linguistics



Demonstration

Reasons to imageArticulatory imaging technologiesHow ultrasound imaging worksInterpreting ultrasound imagesDemonstration

Why to image

Acoustic analysis doesn’t tell us everything about speech.Articulatory imaging is important for:

I analyzing the articulatory parameters of speech, e.g., thingslike:

I studying sounds which can be articulated in more than oneway (e.g. American English /r/)

I studying articulatory trade-offs such as tongue backness/liprounding or tongue height/pharynx expansion.

I real-time feedback for second-language learners and people inspeech therapy.

I . . .




Demonstration


How to image

Several different technologies are available for imaging the vocaltract, such as:

I X-ray

I MRI

I EMMA

I X-ray microbeam

I ultrasound

Each has its own set of advantages and disadvantages. . .




Demonstration


X-ray

X-ray imaging works becausex-rays are variably obstructedby different types of tissueaccording to their density.




Demonstration


X-ray pros and cons

Pros

I high temporal resolution

I locates tongue in mouth

I images entire length ofvocal tract

Cons

I expensive

I not portable

I toxic




Demonstration


MRI (Magnetic Resonance Imaging)(photo from Lee Ryan & the CNL)

MRI works because hydrogenin tissue that has been alignedby a strong magnetic field canbe jiggled by radio waves(fMRI detects hemoglobin).




Demonstration


MRI pros and cons

Pros

I non-toxic

I beautiful pictures

I locates tongue in mouth

I images entire length ofvocal tract

I 3-dimensional imaging

Cons

I scary for some

I subject must lie down andcan’t move

I very noisy

I poor temporal resolution

I expensive

I not portable




Demonstration


EMMA (ElectroMagnetic Midsagittal Articulometry)(photo from Jim Scobbie, QMUC)

EMMA works because receiver coilsglued to the vocal tract report theirdistance from three stationarytransmitter coils.




Demonstration


EMMA pros and cons

Pros

I non-toxic

I precise data points(on tongue)

I high resolution

I good temporal resolution

Cons

I invasive

I expensive

I not portable

I tongue tip and blade only

I must blow-dry tongue(to glue pellets)




Demonstration


X-ray microbeam (images from U. of Wisconsin Med. Sch.)

Uses a narrow x-ray beam to track gold pellets.




Demonstration


X-ray microbeam pros and cons

Pros

I less toxic than older x-rayimaging

I precise data points(on tongue)

I high resolution

I good temporal resolution

Cons

I invasive

I expensive

I not portable

I front of tongue only




Demonstration


Ultrasound

Ultrasound works byemitting high-frequencysound waves which arereflected back to thetransducer by surfaceswith sharp changes indensity.




Demonstration


Ultrasound pros and cons

Pros

I non-toxic

I high temporal resolution(30+ fps)

I inexpensive

I portable

I subjects are comfortable

Cons

I picture can be grainy

I shows tongue only (nopassive articulators)

I limited to area betweenthyroid cartilege and frontof mandible (which castshadows)

I difficult fixed pointidentification




Demonstration


Pros and cons highlights/summary

safe

por

table

inex

pen

sive

hig

hre

solu

tion

hig

hte

mpor

alre

s.

pas

sive

artic.

whol

evo

caltr

act

surfac

es(n

otpoi

nts

)

3D

X-ray x x x x x

MRI x x x x x x

EMA x x x

X-ray microbeam ? x x

Ultrasound x x x x x




Demonstration


Pros and cons highlights/summary

safe

por

table

inex

pen

sive

hig

hre

solu

tion

hig

hte

mpor

alre

s.

pas

sive

artic.

whol

evo

caltr

act

surfac

es(n

otpoi

nts

)

3D

X-ray x x x x x

MRI x x x x x x

EMA x x x

X-ray microbeam ? x x

Ultrasound x x x x ? ? x ?




Demonstration


Ultrasound imaging

I Ultrasound imaging employs high-frequency sound waves togenerate images of objects.

I Abrupt changes in density create echoes.




Demonstration


Ultrasound imaging of the vocal tract

I The tongue-air interface is strongly echogenic (because of thelarge difference in density between air and muscle)

I An ultrasound transducer placed beneath the chin can producea real-time movie of the full length of the tongue surface.




Demonstration


An ultrasound image of the tongue




Demonstration


Things to look at:

I /a/ vs. /i/ vs. /u/

I tense and lax vowels

I Clear /l/ vs. dark /l/

I Retroflex /r/ vs. bunched /r/

I Grooved vs. non-grooved fricatives

I Front /k/ vs. back /k/

I /h/ before different vowels

I . . .




Demonstration

Imaging the palateHead movementUltrasound data extraction and analysis

Palate obscured by air

The ultrasoundtransducer’s field ofview:




Demonstration



Normally there is airbetween the tongue andthe palate.




Demonstration



The tongue-airinterface is quiteechogenic.




Demonstration



This makes the top ofthe tongue imagebrightly.




Demonstration



Nothing beyond thetongue is visible.




Demonstration


Palate visible through water

The space between thetongue and palate canbe filled with water.




Demonstration



Sound waves are nolonger impeded by thesurface of the tongue(because tongue-water isless echogenic thantongue-air)




Demonstration



This makes thesurfaces of the palateappear in the image.




Demonstration



The tongue is lessvisible.




Demonstration


An ultrasound image of the palate




Demonstration


Tongue image. . .




Demonstration


. . . Palate image




Demonstration


Tongue+Palate image




Demonstration


Problem. . .

The head and transducer can move




Demonstration


Flashlights




Demonstration


Head stabilization: HATS (http://speech.umaryland.edu/ahats.html)




Demonstration


Head stabilization: HATS cont’d




Demonstration


Head stabilization: UQAM (Menard et al. 2005)




Demonstration


Head movement correction: UBC’s optotrak(photo from Campbell et al. 2005)




Demonstration


Head movement correction: UBC’s optotrak cont’d




Demonstration


Head movement correction: Palatron (Mielke et al. 2005)




Demonstration


Signal paths




Demonstration


SonoSite TITAN and C-11 transducer




Demonstration


Camcorder and microphone preamplifier




Demonstration


Video mixer and Digital Video recorder




Demonstration


APIL subject in chair




Demonstration


A palate image




Demonstration


The palate image after transformation




Demonstration


Traced palate surfaces




Demonstration


Tracings from several palate images




Demonstration


A composite palate tracing




Demonstration


A tongue image ([u])




Demonstration


The tongue image after transformation




Demonstration


The tongue image after measurement




Demonstration


The tongue image with measurements




Demonstration

In the fieldIn the classroomClinical ultrasoundIn linguisticsExisting labs

Speech applications of ultrasound

I documenting languages in the field or laboratory

I 2nd language acquisition and speech therapy

I in the clinic (e.g. for partial glossectomy patients or patientswith orthodontic appliances)

I in linguistics, analyzing many aspects of speech that are notobvious from the acoustic record




Demonstration


Brugman (2005): Pharyngeal Vowels in N|u




Demonstration


A field setup (Gick and Wilson 2005, Namdaran in prep)

I Special chair

I SonoSite

I Desktop mic standw/counterweighted arm

I Lapel mic

I Mini mic amp

I Sony miniDV cam




Demonstration


The University of South Florida audiovisual phonemedatabase (Frisch et al. 2005)




Demonstration


Ultrasound in the ESL Classroom (Meadows et al. tomorrow)




Demonstration


Impact of an orthodontic palatal expander appliance ontongue movement and speech (Bressmann et al. 2005)




Demonstration


The phonetics of speech errors (Frisch et al. 2005)

Alveolar targets Velar targets




Demonstration


Transparent vowels in Hungarian (Benus 2005)

Transparent vowels in front harmony contexts are less retractedthan transparent vowels in back harmony contexts.




Demonstration


Gestural Timing and Magnitude of English /r/(Campbell et al. 2005)




Demonstration


Syllabic position effects in articulation (Davidson 2005)




Demonstration


American English /r/ production strategies(Mielke, Baker, and Archangeli in prep)

bunched: crow retroflex: prose




Demonstration


Articulatory similarity between segments (Mielke 2005)




Demonstration


Articulatory similarity between segments: results




Demonstration


I University of Arizona

I University of British Columbia

I Haskins Laboratories

I University of Maryland-Baltimore

I New York University

I Universite du Quebec a Montreal

I Queen Margaret University College

I University of South Florida

I University of Toronto




Demonstration


Ultrafest




Demonstration

Setting up an ultrasound labSoftware

Equipment used in an ultrasound lab

Ultrasound machine $30,000video camera $1,000video mixer $1,000computer $1,000video software $500imaging software $100video recorder $1,000microphone $100preamplifier $100chair and tableincidentals such as gel, tissue, wipes




Demonstration

Setting up an ultrasound labSoftware

Software used in an ultrasound lab

Applications

I Image-J (NIH)

I Adobe Photoshop or Corel PHOTO-PAINT (not open source)

I Final Cut Express (not open source)

Ultrsound-specific

I Ultra-CATS (Toronto)

I EdgeTrak (Maryland)

I Surfaces (Maryland)

I QT Extractor (Arizona)

I Palatron (Arizona)

I Glossotron (Arizona)

I Measure-o-tron (Arizona)




Demonstration

Time for trying out the machine. . .


ultrasound research in linguistics

Documents