effects of training on timbre recognition and appraisal … am acad audiol 13: 132-145 (2002)...

J Am Acad Audiol 13 : 132-145 (2002)

Effects of Training on Timbre Recognition and Appraisal by Postlingually Deafened Cochlear Implant Recipients Kate Gfeller* 4 Shelley With Mary Adamek* Maureen Mehr$ Jenny Rogers* Julie Stordahl* Shelly Ringgenberg*

Abstract

The purpose of this study was to compare the effect of structured training on recognition and appraisal of the timbre (tone quality) of musical instruments by postlingually deafened cochlear implant recipients . Twenty-four implant users (Nucleus C124M) were randomly assigned to a con-trol or a training group . The control group experienced only incidental exposure to music in their usual daily routine . The training group participated in 12 weeks of training delivered via a laptop computer in which they were introduced to excerpts of musical instruments representing three fre-quency ranges and four instrumental families . Those implant recipients assigned to the training group showed significant improvement in timbre recognition (p < .0001) and timbre appraisal (p < .02) compared to the control group . Correlations between timbre measures and speech percep-tion measures are discussed .

Key Words : Cochlear implants, music, normal-hearing adults, timbre, tone quality

Abbreviations : ANOVA = analysis of variance ; HINT = Hearing In Noise Test

Sumario

El proposito de este estudio fue comparar el efecto de un entrenamiento estructurado para el reconocimiento y la apreciacion del timbre (calidad tonal) de instrumentos musicales, en sujetos con sorderas post-linguisticas utilizando implantes cocleares . Veinticuatro sujetos implantados (Nucleus C124M) fueron distribuidos al azar en un grupo de entrenamiento y en otro de control . El grupo de control experimento solamente una exposicion incidental a la miisica en su rutina cotidiana usual . El grupo de entrenamiento participo de un entrenamiento de 12 semanas reali-zado a traves de una computadora portatil a la cual se le introdujeron selecciones de instrumentos musicales en tres rangos frecuenciales y con cuatro familias de instrumentos . Aquellos sujetos implantados asignados al grupo de entrenamiento mostraron una mejoria significativa en el reconocimiento del timbre (p < .0001) y en la apreciacion del timbre (p < .02) comparados con el grupo control . Se discuten tambien correlaciones entre las medidas del timbre y las medidas de percepcion auditiva .

Palabras Clave : Implantes cocleares, musica, adultos normo-oyentes, timbre, calidad tonal

Abreviaturas : ANOVA = analisis de variancia ; HINT = Prueba de Audicion en Ruido

*School of Music, The University of Iowa ; tDepartment of Speech Pathology and Audiology, The University of Iowa ; tlowa Cochlear Implant Clinical Research Center, Department of Otolaryngology, University of Iowa Hospitals and Clinics, Iowa City, Iowa

Reprint requests : Kate Gfeller, Department of Otolaryngology, 21033 PFP, University of Iowa Hospitals and Clinics, 200 Hawkins Dr. Iowa City, IA 52242

Music Training and Timbre/Gfeller et al

T

he cochlear implant is a device designed to enhance speech perception for per-sons who are profoundly deaf and who

receive little or no benefit from traditional hear-ing aids . Because implants have been designed primarily to assist spoken communication, the technical features of current devices are less than ideal in encoding and transmitting many aspects of musical sound. For example, adult implant recipients perceive rhythmic features of music similarly to adults with normal hearing (Gfeller and Lansing, 1991, 1992 ; Gfeller et al, 1997) but are significantly less accurate than adults with normal hearing in perception of pitch and pitch patterns (Gfeller and Lansing, 1991, 1992 ; Pij and Schwartz, 1995 ; Gfeller et al, 1997, in press; McDermott and McKay, 1997 ; Pijl, 1997).

This article focuses on the perception of implant recipients on another important aspect of music, timbre . Timbre, represented by the spectral envelope, is "that attribute of auditory sensation in terms of which a listener can judge that two sounds similarly presented and having the same loudness and pitch are dissimilar" (Acoustical Society of America, 1960) . Timbre assists people in recognizing which musical instrument has produced the sound and con-tributes to auditory scene analysis as one listens to complex musical groups such as orchestras . Further, the unique tone quality of various instruments is an important aspect of music's aesthetic quality. Therefore, the extent to which the implant transmits unique and rich timbral representation is not insignificant with regard to listener satisfaction .

With regard to timbre perception, more sub-tle features of the spectral envelope are not faithfully represented with typical speech pro-cessing strategies (Gfeller et al, 1998) . Those few extant studies regarding timbre recognition sug-gest that implant recipients using various devices (Dorman et al, 1991 ; Schultz and Kerber, 1994 ; Gfeller et al, 1998 ; Fujita and Ito, 1999) were less accurate than adults with normal hearing were in open-set recognition of four or five different musical instruments (synthetic representations or recordings) . However, accuracy improved in closed-set testing conditions, particularly fol-lowing brief training (Dorman et al, 1991 ; Schultz and Kerber, 1994 ; Gfeller et al, 1998 ; Fujita and Ito, 1999) .

Appreciation of the sound quality of timbre is possibly of greater importance with regard to implant satisfaction for nonmusicians than is the ability to identify specific instruments because

aesthetic enjoyment is a major function of music . Two studies comparing timbral ratings of adults with that of implant recipients (Schultz and Kerber, 1994; Gfeller et al, 1998) indicate that some instruments sounded significantly more pleasant to normal-hearing listeners than to implant recipients . Gfeller and Lansing (1991) found that eight implant recipients using Ineraid implants (analog coding strategy) rated a larger percentage of musical instruments as pleasant or beautiful than did 10 recipients of the Nucleus 22 device (FOFIF2 feature extractor) . More recent studies with current devices and coding strategies indicate significant differences among users of four different devices (Clarion, Ineraid, Med-El, and Nucleus) with regard to timbral appraisal (Gfeller et al, in press) . Users of all four different devices assigned significantly lower quality ratings than did adults with normal hearing to the timbre of musical instruments played in higher frequency ranges (p < .0008) and for instruments from the string family (p < .04) .

This inferior representation of music via the implant is unfortunate, given that music is a pervasive art form and a common acoustic event in everyday life . It is likely that implant recipi-ents will be exposed to musical sounds regularly, and they are likely to be interested in active lis-tening . Following the perception of speech, the appreciation of music is the next most commonly expressed aspiration by users of cochlear implants (Stainsby et al, 1997). Thus, some level of satis-faction in music listening is a suitable goal for the aural rehabilitation of implant recipients .

Unfortunately, improved music enjoyment is not strongly correlated with length of implant use. That is, mere experience with the device is not associated with improved music perception or enjoyment. However, there are significant cor-relations between self-report of musical enjoy-ment and the amount of time devoted to music listening postimplantation (Gfeller et al, 2000). Many implant recipients indicate that music sounded quite distorted or unpleasant initially. However, informal reports by recipients suggest that some implant users have developed appre-ciation and improved perceptual accuracy for some, though not all, aspects of music listening following focused and repeated practice listening to music over time (Gfeller, 1998). However, there are no data to confirm whether significant improvement occurs specifically for timbre per-ception and appreciation . Further, it is not clear whether these improvements would apply to a randomly selected sample of implant recipients . Modest improvements for timbre recognition fol-

133

Journal of the American Academy of Audiology/Volume 13, Number 3, March 2002

lowing acute laboratory-based training (Fujita and Ito, 1999) are encouraging and suggest that timbre recognition may be enhanced through training . However, benefits from longer and more systematic training for appraisal as well as recog-nition require further study.

Therefore, the purpose of this study was to examine (a) the effect of training on recogni-tion and appraisal (meaning the extent to which the respondent likes or appreciates the quality of sound) of timbre by postlingually deafened implant recipients and (b) the relations between subject characteristics and speech perception scores with timbre recognition and appraisal.

METHOD

Participants

Participants included 24 postlingually deaf-ened adults with 12 to 15 months experience with a Nucleus CI24M implant. Participants were randomly assigned to either the control group (no training-incidental exposure only) or the train-ing group. Nine of those assigned to the control group completed pre- and post-testing . Three additional participants initially assigned to the control group, who completed pretesting, were unable to return for post-testing after 3 months . Two were unable to return owing to medical problems unrelated to the implant (e.g., knee replacement surgery) and one chose not to return for post-testing for personal reasons.

Eleven of the 12 assigned to the training program completed pretesting, the entire 3-month training protocol, and post-testing . The twelfth person assigned to the training group started the home training program but found that he was unable to read the written infor-mation that accompanied the musical stimuli because of very limited reading competency (the program requires a reading ability of approxi-mately fourth grade level) . He struggled through 2 weeks of the program, with the help of his granddaughter, who assisted him as often as possible with the written material . Although he stated that he enjoyed listening to the musical excerpts, the clinical research team determined that the effort required to complete all aspects of the program (reading written information and writing responses to the stimuli) seemed bur-densome for him and his family, and he was excused from the study.

Participant characteristics (gender, age, hearing history, speech perception scores, music listening habits, and past experiences) of those

in the control and the training group appear in Tables 1 and 2. None of the participants in either group was a professional musician or had formal music instruction beyond high school as deter-mined by administering the Iowa Music Back-ground Questionnaire (Gfeller et al, 2000). The amount of music training was a score derived from the total years of training reported for ele-mentary school, high school, college, and adult education. The larger the number, the more musical training the participant has received . A full description of how scores for musical train-ing are obtained appears in Gfeller et al (2000) . Pre- and postimplantation listening habit scores are obtained from a Likert-type rating scale, with points ranging from 2 to 8. A score of 2 indi-cates that listening is/was typically very lim-ited-less than 2 hours of listening weekly. Conversely, a score of 8 signifies that the indi-vidual typically listens/listened to music 9 or more hours per week .

Training Program

The technical features, program format, and content of the training program are described in detail in Gfeller et al (1999) . Briefly, the train-ing program, which consists of 48 lessons over a period of 12 weeks (4 lessons per week), was delivered via a laptop computer with external speakers . The computerized format provided consistent and spaced listening experience for each training participant over time . The lis-tening examples and exercises for timbre required approximately 10 minutes of listening and responses per day. The program provided information about the four families of musical instruments that are characterized by similar principles of sound production : strings, wood-winds, brass, and pitched percussion (specifi-cally, piano) . The timbre training lessons began with an introduction to each family of instru-ments and an introduction to each individual instrument within that family. For example, pictures and sound samples of the violin, viola, cello, string bass, and harp were presented as representatives of the string family. Written descriptions of each instrument included infor-mation about the material from which the instrument is made (e.g ., trumpets are made from brass, whereas clarinets are made from ebony wood), how it is played (e .g ., blowing wind over a reed or striking a key), what styles of music are typically played on the instrument (e.g ., saxophones are often associated with jazz music, and fanfares are often played by trum-

134


Table 1 Characteristics of Participants : Gender, Age, and Speech Scores

Gender Age (yr) LPD IAC IAC in N HINT CNC

Control 1 M 38 15 .00 61 .54 44.87 42.47 34.00 2 F 71 6 , 00 66.67 66.67 98.12 68.00 3 F 75 0 .50 70.51 62.82 90.41 56.00 4 F 70 2 .00 75 .64 71 .79 98.04 72.00

5 M 72 2 .00 57 .69 57.69 97.10 54.00 6 M 33 5 .50 43 .58 37.17 74.20 24.00 7 F 51 0 .17 73 .08 67.95 88.61 78.34 8 M 51 1 .00 28.32 16.00 9 F 40 22 .00 42.31 43.59 56.64 16.00

Mean 55 .7 7 .50 61 .38 56.57 74.88 46.48 SD 16 .57 9 .89 12.79 13 .01 26.34 24.28

Training 1 F 38 7 .00 88.68 59.00 2 F 42 21 .00 51 .15 30.34 3 M 70 10 .00 41 .02 19.23 4 M 60 12 .00 62 .82 66.67 97.12 32.00 5 M 46 33 .00 36.85 19.34 6 F 53 35 .00 55 .12 56.41 99.06 58.00 7 F 52 3 .00 55 .12 47.43 72.64 22.00 8 M 72 4 .00 75 .64 70.51 81 .80 54.00 9 F 42 19 .00 56.41 70.51 87.62 18.00 10 F 75 4 .00 64 .10 61 .54 98.15 56.00 11 M 73 10 .00 37 .18 32.05 35.40 12.00

Mean 56 .6 7 .20 57 .69 56 .41 75.12 32.40 SD 13 .98 7 .53 14 .07 16.57 24.06 20.95

Dashes indicate data missing . LPD = length of profound deafness, IAC = Iowa Consonant Test, IAC in N = Iowa Consonant in Noise Test : HINT = Hearing In Noise

Sentence Test : CNC = CNC Word Recognition Test .

pets), and characteristic sounds that are asso-ciated with each instrument (e.g ., the flute is often described as clear and brilliant, whereas the French horn is often described as mournful in sound quality) .

The lessons began with simple sounds (e.g ., brief melody patterns on solo instruments), and each week, more complex sounds (e.g ., complex songs with accompaniment) were gradually inte-grated into the program. The initial lessons introduced the instruments through pictures and sounds, focusing on one family of instru-ments at a time . Subsequent lessons contrasted high- (e.g., flute), medium- (e.g ., clarinet), and low- (e .g., saxophone) sounding instruments; asked the trainee to identify the instruments from the entire pool of eight instruments; and offered opportunities for the trainees to describe the sounds of the instrument and decide if they like or dislike the sound. Each week began and ended with a review of previously learned infor-mation . There were multiple opportunities each week to hear the various sounds of the instru-

ments, and melodies were repeated throughout the series of lessons.

Audio files of melodies played on each instrument were included in each week's lessons . The audio files included characteris-tic sounds of the instruments (such as the slid-ing pitches of a trombone playing high to low) representing the frequency range of the instru-ment, different types of articulation (staccato, legato, etc .), and a variety of different solo melodies that are representative of the reper-toire of that instrument . For example, sound samples for the clarinet included representa-tions as diverse as Dixieland clarinet music and the clarinet solo heard in Prokofiev's classical composition, "Peter and the Wolf." Pictures of each instrument and short video clips accom-panied the sound files on the weekly lessons so that the listener could form paired associ-ations between the picture of the instrument and its name and the sound of that instru-ment that he/she now hears through the implant .

135


Table 2 Musical Background of Individual Participants and Pre- to Post-test Change in Timbre Recognition

Amount of Musical Training*

Preimplant Listening Habits*

Postimplant Listening Habits*

Timbre Recognition Pre- to Post-test Changet

Control 1 - - 14 .9 2 2 5 -7 .4 3 1 5 2 -3 .7 4 1 3 3 7 .4 5 2 4 3 -7 .4 6 2 6 5 3 .7 7 2 8 5 -3 .7 8 1 4 2 3 .7 9 4 5 6 -7 .4 Mean SD

2.571 3.409

1 .875 0.991

5.000 1 .633

Training 1 4 5 8 37 .1 2 2 4 3 14 .8 3 2 6 3 18 .5 4 4 4 4 26 .0 5 5 3 .7 6 4 3 6 22 .3 7 6 2 14 .8 8 2 6 3 18 .6 9 5 4 3 -11 .2 10 4 6 3 22 .2 11 2 5 3 18 .6 Mean 10.000 3.222 4.900 SD 11 .165 1 .202 1 .101

*The larger the number, the more musical training the individual received ; tpre- to post-test change in percent correct for timbre recognition .

Dashes indicate data missing .

Although all major orchestral instruments from each family were included in the program, the practice examples focused on eight of the most commonly heard instruments from these families : the violin and cello from the string family; the flute, clarinet, and saxophone from the woodwind family ; the trumpet and trom-bone from the brass family ; and the piano from the percussion family. The piano was chosen to represent the percussion family because it is a well-known instrument and because it covers a broad frequency range. In addition, although many percussion instruments do not have a clear fundamental frequency and have many inhar-monic sounds, each piano key has a clearly defined fundamental frequency and overtones and can play easily recognizable melodic patterns such as those used in the listening task for this study. Each of these instruments was represented in the pro-gram an equal number of times over the 12-week period, and each was represented using diverse examples that illustrated the typical frequency range as well as diverse timbral characteristics possible for that instrument.

Pre- and Post-test Measures

Stimuli and Instrumentation

The timbral stimuli used in the three tests described below included eight different musi-cal instruments that (a) are commonly known to nonmusicians, (b) represent three different fun-damental frequency ranges (low = 131-262 Hz, medium = 262-534 Hz, and high = 534-1068 Hz), and (c) represent four different instrumental families based on the principles of sound pro-duction (brass, woodwind, pitched percussion, strings) . Given the dependent variable of recog-nition, less commonly known instruments that could have represented a particular frequency range for a given instrumental family were not included (e.g ., the viola, a mid-range string instrument, which is often confused with the vio-lin) . The trumpet (medium) and trombone (low) represented the brass family. Flute (high), clar-inet (medium), and saxophone (low) represented the woodwind family. The violin (high) and cello (low) represented the string instruments. Each

136


of these instruments was presented in its char-acteristic frequency range. Pitched percussion was represented by piano, which was played in two different frequency ranges (medium and high, both ranges equally characteristic for the piano) .

Past studies of timbre with adults with nor-mal hearing have often used synthesized or highly controlled samples of isolated pitches in timbre testing (Gfeller et al, 1998) . Although isolated and synthesized tones have the advan-tage of greater experimental control, there are difficulties extrapolating the findings from such isolated stimuli to contextualized experiences of "real-life" music listening (i .e ., ecologic valid-ity) (Kendall, 1986; Parncutt, 1989 ; Handel, 1995) . Even trained musicians can have difficulty correctly identifying musical instruments from isolated and synthetic representations (William, 1996) . Consequently, recordings of real instru-ments playing a standardized connected melodic sequence, which include transients that are important cues for recognition, were used in this study.

Specific structural features of the stimuli were selected as follows : The melodic pattern played by each instrument was composed specif-ically for use in these tests . It consisted of a seven-note sequence of equal-duration notes (quarter note = 100 beats per second) that included intervalic changes (stepwise movement and skips) within the basic framework of a C major scale . Eight professional musicians recorded the melodic pattern on their respective instruments at the University of Iowa Elec-tronic Studios using a single Neuman micro-phone located approximately 5 feet from the test instrument at an approximate angle of 10 degrees . The microphone was a Mackie 1402-VLZ Mixer with phantom power. Sound was recorded onto a Panasonic SV-3700 DAT Recorder using Sony PDP-124 digital audio-tapes, which were played back and digitally transferred to a PowerMac 8500/120 computer Digidesign Audiomedia III sound card using Digidesign Sound Designer II . All sounds were extracted into individual aiff mono files with a sample rate of 44.1 kHz (16-bit sound) .

These timbral stimuli were used in three different tasks : (a) Timbre Recognition, that is, identification of the instrument being played by sound alone; (b) Timbre Appraisal Test A: Gen-eral Appraisal, or overall pleasantness of the sound; and (c) Timbre Appraisal Test B: Qual-itative Ratings for Three Dimensions of Tim-bre . The stimuli were presented via an Apple

Computer (model M340-9) with a Sony touch-screen (Model CPD-2000SF), with the sound files presented through Yamaha external speak-ers (Model YST-M15) . The test was adminis-tered in sound field in a quiet room, at 70 dB SPL. Implant recipients were allowed to adjust their processor to a most comfortable level of loudness .

Timbre Recognition

Prior to taking the timbre recognition test, the computer touchscreen displayed a grid fea-turing 16 commonly heard musical instruments (uneven match) and the name of the instru-ment. Subjects were instructed to touch the pic-ture of each musical instrument that they knew by sound prior to deafness . Although instru-ments were chosen that were considered well known to the general public, musical training and experiences differed considerably across the general population . Therefore, it is possible that a person may be unfamiliar with one of the instruments . Those instruments that were not known by an individual prior to deafness, as determined through this preliminary step, were accounted for in the analyses of data .

After completion of two practice items, each instrument was presented three times in random order. After each melody was played, the grid with pictures and names of 16 different instru-ments appeared on the touchscreen, and the individual selected the instrument that he/she thought produced the sound just heard . The computer automatically recorded all responses . No feedback on accuracy was provided during or after testing . Test results were reported as per-cent correct of those instruments known by sound prior to hearing loss .

Timbre Appraisal Test A: General Appraisal

Individual differences in preference for par-ticular instrumental sounds are to be expected, even for normal-hearing listeners. Comparisons of ratings by implant recipients with those of normal-hearing listeners help identify whether the quality of sound heard by implant recipients is generally less pleasant than what is heard by normal-hearing listeners. They also help iden-tify whether there are particular types of sounds that are particularly pleasant or disagreeable for implant recipients .

In Timbre Appraisal Test A, subjects were asked to appraise the overall pleasantness of

137


each item by touching the point along a 100-mm visual analog scale (anchored with the bipolar adjectives of "dislike very much" = 0 to "like very much" = 100) that represented their opinion. Subjects were told in the preliminary instructions that there is no right or wrong answer, that they should give their most hon-est appraisal of the sound quality, and that they should feel free to use the entire range of the 100-mm scale. Each item was played two times in random order. No information about the instrument being played was provided during the task ; therefore, the appraisal was based on sound only as opposed to possible preconceived biases about particular instruments. The com-puter automatically recorded each response . Internal consistency of responses for 48 implant recipients for this Timbre Appraisal Test A has been calculated to be .74 (Gfeller et al, in press) .

Timbre Appraisal Test B: Qualitative Ratings of Three Dimensions of Timbre

Factorial investigations on verbal reports of musical sound quality indicate that 88 percent of variance can be accounted for using three scales : dull-brilliant (or sharp), compact-scattered, and full-empty (Von Bismark, 1974 ; Pratt and Doak, 1976). In prior experiments (Von Bismark, 1974), normal-hearing listeners have judged sounds having more low-frequency energy as more dull (on a continuum of dull to sharp) in quality, whereas sounds having more high-frequency energy were judged as more sharp (brilliant) in quality. Normal-hearing listeners have rated those sounds with more noise as sounding more scattered (on a continuum of com-pact to scattered) . Sounds more rich in harmon-ics tend to be judged as more full (on a continuum of empty to full) (Strong and Plitnick, 1992).

Variety in instrumental timbre (e.g ., the more characteristically hollow sound of the clar-inet vs the very rich and deep sound of a cello) contributes to the novelty and beauty that lis-teners seek in music; thus, one sound being judged as more "empty" than another is not inherently undesirable (Gfeller et al, 1998). However, implant recipients have often described the quality of musical instruments as sounding more thin or shrill than their recollection of how a particular instrument sounded prior to deafness . Furthermore, considerable "noise" (i .e ., scattered) in a musical signal is usually undesirable, other than for special stylistic

effects (e .g ., brief use of a growling tone quality to highlight a particular phrase in the genre, "the blues," or purposeful distortion in "heavy metal" rock music) . Thus, if implant recipients tend to rate sounds as less full, less brilliant, or more scattered than do adults with normal hearing, these data would suggest that the implant trans-mits an uncharacteristic or inferior quality in contrast to what is heard through a normal ear (Gfeller et al, 1998, 2000, in press) . Therefore, these three bipolar adjectives were used as the anchors for three visual analog scales of 100 mm each to measure these three dimensions of sound quality of each timbral item .

Each instrument was presented two times and in random order. After each item, the sub-jects registered their opinion by touching that point on the visual analog scale for each of the three bipolar adjectives . The subject was told that there is no right or wrong answer, that they should give their most honest appraisal of the sound quality, and that they should feel free to use the entire range of the 100-mm scale. The computer automatically recorded each response . No information about the instrument being played was provided .

Correlations with Other Variables

Timbre perception scores were correlated with several demographic variables (age at time of testing and length of profound deafness), which have been found to contribute to vari-ability in patient performance on speech recog-nition and some prior music perception measures (Gfeller et al, 1998, 2000). The timbre perception scores were also correlated with the following speech perception measures : the Hearing In Noise Sentence Test (HINT), the CNC Word Recognition Test, and the Iowa Consonant Test in noise and without noise (sound only). The audiologic test scores from the annual visit, which coincided with the administration of the timbre pretesting, were used in the analyses . Prior studies indicate that the strength of rela-tions between speech recognition and music per-ception varies depending on the specific stimuli and listening task (Gfeller and Lansing, 1991, 1992; Gfeller et al, 1997, 1998, 2000). These correlations then would provide information specific to timbre recognition and appraisal, as well as an indicator regarding the extent to which implant recipients who do well in this aspect of music listening are generally good implant users.


RESULTS

A Ithough the participants were randomly assigned to the groups, to confirm that there were no significant differences on key variables, t-tests were calculated on the follow-ing measures : age, length of profound deafness prior to implantation, speech perception scores (Iowa Consonant Test and Iowa Consonant Test in Noise, HINT, CNC), measures of musical background and listening habits, and pretest scores for timbre recognition (number of items correct and percent correct of those instruments reported familiar prior to hearing loss) (see Table 1) . There were no significant differences found except for the variable addressing the participants' listening habits prior to their hear-ing losses . Although the training group listened to significantly more music before their hearing losses (p < .02), prior research indicates that this variable is only weakly correlated with success in music listening postimplantation (Gfeller et al, 2000). Thus, the two groups were considered equivalent prior to the treatment intervention with regard to all but the variable of preim-plant listening habits, which has not been a strong predictor of implant performance in prior music testing.

The data were analyzed using a mixed gen-eral linear model (repeated-measures design) and t-tests for pairwise comparisons (a priori con-fidence level = .05) . Test results were correlated with post-test scores for recognition of timbre . Because the control and training groups had different experiences during the treatment por-tion of the study, correlations for post-test scores were calculated separately for the control and the training group .

Timbre Recognition

An analysis of variance (ANOVA) with repeated measures revealed no significant dif-ference between the control and training groups on pretest scores for timbre recognition (p < .29) . Thus, the two groups demonstrated a sim-ilar ability to recognize instrumental timbre prior to training . However, the training group had a significantly greater percentage of items correct in the post-test scores than did the con-trol group (p < .002). In addition, the training group achieved a significant increase from pre-to post-test in the number of recognition items correct (p < .0001), whereas the control group achieved no significant increase in items correct (t = 0.16, p < .88) . Figure 1 shows the pre- and

E o`

c 0 a 0 U

4)

r\ v

m

I- v a C) I- 0 0 i

0 0 O c r` v

m c 0 U

c N 1zj' -CC) r c `

c r-- 0 o

- T

'a d a ~ ~ - r-- CO v 'T

C) C) T M C) r I-

c l0 C- 4)

c E 0 t LO ~ 00

ca 0 CO C) C)

k L V 0 U c 0

0 V CO T V N

N E f~ V r I- N `O r N

C m R a Q)

o` E I`f ~1: r CO r

w C 0 c c

0 t I' r V CO N CO LO CA 0 O C) ,- 1- 6) N V cc r N N V m

N d N

c I-- V r LO C) LO C) C) I- r of T W M N

a0 U r C)

M N

ICJ V - rl- r co r C6 Ci~ r rr

0 a v n v r cc N ) U r C) N r T I~

T- N N 2 0 c O M L V r__ N rl-

M co r-- C) N c7 d' N Q

X 0 c a) P, m a QL N ID s

c N

a) E O c

Q)

m a

)=a)o m °- a Q C s Q) C O N .~ O

c d >U=C~C'~ F-

139


Cn E 0

c 0

0 U U Q

m L ro

a .j

C7 0 L

0 0)

.C

C m

c N ro

O) c O 0 .Q

d o- m

c R m C. 0 _E r L m m r 0 U c

a r N O O U! E 0-) (O Ln M CO N 0 r- a

N C

a a

E O O r O r N co co o) c) CO r O N

L w C 0

0 C Q

r N CO Lr) N r cOcOor,ooC3i

O 0 ro

r CO u) "-

m m

N N d O r M O Ch f ++ CO 0-) Cl) C~ 0) b 0) N U ao

N Cr) r O r

N N M Cfl co Cfl

0 'a)

to r r- M T r) U r 0 - CO

c CRrr n r

0 c CR ro a o °o

r

c E 0 . . ~

E a) -r- 0 O O- Q 9 O = -O C O O C O

m=~ .o ? m WD o c d >0EUCn~~

Control Training

Figure 1 Pre- and post-test mean scores for timbre recognition : control and training groups .

post-test mean scores for the two groups . Test-retest reliability (3-month time interval) was calculated by correlating the pre- and post-test scores for the control group (no intervention). The test-retest reliability coefficient was r = .87 . Split-halves reliability was r = .66.

The error patterns for the post-test scores of the control and training groups appear in Tables 3 and 4. The control subjects showed a more diffuse error pattern than do adults with normal hearing tested in prior studies (Gfeller et al, in press) and those in the training group. The percent correct for each instrument appears in boldface type, within the cell for correct responses. The other numbers in each cell of Tables 3 and 4 refer to the percentage of par-ticipants who misidentified a given instrument as another instrument . For example, 3.7 percent of those in the control group identified the piano in the high-frequency range as being a violin . Still another 3.7 percent identified the piano as being a flute. The piano was confused with 8 of the other instruments from the array of 16 options. Pearson correlation coefficients show-ing the relations between timbre recognition and demographic and speech data appear in Table 5.

Timbre Appraisal Test A: General Appraisal

An analysis of variance and post hoc analy-ses revealed a significant increase in general appraisal by the training group from pre- to post-test (p < .02) and no significant change by

140


Table 5 Correlations between Post-test Scores for Recognition and Participant

Characteristics

100---

90 0 0

i 80,-

Number of Items Correct in Post-testing N Y

70~~

Control Training

J O

601- 50- 40'-

Length of Profound Deafness -.28 - .39 Iowa Consonant Test 26 31 30I- Iowa Consonant Test in Noise 42 -.16 0 20'- Hearing In Noise Test 85 21 10r CNC Word Recognition Test 27 68 0'--- Musical background (classes, lessons) 75 -.44 Music listening habits . preimplantation -.81 55 Music listening habits, postimplantation -.97 07 Age at time of testing 93 80 Figure 2

the control group (p < .73) . Table 6 and Figure 2 report the means of the general appraisal (lik-ing) results.

With regard to the spectral characteristics of instrumental families, when members of the family (e.g., violin and cello in the string family or trumpet and trombone in the brass family) are large- and small-scaled versions of one another, the spectral characteristics exhibited by a large instrument in a family will be observed at higher frequencies for the smaller instruments of a fam-ily (Strong and Plitnick, 1992) . However, in the case of these three woodwind instruments, flute, saxophone, and clarinet, there are more struc-tural differences in the instrumental design including the type of reed (e.g., air reed for flute and cane reeds for clarinet and saxophone) and the material from which the body of the instru-ment is made (e.g ., silver, wood, brass) . There-fore, ratings for individual instruments and family groups are worthy of note . In particular, the training group showed a significant increase in the positive appraisal of four specific instru-ments, including the violin (p < .0001), trumpet (p < .0008), saxophone (p < .02), and flute (p < .03) . With regard to preference of instrumental fam-

1 =Control 2 =Training

Condition 8 Pretest N, Post-test

Pre- and post-test mean scores for timbre quality: like versus dislike (control and training) .

ily, t-test results showed significant preference by both control and training participants for piano sounds compared with string (p < .001), woodwind (p < .001), and brass (p < .009) timbres . Mean scores and standard deviations for the pre- and post-test scores of the control and train-ing groups appear in Table 7 (by instrumental family), Table 8 (by frequency range), and Table 9 (by individual instruments) .

Timbre Appraisal Test B: Qualitative Ratings of Three Dimensions of Timbre

Table 6 and Figures 3, 4, and 5 show the means and standard deviations of the two groups for each of the bipolar adjectives (compact = 0, scattered = 100; dull = 0, brilliant = 100; empty = 0, full = 100) . Whereas a larger score for the adjectives brilliant and full indicates better sound quality, a higher score for the adjective scattered indicates more noise and a generally less desirable sound quality.

An ANOVA revealed no significant differ-ences between control and training groups for pretest ratings on descriptors of brilliant or scat-tered. The post-test results yielded a signifi-cant difference in qualitative reports between the

Table 6 Pre- and Post-test Results for Appraisal of Timbre : Control and Training Group

Control

Liking Brilliant Scattered

Training

Pretest Post-test Pretest Post-test

M = 51 .28 (SD = 25.43) M = 52.50 (SD = 23.07) M = 56.20 (SD = 25.23) M = 65.40 (SD = 22.01) M = 54.93 (SD = 27.62) M = 54 .88 (SD = 25.50) M = 59.69 (SD = 25.67) M = 67 .02 (SD = 25.82) M = 43.59 (SD = 25.22) M = 46 .32 (SD = 21 .78) M = 38 .96 (SD = 23.64) M = 34 .06 (SD = 23.74)

Full M=54.13 (SD=25.16) M=60 .74 (SD=21 .35) M=57 .81 (SD=25.68) M=64 .26 (SD=24.06)


Table 7 Pre- and Post-test Results for Appraisal of Liking by Instrumental Family : Control and Training Group

Control Training

Pretest Post-test Pretest Post-test

Piano M = 56.83 (SD = 23.21) M = 55.33 (SD = 24.70) M = 66.50 (SD = 22.57) M = 62.55 (SD = 25.03) Strings M = 40.33 (SD = 26.46) M = 40.51 (SD = 23.00) M = 41 .91 (SD = 32.16) M = 58.34 (SD = 24.63) Woodwinds M = 53.67 (SD = 25.11) M = 57.87 (SD = 22.54) M=61 .46 (SD=19.82) M=72.24 (SD=17.95) Brass M = 53 .08 (SD = 18.49) M = 53.28 (SD = 18 .49) M = 52.84 (SD = 19 .78) M=65.07 (SD=19.21)

Table 8 Pre- and Post-test Results for Appraisal of Liking by Frequency: Control and Training Group

Pretest

Co ntrol

Post-test

Tra

Pretest

ining

Post-test

High Medium Low

M = 40.85 (SD = M = 55.28 (SD = M = 57 .70 (SD =

25 .57) 23 .53) 24 .24)

M = M = M =

43.52 (SD = 55.07 (SD = 59.04 (SD =

26.59) 20.49) 18.84)

M = 46.17 (SD = 30.65) M = 59.68 (SD = 21 .99) M = 62.82 (SD = 72.30)

M = M = M =

57 .23 (SD = 66.68 (SD = 72.30 (SD =

25.16) 20.25) 17.56)

two groups both for ratings of brilliant (p < .05) and scattered (p < .009) . However, pre- to post-test changes were nonsignificant for both con-trol and training groups .

Regarding perceived level of fullness versus emptiness of sound for all instruments com-bined, there were no significant changes from pre- to post-test for either the training or the con-trol group. However, with regard to specific instrumental timbre, the control group rated both the violin (p < .006) and the trumpet (p < .03) timbres as sounding more full during the post-testing than they did during the pretest. The training group judged the piano sound in its

medium register (frequency range) to be signif-icantly more full (p < .03) than it had during pretesting. The t-tests revealed no significant dif-ferences between the two groups for instru-mental families . Correlations between appraisal scores, demographic variables, and speech scores appear in Table 10.

DISCUSSION

T hese results, along with prior findings regarding training by Fujita and Ito (1999),

are promising with regard to the potential ben-efit of training for timbre-related music listen-

Table 9 Mean Appraisal Scores for Individual Instruments: Pre- and Post-test

Control Training

Piano High Medium

Violin Cello low

Pretest

M = 50 .44 (SD = 24.46) M = 63.22 (SD = 22.60) M = 26.67 (SD = 21 .06) M=54.00 (SD=24.54)

Post-test

M = 49.89 (SD = 27.89) M = 60.78 (SD = 20.38) M=26.39 (SD=19.42) M = 55.47 (SD = 16.16)

Pretest

M = 62.05 (SD = 25 .38) M = 70 .96 (SD = 18.89) M=22.73 (SD=30.55) M = 61 .09 (SD = 20.33)

Post-test

M = 59.27 (SD = M = 65.82 (SD = M=46.64 (SD= M = 70.05 (SD =

25.69) 24.50) 26.10) 16 .54)

Flute high M = 45.44 (SD = 25.75) M = 54.28 (SD = 23.93) M = 53.73 (SD = 20.77) M = 65.77 (SD = 20 .47) Clarinet medium M=54.89 (SD=23.50) M=57.11 (SD=20.11) M = 64.18 (SD = 18.75) M=71 .82 (SD=16.19)

Saxophone medium M = 60.67 (SD = 25.00) M = 62.22 (SD = 23 .94) M = 66.45 (SD = 18 .34) M = 79 .14 (SD = 14.92)

Trumpet medium M = 47 .72 (SD = 24.94) M=47.33 (SD=19.65) M = 43.91 (SD = 19 .33) M = 62.41 (SD = 18.95) Trombone low M = 58.44 (SD = 24.09) M=59 .22 (SD=15.60) M = 60.91 (SD = 16.54) M = 67.73 (SD = 19.54)

142


1001 ---

90!-

°0 80F

70= C

`° 60!. . m` 50l

40-

1 =Control 2=Training

Condition

0 Pretest Post-test

1 =Control 2 =Training

Figure 3 Pre- and post-test mean scores for timbre qual-ity: dull versus brilliant (control and training).

ing tasks, particularly for recognition and over-all appraisal (liking) of tone quality. With regard to recognition, evidently, the cochlear implant, though imperfect, does provide enough of the spectral features to permit recognition of vari-ous timbral qualities given some focused lis-tening experience (i .e ., paired association between pictures of the sound source and the tim-bral quality created) or practice . Those implant recipients in the training group showed less dif-fuse errors, thus more nearly resembling the types of errors made by normal-hearing per-sons (Gfeller et al, in press) .

With regard to timbral appraisal, struc-tured listening experience does result in some perceived improvement in overall sound quality, although those in the control group show a lack of improvement over time . More particularly, the post-test scores for participants in the train-ing program showed encouraging improvement from pretest scores on three of the four family groups (all but piano) and even compare favor-

10 0 0 90'-

a 80 ;- 70,L

m 60,,, <n I 0 50,-

', 0 40-

30~_ a E 20i- o U 10i-

0 1 =Control 2 =Training

Figure 4 Pre- and post-test mean scores for ti ity: compact versus scattered (control and training) .

Figure 5 Pre- and post-test mean scores for timbre qual-ity: empty versus full (control and training).

ably with appraisal scores of woodwind and brass instruments in prior studies of adults with normal hearing (Gfeller et al, in press) . The gains are particularly striking for the string family, for those instruments in the higher-frequency range, and the individual instruments violin and trumpet, all of which tended to com-pare especially poorly with prior appraisals by adults with normal hearing (Gfeller et al, in press) . The systematic training that "forces" lis-tening practice may facilitate acclimatization to the various sound qualities of instruments . It is also possible that training alters expectations about what constitutes aesthetic beauty. For example, one participant in the training group, following program completion, commented that although the instruments did not sound the same as they used to (i .e ., prior to deafness), she had learned to appreciate them in a new way.

Table 10 Correlations between Post-test Scores for Appraisal and Participant

Characteristics

Appraisal Ratings in Post-testing

Control Training

Length of Profound Deafness -.25 - .21 Iowa Consonant Test -.27 -.90 Iowa Consonant Test in Noise -.16 -.97 Hearing In Noise Test 08 -.92 CNC Word Recognition Test -.06 -.38

Condition Musical background (classes, lessons) 77 -.40

* Pretest Music listening habits Post-test

preimplantation 81 -.91 Music listening habits .

mbre qual- postimplantation 30 36 Age at time of testing 12 58

143


These findings have important practical impli-cations given that most people choose to listen to music for personal pleasure and enjoyment.

It is worthy of note that 11 of the 12 implant recipients (adults with complex lives) assigned to the training group completed the home-based 3-month-long training program. As indicated earlier, the one noncompliant participant was unable to complete the program because of read-ing problems . In any rehabilitation program of extended duration, compliance can be a problem. This high level of compliance suggests that the program itself is easy to use and has content with enough interest to sustain involvement.

The stability of the pre- to post-test scores for the control group indicates that incidental lis-tening alone does not result in similar improve-ment in timbre recognition as one often observes for speech recognition over that same time period . Furthermore, strong negative correlations between the post-test recognition scores and listening time postimplantation by the control group suggest that informal listening alone over time is not highly correlated with recognition . Perhaps this is because in many listening opportunities (e.g ., listening to the radio or watching a musical ensem-ble on television), the listener is provided with few opportunities to confirm which instrument he/she is listening to, or musical instruments are heard as blends (as in orchestras or bands), and thus identification of individual instruments is not enhanced through incidental listening.

Several past studies have shown weak cor-relations between speech recognition scores and timbre recognition (Gfeller et al, 1998, 2000). This is not especially surprising when one con-siders the difference in tasks; that is, speech recognition often requires identification of a word made up of combined phonemes or the ability to understand connected discourse. Tim-bre recognition requires the ability to match particular spectral features of a given musical instrument with prior memory of that sound. However, this particular study indicates mod-erate to strong relationships between timbre recognition and the Iowa Consonant Test in Noise and the HINT. Those speech recognition tests that require more reliance on isolated speech features represented by the spectral envelope are those most likely to be strongly related to timbre recognition. Another possible explanation for these moderate to strong rela-tionships would be that people who are gener-ally doing well with their implant are more likely to do well on music recognition as well as speech recognition tasks. It is interesting to

contrast the correlations for timbre recognition and speech with those of timbre appraisal and speech, which are quite weak. Providing a qual-itative rating of a sound is a very different task from recognition and calls on different listening skills .

The difference in post-test scores between the control and training group suggest that those in the control group who perform the best in recognition are those individuals who achieve generally superior benefit on various listening tasks. However, with training, the relationships with speech perception performance are much less clear, suggesting that even implant recipi-ents with poorer speech perception performance scores or little prior musical training can improve on timbre recognition given suitable postim-plant training . Thus, although the tone quality is not as positive as that reported by normal-hearing listeners, training can help implant recipients to achieve recognition and apprecia-tion more similar to normal-hearing listeners . This has important implications for quality of life, given the prevalent nature of music in our culture. However, although training has helped implant recipients to improve their performance, their group data are still poor in contrast to many of the outcomes for adults with normal hearing reported in Gfeller et al (in press) . Thus, music perception equivalent to that of normal-hearing listeners will likely require major changes in signal processing.

Acknowledgment . This study was supported by grant P50 D000242 from the National Institute on Deafness and Other Communication Disorders, National Institutes of Health (NIH); grant RR00059 from the General Clinical Research Centers Program, National Center for Research Resources, NIH; and the Iowa Lions Foundation .

REFERENCES

Acoustical Society of America. (1960) . American Standards Acoustic Terminology. New York : Author.

Dorman M, Basham K, McCandless G, Dove H. (1991) . Speech understanding and music appreciation with the Ineraid cochlear implant. Hear J 44(6):32-37 .

Fujita S, Ito J. (1999) . Ability of nucleus cochlear implantees to recognize music. Ann Otol Rhinol Laryngol 108:634-640 .

Gfeller K. (1998) . Music appreciation from the perspec-tive of implant recipients . CONTACT 12(3):24-25 .

Gfeller K, Lansing C. (1991). Melodic, rhythmic, and tim-bral perception of adult cochlear implant users. J Speech Hear Res 34:916-920.

Gfeller K, Lansing C . (1992) . Musical perception of cochlear implant users as measured by the Primary

144

Measures of Music Audiation: an item analysis . J Music Ther 29(1):18-39 .

Gfeller K, Woodworth G, Witt S, et al . (1997) . Perception of rhythmic and sequential pitch patterns by normally hearing adults and adult cochlear implant users. Ear Hear 18:252-260 .

Gfeller K, Knutson JF, Woodworth G, et al . (1998) . Timbral recognition and appraisal by adult cochlear implant users and normal-hearing adults . J Am Acad Audiol 9:1-19.

Gfeller K, Witt S, Kim K-H, et al . (1999) . A computerized music training program for adult cochlear implant recip-ients. JAcad Rehabil Audiol 32:11-27 .

Gfeller K, Christ A, Knutson J, et al . (2000) . The musi-cal backgrounds, listening habits and aesthetic enjoyment of adult cochlear implant recipients . J Am Acad Audiol 11 :390-406 .

Gfeller K, Witt S, Woodworth G, et al . (in press) . The effects of frequency, instrumental family, and cochlear implant type on timbre recognition and appraisal. Ann Otol Rhinol Laryngol

Handel S. (1995) . Timbre perception and auditory object identification . In : Moore BCJ, ed. Hearing. New York : Academic Press, 425-461 .

Kendall RA. (1986) . The role of acoustic signal partitions in listener categorization of musical phrases. Music Percept 4:185-214 .

McDermott H, McKay C. (1997) . Musical pitch percep-tion with electrical stimulation of the cochlea. J Acoust Soc Am 101:1622-1631 .


Parncutt R. (1989) . Harmony : A Psych oacoustical Approach . New York : Springer Verlag .

Pijl S. (1997). Labeling of musical interval size by cochlear implant patients and normally hearing subjects . Ear Hear 18:364-372 .

Pijl S, Schwartz D. (1995) . Melody recognition and musi-cal interval perception by deaf subjects stimulated with electrical pulse trains through single cochlear implant electrodes. JAcoust Soc Am 98:886-895 .

Pratt RL, Doak PE . (1976) . A subjective rating scale for timbre . J Sound Vibr 45:317-328 .

Schultz E, Kerber M. (1994) . Music perception with the MED-EL implant. In : Hochmair-Desoyer JL, Hochmair EC, eds. Aduances in Cochlear Implants. Manz : Wien, 326-332.

Stainsby TH, McDermott HJ, McKay CM, Clark GM. (1997, September) . Preliminary results on spectral shape perception and discrimination of musical sounds by normal hearing subjects and cochlear implantees . In : Proceedings of the International Computer Music Conference. Thessaloniki, Hellas : International Computer Music Association, 11-14.

Strong WJ, Plitnick GR . (1992) . Music, Speech Audio. Provo, UT: Soundprint .

Von Bismark G. (1974) . Timbre of steady sounds : a fac-torial investigation of its verbal attributes . Acustica 30:146-172 .

William DA. (1996) . A study of internal validity of the instrument timbre preference test . J Res Music Music Educ 44:268-277 .

effects of training on timbre recognition and appraisal … am acad audiol 13: 132-145 (2002)...

Documents