kenneth reeder, jon shapiro & jane wakefield the university of british columbia

1Checkpoint Literacy: 15th European Conference on Reading, Berlin 2007

Effectiveness of speech recognition technology in promoting reading proficiency and attitudes for Canadian immigrant children

Kenneth Reeder, Jon Shapiro & Jane Wakefield

The University of British Columbia


What is The Reading Tutor?


Developed by Project LISTEN, Carnegie Mellon University, the

Reading Tutor is:• an automated

Reading Tutor that displays stories on a computer screen, and listens to children read aloud.


The Reading Tutor…

• lets the child choose to read from a menu of high-interest stories.


The Reading Tutor…

• analyzes the student's oral reading and offers help when the reader makes mistakes, gets stuck, clicks for help, or is likely to encounter difficulty.


Why conduct this study?

Edudata Canada, 2002:– reading comprehension testing of

41,962 grade 4 students in the province of British Columbia, Canada:

–32% of students qualifying for ESL support were reading at levels “below expectations”

–19% of non-ESL students reading “below expectations.”


Previous Findings (1)

• Effectiveness of the RT when compared to human tutoring:

Native English speaking children using the RT for 15-20 minutes/day made equal or greater reading gains than a comparison group who received 30 minutes per day of tutoring in oral reading by trained volunteers.

(Reeder et al 2005)


Previous Findings (2)

• English language proficiency and reading gains: The low English proficiency group made greater gains on most measures than the higher proficiency groups and native speakers. (Reeder et al, 2005)


Previous Findings (3)• Attitudes and concept of self and the RT

experience: Results of two tests, the Elementary Reading Attitude Survey (McKenna & Kear, 1990) and Reading Self-Concept Scale (Chapman & Tunmer 1995) did not correlate with reading gains.

• There were however very positive responses toward the RT experience demonstrated in a short post-RT interview. This held for all home language and English proficiency groups included in the study.

(Reeder, et al, 2005)


Goals of the study

1. to determine the role of English language (L2) proficiency in reading gains with the RT

2. to study children’s attitudes toward reading, and self-views as readers following experience with the RT


Participants• 62 students from 3 Vancouver

elementary schools, grades 2-6 (ages 7-13 yrs)

• Receiving ESL support• Grouped by English language

proficiency using Woodcock-Muñoz Language Survey: – low (21) – mid (17) – high (24)


School Settings

• Three Vancouver city schools: system has >50% non-English speaking households.

• Lower income neighborhoods• Reading Tutor systems installed

centrally in each school in a classroom, Library or learning resource room


METHODEnglish proficiency and reading gains:

• Record English proficiency levels (Woodcock-Muñoz)

• Pretest reading performance – WRMT-R• 20 minutes daily practice on the Reading

Tutor for 3-5 months• Record time on task, control its potentially

confounding impact with a covariate analysis

• Re-test reading performance, compare gain scores by groups


Woodcock Reading Mastery Test (Revised)

• Subtests of Total Reading Cluster: (TRC)– Word Identification (WI)– Word Attack (WA)– Word Comprehension (WC)– Passage Comprehension (PC)

• Normed on US populations recently• Used by Project LISTEN, Carnegie

Mellon University


English Proficiency & Gains, Total Reading Cluster WRMT-R

*

430

440

450

460

470

480

490

500

510

Pre Post

Mea

n s

core

Low

Mid

High

Main effect of time (pre/post gains), p=.003, effect size = .146

LOW English proficiency group made greater gains than MID and HIGH groups, p=.005, effect size = .211


Reading Gains, WRMT-R Subtests

SUBTEST p EFFECT SIZE

Word Identification .000 .198

Word Attack NSD --

Word Comprehension .001 .182

Passage Comprehension .003 .146


Language Proficiency X gain effects for LOW group, WRMT-R Subtests

SUBTEST p EFFECT SIZE

Word Identification NSD --

Word Attack .004 .171

Word Comprehension .042 .103

Passage Comprehension .005 .168


Variation in RT usage

Month

Average Minutes

on RT/day

Min Max

October 12.60 12.00 13.20

November 16.94 12.72 24.93

December 17.90 10.28 27.38

January 16.00 10.23 23.00

February 14.87 6.82 27.19

March 16.04 8.67 28.00

April 15.45 4.80 35.20

Reading Level-George G0.0010.00

NovDec Jan Feb MarApr

Reading LevelReading Level-Helen C0.005.00

Nov Dec Jan Feb Mar

Reading Level


Total time on the RT and reading gains• Mean total time spent per child on the

RT: 12.6 hours, SD = 8.05• We found a significant interaction

effect (p = .000, effect size .211) between total time spent per child on the RT and reading gains.

• Thus, our study has controlled this variable effectively.

• Next analysis: examine the relationship of time spent on the RT to reading gains.


Discussion: Reading Gains and English Language Proficiency

• While results suggest that all language proficiency groups made gains on three of the four subtests of our measure, it was the lowest English language proficiency group that appeared to benefit the most from the RT experience.

• These findings fit with our understandings of learning rates in second language acquisition: rapid growth in early learners is typical of SLA.


Reading Attitudes, Self Concept and Students’ Experience

10 item semi-structured clinical interviews administered to participants prior to, and immediately following the RT trial.


Pre- and Post-RT Interview

Post-RT Interview were supplemented with questions that focused specifically on the children’s perceptions about the influence of the RT on their reading skills and abilities.


“Do you think you are getting better at reading? If yes, how do

you know? If no, why not?”

• Yes. Because in the RT, you learn stuff like small words turns to big word. (Kevin,

Low ELP, Cantonese)

• Yeah. Before I went on the RT I didn’t know how to break words apart. (Johnny,

High ELP, Vietnamese)

cont’d…


“Do you think you are getting better at reading? If yes, how do

you know? If no, why not?”, cont’d…

• A little. Because there some new words that I learned that I've never heard of before. The hardest level mostly I know all the words in most some stories but sometimes I don't know most words. (Mubashir, Mid ELP, Urdu)

• Yes. Because when I was reading before didn't stop at the periods but now with the RT, I stop at the periods. (Jennifer, Mid ELP, Spanish)

• Yes. By reading the books I don't know and I read it again and I get it. (Ricky, Mid ELP, Mandarin)


“Is reading easier for you now?”

• Yeah. Whenever like Kindergarten I don't know how to read and the English. The teacher help me read and when I get to gr 1 I try to read myself easy books and gr. 2 do the same thing as gr 1 and gr 3 I try to read a chapter book and now I read chapter books too. (Justin, High ELP, Vietnamese)

• Yes, it is getting more clearer to me. (Mubashir, mid ELP, Urdu)

• Yes. Because before it was hard for me to read chapter books and now the RT made me better at reading chapter books. (Jennifer, mid ELP, Spanish)


“Do you understand more of what you read better than

before?”

• Yes. When I was reading slow, I didn't understand as much and then I started the RT. I started reading slow and then I started reading faster. (Jennifer M, mid ELP, Spanish)

• Yes. Before I didn't quite understand it

but now I know what's happening. (Helena, mid ELP, Spanish)


“Can you figure out words better than before?”

• Last year I always had to sound out, even little words. Now I only sound out bigger words. (Connie, mid ELP, Cantonese)

• Yes. It helps me sound out the word and helps me break apart the words and put it all together. (Jennifer J, high ELP, Cantonese)

• Some of them. (Ricky, mid ELP, Mandarin)


“Do you know more words now?”

• Yes. Usually before a story it told some words that were really hard and it thought were words we wouldn't understand. Sometimes it told us how to spell it and sometimes it just said it told us the meanings. (Mubashir, mid ELP, Urdu)

• Yes. Sometimes when I get stuck on a word all I need to do is click on them and then it will tell me then I usually go back to the start or I just read that word and keep going on. I learned how to read faster. (Johnny, high ELP, Vietnamese)

• A lot more. I learned how to break apart the words and then sound it out all together. (Jennifer J., high ELP, Cantonese)

cont’d…


“Do you know more words now?”(continued)

• Yes, now. When I started reading chapter books I didn't know much words and then the RT helped me read new and more words. I read better than before. (Jennifer M., mid ELP, Spanish)

• Yes. The RT helped me reading stories. Before I didn't read too much. (Helena, mid ELP, Spanish)

• Yeah. Some long words and spelling too. (Ricky, mid

ELP, Mandarin)


Discussion: Interview data on reading attitudes

While analysis of the pre-post responses is on-going, responses regarding the impact of the RT indicate that the children, regardless of first language and English language proficiency:– Enjoy using it, and;– Are aware of the specific impacts on

their growth in reading and writing.


Your questions?

Please ask us here,

or email [email protected],

learn more at Project LISTEN’s web site:

http://www-2.cs.cmu.edu/~listen/

http://www-2.cs.cmu.edu/~listen/


Thank you / Danke!


APPENDICES

Fluency GainsGrade Level Gains


Fluency gains: Internal measures

Average Fluency per month: RT measure

0.005.00

10.0015.0020.0025.0030.0035.0040.00

Flu

en

cy

in W

PM


Grade level gains: RT measure

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

October November December January February March April

RT

Inte

rnal

Gra

de

Lev

el


Grade level gains: internal measures

Reading Level

Month Average Reading Level Lowest Highest

October 1.50 0.00 3.00

November 2.44 0.71 4.69

December 2.94 0.77 5.22

January 2.96 1.12 4.96

February 3.34 1.92 6.00

March 3.40 2.00 5.31

April 3.60 2.00 6.00

kenneth reeder, jon shapiro & jane wakefield the university of british columbia

Documents

reading proficiency

reading tutoranalyzes

reading content

reading tutorlets

students oral reading

automated reading tutor

greater reading gains

shapiro wakefield