captioned video for l2 listening and vocabulary learning: a meta-analysis

20
Captioned video for L2 listening and vocabulary learning: A meta-analysis Maribel Montero Perez * , Wim Van Den Noortgate, Piet Desmet ITEC-iMinds, KU Leuven Kulak, Etienne Sabbelaan 53, 8500 Kortrijk, Belgium Received 30 October 2012; revised 2 May 2013; accepted 19 July 2013 Available online Abstract This study reports on a meta-analysis of the effectiveness of captioned video (i.e. L2 video with L2 subtitles) for listening comprehension and vocabulary learning in the context of second language acquisition. The random-effects meta-analysis provides a quantitative measure of the overall effect of captions on listening comprehension and vocabulary acquisition, as well as an investigation into the relationship between captioning effectiveness and two potential moderators: test type and proficiency level. We conducted a systematic review and calculated effect sizes for 18 retrieved studies. Separate meta-analyses were performed for listening comprehension (including data of 15 studies) and for vocabulary learning (including data from 10 studies). The findings indicate a large effect of captions on listening comprehension as well as on vocabulary acquisition. Test type was found to moderate the effect sizes of listening comprehension. Proficiency level did not moderate the effect sizes of listening comprehension and vocabulary learning. The article concludes with a contextualized discussion of the results and an overview of the limitations of the present meta-analysis as well a number of future research perspectives. Ó 2013 Elsevier Ltd. All rights reserved. Keywords: Captioning; Video; Vocabulary learning; Listening comprehension; Meta-analysis 1. Introduction During the last decades, audiovisual materials (video, TV programs, etc.) have been promoted and exploited as rich language learning resources (for a comprehensive review, see Vanderplank, 2010). The advent of multimedia learning environments (Brett, 1995) made audiovisual material easily accessible and triggered new ways of training foreign language listening comprehension. These multimedia environments did not only make large amounts of (authentic) video available, but they also offered listening support by means of a “technological overlay” (Robin, 2007, p. 109) such as scripts, paced audio delivery, and on-screen text. One particular type of on-screen text is captioning, also * Corresponding author. Tel.: þ32 56 24 60 80. E-mail addresses: [email protected] (M. Montero Perez), [email protected] (W. Van Den Noortgate), [email protected] (P. Desmet). 0346-251X/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.system.2013.07.013 Available online at www.sciencedirect.com ScienceDirect System 41 (2013) 720e739 www.elsevier.com/locate/system

Upload: piet

Post on 14-Dec-2016

270 views

Category:

Documents


9 download

TRANSCRIPT

Page 1: Captioned video for L2 listening and vocabulary learning: A meta-analysis

Available online at www.sciencedirect.com

ScienceDirect

System 41 (2013) 720e739www.elsevier.com/locate/system

Captioned video for L2 listening and vocabulary learning:A meta-analysis

Maribel Montero Perez*, Wim Van Den Noortgate, Piet Desmet

ITEC-iMinds, KU Leuven Kulak, Etienne Sabbelaan 53, 8500 Kortrijk, Belgium

Received 30 October 2012; revised 2 May 2013; accepted 19 July 2013

Available online

Abstract

This study reports on a meta-analysis of the effectiveness of captioned video (i.e. L2 video with L2 subtitles) for listeningcomprehension and vocabulary learning in the context of second language acquisition. The random-effects meta-analysis providesa quantitative measure of the overall effect of captions on listening comprehension and vocabulary acquisition, as well as aninvestigation into the relationship between captioning effectiveness and two potential moderators: test type and proficiency level.

We conducted a systematic review and calculated effect sizes for 18 retrieved studies. Separate meta-analyses were performedfor listening comprehension (including data of 15 studies) and for vocabulary learning (including data from 10 studies). Thefindings indicate a large effect of captions on listening comprehension as well as on vocabulary acquisition. Test type was found tomoderate the effect sizes of listening comprehension. Proficiency level did not moderate the effect sizes of listening comprehensionand vocabulary learning. The article concludes with a contextualized discussion of the results and an overview of the limitations ofthe present meta-analysis as well a number of future research perspectives.� 2013 Elsevier Ltd. All rights reserved.

Keywords: Captioning; Video; Vocabulary learning; Listening comprehension; Meta-analysis

1. Introduction

During the last decades, audiovisual materials (video, TV programs, etc.) have been promoted and exploited as richlanguage learning resources (for a comprehensive review, see Vanderplank, 2010). The advent of multimedia learningenvironments (Brett, 1995) made audiovisual material easily accessible and triggered new ways of training foreignlanguage listening comprehension. These multimedia environments did not only make large amounts of (authentic)video available, but they also offered listening support by means of a “technological overlay” (Robin, 2007, p. 109)such as scripts, paced audio delivery, and on-screen text. One particular type of on-screen text is captioning, also

* Corresponding author. Tel.: þ32 56 24 60 80.

E-mail addresses: [email protected] (M. Montero Perez), [email protected] (W. Van Den

Noortgate), [email protected] (P. Desmet).

0346-251X/$ - see front matter � 2013 Elsevier Ltd. All rights reserved.

http://dx.doi.org/10.1016/j.system.2013.07.013

Page 2: Captioned video for L2 listening and vocabulary learning: A meta-analysis

721M. Montero Perez et al. / System 41 (2013) 720e739

referred to as same-language (Bird and Williams, 2002, p. 509), unilingual or intralingual subtitling (Jung, 1990;Williams and Thorne, 2000, p. 279). Although captions were originally developed for hearing-impaired persons inthe 1980s and made accessible on television through the Teletext system, language teachers and researchers quicklydiscovered the benefits of captions for language learners (Price, 1983; Vanderplank, 1988, 1990): Captioning not onlyoffers the learner support but also provides him with a “target language learning environment” (Bird and Williams,2002, p. 509).

The first large-scale study to investigate the potential of captioning as a pedagogical tool was conducted by Price in1983. Approximately 500 participants watched four clips with or without captions. Findings showed that captioningresulted in superior comprehension independent of background, linguistic, and social variables. Ever since Price’s(1983) pioneer work, language teachers and researchers have considered the simultaneous presentation of secondlanguage (L2) text and video (i.e. bimodal input) as an interesting study object. Studies have yielded encouragingresults concerning the effectiveness of captioning for reinforcing L2 learners’ listening comprehension (Baltova,1999b; Garza, 1991; Guichon and McLornan, 2008; etc.). Researchers also highlighted benefits of captioned videofor stimulating vocabulary learning (Danan, 1992; Sydorenko, 2010; etc.). Yet, the generalizability of much researchon captioning is problematic for two main reasons. First of all, studies have focused on the effectiveness of captions fordifferent proficiency levels. Second, studies have measured different components of listening comprehension andvocabulary learning and have used different tests to do so.

The aim of this paper is to summarize primary research on the effectiveness of captioning for enhancing L2listening comprehension and vocabulary learning of secondary and higher education students. In order to do so, weadopted the methodology of research synthesis and meta-analysis, which consists of a reproducible and “systematicsecondary review of accumulated primary research studies” (Norris and Ortega, 2006, p. 4). This study also analyzedthe relationship between captioning effectiveness and two study characteristics: The participants’ proficiency leveland the type of test employed to assess either listening comprehension or vocabulary learning.

2. Background research and objectives

The background research section first provides a rationale for using captioned video (Section 2.1), before dis-cussing the main benefits and issues studied in captioning literature (Section 2.2). Finally, we provide a rationale forconducting a meta-analysis and present the three main research questions of this study (Section 2.3).

2.1. Rationale for using captioned video

A rationale for using captioned video can be found in (1) research on multimedia materials, (2) research onlistening comprehension, and (3) the dual-coding theory.

(1) With the emergence of multimedia programs, initial research studies on the use of audiovisual material forlistening skill development generally focused on the effectiveness of video by comparing it to traditional audiouse. The use of video was found to “enrich target language processing” (Meskill, 1996, p. 196) and to improvelistening ability (Brett, 1997; Secules et al., 1992; Terrell, 1993), vocabulary acquisition (Brett, 1998; Duquetteand Painchaud, 1996), and oral proficiency (Hung, 2009). However, although video may present learners with“contextually rich materials” (Baltova, 1999a, p. 34), the language and images presented in the video do notnecessarily enhance comprehension (Danan, 2004) or provide opportunities for learning (Baltova, 1999a).Chapelle (2003) argued that input can be made more comprehensible by modifying it. Research questions onmultimedia use thus no longer ask whether or not to use video, but rather how to present video materials in anefficient way (Baltova, 1994). Hence, researchers have investigated the potential of adding on-screen text in theform of traditional native language (L1) subtitles, reversed subtitles (L2 text, L1 video), captioning (L2 video,L2 text), and keyword captioning (L2 video, L2 keywords) and have reported overall positive results concerningtheir effectiveness (e.g. Danan, 1992; Markham, 2001; Winke et al., 2010).

(2) Listening comprehension has often been defined as one of the most complex skills in language learning (Oxford,1993). Because of its real-time nature (Buck, 2001), learners cannot control the speed of the input, which leavesthem with “fewer opportunities to invoke time-consuming explicit knowledge than do the other skills” (Hulstijn,2007, p. 791). This may cause a series of problems, such as speech stream chunking, decoding, and

Page 3: Captioned video for L2 listening and vocabulary learning: A meta-analysis

722 M. Montero Perez et al. / System 41 (2013) 720e739

interpretation (Goh, 2000; Graham, 2006; Rost, 2002), which are related to the lack of automated wordrecognition (de Groot and van Hell, 2005). Research has also suggested that successful listening is highlycorrelated with learners’ vocabulary size (Staehr, 2009; Webb and Rodgers, 2009). Yet, L2 learners havegenerally not developed a sufficiently broad vocabulary to successfully cope with aural input. Hence, researchershave looked at the potential of using on-screen text and captioning more in particular (Markham, 1989; Neumanand Koskinen, 1992; Price, 1983; Vanderplank, 1988), as a means to outweigh deficient vocabulary size andovercome listening problems, while at the same time exposing the learner to the aural and written version of theL2. In this perspective, captioning has been proposed as some kind of “mediating device” (Vanderplank, 1988, p.280), assisting the learner visually where listening skills fall short.

(3) Theoretical underpinnings that have informed studies on captioned video, are generally embedded in theframework of information processing theories. At the heart of these theories is Paivio’s (1986) dual-codingtheory, which states that the presentation of an item in two modes (aural and written) stimulates the verbaland imagery system. This is expected to improve information processing, subsequently leading to greater depthof processing and better recall. Paivio’s theory was also the basis of the cognitive theory of multimedia learning(Mayer, 2001), which was developed in the context of L1 learning and is centered around a series of principles.One of those principles, namely the multimedia principle, states that “people learn better from words andpictures than from words alone” (Fletcher and Tobias, 2005, p. 117). This principle implies that informationconveyed in different formats stimulates the visual and verbal channel, leading to greater processing. Bycontrast, the redundancy principle suggests that the simultaneous presentation of audio, imagery, and on-screentext “interferes with rather than facilitates learning” (Sweller, 2005, p. 159). It is difficult, however, to maintainthis principle in the context of L2 learning since different types of benefits have been associated with the use ofcaptioned video for foreign language learners (see Section 2.2).

2.2. Topics and issues studied in captioning literature

The bulk of literature produced on captioned video has revealed two main linguistic benefits.1 First of all, a series ofstudies have examined whether captioning is effective for improving L2 learners’ comprehension of audiovisualmaterial. Although some studies failed to find differences between captioning and other conditions (Hernandez, 2004;Taylor, 2005), most have shown better comprehension test scores for full captioning groups when compared to nocaptioning control groups (Baltova, 1999b; Chung, 1999; Garza, 1991; Guichon and McLornan, 2008; Huang andEskey, 1999e2000; Markham, 1989, 2001). Second, results of primary research also tentatively suggest that theuse of captioned video may foster vocabulary learning (Bird and Williams, 2002; Chai and Erlam, 2008; Markham,1999; Sydorenko, 2010; Winke et al., 2010). In this perspective, Vanderplank (1990, p. 229) noticed that the use ofcaptions may help learners to carefully study the language of the video, which eventually stimulates “taking out”language from the captioned video.

In spite of the overall positive influence of captions on comprehension and vocabulary learning, a thorough study ofthe existing literature on captioned video revealed two main issues which prevent us from drawing overall conclusionson captioning benefits.

The first one concerns the potentially problematic relationship between participants’ proficiency level andcaptioning effectiveness (Danan, 2004). Research studies showed that the use of captioning has been consideredparticularly controversial for beginning L2 learners. Lambert and Holobow (1984) advocated the use of reversedsubtitling and suggested that beginning learners cannot achieve successful processing if the information is onlyconveyed in the L2 (as is the case for captioning). In a more recent study, Taylor (2005) compared the use of captionedvideo of students in their first year and students having three or four years of Spanish and observed that first-yearstudents scored better in the no captioning group than in the captioning group. Moreover, first- and third-year studentsperformed as well in the no captioning group. Third-year students outperformed the first-year students in the

1 Other benefits are, for instance, reduced listening anxiety of the learners (Vanderplank, 1988) and better oral production (Borras and Lafayette,

1994).

Page 4: Captioned video for L2 listening and vocabulary learning: A meta-analysis

723M. Montero Perez et al. / System 41 (2013) 720e739

captioning group. However, the studies carried out by Guillory (1998), Markham (1989), and Neuman and Koskinen(1992) did reveal beneficial effects of captioning for beginning students.

A second problem that occurs when analyzing existing research concerns the divergence of test measures used toanalyze the dependent variables i.e. listening comprehension and vocabulary learning. Extensive research on vo-cabulary acquisition has shown that complete word knowledge consists of a series of different components, rangingfrom word form recognition to proper use in context (Nation, 2001; Read, 2000). Because captioning effectiveness forvocabulary learning has been measured with different types of receptive and productive vocabulary tests, it is hard tocompare results across studies (Montero Perez and Desmet, 2012). A similar issue was raised by Liversidge (2000) inthe context of listening comprehension. He posited that captioning effectiveness may depend on the type of assess-ment used. In his study, he found that students in the captioning group only scored significantly higher on the storypartial cloze test and not on the multiple choice questions. He therefore argued that studies should include a series ofdifferent tests in order to avoid misperceptions.

2.3. Rationale for using a meta-analytical approach

In Section 2.2 we have shown that 1) captioning effectiveness has been studied extensively and 2) moderatingvariables, i.e. “various descriptive variables” (Lipsey and Wilson, 2001, p. 106) such as proficiency level and test typemay influence captioning effectiveness.

In order to determine the overall effectiveness of captioning and its relationship with the aforementioned studycharacteristics, we conducted a meta-analysis. We considered this methodological approach particularly appropriatefor two reasons. First, by combining the data from multiple studies, a meta-analytic approach allows to estimate theoverall effect of captioning more precisely and to investigate whether the effect varied over studies. Second, one of thestrengths of a meta-analysis lies in its potential to answer research questions regarding moderator effects in order toanalyze whether characteristics of studies correlated with the effects of the study.

The present meta-analysis was guided by the following three research questions:

RQ (1) What is the overall effectiveness of captioning for L2 listening comprehension and vocabularyacquisition?RQ (2) What is the relationship between the test type used and the effect of captioning on L2 listeningcomprehension and vocabulary acquisition?RQ (3) What is the relationship between proficiency level and the effect of captioning on L2 listeningcomprehension and vocabulary acquisition?

3. Method

Conducting a meta-analysis starts with a rigorous literature search in order to identify potentially relevant studies(Section 3.1). Next, all retrieved studies are filtered by using a set of inclusion and exclusion criteria (Section 3.2). Theretained studies are subsequently coded (Section 3.3) and prepared for the meta-analytic procedures (Section 3.4).

3.1. Literature search: study identification and retrieval

In order to identify studies that empirically investigated captioning effectiveness for listening comprehensionand/or vocabulary learning, we used four different procedures including manual and computer searches. First, weconducted a search in the Eric, Academic Search Elite, Web of Science, MLA, and Psycinfo databases. Second, weperformed a search on a series of journals on (computer-assisted) language learning: Alsic, Calico Journal, CALLjournal, Foreign Language Annals, Journal of Educational Technology Systems, Language Learning, LanguageLearning & Technology, the Modern Language Journal, ReCALL, System, and Studies in Second LanguageAcquisition. The journals were searched either through their search interface or the tables of content. Third, wesearched the Proquest database in order to find unpublished doctoral dissertations. Finally, we checked thereference sections of all related articles and comprehensive reviews in order to identify potentially relevantstudies.

Page 5: Captioned video for L2 listening and vocabulary learning: A meta-analysis

724 M. Montero Perez et al. / System 41 (2013) 720e739

All the database and journal searches were based on the same search strings: (listening comprehension orvocabulary)2 and (caption or subtitle); (listening comprehension or vocabulary) and (captioned or subtitled) video;(intralingual subtitle); and (bimodal input). Although we focused exclusively on the use of captioning, we usedboth subtitle and caption in our search strings since both concepts have not always been used in a consistentmanner.3

3.2. Inclusion and exclusion criteria

By using our four component search strategy, we identified over 150 studies on the use of captioning or subtitling.Studies were then reviewed in order to determine their appropriateness to answer our research questions. Only studiesthat complied with the following criteria were considered “similar enough” (Norris and Ortega, 2006, p.16) to be usedin the meta-analysis.

1) The study contained appropriate quantitative and statistical data for calculating effect sizes and correspondingstandard errors.

2) The study focused on the use of video in the context of second language learning and studied L2 on-screen text(captioning) as modification of the original input.

3) The dependent variables measured listening comprehension and/or vocabulary learning.4) The study compared a control condition (video without on-screen text) and a treatment condition (captioned

video), either with two independent groups or with one group measured under both experimental conditions.Results of other conditions (e.g. L1 subtitling) investigated by the same studies are not reported in this meta-analysis.

5) The study accurately described the test tasks in order to correctly code the test characteristics.

A (part of a) study was not included if at least one of the exclusion criteria was met:

1) Studies that were purely descriptive (e.g. Chung, 1996; Danan, 2004; Dıaz Cintas and Fernandez Cruz, 2008;Vanderplank, 2010), did not report the required statistical data (e.g. Baltova, 1999a; Bianchi and Ciabattoni,2008; Garza, 1991; Guichon and McLornan, 2008; Guillory, 1998; Price, 1983; Rowland, 2007; Stewart andPertusa, 2004) or did not include a control group (Zarei, 2011) are excluded from the analysis, as well as ar-ticles that reported incomplete descriptives4 (Markham, 1993) and test descriptions (e.g. Yuksel and Tanriverdi,2009) or focused exclusively on qualitative data (e.g. Vanderplank, 1988, 1990).

2) Studies that focused exclusively on audio materials (e.g. Bird and Williams, 2002; Diao et al., 2007; Lambertand Holobow, 1984) or focused on other types of input modification such as L1 on-screen text (e.g. Araujo,2008; Pavesi and Perego, 2008), reversed subtitles (e.g. d’Ydewalle and Van De Poel, 1999), or annotations(e.g. Jones and Plass, 2002) are not included in the analyses.

3) Studies that focused on the effectiveness of captioning for other audiences than second language learners such ashearing-impaired persons or students with disabilities (e.g. Cambra et al., 2010; Schilperoord et al., 2005) andprimary education children (e.g. d’Ydewalle and Van De Poel, 1999; Kuppens, 2010) are not taken into accountin this review.

4) Studies that measured any other language outcome such as speech production (e.g. Borras and Lafayette, 1994)are not taken into account in this analysis.

2 We used the same search procedures to identify studies on listening comprehension and/or vocabulary learning. Studies were then coded

accordingly (see also the coding categories listed in Appendix A).3 Captioning is generally used to refer to L2 videos with L2 on-screen text, subtitles are used for L2 video with L1 text. By using both concepts,

we could retrieve studies that other searches couldn’t find. However, this also meant that a lot of the retrieved studies were irrelevant in the context

of the present meta-analysis.4 When coding the study, we found that the procedures and reported statistics did not coincide. We consulted the author about this finding and

decided to exclude the data from the meta-analysis.

Page 6: Captioned video for L2 listening and vocabulary learning: A meta-analysis

Fig. 1. Overview of sources and retrieved studies based on study ID (see Appendix B).

725M. Montero Perez et al. / System 41 (2013) 720e739

The application of the criteria mentioned above resulted in the selection of 13 published journal articles and fiveunpublished doctoral dissertations, written between 1989 and 2011. Fig. 1 schematically represents the 18 retrievedstudies per database and/or journal. All the identification numbers are listed in Appendix B.

3.3. Coding of studies

3.3.1. Coding schemeWe established a coding scheme in order to classify and identify common features of the 18 retrieved studies. After

piloting our initial coding book, we established five main coding categories, distinguishing between publicationcharacteristics (author, year, publication type), learner characteristics (L1, proficiency level, level of instruction),research design features (aim, design, target language, sample size, pilot test), video materials (authenticity, content,viewings, length), and test features (proficiency determination, listening test, vocabulary test), as described inAppendix A.

3.3.2. Test type codingIn order to answer our second research question, we needed to carefully consider the test features of each study. In

order to do so, we developed a coding scheme that allowed us to classify studies in function of the test featuresaddressed by the study. We classified test types for listening comprehension on the one hand, and vocabulary learningon the other. The same coding procedures were used for pretests, immediate posttests, and delayed posttests.5

In coding listening comprehension, we identified two main approaches for assessing content comprehension (seeTable 1). We distinguish between tests that are purely receptive and are based on the claim that receptive tests reducethe interference of students’ writing skills (Buck, 2001), and productive tests in which learners are asked to producethe answers themselves. Tests were coded as receptive if learners had to complete written or aural multiple choicequestions (Chung, 1999; Hayati andMohmedi, 2011; Hernandez, 2004; Huang and Eskey, 1999e2000; Hwang, 2003;Liversidge, 2000; Markham, 1989; Markham et al., 2001; Markham and Peter, 2003; Park, 2004; Winke et al., 2010)or a True/False test (Hwang, 2003). Tests were coded as productive if learners had to complete a written recall protocol(Markham et al., 2001; Neuman and Koskinen, 1992; Taylor, 2005), open-ended questions (Baltova, 1999b;Hernandez, 2004), or a fill in the gaps test (Park, 2004).

When coding for type of vocabulary test, we first reviewed all studies in order to find out which tests had been used.Next, we recoded the test used in function of the vocabulary component measured by the test. In order to do so, wemade a first categorization based on Nation’s (2001, p. 347) nine “aspects of word knowledge for testing”. However,this categorization turned out to be too detailed to maintain in further statistical analyses, especially given the smallnumber of study samples. That is why we chose to recode the studies by distinguishing between recognition and recall(see Table 2).

All the coding categories listed in Tables 1 and 2 are included in Appendices A and B, which provide a completeoverview of all coding values and the descriptive statistics for each data set (Appendix A) and the correspondingcoding per study (Appendix B).

5 We only calculated effect sizes for the immediate posttest data since only 2 studies included delayed posttests (Baltova, 1999b; Neuman and

Koskinen, 1992). We listed the posttest criteria in Appendices A and B. Because we focused on the effect of captioning, results of pretests did not

provide answers to our research questions and were therefore considered irrelevant in the context of the present study.

Page 7: Captioned video for L2 listening and vocabulary learning: A meta-analysis

Table 1

Categorization and coding of listening comprehension tests.

Assessment type Tests used Example

Receptive listening

comprehension

- Written multiple choice questions

- Aural multiple choice questions

(stems presented aurally)

- True/false test

“What does Professor Crawford say to Jamal Wallace?- Welcomes him to his class

- Questions his level of work

- Compliments his test scores

- All of the above

- Both b and c” (written multiple choice questions, Hernandez, 2004, p. 174)

Productive listening

comprehension

- Recall protocol

- Open-ended questions

- Fill in the gaps

“Explain what you learned about keeping your body warm when it is very

cold þ 10 blank lines” (written retelling, Neuman and Koskinen, 1992, p. 98)

726 M. Montero Perez et al. / System 41 (2013) 720e739

3.4. Meta-analysis, effect size calculation, and homogeneity testing

We conducted quantitative meta-analyses of the 18 retained studies on captioning effectiveness. As a first step, weconverted the studies’ descriptive and test statistics to effect sizes, and aggregated these effect sizes. Separate meta-analyses were performed for listening comprehension (including data from 15 studies) and for vocabulary learning(including data from 10 studies). Note that 7 studies measured both listening comprehension and vocabulary learning.

The measure of effect size used in this study was Hedges’ g. This is a standardized mean difference, multiplied by acorrection factor for small sample bias. Hedges’ g was calculated using basic descriptive statistics (sample size, meanscores, and standard deviations of the control and treatment group) in the following equation (Lipsey and Wilson,2001, pp. 48e49):

Table 2

Classifi

Test

- Aura

recog

- Liste

mult

ques

- Writ

recog

- Sente

anom

- Word

mult

ques

- Aura

trans

- Writ

trans

- Cloz

g¼�1� 3

4ðnG1 þ nG2Þ � 9

��XG1 �XG2

sp

�ð1Þ

In Equation (1), g refers to a corrected standardized mean effect size, XG1 is used to refer to the group mean of thetreatment group (captioning condition),XG2 refers to the group mean of the control group, nG1 to the size of the

cation and coding of vocabulary tests.

Example of test/instruction Aspect of word knowledge Recoded

l word

nition

ning only

iple choice

tions

- “Now you will hear six words or phrases. Check all of

those that were in the video” (Sydorenko, 2010, p. 72)

- “Expandable throat [.] increase the volume of water”

a. plates b. pleas c. pleats d. plots (Markham, 1999, p. 324)

Receptive: recognize

spoken form

Vocabulary word

recognition

ten word

nition

- Distinguish target word from three distractors

(Neuman and Koskinen, 1992, p. 98):

“atparphic atmosteric atherostic atmospheric”

Receptive: recognize

written form

nce

aly test

Evaluate proper use of word in context

(Neuman and Koskinen, 1992, p. 98):

“It is a natural instinct for animals to search for

food. True e False?

The instinct has been in the house for a week.

True e False?”

Receptive: recall of

form and meaning

Vocabulary recall

meaning

iple choice

tions

“You’d all be pleased to know this year’s

writing competition has now been scheduled.

a. contest, b. sale, c. craft, d. class, e. essay”

(Hernandez, 2004, p. 178)

l word

lation

ten word

lation

“Translate the following word to English” (presented

written or aurally, Sydorenko, 2010, p. 72)

e test Complete a written summary of the video: “Aujourd‘hui

l’arctique est froid et ho... Le soleil ne brille pas

pendant les mois d’hiver.” (Baltova, 1999b, p. 200)

Productive recall of

form and meaning:

produce word form

Page 8: Captioned video for L2 listening and vocabulary learning: A meta-analysis

727M. Montero Perez et al. / System 41 (2013) 720e739

experimental group, nG2 the size of the control group, and sp to the square root of the pooled estimate of the withingroup variance, calculated as a weighted mean of the variances within both groups:

s2p ¼ðnG1 � 1Þs2G1 þ ðnG2 � 1Þs2G2

nG1 þ nG2 � 2ð2Þ

In addition, we estimated the standard errors for the obtained effect sizes, which give an indication of the precisionof the effect size estimates. Standard errors were estimated as follows:

SE gð Þ ¼ 1� 3

4 nG1 þ nG2ð Þ � 9

� � ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffinG1 þ nG2nG1nG2

þ g2

2 nG1 þ nG2ð Þ

sð3Þ

To interpret the magnitude of effect sizes of captioning on listening comprehension and vocabulary learning, weused Cohen’s (1988) rules of thumb for effect size characterization: values lower than or around 0.20 indicate smalleffects; values between 0.21 and 0.80 are interpreted as medium effects; values around 0.80 or higher should beunderstood as large effects.

In the meta-analyses, we weighted each observed effect size according to their precision, defined as the inverse ofthe variance. We used random-effects models, assuming that the variance of the observed effect sizes might be due notonly to sampling variance, but also to systematic variance. In a first step, we did not include any predictors, but ratherestimated the overall treatment effect. We performed a homogeneity test (Q-test) in order to find out if the variabilityof the effect sizes is larger than what can be attributed to sampling error. A statistically significant result from the Q-test indicates that the homogeneity assumption is violated and effect size variation is therefore due to other reasonsthan mere sampling variance. In such cases, one can try to explain these systematic differences between studies bysearching for “moderator variables” (Lipsey andWilson, 2001, p. 118), more specifically, characteristics of the studiesthat are related to the size of the observed effects.

A complicating factor is that for some studiesmore than one effect size can be calculated, because data are reported forsubgroups or formultiple dependent variables. Because it is likely that effect sizes from the same study aremore alike thaneffect sizes fromdifferent studies, effect sizes cannot be regarded as independent andmeta-analytic resultsmight beflawed.We solved this problem by using the mean effect size for each study in the meta-analysis (Lipsey and Wilson, 2001).

A phenomenon that might hamper the conclusions of a meta-analysis is reporting and publication bias: Results thatare not significant or not expected are less likely to be reported by the researcher, or accepted by scientific journals. Oneway to check for bias in the meta-analytic data set is by looking at the relationship between the study size and theobserved effect sizes. If it is reasonable to assume that the size of the study does not affect the size of the effect, a scatterplot with the size of the study (Y-axis) and the observed effect sizes (X-axis) will look like a funnel because the smallerthe study, the more observed effect sizes vary around the population value. If observed effect sizes in one direction areless likely to be reported or published, however there will be a gap in the funnel, resulting in a non-symmetric patternand a correlation between the size of the study and the size of the observed effects. To assess publication bias, wetherefore constructed funnel plots. We also compared the average effect found in published and unpublished studies.

Calculation of effect sizes and the analyses were done using the program Comprehensive Meta-Analysis (CMA)developed by Borenstein et al. (2005).

4. Results

This section describes the results of the two sets of meta-analyses that were conducted: one for listeningcomprehension (Section 4.1) and one for vocabulary learning (Section 4.2).

4.1. Listening comprehension

Of the 18 studies that met all inclusion criteria, we identified 15 studies that investigated listening comprehensionas a dependent variable. Of those studies, 10 were published journal articles, the remaining 5 were unpublished

Page 9: Captioned video for L2 listening and vocabulary learning: A meta-analysis

Table 3

Effectiveness of captioning for listening comprehension by subgroups.

Listening comprehension 95% confidence interval

Subgroup Value k Mean (g) SE p t2 Lower Upper

Overall 15 0.988 .199 <.001 0.50 0.598 1.377

Type of test Receptive test 8 1.436 .313 <.001 0.77 0.822 2.050

Productive test 4 0.547 .431 .205 0.43 �0.298 1.393

Proficiency level Beginning to low intermediate 3 0.657 .573 .251 0.54 �0.466 1.780

Intermediate 6 1.552 .409 <.001 1.03 0.751 2.354

High intermediate to advanced 1 0.711 .990 .473 0.00 �1.231 2.652

Note. t2 refers to the between-study variance in population effect sizes.

728 M. Montero Perez et al. / System 41 (2013) 720e739

doctoral dissertations. Participants involved in the studies were mainly Asian (7 studies) or English (5 studies). Theparticipants of 8 studies were university-level students. Several studies focused on multiple proficiency levels6: Eightstudies involved participants of a beginning level, 11 studies focused on intermediate learners, 4 studies involvedadvanced learners. We found that English was the target language in 10 studies, Spanish in 4 studies and French in onestudy. In order to assess listening comprehension, 8 studies reported the use of a receptive test, whereas 4 studies werebased on a productive one. Three studies measured comprehension using a combination of both types of tests. Themajority of the studies included a documentary as video material. Appendix A lists all coding variables and descriptivestatistics for each category are provided in the fourth column.

4.1.1. Overall effectivenessOur first research question focused on the overall effectiveness of captioning for improving L2 learners’ listening

comprehension. In order to analyze the effect on listening comprehension, we calculated the effect sizes of thecorresponding 15 study samples. Table 3 displays a large average effect size (g ¼ 0.99), demonstrating the overallsuperiority of the captioning condition over the control group. The confidence interval is rather narrow and does notinclude zero, which illustrates that the overall effect is statistically significant (z ¼ 4.97, p < .001).

As can be seen from Appendix A, the study of Hayati and Mohmedi (2011) resulted in a very large effect size(g ¼ 5.38). That is why we added a sensitivity analysis (Borenstein et al., 2009, p. 369): We recalculated the averageeffect size without that study. Results of the sensitivity analysis show that captioning still differs substantially from thecontrol group. According to Cohen’s (1988) criteria, the new effect size shows a medium effect (g ¼ 0.76) and isstatistically significant (p < .001). For all remaining analyses, we provide the results including the effect size of thestudy of Hayati and Mohmedi (2011) in the text, and give the results of the sensitivity analysis, excluding the outlyingvalue, in footnotes.

We subsequently performed a homogeneity test (Q-test). The Q-test revealed that effect sizes show significantlymore variation than can be attributed to mere sampling variance: Q(14) ¼ 110.19, p < .001.7 The I2 statistic is 87.3,which indicates that 87.3% of the variance in the observed effect sizes for the listening studies is due to real differencesrather than to chance. Both the significance and the relatively large size of the heterogeneity warrant an investigationof potential moderating effects of study characteristics. We examined whether two study characteristics could beconsidered moderators of the effect sizes: test type and proficiency level (see Sections 4.1.2 and 4.1.3).

In order to check for publication bias, we constructed a funnel plot. To get a clear view on the distribution of theeffect sizes, we excluded in this plot the outlying observed effect size of the study of Hayati and Mohmedi (2011). Thefunnel plot in Fig. 2 shows that the graph is more or less symmetrically outlined around the mean effect size, whichsuggests that there is no evidence for publication bias. We also compared the effect size of published articles (g¼ 0.99,

6 In order to answer research question three concerning the relationship between proficiency level and captioning effectiveness, the meta-

analysis could only include studies that focused on a single proficiency (10 studies).7 The Q-test computed for the listening studies without (Hayati and Mohmedi, 2011) revealed a similar finding (Q(13) ¼ 41.50, p < .001, SE ¼

.09). The I2 indicates that 68% of the variance is due to real differences.

Page 10: Captioned video for L2 listening and vocabulary learning: A meta-analysis

-2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0

0.0

0.1

0.2

0.3

0.4

0.5

Stan

dard

E

rro

r

Hedges's g

Funnel Plot of Standard Error by Hedges's g

Fig. 2. Funnel plot of listening comprehension studies.

729M. Montero Perez et al. / System 41 (2013) 720e739

p< .001) and unpublished dissertations (g¼ 0.99, p¼ .005) and found that the difference between both types was notsignificant (Q(1) ¼ 2.06, p ¼ .151).

4.1.2. Test typeAs can be seen in Table 3, listening tests were coded as being either receptive or productive.8 We found a large

effect (g¼ 1.44) for receptive tests and the effect was statistically significant (p< .001). We found a medium effect forproductive tests (g ¼ 0.55), but this effect was not statistically significant and the wide confidence interval shows thatmore research is needed in order to obtain statistical significance. A moderator Q-test reveals that the differencebetween both test types is significant9 (Q(1)¼ 10.02, p¼ .002), which shows that captioning had a significantly largereffect when comprehension was measured with a receptive comprehension test. The within group differencesremained however significant (Q(10) ¼ 91.53, p < .001).

4.1.3. Proficiency levelThe third research question sought to determine the relationship between captioning effectiveness and proficiency

level. As shown in Table 3, we found a medium effect size of captioned video for beginning learners (g ¼ 0.66,p ¼ .251) and a large effect was found for intermediate learners (g ¼ 1.55, p < .001). We found a medium effect foradvanced learners (g ¼ 0.71, p ¼ .473). Because effect size calculation for advanced learners was based on a singlestudy, it should be interpreted cautiously. The results of theQ-test indicated that the difference between the proficiencylevels was not significant (Q(2)¼ 2.75, p¼ .252).10 The within group differences remained significant (Q(7)¼ 98.44,p < .001).

4.2. Vocabulary learning

We identified 10 studies that focused on vocabulary learning and complied with our inclusion criteria. Six out of10 studies were journal articles and 4 were unpublished doctoral dissertations. The participants were mainly Asian (4studies) or English (4 studies). Six studies targeted English as L2 whereas 4 other studies targeted Spanish, French (2studies), and Russian respectively. As was the case with listening comprehension, several studies included multipleproficiencies.11 The majority of the participants were university-level students and had a beginning (6 studies) or

8 We only included effect sizes for studies using a single test type.9 After exclusion of Hayati and Mohmedi (2011), the difference between the test types was preserved (Q(1) ¼ 5.53, p ¼ .019).10 A similar pattern was found after exclusion of Hayati and Mohmedi (2011): We observed a large effect for intermediate learners (g ¼ 0.93,

p ¼ .001). There were no differences between the proficiency levels (Q(2) ¼ 0.61, p ¼ .736).11 In order to answer research question three concerning the relationship between proficiency level and captioning effectiveness, the meta-analysis

only included studies that were based on a single proficiency (7 studies).

Page 11: Captioned video for L2 listening and vocabulary learning: A meta-analysis

Table 4

Effectiveness of captioning for vocabulary learning by subgroups.

Vocabulary learning 95% confidence interval

Subgroup Value k Mean (g) SE p t2 Lower Upper

Overall 10 0.866 .147 <.001 0.11 0.578 1.154

Type of test Recognition 2 1.061 .314 .001 0.00 0.446 1.676

Recall 6 0.956 .209 <.001 0.18 0.547 1.366

Proficiency level Beginning to low intermediate 3 1.179 .324 <.001 0.06 0.543 1.814

Intermediate 3 0.821 .269 .002 0.22 0.294 1.347

High intermediate to advanced 1 0.870 .457 .057 0.00 �0.025 1.765

Note. t2 refers to the between-study variance in population effect sizes.

730 M. Montero Perez et al. / System 41 (2013) 720e739

intermediate (6 studies) proficiency level. Two studies involved advanced L2 learners. The dependent variable,vocabulary learning, was measured with a recall test (6 studies), a recognition test (2 studies) or a combination ofboth (2 studies). With respect to the video material used, we found that 5 studies were based on authentic materialsuch as documentaries or movies. Of the remaining studies, 3 videos were based on textbook material (i.e. the videosused in those studies were part of learners’ textbook), 2 other studies reported the use of educational TV programs.All the coding categories and the corresponding descriptive statistics are listed in Appendix A.

4.2.1. Overall effectivenessTable 4 displays the mean effect size of captioning on the immediate vocabulary posttest. The effect size is large

(g ¼ 0.87) and reveals that learners who were exposed to captioned video significantly outperformed learners in thecontrol group (p< .001). We subsequently performed a homogeneity test and found that the homogeneity assumptionwas violated (Q(9) ¼ 19.47, p ¼ .022). The I2 statistic indicates that 53.8% of the variance has to be attributed to realdifferences. In order to account for variation in effect size, we examined the influence of test type and proficiency level(see Sections 4.2.2 and 4.2.3).

The approximately symmetrical distribution of studies’ observed effect sizes around the mean effect size in thefunnel plot (see Fig. 3) suggests that no publication bias is at work. In addition, we compared the effect size of journalarticles (g¼ 0.83, p< .001) and unpublished dissertations (g¼ 0.94, p< .001) and found that the differences betweenboth types of publications were not significant (Q(1) ¼ 0.93, p ¼ .336).

4.2.2. Type of testAs displayed in Table 4, we observed a large effect for recognition tests (g ¼ 1.06) which was also statistically

significant (p ¼ .001). Results were similar for recall tests and demonstrated a large effect of captioning (g ¼ 0.96),

-2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Sta

nd

ard

E

rro

r

Hedges's g

Funnel Plot of Standard Error by Hedges's g

Fig. 3. Funnel plot of vocabulary studies.

Page 12: Captioned video for L2 listening and vocabulary learning: A meta-analysis

731M. Montero Perez et al. / System 41 (2013) 720e739

which was again statistically significant (p < .001). The results of the Q-test showed no differences between the testtypes (Q(1) ¼ 0.69, p ¼ .407); the within groups differences were still significant (Q(6) ¼ 13.6, p ¼ .047).

4.2.3. Proficiency levelWe found a large effect for all proficiency levels (see Table 4). However, only one study was included in the high

intermediate to advanced level category, explaining the large confidence interval. The differences between the pro-ficiencies were not statistically significant: Q(2) ¼ 3.26, p ¼ .196. The differences within the three levels remainedsignificant: Q(4) ¼ 9.68, p ¼ .046.

5. Discussion

The overall objective of this meta-analysis was twofold: First, we tried to determine the overall effectiveness ofcaptioning for two components of language learning, namely listening comprehension and vocabulary learning.Second, we wanted to identify potentially moderating relationships between captioning effectiveness and two studycharacteristics: test type and proficiency level. We did so by conducting a meta-analysis for each study focus: listeningcomprehension and vocabulary learning. Results for each of the three research questions are discussed below.

5.1. Research question 1

Our first research question concerned the overall effect of captioning on two language outcomes: listeningcomprehension and vocabulary learning.

For listening comprehension, we found a significantly large effect of captioning (g ¼ 0.99). The majority of theretained studies revealed a medium or large effect size. Results clearly indicate that learners in the captioning groupsignificantly outperform the control group on listening comprehension posttests. Our findings thus empirically supportearlier research findings that have found overall positive effects of captioned video (e.g. Guillory, 1998; Markham,1989, 2001; Neuman and Koskinen, 1992). Results of this meta-analysis suggest that captioning can be considereda powerful tool for reinforcing learners’ comprehension of video materials. First of all, the presence of captioning mayensure “conscious monitoring of the speech” (Vanderplank, 1988, p. 277), and “bridge the often sizable gap betweenthe development of skills in reading comprehension and listening comprehension” (Garza, 1991, p. 246). By doing so,it may deal with word recognition and speech stream chunking problems (Graham, 2006) and reduce listeners’decoding effort, which would otherwise produce comprehension breakdowns related to the “real-time nature” of theaural input (Buck, 2001, p. 6). The use of captioning can also be considered a means to encourage sound-scriptautomatization and therefore deals with typical problems of the L2 listener (Goh, 2000).

The overall mean effect size for vocabulary learning also revealed a large effect of captioning (g ¼ 0.87) that wasstatistically significant, and empirically confirms the claim that bimodal input fosters vocabulary learning. For all thestudies included in the analysis, we found medium to large effect sizes. Why does captioning foster more vocabularylearning than single modality input? First of all, the presence of captioning may contribute to “a conscious focusing onthe form (especially on correct form), particularly when new or striking expressions are used” (Vanderplank, 1988, p.276). Second, captioning may encourage attention, which is crucial to “taking out” language (Vanderplank, 1990).Third, captions may also support learners in constructing an initial form-meaning link (Winke et al., 2010), whichrefers to the construction of the link between a form, a meaning, and the relationship between both (Van Patten et al.,2004, p. 5). Captioning also seems to help learners to better “isolate” words (Winke et al., 2010, p. 79) and reinforcecontextual clues for inferring word meaning.

From a theoretical point of view, the findings of this meta-analysis confirm Paivio’s (1986) dual-coding theory,which stated that the use of bimodal encoded input leads to better recall of content and vocabulary. On the other hand,the overall effectiveness of captioning for both language outcomes measures provides additional evidence forrejecting the redundancy principle (Sweller, 2005) in the context of L2 learning.

Yet, given the fact that the homogeneity assumption was violated for both language outcomes examined (seeResults section), we also analyzed the presence of potentially moderating variables by doing separate analyses of testtype and proficiency level for both data sets.

Page 13: Captioned video for L2 listening and vocabulary learning: A meta-analysis

732 M. Montero Perez et al. / System 41 (2013) 720e739

5.2. Research question 2

Our second research question asked whether the type of test used to measure either listening comprehension orvocabulary learning moderated the observed effect sizes.

For listening comprehension, results showed that test type moderated the effect sizes. We found the largest andsignificant effect of captions when comprehension was measured by means of a receptive test (g ¼ 1.44). For pro-ductive tests, by contrast, we found a medium, yet non-significant effect (g ¼ 0.55). The findings of the current studycontradict Liversidge’s (2000) findings. In his study, he found evidence for the opposite pattern and claimed thatcaptioning is more effective when followed by a productive task. A possible explanation for our findings may be thatthe test used (e.g. recall protocol) interfered with other language skills (Buck, 2001), e.g. writing skills, thus increasingthe complexity of the task. It is therefore possible that learners understood the video content, yet were unable to reflectcomprehension in their answers. By interfering with other skills, the type of test might reduce the beneficial effects ofcaptioning. Receptive tests do not require learners to produce output which might explain the higher and significanteffect size.

In coding the studies’ immediate vocabulary posttest, we distinguished between recognition and recall tests,based on Nation’s (2001, p. 347) nine components of word knowledge. We found a large and significant effect ofcaptioning on word recognition (g ¼ 1.06). Yet, this finding should be interpreted cautiously as it is only based on2 studies, which is reflected in the wide confidence intervals. On the other hand, we also found a large effect ofcaptioning on recall tests (g ¼ 0.96). The confidence interval for this test type is smaller and indicates a significanteffect. According to this meta-analysis, captioning is equally effective for recognition and recall tests. Yet, weshould note two important limitations. First, results of this analysis are based on imbalanced groups, which meansthat more study samples should be associated with the recognition group in order to verify this finding. Second, alarger number of studies would have allowed us to adopt a more fine-grained coding of the studies’ vocabularytests, which could have led to more diversification in effect sizes assigned to studies in function of test types.

In sum, we have only found significant proof that test type affects captioning effectiveness for listeningcomprehension.

5.3. Research question 3

Our third research question examined whether proficiency level could account for variation in effect sizes.For listening comprehension studies, we found a medium effect size for beginning learners (g ¼ 0.66), which

was, however, not significant. For intermediate learners (g ¼ 1.55), by contrast, we found a large effect size thatsignificantly differed from zero. Results of this analysis show that captioning has been specifically studied as ahelp option for learners whose proficiency level is considered intermediate, yet insufficient for understanding(authentic) video materials. Results suggest that authentic video can indeed be “unlocked” for those intermediate-level learners by using captions (Vanderplank, 1988, p. 278). Results of the advanced level yielded a mediumeffect (g ¼ 0.71) but should be interpreted with care given the fact that only one study involved advanced par-ticipants. We found no difference in the effect of captioning between the proficiency levels for listeningcomprehension. Results of this meta-analysis thus suggest that the relationship between captioning effectivenessand proficiency level may be less problematic than outlined in primary research studies (cf. Background Researchsection). These findings also suggest that other factors might have interfered with the rather straightforwardrelationship between captioning effectiveness and proficiency level. One possible explanation lies in the difficultylevel of the video materials used (Vanderplank, 1988; Winke et al., 2010). In order for captioning to be effective,video materials should be adapted to learners’ actual proficiency level. If not, redundant on-screen text mightindeed result in poorer learning (cf. redundancy principle) and increase the decoding load as captions “cannotcompensate for an excessively wide gap” (Bianchi and Ciabattoni, 2008, p. 70) between the proficiency level andthe level of the video.

Another explanation can be found on the level of learners’ perceptions of captions and their beliefs about theusefulness of on-screen text. Pujola (2002) studied the use of different types of support options in a multimedialistening environment and found that learners’ perceptions of captions varied in function of their proficiency level:Higher-level learners perceived captions “as a backup to their listening activity” (p. 254) whereas lower-level learnersconsidered captions essential for content comprehension. Yet, although learners’ perceptions of captions (Pujola,

Page 14: Captioned video for L2 listening and vocabulary learning: A meta-analysis

733M. Montero Perez et al. / System 41 (2013) 720e739

2002) may vary in function of their proficiency level and consequently influence their use, findings of our meta-analysis suggest that captions still aid content understanding.

For vocabulary learning, the meta-analysis yielded large effect sizes, both significant, for beginning (g¼ 1.18) andintermediate learners (g ¼ 0.82). Different explanations might be found why captioning has the largest effect size forbeginning learners. First of all, beginning learners need extra help on the decoding level. Because captioning visu-alizes word boundaries (Bird and Williams, 2002), learners need to spend less time decoding the speech and cantherefore pay more attention to the language used in the video (Vanderplank, 1990). Moreover, studies have shown thatthe “rich get richer principle” (Neuman and Koskinen, 1992, p. 164) affects vocabulary learning from captioned video(Baltova, 1999b; Neuman and Koskinen, 1992) and that “those who are more fluent in L2 learned more vocabulary”(Neuman and Koskinen, 1992, p. 164). Because beginning learners have not yet developed language fluency,captioning seems crucial in that it might present a means to outweigh deficient L2 knowledge to some degree. Bydoing so, captions may also stimulate language learning. Results for advanced learners (g¼ 0.87) are based on a singlestudy, which is why caution is needed. More primary research results are required if we want to obtain more preciseresults concerning this question.

In sum, these analyses give tentative indications that proficiency level is not a moderator of captioning effec-tiveness for listening comprehension and vocabulary learning. Bimodal input may augment comprehension andvocabulary learning gains irrespective of learners’ proficiency, as long as the video materials match their actuallevel.

6. Conclusions, limitations, and future work

This study was inspired by the need to summarize research that focused on the effectiveness of captioning forlistening comprehension and vocabulary learning. To the best of our knowledge, the present meta-analysis is the firstto empirically synthesize primary research on the benefits of captioning, 30 years after the first large-scale studydemonstrated the learning potential of captioned video (Price, 1983). The overall results of our meta-analysis revealeda large superiority of captioning in that captioning groups significantly outperformed the control group on bothlistening and vocabulary posttests. Results thus support the claim that captioning helps learners to improvecomprehension and fosters vocabulary learning. In order to account for variation in the effect sizes, we also inves-tigated whether two study characteristics moderated the effect sizes. Only the type of listening test was found to be amoderator of effect sizes.

We need to recognize a number of limitations which prevent us from drawing definitive conclusions from the datapresented in this study. First of all, this meta-analysis is based on a limited number of primary research studies. Moreresearch is necessary in order to draw more robust conclusions and identify clearer patterns concerning the effec-tiveness of captioning and the role of potential moderators. Second, moderating effects of other variables need to beexamined, such as the possible effect of contextual factors, i.e. learners’ native language and the target language of thestudy. Because precise information on learners’ native language was lacking in some studies, or native languages weretoo diverse in other studies, we could not obtain meaningful data concerning that issue at this moment. An analysis ontarget language would have been based on largely imbalanced groups. Third, although we performed an extensiveliterature search, we might have missed studies that also complied with our criteria, for instance because we onlyincluded published data and unpublished dissertations that were made available through Proquest. Finally, we shouldalso note that shortcomings of the primary research studies involved are inevitably shortcomings of our meta-analysis.

In spite of the limitations of our study, we hope that this meta-analysis will stimulate further research oncaptioning effectiveness and address some of the issues raised in this review. Further research could focus on therole of the variables discussed in this review, i.e. test type and proficiency level. As shown in this meta-analysis,few studies focused on the effectiveness of captioning for advanced learners. More research is also needed on theeffectiveness of captioning for vocabulary learning and should focus on different types of vocabulary knowledge.Future studies could also try to include delayed posttests or conduct longitudinal studies in order to consider thelong-term effects of captioning on listening development and vocabulary retention (Robin, 2007). The moreprimary research studies, the better a future meta-analysis will deal with the limitations of the present one.

It is also our hope that this meta-analysis will shed more light on the pedagogical potential of captioning, provideaccumulative statistical data in support of earlier findings of comprehensive reviews (Danan, 2004; Vanderplank,2010), and stimulate further research on this topic.

Page 15: Captioned video for L2 listening and vocabulary learning: A meta-analysis

734 M. Montero Perez et al. / System 41 (2013) 720e739

Appendix A. Study coding categories and descriptive statistics

Variable Value Definition Frequency

Listening Vocabulary

Study author

and year

Name

Year

Report name of author(s)

and year of publication

k ¼ 15 k ¼ 10

Publication

type

1 ¼ journal article

2 ¼ unpublished PhD

Variable describing whether

the study was a published

journal article or an

unpublished PhD

k ¼ 15

Journal article k ¼ 10

Unpublished PhD k ¼ 5

k ¼ 10

Journal article k ¼ 6

Unpublished PhD k ¼ 4

Study ID 0001e0018 Unique identification number,

assigned to each study

included in the meta-analysis

(see Appendix B)

k ¼ 15

0001; 0003; 0004; 0005;

0006; 0008; 0009; 0010;

0011; 0012; 0013; 0014;

0015; 0016; 0018

k ¼ 10

0002; 0003; 0004; 0006;

0007; 0008; 0011; 0013;

0016; 0017

Native

languagea1 ¼ mainly Asian

2 ¼ mainly English

3 ¼ mainly Spanish

4 ¼ mainly Persian

5 ¼ not reported

Variable representing the

native language of the

majority of the involved

participants

k ¼ 15

Mainly Asian k ¼ 7

Mainly English k ¼ 5

Mainly Spanish k ¼ 1

Mainly Persian k ¼ 1

Not reported k ¼ 1

k ¼ 10

Mainly Asian k ¼ 4

Mainly English k ¼ 4

Mainly Spanish k ¼ 1

Mainly Persian k ¼ 0

Not reported k ¼ 1

L2 proficiency

level

1 ¼ beginning to low

intermediate

2 ¼ intermediate

3 ¼ high intermediate

to advanced

Variable representing the

target language proficiency

level of the participants

k ¼ 15

Beginning to low

intermediate k ¼ 8

Intermediate k ¼ 11

High intermediate to

advanced k ¼ 4

*Several studies included

multiple proficiencies

k ¼ 10

Beginning to low

intermediate k ¼ 6

Intermediate k ¼ 6

High intermediate

to advanced k ¼ 2

*several studies included

multiple proficiencies

Instructional

level

1 ¼ university level

2 ¼ higher education

3 ¼ secondary education

4 ¼ not reported

Variable representing the

instructional level of the

participants

k ¼ 15

University level k ¼ 8

Higher education k ¼ 4

Secondary education k ¼ 2

Not reported k ¼ 1

k ¼ 10

University level k ¼ 5

Higher education k ¼ 2

Secondary education k ¼ 2

Not reported k ¼ 1

Study aim 1 ¼ listening

comprehension

2 ¼ vocabulary

learning

3 ¼ listening

comprehension &

vocabulary learning

Variable representing whether

the study focused on listening

comprehension only,

vocabulary learning only, or

a combination of both

k ¼ 15

Listening comprehension

k ¼ 8

Listening and vocabulary

k ¼ 7

k ¼ 10

Vocabulary learning k ¼ 3

Listening and vocabulary

k ¼ 7

Design 1 ¼ captioning vs.

control group

2 ¼ captioning vs.

control group vs.

other condition

Variable representing whether

the study included only a

captioning and a control

group, or other conditions

representing other types

of on-screen text

k ¼ 15

Captioning vs. control k ¼ 4

Captioning vs. control vs.

other k ¼ 11

k ¼ 10

Captioning vs. control k ¼ 3

Captioning vs. control

vs. other k ¼ 7

L2 1 ¼ English

2 ¼ Spanish

3 ¼ French

4 ¼ Russian

Variable representing the

target language addressed

by the study sample

k ¼ 15

English k ¼ 10

Spanish k ¼ 4

French k ¼ 1

Russian k ¼ 0

k ¼ 10

English k ¼ 6

Spanish k ¼ 1

French k ¼ 2

Russian k ¼ 1

Sample size Continuous Continuous variable

describing the mean sample

size in the control and

treatment group

k ¼ 15

Control group M ¼ 54,

SD ¼ 39

Range ¼ 8e170Treatment group M ¼ 55,

k ¼ 10

Control group M ¼ 30,

SD ¼ 21

Range ¼ 7e69Treatment group M ¼ 32,

(continued on next page)

Page 16: Captioned video for L2 listening and vocabulary learning: A meta-analysis

(continued )

Variable Value Definition Frequency

Listening Vocabulary

SD ¼ 40

Range ¼ 9e170SD ¼ 24

Range ¼ 5e79

Pilot test 1 ¼ yes

2 ¼ not reported

Variable representing whether

test and video materials

were pilot-tested.

k ¼ 15

Yes k ¼ 9

Not reported k ¼ 6

k ¼ 10

Yes k ¼ 6

Not reported k ¼ 4

Authenticity

of video

documentb

1 ¼ authentic

2 ¼ semi-authentic

3 ¼ not authentic

4 ¼ not reported

Variable describing the

authenticity of the video

documents used

k ¼ 15

Authentic k ¼ 7

Semi-authentic k ¼ 2

Not authentic k ¼ 4

Not reported k ¼ 2

k ¼ 10

Authentic k ¼ 5

Semi-authentic k ¼ 2

Not authentic k ¼ 3

Not reported k ¼ 0

Type of video

content/genre

1 ¼ documentary

2 ¼ educational TV program

3 ¼ textbook video

4 ¼ movie

5 ¼ comedy

Variable representing the

type of video content

k ¼ 15

Documentary k ¼ 7

Educational TV program k ¼ 2

Textbook video k ¼ 4

Movie ¼ 2

Comedy ¼ 0

k ¼ 10

Documentary k ¼ 2

Educational TV

program k ¼ 2

Textbook video k ¼ 3

Movie ¼ 2

Comedy ¼ 1

Number of

viewings

1 ¼ 1 viewing

2 ¼ 2 viewings

3 ¼ 3 viewings

4 ¼ not reported

Variable representing the

number of viewings of

each video clip included

in the study

k ¼ 15

1 viewing k ¼ 6

2 viewings k ¼ 6

3 viewings k ¼ 1

Not reported k ¼ 2

k ¼ 10

1 viewing k ¼ 1

2 viewings k ¼ 5

3 viewings k ¼ 2

Not reported k ¼ 2

Video length Continuous Variable representing the

approximate length of one

viewing of the included

video clip(s)

k ¼ 15

M ¼ 16.05, SD ¼ 13.3

Range ¼ 7e58

k ¼ 10

M ¼ 17.37, SD ¼ 15.7

Range ¼ 5e58

Measure for

determining

proficiency

1 ¼ standardized test

2 ¼ self-designed test

3 ¼ institutional status

4 ¼ researcher’s judgment

5 ¼ combined measurement

Variable describing the

type of measurement

employed to determine

learners’ proficiency level

k ¼ 15

Standardized test k ¼ 7

Self-designed test k ¼ 1

Institutional status k ¼ 5

Researcher’s judgment k ¼ 1

Combined measurement k ¼ 1

k ¼ 10

Standardized test k ¼ 4

Self-designed test k ¼ 0

Institutional status k ¼ 5

Researcher’s judgment k ¼ 1

Combined measurement

k ¼ 0

Type of

immediate

listening test

1 ¼ receptive

2 ¼ productive

3 ¼ combined

Variable representing whether

listening comprehension was

assessed using a receptive

(e.g. multiple choice questions),

a productive (e.g. recall protocol)

test, or a combined measurement

(i.e. when posttest scores do not

clearly distinguish between

both tests)

k ¼ 15

Receptive k ¼ 8

Productive k ¼ 4

Combined k ¼ 3

Type of

immediate

vocabulary

posttest test

1 ¼ recognition

2 ¼ recall

3 ¼ combined

Variable representing whether

vocabulary learning was assessed

with a recognition test, a recall

test, or a combined measurement

(i.e. when posttest scores do not

distinguish between both tests)

k ¼ 10

Recognition k ¼ 2

Recall k ¼ 6

Combined k ¼ 2

Note.a The languages reported refer to the language spoken by the majority of the participants involved in the study. In several articles, participants had

more than 15 L1 backgrounds, which is why we only included the main L1 background per study.b The authenticity of the material was determined as follows. Materials that were made for native speakers were coded as authentic (documentary,

movie, and comedy). Although educational TV programs target native speakers, the language used is simplified which is why those programs were

coded as semi-authentic. Textbook materials, i.e. videos that were specifically made for L2 learners and were part of their textbook, were coded as

not authentic.

735M. Montero Perez et al. / System 41 (2013) 720e739

Page 17: Captioned video for L2 listening and vocabulary learning: A meta-analysis

Appendix B. Overview of study characteristics and effect sizes for all study samples

Publication characteristics Learner characteristics Research design features Video materials Test features Effect size statistics

Study Publication

type

(study ID)

L1 Prof Instr

level

Aim Des L2 N con

group

N treat

group

Pilot

test

Aut

video

Video

con

N� view Length Prof

det

Test

lc

Test

voc

g lc g voc

Markham (1989) 1 (0001) 1 1e3 1 1 1 1 76 76 1 2 2 1 70 2 1 0.541

Danan (1992) 1 (0002) 2 1 1 2 2 3 7 5 1 3 3 3 50 3 2 1.038

Neuman and Koskinen

(1992)

1 (0003) 1 1e3 3 3 2 1 37 32 0 2 2 4 �580 1 2 3 0.195 0.335

Baltova (1999b) 2 (0004) 2 1 3 3 2 3 30 34 1 1 1 3 703000 3 2 2 1.497(***) 1.538(***)

Chung (1999) 1 (0005) 1 1 2 1 2 1 170 170 0 3 3 2 �120 3 1 0.682(**)

Huang and Eskey

(1999e2000)

1 (0006) 5 2 4 3 1 1 15 15 0 3 3 2 140 1 1 2 1.293(***) 1.007(**)

Markham (1999) 1 (0007) 1 3 1 2 1 1 57 61 1 2 2 1 250 1 1 0.870(***)

Liversidge (2000) 2 (0008) 1 1, 2 1 3 2 1 69 79 1 1 4 2 220 4 1 2 0.773(***) 0.732(**)

Markham (2001) 1 (0009) 1 3 2 1 2 1 54 54 0 1 1 1 1903600 1 2 0.711(*)

Markham et al. (2001) 1 (0010) 2 2 1 1 2 2 68 43 1 4 1 1 70 1 3 0.548(**)

Hwang (2003) 2 (0011) 1 2 2 3 1 1 40 40 1 3 3 4 150 1 1 1 1.646(***) 1.214(***)

Markham and Peter

(2003)

1 (0012) 2 2 1 1 2 2 63 85 1 4 1 1 70 1 1 0.967(***)

Hernandez (2004) 2 (0013) 3 2 2 3 2 1 30 32 1 1 4 2 800500 3 3 2 0.277 0.283

Park (2004) 2 (0014) 1 1e3 1 1 2 1 89 89 0 1 1 2 �120 5 3 0.767(**)

Taylor (2005) 1 (0015) 2 1 1 1 1 2 36 35 1 3 3 1 100 3 2 �0.241

Winke et al. (2010) 1 (0016) 2 1, 2 1 3 2 2 8 9 0 1 1 2 �120 3 1 2 1.042(*) 1.366(**)

Sydorenko (2010) 1 (0017) 2 1 1 2 2 4 9 8 0 1 5 2 �703000 3 3 0.672

Hayati and Mohmedi

(2011)

1 (0018) 4 2 1 1 2 1 30 30 1 1 1 1 300 1 1 5.375(***)

Notes.aThe corresponding coding sheet, including the coding scheme used in this table, can be found in Appendix A. Aim ¼ study focus according to research questions; N con group ¼ sample size

control group; N treat group ¼ sample size treatment group; L1 ¼ native language; Prof ¼ proficiency level; Instr level ¼ instructional level; Des ¼ design; L2 ¼ target language; Aut

video¼ authenticity of video materials; Video con¼ video content; N� view¼ number of viewings; Test lc¼ test listening comprehension; Test voc¼ test vocabulary; g lc¼Hedges’ g for studies

on listening comprehension; g voc ¼ Hedges’ g for studies on vocabulary learning.bThe � in the Length column indicates that the materials section of the corresponding text only included estimates of the video length rather than the precise duration of the video clip(s).c(*) significant at the .05 level; (**) significant at the .01 level; (***) significant at the .001 level.

736

M.Montero

Perez

etal./System

41(2013)720e

739

Page 18: Captioned video for L2 listening and vocabulary learning: A meta-analysis

737M. Montero Perez et al. / System 41 (2013) 720e739

References*

Araujo, S., 2008. The educational use of subtitled films in EFL teaching. In: Dıaz Cintas, J. (Ed.), The Didactics of Audiovisual Translation. John

Benjamins Publishing Company, Amsterdam/Philadelphia, pp. 227e238.

Baltova, I., 1994. The impact of video on the comprehension skills of core French students. The Canadian Modern Language Review 50 (3),

507e532.Baltova, I., 1999a. Multisensory language teaching in a multidimensional curriculum: the use of authentic bimodal video in core French. The

Canadian Modern Language Review 56 (1), 31e48.

*Baltova, I., 1999b. The Effect of Subtitled and Staged Video Input on the Learning and Retention of Content and Vocabulary in a Second

Language (Unpublished doctoral dissertation). Ontario Institute for Studies in Education of the University of Toronto, Toronto.

Bianchi, F., Ciabattoni, T., 2008. Captions and subtitles in EFL learning: an investigative study in a comprehensive computer environment. In:

Baldry, A., Pavesi, M., Taylor Torsello, C., Taylor, C. (Eds.), From Didactas to Ecolingua: an Ongoing Research Project on Translation and

Corpus Linguistics. Edizioni Universita di Trieste, Trieste, pp. 69e90.Bird, S.A., Williams, J.N., 2002. The effect of bimodal input on implicit and explicit memory: an investigation into the benefits of within-language

subtitling. Applied Psycholinguistics 23 (4), 509e533.

Borenstein, M., Hedges, L.V., Higgins, J.P.T., Rothstein, H.R., 2005. Comprehensive Meta-analysis, Version 2. Biostat, Englewood, NJ.

Borenstein, M., Hedges, L.V., Higgins, J.P.T., Rothstein, H.R., 2009. Introduction to Meta-analysis. Wiley, Chichester.

Borras, I., Lafayette, R.C., 1994. Effects of multimedia courseware subtitling on the speaking performance of college students of French. The

Modern Language Journal 78, 61e75.

Brett, P., 1995. Multimedia for listening comprehension: the design of a multimedia-based resource for developing listening skills. System 23 (1),

77e85.Brett, P., 1997. A comparative study of the effects of the use of multimedia on listening comprehension. System 25 (1), 39e53.

Brett, P., 1998. Using multimedia: a descriptive investigation of incidental language learning. Computer Assisted Language Learning 11 (2),

179e200.Buck, G., 2001. Assessing Listening. Cambridge University Press, Cambridge.

Cambra, C., Leal, A., Silvestre, N., 2010. How deaf and hearing adolescents comprehend a televised story. Deafness & Education International 12

(1), 34e51.

Chai, J., Erlam, R., 2008. The effect and the influence of the use of video and captions on second language learning. New Zealand Studies in

Applied Linguistics 14 (2), 25e44.

Chapelle, C.A., 2003. English Language Learning and Technology. John Benjamins Publishing Company, Philadelphia.

Chung, J.-mei, 1996. The effects of using advance organizers and captions to introduce video in the foreign language classroom. TESL Canada

Journal 14 (1), 61e65.*Chung, J.-mei, 1999. The effects of using video texts supported with advance organizers and captions on Chinese college students’ listening

comprehension: an Empirical Study. Foreign Language Annals 32 (3), 295e308.

Cohen, J., 1988. Statistical Power Analysis for the Behavioural Sciences. Erlbaum, Hillsdale, NJ.

*Danan, M., 1992. Reversed subtitling and dual coding theory: new directions for foreign language instruction. Language Learning 42 (4),

497e527.

Danan, M., 2004. Captioning and subtitling: undervalued language learning strategies. META 49 (1), 67e77.

de Groot, A., van Hell, J., 2005. The learning of foreign language vocabulary. In: Kroll, J., de Groot, A. (Eds.), Handbook of Bilingualism.

Psycholinguistic Approaches. Oxford University Press, Oxford, pp. 9e29.

Diao, Y., Chandler, P., Sweller, J., 2007. The effect of written text on comprehension of spoken English as a foreign language. American Journal of

Psychology 120 (2), 237e261.

Dıaz Cintas, J., Fernandez Cruz, M., 2008. Using subtitled video materials for foreign language instruction. In: Dıaz Cintas, J. (Ed.), The Didactics

of Audiovisual Translation. John Benjamins Publishing Company, Philadelphia, pp. 201e214.

Duquette, L., Painchaud, G., 1996. A comparison of vocabulary acquisition in audio and video contexts. The Canadian Modern Language Review

53 (1), 143e172.

d’Ydewalle, G., Van De Poel, M., 1999. Incidental foreign-language acquisition by children watching subtitled television programs. Journal of

Psycholinguistic Research 28 (3), 227e245.

Fletcher, J.D., Tobias, S., 2005. The multimedia principle. In: Mayer, R.E. (Ed.), The Cambridge Handbook of Multimedia Learning. Cambridge

University Press, Cambridge, pp. 117e133.Garza, T.J., 1991. Evaluating the use of captioned video materials in advanced foreign language learning. Foreign Language Annals 24 (3),

239e258.

Goh, C.C., 2000. A cognitive perspective on language learners’ listening comprehension problems. System 28 (1), 55e75.

Graham, S., 2006. Listening comprehension: the learners’ perspective. System 34 (2), 165e182.Guichon, N., McLornan, S., 2008. The effects of multimodality on L2 learners: implications for CALL resource design. System 36 (1), 85e93.

Guillory, H.G., 1998. The effects of keyword captions to authentic French video on learner comprehension. CALICO Journal 15 (1e3), 89e108.

*Hayati, A., Mohmedi, F., 2011. The effect of films with and without subtitles on listening comprehension of EFL learners. British Journal of

Educational Technology 42 (1), 181e192.

* Studies included in the present meta-analysis are marked with an asterisk.

Page 19: Captioned video for L2 listening and vocabulary learning: A meta-analysis

738 M. Montero Perez et al. / System 41 (2013) 720e739

*Hernandez, S.S., 2004. The Effects of Video and Captioned Text and the Influence of Verbal and Spatial Abilities on Second Language Listening

Comprehension in a Multimedia Learning Environment (Unpublished doctoral dissertation). New York University, New York.

*Huang, H.-C., Eskey, D.E., 1999e2000. The effects of closed-captioned television on the listening comprehension of intermediate English as a

second language (ESL) students. Journal of Educational Technology Systems 28 (1), 75e96.

Hulstijn, J.H., 2007. Psycholinguistic perspectives on second language acquisition. In: Cummins, J., Davidson, C. (Eds.), The International

Handbook on English Language Teaching. Springer, Norwell, pp. 701e713.

Hung, H.T., 2009. Learners’ perceived value of video as mediation in foreign language learning. Journal of Educational Multimedia and Hy-

permedia 18 (2), 171e190.

*Hwang, Y., 2003. The Effect of the Use of Video Captioning on English as a Foreign Language (EFL) on College Students’ Language Learning

in Taiwan (Unpublished doctoral dissertation). The University of Mississippi, Mississippi.

Jones, L.C., Plass, J.L., 2002. Supporting listening comprehension and vocabulary acquisition in French with multimedia annotations. The Modern

Language Journal 86 (4), 546e561.

Jung, U., 1990. The challenge of broadcast videotex to applied linguistics. IRAL 28 (3), 201e220.

Kuppens, A., 2010. Incidental foreign language acquisition from media exposure. Learning Media and Technology 35 (1), 65e85.Lambert, W.E., Holobow, N.E., 1984. Combinations of printed script and spoken dialogue that show promise for students of a foreign language.

Canadian Journal of Behavioural Science 16 (1), 1e11.

Lipsey, M., Wilson, D., 2001. Practical Meta-analysis. Sage Publications, Thousand Oaks.

*Liversidge, G.B., 2000. The Role of Captioning in Second Language Acquisition (Unpublished doctoral dissertation). Temple University

Philadelphia, Philadelphia.

*Markham, P., 1989. The effects of captioned television videotapes on the listening comprehension of beginning, intermediate and advanced ESL

students. Educational Technology 29, 38e41.Markham, P., 1993. Captioned television videotapes: effects of visual support on second language comprehension. Journal of Educational

Technology Systems 21 (3), 183e191.

*Markham, P., 1999. Captioned videotapes and second-language listening word recognition. Foreign Language Annals 32 (3), 321e328.

*Markham, P., 2001. The influence of culture-specific background knowledge and captions on second language comprehension. Journal of

Educational Technology Systems 29 (4), 331e343.

*Markham, P., Peter, L., 2003. The influence of English language and Spanish language captions on foreign language listening/reading

comprehension. Journal of Educational Technology Systems 31 (3), 331e341.

*Markham, P., Peter, L., McCarthy, T., 2001. The effects of native language vs. target language captions on foreign language students’ DVD video

comprehension. Foreign Language Annals 34 (5), 439e445.

Mayer, R.E., 2001. Multimedia Learning. Cambridge University Press, Cambridge.

Meskill, C., 1996. Listening skills development through multimedia. Journal of Educational Multimedia and Hypermedia 5 (2), 179e201.Montero Perez, M., Desmet, P., 2012. The effect of input enhancement in L2 listening on incidental vocabulary learning: a review. In: Macaire, D.,

Boulton, A. (Eds.), Languages, Cultures and Virtual Communities, Proceedings of EuroCALL 2010, Procedia Social & Behavioral Sciences,

vol. 34, pp. 153e157.

Nation, I.S.P., 2001. Learning Vocabulary in Another Language. Cambridge University Press, Cambridge.

*Neuman, S.B., Koskinen, P., 1992. Captioned television as comprehensible input: effects of incidental word learning from context for language

minority students. Reading Research Quarterly 27 (1), 95e106.

Norris, J.M., Ortega, L., 2006. The value and practice of research synthesis for language learning and teaching. In: Norris, J., Ortega, L. (Eds.),

Synthesizing Research on Language Learning and Teaching. John Benjamins Publishing Company, Philadelphia, pp. 3e52.Oxford, R., 1993. Research update on teaching L2 listening. System 21 (2), 205e211.

Paivio, A., 1986. Mental Representations: A Dual Coding Approach. Oxford University Press, Oxford.

*Park, M., 2004. The Effects of Partial Captions on Korean EFL Learners’ Listening Comprehension (Unpublished doctoral dissertation).

University of Texas at Austin, Austin.

Pavesi, M., Perego, E., 2008. Tailor-made interlingual subtitling as a means to enhance second language acquisition. In: Dıaz Cintas, J. (Ed.), The

Didactics of Audiovisual Translation. John Benjamins Publishing Company, Philadelphia, pp. 215e225.

Pujola, J.-T., 2002. CALLing for help: researching language learning strategies using help facilities in a web-based multimedia program. ReCALL

14 (2), 235e262.

Price, K., 1983. Closed-captioned TV: an untapped resource. MATSOL Newsletter 12, 1e8.

Read, J., 2000. Assessing Vocabulary. Cambridge University Press, Cambridge.

Robin, R., 2007. Commentary: learner-based listening and technological authenticity. Language Learning & Technology 11 (1), 109e115.Rost, M., 2002. Teaching and Researching Listening. Pearson Education Limited, Harlow.

Rowland, J.L., 2007. Closed-captioned video and the ESL classroom: a multi-sensory approach. Journal of Adult Education 36 (2), 35e39.

Schilperoord, J., de Groot, V., van Son, N., 2005. Nonverbatim captioning in Dutch television programs: a text linguistic approach. Journal of Deaf

Studies and Deaf Education 10, 402e416.Secules, T., Herron, C., Tomasello, M., 1992. The effect of video context on foreign language learning. The Modern Language Journal 76 (4), 480e490.

Staehr, L.S., 2009. Vocabulary knowledge and advanced listening comprehension in English as a foreign language. Studies in Second Language

Acquisition 31, 577e607.

Stewart, M.A., Pertusa, I., 2004. Gains to language learners from viewing target language closed-captioned films. Foreign Language Annals 37

(3), 438e442.

Sweller, J., 2005. The redundancy principle in multimedia learning. In: Mayer, R.E. (Ed.), The Cambridge Handbook of Multimedia Learning.

Cambridge University Press, Cambridge, pp. 159e167.

Page 20: Captioned video for L2 listening and vocabulary learning: A meta-analysis

739M. Montero Perez et al. / System 41 (2013) 720e739

*Sydorenko, T., 2010. Modality of input and vocabulary acquisition. Language Learning & Technology 14 (2), 50e73.

*Taylor, G., 2005. Perceived processing strategies of students watching captioned video. Foreign Language Annals 38 (3), 422e427.

Terrell, T.D., 1993. Comprehensible input for intermediate foreign language students via video. Journal of Language Learning Technologies 26

(2), 17e23.

Van Patten, B., Williams, J., Rott, S., 2004. Form-meaning connections in second language acquisition. In: Van Patten, B., Williams, J., Rott, S.,

Overstreet, M. (Eds.), Form-meaning Connections in Second Language Acquisition. Lawrence Erlbaum Associates, Mahwah, pp. 1e26.

Vanderplank, R., 1988. The value of teletext sub-titles in language learning. English Language Teaching Journal 42 (4), 272e281.Vanderplank, R., 1990. Paying attention to the words: practical and theoretical problems in watching television programmes with unilingual

(CEEFAX) sub-titles. System 18 (2), 221e234.

Vanderplank, R., 2010. Deja vu? A decade of research on language laboratories, television and video in language learning. Language Teaching 43

(1), 1e37.

Webb, S., Rodgers, M.P.H., 2009. Vocabulary demands of television programs. Language Learning 59 (2), 335e366.

Williams, H., Thorne, D., 2000. The value of teletext subtitling as a medium for language learning. System 28 (2), 217e228.

*Winke, P., Gass, S., Sydorenko, T., 2010. The effects of captioning videos used for foreign language listening activities. Language Learning &

Technology 14 (1), 65e86.

Yuksel, D., Tanriverdi, B., 2009. Effects of watching captioned movie clip on vocabulary development of EFL learners. The Turkish Online

Journal of Educational Technology 8 (2), 48e54.

Zarei, A., 2011. The effect of interlingual and intralingual, verbatim and nonverbatim subtitles on L2 vocabulary comprehension and production.

Journal of Language Teaching and Research 2 (3), 618e625.