This article was downloaded by: [Simon Fraser University]On: 30 September 2014, At: 10:59Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Computer Assisted Language LearningPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/ncal20

Learner perceptions of reliance oncaptions in EFL multimedia listeningcomprehensionAubrey Neil Leveridge a & Jie Chi Yang aa Graduate Institute of Network Learning Technology , NationalCentral University , Jhongli City , TaiwanPublished online: 15 Mar 2013.

To cite this article: Aubrey Neil Leveridge & Jie Chi Yang (2013): Learner perceptions of relianceon captions in EFL multimedia listening comprehension, Computer Assisted Language Learning, DOI:10.1080/09588221.2013.776968

To link to this article: http://dx.doi.org/10.1080/09588221.2013.776968

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Learner perceptions of reliance on captions in EFL multimedia

listening comprehension

Aubrey Neil Leveridge and Jie Chi Yang�

Graduate Institute of Network Learning Technology, National Central University, No. 300, JhongdaRoad, Jhongli City, Taiwan

Instructional support has been widely discussed as a strategy to optimize student-learning experiences. This study examines instructional support within the context ofa multimedia language-learning environment, with the predominant focus on learners’perceptions of captioning support for listening comprehension. The study seeks to an-swer two questions: (1) do learners’ perceptions regarding dependence on captionsmatch their actual reliance on captioning for listening comprehension? and (2) whichlearners’ perceptions are most influenced by proficiency: low-intermediate, intermedi-ate, or high-intermediate? A total of 139 students from a high school English course innorthern Taiwan, all accustomed to multimedia instruction that includes full captions,completed an English language proficiency test as well as a caption reliance test(CRT), and also provided their perceived degree of reliance on captions for Englishlistening comprehension. The results show that overall perceived reliance was signifi-cantly related to actual reliance as assessed by the CRT. However, proficiency wasalso found to be a mitigating factor in the relationship: while low-intermediate levellearners accurately perceived their reliance, no relation was found for either intermedi-ate or high-intermediate learners, indicating that, at these levels, some learners maynot accurately judge their reliance on captioning. Accordingly, the study offers peda-gogical implications that captioning support, added or removed, based on learner self-reports, may not be inherently beneficial, as perceptions on the reliance of captioningmay be inaccurate.

Keywords: CALL; captions; learner perceptions; EFL listening comprehension;reliance

Introduction

Learning a foreign language is often seen as a difficult and demanding undertaking; each

aspect of language learning includes many peculiarities associated with listening, speak-

ing, reading, or writing. Of these, the process of listening is often the center of frustration

for many learners due to its very nature. When speaking, the learner controls the rate of

speech as well as the content; when reading, the learners read text at their preferred speed;

and when writing, the learner can manipulate, erase, and rewrite text. As such, listening is

unlike other skills, in that the medium involved is transitory, and the speed and content

are typically controlled by the speaker (Brown, 2001; Graham, 2006; Vogely, 1998).

Consequently, instructors must seek out various learning supports to assist learners.

One such support that instructors often turn to is captioning (Chen, 2011; Leveridge &

Yang, in press; Stewart & Pertusa, 2004), which is described as visual text delivered via

multimedia that matches the target language auditory signal verbatim. Since captions

*Corresponding author. Email: yang@cl.ncu.edu.tw

� 2013 Taylor & Francis

Computer Assisted Language Learning, 2013

http://dx.doi.org/10.1080/09588221.2013.776968

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

offer a visual representation of what has been heard, they have been the foci of much re-

search (Danan, 2004; Richards & Gordon, 2004; Stewart & Pertusa, 2004). However, re-

search findings have proven inconclusive; although most support captioning as an aid to

learner comprehension (e.g. Danan, 2004; Hayati & Mohmedi, 2011), some results (e.g.

Stewart & Pertusa, 2004; Vandergrift, 2004; 2007) have indicated that captioning can be

a hindrance. In addition, both Vandergrift (2004) and Yanagawa and Green (2008)

pointed out that the use of captioning might also delay learners’ advancement of their lis-

tening ability, rendering them unable to comprehend the second language (L2) in real-life

situations where captions are not readily available.

Further, since understanding how to successfully employ prior knowledge enables

learners to communicate in the target language (Tuan & Loan, 2010), facilitation of this

understanding is a common aim of L2 listening instruction. The removal of listening sup-

ports that may impede this building process represents a vital objective. However, as

Leveridge and Yang (in press) pointed out, the timing of removal is crucial, as early

removal of captions can cause frustration, while late removal can cause interference.

Consequently, instructors often face a dilemma in terms of the deployment of captions

(Hayati & Mohmedi, 2011).

Eliciting learner perceptions regarding support can assist in ensuring adequate scaf-

folding – it is widely accepted that learner perceptions of instruction and instructional de-

sign play an important role as they direct learning (Frick, Chadha, Watson, Wang, &

Green, 2009; K€onings, Brand-Gruwel, & van Merri€enboer, 2010; Zhu, Valcke, &

Schellens, 2009). These perceptions are often weighted heavily by instructors and often

sway their decisions regarding the addition or removal of learning supports. Accordingly,

instructors often consult learners regarding their reliance on learning supports in an at-

tempt to avoid frustration or interference. Yet this process seldom yields satisfactory

results, as learners tend to self-report in a manner deemed pleasing to their instructors

(Lepper, Corpus, & Iyengar, 2005). In addition, not all learners may be capable of accu-

rately judging their reliance on captioning, further compounding the issue.

Judging personal reliance on captioning may also be problematic since reliance is

not static: learners rely on captioning in varying degrees (Wagner, 2007), i.e. as their

language ability increases, they tend to become less reliant on captioning (Leveridge &

Yang, in press). As such, lower level learners, due to the limited amount of vocabulary

acquired, tend to rely more heavily on captioning support, while advanced learners,

who possess a more substantial vocabulary, tend to rely on captioning support less

(Hayati & Mohmedi, 2011; Jones, 2003). This signifies diminishing degrees of reliance

on captioning at the intermediate level. In other words, the intermediate level is the

bridge between the beginner and advanced levels in which reliance on captioning

support progressively decreases. Thus, it is at this level that the learners’ perception of

reliance is pertinent because instruction, as previously mentioned, is often guided by

these perceptions.

Following in this vein, the aims of the current empirical study are to: (1) determine if

learner perceptions are accurate markers of their reliance on captioning, and (2) discover

whether learners at each of the three levels of proficiency (low-intermediate, intermedi-

ate, or high-intermediate) are able to accurately judge their reliance on captioning. As

such, this study investigates learner reliance on captioning support and learner percep-

tions of personal reliance. To better understand the context of the current study, related

studies highlighting the use of captions and learner perceptions are reviewed, followed by

the methodology and results of the current study. The final section includes a discussion

of the key findings and concluding remarks.

2 A.N. Leveridge and J.C. Yang

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

Review of literature

Appropriate instructional supports are an integral part of learning as they ensure under-

standing of the learning content. As Lee, Srinivasan, Trail, Lewis, and Lopez (2011) sug-

gested, appropriate support strategies that fulfill learners’ requirements and learning

styles are desirable, as they are likely to augment learners’ learning as well as their learn-

ing experiences. This is particularly pertinent to captioning support in multimedia listen-

ing comprehension instruction, as captioning has been found to both augment (Hayati &

Mohmedi, 2011) as well as impede (Taylor, 2005) learning.

Thus far, much of the research regarding captioning support has focused on how cap-

tions benefit certain aspects of L2 comprehension in terms of vocabulary (Stewart &

Pertusa, 2004) and grammar acquisition (Van Lommel, Laenen, & d’Ydewalle, 2006), or

how the addition of captions increases learner comprehension (Chang, Tseng, & Tseng,

2011). While several pertinent studies regarding multimedia-based learning exist, the in-

fluence of multimedia on learning remains questionable (Bhowmick, Khasawneh,

Bowling, Gramopadhye, & Melloy, 2007; Chang et al., 2011). Furthermore, Taylor

(2005) pointed out that the majority of studies on captioning support have involved inter-

mediate or advanced learners, suggesting that captioning may not be an appropriate learn-

ing support for lower proficiency learners. However, the results of several studies

performed over the last two decades that included lower-level learners offer results that

suggest otherwise (see Danan, 2004; Markham, 1989; Neuman & Koskinen, 1992).

A very recent study by Hayati and Mohmedi (2011) explored the effects of films with

and without subtitles on listening comprehension for intermediate English as a foreign

language (EFL) learners in an Iranian university. Two English documentary films were

divided into six episodes and played to 90 university students in one of three treatment

conditions over a six-week period: with English subtitles, with Persian subtitles, and with

no subtitles. To evaluate listening comprehension and establish comparative grounds, a

multiple-choice test was administered following the playing of each episode. The results

show that those participants in the group exposed to native language subtitles scored sig-

nificantly higher than the no-captioning group. However, an even more substantial benefit

was observed for groups with the subtitles in the target language. The authors concluded

that simultaneous reading and listening to messages enhances comprehension of a foreign

language. This was further supported by data collected in a survey of these learners’ per-

ceptions, where learners mentioned that the target language captions did help them to as-

sociate the aural and textual forms of words more easily and quickly than did native

language subtitles or no subtitles. However, no assessment of the degree of reliance on

captioning support for comprehension was administered, which makes these perceptions

questionable, as the students may have been more reliant on captioning than on the aural

signals. Furthermore, learners’ prior experience of instruction supported by captioning

was not disclosed. Thus, it remains possible that the addition of captioning support pro-

duced a novelty effect that in turn, further averted learners’ attention away from the aural

signals in the direction of the visual text.

Taylor (2005) focused on 85 Spanish language learners at a large state university:

14 lower-level first-year and 71 advanced-level third-year students. Participants were

randomly assigned to one of two groups to view a video presented with target language

captions (treatment group) or without captions (control group). While 35% of the first

year students and 11% of the third year students in the treatment group found the captions

distracting or confusing, most displayed a positive attitude towards captioning and many

more perceived the captions as helpful. Furthermore, while approximately 75% of the

Computer Assisted Language Learning 3

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

treatment group reported having attempted to listen to the audio, some students ignored

the audio altogether and focused only on the text. These findings may be due to an over-

load of working memory; another plausible alternative is that learners at lower levels do

not understand how to successfully process the added information of captioning, yet mis-

takenly perceive captioning as helpful, and even necessary. However, similar to Hayati

and Mohmedi’s study (2011), the learners’ prior experience with captioning support was

not indicated, again raising questions regarding extraneous influences such as the previ-

ously mentioned novelty effect.

To uncover the degree to which learners rely on captioning, Leveridge and Yang

(in press) created a novel testing instrument, the caption reliance test (CRT), which

assesses learners’ attention by uncovering what the learner is focusing more heavily on to

gain comprehension: audio or textual information. The testing method includes listening

exercises and auditory multiple-choice questions that are all supported by congruent,

verbatim captioning. To uncover the learners’ focus, incongruences between the audio

and captioning are introduced in the listening exercises. The multiple-choice questions in-

clude one answer that matches the audio and one answer that matches the incongruent

caption. Thus, the learners’ predominant focus may be determined according to the

answers they choose. The CRT was given to three high school EFL classes and the results

then compared to the students’ outcomes garnered from an English language comprehen-

sion test, as well as an English proficiency test. The combined results suggest that all

learners do not rely on captions equally. While lower-level learners rely heavily on

captioning support, higher-level learners are considerably less reliant, possibly due to the

greater amount of prior knowledge they possess as well as their understanding of how to

retrieve this knowledge; this suggests that learners who lack prior knowledge read cap-

tions to gain comprehension. These findings differ from those of Taylor (2005) outlined

above. Moreover, some of Taylor’s (2005) participants ignored the audio altogether,

suggesting a major reliance on captioning. In addition, the participants in Leveridge and

Yang’s (in press) study were accustomed to instruction that included captioning support,

whereas Taylor’s (2005) study does not indicate prior exposure.

In summary, research results regarding captioning are inconsistent. Regardless of

these inconsistencies, captioning support is generally not available in authentic, real-time

listening (Vandergrift, 2004). Accordingly, a general goal of language instruction is to en-

hance learners’ L2 abilities so that they are able to communicate in the target language

without learning supports: as learners progress, instructors seek to remove captioning sup-

port. However, since instructors are frequently unsure as to whether captions should be

retained or removed during listening comprehension activities, they often turn to the

learners for information to help them make appropriate decisions, even though learners

may misjudge their reliance on learning supports (Lepper et al., 2005) resulting in non-

optimal decisions. These problems are reflected in Hayati and Mohmedi’s (2011) state-

ment that the benefits of including subtitles when using movies to teach L2 learners re-

main unresolved. The current study is an attempt to clarify some of these issues by

gathering EFL learners’ perceptions on captioning support, assessing their reliance on

captioning support, and subsequently comparing both in light of individual proficiency.

Research questions

The first research question was based on the assumption that learners’ perceptions of their

reliance on captioning support would match the actual degree of reliance. Furthermore,

as learners’ progress, their level of reliance subsides (Heift, 2006); yielding the

4 A.N. Leveridge and J.C. Yang

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

assumption that learners’ perceptions as to their degree of reliance should also subside,

which is reflected in the second research question.

(1) Do learners’ perceptions of reliance on captioning match their actual degree of

reliance?

(2) Is proficiency a mitigating factor in the accuracy of learner perceptions regarding

reliance?

Methodology

Participants

This study included 139 students, 38 males and 101 females with an average age of 17.

All were studying in one of three classes at a high school in northern Taiwan. All partici-

pants had been enrolled in English as a Foreign Language classes for at least three years.

While all students were of a similar age and in the same grade, their individual English

abilities varied from low-intermediate to high-intermediate. Captioning support was a fa-

miliar medium, as all students had more than two years prior instructional experience in

traditional classrooms enhanced with instructional multimedia, i.e. a multimedia com-

puter system with a projector, including audio lessons supported with captioning. This

instruction consisted of auditory lessons, exercises, and listening comprehension assess-

ments. The auditory lessons, exercises, and assessments were presented in English – the

target language – and were supported with complete, congruent visual text (captions) that

appeared via a projector on a communal screen. It should be pointed out that the partici-

pants did not watch captioned videos; full captions were made available to read on a

screen while listening to an audio lesson. In other words, no video, movies, or animations

were presented with the auditory lessons. In addition, each student was supplied with a

textbook in which they could take notes, answer questions from the lessons and exercises,

and record answers for listening comprehension assessments.

Instruments

The instruments employed in this study were as follows: (1) the intermediate general

English proficiency test (GEPT); (2) the caption reliance test (CRT); (3) a Likert-scale

question regarding learner perceptions; (4) a multimedia system consisting of a computer,

projector, screen, and public address system; and (5) semi-structured interviews.

(1) GEPT: the GEPT, developed in 1999 in Taiwan, provides individual evaluation

of English language proficiency (Roever & Pan, 2008). The current study aimed

to determine the participants’ level of EFL listening comprehension proficiency

variance prior to the experiment using the GEPT results, thereby enabling catego-

rization into one of three proficiency levels: low-intermediate, intermediate, or

high-intermediate. The GEPT scores were divided into three percentiles at 33.3%

intervals, resulting in the following groups: 1 ¼ low-intermediate, 2 ¼ intermedi-

ate, and 3 ¼ high-intermediate.

(2) CRT: the CRT is a multimedia listening test that determines learners’ reliance on

captioning by employing both audio texts and congruent visual textual captions

that include specific incongruent keywords. The content of the CRT may vary,

but the structure remains constant. The CRT comprises a total of 40 multiple-

choice questions, including 30 congruent (see Figure 1) and 10 randomly placed

incongruent (see Figure 2) items.

Computer Assisted Language Learning 5

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

The CRT content was adapted from the audio of the participants’ current listening

comprehension book at the participants’ current stage. Reliability analysis was employed

to check the dependability, consistency and homogeneity of each item in the CRT.

Cronbach’s a for the multiple-choice questions in the CRT was 0.71, which indicates ac-

ceptable reliability. The validity of the CRT was assessed by the participants’ language

instructor as well as a native English speaker to ensure that the test was at an appropriate

level, the grammar was correct, and the testing items were accurate.

(1) A Likert self-report scale: data regarding the subjects’ perceived reliance on cap-

tioning support were collected from self-reports based on a Likert scale with nu-

merical values from 1 to 9. This scale comprised a single question: “How much

do you rely on captioning for comprehension?” where 1 ¼ not at all, 5 ¼ some-

what, 9 ¼ a lot.

(2) Interviews: post experiment semi-structured interviews were given to all partici-

pants with a particular focus on those whose perception of reliance showed the

largest incongruities with the GEPT and the CRT. Incongruence between the

GEPT and the CRT suggests that different types of processing are occurring.

Thus, the questions were designed to first establish if the learner was gaining

Figure 1. Screenshots: CRT audio script congruent question and answer.

Figure 2. Screenshots: Typical incongruent question slides (top row), typical answer slide (bottomright).

6 A.N. Leveridge and J.C. Yang

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

comprehension from the captions, and second to provide possible evidence of al-

ternative cognitive processing taking place. Learners using the captions to gain

comprehension may have tended to read the captions first, while those who read

the captions after listening to the audio may have been using the captions to con-

firm what they heard as correct. As such, two guiding questions led the inter-

views: (1) do captions provide you with the meanings? and (2) do you read the

captions first?

Procedure

The experiment began with 12 weeks of listening comprehension instruction, which in-

cluded captioned content that was integrated into the participants’ regular syllabus and

conducted during the participants’ first semester of their third year of study. The study

took place in the participants’ regular school classrooms with the instructional content in-

tegrated and delivered over 15 weeks. The data were collected during three consecutive

weeks at the end of the semester to ensure that the participants were accustomed to the de-

livery methods. Each of these three weeks corresponded to one of three stages: Stage 1,

the proficiency assessment stage; Stage 2, the reliance/perception assessment stage; and

Stage 3, the interview stage.

Stage 1: assessment of language proficiency. The GEPT was administered to all par-

ticipants in order to categorize the individual learners into low-intermediate, interme-

diate, and high-intermediate proficiency learners. The time allotted for answering the

questions was 50 minutes, equivalent to the listening comprehension section within

the GEPT.

Stage 2: all students participated in the CRT to assess their reliance on captioning

support. Participants’ reliance on captioning was measured through the use of the

CRT (in)congruent questions and triangulated with the GEPT. Immediately after the

CRT was completed, the participants were asked to rate their perceptions of reliance

on captioning utilizing the aforementioned 9-point Likert-type scale; this scale pro-

vided a wider range of responses for the participants, in turn allowing finer results to

be considered. The standard 15 minutes were allotted to complete the CRT and an-

swer the perception question.

Stage 3: semi-structured interviews: individual semi-structured interviews were con-

ducted to provide a deeper understanding of any unexpected results.

Data analysis

After the 139 participants were separated into three percentile groups based on their

GEPT scores, Spearman’s correlation was employed to uncover any correlation between

the three variables. Information collected in this study included: GEPT scores; CRT

scores, and responses to the reliance perception question.

Results

The various results are shown in Table 1. The GEPT results consisted of a minimum score

of 29 and a maximum of 69 out of a possible 100, with a mean of 48.73 (SD ¼ 9.18); the

CRT scores ranged from a minimum of 0 to a maximum of 9, with a mean of 3.09

Computer Assisted Language Learning 7

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

(SD ¼ 1.70); and the nine-point Likert question on perception garnered a minimum of 1,

and a maximum of 9, with a mean of 5.54 (SD ¼ 2.01).

The Spearman’s correlation result (see Table 2) indicates that the participants’ percep-

tions on reliance of captions was significantly correlated to their CRT results (p < 0.01),

suggesting that learners were able to accurately perceive their reliance on captioning

support.

Table 3 illustrates the categorization of learners based on proficiency levels obtained

using the GEPT. Because the GEPT is designed to rank proficiency from the very begin-

ning stages through to fluent levels, the small variation in scores was expected. Three per-

centile groups of learners were created based on these scores, as depicted in Table 3: low-

intermediate (N ¼ 43), with an average of 37.93% (SD ¼ 3.61); intermediate (N ¼ 53),

with an average of 48.81% (SD ¼ 2.96); and high-intermediate (N ¼ 43), with an average

of 59.43% (SD ¼ 3.88).

Following this categorization, Spearman’s correlation was again applied to the three

groups separately. The results showed a positive correlation (Spearman’s rho ¼ .365, p <.05) only between low-intermediate learners and their perception of reliance on caption-

ing, indicating that proficiency was a mitigating factor. While low-intermediate learners

perceived their degree of reliance on captioning accurately, neither intermediate

(Spearman’s rho ¼ .000, p ¼ .997) nor high (Spearman’s rho ¼ .194, p ¼ .213) profi-

ciency learners did so, as shown in Table 4.

Of the 139 participants, a total of 132 partook in the semi-structured interview, as

7 were absent. The responses garnered from the semi-structured interviews were

Table 1. Descriptive statistics from gathered data.

N Minimum Maximum Mean SD

GEPT 139 29 69 48.73 9.18CRT 139 0 9 3.09 1.70Perception 139 1 9 5.54 2.01

Table 2. Spearman’s correlation coefficient between CRT and perception.

CRT Perception

CRT Correlation coefficient 1sig. (2-tailed)

Perception Correlation coefficient .220�� 1sig. (2-tailed) .009

Note: ��p < 0.01.

Table 3. GEPT learner distribution.

Level N GEPT mean SD

Low 43 37.93% 3.61Intermediate 53 48.81% 2.96High 43 59.43% 3.88

8 A.N. Leveridge and J.C. Yang

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

categorized according to the learners’ proficiency levels, and indicate that as learners ad-

vance, they tend to both rely less on captions for comprehension and become less likely

to read the captions first.

To address the first guiding question, “Do captions provide you with the meanings?”

the responses were recorded and categorized by proficiency level. As depicted in Table 5,

positive responses were indicated by of most low-intermediate learners: 95%. Most of the

low-intermediate learners indicated an explicit need for captioning support:

Without the captions, I can’t understand many words.

I heard words I don’t know, but I can recognize them in the captions.

Fewer intermediate learners responded positively: 85.3%. Several intermediate

respondents stated that the captions facilitated comprehension but they did not require

full verbatim captioning:

I only looked at the captions when I heard an unfamiliar word.

I read captions for words I never heard.

I don’t need all the sentences [captions] only for some difficult words.

Even fewer high-intermediate learners responded in a positive manner: 74.5%. Many

high-intermediate respondents stated that the captions were useful to check the accuracy

of their listening comprehension:

Captions help me check [the accuracy of] my listening.

Table 4. Spearman’s correlations on perception and the CRT.

CRT Perception

Low CRT Correlation coefficient 1sig. (2-tailed)

Perception Correlation coefficient .365� 1sig. (2-tailed) .016

Intermediate CRT Correlation coefficient 1sig. (2-tailed)

Perception Correlation coefficient .000 1sig. (2-tailed) .997

High CRT Correlation coefficient 1sig. (2-tailed)

Perception Correlation coefficient .194 1sig. (2-tailed) .213

Note: �p < 0.05.

Table 5. Interview responses in percentages.

N Do captions provide you with the meanings? Do you read the captions first?

Low 40 38 (95.0%) 36 (90.0%)Intermediate 41 35 (85.3%) 33 (80.4%)High 51 38 (74.5%) 32 (62.7%)

Computer Assisted Language Learning 9

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

To address the second guiding question, “Do you read the captions first?” the

responses were recorded and categorized by proficiency level and are presented in

Table 5. Of the low-intermediate participants, 90% indicated positive responses. Most of

the respondents stated that reading the captions first was essential to gain comprehension:

Without the captions, I can’t understand what is happening.

Likewise, 80.4% of the intermediate participants responded positively, stating that

reading the captions prior to listening to the audio was advantageous when encountering

an unfamiliar subject. By contrast, only 62.7% high-intermediate learners indicated posi-

tive responses: several mentioned that they understood the vocabulary and/or were famil-

iar with the content, but read the captions first for different reasons:

Captions give me a clear idea of what is happening before I listen.

I can understand more because I know what to expect, so I can listen to how the speaker talks.

These comments contrasted with views regarding the focus of attention and where it

should be placed for an optimal learning experience. In particular, several high-intermedi-

ate learners were dissatisfied with captioning because the captions thwarted attempts to

rely solely on the auditory message:

When the captions are there, it is like I have to read them.

With captions, I have to close my eyes so I can concentrate only on what I hear.

It would appear that captioning may be a valuable support for some, while a hindrance

for others.

Discussion

The aims of the current study were to: (1) determine if learner perceptions are accurate

markers of their reliance on captioning, and (2) discover whether learners at each of the

three levels of proficiency were able to accurately judge their reliance on captioning.

These aims are discussed sequentially below, followed by short discussions regarding

each proficiency level and the relationship to current language learning theories.

Are learner perceptions accurate markers of reliance on captioning?

The results from the Spearman’s correlation, CRT, and learners’ perceptions of reliance

on captions indicate a correlation between the learners’ perceptions and their assessed de-

pendence, which suggests that EFL learner perceptions of reliance on captioning do match

their actual degree of reliance. It also implies that learners can benefit from instructor

decisions to deploy captioning support based on learner perceptions. However, this result

must be treated with a certain amount of caution, as it only holds true when participants

are assessed as a whole and proficiency is not considered. As such, this result may be

more applicable to heterogeneous groups comprising learners from diverse instructional

backgrounds, with various first languages, and a wide range of proficiency. For more ho-

mogeneous groups comprising learners with similar instructional backgrounds, the same

10 A.N. Leveridge and J.C. Yang

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

first language, and similar levels of proficiency, as in the case of the current study, a sig-

nificant mitigating factor must be considered – the level of proficiency.

Do learners at different proficiency levels accurately judge their reliance?

In contrast to the aforementioned situation where a significant number of all learners

could accurately perceive their reliance on captioning, when proficiency levels were fac-

tored in, only low-intermediate learners provided accurate predictions. One possible ex-

planation may be that the low-intermediate proficiency learners understood their need for

the additional support, based on their realization that they are at a lower level of compre-

hension. Additionally, many intermediate and high-intermediate level learners expressed

a sense of overconfidence or modesty; either may have clouded their perceptions, leading

to an under- or over-estimation of their reliance, respectively. As each proficiency level

was associated with different perceptions, they are discussed individually as follows.

Low-intermediate proficiency learners

In line with Vandergrift (2004), lower proficiency learners reported that they found cap-

tions to be beneficial. These learners noted difficulties in differentiating between words

that seemed to blend together, which is common for native speakers. Furthermore, when

captions are present, there is a visual representation of individual words (Hayati &

Mohmedi, 2011), which is clearly defined and separated. Low-intermediate proficiency

learners relied on the captions to provide an alternative method to comprehend words,

such as the spelling of aurally unrecognized individual words, and to obtain an overall vi-

sual structure of the spoken sentences. This suggests that low-intermediate learners tend

to follow a bottom-up approach: the auditory track is not used to increase sentence com-

prehension, but to clarify the pronunciation of each individual word, with a belief that

“. . . comprehension [is] incomplete unless every word has been recognized” (Field,

2004). This was not, however, the case with higher proficiency learners.

Intermediate proficiency learners

In line with findings by Hayati and Mohmedi (2011), intermediate learners recognized

more individual words, due to their larger and more accessible vocabulary. When identi-

fying fragments of unknown speech, these learners have greater access to a more top-

down approach as well as the ability to incorporate prior knowledge. The captioning pro-

vided the structure of the sentences and could easily be reread to clarify what was heard.

Based on their larger vocabulary, they tended to believe that captions are not required for

comprehension. However, intermediate learners faced greater difficulties in terms of sen-

tence structures and inferring meaning from context. As intermediate learners attempt to

integrate bottom-up with top-down approaches, they may perceive the captions as unnec-

essary, since they understand most words. That said, they still tend to read the captions

for unknown keywords in an attempt to increase overall understanding as reported in the

interviews (see Table 5), similar to the findings reported by Yang and Chang (in press).

High-intermediate proficiency learners

Previous studies (see Chang et al., 2011; Taylor, 2005) noted that high proficiency learn-

ers found captions beneficial. This view is contradictory to the current study’s findings,

Computer Assisted Language Learning 11

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

where the high-intermediate learners expressed that eliminating the captioning helped

them to concentrate solely on listening, thereby increasing both focused cognitive proc-

essing and comprehension via the aural message. Based on comments made during the

interviews, when captions were included, many high-intermediate proficiency learners

first listened to the audio, and then compared what they heard to the captions, leaving lit-

tle time for processing. Furthermore, if the captioning differed slightly from what was

expected, confusion followed, creating further interference. When compared to the inter-

view responses from the low-intermediate and intermediate proficiency learners, fewer

high-intermediate proficiency learners indicated that they read the captions first (see

Table 5). A further comparison shows that a smaller number of these same learners also

reported that comprehension was not achieved by reading the captions. Overall, these

learners may have interpreted this limited processing time as confusing.

Relationship between perception, reliance, and information processing

In the current study, the participants’ reading skills may be more advanced than their lis-

tening skills based on the high percentage of learners who answered positively to both

semi-structured interview questions. This finding is similar to those from prior studies

where the participants did not pay attention to the audio, as they were more experienced

in reading (see Guillory, 1998; Hayati & Mohmedi, 2011; Taylor, 2005). The current em-

pirical results indicate that low-intermediate proficiency learners process captioned audio

differently than do intermediate and high-intermediate proficiency learners, and that

learners’ perceptions of reliance often do not match actual assessed reliance, where inter-

mediate and high-intermediate proficiency learners may be confused as to their degree of

reliance. These findings provide interesting insights into the ongoing processes within

multimedia listening environments when captions are present. Numerous low-intermediate

proficiency learners indicated that they tend to read the captions first, listen to the audio, and

then match the captions to the audio (see Table 5), comparable to bottom-up processing.

However, a higher number of intermediate and high-intermediate proficiency learners indi-

cated that they do not read the captions first, constructing meaning from the audio, which is

a top-down process. As the audio and captions were provided simultaneously, they lacked

sufficient processing time. This in turn led to frustration and may have altered learners’ con-

fidence in their abilities, ultimately changing their perceptions on reliance. However, this ob-

servation serves only as an indication that further research in this area is required.

Conclusions

The current study increases the breadth of empirical knowledge regarding the employ-

ment of captioning support, as it highlights the significance of learner proficiency when

assessing the factors associated with captioning reliance. Captioning can be a valuable

support tool if it is not removed too early, causing frustration, or it is not employed for

too long, causing interference; as such, captioning is simultaneously beneficial to some

and a hindrance to others. In regards to pedagogical implications, this study also provides

evidence that learner perceptions regarding captioning support may be misleading, espe-

cially in terms of those provided by intermediate or high-intermediate proficiency learn-

ers. Thus, one implication of this study is that EFL multimedia instruction might address

the requirements and preferences of individual learners to ensure more beneficial instruc-

tion by taking into account individual differences, specifically levels of proficiency.

12 A.N. Leveridge and J.C. Yang

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

This study also explores a new avenue of the effects of L2 proficiency on learner per-

ceptions of reliance on captioning support within a multimedia listening comprehension

environment; however, there is room for future research to probe or enhance the findings.

Captioning is but one approach that can support L2 listening comprehension instruction.

Additionally, the current study focuses on one particular aspect that influences the effec-

tiveness of this type of support; therefore, it is limited in that various characteristics be-

yond the scope of this study may have influenced the effectiveness of captioning support.

First, the participant group was acquired from a single high school in northern Taiwan; a

fairly homogeneous group where all participants shared the same first language, similar

educational backgrounds, and prior experience of instruction with captioning support.

They were all at an intermediate proficiency level, but were further categorized as low-in-

termediate, intermediate, or high-intermediate. Therefore, the results of this study cannot

be generalized to regions outside of Taiwan, a non-EFL context, or other proficiency lev-

els. Additionally, the participants in this study were accustomed to receiving English lis-

tening comprehension instruction with captioning support provided, which may not be a

customary learning support in other areas of the world. Furthermore, prior studies have

shown that working memory overload may affect language acquisition; however, as this

issue was outside the scope of the current study, future research is required.

To conclude, this study has uncovered the possibility that all learners, depending on

their level of proficiency, may not process audio with captioning support in the same

manner: some learners quickly read the captions and match them with the audio when

there is sufficient processing time, while others wait to hear the audio and then read the

captions, which severely limits the available processing time. Further research on the var-

ious processes involved at different levels of learner proficiency, as well as longitudinal

studies incorporating various types of captioning, would lead to a better understanding of

reliance on captioning, and in turn, improved instructional use. As this study was

designed to explore only learner perceptions of reliance on captioning and to test if profi-

ciency was a mitigating factor of reliance, these issues may be worthy of future investiga-

tion. Finally, future studies should consider other variables, such as individual

differences, learning styles, text characteristics, familiarity of content, and/or task ap-

proach (i.e. pre-task activities or the nature of the comprehension task), as the effective-

ness of this form of support may depend on such variables.

Acknowledgements

The authors would like to thank Mr Christian Venhuizen, from Simon Fraser University, Canada forhis help in improving the readability of this article. The authors would also like to thank all the sub-jects who participated in the study. This study was partially supported by a grant (NSC 100-2628-S-008-002-MY3) from the National Science Council of Taiwan.

Notes on contributors

Aubrey Neil Leveridge is currently a PhD candidate in the Graduate Institute of Network LearningTechnology at National Central University, Taiwan and holds a Master of Education from the Uni-versity of Southern Queensland, Australia. His research focus is mainly on second language acquisi-tion, computer-assisted language learning (CALL), the integration of CALL into classroom settings,and testing for reliance on multimedia learning supports. He has published works in the ComputerAssisted Language Learning and ReCALL journals and is an active member in several CALL spe-cial interest groups.

Computer Assisted Language Learning 13

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

Jie Chi Yang is currently the head of the Graduate Institute of Network Learning Technology and adistinguished professor at the National Central University, Taiwan. With more than a decade of in-volvement in CALL, he has been utilizing various advanced techniques to support language learn-ing, including natural language processing, film-based concordances, video-capture virtual reality,multimedia technologies, and game-based learning strategies. He has published manuscriptsin high-impact journals such as Computers & Education, Journal of Computer Assisted Learning,Educational Technology & Society, Interactive Learning Environments, Computer AssistedLanguage Learning, ReCALL, and CALICO.

References

Bhowmick, A., Khasawneh, M. T., Bowling, S. R., Gramopadhye, A. K., & Melloy, B. J. (2007).Evaluation of alternate multimedia for web-based asynchronous learning. International Journalof Industrial Ergonomics, 37(7), 615–629.

Brown, H. D. (2001). Teaching by principles: An interactive approach to language pedagogy (2nded.). New York: Pearson Education.

Chang, C. C., Tseng, K. H., & Tseng, J. S. (2011). Is single or dual channel better for English listen-ing comprehension? Computers & Education, 57(4), 2313–2321.

Chen, H. J. (2011). Developing and evaluating SynctoLearn, a fully automatic video and transcriptsynchronization tool for EFL learners. Computer Assisted Language Learning, 24(2), 117–130.

Danan, M. (2004). Captioning and subtitling: Undervalued language learning strategies. Meta, 49(1), 67–77.

Field, J. (2004). An insight into listeners’ problems: Too much bottom-up or too much top-down?System, 32(3), 363–377.

Frick, T. W., Chadha, R., Watson, C., Wang, Y., & Green, P. (2009). College student perceptions ofteaching and learning quality. Educational Technology Research and Development, 57(5), 705–720.

Graham, S. (2006). Listening comprehension: The learners’ perspective. System, 34, 165–182.Guillory, H. G. (1998). The effects of keyword captions to authentic French video on learner com-

prehension. CALICO Journal, 15(1), 89–108.Hayati, A., & Mohmedi, F. (2011). The effect of films with and without subtitles on listening com-

prehension of EFL learners. British Journal of Educational Technology, 42(1), 181–192.Heift, T. (2006). Context-sensitive help in CALL. Computer Assisted Language Learning, 19(2),

243–259.Jones, L. (2003). Supporting listening comprehension and vocabulary acquisition with multimedia

annotations: The students’ voice. CALICO Journal, 21(1), 41–65.K€onings, K. D., Brand-Gruwel, S., & van Merri€enboer, J. J. G. (2010). Participatory instructional

redesign by students and teachers in secondary education: Effects on perceptions of instruction.British Journal of Instructional Science, 39(5), 737–762.

Lee, S. J., Srinivasan, S., Trail, T., Lewis, D., & Lopez, S. (2011). Examining the relationshipamong student perception of support, course satisfaction, and learning outcomes in online learn-ing. The Internet and Higher Education, 14(3), 158–163.

Lepper, M. R., Corpus, J. H., & Iyengar, S. S. (2005). Intrinsic and extrinsic motivational orienta-tions in the classroom: Age differences and academic correlates. Journal of Educational Psy-chology, 97(2), 184–196.

Leveridge, A. N., & Yang, J. C. (in press). Testing learner reliance on caption supports in secondlanguage listening comprehension multimedia environments. ReCALL.

Markham, P. L. (1989). The effects of captioned television videotapes on the listening comprehen-sion of beginning, intermediate, and advanced ESL students. Educational Technology, 29(10),38–41.

Neuman, S. B., & Koskinen, P. (1992). Captioned television as ‘comprehensible input’: Effects ofincidental word learning from context for language minority students. Reading Research Quar-terly, 27(1), 95–106.

Richards, J. C., & Gordon, D. B. (2004). New interchange intro: Video teacher’s guide. New York:Cambridge University Press.

Roever, C., & Pan, Y. C. (2008). Test review: GEPT: General English Proficiency Test. LanguageTesting, 25, 403–418.

14 A.N. Leveridge and J.C. Yang

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014

Stewart, M. A., & Pertusa, I. (2004). Gains to language learners from viewing target languageclosed-captioned films. Foreign Language Annals, 37(3), 438–442.

Taylor, G. (2005). Perceived processing strategies of students watching captioned video. ForeignLanguage Annals, 38(3), 422–427.

Tuan, L. T., & Loan, B. T. (2010). Schema-building and listening. Studies in Literature andLanguage, 1(5), 53–65.

Van Lommel, S., Laenen, A., & d’Ydewalle, G. (2006). Foreign-grammar acquisition whilewatching subtitled television programmes. British Journal of Educational Psychology, 76(2),243–258.

Vandergrift, L. (2004). Listening to learn or learning to listen? Annual Review of Applied Linguis-tics, 24, 3–25.

Vandergrift, L. (2007). Recent developments in second and foreign language listening comprehen-sion research. Language Teaching, 40(3), 191–210.

Vogely, A. J. (1998). Listening comprehension anxiety: Students’ reported sources and solutions.Foreign Language Annals, 31(1), 67–80.

Wagner, E. (2007). Are they watching? An investigation of test-taker viewing behavior during anL2 video listening test. Language Learning and Technology, 11(1), 67–86.

Yanagawa, K., & Green, A. (2008). To show or not to show: The effects of item stems and answeroptions on performance on a multiple-choice listening comprehension test. System, 36(1),107–122.

Yang, J. C., & Chang, P. (in press). Captions and reduced forms instruction: The impact on EFLstudents’ listening comprehension. ReCALL.

Zhu, C., Valcke, M., & Schellens, T. (2009). Cultural differences in the perception of a social-constructivist e-learning environment. British Journal of Instructional Science, 40(1), 164–168.

Computer Assisted Language Learning 15

Dow

nloa

ded

by [

Sim

on F

rase

r U

nive

rsity

] at

10:

59 3

0 Se

ptem

ber

2014