listening comprehension.pdf

VALIDATION OF WORLD ENGLISH (WE)

LISTENING COMPREHENSION

SCALE

by

YASUSHI KYUTOKU

Presented to the Faculty of the Graduate School of

The University of Texas at Arlington in Partial Fulfillment

of the Requirements

for the Degree of

MASTER OF SCIENCE IN PSYCHOLOGY

THE UNIVERSITY OF TEXAS AT ARLINGTON

MAY 2007

ii

ACKNOWLEDGEMENTS

I want to make acknowledgments for my supervisor, Dr. Bernstein, for all the

supervision, instruction, advices, helps, kindness, patience, and encouragements in

order to complete my thesis research.

Also, I want to make acknowledgements for my committee members, Dr.

Kimball, Dr. Kopp, and Dr. Gorfein. Without your advices, suggestions, and helps, I

could not complete the thesis research. Again, thanks for devoting your precious time

for my thesis.

Also, I want to thank Dr. Dougall for advices, suggestions, and kindness.

Finally, I want to thank Charles Aden, Chawki Belhadi, Chikako Davidson,

Gabriel Davidson, Sachimi Kyutoku, Tomomi Ogura, and Ryo Yamada, and faculties,

staffs, and colleagues in our departments for inspirations and supports.

March 19, 2007

iii

ABSTRACT

VALIDATION OF WORLD ENGLISH (WE)

LISTENING COMPREHENSION

SCALE

Publication No. _____

Yasushi Kyutoku, M.S.

The University of Texas at Arlington, 2007

Supervising Professor: Bernstein, Ira H

World English (WE) includes any type of dialectical English spoken around the

world (Hinkel, 2005). In the present study, a WE listening test was developed. The

original form of the test consisted of six scales measuring, respectively, demographics,

exposure to non-native English speakers, self perception of WE fluency, WE listening

test, evaluations of non-native speakers' fluency, and attitudes towards non-native

English speakers. Internal consistency, dimensionality, correlations among scales, and

the effects of demographics on the test scores were analyzed based on the data collected

from 32 participants in Study 1. Results indicated moderate to moderately high internal

iv

consistency for the scales except for the attitudes toward non-native English speakers

scale. Study 2 constituted an actual computer administration with 106 participants. SE

listening comprehension test was added to the study. All of the scales except for the SE

test showed moderate to high consistency and unidimensionality.

v

TABLE OF CONTENTS

ACKNOWLEDGEMENTS...................................................................................... ii

ABSTRACT.............................................................................................................. iii

LIST OF ILLUSTRATIONS.................................................................................... ix

LIST OF TABLES.................................................................................................... x

Chapter

1. INTRODUCTION ........................................................................................ 1

1.1 Standard English and World English ...................................................... 1

1.1.1 World English (WE) ................................................................ 1

1.1.2 Reasons to Study WE............................................................... 2

1.2 WE Listening Comprehension Test and Relevant Scales ....................... 4

1.2.1 WE Listening Comprehension Test ......................................... 4

1.2.2 Relevant Scales ........................................................................ 5

1.2.2.1 Exposure Scale ............................................................ 5

1.2.2.2 Attitude Scale .............................................................. 6

1.2.2.3 Self-report Scale .......................................................... 6

1.2.2.4 Evaluation Scale .......................................................... 7

1.3 Purposes of This Study ........................................................................... 7

2. METHODS (STUDY1) ................................................................................ 10

2.1 Participants ............................................................................................. 10

vi

2.2 Materials.................................................................................................. 10

2.3 Design and Procedure ............................................................................. 12

2.4 Data Analyses ......................................................................................... 13

3. RESULTS (STUDY 1) ................................................................................. 14

3.1 Data Screening ........................................................................................ 14

3.2 Validation of Scales ................................................................................ 16

3.2.1 Exposure Scale ................................................................................ 16

3.2.2 Attitude Scale .................................................................................. 17

3.2.3 Self-report Scale.............................................................................. 18

3.2.4 WE Listening Comprehension Test ................................................ 19

3.2.5 Evaluation Scale.............................................................................. 20

3.4 Correlation among Scales ....................................................................... 21

3.5 Effects of Demographics on WE Listening Comprehension Test .......... 23

4. DISCUSSION (STUDY1)............................................................................ 24


4.2 Correlation among Scales ....................................................................... 25

4.3 The Effects of Demographics on WE Listening Comprehension Scale . 25

4.4 Limitations of Study 1............................................................................. 26

5. INTRODUCTION (STUDY 2) .................................................................... 28

6. METHODS (STUDY 2) .............................................................................. 29

6.1 Participants.............................................................................................. 29

6.2 Materials.................................................................................................. 29

vii

6.3 Design and Procedure ............................................................................. 30

6.4 Data Analyses ......................................................................................... 30

7. RESULTS (STUDY 2) ................................................................................ 31

7.1 Data Screening ........................................................................................ 31

7.2 Effects of demographics on WE Listening Comprehension Scale ......... 33


7.3.1 Exposure Scale ................................................................................ 34

7.3.2 Self-report Scale.............................................................................. 36

7.3.3 Evaluation Scale.............................................................................. 37

7.3.4 WE Listening Comprehension Test ................................................ 38

7.3.5 SE Listening Comprehension Test .................................................. 40

7.4 Correlation Analyses among Scales........................................................ 41

8. DISCUSSION (STUDY 2)............................................................................ 42


8.2 Correlation Analyses Among Scales....................................................... 42

8.3 Effects of Demographics on WE Listening Comprehension Test .......... 43

9. GENERAL DISCUSSION ............................................................................ 44

9.1 Conclusions, Limitations, and Future Directions.................................... 44

Appendix

A. WE LISTENING COMPREHENSION SCALE FOR STUDY 1................ 46

B. WE LISTENING COMPREHENSION SCALE FOR STUDY 2............... 56

C. WE CONVERATION SCRIPTS................................................................. 68

viii

REFERENCES.......................................................................................................... 75

BIOGRAPHICAL INFORMATION........................................................................ 79

ix

LIST OF ILLUSTRATIONS

Figure Page

1.1 Sensitive period for the second language learning .......................................... 3

3.1 PA for exposure scale ...................................................................................... 17

3.2 PA for self-report scale.................................................................................... 18

3.3 PA for WE listening comprehension test ........................................................ 20

3.4 PA for evaluation scale ................................................................................... 21

4.1 Distribution of WE listening comprehension test scores ................................ 26

7.1 PA for exposure scale ...................................................................................... 35

7.2 PA for self-report scale.................................................................................... 36

7.3 PA for evaluation scale.................................................................................... 38

7.4 PA for WE listening comprehension test ........................................................ 39

x

LIST OF TABLES

Table Page

3.1 Exposure scale.................................................................................................. 14

3.2 Attitude scale.................................................................................................... 15

3.3 Self-report scale................................................................................................ 15

3.4 WE Listening Comprehension Scale................................................................ 15

3.5 Evaluation scale................................................................................................ 16

3.6 Factor analysis for the exposure scale.............................................................. 17

3.7 Factor analysis for self-report scale ................................................................. 19

3.8 Factor analysis for WE listening comprehension test ...................................... 20

3.9 Factor analysis for evaluation scale ................................................................. 21

3.10 Table of descriptive statistics ......................................................................... 22

3.11 Correlation among the scales ......................................................................... 22

3.12 Effects of demographic data on WE

listening comprehension scale ........................................................................ 23

7.1 Exposure scale.................................................................................................. 31

7.2 Self-report scale................................................................................................ 32

7.3 Evaluation scale................................................................................................ 32

7.4 WE listening comprehension scale .................................................................. 32

7.5 WE listening comprehension scale when only

native speakers were included ......................................................................... 33

7.6 SE listening comprehension test ...................................................................... 33

xi

7.7 Effects of demographics on listening scale ...................................................... 34

7.8 Factor analysis for exposure scale.................................................................... 35

7.9 Factor analysis for self-report scale ................................................................. 37

7.10 Factor analysis for evaluation scale ............................................................... 38

7.11 Factor analysis for WE listening comprehension test .................................... 40

7.12 Table of descriptive statistics ......................................................................... 41

7.13 Correlation among the scales ......................................................................... 41

1

CHAPTER 1

INTRODUCTION

1.1 Standard English and World English

English is spoken as the native language by 3.6 hundred million people and

spoken as the second language by 1-1.5 billion speakers (Crystal, 2000). As a

consequence, English is regarded as the most widely used international language. For

example, factors such as movies, international travel, the internet, and recent

developments in computer networks have all further widened the use of English.

According to Kachru (1985), there are more non-native English speakers than native

ones. Even in English speaking countries, such as the United States, there are

increasing numbers of non-native English speakers because of increasing numbers of

immigrants (Flowerdew & Miller, 1997). Non-native speakers are defined as “people

who already use at least one other language and who live in a community in which

English is not normally used” (Hinkel, 2005).

1.1.1 World English (WE)

English languages can be categorized into three main types according to the

degree of nativity (Hinkel, 2005). The first type of English is Standard English (SE),

which is used in English speaking countries such as the U.S., the UK, Canada,

Australia, and New Zealand. This is commonly referred to as normative, “proper”,

2

“correct”, or “the King’s (Queen’s) English”. A second type of English is non-Standard

English, which is used in countries such as former colonies of English speaking

countries where English is the official or second language but not the dominant one.

There is debate whether specific kinds of native English, such as Ebonics, are standard

or nonstandard. A third type of English is non-native English. Non-native English is

used in countries where English is not official, dominant, or the first language. These

three types of English languages are collectively called World English (WE) as opposed

to SE, which includes only the first type of English. For example, WE speakers may

say “He like you car” instead of “He likes your car.” WE speakers also have relatively

strong accents derived from their dominant language that differ from those used by SE

speakers. They further use a different accent and are less fluent in English than they are

in their dominant language. Another characteristic of both non-standard and non-native

English is the intrusion of vocabulary from the more dominant language. For example,

I, as a native Japanese speaker, tend to say “hai” instead of “yes” (“hai” means yes in

Japanese).

1.1.2 Reasons to Study WE

Several studies reported that effective oral communication depends on the

mutual cooperation of speakers and listeners (Ervin-Tripp, 1970; Grice, 1975).

Although effective communication depends on mutuality, little attention has been paid

to the enhancement of native English speaker’s WE skills (Davies, Hamp-Lyons, &

Kemp, 2003). In other words, the standard of English proficiency in international

settings is based only on SE (Chalhoub-Deville & Wiggleworth, 2005). For example,

3

non-native English speakers are often required to take an examination to qualify for

academic admissions or for employment in English speaking countries. Even in non-

English speaking countries, many people are required to take English proficiency

examinations such as TOEFL when there are academic or business demands to use

English (Davies, Hamp-Lyons, & Kemp, 2003).

The efforts of non-native speakers to improve proficiency can be a potential

cause of miscommunication in various international settings in domains such as

conference, academia, politics, and business (Davies, Hamp-Lyons, & Kemp, 2003).

Further, it is unrealistic for non-native English speakers to achieve the SE fluency of

native speakers (Bouton, 1994; Snow, 1998). Johnson & Newport (1989) reported a

learning curve that describes the relationship between age of arrival to English speaking

country and the test of English grammar score suggested that the age of learning a

second language is important factor (Figure 1.1).

200

210

220

230

240

250

260

270

280

290

native 3 to 7 8 to 10 11 to 15 17 to 39

Age or arrival

Test of English Grammar Score

Figure 1.1Sensitive period for the second language learning

4

That implies that unless a non-native English speaker is exposed to English in early age,

it is hard for him/her to ever achieve SE level of fluency. Also, and perhaps most

importantly, the listening comprehension of native English and non-native speakers is

influenced by the type of English (Major, et al, 2005). Thus, SE is appropriate where

English is normally used while WE is more appropriate and practical in international

settings (Davies, Hamp-Lyons, & Kemp, 2003).

1.2 WE Listening Comprehension Test and Relevant Scales

Despite the importance of WE, it has received very little attention (Davies,

Hamp-Lyons, & Kemp, 2003). As a result, there is not a widely used WE assessment

test. Accurate and valid measurement of WE should be developed in order to enhance

international communication. The present research investigated listening

comprehension and four other related domains of WE proficiency.

1.2.1 WE Listening Comprehension Test

The WE listening comprehension scale was modeled upon a widely used SE

listening comprehension test in the U.S., the Test of English as a Foreign Language

(TOEFL). The TOEFL is developed and administered by Educational Testing Service

(ETS), and according to Bollag (2005) TOEFL “is taken by most foreign students who

are being considered for admission at English-speaking higher-education institutions,

including those in the United States. Last year (2004) nearly 750,000 students took the

test.” In terms of predictive validity (Rosenfeld, Oltman & Sheperd, 2004) and

construct validity (Powers et. al, 1999; Oltman, Stricker, & Barrows, 1990), TOEFL

appears to be a valid measure of English fluency. The TOEFL listening test is

5

composed of questions in which examinees are given native speakers’ conversations,

monologues, and sentence speeches. Respondents are instructed to pick the best

answers based on multiple-choice questions. Similarly, the WE listening

comprehension test is composed of questions in which conversations between a native

Japanese speaker speaking English and an American speaker, monologues of a Japanese

English speaker, and sentence-speeches of a Japanese English speaker are used as the

test material.

1.2.2 Relevant Scales

In addition to the listening comprehension test, exposure to foreign cultures (the

exposure scale), attitudes toward non-native English speakers (the attitude scale), self-

reports of WE listening comprehension skill (the self-report scale), and evaluations of

non-native speaker’s English in the listening tests (the evaluation scale) were developed

to investigate the domains that are related to WE listening comprehension.

1.2.2.1 Exposure Scale

First, the exposure scale was designed to investigate the degree that exposure to

non-native speakers and non-American cultures related to WE listening comprehension

skills. In other words, this scale intends to measure the degree that the amount of

experience to WE influences on the WE listening comprehension test score. Since

Johnson & Newport (1989) suggested that the amount of exposure to spoken language

should not be highly correlated with linguistic skill, amount of exposure to WE should

not be highly correlated with listening performance.

6

1.2.2.2 Attitude Scale

Second, the attitude scale was designed to measure attitudes toward non-native

speakers and cultures related to WE listening comprehension skills. Although some

may believe that attitudes of native speakers toward non-native speakers influence

native speakers’ listening comprehension, Lindemann (2002) failed to find such a

relationship. The present scale was used to reexamine this hypothesis.

1.2.2.3 Self-report Scale

Third, a self-report scale directly asked respondents to judge participants’ own

WE fluency in a similar manner to other metacognitive scales (Metcalfe, 1986;

Nunnally & Bernstein, 1994). This scale was designed to investigate accuracy of self-

perceptions of WE fluency. The scale investigated the following basic conversational

components of WE comprehension in 5-point Likert scales. This scale first prompted

respondents for their self-perceptions of their own WE comprehension skills. It then

assessed the degree and frequency of difficulties in comprehending elements of speech

such as accents (where you stress in a word or a sentence), pronunciation (how you say

a word), and grammar (rules of a language). These factors were included in the scale

because Harris (1994) suggested that the syntactical structure (grammar) and

phonological structures (accent and pronunciation), and the combination of said (overall

fluency) are important factors in the comprehension of spoken English. Various studies

have failed to find a correlation between self-efficacy and actual performance in various

cognitive domains such as memory tasks (Richard, Diefendorff, & Martin, 2006,

7

Shameem, 1998). This current study investigated whether metacognition (self-report) is

related to WE listening comprehension performance.

1.2.2.4 Evaluation Scale

Finally, the scale for the evaluation of non-native speaker’s fluency,

pronunciation, accent, grammar, and the clarity of the speech in the listening

comprehension part was designed to measure whether evaluation of non-native speakers

was associated with examinees comprehension skill.

1.3 Purposes of This Study

Listening and related components of WE were also investigated using standard

classical test theory measures in 2 studies. Study 1 was conducted as a pilot study for

the study 2. These studies involved determining Cronbach’s αof the exposure scale,

attitude scale, self-report scale, evaluation scale, and the WE listening comprehension

test and item/total correlations for the constituent items (rit). Cronbach’s alpha is a

statistical index of internal consistency that is determined by the correlations of items

and the numbers of items in a scale. Higher α implies the higher internal consistency.

It has been suggested that an acceptable scale should produce a Cronbach’s alpha larger

than .7, and each rit should be larger than .2. in its target population (Nunnally &

Bernstein, 1994). Second, exploratory principal component analyses should suggest

that the scales are unidimensional using eigenanalysis and parallel analysis to establish

a baseline for the assessment of dimensionality (Bernstein et al., 2007). As for

principal component factor analyses, a table for each result presented factor pattern (=

factor structure in the following analyses) and communalities (h2). Factor pattern is a

8

coefficient that describes the relationship between a factor and an item. Higher

coefficient value implies the higher relationships. Communality is the sum of the

squared factor pattern for each item. The larger communality implies that the amount

of variance in an item is explained by a particular factor. According to Bernstein et al

(2007):

In PA, one factors a matrix containing the same number

of observations and variables as the real data… but

simulated from a population in which all correlations are

zero. Multiple matrices of this form may be generated.

The results are averaged to provide for a more stable

estimate and standard errors. In order for s solution to

conform to a unidimensional solution, the first principal

component obtained from the real data must be larger

than the first principal component obtained from the

simulation, but all subsequent components from the actual

data must be smaller than their simulated counterparts.

Third, correlations among the exposure scale, the self-report scale, the evaluation scale,

and the listening comprehension test were analyzed. Fourth, the effects of demographic

variables such as gender, nativity/non-nativity of English, and age upon the scales were

also obtained.

9

Both studies to be reported were approved by the University of Texas at

Arlington’s Institutional Review Board. Study 1 was a pilot study designed to provide a

working set of scales.

10

CHAPTER 2

METHODS (STUDY 1)

2.1 Participants

Thirty two students enrolled in introductory psychology at the University of

Texas at Arlington were recruited. The data from one participant was omitted because

that person did not finish the experiment. The sample consisted of 6 males and 29

females, with a mean age of 20.4 years (SD = 4.21). There were 27 SE speakers and 5

non-native speakers. They voluntarily participated in preliminary research in order to

fulfill a course requirement.

2.2 Materials

The first part of all the scales contained demographic information (age, gender

and country of birth). Second, the degree of exposure to foreign cultures (exposure),

attitudes toward non-native English speakers and cultures (attitude), self reports of WE

listening comprehension skill (self-report), Listening comprehension tests (WE test),

and evaluations of non-native speaker’s English in the listening tests (evaluation) were

designed to investigate the domains possibly related to WE listening comprehension.

These items may be found in Appendix A.

a. Scales that preceded the WE listening comprehension scale:

1) Demographic data

11

2) The exposure scale (Part A of Appendix A): measured the degree of

exposure to non-native speakers. In order to eliminate problems

associated with the exposure scale, unit response to items were

standardized.

3) Attitudes toward non-native English speakers were assessed using

five-point Likert scales (Part B of Appendix A).

4) Self-perceptions of WE comprehension skill and the degree and

frequency of difficulties in comprehending elements of speech (Part C of

Appendix A) measured in 5 point Likert scale such as accents (the

position of stressed syllable in a word), pronunciation (the way of

vocalizing a word), and grammar (rules of a language).

b. A scale the presented after the completion of the WE listening comprehension

scale

The evaluation scale (Part H of Appendix A) measured participants’

evaluations of the non-native speaker’s fluency in a conversation (Part D

of Appendix A).

As above noted, the listening comprehension scale of WE (Part D to G of

Appendix A) was modeled upon the TOEFL. In the TOEFL, examinees are presented

with a conversation by native English speakers in SE and asked to pick the best multiple

choice answers. Instead of native English speakers, a conversation between a male

native English speaker using SE and a Female whose native language was Japanese and

who spoke WE was used. Participants were asked comprehension questions using a 4-

12

alternative multiple-choice format. The Japanese speaker’s proficiency level was

certified as “Able to understand and use English well enough for everyday needs and

situations…overseas” by the Society for Testing English Proficiency Inc (STEP, 2004).

A digital audio file was used to reproduce the recorded conversation. Audio

presentation has been found to be a more effective and efficient assessment method of

listening comprehension than video (Read, 2002; Conian, 2001). Also, auditory

presentations remove extraneous variables such as visual information out of the

procedure. Participants listened to digitally recorded conversations through

headphones, and they responded to paper and pencil version of the self-report scale and

listening comprehension scale.

2.3 Design and Procedure

After informed consent was collected, participants received instructions for the

experiment. Then, they proceeded to complete the demographic survey. After the

demographic survey, they responded to parts A (exposure), B (attitude), and C (self-

perceived competency) of the survey, in turn. They then proceeded to part D, where

they heard the conversation once. After listening to the conversation, they then

answered the listening comprehension test of the conversation. They then proceeded to

Parts E, F, and G in the identical manner as Part D. Participants debriefed and released

after completing part H. Thus, all participants went through identical procedures. The

procedure took about 25 to 35 minutes.

13

2.4 Data Analyses

Listening and related components of WE were also investigated using standard

classical test theory measures in 2 studies. This involved determining the internal

consistency (Cronbach’s α) of the exposure scale, attitude scale, self-report scale,

evaluation scale, and the WE listening comprehension test and item/total correlations

for the constituent items (rit). It has been suggested that an acceptable scale should

produce a Cronbach’s alpha larger than .7, and each rit should be larger than .2. in its

target population (Nunnally & Bernstein, 1994). Second, exploratory principal

component analyses should suggest that the scales are unidimensional using

eigenanalysis and parallel analysis to establish a baseline for the assessment of

dimensionality (Bernstein et al., 2007). Third, correlations among the exposure scale,

the self-report scale, the evaluation scale, and the listening comprehension test were

analyzed. Fourth, the effects of demographic variables such as gender, nativity/non-

nativity of English, and age upon the scales were also obtained.

14

CHAPTER 3

RESULTS (STUDY 1)

3.1 Data Screening

The means and standard deviations for each item of each scale are presented in

Table 3.1 for the exposure scale, Table 3.2 for the attitude scale, Table 3.3 for the self-

report scale, Table 3.4 for the WE listening comprehension test, and Table 3.5 for the

evaluation scale. Each scale was subjected to component analysis. The results will be

presented to emphasize the loadings on the first (general) factor.

Table 3.1 Exposure scale

Variable N Mean SD rit

a1 24 48.9 114.7 0.55

a2 24 9.6 7.6 0.70

a3 24 3.8 1.6 0.73

a4 24 3.6 1.6 0.70

a5 24 5.2 8 0.48

a6 24 111.9 299.6 0.32

a7 24 3.4 1.4 0.62

a8 24 3.2 1.4 0.73

a10 24 3 1.6 0.83

Note: Item 9 was eliminated because it measured the degree

of exposure to written language rather than spoken language.

15

Table 3.2 Attitude scale

After revision


b1 31 3.4 0.76 0.38

b2 31 4.6 0.62 0.37

b3 31 4.3 0.92 0.36

b4 31 2.8 1.46 0.32

b5 31 4.3 1.07 0.38

b6 31 3.8 1.19 0.31

b7 31 3.4 1.3 0.2

b8 31 2.6 1.2 0.28

Table 3.3 Self-report scale


c1 32 3.7 0.78 0.48

c2 32 3.4 0.94 0.65

c3 32 3.4 1.01 0.61

c4 32 3.3 0.92 0.41

c5 32 3.3 0.87 0.58

c6 32 3.1 1.00 0.75

c7 32 3.2 0.91 0.55

c8 32 3.4 1.07 0.65

c9 32 3.2 1.03 0.67

Table 3.4 WE listening comprehension scale After Deletion


t11 32 0.50 0.51 0.47

t12 32 0.97 0.18 0.53

t13 32 0.97 0.18 0.53

t15 32 0.75 0.44 0.44

t18 32 0.44 0.50 0.33

t24 32 0.63 0.50 0.28

t27 32 0.84 0.37 0.47

t29 32 0.78 0.42 0.41

t30 32 0.75 0.44 0.40

16

Table 3.5 Evaluation scale


h1 30 4.0 0.93 0.67

h2 30 3.6 0.86 0.70

h3 30 3.1 0.90 0.66

h4 30 3.6 0.89 0.65

h5 30 3.6 0.77 0.60

3.2 Validation of Scales

3.2.1 Exposure Scale

In order to eliminate problems associated with scale units, response to items

were standardized. The internal consistency of the 10-item Exposure scale was high

(Cronbach’s α = .88, n = 24, Table 3.1). Item 9 was eliminated form the analysis

because it was more related to written than spoken English. Consequently it was

dropped from the scale, leaving 9 items (Cronbach’s α = .88, n = 24, Table 3.1). An

exploratory principal component factor analysis was performed. The scale was

unidimensional based upon a parallel analysis criterion (see figure 3.1), and the

resulting pattern appears in Table 3.6.

17

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

1 2 3 4 5 6 7 8 9

Number of factors

Eigen Value

Scree Plot

Randomized data

Figure 3.1 PA for the exposure scale

Table 3.6 Factor analysis for exposure scale

Factor Pattern Communalities

Items Factor 1 h2

1 0.65 0.54

2 0.77 0.60

3 0.81 0.74

4 0.79 0.72

5 0.59 0.83

6 0.36 0.84

7 0.71 0.54

8 0.81 0.69

10 0.89 0.81

3.2.2 Attitude Scale

The internal consistency of the 10-item Attitude scale was low (α = .54, n = 31).

Items 9 and 10 had unacceptable values of rit (-.13 and .01, respectively) so were

18

eliminated. However, the internal consistency remained poor (α = .61, Table 3.2) after

this deletion, the scale as a whole was eliminated. Consequently, the results of factoring

the scale will not be presented.

3.2.3 Self-report Scale

The internal consistency for the 9-item self-perceived competency scale was

high (α = .86, n = 32, Table 3.3). Item level analysis based on rit suggested all items

were related to the scale (Table 3.3) so all original items were retained. An exploratory

principal component factor analysis was performed. The result was that the scale was

unidimensional based upon a parallel analysis criterion (see figure 3.2), and the

resulting factor pattern appears in Table 3.7.

Parallel analysis

0

1

2

3

4

5

1 2 3 4 5 6 7 8 9

Number of factors

Eigen value

Scree Plot

Randomized data

Figure 3.2 PA for self-report scale

19

Table 3.7 Factor analysis for self-report scale


Items Factor 1 h2

1 0.59 0.66

2 0.73 0.70

3 0.72 0.80

4 0.51 0.91

5 0.66 0.83

6 0.83 0.74

7 0.66 0.60

8 0.75 0.80

9 0.76 0.85


In order to achieve the final version of scale, there were four iteration processes.

This iteration process continued until the both the criterion for the Cronbach’s α =.7 and

rit = .2. First, item 8 that had no variance were eliminated (eg. Most participants

answered that question correctly). Then, items that had lower than rit =.2 were

eliminated in each step. In the step 2, items 6, 7, 20, 22, 26, and 28 were eliminated. In

the third step, items 1, 3, 4, 9, 10, 14, 16, 17, 19, and 25 were eliminated. In the fourth

step, items 2, 5, and 23 were eliminated. The final version of the WE listening test

retained 9 items with moderate internal consistency (α = 72, n = 32, Table 3.4). An


unidimensional based on parallel analysis criterion (Figure 3.3) and the resulting factor

pattern appears in Table 3.8.

20

Parallel analysis

0

0.5

1

1.5

2

2.5

3

3.5

1 2 3 4 5 6 7 8 9

Number of factors

Eigne value

Scree plot

Randomized data

Figure 3.3 PA for WE listening comprehension test

Table 3.8 Factor analysis for WE listening comprehension test


Items Factor 1 h2

11 0.52 0.63

12 0.82 0.96

13 0.82 0.96

15 0.58 0.38

18 0.43 0.68

24 0.41 0.56

27 0.63 0.48

29 0.56 0.44

30 0.53 0.59

3.2.5 Evaluation Scale

The internal consistency of the 5-item evaluation scale was high (α = .85, n= 30, Table

5). Each item met the rit = .2 criterion, so all items were retained. An exploratory

principal component factor analysis was performed. The scale was unidimensional

21

based on parallel analysis criterion (Figure 3.4), and the resulting factor pattern appears

in Table 3.9.

Parallel analysis

0

0.5

1

1.5

2

2.5

3

3.5

1 2 3 4 5

Number of factors

Eigen value

Scree plot

Randomized data

Figure 3.4 PA for evaluation scale

Table 3.9 Factor analysis for evaluation scale


Items Factor 1 h2

1 0.80 0.63

2 0.82 0.67

3 0.80 0.64

4 0.78 0.61

5 0.74 0.55

3.3 Correlation among Scales

After the validation of scales, data screening for correlation analyses among

scales was performed. Descriptive statistics appears in table 3.10.

22

Table 3.10 Tables of descriptive statistics

Scale mean SD Skewness Kurtosis n

age 20.34 4.21 3.17 12.24 32

standardized logexp 0 1 1.67 3.28 32

self 29.94 5.92 0.18 1.2 32

eval 17.59 3.57 0.18 -0.64 32

WE 6.63 2.04 -1.31 2.72 32

Table 3.11 contains the scale intercorrelations. As can be seen, the correlation

between the self-perceived competency scale and the listening test was very low.

However, there was a moderate positive correlation between the listening test and the

evaluation scale.

Table 3.11 Correlation among the scales Pearson r (n=32)

Variable WE Self Logexp Eval

WE __ -0.06 .10 0.45*

Self __ 0.15 0.14

Logexp __ -0.07

Eval __

Note: WE = WE listening comprehension scale, Self = Self report scale, Logxp =

standardized and log transformed Exposure scale, and Eval = the Evaluation scale.

*P<.05

23

3.4 Effects of Demographics on WE Listening Comprehension Test

Demographics variables such as gender, native/non-nativity of English, and age

were unrelated to both the listening test (Table 3.12).

There were no trends of note the mean for males was 6.50 and the mean for

females was 7.16, and effect size was small, Cohen's d = .28. It would require about

320 participants to achieve a significant difference with α = .05, and power = .80. The

mean for native English speakers was 6.59 and mean for non-native speakers was 6.80,

and effect size was very small, Cohen's d = .01. It would require about 2874

participants to achieve significant difference with α = .05, and power = .80.

Table 3.12 Effects of demographic data on WE listening comprehension scale

ANOVA

source df F p

Age 1 1.68 0.21

Gender 1 0.65 0.43

C.O.B. 1 1.12 0.30

Error 28 (4.42)

Note: Values in parentheses are mean square errors

24

CHAPTER 4

DISCUSSION (STUDY 1)


The results of the pilot study were that all scales save for attitude met the usual

standards for reliability. Thus, attitude scale did not meet the statistical criterion. Also,

there was theoretical concern about the attitude scale. In other words, it was not clear

whether attitude scale was measuring actual attitude or overt response of participants.

For example, because of social desirability, participants might have reported socially

desirable attitude. Because of these reasons, attitude scale was eliminated from the

analyses. These other scales were then re-examined in a larger sample, but five changes

were made from study 1. First, item 9 was eliminated from the exposure scale, and all

items were changed to be measured in Likert scale. Second, the attitude scale was

eliminated due to low internal consistency as described above. Third, items 1, 2, 3, 4, 5,

6, 7, 8, 9, 10, 14, 16, 17, 19, 20, 21, 22, 23, 25, 26, 28 in the WE listening test were

modified (Appendix 2) if no one picked a particular multiple choice response or

everyone had the same response in a particular question. Therefore, either the questions

or multiple choice answers were modified. For example, multiple choice b) of items 2

in WE test was modified from “Father” to “Friend.” Also, items such as item 8 were

eliminated because everyone could answer that question correctly. “Father” was

25

changed to “Friend” because no one picked that answer. The 9-item exposure scale

(Part A of Appendix 2), the 9-item self-report scale (Part B of Appendix B), the 42-item

WE listening comprehension test (Part C to H of Appendix B), the 5-item English

listening comprehension scale (Part I of Appendix B), and the 5-item evaluation scale

(Part J of Appendix B) were also modified. Fourth, grammatical mistakes and typos

were corrected in the modified scales.


Exposure scale was transformed due to the high deviation from normality.

These previous analyses also suggested two other points: (a) perceived self-

competency was not significantly correlated with test performance, and (b) scores on

the listening test were positively correlated with the evaluation of the fluency of the

non-native speakers.

4.3 The Effect of Demographics on WE Listening Comprehension Test

Demographics such as gender and age did not affect listening test scores.

Notably, native English speakers did not outperform non-native speakers in the WE

listening comprehension scale. This contrasts with findings obtained using SE listening

comprehension scales such as the TOEFL (Stricker, 2004). However, because of the

imbalance in numbers of native and non-native group and, especially, the small sample

size, a null conclusion may be premature.

26

4.4 Limitations of Study 1

The present study has two obvious limitations. As noted, the sample size was

very small. Second, listening test scores may have been range restriction restricted

since all scores distributed fell between 20 to 30 points (Figure 4.1).

Distribution of Scores

0

1

2

3

4

5

6

7

19 20 21 22 23 24 25 26 27 28 29 30

Test Scores

Frequency

Figure 4.1 Distribution of WE listening test scores

This might reflect the sample or the test materials. A practical solution to this limitation

was to modify scales using a larger sample. Since Cronbach’s alpha is not an inferential

measure, it should not be affected by sheer sample size. However, Tabachnik and

Fidell (2001; 2007) suggest a minimal sample size of 90 for a multiple correlation

analysis of the present form (however, since SE listening comprehension test was added

27

later to test discriminant validity, sample size larger than 105 was necessary for the

study 2).

28

CHAPTER 5

INTRODUCTION (STUDY 2)

The purpose of the study 2 was to investigate the modified scales of study 1 in a

larger sample. An SE listening comprehension test was also added to investigate

discriminant validity. The analyses paralleled those of study 1.

29

CHAPTER 6

METHODS (STUDY 2)

6.1 Participants

One hundred and seven students enrolled in introductory psychology at the

University of Texas at Arlington were recruited. The data from one participant was

omitted because that person did not finish the experiment. The sample consisted of 67

females and 39 males, with a mean age = 20.0 years and SD = 2.91. There were 90 SE

speakers and 16 non-native speakers. They voluntarily participated in preliminary

research in order to fulfill a course requirement.

6.2 Material

Five changes were made from study 1, as previously noted. First, item 9 was

eliminated from the exposure scale, and all items were measured using a Likert scale

format. Second, the attitude scale was eliminated. Third, items 1, 2, 3, 4, 5, 6, 7, 8, 9,

10, 14, 16, 17, 19, 20, 21, 22, 23, 25, 26, 28 in the WE listening test were modified

(Appendix B). The modified versions of the 9-item exposure scale (Part A of Appendix

B), the 9-item self-report scale (Part B of Appendix B), the 42-item WE listening

comprehension test (Part C to H of Appendix B), the 5-item Standard English listening

comprehension scale (Part I of Appendix B), and the 5-item evaluation scale (Part J of

30

Appendix B) were also attached as appendix B. Fourth, grammatical mistakes and

typos were collected in the modified scales. Fifth, administration and data collection of

the scales was automated using the E-prime program.

6.3 Design and Procedure

After informed consent was collected, participants were assigned to a computer.

They were asked to start answering the demographic survey and modified exposure

scale, the self-report scale, the WE listening comprehension test, and the evaluation

scale. After completion, participants were debriefed and released. Thus, all participants

went through the identical procedure. The task took approximately 40-45 minutes.

6.4 Data Analyses

Data analyses for the study 2 was identical to the study 1 except that the attitude

scale was excluded for the study 2 and SE listening comprehension scale was added to

the study 2.

31

CHAPTER 7

RESULTS (STUDY 2)

7.1 Data Screening

The means and standard deviations for each item of each scale are presented in

Tables 7.1 to 7.6. Table 7.1 contains the exposure scale. Table 7.2 contains the self-

perceived competency scale. Table 7.3 contains the Evaluation scale. Table 7.4 contains

the WE listening comprehension test, and Table 7.5 contains WE listening test for only

native speakers. Table 7.6 contains SE listening comprehension scale. Severe violation

of assumptions was not detected while screening data.

Table 7.1 Exposure scale


a1 104 3.3 1.8 .69

a2 104 3.0 1.8 .71

a3 104 3.9 1.1 .71

a4 104 3.6 1.3 .56

a5 104 1.7 1.3 .52

a6 104 1.6 1.3 .30

a7 104 3.2 1.3 .72

a8 104 3.0 1.2 .64

a10 104 3.0 1.3 .64

32

Table 7.2 Self-report scale Variable N Mean SD rit

b1 105 3.4 .86 0.67

b2 105 3.1 .87 0.79

b3 105 3.2 .93 0.72

b4 105 3.1 .94 0.65

b5 105 3.2 .90 0.69

b6 105 3.0 .91 0.79

b7 105 3.1 .93 0.83

b8 105 3.1 .92 0.69

b9 105 3.2 .93 0.70

Table 7.3 Evaluation scale Variable N Mean SD rit

j1 106 2.9 0.90 0.73

j2 106 2.7 0.82 0.74

j3 106 2.6 0.74 0.53

j4 106 3.0 0.79 0.47

j5 106 2.6 0.90 0.70

Table 7.4 WE listening comprehension scale Variable N Mean SD rit

t1 106 .86 .35 .40

t2 106 .88 .33 .36

t3 106 .44 .50 .25

t4 106 .51 .50 .29

t8 106 .92 .30 .24

t9 106 .74 .28 .30

t12 106 .72 .44 .23

t13 106 .72 .45 .23

t14 106 .34 .48 .29

t15 106 .91 .29 .25

t16 106 .75 .43 .21

t21 106 .68 .47 .21

t30 106 .87 .34 .23

t31 106 .52 .50 .27

t33 106 .41 .49 .29

t34 106 .55 .50 .30

t35 106 .74 .44 .29

t37 106 .75 .44 .30

t41 106 .42 .50 .29

33

Table 7.5 WE listening comprehension scale when only native speakers were included Variable N Mean SD rit

t1 90 .85 .35 .43

t2 90 .88 .32 .39

t3 90 .44 .50 .32

t4 90 .50 .30 .31

t8 90 .73 .28 .26

t9 90 .61 .44 .29

t12 90 .72 .45 .22

t13 90 .34 .45 .30

t14 90 .91 .48 .31

t15 90 .75 .29 .28

t16 90 .60 .43 .24

t21 90 .97 .47 .25

t30 90 .52 .34 .22

t31 90 .41 .50 .27

t33 90 .55 .49 .32

t34 90 .74 .50 .27

t35 90 .42 .44 .28

t37 90 .42 .44 .32

t41 90 .50 .50 .32

Table 7.6 SE listening comprehension test


item 1 106 .65 0.48 0.19

item 5 106 .58 0.50 0.19

7.2 The Effects of Demographics on WE Listening Comprehension Test

Demographic variables such as gender, native/non-nativity of English, and age

again did not related to scores on the WE listening test. As for the effect on WE test,

there were no trends of note the mean for males was 13.72 and the mean for females

was 13.52, and effect size was very small, Cohen's d = .059. It would require about 480

participants to achieve significant difference with α = .05, and power = .80. There were

34

no trends of note the mean for native English speakers was 13.64 and mean for non-

native speakers was 13.31, and effect size was very small, Cohen's d = .097. It would

require about 9000 participants to achieve significant difference with α = .05, and

power = .8. Since no significant effect of nativity on any scale based on one way

ANOVAs, data from both native and non-native participants were pooled (Table 7.7).

Table 7.7 Effects of demographics on listening scale

ANOVA

source df F p

Age 1 .57 0.45

Gender 1 .00 0.99

C.O.B. 1 .17 0.68

Error 100 (11.56)

Note: Values in parentheses are mean square errors


7.3.1 Exposure Scale

The result suggested moderately high internal consistency of the 9-item

Exposure scale (Cronbach’s α = .87, n =106. Item level analysis based on rit suggested

all items were related to the scale (Table 7.1) so all original items were retained. An

exploratory principal component factor analysis was then performed. The scale was

unidimensional based on parallel analysis criterion (Figure 7.1), and the resulting


35

Parallel analysis

0

1

2

3

4

5

1 2 3 4 5 6 7 8 9

Number of factors

Eigen value

Scree plot

Randomized

Figure 7.1 PA for exposure scale

Table 7.8 Factor analysis for exposure scale


Items Factor 1 h2

1 .77 .60

2 .80 .64

3 .79 .69

4 .67 .56

5 .61 .76

6 .36 .66

7 .80 .66

8 .75 .56

10 .74 .54

36

7.3.2 Self-report Scale

The result suggested an acceptably high internal consistency for the 9-item self-

perceived competency scale (α = .92, n = 106). Item level analysis based on rit

suggested all items were related to the scale (Table 7.2), so all items were retained. An


unidimensional based on parallel analysis criterion (Figure 7.2), and the resulting


Parallel analysis

0

1

2

3

4

5

6

1 2 3 4 5 6 7 8 9

Number of factors

Eigen value

Scree plot

Randomized

Figure 7.2 PA for self-report scale

37

Table 7.9 Factor analysis for self-report scale


Items Factor 1 h2

1 .75 .56

2 .84 .71

3 .79 .62

4 .72 .52

5 .76 .57

6 .85 .72

7 .88 .77

8 .76 .57

9 .76 .58

7.3.3 Evaluation Scale

The result suggested moderately high internal consistency of the 5-item evaluation

scale (α = .83, n = 106). Item level analysis based on rit suggested all items were related

to the scale (Table 7.3), so all items were retained. The scale was unidimensional based

on parallel analysis criterion (Figure 7.3), and the resulting pattern appears in Table

7.10.

38

Parallel analysis

0

0.5

1

1.5

2

2.5

3

3.5

1 2 3 4 5

Number of factors

Eiigen value

Scree Plot

Randomized

Figure 7.3 PA for Evaluation scale

Table 7.10 Factor analysis for evaluation scale


Items Factor 1 h2

1 .85 .72

2 .86 .74

3 .69 .48

4 .62 .38

5 .83 .69


In order to achieve the final version of scale, there were four iteration processes.

This iteration process continued until the both the criterion for the Cronbach’s α =.7 and

rit = .2. First, items (6, 18, 23, 26, 28, 29, 32, 38, and 40) that had very low variance

were eliminated (eg. Everyone answered that question correctly). Then, items that had

lower than rit =.2 were eliminated in each step. In the second step, items 7, 10, 11, 17,

39

19, 20, 22, 24, 25, 26, 27, and 36 were eliminated. In the third step, item 43 was

eliminated. After the elimination, 20 out of 42 items were retained. The result

suggested moderate internal consistency (α = 71, n = 106). Item level analysis based on

rit suggested all retained items were related to the scale (Table 7.4). Internal consistency

of the WE listening comprehension scale for native speakers showed a similar result,

one that was not discussed here (Table 7.5). An exploratory principal component factor

analysis was performed. The scale was unidimensional based on parallel analysis

criterion (Figure 7.4), and the resulting pattern appears in Table 7.11.

Parallel analysis

0

0.5

1

1.5

2

2.5

3

3.5

1 2 3 4 5 6 7 8 9

Number of factors

Eigne value

Scree plot

Randomized data

Figure 7.4 PA for WE listening comprehension test

40

Table 7.11 Factor analysis for WE listening comprehension test


Items Factor 1 h2

1 .54 .66

2 .52 .63

3 .37 .62

4 .40 .77

5 .36 .52

8 .43 .75

9 .36 .54

12 .53 .65

13 .32 .71

14 .38 .66

15 .34 .70

16 .30 .55

21 .30 .66

30 .31 .61

31 .37 .52

33 .39 .58

34 .42 .67

35 .44 .73

37 .43 .58

41 .39 .65

7.3.5 SE Listening Comprehension Test

There were originally 5 items on this scale, but only 2 items were retained due

to low internal consistency (α = .31, n =106, Table 7.6). Due to low internally

consistency and small number of items, meaningful analyses was not able to be

performed further. Low internal consistency seemed to be caused by both low

correlations among items and very small number of items in the scale. Based on the

Spearman-Brown prophecy formula, about 11 more items were needed to have a

reliability of .70.

41


Descriptive statistics after the validation was shown in Table 7.12.

Table 7.12 Table of descriptive statistics

Scale mean SD Skewness Kurtosis

age 19.96 2.01 2.64 9.64

exp 26.18 8.86 0.09 -1.09

self 23.07 2.41 -0.69 0.026

eval 13.8 3.22 0.26 1.24

WE 13.6 3.38 -0.37 -0.33

SE 1.23 0.75 -0.42 -1.11

Results from correlation analyses (Table 7.13) suggested that there were a

medium positive correlation between the WE listening comprehension test and English

listening comprehension test, between the WE listening comprehension test and self-

report, and the listening test and the evaluation scale. The significant correlation

between WE test and self report was unexpected based on the study 1.

Table 7.13 Correlation among the scales Pearson r (N=106)

variable WE SE Self exp eval

WE __ 0.29** 0.36*** 0.02 0.22*

SE __ 0.05 0.03 -0.14

self __ -0.06 0.13

exp __ 0.09

eval __ ___

Note: WE = WE listening comprehension scale, SE = Standard English listening

comprehension test, Self = the Self report scale, Exp = the Exposure scale, and Eval =

the Evaluation scale. *P<.05, **P<.01, ***P<.001

42

CHAPTER 8

DISCUSSION (STUDY 2)


Except for SE test, results from the present study indicated moderate to

moderately high internal consistency for every scales. Based on the Spearman-Brown

prophecy formula, about 11 more items were needed to have a reliability of .70 for SE

test. Principal component analyses suggested the unideminensionality of these scales.


Correlation analyses suggested that the self-report scale was significantly

correlated to performance. That implies that the self-report reflects the WE listening

scale. This conflicts with the findings from Study 1. However, self-report is still far

from accurate measure of WE listening skill because it accounted for only 13% of

variance of the WE test (R2 (106)= .13). Second, those who had higher scores on the

listening test evaluated the fluency of the non-native speaker in the test better than those

who scored lower on the listening test as was the case in Study 1. Significant positive

correlation between WE listening test performance and self-report scale seems to be

incompatible with those of the study 1. That might be attributed to the relatively

43

unreliable WE scale in Study 1. The fact that WE scale in study 2 appeared to be better

scale might be attributed to the modification of items and larger numbers of items in the

scale.

8.3 Effects of Demographics on WE Listening Comprehension Scale

Based on the ANOVA and power analyses, demographics such as gender, age,

and nativity of English did not have a significant effect on listening test scores. Indeed,

effect sizes of demographics were very small. Notably, native speakers did not

outperform non-native speakers in the WE listening comprehension scale as compared

with other SE listening comprehension scales such as the TOEFL (Stricker, 2004) as the

Study 1. However, because of the unequal number of participants in the native (90

participants) and non-native (16 participants) groups, this finding was not strongly

generalizable.

44

CHAPTER 9

GENERAL DISCUSSION

9.1 Conclusions, Limitations, and Future Directions

Because of internal consistency, unidimensionality, and a weak correlation

between the WE and the SE listening scale, WE listening comprehension scale appears

to be valid test to measure WE listening proficiency. However, some modifications

should be considered in further investigations. First, more items should be included in

the WE listening scale in order to make it more useful and reliable. Second, the SE test

should include more items to perform meaningful analyses. Third, a more basic level of

WE such as a word level or phoneme level should be analyzed in the investigation of

WE comprehension because it was difficult to analyze data in the conversation level.

Fourth, the low correlation between the exposure scale and WE listening test may be

caused by the construction of the exposure scale. In other words, the exposure scale

measured exposure to non-English speaking culture in general. Since this current study

included only a Japanese speaker in WE listening scale as a non-native English speaker,

exposure to Japanese English speakers rather than exposure to general non-native

speakers should be measured. Fifth, it will be interesting to use self-report scale after

the WE listening test as well as prior to WE listening comprehension test. Finally,

45

participants may have guessed what the non-native speaker said by using native

speaker's speech as contextual cue in the conversation sections. Thus, participants

might have used native speaker as contextual cue to answer the questions. Therefore,

listening skill to comprehend conversation between non-native speakers' English rather

than conversations between non-native and native speaker should be measured in

conversation section. Logically that will make the listening scale more internally

consistent because listening skill in the comprehension of only a non-native speaker is

measured in the monologue and the sentence sections as opposed to conversation part

that included a native speaker.

46

APPENDIX A

WE LISTENING COMPREHENSION SCALE IN STUDY 1

47

Please answer the following survey and the WE listening comprehension

scale as honestly and accurately as possible. Please answer all parts of the

following survey. In this survey, non-native English speaker is defined

anyone whose first or dominant languages is not English.

Demographic Survey

1. Age ( )

2. Gender ( )

3. Country of Birth ( )

Part A: Answer the following questions based on your experience.

1. How many non-native English speakers do you know?

( )

2. How long have you known them in years (list oldest ones)?

( ) years

3. How close you are to the one you consider yourself closest to?

Not close Very

At all Neutral Close

1 2 3 4 5

4. How often do you see that one person?

Very Very

Rarely Neutral Often

1 2 3 4 5

5. How many times have you visited countries in which English is not the dominant

language?

( )

6. How many days have you spent in countries in which English is not the dominant

language?

( )7. How familiar are you with non-American culture?

Very Very

Unfamiliar Average Familiar

1 2 3 4 5

48

8. How familiar are you with the conversation of non-native English speakers?

Very Very


1 2 3 4 5

9. How familiar are you with the writing of non-native English speakers?

Very Very


1 2 3 4 5

10. How fluent are you in a language other than English?

Not at Very

All Average Fluent

1 2 3 4 5

Part B: Imagine an average non-native English speaker, and answer

following questions.

1. Rate your attitude towards this person’s English fluency.

Not Very Very

Fluent Neutral Fluent

1 2 3 4 5

2. How favorable is your view of non-English cultures?

Very Neutral Very

Negative Positive

1 2 3 4 5

3. Do you want to go to a country in which English is not the native language?

Yes No

1 2 3 4 5

4. Do you want to live in a country in which English is not the native language?

Yes No

1 2 3 4 5

5. Are you interested in a non-American culture?

Yes No

1 2 3 4 5

49

6. Do you pay much attention to foreign cultures?

Yes No

1 2 3 4 5

7. When you do not understand a non-native speakers’ English, do you think that it is

usually something wrong with their fluency?

Yes No

1 2 3 4 5

8. When you do not understand a non-native speakers’ English, do you think that it is

usually something wrong with your listening skill?

Yes No

1 2 3 4 5

9. Do you regard Midwestern American or as the standard form of international

English?

Strongly Strongly

Disagree Neutral Agree

1 2 3 4 5

10. Do you regard British as the standard form of international English?

Strongly Strongly

Disagree Neutral Agree

1 2 3 4 5

Part C: Imagine an average non-native English speaker, and answer


1. How good are you at understanding the English of someone who is not a native

English speaker?

Very Very

Poor Average Good

1 2 3 4 5

2. On average, how difficult is it for you to understand the overall English of someone

who is not a native English speaker?

Very Very

Difficult Neutral Easy

1 2 3 4 5

50

3. How often do you find it difficult for you to understand the overall English of

someone who is not a native English speaker?

Very Not often

Often Neutral At all

1 2 3 4 5

4. On average, how difficult is it for you to understand the English grammar of


Very Very


1 2 3 4 5

5. How often do you find it difficult for you to understand the English grammar of


Very Not often


1 2 3 4 5

6. On average, how difficult is it for you to understand the English pronunciation of


Very Very


1 2 3 4 5

7. How often do you find it difficult for you to understand the English pronunciation

of someone who is not a native English speaker?

Very Not often


1 2 3 4 5

8. On average, how difficult is it for you to understand the English accent of someone


Very Very


1 2 3 4 5

9. How often do you find it difficult for you to understand the English accent of


Very Not often


1 2 3 4 5

51

Before you start Part D, please listen to conversation 1. Then please

answer following questions as accurately as you can.

1. What was the customer looking for?

a) T-shirt b) Statue of Liberty Models c) Some souvenir d) Pewter

model

2. For whom did the customer want the product?

a) Mother b) Father c) Grandmother d) Herself

3. Why did the customer not want the first product the clerk recommended?

a) She did not like the design.

b) She did not like the material.

c) She did not have enough money.

d) She did not think it’ appropriate.

4. Which product was the customer first interested in?

a) T-shirt b) Statue of liberty model c) Silver ring d) Chocolate gift

5. Which word did the customer not understand?

a) Statue b) Copper c) Plastic d) Pewter

6. Which kind of material of the product did the customer buy?

a) Wood b) Copper c) Plastic d) Pewter

7. What kind of wrapping did the customer want?

a) Birthday b) Gift c) Christmas d) Souvenir

8. Which country did the customer come from?

a) Japan b) Jordan c) Korea d) China

9. How many times has the customer visited the U.S.?

a) 1 b) 2 c) 3 d) 4

10. How long does the customer plan to stay in the U.S.?

a) 3 days b) 5 days c) 7 days d) More than 7 days

11. When will the customer leave?

a) Today b) Tomorrow c) 2 days from now d) Next week

12. Other than New York, where did the customer visit?

a) Los Angels b) Philadelphia c) Pittsburgh d) Washington D.C.

52

13. How long did the customer spend there?

a) A day b) 3 days c) A week d) 10 days

14. What was the customer’s purpose of visit to the U.S.?

a) Business b) To see her mother c) To study d) For fun

15. Why did the customer not take the tour?

a) She could not afford it. b) She did not like it.

c) She did not know about it. d) She did not have a chance.

16. How did the customer like New York?

a) She liked it. b) She did not like it.

c) She did not mention like or dislike. d) She liked D.C. better.

17. Which show did the customer watch?

a) Phantom of the Opera b) Miss Saigon c) Cats

d) She did not watch any.

18. Why did the customer go to Hard Rock Café?

a) It is her favorite place.

b) It was her first time to visit there.

c) She likes burger there.

d) She likes their goods.

Before you start Part E, please listen to Monologue 1. Then please answer

following questions as accurately as you can.

1. What is her occupation?

a) Instructor b) Counselor c) Professor d) Principal

2. Where did her husband come from?

a) Africa b) Japan c) Canada d) The U.S.

3. Why was she reading a book about cats?

a) She likes cats.

b) She is studying about cats.

c) It is her assignment.

53

d) She has a cat.

4. What is her dream?

a) To live in foreign countries b) To be a teacher

c) To marry her boy friend d) To write a book

Before you start Part F, please listen to Monologue 2. Then, please answer

following questions as accurately as you can.

1. According to the conversation speaker’s friend Yoko lives in.

a) America b) Russia c) Asia d) Europe

2. How does the speaker evaluate her friend Yoko?

a) She thinks Yoko is a great person.

b) She does not like Yoko.

c) Yoko is nice, but she has to develop more social skills.

d) Yoko cannot speak French.

3. What is the most difficult part of living in Switzerland?

a) She does not like winter.

b) She does not like the area where she lives.

c) Living costs are too expensive where she lives.

d) She does not like to work hard.

4.Which country did the speaker imply Yoko comes from?

a) Japan b) The U.S.A. c) China d) Switzerland

54

Before you start Part G, please listen to Sentences.

1. Why did she not go to the concert?

a) Tickets are too expensive

b) She did not like the band.

c) She did not have the time.

d) She could not get a ticket.

2. Why did she not go to John’s house?

a) It is too far for her. b) It is too cold.

c) She does not like John d) She does not have time to go.

3. How is she regarding Pennsylvania?

a) She is not there right now. b) She is there.c) She will live there. d) She

wishes to live there.

4. What does the sentence imply about Mary

a) One should feel sorry for her.

b) One should apologize to her.

c) One doesn’t need to apologize to her.

d) She should be ignored. .

Part H: Please answer the following questions.

1. Rate the speaker’s fluency in conversation 1.

Very Very

Poor Average Good

1 2 3 4 5

55

2. Rate the speaker’s pronunciation in conversation 1.

Very Very

Poor Average Good

1 2 3 4 5

3. Rate speaker’s accent in conversation 1.

Very Very

Poor Average Good

1 2 3 4 5

4. Rate the speaker’s grammar in conversation 1.

Very Very

Poor Average Good

1 2 3 4 5

5. Rate the speaker’s clarity in conversation 1.

Very Very

Poor Average Good

1 2 3 4 5

56

APPENDIX B

WE LISTENING COMPREHENSION SCALE IN STUDY 2

57

Appendix B: Modified WE Scales

Please answer the following survey and the WE listening comprehension

scale as honestly and accurately as possible. Please answer all parts of the

following survey. In this survey, non-native English speaker is defined

anyone whose first or dominant languages is not English.

Demographic Survey

1. Age ( )

2. Gender ( )

3. Country of Birth ( )

Part A: Answer the following questions based on your experience.

1. How many non-native English speakers do you know?

1) 1-3 2) 4-6 3) 7-9 4) 9-11 5) More than 11

2. How long have you known them in years (list oldest ones)?

1) 1-3 years 2) 4-6 years 3) 7-9 years 4) 9-11 years 5) More than 11

years

3. How close you are to the one you consider yourself closest to?

Not close Very

At all Neutral Close

1 2 3 4 5

4. How often do you see that one person?

Very Very

Rarely Neutral Often

1 2 3 4 5

5. How many times have you visited countries in which English is not the dominant

language?

1) 1-3 2) 4-6 3) 7-9 4) 9-11 5) More than 11

58

6. How many years have you spent in countries in which English is not the dominant

language?

1) 1-3 years 2) 4-6 years 3) 7-9 years 4) 9-11 years 5) More than 11

years

7. How familiar are you with non-American culture?

Very Very


1 2 3 4 5

8. How familiar are you with the conversation of non-native English speakers?

Very Very


1 2 3 4 5

9. How fluent are you in a language other than Standard English?

Not at Very

All Average Fluent

1 2 3 4 5

Part B: Imagine an average non-native English speaker, and answer


1. How good are you at understanding the English of someone who is not a native

English speaker?

Very Very

Poor Average Good

1 2 3 4 5

2. On average, how difficult is it for you to understand the overall English of someone


Very Very


1 2 3 4 5

3. How often do you find it difficult for you to understand the overall English of


Very Not often


1 2 3 4 5

59

4. On average, how difficult is it for you to understand the English grammar of


Very Very


1 2 3 4 5

5. How often do you find it difficult for you to understand the English grammar of


Very Not often


1 2 3 4 5

6. On average, how difficult is it for you to understand the English pronunciation of


Very Very


1 2 3 4 5

7. How often do you find it difficult for you to understand the English pronunciation

of someone who is not a native English speaker?

Very Not often


1 2 3 4 5

8. On average, how difficult is it for you to understand the English accent of someone


Very Very


1 2 3 4 5

9. How often do you find it difficult for you to understand the English accent of


Very Not often


1 2 3 4 5

Before you start Part C (question 1-17), please listen to Part C. You

may listen to the Part C only once. Do not look at or start answering

the questions until you finish listening Part C.

60

1. What was the customer looking for?

a) T-shirt b) Statue of Liberty Model c) Some souvenir d) Specific

souvenir

2. For whom did the customer want the product?

a) Mother b) Her friend c) Grandmother d) Herself

3. Why did the customer not want the first product the clerk recommended?

a) She did not like the design.

b) She did not like the material.

c) It was too expensive.

d) She did not think it’s appropriate.

4. Which product was the customer first interested in?

a) T-shirt b) The medium model c) Silver ring d) The large

model

5. Which word did the customer not understand?

a) Statue b) Copper c) Souvenir d) Pewter

6. Which kind of material of the product did the customer prefer?

a) Pewter b) Copper c) Plastic d) None of the above

7. What kind of wrapping did the customer want?

a) Birthday b) Gift c) Christmas d) Paper

8. How many times has the customer visited the U.S.?

a) 1 b) 2 c) 3 d) 4

9. How long does the customer plan to stay in the U.S.?

a) 3 days b) 5 days c) 7 days d) More than 7 days

10. When will the customer leave?

a) Today b) Tomorrow c) 2 days from now d) Next week

11. Other than New York, where did the customer visit?

a) No where b) Hard Rock Cafe c) Broadway d) Washington D.C.

12. How long did the customer spend there?

a) A day b) 3 days c) A week d) 10 days

13. What was the purpose of visiting there?

a) For shopping b) For museum c) For sightseeing d) For the

tour

61

14. What did she buy there?

a) Souvenir b) Ticket c) Food d) A badge

15. What was the customer’s purpose of visit to the U.S.?

a) Business b) To see her mother c) To shop d) For fun

16. Why did the customer not take the tour?

a) She could not afford it. b) She wanted to go by herself.

c) She did not know about it. d) She did not have a chance.

17. Where she did not go?

a) Steak house b) Museum

c) White House d) Statue of Liberty

18. How did the customer have fun in New York?

a) Food b) Musical c) Sightseeing d) Museum

19. Why did the customer go to Hard Rock Café?

a) It is her favorite place.

b) It was her first time to visit there.

c) She likes burger there.

d) She likes their goods.

Before you start Part D (question 20-27), please listen to Part D. You

may listen to the Part D only once. Do not look at or start answering

the questions until you finish listening Part D.

20. What did she expect that his view of Japanese food?

a) Delicious b) Too heavy c) Too light d) Too salty

21. What did she think about Japanese foods in the U.S.?


22. What did they disagree about the tastes of rice and tofu?

a) He insisted rice and tofu have taste. b) She insisted that rice and tofu

have taste.

c) She does not like rice and tofu. d) She does not like rice and tofu.

62

23. How did she think about his sense of tasting foods?

a) He does not have good sense of tasting any foods.

b) He has good sense of tasting any foods.

c) He does not have good sense of tasting Japanese food.

d) He has a good sense of American foods.

24. How did she evaluate popular American-Japanese food?


25. Which part of his explanation of difference between Japanese and American recipes

she did not understand?

a) Metric system b) Ingredients c) Type of a food d) Cooking

procedure

26. What was her example of wrong American food in Japan?

a) American coffee b) American tea c) American burger d) American

juice

27. How did they settle down about argument?

a) They accepted differences between cultures. b) They did not ague.

c) They changed topic. d) He accepted her view.

Before you start Part E (question 28-31), please listen to Part E. You

may listen to the Part E only once. Do not look at or start answering

the questions until you finish listening Part E.

28. What is her occupation?

a) Instructor b) Counselor c) Professor d) Principal

29. Where did she get married?

a) Africa b) Japan c) The U.S. d) None of the above

30. Why was she reading a book about cats?

a) She likes cats.

b) To write a book.

63

c) It is her assignment.

d) She has a cat.

31. What is her dream?

a) To write a book about cats b) To be a teacher

c) To marry her boy friend d) None of the above

Before you start Part F (question 32-33), please listen to Part F. You

may listen to the Part F only once. Do not look at or start answering

the questions until you finish listening Part F.

32. Why did her husband wanted to see the chef?

a) Because he is a great chef. b) Because he is a TV star.

c) Because he is a famous chef. d) The speaker did not mention why.

33. What kind of restaurant was it?

a) Japanese b) American c) Chinese d) Korean

Before you start Part G (question 34-37), please listen to Part G. You

may listen to the Part F only once. Do not look at or start answering

the questions until you finish listening Part F.

34. According to the conversation speaker’s friend Yoko lives in.

a) America b) Russia c) Asia d) Europe

35. How does the speaker evaluate her friend Yoko?

a) She thinks Yoko is a great person.

b) She does not like Yoko.

c) Yoko is nice, but she has to develop more social skills.

d) Yoko cannot speak French.

64

36. What is the most difficult part of living in Switzerland?

a) She does not like winter.

b) She does not like the area where she lives.

c) Living costs are too expensive where she lives.

d) She does not like to work hard.

37. Which country did the speaker imply Yoko comes from?

a) Japan b) The U.S.A. c) China d) Switzerland

Before you start Part H (question 38-42), please listen to Part H. You

may listen to the Part H only once. Do not look at or start answering

the questions until you finish listening Part H.

38. Why did she not go to the concert?

a) Tickets are too expensive.

b) She did not like the band.

c) She did not have the time.

d) She could not get a ticket.

39. Why did she not go to John’s house?

a) It is too far for her. b) She can go there any time.

c) She does not like John d) She does not have time to go.

40. How is she regarding Pennsylvania?

a) She is not there right now. b) She is there.

c) She will live there. d) She wishes to live there.

65

41. What does the sentence imply about Mary?

a) One should feel sorry for her.

b) One should apologize to her.

c) One doesn’t need to apologize to her.

d) She should be ignored.

42. According to the speaker

a) This fall will be colder.

b) Usually, it is colder in fall.

c) It is a nice day.

d) She complaints about warm weather.

Before you start Part I (question 43-), please listen to Part I. You may

listen to the Part H only once. Do not look at or start answering the

questions until you finish listening Part H.

43. Why does the second speaker wanted to go to England?

a) Because it was on magazine.

b) To see gardening plant.

c) To see a person.

d) Because of special event.

44. Why does the first speaker want to go to Cambodia?

a) Because it was on magazine.

b) To see gardening plant.

c) To see a person.

d) Because of special event.

45. Which country was not mentioned by them?

a) Japan

b) England

66

c) Jordan

d) Iran

46. Which places they agreed that they want to go?

a) Parts of the U.S.

b) Asian countries

c) Egypt

d) Both a and c

47. Which place at least one of them has been to?

a) Cambodia

b) Egypt

c) Hawaii

d) England

Part J: Please answer the following questions.

1. Rate the non-native speaker’s fluency in the Part C to H.

Very Very

Poor Average Good

1 2 3 4 5

2. Rate the non-native speaker’s pronunciation in the Part C to H.

Very Very

Poor Average Good

1 2 3 4 5

3. Rate the non-native speaker’s accent in the Part C to H.

Very Very

Poor Average Good

1 2 3 4 5

67

4. Rate the non-native speaker’s grammar in the Part C to H.

Very Very

Poor Average Good

1 2 3 4 5

5. Rate the non-native speaker’s clarity in the Part C to H.

Very Very

Poor Average Good

1 2 3 4 5

68

APPENDIX C

WE CONVERSATION SCRIPTS

69

Conversation 1

A: Hi, may I help you?

B: Oh, yes please.

A: Are you looking for something in particular?

B: Yes. I want to buy a souvenir for my mom.

A: How about an “I love New York T-shirt” ?

B: Well, my mom doesn’t wear T-shirts. Do you have anything else?

A: I see. We have some Statue of Liberty models.

B: It sounds good. Can I see them?

A: Sure, just over here. We have a lot of different sizes.

B: .........

A: Do you see any that you like?

B: Mmm, this middle size one looks good for her. I’ll take this one.

A: O.K. That one’s pewter, so it’s 15 dollars.

B: What’s pewter?

A: Like silver but soft.

B: Do you have any different materials?

A: Of course. We have gold, silver, wood, copper, plastic. What would you like?

B: Which do you recommend?

A: This copper one is the most realistic.

B: How much is it?

A: This one’s 12 dollars.

C: O.K. I’ll take it.

G: Would you like that gift-wrapped?

C: Oh, can you? Please.

A: No problem. Birthday paper or Christmas paper or anniversary or just normal?

B: Well, Christmas one, please.

A: Sure. Just a moment, please.....So, where are you from?

70

B: I'm from Japan. Have you been to Japan?

A: No, I haven't. Is this your first time to America?

B: Yes. I've been here for 10 days, and I'll leave the day after tomorrow.

A: Are you here on vacation?

B: Yes, I am.

A: Have you spent the whole time in New York, or did you go anywhere alse?

B: I went to Washington D.C.

A: How long did you spend there?

B: For 3 days. I went to see the White House.

A: DId you take a tour?

B: I wanted to. But I couldn' t make a reservation.

A: Oh well, did you go to the Smithsonian?

B: Well I don't know it.

A: It's the big group of museums near the capital building.

B: O.K. I'll go there when I go to Washington D.C. next time.

A: So, you spent the other 7 days in New York?

B: Yes. I enjpyed New York very much.

A: Thanks. So, did you see a Broadway show?

B: Yes, I saw "Cats". It was wonderful.

A: Yeah,everyone likes that one. Are you stying in Manhattown?

B: Yes.

A: Did you go to the Hard Rock Cafe?

B: Yes. I like Hard Rock Cafe. I collect their badges.

A: How many do you have?

B: I have 7. 5 are fom Japan,1 is from Washinton D.C. 1 is from New York.

A: Are you from Tokyo?

B: No, I'm from Osaka, third largest city in Japan.

A: Oh, I'm sorry. O.K. Here you go. One gift-wrapped Statue of Liberty.

71

B: O.K. Here you go.

A: Thank you, and 3 dollars is your change. Have a nice day.

b: Thank you.

72

Conversation 2

A: What did you think the first time you ate Japanese food?

B: Ahh, let’s see, that was when I was in high school… I thought “Wow, this is good. It’s kind of like Chinese food.”

A: Didn’t you think that Japanese food’s flavor was light?

B: Not really, it was pretty salty and strong.

A: Because you had it in America. When I went to America, I thought “This isn’t real Japanese food.”

B: Yeah but I’ve had Japanese food in Japan that’s had pretty strong flavors, haven’t you?

A: Yeah, some are salty. But when I had food in America, everything tasted heavy. Actually your cooking tastes a little heavy.

B: That’s just because you’re used to bland foods like rice and tofu.

A: You usually say that rice and tofu have no flavor but they do! I can tell! I can taste it!

B: It’s just your imagination.

A: You can ask your students! I’m sure they’ll agree with me. And also, water has flavor.

B: Only if the flavor is added, like that C1000 Lemon Water.

A: You don’t understand.

B: Don’t understand what?

A: Even though you like cooking and going to good restaurants, why can’t you tell their flavor? You can tell Coke’s flavor at McDonald’s and so on. Like the syrup is too little or too much.

B: That’s different, that’s like how you can tell the difference between my ribs with five spice powder and the ribs without it.

A: OK, you’ll understand some day, I hope. By the way, what did you have the first time you had Japanese food?

B: I had teppanyaki, at a restaurant in Fort Wayne (the city we lived in in Indiana)..

A: Teppanyaki isn’t so Japanese for me. I know it’s popular in America but Japanese food is tempura or sushi for most Japanese. Isn’t tempura famous in America?

B: It’s kind of famous but less than sushi or teppanyaki. I had heard of sukiyaki and teriyaki in America, too. I had teriyaki before coming to Japan but not sukiyaki.

Anyway don’t you think saying Japanese food is sushi and tempura is stereotyping? Don’t you want other Japanese food to be famous?

73

A: Yes, other Japanese food can be famous, but also I’m proud of sushi and tempura. What Japanese food can be popular in America you think?

B: I think fried noodles and okonomiyaki could be popular. I think yakiniku could be really popular, too, but it probably won’t have a chance because of too many safety regulations.

A: Yeah, okonomiyaki is very good, and battered octopus balls. OK, we can open a restaurant in America!

B: With okonomiyaki and octopus balls? No way, too Osaka! How about Nagoya-style wings and noodles? You know the wings would be popular, remember when I brought them to that party?

A: I liked your wings, but you know I don’t like Nagoya so much. By the way, you read recipes in Japanese and in English. Is there any difference between them?

B: Only in the names of the ingredients and in the measurements, since Japan uses the metric system.

A: What’s the metric system?

B: You know, meters, grams, liters…

A: What system does America use?

B: We use the English system. Feet, pounds, ounces… Anyway, you know a lot of so-called American food in Japan is wrong too, right?

A: You mean American coffee? Light-flavored?

B: That’s one example. It’s like hot brown water, real American coffee is definitely not like that.

A: Things from overseas can be different in different countries.

B: So you shouldn’t complain about Japanese food in America then, right?

A: OK, I’ll try to not complain.

74

Monologue 1

Hello. I am going to tell about myself. I am a teacher for a cram school. Teaching is

interesting for me. And I am married. My husband is American. His name is Gabriel.

He is very sweet. And I like reading books. Now I am reading a novel about cats. I like

cats. Sometime I want to have cats. At last, I will tell you about my dream. I want to be

a translator someday. I want to publish my books. Thank you.

Monologue 2

Hello, I am going to tell you about my experience. Last Tuesday was my husband’s

birthday. So, we went to a good restaurant in Nagoya last Sunday. It’s a famous chef’s

Chinese restaurant. His name is Chin Kenichi. He became famous from Japanese TV

program, Iron Chef. We really enjoyed lunch. When we paid, my husband asked the

staff if Mr. Chin was there. But, later the staff came to us and said “it’s actually secret,

but if you wan to see Chin, He will be here on December 19th and 20

th. We were very

happy to hear it. Probably my husband will go there that day. Thank you.

Monologue 3

Hi, I will tell you about my friend Yoko. I’ve known her for 5 years. Now, she lives in

Switzerland because her husband is Swiss. It can be difficult to live in foreign

countries. And, so it is for Yoko. First, she speaks French there. She had to study

French very hard. Second, it’s very cold in winter and winter starts much faster then in

Japan. Third, everything is expensive there. And she doesn’t like the new or different

things. So it is difficult to get Japanese things. There are some more reasons, but Yoko

always does her best. She never gives up. So, I really respect her. Thank you.

Sentence 1

I would have gone to the concert if I had had a ticket.

Sentence 2

I don’t like to go to John’s house because it’s 5 miles from here.

Sentence 3

I dreamed about living in Pennsylvania.

Sentence 4

You don’t have to say sorry to Mary.

Sentence 5

It’s warm for fall.

75

REFERENCES

Bernstein, I.H., Rush, A.J., Carmody, T.J., Woo, A., & Trivedi, M.H. (2007).

Clinical vs. self--report versions of the quick inventory of depressive symptomology in

a public sector sample. Journal of Psychiatric Research, 41, 239-246.

Boldt, R. F., Larsen-Freeman, D., Reed, M.S., & Courtnery, R.G. (1992).

Distributions of ACTFL Ratings by TOEFL Score Ranges. TOEFL Research Report,

92(59).

Bollag, Burton. (2005). New Test of English as a Foreign language Puts an

Emphasis on Speaking. Chronicle of Higher Education, 52(2), A49.

Bouton, F. L. (1994). Conversational implicature in a second language:

Learned slowly when not deliberately taught. Journal of Pragmatics, 22, 157-167.

Chalhoub-Deville, M., & Wiggleworth, G. (2005). Rater judgment and English

Language speaking proficiency. World Englishes, 24 (3), 383-391.

Conian, D. (2001). The use of audio or video comprehension as an assessment

instrument in the certification of English language teachers. System, 29 (1) , 1-14.

Crystal, D. (2000). Emerging Englishes. English Teaching Professional, (14),

3-6.

Davies, A., Hamp-Lyons, L., & Kemp, C. (2003). Whose norms? International

proficiency tests in English. World Englishes, 22 (4). 571-584.

76

Educational Testing Service (ETS). (2005). Test of English as a Foreign

Language.

Ervin-Tripps, S. (1970). Discourse agreement: How children answer questions.

In Hays (Ed.), Cognition and the development of language. New York: Wiley.

Flowerdrew, J., & Miller, L. (1997). The Teaching Academic Listening

Comprehension and the Question of Authenticity. English for Specific Purposes,16(1),

27-46.

Grice, H. P. (1975). Logic and conversation. In P. Cole & J. Morgan (Eds.),

Speech acts: syntax and semantics. (vol.3, pp41-58). New York: Academic Press.

Harris, J. (1994). English Sound Structure. Cambridge, MT: Blackwell.

Hinkel, E (ed.) (2005). Handbook of Research in Second Language Teaching

and Learning. Mahawah, NJ: LawrenceErlbaum associates.

Johnson, D. W., & Newport, E. L. (1989). Critical period effects in second

language learning: The influence of maturational state on the acquisition of English as a

second language. Cognitive Psychology, 21, 60-99.

Kachuru, Y. (1985). Discourse Analysis, non-native Englishes and second

language acquisition research. World Englishes, 4 (2), 223-232.

Kertoy, M. K., & Goetz, K. M. (1995). The Relationship between Listening

Performance on the Sentence Verification Technique and Other Measures of Listening

Comprehension. Contemporary Educational Psychology, 20, 320-339.

77

Lindemann, S. (2002). Listening with an attitude: A model of native-speaker

comprehension of non-native speakers in the United States. Language in Society, 31(3),

419-441.

Major, R. C., Fitzmaurice, S. M., Bunta, F., & Balsubramanian, C. (2005).

Testing the Effects of Regional, Ethnic, and International Dialects of English on

Listening Comprehension. Language Learning, 55(1), 37-69.

Mecalfe, J. (1986). Feeling of Knowing in Memory and problem Solving.

Journal of Experimental psychology. 12 (2), 288-294.

Nunnly J. C., & Bernstein I. H. (1994). Psychometric Theory (3rd). New York:

McGRAW-HILL.

Oltman, K. P., Stricker, J. L., & Barrows. S. T. (1990). Analyzing Test

Structure by Multidimensional Scaling. Journal of Applied Psychology, 75 (1), 21-27.

Powers, D. E., Scheldi, M. A., Leung, S. W, & Butler, F. A. (1999). Validating

the revised test of spoken English against a criterion of communicative success.

TOEFL Research Report, 99 (5), 63.

Read, J. (2002). The use of interactive input in EAP listening assessment.

Journal of English for Academic Purposes, 1(2), 105-119.

Richard, E. M., Diefendorff, J, M., Martin, J, H., (2006). Revisiting the Within-

Person Self-Efficacy and Performance Relation. Human Performance, 19(1), 67-87.

Rosenfeld, M. Sheppard, P. K., & Sheppard, K. (2004). Investigating the

Validity of TOEFL: A Feasibility Study Using Content and Criterion Relater Studies.

TOEFL Research Report, 3 (18), 71.

78

Shameem, N., (1998). Validating self-reported language proficiency by testing

performance in an immigrant community: the Wellington Indo-Fijians. Language

Testing, 15(1), 86-108.

Snow, S. M. (1998). Economic, Statistical, and linguistic factors affecting

success on the test of English as a foreign language (TOEFL). Information Economics

and Policy, 10 (2), 159-172.

The Society for Testing English Proficiency (STEP). (2004).

Stricker, L. J. (2004). The performance of native speakers of English and ESL

speakers on the computer-based TOEFL and GRE General Test. Language Testing, 21

(2),146-173.

Tabachinick, B. G., & Fidell, L, S. (2001). Multivariate Statistics (4th).

Needham Heights: Allyn and Bacon.

Tabachinick, B. G., & Fidell, L, S. (2007). Multivariate Statistics (5th).

Needham Heights: Allyn and Bacon.

79

BIOGRAPHICAL INFORMATION

Yasushi Kyutoku earned a high schools diploma from Seifu High School

(Osaka, Japan) in March, 1992, and earned B.A. in psychology from the University of

Texas at San Antonio in December, 2001. After the completion of thesis research, he

will continue to study experimental psychology at the University of Texas at Arlington.

He is interested in continuing to conduct researches related to psychometrics such as

validation and application of scales.

listening comprehension.pdf

Documents

listening comprehension skill

listening comprehension test

listening comprehension skills

perceived competency scale

principal component obtained

listening comprehension scale

english speaking countries

point likert scales