listening comprehension.pdf
Post on 11-Nov-2014
43 Views
Preview:
DESCRIPTION
TRANSCRIPT
VALIDATION OF WORLD ENGLISH (WE)
LISTENING COMPREHENSION
SCALE
by
YASUSHI KYUTOKU
Presented to the Faculty of the Graduate School of
The University of Texas at Arlington in Partial Fulfillment
of the Requirements
for the Degree of
MASTER OF SCIENCE IN PSYCHOLOGY
THE UNIVERSITY OF TEXAS AT ARLINGTON
MAY 2007
ii
ACKNOWLEDGEMENTS
I want to make acknowledgments for my supervisor, Dr. Bernstein, for all the
supervision, instruction, advices, helps, kindness, patience, and encouragements in
order to complete my thesis research.
Also, I want to make acknowledgements for my committee members, Dr.
Kimball, Dr. Kopp, and Dr. Gorfein. Without your advices, suggestions, and helps, I
could not complete the thesis research. Again, thanks for devoting your precious time
for my thesis.
Also, I want to thank Dr. Dougall for advices, suggestions, and kindness.
Finally, I want to thank Charles Aden, Chawki Belhadi, Chikako Davidson,
Gabriel Davidson, Sachimi Kyutoku, Tomomi Ogura, and Ryo Yamada, and faculties,
staffs, and colleagues in our departments for inspirations and supports.
March 19, 2007
iii
ABSTRACT
VALIDATION OF WORLD ENGLISH (WE)
LISTENING COMPREHENSION
SCALE
Publication No. _____
Yasushi Kyutoku, M.S.
The University of Texas at Arlington, 2007
Supervising Professor: Bernstein, Ira H
World English (WE) includes any type of dialectical English spoken around the
world (Hinkel, 2005). In the present study, a WE listening test was developed. The
original form of the test consisted of six scales measuring, respectively, demographics,
exposure to non-native English speakers, self perception of WE fluency, WE listening
test, evaluations of non-native speakers' fluency, and attitudes towards non-native
English speakers. Internal consistency, dimensionality, correlations among scales, and
the effects of demographics on the test scores were analyzed based on the data collected
from 32 participants in Study 1. Results indicated moderate to moderately high internal
iv
consistency for the scales except for the attitudes toward non-native English speakers
scale. Study 2 constituted an actual computer administration with 106 participants. SE
listening comprehension test was added to the study. All of the scales except for the SE
test showed moderate to high consistency and unidimensionality.
v
TABLE OF CONTENTS
ACKNOWLEDGEMENTS...................................................................................... ii
ABSTRACT.............................................................................................................. iii
LIST OF ILLUSTRATIONS.................................................................................... ix
LIST OF TABLES.................................................................................................... x
Chapter
1. INTRODUCTION ........................................................................................ 1
1.1 Standard English and World English ...................................................... 1
1.1.1 World English (WE) ................................................................ 1
1.1.2 Reasons to Study WE............................................................... 2
1.2 WE Listening Comprehension Test and Relevant Scales ....................... 4
1.2.1 WE Listening Comprehension Test ......................................... 4
1.2.2 Relevant Scales ........................................................................ 5
1.2.2.1 Exposure Scale ............................................................ 5
1.2.2.2 Attitude Scale .............................................................. 6
1.2.2.3 Self-report Scale .......................................................... 6
1.2.2.4 Evaluation Scale .......................................................... 7
1.3 Purposes of This Study ........................................................................... 7
2. METHODS (STUDY1) ................................................................................ 10
2.1 Participants ............................................................................................. 10
vi
2.2 Materials.................................................................................................. 10
2.3 Design and Procedure ............................................................................. 12
2.4 Data Analyses ......................................................................................... 13
3. RESULTS (STUDY 1) ................................................................................. 14
3.1 Data Screening ........................................................................................ 14
3.2 Validation of Scales ................................................................................ 16
3.2.1 Exposure Scale ................................................................................ 16
3.2.2 Attitude Scale .................................................................................. 17
3.2.3 Self-report Scale.............................................................................. 18
3.2.4 WE Listening Comprehension Test ................................................ 19
3.2.5 Evaluation Scale.............................................................................. 20
3.4 Correlation among Scales ....................................................................... 21
3.5 Effects of Demographics on WE Listening Comprehension Test .......... 23
4. DISCUSSION (STUDY1)............................................................................ 24
4.1 Validation of Scales ................................................................................ 24
4.2 Correlation among Scales ....................................................................... 25
4.3 The Effects of Demographics on WE Listening Comprehension Scale . 25
4.4 Limitations of Study 1............................................................................. 26
5. INTRODUCTION (STUDY 2) .................................................................... 28
6. METHODS (STUDY 2) .............................................................................. 29
6.1 Participants.............................................................................................. 29
6.2 Materials.................................................................................................. 29
vii
6.3 Design and Procedure ............................................................................. 30
6.4 Data Analyses ......................................................................................... 30
7. RESULTS (STUDY 2) ................................................................................ 31
7.1 Data Screening ........................................................................................ 31
7.2 Effects of demographics on WE Listening Comprehension Scale ......... 33
7.3 Validation of Scales ................................................................................ 34
7.3.1 Exposure Scale ................................................................................ 34
7.3.2 Self-report Scale.............................................................................. 36
7.3.3 Evaluation Scale.............................................................................. 37
7.3.4 WE Listening Comprehension Test ................................................ 38
7.3.5 SE Listening Comprehension Test .................................................. 40
7.4 Correlation Analyses among Scales........................................................ 41
8. DISCUSSION (STUDY 2)............................................................................ 42
8.1 Validation of Scales ................................................................................ 42
8.2 Correlation Analyses Among Scales....................................................... 42
8.3 Effects of Demographics on WE Listening Comprehension Test .......... 43
9. GENERAL DISCUSSION ............................................................................ 44
9.1 Conclusions, Limitations, and Future Directions.................................... 44
Appendix
A. WE LISTENING COMPREHENSION SCALE FOR STUDY 1................ 46
B. WE LISTENING COMPREHENSION SCALE FOR STUDY 2............... 56
C. WE CONVERATION SCRIPTS................................................................. 68
viii
REFERENCES.......................................................................................................... 75
BIOGRAPHICAL INFORMATION........................................................................ 79
ix
LIST OF ILLUSTRATIONS
Figure Page
1.1 Sensitive period for the second language learning .......................................... 3
3.1 PA for exposure scale ...................................................................................... 17
3.2 PA for self-report scale.................................................................................... 18
3.3 PA for WE listening comprehension test ........................................................ 20
3.4 PA for evaluation scale ................................................................................... 21
4.1 Distribution of WE listening comprehension test scores ................................ 26
7.1 PA for exposure scale ...................................................................................... 35
7.2 PA for self-report scale.................................................................................... 36
7.3 PA for evaluation scale.................................................................................... 38
7.4 PA for WE listening comprehension test ........................................................ 39
x
LIST OF TABLES
Table Page
3.1 Exposure scale.................................................................................................. 14
3.2 Attitude scale.................................................................................................... 15
3.3 Self-report scale................................................................................................ 15
3.4 WE Listening Comprehension Scale................................................................ 15
3.5 Evaluation scale................................................................................................ 16
3.6 Factor analysis for the exposure scale.............................................................. 17
3.7 Factor analysis for self-report scale ................................................................. 19
3.8 Factor analysis for WE listening comprehension test ...................................... 20
3.9 Factor analysis for evaluation scale ................................................................. 21
3.10 Table of descriptive statistics ......................................................................... 22
3.11 Correlation among the scales ......................................................................... 22
3.12 Effects of demographic data on WE
listening comprehension scale ........................................................................ 23
7.1 Exposure scale.................................................................................................. 31
7.2 Self-report scale................................................................................................ 32
7.3 Evaluation scale................................................................................................ 32
7.4 WE listening comprehension scale .................................................................. 32
7.5 WE listening comprehension scale when only
native speakers were included ......................................................................... 33
7.6 SE listening comprehension test ...................................................................... 33
xi
7.7 Effects of demographics on listening scale ...................................................... 34
7.8 Factor analysis for exposure scale.................................................................... 35
7.9 Factor analysis for self-report scale ................................................................. 37
7.10 Factor analysis for evaluation scale ............................................................... 38
7.11 Factor analysis for WE listening comprehension test .................................... 40
7.12 Table of descriptive statistics ......................................................................... 41
7.13 Correlation among the scales ......................................................................... 41
1
CHAPTER 1
INTRODUCTION
1.1 Standard English and World English
English is spoken as the native language by 3.6 hundred million people and
spoken as the second language by 1-1.5 billion speakers (Crystal, 2000). As a
consequence, English is regarded as the most widely used international language. For
example, factors such as movies, international travel, the internet, and recent
developments in computer networks have all further widened the use of English.
According to Kachru (1985), there are more non-native English speakers than native
ones. Even in English speaking countries, such as the United States, there are
increasing numbers of non-native English speakers because of increasing numbers of
immigrants (Flowerdew & Miller, 1997). Non-native speakers are defined as “people
who already use at least one other language and who live in a community in which
English is not normally used” (Hinkel, 2005).
1.1.1 World English (WE)
English languages can be categorized into three main types according to the
degree of nativity (Hinkel, 2005). The first type of English is Standard English (SE),
which is used in English speaking countries such as the U.S., the UK, Canada,
Australia, and New Zealand. This is commonly referred to as normative, “proper”,
2
“correct”, or “the King’s (Queen’s) English”. A second type of English is non-Standard
English, which is used in countries such as former colonies of English speaking
countries where English is the official or second language but not the dominant one.
There is debate whether specific kinds of native English, such as Ebonics, are standard
or nonstandard. A third type of English is non-native English. Non-native English is
used in countries where English is not official, dominant, or the first language. These
three types of English languages are collectively called World English (WE) as opposed
to SE, which includes only the first type of English. For example, WE speakers may
say “He like you car” instead of “He likes your car.” WE speakers also have relatively
strong accents derived from their dominant language that differ from those used by SE
speakers. They further use a different accent and are less fluent in English than they are
in their dominant language. Another characteristic of both non-standard and non-native
English is the intrusion of vocabulary from the more dominant language. For example,
I, as a native Japanese speaker, tend to say “hai” instead of “yes” (“hai” means yes in
Japanese).
1.1.2 Reasons to Study WE
Several studies reported that effective oral communication depends on the
mutual cooperation of speakers and listeners (Ervin-Tripp, 1970; Grice, 1975).
Although effective communication depends on mutuality, little attention has been paid
to the enhancement of native English speaker’s WE skills (Davies, Hamp-Lyons, &
Kemp, 2003). In other words, the standard of English proficiency in international
settings is based only on SE (Chalhoub-Deville & Wiggleworth, 2005). For example,
3
non-native English speakers are often required to take an examination to qualify for
academic admissions or for employment in English speaking countries. Even in non-
English speaking countries, many people are required to take English proficiency
examinations such as TOEFL when there are academic or business demands to use
English (Davies, Hamp-Lyons, & Kemp, 2003).
The efforts of non-native speakers to improve proficiency can be a potential
cause of miscommunication in various international settings in domains such as
conference, academia, politics, and business (Davies, Hamp-Lyons, & Kemp, 2003).
Further, it is unrealistic for non-native English speakers to achieve the SE fluency of
native speakers (Bouton, 1994; Snow, 1998). Johnson & Newport (1989) reported a
learning curve that describes the relationship between age of arrival to English speaking
country and the test of English grammar score suggested that the age of learning a
second language is important factor (Figure 1.1).
200
210
220
230
240
250
260
270
280
290
native 3 to 7 8 to 10 11 to 15 17 to 39
Age or arrival
Test of English Grammar Score
Figure 1.1Sensitive period for the second language learning
4
That implies that unless a non-native English speaker is exposed to English in early age,
it is hard for him/her to ever achieve SE level of fluency. Also, and perhaps most
importantly, the listening comprehension of native English and non-native speakers is
influenced by the type of English (Major, et al, 2005). Thus, SE is appropriate where
English is normally used while WE is more appropriate and practical in international
settings (Davies, Hamp-Lyons, & Kemp, 2003).
1.2 WE Listening Comprehension Test and Relevant Scales
Despite the importance of WE, it has received very little attention (Davies,
Hamp-Lyons, & Kemp, 2003). As a result, there is not a widely used WE assessment
test. Accurate and valid measurement of WE should be developed in order to enhance
international communication. The present research investigated listening
comprehension and four other related domains of WE proficiency.
1.2.1 WE Listening Comprehension Test
The WE listening comprehension scale was modeled upon a widely used SE
listening comprehension test in the U.S., the Test of English as a Foreign Language
(TOEFL). The TOEFL is developed and administered by Educational Testing Service
(ETS), and according to Bollag (2005) TOEFL “is taken by most foreign students who
are being considered for admission at English-speaking higher-education institutions,
including those in the United States. Last year (2004) nearly 750,000 students took the
test.” In terms of predictive validity (Rosenfeld, Oltman & Sheperd, 2004) and
construct validity (Powers et. al, 1999; Oltman, Stricker, & Barrows, 1990), TOEFL
appears to be a valid measure of English fluency. The TOEFL listening test is
5
composed of questions in which examinees are given native speakers’ conversations,
monologues, and sentence speeches. Respondents are instructed to pick the best
answers based on multiple-choice questions. Similarly, the WE listening
comprehension test is composed of questions in which conversations between a native
Japanese speaker speaking English and an American speaker, monologues of a Japanese
English speaker, and sentence-speeches of a Japanese English speaker are used as the
test material.
1.2.2 Relevant Scales
In addition to the listening comprehension test, exposure to foreign cultures (the
exposure scale), attitudes toward non-native English speakers (the attitude scale), self-
reports of WE listening comprehension skill (the self-report scale), and evaluations of
non-native speaker’s English in the listening tests (the evaluation scale) were developed
to investigate the domains that are related to WE listening comprehension.
1.2.2.1 Exposure Scale
First, the exposure scale was designed to investigate the degree that exposure to
non-native speakers and non-American cultures related to WE listening comprehension
skills. In other words, this scale intends to measure the degree that the amount of
experience to WE influences on the WE listening comprehension test score. Since
Johnson & Newport (1989) suggested that the amount of exposure to spoken language
should not be highly correlated with linguistic skill, amount of exposure to WE should
not be highly correlated with listening performance.
6
1.2.2.2 Attitude Scale
Second, the attitude scale was designed to measure attitudes toward non-native
speakers and cultures related to WE listening comprehension skills. Although some
may believe that attitudes of native speakers toward non-native speakers influence
native speakers’ listening comprehension, Lindemann (2002) failed to find such a
relationship. The present scale was used to reexamine this hypothesis.
1.2.2.3 Self-report Scale
Third, a self-report scale directly asked respondents to judge participants’ own
WE fluency in a similar manner to other metacognitive scales (Metcalfe, 1986;
Nunnally & Bernstein, 1994). This scale was designed to investigate accuracy of self-
perceptions of WE fluency. The scale investigated the following basic conversational
components of WE comprehension in 5-point Likert scales. This scale first prompted
respondents for their self-perceptions of their own WE comprehension skills. It then
assessed the degree and frequency of difficulties in comprehending elements of speech
such as accents (where you stress in a word or a sentence), pronunciation (how you say
a word), and grammar (rules of a language). These factors were included in the scale
because Harris (1994) suggested that the syntactical structure (grammar) and
phonological structures (accent and pronunciation), and the combination of said (overall
fluency) are important factors in the comprehension of spoken English. Various studies
have failed to find a correlation between self-efficacy and actual performance in various
cognitive domains such as memory tasks (Richard, Diefendorff, & Martin, 2006,
7
Shameem, 1998). This current study investigated whether metacognition (self-report) is
related to WE listening comprehension performance.
1.2.2.4 Evaluation Scale
Finally, the scale for the evaluation of non-native speaker’s fluency,
pronunciation, accent, grammar, and the clarity of the speech in the listening
comprehension part was designed to measure whether evaluation of non-native speakers
was associated with examinees comprehension skill.
1.3 Purposes of This Study
Listening and related components of WE were also investigated using standard
classical test theory measures in 2 studies. Study 1 was conducted as a pilot study for
the study 2. These studies involved determining Cronbach’s αof the exposure scale,
attitude scale, self-report scale, evaluation scale, and the WE listening comprehension
test and item/total correlations for the constituent items (rit). Cronbach’s alpha is a
statistical index of internal consistency that is determined by the correlations of items
and the numbers of items in a scale. Higher α implies the higher internal consistency.
It has been suggested that an acceptable scale should produce a Cronbach’s alpha larger
than .7, and each rit should be larger than .2. in its target population (Nunnally &
Bernstein, 1994). Second, exploratory principal component analyses should suggest
that the scales are unidimensional using eigenanalysis and parallel analysis to establish
a baseline for the assessment of dimensionality (Bernstein et al., 2007). As for
principal component factor analyses, a table for each result presented factor pattern (=
factor structure in the following analyses) and communalities (h2). Factor pattern is a
8
coefficient that describes the relationship between a factor and an item. Higher
coefficient value implies the higher relationships. Communality is the sum of the
squared factor pattern for each item. The larger communality implies that the amount
of variance in an item is explained by a particular factor. According to Bernstein et al
(2007):
In PA, one factors a matrix containing the same number
of observations and variables as the real data… but
simulated from a population in which all correlations are
zero. Multiple matrices of this form may be generated.
The results are averaged to provide for a more stable
estimate and standard errors. In order for s solution to
conform to a unidimensional solution, the first principal
component obtained from the real data must be larger
than the first principal component obtained from the
simulation, but all subsequent components from the actual
data must be smaller than their simulated counterparts.
Third, correlations among the exposure scale, the self-report scale, the evaluation scale,
and the listening comprehension test were analyzed. Fourth, the effects of demographic
variables such as gender, nativity/non-nativity of English, and age upon the scales were
also obtained.
9
Both studies to be reported were approved by the University of Texas at
Arlington’s Institutional Review Board. Study 1 was a pilot study designed to provide a
working set of scales.
10
CHAPTER 2
METHODS (STUDY 1)
2.1 Participants
Thirty two students enrolled in introductory psychology at the University of
Texas at Arlington were recruited. The data from one participant was omitted because
that person did not finish the experiment. The sample consisted of 6 males and 29
females, with a mean age of 20.4 years (SD = 4.21). There were 27 SE speakers and 5
non-native speakers. They voluntarily participated in preliminary research in order to
fulfill a course requirement.
2.2 Materials
The first part of all the scales contained demographic information (age, gender
and country of birth). Second, the degree of exposure to foreign cultures (exposure),
attitudes toward non-native English speakers and cultures (attitude), self reports of WE
listening comprehension skill (self-report), Listening comprehension tests (WE test),
and evaluations of non-native speaker’s English in the listening tests (evaluation) were
designed to investigate the domains possibly related to WE listening comprehension.
These items may be found in Appendix A.
a. Scales that preceded the WE listening comprehension scale:
1) Demographic data
11
2) The exposure scale (Part A of Appendix A): measured the degree of
exposure to non-native speakers. In order to eliminate problems
associated with the exposure scale, unit response to items were
standardized.
3) Attitudes toward non-native English speakers were assessed using
five-point Likert scales (Part B of Appendix A).
4) Self-perceptions of WE comprehension skill and the degree and
frequency of difficulties in comprehending elements of speech (Part C of
Appendix A) measured in 5 point Likert scale such as accents (the
position of stressed syllable in a word), pronunciation (the way of
vocalizing a word), and grammar (rules of a language).
b. A scale the presented after the completion of the WE listening comprehension
scale
The evaluation scale (Part H of Appendix A) measured participants’
evaluations of the non-native speaker’s fluency in a conversation (Part D
of Appendix A).
As above noted, the listening comprehension scale of WE (Part D to G of
Appendix A) was modeled upon the TOEFL. In the TOEFL, examinees are presented
with a conversation by native English speakers in SE and asked to pick the best multiple
choice answers. Instead of native English speakers, a conversation between a male
native English speaker using SE and a Female whose native language was Japanese and
who spoke WE was used. Participants were asked comprehension questions using a 4-
12
alternative multiple-choice format. The Japanese speaker’s proficiency level was
certified as “Able to understand and use English well enough for everyday needs and
situations…overseas” by the Society for Testing English Proficiency Inc (STEP, 2004).
A digital audio file was used to reproduce the recorded conversation. Audio
presentation has been found to be a more effective and efficient assessment method of
listening comprehension than video (Read, 2002; Conian, 2001). Also, auditory
presentations remove extraneous variables such as visual information out of the
procedure. Participants listened to digitally recorded conversations through
headphones, and they responded to paper and pencil version of the self-report scale and
listening comprehension scale.
2.3 Design and Procedure
After informed consent was collected, participants received instructions for the
experiment. Then, they proceeded to complete the demographic survey. After the
demographic survey, they responded to parts A (exposure), B (attitude), and C (self-
perceived competency) of the survey, in turn. They then proceeded to part D, where
they heard the conversation once. After listening to the conversation, they then
answered the listening comprehension test of the conversation. They then proceeded to
Parts E, F, and G in the identical manner as Part D. Participants debriefed and released
after completing part H. Thus, all participants went through identical procedures. The
procedure took about 25 to 35 minutes.
13
2.4 Data Analyses
Listening and related components of WE were also investigated using standard
classical test theory measures in 2 studies. This involved determining the internal
consistency (Cronbach’s α) of the exposure scale, attitude scale, self-report scale,
evaluation scale, and the WE listening comprehension test and item/total correlations
for the constituent items (rit). It has been suggested that an acceptable scale should
produce a Cronbach’s alpha larger than .7, and each rit should be larger than .2. in its
target population (Nunnally & Bernstein, 1994). Second, exploratory principal
component analyses should suggest that the scales are unidimensional using
eigenanalysis and parallel analysis to establish a baseline for the assessment of
dimensionality (Bernstein et al., 2007). Third, correlations among the exposure scale,
the self-report scale, the evaluation scale, and the listening comprehension test were
analyzed. Fourth, the effects of demographic variables such as gender, nativity/non-
nativity of English, and age upon the scales were also obtained.
14
CHAPTER 3
RESULTS (STUDY 1)
3.1 Data Screening
The means and standard deviations for each item of each scale are presented in
Table 3.1 for the exposure scale, Table 3.2 for the attitude scale, Table 3.3 for the self-
report scale, Table 3.4 for the WE listening comprehension test, and Table 3.5 for the
evaluation scale. Each scale was subjected to component analysis. The results will be
presented to emphasize the loadings on the first (general) factor.
Table 3.1 Exposure scale
Variable N Mean SD rit
a1 24 48.9 114.7 0.55
a2 24 9.6 7.6 0.70
a3 24 3.8 1.6 0.73
a4 24 3.6 1.6 0.70
a5 24 5.2 8 0.48
a6 24 111.9 299.6 0.32
a7 24 3.4 1.4 0.62
a8 24 3.2 1.4 0.73
a10 24 3 1.6 0.83
Note: Item 9 was eliminated because it measured the degree
of exposure to written language rather than spoken language.
15
Table 3.2 Attitude scale
After revision
Variable N Mean SD rit
b1 31 3.4 0.76 0.38
b2 31 4.6 0.62 0.37
b3 31 4.3 0.92 0.36
b4 31 2.8 1.46 0.32
b5 31 4.3 1.07 0.38
b6 31 3.8 1.19 0.31
b7 31 3.4 1.3 0.2
b8 31 2.6 1.2 0.28
Table 3.3 Self-report scale
Variable N Mean SD rit
c1 32 3.7 0.78 0.48
c2 32 3.4 0.94 0.65
c3 32 3.4 1.01 0.61
c4 32 3.3 0.92 0.41
c5 32 3.3 0.87 0.58
c6 32 3.1 1.00 0.75
c7 32 3.2 0.91 0.55
c8 32 3.4 1.07 0.65
c9 32 3.2 1.03 0.67
Table 3.4 WE listening comprehension scale After Deletion
Variable N Mean SD rit
t11 32 0.50 0.51 0.47
t12 32 0.97 0.18 0.53
t13 32 0.97 0.18 0.53
t15 32 0.75 0.44 0.44
t18 32 0.44 0.50 0.33
t24 32 0.63 0.50 0.28
t27 32 0.84 0.37 0.47
t29 32 0.78 0.42 0.41
t30 32 0.75 0.44 0.40
16
Table 3.5 Evaluation scale
Variable N Mean SD rit
h1 30 4.0 0.93 0.67
h2 30 3.6 0.86 0.70
h3 30 3.1 0.90 0.66
h4 30 3.6 0.89 0.65
h5 30 3.6 0.77 0.60
3.2 Validation of Scales
3.2.1 Exposure Scale
In order to eliminate problems associated with scale units, response to items
were standardized. The internal consistency of the 10-item Exposure scale was high
(Cronbach’s α = .88, n = 24, Table 3.1). Item 9 was eliminated form the analysis
because it was more related to written than spoken English. Consequently it was
dropped from the scale, leaving 9 items (Cronbach’s α = .88, n = 24, Table 3.1). An
exploratory principal component factor analysis was performed. The scale was
unidimensional based upon a parallel analysis criterion (see figure 3.1), and the
resulting pattern appears in Table 3.6.
17
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
1 2 3 4 5 6 7 8 9
Number of factors
Eigen Value
Scree Plot
Randomized data
Figure 3.1 PA for the exposure scale
Table 3.6 Factor analysis for exposure scale
Factor Pattern Communalities
Items Factor 1 h2
1 0.65 0.54
2 0.77 0.60
3 0.81 0.74
4 0.79 0.72
5 0.59 0.83
6 0.36 0.84
7 0.71 0.54
8 0.81 0.69
10 0.89 0.81
3.2.2 Attitude Scale
The internal consistency of the 10-item Attitude scale was low (α = .54, n = 31).
Items 9 and 10 had unacceptable values of rit (-.13 and .01, respectively) so were
18
eliminated. However, the internal consistency remained poor (α = .61, Table 3.2) after
this deletion, the scale as a whole was eliminated. Consequently, the results of factoring
the scale will not be presented.
3.2.3 Self-report Scale
The internal consistency for the 9-item self-perceived competency scale was
high (α = .86, n = 32, Table 3.3). Item level analysis based on rit suggested all items
were related to the scale (Table 3.3) so all original items were retained. An exploratory
principal component factor analysis was performed. The result was that the scale was
unidimensional based upon a parallel analysis criterion (see figure 3.2), and the
resulting factor pattern appears in Table 3.7.
Parallel analysis
0
1
2
3
4
5
1 2 3 4 5 6 7 8 9
Number of factors
Eigen value
Scree Plot
Randomized data
Figure 3.2 PA for self-report scale
19
Table 3.7 Factor analysis for self-report scale
Factor Pattern Communalities
Items Factor 1 h2
1 0.59 0.66
2 0.73 0.70
3 0.72 0.80
4 0.51 0.91
5 0.66 0.83
6 0.83 0.74
7 0.66 0.60
8 0.75 0.80
9 0.76 0.85
3.2.4 WE Listening Comprehension Test
In order to achieve the final version of scale, there were four iteration processes.
This iteration process continued until the both the criterion for the Cronbach’s α =.7 and
rit = .2. First, item 8 that had no variance were eliminated (eg. Most participants
answered that question correctly). Then, items that had lower than rit =.2 were
eliminated in each step. In the step 2, items 6, 7, 20, 22, 26, and 28 were eliminated. In
the third step, items 1, 3, 4, 9, 10, 14, 16, 17, 19, and 25 were eliminated. In the fourth
step, items 2, 5, and 23 were eliminated. The final version of the WE listening test
retained 9 items with moderate internal consistency (α = 72, n = 32, Table 3.4). An
exploratory principal component factor analysis was performed. The scale was
unidimensional based on parallel analysis criterion (Figure 3.3) and the resulting factor
pattern appears in Table 3.8.
20
Parallel analysis
0
0.5
1
1.5
2
2.5
3
3.5
1 2 3 4 5 6 7 8 9
Number of factors
Eigne value
Scree plot
Randomized data
Figure 3.3 PA for WE listening comprehension test
Table 3.8 Factor analysis for WE listening comprehension test
Factor Pattern Communalities
Items Factor 1 h2
11 0.52 0.63
12 0.82 0.96
13 0.82 0.96
15 0.58 0.38
18 0.43 0.68
24 0.41 0.56
27 0.63 0.48
29 0.56 0.44
30 0.53 0.59
3.2.5 Evaluation Scale
The internal consistency of the 5-item evaluation scale was high (α = .85, n= 30, Table
5). Each item met the rit = .2 criterion, so all items were retained. An exploratory
principal component factor analysis was performed. The scale was unidimensional
21
based on parallel analysis criterion (Figure 3.4), and the resulting factor pattern appears
in Table 3.9.
Parallel analysis
0
0.5
1
1.5
2
2.5
3
3.5
1 2 3 4 5
Number of factors
Eigen value
Scree plot
Randomized data
Figure 3.4 PA for evaluation scale
Table 3.9 Factor analysis for evaluation scale
Factor Pattern Communalities
Items Factor 1 h2
1 0.80 0.63
2 0.82 0.67
3 0.80 0.64
4 0.78 0.61
5 0.74 0.55
3.3 Correlation among Scales
After the validation of scales, data screening for correlation analyses among
scales was performed. Descriptive statistics appears in table 3.10.
22
Table 3.10 Tables of descriptive statistics
Scale mean SD Skewness Kurtosis n
age 20.34 4.21 3.17 12.24 32
standardized logexp 0 1 1.67 3.28 32
self 29.94 5.92 0.18 1.2 32
eval 17.59 3.57 0.18 -0.64 32
WE 6.63 2.04 -1.31 2.72 32
Table 3.11 contains the scale intercorrelations. As can be seen, the correlation
between the self-perceived competency scale and the listening test was very low.
However, there was a moderate positive correlation between the listening test and the
evaluation scale.
Table 3.11 Correlation among the scales Pearson r (n=32)
Variable WE Self Logexp Eval
WE __ -0.06 .10 0.45*
Self __ 0.15 0.14
Logexp __ -0.07
Eval __
Note: WE = WE listening comprehension scale, Self = Self report scale, Logxp =
standardized and log transformed Exposure scale, and Eval = the Evaluation scale.
*P<.05
23
3.4 Effects of Demographics on WE Listening Comprehension Test
Demographics variables such as gender, native/non-nativity of English, and age
were unrelated to both the listening test (Table 3.12).
There were no trends of note the mean for males was 6.50 and the mean for
females was 7.16, and effect size was small, Cohen's d = .28. It would require about
320 participants to achieve a significant difference with α = .05, and power = .80. The
mean for native English speakers was 6.59 and mean for non-native speakers was 6.80,
and effect size was very small, Cohen's d = .01. It would require about 2874
participants to achieve significant difference with α = .05, and power = .80.
Table 3.12 Effects of demographic data on WE listening comprehension scale
ANOVA
source df F p
Age 1 1.68 0.21
Gender 1 0.65 0.43
C.O.B. 1 1.12 0.30
Error 28 (4.42)
Note: Values in parentheses are mean square errors
24
CHAPTER 4
DISCUSSION (STUDY 1)
4.1 Validation of Scales
The results of the pilot study were that all scales save for attitude met the usual
standards for reliability. Thus, attitude scale did not meet the statistical criterion. Also,
there was theoretical concern about the attitude scale. In other words, it was not clear
whether attitude scale was measuring actual attitude or overt response of participants.
For example, because of social desirability, participants might have reported socially
desirable attitude. Because of these reasons, attitude scale was eliminated from the
analyses. These other scales were then re-examined in a larger sample, but five changes
were made from study 1. First, item 9 was eliminated from the exposure scale, and all
items were changed to be measured in Likert scale. Second, the attitude scale was
eliminated due to low internal consistency as described above. Third, items 1, 2, 3, 4, 5,
6, 7, 8, 9, 10, 14, 16, 17, 19, 20, 21, 22, 23, 25, 26, 28 in the WE listening test were
modified (Appendix 2) if no one picked a particular multiple choice response or
everyone had the same response in a particular question. Therefore, either the questions
or multiple choice answers were modified. For example, multiple choice b) of items 2
in WE test was modified from “Father” to “Friend.” Also, items such as item 8 were
eliminated because everyone could answer that question correctly. “Father” was
25
changed to “Friend” because no one picked that answer. The 9-item exposure scale
(Part A of Appendix 2), the 9-item self-report scale (Part B of Appendix B), the 42-item
WE listening comprehension test (Part C to H of Appendix B), the 5-item English
listening comprehension scale (Part I of Appendix B), and the 5-item evaluation scale
(Part J of Appendix B) were also modified. Fourth, grammatical mistakes and typos
were corrected in the modified scales.
4.2 Correlation among Scales
Exposure scale was transformed due to the high deviation from normality.
These previous analyses also suggested two other points: (a) perceived self-
competency was not significantly correlated with test performance, and (b) scores on
the listening test were positively correlated with the evaluation of the fluency of the
non-native speakers.
4.3 The Effect of Demographics on WE Listening Comprehension Test
Demographics such as gender and age did not affect listening test scores.
Notably, native English speakers did not outperform non-native speakers in the WE
listening comprehension scale. This contrasts with findings obtained using SE listening
comprehension scales such as the TOEFL (Stricker, 2004). However, because of the
imbalance in numbers of native and non-native group and, especially, the small sample
size, a null conclusion may be premature.
26
4.4 Limitations of Study 1
The present study has two obvious limitations. As noted, the sample size was
very small. Second, listening test scores may have been range restriction restricted
since all scores distributed fell between 20 to 30 points (Figure 4.1).
Distribution of Scores
0
1
2
3
4
5
6
7
19 20 21 22 23 24 25 26 27 28 29 30
Test Scores
Frequency
Figure 4.1 Distribution of WE listening test scores
This might reflect the sample or the test materials. A practical solution to this limitation
was to modify scales using a larger sample. Since Cronbach’s alpha is not an inferential
measure, it should not be affected by sheer sample size. However, Tabachnik and
Fidell (2001; 2007) suggest a minimal sample size of 90 for a multiple correlation
analysis of the present form (however, since SE listening comprehension test was added
27
later to test discriminant validity, sample size larger than 105 was necessary for the
study 2).
28
CHAPTER 5
INTRODUCTION (STUDY 2)
The purpose of the study 2 was to investigate the modified scales of study 1 in a
larger sample. An SE listening comprehension test was also added to investigate
discriminant validity. The analyses paralleled those of study 1.
29
CHAPTER 6
METHODS (STUDY 2)
6.1 Participants
One hundred and seven students enrolled in introductory psychology at the
University of Texas at Arlington were recruited. The data from one participant was
omitted because that person did not finish the experiment. The sample consisted of 67
females and 39 males, with a mean age = 20.0 years and SD = 2.91. There were 90 SE
speakers and 16 non-native speakers. They voluntarily participated in preliminary
research in order to fulfill a course requirement.
6.2 Material
Five changes were made from study 1, as previously noted. First, item 9 was
eliminated from the exposure scale, and all items were measured using a Likert scale
format. Second, the attitude scale was eliminated. Third, items 1, 2, 3, 4, 5, 6, 7, 8, 9,
10, 14, 16, 17, 19, 20, 21, 22, 23, 25, 26, 28 in the WE listening test were modified
(Appendix B). The modified versions of the 9-item exposure scale (Part A of Appendix
B), the 9-item self-report scale (Part B of Appendix B), the 42-item WE listening
comprehension test (Part C to H of Appendix B), the 5-item Standard English listening
comprehension scale (Part I of Appendix B), and the 5-item evaluation scale (Part J of
30
Appendix B) were also attached as appendix B. Fourth, grammatical mistakes and
typos were collected in the modified scales. Fifth, administration and data collection of
the scales was automated using the E-prime program.
6.3 Design and Procedure
After informed consent was collected, participants were assigned to a computer.
They were asked to start answering the demographic survey and modified exposure
scale, the self-report scale, the WE listening comprehension test, and the evaluation
scale. After completion, participants were debriefed and released. Thus, all participants
went through the identical procedure. The task took approximately 40-45 minutes.
6.4 Data Analyses
Data analyses for the study 2 was identical to the study 1 except that the attitude
scale was excluded for the study 2 and SE listening comprehension scale was added to
the study 2.
31
CHAPTER 7
RESULTS (STUDY 2)
7.1 Data Screening
The means and standard deviations for each item of each scale are presented in
Tables 7.1 to 7.6. Table 7.1 contains the exposure scale. Table 7.2 contains the self-
perceived competency scale. Table 7.3 contains the Evaluation scale. Table 7.4 contains
the WE listening comprehension test, and Table 7.5 contains WE listening test for only
native speakers. Table 7.6 contains SE listening comprehension scale. Severe violation
of assumptions was not detected while screening data.
Table 7.1 Exposure scale
Variable N Mean SD rit
a1 104 3.3 1.8 .69
a2 104 3.0 1.8 .71
a3 104 3.9 1.1 .71
a4 104 3.6 1.3 .56
a5 104 1.7 1.3 .52
a6 104 1.6 1.3 .30
a7 104 3.2 1.3 .72
a8 104 3.0 1.2 .64
a10 104 3.0 1.3 .64
32
Table 7.2 Self-report scale Variable N Mean SD rit
b1 105 3.4 .86 0.67
b2 105 3.1 .87 0.79
b3 105 3.2 .93 0.72
b4 105 3.1 .94 0.65
b5 105 3.2 .90 0.69
b6 105 3.0 .91 0.79
b7 105 3.1 .93 0.83
b8 105 3.1 .92 0.69
b9 105 3.2 .93 0.70
Table 7.3 Evaluation scale Variable N Mean SD rit
j1 106 2.9 0.90 0.73
j2 106 2.7 0.82 0.74
j3 106 2.6 0.74 0.53
j4 106 3.0 0.79 0.47
j5 106 2.6 0.90 0.70
Table 7.4 WE listening comprehension scale Variable N Mean SD rit
t1 106 .86 .35 .40
t2 106 .88 .33 .36
t3 106 .44 .50 .25
t4 106 .51 .50 .29
t8 106 .92 .30 .24
t9 106 .74 .28 .30
t12 106 .72 .44 .23
t13 106 .72 .45 .23
t14 106 .34 .48 .29
t15 106 .91 .29 .25
t16 106 .75 .43 .21
t21 106 .68 .47 .21
t30 106 .87 .34 .23
t31 106 .52 .50 .27
t33 106 .41 .49 .29
t34 106 .55 .50 .30
t35 106 .74 .44 .29
t37 106 .75 .44 .30
t41 106 .42 .50 .29
33
Table 7.5 WE listening comprehension scale when only native speakers were included Variable N Mean SD rit
t1 90 .85 .35 .43
t2 90 .88 .32 .39
t3 90 .44 .50 .32
t4 90 .50 .30 .31
t8 90 .73 .28 .26
t9 90 .61 .44 .29
t12 90 .72 .45 .22
t13 90 .34 .45 .30
t14 90 .91 .48 .31
t15 90 .75 .29 .28
t16 90 .60 .43 .24
t21 90 .97 .47 .25
t30 90 .52 .34 .22
t31 90 .41 .50 .27
t33 90 .55 .49 .32
t34 90 .74 .50 .27
t35 90 .42 .44 .28
t37 90 .42 .44 .32
t41 90 .50 .50 .32
Table 7.6 SE listening comprehension test
Variable N Mean SD rit
item 1 106 .65 0.48 0.19
item 5 106 .58 0.50 0.19
7.2 The Effects of Demographics on WE Listening Comprehension Test
Demographic variables such as gender, native/non-nativity of English, and age
again did not related to scores on the WE listening test. As for the effect on WE test,
there were no trends of note the mean for males was 13.72 and the mean for females
was 13.52, and effect size was very small, Cohen's d = .059. It would require about 480
participants to achieve significant difference with α = .05, and power = .80. There were
34
no trends of note the mean for native English speakers was 13.64 and mean for non-
native speakers was 13.31, and effect size was very small, Cohen's d = .097. It would
require about 9000 participants to achieve significant difference with α = .05, and
power = .8. Since no significant effect of nativity on any scale based on one way
ANOVAs, data from both native and non-native participants were pooled (Table 7.7).
Table 7.7 Effects of demographics on listening scale
ANOVA
source df F p
Age 1 .57 0.45
Gender 1 .00 0.99
C.O.B. 1 .17 0.68
Error 100 (11.56)
Note: Values in parentheses are mean square errors
7.3 Validation of Scales
7.3.1 Exposure Scale
The result suggested moderately high internal consistency of the 9-item
Exposure scale (Cronbach’s α = .87, n =106. Item level analysis based on rit suggested
all items were related to the scale (Table 7.1) so all original items were retained. An
exploratory principal component factor analysis was then performed. The scale was
unidimensional based on parallel analysis criterion (Figure 7.1), and the resulting
pattern appears in Table 7.8.
35
Parallel analysis
0
1
2
3
4
5
1 2 3 4 5 6 7 8 9
Number of factors
Eigen value
Scree plot
Randomized
Figure 7.1 PA for exposure scale
Table 7.8 Factor analysis for exposure scale
Factor Pattern Communalities
Items Factor 1 h2
1 .77 .60
2 .80 .64
3 .79 .69
4 .67 .56
5 .61 .76
6 .36 .66
7 .80 .66
8 .75 .56
10 .74 .54
36
7.3.2 Self-report Scale
The result suggested an acceptably high internal consistency for the 9-item self-
perceived competency scale (α = .92, n = 106). Item level analysis based on rit
suggested all items were related to the scale (Table 7.2), so all items were retained. An
exploratory principal component factor analysis was performed. The scale was
unidimensional based on parallel analysis criterion (Figure 7.2), and the resulting
pattern appears in Table 7.9.
Parallel analysis
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8 9
Number of factors
Eigen value
Scree plot
Randomized
Figure 7.2 PA for self-report scale
37
Table 7.9 Factor analysis for self-report scale
Factor Pattern Communalities
Items Factor 1 h2
1 .75 .56
2 .84 .71
3 .79 .62
4 .72 .52
5 .76 .57
6 .85 .72
7 .88 .77
8 .76 .57
9 .76 .58
7.3.3 Evaluation Scale
The result suggested moderately high internal consistency of the 5-item evaluation
scale (α = .83, n = 106). Item level analysis based on rit suggested all items were related
to the scale (Table 7.3), so all items were retained. The scale was unidimensional based
on parallel analysis criterion (Figure 7.3), and the resulting pattern appears in Table
7.10.
38
Parallel analysis
0
0.5
1
1.5
2
2.5
3
3.5
1 2 3 4 5
Number of factors
Eiigen value
Scree Plot
Randomized
Figure 7.3 PA for Evaluation scale
Table 7.10 Factor analysis for evaluation scale
Factor Pattern Communalities
Items Factor 1 h2
1 .85 .72
2 .86 .74
3 .69 .48
4 .62 .38
5 .83 .69
7.3.4 WE Listening Comprehension Test
In order to achieve the final version of scale, there were four iteration processes.
This iteration process continued until the both the criterion for the Cronbach’s α =.7 and
rit = .2. First, items (6, 18, 23, 26, 28, 29, 32, 38, and 40) that had very low variance
were eliminated (eg. Everyone answered that question correctly). Then, items that had
lower than rit =.2 were eliminated in each step. In the second step, items 7, 10, 11, 17,
39
19, 20, 22, 24, 25, 26, 27, and 36 were eliminated. In the third step, item 43 was
eliminated. After the elimination, 20 out of 42 items were retained. The result
suggested moderate internal consistency (α = 71, n = 106). Item level analysis based on
rit suggested all retained items were related to the scale (Table 7.4). Internal consistency
of the WE listening comprehension scale for native speakers showed a similar result,
one that was not discussed here (Table 7.5). An exploratory principal component factor
analysis was performed. The scale was unidimensional based on parallel analysis
criterion (Figure 7.4), and the resulting pattern appears in Table 7.11.
Parallel analysis
0
0.5
1
1.5
2
2.5
3
3.5
1 2 3 4 5 6 7 8 9
Number of factors
Eigne value
Scree plot
Randomized data
Figure 7.4 PA for WE listening comprehension test
40
Table 7.11 Factor analysis for WE listening comprehension test
Factor Pattern Communalities
Items Factor 1 h2
1 .54 .66
2 .52 .63
3 .37 .62
4 .40 .77
5 .36 .52
8 .43 .75
9 .36 .54
12 .53 .65
13 .32 .71
14 .38 .66
15 .34 .70
16 .30 .55
21 .30 .66
30 .31 .61
31 .37 .52
33 .39 .58
34 .42 .67
35 .44 .73
37 .43 .58
41 .39 .65
7.3.5 SE Listening Comprehension Test
There were originally 5 items on this scale, but only 2 items were retained due
to low internal consistency (α = .31, n =106, Table 7.6). Due to low internally
consistency and small number of items, meaningful analyses was not able to be
performed further. Low internal consistency seemed to be caused by both low
correlations among items and very small number of items in the scale. Based on the
Spearman-Brown prophecy formula, about 11 more items were needed to have a
reliability of .70.
41
7.4 Correlation among Scales
Descriptive statistics after the validation was shown in Table 7.12.
Table 7.12 Table of descriptive statistics
Scale mean SD Skewness Kurtosis
age 19.96 2.01 2.64 9.64
exp 26.18 8.86 0.09 -1.09
self 23.07 2.41 -0.69 0.026
eval 13.8 3.22 0.26 1.24
WE 13.6 3.38 -0.37 -0.33
SE 1.23 0.75 -0.42 -1.11
Results from correlation analyses (Table 7.13) suggested that there were a
medium positive correlation between the WE listening comprehension test and English
listening comprehension test, between the WE listening comprehension test and self-
report, and the listening test and the evaluation scale. The significant correlation
between WE test and self report was unexpected based on the study 1.
Table 7.13 Correlation among the scales Pearson r (N=106)
variable WE SE Self exp eval
WE __ 0.29** 0.36*** 0.02 0.22*
SE __ 0.05 0.03 -0.14
self __ -0.06 0.13
exp __ 0.09
eval __ ___
Note: WE = WE listening comprehension scale, SE = Standard English listening
comprehension test, Self = the Self report scale, Exp = the Exposure scale, and Eval =
the Evaluation scale. *P<.05, **P<.01, ***P<.001
42
CHAPTER 8
DISCUSSION (STUDY 2)
8.1 Validation of Scales
Except for SE test, results from the present study indicated moderate to
moderately high internal consistency for every scales. Based on the Spearman-Brown
prophecy formula, about 11 more items were needed to have a reliability of .70 for SE
test. Principal component analyses suggested the unideminensionality of these scales.
8.2 Correlation among Scales
Correlation analyses suggested that the self-report scale was significantly
correlated to performance. That implies that the self-report reflects the WE listening
scale. This conflicts with the findings from Study 1. However, self-report is still far
from accurate measure of WE listening skill because it accounted for only 13% of
variance of the WE test (R2 (106)= .13). Second, those who had higher scores on the
listening test evaluated the fluency of the non-native speaker in the test better than those
who scored lower on the listening test as was the case in Study 1. Significant positive
correlation between WE listening test performance and self-report scale seems to be
incompatible with those of the study 1. That might be attributed to the relatively
43
unreliable WE scale in Study 1. The fact that WE scale in study 2 appeared to be better
scale might be attributed to the modification of items and larger numbers of items in the
scale.
8.3 Effects of Demographics on WE Listening Comprehension Scale
Based on the ANOVA and power analyses, demographics such as gender, age,
and nativity of English did not have a significant effect on listening test scores. Indeed,
effect sizes of demographics were very small. Notably, native speakers did not
outperform non-native speakers in the WE listening comprehension scale as compared
with other SE listening comprehension scales such as the TOEFL (Stricker, 2004) as the
Study 1. However, because of the unequal number of participants in the native (90
participants) and non-native (16 participants) groups, this finding was not strongly
generalizable.
44
CHAPTER 9
GENERAL DISCUSSION
9.1 Conclusions, Limitations, and Future Directions
Because of internal consistency, unidimensionality, and a weak correlation
between the WE and the SE listening scale, WE listening comprehension scale appears
to be valid test to measure WE listening proficiency. However, some modifications
should be considered in further investigations. First, more items should be included in
the WE listening scale in order to make it more useful and reliable. Second, the SE test
should include more items to perform meaningful analyses. Third, a more basic level of
WE such as a word level or phoneme level should be analyzed in the investigation of
WE comprehension because it was difficult to analyze data in the conversation level.
Fourth, the low correlation between the exposure scale and WE listening test may be
caused by the construction of the exposure scale. In other words, the exposure scale
measured exposure to non-English speaking culture in general. Since this current study
included only a Japanese speaker in WE listening scale as a non-native English speaker,
exposure to Japanese English speakers rather than exposure to general non-native
speakers should be measured. Fifth, it will be interesting to use self-report scale after
the WE listening test as well as prior to WE listening comprehension test. Finally,
45
participants may have guessed what the non-native speaker said by using native
speaker's speech as contextual cue in the conversation sections. Thus, participants
might have used native speaker as contextual cue to answer the questions. Therefore,
listening skill to comprehend conversation between non-native speakers' English rather
than conversations between non-native and native speaker should be measured in
conversation section. Logically that will make the listening scale more internally
consistent because listening skill in the comprehension of only a non-native speaker is
measured in the monologue and the sentence sections as opposed to conversation part
that included a native speaker.
46
APPENDIX A
WE LISTENING COMPREHENSION SCALE IN STUDY 1
47
Please answer the following survey and the WE listening comprehension
scale as honestly and accurately as possible. Please answer all parts of the
following survey. In this survey, non-native English speaker is defined
anyone whose first or dominant languages is not English.
Demographic Survey
1. Age ( )
2. Gender ( )
3. Country of Birth ( )
Part A: Answer the following questions based on your experience.
1. How many non-native English speakers do you know?
( )
2. How long have you known them in years (list oldest ones)?
( ) years
3. How close you are to the one you consider yourself closest to?
Not close Very
At all Neutral Close
1 2 3 4 5
4. How often do you see that one person?
Very Very
Rarely Neutral Often
1 2 3 4 5
5. How many times have you visited countries in which English is not the dominant
language?
( )
6. How many days have you spent in countries in which English is not the dominant
language?
( )7. How familiar are you with non-American culture?
Very Very
Unfamiliar Average Familiar
1 2 3 4 5
48
8. How familiar are you with the conversation of non-native English speakers?
Very Very
Unfamiliar Average Familiar
1 2 3 4 5
9. How familiar are you with the writing of non-native English speakers?
Very Very
Unfamiliar Average Familiar
1 2 3 4 5
10. How fluent are you in a language other than English?
Not at Very
All Average Fluent
1 2 3 4 5
Part B: Imagine an average non-native English speaker, and answer
following questions.
1. Rate your attitude towards this person’s English fluency.
Not Very Very
Fluent Neutral Fluent
1 2 3 4 5
2. How favorable is your view of non-English cultures?
Very Neutral Very
Negative Positive
1 2 3 4 5
3. Do you want to go to a country in which English is not the native language?
Yes No
1 2 3 4 5
4. Do you want to live in a country in which English is not the native language?
Yes No
1 2 3 4 5
5. Are you interested in a non-American culture?
Yes No
1 2 3 4 5
49
6. Do you pay much attention to foreign cultures?
Yes No
1 2 3 4 5
7. When you do not understand a non-native speakers’ English, do you think that it is
usually something wrong with their fluency?
Yes No
1 2 3 4 5
8. When you do not understand a non-native speakers’ English, do you think that it is
usually something wrong with your listening skill?
Yes No
1 2 3 4 5
9. Do you regard Midwestern American or as the standard form of international
English?
Strongly Strongly
Disagree Neutral Agree
1 2 3 4 5
10. Do you regard British as the standard form of international English?
Strongly Strongly
Disagree Neutral Agree
1 2 3 4 5
Part C: Imagine an average non-native English speaker, and answer
following questions.
1. How good are you at understanding the English of someone who is not a native
English speaker?
Very Very
Poor Average Good
1 2 3 4 5
2. On average, how difficult is it for you to understand the overall English of someone
who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
50
3. How often do you find it difficult for you to understand the overall English of
someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
4. On average, how difficult is it for you to understand the English grammar of
someone who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
5. How often do you find it difficult for you to understand the English grammar of
someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
6. On average, how difficult is it for you to understand the English pronunciation of
someone who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
7. How often do you find it difficult for you to understand the English pronunciation
of someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
8. On average, how difficult is it for you to understand the English accent of someone
who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
9. How often do you find it difficult for you to understand the English accent of
someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
51
Before you start Part D, please listen to conversation 1. Then please
answer following questions as accurately as you can.
1. What was the customer looking for?
a) T-shirt b) Statue of Liberty Models c) Some souvenir d) Pewter
model
2. For whom did the customer want the product?
a) Mother b) Father c) Grandmother d) Herself
3. Why did the customer not want the first product the clerk recommended?
a) She did not like the design.
b) She did not like the material.
c) She did not have enough money.
d) She did not think it’ appropriate.
4. Which product was the customer first interested in?
a) T-shirt b) Statue of liberty model c) Silver ring d) Chocolate gift
5. Which word did the customer not understand?
a) Statue b) Copper c) Plastic d) Pewter
6. Which kind of material of the product did the customer buy?
a) Wood b) Copper c) Plastic d) Pewter
7. What kind of wrapping did the customer want?
a) Birthday b) Gift c) Christmas d) Souvenir
8. Which country did the customer come from?
a) Japan b) Jordan c) Korea d) China
9. How many times has the customer visited the U.S.?
a) 1 b) 2 c) 3 d) 4
10. How long does the customer plan to stay in the U.S.?
a) 3 days b) 5 days c) 7 days d) More than 7 days
11. When will the customer leave?
a) Today b) Tomorrow c) 2 days from now d) Next week
12. Other than New York, where did the customer visit?
a) Los Angels b) Philadelphia c) Pittsburgh d) Washington D.C.
52
13. How long did the customer spend there?
a) A day b) 3 days c) A week d) 10 days
14. What was the customer’s purpose of visit to the U.S.?
a) Business b) To see her mother c) To study d) For fun
15. Why did the customer not take the tour?
a) She could not afford it. b) She did not like it.
c) She did not know about it. d) She did not have a chance.
16. How did the customer like New York?
a) She liked it. b) She did not like it.
c) She did not mention like or dislike. d) She liked D.C. better.
17. Which show did the customer watch?
a) Phantom of the Opera b) Miss Saigon c) Cats
d) She did not watch any.
18. Why did the customer go to Hard Rock Café?
a) It is her favorite place.
b) It was her first time to visit there.
c) She likes burger there.
d) She likes their goods.
Before you start Part E, please listen to Monologue 1. Then please answer
following questions as accurately as you can.
1. What is her occupation?
a) Instructor b) Counselor c) Professor d) Principal
2. Where did her husband come from?
a) Africa b) Japan c) Canada d) The U.S.
3. Why was she reading a book about cats?
a) She likes cats.
b) She is studying about cats.
c) It is her assignment.
53
d) She has a cat.
4. What is her dream?
a) To live in foreign countries b) To be a teacher
c) To marry her boy friend d) To write a book
Before you start Part F, please listen to Monologue 2. Then, please answer
following questions as accurately as you can.
1. According to the conversation speaker’s friend Yoko lives in.
a) America b) Russia c) Asia d) Europe
2. How does the speaker evaluate her friend Yoko?
a) She thinks Yoko is a great person.
b) She does not like Yoko.
c) Yoko is nice, but she has to develop more social skills.
d) Yoko cannot speak French.
3. What is the most difficult part of living in Switzerland?
a) She does not like winter.
b) She does not like the area where she lives.
c) Living costs are too expensive where she lives.
d) She does not like to work hard.
4.Which country did the speaker imply Yoko comes from?
a) Japan b) The U.S.A. c) China d) Switzerland
54
Before you start Part G, please listen to Sentences.
1. Why did she not go to the concert?
a) Tickets are too expensive
b) She did not like the band.
c) She did not have the time.
d) She could not get a ticket.
2. Why did she not go to John’s house?
a) It is too far for her. b) It is too cold.
c) She does not like John d) She does not have time to go.
3. How is she regarding Pennsylvania?
a) She is not there right now. b) She is there.c) She will live there. d) She
wishes to live there.
4. What does the sentence imply about Mary
a) One should feel sorry for her.
b) One should apologize to her.
c) One doesn’t need to apologize to her.
d) She should be ignored. .
Part H: Please answer the following questions.
1. Rate the speaker’s fluency in conversation 1.
Very Very
Poor Average Good
1 2 3 4 5
55
2. Rate the speaker’s pronunciation in conversation 1.
Very Very
Poor Average Good
1 2 3 4 5
3. Rate speaker’s accent in conversation 1.
Very Very
Poor Average Good
1 2 3 4 5
4. Rate the speaker’s grammar in conversation 1.
Very Very
Poor Average Good
1 2 3 4 5
5. Rate the speaker’s clarity in conversation 1.
Very Very
Poor Average Good
1 2 3 4 5
56
APPENDIX B
WE LISTENING COMPREHENSION SCALE IN STUDY 2
57
Appendix B: Modified WE Scales
Please answer the following survey and the WE listening comprehension
scale as honestly and accurately as possible. Please answer all parts of the
following survey. In this survey, non-native English speaker is defined
anyone whose first or dominant languages is not English.
Demographic Survey
1. Age ( )
2. Gender ( )
3. Country of Birth ( )
Part A: Answer the following questions based on your experience.
1. How many non-native English speakers do you know?
1) 1-3 2) 4-6 3) 7-9 4) 9-11 5) More than 11
2. How long have you known them in years (list oldest ones)?
1) 1-3 years 2) 4-6 years 3) 7-9 years 4) 9-11 years 5) More than 11
years
3. How close you are to the one you consider yourself closest to?
Not close Very
At all Neutral Close
1 2 3 4 5
4. How often do you see that one person?
Very Very
Rarely Neutral Often
1 2 3 4 5
5. How many times have you visited countries in which English is not the dominant
language?
1) 1-3 2) 4-6 3) 7-9 4) 9-11 5) More than 11
58
6. How many years have you spent in countries in which English is not the dominant
language?
1) 1-3 years 2) 4-6 years 3) 7-9 years 4) 9-11 years 5) More than 11
years
7. How familiar are you with non-American culture?
Very Very
Unfamiliar Average Familiar
1 2 3 4 5
8. How familiar are you with the conversation of non-native English speakers?
Very Very
Unfamiliar Average Familiar
1 2 3 4 5
9. How fluent are you in a language other than Standard English?
Not at Very
All Average Fluent
1 2 3 4 5
Part B: Imagine an average non-native English speaker, and answer
following questions.
1. How good are you at understanding the English of someone who is not a native
English speaker?
Very Very
Poor Average Good
1 2 3 4 5
2. On average, how difficult is it for you to understand the overall English of someone
who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
3. How often do you find it difficult for you to understand the overall English of
someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
59
4. On average, how difficult is it for you to understand the English grammar of
someone who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
5. How often do you find it difficult for you to understand the English grammar of
someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
6. On average, how difficult is it for you to understand the English pronunciation of
someone who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
7. How often do you find it difficult for you to understand the English pronunciation
of someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
8. On average, how difficult is it for you to understand the English accent of someone
who is not a native English speaker?
Very Very
Difficult Neutral Easy
1 2 3 4 5
9. How often do you find it difficult for you to understand the English accent of
someone who is not a native English speaker?
Very Not often
Often Neutral At all
1 2 3 4 5
Before you start Part C (question 1-17), please listen to Part C. You
may listen to the Part C only once. Do not look at or start answering
the questions until you finish listening Part C.
60
1. What was the customer looking for?
a) T-shirt b) Statue of Liberty Model c) Some souvenir d) Specific
souvenir
2. For whom did the customer want the product?
a) Mother b) Her friend c) Grandmother d) Herself
3. Why did the customer not want the first product the clerk recommended?
a) She did not like the design.
b) She did not like the material.
c) It was too expensive.
d) She did not think it’s appropriate.
4. Which product was the customer first interested in?
a) T-shirt b) The medium model c) Silver ring d) The large
model
5. Which word did the customer not understand?
a) Statue b) Copper c) Souvenir d) Pewter
6. Which kind of material of the product did the customer prefer?
a) Pewter b) Copper c) Plastic d) None of the above
7. What kind of wrapping did the customer want?
a) Birthday b) Gift c) Christmas d) Paper
8. How many times has the customer visited the U.S.?
a) 1 b) 2 c) 3 d) 4
9. How long does the customer plan to stay in the U.S.?
a) 3 days b) 5 days c) 7 days d) More than 7 days
10. When will the customer leave?
a) Today b) Tomorrow c) 2 days from now d) Next week
11. Other than New York, where did the customer visit?
a) No where b) Hard Rock Cafe c) Broadway d) Washington D.C.
12. How long did the customer spend there?
a) A day b) 3 days c) A week d) 10 days
13. What was the purpose of visiting there?
a) For shopping b) For museum c) For sightseeing d) For the
tour
61
14. What did she buy there?
a) Souvenir b) Ticket c) Food d) A badge
15. What was the customer’s purpose of visit to the U.S.?
a) Business b) To see her mother c) To shop d) For fun
16. Why did the customer not take the tour?
a) She could not afford it. b) She wanted to go by herself.
c) She did not know about it. d) She did not have a chance.
17. Where she did not go?
a) Steak house b) Museum
c) White House d) Statue of Liberty
18. How did the customer have fun in New York?
a) Food b) Musical c) Sightseeing d) Museum
19. Why did the customer go to Hard Rock Café?
a) It is her favorite place.
b) It was her first time to visit there.
c) She likes burger there.
d) She likes their goods.
Before you start Part D (question 20-27), please listen to Part D. You
may listen to the Part D only once. Do not look at or start answering
the questions until you finish listening Part D.
20. What did she expect that his view of Japanese food?
a) Delicious b) Too heavy c) Too light d) Too salty
21. What did she think about Japanese foods in the U.S.?
a) Delicious b) Too heavy c) Too light d) Too salty
22. What did they disagree about the tastes of rice and tofu?
a) He insisted rice and tofu have taste. b) She insisted that rice and tofu
have taste.
c) She does not like rice and tofu. d) She does not like rice and tofu.
62
23. How did she think about his sense of tasting foods?
a) He does not have good sense of tasting any foods.
b) He has good sense of tasting any foods.
c) He does not have good sense of tasting Japanese food.
d) He has a good sense of American foods.
24. How did she evaluate popular American-Japanese food?
a) Delicious b) Too heavy c) Too light d) Too salty
25. Which part of his explanation of difference between Japanese and American recipes
she did not understand?
a) Metric system b) Ingredients c) Type of a food d) Cooking
procedure
26. What was her example of wrong American food in Japan?
a) American coffee b) American tea c) American burger d) American
juice
27. How did they settle down about argument?
a) They accepted differences between cultures. b) They did not ague.
c) They changed topic. d) He accepted her view.
Before you start Part E (question 28-31), please listen to Part E. You
may listen to the Part E only once. Do not look at or start answering
the questions until you finish listening Part E.
28. What is her occupation?
a) Instructor b) Counselor c) Professor d) Principal
29. Where did she get married?
a) Africa b) Japan c) The U.S. d) None of the above
30. Why was she reading a book about cats?
a) She likes cats.
b) To write a book.
63
c) It is her assignment.
d) She has a cat.
31. What is her dream?
a) To write a book about cats b) To be a teacher
c) To marry her boy friend d) None of the above
Before you start Part F (question 32-33), please listen to Part F. You
may listen to the Part F only once. Do not look at or start answering
the questions until you finish listening Part F.
32. Why did her husband wanted to see the chef?
a) Because he is a great chef. b) Because he is a TV star.
c) Because he is a famous chef. d) The speaker did not mention why.
33. What kind of restaurant was it?
a) Japanese b) American c) Chinese d) Korean
Before you start Part G (question 34-37), please listen to Part G. You
may listen to the Part F only once. Do not look at or start answering
the questions until you finish listening Part F.
34. According to the conversation speaker’s friend Yoko lives in.
a) America b) Russia c) Asia d) Europe
35. How does the speaker evaluate her friend Yoko?
a) She thinks Yoko is a great person.
b) She does not like Yoko.
c) Yoko is nice, but she has to develop more social skills.
d) Yoko cannot speak French.
64
36. What is the most difficult part of living in Switzerland?
a) She does not like winter.
b) She does not like the area where she lives.
c) Living costs are too expensive where she lives.
d) She does not like to work hard.
37. Which country did the speaker imply Yoko comes from?
a) Japan b) The U.S.A. c) China d) Switzerland
Before you start Part H (question 38-42), please listen to Part H. You
may listen to the Part H only once. Do not look at or start answering
the questions until you finish listening Part H.
38. Why did she not go to the concert?
a) Tickets are too expensive.
b) She did not like the band.
c) She did not have the time.
d) She could not get a ticket.
39. Why did she not go to John’s house?
a) It is too far for her. b) She can go there any time.
c) She does not like John d) She does not have time to go.
40. How is she regarding Pennsylvania?
a) She is not there right now. b) She is there.
c) She will live there. d) She wishes to live there.
65
41. What does the sentence imply about Mary?
a) One should feel sorry for her.
b) One should apologize to her.
c) One doesn’t need to apologize to her.
d) She should be ignored.
42. According to the speaker
a) This fall will be colder.
b) Usually, it is colder in fall.
c) It is a nice day.
d) She complaints about warm weather.
Before you start Part I (question 43-), please listen to Part I. You may
listen to the Part H only once. Do not look at or start answering the
questions until you finish listening Part H.
43. Why does the second speaker wanted to go to England?
a) Because it was on magazine.
b) To see gardening plant.
c) To see a person.
d) Because of special event.
44. Why does the first speaker want to go to Cambodia?
a) Because it was on magazine.
b) To see gardening plant.
c) To see a person.
d) Because of special event.
45. Which country was not mentioned by them?
a) Japan
b) England
66
c) Jordan
d) Iran
46. Which places they agreed that they want to go?
a) Parts of the U.S.
b) Asian countries
c) Egypt
d) Both a and c
47. Which place at least one of them has been to?
a) Cambodia
b) Egypt
c) Hawaii
d) England
Part J: Please answer the following questions.
1. Rate the non-native speaker’s fluency in the Part C to H.
Very Very
Poor Average Good
1 2 3 4 5
2. Rate the non-native speaker’s pronunciation in the Part C to H.
Very Very
Poor Average Good
1 2 3 4 5
3. Rate the non-native speaker’s accent in the Part C to H.
Very Very
Poor Average Good
1 2 3 4 5
67
4. Rate the non-native speaker’s grammar in the Part C to H.
Very Very
Poor Average Good
1 2 3 4 5
5. Rate the non-native speaker’s clarity in the Part C to H.
Very Very
Poor Average Good
1 2 3 4 5
68
APPENDIX C
WE CONVERSATION SCRIPTS
69
Conversation 1
A: Hi, may I help you?
B: Oh, yes please.
A: Are you looking for something in particular?
B: Yes. I want to buy a souvenir for my mom.
A: How about an “I love New York T-shirt” ?
B: Well, my mom doesn’t wear T-shirts. Do you have anything else?
A: I see. We have some Statue of Liberty models.
B: It sounds good. Can I see them?
A: Sure, just over here. We have a lot of different sizes.
B: .........
A: Do you see any that you like?
B: Mmm, this middle size one looks good for her. I’ll take this one.
A: O.K. That one’s pewter, so it’s 15 dollars.
B: What’s pewter?
A: Like silver but soft.
B: Do you have any different materials?
A: Of course. We have gold, silver, wood, copper, plastic. What would you like?
B: Which do you recommend?
A: This copper one is the most realistic.
B: How much is it?
A: This one’s 12 dollars.
C: O.K. I’ll take it.
G: Would you like that gift-wrapped?
C: Oh, can you? Please.
A: No problem. Birthday paper or Christmas paper or anniversary or just normal?
B: Well, Christmas one, please.
A: Sure. Just a moment, please.....So, where are you from?
70
B: I'm from Japan. Have you been to Japan?
A: No, I haven't. Is this your first time to America?
B: Yes. I've been here for 10 days, and I'll leave the day after tomorrow.
A: Are you here on vacation?
B: Yes, I am.
A: Have you spent the whole time in New York, or did you go anywhere alse?
B: I went to Washington D.C.
A: How long did you spend there?
B: For 3 days. I went to see the White House.
A: DId you take a tour?
B: I wanted to. But I couldn' t make a reservation.
A: Oh well, did you go to the Smithsonian?
B: Well I don't know it.
A: It's the big group of museums near the capital building.
B: O.K. I'll go there when I go to Washington D.C. next time.
A: So, you spent the other 7 days in New York?
B: Yes. I enjpyed New York very much.
A: Thanks. So, did you see a Broadway show?
B: Yes, I saw "Cats". It was wonderful.
A: Yeah,everyone likes that one. Are you stying in Manhattown?
B: Yes.
A: Did you go to the Hard Rock Cafe?
B: Yes. I like Hard Rock Cafe. I collect their badges.
A: How many do you have?
B: I have 7. 5 are fom Japan,1 is from Washinton D.C. 1 is from New York.
A: Are you from Tokyo?
B: No, I'm from Osaka, third largest city in Japan.
A: Oh, I'm sorry. O.K. Here you go. One gift-wrapped Statue of Liberty.
71
B: O.K. Here you go.
A: Thank you, and 3 dollars is your change. Have a nice day.
b: Thank you.
72
Conversation 2
A: What did you think the first time you ate Japanese food?
B: Ahh, let’s see, that was when I was in high school… I thought “Wow, this is good. It’s kind of like Chinese food.”
A: Didn’t you think that Japanese food’s flavor was light?
B: Not really, it was pretty salty and strong.
A: Because you had it in America. When I went to America, I thought “This isn’t real Japanese food.”
B: Yeah but I’ve had Japanese food in Japan that’s had pretty strong flavors, haven’t you?
A: Yeah, some are salty. But when I had food in America, everything tasted heavy. Actually your cooking tastes a little heavy.
B: That’s just because you’re used to bland foods like rice and tofu.
A: You usually say that rice and tofu have no flavor but they do! I can tell! I can taste it!
B: It’s just your imagination.
A: You can ask your students! I’m sure they’ll agree with me. And also, water has flavor.
B: Only if the flavor is added, like that C1000 Lemon Water.
A: You don’t understand.
B: Don’t understand what?
A: Even though you like cooking and going to good restaurants, why can’t you tell their flavor? You can tell Coke’s flavor at McDonald’s and so on. Like the syrup is too little or too much.
B: That’s different, that’s like how you can tell the difference between my ribs with five spice powder and the ribs without it.
A: OK, you’ll understand some day, I hope. By the way, what did you have the first time you had Japanese food?
B: I had teppanyaki, at a restaurant in Fort Wayne (the city we lived in in Indiana)..
A: Teppanyaki isn’t so Japanese for me. I know it’s popular in America but Japanese food is tempura or sushi for most Japanese. Isn’t tempura famous in America?
B: It’s kind of famous but less than sushi or teppanyaki. I had heard of sukiyaki and teriyaki in America, too. I had teriyaki before coming to Japan but not sukiyaki.
Anyway don’t you think saying Japanese food is sushi and tempura is stereotyping? Don’t you want other Japanese food to be famous?
73
A: Yes, other Japanese food can be famous, but also I’m proud of sushi and tempura. What Japanese food can be popular in America you think?
B: I think fried noodles and okonomiyaki could be popular. I think yakiniku could be really popular, too, but it probably won’t have a chance because of too many safety regulations.
A: Yeah, okonomiyaki is very good, and battered octopus balls. OK, we can open a restaurant in America!
B: With okonomiyaki and octopus balls? No way, too Osaka! How about Nagoya-style wings and noodles? You know the wings would be popular, remember when I brought them to that party?
A: I liked your wings, but you know I don’t like Nagoya so much. By the way, you read recipes in Japanese and in English. Is there any difference between them?
B: Only in the names of the ingredients and in the measurements, since Japan uses the metric system.
A: What’s the metric system?
B: You know, meters, grams, liters…
A: What system does America use?
B: We use the English system. Feet, pounds, ounces… Anyway, you know a lot of so-called American food in Japan is wrong too, right?
A: You mean American coffee? Light-flavored?
B: That’s one example. It’s like hot brown water, real American coffee is definitely not like that.
A: Things from overseas can be different in different countries.
B: So you shouldn’t complain about Japanese food in America then, right?
A: OK, I’ll try to not complain.
74
Monologue 1
Hello. I am going to tell about myself. I am a teacher for a cram school. Teaching is
interesting for me. And I am married. My husband is American. His name is Gabriel.
He is very sweet. And I like reading books. Now I am reading a novel about cats. I like
cats. Sometime I want to have cats. At last, I will tell you about my dream. I want to be
a translator someday. I want to publish my books. Thank you.
Monologue 2
Hello, I am going to tell you about my experience. Last Tuesday was my husband’s
birthday. So, we went to a good restaurant in Nagoya last Sunday. It’s a famous chef’s
Chinese restaurant. His name is Chin Kenichi. He became famous from Japanese TV
program, Iron Chef. We really enjoyed lunch. When we paid, my husband asked the
staff if Mr. Chin was there. But, later the staff came to us and said “it’s actually secret,
but if you wan to see Chin, He will be here on December 19th and 20
th. We were very
happy to hear it. Probably my husband will go there that day. Thank you.
Monologue 3
Hi, I will tell you about my friend Yoko. I’ve known her for 5 years. Now, she lives in
Switzerland because her husband is Swiss. It can be difficult to live in foreign
countries. And, so it is for Yoko. First, she speaks French there. She had to study
French very hard. Second, it’s very cold in winter and winter starts much faster then in
Japan. Third, everything is expensive there. And she doesn’t like the new or different
things. So it is difficult to get Japanese things. There are some more reasons, but Yoko
always does her best. She never gives up. So, I really respect her. Thank you.
Sentence 1
I would have gone to the concert if I had had a ticket.
Sentence 2
I don’t like to go to John’s house because it’s 5 miles from here.
Sentence 3
I dreamed about living in Pennsylvania.
Sentence 4
You don’t have to say sorry to Mary.
Sentence 5
It’s warm for fall.
75
REFERENCES
Bernstein, I.H., Rush, A.J., Carmody, T.J., Woo, A., & Trivedi, M.H. (2007).
Clinical vs. self--report versions of the quick inventory of depressive symptomology in
a public sector sample. Journal of Psychiatric Research, 41, 239-246.
Boldt, R. F., Larsen-Freeman, D., Reed, M.S., & Courtnery, R.G. (1992).
Distributions of ACTFL Ratings by TOEFL Score Ranges. TOEFL Research Report,
92(59).
Bollag, Burton. (2005). New Test of English as a Foreign language Puts an
Emphasis on Speaking. Chronicle of Higher Education, 52(2), A49.
Bouton, F. L. (1994). Conversational implicature in a second language:
Learned slowly when not deliberately taught. Journal of Pragmatics, 22, 157-167.
Chalhoub-Deville, M., & Wiggleworth, G. (2005). Rater judgment and English
Language speaking proficiency. World Englishes, 24 (3), 383-391.
Conian, D. (2001). The use of audio or video comprehension as an assessment
instrument in the certification of English language teachers. System, 29 (1) , 1-14.
Crystal, D. (2000). Emerging Englishes. English Teaching Professional, (14),
3-6.
Davies, A., Hamp-Lyons, L., & Kemp, C. (2003). Whose norms? International
proficiency tests in English. World Englishes, 22 (4). 571-584.
76
Educational Testing Service (ETS). (2005). Test of English as a Foreign
Language.
Ervin-Tripps, S. (1970). Discourse agreement: How children answer questions.
In Hays (Ed.), Cognition and the development of language. New York: Wiley.
Flowerdrew, J., & Miller, L. (1997). The Teaching Academic Listening
Comprehension and the Question of Authenticity. English for Specific Purposes,16(1),
27-46.
Grice, H. P. (1975). Logic and conversation. In P. Cole & J. Morgan (Eds.),
Speech acts: syntax and semantics. (vol.3, pp41-58). New York: Academic Press.
Harris, J. (1994). English Sound Structure. Cambridge, MT: Blackwell.
Hinkel, E (ed.) (2005). Handbook of Research in Second Language Teaching
and Learning. Mahawah, NJ: LawrenceErlbaum associates.
Johnson, D. W., & Newport, E. L. (1989). Critical period effects in second
language learning: The influence of maturational state on the acquisition of English as a
second language. Cognitive Psychology, 21, 60-99.
Kachuru, Y. (1985). Discourse Analysis, non-native Englishes and second
language acquisition research. World Englishes, 4 (2), 223-232.
Kertoy, M. K., & Goetz, K. M. (1995). The Relationship between Listening
Performance on the Sentence Verification Technique and Other Measures of Listening
Comprehension. Contemporary Educational Psychology, 20, 320-339.
77
Lindemann, S. (2002). Listening with an attitude: A model of native-speaker
comprehension of non-native speakers in the United States. Language in Society, 31(3),
419-441.
Major, R. C., Fitzmaurice, S. M., Bunta, F., & Balsubramanian, C. (2005).
Testing the Effects of Regional, Ethnic, and International Dialects of English on
Listening Comprehension. Language Learning, 55(1), 37-69.
Mecalfe, J. (1986). Feeling of Knowing in Memory and problem Solving.
Journal of Experimental psychology. 12 (2), 288-294.
Nunnly J. C., & Bernstein I. H. (1994). Psychometric Theory (3rd). New York:
McGRAW-HILL.
Oltman, K. P., Stricker, J. L., & Barrows. S. T. (1990). Analyzing Test
Structure by Multidimensional Scaling. Journal of Applied Psychology, 75 (1), 21-27.
Powers, D. E., Scheldi, M. A., Leung, S. W, & Butler, F. A. (1999). Validating
the revised test of spoken English against a criterion of communicative success.
TOEFL Research Report, 99 (5), 63.
Read, J. (2002). The use of interactive input in EAP listening assessment.
Journal of English for Academic Purposes, 1(2), 105-119.
Richard, E. M., Diefendorff, J, M., Martin, J, H., (2006). Revisiting the Within-
Person Self-Efficacy and Performance Relation. Human Performance, 19(1), 67-87.
Rosenfeld, M. Sheppard, P. K., & Sheppard, K. (2004). Investigating the
Validity of TOEFL: A Feasibility Study Using Content and Criterion Relater Studies.
TOEFL Research Report, 3 (18), 71.
78
Shameem, N., (1998). Validating self-reported language proficiency by testing
performance in an immigrant community: the Wellington Indo-Fijians. Language
Testing, 15(1), 86-108.
Snow, S. M. (1998). Economic, Statistical, and linguistic factors affecting
success on the test of English as a foreign language (TOEFL). Information Economics
and Policy, 10 (2), 159-172.
The Society for Testing English Proficiency (STEP). (2004).
Stricker, L. J. (2004). The performance of native speakers of English and ESL
speakers on the computer-based TOEFL and GRE General Test. Language Testing, 21
(2),146-173.
Tabachinick, B. G., & Fidell, L, S. (2001). Multivariate Statistics (4th).
Needham Heights: Allyn and Bacon.
Tabachinick, B. G., & Fidell, L, S. (2007). Multivariate Statistics (5th).
Needham Heights: Allyn and Bacon.
79
BIOGRAPHICAL INFORMATION
Yasushi Kyutoku earned a high schools diploma from Seifu High School
(Osaka, Japan) in March, 1992, and earned B.A. in psychology from the University of
Texas at San Antonio in December, 2001. After the completion of thesis research, he
will continue to study experimental psychology at the University of Texas at Arlington.
He is interested in continuing to conduct researches related to psychometrics such as
validation and application of scales.
top related