Language Testing

Quiz

• Quiz is an instrument with very limited questions, based on the daily works of the students, in order to check initial comprehension of any segment of material, not final mastery.

• Quiz enables the teacher to find out whether the students have learnt the material or not, in order to improve the students' knowledge on those areas.

• Moreover, the quiz enables the teacher to acquaint (i.e., familiarize) students with types of items that will subsequently be used in test; quiz can help students in their studies by allowing them to see area of emphasis in the course, as well as, the types of items which will be included in the final exam.

• The teacher also will be benefited from the result, by understanding their students' strengths and weaknesses, in order to modify his/her teaching, and improve the learner’s ability.

• Quiz can be unannounced, which may pop up in the class at any moment.

Testing

• Testing is “a method of measuring a person’s ability, knowledge, or performance in a given domain (i.e., a specific sample of behavior)”. (Brown, p.3).

• First,it is a method: it is an instrument- a set of technique, procedures or items- that requires performance on the part of the test taker.

• Second, a test measures. Some tests measure general ability while others focus on very specific competencies or objectives.

• Third, a test measures an individual’s ability, knowledge, or performance.

• Fourth, a test measures a given domain; for example, a test of pronunciation should have items to only measure pronunciation, e.g., by a set of phonetic minimal pairs. A vocabulary test may focus only on the set of words covered in a particular lessons or units.

•

Measurement

• Measurement is “a process of quantifying the characteristics of persons according to explicit procedures or rules”. (Bachman, p.18). This definition includes three distinguishing features:

• 1. Quantification involves the assigning of numbers, and this distinguishes measurement from qualitative descriptions such as verbal accounts, and visual representations. Thus non-numerical categories or ranking such as letter grading (i.e., A, B …) or labels (e.g., excellent, good, average…) will be excluded in the definition of measurement.

• 2. Characteristics are mental attributes and abilities, sometimes called traits or constructs.

• 3. Rules and procedures which means that the ‘blind’ or haphazard assignment of numbers of characteristics of individuals cannot be regarded as assignment. Therefore, quantification must be done according to explicit rules and procedures.

Evaluation

• Evaluation can be defined as “the systematic gathering, both qualitatively and quantitatively, of information for the purpose of making decision” (Weiss, 1972). The information relevant to evaluation can be either qualitative (i.e., non-measurement, e.g., observation, rating) or quantitative measurement (e.g., tests). The evaluation of individuals may involve decisions about entrance of programs, placement, progress, and achievement.

Assessment

• Assessment is collecting, and gathering information through different ways, such as interview, examination, etc., which encompasses a much wider domain than evaluation.

• When a student responds to a question, offers a comment, or tries out a new word or structure, the teacher subconsciously makes an assessment of the student’s performance” (Brown, p.4).

• The difference between test and assessment is that a test is a prepared administrative procedure that occurs at peak performance, knowing that their responses are being measured and evaluated, while assessment can happen at every stage of the course, not just at the peak of performance.

Formative and Summative Assessments

• formative assessment is the ongoing evaluation of students’ progress in the course of teaching, with the aim of understanding the strength and weaknesses of the students, and improving the learner’s ability in the given domain.

• In other words, this kind of assessment is given periodically during a course or instruction to monitor student learning progress and to provide ongoing feedback to students and teacher.

• A formative test typically covers some predefined segment of instruction (e.g. unit, chapter, or particular set of skills), and thus encompasses a rather limited sample of learning tasks.

• The test items may be easy or difficult depending on the learning tasks in the segment of instruction being tested.

• Summative assessment, on the other hand, is a test given at the end of an instructional course which is intended to measure or ‘sum up’ how much a student has learned from a particular course.

• The results of a summative test can also be used for evaluating the usefulness of the instruction.

• A summative test is usually a graded test, i.e., it is marked according to a scale or set of grades.

The pre-scientific movement (Grammar Translation method, or, Essay Translation

Approach)• This method is closely related to the early stages of language testing. That

is, the intuitive (i.e., subjective) stage of language testing• no special skill or expertise in testing is required. That is, testing has been

done based on the teacher's impression, without a specified language testing theory.

• Thus the subjective judgment of the teacher was very crucial. Many teachers in that era emphasized knowing about the language (i.e. the rules of language was very important), without understanding the reliability of the test.

• They emphasized on memorization, memorization of language rules and list of words. Traditional tests also include composition (essay writing), translation (from L1 to L2 or vice-versa), grammatical analysis, spelling, dictation and reading comprehension.

• One problem that arises with these types of test is that they are relatively difficult to score objectively.

Sample Tests• 1. Translate the following sentence from English

to Persian: The man ran out of money.• 2. Write a composition about air pollution.• 3. Fill in the blank with the proper form of the

verb: He………… to the shop tomorrow. (to go)

• The problems with this kind of test are that in addition to lack of consistency in scoring procedures, they also lack of concern about making fair, consistent, and correct decisions about the lives of the students involved.

Psychometric-Structuralism movement (Audio-Lingual method)

• This method is closely affected by behaviorist psychology and structural linguistics.

• Based on behaviorist psychology language is a set of habits, by repeating an utterance or an action you can learn it.

• Also it emphasized that learning happens bit by bit. Structural linguistics emphasizes on language usage, rather than language use. It started by analyzing the components of language (e.g. sounds. morphemes, words, syntax).

• Accordingly objective test were devised, especially discrete-point test, to measure the component of language (listening, speaking, reading, and writing).

The integrative –sociolinguistic movement

• This approach involves the testing of language in context and it, thus, concerned primarily with meaning, the total communicative effect of discourse.

• Beginning with the work of sociolinguistics like Hymes (1967), it was felt that the development of communicative competence depended on more than simple grammatical control of the language; communicative competence also depends on the knowledge of using language appropriately in different situations.

• This approach is best characterized by the use of cloze testing and of dictation, both of which assess the student’s ability to manipulate language within a context of extended text rather than in a collection of discrete-point questions.

• Thus, the principle of cloze testing is based on the Gestalt theory of closure; Gestalt psychologists believe that you cannot understand a person’s response to a situation in terms of a combination of separate stimuli, but that it should be studied as a whole response to the whole situation.

The Communicative Approach to Language Testing

• Communicative tests are tests which require candidates to perform communicatively like that of real situation.

• In such tests, we evaluate samples of performance in certain specific contexts of language use, created under particular test constraints, for what they can tell us about a candidate’s communicative capacity or language ability.

• Communicative approaches, like structural approach, believe that language proficiency is divisible, and try to break the language into different functions and notions, and study them outside the context.

• They believe that by mastering these functions and notions, learners can use them effectively in real life situation.

Canale and Swain (1980) Communicative Competence

• 1. Grammatical competence• Grammatical competence refers to the degree to which

the language user has mastered the linguistic code (i.e., knowledge of vocabulary, rules of pronunciation and spelling, word formation, and sentence structure).

• 2. Sociolinguistic competence• Sociolinguistics competence refers to the degree to which

the language users has mastered in using and understanding linguistic codes appropriately in various contexts to convey specific communicative functions, such as describing, narrating, persuading, eliciting information, and the like.

• 3. Discourse competence• It is the ability to combine ideas to achieve cohesion in form and

coherence in thought. A person who has a highly developed degree of discourse competence will know how to use cohesive devices, such as pronouns and grammatical connectors (i.e., conjunctions, adverbs, and transitional phrases), to achieve unity of thought and continuity in a text. The competent language user will also be skilled in expressing and judging the relationships among the different ideas in a text (coherence)

• 4. Strategic competence• It involves using verbal and nonverbal communication strategies to

compensate for gaps in the language users’ knowledge of the code or for breakdown in communication because of performance factors .

Components of language competence

• Organizational competence• Organizational competence comprises those

abilities involved in controlling the formal structure of language for producing or recognizing grammatically correct sentences, comprehending their propositional content, and ordering them to form texts. These abilities are of two types: grammatical and textual

Organizational Competence– Grammatical competence

• Grammatical competence includes those competencies involved language usage. And it consists of knowledge of vocabulary, morphology, syntax, and phonology/graphology. These govern the choice of words to express specific significations, their forms, and their arrangement in utterances to express propositions, and their physical realization, either as sounds or as written symbols.

• – Textual competence

• Textual competence includes the knowledge of the conventions for joining utterances together to form a text- spoken or written- consisting of two or more utterances or sentences that are constructed according to rules of cohesion and rhetorical organization. Cohesion comprises ways of explicitly marking semantic relationships such as reference, substitution, ellipsis, conjunctions, and lexical cohesion, as well as conventions such as those governing the ordering of old and new information in discourse. Rhetorical organization pertains to the overall conceptual structure of a text, and is related to the effect of the text on the language user. Conventions of rhetorical organization include common methods of development such as narration, description, comparison, classification, and process analysis.

Pragmatic competence

• Pragmatic competence related to knowledge of using language in different situation appropriately.

• Van Dijk (1977) distinguished two aspects of pragmatics: (1) the examination of the ‘pragmatic conditions’ that determine whether or not a given utterance is acceptable to other users of the language as an act, or the performance of an intended function; and (2) the characterization of the conditions that determine which utterances are successful in which situation.

• Pragmatics is thus concerned with the relationship between utterances and the acts or function that speakers (or writers) intend to perform through these utterances.

• According to Bachman and Palmer(1982a) pragmatic competence consisted of the following component.

Pragmatic Competence– Illocutionary competence

• Illocutionary competence is the knowledge which a person has in order to express his intended meaning, whether directly or indirectly. A speaker can signal his intent directly, by announcing its illocutionary force (I request that you leave now) or by using appropriate syntactic form such as the imperative (‘Leave), or by using less direct strategies, by using declarative sentences (e.g., It’s nearly midnight)– Sociolinguistic competence

• Sociolinguistic competence related to the appropriateness of using language in different contexts and discourse. It enables us to perform language functions in ways that are appropriate to that context. Regional and social varieties, or dialects, for instance can be characterized by different conventions, and the appropriateness of their use will vary, depending on the feature of the language use context.

• •

Communicative language ability

• Communicative language ability (CLA) can be described as consisting of both knowledge, or competence, and the capacity for implementing, or executing that competence in appropriate, contextualized comunicative language use.

1. language competence

• Language competence comprises, essentially, a set of specific knowledge components that are utilized in communication via language.

• Grammatical competence includes those competencies involved in language usage, as described by Widdowson (1978).

• These consist of a number of relatively independent competencies such as the knowledge of vocabulary, morphology, syntax, and phonology/graphology.

• These govern the choice of words to express specific significations, their forms, their arrangement in utterances to express propositions, and their physical realizations, either as sounds or as written symbols.

• Suppose, for example, a test taker is shown a picture of two people, a boy and a taller girl, and is

asked to describe it. In so doing, the test taker demonstrates her lexical

competence by choosing words with appropriate significations (boy, girl, tall) to refer to the contents of the picture. She demonstrates her knowledge of morphology by affixing the inflectional morpheme (-er) to ‘tall’.

• She demonstrates her knowledge of syntactic rules by putting the words in the proper order, to compose the sentence ‘The girl is taller than the boy’.

Textual Competence

• Textual competence includes the knowledge of the conventions for joining utterances together to form a text, which is essentially a unit of language - spoken or written - consisting of two or more utterances or sentences that are structured according to rules of cohesion and rhetorical organization.

• Cohesion comprises ways of explicitly marking semantic relationships such as reference, subsituation, ellipsis, conjunction, and lexical cohesion (Halliday and Hasan 1976),

• as well as conventions such as those governing the ordering of old and new information in discourse.

Rhetorical organization

• Rhetorical organization pertains to the overall conceptual structure of a text, and is related to the effect of the text on the language user (van Dijk 1977).

• Conventions of rhetorical organization include common methods of development such as narration, description, comparison, classification, and process analysis (McCrimman 1984).

Pragmatic Competence

• Pragmatics is concerned with the relationships between utterances and the acts or functions that speakers (or writers) intend to perform through these utterances,

Language functions

• Ideational function, by which we express meaning in terms of our experience of the real world (Halliday 1973). This includes the use of language to express propositions or to exchange information about knowledge or feelings. For example, language is used ideationally to present knowledge in lectures or scholarly articles.

• It is also ideationally used to express feelings, as when one pours out one’s emotions to a good friend or in a diary, with or without any intention of eliciting advice or help.

Manipulative

• The manipulative functions are those in which the primary purpose is to affect the world around us.

• instrumental function with which we use language to get things done. For example, we may get so ne, including ourselves, to do something by forming or uttering suggestions, requests, orders, commands, or warnings.

• The regulatory function is used ‘to control the behavior of others

• - to manipulate the persons and, with or without their help, the objects in the environment’ (Halliday 1973:18).

• In addition, this function is performed in formulating and stating rules, laws, and norms of behavior.

• The interactional function of language is its use to form, maintain, or change interpersonal relationships.

• Phatic language use, such as in greetings, ritual inquiries about health, or comments on the weather, is primarily interactional in function.

• The imaginative function of language enables us to create or extend our own environment for humorous or esthetic purposes, where the value derives from the way in which the language itself is used. Examples are telling jokes, constructing and communicating fantasies, creating metaphors or other figurative uses of language, as well as attending plays or films and reading literary works such as novels, short stories, or poetry for enjoyment.

• The heuristic function pertains to the use of language to extend our knowledge of the world around us, and occurs commonly in such acts as teaching, learning, problem solving, and conscious memorizing.

• Teaching and learning may be either formal, as in an academic setting, or informal, as in self-study.

Sociolinguistic competence

• Sociolinguistic competence is the sensitivity to, or control of the conventions of language use that are determined by the features of the specific language use context; it enables us to perform language functions in ways that are appropriate to that context.

Sensitivity to differences in dialect or variety

• These regional and social varieties, or dialects, can be characterized by different conventions, and the appropriateness of their use will vary, depending on the features of the language use context.

Sensitivity to naturalness

• ‘I wish you wouldn’t do that’ with ‘I would feel better by your not doing that’, or ‘I have my doubts’ with ‘I have several doubts’.

Sensitivity to differences in register

• ‘register’ refer to variation in language use within a single dialect or variety.

• field of discourse (lecture, discussion, written expression, registers of playing football)

• mode of discourse (spoken /written)• style of discourse• whether it be spoken, as in lectures or job

interviews, or written, as in business, letters, job announcements, or scholarly papers

Ability to interpret cultural references and figures of speech• knowledge of the extended meanings given by

a specific culture to particular events, places, institutions, or people is required whenever these meanings are referred to in language use.

• A: I hear John didn’t do too well on his final exam.

• B: Yeah, it turned out to be his Waterloo.

• The conventions governing the use of figurative language, as well as the specific meanings and images that are evoked are deeply rooted in the culture of a given society or speech community.

Strategic competence

• The interactional definition, as stated by Tarone (198 l), characterizes a communication strategy as ‘the mutual attempt by two interlocutors to agree on a meaning in situations where the requisite meaning structures do not seem to be shared’ (p. 288).

• production strategy ‘an attempt to use one’s linguistic system efficiently and clearly, with a minimum of effort’ (p. 289). Like communication strategies, production strategies are distinct from the language user’s language competence.

• Strategic competence: mastery of verbal and nonverbal strategies both

• (a) to compensate for breakdowns in communication due to insufficient Competence or to performance limitations

• (b) to enhance the rhetorical effect of utterances.(Canale 1983:339)

Organizational Competence– Grammatical competence

• Grammatical competence includes those competencies involved language usage. And it consists of knowledge of vocabulary, morphology, syntax, and phonology/graphology. These govern the choice of words to express specific significations, their forms, and their arrangement in utterances to express propositions, and their physical realization, either as sounds or as written symbols.

• – Textual competence

• Textual competence includes the knowledge of the conventions for joining utterances together to form a text- spoken or written- consisting of two or more utterances or sentences that are constructed according to rules of cohesion and rhetorical organization. Cohesion comprises ways of explicitly marking semantic relationships such as reference, substitution, ellipsis, conjunctions, and lexical cohesion, as well as conventions such as those governing the ordering of old and new information in discourse. Rhetorical organization pertains to the overall conceptual structure of a text, and is related to the effect of the text on the language user. Conventions of rhetorical organization include common methods of development such as narration, description, comparison, classification, and process analysis.

2. Strategic Competence• Strategic competence characterizes the

mental capacity for implementing the components of language competence in contextualized communicative language use.

• Strategic competence thus provides the means for relating language competencies to features of the context of situation in which language use takes place and to the language user’s knowledge structures (sociocultural knowledge, ‘real-world’ knowledge).

3. Psychophysiological mechanisms

• Psychophysiological mechanisms refer to the neurological and psychological processes involved in the actual execution of language as a physical phenomenon (sound, light).