BASIC PRINCIPLES OF ASSESSMENT

TSL3112 LANGUAGE ASSESSMENTPISMP TESL SEMESTER 6

IPGKDRI

CHARACTERISTICS OF A TEST

• Should be applied to assessments of all kinds in general.

• Questions to ponder:– Can it be given within appropriate administrative

constraints?– Is it dependable?– Does it accurately measure what you want it to measure?– Is the language in the test representative of real-world

language use?– Does the test provide information that is useful for the

learner?

PRACTICALITY

• Refers to the logistical, down-to-earth, administrative issues involved in making, giving, and scoring an assessment instrument.

• These include “costs, the amount of time it takes to construct and to administer, ease of scoring, and ease of interpreting/reporting the results” (Mousavi, 2009).

PRACTICALITY

• A PRACTICAL TEST…– stays within budgetary limits.– can be completed by the test-taker within

appropriate time constraints.– has clear directions for administration.– appropriately utilises available human resources.– does not exceed available material resources.– considers the time and effort involved for both

design and scoring.

OBJECTIVITY

• Refers to the degree to which equally competent scorers obtain the same results.

• Most standardised tests of aptitude and achievement are high in objectivity.

• The test items are objective type (e.g. MCQ), and the resulting scores are not influenced by the scorers’ judgement / opinion.

OBJECTIVITY

• In fact, such tests are usually constructed so that they can be accurately scored by trained clerks and scoring machines.

• Highly objective procedure are used – the reliability of the test results is not affected by the scoring procedures.

• For classroom assessments constructed by teachers or performance-based assessments, objectivity plays an important role in obtaining reliable measures of achievement.

OBJECTIVITY

• Teachers may not only use objective tests, but also other methods of assessment that require judgemental scoring.

• Therefore, to ensure high objectivity:– Select assessment procedures most appropriate

for the learning goals being assessed.– Make the assessment procedure as objective as

possible – e.g. carefully phrasing the questions and providing a standard set of rules for scoring.

WASHBACK EFFECT

• The effect of testing on teaching and learning – e.g. the extent to which assessment affects a student’s future language development.

• Messick (1996) reminded us that the washback effect may refer to both the promotion and the inhibition of learning (beneficial versus harmful/negative) washback.

WASHBACK EFFECT

• Alderson & Wall (1993) – a Washback Hypothesis – how tests influence both teaching and learning.

• A TEST THAT PROVIDES BENEFICIAL WASHBACK…– positively influences what and how teachers teach.– positively influences what and how learners learn.– offers learners a chance to adequately prepare.

WASHBACK EFFECT

– gives learners feedback that enhances their language development.

– is more formative in nature than summative.– provides conditions for peak performance by the

learner.

WASHBACK EFFECT• In large-scale assessment, washback refers to the effects

that tests have on instruction in terms of how students prepare for the test – e.g., cram courses and teaching to the test.

• The current worldwide use of standardised tests for gate-keeping purposes can lead students to focus on gaining an acceptable score rather than on language development.

• Positively, many enrollees in test-preparation courses report increased competence in certain language-related tasks (Chapelle, Enright, & Jamieson, 2008).

WASHBACK EFFECT

• In classroom-based assessment, washback can have a number of positive manifestations, ranging from the benefit of preparing and reviewing for a test to the learning that accrues from feedback on one’s performance.

• Teachers can provide information to students on useful diagnoses of strengths and weaknesses.

WASHBACK EFFECT

• Washback also includes the effects of an assessment on teaching and learning prior to the assessment itself, i.e., on preparation for the assessment.

• The challenge to teachers is to create classroom tests that serve as learning devices through which washback is achieved.

• Washback enhances a number of basic principles of language acquisition: intrinsic motivation, autonomy, self-confidence, language ego, interlanguage, and strategic investment.

WASHBACK EFFECT

• Ways to enhance washback:– To comment generously and specifically on test

performance.– Through a specification of the numerical scores on

the various subsections of the test.– Formative versus summative tests:• Formative tests provide washback in the form of

information to the learner on progress towards goals.• Summative tests provide washback for learners to

initiate further pursuits, more learning, more goals, and more challenges to face.

WASHBACK EFFECT

– To imply that students have ready access to you to discuss the feedback and evaluation you have given.

AUTHENTICITY

• The degree of correspondence of the characteristics of a given language test task to the features of a target language task (Bachman & Palmer, 1996).

• Lewkowicz (2000) discussed the difficulties of operationalising authenticity in language assessment:– Who can certify whether a task or language

sample is “real-world” or not?

AUTHENTICITY

• Such judgements are subjective, and yet authenticity is a concept that language-testing experts have paid a great deal of attention to (Bachman & palmer, 1996; Fulcher & Davidson, 2007).

• Chun (2006) asserts that many test types fail to simulate real-world tasks.

AUTHENTICITY

• AN AUTHENTIC TEST…– contains language that is as natural as possible.– has items that are contextualised rather than

isolated.– includes meaningful, relevant, interesting topics.– provides some thematic organisation to items,

such as through a story line or episode.– offers tasks that replicate real-world tasks.

AUTHENTICITY

• The authenticity of test tasks in recent years has increased noticeably.

• Many large-scale tests nowadays offer simulation of real-world tasks in speaking and writing components, of which the performance of these productive skills were not included previously.

• Reading passages are selected from real-world sources that test-takers are likely to have encountered or will encounter.

AUTHENTICITY

• Listening comprehension sections feature natural language with hesitations, white noise, and interruptions.

• More tests offer items that are episodic in that they are sequenced to form meaningful units, paragraphs, or stories.