11

Lesson OneLesson One

Introduction: Teaching and Introduction: Teaching and Testing/AssessmentTesting/Assessment

22

ContentsContents What is a test?What is a test? Why tests?Why tests? Problems of many testsProblems of many tests Qualities of a good testQualities of a good test Terms: Terms: measurementmeasurement, , testtest, , evaluationevaluation, and , and

assessmentassessment Relationship between measurement, test, evRelationship between measurement, test, ev

aluation, and assessmentaluation, and assessment Relationship between test/ass.& teachingRelationship between test/ass.& teaching Formative and summative assessmentFormative and summative assessment Formal vs. informal assessmentFormal vs. informal assessment Traditional vs. alternative assessmentTraditional vs. alternative assessment Questions to think aboutQuestions to think about HomeworkHomework

33

What Is a Test?What Is a Test?

Unavoidable? Unavoidable? How tests make us feel much of How tests make us feel much of

the time? the time? (sample test)(sample test)– A necessary “evil”?A necessary “evil”?– Anything positive about tests?Anything positive about tests?

Definition: Definition: a method of measuring a a method of measuring a person’s ability, knowledge, or person’s ability, knowledge, or performance in a given domain.performance in a given domain.

44

Sample TestSample Test

Circle the correct answer. You have 3 minutCircle the correct answer. You have 3 minutes to complete this test.es to complete this test.

1. polygene1. polygeneA. the first stratum of lower-order protozoa cA. the first stratum of lower-order protozoa c

ontaining multiple geneontaining multiple geneB. a combination of two or more plastics to B. a combination of two or more plastics to

produce a highly durable material.produce a highly durable material.C. one of a set of cooperating genes, each prC. one of a set of cooperating genes, each pr

oducing a small quantitative effect.oducing a small quantitative effect.D. any of a number of multicellular chromosD. any of a number of multicellular chromos

omes.omes.

55

Why Tests?Why Tests?1. To gather information:1. To gather information: Achievement of learnersAchievement of learners Selection among competitorsSelection among competitors Comparison for levels/rankingComparison for levels/ranking Examination/Evaluation on Examination/Evaluation on

teaching methodsteaching methods Identification of problem areasIdentification of problem areas

2. To reinforce learning & to 2. To reinforce learning & to motivate studentsmotivate students

66

Problems of Many Problems of Many TestsTests

Poor quality (cf. reliability)Poor quality (cf. reliability) Fail to measure accurately what is Fail to measure accurately what is

intended to be measured (cf. intended to be measured (cf. validity)validity)

Not practicalNot practical A harmful effect on A harmful effect on

teaching/learningteaching/learning

(i.e., harmful backwash)(i.e., harmful backwash)

77

Backwash Backwash ((or or Washback)Washback)

Definition: the effect of testing on Definition: the effect of testing on teaching/learning teaching/learning (Bailey 3)(Bailey 3)

Harmful (negative) backwash:Harmful (negative) backwash:– e.g., a writing test with multiple-e.g., a writing test with multiple-

choice itemschoice items Beneficial (positive) backwash:Beneficial (positive) backwash:

– e.g., an oral teste.g., an oral test

88

A Good Test Should …A Good Test Should …

be validbe valid be reliablebe reliable be practicalbe practical have beneficial backwashhave beneficial backwash be authenticbe authentic

99

MeasurementMeasurement Quantifies the characteristics Quantifies the characteristics

(both physical and mental) of (both physical and mental) of personspersons– e.g.: weight, height, motivation, e.g.: weight, height, motivation,

aptitude, intelligenceaptitude, intelligence Involves both tests and non-tests: Involves both tests and non-tests:

test scores, ratingstest scores, ratings

1010

TestTest

Traditional paper-and-pencil testsTraditional paper-and-pencil tests A procedure designed to get a A procedure designed to get a

specific sample of a person’s specific sample of a person’s ability at a given time ability at a given time (so time-(so time-constrained, a limited sample of behavior)constrained, a limited sample of behavior)

A measurement instrumentA measurement instrument

1111

EvaluationEvaluation

QuantitativeQuantitative– Numbers involved; e.g., scoresNumbers involved; e.g., scores

QualitativeQualitative– Analyze data; e.g., letters of Analyze data; e.g., letters of

referencereference Systematic gathering of Systematic gathering of

information for decision makinginformation for decision making Determination of adequacyDetermination of adequacy

1212

AssessmentAssessment

““a set of processes through which wa set of processes through which we make judgments about a learner’s e make judgments about a learner’s level of skills and knowledge” level of skills and knowledge” (Nuna 1(Nuna 1990)990)

Whole situation included over timeWhole situation included over time An on-going processAn on-going process Includes multiple samples of behavioIncludes multiple samples of behavio

r, not just one single judgment or testr, not just one single judgment or test

1313

Assessment

Evaluation

Measurement

(test)

Measurement (non-test)

1414

Evaluation, Evaluation, Measurement & TestMeasurement & Test

Not all measures are tests, Not all measures are tests, not all tests are evaluative, not all tests are evaluative, and not all evaluation involves and not all evaluation involves either measurement or tests.either measurement or tests.

1515

Relationship between Relationship between Teaching & Teaching &

Testing/AssessmentTesting/Assessment A partnership relationshipA partnership relationship Testing not simply follow teachingTesting not simply follow teaching Testing should be supportive of Testing should be supportive of

good teaching & have a corrective good teaching & have a corrective influence on bad teachinginfluence on bad teaching

Depending on purposes of the test:Depending on purposes of the test:– To reinforce learning, motivate To reinforce learning, motivate

studentsstudents– A mans of assessing Ss’ performanceA mans of assessing Ss’ performance

1616

Relationship between Relationship between Test/Assessment & Test/Assessment &

TeachingTeaching

Teaching

Assessment

Tests

(Brown 5)

1717

Teacher’s Role in Teacher’s Role in TestingTesting

Write better tests themselvesWrite better tests themselves– Myth: a qualified T vs. a good test Myth: a qualified T vs. a good test

makermaker

Put pressure on professional Put pressure on professional testers to improve their tests testers to improve their tests – E.g., TWE (Test of Written Eng.) in E.g., TWE (Test of Written Eng.) in

TOEFLTOEFL

1818

Formative & Formative & Summative Summative

Assessment (1)Assessment (1) Formative Assessment:Formative Assessment:

– Evaluating Ss in the process of Evaluating Ss in the process of “forming” their competences and skills“forming” their competences and skills

– Goal: helping them to continue the Goal: helping them to continue the growth process (i.e., future continuation growth process (i.e., future continuation or formation of learning; on-going or formation of learning; on-going development of their language)development of their language)

– Key: the delivery (by the teacher) and Key: the delivery (by the teacher) and internalization (by the student) of internalization (by the student) of appropriate feedback on performanceappropriate feedback on performance

– E.g., most classroom assessmentE.g., most classroom assessment

1919

Formative & Formative & Summative Summative

Assessment (2)Assessment (2) Summative assessment:Summative assessment:

– To measure or summarize what a S To measure or summarize what a S has grasped at the end of a has grasped at the end of a course/unitcourse/unit

– Not necessarily point the way to Not necessarily point the way to future progressfuture progress

– E.g., final exams, general proficiency E.g., final exams, general proficiency examsexams

2020

Formal vs. Informal Formal vs. Informal Assessment (1)Assessment (1)

Formal assessment:Formal assessment:– Systematic, plannedSystematic, planned– All tests are formal All tests are formal

assessments, but not all formal assessments, but not all formal assessment is testing.assessment is testing. E.g., journal, portfolio = formal E.g., journal, portfolio = formal

assessment, but are not “testsassessment, but are not “tests””

2121

Formal vs. Informal Formal vs. Informal Assessment (2)Assessment (2)

Informal assessment:Informal assessment:– Embedded in classroom tasksEmbedded in classroom tasks– To elicit performance without To elicit performance without

recording results and making recording results and making fixed judgments about a fixed judgments about a student’s competence.student’s competence.

– E.g., marginal comments on E.g., marginal comments on paperspapers

2222

Traditional vs. Traditional vs. alternative alternative

assessment (1)assessment (1) Traditional assessment:Traditional assessment:

– One-shot, timedOne-shot, timed– Usually decontextualizedUsually decontextualized– SummativeSummative– Product-orientedProduct-oriented– Focus on the “right” answerFocus on the “right” answer

2323

Traditional vs. Traditional vs. alternative alternative

assessment (2)assessment (2) Alternative assessment:Alternative assessment:

– Continuous long-term assessment, Continuous long-term assessment, untimeduntimed

– Cintextualized communicativeCintextualized communicative– FormativeFormative– Process-orientedProcess-oriented– Open-ended, creative answersOpen-ended, creative answers

2424

Questions to Think Questions to Think aboutabout Are all tests summative?Are all tests summative? Is it possible to convert tests into Is it possible to convert tests into

“learning experiences”?“learning experiences”? Again, is testing a necessary evil?Again, is testing a necessary evil? What can we use tests for?What can we use tests for?

2525

HomeworkHomework

Read Chapter One in both Read Chapter One in both textbooks (Brown; Bailey)textbooks (Brown; Bailey)

Preview: Brown chapter 3 (pp. 43-Preview: Brown chapter 3 (pp. 43-47); Bailey chapters 3 & 6.47); Bailey chapters 3 & 6.

Q to think about:Q to think about:– Why is the course called Language Why is the course called Language

Assessment instead of Language Assessment instead of Language Testing?Testing?