an introduction to testing and assessment san diego state university

1

AN INTRODUCTION TO TESTING AND

ASSESSMENT

SAN DIEGO STATE UNIVERSITYLANGUAGE ACQUISITION RESOURCE CENTER

June 30, 2011

Alan SvidalOffice of Language Acquisition,

San Diego Unified School District

2

An Introduction to Test and Assessment

AGENDA• Gathering background information

• Using backward design

• Some important definitions

• Test and assessment protocols

• Writing tests with practice

• Writing assessments with practice

• Practical considerations about tests and assessments

• Taking a test: level 1 French

• Overview of scoring rubrics

• Writing rubrics f0r reading activity

3

Recall a time you took a test that was an unpleasant experience.

Recall a time it was a pleasant experience.

What made the difference between these experiences?

Gathering background information

4

When is it necessary to test students?

When is it not necessary to test them?

What do teaches hope to learn from tests?

What do students learn from tests?

5

Backward or Reverse Lesson Design

First, I ___________________________________________

Then, I ____________________________________________

Next, I ____________________________________________

Then, I ____________________________________________

Next, I _____________________________________________

Finally, I ___________________________________________

identify target standards. (Which ones will guide the lesson development?)

develop the assessment. (How well can students use the language.?)

select activities that allow students to succeed. (What and how will they practice?)

teach. (What will I do to provide frequent, meaningful practice?)

select functions and knowledge. (What will students need to do and know?)

develop the scoring rubric. (How I will measure student performance?)

6

DEFINITIONS Define the word " ______________"; Define the word " ________________";

test

assessment

7

DEFINITIONS Define the word " ______________";

test

Tests measure academic knowledge, facts, formulas, processes, rules, etc.

Tests measure what a student should know in comparison to other students at the same grade level or in the same course of study.

Tests are summative measures that reveal mastery of language at specific points in time.

8

Which statement about Napoléon II is NOT true:

A. He was the son of Napoléon Bonaparte.B. He never ruled France.C. He was the older brother of Napoléon III.D. He was the king of Rome.

Fill in the blanks with the correct form of the verb “to swim”.

The form may only be used once.

1. I like __________.2. Yesterday, I ________ for two hours.3. By the end of the week, I will have _____ for 20

hours.3. Is _________ you favorite activity?

Match the word in column A to its opposite in column B.

1. Original a. necessary2. Superfluous b. quiet3. Raucous c. redundant4. Salacious d. praiseworthy

e. bodacious

9

DEFINITIONS Define the word " ________________";

assessment

Assessments are ‘performance based’ evaluations that examine behavioral outcomes and the means students use to produce those outcomes.

Assessments do not measure a given body of knowledge but rather how that knowledge is applied to performance.

Assessments are formative measures that highlight the functional use and on-going development of language over time.

10

•You are planning a trip to Paris with friends.

•On a map of Paris, share your itinerary with the class.• Point out what you will see, • where the sites are located, • how you will get there on the metro.

11

Fill in the blank with a single word that fits the context of this paragraph.

Martin lived near Paris with __1___ parents whom he loved very

____2__. They were kind ___3______loving but ____4___ poor. Martin loved

school, ____5___. Every ___6____ , when his mother ____7____ him up, Martin

was _____8____ to get out of ____9_____, put ___10___ his clothes, eat his

___11_____ and __12___ hurry __13___ to school. This morning, however,

Martin looked____14___ the widow but did ___15___ smile. “What’s the

_____16____, Martin?”, his mother ____17___.“Well, ”Martin ____18___, “it’s

the __19__ day of school and I am very ____20____.”

Choose the word that best completes the sentence.

Martin lived near Paris with __1___ parents whom he loved very ____2__. They were kind ___3______loving but ____4___ poor.

1. a. many b. his c. whose d. one2. a. true b. good c. much d. restfully3. a. so b. but c. and d. however4. a. well b. they c. however d. terribly

13

My definition:

Our group’s definition:

What the experts say: “A rubric is…

(1) a scoring guide

(2) that lists key indicators of a performance

(3) in which numeric values are applied

(4) to descriptions of different achievement levels

(5) allowing scorers to differentiate among performance outcomes.

What is a “rubric”?

14

“A rubric is… (1) a scoring guide

(2) that lists significant indicators of a performance (3) in which numeric values are applied (4) to descriptions of different achievement levels(5) allowing scorers to differentiate among performance outcomes.

Outstanding 3

Satisfactory 2

Poor1

Punctuation

Punctuation almost always correct X

Some errors throughout

Careless; numerous errors

Effort More than required

Meets requirements

X

Some items missing, work appears hastily assembled

Creativity Creative, original descriptions; realistic characters; well illustrated; neat

Some creativity; simple descriptions; mostly neat

X

Show no creativity or planning; incomplete descriptions; unrealistic characters, haphazard illustrations or no illustrations

1. Writing Rubric

Ann JonesScore: 7 / 9

15

AN OVERVIEW OFTEST AND ASSESSMENT PROTOCOLS

16

Assessment Protocols: From Linear to Non-linear

We define “linear” assessments as those that produce predictable,

non-spontaneous responses.

Why are standardized tests and computer-based language programs

largely “linear”?

17

Assessment Protocols: From Linear to Non-linear

We define “non-linear” assessments as those that allow for unpredictable, spontaneous responses.

• What are the advantages of “non-linear” assessments?• What are the disadvantages of “non-linear” assessments?

18

1. As you examine the following assessment protocols, identify those that are:

Linear

• Give an example of a “linear” assessment question.• Give an example of a “non-linear” assessment question.

Bridging to non-linear

2. When you have looked at all the assessment options:• Write an example of a “linear” assessment question.• Write an example of a “non-linear” assessment question.

Non-linear

19

TEST AND ASSESSMENT PROTOCOLS

1.DISCRETE POINTSFocus on language details and recall of facts:

True/falseMatchingMultiple choiceFill in blanksVocabulary/grammar drills

20

2. INTEGRATIVEFocus on student control of the language to expand mastery through combining prior and current learning via listening, reading, speaking, writing, and viewing:

ASSESSMENT PROTOCOLS

ClozeDictationWritingInterviews/pollsSummaries

21

3. PROMPTEDFocus on the use of manipulatives, visuals, and realia to stimulate language production:


COCI (Classroom Oral Competency Interview) CWCA (Classroom Writing CompetencyAssessment)

A/B pictures

Story boards/posters

Flash cards

22

4. PERFORMANCEFocus on demonstration, negotiation, interaction, and coping with uncertainties:


Problem solving “games” or info gap act.

Group discussions

Classroom presentations

Reaching consensus

Role-playing

23

5. CREATIVEFocus on individual or group product, reflection, and exposition:


Cultural Expeditions and presentations

Projects (radio, broadcasts, podcasts)

Skits

Puppet shows

Videos

24

6. SUMMATIVEFocus on language acquisition over time:


Flosem

Student Portfolios

Dairies/journals

Student books

25

7. OPEN-ENDEDFocus on non-specified responses, varied contexts and contents, broad range of appropriate responses:


Computer based research

Student constructed tests

“Instruction” by students

Descriptions using pictures

Situation role play

26

10 Ways Language Learners can

Demonstrate Learning

1.

2.

3.

4.

5.

Portfolios

Scrapbooks

Interviews

Phone call to teacher

Skits/charades

27

10 Ways Language Learners can

Demonstrate Learning

6.

7.

8.

9.

10.

Dialogues/conversations

Projects (videos, shows, books)

Writing samples

Storyboards

Tests/quizzes

28

PROCEDURES FOR CONSTRUCTING A READING OR LISTENING TEST

1. Establish goal for summative testing.• achievement test: to measure short-term mastery of knowledge

quizzes • prochievement test: to measure mastery at the end of unit of study

unit test, mid-term exam, final exam • proficiency assessment: to measure language mastery as well as skill

placement exam, end of course exam

2. Decide what type of test is most appropriate for the purpose. • teacher-made test • textbook test • standardized test • modified combination

3. Select reading or listening passage to be tested. • authentic or simulated

• appropriate length • appropriate level of difficulty • relevant/interesting

4. Provide a context or setting, especially for listening. • You are about to hear a weather forecast. • You will read about all the things Dan, an American student,

did during the course of one day in Rome.

29

5. Develop items that test the most significant elements of the content

• be sure students are tested on what they have learned and practiced. • keep the language of test questions at the students’ proficiency level. • do not ask too many questions per reading or listening passage. • write directions in the English. • make sure the choice of answers is not ambiguous• do not ask questions about minor details when testing listening.

6. As you write test questions, consider advantages and disadvantages of test formats

• multiple choice• true or false• matching• etc

30

Key Components for Oral and Written Assessment

Comprehensibility:How well are students understood by others?

ComprehensionHow well do students understand others?

Language Control:How accurate is their language?

Vocabulary Use:How extensive and appropriate is their vocabulary?

Communication Strategies:How do they maintain communication?

Cultural Awareness:How is their cultural understanding reflected in their communication with others?

31

Nicolas and the Bicycle

Linear, bridging, and non-linear assessment

32

ASSESSING ABILTY TO SPEAK OR WRITE IN A SENTENCE

If you want to measure a students ability to speak in the dreaded “COMPLETE SENTENCE”, try this technique that gives students a logical and natural way to write or speak in complete sentences. In natural speech, statements and answers are often single words or phrases. Questions, however, are almost always framed as complete sentences. So, give students an ANSWER and ask them to say or write a logical question. HERE ARE A FEW ANSWERS. WRITE AN APPROPRIATE QUESTION FOR EACH “15 minutes.” “No, I don’t think so.” “Because I said so!” “Friday, before school.” “$35.00 plus tax.”

33

PRACTICAL CONSIDERATIONS ABOUT TESTS AND ASSESSMENTS

•Give advance notice.

•Avoid surprise tests and quizzes

•Plan ahead for possible conflicts.

•Have an alternative test form ready.

•Know what you will do about cheating and tell the class.

•Be aware of the amount of time to take the test and to correct it.

•Give clear directions, in English if necessary.

34

Allow no distractions on the desks or in the room.

Be sure the test will be ready on test day.

Keep all the test materials in a secure place.

If using equipment, be sure it works.

If using rubrics, give them to students early on.

Do not let standardized tests leave the room

35

Using the Framework: Stage I

Under each bullet, provide two ways in which your students might demonstrate that they can fulfill the Stage I functions, one as a test, the other as an assessment.

• express likes and dislikes:Paul loves sports. Circle the activities in this list that fit his interest.

Make a power point slide showing foods you would like to see served in the school cafeteria and tell the class what these foods are.

• make requests:Match the 5 common classroom requests in column A to appropriate

pictures in column B.

Write a note to your parents listing 3 reasons why you would like a bigger allowance.

• obtain information:Read the 5 sentences below about the time each student goes to bed and

then drawn in that time on each student’s clock.

Interview 10 students about how much time they spend watching TV and share the results with your study group.

36

Using the Framework: Stage II

Under each bullet, provide two ways in which your students might demonstrate they can fulfill Stage II functions, one as a test, the other as an assessment.

• express their needs:Match Paul’s needs listed in column A with appropriate solutions in column B.

You are spending the winter break in Alaska. Make a list of things you will need to pack and share this list with your study group.

• make requests:Read the statements below. Then, fill in the blank with the appropriate

question word you would use to ask a question about the statement.

You will role play (as the customer) a phone conversation with your partner (as the travel agent) in which you discuss hotels and sights to see in Paris.

• compare / contrast:Look carefully at pictures A and B. They are similar but not identical. List

five differences you see.

Your cousin from Egypt is coming to stay with your family. Write a letter in which you describe the differences he/she will encounter in San Diego.

37

UNIQUE 1 Vocabulaire .

interrogation quiz

38

UNIQUE•u-•ni-•que

•u-ni-que

•UNIQUE

39

Unique?

• Voilà Notre Dame • unique? • Oui? Non? • __ !

Notre Dame est unique.

Oui

40

Unique?

• Voilà la Tour Eiffel• unique?• Oui? Non? • ___!

La Tour Eiffel est unique.

Oui

41

Unique?• Voilà le drapeau de la France.• unique?• Oui? Non?• ____ !

• Le drapeau de la France est unique.

Oui

42

Interrogation 1a

#A.Notre Dame#B. le drapeau de la France #C.la Tour Eiffel

1. # ________ 2. # ______ 3. # _______

la lettre correcte

43

Interrogation 1a

#A. Notre Dame#B. le drapeau de la France #C. la Tour Eiffel

1. # ______ 2. # ______ 3. # _______AB C

44

Interrogation 1bComplétez la phrase.

1. La Tour Eiffel ______ unique.

2. Notre Dame ______ unique.

3. Le drapeau de la France ________ unique.

45

Interrogation 1bComplétez la phrase.

1. La Tour Eiffel ______ unique.

2. Notre Dame ______ unique.

3. Le drapeau de la France ________ unique.

est

est

est

46

Interrogation 1c

47

Interrogation 1c

Notre Dame est unique.

La Tour Eiffel est unique.

Le drapeau de la France est unique.

48

Meaning from context

Directions:

•Read ALL of the following entries from Madame Videau’s planning agenda.

•You will be asked to respond to some questions after you read.

• You may make notes if necessary.

49

Les dates

Le calendrier de Madame Videau27 Place Édith Cavell

H2Y 1H3 Montréal, Québec

le 5 mai : oncle Bernard arrive de Rome (à l’aéroport) le 9 mai : la famille dîne au restaurant Le lion bleu.

le 10 juin : le 18ième anniversaire de Julie le 26 juin : le mariage de Fabien et d’Alice, église de Ste.-Thérèse

le 1ier juillet: le Jour du Canada, célébration publique (Île Ste-Hélène) le 19 juillet : oncle Bernard retourne en Italie. le 30 juillet : la famille fait du camping aux États-Unis.

50

What do you know ?(Answer as much in French as you can.)

1. List five facts you learned about Bernard ?

2. Name two things that Madame Videau has planned for the family.

3. What two things do you know about Julie?

4. On which date do you suppose there will be a fireworks display in town? Why?

5. How are dates written differently in French than in English? (You should be able to see two differences.)

6. Madame Videau’s birthday is May 25th. Write that date in French and wish her happy birthday.

51


1. List five facts you learned about Bernard ?

• Bernard is an uncle.• He is Madame Videau’s uncle.• He lives in Italy.• He is Italian.• He speaks French.• He is arriving from Rome.• He is arriving on May 5th.• He is arriving at the airport.• He is returning to Italy.• He is returning on July 19th.

52


What two things has Madame Videau planned for the family?

• A dinner on May 9th at the restaurant Le lion bleu.

• The wedding of Fabien and Alice.• To go to the Canada Day celebration.• A camping trip to the U.S. on July 30th.

53


What do you know about Julie?

• Julie’s birthday is June 10th.• Julie will be 18.• Julie is a Gemini.• Julie knows Madame Videau.• Julie is Canadian/ from Quebec.• Julie is a girl.

54


When do you suppose there will be a fireworks display in town?

Why?

• On July 1st. • It is Canada Day• It is a public celebration and a holiday. • It celebrates the confederation of Upper and

Lower Canada.

55


How are dates written differently in French than in English?

(You should see two differences.)

• In French, the date (numeral) comes in front of the month.

• The names of the months are not capitalized.• The word “le” precedes the numeral (date).• In English, we add “-st” to the numeral “1”.• In French, we add “-ier” to the numeral “1”.• In English, we use cardinal numbers with dates.• In French, we use ordinal numbers with dates.• In English, we add “of” but not in French.

56


Madame Videau’s birthday is May 25th. Write that date in

French and wish her “Happy Birthday”.

• le 25 mai.• Bon anniversaire, Madame!

57

TAKING A LOOK AT

SCORING RUBRICS

58

“A rubric is… (1) a scoring guide

(2) that lists significant indicators of a performance (3) in which numeric values are applied (4) to descriptions of different achievement levels(5) allowing scorers to differentiate among performance outcomes.

Outstanding 3

Satisfactory 2

Poor1

Punctuation

Punctuation almost always correct X

Some errors throughout

Careless; numerous errors

Effort More than required

Meets requirements

X

Some items missing, work appears hastily assembled

Creativity Creative, original descriptions; realistic characters; well illustrated; neat

Some creativity; simple descriptions; mostly neat

X

Show no creativity or planning; incomplete descriptions; unrealistic characters, haphazard illustrations or no illustrations

1. Writing Rubric

Ann JonesScore: 7 / 9

59

Select indicators that identify significant components of the instructional activity:

Fluency Pronunciation Vocabulary

Use numerical values to define different achievement levels within each element:

4 - 3 - 2 - 1 (Scales larger than 5 may be difficult to describe.)

Describe each achievement level in specific terms that recognize potential differences among learners and are written in student-friendly language:

4 = speech is sustained: fillers maintain comprehension 3 = speech is hesitant: fillers interrupt comprehension 2 = speech is awkward: fillers interfere with comprehension 1 = speech is broken: fillers prevent comprehension

Points to remember in Developing Rubrics

60

Summary words may be added to qualify numeric values:

(4) Excellent / (3) Good/ (2) Acceptable/ (1) Unacceptable (4) Exemplary / (3) Proficient / (2) Basic / (1) Below basic

With preexisting rubrics, clarify each description by sharing some examples of student work.

Be prepared to edit and revise rubrics that miss the mark you set.

AND MOST IMPORTANTLY:

GIVE STUDENTS THE RUBRIC BEFORE THEY BEGIN THE WORK!

Points to remember in Developing Rubrics

61

Generic Rubrics for Collaborative Work

4 3 2 1

Workload equality

Workload shared equally

Workload somewhat unequal

Workload unequal-done mostly by one or two students

Workload unequal-one student has done all the work

On task All the time Most of the time

Sometimes Little involvement; rarely on task.

Interaction Much discussion; shows respect for others

Some discussion; respectful of others

Little discussion; easily distracted, somewhat disrespectful of others

Shows little interest; disrespectful of others

Critique of Existing Rubrics

62

Generic Rubric for Oral Presentations

Yes No

Accurate pronunciation

Accurate Grammar

63

Generic Rubric for Oral Presentation

4 3 2 1

Pronunciation Accurate throughout, near native

Understandable with very few errors

Some errors but still understandable

Poor pronunciation, very anglicized

Fluency Smooth delivery

Fairly smooth Unnatural pauses

Halting, hesitant, long gaps

Comprehensibility

Easily understood

Understood Difficult to understand

Incomprehen-sible

Vocabulary Extensive use of targeted vocabulary

Some use of targeted vocabulary

Minimal use of targeted vocabulary

Fails to use targeted vocabulary

Credibility (shows knowledge of other culture)

Credible role play; reflects the culture

Credible role play; somewhat reflects the culture

Limited credibility; little connection to target culture

Not credible; no connection to target culture visible

Performance Lively, enthusiastic; good eye contact

General enthusiasm; some eye contact

Low energy; limited eye contact

Reads from cards, monotonous; no eye contact

64

Generic Rubrics for Written Material #1

4

Exceeds Expectations

3 2 1

Grammar Perfect Uses well what is being studied

Some errors with what is being studied

Doesn’t seem to understand what is being studied

Vocabulary Creative use of vocabulary

Vocabulary at present level of study

Some use of current vocabulary; key words missing

Minimal use of targeted vocabulary at present level of study; words used incorrectly

Spelling Perfect Very few errors in spelling and accent marks

Some errors in spelling and accent marks

Many errors in spelling and accent marks

65

La biographie de ma famille5-6 points

o la tache est complétée avec toutes les correctionso il y a des détails supplémentaireo le brouillon est dans le dossiero la composition est très bien organisée et présentéeo tout est fini à l’heureo on a suit toutes les directions

2-4 points

o il n’y a pas de détailles supplémentaireso le brouillon est dans le dossiero le projet est peut-être en retardo l’organisation et la présentation sont acceptables o on a suit toutes les directions

1 point

o on a fait un projet mais pas acceptable parce qu’il ne s’adresse pas aux directions

0 points

o pas de projet5pts –

extraordinaire4pts – excellent3pts – bon2pts – passable1pts – pas

acceptable0pts –pas

acceptable

66

Foreign Language Department

VocabularyA= Rich and extensive vocabulary; very accurate usageB= Occasionally lacks basic words; generally accurate usageC= Often lacks needed words; somewhat inaccurate usageD= Lacks basic words; inadequate usageVerbs/StructureA= Utterances almost always correct, with some minor errorsB= Many correct utterances, but with some structural errorsC= Some utterances rendered correctly, many structural errorsD= very few utterances structurally correctPronunciation/ComprehensibilityA= Entirely comprehensible to native speaker; only an occasional word not comprehensibleB= Some errors of pronunciation, but still mostly comprehensibleC= Many errors about half incomprehensibleD= Mostly incomprehensible; occasional phrases comprehensibleFluencyA= Speech natural and continuous; no unnatural pausesB= Generally natural and continuous; only slight stumbling or unnatural pausesC= Some definite stumbling and hesitation; sentences may be left uncompletedD= Speech halting and fragmentary; long, unnatural pauses

A= 4 pointsB= 3 pointsC= 2 pointsD= 1 point

16 points are possible

67

Creating a Scoring Rubric

You will receive a level one benchmark assessment used by students in San Diego Unified.

Create a rubric that would adequately evaluate student performance on this benchmark.

Afterward, we will examine the district’s rubricas well as a student scoring guide.

68

TECHNIQUES THAT BEST PROMOTE STUDENT REPAIRS

Elicitation Metalinguistic Feedback Clarification request Repetition

ALLOW YOUR STUDENTS TO SELF-REPAIR If you allow time and provide students with the proper cues, they will SELF-REPAIRThe least effective technique to correct a mistake is to give the students the answer!

69

Metalinguistic Clues Without providing the correct form, teacher questions student (for example, “Is it feminine?”, “Do we say it like that?”, etc.

Elicitation Questions that require more than a yes/no answer.”What is the I form of the verb?”

Repetition Teacher repeats the error with high intonation to draw student’s attention to it.

70

Explicit Correction Clearly pointing to the mistake and correcting it.

Recast Teacher repeats student’s utterance but without the mistake.

Clarification Request Using phrases like “Excuse me”, “I don’t understand”, the teacher indicates that the message was not understood

71

WHEN CORRECTING PLEASE…

understand that they will probably not internalize the correction. remember that clarifying meaning is not necessarily error correction. remember that students what to make a point rather than to make a point correctly.

an introduction to testing and assessment san diego state university

Documents

means students

practice writing assessments

important definitions

mastery of language

meaningful practice

student performance

background information

lesson development