leading the way to formative assessment with the...

August 1st

Leading the Way to Formative Assessment

with the KCAS

Introductions

• CCMS Faculty

• Dr. Sue Cain, CPE

• University of Louisville and Bellarmine Faculty

Agenda Housekeeping Norms Surveys On Demand

Writing/Constructed Responses, Vocabulary

September and October dates Knowledge pre-assessment

Website Materials/resources

Where have we been? Lesson Plan guide FRAME Concept Mastery Routine

Where are we going? Classroom assessment Formative assessment Creating higher order

multiple choice, short answer and, essay questions

In-class questioning Question Answer

Relationships (QAR): A strategy for students

Survey September and October

dates

Knowledge post-assessment

Evaluation

Norm! Honor start/stop times

Lunch for today

Silence electronic equipment

Participate actively

Avoid sidebar conversations

Celebrate!

Surveys

• Future PD topics

– On Demand Writing /Constructed Responses

– Vocabulary

• September and October PD dates

– Provide your school name

– Provide two or more dates for each month

• Knowledge pre-assessment

Website

• Currently housed at http://www.bellarmine.edu/education/klandrum.aspx – Ppts

– Links

• Will move to UofL soon – Ppts

– Links

– Videos

http://www.bellarmine.edu/education/klandrum.aspx

http://www.bellarmine.edu/education/klandrum.aspx

Materials and Resources

• Additional copies of information text

– Copies

– Texts and Lessons for Content-Area Reading

Lesson Plan Guide Common Core Standard(s)

Instructional Strategy:

Key Ideas and Details:

Common Assessment (Description):

Formative Summative difference is

USE of

DATA

Assessment

DeSales 2/23/2011 Corrie Orthober, Ph.D.

How Do I Create Good Select-Response and Supply

Response Items?

Types of Selects Response Items

Multiple Choice

Short Answer

Essay

Types of Supply Response Items

Multiple Choice Questions: Some Guidelines to Consider

• Item Stem – *State the item stem in the positive whenever

possible.

– *Ask a complete question to get the item started.

– *Make sure the item stem does not give away the correct answer. – Avoid grammatical clues in an item stem such as plurals

or articles

– Emphasize qualifiers such as most likely and best in the item stem.

If Possible, State the Item Stem in the Positive

Poor Item Better Item

Which of these is not a main idea of the first paragraph?

Which of the following choices best identifies the main idea implied in the

passage?

A. Abraham Lincoln was elected president in 1860.

A. Abraham Lincoln was elected president in 1860.

B. The issue of slavery troubled the nation since its birth.

B. The issue of slavery troubled the nation since 1776.

C. Slaveholding states resolved to fight over slavery.

C. The outbreak of the civil war occurred in 1861.

Prentice Hall Literature, 2007

What are some issues that you have had with stating questions in the negative?

Standard Objective Instructional Strategy

Sample Multiple Choice Assessment Item

RI.6.2. Determine a central idea of a text and how it is conveyed through particular details…

Students will differentiate the central idea in a piece of text from the supporting details.

FRAME Which of the following choices best identifies the main idea implied in the first paragraph? A. Abraham

Lincoln was elected president in 1860.

B. B. The issue of slavery troubled the nation since 1776.

C. C. The outbreak of the civil war occurred in 1861.

Ask Complete Questions


One key detail

A detail that contributes to the main idea is that libraries are most important to

A. is that libraries are important to imagination.

A. imagination

B. is that libraries are important to the poor.

B. the poor

C. is that libraries are important to children.

C. children

D. is that libraries are important to imagination citizens.

D. citizens

What benefits does a complete versus incomplete question create for students?

Adapted from Prentice Hall Literature, 2007

– from “Libraries Face Sad Chapter,” by Pete Hamill



RL.6.2. Determine a theme or central idea of a text and how it is conveyed through particular details…


FRAME A detail that contributes to the main idea is that libraries are most important to A. imagination B. the poor C. children D. citizens

Make Sure the Item Stem Does Not Give Away the Correct Answer


The author introduces the concept of creed in the passage, which of the following are other words that have the same or similar

meaning?

The author introduces the concept of creed in the passage, which of the following is a

word that has the same or similar meaning?

A. protest A. protest

B. belief and faith B. belief

C. recantation C. recantation

What is poor about this item?




RL.6.4. Determine the meaning of words and phrases as they are used in a text…

Students will determine word meanings through the context provided in a given passage.

Think aloud, modeling (definition, synonym, antonym, etc) context clue strategy.

The author introduces the concept of creed in the passage, which of the following is a word that has the same or similar meaning? A. protest B. belief C. recantation

Multiple Choice Questions: Some Guidelines to Consider

• Answer Choices

– *Develop answer choices that are parallel in grammar and in length

• Avoid clues in the answer choices that give away the correct response

– *Make sure that the correct response is the only correct response.

– Make sure the answer choices are plausible

Develop Answer Choices That Are Parallel in Grammar and Length

Which of the following statements represents the best paraphrase of the first

stanza

Which of the following choices best identifies the main idea implied in the

passage?

A. Clouds and daffodils are part of nature, unlike human beings.

A. Clouds and daffodils are part of nature, unlike human beings.

B. I was walking by myself and I was amazed to see a huge field of daffodils.

B. Clouds and fields of daffodils should be treasured.

C. I was by myself when I saw a field of daffodils so I felt lonely.

C. Clouds and fields of daffodils should be shared.

D. It was a windy day but I did not mind because the daffodils swayed in the wind.

D. Clouds and fields of daffodils are beautiful.

Poor Answer Choices

Better Answer Choices

What is poor about these answer items?




RL.7.4. Determine the meaning of words and phrases as they are used in a text, including figurative and connotative meanings, analyze the impact of rhymes and other repetitions of sounds( e.g. alliteration) on a specific verse or stanza of a poem or section of a story or drama.

Students will determine the meaning of phrases in poems.

Modeling, think alouds

Which of the following choices best identifies the main idea implied in the passage? A. Clouds and daffodils are part of nature, unlike human beings. B. Clouds and fields of daffodils should be treasured. C. Clouds and fields of daffodils should be shared. D. Clouds and fields of daffodils are beautiful.

Make sure that the correct response is the only correct response

Poor Answer Choices Better Answer Choices





A. protest A. protest

B. belief B. belief

C. recantation C. recantation

D. faith D. race

What is poor about these answer items?



RL.6.4. Determine the meaning of words and phrases as they are used in a text…



The author introduces the concept of creed in the passage, which of the following is a word that has the same or similar meaning? A. protest B. belief C. recantation D. race

Test of Franzipanics

An illustration of the impact of poorly written test questions

Stiggens, Arter, Chappuis, & Chappuis


The purpose of the cluss in furmpaling is to remove

a. cluss-prags

b. tremalis

c. cloughs

d. plumots

– State the item stem in the positive whenever possible

– Ask a complete questions to get the item started

– Make sure the item stem does not give away the correct answer.

– Avoid grammatical clues in an item stem such as plurals or articles



Trasssig is true when

a. lusp trasses the vom

b. the viskal flans, if the viskal is donwil or zortil

c. the belgo frulls

d. dissles lisk easily

– Make sure the answer choices are plausible

– Develop answer choices that are parallel in grammar and in length

– Avoid clues in the answer choices that give away the correct response

– Make sure that the correct response is the only correct response.



The fribbled breg will minter best with an

a. derst

b. morst

c. sorter

d. ignu

Use the guidelines provided earlier to answer the above question


Your Turn! Evaluate and Revise Test Items

• Examine Your Item Stems

– Do you state them in the positive?

– Do you ask complete questions to get the item started?

– Have you made sure the item stem does not give away the correct answer?

– Do you avoid grammatical clues such as plurals or articles?

– Do you emphasize qualifiers such as most likely and best in the item stem?

• Examine answer choices

– Do you make sure the answer choices are plausible?

– Are your answer choices parallel in grammar and in length?

– Do you avoid clues in the answer choices that give away the correct response?

– Do you make sure that the correct response is the only correct response?

Multiple Choice Items Lend Well to

Recall and Procedural Type Questions But … Multiple Choice Items Can Also Tap Higher Order

Thinking

Multiple Choice Items Can Also Tap Higher Order Thinking

• A multiple choice item developed to assess higher-order thinking usually provides a situation/scenario or graphic/chart followed by several questions that require students to use or apply what they have learned to provide the correct answer.

• Avoid questions, statements or examples used in class or readings for the class. If students can recognize something mentioned in class then they can answer the question correctly simply by memorizing such statements, facts or examples.

• The key to preparing memory-plus application questions is to place the concept in a life situation or context that requires the student to first recall the facts and then apply or transfer the application of those facts into a situation.

http://www.schreyerinstitute.psu.edu/MultipleChoiceItems/

Multiple Choice Items Can Also Encourage Higher Order Thinking

• Sample Item Change To

The main idea of the passage is a. Brooklyn and Queens are important to immigrants. b. There are many new immigrants in Brooklyn and Queens. c. Libraries in Brooklyn and Queens are crowded places. d. Many immigrants in Brooklyn and Queens go to libraries.

The author states that “libraries of Brooklyn and Queens are jammed with the new immigrants.” Which of the following does this detail best support? a. Brooklyn and Queens are important to immigrants. b. There are many new immigrants in Brooklyn and Queens. c. Libraries in Brooklyn and Queens are crowded places. d. Many immigrants in Brooklyn and Queens go to libraries.

Prentice Hall Literature, 2007 – from “Libraries Face Sad Chapter,” by Pete Hamill



RL.6.2. Determine a theme or central idea of a text and how it is conveyed through particular details…


FRAME The author states that “libraries of Brooklyn and Queens are jammed with the new immigrants.” Which of the following does this detail best support? a. Brooklyn and Queens are important to immigrants. b. There are many new immigrants in Brooklyn and Queens. c. Libraries in Brooklyn and Queens are crowded places. d. Many immigrants in Brooklyn and Queens go to libraries.

Techniques for Writing Multiple-Choice Items that Demand Critical

Thinking • Premise – Consequence

• Analogy

• Case Study

Techniques for Writing Multiple-Choice Items that Demand Critical Thinking

Premise - Consequence • Students must identify the correct outcome of a given

circumstance. • Example If the libraries in Brooklyn and Queens were to

close then the people most affected would be: • a) imaginative people

b) poor people c) young children d) American citizens

Note: To increase the difficulty, provide more than one premise.

http://tep.uoregon.edu/resources/assessment/multiplechoicequestions/sometechniques.html

– from “Libraries Face Sad Chapter,” by Pete Hamill



RI.7.3. Analyze the interactions between individuals, events, and ideas in a text (e.g., how ideas influence individuals or events, or how individuals influence ideas or events).

Students will determine the impact of people on events in a piece of text.

Think aloud, modeling.

If the libraries in Brooklyn and Queens were to close then the people most affected would be:

a) imaginative people

b) poor people

c) young children

d) American citizens


Analogy

• Students must map the relationship between two items into a different context:

• Example: Creed is to disbelief as contribute is to

• A). provide B). neglect C). sacrifice D). enrich

http://tep.uoregon.edu/resources/assessment/multiplechoicequestions/sometechniques.html



RI.7.4. Determine the meaning of words and phrases as they are used in a text, including figurative, connotative, and technical meanings; analyze the impact of a specific word choice on meaning and tone.



The author introduces the concept of creed in the passage, which of the following is a word that has the same or similar meaning? A. protest B. belief C. recantation D. race


Case Study

• A single, well-written paragraph can provide material for several follow-up questions.

Example:

The tragedy of Romeo and Juliet is a very famous and simple love story. Written by William Shakespeare in the fifteenth century, the play continues to be enjoyed by audiences more than 500 years later. The reason Romeo and Juliet enjoy such enduring popularity is because

a) Shakespeare is the author

b) It is a great love story

c) Tragedies are popular

d) It is a famous text

Which phrase in the text helps the reader determine the meaning of the word enduring?

a) “written by William Shakespeare”

b) “very famous love story”

c) “more than 500 years”

d) “enjoyed by audiences”

http://tep.uoregon.edu/resources/assessment/multiplechoicequestions/sometechniques.html Prentice Hall Literature, 2007



RI.8.1. Cite the textual evidence that most strongly supports an analysis of what the text says explicitly as well as inferences drawn from the text.

Students will be able to make inferences based on text and background knowledge.

Think aloud, modeling

The reason Romeo and Juliet enjoys such enduring popularity is because

a) Shakespeare is the author

b) It is a great love story

c) Tragedies are popular

d) It is a famous text

Test Item Quality Checklist

Guidelines for Multiple Choice Items

____State whole question in item stem.

____Eliminate repetition of material in response options.

____Be sure there is only one correct or best answer.

____Keep response options brief and parallel.

____Make all response options the same length.

____Limit use of all or none of the above.

____Use “always” and “never” with caution.

Stiggins et al., 2006

Student Documentation of Selected Response Items

Problem #

Learning Target #

Confident Unsure Right Wrong Simple Mistake

Further Study

Stiggins et al., 2006

As you answer each question on the test, decide whether you feel confident in your answer or are unsure about it, and mark the corresponding box.

1. After your test has been corrected, identify which problems you got right and which you got wrong by putting Xs in the “Right” and “Wrong” columns.

2. Of the problems you got wrong, decide which ones were due to simple mistakes and mark the “Simple Mistake” column.

3. For all the remaining wrong answers, mark the “Further Study” column.

Time Permitting

• Transform a recall multiple choice question into a higher order multiple choice question

leading the way to formative assessment with the...

Documents