creating tests 2011
TRANSCRIPT
-
7/30/2019 Creating Tests 2011
1/50
Creating Tests That Assess
Higher Order Thinking Skills
Dr. Jennifer Hurd
Innovative [email protected]
-
7/30/2019 Creating Tests 2011
2/50
Objectives
When you complete this workshop, you
should be able to: Write different types of questions that will assess more
than knowledge and understanding on the Blooms
Taxonomy Scale.
Analyze test items to determine if these test items assess
higher order thinking skills.
Avoid signaling the correct answer by knowing common
errors in teacher-developed exams.
-
7/30/2019 Creating Tests 2011
3/50
Blooms Taxonomy
-
7/30/2019 Creating Tests 2011
4/50
Developing Objectives Using
Blooms Taxonomy
Objectives must be measurable
Objectives must be attainable
Objectives must have a limiting factor
-
7/30/2019 Creating Tests 2011
5/50
Objectives must:
Contain the same key words that we will
discuss for developing questions.
Map to the mission statement or goals for
the entire course or department
Demonstrate the skills that have been
attained for each level of BloomsTaxonomy
-
7/30/2019 Creating Tests 2011
6/50
Example of an ObjectiveAccording to the course material,
85% of the students will be able to
design a business plan for a new
business of their choice.
-
7/30/2019 Creating Tests 2011
7/50
Example of an Application
Objective
According to the course content, 90%
of the students in this course will beable to complete a mock interview
with three of the instructors
-
7/30/2019 Creating Tests 2011
8/50
Learning Outcomes
Are the primary building blocks of good curriculum
design.
Support the goal or learning outcome in that each is asmall step to arriving at what the learner is supposed to
know or be able to do.
Are carefully worded with qualifiers to restrict the
conditions and terms under which the learning outcomesare met.
Are measurable!
-
7/30/2019 Creating Tests 2011
9/50
Learning outcomes must
contain verbs that describeobservable, measurable actions.
-
7/30/2019 Creating Tests 2011
10/50
Learning Outcomes should:
Provide the basis to select
or design instructional
materials, content or
assessments
Provide a framework
within which learners can
organize their efforts to
complete the learning task
-
7/30/2019 Creating Tests 2011
11/50
Planning a test
Determine the method of assessment
Demonstrating knowledge and understanding
Assessing critical thinking skills
Assessing problem solving skills
Assessing performance of procedures
Assessing application skillsAssessing collaborative learning skills
Assessing research skills
-
7/30/2019 Creating Tests 2011
12/50
Planning: Specifying What is to
be Assessed Clarify instructional learning outcomes
Define the purpose of the test
Pre-instructional (pre-test)Tests and quizzes
Final exams
Specify what will be tested and at whatlevel
Develop a test blueprint
-
7/30/2019 Creating Tests 2011
13/50
Blueprint for a Test
Content/
Skill
Area
Knowledge/
Understanding
Critical
Thinking
Skills
Skill
Comprehension
Application
Critical
Thinking
ProblemSolving
Performance
of
Procedures/Demonstrati
ng
Techniques
Collaborative
Learning
Skills
Integrate
Learning
IntoProfessional
Practice
Assessing
Research
Skills
-
7/30/2019 Creating Tests 2011
14/50
Basic Tenants of Test Item
Writing Reading Level
Write items as simply as possible Test for important details
Avoid direct quotes or verbatimstatements
Identify any opinions or authority thatare used in test items
-
7/30/2019 Creating Tests 2011
15/50
Anatomy of an Exam Question
Should have three parts:
Akey wordthat tells the students what they are
expected to doAnobject that tells them what general
information to use to perform their task
A limiting factor or factors that tells whatspecific information to include in the answer
-
7/30/2019 Creating Tests 2011
16/50
Blooms Taxonomy
Lets look at each level of Blooms Taxonomy to
see how we can develop questions for each level.
-
7/30/2019 Creating Tests 2011
17/50
Knowledge Questions
remembering;
memorizing;
recognizing;
recalling identification and
recall of information Who, what, when, where, how ...?
Describe
-
7/30/2019 Creating Tests 2011
18/50
An example of a knowledge
question
According to the instructor,define the key words for
knowledge questions.
-
7/30/2019 Creating Tests 2011
19/50
An example of a knowledge question
According to the instructor, list
the key words for knowledgequestions.
Can you change this question into a M/C
question?
-
7/30/2019 Creating Tests 2011
20/50
Writing a question
-
7/30/2019 Creating Tests 2011
21/50
According to the instructor,which of thefollowing are the key words for knowledge
questions?
A. list, synthesize, evaluate, define, writeB. List, define, dramatize, solve
C. List, define, duplicate, memorize
D. None of the aboveE. A and B only
-
7/30/2019 Creating Tests 2011
22/50
Application Question Key Words
Imagine
Pretend
Make Believe
Visualize
Assume
Put yourself in the
situation of
Use your knowledgeto
Suppose
In groups of three or four write a three-part application
question on any topic or subject area using one of the key
words listed above. Be prepared to share your question.
-
7/30/2019 Creating Tests 2011
23/50
Imagination Activity
Imagine that you are creating your own company
You can hire 5 key people to help you run your
company These 5 people can be anyone from the past orpresent
Who would you hire and why?
You have 3 minutes to decide, then you willdiscuss your response with your table partners.One person from each table will share with thegroup.
-
7/30/2019 Creating Tests 2011
24/50
-
7/30/2019 Creating Tests 2011
25/50
Sample Three-part Evaluation
Question According to the number of safety-related
features each contains, determine which
compact car is safest to drive.
Key Word: Determine
Object: to find out which compact car is safest to drive
Limiting Factor: cars must be compact
Only safety-related features to be
examined
-
7/30/2019 Creating Tests 2011
26/50
A/B Visit
Argue Workshop This workshop allows students to persuade,
convince, determine, decide, recommend,
suggest, support, and explain parts of coursecontent.
Applying course content to other parts of
the course or to their own life makes it evenmore meaningful and worthwhile.
-
7/30/2019 Creating Tests 2011
27/50
A/B Visit
Argue Workshop Set up the room for A/B visits, or have
students get into pairs.
Your topic is to argue whether or not thereshould be a student success course forcollege credit on your campus.
The people on the outside (As) will takethe pro stance.
The people on the inside (Bs) will take thecon stance.
-
7/30/2019 Creating Tests 2011
28/50
Synthesis Question Key Words
Combine
Integrate
Modify Revise
Improve
Rearrange Reconstruct
Hypothesize
Substitute
Create
Generate Devise
Design
Compose Plan
Predict
-
7/30/2019 Creating Tests 2011
29/50
Synthesis Sample Question
Rearrange the amendments to the U.S.
Constitution in order from least to most
important in everyday life. As well, deviseany new amendments that you think would
improve everyday life. Justify your
amendments according to your text or yournotes.
-
7/30/2019 Creating Tests 2011
30/50
Rearrange the amendments to the U.S.
Constitution in order from least to most
important in everyday life. As well, devise
any new ammendments that you think would
improve everyday life.
Key Words: rearrange and devise
Object: To discover which constitutionalamendments you consider most important and
which you would like to add to improve everydaylife
Limiting Factor: deal only with amendments tothe U.S. Constitution.
-
7/30/2019 Creating Tests 2011
31/50
Other Synthesis Assessments
Ask students to create
mnemonic devices to
remember coursecontent.
Students can create
crossword puzzles that
center on a particularpart of course content
Create a visual product
to show course
content. Example: Create a
company logo for
XXX company.
-
7/30/2019 Creating Tests 2011
32/50
Analyzing Test Items toDetermine if they Assess Higher
Order Thinking Skills
In the next three questions, use the
chat feature to tell me whether thequestions assess higher order thinking
skills. Why/why not?
-
7/30/2019 Creating Tests 2011
33/50
According to the text, good money management
skills include
A. Planning, budgeting, and living within yourmeans
B. Living within your means and using credit
cards to help.
C. Setting goals, budgeting, living within your
means, and using credit cards wisely
D. Living on a budget and using credit cards
when necessary
E. Living on a budget without credit cards.
-
7/30/2019 Creating Tests 2011
34/50
Use your knowledge from the text and the
class discussions of Hitlers strategies to
suggest how he lost World War II.
-
7/30/2019 Creating Tests 2011
35/50
According to the text and class discussions, which of the
following would be a reaction of an insecure/avoidant baby
to the strange situation research technique?
A. The baby would show great upset when the parent
departed and would cling to the parent upon his/her
return.
B. The baby may or may not show great upset when the
parent departed, but would cling to the parent uponhis/her return.
C. The baby would show great upset when the parent
departed, but would ignore the parent upon his/her
return.D. The baby may or may not show upset when the parent
departed, but would ignore the parent upon his/her
return.
-
7/30/2019 Creating Tests 2011
36/50
Common Errors Made byTeachers in Constructing Test
Items
Dont give the answer away!
Avoid these common errors.
-
7/30/2019 Creating Tests 2011
37/50
Test Construction Errors to Avoid
Dont use All of the
above or None of
the above only whenthey are the correct
answer.
Avoid distractors that
are a joke or an insult.
-
7/30/2019 Creating Tests 2011
38/50
Test Construction Errors to Avoid
Make sure the stem of
the question and the
distractors/answer aregrammatically correct
Avoid having one
choice that is
extremely longer thanthe other choices.
-
7/30/2019 Creating Tests 2011
39/50
Test Construction Errors to Avoid
If all the distractors
are numbers, 90% of
the time the correctanswer is not the
highest number or the
lowest number.
Avoid using absolute
determiners in the
answer choices.
-
7/30/2019 Creating Tests 2011
40/50
Test Construction Errors to Avoid
Avoid using more
inclusive options.
Avoid answer options
that state a reason.
-
7/30/2019 Creating Tests 2011
41/50
Learning how to play the piano while
singing occurs in theA. Affective domain
B. Motor domain
C. Cognitive domain
D. Motor and cognitive domains
E. None of the above
-
7/30/2019 Creating Tests 2011
42/50
According to the text, those who believe in
the work ethic believe that
A. All people will do good work.
B. Hard work is always rewarded.
C. Work is good and important.
D. Only good workers should receive pay.
-
7/30/2019 Creating Tests 2011
43/50
The normal adult temperature of 98.6
degrees F is equivalent toA. 18 degrees C
B. 27 degrees C
C. 37 degrees C
D. 42 degrees C
-
7/30/2019 Creating Tests 2011
44/50
Summary
Use three-part questions to make sure you areasking questions that cover all of BloomsTaxonomy
Write objectives to measure all aspects of BloomsTaxonomy
Have students create questions and gradequestions
Use other methods instead of just paper/penciltests to assess students in higher order thinkingskills
-
7/30/2019 Creating Tests 2011
45/50
Resources
Parker, J. (2006). Workshops for Active
Learning. Boston: Houghton Mifflin
Company.
This book provides the anatomy of an exam
question process that is used in thispresentation.
-
7/30/2019 Creating Tests 2011
46/50
Resources
http://www.acu.edu/academics/adamscenter/c
ourse_design/syllabus/verbs.html
This link contains a list of learning
outcomes verbs that relate to Blooms
Taxonomy
-
7/30/2019 Creating Tests 2011
47/50
Resources
I included a handout with this presentation
titled Checklist for Evaluating Test Items.
This handout can help you evaluate the
different types of test items to avoid
signaling the answers as we discussed inthis webinar.
-
7/30/2019 Creating Tests 2011
48/50
Resources
http://www.schreyerinstitute.psu.edu/Multiple
ChoiceItems/
This link contains more suggestions for
developing higher level multiple choice test
items.
-
7/30/2019 Creating Tests 2011
49/50
Resources
How to Assess Higher-Order Thinking Skills
in Your Classroom
By Susan M. Brookhart
You can order this book at this website:
Rhttp://www.ascd.org/publications/books/109
111.aspx
http://www.ascd.org/publications/books/109111.aspxhttp://www.ascd.org/publications/books/109111.aspxhttp://www.ascd.org/publications/books/109111.aspxhttp://www.ascd.org/publications/books/109111.aspx -
7/30/2019 Creating Tests 2011
50/50
Thank You!
Feel free to contact me if you have
any other questions
501-268-8860