MMann/SAS

Taxonomic LevelsAnd Rubrics

MMann/SAS

Desired Outcomes •An awareness of taxonomic levels and its purpose•An awareness of the relationship of HCPS III benchmarks and taxonomic levels•An opportunity to match benchmarks and tasks to the taxonomic levels•An awareness of various types of performance assessment rubrics

MMann/SAS

Why do I need to know the taxonomic levels?

Aligning our instruction and assessment to the targets.

MMann/SAS

Alignment – congruence or match between curriculum, instruction and assessment

CurriculumBased on GLOs & HCPS III

InstructionImplementation of the

curriculum

AssessmentMultiple measures of

proficiency of the curriculum

STUDENT ACHIEVEMENT

MMann/SAS

curriculum with standards and

assessment shows a strong relationship

to student achievement.

(Prince-Baugh, 1997; Mitchell, 1998; Wishnick, 1989)

Research on Aligning

MMann/SAS

Identify relevant content standards

Determine acceptable evidence and criteria

Determine learning experiences that will enable students to learn what they need to

know and to do

Teach and collect evidence of student learning

Assess student work to inform instruction or use data to provide feedback

Evaluate student work and make judgment on learning results and

communicate findings

Reteach, or repeat the process with the next set of relevant standards

Teach

er c

olla

bora

tion

thro

ug

hou

t the

pro

cess.

Stu

den

t in

volv

em

en

t th

rou

gh

ou

t th

e

pro

cess.

Adapted from WestEd’s Learning from Assessment

MMann/SAS

All targets, curriculum, instruction, activities and assessments involve some

level of thinking.

• Definition = the science or

technique of classification

MMann/SAS

Cognition Type• or “cognitive demand” - generally

refers to a taxonomy and reflects a classification of thinking rather than a sequential hierarchy.

(understanding prior to application and analysis)

• Cognitive demand is determined by analyzing the context of the lesson.(What support is provided and what are the students being asked to do?)

MMann/SAS

Adapt or adopt a systematic method for assigning performance

expectations. McREL

Taxonomy of objectives = a system for identifying distinct levels of difficulty.

• Bloom’s• Guilford’s • Three-Story Intellect• Marzano’s

MMann/SAS

Marzano’s Taxonomic Levels

• Level 1: Retrieval - recall, execution

• Level 2: Comprehension - integrating, representation, symbolizing

• Level 3: Analysis - matching, classifying, error analysis, generalizing, specifying

• Level 4: Utilization - decision making, problem solving,experimental inquiry, investigation

MMann/SAS

Not used with performance standards, but part of

taxonomy• Level 5: Metacognitive System - goal

setting, process monitoring, monitoring clarity, monitoring accuracy

• Level 6: Self System - examining importance, examining efficacy, examining emotional response, examining motivation.

Adapted from Marzano (2001). “Designing A New Taxonomy of Educational Objectives”.

From Marzano (2001). Designing a New Taxonomy of Educational Objectives

The Three Systems and KnowledgeSelf-System

– Decides to engage

Metacognitive System– Sets goals and

strategies

Cognitive System– Processes relevant

information

Knowledge

Continue current behavior

New Task

MMann/SAS

Marzano’s Taxonomic Levels (Cognitive System)

Level IV Level III Level II Level IKnowledge Utilization

Analysis ComprehensionKnowledge Retrieval

Use ___ to determine ___

Judge the validity of ___

Use___ to solve Generate/Test

hypotheses Analyze using

evidence Investigate

Compare/ contrast Differentiate Categorize Find what is

common among Categorize Determine

reasonableness of information

Predict Determine what

comes next/later

Describe and explain

Explain the concept

Demonstrate and explain

Diagram Illustrate/ describe

how ___ is related to ___

Represent

Recognize Select from a list Recall Give/Provide

examples List Name Read Perform

mathematical operation (by following a set algorithm)

MMann/SAS

MarzanoÕs Taxonomic Levels of Understanding (CognitiveDomain)

LEVEL 1: RETRIEVAL

RecognizingThe student can identify or recognize features of information, butdoes not necessarily understand the structure of knowledge or isable to differentiate critical from non-critical components.

RecallingThe student can recall information, but does not necessarilyunderstand the structure of knowledge or is able to differentiatecritical from non-critical components.

ExecutingThe student can perform a procedure without significant error butdoes not necessarily understand how and why the procedureworks.

LEVEL 2: COMPREHENS ION

IntegratingThe student can identify the basic structure of knowledge and thecritical as opposed to non-critical characteristics of that structure.

SymbolizingThe student can construct an accurate symbolic representat ion ofknowledge, differentiating critical from non-critical elements.

LEVEL 3: ANALYSIS

MatchingThe student can identify important similarities and differences inknowledge or skill.

ClassifyingThe student can identify superordinate and subordinate categoriesrelated to knowledge or skill.

Analyzing ErrorThe student can identify errors in presentat ion or use ofknowledge.

GeneralizingThe student can construct new generalizations or principles basedon knowledge.

SpecifyingThe student can identify specific applications or logicalconsequences of knowledge.

LEVEL 4: UTILIZATION

Decision MakingThe student can use the knowledge to make decisions or thestudent is able to make decisions about the use of the knowledge.

Problem SolvingThe student can use the knowledge to solve problems or to solveproblems about the knowledge.

InquiringExperimentally

The student can use the knowledge to generate and testhypotheses or to generate and test hypotheses about theknowledge.

InvestigatingThe student can use the knowledge to conduct investigations or toconduct investigations about the knowledge.

MMann/SAS

Standards Implementation Process Model

Identify Relevant Standards

Determine Acceptable Evidence and Criteria

Determine Learning Experiences that will Enable Students to Learn what they need to Know & Do

Teach and Collect Evidence of Student learning

Assess Student Work to Inform Instruction or Use Data to Provide Feedback

Evaluate Student Work and Make Judgment on Learning Results and Communicate Findings

Congru

ence

of

Conte

nt,

Conte

xt

& C

ognit

ive D

em

and

MMann/SAS

Let’s Identify Taxonomic Levels1. Retrieve (Marzano)

Recognize, Recall, Execute

2. Comprehension Integrate, Symbolize

3. Analysis Match, Classify, Analyze,

Generalize, Specify

4. Knowledge Utilization Decision Making, Problem

Solving, Inquire Experimentally, Investigate

MMann/SAS

Topic The Universe

Benchmark ES.8.10

Compare different theories concerning the formation of the universe.

Topic Cells, Tissues, Organs, and Organ Systems

Benchmark BS.4.1

Describe different cell parts and their functions.

Level 1 - Knowledge Retrieval

Level 3 - Analysis

MMann/SAS

Drill and Practice

• Work with your table group

• Read each card

• Group by taxonomic level

MMann/SAS

Level of thinking helps determine the appropriate

assessment method

MMann/SAS

Analyze plot, setting, characterization, or conflict to interpret theme in a literary text.

• Describe the setting of the story.• Explain how the author uses his

characters to convey a message.• Compare the plot of this story to

the plot of the previous story.• Chooses a literary element (e.g.,

conflict). Describes how the author treats this element in the story. Assess how this element relates to the theme.

MMann/SAS

Knowing a taxonomy also helps in…

scaffolding instruction.

Identify

Explain

Compare

Create

MMann/SAS

Three Tiers of Skill and Assessment Work

• Drill & Practice

• Rehearsal &

Scrimmage

• Authentic

Performance

Thanks to Heidi Hayes Jacobs

MMann/SAS

The level of thinking in the

benchmark is the level of

thinking required to meet

proficiency.

MMann/SAS

Balanced Assessment Model

MMann/SAS

Performance Assessment is • an assessment (product or performance) based on

observation and judgment about its quality.

• the activities, problems, projects, and assignments students are asked to perform.

• anything from a special task at the end of instruction as in a culminating event, or naturally occurring events during regular instruction.

MMann/SAS

The Importance of Criteria

“Teachers [frequently] ask the wrong question first … “What do we do?” - putting the focus immediately on designing tasks - when they need to ask, “What do we want kids to know and be able to do? How well? What does quality look like? [We} need to ask these questions very clearly first.”

Mike Hibbard, Education Update, 38(4). p.5, ASCD, June, 1996.

MMann/SAS

To Know Criteria Requires ...

• Being exposed to the criteria from the beginning of instruction.

• Having terms defined. (lots of details)• Samples of strong and weak performances.• Practice with feedback using the vocabulary of the

criteria.• Focused revision of work.• Practice articulating the vocabulary for quality and

applying it to many situations.• Instruction consciously focused on subparts of the

criteria. Judy Arter, ATI

MMann/SAS

What is a Rubric?

• A scoring guide designed to provide constructive feedback to students

• Designed to show how important elements of a task would look in a progression from less well developed to exceptional along a continuum (Tomlinson, 2003).

• A Latin word that means “red.”

MMann/SAS

A Rubric =

Dimensions (essential qualities) +Continuum (Scale) +Descriptors of points on the scale +Work samples illustrating

those points.

MMann/SAS

Holistic Rubrics• Holistic rubrics have one performance

expectation description at each numerical level on the rubric.

• The product or performance is evaluated as a whole, and given a single score.

• Used “to obtain the overall impression of the quality of a performance or product.” (Wiggins and McTighe, 1999)

MMann/SAS

Holistic Rubrics

• Quicker to write and to use.– Summative because they evaluate work at

the end of the process.

• Fails to communicate to students, especially low performing students, what their shortcomings are

MMann/SAS

Holistic Rubric Example Fiction Writing Content Rubric

5 The plot, setting, and characters are developed fully and organized well. The who, what, where, when, and why are explained using interesting language and sufficient detail.

4 Most parts of the story mentioned in a score of 5 above are developed and organized well. A couple of aspects may need to be more fully or more interestingly developed.

3 Some aspects of the story are developed and organized well, but not as much detail or organization is expressed as in a score of 4.

2 A few parts of the story are developed somewhat. Organization and language usage need improvement.

1 Parts of the story are addressed without attention to detail or organization.

MMann/SAS

Analytical Rubrics• Use multiple descriptors for each

criterion evaluated.• Type of “task analysis” where teachers

award points on a criterion-by-criterion basis.

• Described as teaching rubrics because their design helps students improve their own performance.

MMann/SAS

Analytic Rubric Example

Fiction Writing Content Rubric

Criteria 4 3 2 1

PLOT: "What" and "Why"

Both plot parts are fully developed.

One of the plot parts is fully developed and the less developed part is at least addressed.

Both plot parts are addressed but not fully developed.

Neither plot parts are fully developed.

SETTING: "When" and "Where"

Both setting parts are fully developed.

One of the setting parts is fully developed and the less developed part is at least addressed.

Both setting parts of the story are addressed but not fully developed.

Neither setting parts are developed.

CHARACTERS: "Who" described by behavior, appearance, personality, and character traits

The main characters are fully developed with much descriptive detail. The reader has a vivid image of the characters.

The main characters are developed with some descriptive detail. The reader has a vague idea of the characters.

The main characters are identified by name only

None of the characters are developed or named.

MMann/SAS

Holistic or Analytical TraitHolistic

Use :• Quick snapshot of overall

status• When speed of scoring is

important• Simple products or

performances

Disadvantages:• 2 students can get same

score for different reasons• Can’t identify strengths &

weaknesses• Not useful for students

AnalyticalUse:• Planning instruction - show

relative strengths & weaknesses

• Details to teach student quality

• Detailed feedback• Precision more important that

speed:Disadvantages:• Scoring is slower• Takes longer to learn

MMann/SAS

Descriptive Terms for Differences in Degree

• Degrees of Understanding

• Degrees of Frequency

• Degrees of Effectiveness

• Degrees of Independence

• Degrees of Accuracy

• Degrees of Clarity

MMann/SAS

Descriptive Terms for Differences in Degrees

Understanding Frequency Accuracy Clarity

thorough/complete

consistently completely accurate

exceptionally clear

substantial generally generally accurate

generally clear

partial/ incomplete

sometimes inaccurate lacks clarity

misunder-standing

rarely major inaccuracies

unclear

MMann/SAS

Options for Selecting Rubrics

• Create your own - build from scratch

• Adopt - use an existing rubric

• Adapt - Modify or combine existing rubrics– Reword parts– Drop or change one or more scales– Omit irrelevant criteria– “Mix” and Match” rubrics– Change a holistic rubric into an analytic rubric– Modify for different grade levels

MMann/SAS

Guidelines for RubricsRubrics are effective when teachers utilize the

following criteria:– Use specific numbers like “2” or “3 or more” rather

than vague words like “some,” “many,” or “few.”– Use specific descriptors, rather than general

descriptors like “good” or “excellent.”– Use the vocabulary of the standards and

benchmarks.– State clear expectations for work so that all

teachers, students, and parents know the criteria for quality and the requirements for earning a grade.

Burke, 2006

MMann/SAS

Resources• Anderson, L.,Krathwohl, D. et al. (2001). A Taxonomy for Learning,

Teaching and Assessing. New York: Longman.

• Curriculum Associates:Assessing Levels of Comprehension.

• Lewin, L. & Shoemaker, B.J. (1998). Great Performances. Virginia: ASCD.

• Marzano, R.J. (2001). Designing a New Taxonomy of Educational Objectives. Thousand Oaks: Corwin Press.

• Popham, W. J. (2002). Classroom Assessment. Boston: Allyn & Bacon.

• Stiggins, R.J. et al. (2004). Classroom Assessment for Student Learning. Portland: ATI.

• Wahlstrom, D. (2002). Designing & Using High- Quality Paper-and-Pencil Tests. Virginia: Successline.

• www.k12.wa.us/CurriculumInstruct/ pubdocs/WERA/WERA2005_Webversion.pp

• http://www.stedwards.edu/cte/content/view/1536/49/