toolkit for evaluating alignment of instruction

8/12/2019 Toolkit for Evaluating Alignment of Instruction

1/143

Instructional Materials Evaluation Tool (IMET) ............III-1

EQuIP Quality Review Rubric .................................. III-21

Assessment Evaluation Tool (AET) ...........................III-27

Assessment Passage and Item Quality Criteria

Checklist ...............................................................III-29

Additional Resources for Evaluating Alignment of

Instructional Materials .............................................. IV-1

Appendix: Publishers Criteria for the Common

Core State Standards ................................................. V-1

TOOLKIT

for Evaluating Alignment ofInstructional and AssessmentMaterials to the Common Core

State StandardsJuly 2013


2/143


3/143

i

I. Introduction ............... ................ ................. ................ ............ I-1

II. Whats in the Toolkit? An Overview ................. ................ ........... II-1

III. The Toolkit: Tools for Evaluating Alignment of

Instructional and Assessment Materials ......................................III

Instructional Materials Evaluation Tool (IMET) ............... ............... III-1

Mathematics, Grades K8 ..............................................................III-1

Mathematics, High School (To Be Completed August 2013)

English Language Arts/Literacy, Grades K2

(To Be Completed August 2013)

English Language Arts/Literacy (Grades 35) and

English Language Arts (Grades 612) .......................................... III-11

EQuIP Quality Review Rubric ....................................................III-21

Mathematics .............................................................................. III-21

English Language Arts/Literacy, Grades K2 .................................III-23

English Language Arts/Literacy (Grades 35) and

English Language Arts (Grades 612) ............... ................. .....III-25

Assessment Evaluation Tool (AET) ............... ................ .............. III-27

Mathematics, Grades KHS........................................................ III-27

English Language Arts/Literacy, Grades 312 ...............................III-34

Assessment Passage and Item Quality Criteria Checklist .................. ..III-39

Mathematics, Grades 3HS........................................................ III-39

English Language Arts/Literacy Passages,

Grades 312 ............................................................................... III-41

English Language Arts/Literacy Items,Grades 312 ............................................................................... III-44

IV. Additional Resources for Evaluating Alignment of

Instructional Materials....................... ................. ................ .... IV-1

Achieve Open Educational Resource Rubrics ................................. IV-1

Qualitative Measures Rubric for Informational Text and

Qualitative Measures Rubric for Literature ...................................... IV-1

CCSS Grade Bands and Quantitative Measures .............................. IV-1

Illustrative Mathematics Task Review Tool ...................................... IV-1

VI. Appendix: The Publishers Criteria for the Common Core

State Standards ................ ................ ................ ........ V-1

Mathematics, Grades K8 ............... ................ ................ ........ V-1

Mathematics, High School ............... ................ ................ ...... V-23

English Language Arts/Literacy, Grades K2 ................. ........... V-43

English Language Arts/Literacy, Grades 312 ............... ........... V-52

TOOLKITfor Evaluating Alignment of Instructional

and Assessment Materials to theCommon Core State Standards


4/143


5/143

TOOLKITfor Evaluating Alignment of Instructional and AssessmentMaterials to the Common Core State Standards

Introduction


6/143

I-1

The Common Core State Standards (CCSS) are a set of academic

standards in mathematics and English language arts/literacy that

are grounded in evidence and designed to ensure that all students

have the academic knowledge and skills they need in these core

subjects to succeed after high school. The CCSS were developed

in a state-led process under the leadership of governors and chief

state school officers with participation from 48 states. The process

included the involvement of state departments of education, dis-

tricts, teachers, community leaders, experts in a wide array of fields,

and professional educator organizations.

A good place to begin to understand the CCSS is through a studyof the standards themselves and the key instructional shifts re-

quired in each discipline. In English language arts/literacy, students

will be exposed to a balance of literary and informational texts to

build a growing base of knowledge and wil l be expected to cite

evidence from within the texts in order to answer questions and

develop written or verbal responses. Students will also be expected

to develop facility with academic language and read texts that in-

crease in complexity as they progress so that all students are ready

for the demands of college- and career-level reading no later than

the end of high school. The instructional shifts in English languagearts/literacy are as follows:1

Building knowledgethrough content-rich nonfiction

Reading, writing, and speaking grounded in evidence from text,

both literary and informational

Regular practice with complex textand academic language

Focus and coherence are the two major evidence-based design

principles of the Common Core State Standards for Mathematics.2

These principles are meant to fuel greater achievement in a deep

and rigorous curriculum, one in which students acquire conceptual

understanding, procedural skill and fluency, and the ability to apply

mathematics to solve problems. Thus, the instructional shifts in

mathematics are as follows:3

Focus:focus strongly where the standards focus

Coherence:think across grades/courses, and link to major

topics in each course

Rigor:in major topics, pursue with equal intensity

conceptual understanding,

procedural skill and fluency, and

applications

To ensure that all students are able to meet these high expec-

tations, educators need access to high-quality and well-alignedinstructional and assessment materials. In support of the work

being done by both educators and developers to meet this need,

Achieve, the Council of Chief State School Officers and Student

Achievement Partners have developed this Toolkit for Evaluating

Alignment of Instructional and Assessment Materials. The purpose

of the Toolkit is to catalyze the impact that the CCSS can have on

student achievement by increasing the prevalence of CCSS-aligned,

high-quality instructional and assessment materials.

I. Introduction

1For more information about the shifts in English language arts/literacy, see achievethecore.org/elalitshifts2For some of the sources of evidence consulted during the standards development process, see pp. 9193 of CCSSM.3For more information about the shifts in mathematics, see achievethecore.org/mathshifts


7/143


8/143


Whats in the Toolkit?

Whats in the Toolkit?An Overview


9/143

II-1

The Toolkit is a set of interrelated, freely available instruments for

evaluating instructional and assessment materials for alignment

to the CCSS. The tools themselves are included in section III; see

Table A for a summary. Each tool in the Toolkit supports the ex-

pectations in the CCSS and derives from the Publishers Criteria

for the Common Core State Standards in English language arts/

literacy and mathematics, which were developed by lead authors of

the CCSS along with the National Governors Association, Council

of Chief State School Officers, Achieve, Council of the Great City

Schools and National Association of State Boards of Education.

The Publishers Criteria provide guidance for both developers and

purchasers of curricular materials by defining quality materials

aligned to the CCSS. The criteria were revised through conversa-

tions with educators, researchers, and other stakeholders to be

purposeful and strategic in both what to include and what to ex-

clude in instructional materials based on the CCSS.

The criteria were developed from the perspective that publishers

and purchasers are equally responsible for ensuring high-quality

instructional materials. They do not define, endorse or prescribe

curriculum; those decisions are, and should be, local within each

state or district. The instruments in this Toolkit do not express novelexpectations, but rather articulations of the Publishers Criteria for

use in practice. It is therefore highly recommended that the Pub-

lishers Criteria be read prior to using any of the included resources.

The Publishers Criteria for the Common Core State Standards can

be found in the Appendix to the Toolkit or online at www.core-

standards.org/resources or

www.achievethecore.org/publisherscriteria.

Educators are encouraged to integrate the Publishers Criteria and

the tools in the Toolkit into CCSS implementation efforts and to

use them to deepen shared understanding and support systematic

application of the criteria for CCSS-aligned instructional and as-

sessment materials. In doing this work, it is important to note that

the included tools do not address all factors that may be important

in determining whether instructional materials and assessments

are appropriate in a given local or state context but instead aim to

clearly articulate the criteria for alignment to the CCSS.

Successful implementation of the CCSS requires many actors

across the educational system to work in concert. Hence, the audi-ence for the Toolkit is intentionally broad, ranging from classroom

teachers to state administrators.

Potential Toolkit users include:

educators and administrators responsible for developing or

evaluating curriculum, or for making purchasing decisions for

comprehensive textbooks and textbook series in print and digi-

tal format;

educators and administrators responsible for developing,evaluating or making purchasing decisions for grade or

course-level assessment materials, including individual or sets

of assessments, item banks or individual assessment items; and

teachers and instructional coaches responsible for creating, or

selecting and reviewing, lesson plans and units.

II. Whats in the Toolkit? An Overview


10/143

II-2

Table A. Types of Tools in the Toolkit

Tools of each type are content specific, and in some cases, grade band specific.

Type of Tool Used for Evaluating

Instructional Materials Evaluation

Tool (IMET)

Comprehensive mathematics and English language arts or reading curricula in print and digital

format.

EQuIP Rubric for Lessons and

Units

Lesson plans and units of instruction in mathematics and English language arts/literacy.

Assessment Evaluation Tool

(AET)

Assessments or sets of assessments and item banks for mathematics and English language arts/

literacy, including interim/benchmark assessments, and classroom assessments designed to ad-

dress a grade or course.

Assessment Passage & Item

Quality Criteria Checklist

Assessment passages and assessment items or tasks.


11/143

II-3

Overview of the Tools in the Toolkit

Instructional Materials Evaluation Tool (IMET)

For each given subject area and grade band, the Instructional Ma-

terials Evaluation Tool (IMET) is used to evaluate a comprehensive

textbook or textbook series for alignment to the CCSS in mathe-

matics and English language arts/literacy. In addition, the IMET can

be used to deepen a shared understanding of the criteria for CCSS-

aligned classroom materials. There are four IMET tools, one each for

K8 Mathematics, High School Mathematics*, K2 English Language

Arts* and a combined tool for 35 English Language Arts/Literacy& 612 English Language Arts.

The IMET should be used for:

Informing decisions about purchasing a comprehensive text-

book or textbook series;

Evaluating previously purchased materials to identify neces-

sary modifications;

Building the capacity of educators to better understand what

CCSS-aligned textbooks look like; and,

Informing publishers of the criteria that consumers will use to

evaluate RFP responses for a comprehensive textbook or text-

book series.

a) Where to find online:

To view and download the IMET, please visit:

www.achievethecore.org/materialsevaluationtoolkit

b) Who uses:

The IMET is designed for use by educators and administrators

responsible for developing, purchasing and/or evaluating a com-

prehensive textbook and/or textbook series. This can include

content specialists, adoption committees and administrators at

the school, district or state level.

c) Target materials:

The IMET is designed to evaluate a comprehensive textbook

and/or textbook series (e.g., basal reading series, mathematics

series, anthologies, student workbooks, teacher editions and

supports) in print and digital format.

d) How to use:

The IMET in both mathematics and English language arts/litera-

cy is organized in two sections:

1. Section I Non-Negotiables: Materials must fully meet all of

the non-negotiables at each grade/course to be aligned to the

CCSS and to continue to Section II.

2. Section II Additional AlignmentCriteria and Indicators of

Quality: The criteria in this section are additional alignmentrequirements that should be met by materials fully aligned

with the CCSS. A higher score in this section indicates that

instructional materials are more closely aligned to the CCSS

than instructional materials that have a lower score.

For each non-negotiable in Section I, reviewers should make a

determination about whether the materials under review have

fully met the criterion based on the metrics provided. For all de-

terminations, reviewers should record a justification to ensure


12/143

II-4

that judgments and determinations are evidence based. Once all

the non-negotiables have been met, then (and only then) should

reviewers evaluate materials based upon Section II: Additional

Alignment Criteria and Indicators of Quality.

*IMET for High School Mathematics and K2 English Language

Arts/Literacy to be completed in August 2013.


13/143

II-5

EQuIP Rubric

Educators Evaluating Quality Instructional Products (EQuIP) is a

collaborative of states working with Achieve to increase the supply

of quality instructional materials that are aligned to the CCSS and

build the capacity of educators to evaluate and improve the quality

of instructional materials for use in their classrooms and schools.

The EQuIP Rubrics are a set of quality review tools to evaluate

the alignment of lessons, units and modules to the CCSS. There

are three EQuIP Rubrics, one each for Mathematics, K2 English

Language Arts/Literacy, and a combined rubric for 35 English

Language Arts/Literacy and 612 English Language Arts. EQuIPbuilds on a collaborative effort of education leaders from Massa-

chusetts, New York and Rhode Island that Achieve facilitated.

The EQuIP Rubrics should be used for:

Guiding the development of lessons and units;

Evaluating existing lessons and units to identify improvements

needed to align with the CCSS;

Building the capacity of teachers to gain a deeper understand-

ing of the instructional demands of the CCSS; and,

Informing publishers of the criteria that will be applied in the

evaluation of proposals and final products.


To view and download the rubrics and related training materials,

please visit: www.achieve.org/equip

b) Who uses:

The EQuIP Rubrics are designed for use by educators and ad-

ministrators responsible for developing, reviewing or making

determinations about materials for use in classrooms. This includes

classroom teachers, instructional coaches, instructional leaders and

administrators at the school, district or state level.


The EQuIP Rubrics are designed to evaluate lessons that include

instructional activities and assessments aligned to the CCSS that

may extend over a few class periods or days as well as units that

include integrated and focused lessons aligned to the CCSS thatextend over a period of several weeks. The rubrics are not designed

to evaluate a single task or activity or portion of a lesson. The ru-

brics intentionally do not require a specific template for lesson or

unit design.

d) How to use:

The EQuIP Rubrics can guide the development of lessons and units

as well as examine and evaluate existing lessons and units to iden-

tify improvements necessary to align with the CCSS. They can be

used by individuals or groups, integrated into formal review pan-els/processes and professional learning communities, and/or used

more informally to guide discussions and decision making.

The criteria in the EQuIP Rubrics are separated into four dimen-

sions: Alignment to the Depth of the CCSS, Key Shifts in the CCSS,

Instructional Supports, and Assessment. The EQuIP quality review

process emphasizes inquiry rather than advocacy; it is intended to

yield observations, judgments, discussions and recommendations

that are criterion- and evidence-based and designed to provide


14/143

II-6

guidance on how to strengthen the lesson or unit. As such, using

the EQuIP rubrics and quality review process leads to concrete sug-

gestions for improvement. Dimension 1, Alignment to the Depth of

the CCSS, is considered non-negotiable. If materials do not meet

many or most of the criteria for Dimension 1 (a rating of 2 or 3)

then no further review takes place. In order to be deemed exempla-

ry, a lesson or unit must receive high ratings in all four dimensions.


15/143

II-7

Assessment Evaluation Tool (AET)

The Assessment Evaluation Tool (AET) is a review tool to evaluate

the alignment of grade or course-level assessment materials for

alignment with the CCSS, including interim or benchmark assess-

ments and classroom assessments. In addition, the AET can also

be used to deepen a shared understanding of the criteria for CCSS-

aligned assessments. There are separate AET tools for KHigh

School Mathematics and 312 English Language Arts/Literacy.

The AET should be used for:

Informing decisions about purchasing assessment materials or

item banks designed to address a grade or course;

Evaluating previously purchased or developed assessment

materials and item banks;

Guiding the development or refinement of individual or sets of

assessments in a district or school;

Building the capacity of educators and content and assess-

ment specialists to better understand what CCSS-aligned

assessments look like; and,

Informing publishers of the criteria that will be applied in the

evaluation of proposals and final products.


To view and download the AET, please visit:


b) Who uses:

The AET is designed for use by educators and administrators

responsible for developing, purchasing and/or evaluating sets of

assessments and item banks. This includes content specialists,

assessment specialists, administrators and educators at the school,

district or state level.


The AET is designed to evaluate grade or course-level assessment

materials for alignment with the CCSS, including interim or bench-

mark assessments and classroom assessments.

d) How to use

The AET is organized as follows:

1. Non-Negotiables: Materials must fully meet all of the relevant

non-negotiables at each grade/course to be aligned to the CCSS.

2. Indicators of Quality: The indicators of quality are additional

dimensions of alignment. Although the assessments may be

aligned without meeting the indicators of quality, assessments

that do reflect these indicators are better aligned. In the AET

for English language arts/literacy, the indicators are incorporat-ed directly into each metric and in the AET for mathematics the

indicators are found in Section II.

For each non-negotiable, reviewers should make a determination

about whether the materials under review have fully met the criteri-

on based on the metrics provided. For all determinations, reviewers

should record a justification to ensure that judgments and determi-

nations are evidence based. Once all the relevant non-negotiables

have been met, then (and only then) should reviewers evaluate

materials based upon the Indicators of Quality.


16/143

II-8

Assessment Passage and Item Quality Criteria Checklists

The Assessment Passage and Item Quality Criteria Checklists are

review tools to evaluate the alignment of individual assessment

passages, items and tasks and to deepen shared understanding of

the criteria for CCSS-aligned assessment items. There are separate

checklist tools for Mathematics Items, English Language Arts/Liter-

acy Passages, and English Language Arts/Literacy Items.

The Assessment Passage and Item Quality Criteria Checklists

should be used for:

Evaluating assessment passages, items and tasks for align-

ment;

Guiding the development or refinement of assessment pas-

sages, items and tasks;

Building the capacity of educators and content and assess-

ment specialists to better understand what CCSS-aligned

passages, items and tasks look like; and

Informing publishers and item writers of criteria that will be

applied to their passages, items or tasks.


To view and download the Assessment Passage and Item Quality

Criteria Checklists, please visit:


b) Who uses:

The Assessment Passage and Item Quality Criteria Checklists

are designed for use by educators and administrators respon-

sible for developing, purchasing and/or evaluating assessment

passages, items or tasks. This includes content specialists and

assessment specialists and educators at the school, district or

state level.


The Assessment Passage and Item Quality Criteria Checklists are

designed to evaluate individual assessment passages, items and

tasks.

d) How to use:

The criteria for the Assessment Passage and Item Quality Cri-

teria Checklists are grouped into gates. Passages, items and

tasks must pass the first gate in order to be considered for an

assessment. The subsequent gates include additional criteria

that passages, items or tasks items should meet in order to be

fully aligned.


17/143The Toolkit: Tools for Evaluating

The Toolkit: Tools forEvaluating Alignmentof Instructional and

Assessment Materials



18/143


19/143


20/143

III-2


21/143

III-3


22/143

III-4


23/143

III-5


24/143

III-6

9

10


25/143

III-7

14


26/143

III-8

15

16

17

18


27/143

III-9

20


28/143

III-10


29/143

III-11


30/143


31/143

III-13


32/143

III-14


33/143

III-15


34/143

III-16


35/143

III-17


36/143

III-18


37/143

III-19


38/143

III-20


39/143


EQuIP Quality

EQuIP QualityReview RubricMathematics .......................................................................................................................................... III-21

English Language Arts/Literacy, Grades K2 .............................................................................................. III-23

English Language Arts/Literacy (Grades 35) and English Language Arts (Grades 612) ................................ III-25


40/143

III-21

EQuIP Rubric for Lessons & Units: MathematicsGrade: Mathematics Lesson/Unit Title: Overall Rating:

"#$ %&'() *'+*,- ,. /$*,0$/ 1*23 4#$ "*,56474$ 8'+*,- 79/ 4#$ -2::7+2*74,0$ /$0$:2;3$94 ;*2-$.. :$/ +< =7..7-#'.$44.> ?$@ A2*B> 79/ 8#2/$ (.:79/ 79/ 17-,:,474$/ +< C-#,$0$D

"#,. 0$*.,29 21 4#$ %&'() *'+*,- ,. -'**$94 7. 21 EF5GH5GID

J,$@ K*$74,0$ K23329. C44*,+'4,29 IDE L9;2*4$/ M,-$9.$ 74 #44;NOO-*$74,0$-23329.D2*PO:,-$9.$.O+ ;:$7.$ 744*,+'4$ %&'() 79/ *$54,4:$D

"# $%&'()*(+ +, +-* .*/+-,0 +-* 1122

""# 3*4 2-&0+5 &( +-* 1122 """# "(5+678+&,(9% 27//,6+5 ":# $55*55)*(+

"#$ :$..29O'9,4 7:,P9. @,4# 4#$

:$44$* 79/ .;,*,4 21 4#$ KK66N

!"#$%&' " '%& )* $#"+%,-%.%- //00 1"&2%1"&34'

'&"5+"#+6'7 &) &2% *8--

+%9&2 )* &2% '&"5+"#+' *)#

&%"4235$ "5+ -%"#535$:

0&"5+"#+' *)#;"&2%1"&34"- "?= "5+ >%--4)55%4&%+ &) &2% 4)5&%5&

@%35$ "++#%''%+:

)#C )* &2%

$#"+% 2".% .3'3@-% 4)55%4&3)5 &) &2% 1"B)# >)#C )* &2% $#"+%

"5+ "#% '8**343%5&-? @#3%*: A%'')5' "5+ 853&' +) 5)& 2)-+ '&8+%5&'

#%'9)5'3@-% *)# 1"&%#3"- *#)1 -"&%# $#"+%':

1,-*6*(8*< !2% 4)5&%5& +%.%-)9' &2#)8$2 #%"')535$ "@)8& &2%5%> 4)54%9&' )5 &2% @"'3' )* 9#%.3)8' 85+%#'&"5+35$': E2%#%

"99#)9#3"&%= 9#).3+%' )99)#&853&3%' *)# '&8+%5&' &) 4)55%4&

C5)>-%+$% "5+ 'C3--' >3&235 )# "4#)'' 4-8'&%#'= +)1"35' "5+

-%"#535$ 9#)$#%''3)5':

=&',6< F%G83#%' '&8+%5&' &) %5$"$% >3&2 "5+ + %1)5'"&%42"--%5$35$ 1"&2%1"&34' >3&2 "99#)9#3"&% @"-"54% "1)5$ &2%

*)--)>35$H

$//%&89+&,(3&2 9%#'3'&%54%=

42))'35$ "5+ "99-?35$ "5 "99#)9#3"&% 1)+%- )# '"&%$? &)

5%> '3&8"&3)5':

1,(8*/+79% >(?*65+9(?&('3&2 4)#%

4"-48-"&3)5' "5+ 1"&2%1"&34"- 9#)4%+8#%' 6>2%5 4"--%+ *)# 35

&2% '&"5+"#+' *)# &2% $#"+%7 &) @% 9%#*)#1%+ G834C-? "5+

"448#"&%-?:

"#$ :$..29O'9,4 ,. *$.;29.,0$ 42 07*,$/ .4'/$94 :$7*9,9P 9$$/.N

J54-8+%' 4-%"# "5+ '8**343%5& $83+"54% &) '899)#& &%"4235$ "5+ -%"#535$ )* &2%&"#$%&%+ '&"5+"#+'= 354-8+35$= >2%5 "99#)9#3"&%= &2% 8'% )* &%425)-)$? "5+

1%+3":

K'%' "5+ %54)8#"$%' 9#%43'% "5+ "448#"&% 1"&2%1"&34'= "4"+%134 -"5$8"$%=&%#135)-)$? "5+ 4)54#%&% )# "@'"4& #%9#%'%5&"&3)5' 6%:$:= 934&8#%'= '?1@)-'=

%D9#%''3)5'= %G8"&3)5'= $#"9234'= 1)+%-'7 35 &2% +3'439-35%:

I5$"$%' '&8+%5&' 35 9#)+84&3.% '$$-% &2#)8$2 #%-%."5&= &2)8$2&,9#).)C35$G8%'&3)5'= 9#)@-%1' "5+ &"'C' &2"& '&318-"&% 35&%#%'& "5+ %-343& 1"&2%1"&34"-

&235C35$:

L++#%''%' 35'T&3)5"- %D9%4&"&3)5' "5+ 3' %"'? &) 85+%#'&"5+ "5+ 8'%: )#C35$ "@).%

$#"+% -%.%-:

C '9,4 2* :29P$* :$..29 .#2':/N

F%4)11%5+ "5+ *"43-3&"&% " 13D )* 35'T&3)5"- "99#)"42%' *)# " ."#3%&? )*-%"#5%#' '842 "' 8'35$ 18-&39-% #%9#%'%5&"&3)5' 6%:$:= 354-8+35$ 1)+%-'= 8'35$ "

#"5$% )* G8%'&3)5'= 42%4C35$ *)# 85+%#'&"5+35$= *-%D3@-% $#)8935$= 9"3#,'2"#%7:

M#"+8"--? #%1).% '899)#&'= #%G83#35$ '&8+%5&' &) +%1)5'"&% &2%3#1"&2%1"&34"- 85+%#'&"5+35$ 35+%9%5+%5&-?:

N%1)5'"&% "5 %**%4&3.% '%G8%54% "5+ " 9#)$#%''3)5 )* -%"#535$ >2%#% &2%4)54%9&' )# 'C3--' "+."54% "5+ +%%9%5 ).%# &31%:

ID9%4&= '899)#& "5+ 9#).3+% $83+%-35%' *)# 9#)4%+8#"- 'C3-- "5+ *-8%54? >3&24)#% 4"-48-"&3)5' "5+ 1"&2%1"&34"- 9#)4%+8#%' 6>2%5 4"--%+ *)# 35 &2%'&"5+"#+' *)# &2% $#"+%7 &) @% 9%#*)#1%+ G834C-? "5+ "448#"&%-?:

"#$ :$..29O'9,4*$P':7*:< 7..$..$.

@#$4#$* .4'/$94. 7*$ 37.4$*,9P

.479/7*/.5+7.$/ -294$94 79/

.B,::.N

J' +%'3$5%+ &) %-343& +3#%4&=)@'%#."@-% %.3+%54% )* &2%

+%$#%% &) >2342 " '&8+%5& 4"5

35+%9%5+%5&-? +%1)5'"&%

&2% &"#$%&%+ //00:

L''%''%' '&8+%5& 9#)*343%54?8'35$ 1%&2)+' &2"& "#%

"44%''3@-% "5+ 85@3"'%+=

354-8+35$ &2% 8'% )* $#"+%,-%.%- -"5$8"$% 35 '&8+%5&

9#)19&':

J54-8+%' "-3$5%+ #8@#34'="5'>%# C%?' "5+ '4)#35$

$83+%-35%' &2"& 9#).3+%

'8**343%5& $83+"54% *)#

35&%#9#%&35$ '&8+%5&

9%#*)#1"54%:

C '9,4 2* :29P$* :$..29 .#2':/N

K'% ."#3%+ 1)+%' )*48##348-81,%1@%++%+

"''%''1%5&' &2"& 1"? 354-8+%

9#%,= *)#1"&3.%= '811"&3.%

"5+ '%-*,"''%''1%5&

1%"'8#%':

=9+&('< B C D E =9+&('< B C D E =9+&('< B C D E =9+&('< B C D E


41/143

III-22

EQuIP Rubric for Lessons & Units: Mathematics

!"#$%&"'()* !"# %&'()*+ ,#-)#. ,&/0)1 203-)4#5 10)*#0)' *3 4#*#06)7# *"# 8&'()*+ '74 '()976#7* 3: (#55375 '74 &7)*5 *3 *"# ;36637 ;30# $-? 5)-. %)# @)%#$%7-:

/3&"(; ,%34$ H'# !"9$()"'() :J ::J :::J :=*


42/143

III-23

EQuIP Rubric for Lessons & Units: ELA/Literacy Grades K-2

Grade: Literacy Lesson/Unit Title: Overall Rating:

!"# %&'() *'+*,- ,. /#*,0#/ 1*23 4"# !*,56474# 8'+*,- 79/ 4"# -2::7+2*74,0# /#0#:2;3#94 ;*2-#.. :#/ +< =7..7-"'.#44.> ?#@ A2*B> 79/ 8"2/# (.:79/ 79/ 17-,:,474#/ +< C-",#0#D

!",. 0#*.,29 21 4"# %&'() *'+*,- ,. -'**#94 7. 21 EF5GH5IJD

K,#@ L*#74,0# L23329. C44*,+'4,29 JDE M9;2*4#/ N,-#9.# 74 "44;OPP-*#74,0#-23329.D2*QP:,-#9.#.P+ ;:#7.# 744*,+'4# %&'() 79/ *# 54,4:#D

!" $%&'()*(+ +, +-* .*/+- ,0 +-* 1122 !!" 3*4 2-&0+5 &( +-* 1122 !!!" !(5+678+&,(9% 27//,6+5 !:" $55*55)*(+

!"# :#..29P'9,4 7:,Q9. @,4" 4"# :#44#* 79/ .;,*,4 21 4"#LL66O

!"#$%&' " '%& )* +,- ./01/2&%#"34 5566 *)#&%"3728$ "89 :%"#828$;

%D?:"#'

A0??%892> FBE "89 "#% )* '=**232%8& '3)?% *)# &7%

'&"&%9 ?=#?)'%;

G#)H29%' )??)#&=82&2%' *)# '&=9%8&' &) ?#%'%8&29%"' "89 28*)#D"&2)8 &7#)=$7 C#2&28$ "891)#

9#"C28$ "89 '?%"I28$ %>?%#2%83%';

C '9,4 2* :29Q#* :#..29 ."2':/O

.D?7"'2J% &7% %>?:232&E '4'&%D"&23 9%H%:)?D%8& )**)=89"&2)8": :2&%#"34 'I2::' A3)83%?&' )* ?#28&E

?7)8):)$23": "C"#%8%''E &7% ":?7"K%&23 ?#2832?":E

72$7 *#%@=%834 '2$7& C)#9'E "89 ?7)823'B;

L%$=:"#:4 283:=9% '?%32*23 *:=%834,K=2:928$&%3782@=%' '=??)#&%9 K4 #%'%"#37 A %;$;E D)82&)#%9

?"#&8%# #%"928$E 37)#": #%"928$E #%?%"&%9 #%"928$'

C2&7 &%>&E *)::)C28$ ":)8$ 28 &7% & %>& C7%8 &%"37%#

)# )&7%# *:=%8& #%"9%# 2' #%"928$ ":)=9E '7)#&

&2D%9 ?#"3&23% &7"& 2' ':2$7&:4 37"::%8$28$ &) &7%

#%"9%#B;

&,K"'%992'3=''2)8' "89 C#2&28$ &7#)=$7 '?%32*23E

&7)=$7&,?#)H)I28$ @=%'&2)8' "K)=& 3)DD)8

&%>&' A283:=928$ #%"9 ":)=9' "89E C7%8

"??:23"K:%E 2::='"&2)8'E "=92)1H29%) "89 )&7%#

D%92"B;

$89?:232&:41"#$%#*0 '("%)*('+ "3"9%D23 H)3"K=:"#4 "89

3)83%?&' )* '48&"> &7#)=$7)=& 28'=3&2)8;

C '9,4 2* :29Q#* :#..29 ."2':/O E69&,3%8&%#%9

:%"#828$ &7"& 2' '%@=%83%9E '3"**):9%9 "89

'=??)#&%9 &) "9H"83% '&=9%8&' &)C"#9

289%?%89%8& $#"9%,:%H%: #%"928$;

A9%9(8* ,0 =*>+5? O)3=' 28'=3&2)8 %@="::4 )8:2&%#"#4 "89 28*)#D"&2)8": &%>&' "' '&2?=:"&%9 28

&7% 5566 A?;PB "89 28923"&%9 K4 28'=3&2)8":

&2D% A37< +# 32*# 7;;:,-7+:# 7-*2.. 7 ?%#2%83% &7% 3)D?:%>2&4 )* &%>&;

O)3='%' )8 '%3&2)8' )* #237 &%>&A'B A283:=928$ #%"9 ":)=9'B &7"& ?#%'%8& &7%$#%"&%'& 37"::%8$%Q ?#)H29%' 92'3=''2)8 @=%'&2)8' "89 )&7%# '=??)#&' &)

?#)D)&% '&=9%8& %8$"$%D%8&E =89%#'&"8928$ "89 ?#)$#%'' &)C"#9

289%?%89%83%;

&%8'2)8' "891)# D)#% "9H"83%9 &%>& *)# '&=9%8&' C7) #%"9 )# C#2&%"K)H% $#"9% :%H%:;

C '9,4 2* :29Q#* :#..29 ."2':/O


43/143

III-24

EQuIP Rubric for Lessons & Units: ELA/Literacy Grades K-2

!"#$%&"'()* !"# %&'()*+ ,#-)#. ,&/0)1 203-)4#5 10)*#0)' *3 4#*#06)7# *"# 8&'()*+ '74 '()976#7* 3: (#55375 '74 &7)*5 *3 *"# ;36637 ;30# 5@ *"'* 6#'5&0# .)*")7 *"# 90'4#A(#-#( 30 *#D* 1362(#D)*+ /'74 '74 '0# 3: 5&::)1)#7* 8&'()*+ '74 5132# :30 *"# 5*'*#4 2&0235#H 5## F6$ G"))"* G"3$ H#/#$ H#/*4/34+

(* I*.1(+6 J/*.:/.$ K3#+AJ(#$3/9D '* ...E130#5*'74'045E309FJSNAS)*#0'1+= '74 *"# H:@@1$)$*# 5"3 K@@$*4(B K% !$8 L$+$/396 "* F$B# G")@1$B(#D /+ 8$11 /+ M:/*#(#/#(7$ /*4 M:/1(#/#(7$ N$/+:3$+'*

...E'1")#-#*"#130#E309F5*#'(A*"#5#A*33(5F*#D*A1362(#D)*+E


44/143

III-25

EQuIP Rubric for Lessons & Units: ELA/Literacy (Grades 3-5) and ELA (Grades 6-12)

Grade: Literacy Lesson/Unit Title: Overall Rating:

!"# %&'() *'+*,- ,. /#*,0#/ 1*23 4"# !*,56474# 8'+*,- 79/ 4"# -2::7+2*74,0# /#0#:2;3#94 ;*2-#.. :#/ +< =7..7-"'.#44.> ?#@ A2*B> 79/ 8"2/# (.:79/ 79/ 17-,:,474#/ +< C-",#0#D

!",. 0#*.,29 21 4"# %&'() *'+*,- ,. -'**#94 7. 21 EF5GH5IJD

K,#@ L*#74,0# L23329. C44*,+'4,29 JDE M9;2*4#/ N,-#9.# 74 "44;OPP-*#74,0#-23329.D2*QP:,-#9.#.P+ ;:#7.# 744*,+'4# %&'() 79/ *#54,4:#D

"# $%&'()*(+ +, +-* .*/+- ,0 +-* 1122 ""# 3*4 2-&0+5 &( +-* 1122 """# "(5+678+&,(9% 27//,6+5 ":# $55*55)*(+!"# :#..29P'9,4 7:,Q9. @,4" 4"# :#44#* 79/

.;,*,4 21 4"# LL66O

!"#$%&' " '%& )* $#"+%,-%.%- //00123425&%#"67 '&"8+"#+'9

:86-;+%' " 6-%"# "8+ %%9$9E 8)&%'E ';AA"#5%'E '@)#& #%'=)8'%'E )# *)#A"- %''"7'?9

$89+5? R5&@58 " 6)--%6&5)8 )* $#"+%,-%.%- ;85&' " C"-"86% )*58*)#A"&5)8"- "8+ -5&%#"#7 &%%9$9E A;-&5=-% +#"*&' "8+ #%.5'5)8' ).%# &5A%? "8+ '@)#&E

*)6;'%+ #%'%"#6@ =#)M%6&'E 586)#=)#"&58$ +5$5&"- &%37< +# 32*# 7;;:,-7+:# 7-*2.. 4"#


45/143

III-26

EQuIP Rubric for Lessons & Units: ELA/Literacy (Grades 3-5) and ELA (Grades 6-12)

!"#$%&"'()* !"# %&'()*+ ,#-)#. ,&/0)1 203-)4#5 10)*#0)' *3 4#*#06)7# *"# 8&'()*+ '74 '()976#7* 3: (#55375 '74 &7)*5 *3 *"# ;36637 ;30# 5@ *"'* 6#'5&0# .)*")7 *"# 90'4#A(#-#( *#D* 1362(#D)*+ /'74 '74 '0# 3: 5&::)1)#7* 8&'()*+ '74 5132# :30 *"# 5*'*#4 2&0235#H 5## F6$ G"))"* G"3$ H#/#$ H#/*4/34+ (*

I*.1(+6 J/*.:/.$ K3#+AJ(#$3/9D '* ...E130#5*'74'045E309FJSNAS)*#0'1+= '74 *"# H:@@1$)$*# 5"3 K@@$*4(B K% !$8 L$+$/396 "* F$B# G")@1$B(#D /+ 8$11 /+ M:/*#(#/#(7$ /*4 M:/1(#/#(7$ N$/+:3$+'*

...E'1")#-#*"#130#E309F5*#'(A*"#5#A*33(5F*#D*A1362(#D)*+E


46/143


Assessment Evaluation

AssessmentEvaluation Tool (AET)Mathematics, Grades KHS ....................................................................................................................... III-27

English Language Arts/Literacy, Grades 312 ............................................................................................... III-34


47/143

III-27


48/143

III-28

!

!


49/143

III-29


50/143

III-30


51/143

III-31


52/143

III-32

o

o

o

o

o

7

http://www.ccssitemdevelopment.org/downloads/Quality%20Criteria%20Checklists%20for%20Items.pdf


53/143

III-33


54/143

III-34


55/143

III-35


56/143

III-36


57/143

III-37


58/143

III-38


59/143

Assessment Passage and

Assessment Passageand Item QualityCriteria Checklist


Mathematics, Grades 3HS ........................................................................................................................ III-39

English Language Arts/Literacy Passages, Grades 312 ................................................................................ III-41

English Language Arts/Literacy Items, Grades 312 ..................................................................................... III-44


60/143


61/143

III-40

2.F

ContextQuality:Whenasituationalorre

al-worldcontextispresentfor

theitem

ortask,isthecontextlogical,con

vincingandnecessaryto

assessthestandard?

2.G

Stimuli:

Whendiagrams,pictures,orillustrationsarepresent:

Aretheyconsistentwiththeindicatedassessmentclaimandstandard(s);

and

Dothey

supportcomprehensionorprovidemathematicalmeaningforthe

item;an

d

Isthepurposeofthestimuliclear?

2.H

Rubric:

Whenarubricispartoftheitemo

rtask:

Doesitcorrectlycommunicatethepurposeoftheitemortask;and

Doesitaccountforallvalidanddistinctso

lutionpathsthatarelikelyto

bedeve

lopedbystudents?

Doespa

rtialcreditcorrespondtopartialfu

lfillmentoftheassessment

targetandstandard(s)athand?

Accepted(allYs)

Acceptedconditionally,withcommentstobeaddressed

Rejected


62/143


63/143


64/143

III-43

3.A

TextG

enres/Characteristics:Dothe

textsorotherstimuliintheset

clearly

reflectthespecificrequirementsoftherelevantpairedormulti-

textstandardsortargets,theitem

ortaskmodel,and/orthetest

blueprint?

Foran

explanationofCCSSrequirementsforpairedormulti-text

stimuli,seethewebsitewww.ccssitem

development.organddownload

thePa

iredPassagesEssay:

http://w

ww.ccssitemdevelopment.org/downloads/Essay%20on%20Pa

ir

ed%20

Passages_September%202012.pdf

3.B

Relati

onshipsAmongTexts:Dothe

texts/stimulihaveaclear

an

d

meaningfulre

lationship,withtestablepointsa

risingfrom

significantpoints

ofcomparison

orintegrationofideas?

3.C

Video

orAudio:Ifthetextisavideooraudiostimulus,doesismeet

thesamequalitycriteriaasforotherte

xts?Inaddition,isthequalityo

f

sound

and/orvideoappropriateforuseonassessments?

If,asaset,th

etextsdonothaveaYinallo

fthecriteriaabove,removethe

mfrom

consideration.Ifthe

textsdomeet

thecriteriainthethirdgate,pro

ceedtothefourthgate

4.FOURTH

GATE:PAIRSORMULTI-TEX

TSTIMULI

Asetoftextsthat

passesthethirdgatemustmeetthefollow

ingcriteria,asapplicable.

Y/N

Explanation

4.A

Ancho

rText:Fortasksthatsimulate

research,isonetextclearly

appropriatetobetheanchortext,prov

idingfoundationalknowledge

andleadingnaturallytoadditionalreadingandexploration?

Ify

es,placeaYinthecolumnto

theright.

Ift

hefirsttextdoesnotmeettherequirementsforananchortext,

pla

ceanNinthecolumntother

ightandsuggestareassignment

for

anexistingtextintheExplanat

ioncolumnorremovetheset

fro

mconsiderationuntilanapprop

riateanchorpassageislocated

.

4.B

Audio

orVisualElements:Dothem

ultimediaelementsaddvalue

tothe

set?(Audioorvisualelementsshouldprovidetestablepoints

ofcom

parisonorintegration,ratherth

ansimplyentertainment.)

Ify

es,pleaseaYinthecolumntotheright.

Ift

heaudioorvisualmaterialdoes

notaddvalue,make

recommendationsforchangesintheExplanationcolumn.

Accepted(all

Y

s)

Acceptedcon

ditionally,withcommentstobe

addressed

Rejected


65/143


66/143


67/143


68/143

III-47

4.A

StandardCoverage:Doestheset

addressasmanydifferent

Stan

dards(andevidencestatements/targets)aspossible,with

item

sbasedontheindividualcharacteristicsofthetextand

focu

sedonkeyaspectsofthetext?

Thesetofitemsshouldbe

exte

nsiveandrobustenoughthata

goodselectionofitemswill

remainafterfieldtesting.

I

fyes,placeaYinthecolumn

totheright.

I

fno,placeanNinthecolumnattherightandgivereasons

i

ntheExplanationcolumn.

4.B

Item

Cluing:Dotheitemsavoidcluingtheanswertootheritems

inth

eset?

I

fyes,placeaYinthecolumn

totheright.

I

fno,placeanNinthecolumnattherightandindicatein

t

heExplanationcolumnwhichitemsclueeachothersothat

t

heycanbemarkedinthebank

appropriately(nottobeused

o

nthesameform).

Accepted(a

llYs)

Acceptedco

nditionally,withcommentstob

eaddressed

Rejected


69/143


70/143


Additional Resources for

Additional Resources forEvaluating Alignment ofInstructional MaterialsAchieve Open Educational Resource Rubrics .................................................................................................. IV-1

Qualitative Measures Rubric for Informational Text and

Qualitative Measures Rubric for Literature ......................................................................................................IV-1

CCSS Grade Bands and Quantitative Measures ...............................................................................................IV-1

Illustrative Mathematics Task Review Tool ...................................................................................................... IV-1


71/143

IV-1

IV. Additional Resources for Evaluating Alignment of

Instructional and Assessment Materials

Achieve Open Educational Resource (OER) Rubrics

Open Educational Resources (OER) are instructional materials,

often in a digital and online format, that contain an open copyright

license that allows educators to share, reuse and edit these

materials. The OER Rubrics can be used in developing or evaluating

OER to help determine the degree of alignment of OER to the

CCSS, and to determine aspects of quality of OER. OER range from

a single lesson or instructional support material (such as a problem

set or game) to a complete unit or set of support materials.

To view and download, please visit:

http://www.achieve.org/oer-rubrics

Qualitative Measures Rubric for Informational Text and Qualitative

Measures Rubric for Literature

Developed by the Council of Chief State School Officers English

Language Arts state collaborative to support qualitative analysis of

what makes a given text complex, these qualitative rubrics guide

educators in measuring features of text complexity, such as: textstructure, language clarity and conventions, knowledge demands,

and levels of meaning and purpose.


http://achievethecore.org/ela-literacy-common-core/

text-complexity/qualitative-measures or

www.ccsso.org/textcomplexity (Launching August 2013)

CCSS Grade Bands and Quantitative Measures

A step-by-step guide to accessing free, online tools that identifythe appropriate grade band for a text.


http://achievethecore.org/ela-literacy-common-core/

text-complexity/quantitative-measures

Illustrative Mathematics Task Review Tool

The Illustrative Mathematics task review criteria are used to

evaluate K12 mathematics tasks designed specifically toillustrate the CCSSM and intended for inclusion on the Illustrative

Mathematics website (http://www.illustrativemathematics.org/).

Each task is intended to be part of a highly crafted set that

illustrates the breadth, depth and nuances of each standard, cluster,

domain, grade level, or conceptual category in the standards. In

order to be published at Illustrative Mathematics, a task must meet

all eight criteria described in the review form.


https://docs.google.com/file/d/0B7UDDaSOTTwkcWRJZjRGNWFWTWs/edit?usp=sharing.


72/143


73/143


Appendix: Publishers Criteria

Appendix: PublishersCriteria for the CommonCore State StandardsMathematics, Grades K8 ............................................................................................................................. V-1

Mathematics, High School ...........................................................................................................................V-23

English Language Arts/Literacy, Grades K2 ..................................................................................................V-43

English Language Arts/Literacy, Grades 312 .................................................................................................V-52


74/143


75/143


76/143


77/143


78/143


79/143


80/143


81/143

V-8

!"#$&

'!()*+

,-./(0102'0!

-34-54,-./

6789$78":;7

"?@A;;=>2=8#?$

@9;9B$'9"?@"7@>

.C!"#$%"'()*"+,"+-./')'0%1'2342+)546%7$54'7%)'574)#84+%$%1'

27849)74+1)3%)%

54%12'45

%:4'57843)+249)*"+170";78

41+7194"'7849)*"+$7E$9B$:;FG>"?@F;B$7$?F$;:9B$

'9"?@"7@>HI;9B">>$>>J$?9F

;?>;798"B"E$@$>8#?"9$@

F=G>9$7>"9$"FB#7"@$=$E$=">J"K;7H"@@898;?"=H;7>GDD;798?#H5L89BF=G>9$7>@$>8#?"9$@">J"K;7

F;JD78>8?

#9B$J"K;7L;7M;:$"FB#7"@$C>9B$N@$@8F"9$9B$="7#$

J"K;789N;

:9B$8798J$.-

;?9B$J"K;7L;

7M;:$"FB#7"@$C

AB8>F789$7

8;?"=>;"DD=8$>9;@8#89"=;7;?

=8?$J"9$78"=>L89B;G9:8O$@D"

F8?#D="?>C'GFB9;;=>"7$

$OD=8F89=N@$>8#?$@:;7:;FG>H>;9B"9>9G@

$?9>>D$?@9B$="7#$J"K;789N

;:9B$8798J$;?9B$J"K;7

L;7M;:$"FB#7"@$C

*;9$9B"9

"?8JD;79"?9%$=%47;:9B$J"K;7L;7M8?#7"@$>P!&8>9B$D

7;#7$>>8;?9B"9=$"@>

9;L"7@J

8@@=$Q>FB;;="=#$I7"R>$$A"I=$

.H?$O9D"#$SC#8E$$>D$F8"==NF"7$:G=97$"9J$?9

9;9B$>$F

=G>9$7>"?@9B$878?9$7F;??$F98;?>C..

8AB$J"9$78"=>>B;G=@@$E;9$"9=$">9TUV"?@GD9;"DD7;

O8J"9$=N&UV;:9B$F=">>98J$9;9B$J"

K;7L;7M;:9B$#7"@$L89B+7"@$>

P!,?$"7$79B$GDD$7$?@;:9B"97"?#$H8C$CH&UVC

5W;7F=G>9$7Q=$E$=$JDB">$>"9#7"@$>P!,H>$$

B99DX44LLLC"FB8$E$9B$F;7$C;7#4@;L?=;"@>4>B

;G=@@$E;9$"9=$">9TUV"?@GD9;"DD7;O8J"9$=N&UV;:9B$F=">>98J$9;9B$J

"K;7L;7M;:9B$#7"@$L89B+7"@$>

P!,?$"7$79B$GDD$7$?@;:9B"97"?#$H8C$CH&UVC

.

.W;7@;J"8?QINQ@

;J"8?D7;#7$>>8;?>8?9B$'9"?@"7@>H>$$B99DX448J$CJ"9BC"78Z;?"C$@G4D7;#7$>>8;?>C


82/143


83/143


84/143


85/143


86/143


87/143


88/143


89/143

" 219>


90/143

V-17

"#$%&'

(")*+,-.&/)0102(0!.34.54-.&/

26789%78:;#9=%>#986?#;@?9#98?986?8;

>#9%78#A?C%7?

S8;6AB@8;$?

W?8;$F#78#CA%?#

;@K7898;$#;@?:AF8;$%XB#98:;?9:

?:AF%G7:CA%>?

)%6:$;8Y8;$#;@

B?8;$G7:G:798:;#A7%A#98:;?=8G?9:

?:AF%G7:CA%>?

,7#G=8;$G7:G:798:;#A7%A#98:;?=8G?#;@A8;%#7

?

V:7E8;$K89=@8?978CB98:;?#

;@>%#?B7%?:GA%G7:C#C

8A89I#;@7#;@:>?#>GA8;$

V:7E8;$K89=C8F#78#9%6#9%$

:786#A@#9#S%H$HJ9K:ZK#I

9#CA%?U

V:7E8;$K89=C8F#78#9%>%#?B7%>%;9@#9#S%H$HJ

?6#99%7GA:9?U#;@A8;%#7>:@

%A?


91/143


92/143


93/143

V-20

"#$%&'

(")*+,

&'-.)/0/1(/!

'23'43&'-.

56%%789:689;:86%?#9@A%B@68;

A%9C8DE%?E=EF#EG=%CGAEG6%H

=@%C


94/143


95/143


96/143


97/143


98/143


99/143


100/143


101/143


102/143

V-29

"#$%&

'()'*)+',-

./01%/0#23/4

#1%/0#56#7893356:50$7%8131;%?/%;%760@%>#1%/0#56A3;%/%715B

07A5C8%#55

321;%61#78#/8607D0$;I35JE01;#>

#K3/01B321;%10>%

8%@31%813

IC05807$1;%?#/10AC5#/L73E5%

8$%#786L05561;#1#/%>361!""#$%!'

!)*

"+'+',-$.$/'

13#E08%/

#7$%32A355%$%>#K3/6#78?3616%A378#/B?/3$/#>6=4#1%/0#

568%@%53?%813?/%?#/%

61C8%71623

/#K3/6%76C/%1;#1C616

1C8B13I%A355%$%#78

A#/%%//%#8BF..#1%/0#56>

C61$0@%%6?%A0#55BA#/%2C5

1/%#1>%71

131;%83>#076JA5C61%/6J#786

1#78#/86079#I5%,J07A5C807$1;%0/071%/A377%A10376#78

1;%0/#??50A#10376!"#$%&'(&)'$"#"+$,('-$./'%01&'/2'(#1=

9;06A/01%/037#563#??50%61380$01#53/375

07%>#1%/0#56E01;3C120S%8?#A07$?5#76=#K3/01B321;%0/10>%37E08%5B

#??50A#I5%

E3/L=


103/143

V-30

"#$% & '()'*)+',-

.#/0% ,1 2345%45 6738 2299: ; ?@@0 "73$7#8B #4= 2#7%%7BI

!"#$%& ()*

+"(),-,./01%$&( 2")3,-4)5 6%4#%,&.

7,(,-5,-35 ()*

8&4$($-0-,./990.-)1 :%. ;(&4# 6&(*%5 ?!@AA

!BC!D C%(0

!"#$%&5J K35L

A0DB5%7B

A0DB5%7 549G-5,-3(,-4)

(99&49&-(,% ,4 G-1G 53G440 $.J

?@@0>


104/143


105/143


106/143


107/143


108/143


109/143

" HJ41>NE


110/143

V-37

"#$%&'

()*(+*,(&-

./#$0#1234#56%23$0#7823#9.2:15;6//%./9

1#4%0/#6.%2/$9%.>;0

6#9$?#

$%.%@%6;71%94AB;4%48#4@#0/%4:/9>;01#42#9.4:7%2;>0%

70%2%94#4/;92!$0#7823

.0#C/9

$23/1#$%23#9.4#56%2/9#../4

/;94;4%=4!48%6#9$?#$%.%1#9.2

48#4D9$6/286#9$?#$%6%#09%02>#


111/143


112/143

"


113/143

V-40

"#$%&'

()*(+*,(&-

.$%/#0012013#4%5%66

71%%829:129;3#6%?012;

>%9@254%=464A#4B6%@B>4B1%/

60%@3:3@;#@C$12B58

C52D>%8$%82524#66B9%1%#8%16:12

9#>>@B>4B1%6A#E%4A#4C52D>%8$%F6390>%%=0>#5#43256

213>>B64

1#4325621A35466@#::2>8@2901

%A%56325GH

71%%829:129B55%@%66#1I>#5$B#$%@290>%=34IH

JA%E36B#>8

%63$5365!483641#@435$21@A#243@?21#39%8#4#8B>40B1@A#6%16?;B43564%#86%1E%625>I

426B00214I2B5$64B8%54635%5$#$35$4A2B$A4:B>>ID34A4A%6B;K%@4H

LB00214:21

M5$>36A>#5$B#$%>%#15%16364A

2B$A4:B>#58A%>064A26%>%#15%16429%%44A%6#9%

64#58#186#

6#>>24A%164B8%546H.>>2D35$M5$>36A>#5$B#$%>%#15%1642@2

>>#;21#4%#64A%I6413E%42

>%#15#586A2DB58%164#5835$35#5%5E31259%54DA%1%M5$>36A36B6%8

#64A%9%83B92:

35641B@4325

D3>>$3E%4A%94A%6B002144A%

I5%%8429%%44A%31#@#8%93@

$2#>6HN#4%13#>6@#5

641B@4B1%35

4%1#@43256350#316?3569#>>$12

B06?#58354A%>#1$%$12B036A>#5$B#$%>%#15%1693$A4;%6AI426A#1%

21#>>ID34A4A%>#1$%

$12B0?;B493$A4524A#E%012;>%96A#135

$21#>>ID34A#69#>>$12B021350#316H#5$B#$%?4A%I

93$A4@A22

6%424A35C#;2B4#58836@B664

A%012;>%96354A%31:3164>#5$B

#$%?#584A%5D211I

#;2B48235$3435M5$>36AHG


114/143

"#

-.

1#


115/143

V-42

#$%&'

'()'*)&'+,

.#/0#10-#-2%33#-.4%0%5#/0-67.4%48$4-94663-.#/0#10-#/01%#08/%--7619#1%%1-#/0#280%

#/$%679633%$%

5#:61-;

-.#/0#10-38?%@;AA;,#

1%B%-..46=$4.67#-0%-918C.86/-67965C6/%/.-?833-

4#.2833B%#CC

38%073%D8B3E0=18/$48$4-946638/.#/0%528.46.4%1-8/.4%9

6=1-%67560%38/$.#-?-

/06.4%1-=B-.#/.8#3#CC389#.86/-;F48-#38$/-

28.4.4%0%5#/0-67C6-.-%96

/0#1E%0=9#.86/761

#1%%1-#/0761

#280%1#/$%679633%$%5#:61-;F4=->24%/48$4-94663-.=0%/.-261?28.4.4%-%

?833-8/48$4-94

663>.4%E#1%/6.261?8/$B%36

2$1#0%3%G%3H/61#1%.4%E1%G

8%28/$;


116/143

DO


117/143

V-44

OCUMENTORGAN

IZATION


118/143


119/143


120/143


121/143


122/143


123/143


124/143


125/143

DOC


126/143

V-53

CUMENTORGANIZATION


127/143


128/143


129/143


130/143


131/143


132/143


133/143


134/143


135/143


136/143


137/143


138/143


139/143


140/143


141/143


142/143


143/143

toolkit for evaluating alignment of instruction

Documents