reviewing and evaluating student learning objectives curriculum, instruction, technology october 12,...

REVIEWING AND EVALUATING STUDENT LEARNING OBJECTIVES

CURRICULUM, INSTRUCTION, TECHNOLOGY

OCTOBER 12, 2012

Everything an Administrator Needs to

Know AboutStudent Learning

Objectives

Created by Jane Boyd, Program Coordinator, Nassau BOCES Student Support Services

1. Assess and identify priorities and academic needs.

2. Identify who will have State-provided growth measures and who must have SLOs as “comparable growth measures.”

3. Determine District rules for how specific SLOs will get set.

4. Establish expectations for scoring SLOs and for determining teacher ratings for the growth component.

5. Determine District-wide processes for setting, reviewing, and assessing SLOs in schools.

Components of a SLO: 5 District Decisions

From NY DOE engageny.org Introduction to SLO Deck

Translating SLO’s to the HEDI Scale

This table illustrates the relationship between the Growth or Comparable Measure component to the Overall Composite Score.

Components of a SLO: NYSED SLO Framework

All SLOs MUST include the following eight basic components:

Student

PopulationWhich students are being addressed?

Learning Content

What is being taught? CCSS/National/State standards? Will this goal apply to all standards applicable to a course or just to specific priority standards?

Interval of Instructional Time

What is the instructional period covered (if not a year, rationale for semester/quarter/etc)?

EvidenceWhat assessment(s) or student work product(s) will be used to measure this goal?

BaselineWhat is the starting level of learning for students covered by this SLO?

Target(s)What is the expected outcome (target) by the end of the instructional period?

HEDI Criteria

How will evaluators determine what range of student performance “meets” the goal (effective) versus “well-below” (ineffective) , “below” (developing), and “well-above” (highly effective)?

Rationale Why choose this learning content, evidence and target?4

From NY DOE engageny.org Introduction to SLO Deck

Student Learning Objective: SLO

Academic GoalSet at the start of the courseRepresents the most important learningSpecific and measurableBased on prior student learning dataAligned to standardsAligned to school or district priorities

Guidance on NYS Dist. Goal-Setting Process: SLO. March 2012

Definition:

Student Learning Objective:

SLOToday!

Where you are and where you are going What are the criteria for each aspect of the SLO How can you manage this process What are your concerns

District rules for how specific SLOs will get set

Will the District require the use of existing, common District-wide assessments for a specific grade/subject?

Are there grades/subjects where the District wants to prioritize building or buying additional District-wide assessments?

Are there groups of teachers where group or team results based on state assessments are appropriate?

Are there grades or subjects where the District can identify priority learning standards or other District-wide guidance for schools and teachers?

What will the District require for any remaining teachers not covered by the above?

District Expectations for Scoring

For each group of grade/subject teachers with similar growth goal approaches, Districts must determine and communicate the District’s expectations for student learning growth relative to baselines and specify how teachers will be awarded HEDI ratings and earn from 0-20 points based on the results obtained, consistent with State Regulations and guidance.

District-wide Processes

What tools will principals use to assess the rigor of teacher-determined targets?

Will Districts review all goals or spot check goals and targets set by schools and teachers in any way to ensure rigor and comparability?

How will procedures to monitor progress of students on SLO targets and final results reviews be handled?

Districts may want to consider other teacher evaluation procedures like evaluator/teacher conferences, and procedures around classroom observations as well as District processes for data-driven inquiry meetings to be sure that sufficient time and coordination is provided.

Room for Decision Making

D23, p.40. In some cases, district and BOCES leaders may develop entire SLOs (or just targets and/or HEDI expectations) for all teachers or certain subject area teachers, and in other instances, district and BOCES leaders may leave the development of the SLO (or just targets and/or HEDI expectations) to principals and/or principals and teachers.

In some cases, the district/BOCES may decide that all SLOs must use the same target expectations based on growth progress norms while in other instances the district/BOCES may allow principals and/or teachers to propose a target based on the starting level of the students covered within the SLO. Decisions as to whether all teachers of the same subject will use a common SLO (or whether principals may even dictate this) are allowable; however, these are decisions first to be made by district/BOCES leaders.


SLOWhen setting SLO’s by course or grade:

Learning content, Interval of Instructional Time, Evidence, Targets, HEDI Criteria are all the same.

Population, Baseline data, Differentiation of targets based upon baseline data, and rationale will be unique to the roster.


SLODifferent district approaches for setting SLO’s:

1. Director’s and Chairpersons create the SLO’s and assessments with Central Office guidance and disseminate them to faculty

2. Director’s and Chairpersons train and over see the creation of SLO’s and assessments within departments or across grade levels with Central Office guidance

3. Teacher’s create SLO’s using district decisions and SLO’s are evaluated by Supervisors – principals, chairpersons, etc.

In all situations, there comes a point where a review needs to take place.


SLODirector’s and Chairpersons create the SLO’s

and assessments and disseminate them to faculty

May select Learning Content – standards Make decisions regarding evidence:

Create baseline assessments or make decisions regarding data to be used as the baseline

Create the summative assessments Apply district decisions for target setting and HEDI scores

Teachers can: Teacher’s describe their population Analyze the data and record that analysis in the “Baseline” Add any information to “Rationale”


SLODirector’s and Chairpersons train and oversee

the creation of SLO’s and assessments within departments or across grade levels

Highly collaborative within the department or grade level Director’s/Chairpersons facilitate as teachers contribute

to conversations regarding learning content for courses/grades

Director’s/Chairpersons create a format for the creation of assessments or use of baseline data

Data is analyzed by teachers and discussed with Director’s/Chairpersons to create appropriate targets.


SLOTeacher’s create SLO’s using district

decisions and SLO’s are evaluated by Supervisors – principals, chairpersons, etc. Teachers receive training and create SLO’s for

their courses using district decisions regarding any learning content, targets and the HEDI scoring.

Appropriate assessments are a collaborative effort and finalized by a supervisor

Data is gathered and the SLO is written with the idea of recommending appropriate targets, etc. with the given district guidelines.

Student Learning Objective: SLOWho needs an SLO?

Right now, it is anyone that is not receiving a state-provided growth measure which is 4-8 ELA and Math. See “Purple Memo” and “Assessment Options for SLO’s”

More than one SLO?Maybe

How do you know what you need? 50% or more of your students must be covered under your

SLO’s. That might mean you need two or maybe even three.

Begin with the courses taught that have the largest number of students, combining sections with common assessments.*

*p10 & 12 SLO Guidance March

Minimum Group Size

Who needs an SLO? Another situation - When the minimum number of students is not met.

Other situations? Part-time teachers? §80-1.1 – Does not discuss part

time classroom teaching service. Long term subs? Librarians – Speech? Second or third SLO just in case my population

changes

Who Needs an SLO?

D43. How are SLOs for Library/Media Specialists established if these teachers do not have regular classes scheduled and only schedule on-demand/teacher-requested basis for specific topics and projects?

Districts/BOCES will need to determine their specific rules around which courses must have SLOs when contact time varies following the State’s rules and the general principle of including the courses with the most students first and making practical judgments about how to consider different course meeting schedules like those in this example.

Classroom Teaching Service Teacher of Record

New Guidance - B12. When must school librarians be evaluated under Education Law § 3012-c?"Classroom teacher" is defined as a teacher in the classroom teaching service who is a teacher of record. Librarians who are certified as a library media specialist or school media specialist (library) are teachers in the classroom teaching service. For the 2012-2013 school year, teacher of record is defined as a teacher who is primarily and directly responsible for a student’s learning activities that are aligned to the performance measures of a course, consistent with guidance.Therefore, a certified librarian who is not a teacher of record is not a "classroom teacher“ and therefore would not need to be evaluated under Education Law § 3012-c. However, if a certified librarian is a teacher of record, he/she would be considered a "classroom teacher" and therefore must be evaluated under Education Law § 3012-c.

Is a Librarian a Classroom Teacher?

Certified Library Media Specialist or School Media Specialist

Teacher of Record

APPR and SLO’s

Teachers in Classroom Teaching Service

Not a Teacher of Record

Other evaluation processes

A teacher who is primarily and directly responsible for a student’s learning activities that are aligned to the performance measures of a course consistent with guidance

M2. What kinds of data will districts and BOCES need to collect in order to determine who is the teacher of record for evaluation purposes?Districts and BOCES will need to collect additional data elements to support teacher of record determinations. (Teacher-Student Data Linkage)

Speech Teacher – Addendum August APPR

B10. If a speech teacher (i.e., certified teacher of the speech and hearing handicapped or certified teacher of the speech and language disabilities) is just providing related services, is he/she required to be evaluated under Education Law §3012-c?

No. If a certified speech and language therapist is providing related services only, as defined in Education Law §4401 and section 200.1(qq) of the Commissioner's Regulations, he/she would not be considered a classroom teacher and would therefore not be required to be evaluated under Education Law §3012-c.

B11. If I am certified as a teacher of the speech and hearing handicapped and provide instructional services, am I required to be evaluated under Education Law §3012-c?

If a teacher who is certified as a teacher of the speech and hearing handicapped is assigned to provide instructional services and he/she is a teacher of record, then he/she is considered a classroom teacher and therefore must be evaluated under Education Law §3012-c.


SLOStudent Population: Students included

in the SLO There is no minimum number of students The population is set on BEDS day Even if the class enrollment changes it is not

abandoned. D27 District set date for adding students Addresses the growth of “each student” (A

ROSTER IS REQUIRED) but then it is the aggregate growth of all the students that determines whether or not the target is met.

Other student information to describe the population may be added (SLO Development Guide)P37, Section D. APPR Guidance April

2012


Population examples Health and Occupations Technology 11th Grade; 24 students,

including 2 with learning disabilities on attached roster 18 Kindergarten Students listed on the attached roster 23 8th grade Math AIS students from 3 sections of the course.

See roster All ___ students enrolled in 2 sections of AP Calculus. See

attached roster All ___ students in 10th Grade English. See attached rosters which

includes course sections, student names, and ID numbers 11th and 12th grade students enrolled in Fitness for Life, see

attached roster which includes students with disabilities

Population*

These are the students assigned to the course section(s) in this SLO - all students who are assigned to the course section(s) must be included in the SLO. (Full class rosters of all students must be provided for all included course sections.)

Population


SLOLearning Content: Content to be taught in the

SLO Identify the source for the standards: Common Core,

State, National, College Board, Industry, AASL Standards, etc.

Include any relevant Common Core Literacy/Math standards with NYS standards (Literacy in History/SS or Science & Technical subjects) – might include district decisions

Name the exact standards and performance indicators As appropriate, choose the “most important learning” or a

“subset of power standards.” What do you traditionally assess and does it give you a picture of the year?

Align to district or school goals and vertically align

SLO Development Checklist OCMBOCES *SLO Template

Learning Content*

What is being taught over the instructional period covered? Standards? Will this goal apply to all standards applicable to a course or just to specific priority standards?

Student Learning Objective: SLOWhere can I find my standards?http://engageny.org/resource/common-core-state-

standards/ for Common Corehttp://www.p12.nysed.gov/ciai/cores.html for Arts,

MST, Social Studies, Science, Health, PE, Family & Consumer Science, etc.

http://www.p12.nysed.gov/cte/cdlearn for Career Development and Occupational Studies

http://www.nbea.org/newsite/curriculum/standards/marketing.html for Business and Marketing

Where else?

Learning Content


Interval of Instructional Time: Timeframe within which the learning content is being taught One academic year, a semester? a quarter? Every other day? - Describe Start and end date

SLO Development Checklist OCMBOCES, SLO Guidance March 2012, APPR Guidance April 2012

Interval of Instructional Time


SLOEvidence: Assessments used to determine

students’ level of learning List the specific baseline assessments (pre-assessment) and

summative assessment(s) that will be used to provide baseline and summative data for the SLO. 1. If the course ends in a NYS assessment or a Regents exam, it

must be used as the summative assessment2. List of 3rd party assessments and Regents equivalents - FLACS3. District, Regional, or BOCES developed. Districts or BOCES

must verify comparability and rigor.

What could the other option have been? (School-or BOCES-wide, group, or team results based

on State assessments)

New York SLO Development Guide SLO Development Checklist OCMBOCES


What if the third option was chosen? District, Regional, or BOCES developed.

Align tightly to learning contentInclude components of good

assessment practicesScoring procedures – “vested interest”

BASELINESVESTED INTEREST

• G4: Does vested interest rule apply to pre-tests given to establish a baseline for a SLO?

• Answer: To the extent practicable, districts or BOCES should ensure that any assessments or measures, including those used for performance-based or performance task assessments that are used to establish a baseline for student growth are not disseminated to students before administration and that teachers and principals do not have a vested interest in the outcome of the assessments they score.

CIT, Lupinskie Center, One Merrick Ave. Westbury, NY 11590

BASELINESVESTED INTEREST

• G4: Does vested interest rule apply to pre-tests given to establish a baseline for a SLO?

• Answer continued: If it is impracticable to comply with this requirement for pre-tests, such as in certain cases when using a performance-based or performance task assessment, the district or BOCES must have adequate procedures in place to ensure that the security of such assessments is not compromised (i.e., as with all SLOs the principal and/or supervisor must ensure the rigor and fairness of the targets and set the goals based on the assessment that is used as the baseline and ensure that such goals are adequately met based on summative data).


Performance Tasks?

Performance TasksHow do SLO’s meet the requirement for

comparability? “Specify priority learning standards in a grade or subject around which assessments or performance tasks for students will be constructed by District Teams” (SLO Guidance March, Page 14)

Includes a “standards-based” rubric (SLO Guidance March, Page 30)


SLO

OR – other baseline options!

Algebra 2/Trig for PhysicsGrade 8 NYS English Assessment for 9th

grade EnglishSecond Grade Math Summative for Third

grade

BASELINES

D28: Question

• The SLO documents state that SLOs “must be specific and measurable, based on available prior student learning data" – what is meant by prior? APPR Guidance June 2012, p37-38

Response

• Teachers are encouraged to use a variety of student historical academic data, if available. This historical academic data might include prior course results, or it might only include the current baseline data from the course taken at the beginning of the interval of instruction.


BASELINES

D29: Question

• What should teachers use as a pre-assessment for establishing a baseline when students enter a sequence for the very first time and have little to no background information of a subject?

•APPR Guidance June 2012, p38

Response• . . .it will be important for

teachers to see what other courses they can draw on to provide other historical data. Any information about what entering students understand from other subject areas based on academic history in other courses, can give a teacher a wealth of information as to where the students may struggle or thrive.

• The teacher may want to also collect other baseline information (e.g., collect a writing sample) in order to more broadly assess the student's skills that may impact instruction for the year.


BASELINES

D47: Question

• Can SLOs for students with disabilities have a different target for growth?

APPR Guidance June 2012, p43

Response• The target for students in

any SLO may be differentiated because of the baseline (starting point of learning) and historical academic data.

• It is important to keep in mind that targets for all students, regardless of any special education classification, should be differentiated because of baseline data and not because of any special education classification.


SUMMATIVE – VESTED INTEREST

• This can be an issue.• Teachers may not score the summative

assessment.• Talk of scoring guidelines for Regents exams to

become more rigorous• Performance task issues• Questions regarding teachers’ involvement in the

creation of summative assessments


Evidence

Student Learning Objective: SLOBaseline: Level of students’ knowledge and skill

in the targeted learning content at the beginning of the interval of instructional time.

Describes how students performed on the pre-assessment or other previous summative assessment

Include any other data sources being considered – that may impact your “target”

Roster Actual baseline scores for each

student are required

Baselines

On the Roster


SLOTarget: Level of growth and skill that

students are expected to achieve at the end point of the interval of instructional time.

NumericRoster – growth goal and scores for each student Written in the form of a statement on the SLO

Different formats consider the individual students growth, might connect baseline to summative, might be a “ n percent of students will ____________.”

Aligned to school/district expectationsIncludes all populations: “including special

populations”

Important to Consider: The Number of Students

Look at Co-taught Living Environment SLO example

How many students are in this class?Describe what you see in the baseline

component. What is the target statement?What HEDI rating is aligned to the target?

Important to Consider: The Number of Students

90% of all students = 90% of 19, which is aligned to 14 points on the HEDI scale

Do the math. 18/19 would be 95% or 18 points

17/19 students would be 89% or 13 points16/19 students would be 84% or 8 HEDI

pointsWhat do you observe?

Differentiated Targets

Take a second look at the Co-taught Regents Living Environment SLO. Examine the Baseline and Target section again. (Regionally developed pre-assessment)

District decisions regarding Targets can be used while differentiating for clusters of student results. Raw score of 60-75 – achieve raw score of 80 or higher Raw score of 30-59 – achieve a raw score of 66 or higher

(mastery) Raw score of 29 or below – achieve a raw score of 41 or

higher (passing)

Target statement: 90% of all students will meet or exceed their goal.

Differentiated Targets

What is the likelihood that students will hit their targets when they are created in this way?

How is this more realistic? Raw score of 60-75 – achieve raw score of 80

or higher Raw score of 30-59 – achieve a raw score of

66 or higher (mastery) Raw score of 29 or below – achieve a raw

score of 41 or higher (passing) Target statement: 90% of all students will

meet or exceed their goal.

Differentiated Targets – Another Example

www.engageNY.orgwww.engageNY.org

Student Population, Baseline, and Target(s)Training SLO Target Approach 1: Set a common growth target.90% of students, including special populations, will grow by 60 percentage points or more on their summative assessment compared to their pre-test for the standards. (e.g., Student E’s target is 60 more than 30, or 90.)

50

Student Pre-Test Score Summative Target

Student A 10 70

Student B 20 80

Student C 5 65

Student D 0 60

Student E 30 90

Student F 10 70


Student Population, Baseline, and Target(s)Training SLO Target Approach 2: Set a growth to mastery target.

85% of students, including special populations, will grow to score 75% or higher on the summative assessment for the selected standards.

51


Student A 10 75

Student B 20 75

Student C 5 75

Student D 0 75

Student E 30 75

Student F 10 75


Student Population, Baseline, and Target(s)Training SLO Target Approach 3: Set differentiated growth targets by student.

85% of students, including special populations, will meet or exceed their individualized target.

52


Student A 10 80

Student B 20 80

Student C 5 75

Student D 0 70

Student E 30 85

Student F 10 80

Target

Student Learning Objective: SLOHEDI Criteria: Translates different levels of

student growth to the four categories, Highly Effective, Effective, Developing, Ineffective.

Allocates points clearly and objectivelyIt must be mathematically possible for a given

teacher to obtain any point value in the scaleDescription

Highly effective – exceeds district/BOCES expectations

Effective – meets district/BOCES expectations Developing – below district/BOCES expectations Ineffective – well-below district/BOCES expectations

Guidance on NYS Dist. Goal-Setting Process: SLO. March 2012

More than one SLO?

HEDI Criteria


SLO

Rationale – Include reasoning behind decisions. Particularly targets and HEDI alignment. That rationale could include Reference to the population and content Baseline data recorded on the roster Other data considered – historical, cohort National

Any information that will help the reader grasp the rigor of the SLO

Rate your SLO! – using the Annotated SLO Rubric Form

Rationale

Resources

APPR Guidance Document, June, 2012SLO Guidance Document, March, 2012From NY DOE engageny.org Introduction to SLO Assessment Options for SLO’s on engageny.orgNys-eval-plans-guidance-Purple memo.pdfIra Schwartz Update on Accountability DATAG July

2012http://www.ocmboces.org/teacherpage.cfm?

teacher=1518 SLO Connections to NYS Teaching Standards.pdfAnnotated SLO Rubric Form

Special Ed, ESL, and More

http://engageny.org/resource/student-learning-objectives-webinar-series-ii-fall-2012/

reviewing and evaluating student learning objectives curriculum, instruction, technology october 12,...

Documents

district rules

district expectations

district decisions

slo deck slide

districtwide guidance

specific slos

districtwide processes

district priorities