Student Learning Objectives (SLO)

Student Learning Objectives(SLO)Anthony ContiAnthony.conti@miu4.orgDr. Cathleen CubelicCathleen.cubelic@miu4.org

Our ObjectivesDefine an SLODesign, Build, and Review an SLOInterpret the SLO TemplateConsider Assessment Quality and PurposeView online toolsPlan for implementation

LEA ResponsibilitiesTo select a minimum group of 6-10 teachers, with aligned principals, to implement the SLO process (to receive training from the IU, using the PDE materials, to design, build and review a minimum of one SLO that is prepared using the online template, and follow the implementation process through the administrative review, monitor and final evaluation stages. THE RESULTS OF EVALUATION ARE NOT EXPECTEDTO BE USED AS A PART OF THE TEACHER EVALUATION.)

LEA Responsibilities2. To assign an implementation team to be trained by the IU and to subsequently train the teachers and principals involved in the pilot, or train the entire staff in preparation for next year. Implementation team members should include, at minimum, but not limited to:a. One administrator, preferably at the curriculum levelb. One principal, preferably one involved in the pilot processc. Two teachers, preferably two involved in the pilot process

LEA Responsibilities3.To allot one day for representatives from the implementation team to meet with the authors of the SLO process. (Substitute and travel expenses would be provided by PDE.)

4. To provide information, in the form of surveys and interviews, to a researcher who will evaluate the SLO process.

67

The Rating Tool [for teacher with Eligible PVAAS Data]The SLO in PA is written in relationship to a specific teacher and a specific class/course/content area for which that teacher provides instruction.

The PSSA test doesnt completely measure my effectiveness.

The SLO [Elective data] is in response to this statement.

10SLO DefinitionA process to document a measure of educator effectiveness based on student achievement of content standards.

The SLO process contains three (3) action components: Design (ing): thinking, conceptualizing, organizing, discussing, researchingBuild (ing): selecting, developing, sharing, completingReview (ing): refining, checking, updating, editing, testing, finalizing

12SLO Process ComponentsKey Points for TrainersExplain that all components are done before the school year (initial conversation with principal) in preparing the SLO; however, the REVIEW component may also continue until the final results are available to determine whether or not the performance expectations have been reached.Clarify the specific timelines for the SLO process will be determined by local education agencies (LEAs) and not by the state; however, a generic timeline for the SLO process should be presented that outlines a before, during, and after school year set of activities. In general:Teacher develops SLO, along with applicable performance measures before school startsPrincipal reviews and discusses with teacher; adjustments may be required.Teacher reviews SLO progress at some midpoint in the yearPrincipal receives mid-year update from teacher; adjustments may be required.Teacher summarizes performance measure data and evaluates each performance indicatorTeacher presents final SLO resultsPrincipal assigns final rating in Section 5

DESIGN: This component is the thinking step in the process used to conceptualize the learning objective in terms of content, students, and performance measures.BUILD: This component is the action step in the process that focuses on completing the SLO Process template and creating and/or selecting performance measures.REVIEW: This component is the reflection step used to examine the three Cs (i.e., Completeness, Comprehensiveness, and Cohesion) of quality.

IMT Orientation Draft 02Sept11-CS1213SLO Process Components DESIGNThinking about what content standards to measureOrganizing standards and measuresDiscussing with colleagues collective goalsResearching what is needed for a high quality SLO

Key Points for TrainersDesigning is planning for the SLO, examining what is needed, and how performance measures are used to collect information about student achievement.Activities during this stage establish the foundation for developing a student learning objective, including such things as:Identifying target content standardsDiscussing Big Idea in the standardsThinking about the goalCollaborating with other teachersBrainstorming the type of performance measures

IMT Orientation Draft 02Sept11-CS1314SLO Process Components BUILDSelecting the performance measure(s) Developing targets and expectationsCompleting the template Sharing the draft materials with other colleaguesDeveloping/Documenting performance task(s)

Key Points for TrainersEnsure the participants understand that the building is an iterative process between the original design and create the SLOs. Often the original designs must be changed once the details on how the standards will be measures and developed performance indicator targets. Activities during this stage complete the SLO Process Template 10.0 and include such things as:Selecting (or creating) the performance measures that are aligned to the targeted content standardsDeveloping mastery and/or growth metrics associated with the performance measuresEstablishing performance indicator targetsIdentifying students included in the SLO dataCreating performance expectations

IMT Orientation Draft 02Sept11-CS1415SLO Process ComponentsREVIEWChecking the drafted SLO (including the performance measures for quality Refining measures and targetsEditing text and preparing discussion points/highlights for principalFinalizing materialsUpdating completed SLOs with performance data

Key Points for TrainersEnsure the participants understand that the review phase requires an extensive evaluation of the SLOs quality in terms of the 3Cs.Completeness, Comprehensiveness, and CoherenceQuality assurance checklist and rubricActivities during this stage occur before and after the presentation to the principal, and include such things as:Finalizing and submitting the proposed SLORefining the SLO based upon feedback from the principalCollecting performance data on student achievementAdjusting SLOs at during the school yearUpdating SLOs with dataEvaluating each performance indicatorDetermining the Elective Rating

IMT Orientation Draft 02Sept11-CS15ActivityCount off by 6 at your table. Each one take an SLO should statement. [next slide]Produce an educator friendly version of that statement.Share within your group. The SLO Should.Represent student performance in a specific course/content area taught by educator.Reflect the diversity of students as learners.Align to a target set of selected academic content standards that represent the depth and breadth of the goal statement.Use metrics based upon multiple events/data collection periods to measure defined levels of student growth or mastery.Be supported by verifiable data that can be collected and scored in a standardized manner.Include a set of independent performance measures directly linked to the established goal.This is a global look at SLOs. Each One Teach One Activity.Groups of 6. Each take one. Produce a educator friendly version of each. Share your statement with the small and large group.17Design19What is a Goal Statement?Definition:Narrative articulating the big idea upon which the SLO is built under which content standards are directly aligned.

Characteristics:Encompasses the enduring understanding of the standardCentral to the content areaFoundational concepts for later subjects/coursesIMT Orientation Draft 02Sept11-CS1920Goal Statement ExampleStudents will apply the concepts and the competencies of nutrition, eating habits, and safe food preparation techniques to overall health and wellness throughout the life cycle at individual, family and societal levels.Does this convey an enduring understanding?Is there a central idea?Is there a foundation for later concepts?IMT Orientation Draft 02Sept11-CS2021 SLO Goal(Template #1)Goal Statement addresses:WHAT the big idea is in the standardsStandardsHOW the skills and knowledge support future learningRationale Statement:WHY the big idea is a central, enduring concept

http://pdesas.org/standard/PACore

Take the participants onto the SAS portal to the Curriculum Framework Select a content area to use as the model. As a district team, complete the template with a selected Goal Statement derived from the Big Ideas!21More Considerations for Goal Statements Do you have previous data to help guide your goal? What does your growth and achievement look like? Is there a building/district-wide goal?

Are there connections to SPP, PVAAS, Danielson areas of focus? Discussion22Activity:Goal Statement (Template #1)Within your team, choose a discipline in which youd like to focus. Preferably, choose a discipline that is taught by one amongst you.Complete Template #1 Goal StatementWe will post them for the entire group. BuildTemplateSection 1

GoalGoal statement should articulate an appropriate big idea. http://pdesas.org/standard/PACore

Standards should be the appropriate Focus Standards supporting the goal.

Rationale statement should be reasons why the Goal statement and the aligned Standards address important concepts for this class/course.

Focus on content shifts, PA Core Focus, Important Standards.26TemplateSection 2

Goal statement should articulate an appropriate big idea

Rationale statement should be reasons why the Goal Statement and the aligned Standards address important learning for this class/course.

27Performance IndicatorDefinition: a description of the expected level of student growth or achievement based on the performance measure Answers two questions.Does the indicator define student success?What is the specific measure linked to the indicator?Example:

28Examples of Performance Indicator TargetsStudents will achieve Advanced or Proficient on all four criteria of the Data Analysis Project rubric.Students will score an average of 3 or better on five different constructed response questions regarding linear modeling according to the general description of scoring guidelines.(http://static.pdesas.org/Content/Documents/Keystone%20Scoring%20Guidelines%20-%20Algebra%20I.pdf)Students will improve a minimum of 10% points from pre- to post-test for material in each semester. Students will show significant improvement in the Domain of Measurement on the Classroom Diagnostic Tools Mathematics Grade 7 assessment from the first to the last administration. Read each example and have participants decide whether or not the indicators are good. Use the criteria previously established. Discuss each as a group (red or green bucket) and give the reasons for the decision.29Performance Indicator Focus student group

A description of the expected level of achievement for each student in a subset of the SLO population (1F) based on the scoring tools used for each performance measure (4A).

Subset populations can be identified through prior student achievement data or through content-specific pretest data.

Examples of Performance Indicator Targets: Focused Student GroupStudents who scored below the 30th percentile on their benchmark AIMSweb R-CBM probe will score above the 30th percentile by the end of the school year using the national norms.Students who scored below a 2 on the pre-test will improve a minimum of one level on the post-test.

What qualifies this as a focused student group? Content-based pretest or prior achievement data? How will the issues of growth and/or achievement factor into the decision about a focused student group indicator?31Activity:Growth and MasteryWhat assessments may be used as growth, mastery or both?

Participants complete a Venn Diagram using names of assessments as well as an example Performance Indicator Target for ALL students.33What are the characteristics of a quality assessment?Write (3).Report out the summary from your table.

34Good assessments haveA specific and defined purposeA mixture of question typesItems/tasks with appropriate DOK levelsItems/tasks that are Standards AlignedA quality rubricA standardized scoring methodAcademic Rigor

A reasonable time limit for completionAn appropriate readability levelMultiple methods of student demonstrationValidity and reliabilityWell-written directions and administration guidelinesCut scores for performance categories

Academic RigorStandards-AlignedDevelopmentally AppropriateFocused on Higher-Order Thinking

Weighting, Linking, or OtherwiseStandardYou may consider each Performance Indicator equal in importance.LinkedYou may link multiple Performance Indicators, if you like. Do this for pass before moving on assessments.3. Weighted You may weight multiple Performance Indicators, if you like. Do this when you believe one or more PIs are more complex or more important than others.

37Standard ScenarioWeighting ScenarioPhysics Class with (3) PI targets:

Total Score = 72.5%TemplateSection 3

40Goal-Indicator-MeasureIndicator #1 might be relating to a growth/progress monitoring PI whereas Indicator #2 may be for a mastery PI41Goal-Indicator-Measure1-1 indicator-assessment ratio42Performance Measure - DescriptionsState the name of the assessment(s).List the type of measure.Explain the purpose, state what the Performance Measure should measure.Identify the timeline and occurrence(s)Scoring Tools should indicate the solution key, rubric, checklist, etc. that is being used to score the PM.Administration & Scoring Personnel should contain who is giving the test and who is scoring it.Performance Reporting should state how others will know which students met the Performance Indicator(s).

Whats the test? [generally] Why am I giving it? How will it be scored? When will it be administered? 43TemplateSection 4

There need not be 5.This is arbitrary. Suggestion is between 2 and 5 PIs44ExpectationsHave audience interpret this slide45Teacher ExpectationsDefinition: identifies each level (Failing, Needs Improvement, Proficient, Distinguished) students are meeting the Performance Indicator Targets.These reflect the continuum established by the teacher prior to the evaluation period.Each level is populated with a percentage range so that there is distribution of performance across levels.Based on the actual performance across all identified Performance Indicators, the evaluator will determine one of the four levels for the SLO. Template Section 5

ReviewSLO Coherency Rubric

School Leaders SLO Checklist

Assessment QA ChecklistTools for Review

Now you are an expert.Take 10-15 minutes to review this SLO example given for a Family and Consumer Science Class.Please make notes on the document where you question or disagree with what is stated and where you think needs clarification. Also, note what suggestions may be given to this instructor for improvement and questions you would ask prior to approving this SLO for the teacher.Please take the first 5 minutes to do this silently on your own. We will review each section and share with the group. Be ready to discuss your thoughts.

50The Online Toolhttp://www.pdesas.org/Use the Homeroom link at bottom rightClick the RIA Homeroom site link in the top paragraphRegister and log in.

Scavenger Hunt Find the online and Word templates we use to complete the SLO writing process.Find the Performance Task Framework Template.Find the handout that helps us define our goal and rationale.Find the Art Grade 8 SLO model SLO.Find the PDF Power Point that discusses Reviewing SLOsFind the Help Desk: Section 1 document

HomeworkYour group return with one SLO written having at least one Performance Indicator/Measure.Complete the template using the MS WORD template and email it to anthony.conti@miu4.orgWe plan to use these as exemplars at our December 12 meeting.Next TimeAssessment LiteracyVetting of Initial SLOsPlanningPERFORMANCE TASK FRAMEWORK

This template is used to organize performance tasks used in the SLO process.

Performance Measure

a.Performance Measure Name

SLO Alignment

b.Class/Course Titlec. Grade(s)/ Level

d.PA Standards

e.Performance Measure Purpose

1. Administration (Teacher)

1a.Administration Frequency

1b.Unique Task Adaptations/Accommodations

1c.Resources/Equipment

2. Process (Student)

2a.Task Scenarios

2b.Process Steps

2c.Requirements

2d.Products

3. Scoring (Teacher)

3a.Scoring Tools

3b.Scoring Guidelines

3c.Score/Performance Reporting

Performance Measure FrameworkGeneral Guidance: Think of the performance task as a single booklet consisting of administration guidelines on Page 1, guidelines to the students on Page 2, and detailed scoring criteria on Page 3, including sample responses aligned to the scoring rubric/tool.

Page 1. Administration (TEACHER)a. Administrative frequency: How many times will the student be given this task within an identified timeframe?b. Unique task adaptations and/or accommodations: How does the task change in either presentation, response options, setting, etc. to accommodate students with disabilities, English language learners, etc.?c. Resources and/or equipment: What equipment, tools, text, artwork, etc. is needed by the student to accomplish the task? What additional personnel are needed to administer the task?Page 2. Task Scenarios, Requirements, Process Steps, Products (STUDENT)a. Task scenario: What information is provided for the student that provides the context necessary to create a response, project, produce, demonstration?b. Requirements: Given the scenario, how are the task requirements articulated to the student in order to establish key criteria by which performance is evaluated? Which requirements are implied, thus requiring deeper understanding of the content being assessed? Which criteria are stated explicitly in order to adhere to the time constraints, product parameters, etc.?c. Process steps: What guidance expresses the sequence of events, steps, or phases of the task? How are extended (multiple days) timelines and demonstrations of progress articulated?d. Products: Given the activities within the task, what products, demonstrations, or performances are expected during and/or at the end of the process? What information is provided about the criteria used to judge student calculations, products, demonstrations, performances, etc.? Scoring (TEACHER)a. Scoring tools: How does the rubric classify different levels of performance, student work, etc.? How is the overall score attained? How well are multiple dimensions aligned to the standards?b. Scoring guidelines: How are the steps that are used to evaluate student products, performances, etc., articulated? What guidance is provided to assign scores for incomplete work? How are additional scoring personnel identified and trained? Given an overall score or classification/performance level, how are examples, models, or demonstrations provided?c. Score/Performance reporting: How are overall results reported back to the student? How are scored results reported for all students?

BuildingTemplate #5-Performance Task Framework

General DirectionsDesigned to examine the alignment characteristics of each Student Learning Objective (SLO), the Coherency Rubric, Version 1, serves as the measurement tool to ensure that each SLO meets the coherency criteria established by the Pennsylvania Department of Education. Consistent with the three-phase SLO design process, the rubric includes the following components: Design, Build, and Review. Each component contains a series of descriptors that specify the criteria for evaluating SLO quality. For this initial year of SLO development, descriptors are evaluated using a simple performance rating scale of Meets Criteria or Not Applicable.Part I: DesignJustify any N/A rating in the space provided at the bottom of the table.Task IDDescriptorRating

Meets CriteriaN/A

1.1The goal statement articulates the big idea (enduring understanding) under which targeted content standards are directly aligned. The statement is concise and free of technical jargon.

|_||_|

1.2Targeted content standards have a direct influence on student performance outcomes and are viewed as central to the subject area.

|_||_|

1.3 The course/subject area associated with the SLO is logically linked to the big idea and targeted content standards.

|_||_|

1.4A blueprint or other design document illustrates relationships among key components (i.e., goal statement, targeted content standards, performance indicators, performance measures, student achievement expectations, and overall teacher rating).

|_||_|

1.5Performance measure is designed to evaluate the targeted content standards (as demonstrated by the performance measures alignment characteristics).

|_||_|

N/A Justifications

Part II: BuildJustify any N/A rating in the space provided at the bottom of the table.Task IDDescriptorRating

Meets CriteriaNeeds Addressed

2.1The goal statement represents a central concept that is enduring, has leverage, and is foundational to further, more complex content standards.

|_||_|

2.2The SLO is supported by a representative sample of the educators students, with a sample size that is sufficient to make valid inferences about student achievement.

|_||_|

2.3Targeted content standards are selected using a valid and reliable approach that is fair and unbiased.

|_||_|

2.4Performance indicators are specific, criteria-focused, challenging (yet attainable), and directly linked to the performance measures.

|_||_|

2.5Performance measures establish benchmarks for two or more points in time within a given school year [Growth]. In addition or alternatively, performance measures establish a clear, date-specific target for an on-demand demonstration of skill and knowledge attainment [Mastery].

|_||_|

2.6The overall rating is directly linked to a performance continuum based on the percentage of students meeting expectations across all performance indicators.

|_||_|

N/A Justifications

Part III: ReviewJustify any N/A rating in the space provided at the bottom of the table.Task IDDescriptorRating

Meets CriteriaNeeds Addressed

3.1The SLO is based on performance measures that are technically sound (i.e., reliable, valid, and fair) and appropriately aligned to the targeted content standards.

|_||_|

3.2The SLO mitigates unintentional consequences and/or potential threats to inferences made about meeting performance expectations.

|_||_|

3.3 The SLO has supporting data and/or evidence to support the assignment of an overall teacher rating (i.e., Failing, Needs Improvement, Proficient, and Distinguished).

|_||_|

3.4The SLO has been reviewed to ensure that it is complete. Meaning, all applicable elements within SLO Template 10.0 have been addressed according to the prescribed business rules.

|_||_|

3.4The SLO has been reviewed to ensure that it is comprehensive. Meaning, all performance measures have been examined to determine that they are appropriate for use in the SLO process.

|_||_|

N/A Justifications

ReviewingSLO Coherency Rubric-PDE-Version4

General DirectionsDesigned to examine the alignment characteristics of each Student Learning Objective (SLO), the Coherency Rubric, Version 1, serves as the measurement tool to ensure that each SLO meets the coherency criteria established by the Pennsylvania Department of Education. Consistent with the three-phase SLO design process, the rubric includes the following components: Design, Build, and Review. Each component contains a series of descriptors that specify the criteria for evaluating SLO quality. For this initial year of SLO development, descriptors are evaluated using a simple performance rating scale of Meets Criteria or Not Applicable.Part I: DesignJustify any N/A rating in the space provided at the bottom of the table.Task IDDescriptorRating

Meets CriteriaN/A

1.1The goal statement articulates the big idea (enduring understanding) under which targeted content standards are directly aligned. The statement is concise and free of technical jargon.

|_||_|

1.2Targeted content standards have a direct influence on student performance outcomes and are viewed as central to the subject area.

|_||_|

1.3 The course/subject area associated with the SLO is logically linked to the big idea and targeted content standards.

|_||_|

1.4A blueprint or other design document illustrates relationships among key components (i.e., goal statement, targeted content standards, performance indicators, performance measures, student achievement expectations, and overall teacher rating).

|_||_|

1.5Performance measure is designed to evaluate the targeted content standards (as demonstrated by the performance measures alignment characteristics).

|_||_|

N/A Justifications

Part II: BuildJustify any N/A rating in the space provided at the bottom of the table.Task IDDescriptorRating

Meets CriteriaNeeds Addressed

2.1The goal statement represents a central concept that is enduring, has leverage, and is foundational to further, more complex content standards.

|_||_|

2.2The SLO is supported by a representative sample of the educators students, with a sample size that is sufficient to make valid inferences about student achievement.

|_||_|

2.3Targeted content standards are selected using a valid and reliable approach that is fair and unbiased.

|_||_|

2.4Performance indicators are specific, criteria-focused, challenging (yet attainable), and directly linked to the performance measures.

|_||_|

2.5Performance measures establish benchmarks for two or more points in time within a given school year [Growth]. In addition or alternatively, performance measures establish a clear, date-specific target for an on-demand demonstration of skill and knowledge attainment [Mastery].

|_||_|

2.6The overall rating is directly linked to a performance continuum based on the percentage of students meeting expectations across all performance indicators.

|_||_|

N/A Justifications

Part III: ReviewJustify any N/A rating in the space provided at the bottom of the table.Task IDDescriptorRating

Meets CriteriaNeeds Addressed

3.1The SLO is based on performance measures that are technically sound (i.e., reliable, valid, and fair) and appropriately aligned to the targeted content standards.

|_||_|

3.2The SLO mitigates unintentional consequences and/or potential threats to inferences made about meeting performance expectations.

|_||_|

3.3 The SLO has supporting data and/or evidence to support the assignment of an overall teacher rating (i.e., Failing, Needs Improvement, Proficient, and Distinguished).

|_||_|

3.4The SLO has been reviewed to ensure that it is complete. Meaning, all applicable elements within SLO Template 10.0 have been addressed according to the prescribed business rules.

|_||_|

3.4The SLO has been reviewed to ensure that it is comprehensive. Meaning, all performance measures have been examined to determine that they are appropriate for use in the SLO process.

|_||_|

N/A Justifications

ReviewingSLO Coherency Rubric-PDE-Version4