development report ensuring time for teaching and … · web viewassabet valley collaborative –...

26
District-Determined Measure for School-level Administrators Ensuring Time for Teaching and Learning Content Area and Grade Range: School Level Administrators, grades 6-12 DDM Summary: This DDM is a post-test only growth measure that assesses the extent to which administrators minimize time lost to scheduling errors, incomplete or inaccurate schedules, unnecessary directed study periods (study halls), or other less productive periods, and thereby maximize students’ and teachers’ time in learning. Developed by: Bennett Miller, Assistant Principal (Tahanto Regional Middle/High school, Berlin-Boylston Public Schools) in consultation with Assabet Valley Collaborative (Marlborough, MA), Risk-Eraser (Falmouth, MA), and the Learning Innovations Program, WestEd (Woburn, MA) Pilot Districts: Berlin-Boylston Public Schools Date updated: June 21, 2015 Table of Contents Introduction ....................................................... 2 Instrument ......................................................... 5 Administration Protocol ............................................ 5 Scoring Guide ...................................................... 8 Measuring Growth and Setting Parameters ........................... 10 Piloting .......................................................... 12 Assessment Blueprint .............................................. 13 Assabet Valley Collaborative – DDM – Time for Teaching and Learning 1

Upload: phungnhan

Post on 19-May-2018

214 views

Category:

Documents


0 download

TRANSCRIPT

District-Determined Measure for School-level Administrators

Ensuring Time for Teaching and Learning

Content Area and Grade Range: School Level Administrators, grades 6-12DDM Summary: This DDM is a post-test only growth measure that assesses the extent to which administrators minimize time lost to scheduling errors, incomplete or inaccurate schedules, unnecessary directed study periods (study halls), or other less productive periods, and thereby maximize students’ and teachers’ time in learning.Developed by: Bennett Miller, Assistant Principal (Tahanto Regional Middle/High school, Berlin-Boylston Public Schools) in consultation with Assabet Valley Collaborative (Marlborough, MA), Risk-Eraser (Falmouth, MA), and the Learning Innovations Program, WestEd (Woburn, MA)Pilot Districts: Berlin-Boylston Public SchoolsDate updated: June 21, 2015

Table of ContentsIntroduction ............................................................................................................................. 2Instrument ............................................................................................................................... 5Administration Protocol ......................................................................................................... 5Scoring Guide ......................................................................................................................... 8Measuring Growth and Setting Parameters ........................................................................ 10Piloting ................................................................................................................................... 12Assessment Blueprint .......................................................................................................... 13

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 1

IntroductionDescription of the MeasureThis DDM is a measure of the school administrator’s indirect impact on students’ learning. By school administrators, we mean any building leader who is responsible for ensuring an effective student scheduling system. Specifically, it is designed to measure growth in students’ available time for learning by decreasing lost learning time due to avoidable student schedule changes and time spent in less productive periods, such as directed study, office-aide, or teaching assistant-type periods instead of instructional periods.

The core measures involve two tasks: (1) documenting and tracking student schedule changes in the first four weeks following the start of school and (2) documenting and tracking student assignments to less productive “schedule fillers,” such as directed study, office aide, or teacher assistant. For the purposes of this DDM, it is a goal to reduce school-related schedule changes and to reduce the number of school-based student assignments. This is further explained in the Administration Protocol below.

RationaleThis DDM is designed as a post-test only growth measure, meaning that data are collected during one period in the year and compared to the prior year’s data. The approach is appropriate because student schedule and assignment changes occur most frequently at the start of the school year and are most reasonably compared across school years, rather than within the school year. For example, higher numbers of schedule changes are expected during the first month of school than during the spring when courses are already underway. A reduction in schedule changes from fall to spring would not reflect growth or improvement, but simply the natural flow of the school year. Therefore, this DDM is designed to compare the percent of student schedule and assignment changes during the first four weeks of school from year to year.

The overarching assumption guiding this DDM is that students who miss early sessions of a class (especially after any “drop/add” time period) miss critical time in learning in that class that may prove difficult to recover. Compounding this, when students are assigned to a class days or weeks after the start of school, the teacher must find a way to bring the student up to speed, or rearrange planned lessons for the entire class to accommodate the new student(s) and potentially hold up the learning for the entire class. By decreasing time lost to making schedule changes after the start of school, or by decreasing the number of students assigned to directed study or other less productive periods, more students will have opportunity to be engaged in productive learning from the first day of school. Further, teacher time in learning will be enhanced due to not having to backtrack for students re-assigned to classes days or weeks into their courses and thereby disrupting the pacing and instructional flow of the course.

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 2

ContextThe design team developed this DDM from the perspective of smaller schools or districts, and modifications may be needed to apply the DDM in larger school or district contexts. For example, the tools in this DDM may need to be modified to reflect a broader range of reasons for student schedule changes or to make analysis of results more feasible or automated when used with particularly large numbers of students.

This DDM is designed as an indirect growth measure rather than a target measure because improvements are being pursued in the designer’s district. Once acceptable levels of student schedule changes and schedule assignments are achieved, the district will revise the estimated parameters to reflect a target measure where the goal will be to remain within a target range of schedule changes and schedule assignments over time. (See Additional Considerations in the Setting Growth Parameters section.)

Description of the Development ProcessThis DDM was developed during October 2014 – June 2015 under a DDM Leadership Grant (FC-217) awarded to the Assabet Valley Collaborative by the Massachusetts Department of Elementary and Secondary Education (ESE). In partnership with Teachers21, Risk Eraser, and with primary support from the Learning Innovations Program at WestEd (Woburn, MA), the Collaborative convened six building administrators, principals, and assistant principals, representing schools spanning grades pre-K-12 from five participating districts. Participants worked in smaller teams of one to three to strengthen and apply their assessment literacy toward the development of several direct and indirect measures of student growth.

Participants grew their expertise over six sessions by engaging in a guided DDM development process framed by a series of questions, including: (1) Why are we doing this work together? (2) What is most important to measure? (3) How shall we measure what’s most important? (4) How can we strengthen and refine our measure? (5) How can we prepare our measure for broader use? (6) How will we measure growth? Throughout, participants engaged in large group discussion and critique, as well as team collaboration and problem solving.

This measure has not yet been piloted. Districts in and beyond the Assabet Valley Collaborative may decide if they would like to pilot and/or modify the attached tools and processes for use as a DDM in their district. Because this is a newly developed measure, it is important that districts engage administrators in examining results from the first year of implementation and identifying, over time, any revisions or refinements that may further strengthen the quality of the assessment, scoring tools, administration protocol, and/or growth parameters to suit the circumstances and realities of each district’s local context.

DDM ObjectiveThis DDM addressed two objectives. The first is to eliminate or minimize preventable schedule changes occurring in the first weeks of a school year. The second is to eliminate or minimize less productive student assignments such as study hall, office aide, and teaching assistant beyond those

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 3

required by IEP, 504, or IST plans, or to accommodate students’ special needs or requests. These less productive student assignments are referred to as “schedule fillers” elsewhere in this DDM.

As shown in the table above, the result of the first objective is weighted to account for 70% of the final assessment score, and the result of the second objectives is weighted to account for 30% of the final score. This weighting suits the particular school that served as the model for this DDM design. This school experiences numerous class changes after the start of each school year, so a key focus for the school’s administrator is to reduce these interruptions to students’ and teachers’ learning time. The number of students assigned to study hall, office aide, and teaching assistant classes, or “schedule fillers” – excluding necessary assignments required by an IEP, 504, and IST plans or to accommodate students’ special needs or requests – is weighted less in this DDM due to this small school’s assignment limitations with a smaller staff and limited course offerings. While this school may realize reduced numbers of students assigned to “schedule fillers,” it is not anticipated that assignments to such classes can be eliminated entirely. Circumstances at other schools may be different and different weighting may be more appropriate.

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 4

Content (Job Responsibility) Weight

Objective-1: Eliminate/minimize preventable schedule changes occurring in the opening weeks of a school year.

Leadership Standard II.C 1- School-Level Administrator Rubric, Time for Teaching and Learning. Creates a master schedule and related systems that set aside instructional time and eliminates unnecessary interruptions to instruction.

70% of the measure

Objective 2- Eliminate or minimize less productive student assignments, such as study hall, office aide, and teaching assistant beyond those required by IEP/504, or to accommodate special needs or requests.

Leadership Standard II.C 1- School-Level Administrator Rubric, Time for Teaching and Learning. Creates a master schedule and related systems that set aside instructional time and eliminates unnecessary interruptions to instruction.

30% of the measure

100%

InstrumentThere are two instruments used to collect data for this DDM. The first is a Schedule Change Request Form and the second, a Study Hall Tracker Form. Both are completed by students when requesting schedule or assignment change; they and are attached to this DDM. Results following four weeks of data collection at the start of the year are categorized and analyzed by the administrator to assess whether the percentage of possible course and assignment changes has been reduced as compared to prior year(s).

The Schedule Change Request Form asks students to complete the following: (1) provide their name, (2) identify their grade, (3) indicate whether the course they want to change is a core (e.g., academic subject such as English/language arts, science, mathematics, foreign language) or non-core course (e.g., physical education, visual or performing arts), (4) indicate the name(s) of the course(s) they want to add or drop, and (5) select or record the reason for the schedule change that best fits their request. This form is used to gather information from students about the specific changes being requested and their reasons for these requests.

The Study Hall Tracker Form asks students to do the following: (1) provide their name, (2) indicate their grade, and (3) select or record the reason for their directed study assignment. This form is also used to survey students assigned to office aide or teaching assistant-type assignments.

Students requesting schedule or assignment changes are asked to complete these forms throughout the year (including the week prior to the start of school); only data collected during the first four weeks of school, however, are analyzed for this DDM. Data collected beyond these first four weeks may inform administrators as they analyze and work to improve the master schedule.

Administration ProtocolThe Administration Protocol addresses how the measure is intended to be implemented to best support a common conversation about student growth across classrooms.

When is the measure administered? Data pertaining to both outcome measures – student schedule changes and student assignments – are collected during the first four weeks of school each year. Data are then categorized and compared to the prior year’s performance to determine growth trends. Thus, the baseline for judging performance is the prior year’s data and trends over time, which are established across these annual data points.

How is the measure administered? The assessment involves collecting two kinds of student data that will serve as annual “outcome measures.” These annual data points will be compared to the previous year’s data to gauge whether growth has occurred.

When students request schedule or assignment changes, the responsible administrator directs them to the relevant Google Form: the Schedule Change Request Form or the Study Hall Tracker

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 5

Form (attached as part of this DDM). Because all students requesting changes or assignments must complete these forms, gaining an acceptable response rate is never in question. Administrators can ask students to complete the form on a nearby computer with Internet access, provide them with the relevant URL link so they can complete the form on their own computers or mobile devices, or provide a paper copy of the form to complete and submit. Administrators are encouraged to make this a predictable routine with which students become familiar throughout their middle and high school years. The administrator or guidance counselor then works with students to make any necessary changes, in consultation with students and teachers, and according to established procedures in the school.

Administrators then analyze the data generated through these student forms. Administrators can easily download Google Form data as an Excel spreadsheet (sample attached as part of this DDM). Alternatively, they may compile data from students’ paper copies of the form.

Student Schedule Change RequestsFor the first outcome measure, administrators use the Schedule Change Request Form to track and categorize all student schedule changes occurring in the first four weeks after the start of the school year. This is when student change requests are likely to be most frequent and most disruptive to student and teacher learning time. Schedule changes are categorized as either school-related or student-related, reflecting the source of the need for change. School-related schedule changes include school-based course conflicts, gaps, overbooking, assignment of inappropriate courses, or other school errors. These are shown as the first three selections on the Schedule Change Request Form and are most directly within the realm of the school to correct. Student-related schedule changes are based in student priorities that have changed since student schedule requests were first submitted. These are reflective of student (and family) preferences and choices that may be beyond the school’s direct control.

Study Hall Tracking Similarly, administrators use the Study Hall Tracker Form to track and categorize all student assignments to study halls based on the reason behind the assignment. School-based assignments may be based on a lack of available, suitable alternative courses; some student assignments to study hall, office aide, or teaching assistant therefore serve as “schedule fillers.” This is the first selection on the attached form. All other choices are considered student-based assignment to such classes and may be based on student requirements (Individualized Education Plans, 504 Plans or other student plans) and specific student requests or preferences. As with the Student Schedule Change Requests, administrators use student responses on the form to evaluate whether the assignments were either school-based or student-based.

A date/time stamp is noted on all Google Form entries, which allows tracking of changes that are requested both before and after the start of the school year. Administrators are also encouraged to examine student-requested schedule and assignment changes that occur before the start of the school year, because these data may inform their scheduling decisions and efforts to improve the master schedule, thereby reducing changes over time.

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 6

How are deviations to protocols addressed? Administrators may find that student response options provided on the example Google Forms in this DDM do not represent the full range of reasons that their students may provide for wanting to change their schedules or assignments. It is important that all administrators using this DDM in a given district conduct pre-assessments and/or interviews with students and guidance counselors to determine the range of reasons students request these changes and to incorporate these into the Google Form or other data collection forms used across their schools. If the response options do not reflect students’ actual reasons for requesting these changes, this may lead students to select inaccurate options and provide the administrator with faulty or misleading data. In the example form provided for schedule changes, the first four choices represent “school-based” items and the remainder are “student-based” items.

It is suggested that the end-users of this DDM review and adapt, as needed, the example forms provided in this DDM for their own local contexts. This approach provides a customized list of student reasons that clearly fall into these two categories of “school-based” and “student-based” reasons for reference by all using this DDM across the district.

How can administrators contribute to growth? Administrators may contribute to growth in each of the outcome measures in multiple ways. For example, to impact the percent of school-related schedule change requests, administrators might consider the following factors:

Timeline - Develop a timeline and clear steps for student scheduling and broadcast to all stakeholders, such as office, guidance, faculty, students, and parents.

Program of Studies – Ensure a complete, accurate, clear, and published Program of Studies per the timeline to support a timely scheduling process.

Teacher Recommendations – Ensure complete and accurate teacher recommendations for courses per the timeline to support the student course request schedule.

Graduation Planning/Path – Support students in the identification of post-graduation goals and guide the course request process in relation to students’ goals.

Student Course Requests – Encourage students to submit their course requests per the timeline to support the Master Schedule; encourage complete and accurate course requests that reflect student needs and wants, with alternative selections identified.

Master Schedule – Ensure that the Master Schedule meets all student required core course graduation requirements.

Master Schedule Optimized – Develop the Master Schedule to meet most student requests for non-required core courses and electives.

Schedules Issued – Ensure student schedules are issued per the timeline with time in current year allotted to request changes for upcoming year.

Schedule Changes Requested – Provide students time to review schedules and request needed schedule changes during the current school year or summer for upcoming school year.

Schedule Changes Made – Ensure students and school resolve schedule change requests before the start of the upcoming school year.

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 7

For example, in the development of this DDM the school pursued a robust blend of these strategies. They invested in training in developing master schedules, published and adhered to a scheduling timeline, integrated results of a graduation planning tool and graduation progress monitoring tool in the Student Information System that students may reference when completing schedule requests, issued schedules well before the end of the year, and held a “Step Up Day” where all incoming high school students went through their actual next year schedules to meet the teachers, review class syllabi, get summer reading and other work, and review expectations with their new teachers. In this way, students had an opportunity to “try before they buy” and work out schedule issues before the end of the current school year to be ready to start the next school year with satisfactory schedules. It is believed these efforts will help the school show growth toward achieving minimal changes to student schedules for any reason after the start of the school year.

Similarly, administrators may contribute to reduced school-related assignment change requests by minimizing assignment to study halls or less productive periods. Careful design of the Master Schedule to distribute suitable core and elective options across the day will ensure some periods do not have under-booked courses, while other periods do not have sufficient choices to avoid the use of study hall-type assignments as “schedule fillers.”

● Study Halls Minimized – Goal: reduce study halls utilized as schedule fillers. ● Non-Productive Classes Minimized – Goal: reduce office aides, teaching assistants, and

other similar schedule assignments utilized as schedule fillers.

Scoring GuideData are collected during the first four weeks after school has started. Administrators analyze these data, coding student schedule changes and student assignments to directed study, office aide, and teaching assistant classes as either school-related or student-related. Those categorized as school-related are the specific target of this DDM, as it is the administrator’s responsibility to reduce school-related disruptions to learning, and those numbers are used in the following scoring calculations.

Step 1. Calculate the percent occurrence for each of the two outcome measures. For both outcome measures – school-related student schedule changes and school-related student assignment to “schedule filler” type classes – the ideal or desired percent of occurrence is zero. The aim is to systematically reduce unnecessary school-related student schedule changes after the start of the school year and avoidable student assignments to classes serving as “schedule fillers.” The closer the calculated percent of occurrence for each measure is to zero, the closer the administrator is to accomplishing this aim.

To calculate the percent of school-related student schedule changes, divide the total number of school-related student schedule changes (or student assignments, for the second outcome) by the total number of students in the school as of end of the fourth week of school. This will provide the percent of school-related schedule (or assignment) changes that occurred during the first four weeks in relation to the total possible number that could have occurred, meaning at least one for every student in the school.

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 8

A hypothetical example of this scoring guidance for the first output measure is as follows: The school had 173 school-related schedule changes during the first four weeks of school in 2014-15 when the school population was 559 students.

173/559 = 30.95%

This indicates that, of all the possible school-related schedule changes that could have occurred in 2014-15 (at least one per student), 30.95% actually occurred.

A hypothetical example of this scoring guide for the second output measure is as follows: The school had 189 students assigned to directed study-type classes as “schedule fillers” in 2014-15 with a school population of 559 students.

189/559 = 33.81%

This indicates that, of all the possible school-related schedule filler assignments that could have occurred in 2014-15 (assuming one per student), 33.81% actually occurred.

Step 2: Weight the results of the two outcome measures.Once the two outcome measures are calculated, their relative weight must be factored in to arrive at a total weighted score for the DDM in the current year. As described in the Assessment Blueprint, the first outcome measure – the percent of school-related student schedule changes - is worth 70% of the final assessment score. The second outcome measure – the percent of school-related student “schedule filler” assignments – is worth 30% of the final score. School districts that adopt this DDM may choose to weight these two parts differently to suit local conditions and priorities.

Using the example above, where the percent of school-related student schedule changes was 30.95% and the percent of school-related student assignment changes was 33.81%, each must be multiplied by their relative weight, then added, to arrive at the final assessment score for 2014-15, as shown below:

Year # St

#School –related Student Course

Changes

% School-related Student Course

Changes

#School-related Student

Assignmts

%School-related Student

Assignmts

Calculation to Account for Weighting

Overall Weighted

%(rounded

)

2014-15 559 173 30.95 189 33.81

(173/559 x .70) + (189/559 x .30)

21.66% + 10.14%32%

2015-16 565 158 27.96 160 28.32

(158/565 x .70) + (160/565 x .30)

19.58% + 8.5%28%

2016-17 550 140 25.45 150 27.27

(140/550 x .70) + (150/550 x .30)

17.82% + 8.18%26%

In this example, the Total Weighted Score indicates the overall weighted percent of school-related schedule and “schedule filler” assignments that occurred during the first four weeks of school in 2014-15.

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 9

Administrators will likely want to track trends across year, so it is recommended that they create a spreadsheet similar to the one shown above. If formulas are entered to automatically calculate the percentages and overall weighted scores, this will allow a view of changes over time in each outcome measure, as well as changes in the overall weighted percent across multiple years (going down each row in the spreadsheet). Administrators can then monitor progress (or lack thereof) in each of the two outcome areas and adjust school management strategies accordingly.

Measuring Growth and Settings ParametersThis DDM is designed as a post-test only growth measure because student schedule and assignment changes occur most frequently at the start of the school year and are reasonably comparable across school years, rather than within the school year. The prior-year performance serves as the pre-test or benchmark score for gauging growth.

To understand growth trends from year to year, calculate the difference in Overall Weighted % (the final column in the chart above) from one year to the next. For example, if the assessment score in 2014-15 was 32%, as shown above (representing the overall weighted occurrence of school-related student course and “schedule filler” assignments), then in 2015-16, after systematically attempting to improve the Master Schedule outcome, it was 28%, growth would be represented by calculating the difference, which in this case would be a decrease of 4 percentage points. Following the same approach and using hypothetical data provided in the chart above, growth between 2015-16 and 2016-17 would be a decrease of 2 percentage points. Across the two years, there was a steady decrease in the percent of both school-related student course changes (from 30.95% to 27.96% to 25.45%) and in the percent of school-related student assignments (from 33.81% to 28.32% to 27.27%). These figures can inform the administrator’s management strategies. The administrator’s evaluator can also see evidence that the administrator has, overall, reduced the disruptions to student and teacher learning time due to course and assignment changes by six percentage points across two years. Determination of growth as low, moderate, or high occurs after benchmark data are collected during the first years of using this DDM.

If there is considerable variation in the percentage of student schedule and “schedule filler” assignments across multiple years, or across multiple schools, it may be more appropriate to compute a difference from the average change rate over time or across a large sample. (See ESE’s Technical Guide B for further explanation.)

Additional ConsiderationsAlthough this DDM is designed as a growth measure for middle and high schools that have significant room for improvement in this area, a district may choose to use it as a target measure instead. This may be more suitable when the middle and high schools in the district have successfully established and want to maintain lower and more manageable numbers of schedule changes after the start of school and low numbers of students assigned “schedule fillers” as a past practice. In this case, maintaining a specific range of performance for each outcome measure may be the target.

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 10

In either case, using this DDM as a growth or target measure, this post-test only reflects change across different samples of students from year to year. Administrators and their supervisors must continually scan for potential threats to the validity of this measure. ESE’s Technical Guide B, for example, suggests that notable shifts in a town’s demographics, perhaps severe school budget cuts, or changes in staff to student ratios, might prevent fair comparison between two different years of results on this DDM. Administrators must identify these potential factors and introduce them as important considerations during final review and discussion with their supervisors. For example, increasing student population without an increase in staffing would likely skew the outcomes in this DDM in an unfavorable light as the student/teacher ratio increases. Conversely, an increase in staffing will likely lead to more student needs being met and better numbers when analyzed using this DDM. Where significant shifts in either student population or staffing numbers occur, districts may choose to “normalize” the outcomes by using a multiplier reflecting student-to-available-course-section ratios.

As an example of “normalizing” the figures from the above chart for swings in student numbers –versus numbers of teaching sections available for scheduling – a ratio of student population divided by the total number of teaching sections will yield a number that can be compared to the ratio of another year and used to modify the outcome for comparison. Using the 2015-16 student population above (565), divided by the nominal teaching sections available (determined by numbers of staff actual teaching period from the master schedule) of say, 214, we get a ratio of 2.64. If, in the following year, our student population drops to 550, as shown above, yet we add two more teachers for 10 more sections to the master schedule, our ratio changes to 2.45. The prior-year ratio divided by the current-year ratio is 1.07. Using this multiplier for the current school year from the above chart, or 26% x 1.07, yields a normalized total of 27.82%. This reflects a more accurate picture of actual improvement considering the changing numbers of students and teaching sections, driven by staffing capacities.

A Note about the Development of Growth ParametersIt is important with post-test only growth measures to gather additional information to help predict the expected level of achievement. During the pilot phase in 2015-16, the developers of this DDM will analyze prior school data on student course changes (fall 2014 vs fall 2015) and use these results to inform estimated growth parameters. Basic data is recoverable from the Student Information System (SIS), but will require thoughtful analysis. The SIS reports simply show dates that courses were dropped and added, with no additional information as to why – information that will be available when the DDM forms (attached) are used in the coming year.

The developers did not have prior years data on student “schedule filler” assignments in their school. Therefore, they plan to administer an alternative Directed Study Survey (see Appendix) to all students assigned study halls, office aide, and teacher assistant classes during the pilot phase of the DDM development. As this is a small school, a 100% survey return rate from these students is feasible. A survey of a sample of these students may be more appropriate for larger schools.

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 11

Estimated Growth Parameters will be developed in 2015-16 based on the above preliminary analyses. Districts should revise these estimates based on careful review of data collected during the first years of administration in their own contexts until historical trends can be established and used to inform and refine these initial estimated parameters.

PilotingDevelopment ProcessDuring initial development, this assessment for administrators received multiple peer and expert reviews. Staff at WestEd (Woburn, MA) and Risk-Eraser critiqued the assessment task and associated tools and an ESE review team also provided critical feedback, which led to further revisions. For example, over multiple meetings the developer received direct input and feedback that guided the focus of this DDM and contributed to its usability. The developer used Google Docs and email to facilitate these exchanges and refinements throughout the development of this DDM.

Preliminary Pilot Subsequently, this measure was refined in response to data collection and analysis during the spring of 2015. During this time, the developers conducted three activities:

1. Ten students at different grade levels who have previously requested course changes and received student assignments were asked to use the tool and provide feedback on the clarity and comprehensiveness of response options.

2. Two administrators were asked to review the DDM document and work with hypothetical data from the tools to provide feedback regarding their clarity, feasibility, and usefulness.

3. The developer analyzed prior course schedule data to determine how common it is for individual students to request multiple course changes. As a single course change request can alter an entire student schedule, for the purposes of this DDM, each student requesting schedule changes is counted as one occurrence, regardless of the actual number of changes made as a result of a given student request.

Preliminary Pilot The students reviewed the forms used to track schedule change requests and the process by which they can be accessed. Due to the small school in which the forms were piloted, paper versions were completed and the corresponding digital version was reviewed. Students sampled indicated that both were user-friendly and self-explanatory.

The administrator review in the guidance office and main office resulted in a re-arrangement of the data forms used by the students when selecting the primary reason they requested a change. The form now lists the reasons for requesting a change that allows for easier sorting of “school-related” versus “student-related” issues. The first three choices are school-related, and the remainder are student-related.

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 12

The review of actual changes indicated that, with very few exceptions, a single change request for a given student results in a domino effect as the student’s schedule is shuffled to accommodate the request. Therefore, a decision was made to track the number of students making a change request versus the actual number of schedule changes necessitated by the request.

Assessment BlueprintThe assessment blueprint is an elaboration of the content table included in the introduction. It was developed before the actual instrument and was revised during development to accurately reflect the final assessment. It serves two purposes: (1) it is a roadmap for the assessment development team to ensure balanced coverage of the most important content and (2) it is a key for other potential users of the assessment by concisely indicating what content the assessment is designed to measure, whether the goal is growth or target attainment, and the difficulty of the items associated with each piece of content. (See pages 12 and 29 of Technical Guide A for more information.)

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 13

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 14

100 points

Step 3: Confirm point al lotments.

Growth TargetEasy

ItemsModerate

ItemsHard Items

70 1 1 1 70

30 1 1 1 30

Totals 100 points 2 items 0 items 0 items 1 items 1 items 2items 100

Leadership Standard II.C 1- School-Level Administrator Rubric, Time for Teaching and Learning. Creates a master schedule and related systems that set aside instructional time and el iminates unnecessary interruptions to instruction. Objective 2- Eliminate or minimize less productive student assignments, such as study hall , office aide, and teaching assistant beyond those required by IEP/504, or to accommodate special needs or requests. [Study Hall Tracker Form] 30%

Leadership Standard II.C 1- School-Level Administrator Rubric, Time for Teaching and Learning. Creates a master schedule and related systems that set aside instructional time and el iminates unnecessary interruptions to instruction.

# of items in content

area

average points per

item

Step 5:. Confirm number of items and average

points per item

edit password: EDIT

Assessment Blueprint - Indirect Measure

Step 1: Enter the total # of points you want to include in the assessment.

Step 2: Enter the relative weight you want to assign to covered content.

Step 4: Describe the types of items you want to develop in terms of cognitive complexity and item difficulty. The corresponding number of items and points per item will be determined.

Content (Job Responsibility)

Weight (% of Overall

Measure)

# of al lotted points

Objective Item Difficulty

Objective-1: Eliminate/minimize preventable schedule changes occurring in the opening weeks of a school year or semester shift (or term or trimester as appropriate). [Course Change Request Form] 70%

100%

Schedule Change Request Form (Google Form)

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 15

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 16

Study Hall Tracker Form (Google Doc) [Insert a period in the fourth bullet from the bottom.]

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 17

Sample Spreadsheet of Responses from Schedule Change Request Form

Sample Spreadsheet of Responses from Study Hall Tracker Form

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 18

DIRECTED STUDY SURVEY

We are looking to improve the scheduling process here at [insert school/district] to ensure that we are serving the needs of the students to the very best of our abilities. With that in mind, we need to get a handle on how we are using our “directed study” periods. Please complete this short survey on why you are in a directed study period so we can improve our schedules for next year.

Name: _________________________________________ Grade: ______*We would like your name in case we have any questions, but it is not required.

What reason below BEST describes your reason for being in a Directed Study? (circle one)

No other classes were available this period that I needed or wanted.

I needed and requested a directed study period to get organized, do homework, etc.

I have a directed study specified in my ED Plan or 504 Plan.

Other reason (please specify below)

Thanks for your help,

[name and role]

Assabet Valley Collaborative – DDM – Time for Teaching and Learning 19