development report social-emotional skill · web viewthe school psychologist should conduct...

District-Determined Measure Example

Social-Emotional Skill Development in the Therapeutic Setting

Content Area and Grade Range: School Psychologists, grades 6-12DDM Summary: This DDM measures growth in grade 6-12 students’ ability to identify their current emotions and corresponding physical symptoms, and to identify and apply a healthy coping strategy to improve their immediate emotional state in the therapeutic setting.Developed by: Sarah Auger, School Psychologist (Stoneham Public Schools, MA) and Carly Greenstein, School Psychologist (North Reading Public Schools, MA)Reviewed by: Names and roles of the individuals who reviewed the measurePilot Districts: North Reading Public Schools, Stoneham Public SchoolsDate last updated: June 2015

Table of ContentsIntroduction 2Instrument 4Administration Protocol 4Scoring Guide 11Measuring Growth and Setting Parameters 13Piloting 13Assessment Blueprint 16

SEEM Collaborative – DDM – School Psychologists 6-12 1

IntroductionDescription of the Measure This DDM is a measure of the school psychologist’s direct impact on students’ growth in their ability to identify their current emotions and corresponding physical symptoms and to identify and apply a healthy coping strategy to improve their immediate emotional state in the therapeutic setting. It is a core function of the school psychologist to support the development of students’ social/emotional skills in these areas, particularly because students who struggle with these skills often experience challenges in school and are subsequently referred to the school psychologist for intervention support. This DDM may also be used by other personnel who instruct and support these students, such as school guidance or adjustment counselors.

This measure is designed for school psychologists who work regularly and intensively with a small caseload of students who receive frequent, ongoing social/emotional intervention and support. When assessing the progress of a small sample of students, it is important to collect sufficient evidence to make valid inferences about these students’ social/emotional growth over time. As a result, this DDM includes multiple measures that are administered throughout the school year. The first is administered to students as a pre- and post-assessment at the start and end of the school year, while the other two solicit students’ journal reflections and the psychologist’s student observations and are administered on a monthly basis. These measures assess only a portion of the content taught by the school psychologist; however, they were chosen as the focus for this DDM because they require students to identify, describe, and analyze emotions, experiences, and coping strategies to improve their immediate emotional state within the therapeutic setting – key skills at the center of the school psychologist’s instruction and support.

Potential Modifications School psychologists with larger caseloads may opt to use this DDM with a randomly selected small sample of three to five students. Alternatively, if they measure growth with their full caseload of students, they may be able to draw valid inferences with fewer data points so may decide to use only two of the three measures described here or, alternatively, to collect data quarterly or every two months instead of monthly. If School Psychologists are unable to administer all three measures, he/she may choose to use the journal prompts as a singular measure to assess students’ growth. This would impact scoring and overall growth parameters.

DDM Objective This measure is aligned to the following Core Course Objective (CCO): Students will identify their current emotions and corresponding physical symptoms, then identify and apply a healthy coping strategy to improve their immediate emotional state in the therapeutic setting.


Content (Standard) Weight

CCO: Students will identify current emotions.Mental Health Standard 5.1- By the end of grade 5, students identify the various feelings that most people experience and describe the physical and emotional reactions of the body to intense positive and negative feelings

Mental Health Standard 5.7- By the end of grade 8, students will identify and describe the experience of different feelings (such as elation, joy, grief, and rage) and how feelings affect daily functioning (MA Comprehensive Health Curriculum Framework)

25% of the measure

CCO: Student will link current feeling to corresponding physical symptoms. Mental Health Standard 5.1- By the end of grade 5, students identify the various feelings that most people experience and describe the physical and emotional reactions of the body to intense positive and negative feelings

Mental Health Standard 5.7- By the end of grade 8, students will identify and describe the experience of different feelings (such as elation, joy, grief, and rage) and how feelings affect daily functioning (MA Comprehensive Health Curriculum Framework)

25% of the measure

CCO: Student will identify healthy coping strategies.Mental Health Standard 5.11- By the end of grade 12, students will analyze healthy ways to express emotions and to cope with feelings, including the common causes of stress, its effects on the body, and managing stress (MA Comprehensive Health Curriculum Framework)

25% of the measure

CCO: Student will implement healthy coping strategies to improve their immediate emotional state.

Mental Health Standard 5.2: By the end of grade 5, students apply methods to accommodate a variety of feelings in a constructive manner in order to promote well being.

25% of the measure

100%1

1 A CCO is a statement that describes core, essential, or high priority content (knowledge, skills, or abilities) identified by those who designed the assessment and drawn, synthesized, or composed from a larger set of curriculum or professional standards. This Core Course Objective aligns with several standards in the Massachusetts Comprehensive Health Curriculum Framework, as shown in the Content Chart, above. These standards are designated for mastery by the end of grade 5, 8, or 12; all are relevant throughout the middle and high school grades for those students who work directly with the school psychologist because these students are typically identified for services due to challenges with these core skills.


InstrumentThis DDM consists of three measures that provide information about students’ understanding and application of the specified social/emotional skills. The first is a Student Survey, consisting of seven open-response questions to which the student responds in writing at the start and end of the school year. This serves as a pre- and post-measure and is intended to highlight growth over time. The second measure is a set of four Journal Prompts, provided to the student in writing, as well as read aloud, and to which the student then responds in writing on a monthly basis. The ongoing journal prompts are based in a belief that writing is a form of reflection and provides time for students to make sense and draw connections that oral assessments may not. The third measure is a Student Interview Checklist, which is completed by the school psychologist monthly based on the student’s oral response to a set of interview prompts. This combination of assessments allows a robust view of growth over time by bringing together several measures that include the observations of both student and school psychologist.

Note that all three measures in this DDM contain items that address each content area described in the Content Chart above. For example, items on the Student Survey address each of the four content areas listed above; items in the student Journal Prompts address each content area above; and items on the school psychologist Interview checklist address each content area above. Point values for the various items have been weighted, from five to nine points each, to reflect the proportions designated in the chart.

Administration ProtocolThe Administration Protocol addresses how the measure is intended to be implemented to best support a common conversation about student growth across classrooms.

Which students should be included? This measure is designed for school psychologists who work intensively with a small caseload of three to five students who receive frequent, ongoing social/emotional crisis intervention and support. If the psychologist has four or five students, all should be included in this DDM. If the psychologist has more students on his or her caseload who receive such ongoing, intensive social/emotional support, students should be randomly selected by taking every other name from an alphabetical listing of last names of students being serviced, up to a maximum of five students. For example, if eight students are being serviced regularly by the school psychologist for social/emotional skill intervention and support (listed here alphabetically by last name) – John Abel, Julie Chase, David Clay, Fred Jones, Tai Moore, Carolina Naroni, Sonia Ramirez, and Tony Romano - the psychologist would include every other student in this DDM – John Abel,


David Clay, and Tai Moore, and Sonia Ramirez - or four in total. This process ensures fair and unbiased selection of a limited number of students whose growth can be tracked using multiple measures during the school year.

When is the measure administered? Students complete the first measure, the Student Survey, during the first week of service with the school psychologist (e.g., September) and once during the last week in May or first week in June. The second measure, Journal Prompts, is administered to the student at monthly intervals throughout the school year, beginning during the second session with the school psychologist. The school psychologist administers the third and final measure, the Student Interview, each month throughout the school year, beginning during the third session with the psychologist. Each of the measures should be administered during different counseling sessions so that the student’s engagement in one activity does not overly influence the ways he or she engages in the others.

How is the measure administered?

Student Survey This is a paper-pencil task for students. The student reads and answers each question on the survey. The school psychologist may answer the student’s questions to clarify the understanding of the assessment task; however, may not give examples of appropriate answers. The school psychologist provides up to 20 minutes of quiet time for the student to complete the task. If the student does not initiate the task, the school psychologist may ask if the student needs clarification on the task or assesses whether special accommodations are needed (see below).

In order to increase consistency in the administration of this assessment across school sites, school psychologists should orally explain the following points to students:

1. This assessment will be given two times per year - at the beginning of the year, before we have worked together in counseling sessions, and again toward the end of the year after we have worked together.

2. The pre-survey portion of the assessment is designed to gather information on what you already know. If you do not know the answers to some of the questions-don’t worry. We will be learning about these topics throughout our work together.

3. Your responses will help guide us on what we should work on throughout our time together.

4. I will give you the survey. Your task is to read the survey and answer the questions. If you have any question, you may ask me. I can clarify the task; however, I am not able to provide you with specific examples on how to answer questions.

5. You will have 20 minutes to work on the survey. If you do not finish in 20 minutes, don’t worry, I can provide more time if needed, but we need to finish today and cannot continue the work into another session.


6. After you hand in the survey, if I have questions about anything that you wrote, I will ask you to clarify during the next time we meet.

7. This is not a test like you typically take in school. You will not be receiving a grade on this task. Just try your best. I will keep your survey results and compare them to your performance on the same assessment at the end of the year. This will help me see how much you have grown throughout our sessions. It will also help me to work better with students in the future.

8. I appreciate your honest answers.

Directions to Students, Post-Survey (last week of May/1st week of June):1. I will use this post assessment to help me see what you have learned throughout

our work together. The post-assessment is exactly the same as the pre-assessment at the start of the year.

2. The skills, topics, conversations, and practice that we have completed throughout our counseling sessions should help you feel prepared to answer these questions.

3. Your responses will help me to provide effective counseling to students in the future.

4. If you have any questions about the survey, you may ask me. I can clarify the task; however, I am not able to provide you with specific examples on how to answer questions.

5. You will have 20 minutes to work on the survey. If you do not finish in 20 minutes, don’t worry, I can provide more time if needed, but we need to finish today and cannot continue the work into another session.

6. After you hand in the survey, if I have any questions about what you wrote, I will ask you to clarify the next time we meet.

7. Remember, this is not a test like you typically take in school. You will not be receiving a grade on this task. Just try your best. I will keep the results and compare them to your performance on the same assessment that you took in September. This will help me see how much you have grown throughout our sessions. This will also help me to work better with students in the future.

Journal Prompts The school psychologist reads the four journal prompts aloud to the student and answers student-generated questions to clarify understanding of the assessment task; however, may not give examples of appropriate answers. The prompt is also provided to the student in a written format to refer to while writing. The school psychologist provides 10-12 minutes to answer the journal prompts, so this is intended as a quick write. If the student does not initiate the task, the school psychologist may ask if the student needs clarification on the task or assesses whether special accommodations are needed (see below).

In order to increase consistency in the administration of this assessment across school SEEM Collaborative – DDM – School Psychologists 6-12 6

sites, school psychologists should orally explain the following points to students:

1. This assessment will be given every month during our counseling sessions. 2. Your responses will help me know what we should work on throughout our time

together. 3. Your responses will also help me monitor your growth throughout counseling. 4. I will read aloud the four journal prompts. I will then give you the prompts in

writing so you can review them on your own. Your task is to respond to each prompt in writing because writing is a form of reflection and allows you some time to think and connect ideas. If you have a question about the journal prompts, you may ask me. I will be able to clarify the task, however, I will not be able to provide you with specific examples on how to answer the prompt.

5. You will have 10-12 minutes to complete your writing. I can provide more time if needed, but you will need to finish the writing today and cannot continue the work into another session.

6. If I have questions about what you wrote, I will ask you to clarify the next time we meet.

7. You will not be receiving a grade on this writing. Just try your best. I will compare your writing to the journal writing you do each month. This will help me see how you are growing throughout our sessions.

Student Interview Checklist The school psychologist completes the interview checklist monthly during the student counseling session. For this activity, the psychologist must have available an age-appropriate set of emotion cards to provide visual scaffolding for this language-based activity, such as the following (a variety of emotion card resources are available online):


and an “Intensity of Emotion” scale on a strip of cardstock that includes visual cues, such as the following:

1 2 3 4 5

CALM MILDLY INTENSE VERY OVERWHELMED INTENSE INTENSE

It is strongly recommended that the visual cues on this Intensity Scale are described to students in terms of intensity of emotion and are clearly distinguished from the emotions themselves, such as anger, jealousy, sadness. To do this, school psychologists may want to discuss and use both the visual emotion cards and the “Intensity of Emotion” scale as a regular part of counseling sessions to engage students in recognizing and naming their emotions, then describing the intensity of their emotional and physical responses.

It is assumed that the psychologist using this DDM will often be intervening with students based on a social/emotional need, so the Interview Checklist is designed as a guided, reflective series of prompts to work through that crisis. The psychologist asks the following series of six questions, providing up to 10 seconds of wait time for each response.

1. What is your current emotion or feeling?

2. [Using the “Intensity of Emotion” scale] On a scale from 1 to 5, with 1 being completely calm and 5 being very intense and overwhelmed, what is the intensity of the emotion you are feeling right now?

3. How is your emotion or feeling affecting your body?

4. Name a healthy strategy for coping with this emotion.

5. Review the steps and try to use this coping strategy right now.

6. Using our scale from 1 to 5, what is the intensity of the emotion or feeling now? SEEM Collaborative – DDM – School Psychologists 6-12 8

(If needed, what could help to further reduce the intensity of the emotion?)

The psychologist only provides additional prompts and support if the student explicitly seeks clarification or support, or if the student is initially unable to respond after the provided wait time. Adhering to the progression of possible student performances, listed below and on the Interview Checklist, the psychologist provides one brief prompt or clarification, if needed and as provided on the Checklist, then provides an additional 10 seconds of wait time for the student’s response. For example, if the student is unable to name his or her current emotion, the psychologist may prompt, “Take another look through the emotion cards. Is there one that shows how you are feeling today?” If after this prompt and an additional 10 seconds of wait time the student is still unable to respond, the psychologist may provide up to two additional prompts, but no more. The psychologist is not allowed to provide suggested responses, such as, “You look very angry today. Are you feeling angry?” Responses must always come from the student; the psychologist’s task is to observe and learn from the student’s ability to respond.

In some cases, it may be possible that one or more of the particular questions is not relevant to a given conversation with the student. For example, it may not be possible or relevant for a student to try a particular identified copying strategy in the therapeutic setting, in which case the psychologist would make note of this on the Student Interview Checklist form by circling n/a (not applicable). In the vast majority of cases, however, it is anticipated that it will be relevant for the student to respond to all six questions in the Interview.

The psychologist records each of the student’s responses in the space provided on the Student Interview Checklist, then circles the associated rating / point value that reflects the demonstrated level of response:

o Yes, Demonstrated Independentlyo Yes, Demonstrated with 1 Prompto Yes, Demonstrated with Multiple Promptso No, Did Not Demonstrate Despite Promptso No Opportunity to Demonstrate, Not Relevant

(Associated point values are differentiated according to the weighting shown in the Assessment Blueprint.) The psychologist then adds the circled point values and records in the final row of the chart as a total out of a possible total of 44 points.

How will accommodations be documented? If the psychologist has observed or received information from teachers or student evaluations that the writing demands of these assessments may not allow students to convey their full understandings of the designated skills, the psychologist may administer the Student Survey and Journal Prompts orally and scribe the student’s oral responses. The psychologist may not provide any prompts, cues, or answers to the student in the process, in accordance with the Administration Protocol above.


In particular, the school psychologist must provide accommodations specified in students’ Individualized Education or 504 Plans. In addition, for Stage 1 ELL students, all instructions and assessment prompts must be translated – in writing and/or orally - to the student’s native language to ensure that these students have opportunity to understand the assessment tasks and demonstrate their understandings. Further, these students must have the option to record their written responses for the Student Survey and Journal Prompts in their native language and to speak responses to the Student Interview in their native language if desired. It is strongly recommended that the psychologist work closely with the school or district EL coordinator to provide these accommodations and to translate and scribe these students’ responses back into English for interpretation during scoring.

All accommodations provided, and the reasons for these accommodations, must be documented on the provided scoring sheets for each of the three measures in this DDM.

How are deviations to protocols addressed? Students are randomly selected for this DDM at the start of the school year; however, if fewer than three students suit the criteria for selection (e.g., work regularly and intensively with the school psychologist on social/emotional skill development) at the start of the school year, additional students may be added as the year progresses. The pre-assessment Student Survey should be administered within two weeks of beginning services with the school psychologist and monthly Journal Prompts and Student Interview Checklist observations should begin during the two weeks following. If any events disrupt the monthly administration of the Journal Prompts or Student Interview Checklist (e.g., student or teacher absence, special school events), the school psychologist should administer the measure at the next scheduled session with the student with the aim of gathering progress data at approximately one month intervals.

Scoring GuideThe school psychologist should conduct and score all student work and performances for this DDM because this person has the relevant professional knowledge to interpret students’ demonstration of social/emotional skills evaluated in these assessments. However, because this DDM is based on multiple one-to-one measures that occur within the therapeutic setting, it is essential that school psychologists calibrate their scoring with same-role colleagues in their school or in other schools in the district (e.g., school psychologists or school guidance counselors who work directly and intensively with students on social/emotional skill development).

Calibration strengthens the likelihood that observations and scoring will be conducted in a consistent and reliable manner over time and across sites. A sample calibration


protocol can be found at the Rhode Island Department of Education website: http://www.ride.ri.gov/Portals/0/Uploads/Documents/Teachers-and-Administrators-Excellent-Educators/Educator-Evaluation/Online-Modules/Calibration_Protocol_for_Scoring_Student_Work.pdf

In particular, to calibrate scoring with this DDM, the school psychologist can bring student Journal Prompts and pre- and post-Student Survey responses to a collaborative session to calibrate scoring. The professionals may also aim to identify anchor papers for each of these measures that represent different levels of performance to guide and support common scoring over time. To calibrate scoring of the Student Interview Checklist, the school psychologist will need to gather video footage of several student interviews to use for a calibration exercise with colleagues.

How Should Scorers Prepare for Scoring?To score each part, the school psychologist should make copies of the provided scoring guides and set up meetings with colleagues to calibrate scoring. This is important to make sure multiple school psychologists are able to score in the same manner, and thereby contribute to fair and reliable scoring across a district.

How Should Assessment Scores Be Calculated?Student Survey The school psychologist consults the Student Survey Scoring Guide (Appendix D). Using the scoring guidance and examples of possible acceptable responses, the psychologist records the student’s name and assessment date at the top of the form, evaluates each of the seven survey items, and circles the corresponding point values indicating whether each student response is Fully Correct, Partially Correct, or Incorrect. Finally, the school psychologist adds the seven scores and records the total in the bottom right cell on the form.

Journal PromptsThe school psychologist consults the Journal Prompts Scoring Guide (Appendix E). Using the scoring guidance and examples of possible acceptable responses, the psychologist records the student’s name and assessment date at the top of the form, evaluates each of the responses to the four journal prompts (and sub-components of each prompt), and circles the corresponding point values indicating whether each student response is Fully Correct, Partially Correct, or Incorrect. Finally, the school psychologist adds the scores and records the total in the bottom right cell on the form.

The School Psychologist transfers total scores from Journal Prompt, Interview Checklist, and Student Survey to the Score Tracking Sheet, then compiles and calculates score from each month using the following steps:

1. Record the student name at the top of the Score Tracking Sheet.2. Locate the appropriate month at the top of the form and record the total score for


http://www.ride.ri.gov/Portals/0/Uploads/Documents/Teachers-and-Administrators-Excellent-Educators/Educator-Evaluation/Online-Modules/Calibration_Protocol_for_Scoring_Student_Work.pdf



each assessment that was completed. The Student Survey should only be completed in September and May. The Interview Checklist and Journal Prompt should be completed once each month September through May. These monthly scores allow the psychologist to track progress over time and to adjust instructional strategies if progress is not evident, increasing the likelihood that gains will be achieved over time.

3. For each month sub-total the points for the Interview Checklist and Journal Prompt out of a total possible 96 points. For September and May, record the total points for the Student Survey, Interview Checklist, and Journal Prompt for a pre-test and post-test score out of a total possible 140 points.

How Should Gain or Growth Scores be Calculated?Although the school psychologist will want to track growth for the Journal Prompts and Interview Checklist from month to month during the year, final growth, or gain, scores are formally calculated by totaling scores from all three measures at the start of services (pre-test) and at the end of services in May (post-test).

After all three measures are completed in September, the scorer adds the results shown in the first column on the Scoring/Tracking Sheet and records the total points out of 140 possible points for September. This process is repeated in May.

To calculate gains, the scorer subtracts the total score from September from the total score from May. For example, if a student scored 50/140 points in September and 105/140 points in May, subtract 50 from 105. The gain score would be 55 points for this particular student.

Measuring Growth and Settings ParametersThe following growth parameters were set with the understanding that the school psychologist often works with several students every year who need crisis intervention support related to self-management and application of social/emotional skills. Although students’ personal situations will likely be varied, the ways the psychologist supports their skill development may be somewhat similar, as reflected in the consistency of items across the three measures in this DDM.

For this reason, growth parameters were set in terms of an overall pre- and post-test gain score, calculated by adding results from all three measures in this DDM at the start of the school psychologist’s services (pre-test score) and the end of services (post-test score). An example is provided in Appendix F, which provides a Score Tracking Sheet.

We anticipate that students receiving services in grades 6-12 will have some basic social/emotional skills, but that they have stalled in their development and, thus, need more intensive support and intervention. Therefore, we expect the upper range of skills may be more challenging for this particular group of students to achieve. For example,


we anticipate that students will start the year scoring approximately 40-60 points and that they will gain their first 20 or 30 points on this DDM more easily than their final 30 points. Therefore, the low growth parameters are broader than the high growth parameters.

Low Growth Moderate Growth High GrowthGain Score 0 – 70 pts 71 – 110 pts 111 – 140 pts

PilotingThese estimated growth parameters will be tested and refined during the pilot phase. Given our short pilot period, we will pilot all three measures with several different grades of students and gauge the span of skills between one grade and the next. Even though these scores will not reflect individual growth, they will provide some measure of average differences at different grade levels, providing an initial rough sense of one year’s potential growth. For example, by gathering end-of-year scores from a sample of 7th graders and comparing their distribution and to a sample of 8th graders’ distributions and mean score, we hope to gain an initial sense of what might be reasonable to expect for one year’s growth on these measures. We will also take into consideration the school psychologists’ observations and judgments of which students they believe made low, moderate, and high growth during the year.

Our pilot plan is for the following educators, representing educators in and beyond the DDM team at a range of schools, districts [and grades 6-11], to pilot this DDM between the end of January 2015 and the end of May 2015:

Name Role School District Grade Level

DDM Team Primary Contact

1 Carly Greenstein

School Psychologist

North Reading Middle School

North Reading

6-8 Carly Greenstein

2 Sylvia Weisz School Psychologist

North Reading Middle School

North Reading

6-8 Carly Greenstein

3 Sarah Auger School Psychologist

Stoneham High School

Stoneham 9-12 Sarah Auger

4 Colleen Goode

Behavior Specialist

Stoneham Public Schools

Stoneham k-12 Sarah Auger


In particular, we aim to learn about the following questions from our pilot:

1. During counseling sessions, are students able to identify their feelings?2. During counseling sessions, are students able to link how their feelings impact their

body?3. During counseling sessions, are students able to identify healthy coping strategies?4. During counseling sessions, are students able to apply coping strategies to improve their

current mood?

Development ProcessThis measure for School Psychologists was refined through both small trials and a longer term pilot from January – June, 2015. During initial development, Carly Greenstein, Sylvia Weisz, Sarah Auger, Colleen Goode conducted small tryouts of Journal Prompts, Interview Checklists, and Student Surveys in North Reading Public Schools and Stoneham Public Schools. These trials led to various refinements of measure’s items and descriptors, as well as to the scoring tools and administration protocol. Due to the feasibility and repetitive nature of the multiple tools, it was determined to solely use the Journal Prompt during the longer pilot period.

In addition, the assessment received multiple peer and expert reviews during initial development. For example, staff at WestEd (Woburn, MA) critiqued the assessment task and associated scoring materials, and an ESE review team also provided critical feedback, which led to further revisions. For example, we changed the scoring of the instruments and the growth parameters.

Finally, a longer term pilot of the assessment in the developers’ own schools and districts will be conducted during the 2015-2016 school year to contribute to the development of increasing specific, feasible, and informative tools for potential users.

Pilot Results

Initial feedback was positive. Those administering the pilot thought that the measures would be feasible throughout counseling sessions. However, throughout the pilot phase the feedback obtained noted that the measures were repetitive in nature. The student journal was recommended as the most beneficial tool. Data patterns and trends over time will be collected throughout the longer pilot phase during the 2015-2016 school year.

Example Data TableApril May Gain

Journal Prompt 0/52 18/52Interview Checklist 9/44 10/44Subtotal 9/96 28/96Student Survey 30/44 37/44Total 39/140 65/140 26


FeedbackStudent Survey

Seemed to be better for younger students. It took approximately 10 mins. to administer.

At times, awkward to administer to high school students. Students felt it was juvenile.

Weird to do so close together but would make sense beginning and end of year

Some questions too complex for 6th graders

Journal Prompt

Responses seemed to be based on students' current emotional state. Is this an accurate assessment of their ability to complete the task, or, an assessment of how they feel in the moment?

Seemed to be tailored for younger participants.

Hard for some students

Student didn't know what the words implementing a coping strategies meant

Student Interview Checklist

Format seemed to be too formal for counseling sessions.

Did not seem to be helpful. Were not applicable to counseling sessions.

Was not applicable. Sessions scheduled at certain time and student usually in a good mood.

Didn't make sense most of the time unless student comes in crisis

In light of the pilot results, we decided to eliminate the interview checklist and student survey. The team determined that the journal prompts allowed students to reflect and report on their current emotion, how their emotion impacted their body, as well as possible coping strategies to improve their overall emotional state. This measure is useful throughout 6-12th grade, is age appropriate, and can be conducted in counseling sessions throughout a school year. This will increase the school psychologist's opportunity to monitor student growth in a feasible and measurable manner. The next steps include solely using the journal prompts as a single measure to assess students’ growth within counseling sessions. Therefore, scoring parameters will need to be changed in order to accurately measure this DDM. However, we do not feel that we have enough information to recalibrate scoring at this time. A longer term pilot is recommended in order to collect data, increase sample size, and improve overall scoring.

If the interview checklist and student survey were to be used, the recommendation of the pilot group would be to amend these items to allow for a wide-range of student emotions at the time of the administration of the survey as this would lead to more practical uses of the tool.

Assessment BlueprintThe assessment blueprint is an elaboration of the content table included in the introduction. It serves two purposes: (1) it is a roadmap for the assessment development team to ensure balanced coverage of the most important content and (2) it is a key for other potential users of the assessment by concisely indicating what content the assessment is designed to measure and the level of rigor with which the content is covered (see pages 12 and 29 of Technical Guide A for more information).


http://www.doe.mass.edu/edeval/ddm/TechnicalGuide.pdf#page=33

http://www.doe.mass.edu/edeval/ddm/TechnicalGuide.pdf#page=15

The DDM developers opted to approximate the total weight of each content area, with 35 items representing approximately 25% of the DDM score in each of the first two content areas and 36 items representing approximately 25% of the DDM score in each of the remaining two content areas.


Appendix A

Student Survey

Name: _____________________________ Date: ______________________

1. Susie was not invited to her classmate’s birthday party over the weekend. a. What emotion is she feeling? b. Explain why you chose this emotion.

2. Bob got 100% on a test that he studied really hard for. a. How is he feeling? b. Explain why you chose this emotion.

3. Consider how you are feeling right now.a. What emotion are you feeling?b. Explain why you are feeling this emotion.

4. How is your body reacting to the emotion that you are feeling right now?

5. Johnny is feeling really nervous about a class presentation. a. How might his body be reacting?

6. Judy is very angry because her parents will not let her go to the movies with her friends.a. How might her body be reacting?

7. Consider the healthy coping strategies you use. a. What healthy coping strategy do you often use to help yourself feel better?b. How does your body react when you use this strategy?

(Continue writing on back)


Appendix B

Student Journal Prompts

1. Think of a situation in the past two weeks when you experienced an unpleasant emotion.

a. Describe the situation.b. Describe the emotion you felt.c. Describe how your feeling affected your body.

2. Think back to the coping strategy that you used to improve this situation.a. Name the coping strategy and explain the steps that you used.

3. Explain why you chose to use this particular coping strategy.

4. Describe how the coping strategy helped.OR If the coping strategy did not help, describe what other healthy coping strategy you ci use if this situation happened again.


Appendix CStudent Interview Checklist (for Teacher Use)

Adult Prompt Acceptable Response YesDemonstrate

d Independentl

y

YesDemonstrate

d with 1 Prompt

YesDemonstrate

d with Multiple Prompts

NoDid Not

DemonstrateDespite Prompts

No Opportunity

to DemonstrateNot Relevant

1. What is your current emotion or feeling?Notes:

Student labels current state with a relevant emotion word (e.g., sad, angry, frustrated)If needed, first prompt: Take another look through the emotion cards. Is there one that shows how you are feeling today?

5 4 2 0 n/a

2. On a scale from 1 to 5, with 1 being completely calm and 5 being very intense and overwhelmed, what is the intensity of the emotion you are feeling right now?Notes:

Student identifies intensity of feeling on a provided scale from 1 to 5. If needed, first prompt: Remember, the first picture shows a calm face because the emotion is very mild at this end of the scale. Each picture shows a little more intensity to the emotion. At this end, you would be having a very hard time controlling your emotions because they would feel super intense and overwhelming. Where on the scale would you say you are feeling right now?

5 4 2 1 n/a

3. How is your emotion or feeling affecting your body?Notes:

Student links current feeling to one or more physical symptoms that typically occurs in concert with the emotion identified.

7 5 3 0 n/a

SEEM Collaborative – DDM – School Psychologists 19

If needed, first prompt: Think about how the inside of your body feels right now - your muscles, your heart beat or pulse, your hands and legs, your head, neck and shoulders. Where in your body do you feel your emotions right now? Can you describe how your emotions are making your body feel?

4. Name a healthy strategy for coping with this emotion.Notes:

Student identifies a coping strategy that would normally reduce the intensity of the identified emotion and support productive interactions.If needed, first prompt:What strategies can you recall that we have worked on together this year? Would any of these be helpful for coping with the emotion you are feeling right now?

9 6 3 0 n/a

5. Review the steps and try to use this coping strategy right now.Notes:

Student tells the steps and applies the coping strategy in the moment.If needed, first prompt: What is the very first step you need to take to use this strategy? What do you need to do after that? Keep going, take me through the steps. Then try the steps.

9 6 3 0 n/a

6. Using our scale from 1 to 5, what is the intensity of the emotion or feeling now? (If the

Student indicates that the intensity of the emotion has decreased and, if needed, describes steps

9 6 3 0 n/a


emotion is not calmed down yet, ask, “What could help to further reduce the intensity of the emotion?”)Notes:

that could further reduce the intensity of the felt emotion.If needed, first prompt: Remember, the first picture shows a calm face because the emotion is very mild at this end of the scale. Each picture shows a little more intensity to the emotion. At this end, you would be having a very hard time controlling your emotions because they would feel super intense and overwhelming. Where on the scale would you say you are feeling right now?

TOTAL SCORE / 44

Special Accommodations Provided: Rationale for Accommodations Provided:


Appendix D

Student Survey Scoring Guide (For Teacher Use)

Fully Correct Responses: - Answers all parts of the question and - Provides relevant and plausible responses for all partsPartially Correct Responses: - Answers some parts of the question or - Answers all parts, but not always relevant or plausible responsesNot Correct Responses: - Answers are incomplete and/or - Answers are not relevant or plausible

Student Name: Date:# Question Guidance Examples of Correct

ResponsesFully

CorrectPartially Correct

Not Correct

1 Susie was not invited to her classmate’s birthday party over the weekend. - What emotion is she feeling? - Explain why you chose this emotion.

Student identifies how he/she might feel and explains why this feeling might be likely after being left out of event by a peer

Sad, mad, lonely

Susie would probably feel angry because it is not kind to leave friends out of a special event.

5 2 0

2 Bob got 100% on a test that he studied really hard for. - How is he feeling? - Explain why you chose this emotion.

Student identifies a positive feeling related to accomplishment

Proud, excited, happy

Bob would feel proud because he probably studied really hard to get every question right.

5 2 0

3 Consider how you are feeling right now.- What emotion are you feeling?- Explain why you are feeling this emotion.

Student names an actual feeling or emotion, rather than a description, an example, a wish, or other response

Angry, frustrated, sad5 2 0

4 How is your body reacting to the emotion that you are feeling right now?

Student describes feeling in the body that connects with the listed emotion

Angry: Trembling, pit in stomachSad: Lack of energyTense: I have a headache

7 3 0

5 Johnny is feeling really nervous about a class presentation. - How might his body be reacting?

Student describes a feeling related to an undesired activity

Sweating, racing heart, stomach ache 7 3 0

6 Judy is very angry bc her parents will not let her go to the movies with her friends.- How might her body be reacting?

Student describes physical symptoms related to anger

Tense, hot, pit in stomach, verbal outburst, shaking 7 3 0

7 Consider healthy coping strategies you use. - What healthy coping strategy do you often use to help yourself feel better?- How does your body react when you use this strategy?

Student identifies a healthy, productive coping strategy and describes how his/her body reacts in response

Go for walk: calms my moodBreathe deeply: relax musclesCount to 100: distracts me 8 4 0

TOTAL SCORE /44


Special Accommodations Provided: Rationale for Accommodations ProvidedAppendix E

Journal Prompts Scoring Guide (For Teacher Use)

Fully Correct Responses: - Answers all parts of the question and - Provides relevant and plausible responses for all partsPartially Correct Responses: - Answers some parts of the question or - Answers all parts, but not always relevant or plausible responsesNot Correct Responses: - Answers are incomplete and/or - Answers are not relevant or plausible

Student Name: Date:

Item # Question Guidance

Examples of CorrectResponses

Acceptable?Fully

CorrectPartially Correct

NotCorrect

1

Prompt: Think of a time in the past two weeks when you experienced an unpleasant feeling or emotion.

.

a. Describe the situation. Student describes an unpleasant situation and what happened.

Student identified an unpleasant experience within the academic, social, and/or personal realm.

5 2 0

b. Describe the emotion you felt. Student correctly identifies an emotion using an emotion word rather than a description of what the emotion was “like” or what the student wanted to do in response to the emotion.

Sad, Anxious, Angry, Jealous, Depressed, Furious, Confused

5 2 0

c. Describe how your feeling affected your body.

Student links feelings with physical state, suggesting connection between emotion and muscles, sweat or tear glands, tensions (headache, neck or shoulder cramps, jittery hands or legs), heart rate, pulse, or energy level, etc.

Lack of energy (sad)Sweaty palms, racing heart (anxious, nervous)Tense, hot, pit in stomach (angry, mad)

6 3 0

2Prompt: Think back to the coping strategy that you used to improve this situation

a. Name a coping strategy and the steps that you used.

Student correctly names a taught coping strategy and explains a correct

Deep breathing.


and complete sequence of steps to implement it.

Be aware of my breathing.2. Slow down my breathing.3. Breathe more deeply and longer.4. Feel my heart beat.

9 4 0

3Prompt: Explain why you chose to use this particular coping strategy.

Student explains at least one plausible reason for choosing a specific strategy.

I felt my heart begin to race so I tried to slow my breathing down and take deeper breaths.

9 4 0

4

Prompt: Describe how the coping strategy helped.

Student describes how the coping strategy helped to address the negative emotion and situation.

Felt calmer so didn’t react strongly or immediately to the other person and was able to walk away or respond calmly.

9 4 0

Prompt: OR if the coping strategy did not help, describe what other healthy coping strategy you could use if this situation happened again.

Student identifies how the strategy failed to improve current emotional state and also identifies an alternative healthy coping strategy to use for the future.

Next time, try to pay attention to the triggers that are upsetting me, such as the person’s use of blaming language.

9 4 0

Total /52

Special Accommodations Provided: Rationale for Accommodations Provided:


Appendix F

Scoring Tracking Sheet

Student: ______________________

Sept Oct Nov Dec Jan Feb March April May Gain

Date: Date: Date: Date: Date: Date: Date: Date: Date: Date:

Journal Prompts

/52 /52 /52 /52 /52 /52 /52 /52 /52

Interview Checklist

/44 /44 /44 /44 /44 /44 /44 /44 /44

Subtotal /96 /96 /96 /96 /96 /96 /96 /96 /96

Student Survey

/44 /44

Total /140 /140

Example

Sept Oct Nov Dec Jan Feb March April May Gain

Date: Date: Date: Date: Date: Date: Date: Date: Date: Date

Journal Prompts

7/52 13/52 13/52 17/52 26/52 17/52 35/52 41/52 41/52 +34 pts

Interview Checklist

10/44 10/44 17/44 10/44 17/44 26/44 26/44 35 /44 44/44 +34 pts

Subtotal 17/96 23/96 30/96 27/96 43/96 43/96 61/96 76/96 85/96 +68/96

Student Survey

16/44 32 /44 +16 pts

Overall Total

33/140 117/140 +84/140 points

This student’s overall gain for the year +84 points.
