the four letter word- data!!!!! 2009 professional school counselors’ conference marksville,...
TRANSCRIPT
The Four Letter Word- DATA!!!!!
2009 Professional School Counselors’ Conference
Marksville, LouisianaDecember 2, 2009
Mark Kuranz, [email protected] Past President
DataA great four letter word!
WHY?
ASCA National Model
Management System
Accountability System
• Results Report – – Program– Guidance Curriculum – Closing the Gap – Impact Over Time
• School Counselor Performance Standards
• The Program Audit
How are students different as a RESULT of what you do?
• Results Report – –Program–Guidance Curriculum –Closing the Gap –Impact Over Time
What Is Your Relationship with Education Data?
Nonexistent?
Reactive?
Proactive?
How Not to Use Data
• To place blame
• To focus on the past
• To maintain the status quo
The Power of Data
• Provide objective snapshots of the students, school, community
• Surface evidence of access or equity issues• Break old myths – eliminate denial• Create urgency/energy for change• Provide direction – data driven decisions
The Power of Data
• Challenge existing behavior, funding patterns, programs, & policies
• Use as an accountability tool• Focuses resources where they are most
needed • Supports grant writing efforts
Using Data to Spur Systemic Change
• School Counselors must be proficient in: – Accessing data – Analyzing data – Interpreting data – Presenting data
• School Counselors must use data to: - Recognize barriers to learning– Point out the system inequities– Advocate for system change– Create urgency for change
WHY?Because…• Drives decisions for student success• Connects to the mission of schools• Demonstrates effectiveness• Incorporates the systemic use of Results
Reports• Ensures accountability
Your Beliefs?
• How much are you willing to learn, do, become frustrated, or challenge the status quo?
• In time, what can you give up?
• Can you advocate for a data-driven program?
Beliefs = Behaviors
• Data • Student characteristics• Ethical standards
Beliefs
What kind of students do we want leaving our schools?
Our answers become a vision for all students.
What kind of students do we want leaving our schools?.
• readers• writer• successful• smart• clear communicator• reliable• appreciative• willing to give back (volunteer)• cautious• risk taker• persistent• confident• moral• spiritually connected• competitive• ability to say no• problem solver• negotiator• willing to compromise
• friendly• team players• honest• questioners• helpful• effective listeners• critical thinkers• life long learners• loyal• responsible• caring• analytical• happy• physically healthy• technologically competent• effective
ASCA Ethical StandardsPreamble
• Each person has the right to be respected, be treated with dignity and have access to a comprehensive school counseling program that advocates for and affirms all students from diverse populations regardless of ethnic/racial status, age, economic status, special needs, English as a second language or other language group, immigration status, sexual orientation, gender, gender identity/expression, family type, religious/spiritual identity and appearance.
• Each person has the right to receive the information and support needed to move toward self-direction and self-development and affirmation within one’s group identities, with special care being given to students who have historically not received adequate educational services: students of color, low socio-economic students, students with disabilities and students with nondominant language backgrounds.
ASCA Ethical Standards
A.1. Responsibilities to Students• The professional school counselor:• b. Is concerned with the educational,
academic, career, personal and social needs and encourages the maximum development of every student.
ASCA Ethical StandardsA.3. Counseling PlansThe professional school counselor:a. Provides students with a comprehensive school counseling
program that includes a strong emphasis on working jointly with all students to develop academic and career goals.
b. Advocates for counseling plans supporting students right to choose from the wide array of options when they leave secondary education. Such plans will be regularly reviewed to update students regarding critical information they need to make informed decisions.
Academic Personal/Social Career
1. 1. 1.
2. 2. 2.
3. 3. 3.
Categorical Brainstorm What projects are possible?
What Data Should You Collect?
Data Elements/Variables Important to Consider: Student Demographics Attainment Achievement Student Behavior Choices Opportunity Gaps
25
What do you want to know? Student Demographics:
What are the characteristics of our students?• Gender• Ethnicity• Socio-economic status (free/reduced lunch)• Limited English Proficiency• Family configuration• Mobility• Zip Code
26
What do you want to know? Attainment:
How many make it to - and beyond key points in the system?
• Advance to next grade• Transition from middle school to high school• Graduation rate• Type of high school diploma• Matriculation to an institution of higher education• Persistence beyond freshman year• Earning a college diploma
27
What do you want to know? Achievement:
What does achievement look like at different levels and with different groups of students?
• Overall Achievement– Grade point average– Standardized test scores– Passing all subjects
• Periodic assessment– Semester grades– End of course tests
• Ongoing classroom assessment– Class assignment grades– Tests
28
What do you want to know? Student Behavior Choices:
What are students doing?• Attendance• Discipline referrals• Classroom behavior• Homework completion• Enrollment patterns
– Algebra in 9th grade– Upper level math and science– Honors, AP, college credit
29
Are some students provided more opportunities than others?
• Access to rigorous curriculum• Access to quality teachers• School policies and climate• Special Education Screening
Placement• School activities• Participation in Support Services
– Tutoring, Mentoring
What do you want to know? Opportunity Gaps:
30
Types of Data•Process•Perception •Results
31
Process data• “What you did for whom” • Evidence that event occurred• How activity was conducted• Did the program follow the prescribed
practice?
32
Process Data - Examples
• Six counseling groups with 8 students each were held
• 1,350 6-8th grade students received the “Time to Tell” guidance lesson
• All high school students seen individually to prepare 4 year plan.
33
Adding Process Data
Activity Process Data
Perception Data Results Data
Question or Demonstration
Pre PostHow will student
behavior change?
Pre Post
Date
# %
Date
# %
Date
# %
Date
# %
34
Perception Data• “What others think, know or demonstrate” data. • Measures competency achieved, knowledge gained
or attitudes beliefs of students – Pre-post– Competency achievement– Surveys– Evaluations
• Measures what students are perceived to have gained in knowledge
35
Perception Data - Examples• Competency Achievement
– Every student in grades 9-12 completed a 4 year plan – Every 10th grade student completed an interest inventory
• Knowledge Gained– 89% of students demonstrate knowledge of promotion/ retention
criteria– 92% can identify Early Warning Signs of violence
• Attitudes or Beliefs– 74%of students believe fighting is wrong– 29% of parents say their child feels safe at school– 58% of teachers say students behavior appropriately in class
36
Adding Perception Data
Activity Process Data
Perception Data Results Data
Question or Demonstration
Pre PostHow will student
behavior change?
Pre Post
Date
# %
Date
# %
Date
# %
Date
# %
37
Results Data• “So WHAT” data• Response to Intervention• How has student behavior changed• Proof activity has (or has not) positively
impacted students ability to utilize the knowledge, attitudes and skills to effect behavior– Attendance– Behavior– Academic achievement
38
Results Data - Examples
• 42 students on the retention list avoided retentions
• Graduation rates improved 14% over three years
• Attendance improved among 9th grade males by 49%
39
Adding Results Data
Activity Process Data
Perception Data Results Data
Question or Demonstration
Pre PostHow will student
behavior change?
Pre Post
Date
# %
Date
# %
Date
# %
Date
# %
40
Examples of Data to ExamineTest Scores
AchievementStateNational
Enrollment Honors/AP Classes College Track Special Education LEP
Graduation Rate By Gender By Ethnicity By SES
Attendance Absences Tardies By Grade Level
Discipline By Classroom Types of Problems Gender
GPA/Class Rank By Gender By Ethnicity By SES
Retention Rates By Subject Area By Grade Level By Gender, Ethnicity Post Secondary Plans
Special Education By Gender By Ethnicity By SES
Dropout Rate Grade Levels Gender, Ethnicity… Reasons Why
So What?Our colleagues are
doing this DATA work!
42
0
5
10
15
20
25
30
Percent
2005-2006 2006-2007
Percentage of Drop OutsCase High School
2005-2006
2006-2007
43
Case High School Graduation
0102030405060708090
100
97-98 98-99 99-00 00-01 01-02 02-03
Bus Referrals
Academic Achievement – Student Focused
AVERAGE GRADE
INCREASE - 18.2%
58.4945.52
67.28
0
20
40
60
80
Semester 1 Quarter 3 Quarter 4Semesters
Gra
des
AVERAGES
Year TwoGrade Point Average
0
0.5
1
1.5
2
2.5
End of yr.
Grade 8
Gr. 9-Qtr.1
Gr. 9 Sem. 1
Ripple--Expertise--Collaborate
1.5
2.5
3.5
studen
t 1
stu
den
t 2
studen
t 3
studen
t 4
studen
t 5
studen
t 6
stu
den
t 7
studen
t 8
studen
t 9
studen
t 10
studen
t 11
studen
t 12
G.P.A. Q1G.P.A. S1
The state per pupil reimbursement is $7,500 for the whole year A 2% increase in overall attendance for Fall Semester 2004 adds 11
students to the state’s bill◦ 94% of 550 9th graders = 517 students◦ 96% of 550 9th graders = 528 students (11 more)
1/2 of $7,500 = $3,750 11 new students times $3,750 = $41,250
The cost of hiring a new starting Masters-level professional school counselor is about $40,000 (not including benefits)
The school has financially gained about what it would cost them to pay for a new professional school counselor for one year.
What are the financial benefits to the school for increasing attendance by 2% for a full semester?
Honors Class Enrollment
0
5
10
15
20
25
30
35
Lang. Arts Math Science Soc. St. Electives World Lang.
Subject Area
Per
cent
age
of s
tude
nts
wit
hin
race
Asian-Pacific
White
Hispanic
Minority AP Enrollment in One Jefferson County High School - System Focused
Student EngagementListening Lesson
51
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Pre
Post
Reading
2000-01 – 3rd grade, 2001-02 – 4th grade, 2002-03 – 5th grade
40
42
44
46
48
50
52
54
56
2000-01 2001-02 2002-03
districtproject students
Math
2000-01 – 3rd grade, 2001-02 – 4th grade, 2002-03 – 5th grade
92
93
93
94
94
95
95
96
96
97
2000-01 2001-02 2002-03
districtproject students
Attendance
2000-01 – 3rd grade, 2001-02 – 4th grade, 2002-03 – 5th grade
55
Difference between students who stayed in program and those who dropped in terms of truancy
0.0
2.0
4.0
6.0
8.0
10.0
12.0
Stayed in Program Dropped out Semester 1
Avera
ge N
um
ber
of
Days T
ruan
t
56
Relationship between amount of face to face contact and truancy
0.0
2.0
4.0
6.0
8.0
10.0
12.0
None 1 to 4 5 to 20 20 to 70 More than 70
Amount of face to face contact
Num
ber
of d
ays
trua
nt
Baseline Days Truant
Q2 - Days Truant
Q3 - Days Truant
For everyone except the people with the highest baseline truancy rate, the second quarter seemed much better than the third in terms of truancy. People who had the most contact also demonstrated the biggest reduction in truancy from baseline.
57
Relationship between amount of face to face contact and truancy
0.0
2.0
4.0
6.0
8.0
10.0
12.0
None 1 to 4 5 to 20 20 to 70 More than 70
Amount of face to face contact
Num
ber
of d
ays
trua
nt
Baseline Days Truant
Q2 - Days Truant
Q3 - Days Truant
For everyone except the people with the highest baseline truancy rate, the second quarter seemed much better than the third in terms of truancy. People who had the most contact also demonstrated the biggest reduction in truancy from baseline.
58
Student Engagement 06-07No data for Q2 truancy on people who never participated.
The reduction observed from baseline to Q2 in truancy is statistically significant, as is the increase in truancy from Q2 to Q3.
Changes in truancy over time by level of involement
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
10.0
Level of involvement
Nu
mb
er
of
Da
ys
Tru
an
t
Baseline Days Truant 3.9 3.9 5.3 8.6
Q2 - Days Truant 0.0 2.7 3.4 4.5
Q3 - Days Truant 2.6 4.9 5.7 5.1
Contact in and out of School
Never Participated Social Worker Truancy Program
59
Student Engagement 06-07It is interesting that people who never participated had the highest GPA at baseline, and the lowest at Q3. Also, there was relatively little movement in GPA’s for people
who did participate. Overall, the decrease from baseline to Q2 GPA is statistically significant, as is the increase from Q2 to Q3.
GPA over time as function of level of involement
0.0
0.5
1.0
1.5
2.0
Level of involement
GP
A
Baseline GPA 1.3 1.8 1.2 1.2
Q1 GPA 1.4 1.6 1.2 1.2
Q2 GPA 1.1 1.2 1.0 0.9
Q3 GPA 1.7 1.3 1.3 1.3
Contact in and out of School
Never Participated or Dropped Sem1
Social Worker Truancy Program
60
Student Engagement 06-07For everyone except the people with the highest baseline truancy rate, the second quarter
seemed much better than the third in terms of truancy. People who had the most contact also demonstrated the biggest reduction in truancy from baseline.
Relationship between amount of face to face contact and truancy
0.0
2.0
4.0
6.0
8.0
10.0
12.0
None 1 to 4 5 to 20 20 to 70 More than 70
Amount of face to face contact
Nu
mb
er o
f d
ays
tru
ant
Baseline Days Truant
Q2 - Days Truant
Q3 - Days Truant
Examples of preliminary gains are as follows:• Teachers reported an increase in student safety at the
project schools. (Source: TUSD School Quality Survey)
• Students responded that students of different races & ethnic backgrounds are getting along better at their school. (Source: TUSD School Quality Survey)
School Climate Results
• Parents indicated an increase in satisfaction with their school. (Source: TUSD School Quality Survey)
• All Principals indicated increased contact with students and a more consistent program (in classrooms, groups and interventions) with increased counseling staff. Principals consider the counselor indispensable. (Source: Principal interviews – March and April 2003)
• 5th grade students reveal that they see positive school changes, have gained many academic and personal skills from their counselor and have a personal vision that includes continuing their education. (Source: Focus groups conducted in March and April at each school2003)
• Principals saw counselor as the primary change agent for the school
• Principals understood how the counselor can contribute to student achievement
• Principals acknowledge that the school counselor knows every student and is the soul or heart of the school.
Principal Comments
Using Charts: Bullying Data
Student Reasons for Being Absent
SHARING PRE & POST DATA IN EXCEL
Questions Pre-test
Post-test
1. Regularly use accountability strategies.
1.4 3
2. Believe SC should address educational issues.
2 4.2
3. Can Conduct program assessments. 2.3 4
4. Know how to organize and make sense of collected data.
3 3.5
5. Know how to share results with stakeholders.
2.5 4.1
SHARING PRE & POST DATA IN CHART
0
2
4
6
Pre-test
Post-test
• Use simple statistics - averages and percentages
• Disaggregate – take apart by meaningful wholes
• Aggregate – condense statistics to meaningful representative numbers
• Cross-tabulate – put data into a chart
• Longitudinal data – look at data over time
Basic Ways to Analyze Data
When you want to know how the group as a whole did on a particular activity or question
Ex., 20 students answered a question on whether they feel safe at school using a 1 to 5 Likert Scale (1-strongly disagree; 2-disagree; 3-unsure; 4-agree; 5-strongly agree)
2 students (1); 3 students (2); 3 students (3); 10 students (4); 2 students (5)
2X1 + 3X2 + 3X3 + 10X4 + 2X5 = 67/20(students) = 3.35 (just above 3-unsure)
When to Use the Average (and how to calculate it)
When you want to know how the group as a whole did on a particular activity or question
Ex., 20 students answered a question on whether they feel safe at school using a 1 to 5 Likert Scale (1-strongly disagree; 2-disagree; 3-unsure; 4-agree; 5-strongly agree)
2 students (1); 3 students (2); 3 students (3); 10 students (4); 2 students (5)
2X1 + 3X2 + 3X3 + 10X4 + 2X5 = 67/20(students) = 3.35 (just above 3-unsure)
When to Use the Average (and how to calculate it)
12 students respond to one question on a questionnaire (1=strongly disagree; 2=disagree; 3=agree; 4=strongly agree)
1 students respond strongly disagree (1) 1X1= 13 students respond disagree(2) 3X2= 65 students respond agree (3) 5X3=153 students respond strongly agree (4) 3X4=12 1+6+15+12= 34 34/12= 2.8 or, the average score on this question for all 12
students is closest to ‘agree.’
Example: Computing Averages
Working with PercentagesThe number over the total = %If 325 out of 350 students graduate, then 92% is the
graduation rate (325/350=92%)
Working with Percent ChangeFinal # (b) – beginning # (a) = change (c)change (c)/beginning # (a) X 100 = % change
c/a = d x 100 = % change 340 graduate 2003; 325 in 2002 (340-325=15)
15/325 X100= 4.5% graduation rate increase!
Statistics Primer
When to use percents?– When you want to know what part of a group did
or did not accomplish somethingFormula: Part of the group (a) /total group (b) = fraction of
the total (c ) X 100 = % Ex., 175 students graduated out of 214 175/214 = . 82 X 100 = 82% graduated
When to Use Percentage(and how to find it)
Ex. On a pretest the students average score on the 1st question about career choices was 1.4 (on a scale of 1-5). The posttest score was 4.2.
Formula: final #(b) - beginning #(a) = change (c) change(c) / beginning #(a)= (d) X 100 = % of change 4.2 – 1.4 = 2.8 2.8/1.4 = 2 X 100 = 200%In other words students increased their knowledge
of career choice by 200%!
Finding the Percentage of Change (Impact or Results)
• Microsoft Excel Chart Online Traininghttp://office.microsoft.com/training/training.aspx?AssetID=RC011055061033
• Free Tech Tools. ASCA School Counselor, 44, pp. 22-29.
Data Support
Where do you begin?• Carol Kaffenberger, Ph.D and Anita Young, Ph.D.
• Eric Sparks - Wake County Public School System
• ASCA Results Reports
• SPARK and SPARK-W
Using DATA to Understand Educational Issues
1. DESIGN: What is the question?2. ASK: How will you answer the question?3. TRACK: How will you make sense of the
data?4. ANNOUNCE: How will you use the findings?
C. Kaffenberger and A. Young
Wake County Public School SystemExample of Goals and Objectives
Area Goal Objective
AcademicTo increase number of promotions
In 06-07, the promotion rate of 3rd grade students will increase by 10% as compared to 05-06
BehaviorTo create a safer school climate
In 06-07, incidents of fighting at school will be reduced by 5% as compared to 05-06
Attendance
To increase the attendance rate
In 06-07, the attendance rate of 9th grade repeating students will increase by 50%
http://www.ascanationalmodel.org/
http://www.schoolcounselor.org/scene
http://www.umass.edu/schoolcounseling/
http://www.cescal.org/
Creating Pre and Post Measures• Classroom teachers asks for help in improving
students’ listening skills• Four listening lessons in over a two month period• Process data - 120 students, 4 – 45 min. lessons• Perception data – pre/post survey• Results data – increase in listening behaviors,
reduction in classroom interruptions, decrease in behavioral referrals
86
87
Creating Pre- & Post Measures
• Think about the specific indicator that you wish to measure
• Delineate the 2-4 main concepts, ideas, or skills contained within the indicator
• Create 1-2 questions for each of these
http://www.sparconline.net/
http://www.doe.state.in.us/octe/goldstar/evaluation.html
90
How are students different as
a result of the school counseling
program?
Remember: The New Question is….
