An Update on Florida’s An Update on Florida’s Charter Schools Program Charter Schools Program

Grant: CAPES External Grant: CAPES External EvaluationEvaluation

2014 Florida Charter Schools Conference: 2014 Florida Charter Schools Conference: Sharing ResponsibilitySharing Responsibility

November 19November 19thth, 2014, 2014

Introduction of PresentersIntroduction of PresentersIntroduction of PresentersIntroduction of Presenters

Principal Investigator: M. David Miller, Principal Investigator: M. David Miller, Ph. D.Ph. D.

Project Manager: Nan Thornqvist, Ed. D.Project Manager: Nan Thornqvist, Ed. D.

Researcher: Wei XuResearcher: Wei Xu

The Charter Schools Program Grant The Charter Schools Program Grant and External Evaluationand External Evaluation

The purpose of the USDE Charter School Program The purpose of the USDE Charter School Program Grant (CSPG) is to increase the national Grant (CSPG) is to increase the national understanding of the charter school model by:understanding of the charter school model by:

1.1. expanding the number of high-quality charter expanding the number of high-quality charter schools by providing financial assistance for the schools by providing financial assistance for the planning, program design, and initial planning, program design, and initial implementation of charter schools, andimplementation of charter schools, and

2.2. by evaluating the effects of charter schools, by evaluating the effects of charter schools, including their effects on student academic including their effects on student academic achievement, staff, and parentsachievement, staff, and parents

FL DOE IEPC Office: FL DOE IEPC Office: Objectives of the CSPGObjectives of the CSPG

Objective 1Objective 1: Increase access to high-quality charter : Increase access to high-quality charter schools for educationally disadvantaged students.schools for educationally disadvantaged students.

Objective 2Objective 2: Improve the authorizing practices and : Improve the authorizing practices and capacity of Local Education Agency Authorizers.capacity of Local Education Agency Authorizers.

Objective 3Objective 3: Increase the number of high-quality : Increase the number of high-quality charter schools in Florida.charter schools in Florida.

Objective 4Objective 4: Increase the academic achievement of : Increase the academic achievement of charter school students in Florida.charter school students in Florida.

Objectives of the External Objectives of the External Evaluation of the CSPGEvaluation of the CSPG

Objective 1Objective 1: The external evaluation team : The external evaluation team will review IEPC activities and documents and will review IEPC activities and documents and report as to the annual progress regarding report as to the annual progress regarding their process and outcome performance their process and outcome performance measures as established to obtain each of the measures as established to obtain each of the four CSP grant objectives.four CSP grant objectives.

Objectives of the External Objectives of the External Evaluation of the CSPGEvaluation of the CSPG

Objective 2Objective 2: The CAPES evaluation team will help determine : The CAPES evaluation team will help determine if the IEPC Office has the appropriate systems in place to if the IEPC Office has the appropriate systems in place to sustain and grow a high-quality charter school sector in sustain and grow a high-quality charter school sector in Florida by determining:Florida by determining:

1)1)What are the strengths and weaknesses in the IEPC What are the strengths and weaknesses in the IEPC Office’s grant application and review processes?Office’s grant application and review processes?

2)2)How can the IEPC Office improve its CSP grant application How can the IEPC Office improve its CSP grant application and review process to better screen for quality?and review process to better screen for quality?

3)3)What is the impact of CSP funding?What is the impact of CSP funding?

4)4)What is the impact of each of the dissemination grants What is the impact of each of the dissemination grants funded by the grant?funded by the grant?

Two Main Goals of EvaluationTwo Main Goals of Evaluation

Examining, improving, and monitoring Examining, improving, and monitoring the grant application and application the grant application and application processprocess

Examining CSP funded schoolsExamining CSP funded schools

Selection of CSP Grant AwardeesSelection of CSP Grant Awardees

Large numbers of applicants each year for Large numbers of applicants each year for approximately 50-60 grant awards (152 approximately 50-60 grant awards (152 applicants in 2011; 112 in 2012; 89 last applicants in 2011; 112 in 2012; 89 last year)year)

All-encompassing online application; All-encompassing online application;

2 grant stages (see handout #2)2 grant stages (see handout #2)

Selection of CSP Grant Selection of CSP Grant Awardees (Cont’d)Awardees (Cont’d)

Ratings/scores based on multiple criteriaRatings/scores based on multiple criteria

Ratings/scores from a panel of five peer Ratings/scores from a panel of five peer reviewersreviewers

The high and low scores are dropped—The high and low scores are dropped—average of the three scores is the final scoreaverage of the three scores is the final score

Evaluation CriteriaEvaluation Criteria

1) Validity of scoring criteria1) Validity of scoring criteria

2) Reliability of scores2) Reliability of scores

ValidityValidity

Content review of grant sectionsContent review of grant sections

Consistency with models used in Consistency with models used in other states including grant other states including grant sections and weighting/scoringsections and weighting/scoring

ReliabilityReliability• Dependability index from Generalizability Dependability index from Generalizability

TheoryTheory• Absolute measurementAbsolute measurement• Allows complex data collection (5 raters Allows complex data collection (5 raters

with each application) with each application) • Allows dropping highest and lowest scoreAllows dropping highest and lowest score• Consistency of ratings of participants (mean Consistency of ratings of participants (mean

and variance of discrepancy from average)and variance of discrepancy from average)

Reliability 2013-14Reliability 2013-14

Area Dependability – 3(drop high and low)

Project Need 0.607

Project Design and Implementation Founding/Governance

0.540

School Leadership 0.694

Business, Finance and Accounting 0.340

Special Populations 0.475

Curriculum and Instruction 0.443

Evaluation 0.596

Dissemination 0.520

Total 0.785

Implications for Grant Implications for Grant Application ProcessApplication Process

Validity of process is highValidity of process is high

Reliability for decision (total) is highReliability for decision (total) is high

Continued monitoring and suggestions will Continued monitoring and suggestions will continue to be made (regarding reviewers, continue to be made (regarding reviewers, grant sections, etc.)grant sections, etc.)

CSP Grant Awardee Data CSP Grant Awardee Data Surveys (IEPC Office personnel, grant reviewers, Surveys (IEPC Office personnel, grant reviewers,

grant applicants, site visited schools’ governing grant applicants, site visited schools’ governing board members, principals, teachers, parents, board members, principals, teachers, parents, community members)community members)

District interviews (five different districts each District interviews (five different districts each year)year)

School site visits (25 schools selected each year; School site visits (25 schools selected each year; document review; direct observation of instruction document review; direct observation of instruction and tour of facilities)and tour of facilities)

Analysis of achievement data and additional Analysis of achievement data and additional research in discovered areas of concern (special research in discovered areas of concern (special populations, teacher attrition)populations, teacher attrition)

Factors Affecting Teacher Factors Affecting Teacher Turnover in Florida Charter Turnover in Florida Charter

schoolsschools

Literature ReviewLiterature Review Teacher attrition rates in charter schools across the Teacher attrition rates in charter schools across the

country is between 20% and 25%. The teacher country is between 20% and 25%. The teacher attrition rate for new teachers reaches 40% attrition rate for new teachers reaches 40% annually (Miron & Applegate, 2007). annually (Miron & Applegate, 2007).

Charter schools are negatively impacted by high Charter schools are negatively impacted by high teacher turnover: teacher turnover:

Has a significant negative impact on academic Has a significant negative impact on academic achievement (Ronfeldt, Loeb,& Wyckoff, 2012). achievement (Ronfeldt, Loeb,& Wyckoff, 2012).

presents a challenge for cultivating high-quality presents a challenge for cultivating high-quality instructional environment (Stuit & Smith, in instructional environment (Stuit & Smith, in press). press).

Literature Review Factors associated with teacher turnover Factors associated with teacher turnover include:include:Teacher background characteristics-e.g. age, gender, Teacher background characteristics-e.g. age, gender, race/ethnicity (Miron & Applegate, 2007; Stuit & Smith, race/ethnicity (Miron & Applegate, 2007; Stuit & Smith, 2012); 2012); School characteristics-e.g. geographical location, School characteristics-e.g. geographical location, institution size (Stuit & Smith, 2012); institution size (Stuit & Smith, 2012); School personnel policies-e.g. union membership School personnel policies-e.g. union membership (Stuit & Smith, 2012); (Stuit & Smith, 2012); Teacher quality-e.g. teaching experiences (Ingle, Teacher quality-e.g. teaching experiences (Ingle, 2007). 2007).

Research Question Research Question In Florida, CSP school site visits have revealed a In Florida, CSP school site visits have revealed a

concern about teacher turnover where many concern about teacher turnover where many teachers are leaving charter schools midyear. teachers are leaving charter schools midyear. Thus, there is a need to get updated information Thus, there is a need to get updated information regarding factors that are associated with teacher regarding factors that are associated with teacher turnover in charter schools including turnover turnover in charter schools including turnover within years as well as annual turnover.within years as well as annual turnover.

To what extent are teacher salary, academic To what extent are teacher salary, academic credentials, as well as years of experience credentials, as well as years of experience associated with teacher turnover in Florida associated with teacher turnover in Florida charter schools?charter schools?

Data Source Data Source Data pertaining to teacher salary, teaching Data pertaining to teacher salary, teaching experience, and attrition (2011-12, 2012-13) experience, and attrition (2011-12, 2012-13) have been obtained from Florida Department have been obtained from Florida Department of Education EIS (Education Information of Education EIS (Education Information Services).Services).

Teacher numbers were taken from Survey 2 Teacher numbers were taken from Survey 2 (early fall) and were compared with Survey 3 (early fall) and were compared with Survey 3 numbers (late spring) to examine teacher numbers (late spring) to examine teacher attrition in Florida charter schools for those attrition in Florida charter schools for those two years.two years.

Descriptive Statistics 2011-12 Descriptive Statistics 2011-12

Descriptive Statistics 2012-13 Descriptive Statistics 2012-13

Descriptive Statistics Descriptive Statistics (start of year)(start of year)

Teacher Salaries Teacher Salaries – Mean $33,175 in 2011-2012Mean $33,175 in 2011-2012– Mean $32,875 in 2012-2013Mean $32,875 in 2012-2013

Teacher Experience Teacher Experience – Mean 4.93 in 2011-2012Mean 4.93 in 2011-2012– Mean 4.97 in 2012-2012Mean 4.97 in 2012-2012

Research Findings - Salary Research Findings - Salary Salary: 2011-12Salary: 2011-12

Teachers who stayed had significantly higher salaries than Teachers who stayed had significantly higher salaries than those who left and those who replaced the teachers who those who left and those who replaced the teachers who left.left.

Replacement teachers had higher salaries than those who Replacement teachers had higher salaries than those who leave but not significantly.leave but not significantly.

Salary: 2012-13Salary: 2012-13Teachers who stayed had significantly higher salaries than Teachers who stayed had significantly higher salaries than

those who left and those who replaced teachers who left.those who left and those who replaced teachers who left.

Replacement teachers had significantly higher salaries Replacement teachers had significantly higher salaries than those who left.than those who left.

Research Findings – Research Findings – Teacher ExperienceTeacher Experience

Teaching Experience: 2011-12Teaching Experience: 2011-12Teachers who stayed had significantly higher Teachers who stayed had significantly higher

experience than those who left.experience than those who left.

New teachers had significantly less experience New teachers had significantly less experience than teachers who stayed or teachers who left.than teachers who stayed or teachers who left.

Teaching Experience: 2012-13Teaching Experience: 2012-13Similar conclusions could be drawn.Similar conclusions could be drawn.

Research Findings – Research Findings – All variablesAll variables

Combining analyses to examine the joint effect of Combining analyses to examine the joint effect of salary, educational level, and years of experience.salary, educational level, and years of experience.

For 2011-12, teachers with more years of working For 2011-12, teachers with more years of working experience, a bachelor degree, and a higher salary experience, a bachelor degree, and a higher salary are more likely to stay in charter schoolsare more likely to stay in charter schools

For 2012-13, teachers with more years of working For 2012-13, teachers with more years of working experience and higher salaries are more likely to experience and higher salaries are more likely to keep working in charter schoolskeep working in charter schools

SummarySummary

IEPC Office has a working CSPG IEPC Office has a working CSPG application and review process in placeapplication and review process in place

Schools are new-need time for Schools are new-need time for programs to show effectsprograms to show effects

Data collection and monitoring will Data collection and monitoring will continuecontinue

Contact InformationContact InformationM. David MillerM. David Miller

The University of Florida and CAPESThe University of Florida and CAPES

dmiller@coe.ufl.edu

Nan Thornqvist Nan Thornqvist

The University of Florida and CAPESThe University of Florida and CAPES

nanthorn@coe.ufl.edu

(352) 538-5187(352) 538-5187

Wei XuWei Xu

The University of Florida and CAPESThe University of Florida and CAPES

xuweiufl@ufl.edu