intensive longitudinal data analysis and visual data ... · intensive longitudinal data analysis...

92
Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in Schools Alex J. Bowers, Ph.D. Associate Professor of Education Leadership Teachers College, Columbia University February 12, 2014

Upload: others

Post on 11-Jun-2020

10 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Intensive Longitudinal Data Analysis and Visual

Data Analytics of Student, Teacher & Leader

Data for Decision Making in Schools

Alex J. Bowers, Ph.D.

Associate Professor of Education Leadership

Teachers College, Columbia University

February 12, 2014

Page 2: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Overview of the Talk

Patterning Student Data to Predict Outcomes & Inform Interventions:

• What are the most accurate “dropout flags”? • Bowers, A.J., Sprott, R., Taff, S.A. (2013) Do we Know Who Will Drop Out? A Review of

the Predictors of Dropping out of High School: Precision, Sensitivity and Specificity. The

High School Journal.96(2), 77-100.

• Are there significantly different trajectories of student grade patterns

that predict high school drop out? • Bowers, A.J., Sprott, R. (2012) Examining the Multiple Trajectories Associated with

Dropping Out of High School: A Growth Mixture Model Analysis. The Journal of Educational

Research, 105(3), 176-195.

• How early can we detect these patterns? • Bowers, A.J. (2010) Analyzing the Longitudinal K-12 Grading Histories of Entire Cohorts of

Students: Grades, Data Driven Decision Making, Dropping Out and Hierarchical Cluster

Analysis. Practical Assessment, Research & Evaluation (PARE), 15(7), 1-18.

Understanding Leadership Effects through Subgroup Trajectory Analysis:

• Does principal training influence school achievement trajectories? • Bowers, A.J., White, B.R. (in press) Do Principal Preparation and Teacher Qualifications

Influence Different Types of School Growth Trajectories in Illinois? A Growth Mixture Model

Analysis.Journal of Educational Administration.

• Are there different types of school leadership? • Bowers, A.J., Blitz, M., Halverson H. (in prep) Is There a Typology of Teacher Responders

to the Comprehensive Assessment of Leadership for Learning (CALL) and Do They

Cluster in Different Types of Schools? A Two-Level Latent Class Analysis of CALL Survey

Data

Bowers 2014

Page 3: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

ELDARG:

Education Leadership Data Analytics Research Group

• Intensive longitudinal data analysis

• Visual data analytics

• School data driven decision making & resource

allocation

• Grades as useful data for decision making

• Early dropout prediction and intervention

• STEM high school pipeline transitions to college

• Mapping effects of high school curricular choices

• Effective school and district leadership practices

• Instructional leadership “best practices”

• Leadership training for school improvement

• School district systems and leadership

• School technology, facilities & finance

• Effects of gaming and computer use on

achievement

• School facility maintenance achievement effects

• School capital facility financing

If you’re interested in any of these research topics, please let me

know. We’re currently recruiting students to join the group!

http://www.tc.columbia.edu/academics/?facid=ab3764

Page 4: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Multi-Level

Subgroup

Identification Longitudinal

(Organization/Classroom Effects)

(Change over time)

(Typology Detection)

Intensive Longitudinal Data Analysis & Visual Data Analytics Bowers 2014

Page 5: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Linear Models

(HLM)

Multi-Level

Subgroup

Identification Longitudinal

(Organization/Classroom Effects)

(Change over time)

(Typology Detection)

Intensive Longitudinal Data Analysis & Visual Data Analytics Bowers 2014

Page 6: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Linear Models

(HLM)

Multi-Level

Subgroup

Identification Longitudinal

(Organization/Classroom Effects)

(Change over time)

(Typology Detection)

Intensive Longitudinal Data Analysis & Visual Data Analytics

Survival Modeling

Hazard Modeling

Longitudinal Risk Models

Bowers 2014

Page 7: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Linear Models

(HLM)

Hierarchical

Growth

Models

Multi-Level

Subgroup

Identification Longitudinal

(Organization/Classroom Effects)

(Change over time)

(Typology Detection)

Intensive Longitudinal Data Analysis & Visual Data Analytics

Survival Modeling

Hazard Modeling

Longitudinal Risk Models

Bowers 2014

Page 8: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Linear Models

(HLM)

Hierarchical

Growth

Models

Multi-Level

Subgroup

Identification Longitudinal

(Organization/Classroom Effects)

(Change over time)

(Typology Detection)

Intensive Longitudinal Data Analysis & Visual Data Analytics

Latent Class Analysis

(LCA)

Survival Modeling

Hazard Modeling

Longitudinal Risk Models

Bowers 2014

Page 9: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Linear Models

(HLM)

Hierarchical

Growth

Models

Multi-Level

Subgroup

Identification Longitudinal

Multilevel

LCA

(Organization/Classroom Effects)

(Change over time)

(Typology Detection)

Intensive Longitudinal Data Analysis & Visual Data Analytics

Latent Class Analysis

(LCA)

Survival Modeling

Hazard Modeling

Longitudinal Risk Models

Bowers 2014

Page 10: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Linear Models

(HLM)

Hierarchical

Growth

Models

Growth Mixture

Modeling

(GMM)

Cluster Analysis

Heatmaps

Multi-Level

Subgroup

Identification Longitudinal

Multilevel

GMM

Multilevel

LCA

(Organization/Classroom Effects)

(Change over time)

(Typology Detection)

Intensive Longitudinal Data Analysis & Visual Data Analytics

Latent Class Analysis

(LCA)

Survival Modeling

Hazard Modeling

Longitudinal Risk Models

Bowers 2014

Page 11: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

K-12 Schooling Outcomes

• Dropout risk

– Accuracy, precision, sensitivity & specificity

• Longitudinal patterns in teacher assigned grades

• Data driven decision making (3DM)

• Targeted and data-informed resource allocation

to improve schooling outcomes

Ryan Sprott

Bowers, A.J., Sprott, R., Taff, S.A. (2013) Do we Know Who Will Drop Out?

A Review of the Predictors of Dropping out of High School: Precision,

Sensitivity and Specificity. The High School Journal, 96(2), 77-100.

doi:10.1353/hsj.2013.0000 ( Preprint Available )

Bowers, A.J., Sprott, R. (2012) Examining the Multiple Trajectories

Associated with Dropping Out of High School: A Growth Mixture Model

Analysis. The Journal of Educational Research, 105(3), 176-195.

doi:10.1080/00220671.2011.552075 ( Preprint Available ) Sherry Taff

Bowers 2014

Page 12: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Issues of Accuracy in Predicting High School Dropout

• Dropping out of high school in the U.S. is associated with a multitude of negative outcomes.

• However, other than a select number of demographic and background variables, we know little about the accuracy of current dropout predictors that are school-related.

• Current predictions accurately predict only about 50-60% of students who actually dropout.

• According to Gleason & Dynarski (2002), accurate prediction of who will dropout is a resource and efficiency issue:

– A large percentage of students are mis-identified as at-risk.

– A large percentage of students who are at-risk are never identified.

• Current reporting of dropout “flags” across the literature is haphazard. – Almost none report accuracy

– Many report specificity or sensitivity, but rarely both

• A dropout flag may be highly precise, in that almost all of the students with the flag dropout, but may not be accurate since the flag may identify only a small proportion of the dropouts.

Balfanz et al. (2007); Bowers (2010); Gleason & Dynarski (2002); Pallas (2003); Rumberger (2004) Bowers 2014

Page 13: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Re-analyzing Past Dropout Flags for Accuracy,

Precision, Sensitivity and Specificity

• Literature search. • Queried multiple databases:

– JSTOR, Google Scholar, EBSCO, Educational Full Text Wilson Web

• Studies were included that: – Were published since 1979 – Examined High School dropout – Examined school-wide characteristics and included all students – A focus on the student level – Reported frequencies for re-analysis

• Initially yield 6,434 overlapping studies – 301 studies were read in full – 140 provided school-wide samples and quantifiable data – 36 articles provided enough data for accuracy re-calculations – Yield 110 separate dropout flags

• Relative Operating Characteristic (ROC) – Hits versus False-Alarms

Bowers 2014

Page 14: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Dropout Graduate

Dropout a

True-positive

(TP)

Correct

b

False-positive

(FP)

Type I Error

a+b

Graduate c

False-negative

(FN)

Type II Error

d

True-negative

(TN)

Correct

c+d

a+c b+d a+b+c+d=N

Event

Pre

dic

tor

Precision = a/(a + b) Positive Predictive Value

True-Positive Proportion = a/(a + c) Sensitivity

True-Negative Proportion = d/(b + d) Specificity

False-Positive Proportion = b/(b + d) 1-Specificity

Event table for calculating dropout contingency proportions

Fawcett (2004); Hanley & McNeil (1982); Swets (1988); Zwieg & Campbell (1993)

“Hits”

“False Alarms”

Bowers 2014

Page 15: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

0

0.2

0.4

0.6

0.8

1.0

Tru

e-p

ositiv

e p

roport

ion (

Sensitiv

ity)

0 0.2 0.4 0.6 0.8 1.0

False-positive proportion (1-Specificity)

Perfect prediction

Better prediction

Worse prediction

Low attendance

An example of the true-positive proportion plotted against the false-positive

proportion for Balfanz et al. (2007) comparing the relative operating characteristics

(ROC) of each dropout flag.

Bowers 2014

Page 16: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

0

0.2

0.4

0.6

0.8

1.0

Tru

e-p

ositiv

e p

roport

ion (

Sensitiv

ity)

0 0.2 0.4 0.6 0.8 1.0

False-positive proportion (1-Specificity)

Relative operating characteristics (ROC) of all dropout flags reviewed, plotted

as the true-positive proportion against the false-positive proportion. Numbers

refer to dropout indicator IDs.

Bowers 2014

Page 17: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Bowers 2014

Page 18: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

0

0.2

0.4

0.6

0.8

1.0

Tru

e-p

ositiv

e p

roport

ion (

Sensitiv

ity)

0 0.2 0.4 0.6 0.8 1.0

False-positive proportion (1-Specificity)

Relative operating characteristics (ROC) of all dropout flags reviewed, plotted

as the true-positive proportion against the false-positive proportion. Numbers

refer to dropout indicator IDs.

Bowers 2014

Page 19: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

0

0.2

0.4

0.6

0.8

1.0

Tru

e-p

ositiv

e p

roport

ion (

Sensitiv

ity)

0 0.2 0.4 0.6 0.8 1.0

False-positive proportion (1-Specificity)

Relative operating characteristics (ROC) of all dropout flags reviewed, plotted

as the true-positive proportion against the false-positive proportion. Numbers

refer to dropout indicator IDs.

Bowers 2014

Page 20: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

0

0.2

0.4

0.6

0.8

1.0

Tru

e-p

ositiv

e p

roport

ion (

Sensitiv

ity)

0 0.2 0.4 0.6 0.8 1.0

False-positive proportion (1-Specificity)

Relative operating characteristics (ROC) of all dropout flags reviewed, plotted

as the true-positive proportion against the false-positive proportion. Numbers

refer to dropout indicator IDs.

Bowers 2014

Page 21: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Demographics:

Student: Female

African American Asian

Hispanic Non-Traditional Family

SES

School: Urban Rural

% Students Free Lunch

Behavior & Structure:

Student: Extracurricular

Retained Negative Behavior

School:

Student-Teacher Ratio Academic Press

Small School Large School

Extra-Large School

High School Dropout

Longitudinal Grades Time1, Time2, Time3

A Growth Mixture Model to Identify Grade Trajectories Associated with Dropout

Bowers 2014

Page 22: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Demographics:

Student: Female

African American Asian

Hispanic Non-Traditional Family

SES

School: Urban Rural

% Students Free Lunch

Behavior & Structure:

Student: Extracurricular

Retained Negative Behavior

School:

Student-Teacher Ratio Academic Press

Small School Large School

Extra-Large School

High School Dropout

A Growth Mixture Model to Identify Grade Trajectories Associated with Dropout

Longitudinal Grades Time1, Time2, Time3

Bowers 2014

Page 23: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Demographics:

Student: Female

African American Asian

Hispanic Non-Traditional Family

SES

School: Urban Rural

% Students Free Lunch

Behavior & Structure:

Student: Extracurricular

Retained Negative Behavior

School:

Student-Teacher Ratio Academic Press

Small School Large School

Extra-Large School

High School Dropout

Latent Trajectory

Classes C

A Growth Mixture Model to Identify Grade Trajectories Associated with Dropout

Longitudinal Grades Time1, Time2, Time3

Bowers 2014

Page 24: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Demographics:

Student: Female

African American Asian

Hispanic Non-Traditional Family

SES

School: Urban Rural

% Students Free Lunch

Behavior & Structure:

Student: Extracurricular

Retained Negative Behavior

School:

Student-Teacher Ratio Academic Press

Small School Large School

Extra-Large School

High School Dropout

Latent Trajectory

Classes C

Intercepts Slopes

GPA 9S1 GPA 9S2 GPA 10S1

A Growth Mixture Model to Identify Grade Trajectories Associated with Dropout

Bowers 2014

Page 25: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Growth mixture model for the simultaneous estimation of latent trajectory classes

using non-cumulative GPA from the first three semesters of high school.

GPA 9S1 GPA 9S2 GPA 10S1

Intercepts Slopes

High School

Dropout

Behavior & Structure:

Student:

Extracurricular

Retained

Negative Behavior

School:

Student-Teacher Ratio

Academic Press

Small School

Large School

Extra-Large School

Demographics:

Student:

Female

African American

Asian

Hispanic

Non-Traditional Family

SES

School:

Urban

Rural

% Students Free Lunch

Latent

Trajectory

Classes

C

Bowers 2014

Page 26: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

A:

B:

0

1

2

3

4

9S1 9S2 10S1Semester

No

n-C

um

ula

tive

GP

A

Mid-DecreasingLow-IncreasingMid-AchievingHigh-Achieving

0

1

2

3

4

9S1 9S2 10S1

Semester

0

1

2

3

4

9S1 9S2 10S1

0

1

2

3

4

0

1

2

3

4

9S1 9S2 10S1

Semester

0

Non

-cum

ula

tive

GP

A

1

2

3

4

9S1 9S2 10S1

Semester

0

1

2

3

4

9S1 9S2 10S1

Semester

0

1

2

3

4

9S1 9S2 10S1

0

1

2

3

4

9S1 9S2 10S1

Semester

0

1

2

3

4

9S1 9S2 10S1

Semester

0

1

2

3

4

9S1 9S2 10S1

0

1

2

3

4

9S1 9S2 10S1

Semester

Low-Increasing: 13.8%Mid-Decreasing: 10.8% Mid-Achieving: 56.5% High-Achieving: 18.9%

Dropout: 39.7% 52.1% 7.0% 1.2%

Longitudinal Non-Cumulative GPA Trajectories in the First

Three Semesters of High School

n=5,400

Mid-decreasing & Low-increasing accounted for:

24.6% of the sample

91.% of the dropouts

Bowers 2014

Page 27: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Growth Mixture Modeling & Student Outcome

Prediction

Next Steps

• Why grades are predictive of outcomes.

• Examine other outcomes – College-going

– College major

– College graduation

– Later employment & careers

• Two-level GMM – What are the high school-level effects on trajectory

probabilities? • Are there some schools that have higher proportions of certain

grade trajectory groups. What predicts it? (aka “dropout factories)

– Leadership and teacher effects on grade trajectories

Bowers 2014

Page 28: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Overview of the Talk

Patterning Student Data to Predict Outcomes & Inform Interventions:

• What are the most accurate “dropout flags”? • Bowers, A.J., Sprott, R., Taff, S.A. (2013) Do we Know Who Will Drop Out? A Review of

the Predictors of Dropping out of High School: Precision, Sensitivity and Specificity. The

High School Journal.96(2), 77-100.

• Are there significantly different trajectories of student grade patterns

that predict high school drop out? • Bowers, A.J., Sprott, R. (2012) Examining the Multiple Trajectories Associated with

Dropping Out of High School: A Growth Mixture Model Analysis. The Journal of Educational

Research, 105(3), 176-195.

• How early can we detect these patterns? • Bowers, A.J. (2010) Analyzing the Longitudinal K-12 Grading Histories of Entire Cohorts of

Students: Grades, Data Driven Decision Making, Dropping Out and Hierarchical Cluster

Analysis. Practical Assessment, Research & Evaluation (PARE), 15(7), 1-18.

Understanding Leadership Effects through Subgroup Trajectory Analysis:

• Does principal training influence school achievement trajectories? • Bowers, A.J., White, B.R. (in press) Do Principal Preparation and Teacher Qualifications

Influence Different Types of School Growth Trajectories in Illinois? A Growth Mixture Model

Analysis.Journal of Educational Administration.

• Are there different types of school leadership? • Bowers, A.J., Blitz, M., Halverson H. (in prep) Is There a Typology of Teacher Responders

to the Comprehensive Assessment of Leadership for Learning (CALL) and Do They

Cluster in Different Types of Schools? A Two-Level Latent Class Analysis of CALL Survey

Data

Bowers 2014

Page 29: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Cluster Analysis & Heatmap Visualizations • Research Question:

– To what extent are there similar user data patterns? – What do the data patterns look like?

• Method: – Cluster analysis is a descriptive statistic

• No hypothesis test

– Similar to the original Netflix and Amazon.com user preference algorithms

• “Other users who also liked/bought/viewed this item, also liked/bought/viewed these other items that you have not yet considered”

– Fewer assumption violation issues than with regression statistics so it can take a wide range of data types

– Attempts to use all of the available data and is robust to missing data issues

• Heatmaps are adapted from bioinformatics and cancer biology

Bowers, A.J. (2010) Analyzing the Longitudinal K-12 Grading Histories of Entire Cohorts of Students:

Grades, Data Driven Decision Making, Dropping Out and Hierarchical Cluster Analysis. Practical

Assessment, Research & Evaluation (PARE), 15(7), 1-18. http://pareonline.net/pdf/v15n7.pdf Bowers 2014

Page 30: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

A

B

C

D

F

K 1 2 3 4 5 6 7 8 9 10 11 12

Grade Year

Gra

de M

ark

ing

Hypothetical Hierarchical Clustering Data

Student 1

Student 2

Student 3

Student 4

Student 5

Student 6

Student 7

Student 8

Bowers 2014

Page 31: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Student 1

Student 2

Student 3

Student 4

Student 5

Student 6

Student 7

Student 8

A

B

C

D

F

K 1 2 3 4 5 6 7 8 9 10 11 12

Grade Year

Gra

de M

ark

ing

Hypothetical Hierarchical Clustering Data

Bowers 2014

Page 32: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Student 1

Student 2

Student 3

Student 4

Student 5

Student 6

Student 7

Student 8

A

B

C

D

F

K 1 2 3 4 5 6 7 8 9 10 11 12

Grade Year

Gra

de M

ark

ing

Hypothetical Hierarchical Clustering Data

Bowers 2014

Page 33: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hypothetical Hierarchical Clustering Data: Clustergram

Student 3

Student 7

Student 5

Student 4

Student 8

Student 6

Student 1

Student 2

A

B

C

D

F

K 1 2 3 4 5 6 7 8 9 10 11 12

Grade Year

Bowers 2014

Page 34: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

K 1 2 3 4 5 6 7 8 9 10 1112

Grade Year

A B C D F

Hypothetical Hierarchical Clustering Data: Clustergram

Student 3

Student 7

Student 5

Student 4

Student 8

Student 6

Student 1

Student 2

A

B

C

D

F

K 1 2 3 4 5 6 7 8 9 10 11 12

Grade Year

Bowers 2014

Page 35: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

K 1 2 3 4 5 6 7 8 9 10 1112

Grade Year

A B C D F

Hypothetical Hierarchical Clustering Data: Clustergram

Student 3

Student 7

Student 5

Student 4

Student 8

Student 6

Student 1

Student 2

Bowers 2014

Page 36: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

K Elementary MS 9th 10th 11th 12th

Hierarchical Clustering of Teacher Assigned Subject-Specific Grades

Bowers 2014

Page 37: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

K Elementary MS 9th 10th 11th 12th

Hierarchical Clustering of Teacher Assigned Subject-Specific Grades

Clu

ste

r T

ree

Bowers 2014

Page 38: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

K Elementary MS 9th 10th 11th 12th

Hierarchical Clustering of Teacher Assigned Subject-Specific Grades

Clu

ste

r T

ree

Bowers 2014

Page 39: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

K Elementary MS 9th 10th 11th 12th

Hierarchical Clustering of Teacher Assigned Subject-Specific Grades

Clu

ste

r T

ree

Su

bje

cts

Bowers 2014

Page 40: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

K Elementary MS 9th 10th 11th 12th

Hierarchical Clustering of Teacher Assigned Subject-Specific Grades

Clu

ste

r T

ree

Su

bje

cts

Students

Bowers 2014

Page 41: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

K Elementary MS 9th 10th 11th 12th

Hierarchical Clustering of Teacher Assigned Subject-Specific Grades

Clu

ste

r T

ree

Su

bje

cts

Students

Bowers 2014

Page 42: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

K Elementary MS 9th 10th 11th 12th

Hierarchical Clustering of Teacher Assigned Subject-Specific Grades

Clu

ste

r T

ree

Su

bje

cts

Students

Bowers 2014

Page 43: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

K Elementary MS 9th 10th 11th 12th

Hierarchical Clustering of Teacher Assigned Subject-Specific Grades

Clu

ste

r T

ree

Su

bje

cts

Students

+3 -3 0 No Data Bowers 2014

Page 44: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Clustering of Grades

K Elementary MS 9th 10th 11th 12th

Bowers 2014

Page 45: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Clustering of Grades

K Elementary MS 9th 10th 11th 12th

Bowers 2014

Page 46: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Clustering of Grades

K Elementary MS 9th 10th 11th 12th

Bowers 2014

Page 47: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Clustering of Grades

K Elementary MS 9th 10th 11th 12th High-High

Bowers 2014

Page 48: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Clustering of Grades

K Elementary MS 9th 10th 11th 12th High-High

Low-Low

Page 49: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Clustering of Grades

K Elementary MS 9th 10th 11th 12th High-High

Low-Low

Low-High

Page 50: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Clustering of Grades

K Elementary MS 9th 10th 11th 12th High-High

Low-Low

Low-High

High-Low

Page 51: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

0

0.5

1

1.5

2

2.5

3

3.5

4

K 1 2 3 4 5 6 7 8 9S1 9S2 10S1 10S2 11S1 11S2 12S1 12S2

Grade-Level

Mean N

on-c

um

ula

tive G

PA

High-High

Mean non-cumulative GPA trends for clusters high-high,

low-low, low-high and high-low, K-12

Bowers 2014

Page 52: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

0

0.5

1

1.5

2

2.5

3

3.5

4

K 1 2 3 4 5 6 7 8 9S1 9S2 10S1 10S2 11S1 11S2 12S1 12S2

Grade-Level

Mean N

on-c

um

ula

tive G

PA

High-High Low-Low

Mean non-cumulative GPA trends for clusters high-high,

low-low, low-high and high-low, K-12

Bowers 2014

Page 53: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

0

0.5

1

1.5

2

2.5

3

3.5

4

K 1 2 3 4 5 6 7 8 9S1 9S2 10S1 10S2 11S1 11S2 12S1 12S2

Grade-Level

Mean N

on-c

um

ula

tive G

PA

Mean non-cumulative GPA trends for clusters high-high,

low-low, low-high and high-low, K-12

High-High Low-Low High-Low

Bowers 2014

Page 54: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

0

0.5

1

1.5

2

2.5

3

3.5

4

K 1 2 3 4 5 6 7 8 9S1 9S2 10S1 10S2 11S1 11S2 12S1 12S2

Grade-Level

Mean N

on-c

um

ula

tive G

PA

Mean non-cumulative GPA trends for clusters high-high,

low-low, low-high and high-low, K-12

High-High Low-Low High-Low Low-High

Bowers 2014

Page 55: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Hierarchical Clustering of Grades

K Elementary MS 9th 10th 11th 12th

Bowers 2014

Page 56: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Cluster analysis future work:

• Replicate with a national sample

• Partner with school districts to replicate and extend the work

• Cluster in two dimensions: Teacher x Student

• Include other types of student data & outcomes

• Test other types of cluster algorithms

• Integrate into data dashboard systems for principals & admin

Bowers 2014

Page 57: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Overview of the Talk

Patterning Student Data to Predict Outcomes & Inform Interventions:

• What are the most accurate “dropout flags”? • Bowers, A.J., Sprott, R., Taff, S.A. (2013) Do we Know Who Will Drop Out? A Review of

the Predictors of Dropping out of High School: Precision, Sensitivity and Specificity. The

High School Journal.96(2), 77-100.

• Are there significantly different trajectories of student grade patterns

that predict high school drop out? • Bowers, A.J., Sprott, R. (2012) Examining the Multiple Trajectories Associated with

Dropping Out of High School: A Growth Mixture Model Analysis. The Journal of Educational

Research, 105(3), 176-195.

• How early can we detect these patterns? • Bowers, A.J. (2010) Analyzing the Longitudinal K-12 Grading Histories of Entire Cohorts of

Students: Grades, Data Driven Decision Making, Dropping Out and Hierarchical Cluster

Analysis. Practical Assessment, Research & Evaluation (PARE), 15(7), 1-18.

Understanding Leadership Effects through Subgroup Trajectory Analysis:

• Does principal training influence school achievement trajectories? • Bowers, A.J., White, B.R. (in press) Do Principal Preparation and Teacher Qualifications

Influence Different Types of School Growth Trajectories in Illinois? A Growth Mixture Model

Analysis.Journal of Educational Administration.

• Are there different types of school leadership? • Bowers, A.J. , Blitz, M., Halverson H. (in prep) Is There a Typology of Teacher

Responders to the Comprehensive Assessment of Leadership for Learning (CALL) and Do

They Cluster in Different Types of Schools? A Two-Level Latent Class Analysis of CALL

Survey Data

Bowers 2014

Page 58: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Do principal training and experience influence

school achievement trajectories?

• Bowers, A.J., White, B.R. (in press) Do Principal Preparation and Teacher

Qualifications Influence Different Types of School Growth Trajectories in

Illinois? A Growth Mixture Model Analysis. Journal of Educational

Administration.

Brad White

Illinois Education Research Council

Bowers 2014

Page 59: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Purpose:

• The purpose of this study is to examine the extent to which principal preparation and professional experience influences different types of school proficiency growth trajectories

Background:

• Principal leadership can have a strong influence on student achievement.

• However, direct effects models are highly problematic (Hallinger & Heck, 1996, 2011).

• Recent research has begun to examine the effects of principal leadership, experience and training on growth in student achievement over time.

– Principal effects appear to be stronger in high poverty schools.

– Principal education appears to be unrelated to student achievement growth in NYC and Illinois.

– Principal on-the-job experiences are related to student achievement growth, such as time as an assistant principal and tenure as principal in the school.

• (Branch, Hanushek & Rivkin, 2009, 2012; Clark, Martorell & Rockoff, 2009; White & Bowers, 2011)

• All of these studies fit the schools to a single proficiency growth trajectory.

• But what if there are multiple statistically significantly different growth trajectories? – Hallinger & Heck (2011) have proposed that leadership by influence different school growth

trajectories in different ways.

Purpose & Background

Bowers 2014

Page 60: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

• To what extent are there different trajectories of elementary and middle school test score proficiency across multiple years of data?

• To what extent are principal background and experience variables related to school proficiency trajectory?

Research Questions

Bowers 2014

Page 61: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Sample:

• All Illinois elementary and middle schools – 2000-2001 through 2005-2006

• Illinois Standard Achievement Test (ISAT) – School-level percent met or exceeded standard

• Non-Chicago (n=2,584) and Chicago schools (n=499)

• Teacher Variables – % Inexperienced teachers

– ITAC (Index of Teacher Academic Capital): • Teacher ACT scores

• Teacher undergraduate college competitiveness

• Proportion of teachers emergency or provisional

• Proportion of teachers who failed the Illinois basic skills test on first attempt

• Principal Variables: – Age

– Female

– Minority

– Selectivity of undergraduate institution (Barron’s)

– Selectivity of graduate institution (Barron’s)

– First year principal or Principal 6+ years (vs. 2-5 years)

– Years principal had served as assistant principal in same school

Sample

Bowers 2014

Page 62: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Method

• Intensive Longitudinal Data (ILD) analysis – Collins, 2006; Shiyko, et al., 2012; Walls & Schafer, 2006 – Hierarchical linear growth models

• Time (level 1) nested within schools (level 2) • Long-format data • Estimates effects of time-varying covariates on:

– Intercepts (year 1 scores) – Slopes (growth in scores over time)

• (Hox, 2010; Raudenbush & Bryk, 2002; Singer & Willet, 2003)

• Growth mixture modeling (GMM)

– To what extent are there statistically significantly different trajectories in proficiency growth?

– Estimate hierarchical linear growth model conditional on trajectory group (latent class).

– Hallinger & Heck, 2011; Muthén, 2004; Shiyko, et al., 2012 Bowers 2014

Page 63: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

2001

Intercepts Slopes

Principal Background &

Experience:

Female

Minority

Age

Selective BA

Selective Masters

First Year Principal

Two-Five Years Principal

Assistant Principal

Taught in Same School

Demographics:

Students:

% African American

% Hispanic

% Asian

% Free & Reduced Lunch

% LEP

Enrollment

Mobility

School:

% Inexperienced Teachers

ITAC

Latent

Trajectory

Classes

C

2002 2003 2004 2005 2006

School % met or exceed standard on ISAT

Growth Mixture Model of Illinois ISAT Elem & MS Growth Trajectories

Bowers, A. J., & White, B. R. (in press). Do Principal Preparation and Teacher Qualifications Influence Different Types of School Growth

Trajectories in Illinois? A Growth Mixture Model Analysis. Journal of Educational Administration.

n=3,154 Elementary and Middle Schools

Bowers 2014

Page 64: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Growth Mixture Model of Illinois ISAT Elem & MS Growth Trajectories

Bowers, A. J., & White, B. R. (in press). Do Principal Preparation and Teacher Qualifications Influence Different Types of School Growth

Trajectories in Illinois? A Growth Mixture Model Analysis. Journal of Educational Administration. Bowers 2014

Page 65: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

0

20

40

60

80

100

% M

et o

r E

xce

ed

ed

Sta

nd

ard

(IS

AT

)

Asst. Princ. Not Asst. Princ.

Chicago

Principal Previously Assistant Principal

High

Low

High

Low

Year Year

0

20

40

60

80

100

% M

et o

r E

xce

ed

ed

Sta

nd

ard

(IS

AT

) High

Low

High

Low

Non-Chicago Chicago

Asst. Princ. Not Asst. Princ.

Taught in same school Taught in diff. school

Taught in same school Taught in diff. school

Principal Previously Taught in Same School

Non-Chicago

Growth Mixture Model of Illinois ISAT Elem & MS Growth Trajectories

Bowers, A. J., & White, B. R. (in press).

Page 66: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Overview of the Talk

Patterning Student Data to Predict Outcomes & Inform Interventions:

• What are the most accurate “dropout flags”? • Bowers, A.J., Sprott, R., Taff, S.A. (2013) Do we Know Who Will Drop Out? A Review of

the Predictors of Dropping out of High School: Precision, Sensitivity and Specificity. The

High School Journal.96(2), 77-100.

• Are there significantly different trajectories of student grade patterns

that predict high school drop out? • Bowers, A.J., Sprott, R. (2012) Examining the Multiple Trajectories Associated with

Dropping Out of High School: A Growth Mixture Model Analysis. The Journal of Educational

Research, 105(3), 176-195.

• How early can we detect these patterns? • Bowers, A.J. (2010) Analyzing the Longitudinal K-12 Grading Histories of Entire Cohorts of

Students: Grades, Data Driven Decision Making, Dropping Out and Hierarchical Cluster

Analysis. Practical Assessment, Research & Evaluation (PARE), 15(7), 1-18.

Understanding Leadership Effects through Subgroup Trajectory Analysis:

• Does principal training influence school achievement trajectories? • Bowers, A.J., White, B.R. (in press) Do Principal Preparation and Teacher Qualifications

Influence Different Types of School Growth Trajectories in Illinois? A Growth Mixture Model

Analysis.Journal of Educational Administration.

• Are there different types of school leadership? • Bowers, A.J., Blitz, M., Halverson H. (in prep) Is There a Typology of Teacher Responders

to the Comprehensive Assessment of Leadership for Learning (CALL) and Do They

Cluster in Different Types of Schools? A Two-Level Latent Class Analysis of CALL Survey

Data

Bowers 2014

Page 67: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

• Latent Class Analysis

– Research question: Is there only one kind of respondent across a survey, or are there statistically significantly different “modes” across the multidimensional response data?

– Said another way:

• Is there a single monolithic response group, or are there statistically significant different subgroups that respond or behave differently?

Bowers 2014

Page 68: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Are there different types of school leadership? • Bowers, A.J., Blitz, M., Halverson, R. (in prep) Is There a Typology of Teacher Responders to the

Comprehensive Assessment of Leadership for Learning (CALL) and Do They Cluster in Different

Types of Schools? A Two-Level Latent Class Analysis of CALL Survey Data.

Angela Urick, University of Oklahoma Urick, A., Bowers, A.J. (2014) What are the Different Types of Principals Across the U.S.? A Latent Class

Analysis of Principal Perception of Leadership. Educational Administration Quarterly, 50(1) 96-134.

Mark Blitz

University of Wisconsin-Madison

Richard Halverson

Bowers 2014

Page 69: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Bowers 2014

Page 70: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Bowers 2014

Page 71: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Background & Purpose

• Are responses across the CALL survey at the teacher and principal levels homogenous or heterogeneous? – This study is in response to recent calls to more appropriately

model the complex nature of teaching and leadership in schools (Hallinger & Heck 2011)

• We used Latent Class Analysis (LCA) to address this question. – LCA does not force everyone in a sample to a single “best fit”

line, but rather examines if there are significantly different subgroups/typology

– Barnes, Camburn & Sebastian (2010); Bowers (2012); Henry & Muthen (2010); Urick & Bowers (in press); Urick (2012)

Bowers 2014

Page 72: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

CALL Domain Subdomain 1 Subdomain 2 Subdomain 3 Subdomain 4 Subdomain 5

1. Focus on learning School-wide focus on

learning

Formal leaders

recognized as

instructional leaders

Collaborative design

of integrated

learning plan

Appropriate services

for students who

struggle

2. Monitoring Teaching

& Learning

Formative eval of

student learning

Summative eval of

student learning

Formative eval of

teaching

Summative eval of

teaching

3. Building Nested

Learning Communities

Collaborative school-

wide focus on

teaching and learning

Professional learning Socially distributed

leadership

Coaching and

mentoring

4. Acquiring & Allocating

Resources

Personnel practices Structuring and

maintaining time

School resources

focused on student

learning

Integrating external

expertise into

instruction

Coordinate & supervise

relations with families

& external communities

5. Maintaining Safe &

Effective Learning

Environment

Clear, consistent &

enforced expectations

for student behavior

Safe learning

environment

Safe haven provided

for struggling

students

Buffering the

teaching

environment

CALL Survey Leadership for Learning Domains & Subdomains

Bowers 2014

Page 73: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Research questions

• To what extent is there a typology of teacher and principal response patterns to CALL?

• To what extent do teacher and principal response patterns to CALL align (or not) across different types of schools and responders?

Bowers 2014

Page 74: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Sample & Method

• CALL Validation sample: – Level 1: n=3,919 teachers

– Level 2: n=109 schools

• Mean scores across the 21 subdomains

• Latent Class Analysis (LCA) – Person-centered statistic, rather than variable

centered.

– Identifies if there is one, or more than one significantly different type of responder.

• Bowers (2012), Henry & Muthen (2010), Muthen (2004), Urick (2012).

Bowers 2014

Page 75: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Two level latent class analysis (LCA) with three classes at level 1

teacher level (CW) and three classes at level 2 school level (CB)

Bowers 2014

Page 76: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Two level latent class analysis (LCA) with three classes at level 1

teacher level (CW) and three classes at level 2 school level (CB)

Bowers 2014

Page 77: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

3 Leader and 3 Teacher Leadership for Learning (LL) Response Subgroups

Bowers 2014

Page 78: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Focus on

Learning Monitor Teaching

& Learning Building Nested

Learning

Communities

Acquiring &

Allocating

Resources Maintaining Safe

& Effective Learning

Environments

3 Leader and 3 Teacher Leadership for Learning (LL) Response Subgroups

Bowers 2014

Page 79: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

A

A A) Providing appropriate services

for struggling students

3 Leader and 3 Teacher Leadership for Learning (LL) Response Subgroups

Focus on

Learning Monitor Teaching

& Learning Building Nested

Learning

Communities

Acquiring &

Allocating

Resources Maintaining Safe

& Effective Learning

Environments

Bowers 2014

Page 80: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

A

B A

B

A) Providing appropriate services

for struggling students

B) Formative evaluation of

teaching

3 Leader and 3 Teacher Leadership for Learning (LL) Response Subgroups

Focus on

Learning Monitor Teaching

& Learning Building Nested

Learning

Communities

Acquiring &

Allocating

Resources Maintaining Safe

& Effective Learning

Environments

Bowers 2014

Page 81: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

A

B C A

B C

A) Providing appropriate services

for struggling students

B) Formative evaluation of

teaching

C) Personnel practices

3 Leader and 3 Teacher Leadership for Learning (LL) Response Subgroups

Focus on

Learning Monitor Teaching

& Learning Building Nested

Learning

Communities

Acquiring &

Allocating

Resources Maintaining Safe

& Effective Learning

Environments

Bowers 2014

Page 82: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Focus on

Learning Monitor Teaching

& Learning Building Nested

Learning

Communities

Acquiring &

Allocating

Resources Maintaining Safe

& Effective Learning

Environments

A

B C

D

A

B C

D

E

A) Providing appropriate services

for struggling students

B) Formative evaluation of

teaching

C) Personnel practices

D) Structuring time, resources &

expertise

E) Buffering the teaching

environment

3 Leader and 3 Teacher Leadership for Learning (LL) Response Subgroups

Bowers 2014

Page 83: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

3 Leader and 3 Teacher Leadership for Learning (LL) Response Subgroups

Bowers 2014

Page 84: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

3 Leader and 3 Teacher Leadership for Learning (LL) Response Subgroups

1.4 Providing appropriate services for

students who traditionally struggle

Bowers 2014

Page 85: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

3 Leader and 3 Teacher Leadership for Learning (LL) Response Subgroups

1.4 Providing appropriate services for

students who traditionally struggle

2.1 – 2.2 Formative & Summative

evaluation of student learning

Bowers 2014

Page 86: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

3 Leader and 3 Teacher Leadership for Learning (LL) Response Subgroups

1.4 Providing appropriate services for

students who traditionally struggle

2.1 – 2.2 Formative & Summative

evaluation of student learning

4.1 Personnel practices

Bowers 2014

Page 87: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

3 Leader and 3 Teacher Leadership for Learning (LL) Response Subgroups

1.4 Providing appropriate services for

students who traditionally struggle

2.1 – 2.2 Formative & Summative

evaluation of student learning

4.1 Personnel practices

4.2 Structuring and maintaining time

4.3 School resources are focused on

student learning

4.4 Integrating external expertise into

school instructional program

Bowers 2014

Page 88: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

3 Leader and 3 Teacher Leadership for Learning (LL) Response Subgroups

Bowers 2014

Page 89: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

0

20

40

60

80

100

Pe

rcen

t o

f Te

ach

ers

0

1

2

3

Avg L

ea

der

Ye

ars

Exp

erie

nce

0

10

20

30

Avg T

ea

ch

er

Ye

ars

Exp

erie

nce

Low LL

Schools Moderate

LL Schools

High LL

Schools

0

1

2

3

4

Ave

rage

Su

rve

y R

esp

onse

Leader High Teachers Mod Teachers

LowTeachers

Three School Contexts of Leadership for Learning

Bowers 2014

Page 90: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Conclusions and Implications

Three significantly different groups of teacher & leaders respondents to CALL

Low (31.3%) • Focus on struggling students

• Less formative eval. of teachers

• Not buffered as much from environ

Moderate (43.7%) High (25.0%) • Collab design of integrated learning plans

• Formative eval. of teaching & learning

• Coaching & mentoring

• Acquisition and allocation of resources

Leadership for Learning Teachers

Leadership for Learning Leaders/Schools

Low (41.5%) • Highest % of Low LL Teachers

• Most experienced teachers

• Least experienced leaders

• Lowest Leader responses

Moderate (41.5%) High (17.0%) • Highest % of High LL Teachers

• Least experienced teachers

• Most experienced leaders

• Strong alignment between teachers and

leaders

Bowers 2014

Page 91: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Overview of the Talk

Patterning Student Data to Predict Outcomes & Inform Interventions:

• What are the most accurate “dropout flags”? • Bowers, A.J., Sprott, R., Taff, S.A. (2013) Do we Know Who Will Drop Out? A Review of

the Predictors of Dropping out of High School: Precision, Sensitivity and Specificity. The

High School Journal.96(2), 77-100.

• Are there significantly different trajectories of student grade patterns

that predict high school drop out? • Bowers, A.J., Sprott, R. (2012) Examining the Multiple Trajectories Associated with

Dropping Out of High School: A Growth Mixture Model Analysis. The Journal of Educational

Research, 105(3), 176-195.

• How early can we detect these patterns? • Bowers, A.J. (2010) Analyzing the Longitudinal K-12 Grading Histories of Entire Cohorts of

Students: Grades, Data Driven Decision Making, Dropping Out and Hierarchical Cluster

Analysis. Practical Assessment, Research & Evaluation (PARE), 15(7), 1-18.

Understanding Leadership Effects through Subgroup Trajectory Analysis:

• Does principal training influence school achievement trajectories? • Bowers, A.J., White, B.R. (in press) Do Principal Preparation and Teacher Qualifications

Influence Different Types of School Growth Trajectories in Illinois? A Growth Mixture Model

Analysis.Journal of Educational Administration.

• Are there different types of school leadership? • Bowers, A.J., Blitz, M., Halverson H. (in prep) Is There a Typology of Teacher Responders

to the Comprehensive Assessment of Leadership for Learning (CALL) and Do They

Cluster in Different Types of Schools? A Two-Level Latent Class Analysis of CALL Survey

Data

Bowers 2014

Page 92: Intensive Longitudinal Data Analysis and Visual Data ... · Intensive Longitudinal Data Analysis and Visual Data Analytics of Student, Teacher & Leader Data for Decision Making in

Thank you!

[email protected]

Bowers 2014