aft 7/12/04 marywood university using data for decision support and planning

AFT 7/12/04 Marywood University

Using Datafor

Decision Supportand

Planning


Presenters

Dr. Marie Loftus Dr. Barbara R. Sadowski

Marywood Universitywww.marywood.edu


Session Overview

Group activity followed by discussion

Starting the research process

Break

Group activity – questionnaire analysis and discussion

Research results

Question and answer


Group Task 1 Develop 2-3 questions of interest

For each question list:

– a) what data could be used – b) factors that might affect the process of

obtaining data

_ c) potential problems with existing data

– d) environmental factors that might bias the data or interpretation of the results

– e) relevant existing data sources


To determine if a teacher’s Thinking Math training has an effect on student achievement in Mathematics.

Research Purpose


Questions Considered

Does the mathematics training of teachers have an effect on the math achievement of students they teach?

Does professional development training of teachers translate into higher student achievement?


Practicality of Research Options

Question 1:

– Difficult to get data on teacher’s background in mathematics

– Measurement difficulty in defining college course work with in-service professional development

Question 2: - Teacher training data available- Non-transient teacher population- Well-documented training program- Student achievement data available


Other Variables Considered

What constituted teacher training? How many TM workshops?

Did TM trained teachers actually implement the principles? Which ones were implemented? How confident did the teachers feel about their ability to implement?

Was years of teaching experience a factor in how well TM was implemented?



Was number of years teaching at grade level a factor?

Was the math background of the teacher a factor in how well TM was implemented?

How much of the time did the teacher actually implement TM when teaching math?



What student variables might affect the score on the achievement test? Reading level? SES? Title 1 support? The number of TM trained teachers they had before reaching fifth grade?

Would school climate affect how the teachers felt about implementing TM math principles? Did the principal support their use of TM?

Ability grouping in the assignment of students


Research Question 1

Is the number of years students have a TM trained math teacher a predictor of how they will perform on a math achievement test?


Total enrollment 8,673 students

Economically disadvantaged 54%

Mobility 25.2%

Schools 18–13 elementary (Pre K-5)

Teachers 712–strong collective bargaining history with the Scranton Federation of Teachers

Scranton PA School District


Data Collection

Survey instrument developed collaboratively by Thinking Math trainers and researchers

Information session held for teachers and administrators

Surveys administered to TM teachers

Student data collected at school sites


Sample

The study sample was teachers and 5th grade students in:

4 of 13 elementary schools

–3 schools with TM trained teachers at the 5th grade level. –1 school with TM trained teachers, none at the 5th grade level


Sample Schools

All offer Title 1 reading, two also offer math

Fifth grade enrollments are 70, 55, 44, and 34


Sample Schools

Percentages of low income students: 28, 36, 46, and 77% (PA Profiles, 1999- 2000)

Class organization

– heterogeneous and homogeneous– departmentalized and self-contained


Group Task 2

Review/analyze teacher questionnaire results

Consider possible reasons for the results of the scale item means

Are the scales correlated? Which ones?

What other variables are related to the scales?

What changes would you make to improve the instrument?


Teacher Survey Results


Teachers Surveyed

33 of 37 are female

Pre-K - 5, Title 1, Learning Support

10% have < ten years teaching experience

70% have twenty or more years of experience

average of 1.5 ER&D courses beyond TM

AFT 7/12/04 Marywood UniversityNCTM 4/ 10/ 03 Scranton SD - Marywood University

Teacher Survey Scales

Confidence Scale ( 7 items)

Rate yourself on your confidence in... e.g., accepting multiple solutions.

1 = Not confident 10 = Very confident

Scale Mean = 8.57 s.d. = 0.80


Teacher Survey Scales

Implementation Scale (7 items)

To what extent do you... e.g., have your students explain and justify their solutions to problems?

1= Not at all 10 = To a great extent

Scale Mean = 8.1 s.d. = 0.89


Influence of TM Scale Items

1. How frequently do you incorporate the TM principles in math instruction?

2. To what degree has your delivery of math instruction changed as a result of TM training?

3. How has the depth of the math content you are teaching changed since your TM training?

4. How much has TM research added to your content knowledge of mathematics?

Scale Mean = 8.19 s.d. = 0.97


Correlations

Confidence Implementation

Influence of TM .395* .429**

Confidence .877**

* p<.05 **p < .01

n = 37, two-tailed


Regression

Implementation as the dependent variable

Influence of TM as independent variable

Confidence in TM Principles as independent variable

89% of the variance in implementation is accounted for by the two independent variables


Experience and Implementation

Teacher Experience N Mean s.d.

Less than 20 years 12 7.86 0.81

More than 20 years 25 8.33 0.95


Teacher Comments on TM

“I now change problems to reflect my class, linking school math to real world experiences.”

“I now use more manipulatives.”

“It has given me new insight for teaching multiple strategies and meeting individual needs.”


Student Achievement Results


SAT Math Problem Solving

PSSA Math

SAT Math Procedures

.798** .777**

SAT Math Problem Solving

.819**

**p < .01, (two-tailed); N = 167

Correlations: SAT – PSSA Math


SAT

Reading PSSA

Reading SAT Math Procedures .626** .620**

SAT Math Problem Solving .694** .648**

PSSA Mathematics .765** .693** **p < .01, (two-tailed); N = 167

Correlations: Reading - Math


Research Hypotheses

Students with 5th grade TM trained teachers will perform significantly better on PSSA 5th grade Math tests compared to the control group.

Students with 5th grade TM trained teachers will perform significantly better on PSSA 5th grade Math tests than the district and state.


Math Performance Levels

Level*** Thinking

Math District

Advanced 19.2% 14.8%

Proficient 26.3 25.9

Basic 36.5 31.2

Below Basic 18.0 28.0

***2 = 14.6; df =3, p < .005


Math Performance Levels

Level Thinking

Math State

Advanced 19.2% 22.5%

Proficient 26.3 30.5

Basic 36.5 24.6

Below Basic 18.0 22.4

Chi-square is not significant.


Mean Comparisons

Thinking

Math Control

PSSA Math** 1307 1216

SAT Math Procedures*

672 644

SAT Problem Solving***

665 631

*t = 2.40 ; df = 39, p < .05

**t = 2.74 ; df = 195, p < .01

***t = 4.0 ; df = 41, p < .000


Mean Comparisons

Thinking

Math District

PSSA Math** 1307 1270

SAT Math

Procedures*** 672 594

SAT Problem Solving***

665 609

**t = 2.91 ; df = 166, p < .005

***t > 20 ; df = 167, p < .000


Mean Comparisons

Thinking

Math State

PSSA Math 1307 1310

Mean difference is not significant.


Teacher-Student Results


Mean Comparisons > 2 TM teachers 2 or fewer

SAT Problem Solving** 665 655

SAT Math Procedures* 671 664

Reading Comprehension*** 671 660

*t = -1.76 ; df = 197, p = .081

**t = -0.91 ; df = 197 p = .081

***t = -2.04; df = 197, p < .05


Research Question

Can math achievement be predicted from teacher experience, professional development and implementation of the TM principles?


Analysis

A multiple regression was run with SAT math procedures as the criterion variable, and three independent variables: years of experience at grade level, professional development and scale score on implementation of TM principles.

A second multiple regression used SAT problem solving as the dependent variable.


Math Procedures Regression

Standardized Coefficients

t

Implementation of TM Principles .497 3.52***

Experience -.158 -1.85*

Professional Development -.191 N.S.

*p =.066

*** p < .001


Math Problem Solving Regression


t

Implementation of TM Principles

.451 3.18**

Experience -.130 N.S.

Professional Development

-.151 N.S.

** p < .05


Reading Comprehension Regression


t

Implementation of TM Principles .549 3.88***

Experience -.077 N.S.

Professional Development -.311 -2.30**

* * p = .023

*** p < .000


Conclusions

Teacher confidence in TM principles and perception of the importance of TM are predictors of implementation which is in turn a predictor of student mathematics achievement.

The number of years a student has a TM trained teacher is a predictor of student achievement in mathematics.


Conclusions

Students who have a 5th grade TM trained teacher performed significantly better on the PSSA 5Th grade Math Test than the control group and the district.


Research Purpose

The research results suggest that TM training does have a significant effect on student achievement.


Topics for Further Research

Does TM training affect student achievement in Reading Comprehension?

What is the relationship between TM training, student reading comprehension and student mathematics problem-solving achievement?


Questions?

aft 7/12/04 marywood university using data for decision support and planning

Documents