The Well- EQUIPped Classroom: Using the Electronic Quality of Inquiry Protocol to evaluate effects of science inquiry professional

development on PreK-3 classroom practice Presented by

Gale A. Mentzer, PhDT. Ryan Duckett, MA

Toledo, OH

Research and Evaluation, LLC1811 N. Reynolds Road, Suite 204Toledo, OH 43615

“Accuracy of observation is the equivalent of accuracy of thinking.”--Wallace Stevens

American Evaluation Association National ConferenceOctober 29, 2016Atlanta, GA

Validity Issues

Surveys Tests Artifacts

EQUIP

• Time

Usage

• Instructio

n

• Discourse

• Assessme

nt

• Curriculu

m

Marshall, J. C., Horton, B., & White, C. (2009). EQUIPping teachers: A protocol to guide and improve inquiry-based instruction. The Science Teacher, 76(4), 46-53.

Example of EQUIP Instructional Factor

EQUIP as Evaluation Tool

Developing (2)

Proficient (3)

Pre-Inquiry (1)

Some More Much More

Exemplary (4)

Rasch at a Glanceexp(Bp - Di)

[1 + exp(Bp - Di)]P{xpi=1|Bp,Di}=

Higher levels of the trait

Level of Ability

(Person Measures)

Lower levels of the trait

Difficult to Endorse

Items that target the latent trait

(Item Measures)

Easy to Endorse

3.0

2.0

1.0

0.0

-1.0

-2.0

-3.0

Rasch Process in Four StepsStep OneAre the items

functioning properly?

Step Two

Step Three Step Four

Are the teachers participating reasonably?

Are you measuring a single latent trait?

How do we use what we know to understand?

Measuring Item Fit

Item FitHarder to Affirm

Teacher provides depth of content that connects big

picture

Lesson allowed for student designed investigation

Lesson integrated content and student

investigation

Student organized information in effective

ways to communicate their learning

Overall assessment of curriculum factor

Easier to Affirm

Difficulty Fit

2.46

1.21

-0.91

-1.26

-1.50

Reliability = .95

0.63

0.93

0.30

1.58

1.07

Rating Scale

Example of a poorly functioning rating scaleRating scale for Spring 2015 Curriculum factor

Building a Staircase

Pre-Inquiry (1)

Developing (2)

Proficient (3)

Exemplary (4)

Person FitSpring 2015 (Pre-intervention)

Fall 2015 (Post-intervention)

Rasch SeparationSeparation statistic of 5.77 = 8 distinct groups!

Dimensionality

Assessing Contrast

Evaluating the Intervention

Scale t α

Instruction 4.623 <.0001

Discourse 3.701 <.0001

Assessment 6.067 <.0001

Curriculum 3.877 <.0001

Assessing long term effects: ANOVA

Instruction 1 < 2 (α = .002) 1 < 3 (α < .0001) 2 = 3 (α = 1.0)

Discourse 1 < 2 (α = .013) 1 < 3 (α < .003) 2 = 3 (α = 1.0)

Assessment 1 < 2 (α < .0001) 1 < 3 (α < .0001) 2 = 3 (α = 1.0)

Curriculum 1 < 2 (α = .004) 1 < 3 (α < .0001) 2 = 3 (α = 1.0)

Any Questions?

This presentation will be uploaded to our website:www.acumenresearcheval

.com