www.sii.soe.umich.edu developing measures of mathematical knowledge for teaching geoffrey phelps,...

www.sii.soe.umich.edu

http://sitemaker.umich.edu/lmt

1

Developing Measures of Mathematical Knowledge for

Teaching

Developing Measures of Mathematical Knowledge for

Teaching

Geoffrey Phelps, Heather Hill, Deborah Loewenberg Ball, Hyman

Bass Study of Instructional ImprovementLearning Mathematics for Teaching

Consortium for Policy Research in EducationUniversity of Michigan

NSF/MSP State Coordinators Meeting

October 20, 2005



2

Overview of today’s session Overview of today’s session

1. LMT/SII Measures Development

2. Some Sample Results

3. LMT/SII Measures and Dissemination



3

Subtract:

What is “Content Knowledge for Teaching”? An Example From

Subtraction

What is “Content Knowledge for Teaching”? An Example From

Subtraction

3002

783-



4

Analyzing Student ErrorsAnalyzing Student Errors

3002

783-

2781

3002 - 783 = 4832



5

Analyzing Unusual Student SolutionsAnalyzing Unusual Student Solutions

3002

783-

299

2219

12 3 0 0 2

7 8 3-

3-7-8-1

2 2 1 9

LMT/SII Measures Development



7

Why Would We Want to “Measure” Teachers’ Content Knowledge for

Teaching?

Why Would We Want to “Measure” Teachers’ Content Knowledge for

Teaching?

• To understand “what” constitutes mathematical knowledge for teaching

• To understand role of teachers’ content knowledge in students’ performance

• To study and compare outcomes of professional development and teacher education

• To inform design of teachers’ opportunities to learn content knowledge



8

Measuring Teachers’ Mathematics Knowledge: Background and History

Measuring Teachers’ Mathematics Knowledge: Background and History

• Research on teacher behavior• Early research on student achievement

– Proxy measures for teacher knowledge– Tests of basic skills

• 1985 on: “the missing paradigm” pedagogical content knowledge

• 1990s: interview studies of teachers’ mathematics knowledge (MSU -- NCRTE)



9

Study of Instructional Improvement

Study of Instructional Improvement

• Study of three Comprehensive School Reforms; teacher knowledge a key variable

• Instrument development goals:– Develop measures of content knowledge

teachers use in teaching • K-6 content for elementary school teachers• Not just what they teach - specialized knowledge

– Develop measures that discriminate among teachers (not criterion referenced)

– Non-ideological

• But we faced significant problems….



10

Problems As We Began This Work

Problems As We Began This Work

• No way to measure teachers’ content knowledge for teaching on a large scale– Small number of items, many written by Ball,

Post, others appeared on every instrument– Nothing known about the statistical qualities

of those items (difficulty, reliability)– Studies relied on single items, yet single

items unlikely valid or reliable measures of teacher knowledge



11

Early Decisions and ActivityEarly Decisions and Activity

• Survey-based measure of CKT-M– 3000 teachers participating in SII– Multiple choice

• Specified domain map

• 5 people + 5 lbs cheese + 5 weeks = 150 items (May 2001)

• Large-scale piloting, summer 2001



12

Early Decisions and ActivityEarly Decisions and Activity

Types of knowledge

Math

em

atica

l co

nte

nt

Content knowledge

Knowledge of content

and students

Number

Operations

Patterns, functions, and algebra



13

Early Analyses and Validity Checks

Early Analyses and Validity Checks

• Results from piloting – We can measure teachers’ CKT – CK reliabilities .70-.90– Factor analysis shows distinct types of

knowledge • Knowledge of content and students (KCS)

separate from CK• Specialized content knowledge (SCK) vs.

common content knowledge (CCK)



14

Overarching Findings: Range of Piloted Reliabilities (IRT)

Overarching Findings: Range of Piloted Reliabilities (IRT)


Knowledge of content and

students

Number and operations (K-6) .71-.89 .51-.78

Patterns, functions, and algebra (K-6) .77-.87

Geometry (3-8) .92



15

Content Knowledge :Number and Operations

Content Knowledge :Number and Operations

• Common knowledge– Number halfway between 1.1 and 1.11

• Specialized knowledge– Representing mathematical ideas and

operations – Providing explanations for mathematical ideas

and procedures– Appraising unusual student methods, claims,

or solutions



16

Representing Number ConceptsRepresenting Number Concepts

Mrs. Johnson thinks it is important to vary the whole when she teaches fractions. For example, she might use five dollars to be the whole, or ten students, or a single rectangle. On one particular day, she uses as the whole a picture of two pizzas. What fraction of the two pizzas is she illustrating below? (Mark ONE answer.)a) 5/4 b) 5/3 c) 5/8 d) 1/4



17

Providing Mathematical Explanations: Divisibility Rules

Providing Mathematical Explanations: Divisibility Rules

Ms. Harris was working with her class on divisibility rules. She told her class that a number is divisible by 4 if and only if the last two digits of the number are divisible by 4. One of her students asked her why the rule for 4 worked. She asked the other students if they could come up with a reason, and several possible reasons were proposed. Which of the following statements comes closest to explaining the reason for the divisibility rule for 4? (Mark ONE answer.)

a) Four is an even number, and odd numbers are not divisible by even numbers.

b) The number 100 is divisible by 4 (and also 1000, 10,000, etc.).

c) Every other even number is divisible by 4, for example, 24 and 28 but not 26.

d) It only works when the sum of the last two digits is an even number.



18

Student A Student B Student C

x32

55 x

32

55 x

3255

+17

25

5+1770

50 1

2550

875+

16

0000

875

875

Which of these students is using a method thatcould be used to multiply any two whole numbers?

Appraising Unusual Student Solutions

Appraising Unusual Student Solutions



19

Common vs. Specialized CKCommon vs. Specialized CK

• Appears in exploratory factor analyses on 2/7 forms; confirmatory on 3/7

• Individuals can be strong in common but not specialized; vice versa

• Suggests there is professional knowledge for teaching



20

Ongoing WorkOngoing Work

• Item and measures development – Middle school national probability study (KCS,

KCT)– Data analysis and probability– PFA equating

• Validation efforts– “Videotape” study– Cognitive tracing studies– Content validity checks

Some Sample Results



22

An Example: Establishing a Relationship to

Student Growth

An Example: Establishing a Relationship to

Student Growth



23

Links to Study of Instructional Improvement Student Achievement

Analysis

Links to Study of Instructional Improvement Student Achievement

Analysis• SII CKT-M measure – 38 items

– SII: .89 IRT reliability

• Model: Student Terra Nova gains predicted by:– Student descriptors (family SES, absence rate)– Teacher characteristics (math methods/content,

content knowledge)

• Teacher content knowledge significant– Small effect (LT 1/10 standard deviation)– But student SES is also on same order of

magnitude



24

A Second Example: Evaluating Teacher Professional

Development

A Second Example: Evaluating Teacher Professional

Development



25

Tracking Teacher GrowthTracking Teacher Growth

• Items piloted in California’s Mathematics Professional Development Institutes (MPDI)– Instructors: Mathematicians and mathematics

educators – 40-120 hours of professional development– Focus is squarely on mathematics content– Summer 2001– Pre/post assessment format (parallel forms)



26

MPDI Teacher Growth (Year 1)

MPDI Teacher Growth (Year 1)

• For all institutes for which we have data, teachers gained .48 logits, or roughly ½ standard deviation

• Translates to 2-3 item increase on assessment

• Considered substantial gain

0

0.2

0.4

0.6

0.8

1

1.2

All institutes

Pre-test

Post-test



27

Results from Sample Institutes

Results from Sample Institutes

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

MPDI I MPDI II MPDI III MPDI IV MPDI V

Pre

Post



28

MPDI Evaluation: Other Findings

MPDI Evaluation: Other Findings

• Length of institute predicts teacher gains– 120-hour institutes most effective, on average– But some 40-hour institutes very effective

• Focus on mathematical analysis, proof, and communication leads to higher gains

• Many questions remain– Effects of content (e.g., mathematics vs.

student thinking)– Treatment of content: common vs. specialized– Effects of teacher motivation

LMT/SII Measures and Dissemination



30

Current Item PoolCurrent Item Pool

• Elementary School (K-6) – Number and operations / Knowledge of

content – Number and operations/ Knowledge of

content and students – Patterns functions and Algebra/




31

Current Item PoolCurrent Item Pool

• Middle School – Number and operations / Knowledge of

content – Patterns functions and Algebra/


• Geometry (3-8)



32

Item Workshops and Dissemination

Item Workshops and Dissemination

• Interested users attend a one-day workshop in Ann Arbor

• We cover – History of item development – Analytic methods and validation studies – How to use technical materials

• Users get– Access to measures – Support materials



33

Dates and Contact Information

Dates and Contact Information

• Learning Mathematics for Teaching– http://sitemaker.umich.edu/lmt

• Dates for LMT Workshops – November 10, 2005– January 13, 2006 – Brenda Ely ([email protected])

• Geoffrey Phelps – [email protected]– 734-615-6076

www.sii.soe.umich.edu developing measures of mathematical knowledge for teaching geoffrey phelps,...

Documents

m3000 teachers

teachers ckt ck reliabilities

teacher educationto

items difficulty

teacher behaviorearly

single items unlikely

students kcs

students performance