http:// t 1 developing measures of mathematical knowledge for teaching geoffrey phelps, heather...

http://www.sii.soe.umich.edu/

http://sitemaker.umich.edu/lmt

1

Developing Measures of Mathematical Knowledge for

Teaching

Developing Measures of Mathematical Knowledge for

Teaching

Geoffrey Phelps, Heather Hill, Deborah Loewenberg Ball, Hyman Bass

Learning Mathematics for TeachingStudy of Instructional Improvement

Consortium for Policy Research in EducationUniversity of Michigan

MSP Regional ConferenceBoston, MA

March 30-31, 2006



2

Overview of today’s session Overview of today’s session

1. LMT/SII Measures Development

2. Some Sample Results

3. LMT/SII Measures and Dissemination



3

Subtract:

What is “Content Knowledge for Teaching”? An Example From

Subtraction

What is “Content Knowledge for Teaching”? An Example From

Subtraction

3002

783-



4

Analyzing Student ErrorsAnalyzing Student Errors

3002

783-

2781

3002 - 783 = 4832



5

Analyzing Unusual Student SolutionsAnalyzing Unusual Student Solutions

3002

783-

299

2219

12 3 0 0 2

7 8 3-

3-7-8-1

2 2 1 9

LMT/SII Measures Development



7

Why Would We Want to “Measure” Teachers’ Content Knowledge for

Teaching?

Why Would We Want to “Measure” Teachers’ Content Knowledge for

Teaching?

• To understand “what” constitutes mathematical knowledge for teaching

• To understand the role of teachers’ content knowledge in students’ performance

• To study and compare outcomes of professional development and teacher education

• To inform design of teachers’ opportunities to learn content knowledge



8

Measuring Teachers’ Mathematics Knowledge: Background and History

Measuring Teachers’ Mathematics Knowledge: Background and History

• Research on teacher behavior• Early research on student achievement

– Proxy measures for teacher knowledge– Tests of basic skills

• 1985 on: “the missing paradigm” pedagogical content knowledge or PCK

• 1990s: interview studies of teachers’ mathematics knowledge (MSU -- NCRTE)



9

Study of Instructional Improvement

Study of Instructional Improvement

• Study of three Comprehensive School Reforms; teacher knowledge a key variable

• Instrument development goals:– Develop measures of content knowledge

teachers use in teaching • K-6 content for elementary school teachers• Not just what they teach - specialized knowledge

– Develop measures that discriminate among teachers (not criterion referenced)

– Non-ideological

• But we faced significant problems….



10

Problems As We Began This Work

Problems As We Began This Work

• No way to measure teachers’ content knowledge for teaching on a large scale– Small number of items, many written by Ball,

Post, others appeared on every instrument– Nothing known about the statistical qualities

of those items (difficulty, reliability)– Studies relied on single items, yet single

items unlikely valid or reliable measures of teacher knowledge



11

Early Decisions and ActivityEarly Decisions and Activity

• Survey-based measure of CKT-M– 3000 teachers participating in SII– Multiple choice

• Specified domain map

• 5 people + 5 lbs cheese + 5 weeks = 150 items (May 2001)

• Large-scale piloting, summer 2001



12

Early Decisions and ActivityEarly Decisions and Activity

Types of knowledge

Math

em

atica

l co

nte

nt

Content knowledge

Knowledge of content

and students

Number

Operations

Patterns, functions, and algebra



13

Early Analyses and Validity Checks

Early Analyses and Validity Checks

• Results from piloting – We can measure teachers’ CKT-M – Reliabilities of .70-.90– Factor analysis shows distinct types of

knowledge • Knowledge of content and students (KCS)

separate from CK• Specialized content knowledge (SCK) vs.

common content knowledge (CCK) Hill, H.C., Schilling, S.G., & Ball, D.L. (2004) Developing measures of

teachers’ mathematics knowledge for teaching. Elementary School Journal 105, 11-30.



14

Reliabilities (1PL-IRT): Elementary

Reliabilities (1PL-IRT): Elementary


Knowledge of content and

students

Number and operations (K-6) .72-.81 .58-.67

Patterns, functions, and algebra (K-6) .70-.85

Geometry (3-8) .85-.86



15

Reliabilities (1PL-IRT): Middle School

Reliabilities (1PL-IRT): Middle School


Knowledge of content and

students

Number and operations (5-9) .74-.75

Patterns, functions, and algebra (5-9) .86-.89

Geometry (3-8) .84-.86



16

Content Knowledge :Number and Operations

Content Knowledge :Number and Operations

• Common knowledge– Number halfway between 1.1 and 1.11

• Specialized knowledge– Representing mathematical ideas and

operations – Providing explanations for mathematical ideas

and procedures– Appraising unusual student methods, claims,

or solutions



17

Representing Number ConceptsRepresenting Number Concepts

Mrs. Johnson thinks it is important to vary the whole when she teaches fractions. For example, she might use five dollars to be the whole, or ten students, or a single rectangle. On one particular day, she uses as the whole a picture of two pizzas. What fraction of the two pizzas is she illustrating below? (Mark ONE answer.)a) 5/4 b) 5/3 c) 5/8 d) 1/4



18

Providing Mathematical Explanations: Divisibility Rules

Providing Mathematical Explanations: Divisibility Rules

Ms. Harris was working with her class on divisibility rules. She told her class that a number is divisible by 4 if and only if the last two digits of the number are divisible by 4. One of her students asked her why the rule for 4 worked. She asked the other students if they could come up with a reason, and several possible reasons were proposed. Which of the following statements comes closest to explaining the reason for the divisibility rule for 4? (Mark ONE answer.)

a) Four is an even number, and odd numbers are not divisible by even numbers.

b) The number 100 is divisible by 4 (and also 1000, 10,000, etc.).

c) Every other even number is divisible by 4, for example, 24 and 28 but not 26.

d) It only works when the sum of the last two digits is an even number.



19

Student A Student B Student C

x32

55 x

32

55 x

3255

+17

25

5+1770

50 1

2550

875+

16

0000

875

875

Which of these students is using a method thatcould be used to multiply any two whole numbers?

Appraising Unusual Student Solutions

Appraising Unusual Student Solutions



20

Common vs. Specialized CKCommon vs. Specialized CK

• Appears in exploratory factor analyses on 2/7 forms; confirmatory on 3/7

• Individuals can be strong in common but not specialized; vice versa

• Support from cognitive interviews of mathematicians

• Suggests there is professional knowledge for teaching



21

Ongoing WorkOngoing Work

• Item and measures development – Middle school national probability study– Develop new measurement modules for

data analysis and for probability• Validation efforts

– “Videotape” study– Cognitive tracing studies– Content validity checks

Some Sample Results



23

An Example: Establishing a Relationship to

Student Growth

An Example: Establishing a Relationship to

Student Growth



24

Links to Study of Instructional Improvement Student Achievement

Analysis

Links to Study of Instructional Improvement Student Achievement

Analysis• SII CKT-M measure – 38 items

– SII: .89 IRT reliability• Model: Student Terra Nova gains

predicted by:– Student descriptors (family SES, absence rate)– Teacher characteristics (math

methods/content, content knowledge) • Teacher content knowledge significant

– Small effect (LT 1/10 standard deviation)– But student SES is also on same order of

magnitude Hill, H.C., Rowan, B., & Ball, D.L. (2005) Effects of teachers'

mathematical knowledge for teaching on student achievement. American Educational Research Journal 42, 371-406.



25

A Second Example: Evaluating Teacher Professional

Development

A Second Example: Evaluating Teacher Professional

Development



26

Tracking Teacher GrowthTracking Teacher Growth

• Items piloted in California’s Mathematics Professional Development Institutes (MPDI)– Instructors: Mathematicians and mathematics

educators – 40-120 hours of professional development– Focus is squarely on mathematics content– Summer 2001– Pre/post assessment format (parallel forms)

Hill, H. C. & Ball, D. L. (2004) Learning mathematics for teaching: Results from California’s Mathematics Professional Development Institutes. Journal of Research in Mathematics Education 35, 330-351.



27

MPDI Teacher Growth (Year 1)

MPDI Teacher Growth (Year 1)

• For all institutes for which we have data, teachers gained .48 logits, or roughly ½ standard deviation

• Translates to 2-3 item increase on assessment

• Considered substantial gain

0

0.2

0.4

0.6

0.8

1

1.2

All institutes

Pre-test

Post-test



28

Results from Sample Institutes

Results from Sample Institutes

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

MPDI I MPDI II MPDI III MPDI IV MPDI V

Pre

Post



29

MPDI Evaluation: Other Findings

MPDI Evaluation: Other Findings

• Length of institute predicts teacher gains– 120-hour institutes most effective, on average– But some 40-hour institutes very effective

• Focus on mathematical analysis, proof, and communication leads to higher gains

• Many questions remain– Effects of content (e.g., mathematics vs. student

thinking)– Treatment of content: common vs. specialized– Effects of teacher motivation– Long term learning from colleagues, curriculum,

practice

LMT/SII Measures and Dissemination



31

Current Item PoolCurrent Item Pool

• Equated forms for elementary school: – Number & operations / Content

knowledge (K-6) – Number & operations/ Knowledge of

content and students (K-6)– Patterns Functions & Algebra/ Content

knowledge (K-6)– Geometry (3-8)



32

Current Item PoolCurrent Item Pool

• Equated forms for middle school:– Number & operations / Content

knowledge (5-9)– Patterns Functions & Algebra/ Content

Knowledge (5-9)– Geometry (3-8)



33

Item Workshops and Dissemination

Item Workshops and Dissemination

• Interested users attend a one-day workshop in Ann Arbor

• We cover – History of item development – Analytic methods and validation studies – How to use technical materials

• Users get– Access to measures – Support materials



34

Dates and Contact Information

Dates and Contact Information

• Learning Mathematics for Teaching– http://sitemaker.umich.edu/lmt

• Dates for LMT Workshops – May 19, 2006– August 10, 2006 – Brenda Ely ([email protected])

• Geoffrey Phelps – [email protected]– 734-615-6076

http:// t 1 developing measures of mathematical knowledge for teaching geoffrey phelps, heather...

Documents

content knowledge slide

edulm t

specialized knowledge

ncrte slide

dissemination slide

subtraction slide

elementary school teachers

teacher education