Improving on the Recitation Section:

Tutorials in Introductory Physics

Wed Brown BagSJP Fa '05

What are Tutorials?

• Research-based• Student-centered• “Elicit-confront-resolve”• Process (discussion, consensus) + Sense-making

The Tutorial space (basement, across from Jerry Leigh)

Tutorials at CU• Used in 1110 & 1120, twice each (+ 1120, now)

• Part of successful courses => motivation to continue

• Today: some data Continue support?

Resources

• Space, material, training, personnel Ongoing support necessary.

• “Learning Assistants” (LA’s) - undergrad STEM majors (interested in teaching)

Courses studied so far Recitation Homework Text:

• Phys 1110 – Fa 03 Tutorials CAPA HRW– Sp 04 Tutorials CAPA HRW– Fa 04 Knight workbook MP Knight– Sp 05 Trad recitations MP Knight

• Phys 1120–Fa 04 Tutorials CAPA HRW–Sp 05 Tutorials CAPA Knight

pre/post measures

• Phys 1110: FMCE• Phys 1120: BEMA• common exam questions• formal + informal survey questions

Phys 1110: Distinguishing features

1: “Tut” (Sp04) Tutorials + CAPA

2: “Workbook” (Fa04) Small groups/Knight (+ Mastering Physics)

3: “(More) Trad” (Sp05) Mostly traditional recit’s (Otherwise much like "2" above)

(different instructors, semesters …)

1110 summary - up front!• Tutorial courses : strongest results on

learning gains and attitude surveys

• Middle course (“2”) (research-based text and hw, clickers + small-group recitations, not Tutorials): good results.

• Last course (like “2” except recitations): gains solid, lowest of three.

Phys 1110 PretestFMCE (Force and Motion concept evaluation)

A nationally validated conceptual exam, like the FCI (but harder)Comparing FMCE pretest

0

510

1520

2530

35

40

0 9 18 27 36 45 55 64 73 82 91 100

Score (%)

% of students

1PreS04 (28% +/-1) 2PreF04 (34%) 3PreS05 (28% )

Matched, valid data only. (Eliminate pre>93%) Ave

Pre ave = 28%(1) / 34%(2) / 28%(3) Spring (2) higher (fall term)

Phys 1110 Posttest

Comparing FMCE posttest

0

510

15

20

2530

35

40

0 9 18 27 36 45 55 64 73 82 91 100Score (%)

% of students

1PreS04 (74% +/-1) 2PreF04 (69%) 3PreS05 (58% +/- 2%)

Ave

Post ave = 74%(1) / 69%(2) / 58%(3)

Phys 1110 normalized gains

gain(1)= .66 +/-.02

distribution of g: compared

05

1015202530

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1normalized gain

% of students

SP04

Phys 1110 normalized gains

gain(1)= .66 +/-.02 g(2)= .585 +/-.02 7.5 points lower => (more than half a letter grade)

distribution of g: compared

05

1015202530

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1normalized gain

% of students

SP04FA04

Phys 1110 normalized gains

gain(1)= .66 +/-.02 g(2)= .585 +/-.02 g(3)= .45 +/-.02(trad recit.) => significantly lower gains. (still, double nat’l standards!)

distribution of g: compared

05

1015202530

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1normalized gain

% of students

SP04SP05

Impact on different pretest populations:"low starters" pretest <=12.5%

normalized gain for low pretest

0

0.1

0.2

0.3

0.4

0.5

0.6

S04 (23%) F04 (20%) S05 (22%)Semester (% of class in this

g

pretest <= 12.5%

(% of class in this pool)

Course (1) (2) (3)

Impact on different pretest populations:"high starters" 50<pre<93%

normalized gain for high pretest

0

0.1

0.20.3

0.4

0.5

0.60.7

0.8

0.9

S04 (13%) F04 (22%) S05 (14%)Semester (% of class in this

g

pretest >50%

(% of class in this pool)

Course (1) (2) (3)

Beyond the FMCE: Exam comparisons

• #2 (Knight workbooks/small groups): 34 common exam q’s

• #3 (Trad recitations): 30 common q’s

(17 are “tutor. materials”, 9 are “quant/trad”)

(12 are “tutor. materials” 6 are “quant/trad”)

Beyond the FMCE: Exam comparisons( Tutorial score - other course score)

All q’ Tut mater. quant/trad(2) Workbook: +6 +7 +6(3) Trad: +10 +14 +10

(All ± 2) N.B. 12 points is roughly 1 letter grade.

Tutorial courses: significantly better exam scores:both conceptual and quantitative/traditional.

Other data• Replication study (compare with UW)• CLASS (attitudes/beliefs)• Surveys (did you like, did it help?)

All favorable (or neutral) for tutorials, ask for details!

1110 Summary• Compare Tut-based with “workbook/small group” measurable diffs (FMCE, exams, CLASS)

• Compare Tut-based with “more trad rec” significant diffs.

• Tutorials only one effect. (Instructors, course structure …!?) • But in 1120, changing instructors + text => no impact

Phys 1120:

Tut1 (Fa04) and Tut2 (Sp05)Attempt at replication.Main differences: Tut2 has… • different instructor• different semester• different textbook• follows up 1110 without Tutorials• no long answer on exams

Summary (up front!)

• Despite changes in course elements, we replicated Tutorials + basic course structure.

• Result: identical (high) learning gains. (Final score matches our juniors.

Hard exam!)

1120 BEMA pre/post

BEMA = “Brief E&M Assessment”, F04 (N=319) Pretest ave 26%

BEMA (matched) (CU scoring) Fa04

02468

101214161820

0 6 12 18 24 30 36 42 48 55 61 67 73 79 85 91 97Score (%) (CU scoring)

% of students

PreF04

1120 BEMA pre/post

F04 (N=319) Pretest: 26% S05 (N=232): 27%

BEMA (matched) (CU scoring) Compare Fa04 and Sp05

02468

101214161820

0 6 12 18 24 30 36 42 48 55 61 67 73 79 85 91 97Score (%) (CU scoring)

% of students

PreF04 PreS05

1120 BEMA pre/post

F04 (N=319) 26% -> 59%, S05 (N=232) 27% -> 59%g(ave, F04) = .44+/- .01 g(ave, S05)=.43+/- .01

BEMA (matched) (CU scoring) Compare Fa04 and Sp05

02468

101214161820

0 6 12 18 24 30 36 42 48 55 61 67 73 79 85 91 97Score (%) (CU scoring)

% of students

PostF04 PostS05

1120 E&M pre/postBEMA (matched) (CU scoring) Fa04

g(ave)=.44, N=331

05

101520253035404550

0 6 12 18 24 30 36 42 48 54 60 66 72 78 84 90 96Score (%) (CU scoring)

# of students

Pre (ave=26)Post (ave=59)

LA pre

LA post

TA post

TA pre

CU upp div post

1120 post

NCSU tradpost

NCSU honor& CMU tradpost

CMU M&Ipost

CMU uppdiv post

Other data

• Common exam questions (no change)• Replication study (compare with UW)• CLASS (attitudes/beliefs)• Surveys (did you like, did it help?)

All results similar for both semesters!

Affect: survey results

1120 (Tut2) followed 1110 using Knight “workbook” small-group recitations.

Asked at end:

Which is better, Tut or trad rec? 39% vs 40%Which would you enjoy more? 39% vs 39%

1120 mini-summaryDifferent instructors, text, exam structures,

semesters… same Tutorials and Conceptests: no sig diff on BEMA, CLASS, exams.

Validated survey scores high.

Slight differences in surveys: ~neutral student satisfaction. No disasters, room to improve

Bottom line

Tutorials successful, productive course elements.

• Cost:$1500/LA/semester * (6-8 LA’s per course)• Need 1.5 hr/week training session (TA’s too!)

• Benefits: Some LA’s => K-12 cert. (+ their learning gains very high)• TA’s exposed to research-based learning.

Recommendation

• We should continue implementing Tutorials (and collecting data)

• Need support for LA’s, and training infrastructure

(=> faculty or experienced grad student assigned to teach the TA’s/LA’s)

Questions, Discussion!

THE END

Replication

UW (no Tut)

UW (with Tut)

CU (with Tut)

Atwood: tension

25 50 55

Identify Newton III partners

15 70 70

McDermott et al., AJP 62, 1994

Replication

UW (no Tut)

UW (with Tut)

CU (with Tut)

Atwood: tension

25 50 55

Identify Newton III partners

15 70 70

1110 Overall CLASS

55

75

12 22% unfavorable

% favorable

1-Tut

3-Trad

2-Workbook

“Beliefs” survey: CLASS pre/post

1110 Overall CLASS

0

20

40

60

80

100

0 20 40 60 80 100% unfavorable

% favorable

CLASS pre/post (full scale)

Tutorial-based course: no shiftTwo others: small but statistically significant declines

CLASS pre/post1120 Overall CLASS

60

70

10 15 20% unfavorable

% favorable

Sp05Tut1

Tut2

pre- to post shifts (attitudes and beliefs survey). Slightly negative (!), though it’s usually worse.

CLASS pre/post1120 Overall CLASS

0

20

40

60

80

100

0 20 40 60 80 100% unfavorable

% favorable

(Shown on full 0-100 scales)

Affect: survey results 1110(Likert scale 1= “no” to 5=“yes”)

• Did the Tutorial help you learn Physics in 1110? 3.53 +/- .05 (20% neg, 60% pos)

• Did you like the Tutorials?2.41 +/- .05 (62% neg, 21% pos)

Affect: survey results 1120Likert scale 1-5: “very bad” to “very good”)

• How do you feel about the use of Tutorials in Phys 1120?

Tut1: 3.32 +/- .07Tut2: 2.96 +/- .09

(33% neg, 53% pos)(46% neg, 43% pos)

• How much help was the Tutorial?Tut1: 3.33 +/- .06Tut2: 2.90 +/- .08

Force Concept Inventory (FCI)

R. Hake, ”…A six-thousand-student survey…” AJP 66, 64-74 (‘98).

<g> = post-pre 100-preFa03/Sp04Fa98

red = trad, blue = interactive engagement

FCI at CU

Beyond the BEMA - exam q’s• 31 common exam q’s (13 explicitly “tutorial material”, rest => other topics)

All Tut material only(Fa04-Sp05) -1.9% ± 2% -1.8%

~40%

~75%

* Mazur, E. Peer Instruction, Prentice Hall