Talks2009-10 End of Year Event

Overview of CWSEI progress (lots of data!)– Carl WiemanImproving Student Study Habits: results of interventionsSara Harris & Louis DeslauriersInteractive Engagement: examples from UBC classes (video)Sarah Gilbert & department members 

Poster session 11‐1:30 room 101Details on everything being done and learnedWorkshop & Discussion 1:30 – 3:00pm, room 101 – How to Most Effectively Measure the Learning that Matters (workshop led by Carl Wieman)g ( p y )3:15 – 4:30pm, room 101 – Incorporating Writing in the Science Curriculum; what and how? (discussion)

CWSEI “Trinity” for each course

1st: Learning  goals. (whatshould students be able to do?)

What shouldstudents

learn?

What arestudents learning?

What shouldstudents

learn?

What arestudents learning?

should students be able to do?)

2nd: Good assessment Which instructionalh

Which instructionalh

(validated tests)

3rd: Improved teaching methods

approachesimprovestudent

learning?

approachesimprovestudent

learning?

3 :  Improved teaching methods (research based, improve learning)

Materials assessment tools homework notesMaterials, assessment tools, homework, notes … saved, reused, improved.  

M ki t hi ff ti d di fMaking teaching more effective, and more rewarding forfaculty and students

Carl Wieman Science Education InitiativeStarted 3 ears ago idespread impro ement in scienceStarted 3 years ago ⇒ widespread improvement in science education. Departments at various scales and levels of maturityLarge scale mature-- Earth and Ocean SciencesLarge scale younger-- Physics and Astronomy

Computer ScienceComputer ScienceMath

Smaller scale programs -- Chemistry Statistics LifeSmaller scale programs Chemistry, Statistics, Life Sciences

$2 M gift from David Cheriton for math and comp sci$2 M gift from David Cheriton for math and comp. sci.

Today--focus on data

1. How many courses/faculty transformed?

2 How much better is the learning?2. How much better is the learning?a. learningb. engagementc. innovative problem solving

3. But does it stay learned? (retention)3. But does it stay learned? (retention)

4. Reaching all students. Turning low performers into high

5. Blizzard of data on improvement from across the departments (appetizer for posters)

1. How widespread is the change-- EOS, most mature, full 3 year effortfull 3 year effort.

24 courses transformed.18 with formal CWSEI supportpp6 with strong informal and moral support

~ 26 faculty involved

typical new things•clearly articulated learning goals for students and facultyfaculty•pre-reading assignments & quizzes•clicker questions and peer discussion•worksheets & in-class group activities•group exams•team projectsteam projects•pre-post testing to measure learning, ...much more active learning and feedback,

EOSC 111:  Laboratory Exploration of Planet EarthS i

‐ Completed‐ First yr Lab

00

‐ All hands on, ‐ lots of group work, i di id l dSara Harris ‐ approx 100 per semester  ‐ individual and group 

quizzesEOSC 112:  The Fluid Earth: Atmosphere and OceanSara Harris, Roger 

‐ been through about 2 years of EOS‐SEI‐ service course for anyone 

‐ clickers ‐ online quizzes ‐ article readings, quizzed,  , g

Francois, William Hsiehy

at UBC‐ about 350/year (split between 2 sections)

g , q ,with feedback (rubrics)

EOSC 114:  Natural Disasters

‐ Completed June’081st year exploratory

Clickerson line assignmentsDisasters

R. Stull and many others‐ 1st year exploratory course‐ over 1000 stu. per year

on‐line assignments

EOSC 210:  Earth Science for Engineers

‐ CompletedLecture and lab

Clicker Qs, in each lecture. Activities and discussionsfor Engineers 

Erik Eberhardt, Uli Mayer, Stuart Sutherland

‐ Lecture and lab‐ 230 each September

Activities and discussions in most lectures. Labs with group work and hands on activities

EOSC 211:  Computer  ‐ Second teaching term  In‐class worksheets, pair‐Methods in the Earth, Ocean and Atmospheric SciencesRichard Pawlowicz, Catherine Johnson

Sept. 2010‐ 2nd year programming course lecture/lab‐ 55 students enrolled last term

programming, name‐sticks, pair and small group discussions, class discussions

Catherine Johnson term

EOSC 212:  Topics in Earth and Planetary SciencesM. Bostock, M. Jellinek

‐ Completed June ‘09‐ 2nd year “science thinking” course‐ 20 to 40 students per 

‐ team‐based quizzes and inclass activities & discussions‐ article reading and p

yearg

question posing workshop style classes‐ peer assessed presentations & posters

EOSC 220: Introduction to Complete 3x5 cards used to answerEOSC 220:  Introduction to MineralogyMary Lou Bevier

‐ Complete ‐mandatory intro. lab course for EOS students ‐ 120 students enrolled

3x5 cards used to answer questions in class, in‐class activities, class discussion, labs have group work and group quizzes

EOSC 221:  Petrology Maya Kopylova

‐ Completed‐ Lecture and lab‐ 100 each January

Wake up exercises (integrating activities into each lecture), some 3x5cards, labs with group work and hands on somework and hands on, some "authentic activity" labs

EOSC 223:  Field techniques Mary Lou Bevier

‐Minor support summer 2009‐ Lectures and Field 

lectures have regular activities and 3x5 cards to get feedback, 

component Field activitiesEOSC 252:  Physics of geologic materialsF. Herrmann

‐ First teach term completed‐ 2nd yr “physics” course‐ 20 – 30 students each

‐ lab exercises‐ in‐class demonstrations with worksheets‐ aiming for interactive20  30 students each 

yearaiming for interactive 

lecturing next yr.

etc. for 3 more pages

2. But do these changes improve student outcomes?(learning, engagement, ...)Hard to tell in most courses because no pre-transform datadata.

Data from example courses where similar transformations, d d t f d t t f d tand good pre transform and post transform data.

Louis Deslauriers and Ellen Schelew (physics)--- cleanest comparison study of teaching methods ever done. Will be landmark in science education researchWill be landmark in science education research (as soon as they write it up for publication)

new-- Louis Deslauriers (PD) and Ellen Schelew (grad std) Perfect comparison of teaching methods: identical sections (260 each), intro phys. 153, same material & time.

___I___________ Experienced highly rated instructor-- trad. lecture & ~2 cl.

_____II_________ Experienced highly rated instructor--trad. lecture & ~2 cl.instructor trad. lecture & 2 cl.

questions

same preparationsame attendance

instructor trad. lecture & 2 cl.questions

wk 1-11wk 1-11same attendance

same engagementsame midterm 1 & 2 grades

elect-mag waves elect mag waves

Wk 12-- competition

elect-mag wavesLouis and Ellen (inexper.)research based teaching

elect-mag wavesregular instructorintently prepared lecture

common exam on EM waves common exam on EM waves

transformed section

•pre class reading assignments with quizzes•pre-class reading assignments with quizzes•in-class small group activities•clicker questions with student-student discussion•targeted instructor feedback guided by observationsof student thinking

ResultsII. Trad I. Transformed.

1. Attendance pre 58% 58% (wk 10 & 11)during 58 % 81%

2. Engagement(back ½ room)

pre 50% 50% (wk 10 & 11)

during 50 % 85%

3. Learning (test) 41(1)% 74(1) %above guess (23%) 18% 51%

trad18

trans51

g ( )S. D. = 13%

trad. ⇒0.58 x 0.5 = 29% engaged 18ad ⇒0 58 0 5 9% e gagedfor above average instructortrans. ⇒0.81 x 0.85 = 69% engaged

other things practiced: scientific discourse, critiquing scientific arguments, sense-making, collaboration.

But how did students feel about it?

“Q1 I really enjoyed the interactive teaching technique during Q1. I really enjoyed the interactive teaching technique during the three lectures on E&M waves (Ch32).”

635760

70

en

ts

1220

30

40

50

um

ber

of

stu

de

2 00

10

Strongly agree

Agree Neutral Disagree Strongly disagree

Nu

stronglyagree

Q2 I feel I would have learned more if the whole phys153 course would have been taught in this highly interactive style.

67

60

70

80

nts

36

2120

30

40

50

60

mb

er

of

stu

de

82

0

10

20

Strongly agree

Agree Neutral Disagree Strongly disagree

Nu

m

Q6 I found the pre-reading to be very helpful to my learning:66

60

70

s

35

2830

40

50

ber

of

stu

den

ts

3 10

10

20

Strongly Agree Neutral Disagree Strongly

Nu

mb

g yagree

g g g ydisagree

Q5 What contributed most to my learning during these three lecture on E&M waves:

22

39

51

30

40

50

60

r o

f st

ud

en

ts

138

0

10

20

Trying to figure out the answer

Trying to work out the answers

The instructor explanation to

The pre-reading The pre-reading quiz

Nu

mb

er

pre-d

in-classclicker instruc.l

pre-readout the answer

to clicker questions

out the answers to the in-class

activities

explanation to the clicker

questions or in-class activities

quizreadingactivityquest. explan.

to c.q. oractiv.

quiz

Q8 In class, the group discussions with my neighbors were very helpful to my learning:

70

60

70

80

s

3340

50

ber

os

stu

den

ts

22

710

20

30

Nu

mb

00

Strongly agree Agree Neutral Disagree Strongly disagree

h d h l l k l kWhat does such a class look like?

See upcoming video clips session. p g p

Measuring student (dis)engagement. Erin LaneWatch random sample group (10-15 students). Check

i t li t f di t b h i h 2 iagainst list of disengagement behaviors each 2 min.

time (minutes)time (minutes)

What about advanced upper division courses?

Ph i 408 d d tiPhysics 408-- advanced optics

Taught by same instructor for several years--ti ll ki t i continually working to improve.

He radically transformed this year.Ended up covering same material in less time.

Midterm exam grades: P t f ti (l t ) 56 +/ 3 1% Pre transformation (lecture) 56 +/-3.1% Post transformation 77 %

(Exams different but three experts did blind rating of the (Exams different, but three experts did blind rating of the exams. All concluded post transformation exam more difficult)

What about learning to think more innovatively?g yLearning to solve challenging novel problems

Jared Taylor and George Spiegelman

“Invention activities” practice coming up withInvention activities -- practice coming up withmechanisms to solve a complex novel problem.Analogous to mechanism in cell.

2008-9-- randomly chosen groups of 30, 8 hours ofinvention activities.This year, run in lecture with 300 students. 8 times per term. (video clip)

Plausible mechanisms for biological process student neverencountered before

Average Number

5.0

6.0

4.0

olut

ions

3.0

mbe

r of S

o

1.0

2.0Num

0.0

1.0

Control Structured Inventions (Outside Inventions (DuringControl StructuredProblems (tutorial)

Inventions (Outsideof Lecture)

Inventions (DuringLecture)

Average Time to First Solution Thread

12.0

14.0

10.0

n)

6.0

8.0

Tim

e (m

in

4.0

0.0

2.0

C t l SPSA (O t id f IA (O t id f IA (D i L t )Control SPSA (Outside ofLecture)

IA (Outside ofLecture)

IA (During Lecture)

3. So research based teaching achieves much better learning & much greater engagement.

Does it stay learned?y(retention)

3. Mastery of quantum mechanics concepts-short & long term Deslauriers & Wieman to be published

score on quantum mech. concept survey

88 8590

100

%)

q p y

68 65

88 85

70

80

ey S

core

(%

interactive engagement/practice

68 65

50

60

nce

pt

Surv

e

superb traditionallecturer

30

40

Con

0 5 10 15 20

Retention interval (Months)

4. Bringing up the bottom of the distributionDeslauriers, Lane, Harris, Wieman

“What do I do with the weakest students? Are they just hopeless, or is there anything I can do to make a difference?”

a. To get such big improvements in average, have to impact entire distribution

b. Data on how to transform lowest performing studentsinto medium and high.

Intervened with bottom 25% of students after midterm 1.

•Phys250 (engphys program, high selective and demanding),y ( gp y p g , g g),bottom 25% averaged +20% improvement on midterm 2!

•EOS climate science course. Very broad range of students.• Averaged +30% improvement!

interventionnonintervention

80

90

100

intervention100

40

50

60

70

ent e

xam scores

scor

e

10

20

30

40

stud M1

MIDTERM 2

0 %

midterm 1 score

0

0 20 40 60 80 100 120 140 160 180student number0 %

student number

What magic does this?,& X, midterm 2 score

What magic does this?Listen to next talk.

All UBC i t d t b f l•~All UBC science students can be successful•A little help on how to learn goes a long way

Large scale survey (~ 600) and interviews on factors that UBC science students perceive as affecting academic

Ashley Welsh

UBC science students perceive as affecting academic performance

An early finding•Students overwhelmingly recognize they do not know how to study effectively Is seen as major barrier to success but findstudy effectively. Is seen as major barrier to success, but find little help in learning how to study.

masses of other datamasses of other datawill overwhelm you with blizzard of info

Go to posters to get details and more resultsGo to posters to get details and more results

Math 152 - Assessment ofMath 152 Assessment of Matlab “for” loop mastery

60%70% Correct

Incorrect

30%40%50%

IncorrectBlank

10%20%30%

0%2008 2009

Math 184--intro calculus “workshops” part of courseintro calculus workshops part of course.Last year collected data on how they were functioning, (observations, surveys, examine correlation of student marks with numerous factors.)

This year, made changes based on the data.s yea , ade c a ges based o t e data

Math 184 Workshops –Correlation between workshop attendance and course gradesg

Pearson Correlation Coeffficient

0.40

0.50

0.60relevant # is (corr. coeff.)2

0.20

0.30

0.40

r

0.00

0.10

2008 2009

Math 184 WorkshopsMath 184 Workshops Student Survey

"… provide useful practice for

The workshop problems ….

"… are related to material coveredp psolving problems on tests"

40%

50%2008

2009

… are related to material covered in class"

40%

50%2008

2009

20%

30%

20%

30%

2009

0%

10%

Strongly Agree Neutral Disagree Strongly0%

10%

Strongly Agree Neutral Disagree StronglyStronglyAgree

Agree Neutral Disagree StronglyDisagree

StronglyAgree

Agree Neutral Disagree StronglyDisagree

EOSC 211: Computer Methods in the Earth Sciences

Introduced technique of “Pair-Programming” from comp sci ed research:Compared to previous year:• Labs are completed about 15% fasterp• Lab marks are about 10% higher• Students are MUCH happier with the

t f dtransformed course

31

EOS Impact of TA training program

10

12

6. I consider myself to be an effective teacher.

2

4

6

8

Pre

Post2008

0

2

Agree Neutral Disagree

6. I consider myself to be an

60708090

yeffective teacher.

2009

102030405060

Pre

Post

2009

0Agree Neutral Disagree

12

ts

Pre & post‐test scores pre

EOS 212-- using model based reasoning

468

10

er  of stude

nt post

02

90‐100 80‐89 70‐79 60‐69 50‐59 <49

Num

be

score (%) bins( )

70%80%90%

100%

MUCH more time on EOSC212

EOS collecting data

10%20%30%40%50%60%70% EOSC212

a LITTLE more time on EOSC212

Roughly equal time on EOSC212.

on time students studying in courses.

0%10%

a LITTLE  less time on EOSC212.

MUCH less time on EOSC212.

Relative amount of time for different

icourses across sci.

Phys 109 & Sci 1 Intro physics lab“invention” activity to develop scientific reasoninginvention activity to develop scientific reasoning

before activity s

1.0

yafter activity

use standarduse histogramsstud

ents

deviation

ctio

n of

s

0student characterization of data

frac

see posters to learn more about these and many more

Conclusions

1. It is possible to make widespread transformationin UBC science teaching-- many courses, many faculty.

2. CWSEI transformations lead to•much greater engagement,•much greater learning•much greater learning,•happier students.

Looking forward to great progress in coming year

third year quantum mechanics course--

Common questions on QM spinpre-transform 2009 final exam 68%+/-3%pre transform 2009 final exam 68%+/ 3%

2010 midterm 76%+/-2%( t h lf h ti t i )(spent half as much time on topic)

physics lab diagnostic measurementsphysics lab diagnostic measurementsshowing improvements, but more work needed

EOSC 211 Lab marks

39