What does research tell us about good teaching?

Liz Angstmann

Where are we heading?• We are all scientists, we should apply the scientific method to our teaching (or even

better to student learning).

• Look at how measurements are made.

• Look at a few of the main findings and then a few practical examples of how we can apply them.

• I shall describe my research into the effectiveness of online courses.

Who?• Physics Education Research people

– People who do serious research in this area

– People who apply this research to their teaching and take measurements to check its effectiveness

• Cognitive psychologists

• Education faculties

So how do we measure the effectiveness of courses or instructors?

Concept inventory tests

Taken from FMCE concept inventory test: Thornton, R. K. & Sokoloff, D. R. (1998). Assessing student learning of Newton’s laws!: The Force and Motion Conceptual Evaluation and the Evaluation of Active Learning Laboratory and Lecture Curricula. American Journal of Physics, 66(4), 338–352.

PhysPort – Supporting physics teaching with research-based resources https://www.physport.org/

How to analyse

Normalized gain =

<post test>�<pre test>full score�<pre test> ⇥ 100%

A number of criteria are usual:Do not include student results of students who score 100% (or close) on the

pretest.

Only include results from students who spend at least 10 minutes on the surveys

Ethics considerations: I only include students who give consent to be included

A bonus mark helps to get students to answer the surveys

Other methods:• Interviews

• Observations of students in classes

• Other measurements of time spent on task (eg. Length of time spent watching a video)

• “Happy sheets” such as CATEI and myExperience

What can we be fairly certain of?

Interactive engagement works!

Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American journal of Physics, 66(1), 64-74.

Large lecture example

https://goo.gl/forms/dB5xj1TyPjjoy6Ik1When the Jacob’s ladder is turned on its side what will

happen?

o The spark will form and move along the ladder as when it was vertical

o The spark will form and move along the ladder faster than when it was vertical

o The spark will form and move along the ladder slower than when it was vertical

o The spark will form and remain stationary

An example of Predict-Observe-Explain

Problem solving workshops• Get students to solve the problems themselves

• Requires some scaffolding

• Careful selection of questions: somewhat open ended, highlighting common misconceptions tend to work well

• OR student led tutorials tend to lead to more interaction, assign students to groups and make them responsible

Student preparation is key!Many studies have shown that when students come prepared

to class they learn a lot more. This holds for both labs and lectures.

Prelab activities

Video summarizes theory and shows the equipment

Quizzes worth a small percentage of course mark to make sure they learn the theory before the lab

UQ: 5 minute physicsA/Prof. Tim McIntyre, School of Mathematics and

Physics, UQ

Dr. Margaret Wegener, School of Mathematics and Physics, UQ

Dominic McGrath, Institute for Teaching and Learning Innovation, UQ

https://teaching.smp.uq.edu.au/fiveminutephysics/https://teaching.smp.uq.edu.au/scims/index.html

Be explicit with students about how they should prepare:• Let them know what chapters of the text book they should

read (the library can give you access to chapters online, for free)

• Tell them which concepts they need to be on top of from the last course for the lecture

There are big differences between how “Experts” and “Novices” think about material

7 ±# rule: The limits of short term memory

6, 1, 3, 7, 4, 0, 0, 2, 8, 6, 4, 6, 8, 7, 1, 0, 4, 2, 9, 8, 5, 6, 6, 6, 3, 1, 1, 1, 5, 4, 5, 9, 5 ,0, 6, 5, 8

Chunking

• Point out links to students so that they can connect what they are learning with what they already know. Eg. When teaching students about rotational properties remind them about similar translational properties.

Revise content to help students move it to long term memory

Cognitive overload• Do not present irrelevant

information.

• Worked examples are useful before students attempt a problem themselves.

• If students are reading text they are not listening to you.

What study techniques work for students?

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4-58.

These work well:Practice testing: includes answering any questions for practice

Distributed practice: spreading out practice over time instead of cramming.

These were ranked as medium utility:Elaborative interrogation: asking “Why?”

Self-explanation: Students explain some aspect of their processing during learning.

Interleaved practice: Doing a little study for one subject, followed by another rather than setting aside large chunks of time for one subject.

These had a low utility:Summarization

Highlighting and underlining

The keyword mnemonic: for learning foreign languages

Imagery use for text learning: Draw/imagine pictures to represent text

Rereading

Do students learn first year physics effectively in an online environment?

Important to ask:• Pressure to “be digital”

• Most studies performed on MOOCs, very different cohort to our undergraduate students

• Offers advantages eg. Allows students to study from anywhere

How much is the right amount?Entirely Online

Entirely face-to-faceBlended

Everyday Physics~200 students each

semester, 3 semesters a year

Introductory, algebra based course

Physics 1A~1600 students in semester 1; 500

in semester 2Introductory, calculus based courseFace-to-face labs and problem

solving workshops

Web-stream lectures~ 200 enrollments in each

semester

Face-to-face lectures

Fundamentals of Physics

~200 students in two semesters

Introductory, algebra based course

Tutorials

Everyday PhysicsVideo lectures (mainly short)

~10 tutorial problems with videoed solutions

A discussion forums

Fortnightly investigations

Quizzes every three weeks

A final report

Videos all available on YouTube, UNSW School of Physics

Everyday Physics

Algebra based course

Fundamentals of PhysicsAlgebra based course

Physics 1A

Everyone has a two hour lab session and one hour problem solving workshop each week.

Some people attend three one hour lectures, others do “interactive lectures” online.

Concept inventory resultsCohort Number

(enrollments)Pretest % Posttest % Normalized

gain %Summer 2015/16 82 (186) 37.5 48.4 17.4

Semester 1, 2016 122 (228) 43.2 49.3 10.8

Semester 2, 2016 89 (243) 43.1 50.5 13.1

Summer 2017/18 63 (227) 37.7 45.6 16.5

Semester 1, 2017 89 (178) 41.8 51.4 16.5

Semester 2, 2017 117 (320) 45.0 50.9 10.8

Semester 1, 2016 43 (164) 35.8 43.0 11.2

Semester 2, 2016 52 (193) 28.3 35.1 9.5

Semester 1, 2017 38 (173) 35.3 46.3 17.1

Semester 2, 2017 74 (253) 33.2 44.0 16.2

Blue: Online course, Everyday Physics

Yellow: Face-to-face course Fundamentals of PhysicsFMCE survey

Physics 1ACohort Number

(enrollments)Pretest % Posttest

%Normalizedgain %

Semester 1, All 776 (1646) 65.3 74.9 27.6

Semester 1, WEB 66 (188) 67.4 74.3 21.4

Semester 1, completed > 80% WEB

27 (19 enrolled in WEB)

73.3 84.6 42.1

Semester 1, All 546 (1646) 62.5 73.0 28.1

Semester 1, WEB 54 (188) 59.9 74.1 35.4

Semester 1, completed > 80% WEB

52 (24 enrolledin WEB)

55.6 71.7 36.1

Semester 2, All 203 (560) 63.8 69.8 16.5

Semester 2, WEB 46 (205) 58.6 66.3 18.7

Semester 2, completed >80% WEB

6 (6 enrolled in WEB)

50.0 82.8 65.6

Blue: FCI survey, mechanicsYellow: TCI survey, thermal

physics

Exam resultsBlue: Physics 1A Yellow: Higher Physics 1A

Cohort Number (enrollments)

Exam Average

Physics 1A (All) 748 (1155) 50.0

Enrolled in web stream 75 (147) 45.8

Completed 80% of web stream mechanics

18 58.7

Higher Physics 1A 329 (435) 62.7

Enrolled in web stream 27 (41) 67.0

Completed 80% of web stream mechanics

13 71.3

Conclusions• No significant difference in performance in concept inventory tests between

entirely online course and face-to-face course

• No significant difference between students who enrolled in online lectures compared with those who attended face-to-face

• Hard to get students to complete online material (but also hard to get them to attend lectures….)

• Students in the more blended course had the highest normalized learning gains. Giving students choice works well. It does not seem that we are disadvantaging students by offering online courses and options.

Discussion