Evaluating the effectiveness of innovative pedagogies –

a personal reflection

Simon Lancaster

The plural of anecdote isnot data.

The plural of anecdote isdata.

57%

43%

The plural of anecdote?

A. The plural of anecdote is not data.

B. The plural of anecdote is data.

10 May 2017

10 May 2017

Professor Mymates module evaluation data

0

1

2

3

4

5

6

0

20

40

60

80

100

120

2008 2009 2010 2011 2012 2013 2014 2015

Mymate module evaluation

NSS Median Mean

Contemporary versus traditional?

<10% 10–30% 30–50% 50–70% 70–90% >90%

17%

20%

7%

23%

20%

13%

What proportion of the time are our students actively thinking during (lecture theatre) teaching?

A. <10%

B. 10–30%

C. 30–50%

D. 50–70%

E. 70–90%

F. >90%

10 May 2017

Our model of lecture flipping

The importance of the question

1 2 3 4 5 6

What is your objective for a question posed during teaching?

10 May 2017

Rank Responses

1

2

3

4

5

6

What is your objective for a question posed during teaching?

The Goldilocks Zone

10 May 2017

Opposites attract, like repels like

Which is favourable?

A. Li Li+ + e−

B. Li+ + e− Li

C. Li+ Li2+ + e−

D. Li− + e- Li2−

Veryconfident

Confident Notconfident

Not at allconfident

0% 0% 0% 0%

How do you feel about your answer?

A. Very confident

B. Confident

C. Not confident

D. Not at all confident

10 May 2017

Which is favourable?A. Li Li+ + e−

B. Li+ + e− Li

C. Li+ Li2+ + e−

D. Li− + e- Li2−

Which is favourable?

A. Li Li+ + e−

B. Li+ + e− Li

C. Li+ Li2+ + e−

D. Li− + e- Li2−

Evidence

9

12

25

54

Student activity

Thinking PI Interactions Lecturing

10 May 2017

Lancaster’s module evaluation data

0

1

2

3

4

5

6

0

20

40

60

80

100

120

2008 2009 2010 2011 2012 2013 2014 2015

Lancaster module evaluation

NSS Median Mean

A. B. C.

0% 0% 0%

What is learning gain?

A. The learning achieved by a student between two points in time.

B. The improvement in knowledge, skills, work-readiness and personal development made by students during their time spent in higher education.

C. The value added by an institution in terms of the increased likelihood of gaining a good honours degree given a certain range of entry qualifications.

10 May 2017

What is learning gain?

A. The learning achieved by a student between two points in time.

B. The improvement in knowledge, skills, work-readiness and personal development made by students during their time spent in higher education.

C. The value added by an institution in terms of the increased likelihood of gaining a good honours degree given a certain range of entry qualifications.

Veryconfident

Confident Notconfident

Not at allconfident

0% 0% 0% 0%

How do you feel about your answer?

A. Very confident

B. Confident

C. Not confident

D. Not at all confident

10 May 2017

What is learning gain?A. The learning achieved by a student between two points in time.

B. The improvement in knowledge, skills, work-readiness and personal development made by

students during their time spent in higher education.

C. The value added by an institution in terms of the increased likelihood of gaining a good

honours degree given a certain range of entry qualifications.

A. B. C.

33% 33% 33%

What is learning gain?

A. The learning achieved by a student between two points in time.

B. The improvement in knowledge, skills, work-readiness and personal development made by students during their time spent in higher education.

C. The value added by an institution in terms of the increased likelihood of gaining a good honours degree given a certain range of entry qualifications.

10 May 2017

The UEA project

At UEA, we are examining 3 ways of measuring Learning Gain:

- Student marks/grades/GPA

- Self-efficacy

- Concept inventories

Concept inventories

Conceptual

Validated

Reliable

10 May 2017

Collaborative concept inventory use

Normalised gain

.

Mean normalised gain for the bonding concepts inventory

0.19

1. Determine an average mark for the quiz.

2. Assign student to above or below average group. 0,1

3. Calculate student average confidence.

4. Assign student to above or below average confidence group.

10 May 2017

Self-efficacy

High confidence Low Confidence

High attainment X

Low attainment

SESSION SIZE P-value AssociationWK2 89 0.058 BORDERWK4 86 0.015 YESWK6 83 0.001 YESWK8 83 0.126 NO

10 May 2017

Self-efficacy over the course of the module

ACKNOWLEDGEMENTS

Dennis Cook

Dr Fabio Arico

Prof Neil Ward

Helena Gillespie

Dr Annamari Ylonen

Dr Adam Longcroft

Prof Eric Mazur

Dr Ross Galloway

Dr Anna Wood

Dr Anne Nortcliffe

HEFCE

Turning Technologies

4* evidence