A MEMBER OF THE RUSSELL GROUP

TRIANGULATING TEL IN HLS:What’s happening; What do stu-dents want; and do staff agree?Peter Reed: Institute of Learning & Teaching

INSTITUTE OF LEARNING & TEACHING

ILT Activity

In response to LGoS, TEL Strategy, etc:

TEL Academic & TEL Support Teams;

A three phase ‘audit’ of TEL across ILT provision.

Current Practice

Staff Perspective Student Perspective

TEL Strategy

Current Practice

Staff Perspective Student Perspective

TEL Strategy

Minimum Standards Online Submission

Lecture Capture

PHASE 1What’s Happening?March - May 2013

Phase 1: What’s Happening?

Involved meeting people(n=40)

And

An audit of VITAL use(~300)

Phase 1: What’s Happening?

Some good (but isolated) practice

Inconsistency in VITAL

Some use of Turnitin (but little formal online submission/feedback)

Recording of lectures/other content is increasingly popular

PHASE 2Staff Survey

n=102July - August 2013

Phase 2: Staff Survey

90%

90% of staff agree with provision of minimum standards (more on this later)

Used MS Word

Used TurnItIn & Grademark

Used Audio Feedback

Used Video Feedback

0 25 50 75 100

92%

28% 62%

48% 12%

7%37%

26%

32%24%

14%

DISAGREE UNAWARE OF LEARN MORE EXPERIENCED

Attitudes & Experience towards electronic marking & feedback

Screencasts

Full ‘live’ lecture capture

Pre-recorded to ‘flip’…

‘Other’ video e.g. interviews

0 25 50 75 100

48%

44% 32%

27%

29%47%

10%

10%15%

19%

Attitudes & Experience towards recording materials

38%

23%

44%

DISAGREE UNAWARE OF LEARN MORE EXPERIENCED

Screencasts

Full ‘live’ lecture capture

Pre-recorded to ‘flip’…

‘Other’ video e.g. interviews

0 25 50 75 100

48%

44% 32%

27%

29%47%

10%

10%15%

19%

DISAGREE UNAWARE OF… LEARN MORE… EXPERIENCED…

Attitudes & Experience towards recording materials

38%

23%

44%

Some concern recordings may impact on attendance

Time

Suppor

t / Tr

ainin

g

Rew

ard S

truct

ure

s

Prio

rities

Ski

lls / L

iter

acie

s

Mot

ivat

ion

Recognition of Barriers to engaging & innovation

61% 32% 31%30%29%5%

PHASE 3Student Survey

n=840October - November 2013

Would like to submit coursework electronically

Would like e-feedback

Would like electronic marks

Would like audio feedback

0 25 50 75 100

55%

DISAGREE INDIFFERENT AGREESTRONGLYAGREE

Attitudes & Experience towards electronic marking & feedback

20%11%

46%24%17%

41%24%17%10%

20%17%26%20%16%

STRONGLY DISAGREE

Would like to submit coursework electronically

Would like e-feedback

Would like electronic marks

Would like audio feedback

0 25 50 75 100

55%

DISAGREE INDIFFERENT AGREESTRONGLYAGREE

Attitudes & Experience towards electronic marking & feedback

20%11%

46%24%17%

41%24%17%10%

20%17%26%20%16%

STRONGLY DISAGREE

Students value feedback (despite some research)

Mostly concerned with e-grades (75% A or SA)

eSubmission popular (69% A or SA) - cost savings

‘Innovative’ approaches such as audio feedback not (38% A or SA%)

Attitudes & Experience towards electronic marking & feedback

Would like to access recorded lectures

Would like synchronised audio with slides

Would like synchronised video with slides

Would like dedicated recordings to watch before class

0 25 50 75 100

48%

DISAGREE INDIFFERENT AGREESTRONGLYAGREE

Attitudes & Experience towards recorded content

17%17%

66%15%

58%17%10%

40%21%21%11%

STRONGLY DISAGREE

Would like to access recorded lectures

Would like synchronised audio with slides

Would like synchronised video with slides

Would like dedicated recordings to watch before class

0 25 50 75 100

48%

DISAGREE INDIFFERENT AGREESTRONGLYAGREE

Attitudes & Experience towards electronic marking & feedback

17%17%

66%15%

58%17%10%

40%21%21%11%

STRONGLY DISAGREE

Students want to access recordings (80% A or SA)

Synchronised Audio & PPT most popular (76% A or SA)

Synchronised Video & PPT is too (65% A or SA)

However….

Attitudes & Experience towards recorded content

Synchronised audio

Pre-r

eco

rded v

ideo

Audio

only

Oth

er

reco

rdin

gs

Synchronised video

50%

33%

1% 7% 9%

Single preferred approach to recorded content

Revisit to clarify

Revise for exam / coursework

92%

87%

7%

56%

46%

Intended use of recorded lectures

70%

Overcome challenges in note-taking

Catch up on missed classes

Overcome concentration

challenges

Instead of attending lectures

9%

Overcome language barriers

Whilst studying at computer

On mobile device e.g. travelling

96%

27%

Context for use of recorded lectures

23%

Multi-tasking

MINIMUM STANDARDS

COMPARING STAFF & STUDENT RESPONSES

Staff & Students were provided a

list

A Welcome to the area

Contact details for Leader

Contact details for others

Module Specification

Timetable / Schedule

Learning Outcomes

Assessment Strategy

Further Reading

Lecture notes/handouts

Past Exam Papers

Online submission

Formative feedback on draft

Online discussion forums

A W

elco

me

to t

he

area

Con

tact

det

ails

for

Lea

der

Con

tact

det

ails

for

oth

ers

Mod

ule

Spec

ifica

tion

Tim

etab

le / S

ched

ule

Lear

nin

g O

utc

omes

Ass

essm

ent

Str

ateg

yFu

rther

Rea

din

gLe

cture

not

es/h

andou

tsPa

st E

xam

Pap

ers

Onlin

e su

bm

issi

on

Form

ativ

e fe

edbac

k on

dra

ftO

nlin

e dis

cuss

ion for

um

s

100%

75%

50%

25%

0%

95%93%

88%87%

84%

54%

71%

79%80%

69%

44%

75%79%

Student suggestions for inclusion in Minimum standards

* ***** *

A W

elco

me

to t

he

area

Con

tact

det

ails

for

Lea

der

Con

tact

det

ails

for

oth

ers

Mod

ule

Spec

ifica

tion

Tim

etab

le / S

ched

ule

Lear

nin

g O

utc

omes

Ass

essm

ent

Str

ateg

yFu

rther

Rea

din

gLe

cture

not

es/h

andou

tsPa

st E

xam

Pap

ers

Onlin

e su

bm

issi

on

Form

ativ

e fe

edbac

k on

dra

ftO

nlin

e dis

cuss

ion for

um

s

100%

75%

50%

25%

0%

72%

47%

65%59%

71%

27%

37%38%

43%

59%

47%

58% 58%

Staff suggestions for inclusion in Minimum standards

A W

elco

me

to t

he

area

Con

tact

det

ails

for

Lea

der

Con

tact

det

ails

for

oth

ers

Mod

ule

Spec

ifica

tion

Tim

etab

le / S

ched

ule

Lear

nin

g O

utc

omes

Ass

essm

ent

Str

ateg

yFu

rther

Rea

din

gLe

cture

not

es/h

andou

tsPa

st E

xam

Pap

ers

Onlin

e su

bm

issi

on

Form

ativ

e fe

edbac

k on

dra

ftO

nlin

e dis

cuss

ion for

um

s

100%

75%

50%

25%

0%

Student v Staff suggestions for inclusion in Minimum standards

A W

elco

me

to t

he

area

Con

tact

det

ails

for

Lea

der

Con

tact

det

ails

for

oth

ers

Mod

ule

Spec

ifica

tion

Tim

etab

le / S

ched

ule

Lear

nin

g O

utc

omes

Ass

essm

ent

Str

ateg

yFu

rther

Rea

din

gLe

cture

not

es/h

andou

tsPa

st E

xam

Pap

ers

Onlin

e su

bm

issi

on

Form

ativ

e fe

edbac

k on

dra

ftO

nlin

e dis

cuss

ion for

um

s

100%

75%

50%

25%

0%

95%93%

88%87%

84%

54%

71%

79%80%

69%

44%

75%79%

Student vs Staff suggestions for inclusion in Minimum standards

72%

47%

65%59%

71%

27%

37%38%

43%

59%

47%

58% 58%

A W

elco

me

to t

he

area

Con

tact

det

ails

for

Lea

der

Con

tact

det

ails

for

oth

ers

Mod

ule

Spec

ifica

tion

Tim

etab

le / S

ched

ule

Lear

nin

g O

utc

omes

Ass

essm

ent

Str

ateg

yFu

rther

Rea

din

gLe

cture

not

es/h

andou

tsPa

st E

xam

Pap

ers

Onlin

e su

bm

issi

on

Form

ativ

e fe

edbac

k on

dra

ftO

nlin

e dis

cuss

ion for

um

s

100%

75%

50%

25%

0%

95%93%

88%87%

84%

54%

71%

79%80%

69%

44%

75%79%

Student vs Staff suggestions for inclusion in Minimum standards

72%

47%

65%59%

71%

27%

37%38%

43%

59%

47%

58% 58%

Statistical TestsPositive correlation between staff & students (r=0.560)

Some significant differences between staff and student responses (p=0.0002)

Significant difference between number of items selected (p=0.0001) - m Staff=9.32: Students=10.91

Significant difference between number of items selected by male and female students (p=0.0012)

A W

elco

me

to t

he

area

Con

tact

det

ails

for

Lea

der

Con

tact

det

ails

for

oth

ers

Mod

ule

Spec

ifica

tion

Tim

etab

le / S

ched

ule

Lear

nin

g O

utc

omes

Ass

essm

ent

Str

ateg

yFu

rther

Rea

din

gLe

cture

not

es/h

andou

tsPa

st E

xam

Pap

ers

Onlin

e su

bm

issi

on

Form

ativ

e fe

edbac

k on

dra

ftO

nlin

e dis

cuss

ion for

um

s

100%

75%

50%

25%

0%

Criteria with largest staff/student difference in Minimum standards

37%

46%

41%34%

A W

elco

me

to t

he

area

Con

tact

det

ails

for

Lea

der

Con

tact

det

ails

for

oth

ers

Mod

ule

Spec

ifica

tion

Tim

etab

le / S

ched

ule

Lear

nin

g O

utc

omes

Ass

essm

ent

Str

ateg

yFu

rther

Rea

din

gLe

cture

not

es/h

andou

tsPa

st E

xam

Pap

ers

Onlin

e su

bm

issi

on

Form

ativ

e fe

edbac

k on

dra

ftO

nlin

e dis

cuss

ion for

um

s

100%

75%

50%

25%

0%

95%93%

88%87%

84%

54%

71%

79%80%

69%

44%

75%79%

Student vs Staff suggestions for inclusion in Minimum standards

72%

47%

65%59%

71%

27%

37%38%

43%

59%

47%

58% 58%

Questions

Which criteria do we include?

How much of the student voice is enough?

Demo PSYC310CSD Automation of Minimum Standards from TULIP

TULIPBB

DemoCSD Automation of Minimum Standards from TULIP

DemoCSD Automation of Minimum Standards from TULIP

DemoCSD Automation of Minimum Standards from TULIP

DemoCSD Automation of Minimum Standards from TULIP

DemoCSD Automation of Minimum Standards from TULIP

Agreement thus farCriteria to be included automatically…

1. Module Title & Module Code 

2. Module Co-ordinator (name and email address)

3. Aims

4. Learning Outcomes

5. Syllabus

6. Teaching & Learning Strategy

7. Assessment requirements 

8. Contact hours

9. Disclaimer

Next StepsHerzberg’s Hygiene Factors

Focus groups

What Else?• Free text questions identified

• Students can’t really identify innovation (mobile or general)

• Lots of mobile usage and expectation (89%)

• Lots of reference to apps to support studies

• Lots of problems with relatively simple tech issues with mobile devices such as wifi, email, library access, etc. Leading to drop-in sessions

What Else?

• Students are generally confident with their digital literacies (79%)….

• But some would like opportunity to develop further (31%)

In Summary

• Expectation and demand from students

• Positive attitudes from staff to engage

• Need support

REFERENCES

Bench Image CC BY-NC-ND Flickr User Madhan Kumar Staff Wanted Image CCBY-NC-SA Flickr User LEOL30

Peter ReedLecturer (Learning Technology)Institute of Learning & TeachingFaculty of Health & Life Science

peter.reed@liverpool.ac.uk@reedyreedles#54332