lrt talks 20140610 efye2014 nottingham

27
Enhancing the First Year Student Experience at MMU: Clearing a path & interpreting digital footprints Professor Mark Stubbs Head of Learning & Research Technologies http://twitter.com/thestubbs http://slideshare.net/markstubbs Nottingham | Jun 2014 #EFYE2014

Upload: markstubbs

Post on 27-Nov-2014

93 views

Category:

Education


1 download

DESCRIPTION

Summary of work undertaken at Manchester Metropolitan University to enhance the student experience, beginning with a rewrite of the first year undergraduate curriculum and including the delivery of personalised systems that wrap institutional resources around the learner

TRANSCRIPT

Page 1: LRT Talks 20140610 EFYE2014 Nottingham

Enhancing the First Year Student Experience at MMU: Clearing a path & interpreting digital footprints

Professor Mark StubbsHead of Learning & Research Technologies

http://twitter.com/thestubbs

http://slideshare.net/markstubbs

Nottingham | Jun 2014#EFYE2014

Page 2: LRT Talks 20140610 EFYE2014 Nottingham

Context | An interesting place to work!

• Large, popular, multi-campus university– 36,000+ students | 1,000+ courses | consolidating from 7 sites

• Pursuing an ambitious transformation initiative– New Buildings, New Curriculum, New Admin Processes & Systems,

New Learning & Mobile Tech, New Quality Processes…

80%+ of 8,125

11/12 UG intake

young from state

schools / colleges

Page 3: LRT Talks 20140610 EFYE2014 Nottingham

Context | Strategic improvement imperative

Student RetentionSuccess & Satisfaction

Student Intake(Aspirations, Attitude

& Abilities)

Learning, Teaching, Assessment & Personal Development

Processes, Facilities& Resources

ReputationMarketing &RecruitmentProcesses

All Year Numbers

Resource allocation

League table rankingsA

A Recruit to target

B

B Improve satisfaction, retention & success

C

C Inform decision-makers

2010 NSS

Where to

intervene?

Page 4: LRT Talks 20140610 EFYE2014 Nottingham

CSI | Student Experience

Careful diagnosis of (diverse) learning experiences• Recurrent messages

– NSS analysis: course organisation determines Q22– Mobile surveys: deadlines, timetables… (m-admin)– Focus groups: consistent, easy to find info

• Great (and not-so great) expectations– “engaging, well-organised courses”– “inspirational tutors who know me”

‘Hygiene

factors’Stimulating

pedagogies

Page 5: LRT Talks 20140610 EFYE2014 Nottingham

CSI | Student Retention & Success

Reflect on…+ Assumptions+ Interventions

Examine data…+ Focus Groups+ Surveys+ Journey journals+ Observations+ National data

Review literature…+ Multi-dimensional+ Belonging+ Habitus+ Polishing+ …

B Improve satisfaction, retention & successLiteratureJourneys

Wed 9, N31 Living

LearningSocial /Community

Assumptions

1st years have requisite motivation, academic ability,

adaptive capacity and resilience to

a) Sort / £ / // / so they can focus

b) Adapt study methods to new academic requirements

c) Perform to required standard

d) Establish supportive social / working relationships

InterventionsR&A / pre-entry, personalised info, early formative feedback

Page 6: LRT Talks 20140610 EFYE2014 Nottingham

Coordinating our response

B Improve satisfaction, retention & success

GreatOnline Experience

(Seamless, personalised)

GreatLearning Spaces

GreatTeaching

SimplifyUnwieldy

Curriculum

EQAL

£350M

Page 7: LRT Talks 20140610 EFYE2014 Nottingham

• A coordinated strike for step-change improvement

EQAL

New Curriculum• designing new units, …

New Admin Systems & Processes• personal timetabling, …

New Virtual Learning Environment• Moodle & myMMU web/mobile, Talis Aspire…

New QA & QE Processes• facilitating curriculum transformation

In the current climate

Diminishing unit of resource

Everything depends on everything else

We are large and risk averse

but

but but

but but

Programme

Board chaired by

DVC Student

Experience

Page 8: LRT Talks 20140610 EFYE2014 Nottingham

MISSION:

2010 Q1: Plan Programme & Projects

2010 Q2: Approve UG curriculum rules

2010 Q3: Develop smart curriculum capture forms

2010 Q4: Enter 800+ new L3 + L4 module

2011 Q1: Approve new modules

2011 Q2: Set up SRS, TT & VLE

2011 Q3: L4 curriculum & systems go live

2011 Q4: Enter new L5 modules…

2012 Q3: L5 goes live

2013 Q3: Entire new UG curriculum live

Page 9: LRT Talks 20140610 EFYE2014 Nottingham

Smart forms

868L3+L4modules

Minimise

‘big ask’

Jul 2010

Page 10: LRT Talks 20140610 EFYE2014 Nottingham

Feb 2011

Core+ VLE blueprint

m

Page 11: LRT Talks 20140610 EFYE2014 Nottingham

More familiar

Page 12: LRT Talks 20140610 EFYE2014 Nottingham

12

Wrapping the institution around the learnerStudent ID

Timetable

This work is licensed under a Attribution-NonCommercial-ShareAlike 2.0 licence

Sync to personal device

Page 13: LRT Talks 20140610 EFYE2014 Nottingham

Pre-entry access to timetables

Less early

access than

expected

Page 14: LRT Talks 20140610 EFYE2014 Nottingham

Wrapping the institution around the learnerStudent ID (+ Unit code)

Deadlines / extensions / feedback return dates / provisional marks

Personalised submission sheet

This work is licensed under a Attribution-NonCommercial-ShareAlike 2.0 licence

Page 15: LRT Talks 20140610 EFYE2014 Nottingham

Wrapping the institution around the learnerUnit code

Resource list

Relevant resource

This work is licensed under a Attribution-NonCommercial-ShareAlike 2.0 licence

Page 16: LRT Talks 20140610 EFYE2014 Nottingham

Wrapping the institution around the learnerUnit code

Past exam papers

Past exam paper

This work is licensed under a Attribution-NonCommercial-ShareAlike 2.0 licence

11,000+ past

exams

uploaded

Page 17: LRT Talks 20140610 EFYE2014 Nottingham

MyMMU App

m-admin

Page 18: LRT Talks 20140610 EFYE2014 Nottingham

Summary | we cleared a path to the information our learners needed…

Page 19: LRT Talks 20140610 EFYE2014 Nottingham

Engaging our students

Providing business intelligenc

e

Automating our

processes

Are we in a better place now?

10,000+ students post

40,000+ comments in

our biannual student

surveys

Submission tracking, Personal timetables, …

Quartile improvements

in course org &

learning resources in

2012 NSS

EQAL wins inaugural Guardian Student Experience Award!

30,000 regular users,

52m+ hits on Moodle,

26,000+ App

registrations!

Page 20: LRT Talks 20140610 EFYE2014 Nottingham

… we can now begin to interpret some digital footprints

Page 21: LRT Talks 20140610 EFYE2014 Nottingham

Using the data

2. Identify relevant data sources

3.Summarise individual data sources

4 Join on common identifiers

5 Prep the data for analysis

6.Analyse & visualise

1.Appreciate the issue

Refining understanding of a

problem space

Page 22: LRT Talks 20140610 EFYE2014 Nottingham

Understanding student satisfaction

2. Identify relevant data sources

3.Summarise individual data sources

4 Join on common identifiers

5 Prep the data for analysis

6.Analyse & visualise

1.Appreciate the issue

Refining understanding of

student satisfaction

Student demographicsEntry profilesInternal survey responses

Join on student ID

Random Forests analysis

Satisfaction predictors↗ Confidence

↘Organisation→Teaching

Sum entry points & qual type

Handle missing values, collapse categories…

Page 23: LRT Talks 20140610 EFYE2014 Nottingham

Understanding student success

2. Identify relevant data sources

3.Summarise individual data sources

4 Join on common identifiers

5 Prep the data for analysis

6.Analyse & visualise

1.Appreciate the issue

Refining understanding of

student continuation

HESA PILeague Tables

Join on UK PRN

Random Forests Regression Tree

PredictorsYNG NSSEC4-7Entry TariffNSS

Add UKPRN

Handle missing values, collapse categories…

Page 24: LRT Talks 20140610 EFYE2014 Nottingham

Conditional Inference Trees

Party Library

Page 25: LRT Talks 20140610 EFYE2014 Nottingham

Limitations

1. Confirmation bias

2. Meaningful comparison

3. Quantitative V Qualitative

4. Data quality

5. Correlation ≠ causation

“Everything that can be counted

does not necessarily count;

everything that counts cannot

necessarily be counted”

Einstein

Reality of our current data warehouse build: careful thought about error handling + painstaking checking of joins!

Page 26: LRT Talks 20140610 EFYE2014 Nottingham

Continuous monitoring & improvement

MDL SRS TT

Data Warehouse Analysis Cubes

Undergraduate Dental Technology Network (11209A)

Overview Students Success Satisfaction Engagement Standards Appl ications Employment

Head count [now] of [all] students for [all modes] [all enrolments]

Enrolled

268 (95%)

Withdrawn

0 (0%)

Suspended

0 (0%)

Repeat w/o attend 14 (5%)

#STUDENTS TOTAL L3 L4 L5 YO L6 L7 Undergraduate Dental Technology Network (11029A)

268 83 73 67 45

- BSc(Hons) Dental Technology (6266) 184 73 67 44 + BSc(Hons) Dental Technology (6266_1F) 73 73 + BSc(Hons) Dental Technology (6266_2F) 67 67 + BSc(Hons) Dental Technology (6266_3F) 44 44 + BSc(Hons) Dental Technology (6267) 1 1 + FDSc Dental Technology (621C) 62 62 + FDSc Dental Technology (6264) 21 21

How has my student cohort changed?

Higher than faculty average

Lower than last year

5%-5%

5%

-5%

PLP

Parents in HE

Mature

Live at home

Higher than last year

Lower than faculty average

Live at home 12% PLP 4% Mature 6% Disability 3% BME 8% Male 42% International 4%

In response to this information, I intend [to highlight the following good practice]

Previously, the Programme Leader committed to…

On 21/03/13 Done? On 22/02/13 raise with the Dean targets for the part-time courses Complete

CMI: Undergraduate Dental Technology Network (11209A)

Overview Students Success Satisfaction Engagement Standards Appl ications Employment

Satisfaction [on all ISS] of [all] students for [all modes] [all enrolments]

Course Satisfaction

Comments Show [all] comments for [all courses] that contain [Click here to enter text.] for [students who are satisfied with confidence building] [Go!] [Clear]

0% 100%50%25% 75%

69%

#Enr

olle

d

Resp

onse

Expl

anati

on

Feed

back

Advi

ce

Org

anis

ation

Reso

urce

s

Confi

denc

e

Ove

rall

Undergraduate Dental Technology Network (11029A) 268 30% 100% 93% 93% 87% 96% 95% 100% - BSc(Hons) Dental Technology (6266) 184 30% 100% 93% 93% 87% 96% 95% 100% - 6266_1F 73 27% 100% 95% 95% 95% 95% 100% 100% - 6266_2F 67 31% 100% 100% 95% 81% 100% 95% 100% - 6266_3F 44 32% 100% 79% 86% 86% 93% 86% 100% + BSc(Hons) Dental Technology (6267) 1 0%

60.0%

65.0%

70.0%

75.0%

80.0%

85.0%

90.0%

95.0%

100.0%

2011

2012

2013

2011

2012

2013

2011

2012

2013

2011

2012

2013

2011

2012

2013

2011

2012

2013

2011

2012

2013

EXPLANATION

FEEDBACK

ADVICE

ORGANISATION

RESOURCES

CONFIDENCE

OVERALL

CMI dashboard for all programme leaders for Sep 2014

C

Page 27: LRT Talks 20140610 EFYE2014 Nottingham

Change management lessons1. Understand important dynamics

2. Be clear what needs to be done

3. Set high-level goals

4. Understand where change is required

5. Be bold and plan holistically

6. Ensure you’ve got the right team to drive

7. Make ‘big asks’ as small as possible

8. Make it stick with reinforcing change

http://tinyurl.com/jiscmmusrcCurriculum Design

Assessment & Feedback http://tinyurl.com/jiscmmutraffic

Find out

more!