Presentation to SCAP of the HEA Research Review, Warwick, 5 July 2013

Prof. David Parsons, Leeds Metropolitan University; HOST Policy Research

A focus on the impact and efficacy of HE-centred teaching and

development programmes An emphasis on new and established academic staff and entrants An evidence-based review - literature and other available evidence UK focus (implications) but international review (evidence-base)

… and with the objective of reviewing: Extent and quality of the evidence base – and gaps. Elements of programme design/delivery with the greatest impact on

staff development and student learning; Scope, method and nature of future research into the efficacy and

impact of these programmes.

Stage 1: Design, team/reviewer co-ordination (and steering).

Stage 2: Mapping wider international experience and sources:

Evidence from HE comparative studies (OECD etc).

Review with international fora, observatories, etc.

Liaison with selected cross-national sectoral/prof. agencies.

Stage 3: Systematic review of national literature etc:

Systematic review of scholarly literature

Official documentation/national evaluations

Evidence call - professional fora and social media

Selective discussions with international ‘experts’

Stage 4: Collation, review (and testing) and reporting

Expanding HE has seen a rising UK policy profile for educational development of academic staff.

The Future of Higher Education (DfES, 2003) provided major public investment (England) – embedded in HEFCE Strategic Plan (2003).

All home-countries emphasised HEI-led approach + partnership support

(eg CETL programme - England and Northern Ireland). Added impetus from adoption of ‘learning outcomes’ in UK from Bologna

Process (and associated Dublin descriptors). Transfer of financial burden to students (England) see’s teaching quality

as key in informed student choice (KIS). Different policy drivers in Scotland, NI and Wales - but with informed

student choice rising up agenda. So - commodification of HE is making teaching quality (and HEI

responses) a strategic asset!!!

Long established scholarly inquiry – 71 diagnosed studies by 1980 (Levinson-Rose and Menges, 1981)

Early (and later) concerns with depth/quality of evidence-base (Weimer and Lenze,

1998). Highly fragmented evidence (Kreber and Brook, 2001; McAlpine, 2003; Prebble et

al., 2004). And for impact-centred research and programme evaluation - evidence drawn from

130 authors (11 countries), but: Little comparative or ‘time-series’ evidence on impacts. Lack of use of baseline’s (eg from tools such as ATI). Available evidence often: single initiative; small scale/quasi-experimental; lacking

contextualisation. Very little on student impacts that was reliable/robust Diverse teacher impact evidence on: attitudes (and conceptions); knowledge/skills;

and transfer to practice.

Rising HEI activity in delivering teaching development strategies

(D’Andrea and Gosling, 2005; Stes et al., 2010; Gibbs, 2012) Great diversity in delivery– part-time/certificated; mandatory/voluntary;

initial/continuing; modular/workshop programmes’ micro-teaching/portfolio’s; etc.

Coherence/cohesion not driven by wider QA - scope via flexible

‘framework’ approach of UKPSF and institutional accreditation. Role of individual qualifications (eg PGCAP) variable – and likely to

remain so unless consistently mandated. Contrast in generic (eg HEI driven) vs specific (eg disciplinary; often

cross-HEI; different disciplinary pedagogies/cultures; etc). So – more activity, more priority (65% HEIs links promotion to teaching

quality) and greater conceptualisation – but lacking ‘common ground’.

‘Control’ diagnosed effect for achieving attitudinal change to teaching

style/facilitating student learning

Positive attitudinal change linked to challenging/informing underpinning conceptual focus for participants.

Systematic shift (participating teachers) towards using inclusive, learner-

centred methods (CCSF etc).

BUT …

A ‘progressive’ process – around a year (or more) of pedagogical training for

positive effects to widely emerge. Evidence that pace in geared to self-efficacy beliefs developing very slowly. Intensive/short duration risks (some) participants being more uncertain

about themselves as teachers

Knowledge and skills impacts also positively affected by programme duration rather than intensity.

Understanding of ‘own approach’ is central to developing ‘professional identity’ – and programme effectiveness.

And beyond ‘duration’ … Direct association of skills impacts with incremental learning approaches

Uncertain effects of mixed-participation and interdisciplinary discourse. Emerging evidence of value of ‘developmental’ peer observation

(affirmation, constructive criticism, experience of how others teach differently).

Mixed evidence and limited comparability but…

Very limited ‘longitudinal’ evidence but some evidence of lasting impact on professional development/application.

Post-hoc analysis (+1 year) of more student-centred practice + stronger transfer

for participants with more credits). Participant motivation is strongest influence (but more for motivation to learn than

transfer of learning to practice. Experienced teachers - more and earlier transfer to practice (but cause or effect). Knowledge transfer potential for ‘novice’/aspiring teachers related to (their) critical

mass of pedagogic knowledge. Transfer potential enhanced by ongoing continuing support – and peer observation. Some negative transfer effects for specific groups (but not well understood). BUT … Knowledge transfer is slow – falls in initial performance?

Longer length/duration positively affects the quality of learning and

transfer potential

Modalities involving ‘on-the-job’ learning have a more positive impact on transfer to practice

BUT … unclear impact effects for other (different) modalities. Interventions extended over time show more positive results for transfer

(ie than one-time interventions). Little evidence (yet) of key motivators and influences to knowledge and

skills transfer to changed teaching behaviour and practice. Novice teachers may need a critical mass of ‘foundation’ pedagogic

knowledge/understanding before effective transfer. Novice teachers relate better to collaborative modes (ie more

experienced colleagues) or communities of practice.

This is a very fragmented and patchy evidence base Available evidence is widely positive … but with chronic gaps in

coherence, comparability – and credibility! More – and more systematic - attention to be paid to assessing

participant impacts … different modality and design effects, transfer determinants, etc.

Additionality of impacts (counterfactual) are very poorly understood –

and crucial for investments. This requires more than post-intervention impact review– it needs

longitudinal review + cross-programme focus Don’t forget end users … what’s added value for/impact on students? BUT … who resources this; who helps institutions/practitioners apply

proven impact methods; who drives it?

See the full research report – available from HEA (printed and on-line)

HEA has also published the full bibliography of the review

HOST Policy Research can be contacted on: info@hostpolicyresearch.com

I can be contacted at my ‘home office’ on: drdj.parsons@gmail.com