pgr-pgt tutorials scheme 1. acknowledgements jenny yiend – project principal investigator...

1

PGR-PGT Tutorials Scheme

KLI excellence in teaching

June 16th 2014

Becci Strawbridge: becci.strawbridge@kcl.ac.uk

Stefania Tognin: stefania.tognin@kcl.ac.uk

2Acknowledgements

Jenny Yiend – Project Principal Investigator

Participating courses & Key Collaborators- MSc Mental Health Studies (Basic Mental Health module):

Programme Leader; Madeleine Oakley; Programme Manager; Dr Katerina Koutsantoni- Research Methods KEATS module: Dr Cathy Fernandes- Carolyn Hodgman - Dr Anna Mountford-Zimdars

Others Involved

Dr Brenda Williams, Dr Eugenia Kravariti, Dr David Hay, Professor Jon Cooper, Professor Susan Lea, Dr Meera Komarraju, Dr Stephen Gilmore, Amy Gillespie

3Project Objectives

1. PGT student requests - more contact time and detailed feedback

2. PGR student requests - increased teaching opportunities

3. Stretched resources – rising PGT numbers, funding decreases

+ KCL teaching and learning key priorities

+ Pedagogical research findings

4Background

Barnett et al., 2008 - educational goals of HE should focus on encouraging students' ‘becoming’ by processes including; student-peer engagement, student activity in own learning and contribution of own ideas in learning situations.

Komarraju et al., 2010 - HE students with frequent interactions with faculty show improved outcomes; less formal communication associated with increased motivation and engagement, and specifically academic discussions outside the classroom have been considered more important.

^ Could be facilitated by small-group tutorial (SGT) framework:- ‘group discussion’- 3-5 students, set of sessions with same tutor - ‘learning focused’

5Support for Small Group Tutorials (SGTs)

- Rudduck (1978) - Evidence of enjoyment and various benefits of small-group tutorials (SGTs)

- Feedback provision as requested by PGT students

- Griffiths et al. (1996) – small-group learning allows exploration of learning styles; personalisation, facilitates ‘deep’ learning

However, tutorials can be resource intensive….

6PGRs as tutors

• Jenkins & Healey (2009) - ‘Research-based’ learning is beneficial to students in HE

• Hay et al. (2013) - researcher-led learning has not been investigated thoroughly but has demonstrated improvements in the quality of learning

• Komarraju et al. (2008) – training graduate teaching assistants can improve enjoyment and sense of self-efficacy in teaching

7Project Plan

1. Data Collection: run Small Group Tutorials (SGTs) at IoP, creating and collecting evaluation data on tutees’ and tutors’ experience

2. Resource Development: create blended learning training materials for tutors and tutees, informed by evaluation data collected in 1;

3. Piloting: of training materials with feedback used to refine resources & optimise learning outcomes

4. School Dissemination: communicate the final tutorial model across the school for use by any programme

5. College Dissemination the tutorial model to Arts & Humanities, assessing generalisability.

8Methods: Training

Small Group Tutorials Blended Learning Training Programme

1. Preparing to Teach (for tutors only)

2. E-learning module• Purpose of tutorials• Role/Skills of Tutor• Tutee’s perspective• Structure and preparation of tutorials

3. Course Specific Briefing

9

10Methods: sample

Two sets of tutorials ran from January-June 2014

Today we report preliminary tutee data from Mental Health Studies tutees

  Basic Mental Health module; MSc Mental Health Studies programme

online KEATs course; Introductory Research Methods

Total numbers

Number of Tutors 

15 9 24

Number of Tutees 

63 38 101

Number of Tutorials  

6 x 1 hour(total 90 contact hours)

5 x 1 hour(total 45 contact hours)

135 contact hours

11

Methods: measures

Quantitative (Pre – Post Evaluation)

• Academic Self-Efficacy Questionnaire

• Rosenberg’s self-esteem scale

• PTES related questions (post-only)

• Bespoke tutorial evaluation questions

Qualitative • Topic-guided free text feedback comments on

a) expectations (pre) and

b) experiences (post)

12

Results: Qualitative (Expectations)

What are you hoping to gain from these tutorials?

Academic writing skills

“Development of academic writing skills”

“how to write essays in exams”

“better ability to apply assessment criteria to my work”

Academic fluency

“Engage in interesting academic debates within a small group”

“Broaden the way we think”

“Opportunity to discuss with others”

Confidence

“Confidence in discussing topics with colleagues & peers […]”

13

Results: Qualitative (Induction)

Do you have any comments on the induction?

“Comprehensive information, very informative”

“All the information covered was in the email and supporting document”

“So far so good, I think we are off to a good start”

“It was very useful”

“Very informative, well organized”

“Lunch was great”

14 Results: Quantitative (standardised measures)

PRE-TUTORIAL POST-TUTORIAL

Academic Self-Efficacy Items

N = 42Mean = 42.74 (4.42)

N = 26Mean = 40.19 (12.03)

Rosenberg Self-Esteem Scale

N = 42Mean = 20.62 (4.75)

N = 22Mean = 21.64 (4.45)

15Results: Quantitative - PTES

• “Tutorials have helped me to better understand in advance the criteria used in marking” (5a)

• “Tutorials have helped me understand how my assessment arrangements and course marking is made fair” (5b)

• “Tutorials have provided me with timely feedback” (5c)

5a 5b 5c0

10

20

30

40

50

60

70

% Agree

% Disagree

PTES question

31%35%

38% 42%

29%

62%

16Results: Quantitative - PTES

• “Tutorials have provided me with detailed and useful feedback (written or oral) on my work” (5e)

• “Feedback received during tutorials has helped me clarify things I did not understand” (6)

48%

32%

50%

8%

5e 60

10

20

30

40

50

60

70

80

% Agree% Disagree

PTES question

17Results: Quantitative – bespoke questions

a. Overall, I felt that I benefited from tutorials

b. I found my tutor to be approachable

c. Tutorials were useful supplement to lectures

d. My knowledge and understanding have improved

a b c d0

10

20

30

40

50

60

70

80

90

% Agree% Disagree

Question

4%

63%

7%

67%

11%

67%

4%

85%

18Results: Quantitative – bespoke questions

e. I found the tutorials motivating/encouraging

f. The tutorials content was appropriate

g. I attended all the tutorials scheduled

h. Timing and location of tutorials suited my needs

e f g h0

10

20

30

40

50

60

70

80

90

% Agree% Disagree

Question

70%

85%89%

7% 7% 7%11%

63%

19

Results: Qualitative – experiences

What were the best features of tutorials for you? The tutor

“Tutor was very supportive, encouraging and made us feel at ease”

“A great teacher helping us to structure work and who gave feedback”

The small group setting

“Small groups allowed discussions that were more relevant to everyone in the group. More comfortable to ask questions/discuss”

“Learning in small groups”

‘Tailor-made’ teaching

“They really tried to get the most out of your knowledge in a particular subject. Our tutor also allowed us to tailor it according to our wishes”

“Supervisor took a big interest in it and did a lot of work tailored to our needs”

Peer advice

“Getting advice on essay writing styles and approach to exams in a small group setting”

“Being able to discuss what others would write in essays and drawing information together to make conclusions”

20

Results: Qualitative – experiences

What were your least favourite aspects of tutorials?

Scheduling

“Little flexibility on times”

“Arranging meeting times that suits all group members”

Structure

“Lack of structure at times”

“A bit unstructured at the beginning but this improved as we got along”

Content

“The tutor did not appear to be very familiar with the content of Basic Mental Health module”

“Tutors could be made more aware of the content of the module so that they have knowledge of it”

21

Results: Qualitative – experiences

Any other comments on the tutorial programme?

Helpfulness

“Very helpful”

“My tutorial leader was helpful, approachable and made the extra effort. Really helped”

“Useful, could be well used for every module”

“It may be more useful to start things earlier in the year as I found the tutorials helped me to focus on my studies further”

“It was a good learning opportunity which helped me gain confidence”

New Ideas

“I think a more structured framework for how to cover topics and the format of information delivery (etc.) is needed. This can be then applied to various topics. This would ensure that different tutorial groups would do similar amount of work each session”

22Summary of Results

• Standardised measures of confidence in academic domain and self-esteem showed no change

• PTES questions show tutorials benefit some areas, but not all

• Bespoke questions show the strongest positive outcomes• Qualitative data provide compelling support and

identify novel areas of potential benefit (e.g. structure)

• Limitations: sample size; lack of control group; short duration

23Conclusions

Preliminary interpretations• The answer you get depends on what questions you ask, and how you ask it, e.g.

- “Timing and location of tutorials suited my needs” - 89% agree, whereas qualitative data theme ‘scheduling’ difficulties; “Little flexibility on times”- “Tutorials have helped me understand how my assessment arrangements and course marking is made fair” – only 35% agree

• What is the best way to assess tutorials?

Future considerations• Standardisation of tutorial structure• Change in evaluation methodology• Standardisation of training• Control group

Supporting evidence: Griffiths et al. (1996) - Peer-led learning proved beneficial for all participants

24Next Steps

1. Data Collection: run Small Group Tutorials (SGTs) at IoP, creating and collecting evaluation data on tutees’ and tutors’ experience

2. Resource Development: create blended learning training materials for tutors and tutees, informed by evaluation data collected in 1;

3. Piloting: of training materials with feedback used to refine resources & optimise learning outcomes

4. School Dissemination: communicate the final tutorial model across the school for use by any programme

5. College Dissemination the tutorial model to Arts & Humanities, assessing generalisability.

25

Project contacts: - Becci Strawbridge, Project Coordinator Becci.Strawbridge@kcl.ac.uk; - Stefania Tognin, CBT module leader Stefania.Tognin@kcl.ac.uk; - Jenny Yiend, Head of PGT Studies Jenny.Yiend@kcl.ac.uk

Questions?

Comments?

26References

• Barnett, R. (2009). Knowing and becoming in the higher education curriculum, Studies in higher education, 34,4, 429-440

• Griffiths, S., Houston, K & Lazenbatt, A. (1996). Enhancing student learning through peer tutoring in higher education, University of Ulster, Coleraine

• Hay, D.B., Williams, D., Stahl, D. & Wingate, R.J. (2013). Using drawings of the brain cell to exhibit expertise in neuroscience: exploring the boundaries of experimental culture. Science Education, 97,3, 468-491

• Jenkins, A. & Healey, M. (2009). Developing the student as a researcher through the curriculum. In C. Rust (Ed.), Improving student learning through the curriculum (pp. 117–132). Oxford, England

• Komarraju, M. (2008). A social-cognitive approach to training teaching assistants. Teaching of Psychology, 35, 327-334.

• Komarraju, M., Musulkin, S. & Bhattacharya, G. (2010). Role of student-faculty interactions in developing college students’ academic self-concept, motivation, and achievement. Journal of College Student Development, 51,332-342.

• Rudduck, J. (1978). Learning Through Small Group Discussion, SRHE, University of Surrey, England