The Activities Matter: Exploring a framework to evaluate the impact of university-led outreach interventions

Annette Hayton and Dr Andrew Bengry-Howell

Podcast available via http://bit.ly/1Av87YM

Evaluating successful interventions

• During an activity such as this you can feel and sense the energy excitement enthusiasm personal transformations

• BUT - is this enough to say that this summer school was an effective intervention?

• What is success – it varies according to your position and interests• Government, OFFA HEFCE University SMTs interested in the long term

impact on progression to HE - longitudinal tracking could provide this but cannot capture the transformative process that appears to take place

• WP practitioners, students and school teachers very interested in the activity itself and its short term impact – success is seen as a perceived positive experience for participants

• Both essential but represent a real dichotomy in how we think about WP interventions

2014 Year 12 Summer School- Clear commitment to move beyond the usual generic

offerings to include subject-based sessions- Although aims were identified absence of overall

framework meant they lacked focus and coherence for example:- Provide an introduction to key aspects of university life- Develop Independent Research and Learning Skills- Raise student attainment

- Evaluation ‘added on’ and not integral to event

2014 Year 12 Summer School

“I was quite worried that I would just be sitting alone all the time in my room, but the organisers made a really good job of getting everyone to talk to each other and get to know each other.”

“I think it has been really good to actually work with the academics, rather than just have someone telling you about the course. The academics are at the cutting edge of the research and are able to tell you about what they are doing right now. That was a great opportunity.”

“I liked working in the labs. It’s so different from college, so it was good to know what sort of environment you would be working in if you did a particular course.”

“Coming on the summer school has made coming to uni less scary … because I didn’t really want to go to uni. I did, but I was really scared. So this has been really helpful.”

Progression to higher education

Capitals

Bourdieu … recognises that social or cultural capitals are expressions of differences in status and power within an unequal social system. In his analysis, economic, social and cultural capitals are all considered in terms of enabling and restricting engagement with education.

He also expanded his concept of capitals in relation to education, distinguishing between intellectual or scientific capital (subject expertise), academic capital (understanding of rules and customs within the academy) and social capital (social connections).

Whitty, Hayton and Tang (forthcoming)

Field of widening participation

Evaluating widening participation outreach

Need to develop:

1. aims for interventions informed by theory and practice

2. content which reflects the aims

3. appropriate methods to generate useful data

4. evidence to demonstrate impact and inform practice and theory

Reflexivity

Evaluation Framework

SOCIAL AND ACADEMIC CAPITAL

HABITUS EDUCATIONAL CAPITAL?

INTELLECTUAL & SUBJECT CAPITAL

PROGRESSION CURRICULUMSTUDENT

IDENTITIESSKILLS

CURRICULUMKNOWLEDGE CURRICULUM

KNOW CHOOSE BECOME PRACTISE UNDERSTANDDevelop students' knowledge and awareness of the benefits of higher education

Develop students' capacity to navigate Higher Education sector and make informed choices

Develop students' confidence and resilience to negotiate the challenges of university life

Develop students' study skills and capacity for academic attainment

Develop students' understanding by contextualising subject knowledge

Reflexivity

Framework enables us to share aims and develop activities with

- Participants- Parents- School teachers- WP outreach workers - University academic and professional support staff

2015 Year 12 Summer School

High intensity intervention

Common top level objectives for a Level 3 event inform discussions and planning

Specific activity objectives developed within the framework

Evaluation approach integral to planning using a range of methods

Common Aims with specific objectives for a Level 3 Intervention

1. Develop students' knowledge and awareness of the benefits of higher educationEnable students to investigate course and placement options, and social and leisure opportunities at the University of Bath and other universities

2. Develop students' capacity to navigate Higher Education sector and make informed choicesEnable students to evaluate university courses and make informed choices that align with personal interests and future aspirations

3. Develop students' confidence and resilience to negotiate the challenge of university lifeEnable students to anticipate challenges they will face in Higher Education and make a successful transition to university

4. Develop students' study skills and capacity for academic attainmentEnable students to enhance their academic skills through collaborative projects that develop capacity for critical thinking, independent research and self-directed learning

5. Develop students' understanding by contextualising subject knowledgeEnable students to:

(a) situate their existing knowledge within wider fields of knowledge and apply to other contexts.

(b) Situate their existing knowledge and interests within the context of university degree programmes and academic disciplines

2015 Year 12 Summer School

• Monitoring participants against agreed WP criteriae.g. State school, socio-economic class, LPN, parental education, gender, ethnicity, disability, Local Authority Area

• Longitudinal tracking – HEAT tracker• Measurable indicators for event objectives• Follow up

2015 Year 12 Summer School

Aim1: Social and Academic Capital: Know

e.g. Specific content and entry requirements of courses at University of Bath- Clickers to record students self-assessed learning- Tests to assess knowledge?- Focus groups would illustrate what they know

Aim 2: Social and Academic Capital: Choose

e.g. Evaluate different types of HEI in terms of personal interests and career aspirations- Facilitated group activity- Reflective discussions conducted by staff

2015 Year 12 Summer School

Aim 3: Habitus: Become

e.g. A positive immersive experience of HE and University of Bath- Visual documentation- Clickers to record students’ satisfaction feedback- Participant observation, field work, focus groups, staff & student

ambassador reflections- Postings on social media- Follow-up online survey

Aim 4: Educational Capital?: Practise

e.g. Enhance communication and presentation skills using different media- Peer evaluation of verbal presentations- Staff assessing posters and giving feedback- Confident participation in debates

2015 Year 12 Summer School

Aim 5: Intellectual Capital: Understand

e.g. Enhance understanding through collaborative projects, which extend knowledge and challenge assumptions

- Collaboration: active learning- Projects: creation of knowledge - Challenge assumptions: encourage new ways of thinking

- A ‘product’ to demonstrate learning and understanding- Ability to put a coherent and robust argument in a debate- Staff and student feedback on capacity of students to apply and develop

their understanding- Participant reflection

2015 Year 12 Summer SchoolAim 1: Develop students' knowledge and awareness of the benefits of higher education

Enable students to:

Investigate course and placement options, and social and leisure opportunities at the University of Bath and other universities

• Discover course and placement opportunities at the University of Bath and opportunities at other universities

• Find out about research areas, expertise and facilities at the University of Bath and new areas of development

• Explore social and leisure, and extra-curricular opportunities at the University of Bath

• Discover career benefits of Higher education and the employment opportunities for University of Bath graduates.

• Find out about academic and information services, facilities and resources at the University of Bath

2015 Year 12 Summer School

Aim 2: Develop students' capacity to navigate Higher Education sector and make informed choices

Enable students to:

Evaluate university courses and make informed choices that align with personal interests and future aspirations

• Evaluate different types of Higher Education Institution in terms of personal interests and career aspirations

• Compare degree courses and study options across a range of universities

• Engage effectively with the UCAS process and generate and submit a strong university application

• Compare student finance, budgeting support and student employment opportunities across a range of universities

2015 Year 12 Summer SchoolAim 3: Develop students' confidence and resilience to negotiate the challenge of university life

Enable students to:

Anticipate challenges they will face in Higher Education and make a successful transition to university• Gain a positive first-hand experience of student life and a university environment

• Become familiar with learning and teaching approaches in Higher Education

• Engage with academic and personal support mechanisms at the University of Bath

• Interact with academic staff and other university employees

• Interact with other students on programme, Student Ambassadors and current University of Bath students.

• Participate in challenging educational activities which are stimulating and motivating

• Celebrate in the achievements of students from similar backgrounds who have progressed to the University of Bath

• Access the information, advice and guidance they need to make a successful transition to the University of Bath or other selective university

2015 Year 12 Summer SchoolAim 4: Develop students' confidence and resilience to negotiate the challenge of university life

Enable students to:

Enhance their academic skills through collaborative projects that develop capacity for critical thinking, independent research and self-directed learning

• Enhance their capacity for independent learning, self-directed study and enterprise

• Enhance capacity for creative problem solving and decision making

• Enhance communication and presentation skills using different mediums.

• Enhance critical thinking skills through experimentation, reflection, analysis, synthesis and evaluation

• Enhance group work skills and capacity to lead and work collaboratively

• Enhance revision skills and expertise in a range of revision techniques

2015 Year 12 Summer SchoolAim 5: Develop students' understanding by contextualising subject knowledge

Enable students to:

(a) Situate their existing knowledge within wider fields of knowledge and apply to other contexts• Extend awareness of the wider applications of knowledge

• Locate existing knowledge within wider fields of knowledge and other contexts

• Enhance understanding through collaborative projects, which extend knowledge and challenge assumptions

(b) Situate existing knowledge and interests within the context of university degree programmes and academic disciplines• Relate existing knowledge and interests to university subject areas and degree

programmes

• Understand how knowledge can be developed within university subject areas and academic disciplines