collaboration on impact learning from a shared measurement approach within post 16 employability...
TRANSCRIPT
Collaboration on Impact Learning from a Shared Measurement Approach
within post 16 Employability Programmes in Northern Ireland
Gladys Swanton - CENI
Lorraine Boyd - NEET Forum
www.inspiringimpact.org
Context for Inspiring Impact
Inspiring Impact UK (2012): 10 year strategy – changing the way that Voluntary/Community sector and its funders think about impact: “putting impact practice at the heart of their work”.
Building Change Trust - Big Lottery £10 million investment focusing on the VCS in Northern Ireland. Funded Inspiring Impact NI Programme for an initial 2-years (2013) (with additional funding from the DSDNI)
IINI built upon existing work in the UK including: an Impact Practice Code; a Measuring Up assessment tool for organisations. IINI has been working across 3 main strands: Support to the VCSE on Impact Practice Engagement with funders to develop impact practice Inspiring Impact Exchange
CENI (Community Evaluation NI) is the strategic partner for the delivery of IINI
THE IDEA FOR JET
3
Shared impact measurement approach•More joined up between funder and funded on impact•Better understanding of the desired impacts and what needs to be measured•Projects supported around a common framework•Potentially more comparable data to get assess achievements & learning
…Journey towards EmploymenT
VOLUNTARY AND COMMUNITY NEET STRATEGY FORUM
We are a group of organisations working with young people and have a shared vision that
by: ‘working together so every young person in Northern Ireland is in
meaningful education, training and employment’
AIM OF THE VOLUNTARY AND COMMUNITY NEET STRATEGY FORUM
The NEET Strategy Forum works collaboratively to inform and showcase proactive work that respects the voice of, and focuses on the right of young people to access, maintain and progress to employment of their choice in Northern Ireland and beyond.
NEET STRATEGY FORUM
T.R.E.E
Oasis Youth Project
Sally Gardens Community Centre
DEVELOPMENT OF THE V&C NEET STRATEGY FORUM
Loose affiliation of organisations came together to :
• to enable those working in the V&C Sector to share information and concerns experienced by young people who experienced difficulties access Education, Training, and Employment
• to influence the development and implementation of a Strategy for young people who in need of education, employment and training
Committee for Employment and LearningInquiry into Young People not in Education, Employment or Training
Volume One
Together with the Minutes of Proceedings, Minutes of Evidence andWritten Submissions Relating to the Report
Ordered by The Committee for Employment and Learning to be printed 15 December 2010
Report: NIA 32/10/11R The Committee for Employment and Learning
NORTHERN IRELAND (NI) EXECUTIVE NEET STRATEGY ‘PATHWAYS TO SUCCESS’ IN JUNE 2012Launch of Pathways to Success Our aspiration is that, by 2020, every young person will not only have an opportunity to access education, training or other preparation for employment but, to the extent that they are able, also avail of that opportunity.
NEETS ADVISORY GROUP (NAG) INTERDEPARTMENTAL GROUP – CHAIRED BY DEPUTY PERMANENT SECRETARY
Chaired by Deputy Permanent Secretary (DEL)
12 Departments
5 Voluntary and Community Sector NEET Strategy Forum
NILGA – Local Government
Employer representative
NEET YOUTH FORUM
• Core Young people who are in need of
education, training and employment
• Young people from across NI, Age
ranging from 16 – 24
• Share experience and knowledge
• Inform their choice of preferred best practice.
• Have genuine constructive input into Northern Ireland NEET reduction agenda
KEY COMPONENTS OF WORK INCLUDE:
Collaboration
Communication
Best Practice
Mapping provision and production of directory
Youth Participation.
• Initial presentation of JET to NEETs Forum
• Building Change Trust support to pilot
• Consultation to develop a draft NI framework
• Testing of JET Framework & Process in NI
• Expression of Interest from 14 C/V/SE groups
• 6 pilots were identified for support
JOURNEY TO EMPLOYMENT (JET) FRAMEWORK – Pilot in NI ………
JOURNEY TO EMPLOYMENT (JET) FRAMEWORK – LAUNCHED MAY 2013
A shared approach to measuring young people’s employability
+ +
+
++
+Intrinsic factors
Extrinsic factors
External factor: The labour market
Personal circumstances
Emotional capabilities
Attitudes
Employability skills
Qualifications education +
training
Experience + involvement
Career management
Journey to EmploymenT (JET) framework
JET OUTCOMES (NI draft)
PERSONAL CIRCUMSTANCES•Access to transport•Personal Safety •Access to the internet•Access to childcare•Access to support for young carers•Supported by family and friends•Access to support for disabled people•Reduced substance abuse•Resisting negative peer influences •Reduced offending/anti-social behaviourEMOTIONAL CAPABILITIES•Self-esteem•Autonomy and control•Grit and determination•EmpathyATTITUDES•Aspirations for education•Attitudes to work•Aspirations for work•Aspirations for the futureEMPLOYABILITY SKILLS•Teamwork•Communication•Problem solving •Self-management•Leadership
QUALIFICATIONS, EDUCATION + TRAINING•Basic skills•Achieving qualifications•Attendance and behaviourEXPERIENCES + INVOLVEMENT•Work experience•Perception of value of work experience•Networks•Volunteering •Inter-community engagement CAREER MANAGEMENT•Career direction•Job search skills•Presentation to employers•Confidence in finding employment •Entrepreneurship
EMPLOYMENT•Entry into employment•Sustaining employment•Quality of employment•Satisfaction with employment
Planning
• Complete theory of change with each pilot group
• Prioritising & matching to JET Framework
Measuring
• Select measures & designing
• Implementing/ sample/timeframe
Analysis & Use
• Pre and post analysis
• Using the learning
Implementing JET in NI
Six Participating Groups
Support delivered May-Oct14
Organisation Pilot Project Area
Barnardo’s Bup4Work
Digital Media Choices
Digital Media Training Programme at Community Centre Belfast
Princes Trust ‘Get Into Social Care’
Sandy Row YES Project - Event Management & Community Engagement
Start 360 Switch On to Employment
Tides Training Tides Training – Shared History Group
JOURNEY TO EMPLOYMENT (JET) FRAMEWORK – Pilot in NI
Step1: THEORY OF CHANGE COMPONENTS
Enabling factors: things that have to exist for the theory of change to happen
Activities: what the charities’ services are
Outputs: what services are provided
Outcomes: Short – Medium term changes that the charity wants to happen
Evidence or assumptions: why you think one outcome will lead to another and cases where it will not
Impact: Ultimate goal - what the charity is trying to achieve
Digital Media Example
Digital media training programme over 30 weeks/one day a week
Enablers:Access to up-to-date quality training, equipment, resources, expertise and practical work experienceLearner friendly approach and peer support
YP have increased knowledge/skills in use of a range of digital media
Y/P have increased confidence/ self-esteem/ achievement
Project activitiesProject Outcomes – Short - Medium
Y/P have improvedemployability skills: •Improved team-working•Improved communication•Improved problem solving•Improved self-management•Showing leadership
Y/P secure jobs in digital media
YP have greater choices whether to pursue further education/ training/ employment /self-employment ie entrepreneurship
YP achieve qualifications and experience required by employers in digital media
YP have positive role models
End Goal/Impact:
Y/P gain entrepreneurial skills / know what is involved in developing a business in the digital media field
Y/P are up-skilled and inspired
Work experience to contribute to digital media projects
Y/P are willing to engage with work/ learning outside of their local area
Priority Outcomes and JET Measures Table Example
Your Theory of Change Outcome
JET Outcome Area
JET Outcome Measure
JET Outcome Tool
Y/P have increased confidence
Emotional Capabilities
Self-esteem Rosenberg’s Self-Esteem (p41)
Y/P have improvedemployability skills
Employability Skills
Employability Skills Confidence/
Employability / Personal Development Scale (pg 52)
Y/P gain entre-preneurial skills / how to developing a business in digital media
Career Management
Entrepreneurship
Entrepreneurship / Starting a business (pg 74)
STEP 4: USE THE JET FRAMEWORK TO SELECT MEASURES
22
Example: Rosenberg Self-esteem scale
Step 5: CHOOSE A RESEARCH DESIGN
23
Before the course:Pre-questionnaire
Intervention
After the coursePost-questionnaire
Long term follow up
Outcomes Prioritised by NI Pilot Projects
JET Outcomes
0 1 2 3 4 5 6
Self- Esteem
Employability Skills
Inter -community engagement
Value of Work Experience
Aspiration for the Future
Attitudes to Work
Communication
Confidence in Finding Employment
Empathy
Entrepreneurship
Personal Safety
FINAL STEPS
STEP 6
IMPLEMENTING MEASUREMENT TOOL
25
STEP 7
ANALYSE THE DATA
STEP 8
LEARN AND IMPROVE
Feedback on the Outcomes and measures…
The JET framework enabled a broader range of outcomes to be captured at a project level; outcomes which are not often articulated or measured
The JET pilot was part of the wider strategic level discussions on impact measurement which added
value to the whole process
“ The JET process supported the development of good impact practice within the organisation including: planning impact, using new tools to measure, building in analysis, learning and reporting
The use of externally validated measures strengthens the evaluation process and provides greater credibility when reporting about achievements
Views on use and implementation of JET resources
The visual presentation on the JET process and the Framework was seen as user friendly and eye catching
Looked at ‘Impact Practice’ in more systematic way. Breaking the process down into 8 distinctive inter-linking steps helped to make it manageable – all the steps were important
It was useful to have a menu of measures to draw from and be able to pick what suited – but there are some challenges with “off-the-shelf” measures
“ Theory of Change was a useful process to think through – making the journey explicit “how to enable young people to move from hopelessness to aspiration”
The pre and post questionnaires were straightforward to use and advice in the JET pack on the practicalities of implementation useful
The pre-coded Excel analysis sheets produced a range of data useful for reporting – however user-friendly formats for presentation of data would be useful
Taking it Forward
• All the pilots groups indicated interest in using JET in their work
• Pilot groups valued the CENI support in navigating their way through the 8 steps - training and support is needed
• Feedback from the pilot support informed the final developmental phase – plans for user-friendly web-based version
• Access to Resources
– JET Framework NI on-line
– Case studies made available on-line
– Process Report on Piloting JET in NI
Thank You
• Lorraine Boyd Regional Project Manager NEET Strategy Forum NI
• E: [email protected]• T: +44(0)2890325 835 • WWW.Neetforumni.org