dyfodol evaluation final report 08.02 › docs › wefo › publications › 130719...• the...
TRANSCRIPT
Mid Term evaluation of the DYFODOL –
Skills in Employment Project
Final Report
January 2013
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 1
Acknowledgements
We would like to offer our thanks to all those interviewed as part of the evaluation who gave their
time so willingly and answered our questions in such detail. This report would not have been
possible without all these contributions.
Report prepared by:
Bob Townley
Report version: Final
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 2
Contents
1. Executive Summary ................................................................................................... 3
1.1. Project background ................................................................................................................. 3
1.2. The evaluation process ........................................................................................................... 3
1.3. Key findings ............................................................................................................................. 3
1.4. Key successes and issues to address ....................................................................................... 6
1.5. Recommendations .................................................................................................................. 6
2. Brief Description of the Project .................................................................................. 7
2.1. Project aims and objectives .................................................................................................... 7
2.2. Project targets ......................................................................................................................... 8
3. Evaluation Overview .................................................................................................. 9
4. Evaluation Findings .................................................................................................. 10
4.1. Progress towards project targets .......................................................................................... 10
4.2. Strategic level feedback ........................................................................................................ 12
4.3. Delivery level feedback ......................................................................................................... 21
4.4. Participant employer feedback ............................................................................................. 31
5. Conclusions and recommendations .......................................................................... 44
5.1. Key successes ........................................................................................................................ 44
5.2. Issues to address ................................................................................................................... 44
5.3. Recommendations ................................................................................................................ 44
5.4. Progress towards targets ...................................................................................................... 45
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 3
1. Executive Summary
1.1. Project background
The aim of the project is to develop and deliver a range of training programmes focused upon the
needs of private sector employers in the North West Wales Convergence area (Anglesey, Conwy,
Denbighshire and Gwynedd). The project involves a partnership of four FE colleges: Coleg Llandrillo
(Lead), Coleg Menai, Deeside College and Coleg Harlech (WEA). Funded through Priority 3, Theme 1,
of the Convergence Programme in Wales, the project commenced in February 2011 and is set to run
for three years until February 2014.
The aim of the project is to develop and deliver a range of accredited training programmes at Levels
1 to 3 focused upon the needs of employers in the area, targeted towards four of the five priority
sectors identified within the Welsh Government’s Economic Renewal Policy (ERP) published in July
2010 and the three additional sectors identified in Autumn 2011. The project offers work-based
NVQs and a range of shorter courses tailored specifically to sector needs and aimed predominately
at employees with low skills, mainly within SME’s. The project is delivered by a team of Business
Development Advisors (BDAs)\Business Liaison Officers (BLOs), peripatetic work-based assessors and
trainers.
1.2. The evaluation process
This Mid Term Evaluation is the first stage of the evaluation process and addresses both the process–
orientated elements of the project and outcomes to date. This report presents findings from a range
of evaluation activities conducted between October and December 2012 including:
• Desk research, including the review of project monitoring information (including Steering Group
and Operational Group minutes etc.);
• Strategic level interviews;
• Delivery level interviews / discussion groups;
• Employer survey.
1.3. Key findings
Findings are reported in relation to the following four elements of the evaluation process:
• Progress towards project targets;
• Strategic level feedback, based on interviews with members of the project steering group,
management team and external stakeholders;
• Delivery level feedback, based on discussion groups with all advisor teams and interviews with
selected curriculum managers;
• Participant employer feedback, based on a telephone survey of 100 participating employers.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 4
The summarised key findings from these elements of the evaluation process are reported,
separately, below.
Progress towards project targets – key findings
• The Partnership is over-achieving against the main targets for companies assisted and employee
participants.
• The Lead Partner is making the most substantial contribution to the achievement of these
targets. While other partners are making a lesser contribution to the total target, they are
largely on profile for the number of companies assisted.
• The Partnership is substantially below the target for the number of employers adopting or
improving equality and diversity strategies. We recognise that this is a common issue across EU-
funded projects and that many projects are currently struggling to meet targets in relation to
cross-cutting themes.
• The Partnership is on target to achieve its other equality targets apart from ‘participants with a
work limiting health condition’.
Strategic level feedback – key findings
• The employer engagement process has been highly successful, particularly in reaching smaller
businesses, and Business Development Advisors have been instrumental in the success of this
process. Partners recognise the value of the project in supporting their organisation’s evolving
role in employer engagement.
• The centrality and effectiveness of the Advisor role in identifying employers’ training needs,
through the standardised Training Needs Analysis process.
• The shared commitment to addressing existing local skills needs; going beyond ‘priority sectors’
to meet evolving local / regional skills needs.
• Concern that the budget spend is currently around £500k under profile. The Partnership is in
discussions with WEFO for a possible extension to address this.
• The requirement for an open procurement process has brought an issue to the surface forcing
the Partnership to acknowledge and address an imbalance in the nature of delivery, towards a
better spread of short courses and longer (NVQ equivalent) qualifications. While causing a hiatus
in delivery, this is not expected to adversely affect project performance.
• That the Partnership is highly unlikely to achieve the employer equality and diversity target but
that it will be important to demonstrate the efforts, processes and resources that have been
directed towards meeting this objective.
• An embedded bilingual approach, with all materials, support and a wide choice of courses in
either medium or bilingually.
• A resounding acknowledgement that the Partnership is working very effectively as a team, with
strong and open communication and effective sharing of best practice to achieve the overall
objectives and targets. The Lead Partner is widely commended for its leadership, professionalism
and support.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 5
Delivery level feedback – key findings
• Advisors are working within strong existing networks of employers, linked to their sector
specialisms, and high levels of engagement have been enabled through these networks.
• The standardised Training Needs Analysis process, linked to business objectives, provides an
opportunity for assessing the extent to which business objectives have been achieved. As
suggested by one group of advisors, this approach also provides a potential framework within
which to integrate Equality and Diversity considerations, linked to business objectives, so that
the TNA takes a holistic approach to identifying the future workforce needs for each business.
• Advisors have responded effectively to specific employer requests for courses not currently
within the course listings, jointly with college curriculum managers and other partners.
• The hiatus in delivery of workplace safety short courses is seen by advisors to have been a
significant issue impacting on their dealings with employers and training delivery but largely
recognise that a process is in place to resolve the issue and effective working relationships with
employers has helped to keep employers ‘on board’.
• Some problems have been encountered in ensuring that employees undertake training once
relevant courses have been identified but new initiatives are being tested to address this.
• Equality and diversity issues have met with much resistance from (especially small) employers. In
some cases advisors appear to take the lead from employers and not pursue this further. In
other cases advisors are taking a more proactive approach to persuade employers of the
potential benefits / value of formal equality and diversity processes.
Participant employer survey – key findings
• Just under half of employers within the sample stated that the TNA process had helped either
“fully or very well” in identifying their business training needs. A further 28% felt that it had
helped “quite well” together accounting for three-quarters of the sample.
• Overall, 84% of employers feel that the skill levels of their staff have increased as a result of the
training received. Two-thirds (66%) feel that these staff are now more productive, 63% feel that
these staff are more likely to progress in the business and 84% feel that these staff are now
more employable in the wider labour market.
• A very high proportion (78%) of participating employers feel that, as a result of their
participation in the project they are either “very likely” or “quite likely” to continue to invest in
the future development of their workforce.
• Nine out of 10 participating employers rate the value for money as either “very high” or “quite
high.”
• Around a quarter (27%) of employers within the sample stated that they had completed an
equality and diversity review.
• The responses indicate that while the nature of the ‘review’ may have been misunderstood a
substantial proportion of participating employers feel that they are now more aware of equality
and diversity issues in relation to their business and that they have experienced a valuable
intervention in relation to these issues.
• A discernible impact has occurred with evidence of new policies and processes created and
recognition of the link between equality and diversity policies and business success. In 56% of
cases where employers state that an equality and diversity review has taken place they have
either developed their existing policies or created new policies and 87% of these employers state
that they will continue to develop these polices in the future.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 6
1.4. Key successes and issues to address
Drawing on these findings, the following successes and issues to address are identified:
Key successes
• The employer engagement process has been highly successful and Business Development
Advisors have been instrumental in the success of this process. Partners recognise the value of
the project in supporting their organisation’s evolving role in employer engagement.
• High levels of employer satisfaction with training delivery and support and a stated willingness
to invest in future training.
• The evolving cross-partnership offer available to participating employers.
• The centrality and effectiveness of the Advisor role in identifying employers’ training needs,
through the standardised Training Needs Analysis process.
• The shared commitment to addressing existing skills needs; going beyond ‘priority sectors’ to
meet evolving local / regional skills needs.
• Strong and effective partnership working, with open communication and effective sharing of
best practice to achieve the overall objectives and targets.
Issues to address
• Adaptation of TNA practices and advisor approaches to address the employer equality and
diversity target.
• Successful completion of the procurement process, addressing a noted imbalance in course
delivery.
• The current budget under-spend.
• Problems encountered in ensuring that employees undertake training once relevant courses
have been identified.
1.5. Recommendations
Drawing on our findings, analysis and discussions with members of the Project Steering Group we
recommend that:
1. The Partnership continue to address the current budget under-spend as a matter of priority
2. A more integrated, business-needs-led, approach may be required, also addressing equality and
diversity issues with further specific training and support for advisors in relation to equality and
diversity issues.
3. Different methods for participant enrolment continue to be tested and implemented (if
effective) in the final stage of the project.
4. Emerging effects / impacts in relation to business objectives are captured by advisors for the
final evaluation.
5. The final evaluation will need to consider the effectiveness of the current procurement process
(and the impact on the wider training market) and the resulting change to the balance of
courses to meet employer needs.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 7
2. Brief Description of the Project
2.1. Project aims and objectives
The aim of the project is to develop and deliver a range of training programmes focused upon the
needs of private sector employers in the North West Wales Convergence area (Anglesey, Conwy,
Denbighshire and Gwynedd). The project involves a partnership of four FE colleges: Coleg Llandrillo
(Lead), Coleg Menai, Deeside College and Coleg Harlech (WEA). The project commenced in February
2011 and is set to run for three years until February 2014.
The total eligible cost of the project is £7,667,909 with a maximum grant available of £4,606,749,
funding through Priority 3, Theme 1, of the Convergence Programme in Wales: Improving Skills
Levels and the Adaptability of the Workforce1, as outlined in the diagram below:
1 http://wales.gov.uk/docs/wefo/publications/strategicframeworks/100202raisingskillsbaseframeworken.pdf
Strategic Framework
Raising the skills base of the workforce and supporting progression in employment through basic
and intermediate level skills
Theme 1
The objective of this theme is raising the
skills base of the workforce and
supporting progression in employment
through basic and intermediate level skills.
> Raising levels of basic literacy and
numeracy in the workforce;
> Supporting the acquisition of ICT,
generic and occupational skills in the
workforce;
> Supporting progression in employment
through flexible training and learning
provision; and
>Helping low skilled workers to gain the
additional skills and qualifications needed
to sustain their employment, improve
productivity and increase earnings via
career progression.
Priority 3
Improving Skills Levels and the Adaptability
of the Workforce
The aim of this Priority is to support
productivity and progression in employment
by raising skill levels at the levels of basic
skills, intermediate and higher level skills, to
tackle skills gaps and shortages, and to
promote gender equality in employment.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 8
According to the Partnership’s Business Plan (April 2012), the aim of the project is to develop and
deliver a range of accredited training programmes at Levels 1 to 3 focused upon the needs of
employers in the area. In terms of the Welsh Government’s Economic Renewal Policy (ERP)
published in July 2010 the project aims to target four of the five sectors identified: ICT; Energy and
Environment; Creative Industries and Financial and Professional services and the three additional
sectors identified in Autumn 2011 (Tourism/Hospitality, Farming and Food and Construction). The
project offers work-based NVQs and a range of shorter courses tailored specifically to sector needs
and aimed predominately at employees with low skills, mainly within SME’s.
The project is delivered by a team of Business Development Advisors (BDAs)\Business Liaison
Officers (BLOs), peripatetic work-based assessors and trainers. The BDAs make contact and visit
individual businesses and undertake training needs analysis and follow-up visits. The training is then
delivered either at college premises or in the workplace. Employers are then able to participate in a
range of NVQs and short courses determined by the Partnership in association with SSCs and local
employers. We understand that the ‘offer’ is subject to evolve during the lifetime of the project to
meet the ever-changing needs and opportunities of the local labour market
2.2. Project targets
The following key indicators and targets are already in place with WEFO and form the basis of
ongoing monitoring and evaluation by the Project Management Team, Steering Group and
Operational Group:
Indicators – Targets and Results Overall Project Target
P3T1
Employed participants 5021
Female participants 60%
Companies assisted 800
Participants gaining a qualification 4571
Level 1 achievement 1142
Level 2 achievement 2560
Level 3 achievement 457
Participants gaining part qualifications 10%
Participants entering further learning 1500
Cross-
cutting
Employers adopting or improving equality and diversity
strategies and monitoring systems 200
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 9
3. Evaluation Overview
This Mid-term Evaluation is the first stage of the evaluation process and addresses both the process–
orientated elements of the project and outcomes to date, across the following three phases:
This report presents findings from a range of evaluation activities conducted between October and
December 2012 including:
• Desk research, including the review of project monitoring information (including Steering Group
and Operational Group minutes etc.);
• Strategic level interviews (x12) (Management Team / Steering Group and external
stakeholders);
• Delivery-level interviews / discussion groups (x4 discussion groups with BDA / BLO teams and
interviews with 4 Curriculum Managers);
• Employer survey (a telephone survey of 100 participating employers).
Through these interviews and discussions, we have addressed a range of evaluation questions,
developed through the inception process captured within discussion guides and a survey
questionnaire pre-agreed with the Project Management team.
Phase 1
2012
• Inception Report and Evaluation Framework: October to November 2012
Phase 2
2012
• Data Collection and Fieldwork: November to December 2012
Phase 3
2013
• Analysis, Reporting and Dissemination: December 2012 to January 2013
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 10
4. Evaluation Findings
The section below presents findings in relation to the following evaluation activities:
• A review of progress against targets;
• Feedback from a series of interviews with Strategic Level partners;
• Feedback from a series of interviews with Delivery Level partners;
• Feedback from a survey of participating employers.
4.1. Progress towards project targets
4.1.1. Summary of key findings
1. The Partnership is over-achieving against the main targets for companies assisted and employee
participants.
2. The Lead Partner is making the most substantial contribution to the achievement of these
targets. While other partners are making a lesser contribution to the total target, they are
largely on profile for the number of companies assisted.
3. The Partnership is substantially below the target for the number of employers adopting or
improving equality and diversity strategies. We recognise that this is a common issue across EU-
funded projects and that many projects are currently struggling to meet targets in relation to
cross-cutting themes.
4. The Partnership is on target to achieve its other equality targets apart from ‘participants with a
work limiting health condition’.
4.1.2. By the Partnership as a whole
As shown in the tables below, the latest data submitted to WEFO demonstrates that the Partnership
is over-achieving against its profile in terms of companies assisted and employed participants;
substantially beyond the final target for companies assisted. As the data shows, the Lead Partner
(Coleg Llandrillo) is making the most substantial contribution to the achievement of these targets,
way beyond the expected profile up to this point in time. While other partners are making a lesser
contribution to the total target, they are largely on profile for the number of companies assisted. As
discussed more fully below in the section on feedback from strategic partners, there have been
some challenges in signing up larger employers and converting employer take-up into course
enrolments. This has led to an increased focus across the Partnership on signing up additional
participants from companies that have been engaged, reviewing processes and sharing successful
practices between partners. The outcome of this shift in focus will hopefully become apparent in the
participant target profile and future progress reports.
The data on qualifications shows that the Partnership is ahead of profile on the overall number of
participants achieving a qualification, with qualifications mainly achieved at Level 2.
The data on the number of employers adopting or improving equality and diversity strategies and
monitoring systems shows that partners are substantially below this target and this issue is
addressed in detail elsewhere in this report. We recognise that this is a common issue and
understand that cross cutting themes are not being met by many projects.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 11
Achievement against Targets - 31st
October 2012 (7th
Claim)
Indicators – Targets and Results Total Target
Profile
Target
Actual / Profile
Difference
P3T1
Employed participants 5021 3002 3117 (+115)
Female participants 60% 60% 49% (-11%)
Companies assisted 800 603 924 (+321)
Participants gaining a qualification 4571 998 1,193 (+195)
Participants entering further learning ** 1500
Cross-
cutting
Employers adopting or improving equality and
diversity strategies and monitoring systems
200 11*
* Does not include companies who have had a review and not yet adopted new practices or companies that have been
reviewed but have not had any improvements recommended
** Data for participants entering further learning has not been collected and discussions are ongoing on how to do this
By Partner
Performance v Total Targets
Companies Participants
Target Actual Target Actual
Llandrillo 320 595 2010 1988
Deeside 104 60 653 328
Menai 288 208 1809 640
Harlech 88 59 553 161
Performance v Profile
Companies Participants
Target Actual Target Actual
Llandrillo 301 595 (+294) 1160 1988 (+828)
Deeside 79 60 (-19) 448 328 (-120)
Menai 172 208 (+36) 1081 640 (-441)
Harlech 51 59 (+8) 313 161 (-152)
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 12
Other equality targets
The Partnership is on target to achieve its other equality targets apart from ‘Participants with a work
limiting health condition’. Partners recognise a range of issues in relation to the achievement of this
target including the problems of definition and reluctance on the part of staff to declare this. We
envisage that other ESF-funded projects across the UK will be encountering similar issues with this
target.
Target Actual
BMEs 2% 1.9%
Older workers (50+) 20% 24% (762 employees)
Participants with a work limiting health condition 10% 1.8% (57 employees)
4.2. Strategic level feedback
This section reports feedback from a series of interviews with those responsible for providing
strategic direction and management (Steering Group members, including the Project Management
team) with some additional comments from one external stakeholder.
Key findings are:
1. The employer engagement process has been highly successful, particularly in reaching smaller
businesses, and Business Development Advisors have been instrumental in the success of this
process. Partners recognise the value of the project in supporting their organisation’s evolving
role in employer engagement.
2. The centrality and effectiveness of the Advisor role in identifying employers’ training needs,
through the standardised Training Needs Analysis process.
3. The shared commitment to addressing existing local skills needs; going beyond ‘priority sectors’
to meet evolving local / regional skills needs.
4. Concern that the budget spend is currently around £500k under profile. The Partnership is in
discussions with WEFO for a possible extension to address this.
5. The requirement for an open procurement process has brought an issue to the surface forcing
the Partnership to acknowledge and address an imbalance in the nature of delivery, towards a
better spread of short courses and longer (NVQ equivalent) qualifications. While causing a
hiatus in delivery, this is not expected to adversely affect project performance.
6. That the partnership is highly unlikely to achieve the employer equality and diversity target but
that it will be important to demonstrate the efforts, processes and resources that have been
directed towards meeting this objective.
7. An embedded bilingual approach, with all materials, support and a wide choice of courses in
either medium or bilingually.
8. A resounding acknowledgement that the Partnership is working very effectively as a team, with
strong and open communication and effective sharing of best practice to achieve the overall
objectives and targets. The Lead Partner is widely commended for its leadership,
professionalism and support.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 13
4.2.1. The value of the project in enhancing organisational strategies
Partners recognise the value of the project in supporting their organisation’s evolving role in
employer engagement, particularly in reaching small and micro businesses and in adding value to
this engagement role through the employment (and development) of the BDA/BLO Business Advisor
team. In the words of one partner: “allowing us to have that better engagement with employers as
well as increasing the number of training opportunities that we are able to deliver.” For one partner
the project is also seen to be enhancing their strategic aims through a focus on the Third Sector and
enhancing Trade Union links, particularly through the Union Learning Fund. While there is a general
recognition of the plethora of other projects / programmes targeted at employers across the region,
several partners also refer to the natural synergy of some of these, existing, projects (including the
Essential Skills and Skills for the Workforce projects). It is the specific role of one member of the
Steering Group to facilitate links between Dyfodol and other ESF-funded projects across North Wales
(as well as informing the project of relevant regional LMI and the skills needs expressed through the
network of Sector Skills Councils).
4.2.2. Developing an offer aligned with wider sectoral priorities
The general consensus among partners is that, while they are aware of the ‘Priority Sectors’,
established in the Welsh Government’s Economic Renewal Policy, and amended for the Dyfodol
Business Plan, the focus has also been on local and regional need which have evolved over the
course of the project. The strategic importance and skills needs of other sectors (eg, Tourism and
Health and Social Care) that were not included in the original list of ‘Priority’ sectors have become
evident through the project (and regional LMI) and led to their prominence in the profile of
supported businesses. It is clear that all partners share a commitment to addressing existing local
skills needs and that they have not been constrained by a focus solely on national policy guidance. In
the words of one partner: the aim of Dyfodol is to improve skills and employability and support
everyone at risk of redundancy “if an employer comes to us we won’t turn them away because
they’re not in a strategic sector.”
In some cases partners have also faced constraints in engaging employers from the key Priority
sectors, including large Energy employers who have experienced difficulties engaging with the
bureaucratic requirement of the ESF funding programme that clashes with a) their internal (UK wide)
HR training systems (and mass procurement of training) and b) data protection rules re employee
payment details and NI numbers. This has also prevented the planned engagement of businesses in
the supply chain.
In other areas partners note that this sectoral objective has clashed with other funding streams,
including the Sector Priorities Fund which is supporting work-based training in key sectors, including
the Creative Industries (through new Apprenticeships). Conversely the partner network is delivering
in other ways to employers across all ‘Priority’ sectors, beyond the Dyfodol project.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 14
4.2.3. Engaging employers and identifying their training needs
Partners refer to a major, multi-faceted, effort in engaging employers (reflected in the current target
profile) but also note the variance across partners in their contribution to this target. On this issue
we note the comment made by one partner in relation to the way in which the employer
contribution is (re)allocated to delivery departments, as an incentive for college Directors to take an
interest in the project. This may provide a model for those partners who have, up until now, made a
lower contribution to the project’s employer engagement targets.
The advisor role is generally seen to be instrumental in the engagement of employers (through
existing relationships and building new alliances / tapping into existing networks). Advisors are seen
to have been particularly successful in engaging smaller employers, while partners also recognise the
value and incentive provided by the subsidy in the current economic climate. In line with the
comments above, partners stress that a strong sales team is being developed through Dyfodol and
that the project is helping to establish Business Point and Linc Menai as recognised and valued local /
regional business support organisations. Partners also emphasise the inherent value in the BDA/BLO
role in promoting a wide range of courses across the provider network; offering employers a broad
choice of courses, provider and location. In terms of the project’s profile, one partner stresses that
the name Y Dyfodol is key, appearing in various related policy reports referring to the ‘future’ as “a
sublime message” to the wider audience of all those interested in skills and business development
across Wales.
We note that partners have used their funding and resources differently in delivering the BDA role
and that one partner has tended to rely more on assessors out in the field with businesses; in their
view using those resources more effectively to engage participants within businesses that assessors
are already working with and looking to the longer term, beyond project funding.
Partners also recognise the centrality and effectiveness of the Advisor role in identifying employers’
training needs, through the standardised Training Needs Analysis process. This process, and its value,
is addressed elsewhere in this report.
4.2.4. Engaging participants
While the project is on profile in meeting its participant target, partners note that there has been
some variation across providers and that, while the project has been highly successful in engaging
employers, there have been some challenges in converting employer take-up into registrations on
courses. This is reflected in the data, where the ratio of participants to employers has (up until now)
fallen behind what was originally anticipated. This has led to an increased focus on signing up
additional employees from employers that have been engaged, reviewing processes and sharing
successful practices (including through discussions at the regular BDA group). Partners also note that
the employer : participant ratio has also been affected by the nature of the cohort of (mainly small)
employers and constraints in signing up some major regional employers (notably Wylfa) that
potentially would have brought large numbers of participants into the project.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 15
4.2.5. Developing / providing courses to meet identified needs
In asking partners about how they have shaped their offer in response to identified needs we also
related this to the extent to which the offer has been determined by the range of courses presented
within the Dyfodol Course List.
In response, partners generally recognise the value of the Course List as a resource created through
the project (in response to national and regional LMI, existing knowledge and discussions with
employer groups and Sector Skills Councils) but also stress that they have sought to go beyond the
range of courses on the list to develop their own college curricula in response to emerging needs. In
the words of one partner: “It’s not about telling them what you can offer but finding out what they
need and seeing if we can deliver.” According to another partner: “It’s a bit of a Catch 22, we can’t
just go on the basis of asking what they want…lots of businesses don’t always know what they
want.”
Partners refer to the development of a range of courses (including Care, Retail, Nuclear, Renewables
and Energy) through the project that weren’t previously available as a package to employers and to
the development of new (online) modes of delivery.
Partners generally acknowledge the proliferation of workplace safety short courses that have been
delivered through the project and that these courses had perhaps become too much of a focus
within the project. One partner notes that this: “has encouraged us to think about what we are
offering and encouraging people to do non Health and Safety courses aswell….it would have been
better if we had been able to do this at the beginning.” While there is frustration among some
partners that a contractual agreement to deliver these courses has been stymied, other partners
refer to an evolving, positive, shift away from these workplace safety courses to other emerging
areas of need. This includes an increasing focus on leadership and management short courses and
towards longer (NVQ equivalent) qualifications with workplace safety as an element of those
courses. This approach is also described, below, in the section on Advisor feedback.
As one partner also notes, from the period of project development to date there has been a growth
in some Welsh Government funding streams such as WBL which has had an impact on the project,
particularly in terms of the take up of longer (NVQ and equivalent) courses.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 16
4.2.6. Identifying employers requiring assistance developing Equality and
Diversity policies and providing support to those employers
Partners generally acknowledge that, although specific processes and resources have been
established to meet this objective, achieving this target has been extremely challenging and highlight
the following constraints and considerations:
• A policy ‘overkill’ in relation to this objective, as a required element within all ESF funded
projects and one that will be addressed in some form with all employers accessing various
support initiatives.
• Whether employers understand the need for the policies, particularly in the case of small and
micro businesses who don’t have basic ‘HR’ policies in place.
• A wariness and suspicion on the part of employers that the project will expose them to other
authorities as not meeting their legal (Equal Opportunities) requirements.
• The extent to which advisors are actively ‘selling’ this element to employers, as a business case.
• That the target only accounts for those employers who can demonstrate they are implementing
new equality and diversity processes (following the project review) and do not account for any
(other) attitudinal changes that may be having an informal impact on the business.
• That advisors do not have the experience and expertise to deliver this and it should be left to
other organisations with this specific experience and expertise.
• That the project / advisors have no ‘stick’ to persuade employers to review their policies and
can only dangle ‘carrots’ (e.g. in terms of making the case for the importance / necessity of
having formal E&D policies in place if / when tendering for public sector contracts).
From our previous experience of evaluating programmes aimed specifically at promoting equality
and diversity policies within small and medium size businesses (across Wales) we support a number
of these observations. We recognise that this is a particularly ‘tough nut to crack’ with smaller
businesses and that employers are naturally suspicious of the motives when the approach is from a
public sector body. But, in our experience, they will respond more positively where they are
presented with potential business benefits that directly relate to their business needs2. In this sense,
presenting equality and diversity policies as a separate, add-on, element to employers is likely to be
met with much resistance and / or lack of interest and we suggest that a more integrated, business-
needs-led, approach may be required (as is currently being tested / demonstrated by certain
advisors). This said, we commend that this objective has been given a high prominence within the
Partnership with a genuine commitment (and a range of processes tested) to achieve it and to
understand the constraints encountered so far.
2 Evaluation of the Pan-Wales SME Equality Project, Wavehill (2006)
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 17
4.2.7. Monitoring progress; milestones and targets and the role of the
Steering Group
While the Partnership is over-achieving on its (final) employer engagement target and is ‘on profile’
for participant numbers there is some concern that the budget spend is currently around £500k
under profile. In terms of projected spend, the Lead Partner is over budget but other partners are
under budget. While the Partnership is addressing this issue with WEFO (through a possible 6-month
extension) and there are some administrative issues (relating to completion of registration data),
that are currently being addressed, there is some concern within the Partnership that this
underspend may grow. There is also a common recognition that the Partnership is highly unlikely to
achieve the equality and diversity target but that it will be important to demonstrate the efforts,
processes and resources that have been directed towards this objective.
One issue reported, separately, by two partners is the effect of a slow start to the project that has
significantly hampered their contribution to the projects targets. In one case the partner referred to
the need to “re-energise” employers after previous funding rounds, with a tendency towards a
hectic final 12 months of delivery.
In a number of cases partners cite the importance of the Steering Group and Operational Group in
continually emphasising the importance of targets as a prominent item on the Steering Group and
Operational Group agendas; with open sharing of support and good practice encouraged between
partners. In this vain, there is a resounding acknowledgement that the Partnership is working very
effectively as a team, with strong and open communication and effective sharing of best practice to
achieve the overall objectives and targets, also successfully overcoming inherent competition
between partners. The Lead Partner is widely commended for its leadership, professionalism and
support.
In terms of structure, one partner refers to a need for greater external (employer) representation on
the Steering Group (while acknowledging the value and contribution of current external members.)
4.2.8. Developing wider regional partnership relations and synergies with
other partners and programmes (including Shaping the Future and the
Essential Skills Employer Pledge)
Several partners refer to links made to the Essential Skills project but two partners note that this
project has not performed well and is a much harder ‘sell’ to employers; that may have worked
better as part of an integrated package with Dyfodol. Little mention is made of links to the Shaping
the Future project although we are aware that efforts have been made at a strategic level to explore
this relationship and that there is a fundamental mismatch between the levels of training offered
through each project. Other links were cited to foundation degrees and Apprenticeship programmes
offered through the college network and some partners refer to a natural, regional, link between
Dyfodol and the complementary Skills for the Workplace project running across North East Wales.
Several partners note the role played by advisors in identifying and promoting complementary
projects and other forms of support for employers.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 18
Other partners have mentioned links made with other external organisations to support the
achievement of specific targets (including with Chwarae Teg in relation to gender targets) and to
complementary activities under the Union Learning Fund, identifying up-skilling needs within
unionised workplaces.
Other issues raised include a perceived state of confusion and competition between projects, with
employers often unsure of who is delivering what. A question was raised as to the lack of a strategic
regional co-ordinating role.
4.2.9. Identifying and managing risks to delivery and the achievement of
targets
The major risk to delivery, referred to by all partners, has been the restriction imposed by WEFO on
the delivery of ‘workplace safety’ short courses on the basis that they were being subsidised through
the project, and offered to employers at a substantially reduced rate which was unfairly disrupting
the market for private training providers. Discussions with partners show that the Partnership has
dealt proactively with this issue and has reached a solution with WEFO where a formal procurement
process has been established for the delivery of these short courses, open to all (public and private)
providers.
While partners note that that this issue has created a hiatus in delivery, adversely affected by the
(acknowledged) over proliferation of these courses, the majority do not feel that this will adversely
affect the overall performance of the project. Indeed, it appears that this has brought an issue to the
surface that perhaps should have been considered at the project’s inception (seen by several
partners as a lesson learnt) and forced the Partnership to acknowledge and address an imbalance in
nature of delivery (although this had been demand-led), towards a better spread of short courses
(including business leadership and supervisory) and longer (NVQ equivalent) qualifications. The need
to address this issue has also led to some innovative solutions, including offering workplace safety
short courses as part of a longer programme, as underpinning knowledge to the overall qualification.
In the words of one partner: “when you’ve worked in ESF a long time you have to roll with the
punches.”
The final evaluation will need to consider the effectiveness of the current procurement process (and
the impact on the wider training market) and the resulting rebalancing of courses to meet employer
needs.
Another issue raised by several partners is a perceived ‘grey area’ in the delivery of courses through
the project that could be perceived as ‘statutory’ (which is disallowed by the terms of the project
funding) rather than mandatory / voluntary and we understand that WEFO have addressed this issue
with the Partnership.
4.2.10. Promoting bilingualism
There is strong agreement that this is a core strength of the project, with the provision of all
materials, support and a wide choice of courses in either medium or bilingually.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 19
4.2.11. Promoting environmental sustainability
There are mixed views on the extent to which the project is promoting environmental sustainability
(as a cross-cutting objective within the project) with some partners noting that these objectives are
embedded within their curriculum (and subject to the Estyn inspection process), and integral to a
range of courses and to organisational policies (including the Green Dragon Environmental
Standard). For others it is embedded in their sectoral focus, where this is largely towards the Energy
and Renewables sectors and to Third Sector organisations involved in community recycling and
environmental issues and one partner notes that the project is also linked to SSCs (Energy and Utility
Skills and Asset Skills) through the ‘Green Deal’ programme.
A minority of partners suggest that more could be done and note a limit to the way that this issue
can be addressed by advisors, with no predetermined routes for signposting to relevant support or
advice.
4.2.12. Lessons learnt to inform the final year of the project
Several partners refer to the project Steering, Operational and Advisor groups as highly effective for
sharing lessons and good practice. The following other ‘lessons learnt’ were also mentioned:
• All projects need to be front loaded and colleges need to move quicker at the start of the
project.
• Keep engagement up – don’t get complacent – the vast majority of firms don’t know that the
project exists.
• Responding to WEFO requirements in terms of changing the curriculum, reviewing sectoral
priorities and developing longer courses / NVQs.
• Developing block courses (to address the minimum 30 hour ruling).
• Delivering workplace safety short courses as underpinning knowledge to the main qualification.
• Make it easier for employers to access short courses – getting procurement working effectively
and short courses up and running again – “this is what employers want” and it leads to NVQs
once had ‘taster’.
• The developing role of college business support services, including Business Point (St Asaph),
Linc Menai and Acquire Training, in business engagement.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 20
4.2.13. Main areas of innovation and / or good practice
The following examples were cited:
• Training on-site and providing the flexibility that is appreciated by companies - “It’s all about
flexibility”;
• Marketing, especially engaging small businesses;
• Opening up curriculum areas to develop new courses;
• The scope to promote each other’s courses – a broad offer to employers;
• Development of online elements and increased use of Moodle (potential innovations) Pwhelli –
Meirion Dwyfor have great video conference facilities that could potentially be used by Dyfodol;
• £50 payment that buys access to a suite of courses across the colleges – giving employers a very
flexible offer, part of a seamless service and all fits with the training needs identified through
the TNA process;
• BDAs can link to other relevant offers / courses / projects when visiting employers – enhances
college access and offer to employers;
• E&D: development / piloting of online E&D questionnaire and sharing of good practice (E&D)
case studies and the outcomes of working with external partners (e.g. Chwarae Teg) to help
achieve aims
• The co-working and the management, drawing things together, “a firm hand on the tiller.”
• Regular reports highlight any variances in performance that are discussed and acted upon and
an open door to the finance team for any queries.
4.2.14. Factors to be considered in the next stage of the evaluation in terms of
measuring and demonstrating the project’s effectiveness and impact
The following factors were suggested by partners, which will be useful in informing the final
evaluation framework:
• What change has occurred in the workplace (e.g. productivity, attrition / retention rates,
morale)?
• Does the profile of training offered match identified needs?
• Have those who have signed up to do a range of courses achieved this (e.g. taking a range of
bite sized courses)?
• Has the project engaged with all types / sizes of business – not just SMEs?
• Has the project increased the volume of courses available across the region?
• What is the impact of procuring short courses - what is the impact of this process on employers
in meeting their needs?
• Have participants progressed in the workplace?
• Have participants progressed into further and higher education?
• What is the impact on business competitiveness?
• What is the impact on the economy of the region as a whole, including moves towards creating
economic sustainability?
• To what extent has the project contributed to wider objectives re increasing GDP and
entrepreneurialism in north Wales?
Another issues raised was that data collected should help to support other, future, bids and
demonstrate the impact and potential for a future all-Wales project.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 21
4.3. Delivery level feedback
This section reports feedback from a series of discussion groups with Business Development
Advisors, across the four partners, and a small number of interviews with Curriculum Development
Managers.
Key findings are:
1. Advisors are working within strong existing networks of employers, linked to their sector
specialisms, and high levels of engagement have been enabled through these networks.
2. The standardised Training Needs Analysis process, linked to business objectives, provides an
opportunity for assessing the extent to which business objectives have been achieved. As
suggested by one group of advisors, this approach also provides a potential framework within
which to integrate Equality and Diversity considerations, linked to business objectives, so that
the TNA takes a holistic approach to identifying the future workforce needs for each business.
3. Advisors have responded effectively to specific employer requests for courses not currently
within the Course Listings, jointly with college Curriculum Managers and other partners.
4. The hiatus in delivery of workplace safety short courses is seen by advisors to have been a
significant issue impacting on their dealings with employers and training delivery but largely
recognise that a process is in place to resolve the issue and effective working relationships with
employers has helped to keep employers ‘on board’.
5. Some problems have been encountered in ensuring that employees undertake training once
relevant courses have been identified but new initiatives are being tested to address this.
6. Equality and diversity issues have met with much resistance from (especially small) employers.
In some cases advisors appear to take the lead from employers and not pursue this further. In
other cases advisors are taking a more proactive approach to persuade employers of the
potential benefits / value of formal equality and diversity processes.
4.3.1. Analysis of training needs, identifying participants and choice of
courses
Employer Engagement
It is clear that advisors are working within strong existing networks of employers, linked to their
sector specialisms, and that (high levels of) engagement have been enabled through these networks
– derived from former (or other existing programmes) including Skills in The Workforce (SITW) or
from existing business networks and internal college networks (including WBL programmes). This
said, it is also clear that new employers have been engaged through a wide range of networking
activities between the advisors (and members of the Steering Group), including hosting /
participating in business events.
Advisors note that the success of previous programmes has helped in the engagement of employers,
providing a client base with a positive experience of training investment. BDAs note that ‘word of
mouth’ has also been an important engagement method within sectors.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 22
The TNA process
Advisors work with a standard template to develop a Training Needs Analysis for each business,
linked to identified business objectives; therefore taking an approach linking training with broader
business needs. As well as making this strategic link for the business owner / HR director this
approach also provides an opportunity for the final evaluation, in assessing to what extent these
objectives have been met through the support of Dyfodol.
Where this process is generally between the advisor and business owner / manager (or HR specialist
in larger firms) one provider is also conducting the process with staff who have been identified as
having a training need, with the aim to match individual and organisation needs and aspirations and
engender employee commitment to training.
Advisors appear to use the Dyfodol Course List as a tool to demonstrate the type of training available
but do not restrict employers to this list and will attempt to address arising needs (not covered on
the list) through other partners, college Curriculum Managers or through signposting to other
relevant programmes.
Advisors have also demonstrated how they have used the process creatively to address identified
business needs through a modular approach, where available courses are presented as modules (or
‘underpinning knowledge’) of a longer-term qualification.
Identifying potential business impact
As mentioned above, this is a key part of the TNA process. We recognise its importance in providing
an opportunity for advisors to initiate a strategic approach, particularly in small and micro
businesses where previous approaches have been largely informal. As noted by one group of
advisors, when first asked about potential business impact: “lots of smaller businesses don’t really
see it like that so we need to coax the reasons out of them.” This highlights a clear area of good
practice within the project, where one group of advisors have worked together to develop a set of
arguments that link potential training needs with SMART objectives, “trying to find a way to explain
it in terms of business impact.” An example was given of persuading businesses of the requirement
of certain qualifications (e.g. CHAS) in the public procurement process and how this could provide
access to public sector tenders. Another group of advisors was able to provide examples where this
public procurement case had had significant impact on participant businesses.
“Shows the employer that investing time and expertise in their business rather than just selling them
a course.”
While issues of compliance / good practice and procurement appear to be important issues,
highlighted by a number of advisors, we also recognise the views also expressed that for many
(particularly micro businesses and sole traders) the primary consideration is business survival and
the skills to enable that. This may be consistent with the reported growth in interest in management
and leadership short courses.
All advisor groups spoke of reviewing these objectives on a regular basis. It is important that the
emerging effects / impacts are captured for the final evaluation.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 23
As suggested by one group of advisors, we believe that this approach also provides a potential
framework within which to integrate Equality and Diversity (and Environmental Sustainability)
considerations, linked to business objectives, so that the TNA takes a holistic approach to identifying
the future workforce needs (skills, recruitment, progression and utilisation / performance) for each
business.
Addressing sectoral priorities
Advisors are aware of the ‘priority’ sectors but state that they also respond to regional / local sector
needs identified through established processes of business support and engagement within the
colleges, the Curriculum Development Managers, the WBL Managers / teams and local sector
networks - and that needs are not static but evolve over time. Curriculum Managers also emphasise
their role in informing project delivery through highlighting their knowledge of sectoral needs and in
working with the project team to develop bespoke packages of courses for specific sectors.
There are some notable differences in the types of sector specialisms across the partner network,
from Energy and Renewables to Health and Social Care and small-scale Creative Industry businesses
and, for one partner, a particular expertise in working with the third sector and social enterprises.
Some partners emphasise how their delivery model has had to adapt to meet the requirements of
the programme, particularly for sectors where qualifications are not seen as a priority and where
previous delivery has been largely community based. This said, it appears that one of the evolving
strengths of the project is the cross-referral network developing between partners as knowledge is
increasingly shared on the sectoral strengths of each provider.
Choice of courses
When asked which courses were the most popular this universally centred on ‘workplace safety’
short courses (including Health and Safety and First Aid), which are in high demand but only form a
small proportion of available courses within the Dyfodol Course List. One group of advisors noted
that these courses had been the focus of the forerunner programme (SITW) and it is clear that this
might be a significant influence. But the main driver is clearly the regulatory agenda which
recommends and / or requires employers across all sectors to equip staff with these qualifications.
This has clearly led the project into a grey area, which potentially conflicts with the restriction
imposed through the funding on the delivery of ‘statutory’ training. In several cases advisors spoke
of directing employers to courses that fulfilled their “statutory” needs. Advisors note that certain
sectors (including Health and Social Care) are largely legislation-led with a strong demand for
workplace safety training but that this training forms part of a workforce progression ladder where
employers meet this requirement first before moving onto higher-level skills.
This said, groups of advisors also noted a growing demand for management and leadership courses.
In one case the provider appears to be developing a range of innovative short courses in this area,
including ‘mindfulness’ and change management; reflecting businesses concerns to manage the
business and the workforce through a period of change and its repercussions (linked to business
survival, growth and competitiveness). “Not focusing specifically on sectors but on softer
management skills and bite sized training.”
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 24
It is also clear that Dyfodol has evolved in responding to specific employer requests for courses not
currently within the Course Listings and that where a request is made this will be investigated with
college Curriculum Managers and other partners to see if there is any ‘currency’ in delivering this
under Dyfodol. An example was given where a range of employers from the Care sector had
expressed a need for courses not on the current list. This request was passed onto the college’s
relevant Curriculum Manager who arranged a meeting with the relevant advisor and assessors and
created new courses to meet this need, “much to the delight of many employees.” Other groups also
highlighted the responsiveness their Curriculum Managers, leading to the addition of new courses
now offered through Dyfodol. One Curriculum Manager emphasised the extent to which the college
has responded where high levels of demand have been identified, referring to a large number of
face-to-face meetings with employers through the Dyfodol project which has enabled the college to
further develop its offer to the local / regional business community (with a 78% increase in short
course delivery over the course of the project).
There is an additional issue on the extent to which different types of employer value qualifications
and certification. While it is clear that, for some, it is the main requirement (particularly where
linked to industry regulations) in other cases (particularly for micro firms, sole traders and social
enterprises) it is not. This latter view expressed by one advisor, reporting client feedback: “I have not
signed up because I want a certificate, I want skills that will allow me to grow and allow my business
to grow and a certificate is not going to allow me to do that.”
The issue of workplace safety courses
As addressed elsewhere in this report, this is also seen by advisors to have been a significant issue
impacting upon their dealings with employers and training delivery. In some cases this brake has
seen to be damaging in their relations to employers, has interrupted the agreed training plans and
employers have withdrawn from the project as a result. For most it appears to have been a
constraint within the project but advisors largely recognise that a process is in place to resolve the
issue and that employers are willing to wait for this resolution (due to their perceived value for
money offered through the project). Advisors also note that the resolution of this issue does not
prevent access to, and delivery of, a wide range of other courses and qualifications, for which (in
particular areas) there has been increased demand.
Additionally, one group of advisors stressed the potential long-term gain in developing courses for
which the project has identified a need, meaning that benefiting employers are likely to continue
their relationship with the NW Wales college network (particularly where they have experienced
cutting-edge facilities and training) rather than seeking provision elsewhere, at the same time
influencing wider curriculum development.
As a result of this process, and responses to the ‘brake’ placed on the delivery of workplace safety
short courses, advisors note a move towards more technical and business / management related
courses and towards longer (NVQ) qualifications. We see this as a positive development arising out
of the range of issues surrounding workplace safety short courses.
We also note the approach taken by one group of advisors in working around the issue through
presenting the shorter workplace safety qualifications as the underpinning knowledge to the main
qualification / certificates (e.g. where the Health and Safety element sits alongside Hospitality Skills
and Customer Services in a (C&G) Certificate in Food and Beverage Services).
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 25
Flexible and responsive delivery methods
A number of examples were given where advisors are helping to promote flexible and responsive
modes of delivery, responding to employer’s requests for courses delivered at evenings and
weekends, where required, and also in core hours: “reflecting what employers want rather than
what we dictate”. Examples were given where advisors had been instrumental in affecting a shift
from multi-week courses to block courses, where it had become evident that courses spread out
over several weeks were a disincentive to employers. Curriculum Managers also provided examples
where delivery had been adapted to specific employer needs arising through the project, including
the adaptation of a multi-week evening course into a block, 1-week, course.
Feedback shows that on-site delivery is often much easier for employers / employees (especially in
rural areas). As noted by one group: “without this flexibility the project could potentially be missing
out on a big chunk of society that, through no fault of their own, can’t get to training.”
Mention was made within different groups of developments in interactive, on-line, approaches to
delivery and that partners were working together on this.
In some cases where relationships are particularly good with certain employers they are able to train
others (from other firms in the sector) on their site.
Identifying lower-skilled workers
While one of the central project objectives is to up-skill low-skilled employees this appears to be an
issue that is not addressed directly by advisors but through the business TNA process, in identifying
the staff that require training at these levels, to meet the wider business objectives. In one case the
advisor group stressed that they do not see it as their role / remit to identify low skilled employees
and this is a sensitive issue that needs to be addressed carefully. In this case the advisors respond to
employer questions about who the course is aimed towards by describing the types and relevance of
courses at the different levels (e.g. a level 2 Health & Safety course is applicable to all and a level 3
course is for supervisors – which helps employers to differentiate.)
One group of advisors also note that the levels of training offered reflect the FE offer (which
removes the need to address this directly), in that the majority of courses offered within FE are at
levels 1-3 and highlight the project’s role in ‘plugging the gap for’ staff who have worked their way
up the business without formal qualifications. Again, the focus is on the wider business need:
“although it’s targeted at the lower skilled workforce we’ve found that the whole business has
benefited from the project really – it might be that they initially sign up lower-skilled workers but
once they see it’s benefited it can spread throughout the company.”
In the case of one provider (where the needs of potential participants are also addressed through
the TNA process) there is a more explicit approach through the BSKB (basic skills and IT test). This
group also noted that the existing level of qualification is evident through the staff declaration form
and the sector (e.g. for care workers) and that in certain industries (land based) employers are
encouraged to undertake an online test to determine staff levels / needs.
The role of Unions was also mentioned (in larger private or public sector firms) where the Union
Learning representative has a role in championing low level (particularly IT) skills development. We
are aware that this only applies to a small minority of employers benefiting from Dyfodol support.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 26
One group of advisors also warned against potential discrepancies in employer and staff perceptions
of their existing skill level, which could potentially discourage employees from learning by forcing
them onto training at too high a level. An example was given where many employers wish their staff
to have ECDL qualifications but many of their staff don’t have the necessary skills to undertake
training at this level, without some form of prior preparation.
Enrolling participants
Some problems have been encountered in ensuring that employees undertake training once needs
and relevant courses have been identified, with staff prevented at the last minute due to work
commitments. In some cases this is seen as affecting the participant target. Possible responses
suggested include being more “bullish” in setting dates and deadlines for employers to complete
their training programmes. Another potential solution discussed is through mixed groups – fee
paying and Dyfodol-funded – sitting the same course, which could potentially be more efficient and
flexible for the employer.
It is notable that providers work in different ways and that attendance can be more problematic
where delivery has traditionally been community-based and where staff are from vulnerable groups.
In these cases it was suggested delivery needs to be “fluid and flexible” and that a college-based
delivery model is not effective in ensuring participation.
Developing wider regional partnership relations and making links with other partners, projects
and programmes
There is an overwhelming consensus that advisors have developed strong relationships through the
project and are working in an effective and co-operative way in sharing leads, knowledge and good
practice. Advisors stress that the working arrangements that have evolved through the project
(including the regular BDA group meetings) have reduced (unhelpful) competition and increased
effective collaboration, ultimately opening up a very broad (cross partnership) offer to employers
with advantages in terms of specialist training and other logistical considerations in terms of the
location and mode of training.
The added value for employers is that advisors also make use of the Dyfodol TNA process to ‘tap in’
to the opportunity to signpost to a wider range of support/provision – other college provision or
other projects (including the Essential Skills project). Conversely, as noted within one group, the
advisor may be contacted in relation to other programmes (e.g. Jobs Growth Wales) and end up
signposting to the Dyfodol offer.
One group of providers raised the issue of progression beyond Dyfodol and viewed this as part of
their role, through making links to wider college programmes (e.g. Apprenticeships). With this in
mind they question the process for referral beyond level 3 (particularly for managers looking to
progress to levels 4 / 5) and feel that this needs greater consideration and strategic co-ordination at
the policy level (as well as potentially adding credibility to the project where there is an evident
strong demand for management and leadership training.)
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 27
4.3.2. Outcomes and impact
This section addresses delivery partners’ perceptions of outcomes and impact to date. We recognise
that a) it may be too early to demonstrate this, b) that this issue will be addressed, in part,
elsewhere in this report through the mid-term feedback from employers, and c) that this will form a
key part of the final evaluation. This said, the discussions with advisors highlights the role of the TNA
process in providing an initial assessment of business objectives (related to identified training needs)
and that progress against these objectives is reviewed on a (regular) follow-up basis.
Advisors made the following comments and observations in relation to the outcomes and impacts
achieved so far.
Employers’ business objectives
Generally speaking, advisors recognise the importance of capturing this information in a systematic
format for the benefit of the project and their wider remit in terms of employer engagement. The
general sense expressed by advisors is that employers have benefited significantly from the support
with several examples given where training through Dyfodol has led to new contracts and / or the
employment of new staff, helping to grow the business and to increase competitiveness (particularly
in terms of public sector procurement). We also note the views expressed by one group of advisors
that employers’ experience of the TNA process, as a distinct element of the support package, is
having an impact on business practices, helping to embed a training needs analysis process into
employers’ internal appraisal systems. This could be viewed (if it is happening more widely) as a
significant element of added value derived from the project, that potentially sustains the process of
linking business objectives and training needs (particularly for small and micro businesses where ‘HR’
processes are largely informal).
Advisors note that employers are generally reluctant to discuss issues of finance and profitability but
that impact can / could be determined in other ways, particularly in terms of new business obtained
and staff changes and through looking at other outcomes (including the change in staff pay rates,
which could be compared with the rates provided at the initial TNA stage.)
In terms of business objectives that were currently not being met (or identified) one group of
advisors stressed a lack of online marketing / e-commerce skills, particularly among their cohort of
small businesses and how this could be a significant issue for many smaller businesses (particularly in
remote / rural areas and seasonal sectors) in expanding their potential market for longer term
survival, with the example given of a potential approach under the banner of “how to survive a
winter in Harlech.”
Employer attitudes to training and development
The general view expressed is that advisors are working in a climate where training is a fairly low
priority for employers and that positive benefits need to be realised for employers to recognise the
value of the training. This is different where employers perceive a direct regulatory requirement for
accreditation / qualification, but even where this is the case advisors have provided examples where
an initial need driven by regulation has led to a change in attitude to the value of training in general.
Through the use of a specific example in the construction sector one group of advisors described the
influence of a positive cycle (where training had enabled tendering which resulted in new work
requiring new staff) and how this had helped to embed the value of training within that firm.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 28
Another group described a food establishment that was putting all of their staff through customer
service training, beyond what they perceived to be the regulatory requirements.
Other issues raised were how to get the messages across to employers in the terms that directly
respond to business issues and needs (e.g. where staff turnover had been an issue and that staff
were now more likely to stay with that employer once they had obtained qualifications, against the
normal perception of staff taking qualifications elsewhere.) Advice from other employers was also
mentioned as an important factor in changing employer attitudes.
Employer attitudes to Equality and Diversity
The main message from advisors is that broaching issues of equality and diversity with employers,
alongside business and training needs, has been a difficult task met with much resistance from
(especially small) employers. In some cases it is felt that employers: “don’t fully understand what’s
being offered” alongside a reluctance to admit that they don’t have policies in place (a ‘shame’
issue). In this case the advisors described how employers often ask if they have to do this, as a
condition of the project, and when the answer has been “no” they can’t see a reason to pursue it.
Another group of advisors expressed the issue as follows: “we bring it up and the uptake on going
further is virtually zero… they don’t understand it a lot of the time…99.9% don’t show an interest and
ignore it once you’ve said it.” [We are] not there to monitor what they do, but to offer support and
say that we have systems in place to assist this.” “We take a softly softly approach just to say that it’s
there but it’s not our job really to force anything on them. If we try to push this at first meeting we
will put them off … Have to be careful how we introduce it, a lot will clam up.”
In some cases, for the reasons described above, advisors appear to take the lead from employers
and not pursue it further if the employer says that they are not interested. In other cases the
advisors outlined approaches that they had taken to persuade employers of the potential value of
formal equality and diversity processes, including helping with recruitment processes, protection
against potential Employment Tribunal cases and with winning tenders (including any advertised
through Sell2Wales) that will require evidence as part of the tender – thereby providing a business
case to the employer. In this case the advisors suggest that resistance is mainly around bureaucracy
rather than a fundamental disagreement with E&D policies and that a drip feed approach can be
used through the review process to gently introduce the potential benefits / rationale. (NB: In all cases
where employers show an interest it appears that they are referred for the review offered through Dyfodol.)
Considering the range of comments it seems that (as with many European funded projects of this
nature) introducing Equality and Diversity policies to employers is seen as an add-on element that
does not fit naturally with the core purpose of the project and a comment from one group of
advisors stands out as part of a potential solution: that Equality and Diversity should be an
integrated part of the offer rather than a stand-alone element, which [the stand-aloneness] is
emphasised by the way the issue is approached / reinforced through the form; requiring a yes / no
response. Equality and Diversity issues should be more of a natural part of the ‘package’ where the
employer approach / needs / issues in relation to Equality & Diversity are addressed in relation to a
wider process of identifying business and training needs – “turning it around”.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 29
Employer attitudes to Environmental Sustainability
Interestingly employers appear to be more receptive to these issues than equality and diversity.
While advisors note that it may be more relevant to some sectors than others (e.g. Energy and
Renewables), where regulatory drivers already exist, there is also a feeling that employers recognise
that this is something they have to do, for business reasons [a distinction was made with general
employer perceptions of E&D issues]. In some cases this is the core business role or ethos, including
some Third Sector organisations and social enterprises. But, as described by one group of advisors,
environmental sustainability issues are the “topic of the day” for many employers, across a range of
sizes and sectors, and moving up the agenda year on year.
4.3.3. Lessons learnt, innovation and good practice
Key lessons learnt that can help inform the final year of the project
The following issues were cited by advisors:
• Workplace safety training - a lesson learnt could be that there was too much emphasis on this
from the beginning…while it has developed this way, workplace safety training does not
dominate the offer but that’s what employers wanted.
• Employee benefits – employees have benefited massively by gaining qualifications and this
needs to be captured and recognized.
• Employer benefits - those who signed up a year ago may have seen lots of benefits – can use
case studies to demonstrate this – can use examples of competitors / employers in other
sectors.
• Participant numbers / targets - finding a way to work with employers and getting them to
commit inc. phoning re course dates rather than emailing [developing the relationship with
employers].
• Potential progression routes beyond level 3: there needs to be a process for signposting (e.g.
to HE providers) to meet wider policy objectives and to enable broader labour market /
employment progression.
• Paperwork – the burden is extremely heavy but partners recognise that this is part of an ESF
project. Could there not be a simpler, electronic, method for capturing data (e.g. a PDA system).
• Short course vs longer qualifications - have had to adapt this to meet needs of employers for
(non qualification based) short, 1-2 day block, courses rather than extended courses over
several weeks.
• Clash of community and business focus – delivery was not front loaded enough at the
beginning and have had to play ‘catch up’ in terms of outputs. A business project that clashed
with provider focus as a community-based training organisation, but now moving forward with
more of a business focus.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 30
Key areas of innovation and / or good practice within the project
Advisors mentioned the following examples of innovation and good practice that have evolved
through the Dyfodol project:
• Adjusting delivery (e.g. from multi-week to a block course) to meet the logistical (time and
travel) needs of employers and staff.
• One designated person to look after training requests and one-way communication with the
client – simplifying the offer for employers and cutting out the need to liaise with other partner
colleges (a single point of contact for employers).
• Developing the online learning curriculum, including a guided course (‘tagged onto’ a customer
service qualification) as an introduction to social platforms; using the latest IT resources
(Chromebooks) and tutor expertise.
• Evolving courses (e.g. hospitality skills and food manufacturing) where the qualifications fit
employer needs and that can be assessed with minimum disruption to the work process.
Factors to be considered in the next stages of the evaluation in terms of measuring and
demonstrating the project’s effectiveness and impact
The following range of factors was mentioned:
• New contracts won.
• Staff retention - has staff turnover increased or decreased?
• Training that participating staff had undertaken 2 years prior to project compared to what they
have undertaken through Dyfodol.
• Comparisons between jobs that individuals are doing before and after Dyfodol support.
• Comparisons of pay levels before and after Dyfodol support.
• Reasons why staff have undertaken training under Dyfodol – the key factor – if this is the value
of the subsidy / £50 fee this could be problem for future sustainability (where some courses
have a commercial value of £800 / £900).
• Should employers be paying more or making a contribution back into the project where they
have directly benefited (e.g. through winning new tenders)?
• Revisiting TNAs and see how far employers have gone in meeting these objectives – and the
impact.
• The impact on the local economy.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 31
4.4. Participant employer feedback
Key issues are:
1. Just under half of employers within the sample stated that the TNA process had helped either
“fully or very well” in identifying their business training needs. A further 28% felt that it has
helped “quite well”; together accounting for three-quarters of the sample.
2. Overall, 84% of employers feel that the skill levels of their staff have increased as a result of the
training received. Two-thirds (66%) feel that these staff are now more productive, 63% feel that
these staff are more likely to progress in the business and 84% feel that these staff are now
more employable in the wider labour market.
3. A very high proportion (78%) of participating employers feel that, as a result of their
participation in the project they are either “very likely” or “quite likely” to continue to invest in
the future development of their workforce.
4. Nine out of 10 participating employers rate the value for money as either “very high” or “quite
high.”
5. Around a quarter (27%) of employers within the sample stated that they had completed an
equality and diversity review.
6. The responses indicate that while the nature of the ‘review’ may have been misunderstood a
substantial proportion of participating employers feel that they are now more aware of equality
and diversity issues in relation to their business and that they have experienced a valuable
intervention in relation to these issues.
7. A discernible impact has occurred with evidence of new policies and processes created and
recognition of the link between equality and diversity policies and business success. In 56% of
cases where employers state that an equality and diversity review has taken place they have
either developed their existing policies or created new policies and 87% of these employers
state that they will continue to develop these polices in the future.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 32
4.4.1. The sample
We recoded the sample to fit with the nine main sectors covered within the survey. The most
prominent of these sectors are Hospitality and Tourism (28 employers / 28%), Care (24 employers /
24%), and Construction/Engineering/Electrical (24 employers / 24%). Finance and Business Services
accounted for seven employers, Agricultural / land based industries for six employers, Retail for five
employers, Health and Beauty for three employers and Energy and Environment for one employer.
Graph 1: Sector of business
Consistent with the overall size profile of participating employers, the majority (80%) of employers
within the survey sample are either small or micro (as defined by the Dyfodol classifications) and just
13% of employers within the sample are classified as medium and 7% as large.
Graph 2: Size of business
2
6
24
24
1
7
3
28
5
100
0 20 40 60 80 100 120
Third Sector
Agricultural/Land based
Care
Construction/Engineering/Electrical
Energy and Environment
Finance and Business Services
Health and Beauty
Hospitality and Tourism
Retail
Total
7%
13%
41%
39%Large
Medium
Micro
Small
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 33
We have sought to achieve a reasonable balance of employers within the sample across the four
main providers (to allow for some comparison between providers) while at the same time reflecting
the actual distribution of participating employers between providers, as illustrated in the chart
below. Coleg Llandrillo and Coleg Menai currently account for the majority of participating
employers within the project, which is reflected to some degree in the sample.
Graph 3: Main provider
We have also sought to achieve a reasonable spread of employers by start date, as shown in the
chart below, with a higher representation (70%) of those employers registering in 2012 than in 2011.
Graph 4: Start Date (quarter of registration)
21%
19%
31%
29% Deeside
Harlech
Llandrillo
Menai
6%
15%
10%
21%22%
17%
9%Apr-11
Jul-11
Oct-11
Jan-12
Apr-12
Jul-12
Oct-12
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 34
Graph 5: Please say how you first heard about the project?
Over half (58%) of participating employers first heard about the project through a direct approach
from one of the colleges, which includes direct contacts, project marketing and contacts made at
networking events, seminars and presentations given by the colleges. The next most effective
process was ‘word of mouth’, at just over a quarter (28%) of participating employers. Assuming that
this message was also generated through the project’s marketing efforts this would account for 86%
of employers who were engaged directly through the project’s multi-faceted marketing approach,
leaving just 14% of employers who had found the project through their own efforts (either online or
by contacting one of the colleges.)
The most common response to this question was “skill development”, cited by 82% of employers.
To “update qualifications” was cited by 17% of employers and “affordable training by 14% of
employers, perhaps surprisingly low when compared to responses, below, in relation to perceived
value for money. The opportunity to “gain a specific qualification” was mentioned by 9% of
employers and “develop a training plan”, “enhance CV” and “develop experience” were each
mentioned by just 1% of employers.
Graph 6: What are you hoping to achieve from your involvement in the project?
43%
28%
9%
8%
7%5%
College approached
me
Word of mouth
Approached the
college
Email from Dyfodol
Project marketing by
post
82
17
14
1
9
1
1
Skill development
Update qualifications
Affordable training
Develop a training plan
Gain a specific qualification
Enhance CV
Develop experience
0 20 40 60 80 100
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 35
4.4.2. Skill development, by sector
Graph 7 below shows the extent to which ‘skill development’ was cited by employers within each
sector. While cited by 100% of employers in four sectors (admittedly, in some cases with very low
numbers of employers in the sample) it is less prominent in two of the key project sectors (Care and
Hospitality and Tourism).
Graphs 7: Skills development within each sector
4.4.3. Training needs and provision
Employers were asked a series of questions about the value of the Training Needs Analysis process
and the role of the Business Development Advisor.
Just under half of employers within the sample stated that the Training Needs Analysis process had
helped either “fully or very well” in identifying their business training needs. A further 28% felt that
it helped “quite well”; together accounting for three-quarters of the sample who felt that the TNA
process had been valuable in determining their training needs. The remaining quarter felt that it had
not helped at all.
Graph 8: To what extent do you feel that the Training Needs Analysis process with the Business
Advisor helped to identify your businesses training needs (in general)?
100
83
67
79
100
100
100
93
60
Third Sector
Agricultural/Land based
Care
Construction/Engineering/Electrical
Energy and Environment
Finance and Business Services
Health and Beauty
Hospitality and Tourism
Retail
0 20 40 60 80 100 120
25%
28%20%
27% Not at all
Quite well
Very well
Fully
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 36
The table below shows some variation when responses to this question are compared by main
provider, with employers citing Coleg Menai as their main provider stating that the TNA process had
been most useful in identifying their business training needs and employers citing Harlech as their
main provider feeling it had been least useful.
Deeside Harlech Llandrillo Menai Total
Very well 35% 15% 20% 30% 20%
Fully 15% 19% 33% 33% 27%
Combined 50% 34% 53% 63% 47%
Q. To what extent do you feel that the Training Needs Analysis process with the Business Advisor
helped you to identify individuals within the workforce who required training?
The findings for this questions are less positive than those for business training needs in general,
with around a third (32%) of employers overall feeling that it had helped to identify individuals who
required training “fully” or “very well” and just over a half (53%) feeling that it had not helped to do
this at all.
Q. To what extent do you feel that the Training Needs Analysis process with the Business Advisor
helped you to identify individuals specific training needs?
Slightly more (38%) employers felt that the process had helped to then identify individual’s specific
training needs “fully” or “very well” and just under half (45%) feeling that it had not helped to do this
at all.
Q. Do you feel that the Training Needs Analysis process with the Business Advisor helped you to
identify ‘low skilled’ workers who required training?
In 79% of cases employers answered ‘no’ to this question.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 37
Graph 9: Do you feel you could have found / accessed this training without the support of the
Business Advisor?
As shown below, just under half (48%) of employers within the sample felt they could not have
accessed the training without the support of the Business Adviser, with a further 7% uncertain. This
suggests that this aspect of the project has been valuable to at least half of participating employers.
4.4.4. Training participation
Employers were then asked for their views on the type and effectiveness of the training undertaken
by their staff.
Q. Please describe the type of training that your staff are undertaking?
Employers were asked to identify all forms of training that their staff had undertaken through the
project. The table below shows the types of training undertaken. ‘General short courses’ are the
most common, with 75% of employers stating that their staff had undertaken this type of training.
This is followed by ‘sector specific short courses’ - with 36% of employers stating that their staff had
undertaken this type of training. A similar proportion (35%) state that their staff had undertaken
‘accredited work-based training and assessment’ and only a small number (3%) state that their staff
have undertaken ‘professional development courses.’
N %
Accredited work-based training and assessment (NVQ or other recognised
qualification) 35 35
General short courses (e.g. First Aid) 75 75
Sector-specific short courses 36 36
Professional Development courses (including equal opportunities and
environmental sustainability) 3 3
7%
48%
45%Don't know
No
Yes
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 38
At the provider level, 90% of employers whose main provider was identified as Coleg Menai have
undertaken general short courses compared to 81% at Deeside, 68% at Harlech and 61% at Coleg
Llandrillo.
Deeside Harlech Llandrillo Menai
Accredited work-based training and assessment (NVQ or
other recognised qualification) 62% 42%
32% 14%
General short courses (e.g. First Aid) 81% 68% 61% 90%
Sector-specific short courses 48% 37% 32% 31%
Professional Development courses (including equal
opportunities and environmental sustainability) 0% 16% 0% 0%
Q. Do you feel that the training course(s) that your staff are undertaking are being delivered in a
way that is flexible and responsive to your / your staff work patterns and business needs?
Overall, 84% of employers answered ‘yes’ to this question, ranging from 100% of employers citing
Harlech as their main provider to 72% of employers citing Menai as their main provider.
Deeside Harlech Llandrillo Menai
Yes 95% 100% 77% 72%
Q. How would you rate the quality of the training that your staff are undertaking through the
project?
Overall, 84% of employers also rate the quality of the training undertaken as either “very high” or
“quite high”; ranging from 100% of employers citing Deeside as their main provider to 68% of
employers citing Harlech as their main provider.
Deeside Harlech Llandrillo Menai
Quite low 0% 5% 3% 0%
Reasonable 0% 26% 13% 17%
Quite high 5% 26% 32% 14%
Very high 95% 42% 52% 69%
Q. How would you rate the appropriateness (relevant to business needs) of the training that
your staff are undertaking through the project?
The majority of providers rated the appropriateness of the training that their staff received through
the project as “very high”, the highest being 83% from the Coleg Menai provider.
Deeside Harlech Llandrillo Menai
Reasonable 10% 32% 10% 7%
Quite high 14% 16% 16% 10%
Very high 76% 53% 74% 83%
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 39
Q. How would you rate the value for money of the training that your staff are undertaking
through the project?
Overall, 91% of employers rated the value for money of the training undertaken as either “very high”
or “quite high”; ranging from 95% of employers citing Deeside as their main provider to 74% of
employers citing Harlech as their main provider.
4.4.5. Training impact
Employers were then asked for their views on the perceived impact of the training undertaken by
their staff.
Q. Do you feel that the skills levels of your staff members who have undertaken training through
the project have increased as a result of the training received?
Overall, 84% of employers answered ‘yes’ to this question, ranging from 100% of employers citing
Deeside as their main provider to 69% of employers citing Menai as their main provider.
Deeside Harlech Llandrillo Menai
Yes 100% 89% 84% 69%
No 0% 11% 10% 31%
Don’t know 0% 0% 6% 0%
Q. Do you feel that your staff members who have undertaken training through the project are
now more productive as a result of the training received?
Overall, 66% of employers answered ‘yes’ to this question, ranging from 81% of employers citing
Deeside as their main provider to 48% of employers citing Menai as their main provider.
Deeside Harlech Llandrillo Menai
Yes 81% 68% 71% 48%
No 5% 21% 19% 45%
Don't know 14% 11% 10% 7%
Q. Do you feel that your staff members who have undertaken training through the project are
now more likely to progress in your business as a result of the training received?
Overall, 63% of employers answered ‘yes’ to this question, ranging from 86% of employers citing
Deeside as their main provider to 38% of employers citing Menai as their main provider.
Deeside Harlech Llandrillo Menai
Yes 86% 58% 74% 38%
No 9% 21% 23% 55%
Don't know 5% 21% 3% 7%
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 40
Q. Do you feel that your staff members who have undertaken training through the project are
now more employable (in the wider labour market) as a result of the training received?
Overall, 84% of employers answered ‘yes’ to this question, ranging from 91% of employers citing
Deeside as their main provider to 79% of employers citing Harlech as their main provider.
Deeside Harlech Llandrillo Menai
Yes 91% 79% 84% 83%
No 5% 11% 10% 17%
Don't know 5% 10% 6% 0%
Q. Do you feel that your staff members who have undertaken training through the project now
have greater job satisfaction as a result of the training received?
Overall, 70% of employers answered ‘yes’ to this question, ranging from 86% of employers citing
Deeside as their main provider to 59% of employers citing Menai as their main provider.
Deeside Harlech Llandrillo Menai
Yes 86% 63% 74% 59%
No 5% 21% 13% 21%
Don't know 10% 16% 13% 21%
Q. Do you feel that your staff members who have undertaken training through the project now
have greater knowledge of environmental sustainability issues (relating to their job and / or
the business) as a result of the training received?
Overall, 45% of employers answered ‘yes’ to this question, ranging from 67% of employers citing
Deeside as their main provider to 28% of employers citing Menai as their main provider.
Deeside Harlech Llandrillo Menai
Yes 67% 32% 55% 28%
No 19% 63% 32% 65%
Don't know 14% 5% 13% 7%
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 41
This graph shows that a very high proportion (78%) of participating employers feel that as a result of
their participation in the project they are either “very likely” or “quite likely” to continue to invest in
the future development of their workforce; a very positive finding in terms of the project’s impact on
participating employers’ attitudes towards training and potential longer term skills investment in the
region. This finding is consistently high across providers.
Graph 10: Overall, as a result of your participation in this project, how likely are you to invest in
the future in developing the skills of more of your workforce?
Equally encouraging is the finding that the majority (60%) of participating employers feel that as a
result of their participation in the project they are either “very likely” or “quite likely” to continue to
invest in the future development of their ‘low skilled’ workers. This finding is also consistently high
across providers.
Graph 11: Overall, as a result of your participation in this project, how likely are you to invest in
the future in developing the skills of more of your ‘low skilled’ workers?
11%
5%
6%
15%63%
Not likely at all
Quite unlikely
Not sure
Quite likely
Very likely
22%
5%
13%
13%
47%
Not likely at all
Quite unlikely
Not sure
Quite likely
Very likely
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 42
4.4.6. Equality and Diversity policy review and training
Employers were then asked for their views on the equality and diversity review process. The
responses indicate that while the nature of the ‘review’ may have been misunderstood, a substantial
proportion of participating employers feel that they are now more aware of equality and diversity
issues in relation to their business and that they have experienced a valuable intervention in relation
to these issues. While what they consider to have been a review is likely to have been more of an
informal discussion as part of the TNA process, a discernible impact has occurred; with evidence of
new policies and processes created and recognition of the link between equality and diversity
policies and business success.
Q. Did you complete an Equality and Diversity review as part of the Dyfodol project?
Overall, 27% of employers within the sample stated that they had completed an equality and
diversity review. This is a surprisingly high proportion when compared to project data and would
suggest that employers have not understood the difference between the discussion of equality and
diversity issues as part of the TNA process and the formal review process undertaken by the project
team. But, this said, it clearly indicates that around a quarter of the sample feel they have had some
intervention in relation to equality and diversity issues. They then go on to answer a number of
further questions on this ‘review’ process that appears to confirm that they believe they have had
some intervention in relation to equality and diversity issues and also highlights some interesting
findings in relation to the impact of this exposure, particularly in relation to raising awareness of
these issues within their business.
Graph 12: Overall, how useful do you feel this review has been?
Nearly two-thirds (64%) of employers who stated that they have completed an Equality and Diversity
review feel that it has been a useful process; 23% state that is has been either “extremely” or “very”
useful.
18%
18%
41%
19%
4%
Not at all useful
Not so useful
Quite useful
Very useful
Extremely useful
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 43
Graph 13: To what extent do you now feel more aware of equality and diversity issues that relate
to your business?
Of those who stated that they had had an equality and diversity review, 59% feel that they are now
more aware of equality and diversity issues; with 26% stating they are now “very” or “highly aware”
of equality and diversity issues.
The verbatim comments below appear to support this assertion:
• “We now have an equality and diversity policy and have also amended our job application
forms. We are also aware of offering training in relation to any disability and feel we have an
obligation to do so.”
• “I have more of a general awareness of it.”
• “I have reviewed my recruitment policy. I have checked that the job application form meets the
equality and diversity needs.”
• “We are more aware of the equality and diversity issues.”
• “It helps me with regard to meeting with clients. I am more aware of any issues that may arise.”
• “It highlighted things that we weren't doing previously.”
• “It has just made me more aware of the issues.”
Q. As a result of your review have you either created new formal Equality and Diversity policies
or updated / developed your existing policies?
Overall, in 56% of cases where employers have stated that a review has taken place, they also state
that they have either developed their existing policies or created new policies (as a result of the
review) and 87% of employers who have updated or created new policies state that they will
continue to develop these polices in the future.
Moreover, in 56% of cases where employers have stated that a review has taken place they also
state that they now see a positive relationship between Equality and Diversity policies and business
success.
41%
22%
11%
19%
7%
No more aware
Slightly more aware
Reasonably aware
Very aware
Highly aware
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 44
5. Conclusions and recommendations
This section sets out a summary of key findings, conclusions and recommendations.
5.1. Key successes
1. The employer engagement process has been highly successful and Business Development
Advisors have been instrumental in the success of this process. Partners recognise the value of
the project in supporting their organisation’s evolving role in employer engagement
2. High levels of employer satisfaction with training delivery and support and a stated willingness
to invest in future training
3. The evolving cross-partnership offer available to participating employers
4. The centrality and effectiveness of the Advisor role in identifying employers’ training needs,
through the standardised Training Needs Analysis process
5. The shared commitment to addressing existing skills needs; going beyond ‘priority sectors’ to
meet, evolving, local / regional skills needs
6. Strong and effective partnership working, with open communication and effective sharing of
best practice to achieve the overall objectives and targets
5.2. Issues to address
1. Adaptation of TNA practices and advisor approaches to address the employer equality and
diversity target
2. Successful completion of the procurement process, addressing a noted imbalance in course
delivery
3. The current budget under-spend
4. Problems encountered in ensuring that employees undertake training once relevant courses
have been identified
5.3. Recommendations
1. That the Partnership continue to address the current budget under-spend as a matter of priority.
2. That a more integrated, business-needs-led, approach may be required, also addressing equality
and diversity issues with further specific training and support for advisors in relation to equality
and diversity issues.
3. That different methods for participant enrolment continue to be tested and implemented (if
effective) in the final stage of the project.
4. That emerging effects / impacts in relation to business objectives are captured by advisors for
the final evaluation.
5. That the final evaluation will need to consider the effectiveness of the current procurement
process (and the impact on the wider training market) and the resulting change to the balance of
courses to meet employer needs.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 45
5.4. Progress towards targets
Employer and participant engagement
The latest data submitted to WEFO demonstrates that the Partnership is over-achieving against its
profile in terms of companies assisted and employed participants; substantially beyond the final
target for companies assisted. The Lead Partner (Coleg Llandrillo) is making the most substantial
contribution to the achievement of these targets, way beyond the expected profile up to this point
in time. While other partners are making a lesser contribution to the total target, they are largely on
profile for the number of companies assisted.
Challenges encountered in signing up larger employers and converting employer take-up into course
enrolments have led to an increased focus across the Partnership on signing up additional
participants from companies that have been engaged, reviewing processes and sharing successful
practices between partners. The outcome of this shift in focus will hopefully become apparent in
future participant target profiles and progress reports.
Project underspend
There is some concern that the budget spend is currently around £500k under profile. In terms of
projected spend, the Lead Partner is over budget but other partners are under budget. While the
Partnership is addressing this issue with WEFO (through a possible 6-month extension) and there are
some administrative issues (relating to completion of registration data) that are currently being
addressed, there is some concern within the Partnership that this underspend may grow.
We therefore urge that the Partnership continues to address this as a matter of priority.
Equality and Diversity policies
Partners are struggling to achieve the target relating to the number of employers adopting or
improving equality and diversity strategies and monitoring systems and highlight a number of
constraints. There is a common recognition that the Partnership is highly unlikely to achieve the
target but that it will be important to demonstrate the efforts, processes and resources that have
been directed towards this objective. We commend that this objective has been given a high
prominence within the Partnership with a genuine commitment (and a range of processes tested) to
achieve it and to understand the constraints encountered so far.
From our previous experience of evaluating programmes aimed specifically at promoting equality
and diversity policies within small and medium sizes businesses (across Wales) we recognise that this
is a particularly ‘tough nut to crack’ with smaller businesses and that employers are naturally
suspicious of the motives when the approach is from a public sector body. But, in our experience,
they will respond more positively where they are presented with potential business benefits that
directly relate to their business needs. In this sense, presenting equality and diversity policies as a
separate, add-on, element to employers is likely to be met with much resistance and / or lack of
interest.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 46
As suggested by one group of advisors, we believe that the current TNA approach also provides a
potential framework within which to integrate Equality and Diversity (and Environmental
Sustainability) considerations, linked to business objectives, so that the TNA takes a holistic
approach to identifying the future workforce needs (skills, recruitment, progression and utilisation /
performance) for each business.
We recommend that a more integrated, business-needs-led, approach may be required.
The feedback from employers demonstrates the extent to which some advisors may not be actively
‘selling’ this element to employers.
We recommend that this issue needs to be addressed through sharing of good practice between
advisors and further specific training and support for advisors in relation to equality and diversity
issues.
The strategic lead on targets and performance
Partners widely cite the importance of the Steering Group and Operational Group in highlighting the
importance of targets as a prominent item on the Steering Group and Operational Group agendas;
with open sharing of support and good practice encouraged between partners. In this vain, there is a
resounding acknowledgement that the Partnership is working very effectively as a team, with strong
and open communication and effective sharing of best practice to achieve the overall objectives and
targets, also successfully overcoming inherent competition between partners. The Lead Partner is
widely commended for its leadership, professionalism and support.
Developing an ‘offer’ aligned with sectoral priorities
The general consensus among strategic and delivery partners is that, while they are aware of the
‘Priority Sectors’, established in the Welsh Government’s Economic Renewal Policy, and amended for
the Dyfodol Business Plan, the focus has also been on local and regional need which has evolved
over the course of the project. The strategic importance and skills needs of other sectors (e.g.
Tourism and Health and Social Care) that were not included in the original list of ‘Priority’ sectors
have become evident through the project (and regional LMI) and led to their prominence in the
profile of supported businesses. It is clear that all partners share a commitment to addressing
existing local skills needs and that they have not been constrained by a focus solely on national
policy guidance.
Engaging employers and identifying their training needs
We recognise the major multi-faceted, effort that has been undertaken in engaging employers
(reflected in the current target profile) but also note the variance across partners in their
contribution to this target. On this issue we note the way in which the employer contribution can be
(re)allocated to college funds, as an incentive for college Directors to take an interest in the project.
In one case 60% of the employer contribution is paid directly into department budgets. We suggest
that this may provide a model for those partners who have, up until now, made a lower contribution
to the project’s employer engagement targets.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 47
The advisor role has been instrumental in the engagement of employers (through existing
relationships and building new alliances / tapping into existing networks) and it is clear that a strong
sales team is being developed through Dyfodol and that the project is helping to establish Business
Point, Acquire Training and Linc Menai as recognised and valued local / regional business support
organisations.
The employer survey demonstrates the high priority given to skill development by employers
engaged in the project.
The advisor role and the strategic value of the TNA process
We recognise the importance of the standardised TNA process in providing an opportunity for
advisors to initiate a strategic approach, particularly in small and micro businesses where previous
approaches have been largely informal.
Employers clearly value the TNA process and support given by advisors in identifying their business
training needs. Just under half of employers within the sample stated that the Training Needs
Analysis process had helped either “fully or very well” in identifying their business training needs. A
further 28% felt that it have helped “quite well”; together accounting for three-quarters of the
sample who felt that the TNA process had been valuable in determining their training needs.
We recognise that the approach of linking the TNA process to business objectives and to
determining potential business impact is valuable in providing indicators for future assessment of
the project’s value, including through reviewing these objectives on a regular basis over the final
stage of delivery.
We recommend that emerging effects / impacts in relation to business objectives are captured by
advisors for the final evaluation.
Engaging participants
While the project is on profile in meeting its participant target, there have been some challenges in
converting employer take-up into registrations on courses. This is reflected in the data, where the
ratio of participants to employers has (up until now) fallen behind what was originally anticipated.
This has led to an increased focus on signing up additional employees from employers that have
been engaged, reviewing processes and sharing successful practices (including through discussions
at the regular BDA group). The employer:participant ratio has also been affected by the nature of
the cohort of (mainly small) employers and constraints in signing up some major regional employers
that potentially would have brought large numbers of participants into the project.
Some problems have also been encountered in ensuring that employees undertake training once
needs and relevant courses have been identified. In some cases this is seen as affecting the
participant target. Partners are looking at different initiatives to address this issue.
We suggest that these different methods continue to be tested and implemented (if effective) in
the final stage of the project.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 48
Addressing risks to delivery
The major risk to delivery, referred to by all partners, has been the requirement of an open
procurement process for ‘workplace safety’ short courses on the basis that they were being
subsidised through the project, and offered to employers at a substantially reduced rate.
The findings from the employer survey confirm the proliferation of short courses delivered, up until
now, through the project. General short courses’ are the most common, with 75% of employers
stating that their staff had undertaken this type of training.
The evaluation has found that the Partnership has dealt proactively with this issue and has reached a
solution with WEFO where a formal procurement process has been established for the delivery of
these short courses, open to all (public and private) providers and that while partners note that that
this issue has created a hiatus in delivery, adversely affected by the (acknowledged) over
proliferation of these courses, the majority do not feel that this will adversely affect the overall
performance of the project. Indeed, it appears that this has brought an issue to the surface that
perhaps should have been considered at the project’s inception (seen by several partners as a lesson
learnt) and forced the Partnership to acknowledge and address an imbalance in nature of delivery
towards a better spread of short courses (including business leadership and management) and
longer (NVQ equivalent) qualifications. We see this as a positive development arising out of the
range of issues surrounding workplace safety short courses.
The final evaluation will need to consider the effectiveness of the current procurement process
(and the impact on the wider training market) and the resulting change to the balance of courses
to meet employer needs.
Developing / providing courses to meet identified needs
We recognise the value of the Course List as a resource created through the project (in response to
national and regional LMI, existing knowledge and discussions with employer groups and Sector
Skills Councils) and that partners have sought to go beyond the range of courses on the list to
develop their own college curricula in response to emerging needs.
Survey feedback shows that the majority of participating employers rated the appropriateness and
quality of the training that their staff received through the project as either “very high” or “quite
high”.
The working arrangements that have evolved through the project have increased effective
collaboration between partners and advisors, ultimately opening up a very broad (cross partnership)
offer to employers with advantages in terms of specialist training and other logistical considerations
in terms of the location and mode of training.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 49
Training impact
The feedback from participating employers on the impact of training undertaken through the project
is highly positive. Overall, 84% of employers feel that the skill levels of their staff have increased as a
result of the training received, 66% feel that these staff are now more productive, 63% feel that
these staff are more likely to progress in the business and 84% feel that these staff are now more
employable in the wider labour market.
Moreover, a very high proportion (78%) of participating employers feel that, as a result of their
participation in the project they are either “very likely” or “quite likely” to continue to invest in the
future development of their workforce. We view this as a very positive finding in terms of the
project’s impact on participating employers’ attitudes towards training and potential longer-term
skills investment in the region.
Value for money
Strategic and delivery partners widely refer to the value for money offered through the project.
Employers clearly recognise this, with 9 out of 10 participating employers rating the value for money
of the training undertaken as either “very high” or “quite high.”
This said, we need to consider these findings in terms of the proliferation of workplace safety
courses that have been funded through the project, at a cost to employers well below the market
rate.
Developing wider regional partnership relations and synergies with other partners and
programmes (including Shaping the Future and the Essential Skills Employer Pledge)
Several partners refer to links made to the Essential Skills project but little mention is made of links
to the Shaping the Future project although we are aware that efforts have been made at a strategic
level to explore this relationship. Other links were cited to foundation degrees and Apprenticeship
programmes offered through the college network and some partners refer to a natural, regional, link
between Dyfodol and the complementary Skills for the Workplace project running across North East
Wales.
The value of the project in enhancing organisational strategies
Partners recognise the value of the project in supporting their organisation’s evolving role in
employer engagement, particularly in reaching small and micro businesses and in adding value to
this engagement role through the employment (and development) of the BDA / BLO Business
Advisor team.
Mid-term Evaluation of the Dyfodol Project
Final Report
Copyright Wavehill Ltd | January 2013 Page | 50
Promoting environmental sustainability
There are mixed views on the extent to which the project is promoting environmental sustainability
(as a cross-cutting objective within the project).
Interestingly, advisors generally feel that employers appear to be more receptive to these issues
than to equality and diversity issues and note that it may be more relevant to some sectors than
others, where regulatory drivers already exist. There is also a feeling that employers recognise that
this is something they have to do, for business reasons (a distinction was made with general
employer perceptions of equality and diversity issues).
Factors to be considered in the next stage of the evaluation in terms of measuring and
demonstrating the project’s effectiveness and impact
The following factors were suggested by partners and advisors, which will be useful in informing the
final evaluation framework:
• What change has occurred in the workplace (e.g. productivity, attrition / retention rates,
morale)?
• Does the profile of training offered match identified needs?
• Have those who have signed up to do a range of courses achieved this (e.g. taking a range of
bite sized courses)?
• Has the project engaged with all types / sizes of business – not just SMEs?
• Has the project increased the volume of courses available across the region?
• What is the impact of procuring short courses - what is the impact of this process on employers
in meeting their needs?
• Have participants progressed in the workplace?
• Have participants progressed into further and higher education?
• Staff retention - has staff turnover increased or decreased?
• Training that participating staff had undertaken two years prior to the project compared to
what they have undertaken through Dyfodol
• Comparisons between jobs that individuals are doing before and after Dyfodol support
• Comparisons of pay levels before and after Dyfodol support
• Reasons why staff have undertaken training under Dyfodol – if this is the value of the subsidy /
£50 fee this could be problem for future sustainability (where some courses have a commercial
value of £800 / £900).
• What is the impact on business competitiveness?
• New contracts won
• Revisiting TNAs and seeing how far employers have gone in meeting these objectives – and the
impact
• What is the impact on the economy of the region as a whole, including moves towards creating
economic sustainability?
• To what extent has the project contributed to wider objectives re increasing GDP and
entrepreneurialism in North Wales?
• Should employers be paying more or making a contribution back into the project where they
have directly benefited (e.g. through winning new tenders)