Providing Inspection Services for

Department of Education Department for Employment and Learning Department of Culture, Arts and Leisure

Dissemination of PSA Level 3 Evaluation

December 2010

Agenda• Terms of Reference • Considerations • Evidence Base • Main Findings • Recommendations • Next Steps • Conclusion and Questions

Considerations ….

Outcomes- summary evaluation report plus four specialist reports

Reports issued to each college last term- follow- up inspection work in four colleges

Same team of specialist inspectors in each college

Evaluation Reports take due account of key policy drivers

Evaluation Reports well- populated with examples of good practice

Dissemination activities ….

Scope of Evaluation- level 3 FE provision in-

• ICT and computing • construction and built environment• electrical and electronic engineering• mechanical and manufacturing

engineering

Terms of Reference 1…

Terms of Reference 2… to inspect and report on

the quality of the curriculum for each priority skill area

the colleges’ strategic planning for the provision

the effectiveness of employer engagement and links with key stakeholders

the quality of the provision for learning

the standards of students’ work

Also to

identify best practice

inform Department on impact of policies

Evidence Base

320 lessons

Small number of lessons to address weaknesses in full-time students’ literacy, numeracy and ICT skills

Sample of mandatory and optional units to improve technical literacy and numeracy skills

College senior management teams curricular team leaders and course teams students

Sample of employers

Meetings with key stakeholders including DEL- Skills Strategy and Sectoral Bodies

Main Findings… overall effectiveness

Overall effectiveness for each college-

it is very good one college

good in three

satisfactory in two

Within each priority skill area:

Electrical and electronic engineering is good or better in 5 colleges- one is outstanding

ICT and computing it is good or better in 4 colleges-one is outstanding

Construction and mechanical and manufacturing engineering only good or better in half the colleges-the provision in mechanical and manufacturing engineering in one college is inadequate

Provision to meet needs of learners and local economy

Full-time provision fine in most colleges but…..

2 year part-time craft and technician courses in nearly all of the priority skills areas is inadequate

1 year or short part-time up-skilling level 3 courses including bespoke courses for industry varies across the priority skills areas and across the colleges

Significant gender gap with few female students recruited to full-time and part-time courses

Leadership and Management…1

STEM Expertise at a High Level…a critical success factor

Fragmented curriculum planning across these areas results in gaps in provision and the underutilisation of resources

Strategic Project Management Methodologies can/do make a difference

Much more needs to be done to foster collaboration within and across colleges

Middle managers have significant responsibilities

Challenges of deploying best staff to meet different priorities

Leadership and Management…2

Quality Assurance arrangements are implemented systematically but are not sufficiently honed to identify and respond to weaknesses

The quality and management of specialist physical resources varies significantly across the sector

Management of specialist resources…a few examples of ineffective management of these resources which has an adverse impact on the students’ learning experiences particularly in construction and mechanical/manufacturing engineering

Leadership and Management..3 Economic Engagement

2 colleges have well -developed strategies to support economic engagement across these areas- they are well-informed and linked effectively to the changes in the wider business environment including sustainable technologies

A common thread in both was the further development of extensive links with industry that had been gained over a lengthy period of time in one or more legacy colleges backed up with action to up-skill the expertise of the lecturing staff

In the other colleges there are examples of good practice in at least one priority skill area that are not shared effectively in the other areas

Leadership and Management..3 Economic Engagement

• Nearly all of the colleges have well- developed links with local and regional employers and sectoral bodies- particularly at senior management level or through whole- college business support units

• These links have a more positive impact when senior managers draw on the skills and expertise of those STEM middle managers and course teams who have well-developed links with industry

• Some examples of good practice in those colleges who have been leading or have participated on projects funded under the Innovation Fund: Employer Support Programme

• WDFs.. there were few examples of changes in their level 3 provision across these areas to meet identified skill needs. Examples of good practice ….

Quality of the Curriculum

In 4 colleges students have good opportunities to develop relevant technical and practical skills

In 2 colleges these opportunities are either limited or vary significantly across the priority skills areas-the curriculum is unduly theory-based and assessment led resulting in lower standards and high levels of student dissatisfaction

Most courses have a good blend of mandatory and optional units to develop the students’ technical literacy and numeracy skills

Additional qualifications can help but..

College admissions criteria vary significantly and are not applied sufficiently rigorously

Quality of Teaching and Learning

• The quality of teaching and learning in the majority (69%) of the lessons is good or better and a significant minority (30%) are very good or outstanding

• A minority (27%) of the lessons are just satisfactory

• Most lecturers make use of ILT in their lessons but insufficient use is made of the more interactive features of educational technology to engage and motivate students

• One example of best practice in enhancing lecturers’ skills in making good use of ILT

Assessment

• More work needed to enable course teams to work collaboratively across campuses in the design and moderation of coursework assignments

• Some examples of good practice to reduce the assessment burden on students- integrated assignments

• For most areas assessment planning is discrete and unit-based which results in an excessive assignment workload for students

CEIAG

• Although students have good access to specialist careers information advice and guidance services in nearly all of the colleges more needs to be done to integrate this into the students’ main vocational units and through their timetabled tutorial programmes

• The use of work-related learning to enhance the students’ knowledge and understanding of the world of work is inadequate in nearly all of the colleges

Student Support

• The student tutorial programmes in nearly all of the colleges are not sufficiently well-organised to support students effectively in their learning particularly in reviewing their progress and setting targets for improvement

• With the exception of 2 colleges electronic or paper-based student learner agreements provide little added value in helping students plan their work effectively

• In some cases, course teams made more widespread use of other online reporting tools that they had developed to track the progress of their students

• In the best practice(1 college) students have access to a well-designed tutorial system which gives good coverage to personal and social development-the college through its student support service has excellent arrangements in place to support students at risk of withdrawing from their courses

Standards and Outcomes..1

• The standards of the students’ work across the priority skills areas are mostly good in 4 colleges

• In the other 2 colleges while there are pockets of good practice the standards of work in most of the priority skills areas are just satisfactory or inadequate

• Most of the students achieve good standards of work in literacy and numeracy within their professional and technical units

• Students can significantly enhance their literacy/ numeracy skills in their professional and technical units

Standards and Outcomes..2

• Well-planned project work relevant to the needs of industry can significantly enhance the standards of students’ work

• Achievement rates vary significantly across the colleges and across the different priority skills area

• Achievement rates for part-time courses are mostly higher than on full-time courses, where high levels of student drop out are a key weakness in a significant minority of courses

Actions required to bring about Improvement 1: Improve the quality of the curriculum across the Professional and Technical Areas

• Ensure that the students enrolled on full-time courses have the relevant academic qualifications and interests to meet the challenges of the courses

• Promote these courses more effectively to female students, where enrolment levels are insignificant

Links to other Evaluation

Aims

• to improve the quality of learning

• to improve standards

• to improve the strategic planning

• to improve the quality of learning

Actions required to bring about Improvement 2: Improve the strategic planning for the provision

• Foster more collaborative approaches to identify and exploit key connections across these PSAs to meet gaps in provision and make more effective use of specialist physical resources and to respond effectively to the needs of industry

Links to other Evaluation

Aims

to improve the quality of the curriculum

develop/ enhance employer engagement

Actions required to bring about Improvement 2: Improve the strategic planning for the provision

• Take action to address provision that is satisfactory or inadequate particularly in construction and mechanical/ manufacturing engineering where only half of the colleges have good or better quality provision

Links to other Evaluation

Aims

• to improve the quality of the curriculum

• to improve the quality of learning

• to improve standards

Actions required to bring about Improvement 2: Improve the strategic planning for the provision

• Ensure that human resource strategies are aligned more closely to curriculum planning particularly where course teams work in industry and in post-primary schools

• Develop more collaborative planning across the sector and within the colleges to widen student access to high quality specialist physical resources particularly expensive advanced manufacturing equipment

Links to other Evaluation

Aims

to improve the quality of learning

to improve standards

• to improve the quality of the curriculum

• develop/ enhance employer engagement

• to improve the quality of learning

• to improve standards

Actions required to bring about Improvement 3: Develop and enhance employer engagement and links with key stakeholders

Make more effective use of the existing expertise of those lecturers and curriculum leaders with a strong background in STEM in curriculum planning, and in developing courses that meet the needs of industry and the NI workforce

• Develop more innovative approaches to promote and deliver part-time courses to meet the needs of the workforce to enhance their skill levels to meet the needs of industry

Links to other EvaluationAims

• to improve the quality of the curriculum

• to improve the strategic planning

• to improve the quality of learning

• to improve the quality of the curriculum

• to improve the strategic planning

Actions required to bring about Improvement 4: Improve the quality of the provision for learning

Provide all full-time students with good opportunities to develop their technical and practical skills and to undertake work-related learning tasks to enhance their employability skills

Links to other Evaluation

Aims

• to improve the quality of the curriculum

• to improve the strategic planning

• develop/ enhance employer engagement

• to improve standards

Actions required to bring about Improvement 5: Improve the standards of students’ work

Implement coherent strategies to improve student retention rates on those full-time courses with low levels of achievement

Links to other Evaluation

Aims

to improve the strategic planning

to improve the quality of learning

Next Steps …

Publication of Reports

Follow-up Inspections

Conclusion/ Questions….