Outputs, Outcomes and Impact

Andrew Harris – Derwen CollegeFiona Voysey – National Star College

Natspec and QSR

• A National ISC PRD group (National Star, RNCB, Treloars, Henshaws & Derwen)

• The data would to used to benchmark across the sector, including GFE LLDD. In addition the data may be included in ILR and FfE.

• For a short period the group was joined by representatives from AoC, NASS & Ofsted.

Outline

• To explore the concept of outcomes and how they relate to ISCs politically and in practice

• To consider the current Natspec QSR project and outcomes for learners with complex needs

How do you know your college is good?

• Ofsted / CSCI inspection• Internal QA• The progress young people make with us

But…

• No national framework for outcomes• What providers value as outcomes are not

always the priorities of others• Confusion and some disagreement over what

we should be measuring and how

Why measure outcomes?

• To show the effect a placement at your college has on a young person.

• To show potential placers that you provide a high quality service –

But- can you meet both functions with the same set of data?

The Project• The challenge: To produce quantitative data on personal/individual success thatallows comparison across providers.

• The Proposal: To produce annual data on the achievement of predicted Every Citizen Matters (ECM) outcomes for individual learners, identifying the numbers and percentages of learners who are ‘ahead/over’, ‘in line with/on’ or ‘behind/under’ the learning needed to meet their goal, amalgamated for the provider as a whole and against each ECM theme.

The Benefits

• Allows personalisation within a nationally recognised framework for consistency (RARPA plus ECM)

• Measures success in outcomes which are valuable to learners and which are controlled by the provider

• Supports self-assessment and evidences ‘distance travelled’/value added

• Does not prescribe or constrain curriculum offer, programme or provider type, enables links to FL

• Links to local authority outcomes and Ofsted inspection• Measures success in outcomes which are valued by

stakeholders and commissioners

Steps…• Pilots to test process, establish guidance and criteria for levels of

performance • Guidance on process including what might be included under each ECM

outcome, including PI’s• Guidance on how to best contextualise the data including use of

evaluative criteria based upon CIF.• Parameters for small numbers of learners• Clarify definitions of complex needs and learners for whom this

approach is appropriate• Validation and quality assurance (requires robust RARPA processes and

self-assessment with validation through peer review and external tests through Ofsted).

• Establish links to ILR

Other recommendations

• The use of destinations against predictions could be a useful indicator but should not be used as a measure of success as there are too many issues outside the control of the provider.

• Students who die or whose health deteriorates such that continued attendance is impossible should be removed from success rates and retention data

Conclusions – June 2010

• Learners individual learning goals (ILGs) and ECM outcomes?

• Are your RARPA processes robust? How do you achieve this?

• The concept of ‘ahead/over target’, ‘in line with/on target’, behind/under target.

• Consider the use of percentages in data collection.