equabench—benchmarking european excellence€¦ · education and training ... organisations...

36
Results Orientation Innovative Practice Guide EQUABENCH—Benchmarking European Excellence

Upload: tranthien

Post on 08-Aug-2018

213 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

Results Orientation

Innovative Practice Guide

EQUABENCH—Benchmarking European Excellence

Page 2: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

2

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

PROJECT PARTNERS

The Cedar Foundation (Cedar) Centro Studi Opera Don Calabria (Centro Studi)

31 Ulsterville Avenue Belfast BT9 7AS Northern Ireland Telephone: +44 28 90 666188 Email: [email protected] Website: www.cedar-foundation.org Contact: Stella Maguire, Head of Organisational and Service Development

Viale Don Calabria 13 Cap 44100 Comune Ferrara Provincia Ferrara Italy Telephone: +39 0532 741515 Email: [email protected] Website: www.centrodoncalabria.it Contact: Giuseppe Sarti, Coordinator

European Platform for Rehabilitation (EPR) Pluryn Arbeid (Pluryn)

15, Rue de Spa 1000 Brussels Belgium Telephone: + 32 2 736 54 44 Email: [email protected] Website: www.epr.eu Contact: Simona Giarratano, EQUASS Coordinator

Nijmeegsebaan 9 6561 KE Groesbeek Holland Telephone: +31610914930 Email: [email protected] Website: www.pluryn.nl Contact: Birgit Grimbergen, Quality Professional

Josefsheim gGmbH (Josefsheim) Centro de Reabilitação Profissional de Gaia (CRPG)

Heinrich-Sommer-Straße 13 59939 Olsberg Germany Telephone: +49 (0) 2962 800-469 Email: [email protected] Website: www.josefsheim-bigge.de Contact: Martin Künemund, Project Manager Corporate Development

Avenida João Paulo II, 4410-406 Arcozelo, Vila Nova de Gaia Portugal Telephone: +351227537705 Email: [email protected] Website: www.crpg.pt Contact: António Rilho, Quality Professional

National Learning Network (NLN)

University Rehabilitation Institute

Republic of Slovenia (URI)

Roslyn Park Sandymount Dublin 4 Ireland Telephone: +353 1 2057276 Email: [email protected] Website: www.rehab.ie Contact: David Muldoon, Head of Accreditation, Standards & Supports

Linhartova 51, 1000 Ljubljana Slovenia Telephone: +386 1 4758176 Email: [email protected] Website: www.ir-rs.si Contact: Jurij Svajger, Business Director of Clinical Hospital for Physical Medicine and Rehabilitation

Page 3: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

3

Introduction

In October 2008, the EQUABENCH project brought together eight vocational, education and training (VET) organisations with a proven track record in innovation and continuous improvement. Partner organisations from Italy, Slovenia, Northern Ireland, Ireland, Belgium, Germany, Portugal and the Netherlands followed a two year work programme to share expertise and best business practice at workshops, benchmarking activities and dissemination events. This Innovative Practice Guide was produced by the EQUABENCH network as part of a Leonardo da Vinci funded project running from 2008 to 2010. This Guide explores the innovative practice used by a number of Vocational Education and Training (VET) organisations from across Europe in relation to the EFQM Fundamental Concept of Excellence – Results Orientation. Titles in the series are:

•Leadership and Constancy of Purpose

•Management by Processes and Facts

•People Development and Involvement

•Customer Focus

•Results Orientation

•Continuous Learning, Innovation and Improvement These guides consist of: 1. An overview of the practical steps that VET organisations should apply to assist

in the development of innovative practice. 2. An Appendix of detailed case studies from the participating VET organisations

which have informed the Innovative Practice Guides. Guides are freely available for download from partner websites and from the following sources:

http://equabench.epr.eu www.adam-europe.eu/adam/homepage

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

Page 4: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

4

Results Orientation - The EFQM Fundamental Concept

The Concept Excellence is achieving results that delight all the organisation’s stakeholders.

How the Concept is put into Practice In the fast changing environment that is today’s world, excellent organisations are agile, flexible and responsive as stakeholder needs and expectations change, often frequently and quickly. Excellent organisations measure and anticipate the needs and expectations of their stakeholders, monitor their experiences and perceptions, and monitor and review the performance of other organisations. Information is gathered from both current and future stakeholders. This information is used in order to set, implement and review their policies, strategies, objectives, targets, measures and plans, for the short, medium and longer term. The information gathered also helps the organisation to develop and achieve a balanced set of stakeholder results.

The Benefits

• Added value for all stakeholders. • Sustainable success for all stakeholders. • Understanding of the current and future requirements for performance in order to

set targets. • Alignment and focus throughout the organisation. • Delighted stakeholders.

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Page 5: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

5

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

The Practical Steps

The application of key aspects of the EFQM Fundamental Concept of Excellence within VET organisations is explored by the case studies included in the appendix of the Equabench Innovative Practice Guides. While these highlight a variety of innovative approaches and techniques relating to each Fundamental Concept there are common practical steps that must be applied by any organisation seeking to improve any part of their business. Each of the Guides in this series structures information and advice for VET organisations against these steps in relation to the Fundamental Concept under consideration. The steps have been designed to assist VET organisations to better apply the principles of the EFQM continuous improvement cycle to their context as is illustrated below.

Vision

Mission

Values

Objectives

Performance Indicators

Stakeholder Feedback

Results Co

ntin

uo

us L

ea

rnin

g, In

no

va

tio

n a

nd

Im

pro

vem

en

t

Page 6: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

6

The Practical Steps to Providing Effective Results

Orientation

Vision and Values

The EFQM Fundamental Concept of Excellence views excellence as achieving results that delight all the organisation’s stakeholders. The starting point for a VET organisation concerned with delivering or improving its Results Orientation should be its Vision. The Vision is an aspirational statement of what the organisation wants to achieve and it must be shared by staff and the organisation as a whole. The Vision statement should refer to the organisation’s key customer group and describe in broad terms the ultimate future goal the organisation is seeking to secure for the customer group, in other words it should be outcome orientated. For example, Cedar’s Vision clearly and simply describes what the organisation wants to achieve for its key customer group – a society in which disabled adults and children are fully included citizens. Central to this is clarity around the results the organisation wishes to achieve within each of its services and ensuring these are aligned to the organisation’s Vision and Mission. A Vision statement which orientates an organisation towards the results to be achieved is of paramount importance. Beneath the high level Vision statement, organisations that are seeking to improve Results Orientation also need to have a clear vision of what they want to achieve. In these circumstances the Vision recognises the issue to be addressed and identifies the improvements to be made. For example, at the outset of their project to establish the Objectives Matrix, Josefsheim Bigge had a clear Vision for improvement. Josefsheim Bigge lacked an instrument that could clearly define, bring form and structure to the fixed objectives of the entire organisation, set deadlines, as well as establish responsibilities. In contrast, NLN’s Vision in relation to Results Orientation is already well established and reflected in practice. The organisation’s attitude to Results Orientation is reflected in its outcomes focused approach at National, Regional, Centre and Programme level. The organisation’s Key Performance Results are established based on factual information and are reviewed using reports on organisational, functional and individual performance. Organisational Values are also an important element in delivering effective Results Orientation. Values define what is important to an organisation and will therefore inform what Results the organisation should measure. For example Centro Studi’s Medialabor project is founded in four guiding principles:

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Page 7: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

7

• Comprehensiveness: Attention to the holistic needs of the human being • Person Centred Personalised approach • A network approach • European and International benchmarking

This informed the type of Results examined by this project e.g. the connections between work performance and other aspects of life were assessed. Furthermore, inherent Values informed Josefsheim Bigge’s approach to Results Orientation. For example, the organisation’s commitment to the empowerment concept, the desire to have in-built programme flexibility to support person centred programmes and the aspiration to move ‘from the object to the subject’ in their approach. Pluryn goes further, with the organisation’s culture and ethos having a tangible impact on their Vision for Results Orientation. Whilst Pluryn use results to set, implement and review service level objective, targets, measures and plans for the short, medium and longer term in regard to stakeholders they feel it is important to ensure that time bound, outcome driven services do not work to the detriment of disabled or disadvantaged service users. Therefore, Pluryn seeks to temper their Results Orientated approach with ‘fuzzy-ness’ when this is in the client’s interest.

Mission

In order to achieve its Vision an organisation must develop a supporting Mission, which is a statement of purpose or intent which clarifies the organisation’s specific role to achieve the Vision. The Mission should be developed in a group setting either by the entire organisation, or in the case of larger organisations, by the senior management team. The process should involve consultation with stakeholders. For VET organisations the Mission provides a vital link between Vision and Strategy as noted by NLN: the organisation is committed to achieving its Mission through defining a set of Key Performance Results (KPRs) and implementing a quality management system to monitor and review these key performance areas. The KPRs translate the organisation’s Vision, Mission and Strategy into a coherent set of performance measures. Those organisations seeking to improve their Results Orientation need to establish their Mission or the approach they will take in order to achieve this. For example, Centro Studi decided to apply the Balanced Scorecard within their VET organisation mindful that VET organisations strive to deliver Mission outcomes, not superior financial performance. Yet the organisation recognised that a range of performance needed to be measured and that a system was required to support this.

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

Page 8: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

8

Mission statements can also add Results Orientation at project level. For example, the Mission of Centro Studi’s Medialabor project is to increase the occupational, personal and social autonomy of disadvantaged people, and people with disabilities in particular, and to increase their participation in the community. This statement links directly to the results measured – percentage of placements in the open labour market and number of companies involved. Ultimately, an organisation’s Mission in relation to Results Orientation is defined by the Fundamental Concept: in a fast changing environment that is today’s world, excellent organisations are agile, flexible and responsive as stakeholder needs and expectations change, often frequently and quickly. These changes will be reflected in the results achieved and therefore, an organisation needs to have access to the right results at the right time. As Cedar notes, the underlying belief is what gets measured gets done and that the right measures and targets should be in place to provide clarity about what constitutes organisational success.

Objectives

Vision, Mission and Value statements allow organisations to set the direction for the organisation and projects within the organisation. However, these statements fall short in detail and it is essential to provide clear statements of intent which describe the tasks that need to be completed to realise the Mission and progress the Vision. This is the function of objectives. Objectives provide the link between what is planned and what is done and they should reflect the results are being targeted. In terms of Results Orientation, objectives should be:

• A mix of strategic and operational tasks which relate both to the Vision, Mission and Corporate Strategy e.g. Cedar’s Balanced Scorecard system ensures a strong cascade between high level objectives within the Organisational Scorecard, Service Level Scorecard objectives and ultimately individual objectives within Work Plans;

• Developed across all the departments and functions of the VET organisation by relevant members of each team to promote ownership, realistic and achievable goal setting e.g. Centro Studi are considering the application of the Balanced Scorecard as a means of deploying a clear strategy through the single areas of the organisation including: Vocational Training, Rehabilitation, Social Area and Sports Activities;

• Communicated to all other staff members to ensure clarity of purpose e.g. Josefsheim has made their Objectives Matrix available to all staff via the intranet and this has resulted in all employees being aware of and understanding the objectives of the company and orientating their daily work towards them;

• SMART (Specific, Measurable, Achievable, Realistic/Relevant and Time-bound);

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Page 9: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

9

• Hierarchical in terms of importance to the organisation e.g. the prime objectives in Josefsheim are to secure the provision of service and the continued existence of the company. Performance, Economic, Organisational, Employee and Political Objectives are monitored in order to achieve these top objectives;

• Interconnected to the teams and individuals responsible for achieving them; and

• Sufficiently flexible to best fit the needs of the client group e.g. Pluryn recognises the importance of setting objectives which are SMART but also acknowledges that due to the nature of client groups and the rehabilitation sector that some objectives need to be more flexible. These flexible objectives need to deal with unexpected positives and negative incidents and occurrences.

Performance Indicators and Stakeholder Feedback

The design and deployment of Key Performance Indicators and mechanisms to capture stakeholder feedback are core to Results Orientation. As the Fundamental Concept of Excellence states: Excellent organisations measure and anticipate the needs and expectations of their stakeholders, monitor their experiences and perceptions and monitor and review performance in the organisation. Information is gathered from both current and future stakeholders. Performance Indicators These are measurable indicators of performance or a stepping stone to be used along an agreed pathway that will show whether the organisation is making progress towards achieving its objectives and targeted results. Performance Indicators should be closely aligned to objectives and may in effect provide the ‘measurable’ element of the SMART acronym. The selection of Performance Indicators should not be rushed as it is important to ensure that the measure selected validly represents performance against the objective and that the measure is not selected solely on the basis that information is readily available. Within the VET sector an organisation may have Performance Indicators imposed upon them from an external source. For example, URI is subject to a range of legislation and statutory requirements. The Slovenian Ministry of Labour, Family and Social Affairs evaluates and compares the performance of Vocational Rehabilitation providers against a range of outcome measures including: outcomes in employment; and the amount / diversity of employers. Therefore URI’s Professional Team needs to meet several levels of demands or standards: • Standards of vocational rehabilitation services • The national norm regarding number of clients per team per year • Expected Outcomes:

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

Page 10: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

10

1. External: employments / active measures, increased employability, number of employers

2. Internal: employers in ordinary labour market, variety of employers, long-term solution of employment problems.

A Results Orientated VET organisation will deploy Performance Indicators to track results at a strategic and operational level and everything an organisation sets out to achieve should have an associated Performance Indicator. The following learning and practice within the EQUABENCH network can be applied by any VET organisation.

• CRPG use a comprehensive, analytical model on outcome evaluation which is based on the Quality of Life conceptual model. The model consists of 3 dimensions: Personal Development, Social Inclusion and Well Being and 8 associated variables. 51 metric indicators have been distributed for the 8 variables to assess the effectiveness of CRPG’s programmes in improving an individual’s quality of life.

• NLN utilise a model which focuses on using Key Performance Results to translate the organisation’s Mission, Vision and Strategy into a coherent set of performance measures. National KPRs are agreed by the Senior Management Team and are deployed through the organisation’s three regions and functional departments.

• The definition and measurement of results within Cedar is categorised with three specific areas:

1. High level Key Performance Results (Organisational) 2. Service level Key Performance Indicators (Principal and Secondary) 3. Generic Key Performance Indicators (Organisational Wide, but at a Service

Level) Stakeholder Feedback The work of VET sector organisations must be very person centred, especially for those organisations working with a disadvantaged client group. The Vision, Values, Mission, Objectives, Performance Indicators and Results should focus on the trainee and be developed to ensure trainees’ needs are being met and measured. Feedback from other stakeholders such as funders, carers and other professionals is essential in designing a balanced range of Performance Indicators and securing relevant results. Stakeholder feedback should be used by VET organisations to shape the type of Results being measured. This stakeholder engagement will quality assure an organisation’s Results Orientation approach ensuring they are measuring the right Results in the most appropriate way. For example:

• Cedar measures a range of results to reflect the agenda of all stakeholders. Cedar segments the results it wishes to achieve in terms of customer, people, society and

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Page 11: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

11

key performance. Customer Measures include service user satisfaction, sustained outcomes, customer demand, complaints and feedback from internal customers. People Measures include staff satisfaction, levels of investment in training and development, staff absence, staff involvement in improvement activity and take up of employee benefits. Society Results include perceptions of services, number of volunteers and number of environmental measures in place. Key Performance Measures include the level of service growth and expansion, securing funds in a competitive market and improvement in technological resources.

• CRPG’s Outcome Evaluation recognises the need to gather and analyse information that comments on the effects of multi level interventions.

A paramount consideration for a VET organisation concerned with Results Orientation should be direct feedback from their trainees to comment on the effectiveness and efficiency of the intervention provided. This is usually completed by surveying clients and in some instances their case manager to ascertain their views. For example: • CRPG used the Quality of Life Model to identify the success of their interventions

on their client group. In this case they conducted 60 closed questions and 1 open question for comments and suggestions. This model focuses on assessing the success of their rehabilitation programme by measuring personal development outcomes; well-being outcomes; and social inclusion outcomes. The process also involves surveying case managers to ascertain their views on the success or otherwise of the intervention. Best practice recommends that this should be carried out on a face to face basis as both quantitative, qualitative and anecdotal data will be considered in evaluating success.

• Centro Studi uses a Client Centred Results Orientated model which focuses on setting objectives and measuring the results in achieving these objectives. An integral part of this approach was defining both processes and procedures. Every 6 months there is a process revision undertaken by top management to define corrective actions on the basis of objective evaluation and the results of client satisfaction questionnaires.

Other means of securing feedback are described within the Customer Focus Innovation Practice Guide.

Results Organisations must ensure that the qualitative and quantitative Results are collated using robust mechanisms to determine whether Performance Indicators have been achieved. Analysis must then be undertaken to assess whether the intervention has had the desired impact. As the Fundamental Concept states: This information is used in order to set, implement and review their policies, strategies, objectives, targets,

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

Page 12: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

12

measures and plans for the short, medium and longer term. This is core to a VET organisation achieving true Results Orientation. It is also essential that the mechanisms used to collect and collate results are fit for purpose and appropriate to the range of measures being used. For example Cedar focuses on 3 Key Performance Results which demonstrate achievement of the organisation’s Mission. These include:

• The number of disabled people served by the organisation;

• The level of customer satisfaction with services received; and

• The number of services being provided to meet the needs of disabled people across Northern Ireland.

Measuring, tracking and managing Results is achieved through the use of a Balanced Scorecard and Traffic Lights system. Together they constitute an effective reporting and management response system to assure measures are deployed and targets are met. Results achieved are tracked at quarterly management review and bi-monthly review of organisational Key Performance Results by the senior management team. Similarly, Josefsheim Bigge’s Objectives Matrix is reviewed quarterly to determine the fulfilment and implementation of the planned objectives. The achievement of individual objectives and measures within projects is assessed using a Traffic Light System. NLN monitors and reviews its Results against annual plans as follows:

• Information Cascade – Monthly meetings and feedback at all levels

• Quarterly Reviews – using the monthly meetings structure

• Compass Reporting System – service user data

• Sun Reporting System – financial data

• Source Reporting System – human resource data

Continuous Learning, Innovation and Improvement An organisation’s Results Orientation is incomplete without addressing the feedback loop or continuous improvement cycle. Results captured must be reviewed and analysed and on the basis of these facts, improvement must be planned and enhanced results targeted. By analysing and reflecting upon the range of Results measured and the actual Results achieved VET organisations can establish:

• Whether they are measuring the range of Results necessary to achieve a balanced set of stakeholder Results; and

• Whether the Results achieved demonstrate targeted levels of success.

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Page 13: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

13

The former consideration will allow VET organisations to make improvements to their Results Orientation practice in terms of shaping and informing what is measured. The second consideration will enable VET organisations to target improved performance by amending and refining their practices. Josefsheim Bigge recognises the need for continued improvement of its system for Results Orientation. They view the Objectives Matrix as currently too comprehensive and will seek to improve the model’s brevity, clarity and simplicity to enhance the understanding of everyone in Josefsheim. Cedar’s system is designed to assure ongoing refinement of the Results measured and to drive innovation. To this end Cedar:

• Assesses the currency of measures and targets;

• Systematically reviews Results;

• Ensures there is follow up action if targets are not met; and

• Revises measures and targets as required. Furthermore, Cedar stresses the importance of having a clear rationale for targets and measures, the need for a numerical target and the value of comparative benchmarking. Centro Studi describes improved practice based on analysis of Results. The Self Help Project was initiated to address issues arising from the following Results:

• A low percentage of work placement for people with psychiatric disorders; and

• Difficulties with job retention within this client group. The organisation responded with a changed approach and improved Results were subsequently achieved i.e. a higher percentage of people with psychiatric disorders were employed and a higher percentage of jobs were retained by this client group. NLN also provides an example of adjusting plans based on Results achieved. During reviews in 2009 it became obvious that progression to employment had slowed due to the economic crisis and adjustments were made to outcome targets in 2010. New targets were set for learner certification as a result of ongoing analysis of outcomes during the last year. CRPG notes that the knowledge developed from outcome evaluation is useful for the restructuring of programmes, improving their effectiveness. By maintaining a sound Results Orientation, VET organisations are equipped to add value and achieve sustainable success for all stakeholders and to meet the changing requirements of their customers by continually improving their services.

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

Page 14: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

14

Appendix - The Case Studies 1. Key Factors for the successful implementation of the Balanced

Scorecard in a VET Organisation—Centro Studi The Balanced Scorecard (BSC) Model is a facilitative tool for effectively managing results orientated organisations. This paper examines its application in VET organisations and makes recommendations about how to heighten the impact of the model in the VET sector. Mission Management Support The BSC is a strategic management system that translates an organisation’s mission and strategy into tangible objectives and measures. It requires identification of performance drivers and then acting on the levers in order to achieve the final result. The model suggests viewing the organisation from different perspectives rather than just financial in order to monitor key success factors from a balanced point of view to ensure the realisation of its mission. Other perspectives include Stakeholder, Processes and Learning & Growth. However, being successful means being effective and efficient which means getting outcomes at a lower cost.

Which factors are of particular importance if we apply the BSC model in VET organisations? VET organisations strive to deliver mission outcomes, not superior financial performance. Therefore it should be naturally easier to apply the approach that the BSC model proposes, although there is still a very strong emphasis on the financial perspective. The strength of the values that underline a VET organisation can temper this and lead to a more balanced approach. The BSC model is about improving performance. In order to improve performance, we need to learn how to measure it. In the case of VET organisations the objectives are difficult to define in a quantitative manner. For example, Don Calabria, the founder of Centro Don Calabria, wanted to help others that were in difficulty especially disadvantaged young people who were marginalised from society. According to him, the best way to do this was through education and employment. The realisation of his vision subsequently led to the development of the organisation’s Centro Don Calabria. However, it is a major managerial challenge to define the objectives of social inclusion in a way that can be measured. VET organisations deliver services not products. When we think about approaching the needs or the ‘market’, we must consider the so-called four ‘P’s – Product, Price, Place and Promotion. When we deliver ‘services’ instead of ‘products’ there are 3 more ‘P’s that we should consider – Processes, Physical Evidence and People. Marketing a service based organisation is different from marketing a goods based organisation. These differences include:

• What the user or buyer receives or purchases is intangible

• It is difficult to compare the quality of similar services

• The user might not appreciate the full worth of the service if they do not have a good

benchmark for comparison

• The user or buyer cannot return the service

This requires an emphasis on the importance of measuring performance and it means that VET organisations should identify and define their processes to support this. The level of personal satisfaction and training is also of particular importance and the role that competent

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Page 15: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

15

and satisfied staff play in the delivery of satisfactory services also needs to be addressed.

Today organisations operate in a business environment that keeps changing quickly and very often discontinuously. The only effective way to create an organisation that is flexible enough to adjust to the changes of external factors, at least as quickly as they keep changing, is the management of human resources. Therefore when constructing a BSC model particular attention should be paid to this process. What could be the opportunities for Centro Studi to apply the Balanced Scorecard? The implementation of the BSC model requires a clear strategy that could be deployed through single areas of the organisations including: Vocational Training; Rehabilitation; Social Area; and Sports Activities. The model could help to introduce a process approach and quality management systems as the implementation and management of BSC requires the identification and focus on internal processes that are crucial for the achievement of the results and strategy implementation. The BSC model also requires identification of objectives and key performance indicators that allow for the measurement and monitoring of components; this is an approach that brings a cultural change in the organisation. Having regard to the stand alone nature of the four strands of the Don Calabria Centre, the BSC model could be applied at the level of single areas or even smaller organisational units, e.g. an adult vocational training area. What are the rules of road to follow to enhance the implementation of the model?

• Define an organisational area (unit) where you want to use the BSC as a managerial tool

• Make it a priority

• Make it simple

• Work on the initiatives that you need to accomplish and once they are completed, identify

the measures

• Work with staff who understand the need for change and are willing to undertake the

challenge

• Keep it informal, don’t formalise it unless you have real results that become arguments

• Create the quality culture and start introducing elements of the quality management system

• Construct and implement. Prove it to improve it

• Use it as a communication tool

• Monitor the competence of the staff in order to plan and deliver appropriate training

• Monitor staff satisfaction

• Develop a programme of staff evaluation and promotion

Usually organisations need 2 to 3 years before they implement BSCs as a management tool. This time is required to design, build and implement the whole model. Therefore, invest time in single phases of the application of the model. For example:

• provide training to reduce change resistance

• monitor and provide feedback on activity development

• compare results with those planned

• verify if there are any changes in the business environment (single and double loop

learning)

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

Page 16: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

16

2. A results orientated employment access service —Centro Studi

Opera Don Calabria

1 MEDIALABOR Medialabor, founded in 2000, is an employment access service within the Vocational Training Area of Centro Don Calabria. It collaborates with the other areas of the Centre: the Rehabilitation Area and the Social Area according to a global approach. The mission is to increase the occupational, personal and social autonomy of disadvantaged people, and of people with disabilities in particular, and to increase their participation in the community. Key aspects of the service include: • 4 Employment Access Case Managers for people with disabilities (pwd) • 5 Employment Access Case Managers for disadvantaged people • 3 Disability Management Experts • 43% of placements of pwd in the open labour market • 450 Companies The guiding principles of Medialabor are • Comprehensiveness : Attention to the holistic needs of the human being • Person Centred Personalised approach • A network approach • European and International benchmarking

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

MEDIALABOR

Page 17: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

17

2 RESULTS ORIENTATED APPROACH Part of the client results orientated approach was the definition of processes and procedures as follows

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

PROCESSES

PROCEDURES

SP01 COMPLAINT MANAGEMENT

P.1.1 Complaint management

SP02 EMPLOYMENT ACCESS

P.2.1 In charge process P.2.2 Needs assessment P.2.3 Company identification P.2.4 Programme Agreement P.2.5 Work experience P.2.6 Hiring advice

SP03 EMPLOYMENT MONITORING

P.3.1 On going Employment monitoring

SP04 SATISFACTION CHECK

P.4.1 Users questionnaire P.4.2 Companies questionnaire P.4.3 Personnel questionnaire

SP05 USERS ADVICE P.5.1 Users advice P.5.2 Traumatic Brain Injury (TBI) Users advice

SP06 JOB REINTEGRATION (especially TBI) P.6.1 TBI Job reintegration

SP07 DISABILITY MANAGEMENT P.7.1 Disability Management

SP08 STAFF TRAINING P.8.1 Staff training

SP09 MANAGEMENT REVIEW P.9.1 Management Review

Page 18: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

18

Objectives are defined on annual basis. 3 PROCESS REVISION SUMMARY Every six months there is a process revision with the top management in order to define corrective actions on the basis of: • Objective evaluation; and • Results of Client satisfaction questionnaire. 4 EXAMPLES OF IMPROVEMENT ACTIVITY 4.1 SELF HELP PROJECT As a result of analysing data about PWD Placement we identified some issues: • a low percentage of placement for people with psychiatric disorders; and • difficulty in job retention for people with psychiatric disorders.

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

OBJECTIVES 2009 INDEXES 2009 RESULTS 2009

RESULTS 2008

New employment access plans

70 53 59

Employment percentage 42% 37% 42%

New companies

41 30 41

Number of rehabilitation treatments for employment access

617 496 617

Public recognition of Medialabor

To obtain a regional accreditation of Medialabor as Employment Access Service

Regional accreditation reached in June 2009

Request of accreditation not successful

Disability management projects

4 projects 3 projects 2 projects

Definition of collaboration with On Going training area

Define a procedure of collaboration between Medialabor and On Going Training Service

A common procedure in the field of ESF Project Management has been defined in November

Page 19: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

19

In line with our network approach we defined a partnership with a Self Help association (According to the Self Help Method clients are the principal protagonists of their own rehabilitation). We more clearly defined the roles of partners: Medialabor: Employment orientation and guidance Self Help Association: Therapeutic/Pharmacological aspects, Housing, Leisure time Results: • Updated information between partners as regards psychological health • Connections between work performance and other aspects of life • Prevention • Higher percentage of placement • Higher percentage of job retention 4.2 DISABILITY MANAGEMENT PROJECTS From analysis of data about companies’ satisfaction it was possible to identify the need for improving the companies’ abilities in managing disadvantaged employees (not only pwd). According to the European and international dimension we updated our knowledge and competencies in the field of disability management in 2005. • Return to Work Co-ordinator Certificate accredited through the National Institute of

Disability Management and Research in Canada in 2005 • Post graduate diploma in Trust Counselling through the Jurisprudence Studies

Department of Verona, Italy in 2008 • Benchmarking within the European Platform COMPANY BASED DISABILITY MANAGEMENT ACTIVITIES: • Analysis of the companies' needs and characteristics • Analysis of the employees' needs (questionnaire about well being) • Training sessions on topics connected to wellbeing, addressed to managers and

employees (health and safety issues, healthy lifestyle, addicts, accidents at work, etc.) • Training sessions about disability awareness, group leading capabilities, communication

skills • Booklet with information on topics of wellbeing and prevention • Analysis of risky jobs • Back to study courses for interested employees • Case management for disadvantaged employees Collaboration with 4 companies • A Company dealing with public utilities in Verona 5 year project • A Pharmaceutical Company located in Milan 2 year project • The Town Council of Verona 2 year project • An important National Bank 4 year project

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

Page 20: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

20

3. The effective deployment of The Balanced Scorecard as a

performance management system—Cedar

Over a period of 15 years the Cedar Foundation has developed a performance management system through the effective deployment of Balanced Scorecards. This approach is integral to Cedar’s drive to develop and deliver quality services in achieving its vision of a society in which disabled adults and children are fully included citizens. Central to this is clarity around the results the organisation wishes to achieve within each service area, ensuring that these are aligned to the organisation’s vision and mission. The underlining belief is what gets measured gets done and that the right measures and targets are in place to provide clarity about what constitutes organisational success. The definition and measurement of results within Cedar is categorised within 3 specific areas: • High level Key Performance Results (Organisational) • Service level Key Performance Indicators (Both Principal and Secondary) • Generic Key Performance Indicators (organisation wide but at a service level) High Level Organisational Key Performance Results examples include: • Number of disabled people being served by the organisation • Level of customer satisfaction with the services received • Number of services being provided to meet the needs of people with disabilities across

Northern Ireland The organisation has sustained a high level of performance against these areas over a number of years ensuring that it is reaching the maximum number of people with disabilities and that services are delivered to high levels of customer satisfaction.

Key Performance Indicator Customer Satisfaction Table

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Page 21: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

21

Service Level Key Performance Indicators are segmented on the basis of Principal and Secondary Indicators. For example, within our Training & Brain Injury Services a Principal Indicator includes positive outcomes achieved by the leavers from the services including progression to employment or further training. Secondary Indicators include the number of training hours delivered, programme occupancy and qualifications gained. Generic Performance Indicators include: staff absence, staff satisfaction, financial turnover, purchaser satisfaction, environmental performance and number of volunteers employed. These are measured across the organisation at a service level. Using the Business Excellence Model, the organisation has segmented the results it wishes to achieve in terms of – customer, people, society and key performance results. Customer measures include service user satisfaction, sustained outcomes, customer demand, complaints and feedback from internal customers. People measures include staff satisfaction, level of investment in training and development, staff absence, staff involvement in improvement activity and take up of employee benefits. Society results include perception of services, number of volunteers and number of environmental measures in place. Key Performance Measures include the level of service growth and expansion, securing funds in a competitive market and the improvement in technological resources. Measuring, tracking and managing results is achieved through the use of a Balanced Scorecard and Traffic Lights System. Together they constitute an effective reporting and management response system to assure measures are deployed and targets are met. The deployment of the system is achieved via the cascade and aggregation of measures and targets, service level tracking through quarterly management review and bimonthly review of Organisational KPI’s by the Senior Management Team.

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

Page 22: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

22

The system ensures a strong cascade between high level objectives, targets and measures within the Organisational Scorecard and those within Service Level and ultimately Individual Scorecard (Workplans). In reverse, the system facilitates an aggregation of individual results achieved. These are fed into Service Level Scorecards and finally aligned to the Organisational high level results. Cedar’s experience is that measurement and target setting alone is insufficient; there also needs to be an effective system in place to be assured that: • The measures and targets are current and valid; • Results are systematically reviewed; • There is follow up action if targets are not being met; and • Measures and targets are revised as required.

The system is designed to assure ongoing refinement and to drive innovation. Experience within the Cedar Foundation has indicated the following key learning points: • It is essential to get the measure right at the outset but be prepared to look for alternative

measures against key objectives. • All objectives should be SMART and there should be a clear rationale in relation to the

individual target such as historical performance. • Where possible, the target should be clearly defined in numerical figures e.g.

percentage figures. • Benchmarking needs to be an integral activity to provide stretch targets. In general Cedar has found the application of the Scorecard provides for an effective performance management system in ensuring that we measure and set targets appropriately against key objectives. This approach provides clarity of expectations on what can be achieved against objectives, it facilitates planning, implementation and evaluation and promotes accountability and transparency. Most of all it allows a realistic setting of challenging targets for improvement and innovation.

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Page 23: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

23

4. The objectives matrix as a higher instrument for the planning

and controlling of a company—Josefsheim Bigge

1. Current situation and reason for the development of the objectives matrix in the

Josefsheim Bigge

“Only one who knows the objective will find the way”. This proverb from the Chinese philosopher Laozi from the 6th century B.C. shows how important clearly and precisely formulated objectives are. In the past, the company objectives of Josefsheim were not transparent or comprehensible for all employees. There was no target setting system that could make the objectives of the company available and understandable to all. The strategy of the managers in the individual departments of Josefsheim only applied to their own respective areas of responsibility. Beyond the objectives of the individual departments, there was often unclarity or, at least, uncertainty. Within the framework of the objective and strategy development of the company, a desire was expressed for change to a more holistic way of thinking. The aim was to loosen the organisational structure and, in future, to orientate more on the contents of the objectives. Further guidelines in the development process of the company lead from the assignment of care to the self-determination of people with disabilities (empowerment concept). The future-oriented planning process requires that the system structure is flexible and can allow for adjustment. The changes should not take place with the system, but rather on an individual basis with people. The guideline “From the object to the subject” also expresses the aspired pedagogic in Josefsheim. Josefsheim Bigge was lacking an instrument that could clearly define, bring form and structure to the fixed objectives of the entire organisation, set deadlines, as well as establish responsibilities. There was need to develop an instrument that was able to recognise possible existing problems by means of target/actual comparisons and to work on solutions. Furthermore, the instrument should coordinate the diverse activities inside and outside Josefsheim and provide orientation to assist employees with their decisions and actions. Finally, it is imperative to know which objectives should be followed by the individual employees with how much priority for a sensible distribution of the scarce financial, personnel and material resources. In order for this to happen, the types of objective there are in Josefsheim and how they relate to each other must be clarified. 2. Objectives in Josefsheim Bigge The prime objectives of Josefsheim are to secure the provision of service and the continuing existence of the company. In order to achieve these top objectives, it is important to monitor the following: • Performance objectives • Economic objectives • Organisational objectives • Employee goals • Political objectives

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

Page 24: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

24

All objectives are of equal value and are linked to one another. Only together can they secure the progress of the company in the long term.

Figure I: Relationship of the objectives of non-profit organisations

The short-term and middle-term strategic and operative secondary objectives of Josefsheim have been identified, defined and fixed in their core purpose with the advisory boards and employees. Attention has, of course, been paid to make sure that the defined objectives of Josefsheim Bigge are in compliance with the strategic objectives of the Josefs-Gesellschaft Group as a whole.

3. The objectives matrix The developed objectives matrix is the higher instrument for planning and controlling in Josefsheim. By using it, company strategy with specific objectives and measures can be implemented. The objectives matrix visualises the objectives to be reached in a period of 3 to 5 years. Due to internal and external influences, it is in a continuous state of flux.

Figure II: The objectives matrix in the Plan-Do-Check-Act (PDCA) cycle as a basis for our learning organisation

The objectives matrix defines, specifies and gives a clear structure to the fixed objectives of the entire establishment of Josefsheim Bigge.

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Secure the provision of

Service and Support the Organisation

Secure the

provision of Service and Support the

Organisation

Political

objectives

Political objectives

Organisational

objectives

Organisationalobjectives

Economic objectives

Economic

objectivesPerformance

objectives

Performance

objectives

Employee

goals

Employeegoals

QM

Controlling

QMControlling

Secure the provision of

Service and Support the Organisation

Secure the

provision of Service and Support the

Organisation

Political

objectives

Political objectives

Organisational

objectives

Organisationalobjectives

Economic objectives

Economic

objectivesPerformance

objectives

Performance

objectives

Employee

goals

Employeegoals

QM

Controlling

QMControlling

Implementation

Assignment of

objectives/planning

Analysis/

diagnosisAnalysis/

diagnosis

Review

e.g. suggestion for improvement

Visualised: e.g.

annual report

Visualised: e.g.

objectives matrixImplementation

Assignment of

objectives/planning

Analysis/

diagnosisAnalysis/

diagnosis

Review

e.g. suggestion for improvement

Visualised: e.g.

annual report

Visualised: e.g.

objectives matrix

Page 25: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

25

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

The matrix is given a quarterly review on the fulfilment and implementation of the objectives. Have the agreed individual objectives been achieved by the projects and measures which were undertaken? The assessment of the individual objectives is shown visually with the help of a “traffic light” system. 4. Experiences with the objectives matrix • The company strategy of the Josefsheim and operative actions have been linked

together by means of the objectives matrix. • The objectives matrix clearly indicates what categories (e.g. performance objectives,

economic objectives, etc.) match the individual objective targets of the company. • The objectives of Josefsheim are now transparent, operational and understandable. The

objectives matrix has been made available to all employees via the intranet. All employees are aware of and understand the objectives of the company and orientate their daily work towards them.

• Our objectives matrix is still too comprehensive. It should be kept as brief, clear and as simple as possible so that it can be understood by everyone in Josefsheim. The necessary clarity of objectives can be achieved through classification in a hierarchy of objectives and a well-defined prioritisation of the individual objectives. It should be clarified how urgent each objective is and what importance it holds for Josefsheim.

• Further questions to be resolved concerning the objectives matrix include: • Are the guidelines clearly defined and differentiated? • Are the present guidelines correct? • Do the guidelines help us with the development of our organisation?

The objectives matrix is currently under further development. Further reading: The objectives matrix was developed specifically to our requirements based on our practical experience. We found suggestions for development in the following book: Bachert, Robert; Vahs, Dietmar (2007): Change Management in Non-profit Organisations, Schäffer-Poeschel Verlag, Stuttgart.

Page 26: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

26

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Fig

ure

III: O

bje

ctives m

atr

ix

Page 27: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

27

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

5. How results influence strategic planning—National Learning

Network

Excellent organisations meet their Mission and progress towards their Vision through planning and achieving a balanced set of results that meet both the short and long term needs of their stakeholders and, where relevant, exceed them. Excellent organisations use this information to set, implement and review their policies, strategies, objectives, targets, measures and plans for the short, medium and long term (EFQM). Approach National Learning Network is committed to achieving its mission through defining a set of Key Performance Results (KPRs) and implementing a quality management system to monitor and review these key performance areas. The KPRs translate the organisation’s Mission, Vision and Strategy into a coherent set of performance measures. The organisation’s attitude to Results Orientation is reflected in its outcomes focused approach at National, Regional, Centre and Programme level. The organisation’s KPRs are established based on factual information and are reviewed using reports on organisational, functional and individual performance. Deployment National KPRs are agreed by the Senior Management Team and are deployed through the three regions and functional departments. The regions, functional departments, centres and individual courses prepare plans which feed into the National KPRs. Management information systems and reporting processes are used to monitor and compare the key management and service delivery functions of the organisation. All staff members are involved in the process through planning and review meetings. Assessment and Review National Learning Network monitors and reviews its annual plans as follows:

• Information Cascade – Monthly meetings and feedback at all levels

• Quarterly Reviews – Using the monthly meetings structure

• Compass Reporting System – Service User Data

• Sun Reporting System – Financial Data

• Source Reporting System – Human Resources Data

Page 28: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

28

Information Cascade

Plans are reviewed, and revised where necessary on a quarterly basis at all levels using reports produced monthly through the global IT System. NLN also has a number of qualitative and quantitative measures which enable it to determine if it has achieved its goals or if corrective action is required. For example during reviews in 2009 it became obvious that progression to employment had slowed due to the economic crisis and adjustments were made to outcome targets for 2010. New targets for learner certification were set as a result of ongoing analysis of outcomes during the last year.

Advantages of NLN’s Quality Management System

• Provides clear focus on what we want to achieve

• Sets clear and measurable targets

• Leads to defining plans on how to achieve targets

• Promotes staff engagement & accountability

• Provides framework for review

Results - Quantitative Results - Qualitative

Employment Outcomes Full & Part-Time Supported Employment Progression Outcomes Further Training Further Education Qualifications Achieved

Health & Social Gain Perception Measures Satisfaction Surveys Exit Questionnaires Complaints and Compliments Feedback from Learner Representatives, Individual Planning, Monitoring & Evaluation

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Network Management Team (NMT)

Regional Meetings Regional Director, Area Managers,

Regional Team

Area Meetings Area Manager, Instructors, Support Staff

Page 29: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

29

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

6. Quality of life model as a tool for Outcome Evaluation.—CRPG

The outcome evaluation arising from the programmes implemented in the field of people with disabilities is a process of gathering and analysing information that allows the extraction of knowledge about the effects of multilevel interventions, being of high relevance for the mechanisms of decision making in the process of continuous improvement. The conclusions generated by the outcome evaluation allow for maximising the adjustment of the programmes to the characteristics and peculiarities of the target-audience. Thus, the knowledge developed from the outcome evaluation is useful for the restructuring of programmes to improve their effectiveness. It is also a way of measuring the impact of such interventions, and justifying their need. In the current context, where funding indexed to results is already a practice in some of the European countries, the outcome evaluation can also be useful to understand the soft indicators concerning the main goal of interventions - promoting clients’ quality of life. Purpose The outcome evaluation has the purpose of analysing the consistency and sustainability in the time span of the changes arising from the actions implemented. Analytical model Given that enhancing the clients’ quality of life is the main goal of an habilitation/rehabilitation process, the analytical model chosen to evaluate those interventions/outcomes is the Quality of Life Model. Based on the WHOQOL Group

1 definition, quality of life can be understood as the individual’s

perception about his/her position in life, according to the cultural context and the systems of values in which he/she lives, as the result of the interaction between his/her objectives and expectations and the objective indicators defined for his/her social and cultural environment. Different authors have been studying the domains that compose quality of life concept. Schalock

2 has identified eight critical components: emotional; well-being; interpersonal

relations; material well-being; personal development; physical well-being; self determination; social inclusion; rights. In order to fine-tune this model for the outcome evaluation purpose, its architecture was adapted, integrating a multidimensional organisation, assimilating 3 structural dimensions, each one composed by different variables:

1 The WHOQOL Group – The World Health Organization Quality of Life Assessment (WHOQOL): Devel-opment and General Psychometric Properties. Social Science & Medicine, 46, 12 (1998) 1569-1585 (adapted). 2SCHALOCK, R. (ed.) – Quality of Life 1: Conceptualization and measurement. Washington: AAMR, 1996.

Page 30: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

30

Operational diagram of the analytical model on outcome evaluation, based on the quality of life conceptual model

Dimensions definition:

• Personal development - refers to the range of relationships that shape the competence

structures, articulating with the patterns of human activity. This process is characterised by a mechanism through which a person acquires competence, control and influence over a number of significant issues related to multiple relationships within the contexts in which they are included. Thus, the dimension incorporates the perception of personal competence in an interactional situation, either in the context of interpersonal relationships or in the exercise of self-determination. In this perspective, the dimension is segmented in 2 variables: Interpersonal Relations and Self-determination.

• Well-being – refers to the conditions of life understood as desirable by the person in 3 key

areas: emotional well-being, physical well-being and material well-being. This dimension focuses on the way people think about themselves, including specific areas of acceptance, perception of their disability, satisfaction of the interaction with the contexts of life, and individual perception about the relationship between aspiration and achievement in a number of areas. These areas include mobility, leisure, daily life activities, property, and income, among others. Thus, this dimension is segmented into 3 variables: Emotional, Physical and Material.

• Social inclusion - refers to the opportunities to control the interaction with the

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Page 31: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

31

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

environment and to influence the decisions which have an impact on their life projects. This dimension integrates a set of mechanisms, through which the individuals learn to identify close relations between their aims and the way to achieve them, gaining a broader access and control over the resources. In this sense, it is useful to evaluate the outcomes on the domains of employability, citizenship and rights, analysing the current situation in their professional field, competences for retaining the job, career management and development, level of participation in the community and perception regarding the control and efficiency over the collective phenomenon. Therefore, this dimension is segmented in 3 variables: Employability, Citizenship and Rights.

Methodological guidelines Instrument The chosen measure instrument is a questionnaire, which is composed of 60 closed questions and 1 open question for comments and suggestions. This methodology has the advantage of fast data collection, analysis and comparison; lowest cost for the greatest amount of information. Operationalisation of the metric indicators The operationalisation of the outcome indicators has resulted in 51 metric indicators, distributed for the 8 variables which integrate the quality of life model. Considering that professional rehabilitation programmes have a special focus on employability, the satisfaction and outcome axes were considered. Therefore, it is possible to identify potential relations within the data analysis and information extraction. Time frame The outcome evaluation is implemented 1 year after the conclusion of service provision. The completion of the questionnaire can vary within a 2 months timescale. Comprehensiveness The questionnaire is administered to all clients of the rehabilitation/habilitation pathways who have completed 1 or more interventions, with a timeframe of not less than 2 months. However, other timeframes for service provision can be considered for specific services, if judged appropriate.

Page 32: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

32

7. Enhancing Vocational Rehabilitation Results: A Case

Management approach to co-operation with employers—

University Rehabilitation Institute, Republic of Slovenia In Slovenia vocational rehabilitation is defined in the Vocational Rehabilitation and Employment of Persons with Disability Act (enforced since 2006) by 14 services, describing the vocational rehabilitation process and setting common standards of practice for service providers. The Ministry of Labour, Family and Social Affairs has also set the norm, defining the VR team and the work load that such a team has to manage on an annual level. The outcomes of the teams are evaluated annually against this norm and against other outcome measures such as the rate of the outcomes in employment and the amount/diversity of the network of employers.

The Professional Team needs to meet several levels of demands or standards:

• Standards of vocational rehabilitation services

• The national norm regarding number of clients per team/year

• Expected outcomes:

• External: employments/active measures, increased employability, number of employers

• Internal: employments in ordinary labour market, variety of employers, long-term solution of employment problems

The standards of vocational rehabilitation services emphasise the importance of on-the-job training for the final outcomes of the process. This has also been supported by new legislation in this area. For all service providers this means a substantial challenge in organising and managing the shift of focus of the rehabilitative training from centre-based to employer-based.

In order to increase the ability of a professional team to manage the cooperation with increasing numbers and variety of the employers, the Ljubljana team of the Vocational Rehabilitation Centre (University Rehabilitation Institute, Republic of Slovenia) has introduced a case management approach. The main change that this approach imposes is reflected in the fact that it increases the level of professional activities that are shared between team members. This means that every team member is able to manage the vocational rehabilitation process of a client.

This approach is being used also on the level of managing the entire rehabilitation process. In order to be efficient this approach needs to be addressed by several aspects of management at the same time:

• the case management approach to team work,

• support from the quality system and

• other staff supporting activities

The model is illustrated by the approach that is being used for cooperating with employers:

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Page 33: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

33

*An example of a supportive document is illustrated below.

Impact of the approach More efficient and balanced work load management as reflected in: • Increased ability to respond and flexibility • Shared responsibilities • Shared ownership of the Vocational Rehabilitation process • Shared work load/burnout prevention Enhanced results (quality and quantity) – expectations for 2009: • 68% active outcomes • 23% level of employment of people with disability (national level for 2009 – 11%) • 73% employments/trainings in mainstream environment

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

COOPERATING WITH EMPLOYERS CASE MANAGEMENT TEAM

APPROACH QUALITY SYSTEM

SUPPORT STAFF SUPPORTING

ACTIVITIES

• Sharing client load

• Sharing employer load

• Applying client centred approach

• Empowering clients/ employers

• Seeking/providing/exchanging specialised expertise (team members, environment...)

Providing easy (via intranet) access to:

• Standards of services

• Organisational procedures (description of processes, activities...)

• Supportive documents*

• Work instructions

• Learning materials

Continuous professional development e.g.:

• Training in case

management

• Training in empowering clients and creating an empowering environment (e.g. Vrij Baan, Request)

• Training in supported employment strategies

Professional burnout and stress prevention:

• Intervision

• Supervision

Employer-related case management checklist Task Document Team member

Contacting, communicating OB SIS 04 Case manager, social worker

Workplace analysis OB CPR 061 Physician, others

Developing a job description OB CPR 014 Employer, physician, others

Summarising health risks and limitations OB CPR 013 Physician

Managing the on-the-job training contract OB CPR 016 Case manager, employer

Developing workplace adaptation/ support plan External template Case manager, team

Lending/ installing adaptive equipment OB CPR 079 Case manager, others

Following up and support in on-the-job training OB CPR 071 Case manager

Assessment of training results External template Case manager, physician, social worker, technologist, others

Counselling and support to employer in realising the employment

OB SIS 004 Case manager, others

Counselling and support to employer at obtaining incentives for employing PWD

OB SIS 004 Case manager, social worker

Following up in employment OB SIS 004 Case manager, others

Page 34: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

34

1 note: Ir. P.H. Markenstein 2004, Jan Kaldenbach 2007

8. A philosophical approach to measuring and anticipating the

needs and expectations of stakeholders, especially the client.

—Pluryn

Excellent organisations measure and anticipate the needs and expectations of their stakeholders, monitor their experiences and perceptions. In Pluryn it is our common procedure before the start of the vocational training of the individual client, to determine • the goal aimed for; • the content of the vocational training; • the personal training that is foreseen; • the evaluations that guide and follow this developmental process; • the design and development of the personal manual ; and/or • the impact of environmental conditions i.e. things that you cannot change by training the

person.

All of these are discussed and planned in dialogue with the client and their case manager or referring organisation. When the assessment is accomplished there will be a formulation of the (hypothetical) end result of the course, including qualifications/certifications, and placement in the desired job. This end goal will be the leading objective but will not be fixed in a stringent way. The way to get at the desired end results are described in a SMART (Specific, Measurable, Achievable, Relevant and Time Bound) way and laid down in a tailor made “Individual Reintegration Plan” (IRP). This plan agrees to conditions set by the funding partner, UWV. While we use this information in order to set, implement and review our service level objectives and targets, measures and plans, for the short, medium and longer term in regard to our stakeholders we feel it is important to assure that time bound, outcome driven services do not work to the detriment of disabled or disadvantaged service users. We therefore seek to ensure that our results orientated approach is tempered by fuzzy-ness when this is in the client’s interest. An Important Leading Thread in Practice is “FUZZY-ness”

1 :

In Pluryn we believe that it is important to acknowledge that the needs and expectations and goal setting in practice will not be fulfilled only by “SMART-ness”. We need a lot of flexibility and space to deal with unexpected positive, and sometimes negative, incidents and occurrences; we need FUZZY-ness. In many cases the goals of clients cannot be clearly and exactly described in their full potency because there is a lack of knowledge of the full spectrum of possibilities that the person has. For both the client as well as the case manager there can be unforeseen possibilities as a

LEADERSHIP AND CONSTANCY OF PURPOSE INNOVATIVE PRACTICE GUIDE

RESULTS ORIENTATION INNOVATIVE PRACTICE GUIDE

Page 35: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

35

consequence of the different conditions and different settings where the client has to perform and achieve. The impact of the client who is often a young person, in a new environment, with new professionals may give rise to other perspectives. In some ways the Vocational Trainer’s role is that of a detective, you have some clues to seek and you do not want to close down ways at a pre-emptive phase . There must be space for discovery/detection/development of new fields, compensatory strategies and unexpected talents. There must also be space for disappointments and restrictions that were not foreseen at initial assessment; factors such as the progression of disease, changing preconditions in the client’s life circumstances, or in the circumstances of significant others in their life which may have an impact on their vocational training pathway. Vocational Trainers can also fall into the pitfall, or danger, of allowing a SMART approach to result in setting safe and easy to reach goals with clients; goals that assure targets that the Trainer fulfils their promise and delivers qualification standards. However these are no longer client-focused and can result in the client’s motivation being easily lost along with their ambition and they end up totally disillusioned with the process. We therefore propose the theory of FUZZY-ness as opposed to SMART-ness underpinning your results orientation in service development and delivery. It is the responsibility of the Vocational Trainer to work with the client in way that assures that the plans you have made together are ambitious and inspiring. A FUZZY goal is a : F = Feest =celebration U= Uitdaging=challenge, ambitious Z= Zuiver=authentic, genuine Z= Zinnelijk=passionate, inspiring Y=Yes= yes

EQUABENCH— BENCHMARKING EUROPEAN EXCELLENCE

Page 36: EQUABENCH—Benchmarking European Excellence€¦ · education and training ... organisations against these steps in relation to the ... realistic and achievable goal setting e.g

36

Results Orientation

Innovative Practice Guide

EQUABENCH—Benchmarking European Excellence

This project has been funded with support from the European Commission. This publication [communication] reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

The Cedar Foundation www.cedar-foundation.org

European Platform for Rehabilitation www.epr.eu

Josefsheim gGmbH www.josefsheim-bigge.de

National Learning Network www.rehab.ie

Centro Studi Opera Don Calabria www.centrodoncalabria.it

Pluryn Arbeid www.pluryn.nl

Centro de Reabilitação Profissional de Gaia www.crpg.pt

University Rehabilitation Institute Republic of Slovenia

www.ir-rs.si

The Equabench Innovative Practice Guides are available in a range of alternative,

accessible formats on request from The Cedar Foundation

Project Partners