caf works eipa

8/8/2019 Caf Works Eipa

1/101

works better service for the citizens by using CA


2/101


3/101

Foreword

5

European cooperation and work on quality ma-nagement has significantly improved over thelast few years. During Austria's first Presidency

in 1998 the aim was to take specific steps towardsimproving public service for the citizens. The idea of aEuropean Public Administration Quality Award provednot to be the right tool at that time. So the idea of aCommon Assessment Framework to provide publicadministrations with a quality management tool andgive an impetus for improvements was born. Toget-her with EIPA and the members of the European Pu-blic Administration Network (EPAN) CAF was develo-ped and finally presented at the 1st EU-Quality Confe-rence in Portugal in the year 2000. Since then, the3rd version of CAF has been finalised in the light ofthe experiences of the more than 900 CAF Users.The CAF 2006 version will be officially presented (dis-tributed) at the 4th EU Quality Conference in Tamperein September 2006.

As CAF has continued to spread all over Europe thereis increasing interest in more in-depth knowledge of theuse of CAF and the results achieved. The need for ap-propriate benchlearning partners in the different fieldsof public administrations is growing. It is now time tohave a first evaluation of whether CAF works in Europe.The overall objectives of the publication CAF works better results for the citizens by using CAF are

to bring the CAF self-assessment to life by showingspecific results and improvements related to theCAF self-assessment.

to raise awareness of CAF as an instrument forquality management throughout Europe's publicsector organisations.

to increase benchlearning between CAF users.

For the first time, information is being provided onconcrete results for citizens, people within the organi-sation and society, the way CAF has been used andimprovement actions which lead to improved resultsin the organisations.

On behalf of the Austrian Presidency I would like tothank all who have participated in this publication: themembers of the European CAF Correspondents Net-work, the public administrations which have deliveredtheir case descriptions. Also to EIPA and the AustrianKDZ-Centre for Public Administration Research whohave done an excellent job in guiding the case descrip-tions and extracting general results and conclusions.

I hope this publication offers interesting experiencesand information for CAF users who are interested inBenchlearning and that it may also encourage thosewho still hesitate to use CAF to join the Europeancommunity of CAF users.

Emmerich Bachmayer, Director-General,Public Service and Administrative Development


4/101


5/101

Content

Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Lessons Learned on the use of CAF. . . . . . . . . . . . . . . . . . . . . . . . . . . 84Publication CAF works in a nutshell. . . . . . . . . . . . . . . . . . . . . . . . . . 88Results and Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96Management Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

3


6/101

Introduction

The Common Assessment Frameworkin the context of the Lisbon strategy towardsa better-performing public sector

The European Council of Lisbon in March 2000stressed that the European Union should beco-me the most competitive and dynamic knowled-

ge-based economy in the world, capable of sustaina-ble growth with more and better job opportunities,greater social cohesion and a simplified regulatory en-vironment. The successful implementation of thisstrategy depends to a large extent on a highly compe-tent, professional and competitive public sector, cha-racterised by effective governance structures, innova-tive and effective policy-making practices and a per-formance-oriented provision of services.The Ministers responsible for Public Administration inthe European Union agreed in 2004 that more attenti-on and effort is necessary from the Member Statesand from the European institutions in order to improvePublic Administration, so that it can become an impor-tant factor in the achievement of the Lisbon objectives.At the end of the Luxembourg presidency on 8 June2005, they congratulated the working groups of theEuropean Public Administration Network EPAN for thesuccessful implementation and development of activi-ties and common tools, encouraged an innovationagenda in the fields of benchmarking and best practi-ces as well as in terms of the development of theCommon Assessment Framework (CAF) and otherquality tools and requested that this work be better in-tegrated with the Lisbon agenda. As a result, in thenew Mid-Term Programme for the period 2006-2007,they agreed to focus attention on the further develop-ment of quality management in the public sector.With respect to CAF, the Directors-General of PublicAdministration agreed at their meeting in Luxem-bourg to promote its further dissemination withinEurope. In the meeting under the UK Presidency theyset a specific target of 2,000 registered CAF applica-tions by 2010. The Directors-General welcomed theinitiative from Austria to produce a publication CAFworks Better results for the citizens by using CAFduring the Austrian Presidency.

Purpose of the book

This publication shows a selection from the current900 CAF users in different European countries thatachieved remarkable results through using the CAF.During the two European CAF Users Events in 2003in Rome and in 2005 in Luxembourg, around 300 CAFusers exchanged experiences, the obstacles they metand the lessons they learned. The Austrian presidencyconsidered the time right to share these and otherexperiences of the Common Assessment Framework with a bigger audience by means of a publication.The brochure aims to provide a representative exam-ple of the way in which CAF is used in Europe. 29 ap-plications in 15 countries have been selected. Theyfirst of all illustrate the extent to which the CAF isused as a common framework all over Europe to as-sess the quality of public administrations and to inspi-re them in their way to excellence. Common meansthat the same structure and scoring system is usedby public organisations from different countries, withdifferent socio-economic backgrounds and differentadministrative cultures. Framework indicates thatthis structure is just a set of quality principles put to-gether in a logical and coherent way, covering all theaspects of the functioning of an organisation, so as tobe confronted with the reality of a living organisationin order to make an accurate diagnosis of its health.This assessment must lead to actions on the areasidentified for improvement.In focusing on this last point, the publication goesfurther than the previous studies published by the Eu-ropean Institute of Public Administration in preparati-on of the 2 European CAF Users Events. These weresurveys of the practice of self-assessment with theCAF and its evolution since CAF had been launched inMay 2000. The publication CAF works aims to closethe circle of quality and present for the first time thetype of improvement actions the self-assessmentshave brought about and details of the way these im-provement actions have improved the results of theorganisations, especially in the field of citizen/custo-mer satisfaction. Most of the CAF applications havebeen carried out with the 2002 version of the model

8


7/1019

Introduction

and 3 years is a rather short period for a completequality experience, starting with self-assessment andending with better results. Nevertheless, the selectedcases have run through their first round of continuousimprovement in that period and should be inspiring forother public organisations still doubting on the effecti-veness of this approach. They can be considered asgood practices on the use of CAF and will be presen-ted as such in the European CAF database at EIPA.They can be an interesting source for benchlearning inthe future.

Philosophy of the book

Each successful improvement programme starts witha clear diagnosis of the current situation. CAF as aholistic diagnosis gives information on the quality ofthe managerial practices of an organisation in theareas of the enablers criteria and on the trends in theresults achieved in the results criteria. In the resultscriteria in particular, the assessment is based on theconclusions of measurements of perceptions and spe-cific indicators. Given the cross-connection betweenthe enablers and results criteria and subcriteria, theanalysis indicates which enablers should be better de-veloped in order to improve low results and what ac-tions should be undertaken in that respect. After the-se improvement actions, the specific measurementsand a new overall CAF assessment should show bet-ter results. If not, the improvement actions have notbeen effective enough and will have to be adjusted.

The publication gives examples of effective improve-ment actions in public organisations of different Euro-pean countries, based on the use of CAF. In this re-spect measurement is the key word and this publicati-on attempts to underline its importance in themanagement and improvement of public administrati-ons.

The selection procedure

The national CAF correspondents were asked to pre-sent a number of CAF applications they considered tobe suitable for this study. The CAF Resource Centre

received details of 62 organisations which had beennominated. 48 of them introduced 50 case descripti-ons on the basis of the proposed format, reflectingthe objectives of the study. A jury made up of the Au-strian Chancellery, the Austrian KDZ-Centre for PublicAdministration Research and EIPA selected 29 casesfor the brochure. They used the following criteria: theconformity of the case description with the spirit ofthe format, the inner coherence of the case descripti-on, the presentation of clear information on its re-sults and measurements, the spread of countries andsectors.

The structure of the book

The 29 case descriptions are the core of this publica-tion. They are presented by sector: Customs, Tax andFinances, Economy and Agriculture, Health, SocialServices, Education and Research, Local Administra-tion, Regional Administration and Transport and Infra-structure.The heading for each description consists of: the na-

me of the organisation, the country, the address of itswebsite, the numbers of CAF use, the key words ithas chosen and the results criteria it focuses on. Af-ter a short presentation of the mission, vision andmost important resources, the major results are des-cribed. How these results were achieved is explainedin the following paragraph: our way to improved re-sults. The CAF in practice illustrates in a few lineshow the self-assessment took place. In addition, theauthors of this book selected a key statement foreach organisation.The chapter CAF works in a nutshell provides anoverview of the 29 cases and a short case descriptionof each case. In the chapter lessons learned usingthe CAF, the authors try to summarise the com-ments made on the use of CAF itself. Many of thesecomments are a source of inspiration for the revisionof the CAF that is currently taking place.In the conclusions, the authors describe the lessonswhich readers can learn from these cases.


8/101

Customs, Tax and Finance

Tax Office Plonsk

The Office's aim is to ensure that constitutionaltax obligations are fulfilled by citizens accordingto the tax law.

Our missionBy respecting every individual's dignity and rights andby building up a trustworthy image of the Civil Servicein the public's view, we secure the budget revenues inthe most effective, efficient and unbiased way, thusenabling the Council of Ministers to achieve the eco-nomic, social and administrative goals, and fosteringthe economic development of the region at the sametime.

Our visionWe aim to be a high-quality and trustworthy instituti-on that, owing to the staff's valuable experience andqualifications, fulfils the administrative tasks and con-tinues to improve its organisation.

The Tax Office in Plonsk is one of 400 tax offices inPoland and one of 51 tax offices under the Tax Cham-ber Warsaw. The area where Tax Chamber Warsaw(TCW) operates is the district of Mazovia, located inthe centre of Poland, including Warsaw - the capital.We currently deliver services to 67,000 tax-payers inthe Plonsk district and employ 73 people. 97% of ouremployees are members of the Civil Service Corps.57% of our employees have university degrees.

Our major results

As far as customer satisfaction is concerned in 20012003, we observed a positive tendency in results of60.2%, 71.2% and 79.5%. However, the results inthe area of information availability were alarming.These were 57%, 55% and 59% and were lower thanthe assumed objective of 60%. When CAF is in use,the customer satisfaction should reach 90% level.The results of our systematic research on employeesatisfaction for 20012004 were 56%, 69%, 71%and 64%. The lack of stability of those results wasthe mean reason for adopting measures relating torecruitment.While implementing the system of quality manage-ment, we took advantage of the know-how of local go-vernment administration as well as PHARE program-me experts to work out a method of evaluating custo-mer and employee satisfaction.Citizen satisfaction is measured by means of a quanti-ty method at least twice a year. Our customers arerequested to fill in a closed question questionnaire.The total satisfaction level specified as a percentageis calculated as the arithmetical average of positivecomments in eight questions. All answers are subjectto detailed comparative analysis. We therefore recei-ve information on employees' competence and polite-ness, availability of information, organisation and effi-ciency of work and our customers' preferences.Employee satisfaction is measured twice a year. Em-

10

Tax Office PlonskCountry: PolandWebsite: www.izba-skarbowa.waw.plCAF uses: 2 xKey words: information, partnership, recruitment

Focus:citizen resultspeople results


9/101


10/101


Fund Management Institute (FMI)

Instituto de Gesto de Fundos (Fund ManagementInstitute FMI) is a public body under the aegis ofthe Ministry of Social Security. It manages the So-

cial Security Reserve Fund (EUR 6,000 million), whichis invested in financial assets issued by OECD issu-ers. It has also been entitled to manage the War Ve-terans Fund since 2003.FMI seeks to be the asset manager of the PortugueseState. Its mission is to develop an asset managementactivity designing integrated solutions that fill theneeds, the risk profile and the purposes of theirclients.FMI activity consists of fund management of State-owned funds and it has an annual budget of EUR 2.7million and 26 employees within 4 departments.Headquarters are located in Oporto.In order to be recognised by their potential clients asa qualified fund management provider, FMI wants allits personnel to be committed to the use of bestpractices in order to control the operational risk of itsprocesses. By opting for benchmarking with other in-stitutions, FMI seeks to promote an internal cultureof commitment to excellence.

Our major results

FMI used CAF for the first time in April 2004, and thevision and mission were subsequently clearly definedand communicated to all staff. A search for potentialclients (trusts of State-owned assets) was performed

and initiatives were launched to present FMI as a fundmanagement provider. Three formal proposals werecompleted and presented to the respective governingbodies. Regular reporting and operative tasks werescheduled in an intranet application, resulting in animprovement from 87% to 95%. Trading and settle-ment efficiency improved from 84% to 90% and theoverall operative index improved from 59% to 81%.

The strategic investment objectives of the Social Se-curity Reserve Fund were achieved and its perfor-mance improved: the real rate of return in the past 3years went from 2.67% to 3.99% and the 5-yearaverage return above cost of Portuguese public debtincreased from 0.35% to 0.45%.FMI applied for C2E (Committed to excellence) EFQMcertification with 3 special actions: Implementation of Information Security best practi-

ces according to ISO 17799 and ISO 27001; Documentation and owner attribution of key proces-

ses, according to ISO 9001:2000; Establishment of a strategic planning process.

Our way to improved results

Actions were undertaken in order to develop the inter-nal information system with a final purpose: "measureto manage". FMI entered a process of negotiation ofinternal objectives according to 3 different levels: or-ganisational, departmental and individual.

12

Fund Management Institute (FMI)Instituto de Gesto de Fundos (IGF)Country: PortugalWebsite: www.seg-social.ptCAF uses: 2 xKey words: plan, communicate, train, measure

Focus:key performance results


11/101


Service-level agreements between departments andbetween departments and the Board were concluded.Regular tasks were scheduled in an intranet applicati-on and are validated by internal clients. Indicators we-re defined, concluded with each department and pu-blished on the intranet. Monthly meetings with seniormanagement were scheduled for continuous monito-ring. FMI has been running an individual evaluation sy-stem since 2000. The asset management activitywas benchmarked in relation to a market compositeindex (including bonds, equities and real estate) andagainst the cost of the Portuguese public debt portfo-lio. In order to be recognised as a fund managementprovider (additional clients/funds), FMI will have aquality label based on track record, skill, best practi-ces, operational risk management and commitment tocontinuous improvement. Strategic planning, commu-nication, training, information security managementand processes documentation are keys for acquiringsuch recognition.In addition, FMI launched a 4th HR adequacy actionwith the aim of improving skills and programming thetraining courses for the near future.Communication (internal and external) was also onthe agenda and several initiatives were launched, e.g.designation of an internal communication provider,elaboration of booklets presenting FMI and its ser-vices.

The CAF in practice

Intradepartmental collection of evidence ofstrengths and improvement suggestions (involve-ment of all staff members).

Six half-day meetings: senior management presen-tation, discussion and rating of each CAF criterionbased on the previous collection of evidence andsuggestions (2 board members, 4 department ma-nagers and 1 member from the Board secretariat).

Filling in the CAF questionnaire. Definition of prioritisation criteria. Prioritisation of improvement actions according to

strategic objectives. Planning the implementation of the improvement ac-

tions.

This evaluation process has been concluded within1.5 months.The participation of al 26 staff members, the periodof collection of evidence and suggestions and enlarge-ment of the timeframe (from 1 to 1.5 months) weresuccessful elements in our 2nd CAF application.

ContactMr Henrique CruzBoard MemberE-mail: [email protected]

13

Lessons learnedWe need to formalise our procedures and KPIs, todefine and communicate our objectives, and imple-ment a system to control our objectives.


12/101

Economy and Agriculture

Estonian Animal Recording Centre

The Estonian Animal Recording Centre (ARC) is agovernment institution under the administrationof the Ministry of Agriculture. The main task of

ARC is the improvement of the efficiency of animalhusbandry by performing animal recording and inde-pendent testing of the quality of raw milk.ARC records the performance of dairy cattle, beefanimals, pigs and goats, and performs genetic evalua-tion of livestock and independent testing of the qualityof raw milk. The Milk Analysis Laboratory of AnimalRecording Centre analyses the animal recording milk samples and the raw milk samples from dairies.ARC has approximately 2,500 animal keepers asclients. The Estonian dairy industry as a whole is alsoour client. ARC sells the ear tags for cattle, goats, pigsand sheep and also issues replacement tags for lost ordamaged ones. Sixty people work for ARC. Sixteenwork in the analysis laboratory and twenty-four withclient service. ARC also has four programmers whocreate applications for internal use. ARC's 2005 bud-get is EUR 1.3 million. The Estonian Accreditation Cen-tre issued to The Milk Analysis Laboratory an accredi-tation certificate in January 1999, which certifies itscompetence for analysing milk samples in conformitywith the requirements of EVS-EN ISO/IEC 17025. As of1998, the laboratory of analyses participates twice ayear in international collaborative studies for identifica-tion of milk components. The Milk Analysis Laboratoryconducts international comparative tests with regardto all milk components identified in its laboratory using

internationally recognised methods or analysers.We used the CAF model in 2003. In 2002 and 2004we participated in the Estonian Quality Award contest(EFQM model). In 2004 ARC was the best organisati-on in the public sector.

Our major results

CitizensARC has always considered satisfying its clients' wishesto be very important. We also believe that our clientsare happy with our services. In starting to employ quali-ty models, it became evident that we do not have quan-tifiable results and have lacked clear objectives.By the time we were using the CAF model, we hadmanaged to introduce market research among cattlekeepers. According to this research, clients werehappy with ARC's services, with evaluations on a 5-point scale of:Accommodating, willing to cooperate: 4.42Trustworthy: 4.38

There were few performance indicators to present,which should have been consistently monitored anddata gathered, and moreover, the existing indicatorsdid not cover all the main services.Replacement ear markers must be supplied to clientswithin 20 working days. This objective was reached in90% of cases in 2002 and 94% of cases in 2004.ARC has created the Bossy programme for keepers

14

Estonian Animal Recording Centre (ARC)Country: EstoniaWebsite: www.jkkeskus.eeCAF uses: 1xKey words:citizens (client) feedback, people feedback

Focus:citizen results


13/101


of dairy cows. The programme has different electro-nic applications for people with Internet connectionsof differing speeds. ARC has monitored the number ofusers of the programme and the frequency of its use.

Two main approaches are used for the measurement:1. Written questionnaires. This was how ARC's 2003

survey of our customer's satisfaction with our mainservices was conducted. As data is exchanged withthe majority of clients on paper and they are loca-ted in rural areas where Internet coverage in 2003was not extensive, written questionnaires weremore practical. Feedback forms are also filled in byhand at the end of information days organised forclients.

2. Gathering and publishing service-related perfor-mance indicators through electronic means. All de-partments disclose their performance indicatorson the intranet on a monthly basis. Doing so usingthe intranet is technically quite simple: a depart-ment worker uploads an MS Excel file of the up-dated data, which features both numerical informa-tion and illustrative material.


Process and change management & develop and im-plement partnerships with customers/citizens.1. In using quality models, it became clear that there

has been little inclusion of clients and workers inthe development of services. It was found throughanalysis that the majority of development proposalsare made verbally. In order to ensure that propo-sals reach the decision-makers, the decision wastaken to create the IDEA electronic application forverbal proposals. All workers have the chance toregister their own or their client's proposals orcomplaints in this programme. The worker thensends the proposal to be answered by the head ofthe relevant department with the simple click of abutton. It was agreed that answers would be givenwithin a week and would be posted in IDEA for all tosee. Making answers public in this way has alsohelped to improve communication. It was alsoagreed who would be responsible for giving feed-

back to clients on their proposals. Our workers ha-ve taken to IDEA and the management has givenstrong encouragement for its use.

2. ARC values client feedback. In recognition of clientswho make proposals, it was decided to inauguratethe "ARC's Most Instructive Client" award. The ob-jective of the award is to give recognition to clientswho have actively participated in the improvementof ARC products and services. The award is pre-sented annually.

The CAF in practice

All workers were informed about the performance ofself-assessment. The self-assessment group compri-sed six members with one meeting leader (from theupper management level without voting rights). All di-visions and management levels were represented inthe group. The self-assessment exercise lasted fourto five days. Two members of the group had done trai-ning based on the CAF model, and they trained the re-maining group members. The three-hour training ses-sion was made up of both theoretical and practicalparts. In the practical part, assessment was introdu-ced in terms of one enabler and one result criterion.After training, the self-assessment team was givendata forming the basis for the evaluation. Four dayslater the consensus meeting was held.

ContactEneken Ulmas, Internal AuditorE-mail: [email protected]

15

PDCA, Plan-Do-Check-ActWe have started using approach reviews. Thismeans we analyse whether a specific activity is thebest way of achieving a given goal. Use of the CAFmodel over several years has led to cases where agood beginning has remained a one-off activity.


14/101


The Pirkanmaa Employment and Economic Development Centre

Mission

The mission of The Employment and EconomicDevelopment Centre (TE-Centre) of Pirkanmaais to promote entrepreneurship, competitive-

ness, growth of companies, employment and regionaldevelopment in its region.

VisionOur vision is to be an active and efficient partner withexpertise in promoting diverse and competitive eco-nomics and employment. As a result of this co-ope-ration, Pirkanmaa is an attractive location for suc-cessful business operations, based on expertise andspecialisation.

ValuesValues guiding our actions are customer-orientation,effectiveness, as well as mutual respect and co-opera-tion. We have 143 employees and our yearly budget isaround EUR 5.5 million. TE-centres were founded in1997 as the result of an assimilation of aspects of theMinistry of Trade and Industry, the Ministry of Agricul-ture and Forestry and the Ministry of Labour.

Our major results

Citizens/CustomersCustomer visits are increasing and customers are

satisfied. The service quality experienced by our cus-tomers (internal customers excluded) has developed(on the scale from 4 to 10) from 8.1 in 2004 to 8.2in 2005 and we are aiming at 8.3 in 2006. The use-fulness of the services received (on the scale from 1to 5) has remained at 4.4 in both 2004 and 2005and we aim to approach 4.5 in 2006.The level of customer visits has continued to grow inrecent years from 15,475 to 21,985 and we expectthe growth to continue to around 23,500.

PeoplePeople satisfaction has long been rated as 3.3 on thescale from 1 to 5. Changes among permanent staffhave been few, approximately 1 to 2 % per year,which we see as a clear sign of a good working envi-ronment. Sick leave has declined by 19% from 10.37to 8.4 days per person per year and longer periodsof absence or sick leave have declined. Participationin training has grown by 55 % in a year (from 2.8 to4.2 days per person per year).

Society resultsContacts with city officials and members of parlia-ment are strengthening. On average, the PirkanmaaTE-centre is better known and has a better imagethan other TE-centres. Good results are obtained re-garding environmental issues, e.g. saving paper as aresult of moving to electric service models, etc.

16

The Pirkanmaa Employment andEconomic Development CentreCountry: FinlandWebsite: www.te-keskus.fi/pirkanmaaCAF uses: 1 xKey words: prioritised, immediate,participative, followed-up

Focus:citizen results, people results, society resultskey performance results


15/101


Key performance resultsGoals have been met which led to improved results ininspections reports. We have managed to make around6% savings on our yearly budget.

MethodologyInformation on citizens/customers satisfaction is col-lected by a yearly electronic survey conducted togetherwith the other 14 TE-centres in order to facilitate a sa-tisfactory comparison. By following up customer visits,we anticipate future customer needs. An electronicsurvey of all staff carried out with the other 14 TE-cen-tres provides information on employee satisfaction.Other information is collected by monitoring sick leave,staff changes, participation in training, etc.Society results are registered by a survey of the fami-liarity with and image of TE-centre among non-custo-mers. Other indicators are based on follow-up informa-tion. Key performance results are based on productivityinformation, the follow-up to the organisation's yearlygoals (based on strategy), how these goals have beenmet, and the budget follow-up.


By good strategic planning, yearly planning and antici-pation, we try to arrive at better customer results.People results are strongly influenced by developed andsystematic models of leadership and leadership trai-ning, as well as proper people management, e.g. byclear rules of the game, free-time activities and a se-rious approach to well-being at work, participation pos-sibilities, information exchange systems, etc. Well-ma-naged partnerships and resources create good societyresults. Actions on leadership, strategic planning, part-nerships and resources management, processes andchange management affect the key results. In our self-evaluation, one of the most important and most neededimprovement actions was how to develop our social ca-pital, the knowledge of how to meet the needs of our fu-ture organisation more effectively in the light of sub-stantial changes to its structure in the coming years.We agreed to analyse our competences by defining thefuture organisation and the knowledge it requires andsetting goals for the future. On the basis of the consul-

tative analysis, we decided to draw up a strategic planon how to develop both our organisation and the com-petences of our employees. We have received fundingfor the two-year project and are eager to get started.

The CAF in practice

We started by updating our CAF description. Our pre-vious description was an EFQM, so the rewriting pro-cess was also very useful. The CAF description was deli-vered to all staff and presented to everyone at what istermed a morning coffee training. As for the evaluation,we organised a one-day workshop. The group consistedof 38 persons. We had invited everyone who was inter-ested. We also gathered a group that represented theentire organisation: the board, all people in senior posi-tions and employees from all different occupationalgroups. The method of working was the same foreveryone so that position was unimportant. We startedby presenting the model once again and splitting intogroups to discuss the criteria: defining strengths andweaknesses and suggesting improvement actions andscores. After the group work, each group presentedtheir opinions to the others and explained why they hadgiven certain scores. After each presentation, the entiregroup voted using a Global Response electric voting ma-chine. Consequently, the opinions were immediately visi-ble and we had the possibility to discuss and reach con-sensus on the final scores. After going through each cri-terion, we came up with a list of strengths, weaknessesand scores, as well as a list of suggestions. In the end,the quality group and the board went through the resultand improvement actions again and agreed on responsi-bilities, timetables, etc. We have also informed staff ofhow actions are being implemented.

ContactMs Anna-Kaisa Ikonen, Development ManagerE-mail: [email protected]

17

Lessons learnedMoving from EFQM to CAF is worthwhile when itcomes to participation.


16/101


The Regional Directorates for Industry, Researchand the Environment (DRIRE) perform a numberof activities which come under the authority of

the French government (Ministry of Industry) as wellas activities to stimulate business.The DRIREs are responsible for ensuring that regula-tions are properly applied by industry with regard tothe environment (waste and exploitation of resour-ces), nuclear safety, industrial safety and the safetyof vehicles. To this end, they implement inspectionprogrammes on the premises of all the industrial en-terprises in Aquitaine while also drawing up technicalor regulatory specifications (operating licences).DRIRE Aquitaine currently has 160 inspectors, theoverwhelming majority of them technicians.

Our major resultsThe results are measured both by the CAF self-eva-luation and by an internal barometer specific to DRI-RE Aquitaine. This barometer is a periodic question-naire that evaluates employee satisfaction but also

their perception of quality policy and is an opportunityfor employees to make suggestions.A permanent sug-gestion box system is in place, allowing employees tobe the driving force in the improvement process.An annual interview consisting of a skills appraisal foreach inspector completes the system. It is based ona benchmark established for each occupation. The ac-tivities subject to quality assurance are carried outsolely by qualified or authorised staff. An appraisal foreach manager is conducted by the director on the ba-sis of an internal survey. This appraisal assists notonly in evaluating the managers but also, more impor-tantly, in fine-tuning the management of DRIRE Aqui-taine. Developments are followed up annually.


An effort has been made to structure our organisati-on, our management and our approach (feedback andsatisfaction) to customers. Improvements have beenachieved by implementing a system of managementby quality based on compliance with standard ISO9001 and approved in 2004 through the renewal ofour ISO 9001 certification. Since the first self-evalua-tion, the management has provided a certain numberof internal communication tools (internal letters, pre-sentation meetings for new projects, intranet, etc.).Actions to obtain staff feedback have been increased.Various tools have been put in place for obtaining cus-tomer feedback and measuring customer satisfaction.Suggestions for improvement and for rectifying any

18

DRIRE Aquitaine

DRIRE AquitaineCountry: FranceWebsite: www.aquitaine.drire.gouv.frCAF uses: 4 xKey words: Skill (of inspectors),Preparation (for actions); Control; Evaluation

Focus:Citizen/Customer results, People resultsKey Performance Results

CRITERION Score Score Score Score2001* 2002 2003 2005

1. Leadership 2 1.9 1.7 3

2. Policy and strategy 1.6 1.7 2 2.9

3. Human resource management 1.5 1.3 1.8 2.7

4. Partnerships and resources 2.1 1.8 2.7 3

5. Process and change management 1.4 1.8 2.1 3.6

6. Citizen/customer results 0.4 0.8 0.9 2.5

7. People results 0.4 1.2 1.4 2.5

8. Society results 1.1 1.4 1.4 1.5

9. Key performance results 0.9 1.8 2.3 2.3


17/101


weaknesses identified have been implemented syste-matically. A tool for evaluating managers has beenprovided on the basis of compliance with the main re-quirements of standard ISO 9001 and the CAF crite-ria. Each activity is matched to quality criteria, check points and actions for obtaining customer feedback. Aseries of indicators followed by front-line managers orby the management committee is used to measurethe results obtained at all levels.Another route to improvement developed out of CAFexercises has been to adopt a process system. TheCAF enabled us to identify the need to improve opera-tional communication and adopt collective responsibi-lity in devising products. Collective preparation of each activity Checking the balance between what is being done

and what is at stake Simplifying procedures by improving the skills of

the inspectors Standardising products by consolidating feedback

from tests Single final inspection of products, quality control

having been carried out during the preparation andelaboration phase.

In this way, because the CAF is based on internal reflec-tion, it has been possible to identify malfunctions duringthe product elaboration phase, non-conformities in de-sign and difficulties associated with the organisation orlack of training and deal with them more effectively. Theemphasis has been placed on training rather than onprocedures and controls in order to provide a satisfacto-ry standard of quality without making the procedurescumbersome. The solutions that have been implementedderive directly from the analysis made by the inspectorsand assist in resolving the difficulties they encounter.

The CAF in practice

The self-evaluation leaderEach year the self-evaluation is led by one of the can-didate self-evaluators selected by the managementcommittee. It is necessary to choose a leader who isrecognised for his ability to see the whole picture andhis neutrality. He is preferably chosen from outsidethe management committee.

The self-evaluatorsThe self-evaluation is carried out by a team of self-eva-luators chosen by the leader from among staff volun-teers. They should ideally be between 3 and 9 in number.The preparationThis takes half a day and is led by the CAF consultant.Each criterion is explained to the self-evaluators untila good general level of understanding and coherencehas been attained.The self-evaluationEach self-evaluator is given about fifteen days to com-plete his form. The self-evaluator must evaluate theDRIRE according to his knowledge of it and not confi-ne himself to his own unit. The self-evaluator worksalone and does not interview the inspectors. The self-evaluation work takes between 8 and 10 hours.The report and the feedback sessionThe self-evaluation report is prepared by the leaderand handed to the consultant at least 48 hours beforethe day-long feedback meeting. During this meetingthe consultant reviews the signs of progress identi-fied, the objective being to obtain a consensus fromthe self-evaluators' assessments.The follow-upWithin about two weeks the leader asks the self-eva-luators to specify pathways towards achieving pro-gress and to propose lines of action and then to clas-sify these progress pathways according to the follow-ing priority criteria: External impact / Internal impact / Work unit / Cost. On the basis of these elements, theleader prepares the final self-evaluation report. Thisreport is submitted to the management committee.

ContactLaurent Balahy, Quality ManagerE-mail: [email protected]

19

Lessons learnedReaders of the publication CAF works who are in-volved in an ISO 9001 quality certification processshould be aware that the CAF is an effective tool fordetecting potential improvements which would notcome to light as quickly without it. It is a necessarycomplement to a standardised quality process.


18/101

Education and research

Mission

The main tasks of the Austrian Study Grant Aut-hority are the handling of applications for studygrants, grants for study abroad and grants upon

the completion of studies as well as the provision ofinformation and advice.

VisionDuring the past years, the Austrian Study Grant Aut-hority has become a service-orientated organisation.The main fields of development have been customerfocus, focus on people, the quality management sy-stem, the certification by the international standardISO 9001, personnel and organisational development,cost and activity accounting, controlling, support of135 junior executives, projects to improve the coreprocesses and self assessments. These processeshave been carried out by engagement and motivationof the people in the organisation. All development pro-cesses have been aimed at satisfaction of customers

and employees and efficient allocation of the substan-tial public funds.The Austrian Study Grant Authority fulfils itstasks at 6 locations with a total of 132 emp-loyees. The total amount of direct support gi-ven in 2004/05 was about EUR 187 million.The internal administration costs were aboutEUR 6.5 million.

Our major results

The Study Grant Authority has had a long tradition ofasking students for their opinion. The first survey wasconducted in the early nineties. Since the quality ma-nagement system was implemented, the surveys havebeen conducted on a regular basis and more syste-matically. In the early stages it consisted of a four-pa-ge questionnaire that has now been replaced by anelectronic questionnaire on the website. In addition, atelephone survey consisting of standardised questi-ons has been conducted. The customer satisfactionhas increased with respect to the following: shorte-ning of processing time, simplification of the processand improvement of consultation.Handling applications for a study grant is the StudyGrant Authority's core process. Approximately 50%of the total working time is used to perform this task.More than 63,500 applications were handled in thepast academic year. The following criteria are definedas measuring figures: date of application, date of

completeness, date of transaction, date of approbati-

20

The Austrian Study Grant Authority

StudienbeihilfenbehrdeCountry: AustriaWebsite: www.stipendium.at/stbh/behoerde.htmlCAF uses: 3 xKey words: customer orientation, e-government,optimise processes, advice

Focus:citizen results, key performance results

18.11.2004 18.11.2005

Application for a study grant 36,440 44,898

Handled applications 23,631 32,259

Degree of handling 65% 72%

Degree of handling within 3 weeks of the application 42 % 69%

Processing time 2001/02 2004/05

Processing time in days 38 30

Transaction within 3 weeks of the day of the application 32% 42%


19/101


on and date of decision-making (notice). The followingfactors impact the duration of the process: complexi-ty of the Austrian Student Support Act 1992, com-prehensibility of the applications forms, necessity tosubmit different records and confirmations, internalworkflow, IT-support, employees and applicants. Long-term measures have been introduced to optimise thecore process of implementing the applications. Thefirst figures in the current winter term 2005/2006show the successful implementation of these measu-res. Despite the higher number of applications, thedegree of handling is significantly higher.


e-scholarship projectThe customer surveys have been used as a basis todevelop the e-scholarship project. The aim of thisproject is to transform the study grant procedure intoa full electronic procedure in terms of e-government.1. Expansion of data exchange regarding study suc-

cess with educational institutions (universities andteacher training colleges), family status regardingsocial security, family allowance and additional in-come data.

2. Technical possibility to submit an application via In-ternet.

3. Repeated application: one application is submittedat the beginning of the academic education and willbe automatically followed up by verifying the infor-mation regarding the preconditions for receiving astudy grant (income, success in study, family sta-tus) electronically provided.

The first electronic application for a study grant isnow manageable via www.help.gv.at. At present, the

Brgerkarte (citizen card) and the A1 signature ser-ve as a digital signature. The implementation of thedata exchange with other institutions apart fromthe data concerning the success in study has beencompleted and therefore no other documents need tobe submitted. The third step has been in place sincethe end of 2005.

Consultation projectBased on the requirements of customers as revealed

by the surveys, the need to intensify consultancy hasbeen highlighted. The new e-government applicationmakes it possible to enhance the consultancy pro-cess. The aim of the project is to improve consultati-on, to develop quality criteria and to enhance em-ployees' qualification. As a first step, 16 employeesqualified as special consultants, so that starting inthe summer term 2006, improved consultation forthe students can be offered.

The CAF in practice

In 1995, the Austrian Study Grant Authority startedto develop a quality management system. It has recei-ved ISO 9001 certification. The knowledge requiredfor a self-assessment process has been in place since2002, so it was possible to conduct an initial self-as-sessment using the CAF. The expanded quality teamperformed the self-assessment. The Austrian StudyGrant Authority has been involved in quality issues formany years. Quality has been a focus of interest formore than 10 years, this is reflected in the quality sy-stem, ISO 9001, the Speyer Quality Award and since2002, the CAF. Experience has shown that there is anincreasing demand for quality. We observe that themembers of the quality team are becoming more andmore demanding. They take a critical view of the per-formance, which is reflected in the evaluation of evi-dence, and in assigning scores.

ContactMrs. Ursula FehlingerHead of Controlling and Budget DepartmentE-mail: [email protected]

21

Lessons learnedThe main challenge for the assessment processwas time. Increasing duties and many routine acti-vities make it difficult to find time for apparently

non-essential work. However, it also offers an op-portunity to view the organisation externally. Usingthe CAF helps to add new impetus to quality issu-es, to increase sensitivity to quality, to refocus thewhole organisation on quality.


20/101


The Engineering College of Aarhus educates Ba-chelors of Engineering and has an approach gea-red particularly to the profession. The College

has close ties to regional industry and emphasisescollaboration with regard to development, engineeringtraining and student project work. It has 1,500 stu-dents, 150 employees and had in 2005 a turnover ofDKK 100 million (Euro 22.5 million). The College en-joys close collaboration with The University of Aarhus,which offers a number of Master of Engineering de-grees. The Engineering College of Aarhus has enteredinto a preliminary agreement with four other collegesof education with the aim of creating a new educationcentre (10,000 students) at undergraduate level. TheEngineering College has established a matrix organi-sation to strengthen professional development, to fa-cilitate co-operation with external industrial partnersmore effectively and allow for flexibility in educationaldevelopment.

MissionThe Engineering College of Aarhus educates profes-sional engineers heading for a job in industry and de-velops new advanced technologies within a framework of industry-related laboratories in order to improvestudies and to support regional industry's competiti-veness.

VisionIn 2010, the Engineering College of Aarhus will beamong the five leading institutions of undergraduateengineering education in Europe and Danish industry'spreferred partner within the realm of industrial deve-lopment and the associated development of enginee-ring education. The quality development work is mana-ged by a coordinating group (hereafter Task Force)with eight representatives from management, lectu-rers, administrative staff and two students.

Our major results

In spring 2004, the Engineering College of Aarhus hadno formal procedures for introducing new staff to thetask of professional teaching. In the autumn of 2004, aguide and a procedure were drawn up. The effects ofthe new procedure were measured in the autumn of2005. All recently employed lecturers assembled at anevaluation meeting discussing the procedures for im-plementing the resulting guide. Satisfaction regardingthe initiatives was expressed and suggestions for im-provements were incorporated in a revised guide.In the autumn of 2004, only 30% of the employeeshad received their personal development dialogue wit-hin the last two years. Another 30% never receivedthis dialogue. In spring 2005, however, all employeeshad received their personal development dialogue.Subsequently, overall satisfaction with form and re-sults were expressed. To measure the quality of a fol-

22

Engineering College of Aarhus

Engineering College of AarhusCountry: Denmark Website: www.iha.dk CAF uses: 1 xKey words: employee satisfaction,quality assessment, efficiency, professionalism

Focus:people resultskey performance results


21/101


23

low-up relating to the conclusions, a questionnaire willbe carried out in February 2006, also including que-stions concerning satisfaction with the completion ofthe personal developmental dialogues.In spring 2004, the College did not use informationtechnology at in-house meetings. In November 2004,approx. 60% of employees were using the joint calen-dar system. In the autumn of 2005, the meeting plan-ner system was being used by all secretaries and lec-turers.In spring 2005, an external consultancy (professionalbusiness psychologists) carried out an evaluation ofemployees' satisfaction and their level of personal andprofessional resources. Following this evaluation,measurements and improvements were implemented.In 2006, the improvements will be assessed by a newexternal evaluation of employees' satisfaction.Measurements are generally carried out among fairlylarge groups of staff and students by means of elec-tronic questionnaires, which enable respondents tochoose from a number of statements. We take greatcare not to use this facility excessively, as employeesmust be able to maintain a normal working schedule.We are also aware that, when asking about and mea-suring attitudes, a follow-up and visualisation is re-quested. Furthermore, personal interviews are car-ried out face-to-face or by telephone. Finally, conclu-ding meetings are held in fairly small groups allowingfor close discussions and effective dissemination.All measurements are designed to measure and isola-te key results. In the case of particular improvementsof work conditions not directly measurable, comple-menting questionnaires and interviews are carried outdepending on the field of improvement. Until now, va-rious measurements do not always indicate if key re-sults are obtained. If this is the case, we need topractice before we are able to locate correct indica-tors and consequently approach objectives.

KVIK(Quality Tool for Developing Innovation andCompetencies) is the Danish version of CAF. It issupported by an electronic tool developed by theSKK, the Centre for Development of Human Re-sources and Quality Management


Using the KVIK (CAF) self-evaluation, we have identi-fied approx. 80 tasks where action is needed. A neworganisation with new leaders and managers was im-plemented in 2004/2005. For different reasons wehad to focus on fundamental problems relating topeople. The majority of improvements will most likelyprovide a positive spin-off in strengthening the stu-dents' potential and are therefore relevant for othercriteria. The following issues were analysed in workinggroups consisting of staff (and students in somegroups) in 2004 and 2005. The working groups referto the Task Force.

Human resources management

Staff policyA policy geared to senior staff older than 58 has beendeveloped and implemented. The working group isnow concentrating on the subject workplaces athome.

Guide for engaging new lecturers(associated professors)The College is engaged in a major renewal of its staff.New lecturers will have to be employed in the years tocome, and this requires a solid process to secure theinduction of new staff. Guidelines have been workedout and tested on recently employed lecturers and in-terviews have been carried out.

Dialogues with staff on their personal development(MUS)Annual MUS is a procedure embedded in Danish legis-lation. A procedure for the Engineering College of Aar-hus has been developed and the first dialogues with allstaff members were carried out in the spring of 2005.The effects will be measured in February 2006.

Processes

ITA very thorough survey has measured employee per-ception of the nature and extent of IT use and has al-


22/101


so pointed to a range of proposals for improvement.The introduction and use of shared calendars for all150 members of staff was organised in this workinggroup. Subjects have been prioritised and the groupwill continue its work within key areas of lecturing inspring 2006.

New OrganisationIn less than three months in 2005, a working groupdrew up a proposal for a new organisation in order tostrengthen the professional engineering qualities ofstaff, its ability to cooperate with external industrialpartners and the dynamics in developing new studiesand/or specialisations of existing study areas. The ef-fect of the new organisation will be measured duringthe coming year.

Evaluation of education proceduresA working group of lecturers and students are pre-sently drawing up new guidelines for the current eva-luation of all educational activities. Once the guideli-nes are ready, they will be implemented, and after acertain period the effects will be measured and eva-luated.

Facilities for educationThe working group will focus on the furnishing andsetting up of class rooms, etc. and propose measure-ments for improvement of these facilities.

The CAF in practice

The KVIK has been used during our first self-evaluati-on process in 2004. It was developed to enable ma-nagers and staff to jointly evaluate the organisation.We used it as the initial step to identify actions to betaken.The Task Force chose KVIK partly because it is easyto use and partly because of the convenient electronictool. The self-evaluation was carried out among the10 members of the Task Force who fully represent themake-up of staff. The self-evaluation consisted of anindividual preparation and a full-time consensus mee-ting where strengths and weaknesses where discus-sed in two groups. At the meeting, four topics were

prioritised and a plan of action with clear responsibili-ty for the future work was made. Mission, vision andstrategies were also thoroughly discussed in order toestablish the foundation for the working groups. Theresults were presented to the organisation at an in-formation meeting and all employees were encoura-ged to participate in the groups.

ContactOve PoulsenRectorEmail: [email protected]

24

Lessons learnedIn retrospect, it was right to carry out the self-eva-luation with the ten members of the Task Force. Aself-evaluation including all employees would havebeen an enormous task due to the handling of ans-wers and prioritising key areas.


23/10125


24/101


The main goal of Bygholm Agricultural College isto offer education, supplementary training andconsultancy related to the agricultural sector.

Our mission is to educate future farmers to be inde-pendent with high professional standards and respon-sibility concerning production of quality foodstuffs,animal welfare and environmental management. Ourvision is to be the preferred agricultural College inDenmark with a high level of activity and high stan-dards within farming and management.Bygholm Agricultural College employs 40 people (pro-fessors, administrative workers, boarding school per-sonnel and school farm personnel).150 full-time students attend Bygholm AgriculturalCollege every year.

Our major results

CitizensThe main results obtained relate to student satisfacti-on measurements and rates.Before the CAF self-evaluation, student satisfactionmeasurements were not systematically linked to thestrategic objectives of the organisation and the re-sults of the satisfaction measurements were not regi-stered systematically or published.

Selected resultsOne strategic object is that 80% of the students pre-ferred the school because of its good reputation orthe school was recommended by former students.The result of the first student satisfaction measure-ment was 90% and the second measurement was98.5%.Another strategic object is that all students considerthe lessons to be of a high professional standard. Ona scale from 1 to 5, the goal is a student satisfactionrate of 4. The results of the measurements duringthe period of CAF were 4.1, 4.3, 4.0 and 4.0 in fourdifferent classes.Student satisfaction rates are measured via a questi-onnaire. Three persons from the self-evaluation groupparticipated in a learning seminar at SCKK. This se-minar provided these persons with knowledge aboutmeasurements: what can be measured, what are thepitfalls of measuring via questionnaires and what can-not be measured?As a consequence of the seminar, we have developednew questionnaires where the students give their opi-nion on a scale of 1 to 5. The questions are designedtaking into account the strategy and the BalancedScore Card (BSC) based on this strategy. The studentsatisfaction rates are calculated and compared toprevious rates. The student reply rate to the questi-onnaire is 95100%. The result is presented on theschool website. In our view, the student satisfactionmeasurements are useful and have become more

26

Bygholm Agricultural College

Bygholm LandbrugsskoleCountry: Denmark Website: www.bygholm.dk CAF uses: 1 xKey words: formulate strategy,implementation BSC

Focus:citizen results


25/101


27

successful as a tool to work on improvements as wehave designed questions in relation to the strategyand our BSC.


Strategy and PlanningThe selected results in criteria 6 are linked to criteria2. With respect to the organisation, we have beenworking with developing tools to implement the stra-tegy (criteria 2) and developing new methods to mea-sure whether this implementation is resulting in im-provements on critical issues (criteria 6).After completing the self-evaluation and writing theconsensus paper, the self-evaluation group formulateda strategy for the entire organisation. The group helda one-day workshop with all employees at the school.The overall strategy was presented and the workshopparticipants formulated proposals for objectives andconcrete actions that would implement the strategy.The self-evaluation group then collected the propo-sals, prioritised them, and drew up a balanced score-card with measurements and targets.The BSC is now one of the main tools to focus on theimprovement actions in the organisation. Once a year,the professorial staff draw up new targets, definenew initiatives, and decide how to measure whetherthe actions result in the target.We have succeeded in implementing BSC. The schoolmanagement is using the BSC when prioritising re-sources between initiatives, and professorial teamsuse it to define which actions to take, e.g. to improvetraining programmes, lectures or questionnaires.

The CAF in practice

A group of six people were selected by the schoolprincipal. However, the principal gave professors theoption to volunteer for the group, and one professoractually did so. The group consisted of the principal,the head of education, two professors selected to berepresentative to the group of professors (one experi-enced and one new), the professor who volunteeredand one person from the non-teaching staff (head ofthe boarding school). The group members answered

the self-evaluation-questionnaire individually and thenparticipated in a two-day workshop. At the workshop,SCKK consultants supported the group through theprocess of obtaining consensus on the organisation'sstrengths and areas of improvement. The group esta-blished the order of priorities and decided to focusthe improvement action on criteria 2 and 6. Thegroup members then participated in a three-day lear-ning seminar on strategy and a three-day learning se-minar on evaluation and measuring methods.

ContactLotte IpsenHead of educationE-mail: [email protected]

Lessons learnedCAF can help an organisation to work systematically.


26/101


ITC Romanazzi is a technical high school withlanguage, accounting and computer courses rela-ted to business and tourism. Furthermore, there

is a specific business-oriented computer course foradults. New technologies are used in educational acti-vities supported by the Multimedia MultifunctionalService Centre Dilos Centre run by a group ofteachers from the school. The Dilos Centre Team(DCT) provides projects management, partnershipsand collaborations with local and European schools,agencies, companies, and the diffusion of ICT. Themission of the Institute is the growth of the culturalbasis and know-how of the students, the developmentof their abilities and knowledge particularly concer-ning ICT and foreign languages through integratedtraining activities. The number of students attendingis increasing (840 in 2003, 920 in 2004 and 940 in2005) due to the innovative didactic methods andschool-job training based on the use of ICT.

The Institute has about 120 teachers and 25 em-ployees (administration and technical assistants). Thebudget granted by the Italian National Agency for Edu-cation (M.I.U.R), is supplemented by financial supportfor specific projects from local and national agencies,the European Commission (Leonardo Programme, So-crates, etc.) and others. The DCT plans and managesthese projects, encouraging innovation and best prac-

tices. The DCT suggested the application of CAF tothe School Principal and carried out the entire pro-cess (from information distribution actions until theself-assessment Report and the Improvement Plan).As a result of the activities geared towards innovationand quality promoted and supported by the DCT -the School has been awarded prizes from Nationaland European Agencies.

Our major results

The most relevant results are connected to the im-proved use of information technology and didactic in-novation. There has been a general increase in the le-vel of customer satisfaction, particularly among stu-dents, parents, teachers and employees involved ininnovative actions and projects. The customer satis-faction score targeted at incoming students in No-vember 2005 (55.52%) is better than the score ofthe same survey conducted in January 2005 for a si-milar target group (40.94%). The areas of satisfacti-on included the use of ICT in didactics, the availabilityof facilities such as opening and closing hours, the in-ternal relationships, the integration in the school envi-ronment, the openness to change.Stakeholder satisfaction has been measured by usingsurveys together with complementary tools like focusgroups, ad hoc questionnaire and complaint boxes.

28

The Institute D. Romanazzi of Bari

Istituto Tecnico CommercialeDomenico Romanazzi BariCountry: ItalyWebsite: www.itcromanazzi.itCAF uses: 1 xKey words: innovation, communication, integration,new technologies

Focus:citizen results, people results, society resultskey performance results


27/101


29

The students have been involved in planning question-naires (suggesting specific items), data collecting,data input, statistical analysis of survey results, usingICT. The indicators concerned the main aspects re-garding the students (drop-out, success and interestin school activities) and their parents (image of theschool, satisfaction with education of students andcurriculum, relationships with teachers, employeesand leadership, etc). Furthermore, an increasing in-volvement of teachers and employees in focus groupsand decision-making processes has been an impor-tant step in increasing the sense of belonging to theorganisation and achieving a better working atmo-sphere (criterion 7) resulting from the increasedpeople satisfaction scores.

Transparency of information and evidence of improveduse of information technology in internal/externalcommunication and networking has led to a betterimage of the organisation. Different awards were re-ceived such as the Best Practice LAN, Netdays2004 best Italian product of a school, Best practicein integrated innovative training methods and The in-novative activities of the Service Centre Dilos. Part-nerships and interactions with local and internationalschools are growing, e.g. by membership of ENIS (Eu-ropean Network of Innovative Schools).The key performances results show improved qualityof service and products, better cost-effectiveness, in-creased financial grants from external agencies andpositive inspection and audit results. The measure-ment of these results has involved Administration em-ployees and has been developed by queries to theschool database.

Our way to improved better results

Although applied only once, in 2005, the use of theCAF has been a very important tool in planning impro-vement actions and activities for the coming year. Theschool actually has a formal document (POF: Pianodell'Offerta Formativa - 2005-2006) that is the fol-low-up of the CAF application and is based on the Im-provement Plan. The action plan has been approved inSeptember 2005 by the Teachers Board and the

School Executive Council and in October 2005, a ge-neral meeting was held for the distribution of indivi-dual tasks among the people involved.

The start-up of the projects included in the Improve-ment Plan, concerning the gathering of informationabout stakeholders' expectations and needs and thedevelopment of internal channels of information (crite-rion 2), has already produced an increasing participa-tion in innovative activities and motivation of teachers(criterion 7) and students, as well as a general satis-faction of parents (criterion 6). At the same time, thepromotion of open communication, considering sugge-stions and ideas from personnel, and their involve-ment in the process of updating the School Plan of ac-tivities (criterion 3) led to the same results (criterion7), as well as the implementation and diffusion of theuse of technologies in all sectors of activities (criteri-on 4).

The Improvement Plan includes three main Projects:Comunicazione, Successo Scolastico and Organizza-zione.

The first project aims to involve all relevant stakehol-ders in order to consolidate the image of the organi-sation and its educational mission, translating theminto strategic and operational actions whose goal isto increase the number of enrolments, improving atthe same time the quality of the teaching on offer. To-gether with the second project, it addresses the de-velopment of fields of activities such as teaching me-thodology and technological innovation. The evidenceof success will be in the improved use of ICT in tea-ching and classroom activities, encouraging initiativeswhich will arouse a new interest with respect to theorganisation and better results in student involve-ment and interest for learning and training.The third, but no less important project, involves allrelevant actions of planning and management thatsupport the development of ordinary and extraordina-ry activities of the School. The party responsible forthis project is, of course, the Principal, but she hasthe support of a large number of teachers responsiblefor the various sectors of activities.


28/101


The CAF in practice

The DCT - after taking part in a national event for thediffusion of CAF at the end of 2004 suggested itsapplication to the leadership of ITC Romanazzi. Thesuggestion was accepted and approved by the Tea-chers Board and the School Executive Council. TheD.C. team was authorised, as the assessment group,to apply the Model to the whole organisation. Tutoredby Formez experts, the CAF team started work in Fe-bruary 2005. The work involved teachers and em-ployees of the school in briefings and focus groups ai-ming at the diffusion of the Model. Moreover, theteam started data detection and surveys on customersatisfaction involving in these activities the relevantinternal stakeholders. Following the suggestions ofFormez experts, the team organised results and datafor benchmarking with other schools/organisations.In July 2005, at the end of the school year, a firstdraft assessment report was completed, followed bya definitive report, in September, with the analyticdescription of results, strengths and improvementareas. The consensus on CAF application by relevantstakeholders of the organisation was an importantstep in leading the Assessment Group to plan the ac-

tions and activities for the School Annual Plan (P.O.F.:Piano dell'Offerta Formativa). In addition to the nume-rical results of the CAF, the main goal achieved by itsapplication has been specific evidence of efforts madeand the launch of a process of continuous improve-ment. The next step is the application submission forthe 2006 Quality Prize of the Italian Department forPublic Administration.

ContactDilos Center Team:Francesco Maiorano, Germana Berardi, ElisabettaDalfino, Anna Maria Papapicco.Cecilia Pirolo, PrincipalE-mail: [email protected]

30

Lessons learnedThe CAF application has highlighted the opportunityto consider positive or critical aspects of the orga-nisation which are normally underestimated or notdiscussed, and, above all, the measurable connec-tion between actions and effects through the enab-lers results.


29/10131


30/101


Escuela Infantil Municipal Hello Rochapea is aNursery School (0-3 years) under the auspicesof the City Council of Pamplona in the Autono-mous Community of Navarre. It offers a unique Eng-lish-Spanish bilingual experience. The City CouncilEducation Department has contracted AVANVIDA forthe management. A specific autonomous institution(OAEIM) is responsible for the coordination of the en-tire public service called Escuelas Infantiles Municipa-les de Pamplona.Ourmission is to contribute and ensure an appropria-te development of the child through care and educati-on in direct collaboration with the families in order toimprove the quality of life and social community cohe-sion.Ourvision is to be a reference model in bilingual edu-cation in our community with a team committed toprinciples and values ensuring a specific and motivat-ed development of the personal and social capacitiesof the children in terms of well-being and happiness.Our values are: commitment, guidance, team work,change, improvement, and success.Escuela Infantil Municipal Hello Rochapea employs 19people (teachers, nurses, cleaning staff and cook) andcares for 123 children.

Our major results

The measures adopted led to very positive results inevery parameter where improvement actions were ta-ken.

A few citizens/customers results: 97% of complaints were resolved in less than 15

days; 95% of citizens continued to use the school throug-

hout the year; 100% of families rated satisfaction > 7 (average =

8.4 out of 10).

It was the school most in demand in the region with80 applications above the average number. The staffincreased their participation in organisational tasksby 65%, and their satisfaction rate increased from6.8 to 7.6.

As a key performance result, we refer to the 7.3 sa-tisfaction rate of the City Council and the EuropeanBronze Medal EFQM award for the management ofthe school.

Satisfaction rates were measured and evaluated ex-ternally in order to obtain the recognition of the regio-nal government by the Management Club of Excellen-cy EFQM.

32

Municipal Nursery School Hello Rochapea

Escuela Infantil Municipal Hello Rochapea(AVANVIDA-Ayuntamiento de Pamplona)Country: SpainWebsite: www.avanvida.netCAF uses: 1 xKey words: prospects, participation, confidence,fidelity

Focus:citizen results, people results,key performance


31/101


32/101


1. OrganisationAppointing a CAF Coordinator, consulting with ma-nagement regarding people, resources, objectives,terms, communicating the SA project to all, for-ming an SA group.

2. TrainingProviding training relating to the CAF model andthe SA.

3. Self-assessment Completing an individual SA sheet: focus, evidence,

strengths, improvement areas, scoring._ Coordinating the SA group, identifying agreement

and disagreement points and discussing evidencewhen differences occur (e 2 points), reaching con-sensus.

4. Improvement Plan Prioritising ten main improvement areas with ma-

nagement, defining action plan with management,EFQM bronze prize actions, action plan.

Communicating the SA report, presenting resultsto line management and staff, preparing reportsfor award, presentation of the bronze prize.

Implementing the action plan, implementing impro-vement actions by process heads (with participati-on of SA team.)

Evaluating the action plan, checking periodicallythat agreed actions are will eventually be success-ful, performing a periodic SA (every 1 or 2 years).

5. BenchmarkingImplementing benchmarking actions, registering atthe CAF database, possibly exchanging practiceswith Spanish and European organisations via theCAF database. Frame used: CAF database.

ContactMs Emma Labayen BerazaPrincipal Nursery School AVANVIDA Hello RochapeaE-mail: [email protected]

34

Lessons learnedIt is important to relate improvements to resultsto avoid dispersion of effort and to sustain ma-nagement support.


33/10135


34/101


This case is not based on a single organisation,but rather a group of organisations. The umbrel-la organisation, the Norwegian Council of Music

and Art Schools represents approximately 95% of the434 municipalities in Norway, and takes responsibilityfor organisational development of the schools at na-tional level. The council took the initiative to develop abranch-specific quality development system, includingCAF. The entire system was completed in 2001 andimplemented in 2002 as a pilot project.The pilot project was intended to test, implement andmarket the system, to gain experience and acquireknowledge in some schools. Seven municipal musicand art schools took part. The project was very suc-cessful and was concluded in 2004. After the project,the system was offered to all municipalities/schools inNorway via the book by Even Fossum Svendsen: Be-dre og bedre? Kvalitetsutvikling i kulturskolen. TheMinister of Education has given the quality systemstrong support and recommendation.The quality system also contained service declarati-ons and consumer polls. CAF was an important com-ponent as a TQM quality model and self-assessmentmethod. The CAF model was tailor-made to thebranch by adjusting the examples, or indicators, lin-guistically and organisationally. The participants le-arnt to regard themselves as customer service pro-ducers and were challenged regarding their own po-tential for improvement. The CAF alone does notprovide quality, but incorporated into a system of con-

tinuous quality development, CAF provides a trust-worthy description of the organisation, and pinpointsits strengths and weaknesses.

Our major results

All the schools improved their scores in the secondCAF assessment during the project. The average in-crease of all the 9 CAF criteria for all schools was12.3 % with respect to the first CAF assessment.Among the results-criteria, the customer/citizens andpeople results were the two most improved. In this ca-se description, I use the average score of all 7 munici-palities.Since the project included the work with service decla-rations and consumer polls, the focus was already di-rected to the systematic collecting of important infor-mation from the customer or consumers. The qualitysystem considered the quality concept as a way of ta-king care of the needs and expectations of the employ-ers, consumers and professionals. Therefore, the im-provement in the field of customer results will have astrong link to the work with consumer polls. There isalso a close correlation to customer expectations. Thegoal of a small gap between customer expectationsand the customer results, is derived from the princi-ples of correct quality as a correlation between resultsand expectations in a 1/1 relationship.To understand the effect of the strategic improvementwork connected to the CAF assessment, we should look

36

Seven municipal Music and Art schools

Seven municipal Music and Art schools(coordinated by the Norwegian Council of Musicand Art Schools)

Country: NorwayWebsite: www.kulturskoleradet.noCAF uses: 2 xKey words: process management, planned quality

Focus:citizen results, people results


35/101


37

at the gap between the first and second assessment.Of all the criteria, the biggest increase between theassessments was found in Leadership, closely follo-wed by Strategy and planning. As regards results,the Customer/citizen showed a satisfactory increa-se, followed by the People results.


The work with improvement was reported to the pro-ject leader from each municipality. They each chose 1to 3 improvement areas, followed by a strategic qualityplan for the effective process. This work was done verycarefully and was given a high priority. The strongestimprovement result was found in criteria 1 and 2, follo-wed by 6 and 7. Considerable efforts were made regar-ding the organisations' management approach and thestrategic basis. The complementary roles of the firstand second CAF criteria are a driving force behind theresults of criteria 6 and 7. In all 7 organisations, thefocus on consumer polls and service declarations was

strong. Most of the organisations implemented thesetools and achieved satisfactory effects. Several otherimprovement actions were also launched.In the quality development, it is important to improvethe results as well as to adjust consumers' expectati-ons. The right way to work with these adjustments isto make it very clear what kind of product, servicesand quality customers expect. The service declarationis an effective tool to clarify these expectations. Thequality term is considered to have achieved the cor-rect quality when the gap between customer expecta-tions and customer results is as small as possible.

The CAF in practice

The different SAGs were small, 5 to 7 people. Therewas at least one representative from the managementin the group. They spent at least an entire day on theassessment, some organisations spent 2 days. The in-volvement of employees (teachers) was quite success-ful and led to a responsible way of working with the im-provement processes. It also strengthened the ac-countability for ensuring proper quality at allorganisational levels. At least 3 persons in the SAGhad been trained specially in development work, qualitywork and CAF before the assessment began. Thesepeople were brought together for 2 days before theassessment, and we continued using this platform forlearning and holding discussions across organisationalborders. This was later assessed as an importantstrength in the process of quality development.

ContactEven Fossum Svendsen, Deputy Director of EducationCounty Council administration of Nord-TrndelagE-mail: [email protected]

Average results from the 7 municipalities relating to all 9 CAF criteria

1 Leadership2 Stretegy and planning9 Key performance results

3 HRM8 Societyresults

4 Partnershipsand resources

7 People results

5 Process andchange management

6 Customer/citizen-oriented results

Improvement within each of the CAF-criteria. Average of 7 organisations

Lessons learnedThere were not many improvement actions in eachorganisation. All the advice from the assessment/ development groups in the project pointed to theimportance of not working with too many improve-ment actions. Limit your choice; do it properly!


36/101

Health

The primary catchment area of the PirkanmaaHospital District includes 463,000 people andthe secondary catchment area 1.2 million

people. The district includes Tampere University Hos-pital, three district hospitals and two local hospitals.There are approximately 6,600 hospital district staffmembers. The total number of beds in the hospital di-strict is 1,553, producing 78,800 care periods peryear. The annual outpatient attendance is 374,000.The income of Pirkanmaa Hospital District in 2005was approximately EUR 441 million. The greater partof this comprised income from sales, accounting for93.6 %. The expenses comprised staff costs (61.5%), material acquisition (20.8 %), service acquisition(15.5 %) and other operating costs (2.1 %).The ethical principles of the organisation are propermedical care, respect for people and recognition ofskills.According to itsvision: The Pirkanmaa Hospital Di-strict provides high-quality services in specialisedmedical care in a humane economical and productivemanner. Services are provided in accordance withcustomer needs. Patients are content with the recei-ved care. The Tampere University Hospital is recogni-sed nationally and internationally as a centre of edu-cation and research. The Pirkanmaa Hospital Districtprovides a dynamic and challenging working environ-ment for its staff, in which they receive due apprecia-tion for their contributions.

Our major results

Customer satisfaction (CAF criterion 6) is measuredon scale 4-10. The result is considered to be satis-factory if 75% of respondents give a score 9 or 10.Regarding inpatient care, the trend has been positivesince 2000 with 5 out of 6 units meeting or excee-ding the target of 75% every year.The key performance results (CAF criterion 9) aremainly positive and higher than in compared organisa-tions. Tampere University Hospital is 11% more pro-ductive than average university hospitals in Finland.The productivity measured in episodes is higher thanin other compared hospitals. The healthcare costsper capita are lower than average in Finland. Thegoals set in service agreements with the membercommunities of the district have been met. The finan-cial development trend is positive.Customer results are measured extensively (before,during and after the care period) and they show posi-tive development trends. Satisfaction of internal cus-tomers is also measured in line with the process de-velopment work performed in the hospital district.The customer satisfaction results of different clinicsand departments inside the organisation are bench-marked. A specific questionnaire is used to collectthe information centrally to ensure comparability ofthe results. Different units can, however, add theirown questions to the standard questionnaire if desi-red. The information is collected annually or continu-

38

Pirkanmaa Hospital District

Pirkanmaa Hospital DistrictCountry: FinlandWebsite: www.pshp.fi/ (only in Finnish)CAF uses: 3 xKey words: continuous quality improvement,process improvement, implementation BSC.

Focus:citizen results, key performance results


37/101

Health

39

ously during the whole year. Parts of the key resultsare reported in national benchmarking data of health-care. These indicators are collected and reported an-nually by the National Research and DevelopmentCentre for Welfare and Health (STAKES). This infor-mation is used in further developing the performanceof the organisation. All the quality and performanceindicators and their target levels are included in theBSC model of the hospital district.


The selected results in criterion 6 are linked to crite-ria 2 and 5. The selected results in criterion 9 are lin-ked to criteria 1, 2 and 5.The main result of the long-term quality work (usingEFQM and CAF) has been the systematic improvementof management practices and coherent leadership. Ithas included training leaders, synchronising annualplanning processes and evaluating various units andprocesses. During the development period, quality ma-nagement has been integrated into routine businessmanagement of the organisation. A great deal of at-tention has been devoted to the development of mea-surement tools and indicators as well as new processinnovations resulting in higher performance.

The CAF in practice

Every hospital district operational unit performs theself-evaluation annually so that the results can beused in a planning process for the following year. Inevery profit area there is a trained member of thepersonnel who leads the evaluation process. The mainaim is to identify the key strengths relative to theunits operations and the most important areas forimprovement. The main areas of improvement of dif-ferent units are discussed at profit area level, wherethey are prioritised according to the strategic plansof the whole organisation.

ContactMr Juha HeinoMD, MQ, Development ManagerE-mail: [email protected]

Lessons learnedThe quality development of a large organisation is astep-by-step process. Annual self-assessmentsprovide satisfactory information for the annual im-plementation of specific development actions.


38/101

Local Administration

Vienna has 19 Municipal District Offices (MDO)that are part of Vienna's municipality and directlysubordinated to the Mayor of Vienna and the Chief

Executive Director. Each MDO is headed by a civil ser-vant trained in law.

MissionA MDO is the first contact point for our citizens in theareas of commercial law matters, conducting admini-strative penalty procedures, administration of TreeProtection Act, residence registration, passports,lost property office, elections, referendums, opinionpolls, certificates of citizenship, administration ofparking space.

VisionOur organisation wants to change from an old bureau-cracy administration to new standards bas

caf works eipa

Documents