customer satisfaction report

8/3/2019 Customer Satisfaction Report

1/16

EUROCONTROL

A Centre of Excellence in ATM Training

IANSCustomer Satisfaction

Repor

Report

Sieu thi dien may Viet Long - www.vietlongplaza.com.vn


2/16

ii Copyright 2006 EUROCONTROL

Copyright Notice

2006 European Organisation for the Safety of Air Navigation (EUROCONTROL). All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic,

mechanical, photocopying, recording or otherwise.

Any other use is subject to prior written consent by EUROCONTROL. Request shall be addressed to:

Director of the Institute of Air Navigation Services, 12, rue Antoine de Saint-Exupry, L-1432 Kirchberg/Luxembourg.

The EUROCONTROL Institute of Air Navigation Services aims to provide the services that you wantand to make your stay in the Institute as enjoyable as possible. All Institute personnel are there toensure that your stay at the Institute is successful. However, if you do have a complaint (or acompliment) please tell us. If you are not satisfied with the service we provide or you would like topropose an improvement then please fill out the form at http://www.eurocontrol.int/ians/complaint.html,

or contact [email protected] directly.


3/16

Printed on 30/11/2004 Produced by the Business and Administration Unit 3

Table of Contents

INTRODUCTION..................................................................................................................................... 4ATM TRAINING......................................................................................................................................5ATC TRAINING ....................................................................................................................................12E-LEARNING........................................................................................................................................ 13STAKEHOLDER FEEDBACK.............................................................................................................. 14COMPLAINTS ......................................................................................................................................15CONCLUSION......................................................................................................................................16


4/16

Customer Satisfaction Report

4 Copyright 2006 EUROCONTROL

Introduction

This is the 7th

report that the Institute devotes to theassessment of the level of satisfaction of its

customers. The study on which this report is basedhas been conducted in the framework of the qualityimprovement programme implemented at theInstitute since end of 1996.

Measuring customer satisfaction on a regular basisis one of the key elements of the quality strategyimplemented by the Institute since 1997.

Our objective is to understand exactly how ourcustomers perceive the Institute and, based on theirlevel of satisfaction, to improve our services andproducts.

This report proposes a summary and an analysis ofthe data collected in 2005.

The customer segmentsassessed in 2005

IANS applies a policy to measure the satisfaction ofits customers with all its products and services.These include:

The ATM course participants

The ATC course participants

The E-Learning Students

Training Sponsors

Consultancy Bodies.

User Groups

Assessment Tools

The assessment is based on a set of assessmenttools and measurement methods. During 2005further work has been done to increase the integrityof the data collection.

Evaluation Forms

To measure the satisfaction of its customers withthe products and services IANS uses the followingevaluation forms:

A Student Evaluation Form filled in by thestudents at the end of a course. This evaluation

form is used for both the ATM courseparticipants and ATC course participants.

An E-Learning Evaluation Form filled in by astudent upon completion of an E-LearningModule.

A generic Stakeholder Feedback Form filled inby various stakeholders. The form is availableathttp://www.eurocontrol.int/ians/feedback.html.

Customer Visits

IANS visits a sample of its customers on a yearlybasis. The customer visits serve as a main source

of feedback on existing products and a source ofinformation for improvements and new products.The customer visits include visits to Air NavigationService Providers and civil and military StateAuthorities.

Complaint Procedure andComplaint Form

IANS uses an anonymous form, available on itsweb-site for the collection of complaints related toits products, services and staff. A reference to thecomplaint procedure and associated form is madein all products and services. The complaint form isavailable athttp://www.eurocontrol.int/ians/complaint.html.


5/16


ATM Training

The Satisfaction of Students with the ATM Training Products is measured through the use of a Course EvaluationForm.

MeasurementMethod

At the end of each(classroom) course,students are presentedwith an evaluation formcontaining a number ofcriteria to be assessed.Each criterion can begiven a result between 1and 4 meaningunacceptable (1),unsatisfactory (2), good

(3) an excellent (4).

The complete set ofcriteria are grouped infive main categories:

Training Content

Instructional Team

Documentation

Equipment

Administration

The first three categoriesare directly related to the

course. The last twocategories are directlyrelated to the supportingenvironment andorganisation.

Figure 1 Student Evaluation Form


6/16



From Forms toResults

We map the marks of thestudents on a scale from

0 (all students markUnacceptable) to +100(all students markExcellent). When allstudents mark Goodthen the result is 75 andwhen students mark, onaverage, between Goodand Excellent then theresult will be around 85.Likewise, if students fillin 'all-Uncacceptable'then the result will be 0.If students fill in 'all-Un-Satisfactory' then theresult will be 25.

1Copyright 2003 EUROCONTROL

Measuring SatisfactionMeasuring Satisfaction

1

Unsatisfactory Satisfactory

2

Good

3

Excellent

4

Marks from Students

ResultResult

All 3All 3All 2All 2All 1All 1

2525 7575 10010000

All 4All 4

Figure 2 How we measure satisfaction

IANS aims to meet our objective of being a Centreof Excellence in ATM Training. For us this meanswe aim to achieve results on any criterion between75 and 100. As an example: the overall average (allcourses, all criteria, 3090 forms) for IANS in 2005 is87.5 compared to 85 in 2004. The highest resulton a single criterion is 95 (Subject Knowledge of theInstructor). The lowest result remains pre-courseinformation (76.5 compared to 70 in 2004).

Results

The results presented below are extracted from theautomated student satisfaction evaluation system.Results can be presented per category, per trainingdomain and per criterion. We have selected the

most significant results to be included in this report.


7/16


0

500

1000

1500

2000

2500

3000

3500

Number of Forms 1937 1915 2624 3090

2002 2003 2004 2005

Figure 3 Number of Evaluation Forms

Size of thePopulation

This graph shows thenumber of evaluationforms received and

processed by IANS. In2005 the Institutecontinued to optimise theprocessing of the forms.

This led to anothersignificant increase ofthe number of processedforms in 2005.

Composition ofthe Population

Students indicate on theevaluation form whichtype of staff they are.

This chart shows that amajority of Students atIANS in 2005 areOperational andTechnical Staff.

These results areconsistent with theresults of the previousyears.

192

345

977

693

223

600

1298

824

0

200

400

600

800

1000

1200

1400

Administrative Managerial Operational Technical

20042005

Figure 4 Composition of the population (number of students)


8/16



50

55

60

65

70

75

80

85

90

95

100

Course Results 83 84,5 86 87,5

2002 2003 2004 2005

Figure 5 ATM Course Results

Satisfaction

This chart shows thesatisfaction of thestudents with the ATMcourses. The result isobtained by averagingthe results obtained onevery criterion in thecategories TrainingContent, InstructionalTeam andDocumentation.

A result of 87 indicatesthat, on average,students mark criteriabetween Good andExcellent.

The increase insatisfaction has been

achieved throughdetailed analysis of theresults of each courseand introducingcorrective actionwherever required.

50

55

60

65

70

75

80

8590

95

100

Content 81,5 83 84,5 86,5

Instructor 88 89 91 92

Documentation 80 80,5 83,5 85

2002 2003 2004 2005

Figure 6 ATM Course Results: per Category

Segmented View

This chart presents theresults per measurement

Category. These areTraining Content,Instructional Team andDocumentation.

IANS has improved theresults on each of thesethree categories duringthe last four years.

IANS will continue toanalyse the results andmake improvementswhere possible andpracticable.


9/16


Best Results

Customers often praiseIANS for its staff andexcellent infrastructure.

This informal feedback is

confirmed by the courseevaluation results. Thecriteria InstructorSubject Knowledge andClassroom Equipmenthave consistentlyreceived the highestmarks from the studentsduring the last threeyears.

IANS will continue toinvest in the competenceof its instructors bycontinuation training and

competency assessmentand it will maintain thebuilding at its highstandard.

50

60

70

80

90

100

Instructor Subject

Knowledge

92,5 93 94,5 95,5

Classroom

Equipment

88,5 89 90,5 91

2002 2003 2004 2005

Figure 7 Criteria with Best Results

50

60

70

80

90

100

Pre Course

Information

61,5 64 70,5 78

Course Duration 71 74 76 76

Canteen 73,5 72,5 75 77,5

2002 2003 2004 2005

Figure 8 Criteria with less positive Results

Areas forImprovement

IANS continuously looksfor opportunities toimprove. Criteria with theless positive results arethe Pre-CourseInformation and theCourse Duration.

In 2004 IANS has put inplace an action plan toimprove the Pre-Courseinformation andsignificant progress hasbeen made in thisdomain in 2004 and2005.

The optimum duration foreach course will bereviewed on a course-by-course basis.

In 2005 IANS changedthe layout of the coffeearea and installed newequipment in the kitchento improve the flow ofstudents.


10/16



Focus on Safety

IANS is heavily involvedin the development anddelivery of SafetyCourses to supportSafety Regulation andSafety Management.

In 2005 the number ofcourses delivered in theSafety domain increasedfrom 36 to 50 and thenumber of participantsincreased from 614 to844.

We are pleased to notethat the results in thisrelatively new area forIANS equal the averageobtained in other, more

mature, trainingdomains.

It is a clear indicationthat the rigorousdevelopment approachat IANS leads toproducts that meetcustomer expectation.

50

55

60

65

70

75

80

85

90

95

100

Safety 86 88,5 90

Overall 84,5 86 87,5

2003 2004 2005

Figure 9 Safety Domain Course Results

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%100%

2004 91% 1% 8%2005 92% 1% 7%

Yes No Blank

Figure 10 Would recommend course to a colleague

Measuring

LoyaltyIn 2004 we introduced anew question on theevaluation form: Wouldyou recommend thecourse to a colleague.

In 2005, like in 2004,more than 90% of thestudents wouldrecommend the coursethat they attended to acolleague.


11/16


0,00%

1,00%

2,00%

3,00%

4,00%

5,00%

6,00%

7,00%

8,00%

9,00%

10,00%

Percentage of'Unsatisfactory'

Results

0,91% 0,90% 0,83% 0,59%

2002 2003 2004 2005

Figure 11 Level of dissatisfaction

Measuringdissatisfaction

To measuredissatisfaction IANS usesthe percentage of

unsatisfactory marks onthe evaluation forms.

This chart shows theproportion of un-satisfactory marks on theevaluation forms for allcriteria.

The excellent results in2005 are spread over theentire spectrum of criteria.

Note that that anunsatisfactory mark onone criterion does not

necessarily mean that thestudent is not satisfiedwith the whole product.


12/16



ATC Training

The satisfaction of ATC ab-initio students ismeasured using the same evaluation as the systemused for measuring the satisfaction of the ATM

students. Satisfaction figures of ATM and ATCstudents must however not be compared. Thecontext of these courses, although delivered in thesame building, is completely different:

The duration of ATM courses is maximum oftwo weeks whereas ATC have a duration ofseveral months.

The student population of ATM courses arestaff members from service providers andmember states.

The student population of ATC courses are ab-inito ATC students

Last but not least, the ATC ab-initio studentsare evaluated at the end of the course. At theend of the evaluation they either pass or fail.

The outcome of the evaluation tends to have asignificant impact on the satisfaction of thestudents.

IANS believes that the critical success factor forATC students is their validation rate (i.e. theproportion of students that obtain an ATC license).We do however present the satisfaction data in thisreport as it is part of our quality management tools.

50

60

70

80

90

100

Training Content 79,5 79 75 78

Instructors 76,5 73 72,5 77

TrainingDocumentation

78 69 76 78

2002 2003 2004 2005

Figure 12 ATC Student Satisfaction

ATC Student

SatisfactionThis chart shows thesatisfaction of ATC ab-initio students.

The last few years ATCab-initio training has beencharacterised by highrotation of staff andcontinuous changes to thetraining content andstructure as a result ofvariations in demand andnew regulatory

requirements.In 2005, IANS significantlyimproved thedocumentation andtraining of the ATCInstructors.

Compared to 2004 thesatisfaction of thestudents has increased foreach segment.


13/16


E-Learning

IANS introduced satisfaction measurement for its e-learning products in the middle of 2004. Studentsare given an opportunity to evaluate each e-learning

module at the end of the module. The figurespresented in this report cover all the modules on theEUROCONTROL IANS on-line TrainingZone,including the CFMU modules.

In 2005, students filled in 2774 forms (800 in 2004)evaluation forms were filled in. 90% of the studentsidentified to be satisfied with the modules and

more than 88% of the students indicated that themodule met the objectives.

These figures indicate that the overall satisfactionwith the e-learning modules is excellent.

1. Why are you studying this module? Personal reasons (e.g. to further career, out of personal interest etc). My employer decided I should. It was a prerequisite for a class-room based course or programme.

2. The module agree not sure-n/a disagree

followed a logical sequence, O O Owas not too easy or too difficult, O O Owas clear and easy to understand. O O O

3. The module was agree not sure-n/a disagree

easy to use, O O Ointeractive and interesting. O O O

4. In order to successfully access this module, did you have to

change your browser (Internet Explorer) settings? install new software (flash plug-in, java plug-in, internet explorer)? disable a popup-killer or anti-intrusion programme (e.g. Norton Internet

Security)?

5. The module agree not sure-n/a disagree

met the objectives set at the beginning,O O Oallowed me to check my understanding. O O O

6. Did you enjoy studying this module: O yes O no

7. Will this module

Help you directly in your current job? Help you in a future job? Increase your understanding of the industry without direct impact on your

current/future job? Help you personally (e.g. out of personal interest)?

E-LearningEvaluation Form

A customised form wasdeveloped to capturestudent satisfaction withthe e-learning modules.The form collectsinformation related to thereasons for studying, thecomplexity of the module,the objectives of themodule and the impact ofthe module on thestudents job.

0%

10%

20%

30%

40%

50%

60%70%

80%

90%

100%

2004 92% 73% 86%

2005 95% 80% 87%

structure level clarity of language

Figure 13 Structure of E-Learning Modules

Structure of E-Learning Modules

This chart shows thepercentage of students

that answered yes on thefollowing questions:

Structure of the modulemeets expectations.

Level of the modulemeets the expectations

Module uses clearlanguage.

These results indicate thatthe structure and contentof the E-Learning modulesmeet the needs of the

students


14/16



0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Self-driven 43% 28%

Employer-driven 30% 39%

Prerequisite 27% 33%

2004 2005

Figure 14 Why did students study e-learning modules

Origin of E-LearningStudents

It is IANS strategy tointegrate e-learning in

well defined trainingprogrammes definedand implemented by thestudents employer - andas pre-requisite modulesfor classroom courses.

These results indicatethat the proportion ofstudents registeredthrough their employer orstudying as a pre-requisite module hassignificantly increasedcompared to the number

of students that study outof their own initiative.

The number of studentsthat see a direct impacton their job fromstudying e-learningmodules has increasedfrom 56% in 2004 to68% in 2005.


15/16


Stakeholder Feedback

Training Consultation Group

The Stakeholder Feedback Form was distributed tothe meeting participants at the end of each TrainingConsultation Group Meeting. The majority of thefeedback indicates that the overall perception of theInstitute lies between Good and Excellent (on ascale unsatisfactory satisfactory good excellent).

Compared to 2004 the results related tocommunication (content, format and timeliness)have improved. We will continue to review and

improve our communication with the members ofthe consultation group.

Other stakeholdersThe stakeholder evaluation form was alsodistributed to:

Participants of the training focus group.

The training partners at IATA.

The results indicate an equally high level ofsatisfaction of working together with IANS.

Customer VisitsThe Customer Visits are a process in which IANSmeets stakeholders in member states on a bi-lateralbasis. It is a mechanism to collect qualitativefeedback from various stakeholders. The countriesvisited in 2005 are: Austria, Belgium, BIH, Bulgaria,Greece, Hungary, Spain, Sweden and Turkey

During the meeting of the Training ConsultationGroup in September 2005 it was agreed that IANSwould include in its Customer Satisfaction Reportinformation gathered during the Customer Visits.

During these customer visits, stakeholders have in

general expressed a high degree of satisfaction withIANS, its staff and the services it offers. Thefollowing points have been raised:

OJTI: The delivery of OJTI courses throughpartnerships is often perceived as anexpensive solution (by stakeholders receivingthe services). The solution is not perceived asentirely satisfactory.

ATSEP Training: Many of the visitedstakeholders are currently in the process of

defining basic training for Air Traffic SafetyElectronics Personnel (ATSEP). Somecollaboration with EUROCONTROL and othermember states could avoid duplication of effort.

On-Site Courses: In general the demand foron-site course (i.e. courses at stakeholderpremises) exceeds the capacity of IANS.

Although most stakeholders are aware of theE-Learning products of the Institute many ofthem have not yet been able to assess the

feasibility to use the products available. ATC Training Notes: IANS should develop a

policy to distribute CCC aligned ATC TrainingNotes.

Complaints

Every training note distributed to students and all

pages on the On-Line Trainingzone contain aclearly visible reference to IANS complaintprocedure. The procedure enables stakeholders tosubmit complaints in an anonymous way or anidentified complaint through e-mail. The text is asfollows:

The EUROCONTROL Institute of Air NavigationServices aims to provide the services that you wantand to make your stay in the Institute as enjoyableas possible. All Institute personnel are there toensure that your stay at the Institute is successful.However, if you do have a complaint (or a

compliment) please tell us. If you are not satisfied

with the service we provide or you would like topropose an improvement then please fill out theform athttp://www.eurocontrol.int/ians/complaint.html, orcontact [email protected] directly.

Although the procedure is clearly marked in alldocumentation, IANS did not receive formalcomplaints in 2005.


16/16



Conclusion

IANS aims to be recognised by its stakeholders aCentre of Excellence in ATM Training. We

continuously improve our products and processes.We measure the satisfaction of our customers withthe products that we provide.

This document presented the results of thesatisfaction measurement in 2005. The satisfactionwith our ATM Training Products shows an upwardtrend in all domains for the fourth year in a row. The

satisfaction of the ATC Ab-initio students with theATC training started increasing again in 2005. In

2005 we have further exploited satisfactionmeasures for the E-Learning Products and for theStakeholders that we work with. The results in thesedomains show a very high degree of stakeholdersatisfaction.

END OF DOCUMENT.

customer satisfaction report

Documents