total quality management based on self-assessment
DESCRIPTION
Total quality management based on self-assessment. Andre Vyt. The experience with in Flanders. PARTNERS IN QUALITY MANAGEMENT. PROZA exists in different versions. Higher education Secondary education Vocational training for handicapped persons Adult education & training centers - PowerPoint PPT PresentationTRANSCRIPT
Total quality managementbased on self-assessment
Andre Vyt
PARTNERS IN QUALITY MANAGEMENT
The experience with in Flanders
PROZA exists in different versions
• Higher education• Secondary education• Vocational training for handicapped
persons
• Adult education & training centers• Health Care & Social care centers
The goal of is
• to diagnose strengths and weaknesses of all relevant aspects in the organization
• inspired by an all-encompassing and recognized framework (EFQM Excellence model)
• by the process-owners (lecturers, researchers, administrators, technicians ...) in quality teams
• together with all parties involved: supervisors, colleagues, and clients (students, qualified professionals…)
• to set up well-supported specific actions for improvement
Inspired by EFQM Excellence
Basic conditions
Results
Core processes
Core processes
Leadership
Human resources
policy
Appreciation by society
Performance of the
organization
Resources and co-operation
Student satisfaction
Policy and strategy
Personnel satisfaction
The 9 areas of attention
The content of PROZA
• It contains more than 60 focal points over the 9 areas of attention
• In a focal point 25 specific items are to be checked/answered
• The 25 items are grouped into 5 phases, according to the level of quality management
Areas and points of attention
Focal points in core processes
• Admissions coaching• Communication with students• Curriculum design• Design of course units• Implementation of the curriculum• Teaching and tutoring• Student coaching
Focal points in core processes
• Practice-based learning• Internationalisation• Testing and assessment• Graduation and alumni-oriented
activities• Postgraduate education• Public service provision• Research• Development and practice of arts
The 5 phases of development
• Quality is variableQuality is linked to the individual
• Initiation of process-based thinking/actingStarting to work systematically
• (High level of) ProfessionalismQuality is guaranteed
• Systematic innovationContinuous improvement
• Aiming for excellenceRecognition of expertise and high level of quality
The PDCA-cycle of QM (Deming)
quality
plan
do
act
check
time
improvementassurance
evaluate - control
The higher the level attained,
the better the
• Formulation and management of processesMeasurements and information gathering
• Systematisation and reliability Openness and transparency
• Attunement and integration Innovation and improvement
• Involvement of different persons Client and external orientation
1: Quality linked to the individual
• Quality is the result of individual effort
• Short-term thinking• Problems are resolved when they
arise• Limited exchange of information• Lack of follow-up commitments• A minimum of quality is present
2: Initiating process-based thinking/acting
• Formulation of needs and priorities• Overall commitments enforceable to a
limited extent• Increasing attention to information
management• Real causes of most problems are
known• Performance is evaluated regularly• Improvement is clearly noticeable
3: (High level of) Professionalism
• Planned approach• Availability of reliable data• Binding by written commitments• Continuity of service is guaranteed• Level of quality achieved is
supported by assessment data• Information flow is effective
4: Systematic innovation
• Systematic analysis and assessment of operations
• Use of quality indicators as a policy instrument
• Structural problem detection & handling
• Innovative thinking• Emergence of synergy effects
5: (Aiming for) excellence
• Frames of reference, indicators and working methods internationally recognized as best practice
• Broad base of support for moving beyond current levels of quality
• All quality indicators are moving in a positive direction
• Large number of personnel are internationally recognized experts
E.g., the focal point “course units” has items
such as:
1. Do lecturers know the function of their course unit in the curriculum?
2. Do colleagues work together in a team to design course components?
3. Is the design of nearly all course units geared to the frame of reference of the department?
4. Are goals of improvement defined on a yearly basis for all course units?
5. Is the design of specialized course units entrusted to experts?
Testing and assessment
1. Do all students receive a copy of the exam regulations in time?
2. Is there an attempt to safeguard objective assessment by involving more assessors?
3. Are assessment methods for each course component clearly geared to the objectives?
4. Are assessment methods systematically adjusted following evaluation?
5. Is the quality of the assessment methods regularly tested by external experts?
Student admission coaching
Answering the questions
• Answering categories: yes / no / not applicable / impossible to answer
• Lack of measurement data provides relevant information on the need to professionalize the management of information
• Concrete questions provide immediate goals of improvement
The method used in PROZA
• The checklists are filled out individually by the team members
• They gather to agree in consensus on the scores of the items and deduce the level of quality management
• but also to identify immediately specific goals of improvement
• that are selected according priority and written down in a self-evaluation report
Score (in consensus)
• Actual level: phase within which 3 items are positive
• Potential or growth level: phase within which 1 item is positive
• … each time on condition that in previous phases at least 3 items are positive
Chart of scores within an area
Chart of scores over the 9 areas
The PROZAself-assessment process
• Preparation• Diagnosis• Improvement
Preparation
• Policy decision to use PROZA and formation of a steering group
• Getting people on board by engagements and good communication
• Initial screening with quickscan to determine which areas (and foci) to address
• Formation of self-assessment groups• Agreement on working method and prior
clarification of terminology and gradation words
Screening with the quickscan
• 200 statements that probe the overall level of quality (20 per area, 40 for core processes)
• To be answered with– Adequate (A)– Must be improved / not adequate (NA)– Important (I)– Not so important (NSI)
• Preferably by the steering group or key persons
The Quickscan
The Quickscan
Carrying out the diagnosis
• Relevant information on the foci can be made/kept available
• Gaining a view on hot issues by an inventory of individual scores before the meeting
• Value of individual comments on the questions
• Short-circuiting endless discussions:– Right/wrong is less essential than identifying
improvement– Eventual agreement on skipping items on which
only one member answered negatively
From diagnosis to improvement
• Diagnosis and proposals for improvement are formulated on a report form
• Assignment of a priority value• Definitive selection of ultimate priorities
by steering group or management • For each selected goal, an action plan
must be drawn up on a predefined form by an improvement team
Work sheets in PROZA
Action plan
• Concrete goal• Responsible person• Actions that have to be taken• Persons involved• Time plan• Expected results / deliveries• Budget
Client-orientation of PROZA
• Evaluation criteria and items were tested on content and applicability in different HE institutions.
• Assessment questions have grown out of practical experience and are based on a consensus between TQ managers
• The checklists are accompanied by a users guide and a list of concepts
• The checklists are delivered electronically, so that institutes can adjust words to match the structure of the organisation
Some pro’s of PROZA
• Exhaustive and flexible: separate areas and focal points can be analysed for different units in the organisation
• It is easy to deduce questionnaires• More than 1000 items result in very
specific goals of quality improvement• Based on individual analysis and on
consultation with others in teams
Links between self-evaluations and external
quality assurance
Quality management policyHuman resourcesMeans and financesLearning goalsProgramme content and organizationOutside school practice trainingInternational policy and activitiesTraining methods and toolsAssessment and examinationCounseling and guidanceStudy load and effectivenessResearch, arts and societal servicesGraduates
Themes for auditsSelf-evaluation
Implementation requirements
• Improvement as the final goal: diagnosis and scoring is only the starting point.
• Explicit policy and engagement on all levels of the management.
• Agreement on aspects of confidentiality and openness of reports
• Explicit rules about implementation of selected goals, follow-up and feedback
Avoiding pitfalls
• If not properly introduced and coached, it may be perceived and labeled as a mass of paperwork which demands more energy than that it should help to improve quality.
• If not complemented by a leading and steering management, self-evaluation may bog down in the diagnostic process.
• If the instrument is put forward as to be used primarily on teaching processes , it may result in adverse effects of “shooting the piano-player” instead of improving the piano.
Guaranteeing the achievement
of goals
• Diagnoses by self-evaluation teams can exist alongside the management instead of in the heart of the management, leading to quality management on paper instead of effective improvement in reality
• Avoid misuse by management to “solve” management problems, or to evaluate employees.
• Diagnoses must not only result in clear objectives of improvement but must also be made concrete in realistic project planning according to SMART rules.
Criteria for project goals
•Relevant
•Understandable
•Measurable
•Behaviorable
•Attainable
•Specific
•Measurable
•Agreed/acceptable
•Relevant/realistic
•Time-related