presentacio cbuc 08.04.14 v.def
DESCRIPTION
Presentation by Nuria Balague (Universitat Autonoma de Catalunya) about Quality Programs. April 14th, 2008, BarcelonaTRANSCRIPT
1
EVALUATION AND QUALITY IN SPANISH ACADEMIC LIBRARIES
Núria BalaguéDeputy Librarian
Barcelona, 14 April 2008
2
Outline:
1. Evaluation and Quality in libraries2. The three levels of quality
assurance1. European level2. National level3. Institutional level
3. Summing-up ideas
1.Evaluation and quality in libraries
4
library resources and planning
library results
user needs and expectations
The quality gap
Library information services value chain
5
= Measurement + Improvement
• Evaluation and quality are complementary terms
• Why to evaluate if we do not intend to improve detected weaknesses ?
• How do we know that we are improving in the right direction if we do not submit to periodic evaluations to confirm it and to help us to define new lines of improvement?
6
Quality Approaches in libraries
• 1960-1990 Statistics, indicators and standards
• 1989-2000 Evaluation and TQM • 1990 – EFQM, ISO 9001, LibQual +
Inputs -> Outputs -> OutcomesResources -> Services -> Users ->
Continuous improvement
7
About Quality • what customers perceive as a result
of the comparison of products or services with others, and through reflecting on their own expectations.
• is dynamic; it is a social construct, always relative and continuously evolving: what satisfies the customer today, will fail to do so tomorrow.
• educational sector has become a consumer item governed by the laws of the market and by the impulses and trends influencing citizens’ perception
8
• Libraries are in transition. Defining and measuring what they do, how much they do, and how well they do it is difficult, because what they do is constantly changing.
(Denise Troll Covey, 2002)
9
A quality-assurance system, a good investment• set of structured processes that serve to plan, direct,
communicate, control and improve the quality within an organization.
• process approach helps to gather, organize and distribute suitable information to suitable people, in optimum conditions of costs and time, with the purpose of being able to make the most suitable decisions.
• develop aspects such as quality policies and quality objectives, establish the functions and responsibilities of staff members, define products and services, identify working processes and the necessary resources to carry them out, and fix mechanisms of supervision, control, prevention, correction and continuous improvement
10
Developing and implementing a quality management system consists of several steps including the following:
• a) determining the needs and expectations of customers and other interested parties.
• b) establishing the quality policy and quality objectives of the organization.
• c) determining the processes and responsibilities necessary to attain the quality objectives.
Quality management systems approach(1)
11
Quality management systems approach(2)
• d) determining and providing the resources necessary to attain the quality objectives.
• e) establishing methods to measure the effectiveness and efficiency of each process.
• f) applying these measures to determine the effectiveness and efficiency of each process.
• g) determining means of preventing nonconformities and eliminating their causes.
• h) establishing and applying a process for continual improvement of the quality management system.
2.The three levels of quality assurance
1. European level
13
EUROPE
• Bologna Declaration (June 1999) European Higher Education Area (EHEA).
• Bergen Conference (May 2005) Standards and Guidelines for Quality Assurance in the European Higher Education Area
proposed by the European Network for Quality Assurance in Higher Education (ENQA).
14
Guidelines for Quality Assurance in the EHEA, Section 1.5
“In addition to their teachers, students rely on a range of resources to assist their learning. These vary from physical resources such as libraries or computing facilities to human support in the form of tutors, counsellors, and other advisers. Learning resources and other support mechanisms should be readily accessible to students, designed with their needs in mind and responsive to feedback from those who use the services provided. Institutions should routinely monitor, review and improve the effectiveness of the support services available to their students.”
2.The three levels of quality assurance
2. National level
16
STRATEGIC LINE No. 3: REBIUN AND QUALITY To enhance, promote and
share resources and expertise that favor potential development of the people who work in University libraries and the quality improvement of management and services
Strategic goals 1st Strategic Objective: Evaluation and Best Practices • To promote, disseminate, lead and
share evaluation processes for the improvement and assurance of quality of library services, the effectiveness and efficiency of the allocated resources and accountability to the institution.
2nd Strategic Objective:Leadership, management and
skills • To continue joint actions to improve
and increase the professionalism of University libraries workers to cope with new challenges
17
1994--
18
19
Spanish National Plans for Evaluation of the Quality of Universities1st Plan (1996-2000)• To promote the institutional evaluation.• To compile a homogeneous methodology and common criteria.• To provide society with relevant and objective information on the level of quality of Spanish universities.
2nd. Quality Plan (2001-2006) continued strategic lines of its predecessor. It also sought to promote the participation of the Autonomous Communities by creating autonomic agencies for the quality assessment of higher education.
72 universities 1.450.000 students
• To provide educational authorities with objective information for making the decisions under their power.
20
Spanish University Libraries Evaluations 1998-2006
• [Previously 3 libraries were evaluated between 1996-98 without specific guide for libraries]
• Some 50 Spanish university libraries were subjected to a formal evaluation process (about 75% of all Spanish university libraries).
• 4 different evaluation guides (high coincidences in the key aspects)
– AQU-CBUC, then assumed by the Council of Universities (based on evaluation UK)
– ACUA (based on EFQM) – ANECA (based on EFQM)– AQU-CBUC (new guide, 2006)
• Same main components of the overall process:
– a self-evaluation
– an external evaluation
– a final report
21
Spanish National Agency for Quality Assessment and Accreditation
Its activities include:
•Improving university teaching, research and management activities.
•Fostering the performance of Higher Education Institutions by monitoring objective and transparent processes.
•Providing public administrations with appropriate information for decision-making within the scope of their authority.
•Providing society with information about the achievement of universities’ aims.
22
Certification of the Quality of University Library Services
EVALUATION CRITERIA Ponderation
1.ELEMENTS BASED ON DATA
1.1. Resources: financial, bibliographical, physical space, etc. 10%
1.2. Results: circulation, interlibrary loans, user education use of electronic resources, etc.
15%
1.3. User satisfaction level. 10%
2. ELEMENTS RELATED TO THE PROCESSES
2.1. Level of integration of the library service in the institution 10%
2.2. Level of relation between the library service and its users: library / teaching, library/ research.
5%
2.3. Management, organization and processes. 10%
2.4. Personnel Policy: education, training and promotion. 10%
3. ELEMENTS RELATED TO IMPROVEMENTS DERIVED FROM THE EVALUATION PROCESS
3.1. Criteria related to the development of improvement plans. 20%
3.2. Criteria related to active participation of library service in innovation and cooperation projects.
10%
23
Objectives• Promote evaluation
• Improve library services• Public recognition
Certification of the Quality of University Library Services
Year Library candidates
Certified libraries Grants total amount
2004 16 7 873,000 €
2005 16 12 877,695 €
2006 9 6 889,644 €
Total 25 2,640,339 €
INTERRUPTED PROGRAMME
24
Evaluation of Catalan university libraries
1st. evaluation of Catalan university libraries (1999-2001). Main aims:• to evaluate library processes and the organizations supporting them.• to assess the added value of libraries in terms of how they strengthen student learning and contribute to the betterment of research.• a cross evaluation of Library Services was performed and a final report was prepared.• the Library Services Evaluation Guide (1999) was based on the methodology proposed in “The effective academic library” (UK, 1995).
2nd. evaluation of Catalan university libraries (2006-2007).• new guide: Evaluation Guide of Library Services and their contribution to the quality of learning and research (2006).• a final report of the cross-evaluation will be published in 2008.
• autonomous region of Spain, 7,000,000 inhabitants, own language (Catalan).• 8 public and 5 private universities. • 200,000 students, 15,000 lecturers and 7,000 administrative staff
25
Evaluation criteria• The library within the university
1. 1. The regulatory framework 1.2. Strategic planning 1.3. Towards the transformation of LS 1.4. LS's relations with CBUC.
• Core programmes Related to : 2.1. Teaching and autonomous learning 2.2. Research 2.3. Digital library and the use of IT 2.4. Cooperation and innovation.
• Organization, management and processes 3.1. Organization 3.2. Processes 3.3. Management of participation and communication.
• Quality assurance 4.1. Structure and organization 4.2. Monitoring mechanisms (improvement plans) 4.3. Mechanisms to know user satisfaction.
• Resources 5.1. Staff 5.2. Buildings and equipments 5.3. Collections 5.4. Budget.
• Results Related to: 6.1. Provision of services and volume of activity 6.2. Satisfaction of users and the staff 6.3. Efficiency in the provision of service
26
Evaluation of Catalan university libraries
First evaluation (1999-2001)
• UB• UAB• UPC• UPF• UdG• UdL• URV
Second Evaluation(2006-2007)
• UB• UAB• UPC• UPF• UdG• UdL• URV• UAO• UVic• UOC
•different guide•more evaluation experience•(almost) similar problems with data gathering
•more libraries•more cooperation•more IT• more user -focused
27
Evaluation of Catalan university libraries
• No negative reactions• Improvements are
evident• Training in quality
improvement• We know better our
weaknesses
• Lack of community involvement
• Some improvements are in hands of the university (not libraries)
• Little support for improvement
External assessment is useful in providing advice, new ideas, and some help from outside the organization, and in the validation of the views of internal reviewers. It works on relevant information (evidences not on intuitions).
2.The three levels of quality assurance
3. Initiatives at an Institutional level
29
Best Practices (UAM)
Quality teams, TQM (UCarlos III, UAB,…)
30
Strategic plans (UPC, UNED, La Rioja...)
31
Balanced scorecard (UPF,...)
32
Service charts (UAB, UPF, UPC, …)
33
User surveys (UPC, …)
34
Affect of Service
Empathy
Information Access
Personal Control
Responsiveness
Symbol
Utilitarian space
Assurance
Scope
Timeliness
Convenience
Library as Place
LibraryServiceQuality
Refuge
Reliability
Ease of Navigation
Convenience
Modern Equipment
LibQual+ (UB, Granada,… )
35
EFQM (Cádiz, …)
36
Acquisitions Database
Non ConformingDatabase
Non Documentary Vendors Database
Reading RoomsDatabase
ILLDatabase
VTLS
Corrective and Preventive Actions
Vendors Continuous Evaluation
UAB Libraries ISO 9001 Quality System
Users Surveys, Suggestions and Claims
SERVICE LEVEL AGREEMENT
Internal & External Audits
Annual Review of the Quality System
Quality Indicators
tra cea bi li ty
continuous improvement
37
UAB Libraries quality journey
90
91
92
93
94
95
96
97
98
99
00
01
02
03
04
05
06
European projects
Quality groups
ISO 9000 quality system certification
AQU evaluation
ANECAQuality Certification
3. Summing-up ideas
39
ANECA
ISO
CGC-EFQM
Biblioteca universitària i (multi)certificació de qualitat
Multicertification?
40
Quality: Benefits and Opportunities (1)• The adoption of a quality management system is a strategic
decision of each organization• It is not a purpose itself, but a means for the development of
a continuous improvement dynamics within the library.• We may use:
– a model (as EFQM)– a standard (as ISO 9001)– the criteria of some quality prize or quality agency– using only the common sense, unrelated to any “official”
reference.• Different approaches are not incompatible or exclussive, and
can be simultaneous
• Not a restrictive positioning but an integrative approach in a quality management environment opened to different techniques and models
41
Quality: Benefits and Opportunities (2)
• Today’s organizations are being over audited, which indirectly means that large amount of money and time are (and will be) invested on evaluation.
• If libraries establish a solid foundation for a formal quality-management system, it will be much easier "to adapt the output format" to the specific requirements of any given evaluation that they will have to undergo in order to support university strategies.
42
Evaluation: Benefits and Opportunities (1)• To confront the challenges and the demands of the European
Higher Education Area (EHEA).
• To consolidate the culture of quality.
• To promote appropriate ethics and values within the organization.
• To find out if there is an overload of bureaucracy.
• To find out gaps in the quality management system.
• A mean to constantly analyze the processes performances.
• No “no-go” areas.
• Detection of incipient problems.
• Possibility of immediate implantation of preventive actions.
43
Evaluation: Benefits and Opportunities (2)
• Evaluation as a routine exercise (positive sense)
• Work team reinforcement
• Encourage openness
• Self-reflection on individual tasks
• Self-consciousness of the importance of one’s own work
• It develops a problem-solving culture
• To provide information, advice, challenge and support for the decision-making
44
But...• It is impossible to evaluate everything and overcontrol is
as bad as undercontrol.
• Creating a culture of assessment is not simple, it means changing the mind set and daily activity of librarians and support staff.
• Internet and e-technologies impact the evaluation process, in that evaluations need to become cognizant of the new structures and paradigms of the e-library.
“The methods for assessing new resource delivery and the skills required to conduct such assessments evolve at a slower rate” (D.Troll).
• The objective and criteria of quality evaluations should be gradually improved.
45