who guideline on the implementation of quality …

Working document QAS/19.783

January 2019

Draft document for comments – prepared by EMP/RSS

1

WHO GUIDELINE ON THE IMPLEMENTATION 2

OF QUALITY MANAGEMENT SYSTEMS 3

FOR NATIONAL REGULATORY AUTHORITIES 4

(January 2019) 5

DRAFT FOR COMMENTS 6

7

8 © World Health Organization 2019 9 10 All rights reserved. 11 12 This draft is intended for a restricted audience only, i.e. the individuals and organizations having received this draft. The draft 13 may not be reviewed, abstracted, quoted, reproduced, transmitted, distributed, translated or adapted, in part or in whole, in any 14 form or by any means outside these individuals and organizations (including the organizations' concerned staff and member 15 organizations) without the permission of the World Health Organization. The draft should not be displayed on any website. 16 17 Please send any request for permission to: 18 19 Dr Sabine Kopp, Group Lead, Medicines Quality Assurance, Technologies Standards and Norms, Department of Essential 20 Medicines and Health Products, World Health Organization, CH-1211 Geneva 27, Switzerland, fax: (41 22) 791 4856, email: 21 [email protected]. 22 23 The designations employed and the presentation of the material in this draft do not imply the expression of any opinion 24 whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or 25 of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate 26 border lines for which there may not yet be full agreement. 27 28 The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or 29 recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and 30 omissions excepted, the names of proprietary products are distinguished by initial capital letters. 31 32 All reasonable precautions have been taken by the World Health Organization to verify the information contained in this draft. 33 However, the printed material is being distributed without warranty of any kind, either expressed or implied. The responsibility 34 for the interpretation and use of the material lies with the reader. In no event shall the World Health Organization be liable for 35 damages arising from its use. 36 37 This draft does not necessarily represent the decisions or the stated policy of the World Health Organization. 38

39

Please send any comments you may have to Dr S. Kopp, Group Lead, Medicines Quality Assurance,

Technologies Standards and Norms ([email protected]), with a copy to Ms Sinéad Jones ([email protected])

by 30 March 2019.

Medicines Quality Assurance working documents will be sent out electronically only. They will also

be placed on the Medicines website for comment under “Current projects”. If you have not already

received our draft working documents, please send your email address (to [email protected]) and we

will add you to our electronic mailing list.

.

mailto:[email protected]




SCHEDULE FOR DRAFT WORKING DOCUMENT QAS/19.783: 40

41

WHO GUIDELINE ON THE IMPLEMENTATION OF 42

QUALITY MANAGEMENT SYSTEMS FOR 43

NATIONAL REGULATORY AUTHORITIES 44

45

Activity Date

QMS workshop in Zimbabwe. Complete self-benchmarking by end

of workshop. Conduct survey and summarize countries' expression

of needs for WHO guidelines on QMS implementation.

9-13 Oct 2017

Prepare and present concept paper to recommend the development

of the 'WHO guidelines on implementation of quality management

systems (QMS) for national medicines regulatory authorities'.

16-20 Oct 2017

Develop the TOR for drafting group members, including the

selection criteria and invitation letter to join the drafting group.

15 Nov 2017

Contact selected countries and call for experts to join the drafting

group according to selection criteria and follow up on

communications. Select and appoint drafting group.

15 Nov 2017 to 15 Jan 2018

Select and appoint drafting group based on meeting specified criteria

and invite them officially to join the group, WebEx conference, and

first face-to-face drafting group meeting.

15 Jan 2018

Kick-off drafting group meetings (WebEx). 13-15 Feb 2018

First face-to-face meeting with drafting group, Tunis, Tunisia. 12-16 Mar 2018

QMS workshop in Burkina Faso. Complete self -benchmarking by

end of workshop. Run survey and summarize countries' expression

of needs for WHO guideline on QMS implementation.

16-20 Apr 2018

Produce first draft (V1) of the guidelines. 29 Jun 2018

Review of draft V1 by CRS, rapporteur and ISO expert. 30 Jun to 10 Jul 2018

Deliver final V1 for review by drafting group. 11 Jul 2018

Conduct WebEx meetings with drafting group. 7-9 Aug 2018

Drafting group members submit comments on V1. 31 Jul 2018

Address comments from drafting group and produce V2 for review

by CRS and hand over to TSN.

31 Aug 2018

ECSPP meeting. Update on status of guidelines development. 15 Oct 2018


3

Submit to TSN for Public Consultation 1. 31 Oct 2018

Public consultation 1. 31 Oct – 31 Dec 2018

Address and incorporate comments from public consultation 1,

produce draft to share with drafting group, address comments from

drafting group, prepare (V3) of the guidelines.

31 Dec 2018

Provide opportunity for the drafting group to comment on current

version via WebEx.

1 Feb 2019

Organize and conduct informal consultation with international

stakeholders and drafting group (three-day meeting)

10 Feb 2019

Informal consultation on V3, Tunis, Tunisia. mid-March 2019

Post-meeting edits/clean up and submit to TSN for public

consultation 2 (V4).

Public consultation 2. 15 Mar to 15 Jun 2019

Collate comments and circulate for final review by drafting group. 15 Jul 2019

Hand-off to TSN for review by ECSPP in advance to annual meeting

in October where the guidelines will be presented for adoption.

31 Jul 2019

Review guidelines for possible endorsement. Oct 2019

Address all comments and requests from ECSPP, produce final

version of guidelines (V5). 31 Dec 2019

46


Table of Contents 47

Abbreviations .......................................................................................................................................... 6 48

1. Introduction ..................................................................................................................................... 7 49

1.1 Background ................................................................................................................................... 7 50

1.2 Basis for the guideline .................................................................................................................. 8 51

1.3 Objective ....................................................................................................................................... 8 52

1.4 Scope ............................................................................................................................................. 8 53

1.5 Instructions for using the guideline ............................................................................................... 9 54

2. General considerations ...................................................................................................................... 10 55

3. Definition of terms ............................................................................................................................ 12 56

4. Translation of ISO Standard 9001:2015 to the specific needs of NRAs ........................................... 15 57

4.1 Requirements .............................................................................................................................. 15 58

4.2 High level structure of ISO 9001:2015 ....................................................................................... 15 59

4.3 Clause by clause guidance for NRAs on the requirements for ISO Standard 9001:2015 ........... 18 60

Clause 0 Introduction .................................................................................................................... 18 61

Clause 1. Scope ............................................................................................................................. 22 62

Clause 2. Normative references .................................................................................................... 22 63

Clause 3. Terms and definitions .................................................................................................... 22 64

Clause 4. Context of the organization ........................................................................................... 23 65

Clause 5. Leadership ..................................................................................................................... 45 66

Clause 6. Planning ........................................................................................................................ 49 67

Clause 7. Support .......................................................................................................................... 54 68

Clause 8. Operations ..................................................................................................................... 64 69

Clause 9. Performance evaluation ................................................................................................. 75 70

Clause 10. Improvement ............................................................................................................... 84 71

5. QMS implementation methodology .................................................................................................. 87 72

6. Considerations to ensure integrated implementation of QMS in NRA ............................................. 90 73

References and further reading ............................................................................................................. 92 74

Authors and acknowledgements ........................................................................................................... 94 75


5

Appendix 1. Integration of QMS into the WHO Global Benchmarking Tool ...................................... 95 76

77

78


Abbreviations 79

NOTE: This section will be updated in the final stages of guideline development. 80

81

CAPA Corrective action and preventive action 82

CI Continual improvement 83

DI Documented information 84

GBT Global Benchmarking Tool 85

GCP Good Clinical Practice 86

GMP Good Manufacturing Practice 87

GRP Good Regulatory Practice 88

ICT Information and Communication Technology 89

IMS Integrated Management System 90

ISO International Standards Organization 91

KPI Key Performance Indicators 92

LIMS laboratory Information Management System 93

MA Marketing Authorization 94

M and M Monitoring and Measurements 95

MC Market Surveillance and Control 96

MOF Ministry of Finance 97

MOH Ministry of Health 98

MRM Management Review Meeting 99

MS Member States 100

NCL National Control Laboratory 101

NRA National Regulatory Authority 102

PDCA Plan, do, check and act 103

P and S Products and Services 104

QMP Quality Management Principles 105

QMS Quality Management System 106

SF Substandard and Falsified 107

SLP Summary Lot Protocol 108

SOP Standard Operating Procedure 109

TM Top Management 110

TRM Technical Review Meeting 111

VL Vigilance (one of the NRA regulatory functions) 112

WHO World Health Organization 113


7

1. Introduction 114

115

1.1 Background 116

117

Implementation of the Thirteenth World Health Organization (WHO) General Programme of Work 118

(2019-2023) as adopted by the Seventy-first World Health Assembly (2018) and the WHO Leadership 119

Priorities, has attracted much international public health attention to the theme of Universal Health 120

Coverage (UHC) and increased access to safe and effective medical products. 121

122

Several World Health Assembly (WHA) resolutions, including WHA67.20 (2014), mandate WHO to 123

provide support to its Member States (MS) in strengthening national regulatory systems for medical 124

products. It recognizes that “effective regulatory systems are an essential component of health system 125

strengthening and contribute to better public health outcomes; that regulators are an essential part of the 126

health workforce, and that inefficient regulatory systems themselves can be a barrier to access to safe, 127

effective and quality medical products” [1]. Accordingly, WHO’s vision is for all MS to have a 128

regulatory system that ensures medical products and other health technologies in the market meet 129

internationally recognized standards of quality, safety, and efficacy to facilitate access to these products. 130

131

National Regulatory Authorities (NRAs) are responsible for facilitating access to safe, quality and 132

effective medical products within the respective MS and for consistently demonstrating that the services 133

they provide meet legal and regulatory requirements; they deliver effective and efficient services; and 134

they can evaluate performance and make improvements. A quality management system (QMS) can 135

ensure that the products or services an NRA provides consistently meet statutory and regulatory 136

standards and meet customers’ expectations. A QMS provides opportunities to enhance customer 137

satisfaction; address context-associated risks and opportunities for continuing improvement; 138

demonstrate conformity to specific QMS requirements; and assure the quality, safety and efficacy of 139

medical products. 140

141

In 2015, WHO developed and launched the Global Benchmarking Tool (WHO GBT). This tool assists 142

regulators worldwide in evaluating the developmental status of their regulatory system and its related 143

functions. The GBT includes one indicator that assesses the NRAs’ level of development with respect 144

to QMS.1 Benchmarking results of 43 low and middle-income countries indicate that most NRAs need 145

to establish and implement, or if already established, enhance and maintain QMS. 146

1 Appendix 1 describes the relationship between QMS and the WHO GBT.


QMS implementation is challenging for NRAs due to the diversity of NRA organizational structures, 147

the different levels of NRA development and the number of regulatory functions that need to be 148

addressed. Several international guidelines on QMS have been published; however, none of these 149

focuses specifically on NRAs. Other existing guidelines are field specific [11-20]. At the request of 150

MS, WHO developed this document to provide tailored guidance to NRAs on QMS implementation. 151

152

1.2 Basis for the guideline 153

154

ISO Standard 9001:2015 ‘Quality management systems- Requirements’ is a well-known international 155

standard published by the International Organization for Standardization (ISO). The standard is 156

applicable to both products and services and provides requirements for establishing a QMS that can be 157

applied to any organization, public or private, big or small, and to a variety of fields. The WHO GBT 158

QMS sub-indicators are based on this standard. Accordingly, ISO standard 9001:2015 offers a practical 159

model to establish and implement QMS for NRAs [7]. 160

161

1.3 Objective 162

163

The objective of this guideline is to assist NRAs to develop, implement or improve quality systems 164

using ISO Standard 9001:2015, and subsequent updates, as a basis. The expectation is that this will 165

increase the reproducibility of the quality and consistency of the outputs (products and services), 166

customer focus and stakeholder satisfaction. 167

168

1.4 Scope 169

170

This is an overarching guideline that can be applied across all regulatory functions, including 171

registration and marketing authorization, vigilance, market surveillance and control, licensing 172

establishments, regulatory inspections, laboratory access and testing, clinical trials oversight and lot 173

release. 174

175

This guideline, for practical and illustrative purposes, only provides examples for the following four 176

NRA functions: 177

178

• Registration and marketing authorization (MA) 179

• Vigilance (VL) 180

• Lot release (LR) 181


9

• Market surveillance and control (MC). 182

183

Each of these functions was selected for different reasons: MA is a critical function, but no specific 184

guidance is available; VL is the weakest function as evidenced by the results from WHO benchmarking 185

of NRAs; LR is the vaccine-specific regulatory function and requires particular attention; and MC was 186

selected because sub-standard and falsified medical products are a major issue in developing countries. 187

Although the examples are specific to these four functions, it is important to note that the principles can 188

be applied to any regulatory function. 189

190

This guideline can be utilized by all institutions responsible for regulatory oversight of medical products 191

including the national control laboratory (NCL) and any other agency or institute involved in regulatory 192

oversight, as well as customers and other stakeholders. 193

194

1.5 Instructions for using the guideline 195

196

This guideline provides interpretation for implementation purposes specific to NRAs. It does not 197

duplicate the text from the ISO Standard 9001:2015, therefore it is important to read this guideline in 198

conjunction with ISO Standard 9001:2015 and its supporting documents. 199

200

1.5.1 Required documents 201

202

As a first step, NRAs should have the following three documents on hand. 203

204

• ISO Standard 9001:2015 is the standard upon which this guideline is based. 205

• ISO Standard 9000: 2015 provides QMS-related vocabulary (terms and definitions) and describes 206

the fundamentals and principles of QMS [8]. 207

• ISO/TS 9002:2016 provides generic guidance (not specific to NRAs) on how to apply ISO Standard 208

9001:2015 by describing individual clauses and giving examples of steps any organization can take 209

to meet the requirements [21]. 210

211

1.5.2 Recommended documents 212

213

The additional documents listed below may be of interest to NRAs but are not required. 214

215

• Quality management principles (ISO brochure) [34] 216

• ISO Standard 9001:2015 for small enterprises – What to do? Advice from ISO/TC 176 217


• ISO/IEC 17020 covering requirements for the operation of various types of bodies performing 218

inspection including regulatory inspections [22] 219

• ISO/IEC 17025: 2017 is the existing standard providing general requirements for the competence of 220

the testing and calibration laboratories [23] 221

• ISO 19011:2018 Guidelines for auditing management systems [24] 222

• ISO 31000:2018 Risk management- Guidelines [25] 223

• ISO 9004: 2018 Quality management - Quality of an organization - Guidance to achieve sustained 224

success [26] 225

226

1.5.3 Accessing the documents 227

228

Complete information on QMS-related ISO standards, free brochures and publications and links for 229

purchase of these documents are available on the ISO website https://www.iso.org/iso-9001-quality-230

management.html. Alternatively, ISO standards can also be purchased from the national standard body 231

in the NRA’s country. 232

233

1.5.4 Guidance for NRAs on the requirements for ISO 9001:2015 234

235

This document provides guidance on how the ISO Standard 9001:2015 requirements can be applied to 236

QMS implementation for NRAs. Section 4.3 of this guideline is a clause by clause correlation to 237

Clauses 0 to 10 of ISO Standard 9001:2015. NRAs are advised to use the following step-wise approach: 238

239

1. Review the clause in ISO Standard 9001:2015. 240

2. Refer to the corresponding clause and accompanying guidance in ISO/TS 9002:2016. 241

3. Refer to the corresponding clause in section 4.3 of this guideline, which contains guidance and 242

examples specific to NRAs. 243

244

NRAs are referred to ISO Standard 9000: 2015 [8] for terms and definitions. Definitions of some 245

important terms are included in Section 3 of this guideline. 246

2. General considerations 247

248

Use of this guideline is voluntary. NRAs are free to use this guideline or to choose other methods for 249

implementing QMS. 250

251

https://www.iso.org/iso-9001-quality-management.html

https://www.iso.org/iso-9001-quality-management.html


11

NRAs from any country can use this guideline regardless of the NRA’s size or organizational structure. 252

WHO considered three NRA organizational models (centralized, decentralized and discrete)2 and 253

examined whether the differences in their organizational structure impacted the approach to QMS 254

implementation. WHO concluded that regardless of the organizational structure of the NRA, each 255

institution involved in the regulatory oversight of medical products should establish its own QMS for 256

the products and services it provides. 257

258

Preferably, all regulatory institutions in a single country should follow the same standard for 259

consistency and coherency, ideally ISO Standard 9001:2015, since this standard provides the basis 260

(principles and requirements) for adaptation and implementation of QMS to any field. 261

262

Certain ISO 9001:2015 standard elements are systemic while others are functional. This guideline 263

discusses them in an integrated manner tailored to the specific needs of NRAs. 264

265

WHO does not provide QMS certification services and cannot issue QMS certification or conduct 266

official QMS audits of NRAs. However, as part of the regulatory systems strengthening program, WHO 267

can provide technical support for benchmarking. 268

269

Good Regulatory Practices (GRP) provide a means for establishing sound, affordable, and effective 270

regulation of medical products as an important part of health system strengthening. GRP are a set of 271

practices applied to the development, implementation and maintenance of controls, including laws, 272

regulations and guidelines, to achieve a public policy objective. There are nine GRP principles [6]. 273

274

• Legality: Regulation should have a sound legal basis and should be consistent with existing 275

legislation, including international norms or agreements. 276

• Impartiality: Regulation and regulatory decisions should be impartial to be fair and 277

to avoid conflicts of interest, unfounded bias or improper influence from stakeholders. 278

• Consistency: Regulations should be clear and predictable; both the regulator and the 279

regulated party should understand the behaviour and the conduct that are expected and the 280

consequences of noncompliance. 281

• Proportionality: Regulations and regulatory decisions should be proportional to the risk and 282

should not exceed what is necessary to achieve the objectives. 283

• Flexibility: Regulations should not be prescriptive; they should allow flexibility in 284

2 A more detailed description of the NRA models is provided in practical help box 1.


responding to a changing regulated environment and different or unforeseen circumstances. 285

286

• Effectiveness: Regulations should produce the intended result. 287

• Efficiency: Regulations should achieve their goals within the required time, effort and cost. 288

• Clarity: Regulations should be accessible to, and understood by, the users; 289

• Transparency: Regulatory systems should be transparent; requirements and decisions should be 290

made known to affected parties and, where appropriate, to the public in general. 291

292

These principles may be used while framing the quality policy and objectives of the NRA and 293

compliance achieved through the implementation of QMS. A WHO guidance document on GRP is in 294

development [6]. 295

3. Definition of terms 296

297

NOTE: This section will be refined and updated in the final stages of guideline development. 298

299

The definitions provided below apply exclusively to the terms used in this guidance document. 300

301

Terminology and definitions in this document that are specific to NRAs are those of WHO [32, 33]. 302

Additional terms related to QMS can be found in ISO Standard 9000:2015 [8]. 303

304

A 305

• Activities- project management smallest identified object of work in a project 306

• Assessment- Systematic, independent and documented process for obtaining assessment evidence 307

and evaluating it objectively to determine the extent to which assessment criteria are fulfilled 308

• Audit- Systematic, independent and documented process for obtaining objective evidence and 309

evaluating it objectively to determine the extent to which the audit criteria are fulfilled 310

311

B 312

• Batch- A defined quantity of product processed in a single process or series of processes and 313

therefore, expected to be homogeneous 314

315

C 316

• Certification- The term applied to third party attestation related to products, processes, systems or 317

persons 318

• Competence- Ability to apply knowledge and skills to achieve intended results. Commitment 319

• Conformity- Fulfilment of a requirement 320

• Continual improvement- Recurring activity to enhance performance 321

• Control- The taking of all necessary actions to ensure and maintain compliance with the criteria 322

established in the statutory and regulatory requirements 323

• Corrective action- Action to eliminate the cause of nonconformity and to prevent recurrence 324


13

• Counterfeit- А counterfeit medicine is one which is deliberately and fraudulently mislabelled with 325

respect to identity and/or source 326

• Customer- Person or organization that could or does receive a product or a service that is intended 327

for or required by this person or organization 328

• Customer satisfaction- Customer’s perception of the degree to which the customer’s expectations 329

have been fulfilled 330

• Customer service interaction of the organization with the customer throughout the life cycle of a 331

product or a service 332

333

D 334

• Defect- Non-fulfilment of a requirement related to a specified use 335

• Documented information- information required to be controlled and maintained by an organization 336

and the medium on which it is contained 337

338

E 339

• Effectiveness Extent to which planned activities are realized and results achieved 340

• Efficiency Relationship between the result achieved and the resources used 341

342

F 343

Feedback- customer satisfaction opinions, comments and expressions of interest in a product, a service 344

or a complaints-handling process 345

346

G 347

• Good manufacturing practice That part of quality assurance which ensures that products are 348

consistently produced and controlled to the quality standards appropriate to their intended use and 349

as required by the marketing authorization 350

• Good regulatory practice Set of practices that are to be applied to the development, implementation 351 and maintenance of controls – including laws, regulations and guidelines – to achieve a public policy 352 objective 353

• Governance Refers to the different ways that organizations, institutions, businesses and 354

governments manage their affairs. Governance is the act of governing and thus involves the 355

application of laws and regulations, but also of customs, ethical standards and norms 356

357

K 358

• Key performance indicator- A quantifiable measure used to evaluate the success of an organization, 359

employee, etc. in meeting objectives for performance 360

361

M 362

• Management system- System to establish policy and objectives and to achieve those objectives 363

• Measurement management system- Set of interrelated and interacting elements necessary to achieve 364

metrological confirmation and continual control of measurement processes 365

• Medical products- A term that includes medicines, vaccines, diagnostics and medical devices 366

367

N 368

• National Regulatory Authority- WHO terminology for national medicines regulatory authorities. 369

NRAs should promulgate and enforce medicines regulations 370

• Non-conformity- Non-fulfilment of a requirement 371

372

P 373

• Procedure- Specified way to carry out an activity or a process 374

• Process- Set of interrelated or interacting activities that use inputs to deliver an intended result 375

• Process approach- Any activity or set of activities, that uses resources to transform inputs into 376

outputs can be considered a process 377


• Product- Output of an organization that can be produced without any transaction taking place 378

between the organization and the customer 379

• Provider (Supplier)- Organization that provides a product or a service 380

381

Q 382

• Qualification Action of proving that any premises, systems and items of equipment work correctly 383

and lead to the expected results 384

• Quality Degree to which a set of inherent characteristics of an object fulfils requirements 385

• Quality assurance Part of quality management focused on providing confidence that the quality 386

requirements will be fulfilled 387

• Quality characteristic Inherent characteristic of an object related to a requirement 388

• Quality control- Part of quality management focused on fulfilling quality requirements 389

• Quality management- Management with regard to quality 390

• Quality management system Part of management system (set of interrelated or interacting elements 391

of an organization to establish policies, objectives, and processes to achieve those objectives) with 392

regard to quality 393

• Quality manual- Specification for the quality management system of an organization 394

• Quality planning Part of quality management focused on setting quality objectives and specifying 395

necessary operational processes and related resources to fulfil the quality objectives 396

• Quality policy Overall intentions and direction of an organization related to quality as formally 397

expressed by top management 398

399

R 400

• Regulation- A written instrument containing rules having the force of law. Regulatory requirement- 401

Obligatory requirement specified by an authority mandated by a legislative body 402

• Release- Permission to proceed to the next step of the process or to the next process 403

• Requirement- Need or expectation that is stated, generally implied or obligatory. Generally implied 404

means that it is a custom or common practice for the organization, its customers or other interested 405

parties, that the need or expectation under consideration is implied 406

• Review- Determination of the suitability, adequacy and effectiveness of an object to achieve 407

established objectives 408

• Risk- Effect of uncertainty 409

410

S 411

• Service- Output of an organization with at least one activity necessarily performed between the 412

organization and the customer 413

• Statutory requirement- Obligatory requirement specified by a legislative body 414

415

V 416

• Validation Confirmation, through the provision of objective evidence, that the requirements for a 417

specific intended use or application, have been fulfilled. 418

419

420


15

4. Translation of ISO Standard 9001:2015 to the specific needs of 421

NRAs 422

423

4.1 Requirements 424

425

This section of the guideline requires NRAs to have access to three standards: ISO 9001:2015, ISO 426

9000:2015 and ISO/TS 9002:2016 [21] (see 1.5.1 above). 427

428

4.2 High level structure of ISO 9001:2015 429

430

ISO Standard 9001: 2015 contains an introduction section (Clause 0) and 10 clauses as presented in 431

Table 1. Clauses 1 to 3 set the stage for the requirements. Clauses 4 to 10 represent the actual 432

requirements, with Clause 4 providing an overview of considerations regarding the context of the 433

organization and how to apply the process approach. These considerations are addressed in detail in 434

Clauses 5 to 10. Table 1 provides an overview of the structure of ISO Standard 9001:2015 and briefly 435

describes the intent of each clause. 436


Table 1. Structure of ISO 9001:2015, its clauses and brief description of intent for each clause 437

438

0 Introduction Describes benefits of QMS, quality management principles, concept of process approach and Plan-Do-Check-Act (PDCA),

concept of risk-based thinking and relationship with other management system standards.

1 Scope Provides purpose of QMS for an organization (i.e. NRA)

2 Normative

references

ISO Standard 9000:2015 should be used as the reference standard which defines the terms used in ISO Standard 9001:2015.

3 Terms and

definitions

Terms and definitions are given in ISO Standard 9000:2015.

4 Context of the

organization

4.1 To determine issues (strengths and areas for improvement) internal and external to the NRA which may affect its ability to

meet the expected results

4.2 To determine the interested parties (stakeholders) and capture their needs and expectations relevant to the QMS

4.3 The organization (i.e. NRA) to decide the scope (boundaries) of the QMS

4.4 Provides a template for process approach (PDCA) and documented information needed for QMS

5 Leadership 5.1 Responsibilities/actions of/by top management (TM) to demonstrate leadership and commitment towards QMS, including

customer focus

5.2 Development of a quality policy by TM and ensuring its application

5.3 Definition of roles, responsibilities and lines of authority by TM

6 Planning

6.1 Determining risks and opportunities (using information from 4.1 and 4.2) and planning actions on risks and opportunities

6.2 Establishing quality objectives and making plans to achieve them

6.3 Planning for changes, if any, in QMS

7 Support 7.1 Providing resources for QMS (people, infrastructure, measuring equipment and organizational knowledge)

7.2 Ensuring staff are competent

7.3 Ensuring people are aware of quality policy, quality objectives, importance of their contributions to the effectiveness of QMS

and knowing the consequences for not doing work as per QMS

7.4 Establishing internal and external communication processes

7.5 Creation and control of documented information (procedures and records)

8 Operation 8.1 To address operational planning and control


17

8.2 Requirements for products and services (P and S) covering communication with customers, developing and reviewing

requirements for P and S and to document changes to P and S requirements

8.3 To develop processes for designing and developing P and S

8.4 To develop processes for procurement of the right P and S

8.5 To carry out provision of services under controlled conditions, including post-delivery activities

8.6 To ensure authorized release of P and S

8.7 To ensure outputs (products, services or other) which are not conforming are controlled

9 Performance

evaluation

9.1 Monitoring, measurement, analysis and evaluation (Check part of PDCA) covering plan for monitoring and measurements

(M and M) of P and S, processes and system and for analysis and evaluation of M and M data and establishing a process for

obtaining customer feedback for assessing the degree of customer satisfaction

9.2 Process of planning and conducting internal QMS audits and reporting results internally

9.3 Management review covering purpose of review, inputs to be considered by TM and outputs of review with decisions and

actions relating to opportunities for improvement, changes needed in QMS and resource needs

10 Improvement 10.1 To determine opportunities for improvement with focus on enhancing customer satisfaction

10.2 Nonconformity and corrective action. Actions to control or correct a nonconformity should be taken promptly, this can be

achieved by containing the problem while investigations continue to eliminate its cause to avoid its recurrence

10.3 Using outputs from 9.1.3 and improvement decisions taken during management review to initiate continual improvements

Annex A Clarification of new structure of the standard, terminology in the standard and concepts

Annex B Other international standards on quality management and QMS developed by ISO/TC 176

Bibliography Useful list of supporting ISO standards and websites

439


4.3 Clause by clause guidance for NRAs on the requirements for ISO 440

Standard 9001:2015 441

442

Clause 0 Introduction 443

444

This clause describes the benefits of QMS; quality management principles; the concept of process 445

approach and Plan-Do-Check-Act (PDCA); the concept of risk-based thinking; and relationship with 446

other management system standards. 447

448

Clause 0.1 General 449

450

ISO Standard 9001:2015 has been adopted by more than 130 countries, it is internationally accepted 451

and has become a world benchmark for good management practice. More than one million certificates 452

of conformity to ISO Standard 9001 have been issued worldwide. ISO Standard 9001:2015 is not a 453

product standard but a system standard. It gives “what” an NRA should do for its QMS and leaves the 454

“how” to be decided by the NRA. 455

456

Benefits for NRAs using this ISO standard include: 457

458

• the possibility to standardize operations, leading to uniformity; 459

• availability of up-to-date manuals, instructions and procedures; 460

• clarity and transparency of responsibilities; 461

• systematic training and development of human resources; 462

• structured and smooth vertical and horizontal communication; 463

• in-built process performance monitoring and improvement mechanism; 464

• systematic processing of customer feedback; 465

• standard system of detections, investigation and correction of errors; 466

• system for addressing customers’ complaints; and 467

• a means to demonstrate the ability to consistently provide quality products and services. 468

469

QMS are influenced by the different policies, objectives, diverse work methods, resource availability 470

and administrative practices specific to each NRA. Therefore, the details of each QMS will be different 471

in each NRA. While the detailed method of QMS implementation is important, what matters most is 472

that the QMS yields effective, consistent and reliable results. The QMS must be as simple and 473

understandable as possible to function properly and meet the quality policies and objectives of the NRA. 474


19

An NRA may decide to have its QMS assessed for certification or not. Regardless of whether it seeks 475

assessment and/or certification by a third party, the NRA will still benefit from the implementation and 476

maintenance of an effective QMS. 477

478

The ISO Standard 9001:2015 standard adopts the “process approach”, which enables the NRA to plan 479

processes and its interactions. It also incorporates the concepts of the PDCA cycle and risk-based 480

thinking. Risk-based thinking enables an NRA to identify factors that could cause its processes and 481

QMS to deviate from the planned results; put in place preventive controls to minimize negative effects; 482

and leverage opportunities as they arise. 483

484

Clause 0.2 Quality Management Principles 485

486

ISO Standard 9001:2015 supports the application of the seven quality management principles (QMPs) 487

described in ISO Standard 9000:2005 which are applicable in the context of NRAs. 488

489

• Customer focus. The primary focus of quality management is to meet customer requirements and 490

to strive to exceed customer expectations. 491

• Leadership. Leaders at all levels establish unity of purpose and direction and create conditions in 492

which people are engaged in achieving the organization’s quality objectives. 493

• Engagement of people. Competent, empowered and engaged people at all levels throughout the 494

organization are essential to enhance the organization’s capability to create and deliver value. 495

• Process approach. Consistent and predictable results are achieved more effectively and efficiently 496

when activities are understood and managed as interrelated processes that function as a coherent 497

system. 498

• Improvement. Successful organizations have an ongoing focus on improvement. 499

• Evidence-based decision-making. Decisions based on the analysis and evaluation of data and 500

information are more likely to produce desired results. 501

• Relationship management. For sustained success; organizations manage their relationships with 502

relevant interested parties, such as providers. 503

504

An ISO brochure titled ‘Quality management principles’ provides the full text of the QMPs. The 505

brochure can be downloaded from http://www.iso.org/iso/pub100080.pdf at no charge. ISO 9000:2005 506

and the ISO brochure, contain supporting information on the QMPs, including the rationale, key 507

benefits and possible actions associated with each QMP. These principles also provide a sound basis 508

for establishing quality policy and quality objectives. 509

510

http://www.iso.org/iso/pub100080.pdf


Clause 0.3 Process approach 511

512

ISO 9001:2015 advocates the use of a process approach for the development, implementation and 513

enhancement of the effectiveness of a QMS, with the aim of increasing customer satisfaction by meeting 514

their requirements. Understanding and managing the interconnected processes in a regulatory system 515

contributes to the NRA’s effectiveness and efficiency in achieving its quality objectives and intended 516

results. In addition, this approach helps NRAs identify the management capacity needed to produce the 517

desired outputs. 518

519

The NRA should identify the following elements for each process: 520

• the main inputs to the process, for example: information, legal requirements, national and/or 521

regional government policies, materials, energy, human and financial resources; 522

• the desired outputs, for example, the characteristics of the product/service to be provided; 523

• controls and indicators needed to verify the process performance and/or results; and 524

• interaction with other processes (outputs from one process typically form inputs into other 525

processes). 526

PDCA 527

528

The regulatory system, the QMS and related processes can be managed using the Plan-Do-Check-Act 529

cycle (PDCA) cycle with an overall focus on risk-based thinking to leverage opportunities and prevent 530

undesirable results. ISO 9001:2015 provides the following brief description of the PDCA process: 531

532

• Plan: establish the objectives of the system and its processes, and the resources needed to deliver 533

results in accordance with customers’ requirements and the NRA’s policies, and identify and 534

address risks and opportunities; 535

• Do: implement what was planned; 536

• Check: monitor and, where applicable, measure processes and the resulting products and services 537

against policies, objectives, requirements and planned activities and report the results: 538

• Act: take actions to improve performance, as necessary. 539

540

Clauses 6 to 10 each focus on one stage of the PDCA cycle: 541

542

• Clause 6 –Planning – Plan 543

• Clause 7 – Support – Do 544


21

• Clause 8 – Operation – Do 545

• Clause 9 – Performance evaluation – Check 546

• Clause 10 – Improvement – Act 547

Figure 1. ISO Standard 9001:2015 clauses viewed in relation to the PDCA cycle. 548

549

550

Risk-based thinking 551

552

According to the ISO standard 9001:2015, risk-based thinking is an essential component of QMS. Risks 553

and opportunities should be identified during the planning stage. 554

555

Risk is the effect of uncertainty, and any uncertainty can have positive or negative effects on one or 556

more objectives. Uncertainties can emerge due to changes in the operational environment, political 557

decisions, lack of information or unknown information or a variety of aspects. NRAs should plan and 558

implement actions to address risks and opportunities to prevent negative effects and improve results. 559

560

Opportunities can arise due to situations favourable to the achievement of a desirable result. For 561

example, a change in the structure of the NRA can create opportunities to improve the efficiency in the 562

organization. It can also carry some risks. Actions taken to leverage the opportunities should also 563

include consideration of the associated risks. 564

565

566


0.4 Relationship with other management system standards 567

568

It is important to note that ISO Standard 9001:2015 was developed following the same high-level 569

structure used in all ISO management systems standards. This structure (10 clauses with the same 570

headings) facilitates the integration between different standards enabling NRAs to develop an integrated 571

management system (IMS) if they wish to implement other management system standards. The 572

following standards may be of interest to NRAs. 573

574

• ISO Standard 37001:2016 anti-bribery management systems (24) 575

• ISO/IEC 27001:2013 information security management systems (25) 576

577

ISO Standard 9004:2018 (Managing for sustained success of an organization) (9), provides guidelines 578

which NRAs may use for initiating improvements in the QMS. 579

580

Clause 1. Scope 581

582

The scope explains the purpose of the standard. This clause states that the ISO 9001:2015 requirements 583

are for a QMS, and not for products or services. It also indicates that ISO Standard 9001:2015 is 584

intended to be generic and applicable to all organizations, regardless of their type, size, or the products 585

and services they provide. 586

587

By implementing this standard, the NRA can demonstrate its ability to consistently provide products 588

and services that meet customer, statutory and regulatory requirements and can enhance customer 589

satisfaction. This guideline aims to provide guidance to adapt the ISO Standard 9001:2015 590

requirements to the needs of NRAs with respect to all of its regulatory functions, including the 591

supporting processes. 592

593

Clause 2. Normative references 594

595

ISO Standard 9000:2005 - Quality management systems- Fundamentals and vocabulary is an integral 596

part of ISO Standard 9001:2015 and is the source for the definitions of the terms used in ISO Standard 597

9001:2015. 598

599

Clause 3. Terms and definitions 600

601

As all terminology required for the use given in ISO Standard 9000:2005, no additional terms are 602


23

included here. Specific NRA-related terminology (not included in the ISO standard) is listed and 603

definitions are those provided in the relevant WHO documents. 604

605

The ISO 9000 family of standards uses generic terms to describe the relationship between the parties 606

involved. For the purposes of this guideline the term “organization” means NRA. “External providers” 607

are people or companies from whom the NRA receives products and services (e.g. suppliers). 608

“Customers” are people or organizations who receive products and services from the NRA. 609

610

ISO’s “Online Browsing Platform” can be used to search for information on terms and definitions 611

included in ISO 9000:2005, see: https://www.iso.org/obp/ui/ 612

Clause 4. Context of the organization 613

614

4.1 Understanding the organization and its context 615

616

Guidance 617

618

The intent of this clause is to understand the external and internal issues relevant to the NRA’s purpose 619

and strategic direction that can impact its ability to achieve the planned quality objectives of its 620

QMS. 621

622

There are many sources of information about internal and external issues that can affect the effective 623

implementation of the QMS. The issues are categorized as either related to statutory or regulatory 624

requirements. Statutory issues are considered for both internal and external cases. They provide the 625

boundaries within which the QMS can be implemented while complying with national laws (pieces of 626

legislature). Regulatory issues relate to professional regulatory bodies for personnel, materials, 627

environmental, financial and other areas that affect internal and external implementation of the QMS. 628

There are many sources for information about external and internal issues, such as internal documented 629

information and meetings, national and international press, websites, publications from national 630

statistics offices and other government departments, professional and technical publications, 631

conferences and meetings with relevant agencies, meetings with customers and relevant interested 632

parties, and professional associations. 633

634

External and internal issues can change, and therefore should be monitored and reviewed. The NRA 635

can conduct reviews of its context at planned intervals and through activities such as management 636

review. 637

https://www.iso.org/obp/ui/


638

An NRA must /should understand the context to provide the foundation for determining the scope of its 639

QMS, quality policy, quality objectives, and risks and opportunities. 640

641

Practical help box 1. Guidance to assist in the interpretation of clause 4.1 642

643

Understanding the NRA and its context 644

645

NRAs can be organized according to different models: 646

647

• In the centralized model, all regulatory functions are under the same organization and TM. 648

• In the decentralized model, a central office is generally located in the capital city and subsidiary offices 649

in states or provinces. The roles and responsibilities of these offices can be different; some functions may be 650

carried out at central level while others are delegated to the decentralized offices. 651

• In the discrete model, different institutions are responsible for different regulatory functions. Each of 652

them reports independently, usually to the Ministry of Health. 653

654

The specific characteristics of the NRA in question must be carefully analyzed when considering the context of 655

the organization. WHO concluded through discussions with the drafting group, that independently from the 656

organizational structure of the NRA, each institution involved in the regulatory oversight of medical products 657

should establish their own QMS in accordance with their specific processes. 658

659

Examples of internal/external issues 660

661

Internal issues to be taken into consideration include: 662

663

• resource factors, including infrastructure, governance, environment for the operation of the processes, 664

organizational knowledge, workforce and financial considerations; 665

• human aspects such as competence of persons, organizational culture and values, relationships with unions; 666

• operational factors such as process capabilities, performance of the quality management system, customer 667

evaluation; and 668

• factors in the governance of the organization, such as rules and procedures for decision making or 669

organizational structure. 670

External issues to be taken into consideration include: 671

672

• macro-economic factors such as money exchange rate predictions, economic situation, inflation forecast, 673

credit availability; 674

• political factors such as political stability, public investments, local infrastructure, international trade 675


25

agreements; and 676

• technological factors such as new sector technology, materials and equipment, patent expirations, 677

professional code of ethics. 678

679

NRAs can use tools such as strengths, weaknesses, opportunities and threats analysis (SWOT) and political, 680

economic, social, technological, legal, environmental analysis (PESTLE) to identify issues. Alternatively, simpler 681

approaches can be useful, depending on the size and complexity of the NRA’s operations, such as brainstorming 682

and asking "what if" questions. 683

684

4.2 Understanding the needs and expectations of interested parties 685

686

Guidance 687

688

The intent of this clause is to ensure that the NRA considers the requirements of relevant interested 689

parties, beyond just those of its direct customers. NRAs should focus only on parties that can have a 690

direct or indirect impact on the NRA’s ability to provide products, and services that meet requirements 691

customer’s and statutory and/or regulatory requirements. The NRA may consider external and internal 692

issues (decided under clause 4.1) for determining relevant interested parties. 693

694

The NRA should have a robust system in place to monitor and review the relevant requirements of its 695

interested parties at planned intervals. The information resulting from these activities should be 696

considered when determining the scope of the QMS (see 4.3) and for determining risks and 697

opportunities (see 6.1). 698

699


701

Interested parties and examples of their requirements 702

703

Examples of NRAs interested parties include: 704

705

manufacturers, researchers, sponsors for new product development, civil society, consumers, patients, healthcare 706

providers, distributors, exporters, importers, wholesalers, pharmacists, government partners (MOH, MOF, other), 707

parliament members and commissions, national and international pharmacopoeias, health system in general and 708

immunization program in particular, provincial NRA offices in the case of decentralized models, other institutions 709

with regulatory responsibilities as in the case of the discrete model. 710

711

712

713


Examples of requirements include: 714

715

availability of affordable medical products of assured quality, safety and efficacy, effective and efficient service, 716

legality, transparency, good communication skills, confidentiality, courtesy, compliance with laws, regulations 717

and requirements and responsiveness, good governance, impartiality, clarity, consistency and flexibility. 718

719

4.3 Determining the scope of the QMS 720

Guidance 721

722

The scope for the QMS should be established based on the following information: 723

724

• the external and internal issues as determined by the requirements of clause 4.1; 725

• the requirements of relevant interested parties (such as regulators and customers) as determined 726

in accordance with the requirements in clause 4.2; and 727

• the products and services provided by the NRA. 728

729

In determining the scope of the QMS, the NRA shall also establish the boundaries of the QMS by 730

considering issues such as: the infrastructure of the NRA; the NRA’s different sites/offices and 731

activities; and centralized, decentralized or externally provided functions, activities, processes, products 732

and services. 733

734

The NRA should carefully review each individual requirement within a clause to determine whether it 735

is applicable. Some or all the requirements in a clause may be applicable. NRAs should not decide a 736

clause is not applicable without careful consideration of each requirement. The documented scope 737

should include details of the products and services covered as well as justification for any requirements 738

determined to be not applicable. 739

740

The scope should be maintained as documented information using whatever method meets the NRA’s 741

needs, such as in the quality manual or a website. 742

743


745

Defining the scope of a QMS in an NRA 746

747

The NRA should determine the scope of the QMS based on the services to be provided, requirements of interested 748

parties, processes, infrastructure, and activities and resources available for each organization. The scope should 749

match the roles and responsibilities of the NRA and address all regulatory functions. In the case of a decentralized 750


27

or discrete NRA, if a certain institution is responsible for vigilance and another institution is responsible for 751

inspections; the scope for each institution should cover the services each one provides. Both institutions should 752

establish a QMS, preferably using the same standard, ideally ISO Standard 9001:2015. 753

754

Example of scope statement: 755

756

The Quality Management System at country X National Regulatory Authority (XNRA) covers all the procedural, 757

executive and supervisory functions of the X National Regulatory Authority in order to ensure the safety of food, 758

the safety and quality of the human and animal medicines, and the safety and efficiency of medical devices and 759

supplies through the establishment of an effective regulatory body in all sectors of the XNRA and all of its 760

branches in the country. 761

762

Excluding: 763

764

1. All the technical procedures and tests carried out in the laboratories, which will be covered through the 765

application of quality management system for the laboratories based on the international standards ISO Standard 766

17025, 767

768

2. All procedures for sampling during the inspection of establishments and at the ports of entry as well as 769

inspection tools used in this regard, will be covered by the implementation of the quality system of inspection ISO 770

Standard 17020. 771

772

4.4 QMS and its processes 773

774

Guidance 775

776

This clause provides a template for the process approach (PDCA). It focuses on the processes needed 777

for the QMS in accordance with ISO Standard 9001:2015 and the related documented information 778

needed. 779

780

When referring to the processes required by NRAs to carry out the different functions, it includes not 781

only the processes for service provision, but also the processes needed for the effective implementation 782

of the system, such as internal audits, management review and others (including processes that are 783

performed by external providers). 784

785

A process is a set of interrelated or interacting activities that use inputs to deliver intended results. 786

787

a) The NRA should determine the inputs required (what is required for the implementation of the 788


processes as planned) and the outputs expected from its processes (either by the customers or the 789

subsequent processes). Inputs and outputs can be tangible (e.g. materials, components or equipment) 790

or intangible (e.g. data, information or knowledge); 791

792

b) When determining and organizing the sequence and interaction of these processes, different methods 793

can be used such as process maps or flow diagrams (see figure 2 as example); 794

795

c) To make sure that processes are effective (i.e. deliver the planned results), the process control criteria 796

and methods should be determined and applied, criteria for monitoring and measurement can be process 797

parameters, or specifications of services; performance indicators related to quality objectives or other; 798

799

d) The NRA should determine the resources needed for processes such as people, infrastructure and 800

environment for the operation of the processes, organizational knowledge etc; 801

802

e) The NRA should assign the responsibilities and authorities for its processes by first determining the 803

activities of the process and then determining the persons who will perform the activity; 804

805

f) The NRA should ensure that any actions needed to address risks and opportunities associated with 806

the processes are implemented; 807

808

g) The NRA should analyse and evaluate monitoring and measuring data (see c above); and implement 809

any changes needed to ensure that these processes consistently achieve their intended results; and 810

811

h) The NRA can use the results of analysis and evaluation (see ‘g’ above) to determine the necessary 812

actions for improvement. 813


29

Figure 2. Example of interaction of processes 814

815

Business processes of the NRAs are shown in the centre of figure 2 (operation box) which includes four 816

processes of NRAs related to MA, VL, MC, & LR. Horizontally, the customer requirements are 817

captured to the left of the operation box and, to the right, delivery of products and services to the 818

customer is shown. Vertically, top management, at the bottom, provides its leadership and commitment 819

for QMS to achieve its intended results. Monitoring and measurement data of the processes/services 820

and customer feedback data, when analysed and evaluated, provide information on performance of 821

QMS. Output of performance evaluation can be used for initiating improvement of QMS/services. Two 822

vertical bars on left and right of the figure demonstrate that the QMS is based upon the context of the 823

organization and planning of QMS has been done based upon the context information. 824

825

MA, LR, VL and MC are used as models throughout this guideline; however, a similar approach can 826

be taken to address the other regulatory functions. Practical help boxes 4 to 8 and Table 2 illustrate the 827

processes for carrying out each of the above-mentioned functions. It is worth noting that it is not the 828

purpose of the process flows provided in figures 3 to 6 to represent a recommendation about the steps 829

required to exercise each of these functions. These are just provided as examples to explain the 830

relationship between the processes, the related inputs/outputs, monitoring points and established 831

controls, indicators used, resources needed including roles and responsibilities, authorities and risks and 832

opportunities for improvement described in the help boxes. Different NRAs can have different ways 833

of approaching the functions and, hence, the steps involved and the relationships between steps and/or 834

inter related processes may differ. 835

836



838

Characteristics of the processes and their interrelationships involved in the MA function 839

840

Figure 3. Processes and their interrelationships in conducting the MA function 841

842

843

Inputs 844

845

Laws, regulations, mandate, guidelines, access to laboratory for sample testing, market authorization applications, 846

site master files, outcome of inspections, including: good manufacturing practice (GMP), good clinical practice 847

(GCP), good distribution practice (GDP), others 848

849

Steps 850

851

Dossier screening, dossier evaluation (quality, safety, efficacy), laboratory analysis, inspection report, expert 852

committee review and decision making, final approval by Top Management and update of the list of registered 853

medicines. 854

855

Outputs 856

857

Screening results (checklist), dossier acceptance letter, dossier review reports, rounds of questions to the 858

manufacturer and other pertinent communications, test results, inspection reports and certificates. Update of 859

database, committee decision and MA approval or rejection. 860

861

862


31

Main processes 863

864

Receipt of application, screening, evaluation, including cycles of questions and responses, granting MA or 865

rejection 866

867

Interacting processes 868

869

Laboratory analysis, GMP inspection, review by expert committee, email communications, meeting minutes, IT 870

platform, official files, letters, meetings (expert meetings) 871

872

Examples of criteria and methods in place to ensure effective operation and control of processes: control 873

points and performance indicators 874

875

Criteria that must be monitored to ensure that processes are properly executed will be based on the criticality of 876

the processes and steps of the processes. Guidelines and SOPs will define elements such as the target evaluation 877

timeframe. Control points will be defined, and indicators chosen in such a way as to allow to monitor parameters 878

of performance. 879

880

Control points 881

882

Screening and dossier review 883

884

Performance indicators 885

886

Key performance indicators (KPI) that measure the actions and events that lead to a result as well as the frequency 887

of the evaluation must be established. The KPIs, particularly if they are carefully developed, represent an excellent 888

tool to monitor performance of the NRA. When setting KPIs use a quantitative method whenever possible and 889

determine appropriate numerators and denominators. [27]. 890

891

Examples of performance indicators include 892

893

Percentage of applications that have been screened within the specified timeline. 894

895

Compliance with defined review timeline 896

897

Quality of the evaluation reports, e.g. evaluation report assessed by three evaluators of different level of seniority 898

yields similar results 899

900

Number of new products listed in the register in a year in relation to the number of applications received 901

902


Potential indicators for other possible control points 903

904

Compliance with overall timeline for registration 905

906

Customer satisfaction evaluated through complaints, surveys, questionnaires, percentage of approved appeals, 907

others 908

909

Use of internal audits to assess performance 910

911

912

913


33


915

Characteristics of the processes and their interrelationships involved in the VL function 916

917

Figure 4. Processes and their interrelationships involved in conducting the VL function 918

919

920

Example of inputs, steps and outputs of processes and processes interrelationships 921

922

Inputs 923

924

Information received from patients, health professionals, international vigilance (VL) networks, industry, media, 925

risk management plans, clinical trials or PMS, suspect product, adverse event reporting, risk management plan, 926

post- market surveillance, clinical trial data. 927

928

Steps 929

930

Receipt, analysis, conclusion, reporting, feedback 931

932

Outputs 933

934

Communication of outcome (positive or negative), regulatory measures, alerts, recalls, risk minimization plan, 935

medical product information provided to patients, health professionals, international VL networks, industry, media 936

and feedback to reporting source 937

938

939




942




of performance. 946

947

Criteria for monitoring performance can include 948

949

• Structural indicators should measure systems and physical infrastructure; 950

• Assessments/evaluations/reviews should be timely (according to severity of signals); 951

• Evaluation should address all relevant aspects of the VL system (quality of the evaluation); 952

• The evaluation strategy should include outcomes that can be realistically measured, to avoid inaccurate or 953

misleading data; 954

• Indicators should provide an assessment of current PV documentation and resource compliance with 955

regulatory VL expectations and requirements. 956

• KPI should be re-evaluated to assess their relevance as indicators, and targets can be re-set when deemed 957

appropriate. 958

• As a consequence of monitoring VL System performance, corrective and preventive measures must be 959

implemented, resulting in continuous improvements to the VL System. 960

961

Control points 962

963

Triage/prioritization, data collection and verification, coding of adverse event descriptions, 964

quality of case causality assessment, timeliness, dissemination. 965

966


968


of the evaluation must be established. When setting KPIs use a quantitative method whenever possible and 970


972


974

Examples of performance indicators include the Number of vigilance inspections performed against planned based 975

on prioritization criteria for inspection. 976

977


35

Number of Adverse Drug Reaction reports received from healthcare professionals, from the media (data collection 978

mechanisms). 979

980

Percentage of fatal adverse drug reactions analyzed within target timeline, percentage of serious adverse drug 981

reactions analyzed within target timeline. 982

983

Number of complaints addressed vs. total number of complaints received by the VL department. 984

985

Number of recalled products “controlled” vs recalled products. 986

987

Internal audit findings. 988



Characteristics of the processes and their interrelationships involved in the MC function 990

Figure 5. Processes and their interrelationships involved in conducting the market surveillance 991

and control (MC) function 992

993

994

995

996

997

998

999

1000

1001

1002

1003

1004

1005

1006

1007

1008

1009

1010

1011

1012

1013

1014

1015

1016

1017

1018

1019

1020

1021

1022

1023

1024

Yes

No

Start

Medical Product in Market

Sampling

Testing

Identify SF

End

Sampling Plan

Market Complaint

Inspection report

GDP

Control of import/ export

Internet Sales

Test

report

Control

point

Identification of Products & Personnel

involved in SF

Involvement of NRA, Intelligence,

Enforcement, Police, Whistle Blow,

Manufacturer, Distribution, Retailer

Recall &

Reconciliation of

Quantity

Communication of SF to all Stake

holders: NRA, All within NRA, MAH,

Supply chain, User (Patient), Healthcare

professional, Other NRA, Public domain

Disposal of Recalled

products

Control

point

Control

point

Summary

report

Outcomes of MC

activities shared with

NRAs and other

stakeholders

Report of defective

product/ Rapid Alert

Notification

Control

point


37

MC requires the NRA (in collaboration with other relevant authorities e.g. customs) to ensure that substandard 1025

and falsified (SF) products do not enter/or are removed from the national market. It mandates the NRA to ask for 1026

all transactions relating to importation and/or exportation of consignments of medical products to be conducted 1027

by licensed entities and that good storage and distribution practices be followed. 1028

1029

Example of inputs, steps and outputs of processes, processes interrelationships 1030

1031

Inputs 1032

1033

Market complaint, market intelligence, inspection reports, sampling plan outcome, feedback from import and 1034

export activities and internet pharmacy. 1035

1036

Steps 1037

1038

Risk-based sampling, testing, identifying SF, decision on recall and communication to all stakeholders, including 1039

all relevant parties within NRA, market authorisation holder, supply chain, health care professionals, patients, 1040

international organisations others. 1041

1042

Outputs 1043

1044

Identification of SF, alerts, recalls, communication to all stakeholders and database for the SF. 1045

1046



1049

Control points 1050

1051

Sampling and testing, identification of SF, recalls and related reconciliation and effective communication to 1052

stakeholders 1053

1054


1056


of the evaluation must be established. When setting up KPIs use a quantitative method whenever possible and 1058


1060


1062

Number of consignments received through the port of entry. 1063

Number of samples drawn against planned. 1064


Number of samples sent for testing and number tested. 1065

Time taken to generate test report against the target timeline. 1066

Time taken to evaluate suspected products against the target timeline. 1067


39


1069

Characteristics of the processes and their interrelationships involved in the LR function 1070

1071

Figure 6. Processes and their interrelationships involved in conducting the LR function 1072

1073

1074

Inputs 1075

1076

Cover letter, summary lot protocol (SLP), samples, marketing authorization specifications, information on adverse 1077

events, surveillance data (test results on samples retrieved from the market). 1078

1079

Steps 1080

1081

Screening documents, request testing, perform testing, evaluation of SLP and testing results, decision, refer for 1082

review by technical committee. 1083

1084

Outputs 1085

1086

Notification of rejection, lot release certificate. 1087

1088

1089

1090


Main processes 1091

1092

Screening documents (cover letter and SLP), evaluation of SLP, review by the technical committee, decision-1093

making process. 1094

1095

Interacting processes 1096

1097

Sample testing can be considered an interacting process if this is contracted out to a third-party laboratory. 1098

Otherwise, it is a process that must be performed to yield an output. 1099

1100



1103




of performance. 1107

1108

Control points 1109

1110

Evaluation of the SLP. 1111

Expert committee review (of the report). 1112

1113


1115


of the evaluation must be established. When setting KPIs use a quantitative method whenever possible and 1117


1119


1121

Compliance with evaluation timelines. 1122

Check inputs in laboratory information management system (LIMS), checklist, outputs - percentage of outputs 1123

verified/validated in quality review. 1124

Trend analysis for test results. 1125

Percentage of timely reviews by the Expert review committee. 1126

1127

1128

1129


41

Potential indicators for other possible control points 1130

1131

Customer satisfaction evaluated through complaints, surveys, questionnaires. 1132

Use of internal audits to assess performance. 1133

1134

Practical help box 8. Guidance to assist in the interpretation of clause 4.4 relating to resources 1135

1136

Resources, roles and responsibilities, and authorities required to ensure adequate performance of processes 1137

for delivery of quality services by NRAs (common to all functions). 1138

1139

Resources, roles and responsibilities, authorities 1140

1141

Human resources should be allocated in line with the processes to be executed as well as the workload. Each 1142

employee has a job description and needs to be trained and qualified to perform his/her job. Roles and 1143

responsibilities and lines of authority should be detailed in the job description and organizational chart. Each 1144

process should have appropriate staffing and managers responsible for it. Staff performance, including 1145

performance of managers, should be evaluated regularly and re-training provided as needed. 1146

1147

Human resources 1148

1149

May include receptionist, administrative staff, screening officer, case investigation experts, evaluators, leadership, 1150

process supervisors, laboratory analysts, IT staff, human resources staff, expert committee members, regulatory 1151

inspectors, housekeeping staff, driver, other. 1152

1153

Proper infrastructure should be in place to carry out the activities (processes), e.g. if adequate laboratory 1154

infrastructure is not in place, consider contracting the service of a qualified laboratory. 1155

1156

Examples of aspects to be considered in terms of infrastructure, including facilities, IT, financial resources, 1157

documentation and work environment. 1158

1159

Infrastructure 1160

1161

• Adequate work space 1162

• Equipment as needed 1163

• Fully established lab or access to a contracted laboratory 1164

• Means of transportation 1165

• IT system, computers and software, databases, archiving system 1166

1167

1168


Financial resources 1169

1170

Financial resources to buy appropriate equipment, secure its maintenance and procure consumables. Computer 1171

systems and databases need to be validated. Hiring and retaining a sufficient number of qualified staff requires 1172

competitive salaries 1173

1174

Documentation System 1175

1176

Required documents include: strategic direction, vision and mission, laws and regulations, quality policy, 1177

guidelines, lot release policy, SOPs, forms, instructions and checklists. The NRA should establish a system for 1178

documentation preparation, review and approval as well as documentation control, revision and recordkeeping 1179

(see also guidance under clause 7.0). 1180

1181

Work environment 1182

1183

Social, physical and psychological factors all contribute to establishing an environment conducive to quality work; 1184

e.g. non-discriminatory, non-confrontational, stress-reducing, physically comfortable (lighting, temperature, 1185

ventilation, others). 1186

1187

During the planning stage, the NRA should address risks and opportunities in accordance with the 1188

requirements set forth in 6.1. 1189

1190

Table 2. Risks and opportunities affecting MA, LR, VL and MC 1191

1192

NOTE: Text is presented in plain format below to ensure it has line numbers consistent with the rest of 1193

the document, thereby facilitating use of the comment form during public consultation. The table will 1194

be appropriately formatted in the final stages of guideline development. 1195

1196

Marketing authorization 1197

1198

Transparency of NRAs and their work is one of the principles of GRP. Posting as much information as 1199

possible on the internet helps NRAs increase transparency. This information can include the 1200

registration procedure steps and timelines, related regulations and guidelines, charts indicating actual 1201

level of compliance with the target timelines, evaluation reports, others. Posting sensitive information 1202

on the web (e.g. performance charts) constitutes a risk of potential complaints or criticism, but at the 1203

same time offers opportunities for improvement, advocacy of the work performed, reliability, others. 1204

1205


43

The evaluation process should be properly monitored and evaluated, including the experts who conduct 1206

the evaluation. Failure to do so can lead to the risk of granting a MA based on an insufficient or 1207

inadequate data package. Risks usually also entail opportunities. In this example, if an NRA does not 1208

have the adequate expertise / resources to evaluate a certain product, reliance on other agencies can be 1209

an option. 1210

1211

Lot release 1212

1213

Tests to be performed as part of lot release must be appropriately validated, including the equipment 1214

used (properly calibrated), the consumables properly tested, released and used before expiry, qualified 1215

analysts to perform the tests who are regularly re-qualified and test performance monitored. Failure to 1216

meet all these requirements lead to the risk of either releasing a lot that does not meet the requirements 1217

or rejecting a lot that meets the specifications. Identification of the specific constraints may also bring 1218

about opportunities to improve the planning for procurement of consumables or equipment that may be 1219

required. 1220

1221

IT failures pose a risk for timely delivery of the service (release of lots), it raises at the same time an 1222

opportunity for renewing the system (hardware/software). 1223

1224

Vigilance 1225

1226

VL function carries a risk of potentially missing out on important signals because of underreporting or 1227

poor analysis and interpretation of the reports. The consequence is harm to the public and loss of 1228

reputation for the NRA. Lack of, or poor communication about, the safety of a product that has been 1229

suspected (as a result of analysis of the signals) may result in public panic and loss of trust on the NRA. 1230

Such failures can offer at the same time an opportunity for improvement and strengthening of the system 1231

1232

There is an increase in the number of registered new medicines (biologicals, biosimilars, others) so that 1233

a robust vigilance system needs to be in place. To establish a robust VL system, a database is developed 1234

to monitor implementation of all the approved Risk Management Plans (RMPs) and to measure their 1235

effectiveness. 1236

1237

The data base is also used for all medicines subject to additional post-marketing monitoring to track 1238

potential safety concerns. Vigilance inspection is one of the tools used to monitor and maintain the VL 1239

system within local companies and agents. 1240

1241


Market surveillance and control 1242

1243

• Risk of failure to test the sampled products due to limited capacity for testing e.g. reagents, 1244

standards, staffing, other. This may lead to failure in identifying products that have been damaged 1245

in the distribution/ storage chain. This may adversely impact the reputation of the NRA. At the 1246

same time, such an event provides an opportunity to convince TM of the constraints and the need 1247

for resources to prevent a recurrence. 1248

• Lack of expertise for SF case investigation poses a serious risk of missing SF products. It provides 1249

an opportunity to establish or review methods and training of staff in medicine production facilities. 1250

• Unclear communication strategy may lead to mis-communication or missing communicating an SF 1251

to certain relevant stakeholders. This may adversely impact the reputation of the NRA. It provides 1252

an opportunity to review the procedures and personnel responsible for communication of such 1253

events. 1254

• In case a recall of a product is mandated, there is a risk not to recall and dispose the whole batch. 1255

Recall from remote areas may be challenging. It provides an opportunity to empower the national 1256

vigilance system, involve and commit other institutions in the dissemination of regulatory measures 1257

and recall products that are damaged or do not meet the required quality standards or are SF. 1258

1259

Documented information for QMS1260

1261

The strategic direction of the NRA, mission, vision, policies, quality objectives, as well as procedures 1262

and other information, should be documented (4.4.2). In addition to this, NRAs will also need 1263

Documented Information to support its operations. As per definition, documented information is 1264

information required to be controlled and maintained by NRAs. 1265

1266

At several places ISO Standard 9001:2015 requires: 1267

1268

a) maintaining documented information (which means a manual, procedure, instruction, checklist, 1269

vision/mission/policy/objectives statements, guidelines, specifications, drawings, websites, circulars, 1270

government orders etc.); and 1271

1272

b) retaining documented information (which means records, reports, minutes of the meetings or 1273

any document which provides evidence that the activity has been performed as per applicable 1274

criteria/methods etc.). 1275

1276

Some of the documented information to be retained may be formal (validation reports, audit reports, 1277


45

others) and other informal (meeting minutes). NRAs should have such documented information (see 1278

clause 7.5 for details of DI). 1279

1280

Practical help box 9. Guidance for interpretation of clause 4.4 relating to documented information 1281

1282

High-level NRA documentation can include the legal basis, regulations, decrees, strategic plan, vision and 1283

mission, overall objectives of the organization, quality objectives, quality policy, quality manual, others. 1284

1285

Lower-level NRA documents can be divided into two categories: documents and records. 1286

1287

Examples of documents include SOPs, instructions and forms and checklists. Examples of records include 1288

assessment reports, inspection reports, test results, application submissions from manufacturers, marketing 1289

authorization dossiers, SLPs, correspondence, trending data and its analysis, validation and qualification protocols 1290

and reports, calibration data, training plans and records, analysts and evaluators qualification information, 1291

maintenance program, training program and records, internal audit plans and reports, corrective and preventive 1292

action (CAPA) plans and compliance reports, others. 1293

1294

Documentation and records must be controlled. A system should be in place whereby documents are reviewed, 1295

authorized and approved. Newer versions replace older ones which become obsolete. Documentation should not 1296

only be maintained, but also retained for established periods of time, which are defined by each authority 1297

according to established rules (generally not less than five years for some documents and not less than 10 years 1298

for others). A documentation control system should be established to ensure that all relevant areas have the 1299

required documented information and that only the latest (current) version is available at any point in time. (See 1300

also clause 7.0). 1301

Clause 5. Leadership 1302

1303

5.1 Leadership and commitment 1304

1305

Guidance 1306

1307

The intent of this clause is to ensure that NRA TM demonstrate leadership and commitment by taking 1308

an active role in engaging, promoting, communicating and monitoring the performance and 1309

effectiveness of the QMS. 1310

1311

The clause requires TM to demonstrate leadership and commitment with respect to the QMS. To 1312

achieve this, TM should comply with conditions (a-j) listed in the standard. In brief, TM is accountable 1313

for the effectiveness of the QMS and its integration into the core processes of the NRA by supporting 1314

the critical characteristics of the QMS as defined and described in ISO Standard 9001:2015. This 1315


includes promoting the use of the process approach through the PDCA cycle and risk-based thinking to 1316

ensure that the quality policy and quality objectives are compatible with the context and strategic 1317

direction of the NRA; ensuring that the required resources are available to support staff to contribute to 1318

the effectiveness of the system; supporting other managers in their respective roles to demonstrate their 1319

leadership; and promoting improvement. Experience shows that leadership and commitment are 1320

essential requirements for successful implementation of a QMS. 1321

1322

An important element of the ISO Standard 9001:2015 standard is its emphasis on customer focus. In 1323

this respect, TM should demonstrate its leadership and commitment to the QMS by continually 1324

identifying the needs and expectations of its customers (Drug importers, Drug manufacturers, Drug 1325

outlet operators, Patients, Medical Practitioners, Research institutions etc), as well as ensuring that the 1326

NRA fulfils applicable statutory and regulatory requirements. 1327

1328

In many cases, a focus for on-time delivery performance and on customer complaints can provide 1329

information on any actions that might be necessary to achieve or improve customer satisfaction. 1330

1331

The NRA also needs to ensure that appropriate actions are implemented to address risks and 1332

opportunities that can affect customer satisfaction. To increase customer satisfaction, innovation and 1333

best practices can be introduced into the NRA processes. 1334

1335

5.2 Quality Policy 1336

1337

Guidance 1338

1339

Two key aspects are covered in this clause, namely the development of a quality policy and 1340

communicating the quality policy to the NRA personnel. 1341

1342

The quality policy is a powerful and highly visible statement of intent towards quality services by the 1343

TM signed by the Director or Head of the NRA (in the case of a discrete NRAs, this responsibility may 1344

fall in the hands of the Ministry of Health). 1345

1346

While establishing a quality policy, the NRA TM should keep in view its purpose and strategic direction 1347

(mission, vision, guiding principles and core values). Good regulatory practices (see 2.0 General 1348

considerations of this guideline) and quality management principles (see clause 0.2) can also be used 1349

for establishing commitment of the NRA TM towards quality services. 1350

Two commitments should come out clearly in the statement of quality policy: 1351


47

1352

• Commitment to satisfy customer or stakeholders requirements as well as applicable statutory and 1353

regulatory requirements; and 1354

• Commitment to continual improvement of the QMS. 1355

1356

The policy should also provide a framework for setting quality objectives (which means any claims in 1357

the quality policy should be measurable when converted into objective). 1358

1359

TM should ensure that the quality policy is communicated, understood and applied by persons of the 1360

NRA, so they are able to contribute to the effectiveness of the QMS. The policy can be communicated 1361

by different methods such as via noticeboards, screensavers, by the organization’s website, or during 1362

routine meetings. In addition, the NRA can make the quality policy available, as appropriate, to relevant 1363

interested parties such as external providers (service providers/suppliers), partners, customers and 1364

governmental agencies, for example, by displaying it on NRA’s website. 1365

1366

Practical help box 10. Guidance for interpretation of clause 5.2 1367

1368

Example of quality policy for NRAs from countries X, Y and Z. 1369

1370

1) Country X National Regulatory Authority (XNRA) quality policy. (Approved by XNRA management) 1371

1372

XNRA is committed to meet the needs and expectations of customers through continual improvement of its 1373

processes and quality services by implementing QMS effectively according to ISO Standard 9001:2015 1374

requirements. We will ensure quality, safety and/or efficacy of food, medicines, cosmetics and medical devices 1375

in compliance with the XNRA Drug Act 1:2006. We should establish objectives at system and departmental level 1376

that ensure that the requirements of this policy are met. Top Management is committed to providing the necessary 1377

resources to ensure maintenance and continuous improvement of QMS. 1378

1379

Executive Director signature 1380

“Together we protect public health” 1381

1382

2) The YNRA is committed to protect the health of people of the country and fulfil its duties with professional 1383

and scientific rigor, while ensuring safety, efficacy and quality of Allopathic, Homeopathic and Herbal 1384

medicines, vaccines, and biological products according to the Drugs Act and Rules and future amendments. 1385

1386

YNRA should work in effective, transparent and timely manner, ensuring implementation of Quality Management 1387

System and to ensure its continuing improvement. 1388

To meet our commitment, we must: 1389


1390

— Foster a team approach. 1391

— Emphasize appropriate training for all employees. 1392

— Recognize each employee's responsibility for quality. 1393

— Provide regulations with timely written corrective actions. 1394

— Earn recognition of our quality process and progress. 1395

— Provide a framework for establishing and reviewing quality objectives. 1396

— Develop and achieve Quality Improvement Goals. 1397

— Maintain our honesty and integrity by following our Code of Conduct. 1398

— Review and renew this Quality Policy on a regular basis. 1399

1400

3) “ZNRA is committed to provide quality services in response to customer needs and expectations. We should 1401

strive to balance the interests of our stakeholders without compromising quality, safety and/or effectiveness 1402

of food, drugs, cosmetics and medical devices by managing the Authority with utmost professionalism. We 1403

commit ourselves to comply with requirements of the ISO 9001:2008 standard and continually improve 1404

effectiveness of Quality Management System. We should manage and provide resources for continuous 1405

improvement of our services to ensure customer satisfaction”. 1406

1407

5.3 Organizational roles, responsibilities and authorities 1408

1409

Guidance 1410

1411

The NRA TM will need to establish specific responsibilities and authorities for the assigned roles and 1412

ensure that persons of the NRA understand and are aware of their assignments. These could be 1413

communicated through job descriptions, work instructions, duty statements, organization charts, 1414

manuals, procedures, others. Adequate resources are required to match the needs. 1415

1416

Items a) and b) in the clause - the QMS conforms to the requirements of the ISO Standard 9001:2015 1417

[7] and processes are delivering the intended outputs - describe roles to be assigned to each process 1418

owner (managers), while items c) to e) in the clause - reporting on QMS performance, promoting 1419

customer focus and maintaining the integrity of the QMS when changes are made - describe roles to be 1420

assigned to specific persons. Although certain responsibilities are delegated, the overall responsibility 1421

and accountability for the QMS remains with TM. 1422

1423

1424

1425



49

1427

Ideally, NRAs will have a QMS unit in place, or a responsible officer as a minimum within each Unit, Department 1428

or Directorate, as management representative or QMS coordinator who can report on QMS performance, promote 1429

customer focus and maintain the integrity of the QMS when changes are made. 1430

1431

Responsibility for ensuring that the QMS conforms to the requirements of the ISO Standard 9001:2015 and 1432

processes are delivering the intended outputs should be included in staff job descriptions and assessed during 1433

performance evaluation. Staff should be trained in QMS (conferences, meetings, online platform). Trainings 1434

should be relevant to the regulatory functions and reflected in documented information. 1435

1436

Example of country YNRA 1437

1438

The Deputy Director of the agency has been designated as the representative of the agency in quality management. 1439

His responsibilities and authority include: 1440

1441

• To ensure that the necessary processes for quality management are established, implemented and maintained. 1442

• To inform senior management of system operation, including the needs for improvement. 1443

• To promote awareness of customer requirements at all levels of the organization. 1444

1445

The responsibility of the management representative includes relationships with external parties on matters related 1446

to the system. He is the designated Quality Assurance Manager. 1447

1448

Clause 6. Planning 1449

1450

6.1 Actions to address risks and opportunities 1451

1452

Guidance 1453

1454

The intent of clause 6.1 is to ensure that when the NRA plans its QMS processes, it identifies its risks 1455

and opportunities and plans actions to address them. The purpose of this clause is to prevent 1456

nonconformities, including errors in outputs, and to determine opportunities that might enhance 1457

customer satisfaction or achieve quality objectives. 1458

When determining risks and opportunities, the NRA should focus on enhancing desirable effects, 1459

preventing or reducing undesired effects (through preventive actions or risk reduction). This is adopting 1460

a "risk-based approach" and the NRA should consider the application of this approach to all processes 1461

required for its QMS. 1462


The NRA can choose the methods of risk determination that suit its needs. The simpler approaches 1463

include techniques such as structured brainstorming, "what if?” method, consequences/probability 1464

matrices, etc. For guidance, the NRAs may refer to the international standard ISO/IEC 31010 that 1465

provides a list of risk assessment tools and techniques. 1466

1467

When examining opportunities, potential risks to the QMS associated with them should also be 1468

determined; the results of such determinations should be used when making decisions on whether to 1469

implement the opportunities. 1470

1471

The application of risk-based thinking can also help the NRAs to develop a proactive and preventive 1472

culture focused on doing things better and improving how work is done in general. 1473

Once the risks and opportunities are identified, actions must be planned to address them. Actions are 1474

planned, implemented, analysed and evaluated to assess their effectiveness. 1475

The actions taken to address risks will depend on the nature of the risk (its probability/frequency and 1476

severity), for example: 1477

a) The risk can be avoided by no longer performing the process where the risk can be encountered 1478

(risk is terminated). 1479

b) The risk can be eliminated by assisting persons in the organization with less experience or by 1480

capacity building (risk is treated). 1481

c) The risk can be shared by out sourcing the process or taking an insurance cover (risk is transferred). 1482

d) The risk can be accepted, and no action taken, based on its potential effect or the cost of the needed 1483

action (risk is tolerated). 1484

1485

The above alternative actions are also termed as 4T (terminate, treat, transfer or tolerate) methods of 1486

treating the risks. 1487

1488


1490

Example of actions to address risks and opportunities for lot release process 1491

1492

It is foreseen that there will be an increased demand for release of batches of a certain vaccine the following year. 1493

The analysis of risks and opportunities to meet the increase in demand includes an assessment of the current 1494

situation (process capacity), an analysis of the risks of attempting to meet the demand under the present conditions, 1495

and the opportunities that arise from this new situation. The release process requires analysts to test the vaccine 1496

batches, reviewers to go through the SLP, and professionals to prepare the report to be reviewed by the Technical 1497


51

Committee, plus the final approval by the Head of Agency. The risks associated with addressing the increased 1498

demand include the fact that testing as well as review capacity may not be sufficient, and since the Technical 1499

Committee meets only once per month, the response capacity may not be enough. 1500

1501

As part of the planning process, the team needs to assess the risk of releasing lots that do not meet specifications 1502

on one hand; and the risk of not releasing a lot that meets the specifications due to limited capacity on the other. 1503

In addition, the likelihood (probability) of failure in service providing, the impact on the quality of the products 1504

released, the estimated frequency at which errors could happen, the potential impact on customer’s satisfaction 1505

and the credibility of the institution (seriousness) in case of not meeting the increased demand, or in case the 1506

service provided is inadequate, should all be considered. 1507

1508

The analysis of risk and opportunities provides projections for changes to be introduced in the system to effectively 1509

and efficiently address the increased demand. For example, if surge in testing capacity cannot be implemented or 1510

is not economically feasible, a new prioritization mechanism based on knowledge of the different products to be 1511

released and record of the manufacturers could be put in place. 1512

1513

Through this analysis, it may be estimated that by increasing the staff by one analyst and increasing committee 1514

meetings to two per month, the demand could be appropriately addressed (opportunity for increased resources). 1515

1516

6.2 Quality objectives and planning to achieve them 1517

1518

Guidance 1519

1520

The intent of this clause is to ensure that the NRA establishes quality objectives and plans appropriate 1521

actions to achieve them. Quality objectives should be established for relevant functions, levels and 1522

processes, as appropriate, to ensure the effective deployment of the NRA’s strategic direction (plans) 1523

and its quality policy. Whenever possible, they should be SMART objectives (Specific, measurable, 1524

achievable, realistic and time bound. The following table provides guidance on implementation of 1525

bullets ‘a’ to ‘g’ of clause 6.2 1526

1527

Table 3. Guidance for development of quality objectives 1528

1529

Requirement Intent with example

a. Be consistent with the quality policy Use commitments made in quality policy for setting quality

objectives e. g. setting objectives on continual improvement of

QMS as committed in quality policy

b. Be measurable Define quantity or period e. g. processing time of customer

request will be reduced from 2 to 1 day


c. Address applicable requirements For example, if certain regulatory requirement relating to

product/service are applicable, setting objectives for that

Using WHO GRP for setting objectives

d. Be relevant to conformity of products

and services and enhanced customer

satisfaction

For example, ‘On Time and In Full’ delivery of service, setting

targets for achieving higher level of customer satisfaction

e. Be monitored

Means being reviewed for progress being made in achieving

the quality objective; this could be carried out through analysis

of process monitoring and customer feedback data and

comparing results with set targets

f. Be communicated For example, through circulation of minutes of meetings

internally and to external interested parties viz suppliers by

signing agreements

g. Be updated as appropriate Potential or actual changes that can impact on the ability to

achieve quality objectives need to be considered and action

taken as necessary, to ensure new issues or requirements are

addressed.

1530

A plan should be in place to ensure that the set objectives will be met. The planning exercise includes 1531

determining the actions that will need to be taken, the resources that will be required (e.g. human and 1532

financial to purchase equipment and the required supplies), assigning responsibilities to staff for specific 1533

tasks, determining timelines for completion of each step and deciding means to be used for measuring 1534

and evaluating whether the objectives have been achieved or not (see also clauses 9.1 to 9.3). 1535

1536


1538

Example of mission, vision and quality objectives for an NRA 1539

1540

XNRA mission statement 1541

1542

The mission of XNRA is to protect and promote public health by ensuring quality, safety and/or efficacy of food, 1543

medicines, cosmetics and medical devices. 1544

1545

XNRA vision statement 1546

1547

The vision of XNRA is to provide the best regulatory services to ensure the quality of food, drugs and cosmetics 1548

in the southern hemisphere by 2020. 1549

1550

XNRA quality objectives 1551

1552


53

The quality objectives of XNRA are established in line with the goals outlined in the XNRA strategic plan for the 1553

period of 2015-2020. These objectives are: 1554

1555

a) Maintain good governance and management of the agency with view at ensuring continuing improvement of 1556

QMS. 1557

b) Continuously improve the quality of service through regular training of staff, monitoring of performance and 1558

monitoring compliance with set review timelines. 1559

c) Strengthen laboratory services by conducting the validation of all test methods used and the qualification of 1560

staff involved in such tests by 2020. 1561

d) Strengthen cooperation and collaboration with relevant organizations and government agencies. 1562

1563

Planning to achieve quality objectives 1564

1565

The achievement of XNRA quality objectives should be through implementation of specific actions as detailed in 1566

the current XNRA strategic plan. 1567

1568

XNRA determines and provides resources (including human, financial, infrastructure, technology, work 1569

environment and organizational knowledge needed to establish, implement, maintain and continuously improve 1570

QMS. The resource requirements are defined through budgeting and other business management processes 1571

including planning and management review. 1572

1573

TM is ultimately responsible for quality of XNRA services by ensuring the resources, systems and processes 1574

needed to implement and improve QMS and for undertaking management review meetings. All employees are 1575

responsible for the quality of their work and implementation of the policies and procedures applicable to the 1576

processes they perform. 1577

1578

The quality objectives should be achieved by 2020 and will be evaluated by undertaking quality internal audits 1579

and analyzing performance data for continual improvement of the system with the overall aim of meeting 1580

customers’ needs and expectations. 1581

1582

Another example - ZNRA Quality objectives 1583

1584

Objective 1: The rate of counterfeit and substandard food, medicine, cosmetics and medical devices circulating 1585

in the country reduced by 50% by June 2020 1586

Objective 2: Customer satisfaction for services offered by ZNRA increased by 80% for both internal and external 1587

customers from 63% and 66% respectively by June 2020 1588

Objective 3: ZNRA self-sustained financially from 60% to 80% by June 2020 1589

Objective 4: 90% of human resources recruited and retained by June 2020 1590

6.3 Planning of changes 1591

1592


Guidance 1593

1594

The intent of this clause is to determine the need for changes to QMS to adapt to changes in the NRA’s 1595

context/business environment, as well as to ensure that any proposed changes are planned, introduced 1596

and implemented in a controlled manner. 1597

1598

The purpose of planning the change is to maintain the integrity of the QMS and ensure the NRA’s 1599

ability to continue to provide conforming products and services during the change. 1600

1601

The need for changes can result from, changing needs of customers and other relevant interested parties, 1602

new products to be evaluated to grant market authorization, changing process methods to improve 1603

trends in non-conforming outputs, using new information and communication technology (ICT) for a 1604

service or process, outsourcing important processes, persons in key roles leaving (either due to 1605

retirement, job change or other), or moving to online service provision. 1606

1607

The NRA should consider the availability of resources and necessary allocation or reallocation of 1608

responsibilities for any change. This could be done by assigning persons to a team to manage the 1609

change, or by delaying the change until the right resources are available. 1610

1611

Practical help box 14. Guidance for the interpretation of clause 6.3 1612

1613

In planning for an increased demand for lot release the following year, a risk analysis determines that the NCL 1614

will need to add one more SLP evaluator, increase the technical committee meetings to twice per month, and 1615

prioritize lot testing. These measures entail a change in the processes which must be planned, documented, 1616

integrated to the QMS and properly monitored. In this manner, the organization is considering the potential impact 1617

of the change, the availability of resources, and the allocation or reallocation of responsibilities thereby conserving 1618

the integrity of the QMS. 1619

1620

1621

1622

1623

1624

1625

Clause 7. Support 1626

1627

7.1 Resources 1628


55

1629

Guidance 1630

1631

The intent of this clause is to ensure that the resources necessary for the establishment, implementation, 1632

maintenance and continual improvement of QMS are available to the NRA for its effective operation. 1633

1634

In determining the resources that need to be provided, the NRA should consider the current capabilities 1635

of its internal resources (e.g. people, capability of equipment, organizational knowledge) and any 1636

constraints (e.g. budget, number of resources, schedule). A decision should then be made on the 1637

resources needed, including those to be sourced externally, and the necessary actions taken to ensure 1638

the resources needed are provided; this applies to all resources listed under sub-clauses of resources 1639

from 7.1.2 to 7.1.6 of the Standard. 1640

1641

Three important categories of resources need to be considered; people, infrastructure and environment 1642

for the operation of processes. To plan for adequate resources in quality and quantity, three steps are 1643

to be followed: 1644

1645

a) Determine what resources are needed (number of people and level of competence required, utilities, 1646

facilities, equipment including hardware and software needed as well as good working conditions; 1647

physical such as temperature control, level of lightening, etc and human conditions to ensure an 1648

adequate work environment). 1649

b) Plan how and when these are going to be provided. 1650

c) Plan for the means to ensure that the resources provided are maintained (periodic preventive 1651

maintenance) and controlled as needed. 1652

1653

Monitoring and measurement resources 1654

1655

The intent of the clause is also to ensure that the NRA determines and provides suitable resources 1656

(measurement equipment, instruments, etc.) to ensure valid and reliable monitoring and measuring 1657

results when evaluating the conformity of its products and services. 1658

1659

Monitoring implies critical observation, supervision and checks to determine the quantitative or 1660

qualitative status (or both) of an activity, a process, a product, or a service. Measurement considers the 1661

determination of a quantity, magnitude, or speed, by using suitable measuring resources. This can 1662

include the use of calibrated or verified equipment that is traceable to national or international 1663

measurement standards. For services, it can include the use of known and validated models for service 1664


feedback, for example social service models. 1665

1666

In determining the criticality of monitoring and measurements to ensure valid results, the NRA should 1667

determine what needs to be monitored and/or measured for its processes, products and services. It 1668

should then determine the resources needed for this monitoring and measuring; ensuring its 1669

suitability/fitness for the purpose. Resources used should be maintained for their continuing fitness. 1670

1671

Documented information to be retained can include schedules outlining how often checks are needed 1672

to ensure valid results as well as reports of results/outcomes. 1673

1674

Measurements need to be traceable (to national and/or international measurement standards) when it is 1675

a requirement or when the NRA determines it to be necessary to have confidence in the validity of the 1676

measurement results. 1677

1678

If measuring equipment is used to verify conformity to requirements and to provide confidence in the 1679

validity of measurement results, the NRA should consider how the measuring equipment is verified 1680

and/or calibrated, identified with calibration status, safeguarded from adjustments, stored, used and 1681

maintained. 1682

1683

If measuring equipment is found to be unfit for the intended purpose, the potential impact on compliance 1684

with measurement requirements should be reviewed and necessary actions taken. The results of such a 1685

review can also indicate that no action is required or, alternatively, that a service needs to be performed, 1686

products in stock need to be investigated, relevant customers must be informed, or even that a product 1687

recall is required. The level of action needed depends on the conformity of products and services. 1688

1689

Organizational knowledge 1690

1691

The clause also focuses on the need to maintain the knowledge determined as necessary, by the NRA, 1692

for the operation of its processes and to achieve conformity of products and services, as well as to 1693

encourage the acquisition of necessary knowledge based on changing needs and trends. 1694

1695

Organizational knowledge is the specific knowledge of an organization coming either from its collective 1696

experience or from the individual experience of its persons. This knowledge is or can be used to achieve 1697

the organization’s intended results. 1698

1699

The NRAs should consider how to determine and manage the organizational knowledge required to 1700


57

meet NRA’s present and future needs. Persons and their experience are the foundation of organizational 1701

knowledge. Capturing their experience and knowledge can generate synergies leading to the creation 1702

of new or updated organizational knowledge. 1703

1704

In determining, maintaining and making available organizational knowledge, NRAs can consider:a) 1705

learning from failures, near miss situations and successes;b) gathering knowledge from stakeholders, 1706

experts and partners;c) capturing knowledge that exists within itself. 1707

1708

The tools for maintenance and distribution of organizational knowledge can include the intranet, 1709

libraries, awareness sessions, newsletters, others. 1710

1711

Practical help box 15. Guidance for the interpretation of clause 7.1 relating to organizational 1712

knowledge 1713

1714

The NRA should safeguard the knowledge necessary for its operation and achievement of conformity of products 1715

and services. It should also encourage gaining new knowledge to meet its current and future needs. 1716

1717

Example of measures taken by the NRA of country Y to maintain and update organizational knowledge: 1718

1719

a) YNRA has developed, and updates as needed, detailed job descriptions of personnel responsible for key 1720

processes in the chain that leads to their outputs (products and services), 1721

b) YNRA carries out initial training of new staff and refreshment training of staff at different levels with 1722

new information relevant to their respective positions to keep their competence up to date, 1723

c) Recruitment of new staff is based on job description, the position posted on the website, and candidates 1724

subject to a test and interview before decision-making, 1725

d) In case of staff turnover, and whenever possible, an overlap between the staff leaving the position and 1726

the new staff is sought, so that knowledge is properly transferred to the new staff and opportunity is given to the 1727

incoming person to practice under advice of the person leaving the position, 1728

e) In case of retirement, succession is properly planned through timely recruitment of successor 1729

f) Organizational knowledge refers not only to processes for service delivery, it also includes the broader 1730

perspective (e.g. knowledge of mission, vision, quality policy and objectives, strategic plan and strategic 1731

objectives of the NRA, understanding the context, internal and external issues, customers’ expectations, statutory 1732

and regulatory requirements, relationship with customers and suppliers and with other relevant organizations or 1733

agencies, others). To ensure that this knowledge is maintained and properly communicated to personnel internally, 1734

YNRA organizes meetings at regular intervals where issues are discussed; a newsletter is produced and circulated 1735

through the intranet on monthly basis; in case of urgency email communications are sent to all relevant personnel, 1736


g) YNRA provides opportunities for training of staff outside the NRA, by participating in technical courses 1737

and through attendance to scientific meetings. Personnel that benefits from such opportunities are required to write 1738

a meeting or training report and to deliver a lecture to colleagues in the NRA for knowledge/ information sharing. 1739

h) Staff benefitting from fellowships abroad are required in addition to g) to stay in the NRA for a time 1740

equal to the double of the duration of the fellow ship. Monthly seminars are organized in which personnel share 1741

experiences from their work with others (e.g. a rejected market authorization application, information on an 1742

innovative product, feedback from the field regarding safety profile of recently registered and commercialized 1743

vaccines). 1744

1745

7.2 Competence 1746

1747

Guidance 1748

1749

The intent of this clause is to identify the necessary competence required to perform individual roles 1750

and responsibilities and to ensure persons carrying out work are competent, based on training, education 1751

and/or experience. The term ‘persons’ includes managers, existing employees, temporary employees, 1752

sub-contractors and their employees, and outsourced persons. 1753

1754

Competence is the ability to apply knowledge and skills to achieve intended results. Demonstrated 1755

competence is sometimes referred to as ‘Qualification’ or ‘licensed person’. 1756

1757

Competence requirements can be determined by, for example: 1758

1759

• Specified performance criteria 1760

• Awareness of specified requirements and acceptance criteria 1761

• Knowledge of processes and controls operated by the organization. 1762

1763

When a person does not meet, or no longer meets, the competence requirements, then actions should be 1764

taken; such as, for example: 1765

1766

• Mentoring the employee 1767

• Providing training 1768

• Simplifying the process so that the person can carry it out successfully 1769

• Reassigning the employee to another position. 1770

Evaluation of competence can be done in several ways, including: 1771

1772


59

• Regular supervisor or manager evaluation of persons performing tasks and the operation of 1773

processes; and 1774

• Benchmarking against service performance requirements. 1775

1776

Appropriate documented information that provides evidence of an employee’s competence includes, 1777

e.g. diplomas/degrees, completion of training, resumes, performance reviews and licenses should be 1778

retained. 1779

1780


1782

The practical help box 15 refers on point c) to the selection and recruitment of staff. The selection process for 1783

recruitment is critical to identify persons competent for the job, the requirements of the position, the acceptance 1784

criteria and the position specific knowledge are reflected in the job description published on YNRA website. The 1785

selection process includes a test and one or more interviews before a decision is made regarding the best candidate. 1786

This process is likely to successfully identify competent candidates. YNRA invests in maintaining competence 1787

of its employees, and in reducing turnover to the maximum possible extent. It has provisions to update knowledge 1788

of personnel through regular refreshment training, participation in scientific/technical meetings and other means. 1789

Competence records are kept as part of the process of acquiring competence. 1790

1791

In case a person no longer meets the requirements of the position, YNRA offers mentoring and retraining, and as 1792

a last option reassigns the person to a different position. There are instances where reassignment of a person to a 1793

different position is not the result of failure but of the initiative of the person to gain experience in a different area 1794

of expertise. Rotation of personnel is also a regular practice in YNRA to facilitate the acquisition of additional 1795

skills/ competencies and to provide incentives. 1796

1797

7.3 Awareness 1798

1799

Guidance 1800

1801

The intent of this clause is to ensure that persons are aware of the quality policy, relevant quality 1802

objectives, their contribution to the effectiveness of QMS and the implications of not conforming to 1803

QMS requirements. 1804

1805

Persons can demonstrate their awareness in day-to-day activities by distinguishing between what is 1806

acceptable and what is not, and by taking appropriate action when processes, products and services do 1807

not meet agreed specifications. 1808

1809


Depending on the nature of the work that the persons perform, the actions for creating awareness can 1810

vary. Awareness can be created through regular review meetings, gathering feedback and ensuring this 1811

feedback is made known to relevant persons. 1812

1813

NRA staff should be aware of the quality policy and objectives, their role and contribution to an 1814

effective QMS and the implications of not conforming to the QMS requirements. A common practice 1815

is for the TM to post the mission, vision, quality policy, and quality objectives in the entrance of the 1816

NRA or other strategic locations for all staff and customers to see. 1817

1818


1820

To ensure that personnel are aware of the quality policy and relevant objectives, ZNRA posts them at the entrance 1821

of the building, in every bathroom, library and every meeting room. In addition, ZNRA has distributed slogan 1822

bottoms stating “I adhere to QM policy and objectives” for every single worker in the organization. ZNRA also 1823

placed posters of the Quality Management principles and the benefits of the QMS in improving performance and 1824

implications to the health of the public if not conforming to QMS. ZNRA TM speaks in QMS terms. 1825

1826

7.4 Communication 1827

1828

Guidance 1829

1830

The intent of this clause is to establish the process of internal and external communications. NRAs 1831

needs to decide what needs to be communicated and who needs this information, to determine the most 1832

effective communication method and timing; including who provides the communication. 1833

1834

The NRA should identify those parties with whom they should communicate, to ensure the effective 1835

operation of the QMS, such as customers, suppliers, experts, ministry of health, media and other 1836

stakeholders. 1837

1838

More formal communication might be required for external interested parties, such as reports, invoices 1839

or service level agreements, press briefs, etc. Internal communications can use methods such as regular 1840

department meetings, briefing sessions, email or the intranet. More formal methods, such as written 1841

reports or minutes of the meetings etc., can also be required for internal communication, depending on 1842

the nature of the information and how critical the issues are that need to be communicated. 1843

It is also common for an NRA to designate a specific communication officer who has been trained on 1844

what, how, when, and to whom communicate depending on the matter. Communications outside the 1845


61

NRA, e.g. the media, should be carried out exclusively by communications-trained officers designated 1846

by NRA TM. 1847

1848


1850

Example of internal communication requirements at YNRA 1851

1852

As a minimum internal communication is done through the following channels: 1853

1854

• Meeting with Ministry-Joint Secretary (once in a quarter) 1855

• Senior Staff meetings (monthly) 1856

• Full Departmental meetings (weekly) 1857

• Daily operational meeting of the departments and administrative areas 1858

• Email, internet platform and/or telephone. 1859

1860

Example of communication with the customer at YNRA 1861

1862

Communication with customers is maintained through: 1863

1864

• Internet platform, fax, email or post 1865

• Surveys and interviews are used to obtain customers feedback 1866

• Meetings and exchanges between specialists and managers 1867

• Complaints and grievances are handled in quality management following the instructions for handling 1868

complaints and grievances. 1869

1870

7.5 Documented information 1871

1872

Guidance 1873

1874

The intent of this clause is to put the documented information into two categories; information that 1875

needs to be maintained and information that needs to be retained. The wording “maintain documented 1876

information” means the information contained in documented procedures, manuals, forms, checklists 1877

etc. The other examples are QMS scope statement, quality policy and quality objectives statements. 1878

These are popularly called ‘documents’. 1879

1880

The wording “retain documented information” means ensuring that information that is used to provide 1881

evidence about whether a requirement has been fulfilled needs to be kept/ retained. These are popularly 1882

called ‘records/ evidences. 1883


1884

In general, ISO Standard 9001:2015 is not prescriptive in terms of the extent of documented information 1885

needed. This will vary from organization to organization depending on the size and complexity of their 1886

operations and processes; customers, statutory and regulatory requirements; and the competence of the 1887

persons involved. 1888

The following is the typical structure of documented information for QMS: 1889

1890

• Quality manual – a high level document providing intentions and commitments of the NRA about 1891

each requirement of ISO Standard 9001:2015 and providing references to lower level documents 1892

such as procedures etc. The manual could also include statement of scope of QMS, quality policy 1893

and quality objectives. 1894

• System procedures – such as procedures for risk determination and risk control, maintenance of 1895

infrastructure, maintenance and calibration of monitoring and measuring resources, creation and 1896

control of documented information, internal audit, management review, customer complaints and 1897

feedback, corrective action etc. 1898

• Standard operating procedures (SOPs) – for operational processes for each of the regulatory 1899

functions 1900

• Forms/formats/templates – as needed in the above procedures 1901

• Records – as evidence of demonstrating conformance to the prescribed requirements 1902

1903

While creating and updating documented information (DI) for QMS, an appropriate identification and 1904

format is used, and that DI is duly reviewed and approved. 1905

1906

The Identification and description of DI may be assured by the title, date, author, or reference number 1907

(or a combination of these), the format for the DI can be hard copy, electronic or both. It could also be 1908

in more than one language, based on the culture of the organization. 1909

1910

The method for the review and approval of DI should be decided, e.g. having an identified person with 1911

the authority to review and approve the DI or having one or more reviewers and one person who takes 1912

the responsibility for approval. 1913

1914

Documented information should be available in a suitable format and be adequately protected. The DI 1915

should also be in a form that is suitable for the intended use, for example, a written technical service 1916

agreement for an external service provider, or process parameter information in electronic format that 1917

can be used/downloaded at the process interface internally. 1918


63

1919

Controls on DI include its availability, distribution and protection (for example from loss of data), its 1920

confidentiality, improper use and unintended changes. This can be done in many ways, including in 1921

electronic systems with ‘read-only’ access and specified permissions to access, password protection or 1922

identification (ID) entry. Information security issues and data backup should also be taken into 1923

consideration. 1924

1925

The control of documented information also must address distribution, access, retrieval and use, storage 1926

and preservation, control of changes, retention and disposition of DI. 1927

1928

Documented information can change and develop as an organization develops its QMS. There is also 1929

a need to consider how historical documented information is maintained, stored and retrieved as 1930

necessary for subsequent use. 1931

1932

The retention time for documented information could be a statutory or regulatory requirement, a 1933

contractual requirement, or can be determined by the NRA. 1934

1935

Documented information of external origin such as customer’s given DI (application files, SLPs), 1936

government orders, national/regional/international standards, calibration reports given by outside 1937

laboratories etc., as necessary for QMS should be identified appropriately and controlled in line with 1938

other DI. 1939

1940

When documented information is retained as evidence of conformity (records), it should be protected 1941

from unintended alterations, e.g. in case of soft copies, only giving controlled access (‘read only’) to 1942

such information. 1943

1944

Practical help box 19. Guidance for the interpretation of Clause 7.5 1945

1946

Examples of documented information that needs to be maintained by NRAs 1947

1948

Organizational chart, job descriptions, list of personnel and their roles and responsibilities, operational processes 1949

flow charts, mission and vision policy statements, strategic plans, QMS scope statement, quality manual and 1950

quality objectives, standard operating procedures, instructions, forms among others. 1951

1952

1953

Examples of documented information that needs to be retained by NRAs 1954

1955


Marketing authorization files, summary lot protocols, certificates of compliance/ non-compliance, lot release 1956

certificates, records of test results, reports, annual product review reports, testing methods and related validation 1957

reports, reports of adverse events following immunization and case investigation reports, complaints and related 1958

reports, personnel qualification records, personnel training records, personnel health records, internal/external 1959

audit plans and related reports, management review meetings’ agenda and minutes, validation, qualification or 1960

calibration records or reports among others. 1961

1962

More information on good data and record management can be found in the recently published WHO guideline. 1963

(28) 1964

1965

Clause 8. Operations 1966

1967

8.1 Operational planning and control 1968

1969

Guidance 1970

1971

The intent of this clause is to ensure that NRAs plan, implement and control the operational processes 1972

(MA, VL. MC and LR processes) that are necessary to meet the requirements of service provision, 1973

including any externally provided (outsourced) processes. 1974

1975

The following are the components of the plan: 1976

1977

• Determining requirements for products and services - consider customer, statutory and regulatory 1978

requirements, organizational requirements including requirements relating to relevant interested 1979

parties (stakeholders). 1980

• Establishing criteria (methods/procedures/KPIs) for the control of processes and acceptance of 1981

products and services consider; a) risks and opportunities; b) quality objectives; c) requirements for 1982

products and services. 1983

• Determining what resources are needed and if the current resources suffice. 1984

• Planned and potential unintended changes, and how these changes can affect the operations. 1985

• Determining documented information that needs to be maintained and that which needs to be 1986

retained 1987

1988

The output of the above planning should be used as inputs to operations. It should be kept in suitable 1989

format and media for those who need to use it. Practical help boxes 5 to 8 give examples of the details 1990

of the processes involved in MA, LR, VL and MC, and possible KPIs, and required resources. 1991


65

1992

8.2 Requirements for products and services 1993

1994

Guidance 1995

1996

The first part of the clause refers to communication with customers and focuses on five communication 1997

areas: 1998

1999

a) detailed communication of the products and services offered so that the customer understands the 2000

requirements, to be provided to the customer through the website, pre-submission meetings, 2001

scientific advice, telephone or other, 2002

b) clear communication on how the customer can contact the NRA in case of questions or other 2003

services, or how the NRA would contact the customer in case of questions or other, 2004

c) establishing channels to gain information from customers such as concerns, complaints, positive 2005

and negative feedback, for example a web-based platform, phone calls, surveys, etc., 2006

d) inform customers how the customer property (documents, samples, dossiers etc.) is handled, where 2007

appropriate, and 2008

e) ensure that the NRA is proactive in communicating with the customer about possible contingency 2009

actions that can be taken, if the need occurs, such as natural disasters, epidemics, shortfall of staff 2010

or others. 2011

2012

Proactive communication enables the customer to understand what the NRA can or intends to provide 2013

and enables the NRA to understand or confirm the needs and expectations of the customer and bring 2014

greater transparency and public accountability. 2015

2016

The second part of the clause refers to determining the requirements of products and services, which in 2017

the case of NRAs are mostly statutory and regulatory requirements such as the Food and Drugs Act, the 2018

pharmacopoeia, monographs, etc.; but also, timelines for service delivery, fees charged for service, 2019

hours for customers service, acceptable waiting/or response time, etc. This information should be 2020

transparently communicated to customers and the public in general, usually through the NRA website. 2021

2022

The NRA should review the commitments it makes to a customer and ensure it can meet them. The 2023

review allows to reduce the risk of issues arising during operations. 2024

2025

The NRA should ensure that request for service received from customer is complete and is in conformity 2026

with service requirements. When there is a difference between the requirements for products or services 2027


as requested by customer and the one prescribed by the NRA, the same should be communicated to the 2028

customer and resolved before processing the request. Any verbal request/change in the requirements, 2029

either by the NRA or by the customer, should be confirmed before service is provided. 2030

2031

When the requirements for products and services are changed due to any reason, the NRA should take 2032

measures to inform all relevant interested parties. 2033

2034

The NRA should retain evidence of the results of the revisions to the requirements of products and 2035

services and any new requirements for the products and services that are provided. 2036

2037

Practical help box20. Guidance for the interpretation of Clause 8.2 2038

2039

Example of communication with customers 2040

2041

a) Customer Communication 2042

2043

Communication with customers aims at collecting information about their needs and expectations, as well as the 2044

reception of doubts, suggestions and complaints about the current work process and the engagement of 2045

stakeholders. 2046

2047

Phases: 2048

2049

- meetings with internal customers - managers and their teams - to identify problems and propose solutions; 2050

- meetings with regulated sector and other stakeholders to identify difficulties with the products and services 2051

offered by the regulatory authority and seek suggestions for improvements; 2052

- meeting with managers, to present the methodology and the schedule of actions. 2053

2054

Products: 2055

2056

- user's journey map: tool to identify all the points of contact of a user with a product or service and understand 2057

their needs, feelings, desires and pains related to the product or service, to promote the necessary improvements; 2058

- communication plan: a tool that establishes strategies for communicating with customers and other stakeholders 2059

involved during all stages of the process improvement initiative and raising awareness of the new way of working. 2060

2061

2062

2063

b) Determination and critical analysis of requirements 2064

2065


67

To determine the requirements, it is necessary to understand the challenges, make the diagnosis and immerse in 2066

the problem, through five stages. 2067

2068

Phases: 2069

2070

- definition of the scope of processes; 2071

- collection of quantitative and qualitative data; 2072

- mapping the current situation of the processes; 2073

- definition of performance gains; 2074

- definition of the team committed to the initiative. 2075

2076

Products: 2077

2078

- planning the initiative; 2079

- quantitative and qualitative analysis of processes; 2080

- flowcharts and checklists AS IS (as it is). 2081

2082

c) Changes on requirements 2083

2084

When changes of requirements occur, regardless of the reason, process documentation is reviewed, changes are 2085

recorded as well as communicated to all those involved in the chain. 2086

2087

Phases: 2088

2089

- meetings with customers to understand the dynamics of the necessary changes; 2090

- revision of the documentation (scope, schedule, actors, communication plan); 2091

- communication to all those involved. 2092

2093

Product: 2094

2095

- new planning of the process; transformation initiatives (scope, schedule, actors, communication plan). 2096

2097

8.3 Design and development of products and services 2098

2099

Guidance 2100

2101

The intent of this clause is to ensure that NRAs establish, implement and maintain a design and 2102

development process in order to ensure that new products and services meet requirements. The design 2103

and development process define the characteristics of the products and services. 2104


2105

The design and development of products and services consists of a set of processes that use ideas or 2106

requirements for a product or service. These ideas or requirements can come from customers, end-2107

users, regulations, the organization or other interested parties including WHO. The ideas or 2108

requirements are processed to develop more detailed requirements that finally define the characteristics 2109

of the product or service. An example of an instance where an NRA may need to go through design 2110

and development process is in case it decides to perform a regulatory function that was not in place 2111

until then, or a new process within a function. For example, NRA from country X does not inspect 2112

clinical sites and is now able to introduce this new process within their activities to regulate clinical 2113

trials. To introduce this process, they will follow international guidance, however specificities of the 2114

process to be followed will be designed in-house. If an organization only uses ideas or requirements 2115

provided by regulation, customers or end-users, without adding more detail, it does not have design and 2116

development activities. In such cases, an explanation as to why the NRA is not applying clause 8.3 in 2117

its QMS can be included in the QMS scope statement (see clause 4.3). 2118

2119

The design and development of products and services requires several phases or steps 2120

2121

• Design and development planning: to determine the necessary design and development activities 2122

and tasks. This plan should include, required process stages; design inputs, design review, design 2123

verification and design validation, resource needs; as well as a clear definition of roles and 2124

responsibilities. 2125

• Design and development inputs: determines the inputs for design and development projects. These 2126

inputs need to be unambiguous, complete, and consistent with the requirements that define the 2127

characteristics of the product or service. It is important to consider the functional and performance 2128

requirements, statutory and regulatory requirements as well as additional standards or codes of 2129

practice. 2130

• Design and development controls: to ensure that once the inputs have been determined, the design 2131

and development activities and controls are implemented in accordance with the planning, to ensure 2132

process is effective. 2133

• Design and development outputs: to ensure that design and development outputs (service provision, 2134

standard operating procedure or service provision manual) give the necessary information for all 2135

the processes needed to provide intended products and services (including information to be 2136

provided by service recipients, service provision process, and post-delivery activities, if any). 2137

• Design and development changes: to determine, review and control changes made during or after 2138

the design and development process. Changes can arise during the design and development process 2139

(because of design review, verification or validation activity), after the release and approval of the 2140


69

design and development outputs and during implementation of the same, as a result of monitoring 2141

customer satisfaction and interested parties’ feedback or as result of changes, or new, statutory and 2142

regulatory requirements. 2143

2144

Once these development phases are completed, a review by person(s) who are not involved in design 2145

activity) is required to ensure that design and development planning stages and the output of each stage 2146

are in place. Verification (comparing the new design with a similar proven design) and validation 2147

(testing under intended user conditions) activities are essential for controlling the design and 2148

development process and need to be implemented effectively. 2149

2150

NO EXAMPLE AVAILABLE AS YET 2151

2152

8.4 Control of externally provided processes, products and services 2153

2154

Guidance 2155

2156

The intent of this clause is to control processes, products and services that are provided by an external 2157

provider. External providers could include Government’s central procurement agency, suppliers of 2158

products and services, experts and consultants or someone to whom the NRA decides to outsource a 2159

process. 2160

2161

The NRA is responsible for ensuring that externally provided processes, products and services conform 2162

to requirements (e.g. through incoming goods inspection, or surveillance of an outsourced service 2163

provider). 2164

2165

The NRA should clearly identify its requirements (specifications) for the product and service to be 2166

purchased to ensure that externally provided processes, services or products do not have a negative 2167

effect on its operations or on customer satisfaction. 2168

2169

The NRA should ensure, its requirements are complete, clear and address any potential issues. It should 2170

clearly communicate the requirements and controls to be applied to the external provider and both 2171

parties should agree as to what is required. This understanding of requirements is usually reflected in 2172

a technical service agreement and/or through a purchase order/contract. 2173

2174

The NRA needs to determine and apply criteria for the evaluation, selection, monitoring of performance, 2175

and re-evaluation of external providers. The type and extent of control to be exercised is based on how 2176


much effect the externally provided process, product or service can have on the conformity to 2177

requirements of the NRA’s products or services. The NRA should determine which specific controls 2178

are to be implemented to an external provider. Control activities that may be considered include 2179

inspections, certificates of analysis or testing, second party audits, evaluation of statistical data and 2180

performance indicators. 2181

2182

The NRA should maintain up-to-date information related to its external providers, evaluated on their 2183

ability to comply with purchasing requirements both in terms of conformity of the product and service 2184

provided and delivery performance. A list of providers could serve as a basis for external provider 2185

selection and management of relation with current external providers. 2186

2187


2189

For the MA process, the NRA has as external provider with the figure of "Third Authorized Party” (TAP), which 2190

are persons authorized by the NRA to perform a preliminary review of marketing authorization dossiers and to 2191

issue, if applicable, a Favorable Technical Reports (FTR) for registration, modification or extension of medicines 2192

MA based on compliance with the requirements established by the Ministry of Health in the corresponding 2193

regulations for the completion of procedures. 2194

2195

The process of selection of a TAP by the NRA is done through an announcement published by the Ministry of 2196

Health in which the requirements that must be fulfilled by interested candidates are established. The authorization 2197

to act as a TAP has a validity of two years. TAPs can be individuals or companies. 2198

2199

The NRA recognizes the technical competence of the TAP ensuring that the process of evaluation and release of 2200

an FTR is managed effectively in accordance with the provisions of the current regulations through the 2201

establishment of policies, responsibilities and activities to be fulfilled by the TAP. The TAP is also subject to 2202

controls for evaluation and monitoring. 2203

2204

The controls applied by the NRA fall under the following categories: 2205

2206

• Technical supervision 2207

• Supervision of the records reviewed by the TAP 2208

2209

Verify the technical qualification, training and experience of the personnel involved in the review. 2210

Verify the documented training system that ensures competencies in the technical aspects. 2211

Verify that there are adequate tools, references and bibliography that allow technical activities. 2212

Review of the technical procedures and the homologation criteria of the reviewers (TAP). 2213

2214


71

Follow up on the corrective or preventive actions derived from the non-conformities detected in the supervision 2215

visits. 2216

2217

TAP verification actions allow to detect FTR with inconsistencies or non-compliance with the regulatory 2218

provisions. 2219

2220

The qualification of the TAP has been assigned a level of confidentiality from I to IV, being I "highly reliable" 2221

and IV "not reliable", which will provide the level of review (reduce, regular or strict) by the NRA of the 2222

procedures entered with FTR of the TAP. 2223

2224

2225

8.5 Production and service provision 2226

2227

Guidance 2228

2229

The intent of this clause is for the NRA to establish controls to ensure that the intended results are 2230

achieved (products and services), by reducing the potential for errors/nonconforming outputs. The 2231

clause also focuses on the preservation of data and physical property, traceability, control of changes 2232

and the responsibility for post-delivery activities. 2233

Items a) to h) of clause 8.5.1 provide suggested controls to be applied to service provision to ensure that 2234

the criteria determined in clause 8.1 are met. These include documented information about the 2235

procedures used and the results of monitoring activities including measurements if applicable; checks 2236

to ensure that the necessary infrastructure is in place as well as availability of competent personnel, that 2237

processes are validated, actions taken to prevent human error including appropriate training of personnel 2238

provided, and controls are in place for release, delivery and post-delivery activities including 2239

confirmation that authorized personnel for these activities is in place. 2240

2241

To properly monitor the status of product and service provision throughout the service provision 2242

process, products and services should be identified (reference number of service request, batch number 2243

for drugs, code no of product, others) and traceability ensured. Product and service identification 2244

prevents unintended mix up of the service requests and allows tracing of the events for processing 2245

service requests, updating customer about status of service delivery, investigation of customer 2246

complaints, etc. 2247

2248

The NRA, due to its mandated role and responsibilities, has access to property that does not belong to 2249

the NRA, but which is under the NRA’s control; this property can be tangible or intangible. Examples 2250


include marketing authorization dossiers, SLPs, vaccine samples for testing, intellectual property or 2251

personal data, others. The NRA should make provisions to ensure the protection of such property. 2252

2253

The actions taken to protect it, will depend on the type of property. The owner of the property should 2254

be clearly identified and made known within the NRA. Protection of data could be ensured through a 2255

specific password protected electronic location or file with restricted access to store customer’s 2256

intellectual data, patent information, performance and sales figures, etc. Data integrity can be ensured 2257

by regular back-ups and virus protection, storage of magnetic media (e.g. video tapes, audio tapes and 2258

computer disks) in a non-magnetic environment, others. 2259

2260

When the NRA takes control of the property its verification is important (e.g. state or physical condition, 2261

accuracy of personal data, completeness of the dossier). 2262

2263

The customer or external provider should be accurately informed if property is lost, damaged or 2264

otherwise found to be unsuitable or incapable of use. This will require to be documented. 2265

2266

Outputs from different processes and products and services, should be preserved from damage or loss 2267

at all stages during service provision. The NRA should determine which are the outputs, products and 2268

services that can deteriorate or degrade and implement appropriate preservation methods. 2269

2270

In case that changes occur during service provision, these must be reviewed and controlled. 2271

2272

There may be numerous reasons why changes occur; for example, a change initiated by an external 2273

provider (e.g. delays in getting expert’s opinion or test reports from external labs), due to internal issues 2274

(e.g. critical equipment failure, internet connectivity issues) or to an external issue (e.g. new or modified 2275

customer or statutory and regulatory requirements). 2276

2277

Changes must be controlled, and the relevant documented information retained. Examples include: a) 2278

minutes of the review activities; b) description of the change; c) details of the person(s) or a customer 2279

authorizing the change. 2280

2281

The responsibility of the NRA does not end with product and service delivery, the clause also 2282

emphasizes the need to determine the post-delivery activities in which the NRA is engaged. For this, it 2283

should consider if the post-delivery activity is part of a contractual requirement or is a regulatory 2284

requirement such as marketing authorization renewals, approval of changes (variations), market 2285

surveillance or to address a potential complaint from customers (customers dissatisfaction). 2286


73

2287

Other examples of post-delivery activities include: 2288

2289

• Engagement with customers to determine if the products or services were to their satisfaction 2290

through customer feedback, resolving customer complaints, customer compliments, media reports 2291

etc; and 2292

• Customer access to on-line information required after delivery. 2293

2294

Practical help box 22. Guidance for the interpretation of clause 8.5 relating to preservation of 2295

physical property 2296

2297

Example of infrastructure required for the preservation of vaccines until their expiry 2298

2299

An NRA may receive vaccine samples for visual inspection or testing during the lot release process, and in some 2300

cases also during the marketing authorization evaluation process, although at this stage this is not required. 2301

2302

In case samples are requested, these need to be properly stored and kept until the expiry date. The NRA needs 2303

adequate infrastructure to keep vaccines at 2-8°C during the whole shelf life. Adequately validated and regularly 2304

monitored refrigerators are needed. Back up measures are required such as an alarm system (ideally centralized), 2305

which ensures that in case of electricity failure or break down of the equipment, a responsible officer is 2306

immediately informed. A back up refrigerator or electricity generator, depending on the source of the failure, 2307

must be available for such situations 2308

2309

8.6 Release of products and services 2310

2311

Guidance 2312

2313

The intent of this clause is to ensure that products and services are checked for conformity for all 2314

applicable requirements, at appropriate stages of the service provision process, before they are released 2315

for delivery to the customer, for example, issue of market authorization certificate/letter. 2316

2317

Approval by a relevant authority may be required when all checks for conformity have not been 2318

satisfactorily completed - in some cases, this could be the customer. 2319

2320

The release of products and services should be suitably documented (DI). The DI should include 2321

evidence that the product or service conforms to all acceptance criteria and be traceable to the person 2322

authorized to release products and services. 2323


2324

The person(s) who authorize(s) final release of the product or service should be suitably defined by, for 2325

example, their job description or authority level. 2326

2327


2329

Products of XYN Drug Authority include reports, certificates, licences, permits and authorization letters. These 2330

are checked by the respective supervisors and signed by the Executive Director or other senior officer(s) 2331

authorised by the Governing Board to do so as per section XX of the National FDA Act, before delivery to the 2332

applicant or entity that requests for them. The list of authorised persons (Authorised Persons to Release the XYN 2333

Drug Authority products to applicants) is updated from time to time and communicated to all staff via the posted 2334

on the XYN Drug Authority Intranet. 2335

2336

The release of reports, certificates, licences, permits and delivery to the applicants does not proceed until the 2337

requirements have been satisfactorily met (e.g. certificates for GMP compliance are not issued until the evidence 2338

of corrective and preventive actions taken by the manufacturer are submitted and evaluated by the XYN Drug 2339

Authority and found to be satisfactory). 2340

2341

8.7 Control of non-conforming outputs 2342

2343

Guidance 2344

2345

The intent of this clause is to prevent non-conforming outputs from progressing to the next stage or to 2346

the customer. 2347

2348

There are different ways to control non-conforming outputs: 2349

2350

• Correcting (rework, repair) the nonconformity to ensure it does conform 2351

• Removing the nonconformity from the process entirely (rejecting or scrapping the output/product) 2352

• Obtaining authorization for release under concession 2353

2354

The extent of control that an organization needs to take depends on the nature of the nonconformity and 2355

its potential effects. 2356

2357

If the nonconformity is discovered after it has progressed to the next stage, or been delivered to the 2358

customer, the NRA should take appropriate actions to prevent unintended use or undesired 2359

consequences, and take measures such as issuing a re-call, suspension, re-processing, eliminating or 2360


75

reducing the NC to acceptable level (concession). In case of concession, authorization should be given 2361

by the appropriate person(s) or, where relevant, the customer. 2362

2363

The NRA should ensure that the documented information retained includes details of the 2364

nonconformity, the actions taken to correct, mitigate or communicate it, any concessions obtained (e.g. 2365

agreement with the customer that the product or service could be used despite the nonconformity) and 2366

who authorized the actions taken. 2367

2368

Retaining documented information on the above ensures that processes are improved and optimized; 2369

corrected work instructions, processes and procedures are detailed for future use. 2370

2371

Practical help box 24 Guidance for the interpretation of clause 8.7 Control of non-performing 2372

outputs 2373

2374

When a nonconforming output is detected before or after delivery to the customer, it is registered by the process 2375

owner and an investigation form e.g. complaint investigation in-process form (in case of market/customer 2376

complaints); or the OOS investigation form (in case of the QC Laboratory); or the corrective action request (CAR) 2377

form (for others, e.g. arising out of quality audits); is raised for investigation to be initiated in order to find out the 2378

root cause or assignable cause. Correction and corrective action are then taken by the respective process owner. 2379

2380

A non-conforming output may include any of the following items that is found to have an error, mistake or defect 2381

before or after delivery to the applicant (customer): 2382

2383

a) Marketing authorization certificate, GMP certificate, import/export permit, manufacturing licence, licence to 2384

sell drugs, or a laboratory test report or certificate of analysis, 2385

b) Published adverse event report, 2386

c) Clinical/field trial assessment monitoring report, 2387

d) Promotional material vetting report. 2388

2389

In all cases, correction and corrective action are taken and the certificate, permit, licence or report that has an 2390

error, mistake or defect is either cancelled or withdrawn (without prejudice), or both and replaced with a corrected 2391

one. However, the validity period and the applicable conditions remain the same. 2392

2393

2394

2395

Clause 9. Performance evaluation 2396

2397

Practical help box 25 Guidance for the interpretation of clause 9 2398


2399

A monitoring and evaluation framework that tracks process activities, targets, key performance 2400

indicators, and outputs is used to monitor progress of processes. Performance reports (quarterly, semi-2401

annual, and annual) are made and their information analyzed and used as input in management reviews. 2402

2403

9.1. Monitoring, measurement, analysis and evaluation 2404

2405

Guidance 2406

2407

The intent of this clause is to ensure that NRAs conduct monitoring, measurement, analysis and 2408

evaluation to determine if the intended results are being achieved. The NRAs should determine what 2409

needs to be monitored and measured (according to the characteristics of processes, products, services 2410

and risks involved) and the methods to be used to analyse and evaluate the performance and 2411

effectiveness of QMS. The NRAs should also determine how and when the monitoring, measurement, 2412

analysis and evaluation will be carried out, and the resources that will be needed for the same. 2413

2414

The NRAs should decide on what documented information relating to monitoring, measurement, 2415

analysis and evaluation will need to be retained as evidence of the results. 2416

2417

One way of monitoring performance is through feedback from customers. It allows to evaluate the 2418

degree of customers’ satisfaction and to determine opportunities for improvement. The NRA may 2419

choose to seek feedback from a selected population of customers or from every customer at the end of 2420

a transaction. Means to obtain feedback are offered through the social and published media such as 2421

web sites and message boards, opinion surveys and compliments or complaints. 2422

2423

The NRAs should be able to determine the degree of customer satisfaction after the results are analysed 2424

and evaluated; and act based on this information. This information should be an input to management 2425

review (9.3) and can be used for determining if actions are necessary to improve customer satisfaction. 2426

2427

The results of monitoring and measurement (data and information) must be analysed to determine if 2428

processes, products and services meet requirements and whether there are any needed actions and 2429

opportunities for improvement. The purpose of analysis and evaluation of data from monitoring and 2430

measurement activities include assessing the level of customer satisfaction, assessing whether plans are 2431

being met, assessing performance of external providers, how successful the NRA has been in addressing 2432

risks and opportunities, status of performance and effectiveness of QMS and need for improvements. 2433

2434


77

Data sources that could be used for analysis and evaluation include the monitoring of customer 2435

perception (9.1), feedback from media and the public in general (9.1), monitoring quality objectives 2436

(6.2), timeliness of service delivery (8.5), data related to corrective or preventive actions taken (8.7), 2437

non-compliances detected during internal audits (9.2), others. 2438

2439

The output from analysis and evaluation is generally in the form of documented information such as 2440

trend analyses or reports and becomes an input to management review (see clause 9.3). 2441

2442

EXAMPLE NOT AVAILABLE AS YET 2443

2444

9.2 Internal audit 2445

2446

Guidance 2447

2448

The intent of the clause is, for the management of NRA, to obtain information through internal audits 2449

about continued conformance and effectiveness of QMS. 2450

2451

The internal audits are conducted at planned intervals to verify if the NRA’s activities and processes 2452

continue to meet the prescribed system as defined in NRA’s documented information (e.g. Quality 2453

manual, quality policy, quality objectives, procedures, instructions, risk control plans and other plans), 2454

if the QMS continues to meet the requirements of ISO Standard 9001:2015 standard and if QMS is 2455

effectively implemented and maintained. 2456

2457

Internal audits should be planned, results reported and timely actions on the audit findings taken. 2458

2459

Items a) to f) under clause 9.2.2 of the ISO Standard 9001:2015 provide details on how audits must be 2460

planned. The planning process includes developing the audit programme (calendar of audits over a 2461

period of time - for example, one year - which also means setting the frequency of audit of each area 2462

and related processes over a one-year time), defining the criteria to be used (standard, requirements), 2463

the scope of the audit and the methodology to be used during the audit (interviews, documented 2464

information review, results, trends, etc.). Auditors should be selected, usually from sectors within the 2465

regulatory agency not affected by that particular audit (cross functional audit) who have been trained in 2466

QMS auditing. Sometimes external auditors may be used if independence within the NRA is difficult 2467

to ensure. Corrective actions must be taken promptly to address the findings of the audit. Documented 2468

information of the programme and audit results should be maintained. 2469

2470


Responsibility for planning of audits lies with the person who is directing and managing the internal 2471

audit process (audit manager) such as QMS coordinator or some other person as authorized by 2472

management of NRA. 2473

2474

While developing the programme, the audit manager should apply risk-based thinking and consider 2475

how often the process is performed as well as its maturity and complexity, whether any changes in the 2476

process were introduced, or other changes affecting the NRA, the process performance, results from 2477

previous audits and history of complaints. 2478

2479

After each internal audit is completed, the results should be reported to relevant management. Based 2480

on these results, appropriate correction and/or corrective actions can be necessary. Typically, 2481

organizations establish a time to respond and correct nonconformities to ensure they are fixed in a timely 2482

manner. 2483

2484

During the audit, auditors might bring up a potential weakness in the QMS, which may represent 2485

opportunity for improvement. Such information can help management to decide if it is appropriate to 2486

initiate action for improvement. 2487

2488

It is important that management of NRA fosters an open-minded culture where quality audits are 2489

perceived as a means to improve performance and not to assign blame for any non-conformities found. 2490

2491


2493

Planning internal audits by ANRA, the NRA from Country A 2494

2495

Scope of the auditing programme: legal services, internal audit, procurement management, communication and 2496

public education, finance and accounts, information and communication technologies and statistics, human 2497

resources and administration, planning monitoring and evaluation, food inspection and enforcement, food 2498

registration food risk assessment, clinical trials control and pharmacovigilance, medicines and complementary 2499

products inspection and enforcement, medicines registration, medical devices, diagnostics and cosmetics control 2500

Timelines: November and December 2018 2501

2502

Processes being audited clearly defined, methodology applied is interviews and documented information review. 2503

Audit duration: One day for each section/unit. 2504

Auditors: Two internal auditors (cross functional) are selected based on expertise. 2505

2506

ANRA is a decentralized authority with five zonal offices. The auditing programme is the following: 2507


79

2508

Scope: The five zonal offices 2509

Processes: QMS, premises licensing, inspection and enforcement, import and export control, registry, customer 2510

complaints and procurement and finance 2511

Audit duration: One day for each zonal office 2512

Timelines: October 2018 2513

Auditors: Two internal auditors (cross functional) are selected based on expertise. 2514

2515

Summary of the procedure for conducting internal audits by ANRA 2516

2517

The purpose is to provide details on how internal audits should be conducted to check and evaluate the efficacy 2518

and effectiveness of the QMS for continual improvement. 2519

2520

The quality manager (QM) appoints the auditors and prepares the audit programme for HQ and Zone offices. The 2521

QM informs auditors and auditees in writing about the programme. The audit team prepares the audit checklist 2522

based on previous audit findings and QMS documentation. During the audit it is the responsibility of the team 2523

leader to open the audit and to verify attendance. The audit team should collect and verify information for specific 2524

processes, procedures, functions, sites, areas and activities. It records the findings and prepares the audit report. 2525

The audit team leader closes the audit. Non-conformities are categorized in minor, major and opportunity for 2526

improvement. The audit report is reviewed and agreed upon by auditors and auditee. This is then forwarded to 2527

the QM. The auditee should prepare a CAPA and the QM will take measures for timely follow up. If corrective 2528

or preventive actions are properly implemented, the audit is closed, otherwise another follow up form is opened. 2529

2530

The audit reports, corrective and preventive actions reports, attendance register, checklists and audit programme 2531

should be kept at QM office and maintained for a period of five years and then destroyed by tearing, shredding, 2532

burning or other appropriate means. 2533

2534

9.3 Management review 2535

2536

Guidance 2537

2538

The intent of this clause is to ensure that NRA’s top management conducts periodic review of 2539

performance of its QMS. The purpose of such review is to determine if NRA’s QMS continues to be 2540

suitable (fitting the purpose), adequate (sufficient), effective in achieving the intended results and 2541

continues to be aligned with the strategic directions of the NRA. 2542

2543

Management review should be conducted at planned intervals; this could be daily, weekly, monthly, 2544

quarterly, semi-annually or annually, depending on the situations facing the NRA. If various levels of 2545


the management carry out management review activities, the results should be made available to its top 2546

management for final decision/approval. 2547

2548

Management reviews could be a standalone activity or in a combination of related activities (e.g. 2549

strategic planning, business planning, annual meeting, operations meetings, other management 2550

reviews). 2551


81

Table 4. Management Review Meetings (MRM) inputs and outputs 2552

2553

INPUTS OUTPUTS

Review implementation of actions to be taken

following previous review meetings

• Decisions and actions related to

opportunities for improvement (10.1)

• Decisions and actions related to changes

required in the QMS (6.3)

• Need for additional resources to implement

improvement initiatives and changes

suggested in QMS and for other areas where

resources (including human resource) are

not adequate (7.1).

• Progress towards quality objectives (6.2)

• Management review meeting minutes to be

retained as documented information and

communicated in an adequate way to all

concerned.

• Outputs from MRM will be inputs for the

following meeting

Updated analysis of internal and external context of

NRA (4.1)

Performance and effectiveness of QMS (9.1)

through:

Customers satisfaction (9.1), feedback from

interested parties (4.2), implementation of quality

objectives (6.1), monitoring processes through KPIs

(8.5), conformity of products and services (8.6),

status of non-conformities including response to

complaints and corrective or preventive actions

(10.2), monitoring and measurement results (9.1),

audits outcome (9.2) and performance of external

providers (8.4)

Adequacy of resources (7.1)

Data about the effectiveness of the actions to

confront risks and opportunities (see 6.1)

Opportunities for improvement (10.1 and 10.3)

2554


2556

Country A NRA Summary Procedure for Management Review Meetings 2557

2558

The QM should call management review meetings at a minimum once per year but should attempt to conduct 2559

them quarterly. Attendees must include the Director General, Directors of Units, Management representative, 2560

Legal counselor and any other personnel as deemed necessary by the management to attend. The QM should 2561

record the minutes of the meeting. Analysis of data presented (according to table 4) should be performed to look 2562

for areas of improvement. Improvement items and follow up actions should be implemented as Action items. The 2563

agenda should include status of action of previous MRM and changes in internal and external issues that are 2564

relevant to the QMS 2565

2566

Some NRAs have two types of review meetings: 2567

2568

1. Technical review meetings (TRM) membership is QM focal points in different technical units. They take 2569

place frequently (usually monthly) and focus specifically on technical aspects such as KPIs, implementation 2570


of corrective or preventive actions, conformity of products and services, monitoring and measurement results 2571

including trends, audit outcomes and performance of external providers 2572

2573

2. Management review meetings membership is as for the (TRM) plus higher-level management as indicated 2574

for Country A above. These take place on quarterly, bi-annual or annual basis. Outputs from the TRM are 2575

inputs for MRM as well as other QMS related aspects not addressed in the TRM (see table 4) 2576

2577

2578

2579

2580

2581

2582

2583

2584

2585

2586

2587

2588

2589

2590

2591

2592

2593

2594

2595

2596

2597

2598

2599

2600

2601

2602

2603

2604

2605

2606


83

XYN Drug Authority Management Review Flow Chart 2607

Quality Management Department Top Management

Schedule Management Review Meeting & Issue

Agenda

Analyze Performance,

Results & Trends

Management Review Outputs

• Opportunities for improvement;

• Changes to the QMS;

• Resource needs.

Agree & Implement

System Changes No System Changes

Suitable &

Adequate

Not suitable or

Not adequate

Documentation & Records

Suitability &

Adequacy

Management Review Inputs (Agenda)

1) The status of actions from previous management reviews;

2) Reports on process performance, conformity of

services and the adequacy of resources

(including effectiveness of processes;

monitoring and evaluation results and trends;

changes in external and internal issues;

resources available e.g. human resource,

time, equipment and technology used,

financial, etc.) with respect to the following

key drug regulatory processes:

a) Assessment and registration of medicines; b) Inspection and licensing/certification of

• pharmacies, medicine shops

• pharmaceutical manufacturers (Good Manufacturing practice);

c) Control of pharmaceutical imports and exports;

d) Pharmacovigilance; e) Clinical Trials; f) Vetting of drug promotional materials; g) Post-marketing surveillance; and h) Enforcement.

3) Report and trends on quality objectives (extent to which objectives related to customer satisfaction, e.g. service delivery objectives have been met) for the key drug regulatory processes listed in 2 above and all the support processes, e.g. Finance and administration, human resource, procurement, legal services, information technology, internal audit, quality management, and public relations;

4) Report and trends on client/customer complaints (market complaints, including appeals)

5) Information from the recent customer satisfaction survey report;

6) Internal quality audit results (from Internal quality audit reports, including second party audits);

7) Report on nonconformities and corrective actions;

8) Performance of external providers; 9) Effectiveness of actions taken to address risks

and opportunities for all key drug regulatory processes and all support processes;

10) Identifying opportunities for improvement for all processes.

Documentation & Records


Clause 10. Improvement 2608

2609

10.1 General 2610

2611

Guidance 2612

2613

The intent of this requirement is to ensure that the NRA determines opportunities for improvement, 2614

plans and implements actions to achieve the intended results and to enhance customer satisfaction. 2615

2616

Improvements can help the NRA to keep meeting customer requirements and expectations by 2617

improving its products and services, correcting or preventing undesired effects, and improving the 2618

performance and effectiveness of QMS. 2619

2620

There are different methods to conduct improvement, such as correcting existing nonconformities and 2621

taking actions on their causes to prevent recurrence, or making small-step-ongoing improvement 2622

activities or through breakthrough projects leading to innovation, revision and improvement of existing 2623

processes or the implementation of new processes; 2624

2625

10.2 Non-conformity and Corrective Action 2626

2627

Guidance 2628

2629

The intent of this clause is to ensure that the NRA manages nonconformities, and implements corrective 2630

action, appropriately. 2631

2632

Non-conformity means ‘non-fulfilment of a requirement’ related to a product, service, process or QMS. 2633

These requirements may come from the customers, from relevant interested parties, from statutory and 2634

regulatory requirements, or they may be internal requirements defined by the NRA in its policies, 2635

manuals, procedures, quality objectives, etc. 2636

2637

A non-conformity could be identified from customer complaints or from non-conforming outputs, 2638

problems arising from relevant interested parties, audit results, the effects of unplanned changes, etc. 2639

2640

The immediate action needed is to control or correct any non-conformity. This can be achieved by 2641

containing the problem while the investigation continues. For example, making customers aware of a 2642

non-conformity and to provide information about the potential or actual effects on the product provided 2643


85

or service delivered and also correcting the situation. 2644

2645

To make corrections and introduce corrective or preventive actions, the NRA should follow the 2646

following steps: 2647

2648

1. Review and analyse the non-conformity to determine its cause by using methods such as, 5-why 2649

method or cause-and-effect-analysis diagrams or simply by brainstorming with a cross functional 2650

team. Examples of typical root causes include lack of understanding of requirement, lack of 2651

resources, process not well-defined, etc. 2652

2653

2. Determine the extent of the actions that need to be taken to eliminate the cause determined at 1 2654

above. There might be instances where the cause of the non-conformity cannot be eliminated, 2655

therefore the NRA should consider taking actions to detect and minimize the effects of the non-2656

conformity if it were to occur again. 2657

2658

3. Implement any needed actions as decided at 2 above. This may include making changes in 2659

process/procedure, providing resources, retraining persons, ensuring better adherence to defined 2660

process, etc. It should be ensured that corrective action taken in one area should not cause adverse 2661

effects in another area of the NRA. 2662

2663

4. Review the effectiveness of corrective or preventive actions taken by confirming (through 2664

evidence) that the actions have been implemented. This may be accomplished by observing the 2665

performance of processes or reviewing documented information or verifying during internal audits 2666

that the same non-conformity is not repeated. This review should be done after a reasonable time 2667

needed to implement corrective action has elapsed. 2668

2669

5. After the review of corrective or preventive actions, the NRA should consider whether there is a 2670

new risk or opportunity that was not determined during planning (see 6.1) and planning should be 2671

updated as necessary. 2672

2673

The NRA should retain documented information showing what correction or corrective or preventive 2674

actions were taken, including the nature of the non-conformity (e.g. nonconformity statement); 2675

examples include corrective action forms or databases and evidence demonstrating that actions have 2676

been taken. 2677

2678

2679



2681

An example of dealing with nonconformity using a Corrective Action Request Form. 2682

XYN Drug Authority Corrective Action Request (CAR) Form 2683

(To be used to request for corrective action of a nonconformity in the NRA quality system) 2684

2685

Nonconformity Root Cause

(from 5-Why Root

Cause Analysis

Form attached)

Corrective

Action

The steps that have or will be

taken for the demonstration

of effectiveness of the actions

taken

Timeline

1.1 There was no

evidence of

monitoring of the

quality objective in

the Post Marketing

Surveillance

department,

contrary to the

requirements of the

standard.

The parameters to

be monitored and

the tool to be used

had not been

established.

Develop a format

for monitoring the

quality objective

clearly indicating

the parameters.

1. Developed a format with

key parameters as listed

a) Products

b) Annual and Quarterly

targets

2. Create a database on the

NRA server for

monitoring the quality

objectives and train the

NRA staff in using it.

Oct 2018

Nov 2018

A root cause analysis must be attached. See example below: 2686

XYN Drug Authority Root Cause Analysis Form 2687

Directorate/ Department / Unit/ Area: Inspection & Licensing Dept. Representative: Dr. Brian Harvey Date: 7th Nov 2018

Category of Problem: Nonconformity Nonconforming output Market Complaint Other (please specify)

(Check applicable box above by double clicking on it) 2688

2689

Problem / Issue Why 1 Why 2 Why 3 Why 4 Why 5

1.1 There was no

evidence of

monitoring of the

quality objectives

in for the

Inspection and

Licensing (I&L)

processes

Why was there no

evidence of monitoring

of the quality

objectives?

Because the quality

objectives had just been

developed and

monitoring them had

not started.

Why was

monitoring of the

quality objectives

not yet started?

Because the quality

objectives had not

been communicated

to the relevant

personnel at all

levels.

Why were the quality

objectives not yet

communicated to the

relevant personnel at

all levels?

Because the system of

monitoring them was

not well established.

Why was the system

of monitoring the

quality objectives not

well established?

Because the

parameters to be

monitored and the

tool to be used had

not been

established.

This is the root cause

that must be taken to

the CAR form above

to determine the

corrective action

2690

Note: Although this technique is called “5 Whys,” you may need to ask the question fewer or more times than five before you 2691

find the root cause of the problem or non-conformity. 2692

2693


87

10.3 Continual improvement 2694

2695

Guidance 2696

2697

The intent of this clause is that NRA should continually improve the suitability, adequacy and 2698

effectiveness of its QMS. 2699

2700

Continual improvement can include actions to increase consistency of process outputs and products and 2701

services; improve process capability and reduce process variation. This is done to enhance the NRA’s 2702

performance and benefit its customers and interested parties. The results from analysis (9.1) and 2703

evaluation and management review (9.3) are used to decide whether continual improvement actions are 2704

needed and what they should be. 2705

2706

Examples of CI include reducing errors, rework, complaints, non-conformity, breakdown of equipment, 2707

delays of promised services etc. and improving customer satisfaction, improving employees’ 2708

involvement, etc. 2709

2710

There are several methodologies and tools that NRA can consider to initiate continual improvement 2711

activities, for example, Kaizen, benchmarking, use of self-assessment models. etc. 2712

2713

Practical help box 27. Guidance for the interpretation of clause 10.3. 2714

2715

As part of continual improvement, XYN Drug Authority uses trending of quarterly, semi-annual and 2716

annual performance of the key drug regulatory process and all support processes; and from results of 2717

management review, to determine areas of underperformance and to identify any opportunities for 2718

improvement. 2719

2720

5. QMS implementation methodology 2721

2722

For the successful implementation of QMS, full commitment of Head of NRA (top management) will be 2723

necessary with respect to provision of timely resources (human and others) for implementation of QMS 2724

and by demonstrating his/her leadership, commitment and customer focus (see guidance under clause 5.1 2725

of this guideline) through all stages of the implementation of QMS. 2726

2727

A systematic way of implementing the QMS will include the following steps: 2728


Step Activity Refer guidance

under clause

Responsibility

within the NRA

A. Documenting QMS

1. Appoint a Core Team (CT) with members from various

functions of NRA with one person as team leader, who

subsequently could be designated as QMS Coordinator

- Head of the NRA

2. Persons in CT should fully understand the QMS

requirements either through study of this guideline or

undergo a formal training on the subject

Full guideline CT

3. Develop current Context statement (SWOT analysis) of

NRA or use one, if already available

4.1 CT

4. Determine and document requirements (needs and

expectations) of interested parties/stakeholders (both

external and internal) relevant to QMS

4.2 CT

5. Determine and document the Scope of QMS (could be

whole NRA or specific functions) with NRA’s products and

services within the scope listed in it.

If any of the requirements of ISO 9001 is not applicable,

provide its justification within scope statement

4.3 CT and Head of

NRA

6. Develop and document Quality Policy, keeping in view the

purpose (vision and mission), context and strategic direction

of NRA.

Policy statement could be communicated through display

within NRA office(s) or otherwise communicated to all, for

its understanding and application.

5.2 CT and Head of

NRA

7. Develop and document QMS related responsibilities and

authorities at different levels of NRA staff and

communicate to all concerned.

5.3 CT and Head of

NRA

8. Use information from step 3 and 4 above, as input, to

determine risks and opportunities and develop risk control

plan.

6.1 CT

9. Develop and document measurable and time bound quality

objectives including plan for monitoring and achieving

them and communicate quality objectives to all concerned.

6.2 CT and Head of

NRA

10. Carry out a gap analysis with respect to support processes

covering human resources, infrastructure (equipment,

hardware, software, facilities etc.), process environment

(heating, lighting etc), measuring equipment, organizational

knowledge and communication and fill the gaps, if any.

Develop new or harmonize existing Standard Operating

Procedures (SOPs) for control of measuring equipment,

organizational knowledge, training and communication

7.1, 7.2, 7.4 CT

11. For internal support services provided by viz

administration, HR, ICT systems, maintenance, logistics,

procurement etc, it is good to develop and practice Service

Level Agreements (SLAs) covering service standards (time

lines) and responsibilities of each party (internal service

provider and service recipient)

7.1, 7.2, 7.4 CT

12. Conduct gap analysis to assess the extent to which the

existing NRA policies, procedures/manuals and practices

relating to regulatory functions (MA, VL, MC, LR or

others) are in line with service provision processes (8.1 to

8.7) of ISO 9001 and harmonize existing SOPs or develop

8.1, 8.2, 8.3, 8.4,

8.5, 8.6 and 8.7

CT


89

additional processes and related SOPs and SLAs with

customers. Also integrate risk control plan in the relevant

SOPs.

Guidance under clause 4.4.1 will facilitate to harmonize

SOPs with ISO 9001 requirements.

4.4.1

13. Develop quality system procedures (QSPs) for monitoring

of customer satisfaction, internal audit, management

review, complaints handling, correction and corrective

actions, improvement; and put them in practice.

9.1.2, 9.2, 9.3,

10.1, 10.2 and

10.3

CT

14. Develop and document a Quality Manual (QM) stating as to

how NRA intents to meet each requirement of ISO 9001

with scope and quality policy statement (at 5 and 6 above)

included into it. All other documents viz SOPs, QSPs,

SLAs, forms/formats/templates referred in

QSPs/SOPs/SLAs may be added as annexes to QM or kept

separately as standalone folders.

All above documented information (DI) including records

could be either in hard or soft version. A QSP for creation,

updating and control of DI will also be needed and also a

QSP on Planning for changes.

7.5, 6.3 QMS Coordinator

B. Practicing QMS

15. It is good practice that documents as they get developed are

communicated to all concerned and put into implementation

mode.

- QMS Coordinator

and all concerned

16. Formal awareness sessions may be held by CT for people to

understand and apply the policies, objectives, SOPs, QSPs,

SLAs etc and if necessary train people on how to use new

QSPs/SOPs/SLAs

7.3 CT/QMS

Coordinator and

all concerned

17. Monitoring of products, services and processes should

continue to happen against defined KPIs, risk control plan,

and through monitoring of applicable quality objectives.

The monitoring data should be analysed and evaluated.

9.1.1 & 9.1.3 CT/QMS

Coordinator and

all concerned

18. After formal implementation of QMS, for at least a period

of 3 months, an internal audit, followed by

corrections/corrective actions on the audit findings and

management review should be carried out

9.2, 10.2 & 9.3

and related QSPs

QMS

Coordinator, all

concerned and

Head of NRA

19. After each management review there will be follow up

actions on the decisions taken during review and taking

forward improvements where ever identified during review.

9.3, 10.1, 10.2 &

10.3 and related

QSPs

QMS Coordinator

20. Steps 17 to 19 are ongoing - All concerned

2729

It will be realistic to give a time frame of 9 to 12 months for completing all the above steps well. 2730

2731

If the NRA considers necessary to take help of a QMS consultant for implementation of ISO 9001 QMS, 2732

the NRA may appoint one. 2733

2734

2735

2736

2737


C. Certification of QMS

Although not mandated by ISO 9001, If the NRA management wishes to obtain a third-party

certification, the NRA may select an accredited Registrar/Certification body (data available on ISO

website) at an appropriate time, for example during step 18 above.

The selected certification body (CB) will first examine the NRA’s documents (quality manual, QSPs,

SOPs & other documents) for their conformity to ISO 9001. Thereafter once NRA has completed all

activities satisfactorily (i.e. up to step 19 above), the CB will arrange an audit of the NRA’s QMS and

based upon the audit results will issue the certificate. Certification is generally valid for a period of 3

years and for the maintenance of certification, annual surveillance audits are also carried out by same

CB after certification.

2738

2739

6. Considerations to ensure integrated implementation of QMS in 2740

NRA 2741

2742

Implementing ISO Standard 9001:2015 in departments or institutions of the NRA is feasible; however, 2743

integration of several functions of NRAs into one comprehensive and effective system represents a 2744

challenge. Executive coordination between different departments or offices in the decentralized model 2745

or institutions in the discrete model is critical and challenging. This is not just a question of regulatory 2746

affairs because the challenges are widespread. Capacity building and strengthening is essential for 2747

coordinating efforts between institutions involved in achieving good implementation of drug policies, 2748

frameworks, others. 2749

2750

Potential mechanisms that can help in QMS implementation: 2751

2752

• Strong coordination mechanism is established including communication, 2753

• High level support from TM for QMS implementation, 2754

• Assembly of a high-level executive committee to enforce understanding and commitment to QMS 2755

implementation and maintenance, 2756

• Empowerment of the NRA by the MoH with authority to drive QMS implementation, 2757

• Sustainability of QMS would be facilitated if part of the legal framework, e.g. a decree/mandate- 2758

or other legal means supported it, 2759

• Including QMS in the national medicine policy, 2760

• Including responsibility for contributing to QMS in staff job description, 2761


91

• Training all staff in QMS via courses, conferences, meetings, online platform. Trainings should be 2762

relevant to the regulatory functions, 2763

• Creation of a QMS unit in the NRA or as a minimum appointment of a QMS responsible officer 2764

with the appropriate level of authority, 2765

• Engagement of all stakeholders, 2766

• Creation of a portal for information-sharing among different stakeholders which would allow NRAs 2767

to share their procedures for QMS implementation, documentation, models, frameworks, tools. It 2768

would provide an opportunity for QMS networking among NRAs, 2769

• Coordination of KPIs between different functions so that all in the NRA speak the same QMS 2770

language, integrated and responsive to achieving the strategic plan and the establishment and 2771

maintenance of the QMS 2772

• Monitoring of the process flow between each party, e.g. MA sharing data with NCL and other 2773

functions as needed, 2774

• WHO recommendation for the TM to enforce QMS for all parties of the NRA, 2775

• As a part of the enforcement concept, and for some NRA models, creation of a high level (e.g. in 2776

the MOH) QMS unit with an external audit team, 2777

• Creation of a technical unit with one representative from each organization or department which 2778

meets at regular intervals for coordination, communication and data analysis, 2779

• Practice by many vaccine producers is to have a two-level system: a management review team 2780

which is high-level management who meet once a year and a quality management team which is 2781

more technical. The technical team meets and discusses KPIs monthly or quarterly. A similar 2782

approach could be considered by NRAs. 2783


References and further reading 2784

NOTE: References listed in this section, and their numbering throughout the guideline, will be 2785

corrected and updated in the final stages of guideline development. 2786

2787

[1] WHA67.20 Resolution http:// http://apps.who.int/gb/ebwha/pdf_files/WHA67/A67_R20-en.pdf 2788

Accessed on 30 September 2017. 2789

[2] WHO Expert Committee on Specifications for Pharmaceutical Preparations. Fiftieth Report. WHO 2790

Technical Report Series 996, 2016; pp 3-4. 2791

[3] Establishment of the WHO Expert Committee on Biological Standardization 2792

http://www.who.int/biologicals/expert_committee/en/ (last accessed 20.03.13). 2793

[4] WHO Expert Committee on Biological Standardization. Forty second Report. Guidelines for 2794

national authorities on quality assurance for biological products. WHO Technical Report Series 822, 2795

1992; Annex 2. 2796

[5] WHO Expert Committee on Biological Standardization. Forty-fifth Report. Regulation and 2797

licensing of biological products in countries with newly developing regulatory authorities. 2798

WHO Technical Report Series 858, 1995; Annex 1. 2799

[6] Good Regulatory Practices: Guideline for National Regulatory Authorities for Medical Products. 2800

WHO/DRAFT/ September 2016. 2801

[7] International Organization for Standardization, ISO 9001:2015. Quality management systems- 2802

Requirements Fifth edition 2015-09-15. Reference number: ISO 9001:2015 (E) 2803

[8] International Organization for Standardization, ISO 9000:2015. Quality management systems- 2804

Fundamentals and vocabulary. Edition 2015 2805

[9] International Organization for Standardization, ISO 9004. Quality of an Organization -Guidance to 2806

achieve sustained results. ISO 9004:2018 2807

[10] International Organization for Standardization, ISO 19011: 2018 provides guidance on auditing 2808

management systems. Second edition 2011-11-11 2809

[11] Guide to the Implementation of a Quality Management System for National Meteorological and 2810

Hydrological Services (2013 edition), World Meteorological Organization, WMO-No. 1100, 2013, 2811

http://www.wmo.int/pages/prog/hwrp/qmf-h/documents/ext/wmo_1100_en.pdf , accessed on 30 2812

September 2017 2813

[12] Guidance for the Implementation of a Quality Management System in Drug Testing 2814

Laboratories: https://www.unodc.org/documents/scientific/QMS_Ebook.pdf , accessed on 30 2815

September 2017. 2816

[13] EDQM, Quality Management Documents https://www.edqm.eu/en/quality-management-2817

guidelines-86.html. Accessed on 30 September 2017. 2818

http://apps.who.int/gb/ebwha/pdf_files/WHA67/A67_R20-en.pdf

http://www.who.int/biologicals/expert_committee/en/

http://www.wmo.int/pages/prog/hwrp/qmf-h/documents/ext/wmo_1100_en.pdf

https://www.unodc.org/documents/scientific/QMS_Ebook.pdf

https://www.edqm.eu/en/quality-management-guidelines-86.html

https://www.edqm.eu/en/quality-management-guidelines-86.html


93

[14] ICH Harmonised Tripartite Guideline. Pharmaceutical Quality System Q 10. 2819

Current Step 4 version, 4 June 2008 2820

[15] Quality Management Training for Blood Transfusion Services. 2821

http://www.who.int/bloodsafety/publications/who_eht_05_03a.pdf, accessed on 30 September 2017 2822

[16] Quality systems requirements for national good manufacturing practice inspectorates. 2823

http:// http://apps.who.int/medicinedocs/documents/s22112en/s22112en.pdf, accessed on 30 2824

September 2017 2825

[17] WHO good practices for pharmaceutical quality control laboratories. 2826

http://www.who.int/medicines/areas/quality_safety/quality_assurance/GoodpracticesPharmaceuticalQ2827

ualityControlLaboratoriesTRS957Annex1.pdf, accessed on 30 September 2017 2828

[18] A new evolution for quality management in the automotive industry. 2829

https://www.iso.org/news/2016/08/Ref2109.html, accessed on 30 September 2017 2830

[19] Guidelines on Quality Management for Multidisciplinary Occupational Health Services. WHO 2831

European Centre for Environment and Health, Bilthoven 2832

http://apps.who.int/iris/bitstream/10665/108268/1/E68239.pdf, accessed on 30 September 2017. 2833

[20] Manual on the Quality Management System for Aeronautical Information Services. International 2834

Civil Aviation Organization. First Edition-2010 2835

https://www.icao.int/APAC/Meetings/2011_AAITF6/QMS%20manual%20formated%20for%20edit2836

%20and%20submission%20to%20DOC%20Control%20_MH_.pdf, accessed on 30 September 2017. 2837

[21] ISO Technical Specification ISO/TS 9002: 2016 “Quality management systems –Guidelines for 2838

the application of ISO 9001:2015” 2839

[22] ISO/IEC 17020:2012 Conformity assessment- Requirements for the operation of various types of 2840

bodies performing inspection. 2841

[23] ISO/IEC 17025: 2017 General requirements for the competence of testing and calibration 2842

laboratories 2843

[24] ISO 19011:2018 Guidelines for auditing management systems 2844

[25] ISO 31000:2018 Risk management- Guidelines 2845

[26] ISO 9004: 2018 Quality management - Quality of an organization - Guidance to achieve 2846

sustained success 2847

[27] ISO 37001:2016 Anti-bribery management systems- Requirements with guidance for use 2848

[28]ISO/IEC 27001:2013 Information security management- Requirements 2849

[29] ISO 14001:2015 Environmental management systems- Requirements with guidance for use. 2850

[30] Request for Quality Metrics- Guidance for Industry. U.S. Department of Health and Human 2851

Services Food and Drug Administration. Center for Drug Evaluation and Research (CDER). Center 2852

for Biologics Evaluation and Research (CBER). July 2015. Pharmaceutical Quality/CMC. Current 2853

Good Manufacturing Practices (CGMPs) 2854

http://www.who.int/bloodsafety/publications/who_eht_05_03a.pdf

http://apps.who.int/medicinedocs/documents/s22112en/s22112en.pdf

http://www.who.int/medicines/areas/quality_safety/quality_assurance/GoodpracticesPharmaceuticalQualityControlLaboratoriesTRS957Annex1.pdf

http://www.who.int/medicines/areas/quality_safety/quality_assurance/GoodpracticesPharmaceuticalQualityControlLaboratoriesTRS957Annex1.pdf

https://www.iso.org/news/2016/08/Ref2109.html

http://apps.who.int/iris/bitstream/10665/108268/1/E68239.pdf

https://www.icao.int/APAC/Meetings/2011_AAITF6/QMS%20manual%20formated%20for%20edit%20and%20submission%20to%20DOC%20Control%20_MH_.pdf

https://www.icao.int/APAC/Meetings/2011_AAITF6/QMS%20manual%20formated%20for%20edit%20and%20submission%20to%20DOC%20Control%20_MH_.pdf


[31] Guidance on good data and record management practices. WHO Technical Report Series 996, 2855

annex 5; 2016. 2856

[32] Guidelines on Clinical Evaluation of Vaccines: Regulatory Expectations. WHO/Draft/29 2857

October 2015 2858

[33] Quality Assurance of Medicines terminology database. List of terms and related guidelines 2859

http://www.who.int/medicines/services/expertcommittees/pharmprep/20160302_QASterminologyDB.2860

pdf 2861

[34] Quality Management Principles. ISO brochure. 2862

https://www.iso.org/files/live/sites/isoorg/files/archive/pdf/en/pub100080.pdf 2863

2864

Authors and acknowledgements 2865

2866

Mr Y. Al-Nujaym, Saudi Food and Drug Authority, Saudi Arabia; Dr O.A.M.A. Badary, National 2867

Organization for Drug Control and Research, Egypt; Ms G.F. Ferreiro, Centro para el Control Estatal 2868

de Medicamento, Equipos y Dispositivos Médicos, Cuba; Ms A. Julsing, Medicines Control Council, 2869

South Africa; Ms Y. Lee, Ministry of Food and Drug Safety, Republic of Korea; Dr R. Lino de Brito, 2870

Agencia Nacional de Vigilancia Sanitaria, Brazil; Ms L. Margaryants, Scientific Centre of Drug and 2871

Medical Technology Expertise, Armenia; Ms G. Mkomagi, Tanzania Food and Drug Authority, 2872

Tanzania; Ms M. Muñozcano Quintanar, Comisión Federal para la Protección contra Riesgos 2873

Sanitarios, Mexico; Mr G. Muthuri Francis, Pharmacy and Poisons Board, Kenya; Ms A. Olivares, 2874

Comisión Federal para la Protección contra Riesgos Sanitarios, Mexico; Mr P. Osatapirat, Thailand 2875

Food and Drug Administration, Thailand; Ms H. Qorani, Jordan Food and Drug Administration, 2876

Jordan; and Dr C. P. Alfonso, Mr S. Arora, Dr R.O.A. Dehaghi, Dr N. Dellepiane, and Ms S. 2877

Ramirez, World Health Organization, Switzerland.2878


95

Appendix 1. Integration of QMS into the WHO Global Benchmarking Tool 2879

The WHO Global Benchmarking Tool (GBT) is used to assess the level of implementation of QMS in NRA. The QMS indicator consists of 14 self-scored 2880

sub-indicators to identify the degree of QMS implementation and the existing gaps across the NRA. The equivalence of the 14 sub-indicators in the GBT 2881

with the ISO 9001: 2015 requirements is shown in Table 1. 2882

The concept of maturity level (ML) from ISO 9004:2009 into the stratification of QMS scores is incorporated in the GBT and has been implemented for 2883

some time by the Benchmarking of the European Medicines Agencies (BEMA). Maturity levels take into account the criticality of indicators and the minimal 2884

required capacity of a regulatory system to control and implement proper oversight of health products. 2885

2886

The maturity level indicates the status at which each sub-indicator performs. As maturity level one addresses the legal framework of the regulatory system, 2887

the QMS gap analysis starts at maturity level two. In maturity level two, an organization operates in a reactive mode with an evolving national regulatory 2888

system that partially performs essential regulatory functions. In maturity level three, an organization has a stable, well-functioning and integrated regulatory 2889

system which is accompanied with essential capacity implemented in all functions. Maturity level four implies a regulatory system that performs at advanced 2890

level with continual improvement which support all implemented regulatory functions. The basis for climbing the maturity level ladder for robust regulatory 2891

systems strengthening is for NRAs to fill the gaps at the lowest level before taking a step up. Maturity level three is currently being considered as the target 2892

performing level of national regulatory authorities for implementation of the WHA Resolution 67.20. 2893

2894

Table 1. Equivalence of the 14 GBT sub-indicators with the ISO 9001: 2015 requirements 2895

Sub-

indicator Description

ISO 9001:2015

Clause Requirements ML

RS05.07 Requirements for documentation management as well as

traceability of regulatory activities are established 7.5 Documented information 2

RS05.01 Top management demonstrates commitment and leadership to

develop and implement QMS 5.1.1 Leadership and commitment 3

RS05.02 Quality policy, objectives, scope and action plans for establishment

of the QMS are in place and communicated to all levels 4.3, 5.2.1, 5.2.2, 6.2

Quality policy, objectives,

scope, and action plans 3

RS05.03 Organizational chart, roles and responsibilities to establish the

QMS are defined and in place 5.3

Organizational roles, and

responsibilities 3


RS05.04 Enough competent staff is assigned to develop, implement and

maintain the QMS 7.1.2, 7.2

Human resources and

competency 3

RS05.09 The externally provided products and services relevant to

regulatory activities are controlled through established mechanisms 8.4

Control of externally provided

process, products and services 3

RS05.11 Internal and/or external audits of the QMS are established and

conducted at planned intervals 9.2 and ISO 19011 Internal audit 3

RS05.05 The regulatory authority establishes required mechanisms to

continually improve the QMS 10.3 Continual improvement 4

RS05.06

The NRA has identified its regulatory processes, determined their

interactions and defined the methods needed to control these

processes

4.4, 8.3, 8.5.1 Operation 4

RS05.08 External and internal issues including relevant potential risks are

defined and assessed periodically for proper risk mitigation 6.1, 4.1

Actions to address risk and

opportunities 4

RS05.10

A mechanism to evaluate the satisfaction of internal and external

customers and other interested parties is in place for system

improvement

9.1.2, 9.1.3 Customer satisfaction 4

RS05.12

Corrective actions, and actions to address risks and opportunities,

are implemented and documented and their effectiveness is

verified

9.1.3, 10.2 Non-conformity and corrective

action 4

RS05.13 Top management reviews and documents the organization’s QMS

at planned intervals (management review) 9.3 Management review 4

RS05.14 A mechanism is established to evaluate and demonstrate the

effectiveness of training activities 7, 7.2 Training 4

2896