national external quality assessment (eqa) scheme for ... · 3 cervical screening programme -...

Cervical Screening Programme National external quality assessment (EQA) scheme for gynaecological cytopathology

Ninth edition, March 2018 Public Health England leads the NHS Screening Programmes

2

Cervical Screening Programme - National external quality assessment (EQA) scheme for gynaecological cytopathology

About Public Health England

Public Health England exists to protect and improve the nation’s health and wellbeing,

and reduce health inequalities. We do this through world-leading science, knowledge

and intelligence, advocacy, partnerships and the delivery of specialist public health

services. We are an executive agency of the Department of Health and Social Care,

and a distinct delivery organisation with operational autonomy. We provide government,

local government, the NHS, Parliament, industry and the public with evidence-based

professional, scientific and delivery expertise and support.

Public Health England, Wellington House, 133-155 Waterloo Road, London SE1 8UG

Tel: 020 7654 8000 www.gov.uk/phe

Twitter: @PHE_uk Facebook: www.facebook.com/PublicHealthEngland

About PHE Screening

Screening identifies apparently healthy people who may be at increased risk of a disease

or condition, enabling earlier treatment or better informed decisions. National population

screening programmes are implemented in the NHS on the advice of the UK National

Screening Committee (UK NSC), which makes independent, evidence-based

recommendations to ministers in the 4 UK countries. The Screening Quality Assurance

Service ensures programmes are safe and effective by checking that national standards

are met. PHE leads the NHS Screening Programmes and hosts the UK NSC secretariat.

www.gov.uk/phe/screening

Twitter: @PHE_Screening Blog: phescreening.blog.gov.uk

Prepared by: Sharon Whitehurst

For queries relating to this document, please contact: [email protected]

© Crown copyright 2018

You may re-use this information (excluding logos) free of charge in any format or

medium, under the terms of the Open Government Licence v3.0. To view this licence,

visit OGL. Where we have identified any third party copyright information you will need

to obtain permission from the copyright holders concerned.

Published: February 2018

PHE publications PHE supports the UN

gateway number: 2017796 Sustainable Development Goals

http://www.gov.uk/phe

https://twitter.com/PHE_uk

http://www.facebook.com/PublicHealthEngland

http://www.gov.uk/phe/screening

https://twitter.com/phe_screening

https://phescreening.blog.gov.uk/

mailto:[email protected]

https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/

https://www.gov.uk/government/collections/sustainability-and-public-health-a-guide-to-good-practice

3


Document information

Only the website version of this document is controlled and located at

www.gov.uk/government/collections/cervical-screening-professional-guidance.

Any downloaded or electronically transmitted file is not a controlled document and it is

the responsibility of individuals to make sure that they are accessing the most recent

version.

The EQA Management Board has approved this document (January 2018).

UKAS accredited Proficiency Testing Provider number 7597.

Accreditation schedule.

https://www.gov.uk/government/collections/cervical-screening-professional-guidance

https://www.ukas.com/wp-content/uploads/schedule_uploads/00013/7597Proficiency%20Testing%20Multiple.pdf

4


Contents

About Public Health England 2

About PHE Screening 2

Document information 3

Contents 4

1. Introduction 7

2. Summary of changes from eighth edition 7

3. Joint Working Group for Quality Assessment in Pathology 9

4. Scope 10

5. Aims 10

6. General information 11

6.1. Scheme provider 11

7. Scheme enquiries 11

8. Organisation and management 12

8.1. Host organisation 12

8.2. Staffing 12 8.3. Governance 14

9. EQA agreement 17

9.1. General terms and conditions of participation 17 9.2. Eligibility requirements 18

9.3. Enrolment 20 9.4. Participant responsibilities 20

9.5. Confidentiality 21 9.6. Subcontracting laboratories 22

9.7. Review of submitted slides 26 9.8. Professional code of conduct 27 9.9. Financial aspects 28

10. Preparation for the EQA round 29

10.1. Organising transfer of slides 29 10.2. Tracking the location of slides 29 10.3. Slide bank 29

10.4. Slide breakage, damage or loss 30 10.5. Assembly of slide sets 30

10.6. Information and instructions for participants 31 10.7. Other documentation 32

11. Conditions for examining EQA slides 34

11.1. Non-medical staff 34 11.2. Consultant biomedical scientists in cervical cytology 35 11.3. Medical staff 36

5


11.4. Trainee staff 37 11.5. Additional slide sets 37

12. The assessment 38

12.1. Recording and submitting responses 38

12.2. Following assessment 40 12.3. Reports 41 12.4. Certificates of participation 41 12.5. CPD and CME credits 41

13. Slide consensus 42

13.1. Consensus opinion 42

13.2. Data handling and calculations 42 13.3. Slides with consensus different to the submitted classification 42 13.4. Consensus on negative, inadequate and normal slides 43 13.5. Identifying missed dyskaryosis 44

13.6. Grading consensus for abnormal slides 44 13.7. Slides not reaching consensus 45 13.8. Slides with abnormal consensus but no grade consensus 45

14. Scoring 47

14.1. Consensus slides 47 14.2. Cytology screeners and checkers 47

14.3. Pathologists and consultant biomedical scientists 48 14.4. Transcription errors 49

14.5. Final scores 49

15. Assessment of performance 50

15.1. Cytology screeners and checkers 50

15.2. Pathologists and consultant biomedical scientists 50 15.3. Identification of substandard performance 50 15.4. Persistent substandard performance 52

15.5. Action points for persistent substandard performance 53

16. Communication to and from the EQA scheme 55

16.1. Constructive criticism, comments and compliments 55

16.2. Formal complaints 55 16.3. Formal appeals 55 16.4. Protocol changes 56 16.5. Participants’ meetings 56 16.6. Audit 56 16.7. Surveys 57 16.8. Annual report 58

References 59

Appendix 1: procedure for reporting and investigating alleged collusion 60

Appendix 2: EQA timeline 63

Appendix 3: referral sheet 64

Appendix 4: scoring matrix for cytology screeners/checkers 65

Appendix 5: scoring matrix for pathologists/consultant biomedical scientists 66

6


Appendix 6: table of performance examples with action points 67

Appendix 7.1: flowchart of actions taken when first action point is reached 68

Appendix 7.2: flowchart of actions taken when second action point is reached 69

Appendix 8: formal complaints procedure 70

Appendix 9: appeals procedure 71

7


1. Introduction

This document provides guidance on the scheme design, interpretation of results and

the general requirements and responsibilities for laboratories and participants. It also

sets out the principles and methodology of operation for a consistent delivery across

the nation.

2. Summary of changes from eighth

edition

Section

Sub-section

Summary

8 8.3.1 Clarified that the EQA management board responsibilities are documented in the terms of reference for that committee.

9 9.1 Sentence inserted to confirm that all scheme documents, electronic and paper, are retained in accordance with ‘The retention and storage of pathological records and specimens’, fifth edition April 2015.

9.1.1 Clarified that Royal Mail Special Delivery is the method of approved transport. Deleted sentence, which referred to ‘inter-laboratory’ services and international couriers.

9.5 Clarified that laboratory managers must have systems in place to monitor and review on-going staff participation and performance in EQA, to monitor trends in results as appropriate and recommended that this be formally recorded within the departmental appraisal process. Deleted previous sentence…’participants will be asked to disclose their individual results to laboratory management’.

9.6.2 Clarified that only copy slides must be submitted for inclusion in the EQA rounds.

9.6.7 Clarified that where original material is submitted for EQA then this contravenes the scheme protocol.

8


9.8 This section was revised to describe the intentions of the scheme in respect of potential collusion.

12 12.1 Paragraph 9, heading changed to ‘Recording and submitting results’. Clarified that participants must upload their results immediately after screening their EQA slides and while the slides are still in the laboratory.

16 16.4 Sentence inserted to clarify that the EQA facilitator notifies participants about agreed changes to the scheme and when they come into effect.

Appendix 1 Revised to describe the process for reporting and investigating alleged collusion.

9


3. Joint Working Group for Quality

Assessment in Pathology

The Joint Working Group for Quality Assessment in Pathology (JWG) is a committee of

the Royal College of Pathologists (RCPath), reporting to the Professional Performance

Panel. The JWG is responsible for the oversight of all pathology external quality

assessment (EQA) in the UK, including the approval and registering of schemes, and

the setting of policy and maintenance of appropriate professional standards. As part of

this remit, the JWG monitors the EQA performance of clinical laboratories in the UK.

JWG uses discipline-specific panels, known as National Quality Assurance Advisory

Panels (NQAAP), which report those laboratories that have failed to rectify quality

problems to the JWG.

The JWG subsequently works with failing laboratories to improve standards and has

the responsibility to report persistent poor performance to the Care Quality Commission

and UKAS. Figure 1 outlines the pathway for pathology EQA in the UK.

Figure 1. Pathology EQA in the UK

10


4. Scope

The NHS Cervical Screening Programme (NHSCSP) External Quality Assessment

Scheme for Gynaecological Cytopathology and the guidance set out in this document

applies to NHS cervical screening laboratories in England and Wales, and laboratories

outside the United Kingdom which are signed up to the EQA agreement.

5. Aims

The scheme aims to provide an acceptable degree of reliability and consistency by

educating, advising and supporting all participants.

The scheme is based on the assessment of slide sets by all staff who report cervical

cytology on behalf of the NHSCSP. Individual reporting is compared to consensus

reporting to determine whether an individual’s performance falls within an acceptable

range.

There are 2 rounds (2 circulations of slide sets) in each EQA year. An EQA year runs

from 1 April to 31 March.

There are 10 slides in each slide set per round, which are used for performance

analysis.

Up to 2 additional ‘special interest’ slides may be circulated with the set. These are

clearly identified as being for educational interest, and are not included in the

calculation of final assessment results.

11


6. General information

6.1. Scheme provider

The provider organisation is a Public Health England (PHE) executive agency of the

Department of Health for its Young Person and Adult Screening Programmes.

Postal address:

Young Person and Adult Screening Programmes (YPASPs)

Public Health England

Level 2, Vulcan House Steel

6 Millsands

Sheffield

S3 8NH

Corporate information: www.gov.uk/PHE

7. Scheme enquiries

All general enquiries must be made directly to the National EQA Team:

National EQA Team

Screening QA Service (Midlands & East) 1st Floor

5 St Philip’s Place

Birmingham

B3 2PW

Tel: (0121) 214 9130

Email: [email protected]

The telephone line and generic email account are manned on weekdays during normal

office hours (9am to 5pm). Staff with dedicated responsibility for EQA services deal with

all enquiries.

http://www.gov.uk/PHE

mailto:[email protected]

12


8. Organisation and management

8.1. Host organisation

The Screening Quality Assurance (QA) Service (Midlands & East) hosts and operates

the EQA scheme.

The head of screening QA service (Midlands and East) has national responsibility for

external quality assessment as part of their role for cervical screening. This includes:

day to day support and advice to the EQA operations team

line management for the EQA facilitator and administrative function

responsibility for EQA online through the line management of the SQAS staff

who develop and maintain the system

joint authorisation, with the EQA facilitator, of individual end of round reports

review and authorisation of all operational work instructions and associated

documentation

chairs the EQA operational group meetings

A dedicated EQA manager and administrative team operate the day-to-day

organisation and management of the scheme.

8.2. Staffing

8.2.1. Scheme Organiser

PHE subcontracts the scheme organiser and deputy scheme organiser roles.

The scheme organiser and deputy:

is a consultant cytopathologist, a consultant histopathologist with an interest in

cervical cytopathology or a consultant biomedical scientist who holds the

advanced specialist diploma in cervical cytology

is professionally regulated through their registration body

is a participant in the scheme and has experience of selecting and reviewing

slides for EQA

holds a senior post in a laboratory providing gynaecological cytopathology for

the NHSCSP

participates in an appropriate CPD/CME scheme, and fulfils NHSCSP and pre-

existing professional body requirements for continuing professional development

has some knowledge of quality management systems and accreditation

requirements

13


may also be the chair of the scheme’s EQA steering group

The scheme organiser and his or her deputy:

collaborates with the provider operation in setting annual objectives for the EQA

scheme and its strategic direction

receives advice from the EQA steering group on the practical aspects of scheme

design and operation

is responsible for the oversight of poor performance and liaises with NQAAP in

the event of persistent substandard performance

oversees difficult and sensitive situations associated with participant poor

performance, complaints and appeals

approves the scheme’s annual report(s) for service users and management

promotes the value of the EQA scheme and its activities at relevant professional

meetings

promotes and provides advice on the educational elements of the scheme

responds urgently to any situation arising during the EQA round which has the

potential to adversely affect a patient’s management

8.2.2. EQA facilitator

The national external quality assessment (EQA) facilitator is employed by PHE. This

individual has the authority to manage the day-to-day operation of the scheme in line

with published protocols and guidelines. This includes responsibility for the

management and operation of the quality management system. The EQA facilitator

reports to the scheme organiser in a professional capacity, and is accountable to the

head of screening quality assurance (Midlands and East) who is also their direct line

manager.

The EQA facilitator is responsible for:

day to day management of the EQA scheme

development and maintenance of local administrative and quality systems in line

with published protocols and quality procedures

undertaking the role of scheme quality manager

overseeing arrangements for giving laboratories and participants appropriate

notice of EQA requirements

identifying and highlighting actual or potential issues in relation to the scheme’s

operation and bringing these to the attention of the appropriate manager

maintaining all aspects of confidentiality and for the safe storage, recording and

tracking of samples submitted for use in EQA

working closely with the local support team to identify areas for development in

relation to the online EQA reporting system

14


preparing the audit strategy and internal audit programme and making sure

findings are recorded and completed appropriately

preparing for external audits and maintenance of routine activities in line with

UKAS requirements

producing accurate reports as requested for management, steering group and

operational review meetings

taking action in the event of poor performance as documented in this protocol,

and act as the scheme secretary as required

The EQA facilitator is supported by a local administration team hereafter termed

national EQA team. The head of screening quality assurance and 2 administrative

assistants provide cover for the EQA facilitator in his or her absence, appropriate to the

cover required.

The national EQA team comprises staff with defined responsibilities to support the EQA

facilitator as well as responsibilities for management and IT/data support. The cytology

education manager (under contract to PHE Screening) maintains the national scheme

protocol. Appropriate cover is provided in his or her absence.

8.3. Governance

Figure 2 below illustrates the EQA relationships within the scheme.

8.3.1. EQA management board

The EQA management board is an executive team which provides a review and

governance mechanism for the scheme. The EQA Management Board terms of

reference provide clear details of its roles and responsibilities.

The EQA management board acts on behalf of and has direct access to highest-level

management Membership includes: national cervical screening programme manager,

scheme organiser, deputy organiser, EQA facilitator and head of screening quality

assurance (Midlands and East).

The national cervical screening programme manager has oversight of the EQA scheme

and reports directly to the national programmes lead, young person and adult screening

programmes. The national cervical screening programme manager does not have a

role in operational delivery and is chair of the EQA management board. The meetings

are administered by the national EQA team.

The EQA management board is responsible for the strategic direction, planning and

development of the scheme. The EQA management board has considered all the

essential requirements of planning, including:

15


statistical design

suitability of test items

frequency of assessment and evaluation criteria

operational activity

quality management system

resources required to implement the processes

number of participants

cross-boundary working

8.3.2. EQA steering group

The EQA steering group is set up to provide a supporting and advisory role and meets

on a quarterly basis. Members include individuals from participating laboratories to

make sure the requirements and interests of participants are represented. The EQA

steering group provides technical advice and assistance on scheme design and

operational issues. Its role does not extend to involvement in performance issues

unless these are related to the scheme’s design and performance monitoring systems.

The scheme organiser chairs the meetings. The meetings are administered by the

national EQA team.

8.3.3. Operational group

The operational group is set up to discuss all operational activity and meets on a

quarterly basis. Members include the national EQA administration team, staff

responsible for the maintenance of EQA online and a representative from the national

programme team responsible for protocol maintenance. The scheme organiser or

deputy organiser may attend to provide advice (as necessary). The Head of Screening

QA Services (Midlands and East) chairs the meetings. The meetings are administered

by the national EQA team.

8.3.4. Online web-based service for participants

The nationwide transition to online data entry and results management was made on 1

April 2015. The bespoke EQA website (hereafter termed ‘EQA online’) is secure and

access is via an N3 connection.

EQA online is managed by the national EQA team in Birmingham and provides a

secure facility for existing participants to enter their results for the current round, and to

view, download and/or print their individual personal reports.

Suitable provision is made for individuals who are unable to access the website through

the use of manual recording and data entry systems. Laboratories without an N3

connection can scan and email their EQA response forms securely.

16


Access to this service is achieved through the participant registration and sign up

process.

An email facility is provided for account queries including lost or forgotten passwords

and login details.

Figure 2. Inter-relationships within the EQA scheme

DIRECTOR OF SCREENING Public Health England

Screening Programmes and Screening QA Service

Young Person and Adult

Screening Programmes

Cervical Screening Programme

SCHEME PROVIDER EQA Management Board

Screening Quality

Assurance Service

SCHEME OPERATION & QUALITY

MANAGEMENT National EQA Team

SCHEME ORGANISER

(Professional Accountability)

EQA STEERING GROUP (Advisory)

PARTICIPANTS/ LABORATORIES

Slide selection and review

17


9. EQA agreement

There is an agreement between the PHE Screening QA Service and NHS cervical

screening laboratories and their staff who report gynaecological cytopathology for the

NHS Cervical Screening Programme. It is also applicable to private laboratories and

those from outside the UK who participate in the scheme. The general terms and

conditions of participation for all parties are documented in the agreement. The

agreement is subject to periodic review.

9.1. General terms and conditions of participation

Participation in the EQA scheme is mandated for all staff who report gynaecological

cytology for the NHSCSP. This applies to all:

cytology screeners

biomedical scientists

consultant biomedical scientists (also known as advanced BMS practitioners in

cervical cytology)

pathologists

Participation is mandated for all staff whether employed on a full-time, part-time,

permanent or temporary basis.

Participation in relevant accredited EQA schemes is a prerequisite for laboratories

registered with UKAS and seeking accreditation to ISO 15189:2012.2

Laboratories are responsible for notifying the national EQA team of any changes in

service provision, and changes in staffing (whether permanent or temporary) including

new recruits, leavers and locums.

Where possible, the EQA team will make reasonable adjustments to facilitate access to

the scheme for all potential participants. Laboratories are responsible for notifying the

national EQA team if a member of staff has any particular needs, which may require an

adjustment as to how they participate in EQA.

Participants must use the LBC methodology that they routinely report in - ThinPrep or

SurePath. Individuals who routinely report in both ThinPrep and SurePath must

alternate their participation between the 2 technologies by agreement with the national

EQA team. These individuals must not participate in both technologies in a single EQA

round.

18


Laboratories must confirm their acceptance of all terms and conditions described in the

EQA agreement in order that their staff can enrol on the scheme. There is no

requirement for annual re-enrolment as this is managed through periodic review and

renewal of the EQA agreement.

Laboratories are responsible for the safekeeping of EQA slides during each round.

Laboratories must report damage or breakages to the EQA facilitator immediately via

email. The EQA facilitator advises the laboratory how to proceed in these

circumstances.

Slides must be protected from sunlight when not in use in order to preserve their quality

for subsequent participants.

All data entry and result reports are managed via the EQA website.

All scheme documents, electronic and paper, are retained in accordance with ‘The

retention and storage of pathological records and specimens’, fifth edition April 2015. 5

9.1.1. Labelling, packaging and transport

Laboratories are provided with detailed instructions to make sure that EQA slides are

correctly and safely packaged and labelled in line with Royal Mail requirements. Slides

must be transported by Royal Mail Special Delivery. This method allows packages to

be tracked during transit and upon receipt. Other methods of transport may be used

subject to approval by the scheme’s management. Laboratories must not use

unauthorised methods of slide transport.

9.1.2. Sanctions for non-compliance with the EQA agreement

Laboratories, which do not conform to the protocol requirements, are in breach of the

EQA agreement and may be removed from the scheme. The scheme has established a

formal escalation policy. Removal from the scheme presents a risk to a laboratory

achieving or maintaining ISO 15189:2012 accreditation.

9.2. Eligibility requirements

9.2.1. Students and trainees

Students and trainees who intend to pursue a career that includes cervical cytology,

and who are working in an NHSCSP cervical screening laboratory or a cervical

screening laboratory under contract to the NHS, are eligible to participate in the

scheme for educational purposes only. Scores are excluded from the performance

analysis. Laboratory managers must make sure that this staff group discusses results

with the relevant training officer/supervisor.

https://www.rcpath.org/resourceLibrary/the-retention-and-storage-of-pathological-records-and-specimens--5th-edition-.html

https://www.rcpath.org/resourceLibrary/the-retention-and-storage-of-pathological-records-and-specimens--5th-edition-.html

19


9.2.2. Locum staff

Locum staff participate in the national scheme and comply fully with all conditions and

arrangements if they work:

in an NHSCSP cervical screening laboratory

in a cervical screening laboratory under contract to the NHS

Note that production of a satisfactory report and/or evidence of participation in EQA

does not guarantee eligibility for employment in an NHSCSP cervical screening

laboratory or a cervical screening laboratory under contract to the NHS.

9.2.3. Staff working in private laboratories under contract to the NHS

Staff working in private laboratories where screening is undertaken for the NHSCSP

participate in the scheme must comply fully with its conditions and arrangements.

9.2.4. Staff working in private laboratories in the UK

The EQA agreement also describes the conditions of participation for staff working in

private laboratories (non-NHSCSP screening) within the UK, and staff working in

cervical cytology laboratories outside the UK.

Staff working in private (non-NHS) laboratories where non-NHSCSP screening is

performed may wish to subscribe to the scheme. Participation is not mandated for

these individuals.

Individuals working in private laboratories who choose to participate in the scheme

must note that their responses are not included in the consensus calculations to

determine slide classifications. In addition, their overall scores are not included in the

statistical analysis used to determine individual performance.

Results and feedback to these participants are determined by comparison against

participants who do perform screening work for the NHSCSP.

9.2.5. Staff working in laboratories outside the UK

Subject to operational capability and resources, the scheme is open to laboratories

from outside the UK subject to their meeting specified terms and conditions of the EQA

agreement.

Participants working in laboratories outside the UK must note that their responses are

not included in the consensus calculations to determine slide classifications. In addition,

20


their overall scores are not included in the statistical analysis used to determine

individual performance.

Non-UK laboratories declare the pathway for professional accountability within their

employing organisation and regulatory body under the terms and conditions of the EQA

agreement. Persistent poor performance is reported to the relevant regulatory body.

9.2.6. Ineligible staff

Individuals are not eligible to participate in the EQA scheme if they are not currently

screening and reporting.

9.3. Enrolment

On enrolment, participants are directed to the secure EQA website for general

guidance and instructions on how to set up and manage their EQA account.

Participants must use an employer issued email address to set up and operate an EQA

account. Participants unable to access EQA online using an N3 connection will be

advised by the national EQA team of the alternative arrangements for participation

using established manual systems.

Participants will be asked to confirm they have read and understood this document.

Participants who are unsure about any EQA website content or a scheme operational

matter must contact the EQA facilitator for guidance.

9.4. Participant responsibilities

9.4.1. Late participation

Participants must be available to complete the EQA during the time the slides are

present in the laboratory.

Provision can be made for staff who are unavailable on the agreed EQA dates for a

legitimate reason (for example due to pre-arranged annual leave or a short-term

authorised absence). These individuals may participate at a neighbouring laboratory, or

slides can be made available on an alternative date at the end of the round or at

another suitable point in the cycle (only one further date will be offered). Local

management is responsible for making sure that staff are allocated time away from the

laboratory to participate in EQA elsewhere if necessary.

21


9.4.2. Non-participation

Participants need to be aware of the consequences of non-participation in EQA.

An individual who is absent on the agreed EQA date, or who does not participate

without a legitimate reason, is classed as a non-participant without a valid reason. This

may lead to a result of poor performance being recorded.

If an individual fails to participate in 2 out of 3 rounds without legitimate reasons, this

information is passed to the:

scheme organiser

head of screening QA

head of screening QA responsible for the relevant laboratory (who alerts the

commissioners of the service)

chair of NQAAP

Legitimate reasons for non-participation include:

short or long-term authorised absence

maternity leave

pre-arranged annual leave

an extended period of sabbatical leave

9.5. Confidentiality

Information disclosure liabilities, responsibilities and procedures are communicated to

and followed by all EQA staff. This includes staff employed directly by and/or under

contract to PHE.

All participant data is confidential. All electronic data is stored in password-protected

files or in areas accessible only to authorised EQA staff. Paper based data is stored in

a secure, locked cabinet and access is restricted and limited to only authorised EQA

staff.

The scheme is confidential under the conditions of participation in EQA schemes

determined by the professional bodies through the Joint Working Group for Quality

Assurance (JWG).

Participants may choose to waive confidentiality for the purposes of discussion and

mutual assistance. They can also choose to waive confidentiality for regulatory or

recognition purposes.

22


The identities of participants are not made available except under circumstances stated

in the terms and conditions of participation. The EQA facilitator will only provide

participant names and corresponding codes:

when a participant is enrolled on the scheme

when a participant has requested a reminder of his or her code number either in

writing or e-mail or via the facility provided in their online EQA account

when circumstances dictate that the chair of the JWG on quality assurance is

notified of a member of medical staff or a consultant biomedical scientist who

has reached the second action point (see section 15.5.2). The chair of the JWG

may request that confidentiality is not broken in these circumstances

Laboratory managers must have systems in place to monitor and review on-going staff

participation and performance in EQA, and to monitor trends in results as appropriate. It

is recommended that this is formally recorded within the departmental appraisal

process.

At present, it is considered extremely unlikely that any ‘interested parties’ or

government regulatory authorities require participants’ results to be provided directly to

them.

If in the unlikely event the provider organisation receives a third party request from a

regulatory authority requiring EQA results, the source will be investigated and verified

by the national EQA team. The provider organisation will comply with a valid and

verified request, and notify the affected participants of this action in writing within 5

working days.

9.6. Subcontracting laboratories

PHE subcontracts with the NHSCSP cervical screening laboratories that provide slide

material, and the staff who review it. The provision of suitable slides and qualified staff

who perform the slide reviews is critical to the success and integrity of the scheme.

Cervical screening laboratories not under contract to the NHS (including those outside

the UK) are not quality assured by PHE or its equivalent in other parts of the UK. They

are not eligible therefore to submit slide material or provide staff to undertake slide

reviews since these services are not required to conform to national screening

programme standards and guidelines.

NHSCSP cervical screening laboratories or those under contract to the NHS are

accredited by or registered for accreditation with United Kingdom Accreditation Service

(UKAS). Laboratories that are not yet accredited to ISO 15189:2012 must make sure

that only qualified and competent staff undertake slide selection, submission and slide

review activities for the scheme.

23


Staff are deemed competent on the basis of:

qualification

experience

satisfactory participation in the scheme

employment by a laboratory which is contracted to provide a cervical screening

service to the NHSCSP

9.6.1. Source of slides

All NHSCSP cervical screening laboratories (or cervical screening laboratories under

contract to the NHS) are required to submit slides under the terms and conditions of the

EQA agreement.

Controls are in place to minimise loss or damage to slides during EQA activity and this

is monitored annually. Any loss or damage to stakeholder property which renders it

unsuitable for use is recorded and reported to the stakeholder. (Stakeholder property is

physical property including glass slides, participant information and scheme data.)

The total number of slides requested for any particular round is at least 3 times the

number that will be needed. This is because up to two-thirds of slides submitted may be

unsuitable due to poor technical quality or a lack of agreement on the classification of

the slide.

‘Spoke’ laboratories for liquid based cytology (LBC) processing arrangements are also

eligible to submit slides, provided that both the ‘hub’ and ‘spoke’ laboratories meet the

terms and conditions described in the EQA agreement.

Laboratories must keep a record of the slides and supporting documents submitted for

EQA purposes.

The national EQA team returns all slides to the submitting laboratory at the end of the

EQA year (or sooner) by arrangement.

The national EQA team provides feedback to submitting laboratories on panel

consensus and laboratories are required to consider this information during their review

and EQA performance monitoring processes.

9.6.2. Copy slides

Only copy slides must be submitted to the scheme for inclusion in EQA rounds. Copy

slides (repeat preparations) produced from a single LBC sample are requested from

each laboratory. If the laboratory produces more than one copy from the same sample

24


then each is considered as an entirely separate case for EQA purposes. Identical

consensus for each of the copy slides must not be assumed.

9.6.3. Slides selected for EQA

Laboratories must provide good (not necessarily easy) examples for potential use in

EQA.

The slides selected for EQA range across the classifications recognised by the British

Association for Cytopathology (BAC). The following classifications are covered:

inadequate

negative

borderline change in squamous cells

borderline change in endocervical cells

low-grade dyskaryosis

high-grade dyskaryosis (moderate)

high-grade dyskaryosis (severe)

invasive squamous carcinoma

glandular neoplasia of endocervical type

glandular neoplasia (non-cervical)

For further explanation see ‘Achievable Standards, Benchmarks for Reporting and

Criteria for Evaluating Cervical Cytopathology’.3

Slides containing infections may be included, although there is no requirement to record

these on the response form. Interesting cases may be included to add educational

value to the scheme. Rarities are not included, nor are slides from unusual or problem

cases unless deemed ‘difficult and clear cut’.

Slides must have histological confirmation of diagnosis if they show:

high-grade dyskaryosis (moderate)

high-grade dyskaryosis (severe)


glandular neoplasia of endocervical type

glandular neoplasia (non-cervical)

Slides that are high-grade (moderate) or high-grade (severe) must be histologically

confirmed as either CIN2 or CIN3 (an exact match is unnecessary). Slides showing

borderline change in squamous cells, borderline change in endocervical cells, low-

grade dyskaryosis, or negative cytology should be submitted independently of any

associated HR-HPV test results.

25


9.6.4. Slides agreed for submission

Submitted slides must have achieved an in-house consensus opinion among staff,

including a cytology screener, a checker and a pathologist/consultant biomedical

scientist.

Staff who can select slides include cytology screeners, checkers, consultant biomedical

scientists and pathologists. These individuals are qualified and trained to report cervical

cytology, and are currently reporting cervical cytology on behalf of the NHSCSP. These

individuals are responsible for assessing the suitability and technical quality of slides for

potential use in EQA.

9.6.5. Supporting information for submitted slides

The laboratory provides the following information with each slide submitted:

the consensus opinion on the classification of each slide, based on agreement

between at least one cytology screener, one checker, and one

pathologist/consultant biomedical scientist

clinical information derived from the HMR101 form or its electronic equivalent

(age of individual and last menstrual period (LMP)/cycle stage)

histological confirmation of diagnosis for any cases with a consensus opinion of

high-grade dyskaryosis (moderate) or worse

Cytology samples showing low-grade dyskaryosis or borderline change in squamous or

endocervical cells may not have had further follow-up, since the woman may have been

returned to routine recall after HPV triage. In such cases, it may not be possible to

obtain histological confirmation, and slides falling into these reporting categories do not

need to be supported by further slides and/or histology.

Negative or inadequate slides do not require a second slide confirming the classification

in order to be included in the EQA scheme.

If copy reports are provided, only the corresponding slide number must be used for

identification. Any patient identifiable information must be removed.

26


9.6.6. Slide confidentiality

The laboratory is responsible for concealing any patient identifiable information on the

slides before submission. The laboratory must obscure the names of the patient and

the submitting laboratory for the purposes of the EQA round but not remove them from

the slide. The laboratory must make sure that the slides are sufficiently labelled to allow

the EQA facilitator to match slides to the relevant accompanying information.

9.6.7. Recording of dots or markers

Where a laboratory submits original material, in contravention with the scheme protocol

it is responsible for keeping a record of the coordinates of any dots or markers that

have been placed on the slides during screening or reporting. This is for medico-legal

reasons. The laboratory is also responsible for removing these dots or markers prior to

submission of the slide.

9.6.8. Dotted templates

EQA does not permit the preparation of dotted templates for staff on ‘catch-up’ prior to

their participation.

9.7. Review of submitted slides

Slides submitted by laboratories undergo a further review before they are included in

the scheme. The EQA facilitator is responsible for organising the slide review.

9.7.1. Reviewers

Laboratories are responsible for providing qualified and competent staff to undertake

the slide review. A cytology screener, a checker and a pathologist/consultant

biomedical scientist undertake the review.

Slide reviewers can report slides, which they may have examined as part of the review

process. To reduce the risk of recognition, slides are anonymised for the review

process, and anonymised again if used in the EQA round. Slide sets generated for

EQA, as far as possible, minimise the number of slides included from a single

laboratory.

9.7.2. Review method

The reviewers agree the classification of a slide that is consistent with the submitting

laboratory opinion, patient history, or histology where appropriate (for example a

cytological report of high-grade dyskaryosis (moderate) matching a histological

diagnosis of CIN II). The panel reaches its consensus without prior knowledge of the

submitting laboratory’s classification.

27


Slides accepted for EQA are assessed for their technical quality, including stain quality,

mounting and any cracks. Slides that are not technically adequate are rejected.

9.7.3. Discordant opinions

If the panel opinion differs from the submitted opinion and would result in the woman

being managed differently, the EQA facilitator notifies the originating laboratory and

returns the slide(s) immediately. For example, there may be disagreement on whether

to report the slide as negative, inadequate, or abnormal. Alternatively, the agreed panel

opinion may be that the slide should be reported as high-grade (moderate) or worse,

whereas the laboratory may have reported the slide as borderline change in squamous

or endocervical cells, or low-grade dyskaryosis.

It is the responsibility of the medical head of the submitting laboratory to take any

necessary follow up action in the event of such notification.

In the rare event that a participant discovers any unexpected clinical findings (for

example, psammoma bodies) during an EQA round, they must bring this to the

attention of the EQA facilitator in the first instance. The EQA facilitator will speak with

the scheme organiser to confirm the findings and ascertain the required action. The

scheme organiser will contact the submitting laboratory immediately to agree the

appropriate action. Follow up regarding any clinical implications for the woman is the

responsibility of the submitting trust.

9.8. Professional code of conduct

The scheme relies on each participant’s professional honesty to make sure that cases

are not discussed until all staff in the laboratory (excluding absentees) have completed

and submitted their EQA responses.

Participants must not pass on information during the EQA round to allow his or her

opinions to be submitted by another. The scheme defines such behaviour as collusion.

Collusion is a behaviour, which could compromise the validity of results and

consequently undermine the scheme’s integrity, as well as damage an individual’s

professional integrity.

At the beginning of every EQA round, laboratory managers must highlight the potential

consequences of collusion:

any evaluation of personal performance is meaningless

it allows an individual(s) to gain unfair advantage over his or her colleagues

the risk of submitting false responses is increased and this could adversely

affect individual performance

28


whether proven or unproven, collusion seriously diminishes an individual’s

professional integrity

the value and integrity of the EQA scheme is undermined

Laboratory managers must be sensitive when dealing with poor performance and

provide support for underperforming staff. Underperforming staff must avail themselves

of the support provided by management.

These measures are in the interests of all participants with the aim of avoiding adverse

effects on individual performance.

The EQA facilitator must be notified, in the first instance, if there is evidence of or a

strong suspicion of collusion. The scheme organiser will collaborate with the relevant

laboratory. The laboratory is responsible for investigating the allegation at local level

and reporting to the scheme organiser. If collusion is proven or admitted, then the

matter will be escalated immediately. Notification will be sent to NQAAP, the employer,

SQAS, and the relevant regulatory organisation.

The procedure for reporting and investigating alleged collusion is described in

Appendix 1.

9.9. Financial aspects

The funding of the scheme is included in the budget of the screening division of PHE.

Employing laboratories must meet costs incurred by individuals needing to travel to

another location to participate in EQA (such as absentees or locum staff).

The EQA scheme is free of charge for laboratories in England providing a cervical

screening service to the NHSCSP. A charge is made for providing the EQA scheme to

NHS laboratories outside England and to private laboratories that are not contracted to

the NHSCSP.

29


10. Preparation for the EQA round

Participants are provided with detailed information about the scheme, including their

rights and responsibilities, in advance of the EQA round. This information is provided

via the web-based EQA system or email if the system is not available to them.

10.1. Organising transfer of slides

The EQA facilitator is responsible for working with laboratories to agree dates for slide

receipt, onward transmission or return of slides, and a closing date and time for

submission of responses. See Appendix 2 for the timeline for the EQA schedule.

10.2. Tracking the location of slides

The EQA facilitator is responsible for knowing the location of each and every slide

during an EQA round, and ensuring that this information is documented.

10.3. Slide bank

The review panel(s) selects slides used in the EQA rounds. The EQA facilitator is

responsible for establishing a comprehensive slide bank record and its maintenance.

10.3.1. Obscuring laboratory details

The EQA facilitator must make sure that any laboratory details that would disclose the

origin of a slide are obscured for the duration of a round. This information is not

removed from the slide, but covered by a label that only shows the unique identifying

number of the slide and set.

10.3.2. Removal of dots/markers

The EQA facilitator must check that all dots/markers have been removed from

submitted slides.

10.3.3. Preservation of staining quality

Within each new round, the EQA facilitator must vary the order in which slide sets are

circulated between laboratories. This will make the distribution of pale slides more

equitable if staining deteriorates as a result of repeated microscopy during the EQA

round. Laboratories must protect the slides from sunlight in order to preserve the quality

of the slide for subsequent participants. Participants are provided with instructions on

handling slides, their preservation during EQA and storage prior to their return.

30


10.3.4. Slides used in more than one round

If EQA slides are to be used for a third round, then they must be reassessed

beforehand to make sure that their technical quality has not diminished. Slides retained

for future use, or as a contingency measure, must be stored in a suitable light-

impervious container within a locked drawer or cabinet.

10.4. Slide breakage, damage or loss

The laboratory lead must notify the EQA facilitator immediately via email if slides are

reported as broken, damaged or missing either in transit or during the EQA round. The

EQA facilitator will advise the laboratory lead regarding next steps.

The nominated QA biomedical scientist must make an assessment of the slide and

advise if the slide can be repaired, if the damage is not sufficient to disadvantage any

participants, or if the slide must be removed from the circulation. Repairs must not be

attempted without the written consent of the submitting laboratory.

The scheme considers lost or broken (irreparable) slides to be non-consensus and they

are not included in performance calculations. It is impossible to replace damaged or

broken slides with others from the same category, as no two slides are identical. In

such instances, the EQA facilitator notifies the submitting laboratory of the breakage.

At the end of the round, broken or damaged slides are returned to the submitting

laboratory with the rest of the batch. If any slides returned to the national EQA team are

found to be broken upon receipt then repairs are only attempted with the written

consent of the submitting laboratory.

10.5. Assembly of slide sets

The EQA facilitator is responsible for making sure the slide sets are assembled and

numbered accurately.

10.5.1. Number of sets required for cytology screeners/checkers

The EQA facilitator will determine how many slide sets are required to complete the

round. A minimum of 45 participants (excluding trainees and participants working in

laboratories that only screen private work) will see each slide within a set. If a slide

circulation has 200 cytology screeners/checkers, then the maximum number of slide

sets is 5 (4 plus one for the pathologists/consultant biomedical scientists).

The EQA facilitator makes sure that each peer group contains a minimum number of 45

people (preferably more) before the round begins.

31


10.5.2. Composition of slide sets

Each slide set contains 10 EQA slides plus any additional slides included for

educational interest. These additional slides are clearly labelled as such, and do not

form part of the assessment. The EQA facilitator determines the final composition of the

slide sets using the following criteria.

1. Slides are selected from all the categories recognised by the BAC. Slide sets

must comprise a greater proportion of abnormal cases than is seen in routine

practice.

2. Each slide set contains between 2 and 6 negative or inadequate slides. The

EQA facilitator must vary the ratio of negative or inadequate slides to slides of

other types for each round.

3. Where more than one slide set is used for a peer group, each slide set must

contain the same number of slides from each classification in each round.

4. All classifications must be covered in any one EQA year (within two consecutive

rounds). This includes negative, inadequate, borderline change (of either type),

low-grade dyskaryosis, high-grade dyskaryosis (moderate), high-grade

dyskaryosis (severe), and wherever possible, ?invasive squamous carcinoma

and ?glandular neoplasia.

10.5.3. Pathologists’ slide set

Slide set(s) allocated for the sole use of pathologists/consultant biomedical scientists

must be clearly labelled as such. They are selected at random from the available sets

and are, ideally, seen by at least 45 participants. In the event that the number of

pathologists/consultant biomedical scientists falls below this number, then a minimum

of 25 participants is acceptable for statistical validity.

10.6. Information and instructions for participants

Participants receive supporting information and instructions for use prior to the start of

an EQA round and the circulation of slide sets. The EQA facilitator instructs participants

on where and how to access the relevant information online.

Supporting information includes:

a response form

a referral sheet

slide set details

participant instructions for recording and submitting responses

relevant information for special interest slides

a complaints procedure

32


an appeals procedure

Participants using manual systems will receive paper documentation before the EQA

round starts.

10.6.1. Response form

The paper version of the response form replicates the content of the online version. For

manual systems, each participant receives their response form in a sealed envelope or

by secure email.

10.6.2. Referral sheets

In each round, one or more cytology screeners from each laboratory must mark the

pathologists’ slide set in addition to their EQA. These individuals are required to

complete the referral sheet (Appendix 3). In addition to completing their own EQA, one

or more checkers must assess each slide that the cytology screener considers

potentially abnormal and to complete the standard referral sheet.

The completed referral sheet is circulated to all participants viewing the pathologists’

slide set by the laboratory manager or nominated deputy. The original referral sheet is

retained by the laboratory. There is no requirement to submit this form as part of the

assessment.

10.6.3. Slide set details

Slide information accompanies each slide set. For each slide within the set, the

following information is given:

the slide number

the age of the woman

LMP

HR-HPV test results do not accompany the EQA slides, as this would give an indication

of the cytological classification of the slide. The classification of slides

included for educational interest, with any further comments, accompany the slide set.

10.7. Other documentation

The following areas of guidance are made available to participants in either written or

electronic format, or reference is provided to the location of controlled documents. This

includes instructions and/or action to take regarding:

receipt of broken or damaged slides

slides damaged or broken during the EQA round

33


handling, cleaning and storing slides during the EQA round

unexpected findings during a live round

safe packaging and dispatch of slides

34


11. Conditions for examining EQA slides

Participants examine EQA slides under conditions, which resemble those used in their

routine practice as closely as possible. They will not undertake the assessment in an

examination setting. Participants are required to evaluate their own diagnostic opinion

against those of their peers. Participants must not discuss the slides in any way before

all participants (excluding absentees) have viewed them and recorded and submitted

their results.

In each round, there are 2 peer groups: cytology screeners and checkers, and

pathologists and consultant biomedical scientists. Where a peer group uses more than

one slide set, participants are each allocated a set. The EQA facilitator makes sure that

the number of participants viewing each set is sufficient for a consensus calculation to

be valid.

The slide categories are representative of the categories of slide result found in routine

workload. The nature and numbers of slides circulated and the results analysis are key

features of the scheme.

11.1. Non-medical staff

In NHSCSP, laboratories there are 2 clearly identifiable tiers of activity undertaken by

non-medical staff: primary screening and checking. At present, staff (other than

consultant biomedical scientists) who are not medically qualified do not report abnormal

slides. They sort slides into those they report as negative or inadequate, and those that

they pass to more senior colleagues for further review. Non-medical staff are assessed

on the basis of their ability to distinguish between negative, inadequate, and abnormal

slides. Many laboratories routinely encourage their checkers to suggest a classification.

The scoring scheme covers all staff in order to enhance the educational benefit of the

scheme.

Individuals (usually cytology screeners and biomedical scientists) who undertake

primary screening decide whether a slide is negative, inadequate, or potentially

abnormal. They refer abnormal slides on to senior staff for more detailed reporting.

Individuals who routinely carry out primary screening undertake EQA with unscreened

and unmarked slides.

There is no limit set on the maximum time that cytology screeners can look at a

particular EQA slide. As assessment conditions resemble those of normal working

practice, the expectation is that cytology screeners are able to screen the 10 EQA

slides in approximately 2 hours.

35


A cytology screener is assessed on only one set of slides. Performance is measured on

his or her ability to differentiate between negative, inadequate, and potentially abnormal

slides, and on any instances of missed dyskaryosis. Cytology screeners are required to

suggest a grading classification for abnormal slides, and quantitative and qualitative

feedback based on these opinions is provided for personal educational purposes. This

feedback is not included in the performance analysis.

In addition, cytology screeners can view and report slides used in the EQA scheme for

pathologists/consultant biomedical scientists. Their results are treated as purely

educational and are not used in performance analysis.

Checking

Experienced cytology screeners and biomedical scientists have varied duties. Checkers

usually undertake some primary screening, rapid review, and checking of cervical

slides that appear potentially abnormal. Checkers, like cytology screeners, report one

set of 10 unmarked slides.

There may be some checkers who do not perform any primary screening and who look

only at marked slides. These individuals are allowed to perform their EQA on marked

slides, however they report all 10 slides within the slide set, not only those that have

been marked. Cytology screeners within the laboratory must mark the slides as they

would in routine practice before the checkers perform their EQA.

Checkers who receive marked slides do not receive a copy of cytology screener

opinions. The performance of those checkers who do not engage in primary screening

is calculated in the same way as that of cytology screeners and checkers who do.

Checkers receive quantitative and qualitative feedback for personal educational

purposes on their grading of abnormalities. Assessment of their performance is based

on their ability to distinguish between negative, inadequate, and potentially abnormal

cervical samples and on any instances of missed dyskaryosis.

Checkers can view and report slides used in the EQA scheme for

pathologists/consultant biomedical scientists. Their results are treated as purely

educational and not used in performance analysis.

11.2. Consultant biomedical scientists in cervical cytology

Consultant biomedical scientists sign out abnormal slides and provide management

recommendations. They may also undertake checking duties and report unchecked

slides, effectively acting as their own checker. EQA for these staff uses slides that have

been screened by cytology screeners and other checkers, in line with routine practice in

the laboratory.

36


While consultant biomedical scientists act under the direction of the consultant

pathologist, they are not under his or her direct supervision. From an EQA point of

view, it is therefore appropriate that they are included alongside medical practitioners.

There may be some consultant biomedical scientists who perform primary screening.

These individuals are not expected to participate in EQA as a cytology screener.

11.3. Medical staff

The 2 major activities undertaken by most pathologists who routinely report for the

NHSCSP are as follows.

1. Reporting of slides referred from cytology screeners and checkers as potentially

abnormal.

2. Review of slides that have previously been reported as negative or inadequate

by cytology screeners or checkers, and have later been identified as requiring

medical review.

The EQA scheme for pathologists assesses an individual’s performance in providing an

opinion on slides identified as potentially abnormal, and in reviewing negative and

inadequate slides that have been through primary screening. Cytology screeners and

checkers may wish to participate in this type of EQA in addition to their regular EQA

Their responses are not used for performance assessment, and are not included in the

analysis of the slide set.

EQA for medical staff is based on slides that have been screened by cytology

screeners and checkers in line with routine practice in the laboratory. Practice varies

between laboratories, but cytology screeners and checkers often mark slides. Where

this is normal practice, marked slides are retained for EQA purposes. Pathologists and

consultant biomedical scientists know which slides have been referred by cytology

screeners and which by checkers, but not the suggested classifications of the slides

offered by either group.

The response form used by medical staff when reporting slides is in line with

recommendations from the British Association for Cytopathology, in particular their

Code of Practice for Cytopathology Laboratories.4 This states that the pathologist

should see all abnormal material and a proportion of negative material to make sure

that laboratory is maintaining accuracy and quality.

A pathologist has experience of screening unmarked slides (and, in particular, of

rescreening negative slides when abnormalities have subsequently been found), and of

rescreening an entire slide when equivocal cell groups have been identified as requiring

an opinion. Pathologists who undertake EQA assessment examine the whole of every

37


slide, including slides that have already been marked by cytology screeners and

checkers.

The cytology screeners selected to screen slides in the pathologists’ set mark these as

they would in routine practice. They indicate (with an ‘R’ on the referral sheet), any

slides that they consider potentially abnormal. The checkers then look at the slides

marked with an ‘R’ and indicate (with an ‘A’ on the referral sheet) any slides that they

would refer to the pathologist/consultant biomedical scientist as potentially abnormal.

The checkers also mark the slides as they would in routine practice.

The pathologists and consultant biomedical scientists then perform their EQA on the

marked slide set using only the referral sheet for reference (they remain unaware of

cytology screener/checker opinions concerning the classification of the slides). All EQA

slides within the set are examined, and pathologists/consultant biomedical scientists

check the entire slide and not just areas marked as potentially abnormal.

All 10 slides count towards the assessment. Assessment is based on the ability to

distinguish between negative, inadequate, and abnormal slides, the grading of each

abnormal slide, and any instances of missed dyskaryosis.

Pathologists and consultant biomedical scientists may also choose to examine the non-

medical slides for educational purposes, and the EQA facilitator makes sure that these

scores are not included in the performance analysis.

11.4. Trainee staff

Trainee cytology screeners, trainee biomedical scientists and trainee medical staff who

intend to work in the field of cervical cytology are encouraged to participate in EQA.

The scheme is considered to be of educational value for these staff and, although they

are allocated a mark and given the same level of feedback as qualified staff within their

peer group, their results are excluded from any consensus calculation or performance

analysis of the slide set.

11.5. Additional slide sets

If participants choose to view extra slide sets in addition to their allocated EQA slide set,

results from these additional sets are not included in the performance analysis and not

recorded on EQA online.

38


12. The assessment

12.1. Recording and submitting responses

All participants in the EQA scheme register their opinion concerning the cytological

pattern on an EQA slide in a format consistent with both standard British Association for

Cytopathology reporting and the standard request form (HMR 101).

Figure 3 equates to the screen viewed by participants in EQA online.

Columns added to the table for special interest slides are not part of the formal

assessment.

The response form is available in paper or electronic format for those participants

unable to access EQA online.

The response form includes mandatory fields for:

personal identity code (only on paper or electronic version, not online)

laboratory code

date of EQA

test set identity

Participants receive instruction on completing whichever format form they use.

Participants must note that:

only one box must be marked in the result column of the cytological pattern

section for each slide

responses must be in the form of a tick, cross or strike (paper/electronic version),

and any other marks are not acceptable and will be recorded as a ‘no response’

a mark (tick, cross or strike) which spans more than one box is recorded as a

‘transcription error’

if a box is marked in error, it must be blocked out completely using ink or

correction fluid (paper version)

failure to mark one box or marking more than one box in any column is recorded

as a ‘transcription error’

Participants are responsible for making sure their responses are entered as intended,

as these cannot be changed after submission. Participants entering their responses

online are prompted to check and confirm their answers before submitting them.

39


Participants who submit their responses via paper form are made aware of the

penalties for transcription errors. Forms submitted to the EQA facilitator are scored only

as received. There is no opportunity for ‘modifying’ forms prior to marking. It remains

the responsibility of the individual participant to make sure that responses are recorded

correctly.

Participants using EQA online must upload their results immediately after screening

their slides, and while the slides are still in the laboratory.

All participants must keep a paper record of their responses once these have been

submitted until the final results are issued by the scheme. This includes participants

recording their results using EQA online. This is to mitigate against potential loss of

EQA data in the event of IT system failure.

40


Cytological pattern Result

Result code

HMR101 category Slide number 1 2 3 4 5 6 7 8 9 10

Inadequate (1)

Negative (2)

Borderline change in squamous cells

(8)

Borderline change in endocervical cells

(9)

Low-grade dyskaryosis (3)

High-grade dyskaryosis (moderate) (7)

High-grade dyskaryosis (severe) (4)

invasive squamous carcinoma (5)

Glandular neoplasia of endocervical type

(6)

Glandular neoplasia

(non-cervical)

(0)

Figure 3. Response form

12.2. Following assessment

12.2.1. Release of submitted opinions

Once all participants (or one of the peer groups) within the laboratory have viewed the

EQA slides and recorded and submitted their results, the opinions that have been

provided by the submitting laboratory are released to those participants while the EQA

slides are still in the laboratory.

12.2.2. Interim feedback

EQA online makes interim feedback available to view in the form of submitted opinions.

The scheme makes paper or electronic format interim feedback available for

participants unable to access EQA online. This allows individuals to compare their own

responses with the submitted opinions for the EQA slides at an early point.

Interim feedback is provisional and participants must not assume this will necessarily

match their final result. The interim report form clearly states that the results are

confidential to the individual participant.

Final EQA results are based on consensus opinions that are only available at the end

of the round.

41


12.2.3. Final feedback

The scheme provides confidential final feedback to each participant at the end of the

EQA round. This is in a report (or series of reports) and includes an overall score of

performance in the scheme. Verbal feedback on individual performance is not

available.

12.3. Reports

12.3.1. Participant reports

The head of screening quality assurance and the EQA facilitator have joint

authorisation to issue reports to individuals at the end of each completed round.

Participant reports are issued within the 6-month period allocated for each round (no

later than 30 September for the first round, and no later than 31 March for the second

round). Participants must keep their own records of performance in proficiency testing,

including the outcomes of investigations of any unsatisfactory results.

Performance results are a useful record for participants. Limitations are placed on using

these to determine competence. Successful performance may represent evidence of

competency for a particular round but may not reflect ongoing competence. Similarly,

unsatisfactory performance in a particular round may reflect a random departure from a

participant’s normal competence. [1]

12.3.2. Annual report

The annual report includes a summary of participant results and is widely distributed

within the programme and to key stakeholders

12.4. Certificates of participation

All eligible individuals will receive a certificate of participation once the final results

reports have been validated and issued. Eligible laboratories will also receive a

certificate of participation.

12.5. CPD and CME credits

For the purposes of continuing professional development, participants may claim 2

hours CPD/CME credit per completed EQA round.

42


13. Slide consensus

13.1. Consensus opinion

Final analysis of participants’ performance is based on consensus agreement. The

correct classification of a slide is based on consensus opinion determined at the end of

the round and is based on an 80% consensus of participant results for each peer

group.

Within the appropriate peer group, which has submitted a valid response for an EQA

slide, at least 80% of the participants will have agreed in their classification of that slide

in order for it to be used to analyse performance. Such calculations only take into

account EQA responses from participants who participate in primary screening and/or

the issuing of reports for the NHSCSP. Responses from participants working in

laboratories that perform only private screening work are excluded from the consensus

calculations.

If a participant fails to provide a response for a slide, or provides multiple responses for

a slide, they are excluded from the consensus calculations for that slide and scored

according to the guidance.

13.2. Data handling and calculations

To simplify data handling and calculations, the EQA facilitator keeps a record of

responses by participant number (not name) for each EQA slide.

Automatic electronic calculation of consensus scoring is acceptable, provided that

these calculations have been previously audited and validated.

Although the calculation of consensus results in a delay in the provision of final

feedback to participants, the difficulties inherent in reaching an agreement on

cytological classification mean that this approach is necessary.

13.3. Slides with consensus different to the submitted classification

If the consensus opinion differs significantly from that of the submitting laboratory and

review panel then the EQA facilitator informs the laboratory and returns the slide

immediately.

This includes any slides where there is a mismatch between the submitted opinion and

the consensus opinion as to whether the slide is negative, inadequate, or abnormal. It

43


also applies to slides in the pathologists’ set that are reported with 80% consensus as

high-grade dyskaryosis (moderate) or worse, but have been submitted as showing:

borderline change in squamous cells

borderline change in endocervical cells

low-grade dyskaryosis

The scoring of participants on such slides is based on the consensus opinion.

It is the responsibility of the medical head of the submitting laboratory to take any

necessary action following mismatch between submitted and consensus opinion.

13.4. Consensus on negative, inadequate and normal slides

Consensus agreement on negative, inadequate and abnormal slides is based on all

valid participant responses in the peer group for the particular slide.

13.4.1. Consensus calculations for the distinction between negative, inadequate and

abnormal

To calculate the consensus agreement for the distinction between negative,

inadequate, and abnormal slides, the following codes on the response form are all

deemed abnormal:

3 (low-grade dyskaryosis)

4 (high-grade dyskaryosis (severe))

5 (invasive squamous carcinoma)

6 (glandular neoplasia of endocervical type);

7 (high-grade dyskaryosis (moderate))

8 (borderline change in squamous cells)

9 (borderline change in endocervical cells)

0 (glandular neoplasia (non-cervical)

For each slide, the number of participants who have classified the slide as negative,

inadequate, or abnormal is totalled. If all participants have submitted the same

response then the consensus agreement for the slide is 100%. For slides where there

is more than one classification suggested, the consensus agreement is calculated

using the classification that has the highest number of responses, as follows:

highest number of participants giving the same classification % consensus = x 100

total number of participants giving a valid response for the slide

44


13.5. Identifying missed dyskaryosis

Missed dyskaryosis is defined as a participant reporting a slide as negative or

inadequate which is classified by consensus as dyskaryotic.

13.5.1. Consensus for identifying missed dyskaryosis

This is calculated by assessing whether all responses of low-grade dyskaryosis or

worse within the group total 80% or greater.

13.6. Grading consensus for abnormal slides

Consensus agreement on the grading of abnormal slides is based on all valid opinions

reported on the slide by each peer group. The grading consensus for cytology

screeners and checkers is provided for educational feedback only.

The consensus opinion and scoring of pathologists and consultant biomedical scientists

for the grading of abnormal slides is based on EQA responses from pathologists and

consultant biomedical scientists only.

If the consensus agreement (calculated in section 13.4.1) was 80% or greater for an

abnormal classification of a particular slide, then the grading consensus is calculated

for that slide as follows:

highest number of participants giving the same grade % consensus =

x 100 total number of participants giving a valid response for the slide

If 80% consensus is not reached, but the amalgamation of adjacent grades would

enable it to be reached, then this is permitted. The grade with the highest level of

consensus is combined with the grade with the next highest consensus, provided that

the 2 grades in question conform to the following groupings:

borderline change in squamous cells and borderline change in endocervical

cells.

borderline change (squamous and endocervical cells) and low-grade dyskaryosis

low-grade dyskaryosis and high-grade dyskaryosis (moderate)

high-grade dyskaryosis (moderate) and high-grade dyskaryosis (severe)

high-grade dyskaryosis (severe) and invasive squamous carcinoma

high-grade dyskaryosis (severe) and glandular neoplasia of endocervical type,

glandular neoplasia (non-cervical), or both glandular neoplasia categories

invasive squamous carcinoma and either glandular neoplasia of endocervical

type, or

glandular neoplasia (non-cervical), or both glandular neoplasia categories

45


glandular neoplasia of endocervical type and glandular neoplasia (non-cervical)

If combining the different categories of borderline change still does not achieve 80%

consensus, amalgamation with a third grade (low-grade dyskaryosis) may be

performed. Similarly, the glandular neoplasia categories may be combined with each

other or another adjacent grade to reach consensus. The amalgamation of both

glandular neoplasia categories plus one other adjacent grade (which has the third

highest level of consensus) is also permitted.

In the unlikely event that the categories on either side of the grade with the highest

level of consensus have the same percentage consensus, follow the guidance given in

Table 1.

These amalgamations exist for scoring purposes only and may not be compatible with

any specific management recommendation.

13.7. Slides not reaching consensus

Slides that have not been reported with 80% consensus as negative, inadequate, or

abnormal are removed from any performance analysis. This means that participants

are scored only against the remaining slides in the set that reach consensus. To

achieve statistical validity, a minimum of 6 slides within any set has to reach

consensus.

13.8. Slides with abnormal consensus but no grade consensus

If a slide has at least 80% consensus for an abnormal classification, but 80%

consensus is not reached for the grading of that slide (even when adjacent grades are

amalgamated), no score is allocated for grading.

Table 1. Amalgamation of grades when both adjacent grades have equal consensus

Grade with highest consensus

Grades with next highest (equal) consensus

Consensus

Low-grade dyskaryosis

Borderline change (squamous and endocervical)

Low-grade dyskaryosis/Borderline change (squamous and endocervical)

High-grade dyskaryosis (moderate)

High-grade dyskaryosis (moderate)

Low-grade dyskaryosis High-grade dyskaryosis (moderate)/High-grade dyskaryosis (severe)

High-grade dyskaryosis (severe)


High-grade dyskaryosis (moderate) High-grade dyskaryosis (severe)/High-grade dyskaryosis (moderate)


46



invasive squamous carcinoma High-grade dyskaryosis (severe)/ invasive squamous carcinoma Glandular neoplasia of endocervical type

or

?Glandular neoplasia (non-cervical)


Glandular neoplasia of endocervical type High-grade dyskaryosis (severe)/Glandular neoplasia of endocervical type

Glandular neoplasia (non-cervical)



invasive squamous carcinoma/High-grade dyskaryosis (severe)

Glandular neoplasia of endocervical type or Glandular neoplasia (non-cervical)


Glandular neoplasia of endocervical type invasive squamous carcinoma/Glandular neoplasia (non-cervical)



High-grade dyskaryosis (severe) Glandular neoplasia of endocervical type/ High-grade dyskaryosis (severe)


Glandular neoplasia (non- cervical)

High-grade dyskaryosis (severe) Glandular neoplasia (non-cervical)/?invasive squamous carcinoma



High-grade dyskaryosis (severe) or invasive squamous carcinoma Glandular neoplasia of

endocervical type/?Glandular neoplasia (non-cervical)


Glandular neoplasia (non- cervical)

High-grade dyskaryosis (severe) or invasive squamous carcinoma Glandular neoplasia of

endocervical type/Glandular neoplasia (non-cervical)


Glandular neoplasia combined (endocervical and non-cervical)

High-grade dyskaryosis (severe) Glandular neoplasia (endocervical and non- cervical)/?invasive squamous carcinoma


47


14. Scoring

14.1. Consensus slides

Participants are only scored on slides in their EQA slide set that reach 80% consensus.

Participants from laboratories where only private screening work is undertaken are

scored against the consensus for each slide calculated using responses from

participants who work in the NHSCSP.

EQA schemes are generally based on 80% consensus, and that consensus may or

may not be correct.

14.2. Cytology screeners and checkers

14.2.1. NIA score

Cytology screeners and checkers are marked on their ability to distinguish between

negative, inadequate, and abnormal slides (an NIA score) as follows:

slide correctly interpreted as negative/inadequate/abnormal = 2 marks

slide incorrectly interpreted = 0 marks

Incorrect interpretations can fall into the following categories:

the slide has a negative consensus and the participant has reported it as

inadequate or abnormal

the slide has an inadequate consensus and the participant has reported it as

negative or abnormal

the slide has an abnormal consensus and the participant has reported it as

negative or inadequate

A scoring matrix for cytology screeners and checkers is given in Appendix 4.

14.2.2. Missed dyskaryosis

Where cytology screeners and checkers have reached consensus that a slide shows

dyskaryosis (80% or more of participants have classified it as low-grade dyskaryosis or

worse), a participant who reports the slide as negative or inadequate is recorded as

having missed dyskaryosis. This counts as substandard performance.

48


14.2.3. Educational value

To enhance the educational value of the scheme, qualitative and quantitative feedback

is given to cytology screeners and checkers on their grading of abnormal slides.

14.3. Pathologists and consultant biomedical scientists

14.3.1. NIA score

Pathologists and consultant biomedical scientists are scored on their ability to

distinguish between negative, inadequate, and abnormal slides (an NIA score).

In addition to the NIA score, extra marks are allocated for grading of consensus

abnormal slides:

correct grading of slide = 2 marks

grading of slide within one category either side = 1 mark

grading of slide wider than one category either side = 0 marks

Participants do not lose a mark for reporting a slide that has a consensus report of

borderline change in squamous cells as borderline change in endocervical cells, and

vice versa.

A scoring matrix for pathologists and consultant biomedical scientists is in Appendix 5.


If a slide has reached 80% consensus among pathologists and consultant biomedical

scientists as showing dyskaryosis (80% or more of participants have classified it as low-

grade dyskaryosis or worse), a participant who grades the slide as negative or

inadequate is recorded as having missed dyskaryosis. This counts as substandard

performance.

49


14.4. Transcription errors

If a participant fails to tick a box on the response form for a slide, or ticks more than one

box for a single slide, this is penalised as an overcall or undercall as follows.

If no box is ticked

No mark is given. If the consensus is that the slide should be reported as dyskaryosis,

this is counted as missed dyskaryosis, and thus as substandard performance.

If 2 boxes are ticked (both abnormal)

If the consensus is abnormal, 2 marks are given for correctly classifying slide as

abnormal (NIA score), then marked for grading using the ticked box that is furthest

away in grade from the consensus grade. For example, if low-grade dyskaryosis and

high-grade dyskaryosis (severe) are both ticked, and the consensus is invasive

squamous carcinoma, the mark is as if the participant has only ticked low-grade

dyskaryosis. If the consensus is negative or inadequate, the participant receives zero

marks.

If negative or abnormal are ticked, and abnormal is ticked


this is counted as missed dyskaryosis, and thus as substandard performance.

If one box is marked inadequate and one is marked negative


this is counted as missed dyskaryosis and thus as substandard performance.

Where a participant has given 2 or more responses for a single slide these responses

are excluded from the consensus calculation for that slide.

14.5. Final scores

In order to make sure that scores are comparable within peer groups (cytology

screeners/checkers or pathologists/consultant biomedical scientists), the total score for

each participant is converted into a percentage of the possible score for the particular

slide set that was used (after any non-consensus slides have been removed).

participant’s score % score =

maximum possible score for the slide set

x 100

50


The scheme produces separate lists of scores for cytology screeners and checkers,

and pathologists and consultant biomedical scientists. Scores are listed by participant

code and ranked in percentage score order. Reported scores do not include trainee

staff or staff from laboratories that do not participate in the NHSCSP.

Percentage scores are expressed to 2 decimal places. Values below 4 in the third

decimal place are rounded down. Values of 5 or above are rounded up.

15. Assessment of performance

The formal assessment of personal performance differs for the two peer groups of

participants.

15.1. Cytology screeners and checkers

For cytology screeners and checkers, personal performance is assessed on the

individual’s ability to distinguish between negative, inadequate, and abnormal slides.

Any instance of missed dyskaryosis is also counted.

15.2. Pathologists and consultant biomedical scientists

For pathologists and consultant biomedical scientists, personal performance is

assessed on an individual’s ability to distinguish between negative, inadequate, and

abnormal slides, the grading of cytological patterns for abnormal slides, and any

instance of missed dyskaryosis.

15.3. Identification of substandard performance

Substandard performance is specified as:

a score that falls below the bottom 2.5 percentile point when percentage scores

are placed in rank order for the slide circulation

an instance of missed dyskaryosis

15.3.1. Calculation of the bottom 2.5 percentile point

It is possible to calculate the bottom 2.5 percentile point in Excel by using the inbuilt

function key (fx) on the tool bar. The percentile function can be found in the statistical

function category. The ‘array’ of data is all percentage scores in the rank and the ‘K’

value is the percentile value sought, expressed as a decimal (such as 0.025). The

value obtained is the bottom 2.5 percentile point of all the results. Results falling

51


beneath this value are identified as substandard for the purposes of EQA performance

assessment.

Automatic electronic calculation of the bottom 2.5 percentile point is acceptable,

provided that these calculations have been previously audited and validated.

It is also possible to calculate the score at the 2.5 percentile manually, as shown in the

following example:

The calculation is based on an EQA round consisting of 68 percentage scores that range from 100% at

the upper level to 60% at the lower level. The score of the second lowest participant (R2) was 66%; that of the third lowest participant (R3) was 72%.

The formula used is

1 + P(N – 1) = a + b

where a is the integer of the result indicating the ranked position of the score, b is the fraction indicating the proportion of the difference between scores at (a) and (a + 1), P is the percentile (0.025), and N is the

number of participants (68).

1 + 0.025(68 – 1) = a + b

1 + (0.025 x 67) = a + b

1 + 1.675 = a +

b a + b = 2.675

a = 2

b = 0.675

Since a = 2, the score of the second lowest participant is taken as the baseline; if the calculation resulted in a = 1 then the score of the lowest participant would be used as the baseline, or the third lowest participant if the result was a = 3 and so on. With a = 2, the value of b, 0.675, relates to the proportion of the difference between the scores of the second and third lowest participants (if the calculation resulted in a = 1 then the value of b would relate to the difference between the scores of the lowest and second lowest participant and so on).

The proportional difference between the two figures is calculated as follows:

(R3 – R2) x 0.675

(72 – 66) x 0.675

6 x 0.675 = 4.05

The desired percentile is then calculated by adding this difference to the score of the second lowest participant (as a = 2 in the first part of the calculation) as follows:

66 + 4.05 = 70.05%

Any scores below 70.05 are below the 2.5 percentile point and considered to be substandard performance.

Owing to the nature of the scoring system, on some occasions there may be individuals

whose scores equal the 2.5 percentile point with no scores falling below the value. In

52


such cases, the scheme does not consider these individuals to have shown

substandard performance.

Participants from private (non-NHS) laboratories or overseas laboratories have their

percentage scores compared to the 2.5 percentile point after calculations are

completed for the other participants. Their individual percentage scores are therefore

not included in the calculations to determine the 2.5 percentile.

Substandard performance is identified by placing the scores for each peer group in rank

order for each circulation. Individuals with scores below the 2.5 percentile point are

classed as having substandard performance for that EQA round.


Missed dyskaryosis is identified as a participant report of negative or inadequate for a

slide that has a consensus classification from the relevant peer group of dyskaryosis.

This is reached by combining all valid responses of low-grade dyskaryosis, high-grade

dyskaryosis (moderate), high-grade dyskaryosis (severe), invasive squamous

carcinoma, glandular neoplasia of endocervical type and glandular neoplasia (non-

cervical).

15.3.3. Notification of substandard performance

If a participant scores below the 2.5 percentile and misses a dyskaryosis, this is

deemed to be a single instance of substandard performance for that round.

Any participant with substandard performance receives a confidential letter notifying

them of this, and offering support from the scheme organiser. Although this does not

constitute persistent substandard performance, staff are encouraged to discuss a result

of this nature with their training officer or line manager.

15.4. Persistent substandard performance

Persistent substandard performance is defined as the occurrence of either of the

following in 2 out of 3 consecutive EQA rounds:

scoring below the 2.5 percentile point

missed dyskaryosis

After each round, results are analysed to identify persistent substandard performance.

This is based on the participant’s EQA results for the three most recent rounds. The

EQA facilitator keeps a record of participants’ previous scores complete with comments

so that persistent substandard performance can be easily identified.

53


A participant is not classified as showing persistent substandard performance on the

basis of the results of a single round. For example, if a participant misses dyskaryosis

on 2 or more occasions in one round, or misses dyskaryosis on one or more occasions

and also falls below the 2.5 percentile point in that round, his or her performance is

graded as substandard for that round, but not as persistent substandard performance.

15.5. Action points for persistent substandard performance

Action points, and the remedial measures taken when they are reached, are defined by

the NQAAP (histopathology and cytopathology).

15.5.1. First action point

The first action point is reached when persistent substandard performance is first

identified.

15.5.2. Second action point

The second action point is reached when a participant’s performance continues to be

substandard in 2 out of 3 consecutive rounds after the first action point has been

reached. Persistent substandard performance is determined on a ‘rolling’ basis (using

the results of the last 3 consecutive rounds). Substandard performance in either of the

next 2 rounds after triggering the first action point means that the participant has

reached the second action point.

A table, which demonstrates the various combinations of results that can lead to the

first and second action points, depending on an individual’s pattern of persistent

substandard performance, is in Appendix 6.

The specific action to be taken when action points are reached is different for each peer

group. The pathways for each action point are in Appendices 7.1 and 7.2.

15.5.3. Developing a remedial action plan

Participants identified as persistent substandard performers are required to produce

and implement a remedial action plan. The participant (jointly with the laboratory

manager when necessary) will submit a declaration to the scheme organiser to confirm

and evidence that the plan has been implemented and appropriate action taken. The

pathway is outlined in Appendix 7.1.

54


The minimum requirements for a remedial action plan are:

explanation of the reason for substandard performance

review of workload figures for cervical cytology

review of screener and checker performance data for the period of EQA

substandard performance consisting of abnormal sensitivities, high grade

sensitivities and reporting profiles

review of pathologist and consultant biomedical scientist performance data for

the period of EQA substandard performance consisting of positive predictive

value (PPV), abnormal predictive value (APV) and Referral Value (RV)

proposals for a review of in-house teaching sets or the scheduled attendance of

suitable formal training

reflective statement taking into account the wider perspective of the working

environment and current performance

15.5.4. Participants from outside the UK

Non-UK laboratories declare their professional lines of accountability within their

employing organisation and regulatory body under the terms and conditions of the EQA

agreement. The EQA scheme reports persistent poor performance to the relevant

regulatory body.

55


16. Communication to and from the EQA

scheme

16.1. Constructive criticism, comments and compliments

The EQA scheme for gynaecological cytopathology considers any concern or negative

comment relating to any aspect of its operation or management to be a complaint.

Scheme participants are welcome to express concerns or suggest improvements via

email, telephone or letter. The EQA service actively seeks feedback (both positive and

negative) through a national annual survey and a participants’ forum.

Verbal and written communications are managed locally. The scheme defines the term

‘complaint’ as dissatisfaction about or criticism of the EQA operation or management. A

complaint does not have to be labelled as such by the sender for it to be treated as a

complaint by the scheme if it displays the criteria indicated. All communications are

logged and acted on as needed, and used to help identify system improvements.

16.2. Formal complaints

Formal complaints go directly to the EQA facilitator in the first instance. In the event of

failure to resolve a complaint, the individual in question can complain directly to the

scheme organiser. If the issue is still unresolved, the participant can then refer his or

her complaint to the chair of NQAAP. The NQAAP decision is final.

The complaints procedure (Appendix 8) describes how participants can lodge a formal

complaint regarding any aspect of scheme operation or management.

16.3. Formal appeals

If EQA participants wish to formally appeal regarding their performance assessment or

results, they must contact the EQA facilitator in the first instance. In the event of

escalation, the individual in question can complain directly to the scheme organiser.

Ultimately, the participant can refer their appeal to the chair of NQAAP. The NQAAP

decision is final.

The appeals procedure (Appendix 9) describes how participants can lodge a formal

appeal against an evaluation of their EQA performance.

56


16.4. Protocol changes

The protocol may undergo changes when improvement opportunities are identified, or

when non-conformity within a process has been identified and requires corrective

and/or preventive action.

The changes are listed in the revision history at the beginning of the protocol. The

revised protocol will be uploaded to EQA online, and a PDF version provided to

laboratories that submit their responses manually. All participants are required to

confirm, via EQA online or through the submission of a written declaration, they have

read and accept the terms specified in the protocol.

Changes may also arise as a consequence of participant feedback or direct request

from the operational group, steering group or management board.

The scheme uses a change control procedure to prioritise and manage all changes.

The EQA facilitator will notify participants of agreed changes to the scheme and the

date when they are to come into effect.

16.5. Participants’ meetings

Participants’ meetings/workshops provide a forum for staff to view scheme results

comparisons, discuss operational aspects of the scheme and review any challenging

slides and/or educational cases. The feedback from the participants’ annual meeting is

managed through the EQA operational group. A summary is provided in the annual

report.

The meetings are held annually and open to all staff. Laboratories are encouraged to

send at least one member of staff who can then cascade information to colleagues.

16.6. Audit

The EQA operational group is responsible for setting the scope of the audit strategy

and preparing the internal audit programme. The EQA management board is

responsible for authorising the audit strategy and programme and may advise on

adjustments if considered necessary.

The scheme organiser is responsible for notifying participants if audit findings indicate

that non-conformities may have affected participant EQA results.

The head of screening quality assurance is responsible for all UKAS related

communications, and for making sure the overall requirements of external assessments

are met.

57


16.7. Surveys

Short surveys are used periodically to obtain feedback or views from participants on

actual or proposed scheme activity.

There is an annual survey, which all participants are encouraged to complete, either to

register their satisfaction with the EQA service or use as an opportunity to provide

constructive criticism, comments and/or suggestions for improvement.

Findings from the participant surveys are managed through the EQA operational group.

A summary is included in the annual report.

58


16.8. Annual report

The scheme produces an annual report for:

participants

members of the EQA management board, steering group and operational group

PHE director of screening

PHE head of YPA screening programmes

PHE national screening QA service lead

PHE regional cervical screening QA portfolio leads (for cascade to regional

cervical screening QA staff)

National Quality Assurance Advisory Panel (NQAAP) for Cellular Pathology

59


References

1. ISO 17043:2010 Conformity assessment – General requirements for proficiency

testing.

2. ISO 15189:2012 Medical Laboratories – Requirements for quality and

competence.

3. Achievable Standards, Benchmarks for Reporting and Criteria for Evaluating

Cervical Cytopathology, 3rd edition. NHS Cancer Screening Programmes, 2013

(NHSCSP Publication No 1).

4. Recommended Code of Practice for Laboratories Participating in the UK Cervical

Screening Programmes. London, British Association for Cytopathology, 2015.

5. ‘The retention and storage of pathological records and specimens’, fifth edition

April 2015

60


Appendix 1: procedure for reporting and

investigating alleged collusion

1. Purpose

To provide a procedure for reporting and investigating collusion or any behaviour

witnessed or strongly suspected, whether deliberate or not, which could compromise

the validity of EQA results and consequently damage the integrity of the scheme.

2. Scope

This procedure applies to all scheme participants and EQA staff.

3. Policy

Participants must not pass on information during the EQA round to allow his or her

opinions to be submitted by another. The scheme defines such behaviour as collusion.

Collusion or any behaviour that threatens the integrity of the scheme is taken seriously

and will be escalated to the highest authorities.

4. Responsibilities and authorities

All participants and EQA staff have a responsibility for reporting collusion or any

behaviour, which could potentially undermine the scheme’s integrity.

The scheme organiser is responsible for responding to a reported allegation of

collusion. The scheme organiser is responsible for collaborating with the individuals

involved and the appropriate laboratory lead.

The appropriate laboratory leads are responsible for any local investigation and

notifying the outcome to the scheme organiser.

If collusion is proven, the appropriate laboratory lead is obligated to refer the matter to

the relevant professional regulatory body and notify the scheme organiser.

5. Procedure

In the first instance, the EQA facilitator must be notified in writing if collusion is

witnessed or strongly suspected. The EQA facilitator refers the information to the

scheme organiser.

61


The procedure for managing allegations of collusion is set out below.

5.1 Medical staff

The scheme organiser writes to individuals separately via the EQA facilitator within 7

days of being notified about the allegation.

The recipients are asked to acknowledge receipt of the letter and respond to the

scheme organiser within 14 days. The recipients are advised to identify themselves to

the clinical head of the laboratory and offer an explanation for the allegation of

collusion.

The scheme organiser writes to the clinical head of the laboratory setting out the

concern, and explains that the individuals are required to identify themselves to him or

her. The email does not identify the individuals concerned. The clinical head of the

laboratory is asked to acknowledge the letter and respond to the scheme organiser

within 14 days of receipt.

The clinical head of the laboratory provides appropriate feedback and guidance to the

individuals concerned. The clinical head of the laboratory writes to the scheme

organiser to confirm the outcome of any local investigation within a further 14 days.

If no acknowledgement is forthcoming by 21 days then a reminder is sent. If no reply is

received after a further 7 days, the scheme organiser notifies the chair of NQAAP. The

NQAAP chair speaks with the head of screening QA about further action.

If collusion is proven, the EQA round of the individuals concerned is invalidated and

counted as sub-standard performance. The clinical laboratory lead reports the matter to

the organisation’s HR department, SQAS and relevant regulatory body. The clinical

laboratory lead notifies the scheme organiser who in turn notifies NQAAP.

5.2 Non-medical staff

The scheme organiser writes to individuals separately via the EQA facilitator within 7

days of being notified about the allegation.

The recipients are asked to acknowledge the letter and respond to the scheme

organiser within 14 days of receipt. The recipients are advised to identify themselves to

the scientific head of the laboratory and offer an explanation for the allegation of

collusion.

The scheme organiser writes to the laboratory scientific head for cervical cytology

setting out the concern, and confirms that the individuals have been advised to identify

themselves to him or her. The email does not identify the individuals concerned.

62


The scientific head of laboratory is asked to acknowledge the letter and respond to the

scheme organiser within 14 days of receipt.

The scientific head of laboratory provides appropriate feedback and guidance to the

individuals concerned. The scientific head of the laboratory writes to the scheme

organiser to confirm the outcome of any local investigation within a further 14 days.

The laboratory’s scientific head informs the lead consultant for the cytology service and

the medical head of department. If it is the scientific head that is under investigation,

then the copy letter goes to the consultant pathologist responsible for reporting cervical

cytology.

If no acknowledgement is forthcoming by 21 days then a reminder is sent. If there is still

no response after a further 7 days, then the scheme organiser notifies the NQAAP

chair. The NQAAP chair speaks with the head of screening QA about further action.

If collusion is proven, the EQA round of the individuals concerned is invalidated and

counted as sub-standard performance. The scientific laboratory lead reports the matter

to the organisation’s HR department, SQAS and the relevant regulatory body. The

scientific laboratory lead notifies the scheme organiser who in turn notifies NQAAP.

National external quality assessment (EQA) scheme for gynaecological cytopathology: scheme protocol

63

Slides cleaned and delivered to next

laboratory

Identify

participants

and prepare

forms

Slide

circulation

begins

Slides/forms delivered to laboratory

Submitted classifications released once EQA complete

Interim feedback given

Appendix 2: EQA timeline

Contact laboratories to request slides for inclusion in the rounds

Slide

review

Assembly of

slide sets

Set dates for

participation

Late

participation

Statistical analysis and identification of persistent substandard performance

Final

feedback

issued

Slide

circulation

ends

Slides returned.

Participants’ meeting held

annually following

completion of two EQA rounds

64

Appendix 3: referral sheet

Referral of abnormal slides for reporting

Referral for reporting is on the basis that the slides indicated are, in your opinion,

potentially or probably abnormal. You will not be marked on the basis of your responses

on this form. Please treat the slides as you would in normal practice within your

laboratory (for example, ringing, dotting or marking as appropriate).

Cytology screener

If you are a cytology screener, please indicate with an ‘R’ the slides you would refer to a

checker or pathologist for reporting on the basis of your opinion. The pathologist

examines the slide according to his or her normal practice and forms his or her own

opinion.

Checker

If you are a checker, please indicate with an ‘A’ the slides you would refer to a

pathologist for reporting on the basis of your opinion. Please look only at slides marked

‘R’ by the cytology screener. The pathologist examines the slide according to his or her

normal practice and forms his or her own opinion.

Hospital ....................................

Slide set identification ..............

Please mark in the appropriate box

Slide number

1

2

3

4

5

6

7

8

9

10

Cytology screener

Checker

Note that this form is not part of the assessment and is retained for laboratory records

only.

65

Appendix 4: scoring matrix for cytology

screeners/checkers

Consensus

Negative Inadequate Abnormal

Negative 2 0 0

Inadequate 0 2 0

Borderline changes 0 0 2

Low-grade dyskaryosis 0 0 2

High-grade dyskaryosis (moderate) 0 0 2

High-grade dyskaryosis (severe) 0 0 2

invasive squamous carcinoma 0 0 2

Glandular neoplasia of endocervical type 0 0 2

Glandular neoplasia (non-cervical) 0 0 2

Slide correctly interpreted Slide incorrectly interpreted


66

Appendix 5: scoring matrix for pathologists/

consultant biomedical scientists

Response

Inadequate

Negative

Borderline Low- grade

High- grade Moderate

High- grade Severe

Invasive GN endocx

GN non-cx

Co

ns

en

su

s

Inadequate 2 0 0 0 0 0 0 0 0

Negative 0 2 0 0 0 0 0 0 0

Abnormal/ no grade

0 0 2 2 2 2 2 2 2

Borderline 0 0 4 3 2 2 2 2 2

Bord/Low- grade

0 0 4 4 3 2 2 2 2

Low-grade 0 0 3 4 3 2 2 2 2

Low- grade/ HG Mod

0

0

3

4

4

3

2

2

2

High-grade Moderate

0 0 2 3 4 3 2 2 2

HG Mod/ HG Sev

0 0 2 3 4 4 3 3 3

High-grade Severe

0 0 2 2 3 4 3 3 3

HG Sev/ Inv

0 0 2 2 3 4 4 3 3

HG Sev/ GN endocx

0

0

2

2

3

4

3

4

3

HG Sev/ GN non- cx

0

0

2

2

3

4

3

3

4

HG Sev/ GN

0 0 2 2 3 4 3 4 4

Invasive 0 0 2 2 2 3 4 3 3

Inv/ ?GN endocx

0

0

2

2

2

3

4

4

3

Inv/ GN non-cx

0

0

2

2

2

3

4

3

4

Inv/ GN 0 0 2 2 2 3 4 4 4

GN endocx

0 0 2 2 2 3 3 4 3

GN non- cx

0 0 2 2 2 3 3 3 4

GN (GN endocx/ GN non- cx)

0

0

2

2

2

3

3

4

4

Slide correctly interpreted Slide incorrectly interpreted

One grade from consensus opinion > one grade from consensus opinion



67

Appendix 6: table of performance examples with action points

1st Round 2nd Round 3rd Round 4th Round 5th Round 6th Round 7th Round

Rounds contributing to 1st Action Point

Rounds contributing to 2nd Action Point

Participant A Substandard Satisfactory Substandard 1st Action Point

Substandard 2nd Action Point

1, 2 & 3 2, 3 & 4

Participant B Substandard Satisfactory Substandard 1st Action Point

Satisfactory Substandard 2nd Action Point

1, 2 & 3 3, 4 & 5

Participant C Substandard Non-valid non-participation 1st Action Point


1 & 2 2, 3 & 4

Participant D Substandard Substandard 1st Action Point

Satisfactory Satisfactory Substandard Satisfactory Substandard 1st Action Point

1 & 2 5, 6 & 7

-

Participant E Substandard Substandard 1st Action Point

Substandard 2nd Action Point

1 & 2 1, 2 & 3

Participant F Substandard Substandard 1st Action Point


1 & 2 2, 3 & 4

Participant G Substandard Substandard 1st Action Point

Satisfactory Satisfactory Substandard Substandard 1st Action Point

1 & 2 4, 5 & 6

-

Participant H Substandard Satisfactory Substandard 1st Action Point

Non-valid non-participation 2nd Action Point

1, 2 & 3 2, 3 & 4

Participant I Substandard Substandard 1st Action Point

Satisfactory Non-valid non-participation 2nd Action Point

1 & 2 2, 3 & 4

Participant J Non-valid non-participation

Substandard 1st Action Point

Satisfactory Satisfactory Non-valid non-participation

Satisfactory Satisfactory 1 & 2 -

Participant K Satisfactory Non-valid non-participation

Satisfactory

Non-valid non-participation Cessation of screening

- -

Participant L Substandard Satisfactory Non-valid Non-participation 1st Action Point

Satisfactory

Non-valid Non-pariticpation Cessation of screening

1, 2 & 3 -


68

Appendix 7.1: flowchart of actions taken

when first action point is reached

EQA facilitator informs scheme

organiser

Scheme organiser sends

reminder

If no reply within 3 weeks

NQAAP chair informed

First action point is

reached

Medical staff/Consultant

biomedical scientists Non-medical staff

EQA facilitator informs scheme

organiser

Scheme organiser writes to

participant/s

Participant confirms and

evidences implementation of

action plan to scheme

organiser within 4 weeks

Scheme organiser writes to

participant/s (cc letter to

scientific head and medical

head of department)

Scientific head confirms and

evidences implementation of

action plan to scheme

organiser within 4 weeks

Implementation of action plan

not confirmed

Scheme organiser sends

reminder

If no reply within 3 weeks

NQAAP chair informed


69

Appendix 7.2: flowchart of actions taken

when second action point is reached

Chair corresponds with

individual via EQA facilitator

If lack of co-operation or no

reasonable explanation

Chair of Joint Working Group on

QA informed

Professional performance

committee of RCPath/Trust

medical director/Health & Care

Professions Council informed

Professional performance

committee review panel

convened

Second action point is

reached

Medical staff/Consultant

biomedical scientists Non-medical staff

Scheme organiser informs

NQAPP chair

Investigation panel convened

Panel seeks possible

explanations

Scheme organiser informs

NQAPP chair and writes to

individual/s (copy scientific

head and medical head of

department)

NQAPP chair and head of

screening QA agree further

action

To

be

co

mp

lete

d w

ith

in 4

wee

ks


70

Appendix 8: formal complaints procedure

1. Purpose

This guidance provides a procedure for dealing with complaints from all stakeholders in

the event of dissatisfaction with any aspect of scheme operation or management.

2. Scope

This procedure applies to all participants, EQA staff and stakeholders.


Participants are responsible for following the procedure if making a complaint. EQA

staff are responsible for taking appropriate action and offering a solution within the

required timeframes.

4. Procedure

If a participant wishes to make a formal complaint, then he or she must do so by writing

to the EQA facilitator.

The EQA facilitator sends a holding response within three working days of receiving the

complaint.

The complaint is logged together with a summary of the communication and

subsequent discussions that may involve other EQA staff including the scheme

organiser.

The complainant can expect to receive a response offering a solution within 28 working

days.

If the matter is not resolved to the satisfaction of the complainant then the EQA

facilitator will refer the matter to the scheme organiser. The scheme organiser will

provide a response within 28 working days.

If the matter is still not resolved then the scheme organiser will escalate it to the EQA

management board. The EQA management board decision is final. A final response

can be expected within 28 working days.

If the complainant remains dissatisfied with the outcome, or the matter becomes

intractable, then they can self-refer the matter directly to NQAAP.


71

Appendix 9: appeals procedure

1. Purpose

This guidance provides a procedure for dealing with participants in the event of an

appeal against an evaluation of their EQA performance.

2. Scope

This procedure applies to all participants.


Participants are responsible for making a timely appeal against an evaluation of their

EQA performance. EQA staff are responsible for taking appropriate action and offering

a solution within the required timeframes.

4. Procedure

If a participant wishes to appeal against an evaluation of their EQA performance, then

he or she may do so by writing to the EQA facilitator within 28 working days upon

receipt of their final results report.

The EQA facilitator sends a holding response within three working days of receiving the

appeal.

The appeal is logged together with a summary of the communication and subsequent

discussions that may involve other EQA staff including the scheme organiser.

The appellant can expect to receive a response offering a solution within 28 working

days.

If the matter is not resolved to the satisfaction of the appellant then the matter is

referred to the scheme organiser. A response is provided within 28 working days.

If the matter is still not resolved then it will be escalated to the EQA management

board. The EQA management board decision is final. A final response can be expected

within 28 working days.

If the appellant remains dissatisfied with the outcome, or the matter becomes intractable

then they can self-refer the matter directly to NQAAP.

national external quality assessment (eqa) scheme for ... · 3 cervical screening programme -...

Documents