current trends for quality assurance in radiotherapy

Current Trends for Quality Assurance in Radiotherapy

Tommy Knöös, Sweden

3

Objectives

Can we improve the efficency of our quality assurance?

Quality assurance efforts are increasing with new modalites and technology

Can we change our approach?

Can we learn from other organisations?

2010-05-27 Dublin

Dublin2010-05-27

Background The scientific world is full of

• Suspension/Tolerance levels• Action levels• Time intervals

For testing a huge amount of parameters• Treatment units• CT scanners• Treatment planning systems• Especially technical and physical ones

Focus of current QM/QA guidelines• measure functional performances of

radiotherapy equipment by measurable parameters

• set tolerances at desirable but achievable values

What is on a cancer patient’s mind?

• Safety

• Radiation therapy accident article published in NY Times

AAPM Summer School 20115Saiful Huq - New Paradigms for QM in RT

Dublin

6

How to avoid these?

2010-05-27

Lisa Norris

Brian Lara Cancer Centre Trinidad and Tobago

Can current QM standards prevent such events and meet the safety standards of

new clinical challenges?

Failure of QM:

• Insignificant errors in dose to the target • Major events that can injure or kill patients

Main goal of QM:

• Patients will receive the prescribed treatment correctly


Examples of current QA paradigms: Annual calibration of linac

that is it could have been spent checking things with a higher likelihood of failure

• The annual QA takes several days to complete

• If everything checks out, the effort was mostly wasted

• If some problem was found, how long had it been wrong?

• If the problem was significant then a daily check should be in place to prevent it


• We measure fluence maps for IMRT QA• This is certainly something that we would want to

have correct• However, if the system has been commissioned, and

then used correctly, why would this be wrong for the patient?

• If you worry about the leaf movement, even if the fluence map indicates correct movement for the first day of treatment, why do you think it might be correct in a week?

• Are there problems found this way? Almost never• Do we need QA for leaf movement, maybe daily???Saiful Huq - New Paradigms for QM in RT 9

AAPM Summer School 2011

Xample: IMRT QC

Problems with current QA paradigm

• These examples suggest that QA as performed currently might not be the optimal use of a physicist’s time

• Just performing all the recommended QA might not provide protection against events

Saiful Huq - New Paradigms for QM in RT 10AAPM Summer School 2011

Implementation of industrial tools

Dublin2010-05-27

Quality management tools

Process mapping• Processes and their sub processes

Identify the processes which are more error prone• FMEA (Failure Mode and Error Analysis)• RCA (Root Cause Analysis)

Identify measures for control• SPC (Statistical Process Control)

Measure the quality of the measures/control points• Score actual and potential deviations (e.g.

ROSIS)

ICARO 20092009-04-28

Process maps Process map is a visual demonstration of work processes The map shows how inputs, outputs, and tasks are linked together

• Identifies the major steps in the process• Who performs the steps

Provides a visual picture of the process Identifies and understand process deviations and vulnerabilities,

breakdowns, mistakes, delays - detects where improvements may be made

A.k.a flow charting Numerous formats or approaches exist - As-is and/or To-be

Imaging Volumes Planning Review

Prescription

Trearment (1-n)

Accelerator

Treatment finished

Treatment starts

TPSCT, PET/CT,

MR ...

ICARO 20092009-04-28

Work plan

1. Establish process boundaries

2. Develop the data gathering plan

3. Interview the process participants

4. Generate the process map

5. Analyze and use the map

ICARO 20092009-04-28

From Ford et al IJROBP, Vol. 74, No. 3, pp. 852–858, 2009

Dublin

17

2010-05-27

Dublin

18

Process mapping

2010-05-27

S Huq et al. (Red Journal QA issue)

ICARO 20092009-04-28

Example of “swim lane” flow chart: Patient to have an diagnostic x-ray

From “Japanese Association of Healthcare Information Systems Industry“

Dublin2010-05-27






ROSIS)

Use process maps to identify critical steps in the process and/or when investigating incidents and

accidents

ICARO 20092009-04-28

Identifying hazards and error prone sub-processes

FMEA – Failure Mode and Error Analysis During the FMEA, which is a proactive method, the

following questions have to be answered for each step in each sub-process; • a) what could possibly go wrong (potential failure mode), • b) how could that happen (i.e., what are the causes that

result in a failure mode) and finally • c) what effects would this failure mode produce (potential

effects of failure) Suitable for creating Fault Trees

ICARO 20092009-04-28

FMEA

For each failure mode, estimate:• The probability the failure occurs (“O”)• The severity of it’s effects (“S”)• The probability that the failure will be

undetected (“D”) Form the “Risk Priority Number” RPN:

• RPN=O*S*D O,S,D range [1,10] Concentrate on the highest RPN

Step Potential failure modes

Potential causes

of failure

Potential

effects of

failure

O S D RPN

Comment

FMEAFor a given sub-process:

RPN = O x S x D [ 1 ≤ RPN ≤ 1000 ]

AAPM Summer School 2011Saiful Huq - New Paradigms for QM in RT 24

O, S, and D values used in FMEARank Occurrence (O) Severity (S) Detectability

(D)Qualitative Frequency Qualitative Categorization Estimated

Probability of going

undetected in %

1 Failure unlikely

1/10,000 No effect 0.01

2 2/10,000 Inconvenience Inconvenience 0.2

3 Relatively few failures

5/10,000 0.5

4 1/1,000 Minor dosimetric error

Suboptimal plan or treatment

1.0

5 <0.2% Limited toxicity or under-dose

Wrong dose, dose

distribution, location or

volume

2.0

6 Occasional failures

<0.5% 5.0

7 <1% Potentially serious toxicity or under-dose

10

8 Repeated failures

<2% 15

9 <5% Possible very serious toxicity

Very wrong dose, dose

distribution, location or

volume

20

10 Failures inevitable

>5% Catastrophic >20

25

Major proce

ss

Step Potential

failure mode

Potential causes

of failure

Potential effects

of failure

O S D RPN

Transfer images &

DICOM Data

Transfer Primary data set

Incorrect dataset

associated with

patient

Pull up wrong

patient’s record

Very wrong dose disn

Very wrong vol

1 9 9 81

Phy B 5 9 3 135

Phy C 6 8 7 336

Phy D 3 9 6 162

Phy E 3 9 8 216

Phy F 3 8 3 72

Phy G 5 9 3 135

Phy H 5 9 8 360

Avg 3.88 8.75 5.88

188

FMEA

26

ICARO 20092009-04-28

Example of a FMEA analysis – Setting up a patient and deliver a treatment at an incorrect position

Potential failure mode

Potential cause(s) of failure Potential effect(s) of failure

O S D RPN Proposed action(s)

Incorrect isocentre

Shift between reference point and isocentre not present in R/V system

Dose delivered to wrong volume, PTV under-dosed and/or OAR overdosed during all fractions

4[1] 10[2] 3[3] 120 Second check of all parameters, review methods

Shift specified incorrectly in set-up instructions

See above 6[4] 10 5[5] 300 Training and second check of all parameters, review methods

Shift specified correctly but made incorrectly

Dose delivered to wrong volume, PTV under-dosed and/or OAR overdosed during one fraction

4 6[6] 3 72 Training, verify table position

Staff omitted to make shift See above 2 6 10 120 Training, verify table position

[1] The value was chosen based on that this do not happens to often. Suggestion is that the data is missing once per 2000 cases. In Lund we have about 2600 patients annually and it seems to be an accurate number that this data is missing not more than once a year.[2] If this will be undetected though out the full treatment the ranking of 10 maybe be adequate.[3] This number is incredible hard to set since it depends very much on the clinical environment. If for example it is possible to acquire the patient position (treatment position) by a simple action by the therapist this can be added to the setup and the patient will be treated “correctly” at each fraction. On the other hand since the data is missing one may step back in the process to get the correct data. Based on the latter process a low number is assigned.[4] This probably occurs much more often than the above case.[5] The delectability is definitely lower in this case compared to the first failure situation.[6] Having a lower severity for this failure mode seems appropriate since there are chances to detect this at the following treatment sessions.

Dublin2010-05-27






ROSIS)

Dublin

29

2010-05-27

From Todd Pawlicki

Dublin

30

2010-05-27

From Todd Pawlicki

2010-05-27 Dublin

Statistical Process Control -

SPC An effective method of monitoring a process

through the use of control charts Control charts distinguish background

variation from events of significance based on statistical techniques • Random variation in the data is de-emphasized

by the way the process behaviour limits are constructed

Using process behaviour charts is an interactive procedure that requires process knowledge and interpretation by the user

Dublin

32

Example of SPC for IMRT verification

2010-05-27

Clin tolerance

Clin tolerance

UCL

LCL𝑋

Todd Pawlicki et al.

SD

SD

Dublin

33

Example of SPC for independent monitor unit check

2010-05-27

Private communication F Nordström

Dublin

34

Four states can be identified

State I• in control and within specifications (IN-IN)• Action required - maintain process control by continuing to monitor the

process State II

• in control and out of specifications (IN-OUT)• Action required - change the equipment and/or procedures of the

process State III

• out of control and within specifications (OUT-IN)• Action required - identify and remove systematic errors in the process

State IV• out of control and out of specifications (OUT-OUT)• Action required - identify/remove systematic errors and/or change the

equipment and/or procedures of the process

2010-05-27

Dublin

35

2010-05-27

From Todd Pawlicki

Dublin

36

QC of IMRT Γ - analysis

2010-05-27 From Létourneau et al 2010 in Med Phys

3% of dose difference (%ΔD) or 2 mm of distance to

agreement (DTA) and an inclusion threshold (%Th) of 10%

Initial model

Initial model

Optimized model

Optimized model

Prostate

Paraspinal SRT

SunN

ucl

ear

MapC

heck

Simple rules to analyse control charts

2011-May ESTRO Anniversary

Dublin2010-05-27






ROSIS)

Check sheets

What• A pre-defined format for data

collection• Categories are created in advance

but my be added• A mark is added for each example

When To keep track of events/parameters

in a process• Number of correct/incorrect outcome

of certain processes• Number of errors in certain important

parameters


39

Probably suitable for registration the

efficiency in various defence layers/ in your

QA system

We have performed 140 000 MU checks with an

error frequency of 0.003%

http://qatestlab.com/

ICARO 20092009-04-28

What we need?

What errorsorrurs? Or• What incidents?

The frequency One have to

remember – low probability

Learn from each other!!!

A radiotherapy example


41

Thanks Petra Reijnders-Thijssen for the data

Start here Take these later

O – OrganisationH – Human

T – Technical P – Patient

ICARO 20092009-04-28

ROSIS - SAFRON

www.rosis.info

Dublin

43

Summary

Using a process characterization scheme, the control chart approach provides better discrimination of process performance than the standard deviation approach.

Control charts also have implications for clinical trials QA where they may be used to appropriately categorize process performance, minimize protocol variations and guide QA process improvements.

A LEAN approach

2010-05-27

Dublin

44

The Problem!!! RT QA guidance is focused on equipment

performance even though most RT events have resulted from human performance failures rather than equipment failure

Lack of adequate guidance for resource allocation• The premise: If it can be checked, check it• That is…..everything

Lack of adequate personnel

2010-05-27

Dublin

45

The Problem (cont)

Pressure on medical physicists to implement new technologies

Lack of guidance on how to implement these technologies

Delays in publishing QA protocols or guidelines for emerging technologies

2010-05-27

Where are the resources and time to do it all?

What should a time starved health worker do?

What to Do?

46Saiful Huq - New Paradigms for QM in RT AAPM Summer School 2011

current trends for quality assurance in radiotherapy

Documents