use of microbial risk assessment in decision-making david vose consultancy 24400 les lèches...

Use of Microbial Risk Assessment in Decision-Making

David Vose Consultancy

24400 Les Lèches

Dordogne

France

www.risk-modelling.com

Email

David Vose's secretary David Vose

Slide show on: www.risk-modelling.com/firstmicrobial.htm

Note the 2 ‘l’s !

David Vose Consultancy Ltd

www.risk-modelling.comMicrobial risk analysis in

decision making

IntroductionApplying CODEX guidelines in reality

DifficultiesOther ways of thinking

Experience with microbial modellingSome survey resultsThe Dutch experienceSome US experience

Modelling challengesComparison of some complete modelsReviewing a model in context



decision making

Microbial risk assessment is a scientifically-based process consisting of four steps:

1. Hazard Identification The identification of known or potential health affects associated with a particular agent;

2. Exposure Assessment The qualitative and/or quantitative evaluation of the degree of intake likely to occur;

3. Hazard Characterization The quantitative and/or qualitative evaluation of the nature of the adverse effects associated with biological, chemical and physical agents that may be present in food… For biological agents… a dose-response assessment should be performed if the data is available;

4. Risk Characterization Integration of Hazard Identification, Hazard Characterization and Exposure Assessment into an estimation of the adverse effects likely to occur in a given population, including attendant uncertainties.

Codex Alimentarius CommissionFAO/WHO (1995)



decision making

OIE experience

OIE produced guidelines for animal import risk assessments (for the management of disease spread)Now in its second editionGuidelines were offered as a way to help member (including developing) countries understand how to perform a r.a.First Ed. guidelines were used too literally, both by analysts and lawyers, and found to be often impractical or irrelevant to the risk questionLesson: keep guidelines non-specific, encourage understanding rather than prescribing a formulaic approachPopular interpretation of CODEX guidelines suffer similarly



decision making

(1) Risk analysis uses observations about what we know to make predictions about what we don’t know. Risk analysis is a fundamentally science-based process that strives to reflect the realities of Nature in order to provide useful information for decisions about managing risks. Risk analysis seeks to inform, not to dictate, the complex and difficult choices among possible measures to mitigate risks...

Society for Risk AnalysisPrinciples for Risk Analysis



decision making

(2) Risk analysis seeks to integrate knowledge about the fundamental physical, biological, social, cultural, and economic processes that determine human, environmental, and technological responses to a diverse set of circumstances. Because decisions about risks are usually needed when knowledge is incomplete, risk analysts rely on informed judgment and on models reflecting plausible interpretations of the realities of Nature. We do this with a commitment to assess and disclose the basis of our judgments and the uncertainties in our knowledge.

Society for Risk AnalysisPrinciples for Risk Analysis



decision making

Current modelling

Microbial QRA is a developing scienceWe’re making a lot of progress, but it is still in infancy

Mostly producing ‘farm-to-fork’Models the whole system but very poorly

Not designed to model any decision question well

Often relies on poor data, surrogates, and guessesAlmost never is a decision question posed beforehandAssessors have probably over-sold QRA’s usefulnessManagers have expected too much



decision making

F2F Achilles’ HeelsVery little data available, system being modelled is hugely complex!

Uncertainty, variability, inter-individual variablilityTake too long to complete, too easy to make mistakesF2F considers only pathogen on the food source

E.g. not E.coli produced during life of animal, appearing in water, vegetables, farmers’ exposure

Predictive microbiology still unreliableBroth data doesn’t translate well to food (usually overestimate, but some data – Tamplin, USDA – shows lag period can be shorter, e.g. E.coli in ground beef, Listeria in processed hams)Models often not based on physical/biological ideas, so we don’t learnAttenuation may not be death, and ignores reactivation of bacteria

D-R models inadequateDon’t describe variability observedP(ill|dose, infected) = P(ill|infected)?Feeding trial data don’t match epi data – can hugely underestimate the risk

Little cost-benefit analysis effort madeIncluding actions affecting several risk issues

Requires enormous resources – impractical for many countries



decision making

The lessons learnt from risk analysis experiences:1. Risk management has not always been an integral part of risk

analysis so far;

2. Risk managers should be trained to understand risk assessment, and risk assessors should be trained to explain their work;

3. Available data are often of limited use for risk assessment and communication of data needs between risk assessors, food scientists and risk managers is a critical issue;

4. The risk manager questions usually require rapid results, whereas (farm-to-fork) risk assessment projects require several years to complete. Solving this conflict requires open communication;

5. Uncertainty is often large.

Dutch observations on past QRAHavelaar, Jansen (2002)



decision making

Our surveyInternet based, voluntary participation, 39 valid responses

Do you consider that risk analysis has brought about the following improvement to government decision-making?

0%

10%

20%

30%

40%

50%

60%

Fairness Rationality Consistency Transparency

Very much

Much

Some

Little

Very little



decision making

What factors jeopardise the value of an assessment?

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Insufficient human resources tocomplete the assessment

Insufficient time to complete theassessment

Insufficient data to support therisk assessment

Insufficient in-house expertise inthe ares

Insufficient general scientificknowledge of the area

Always

Usually

50:50

Seldom

Never



decision making

What factors jeopardise the appropriate implementation of a risk management decisions?

0%

10%

20%

30%

40%

50%

60%

Politics Other issues take precedence Legal restrictions Insufficient resources toimplement the action

Risk assessment toocomplicated

Risk assessment not acceptedas valid

Insufficient time

Always

Usually

50:50

Seldom

Never



decision making

Do decision makers:

0%

10%

20%

30%

40%

50%

60%

70%

9) Assign sufficientresources and time to

complete the riskassessment?

10) Encourage assessorsto suggest alternative

approaches to theassessment?

11) Require theassessment to be simplif iedfor ease of understanding,at the expense of technical

accuracy?

12) Make the resultsavailable only if it suits their

purposes?

13) Encourage assessorsto produce an assessmentto support a predetermined

position?

14) Involve risk assessorsin the decision-making?

15) Allow /expect the riskassessors to make the

decision?

Always

Usually

50:50

Seldom

Never

Do decision makers:

0%

10%

20%

30%

40%

50%

60%

70%

1) Understand and usethe results of a risk

assessment

2) Encourageinvolvement of

stakeholders andexternal expertise?

3) Encourageassessors to explain

w hat may and may notbe possible to achieve?

4) Put a lot of emphasison receiving commentsat the planning stage of

a risk assessment

5) Involve riskassessors in planninghow to communicatethe risk assessment

results?

6) Involve stakeholdersafter completion of the

assessment of theappropriate risk

management action totake?

7) Expect too much of arisk assessment?

8) Expect too little of arisk assessment?

Always

Usually

50:50

Seldom

Never



decision making

Completion times of some farm-to-fork QRAs

Jan-96 Jan-97 Jan-98 Jan-99 Jan-00 Jan-01 Jan-02

US FSIS SE

USDA Vibrio

FDA Listeria

FSIS E. Coli

CVM Campy

Harvard BSE Final report

Draft report

Being revised

Draft report

Final report

Final report



decision making

S. enteritidis only: mean outbreak attack rates (plus 90% confidence intervals)

0%

20%

40%

60%

80%

100%

1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 1.E+08 1.E+09 1.E+10

Dose

Inte

rnat

iona

l out

brea

k at

tack

rat

e

Normal; Normal food

Normal; Fatty food

Susceptible; Normal Food

Susceptible; Fatty food

All participants

Naive participants

S. typhirurium only: mean outbreak attack rates (plus 90% confidence intervals)

0%

20%

40%

60%

80%

100%

1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 1.E+08 1.E+09 1.E+10

Dose

Inte

rnat

iona

l out

brea

k at

tack

rat

e

Normal; Normal food

Normal; Fatty food



All participants

Naive participants

Salmonella dose-responseEpi and feeding trial comparison

All Salmonella spp.: Mean outbreak attack rates (plus 90% confidence intervals)

0%

20%

40%

60%

80%

100%

1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 1.E+08 1.E+09 1.E+10

Dose

Inte

rnat

iona

l out

brea

k at

tack

rat

e

Normal; Normal food

Normal; Fatty food



All participants

Naive participants

Review by Amir Fazil in FAO/WHO (2001)

D-R mathematical models review by Haas (2002)



decision making

“Although the goal was to make the model comprehensive, it has some important limitations. It is a static model and does not incorporate possible changes in SE over time as either host, environment or agent factor change. For many variables, data were limited or nonexistent. Some obvious sources of contamination, such as food handlers, restaurant environment, or other possible sites of contamination on or in the egg (such as the yolk), were not included. And, as complex as the model is, it still represents a simplistic view of the entire farm-to-table continuum. Finally, the model does not yet separate our uncertainty from the inherent variability of the system. Much more work is needed to address this, and all other, limitations.”

USDA-FSIS-FDA Salmonella Enteritidis



decision making

USDA-FSIS-FDA Salmonella Enteritidis

Original model impetus was to evaluate effect of refrigeration temp from laying to retail on food safety

Empirically must have little affect since it only deals with a few days in the life of an egg

No cost-benefit attached

Now being redone to focus on level of performance required for shell, and liquid egg pasteurisation

i.e. much more decision focused



decision making

FDA Listeria risk assessment

No specific decision questions attached

Attempted to look at relative importance of a large list of Listeria-carrying foods

Given the data available, perhaps the only method possible to estimate which food types contribute the greatest risk

So a good QRA application



decision making

Remedies: focusing on decisionsConsider what is known about the risk problem, and data available immediately or within acceptable time frame

Use epi data as much as possibleCollect more epi data (e.g. Japan, Denmark)

Consider what analysis could be done with this knowledgei.e. a risk-based reasoned argument for evaluating particular actions

Estimate the possible magnitude of benefit for a risk actionNote that it may not be possible to evaluate all actions

Perform a cost-benefit analysis on these actionsPerform a Value of Information analysis

Determines whether it is worth collecting more data before making a decision

Consider strategy to validate whether predicted improvement occursTrain data producers to supply maximally useful data

E.g. microbiologists taken more than one cfu from a plateMore inter-agency unity

E.g. Farm (APHIS) Slaughter (FSIS) Retail (FDA)



decision making

Make it as simple as possible: example

Risk: Human illness from SE in eggsA shell-egg selection system proposed that will reduce by 30% the number of contaminated eggs going to marketCurrently, 30,000 people a year suffer from SE from eggsWhat will be the reduction in cases if the new system is implemented?

Reduction in cases = 30%*30,000 = 9,000 people/year

No need for models of D-R, bacterial growth, handling, etc.

Vulnerability to assumptions smaller than from using F2F model



decision making

Campylobacter Risk Management and Assessment

Dutch proposal

The main objectives of the project are to advise on the effectiveness and efficiency of measures aimed at reducing campylobacteriosis in the Dutch population. The two key questions are:

1.What are the most important routes (quantifiable?)?

2.Which (sets of) measures can be taken to reduce the exposure to Campylobacter, what is their expected efficiency and societal support?

An example of the way forwardHavelaar, Jansen 2002



decision making

The target of the assessment is not limited to estimating the possible reduction in disease incidence but to evaluate both costs and benefits of possible interventions and to access their acceptance by stakeholders.

Interventions with low social support will require more effort to uphold, which increases their costs and reduces their efficacy.

The way forward – cont.



decision making

Danish Vet Service Salmonella QRA“A Bayesian Approach to Quantify the Contribution of Animal-food Sources to

Human Salmonellosis” - Hald, Vose, Koupeev (2002)

0 200 400 600 800 1,000 1,200 1,400

Unknown source

Beef

Ducks

Turkeys

Imported beef

Broilers

Imported pork

Imported poultry

Pork

Outbreak

Travel

Eggs

Estimated number of cases

97.5% percentile

Mean

2.5% percentile

Estimated number of cases of human salmonellosis in Denmark in 1999 according to source

Model ranks food sources by risk. Easily updateable with each year’s data. Bayesian update improves estimate and checks validity of assumptions.



decision making

Fluoroquinolone-resistant

Campylobacter risk assessment

Section 1Campylobacter culture

confirmed cases observablein US population

Section 2Total number of

Campylobacter infections inyear in US population

Section 3Number of those with

Fluoroquinolone-resistancefrom chickens and

administeredFluoroquinolone

Section 4Number of Fluoroquinolone

resistant Campylobactercontaminated chicken

carcasses consumed in year

Section 5Using the model to manage risk.

Measuring the level of risk.Controlling the risk.

Model:

Contaminated carcasses after slaughter plant * probability = affected people



decision making

Broiler house

Transport

Slaughter house Hanging Scalding

Defeathering Evisceration

Washing Chilling

Export Chicken parts Whole chickens

Chilled Frozen Import

Catering Cross contamination

Heat treatment

Retail

Consumer Cross contamination

Heat treatment

Dose response

Further Processing

Risk estimation

Slaughterhouse model

Consumer model

Example of Farm-to-Fork

model

Campylobacter in poultry

Draft report 2001

Institute of Food Safety and Toxicology

Division of Microbiological Safety

Danish Veterinay and Food Administration

Behaves the same way as CVM model if prevalence is reduced



decision making

Reviewing a risk assessment

Risk assessment should be decision focusedIt is not appropriate to review a risk assessment independently from the question(s) the assessment is addressing

Eg because a point is moot if the decision is insensitive to the argument

It uses science but is not itself scientific research

So we have to go with the best we’ve got



decision making

Finally - risk assessors should gain hands-on experience to ensure their models reflect the

real world



decision making

References

• Haas, C.N., (2002), Conditional Dose-Response Relationships for Micro-organisms: Development and Applications. Risk Analysis 22 (3): 455-464.

• Havelaar, H. and J. Jansen, (2002), Practical Experience in the Netherlands with quantitative microbiological risk assessment and its use in food safety policy. Draft paper, RIVM, Bilthoven, The Netherlands.

• Hope, B.K., et al. , (2002), An overview of the Salmonella Enteritidis Risk Assessment for Shell Eggs and Egg Products. Risk Analysis 22 (3): 455-464.

• Joint FAO/WHO Expert Consultation on the Application of Risk Analysis to Food Standards Issues (Joint FAO/WHO, 1995).

• Joint FAO/WHO Expert Consultation on Risk Assessment of Microbiological Hazards in Foods: Risk characterization of Salmonella spp. in eggs and broiler chickens and Listeria monocytogenes in ready-to-eat foods. (2001), FAO headquarters, Rome.

• Teunis, P.F.M. and A.H.Havelaar, (2001), The Beta-Poisson Dose-Response Model Is Not a Single-Hit Model. Risk Analysis 20 (4): 513-520.

use of microbial risk assessment in decision-making david vose consultancy 24400 les lèches...

Documents

risk analysis slide

microbial risk analysis

risk analysis principles

risk question lesson

context slide

microbial modelling

animal import risk assessments

hazard characterization