assessing the adequacy of epidemic models using hybrid approaches gavin j gibson & george...
Post on 21-Dec-2015
215 views
TRANSCRIPT
Assessing the Adequacy of Epidemic Models Using Hybrid Approaches
Gavin J Gibson & George StreftarisMaxwell Institute for Mathematical Sciences
Heriot-Watt University
Collaborators: Alex Cook, Chris Gilligan, Tim Gottwald, Glenn Marion, Mark Woolhouse, Joao Filipe,..
Funding: BBSRC, USDA
InFER2011 (Inference For Epidemic-related Risk) 28th March - 1st April 2011
Outline
• Modern algorithms allow computation of solutions to complex problems in inference
• Understanding and interpreting solutions is not always so easy!
• True for fitting/testing/comparing epidemic models
• Hybrid approaches & the Freudian Metaphor
• Examples and alternatives
• Questions and challenges
Bayesian inference for epidemics Experiment: yields partial data y. Stochastic model: with parameter specifying (y|). Aim: Express belief re as a probability density (|y).
Bayesian solution: Assign prior distribution (), to yield posterior (|y) ()(y|).
Problem: (y|) is often intractable integral.
Data augmentation: Consider data x from ‘richer’ experiment (i.e. y = f(x)) for which (x|) is tractable. Consider (, x | y) ()(x |)(y|x). Often straightforward to simulate from (, x |y) e.g. using MCMC.
Basic SEIR model
S E: If j is in state S at time t, then Pr(j is exposed in (t, t+dt)) = I(t)dt
E I: (random time in E)I R: (random time in I)
~E
j EET
~I
j IIT
Parameters: = (, E, I)
If times of transitions are observed (x), then likelihood (x | ) tractable.
Data y usually heavily censored/filtered (e.g. only removals are observed, weekly totals of new infections)
Fitting using McMC
(, x |y) ()(x, y|)
•Construct Markov chain with stationary distribution (, x |y)
•Iterate by proposing & accepting/rejecting changes to the current state (i, xi) to obtain (i+1, xi+1).
Updates to can often be carried out by Gibbs steps.
Updates to x, usually require Metropolis-Hastings and Reversible-Jump type approaches.
Iterate chain to produce sample from (, x | y). (See e.g. GJG + ER, 1998,O’Neill & Roberts, 1999, Streftaris & GJG, 2004, Forrester et al, 2007, Gibson et al., 2006, Chis-Ster et al. 2008, Starr et al. 2009,…)
Sensitivity to prior
Removals from smallpox epidemic (Bailey)
Markovian SEIR model: : contact rate,: removal rate, : E → I rate
1000 samples from posterior using uniform prior over cuboid.
Similar difficulties arise e.g. when considering infection processes incorporating primary (a) and secondary infection (b) rates. (Gibson & Renshaw, 2001)
Extension to spatio-temporal SI models
Susceptible j acquires infection at rate:
Rj = f(t; )( + i K(dij, ))
Spatial kernels (examples):
1. K(d, ) = exp(-d)
2. K(d, ) = exp(-d2)
3. K(d, ) = (d+1)-
j
Can be fitted using standard Bayesian/data augmentation/MCMC approach
See GJG (1997), Jamieson (2004), Cook et al (2008) , Chis-Ster & Ferguson (2009)
If models are used to design control strategy – e.g. spatial eradication programmes – then model choice can be crucial.
Example: Miami Citrus Canker epidemic (Gottwald et al., Phytopathology, 2002; Jamieson, PhD Thesis, U. Cambridge, 2004, Cook et al. 2008)
Data: Dade county, Miami
Optimal strategy for eradication sensitive to model choice
6056 susceptibles, 1124 infections after 12 30-day periods
Control strategies can be controversial
Classical-Bayesian Spectrum‘Classical’ model: fixed, model specifies (x |) , where x represents quantities varying between replicate experiments. Predicts frequencies for x given .
‘Bayesian’ model: Uncertainty in modelled as prior () giving (, x) ()(x | ). A framework for both prediction of x and learning about .
How ‘large’ should our space of possible be?
Very large - less need to benchmark against alternatives – but problems of prior representation and sensitivity, computational complexity
Very small – greater need to assess adequacy – sensitivity and complexity of inference reduced
Hybrid approaches to model adequacy
Example: Posterior predictive p-values (e.g. Rubin, 1984, XL Meng, 1994).
To test H0: the model is valid. Observe y, calculate teststatistic T(y), then consider
dTTPp )|)((| yy
Interpretation:
•Posterior probability of more extreme value of T in next experiment.•Posterior expectation of the classical p-value P(T > T(y); ), computed by classical statistician with knowledge of .
*in Handbook for data analysis in the behavioral sciences
EGO: Reason, common sense, translates the appetites if ID into action
SUPEREGO: Conscience, criticism of EGO
ID: Basic Instincts & Drives
The Freudian Metaphor (see e.g. Gigerenzer*, 1993) Co-existence of multiple facets of the statistical personality.
The Freudian Metaphor (see e.g. Gigerenzer*, 1993) Co-existence of multiple facets of the statistical personality.
*in Handbook for data analysis in the behavioral sciences
EGO: Reason, common sense, translates the appetites if ID into action
SUPEREGO: Conscience, criticism of EGO
ID: Basic Instincts & Drives
Gigerenzer
Neyman-Pearson
Fisher
Bayesians
*in Handbook for data analysis in the behavioral sciences
EGO: Reason, common sense, translates the appetites if ID into action
SUPEREGO: Conscience, criticism of EGO
ID: Basic Instincts & Drives
Gigerenzer
Neyman-Pearson
Fisher
Bayesians
GJG
Classical
Bayesian
The Freudian Metaphor (see e.g. Gigerenzer*, 1993) Co-existence of multiple facets of the statistical personality.
*in Handbook for data analysis in the behavioral sciences
EGO: Reason, common sense, translates the appetites if ID into action
SUPEREGO: Conscience, criticism of EGO
ID: Basic Instincts & Drives
Gigerenzer
Neyman-Pearson
Fisher
Bayesians
GJG
Classical
Bayesian
Physicists?
The Freudian Metaphor (see e.g. Gigerenzer*, 1993) Co-existence of multiple facets of the statistical personality.
Observes:
Imputes:
y
S
( |y)
p(T(y), ))
(p |y)
E Asserts model and ()
0 1
Schematic diagram of PP p-value
Large probability of small p-value indicates conflict between E and S
WORLD OF THE EGO
Suppose S applies Likelihood Ratio Test, for example
• T(y, ) = (y | )/1(y) where 1(y) denotes sampling density of y under an alternative model. Problem of intractable likelihood (y | ) arises again!
• Impute S’s response to observation of latent process x?
• So long as both and 1 specify a tractable sampling density for x, then (x|)/1(x) can be imputed (e.g. from MCMC).
PP p-value for model comparison
Observes:
Imputes:
y
x S
(x |y)
p(T(x, ))
(p |y)
E Asserts model and ()
0 1
Imputed p-value from a latent process
Large probability of small p-value indicates conflict between E and S
R solani in radish (GJG, et al., 2006)
18 x 23 grids of plants, daily sampling (approx):High inoculum: 45 randomly chosen sites (13 reps)Low inoculum: 15 randomly chosen sites (13 reps)
•Model: SI with primary infection (a), nearest-neighbour secondary infection (b0, b1, b2) representing max rate, variability and peak timing.
•Fitted using MCMC methods
Results: Replicates fitted jointly (assuming common parameters) and separately.
Model checking with imputed p-values
Symptomatic day 9 X
Missing O, Primary inoc. +
Symptomatic day 9 X
Sample of data (high inoculum)
High inoculum
Low inoculum
Posterior densities (primary infection rate, a)
High inoculum
Low inoculum
Posterior densities (peak secondary rate, b0)
Posterior densities (secondary variability, b1)
High inoculum
Low inoculum
Posterior densities (secondary peak time, b2)
High inoculum
Low inoculum
Posterior predictive envelope of I(t) (joint fit posterior mean parameters)
High
Low
Checking using ‘Sellke’ residuals
If Ri(t) denotes infectious challenge to i at time t,
where xi denotes infection time.
Impute latent ‘Sellke’ thresholds for each site and S’s p-value from K-S test, generating posterior distribution of p-values.
dttRix
ii 0
~ Exp(1)
E
(, x, ) S p()
()
Observation y
(p|y)
mean median LQ UQ Pr(p<5%) rep 0.013454 0.008933 0.003938 0.017715 0.9706 H1 0.000024 0.000006 0.000002 0.000020 1.0000 H2 0.000048 0.000015 0.000005 0.000046 1.0000 H3 0.002916 0.002039 0.001029 0.003757 1.0000 H4 0.136859 0.121301 0.070754 0.189353 0.1508 H5 0.075692 0.061481 0.033481 0.101138 0.4054 H6 0.014030 0.010434 0.005457 0.018893 0.9794 H7 0.000000 0.000000 0.000000 0.000000 1.0000 H8 0.000024 0.000009 0.000003 0.000026 1.0000 H9 0.552893 0.540598 0.388507 0.707525 0.0014 H10 0.000002 0.000000 0.000000 0.000002 0.9844 H11 0.000000 0.000000 0.000000 0.000000 1.0000 H12 0.086550 0.067288 0.035668 0.116220 0.3708 H13
mean median LQ UQ Pr(p<5%) rep 0.085966 0.063765 0.030051 0.118423 0.4036 L1 0.016515 0.011087 0.004239 0.022995 0.9412 L2 0.025164 0.018007 0.008519 0.034758 0.8724 L3 0.001911 0.001254 0.000602 0.002506 1.0000 L4 0.000126 0.000034 0.000009 0.000116 1.0000 L5 0.067819 0.052805 0.024010 0.097047 0.4804 L6 0.000045 0.000017 0.000005 0.000048 1.0000 L7 0.000004 0.000000 0.000000 0.000001 1.0000 L8 0.000030 0.000006 0.000001 0.000022 1.0000 L9 0.003341 0.001897 0.000737 0.004192 0.9998 L10 0.000001 0.000000 0.000000 0.000001 1.0000 L11 0.001243 0.000678 0.000257 0.001549 1.0000 L12 0.051377 0.027275 0.008475 0.069362 0.6560 L13
High inoculum - joint fit p-val. posterior summaries
Low inoculum - joint fit p-val. posterior summaries
Posteriors for p indicate lack of fit…….
Model comparison using imputed p-values
Streftaris &Gibson (PRSB, 2004) implicitly followed this approach
•Analysed data from 2 experiments of FMD in 2 populations of sheep.•2 groups of 32 sheep each subdivided into 4 sub-groups.•Group 1 exposed to FMD, subsequent epidemic observed.
Data: Censored estimates of infectious period for each sheep, and measures of peak viraemic load for each animal.
Question: Does viraemia decline as we go down the infection tree?
Experiment (Hughes et al., J. Gen. Virol. 2002):
32 sheep allocated to 4 groups G1, .., G4. Animals in G1 inoculated with FMD virus (4 at t=0, 4 at t=1 day). Thereafter animals mix according to the following scheme:
Day 1 2 3 4 .
G1 G2 G3 G4
Idea is to ‘force’ higher groups further down the chain of infection.Data: Daily tests on each animal, summarised by:y = (time of 1st +ve test, last +ve test, peak viraemic level)
Model: SEIR-Relationship between infectivity and viraemic load-Weibull distributions for sojourn in E and I classes-Peak viraemia independent of depth-Vague priors for model parameters ().
Depth 0
Depth 2
Depth 1
Depth 4
S conducts 1-way ANOVA on peak viraemic levels, to generate p-value.
E considers (p|y) to identify potential conflict.
Let x denote the infection network – must be imputed using MCMC
x
Group 1
Group 2
Arguably, a little too strongly stated.
From Streftaris & Gibson, PRSB (2004)
If we accept the modelling assumptions we must nevertheless concede that, with high posterior probability, an ANOVA test would provide significant evidence of differences in viraemia with depth in the infection chain.
A general infection process (Streftaris & Gibson, 2011, in preparation)
Assume ‘Sellke’ thresholds drawn from unit mean Weibull with shape parameter . (NB = 1 is exponential).
Is there evidence against the exponential model in favour of this new model for Experiments 1 & 2?
A: Full Bayesian – include as additional parameter and consider (|data)
B: Latent KS-test applied to imputed thresholds for exponential model
C: ‘Latent’ LRT (against Weibull alternative) applied to imputed thresholds for exponential model
Results for Weibull threshold model
A
C
B
EXPERIMENT 1
Imputation ‘reinforces’ the model
Observes:
Imputes:
y
x S
(x |y)
p(T(x, ))
(p |y)
E Asserts model and ()
0 1
Large probability of small p-value indicates conflict between E and S
Power of tests applied to x should be expected to diminish with amount of imputation.
Loss of ‘power’ as the ‘richness’ of x increases
Consider simple hypotheses regarding distribution of x.
E asserts x ~ 0(x) = (x | 0). S checks against alternative 1(x)
Observe y = f(x).
E imputes x ~ 0(x|y) and result of S’s test based on 0(x)/1(x).
Should use 0(x)/(1(y)0(x|y)) = 0(y)/1(y).
If we use z instead of x, where x = f(z), the corresponding mis-match between denominators increases (as measured by K-L divergence).
Comparing models - ‘Symmetric’ Approaches
1. Bayesian Model Choice
• Embed ‘competing’ models i = 1, … , k in an expanded model space equipped with prior for models (p1, …, pk) and parameters (i), i = 1, ..., k.
• Increased complexity makes implementation of MCMC harder.
• Model posterior probabilities sensitive to choice of prior (i).
Power-law decay, a (transformed)
CTV spread by melon aphid: Model: Rj = + i dij-
Gottwald et al., 1996,GJG 1997
1 year 1 year
Posterior contour plots: Melon aphid (3 epidemics) v Brown citrus aphid (3 epidemics)
MELON APHID(B + NN)
BC APHID(Local not NN)
Gottwald, et al (1999)
n1 infections n2 infections
Local parameter a
b
Analysis of such historical data could provide informative priors for comparison of MA and BCA ‘models’ fitted to a new data set.
MA prior
BCA prior
3rd model representingunspecified alternative (characterised by vague prior) may not be favoured in Bayesian model comparison.
Leads to comparisons based on separate fitting of models.
Backgroundinfection
2. Posterior Bayes Factors / DIC
•PBF (Aitken, 1991) compare models on basis of
•DIC (Spiegelhalter et al, 2002) uses D() = -2 log (y|). Formally
where is measure of complexity and expectations are taken over (|y). DIC is then computed across the models to be compared.
22222
11111
||
||
dyy
dyy Ratio of posterior expectations of the likelihood
DpDDIC 2)~
(
~)( DDpD
DIC for epidemic modelling?
We may need to consider augmented parameter vector ′ = (, x) where x are unobserved components so that (y, x|) is tractable.
•No unique choice of x!
•Dimension of imputed x may approach (or exceed) dimension of data set y.
See Celeux at al, 2006, DIC for missing data models for extensive range of alternative ways to define DIC
•Bayesian relevance of comparing DICs across models?
~
DIC
Philosophical difficulties with DIC/PBF?
E1
1S
Observation y
(|y)
E2
2
()
(|y)
2 or more Egos required!“Batesian” rather than Bayesian?
E1
S1
E2
S2
DIC1(y) DIC2(y)
DIC1(y) DIC2(y)
Interpretation 1 Interpretation 2
()
2 or more statisticians: DIC interpreted by some external arbiter.
DIC
Philosophical difficulties with DIC/PBF?
E1
1S
Observation y
(|y)
E2
2
()
(|y)
2 or more Egos required!“Batesian” rather than Bayesian?
E1
S1
E2
S2
DIC1(y) DIC2(y)
DIC1(y) DIC2(y)
Interpretation 1 Interpretation 2
()
2 or more statisticians: DIC interpreted by some external arbiter.
12
(p|y) (p|y)
Summing up
•Many ‘tensions’ in Bayesian methods come to the fore in the context of dynamical epidemic models
•Hybrid approaches may offer a way of addressing these tensions by applying Bayesian methods to low-complexity models checked in a classical approach
•Perhaps we need to underplay the importance of models as predictive tools as opposed to interpretive tools.
•Qualitative conclusions that are robust to model choice may be seen as extremely valuable
Model: SEI with ‘quenching’, primary and secondary infection constant latent period.
3 ‘submodels’:
1. Latent period = 0 (SI)2. SEI with observations recording I3. SEI with observations recording E+I
No trichoderma
Trichoderma
Final example: R solani in radish re-visited
1. SI model 2. SEI model, I observed
PrimaryPrimary
Secondary
Secondary
Latent
Quenching
Quenching
Although quantitative estimatesof parameters changes with model the qualitative conclusion seems robust.
There is consistent evidence that T viride appears to affect the primary infection parameter.
Models are useful ‘lenses’ even if they cannot be used as ‘crystal balls’!
3. SEI model, E+I observed
Primary Secondary
Quenching Latent