Bias in estimates of HIV Bias in estimates of HIV incidence based on the detuned incidence based on the detuned assay: assay: A proposed solutionA proposed solution

Robert S Remis, Robert WH Palmer, Robert S Remis, Robert WH Palmer,

Janet M RaboudJanet M Raboud

Department of Public Health Sciences, University of TorontoDepartment of Public Health Sciences, University of Toronto

Mount Sinai Hospital, Toronto, OntarioMount Sinai Hospital, Toronto, Ontario

STARHS satellite meetingSTARHS satellite meeting

Bangkok, Thailand, July 11, 2004Bangkok, Thailand, July 11, 2004

MOHLTC, Laboratories Branch, IMC – 2001

BackgroundBackground

• STARHS assay of HIV-positive specimens STARHS assay of HIV-positive specimens identifies recent infections identifies recent infections

• Used to calculate HIV incidence density, a Used to calculate HIV incidence density, a critical indicator usually difficult to critical indicator usually difficult to measuremeasure

• Numerator is discordant specimens; Numerator is discordant specimens; denominator is person-time from window denominator is person-time from window periodperiod

• ButBut analysis using diagnostic specimens analysis using diagnostic specimens may be subject to strong testing biasmay be subject to strong testing bias

MOHLTC, Laboratories Branch, IMC – 2001

Problem of biasProblem of bias

• In 2002, assessed sources, direction and strength of In 2002, assessed sources, direction and strength of bias with diagnostic specimens (bias with diagnostic specimens (Remis et al, XIV ICARemis et al, XIV ICA))

• For MSM, bias up to 7.3 fold, with plausible For MSM, bias up to 7.3 fold, with plausible parameter values yielding bias of 2-3 foldparameter values yielding bias of 2-3 fold

• Principal source of bias Principal source of bias “seroconversion effect”“seroconversion effect” i.e. i.e. increased likelihood of HIV testing following infection increased likelihood of HIV testing following infection due to seroconversion illness or to high risk due to seroconversion illness or to high risk exposureexposure

• Quantified as proportion of subjects who test within Quantified as proportion of subjects who test within 90 days after HIV infection (Psce)90 days after HIV infection (Psce)

MOHLTC, Laboratories Branch, IMC – 2001

Proposed solution #1Proposed solution #1

• Incidence density calculated using Incidence density calculated using STARHS assay with diagnostic specimens STARHS assay with diagnostic specimens must be interpreted with great cautionmust be interpreted with great caution

• Need to adjust calculated HIV incidence Need to adjust calculated HIV incidence taking into account bias due to Ptaking into account bias due to Pscesce

• Originally proposed studies to measure Originally proposed studies to measure knowledge of seroconversion illness and knowledge of seroconversion illness and assess likelihood of immediate HIV assess likelihood of immediate HIV testing under various scenariostesting under various scenarios

MOHLTC, Laboratories Branch, IMC – 2001

Not so fastNot so fast

• Studies take time and cost moneyStudies take time and cost money• Population studied may not be Population studied may not be

representative (may need many surveys to representative (may need many surveys to include different populations)include different populations)

• Questions about likely HIV testing Questions about likely HIV testing hypothetical; answers may not be validhypothetical; answers may not be valid

• No help with historical specimensNo help with historical specimens

MOHLTC, Laboratories Branch, IMC – 2001

Eureka!Eureka!

• HIV incidence calculated from STARHS HIV incidence calculated from STARHS assay at different window periods assay at different window periods provides empirical evidence of Psceprovides empirical evidence of Psce

• Slope of HIV incidence at different Slope of HIV incidence at different window periods is direct and quantitative window periods is direct and quantitative indicator of strength of Psceindicator of strength of Psce

MOHLTC, Laboratories Branch, IMC – 2001

Incidence calculated using different Incidence calculated using different window window periods with Vironostika assay, 2001periods with Vironostika assay, 2001

0.0

0.5

1.0

1.5

2.0

2.5

133 170 336

Window period (days)

Inci

dence

(per

100 p

ers

on-y

ears

) MSM

MSM-IDU

IDU

HR hetero

LR hetero

MOHLTC, Laboratories Branch, IMC – 2001

Determination of Psce and true Determination of Psce and true incidence using empirical dataincidence using empirical data

• Algebraic formula developed in 2002 Algebraic formula developed in 2002 expressed measured incidence density expressed measured incidence density as a function of true incidence density, as a function of true incidence density, PPscesce and HIV testing parameters and HIV testing parameters

MOHLTC, Laboratories Branch, IMC – 2001

Measured incidence as Measured incidence as function of Pfunction of Pscesce and true and true incidenceincidence

MOHLTC, Laboratories Branch, IMC – 2001

Measured incidence as Measured incidence as function of Pfunction of Pscesce and true and true incidenceincidence

Where:Where:

I’I’estest = measured incidence density = measured incidence density

N = study populationN = study population

TTobsobs = study period = study period

TTwinwin = detuned window period = detuned window period

IItruetrue = true incidence densit = true incidence densit

TTtesttest = mean inter-test interval = mean inter-test interval

PPscesce = proportion seroconverting <90 days after infection = proportion seroconverting <90 days after infection

MOHLTC, Laboratories Branch, IMC – 2001

Determination of Psce and true Determination of Psce and true incidence using empirical dataincidence using empirical data

• True incidence density is unknownTrue incidence density is unknown• Can determine value of Psce and true Can determine value of Psce and true

incidence density by varying values through incidence density by varying values through range to fit to measured incidence densityrange to fit to measured incidence density

• Repeated at discrete values of window Repeated at discrete values of window period and modeled incidence is fit to period and modeled incidence is fit to observed incidence by selecting values of observed incidence by selecting values of true incidence density and Psce that true incidence density and Psce that minimize the differenceminimize the difference

• Minimize sum of squares of difference Minimize sum of squares of difference (goodness-of-fit)(goodness-of-fit)

MOHLTC, Laboratories Branch, IMC – 2001

Determination of Psce and true Determination of Psce and true incidence using empirical dataincidence using empirical data

• Programmed software in APLProgrammed software in APL• Vary Psce from 0% to 50% in increments of Vary Psce from 0% to 50% in increments of

0.1%0.1%• Vary true HIV incidence from 0 to 20 per 100 Vary true HIV incidence from 0 to 20 per 100

person-years in increments of 0.01person-years in increments of 0.01• Program selects values of Psce and incidence Program selects values of Psce and incidence

for which sum of squares of difference for which sum of squares of difference between observed and modeled incidence is between observed and modeled incidence is lowestlowest

MOHLTC, Laboratories Branch, IMC – 2001

Psce, measured and true HIV incidence by Psce, measured and true HIV incidence by year and health region among MSM, 2001-year and health region among MSM, 2001-03 03

1.18

1.56

1.05

0.98

1.80

1.41

0.60

0.56

0.59

1.891.89

2.012.01

1.801.80

2.462.46

1.871.87

2.062.06

0.780.78

0.560.56

0.580.58

9.1%9.1%

4.6%4.6%

10.5%10.5%

21.1%21.1%

0.4%0.4%

6.1%6.1%

3.9%3.9%

0.0%0.0%

14.7%14.7%

TORONTOTORONTO

20012001

20022002

20032003

OTTAWAOTTAWA

20012001

20022002

20032003

OTHEROTHER

20012001

20022002

20032003

TrueTrue

incidenceincidence

MeasuredMeasured

incidenceincidencePscePsce

MOHLTC, Laboratories Branch, IMC – 2001

Crude and adjusted HIV incidence Crude and adjusted HIV incidence among MSM and IDU, Toronto, 1999-among MSM and IDU, Toronto, 1999-20032003

0.00

0.50

1.00

1.50

2.00

2.50

3.00

1999 2000 2001 2002 2003Period

HIV

in

cid

en

ce (

per

100 p

y)

MSMmeasMSMtrueIDUmeasIDUtrue

MOHLTC, Laboratories Branch, IMC – 2001

Crude and adjusted HIV incidence Crude and adjusted HIV incidence among MSM and IDU, Ottawa, 1999-among MSM and IDU, Ottawa, 1999-20032003

0.00

0.50

1.00

1.50

2.00

2.50

3.00

1999 2000 2001 2002 2003

Period

HIV

inci

dence

(p

er

100 p

y)

MSMmeasMSMtrueIDUmeasIDUtrue

MOHLTC, Laboratories Branch, IMC – 2001

Summary of findingsSummary of findingsAdjustment of HIV incidenceAdjustment of HIV incidence

• Goodness-of-fit approach allowed Goodness-of-fit approach allowed adjustment to remove testing biasadjustment to remove testing bias

• Modelled HIV incidence fit very well to Modelled HIV incidence fit very well to observed HIV incidenceobserved HIV incidence

• Data using specimens from diagnostic HIV Data using specimens from diagnostic HIV testing should be presented with both crude testing should be presented with both crude and adjusted values of HIV incidenceand adjusted values of HIV incidence

MOHLTC, Laboratories Branch, IMC – 2001

AcknowledgementsAcknowledgements

• Ontario Laboratory Enhancement Study Ontario Laboratory Enhancement Study fundingfunding• Ontario HIV Treatment NetworkOntario HIV Treatment Network• Centre for Infectious Disease Prevention Centre for Infectious Disease Prevention

and Control, Health Canada and Control, Health Canada • Neil Hershfield developed custom software Neil Hershfield developed custom software

to adjust HIV incidenceto adjust HIV incidence