matthew lamb mrl2013@columbia icap-m&e ny

Matthew Lamb [email protected]

ICAP-M&E NY

Using routinely-collected data to estimate patient retention in care and loss to follow-up

ICAP Methodology WebinarJanuary 19, 2012

Upcoming methodology webinars

February 9

Overview of ICAP Geographic Information System Resources

Charon Gwynn, Yingfeng Wu, and Mark Becker

Future methodology webinar ideas?

email the methodology webinar coordinator, Bill Reidy: [email protected]

mailto:[email protected]

Matthew Lamb [email protected]

ICAP-M&E NY

Using routinely-collected data to estimate patient retention in care and loss to follow-up

ICAP Methodology WebinarJanuary 19, 2012

Outline

• Defining retention• Why is retention a useful outcome, compared to

mortality?• How does loss to follow-up impact our measures of

patient survival?• How can we use retention to assess patient and

programmatic outcomes?• How can we measure retention using routinely-collected

data?• Patient-level• ART cohort• Aggregate

Enrollment into HIV care and treatment clinic

Already on ARTART ineligible at

enrollment (window)

Unknown eligibility at enrollment

(window)

ART eligible at enrollment

Initiates ART

Becomes ART eligible

Follow-up after ART initiationFollow-up after ART initiation

LTF

Death

LTFDeath

LTFDeath

LTFDeath

LTFDeath

Transfer

Transfer

Transfer

Transfer

Transfer

Transfer

Working definitions

• Retained• Known to be alive and engaged in care

• Retained on ART• Known to be alive, engaged in care, and on ART

• Retained in pre-ART care• Known to be alive and engaged in care, but not yet on ART

• Known dead• Death known to clinic and documented

• Transferred out• Patient transfer to another clinic known and documented

• Lost to follow-up• Patient not known to be dead or transferred, treatment

status and whereabouts unknown

Known dead

Lost to follow-up

+

Non-retained =

Outline






Non-retention combines “bad” outcomes of death and loss to follow-up

LTF Death

Treatment interruption/stoppageLack of monitoringLack of services

Death LTF

Unmeasured death Non-retained

Loss to follow-up results in underestimates of patient mortality

Measured death

Outline






LTF influences our measures of survival

(example)

• Suppose you live in a universe where HIV clinics have perfect documentation, and all patients who enroll into HIV care attend every one of their scheduled visits and take all of their medication.

Measuring risk of death in a cohort with no LTF

Time (months) sinceART initiation

Incidence proportion of death:

4/20(20%)

Retention proportion:16/20(80%)

1 2 3 4 5 6 7 8 9 10 11 12


4/20(20%)



3/20(15%)


1 2 3 4 5 6 7 8 9 10 11 12


LTF underestimating the risk of death

Introducing incidence rates

• To account for unequal follow-up time in the presence of loss to follow-up, we measure incidence using incidence rates instead of incidence proportions

Incidence proportion Incidence rate

number of eventspopulation at risk

number of eventsperson−time at risk

Estimating mortality rates in the presence of LTF


4/20(20%)



3/20(15%)


1 2 3 4 5 6 7 8 9 10 11 12

125.51212

7.7512121212121212129

1212129

1212

223.25 person-months

Incidence rateof death

4/223.25 pm21.5 per 100 py

Non-retention rate is the same here

122.251212

7.7512121212129

1249

128

1296

12

199 person-months

Incidence rateof death

3/199 pm18.1 per 100 py

Non-retention rate

8/199 pm48.2 per 100 py


Months of observation for each patient

Total person-time

Incidence rates give us closer estimates to “the truth” in populations with loss to follow-up

No LTF LTF % Difference

Incidence Proportion

20% 15% 20%

Incidence Rate

21.5 per 100 py

18.1 per 100 py

16%

Using incidence proportions when follow-up time is unequal biases our interpretation of outcome occurrence

3m 6m 9m 12mIncidence proportion: 4/(4+10) = 4/ 14 = 29%Incidence rate:4/((4*1m) + (10*12m)) =

4/124 person-months = 39 per 100 person-years

3m 6m 9m 12mIncidence proportion: 4/(4+10) = 4/ 14 = 29%Incidence rate:4/((4*11m) + (10*12m)) =

4/164 person-months= 29 per 100 person-years

Risk Ratio = 29%/29% = 1

Rate Ratio = 39/29 = 1.3

Outline






Comparing retention between clinics allows us to assess factors that may improve retention

Age group Proportion retained 1 year after ART initiation

5-10y 12%

11-14y 15%

15-24y 30%

25-54y 19%

55 and over 17%

*Among patients initiating ART April 2008-March 2010. Loss to follow-up is defined as patients not known to have died or transferred without a visit in the last 6 months of data collection (ART). Patients LTF are censored 15 days after their last visit (ART).

Comparing retention between clinics allows us to assess factors that may improve retention

Source: Lambdin et al. JAIDS. Volume 57(3), 1 July 2011, pp e33-e39

http://www.ncbi.nlm.nih.gov.ezproxy.cul.columbia.edu/pubmed/18670668

Outline






Transforming patient-level data into follow-up cohorts

Working example: Non-retention one year after ART initiation• Select study population and time period for enrollment

and follow-up of patients• Define loss to follow-up

• No visit in the last 6 months of data collection• Therefore need to extend follow-up time for 6

months after enrollment period• Define “zero time”• Calculate person-time for each patient• Calculate the incidence rate for the outcome of interest

Q1 Q2 Q3 Q4 FU1 FU2

13 Non-retained within 1 year

153.5 person-months

Non-retention rate = 13/153.5 person-monthsNon-retention rate = 102 per 100 person-years

Calculating retention from patient-level data

• Study population: patients initiating ART between Q1 and Q4• Exclude patients initiating prior to Q1

• Outcome = non-retention (LTF or death)• Make sure all patients have sufficient opportunity to meet definition of LTF

• Extend follow-up period to 6 months after Q4 • Start following patients from their ART start date until they become non-retained or the study period ends• Calculate person-time• Calculate non-retention rates

7.54

4.57.5127.512127.5125

7.57.57.57.52.597

6.57

Limit follow-up to1 year

Slight diversion: what to do about transfers

Transfers prevent us from knowing the true retention status of the patient after they transfer.

We say that patients who transfer are “censored,” meaning that we do not have complete information on their retention status, had they remained at their initiating clinic

We allow transfers to contribute person-time to the denominator until their transfer date

Summary: calculating non-retention from patient-level data

• Clearly define• Study population• Study period of enrollment• Follow-up time• Loss to follow-up• Outcome of interest• Zero time

• Calculate each person’s follow-up time from zero time until reaching outcome of interest, censoring, or end of study

• Calculate incidence rate

Patient-level data is awesome, but…

• Not everyone has it• It requires some “work” to analyze

Thankfully, there is other information that we routinely collect

Outline






12 month aggregate ART cohort data (URS)

Quarter of ART initiation:May-July 2010

Number initiating ART inMay-July 2010:

269

Number on ART July-Sep 2011

214

Proportion retained = 214/269 = 80%

Q1 Q2 Q3 Q4 FU1 FU2

12 month aggregate ART cohort data

28

Count patients initiating within the same quarter

5

Count how many of theseare on ART 11-16 months later

1

Proportion retained = 1/520%

Weighted average retention measures from aggregate cohort data

ART initiation quarter

Number initiating ART during quarter

Number alive and on ART 12 months later

Retention proportion

Jan-Mar 5 1 20%

Apr-Jun 100 96 96%

Jul-Sep 150 146 97%

Oct-Dec 200 196 98%

Total 455 439 96%

Weighted-average 12-month retention incidence proportion

(5∗0.2 )+ (100∗0.96 )+(150∗0.97 )+(200∗0.98)5+100+150+200

¿96%

12 month aggregate ART cohort data: caveats

• Transfers should be excluded from numerator and denominator

• Does not separate non-retention into LTF, death• Can not directly calculate incidence rates• Only collected at 12 months• Can combine several cohort measures of

retention to obtain a clinic-specific average 12-month retention

Outline






HIV care and treatment clinics routinely report the following

• Cumulative number of patients on ART at the end of the previous quarter• (which equals the cumulative number of patients on

ART at the beginning of the current quarter)

• Number newly initiating ART during the quarter• Cumulative number of Deaths, Transfer, LTF,

ART discontinuation through the end of the quarter

• From these, we can calculate overall retention estimates

Q1 Q2 Q3 Q4 FU1 FU2

Aggregate retention

2

5

7

5 5 5

9 13 14

3 1 4

Q1 Q2 Q3 Q4

Aggregate retention

2

5

7

5 5 5

9 13 14

3 1 4

Assume 3 months follow-up timefor each individual on ART at beginning of the quarter

Assume 1.5 monthsfollow-up time for each individual initiating during the quarter

Subtract 1.5 months follow-up time for each individual exiting during the quarter

2*3 + 5*1.5

=13.5 pm

7*3 + 5*1.5 –3*1.5

=24 pm

9*3 + 5*1.5 –1*1.5

=33 pm

13*3 + 5*1.5 –4*1.5

=40.5 pm

Q1 Q2 Q3 Q4

Aggregate retention

2

5

7

5 5 5

9 13 14

3 1 4

Overall one-year non-retention rate:

7/(13.5+24+33+40.5)

7 per 111 person-months

76 per 100 person-years

2*3 + 5*1.5

=13.5 pm

7*3 + 5*1.5 –3*1.5

=24 pm

9*3 + 5*1.5 –1*1.5

=33 pm

13*3 + 5*1.5 –4*1.5

=40.5 pm

Aggregate retention: important caveats

• Aggregate retention does not estimate a patient’s risk of being non-retained in one year• It is a clinic-level estimate of the flow of

patients into and out of the clinic• We can use the same procedure to calculate

aggregate LTF and mortality rates• Comparing aggregate retention rates between

clinics is very useful

Summary

• Loss to follow-up results in underestimates of survival• In situations with high loss to follow-up, non-retention

provides us with a more conservative estimate of undesirable patient outcomes

• Incidence rates are the preferred measure of non-retention incidence, but are not always available

• Routinely-collected Patient-level, ART cohort, and cumulative aggregate data can be used to estimate non-retention incidence

• Comparing non-retention between populations and clinics can help us identify areas that affect patient outcomes…

Using non-retention: CROI 2012 Abstracts

• Lamb et al. Factors Associated With High Loss To Follow-Up Among sub-Saharan African Youth 15-24 Years of Age Enrolled in HIV Care

• Elul et al. Six- and 12-month Non-retention Over Time among 5,690 Cohorts with 316,762 Patients Initiating Antiretroviral Therapy (ART) in 9 Countries in Sub-Saharan Africa

• Mcnairy et al. Retention of HIV-infected Children on ART in ICAP-supported HIV Care and Treatment Programs

matthew lamb mrl2013@columbia icap-m&e ny

Documents

patient retention

icap data

measuring retention

matt lamb of icap

todays webinar

webinar series

data analyst

icaps monitoring evaluation