Trutz Haase

Jonathan Pratschke

Feline Engling

EMCDDA

Methodological Toolkit for the Estimation of the Number of People in Drug Treatment

Presentation to National Focal Points, EMCDDA, 24th -26th June 2013

Trutz Haase

Jonathan Pratschke

Feline Engling

BACKGROUND TO THE STUDY

Phase 1 of the Study:

� Development of a Generic Mapping System

� Workshop in January 2012 – 8 Focal Points participating

� Final Report in July 2012

Phase 2 of the Study:

� Development of a Methodological Toolkit

� to assist national Focal Points in dealing with double counting

� to improve national estimates along the lines of the Generic Mapping System

A SAMPLE MAP OF A COUNTRY’S TREATMENT SYSTEM

Outpatient Network

Specialised Treatment Centres

Low Threshold Agencies

PrisonsOpioid

Substitution Treatment

General Practitioners

Day Care Centres

Country

Inpatient Network

Hospital: Detoxification/

Emergency

Hospital: Rehabilitation/

Psychiatric

Prisons

Residential Communities

Therapeutic Communities

Total Drug Treatment

A GENERIC MAPOF SERVICE PROVISION AND TREATMENT DATA

Sub-total: Outpatient Network

Specialised Drug Treatment Centres

Low-threshold Agencies

Treatment Units in Prisons

Opioid Substitution Treatment

Other Outpatient Treatment

General Practitioners

General Health Care Centres

Country

Outpatient Network

Sub-total: Inpatient Network

Medical Detoxification Treatment

Hospital-based Residential Drug Treatment Units

Treatment Units in Prisons

Other Inpatient Treatment (1)

Other Inpatient Treatment (2)

Therapeutic Communities

Residential Drug Treatment Centres

Total Drug Treatment

Inpatient Network

A SAMPLE MAPOF SERVICES PROVISION AND TREATMENT DATA

WHICH INFORMATION? THE BASIC “BOX”

repNR Unit Count M rep NR Patient Count M

calcTDI Unit Count L calc TDI Patient Count LL

Units Patients

Comment Comment

Generic Category Country-specific Category

Source Indicator

calc calculated

rep reported

est Estimated

nnn Source

?? Check

Evaluation Flag

HH Very high

H High

M Medium

L Low

LL Very low

ADVANTAGES OF GENERIC MAPPING SYSTEM

� Same map/format for each country

� You understand your own map/data, you understand everyone else’s

� Parallel use of generic and country-specific terms of classification

� Clear distinction between where information is missing (ni) and valid zeros (0)

� Thus drawing attention to missing information (with positive knock-on effects)

� Data can be aggregated to country level sub-totals and totals

� Data can be aggregated across groups of countries and EMCDDA-wide

� All data can be instantly shown on a per 1,000 capita basis

� Cross-country per capita calculations can be used for evaluation of counts

� Data can be easily updated

� Standardised reports can be easily produced

TASKS FOR PHASE 2 OF THE PROJECT

� The challenge ahead is to derive appropriate algorithms for estimating missing

data and to optimise country-specific estimates.

� Particular attention needs to be paid to the sensitivity of the Generic Mapping

System to double counting.

� Hence the Focus of Phase 2 is on developing a “Methodological Toolkit for the

Estimation of the Number of People in Drug Treatment”

OBSERVATIONS FROM THE 2011 NATIONAL REPORTSPDU PREVALENCE ESTIMATES (1)

� Definition of PDU

� Most countries have adopted EMCDDA definition, though some continue to use alternative (historically-motivated) definitions.

� Estimation Basis: Large variations in data used, including:

a) police reports related to opioids and/or other illicit drug use

b) drug-related deaths

c) substitution registry and/or other OST related-data

d) drug treatment data from inpatient, out-patient and other drug treatment facilities

OBSERVATIONS FROM THE 2011 NATIONAL REPORTSPDU PREVALENCE ESTIMATES (2)

� Multiplier Construction

� General population surveys unsuitable, hence employment of either capture re-capture (CRC) or respondent-driven sampling (RDS) methods

� Accuracy depends on both the accuracy of the base indicator and the nature of the survey used to calculate weights/multipliers

� Surveys tend to be infrequent and limited in size (both institutionally and geographically)

� Resulting PDU estimates tend to have large confidence intervals

� There is limited scope for “borrowing strength” across countries, as each multiplier is specific to the dataset used

OBSERVATIONS FROM THE 2011 NATIONAL REPORTSPDU PREVALENCE ESTIMATES (3)

� Implications for EMCDDA

� Differences in the methodologies adopted to estimate PDU are a problem when making comparisons between countries, or when estimating PDU across the EMCDDA 30 countries

� They are less problematic when monitoring changes over time within individual countries, as long as the methodology remains the same

� There exists a potential trade-off between adopting a common methodology across the EMCDDA 30, and the discontinuity that such a change may entail for a particular country

� There is limited scope to “borrow strength” across multiple countries, as each multiplier is highly specific to the base dataset being used

OBSERVATIONS FROM THE 2011 NATIONAL REPORTSTREATMENT DATA (1)

� Centralisation

� Does not necessarily entail existence of a single integrated system of individual client records

� Most countries operate a single national reporting system in which pre-aggregated data are drawn together from different sub-systems

� The organisational structures of such sub-systems tend to reflect either organisational/institutional distinctions, or geographical region

� In the most basic approach, double-counting is eliminated at the level of individual treatment facilities

� More advanced systems eliminate double counting at the level of sub-systems

� Few countries have developed a process of client identification applied across the entire national system

OBSERVATIONS FROM THE 2011 NATIONAL REPORTSTREATMENT DATA (2)

� Integration

� To date, most national reporting systems follow the reporting structure of TDI, but exclude GPs and low-threshold agencies from core datasets

� Not all treatment centres are included in counts and data collection may focus on larger treatment units only

� This can be corrected for by extrapolating to all facilities, taking account of their relative size

� This process of extrapolation might be improved upon using the facility surveys that are planned in EMCDDA countries

OBSERVATIONS FROM THE 2011 NATIONAL REPORTSTREATMENT DATA (3)

� Overcoming Double-Counting

� The most effective way to eliminate double-counting is to use client IDs

� Client IDs are widely applied within individual treatment facilities, and double-counting as a result of multiple treatment episodes within a given centre is largely eliminated from reported TDI data

� The introduction of client IDs across multiple facilities – generally using pseudonyms – remain the exception rather than rule

� Where studies exist on the overlap between sub-systems, these tend to relate to the broad TDI categories

� Overlaps between facility types as specified in the Generic Mapping System have not yet been reported

TOWARDS A METHODOLOGICAL TOOLKIT (1)

� Double Counting as a Result of Multiple Service Use

TOWARDS A METHODOLOGICAL TOOLKIT (2)

� Overcoming Double Counting

� The most accurate and flexible way of tackling the methodological challenge of multiple service use involves the use of personal identifiers

� The anonymity of service users has to be protected by using special identification codes that can only be linked with individuals by a “trusted third party”

� The additional information provided by personal identifiers can, at least in theory, be used to identify overlaps between services and to count the number of interventions received by an individual

� Whilst technically possible, “pseudonomisation” is IT intensive, technically demanding and can generate additional difficulties in terms of data access and analysis, particularly if these are not planned from the outset

TOWARDS A METHODOLOGICAL TOOLKIT (3)

� Implementing the Use of Client IDs

� Decision at the aggregate level (EMCDDA countries)

� Decision at national level

� Achieving agreement/participation of treatment providers

� Tendering and commissioning of “Trusted Third Party” (TTP)

� Development and distribution of client software

� Implementation at level of treatment facility

� Collation of data and data aggregation

� Statistical analysis of data records at national level

� Reporting back to EMCDDA (using TDI/Generic Mapping System)

� EMCDDA 30 overall comparative analysis

TOWARDS A METHODOLOGICAL TOOLKIT (4)

� The Alternative: A Survey of Service Users

� A census of treatment providers could be used as sampling frame

� A stratified sample of treatment providers would be extracted

� A sample of service users would then be carried out

� Survey instruments would inquire about use of services and treatments received over previous year

� Results could be generalised to the total population of people in treatment

� The survey would provide reliable estimates of overlaps – multiple episodes and multiple services use

TOWARDS A METHODOLOGICAL TOOLKIT (4)

� A Synthesis: Mixed Methods Approach

� The calculation of overlaps requires either system-wide client IDs or a survey of service users

� Either approach would be adequate and it is not necessary for all EMCDDA countries to apply the same method

� Results can be used even if a minority of countries provide data on overlaps between facility types specified in the Generic Mapping System

� Results from individual countries may be extrapolated to “similar” countries or adapted using additional assumptions or models

� This “agenda” has the potential to yield considerable improvements in EMCDDA-wide estimates of the number of people in drug treatment

� Other benefits include policy-relevant insights into similarities/differences across countries and regions and over time

www. t r u t z h a a s e . e u