predictive models for health and social care: a feasability study
DESCRIPTION
TRANSCRIPT
© Nuffield Trust
Predictive Models for Health and Social Care: A Feasibility Study
Authors: Bardsley M, Billings J, Dixon J, Georghiou T, Lewis GH, Steventon A (2011) ‘Predicting who will use intensive social care: case finding tools based on linked health and social care data’, Age and Ageing 40(2): 265-270
February 2011
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Background
Key points
Evidence that admission to a nursing home can be delayed or avoided by means of preventative ‘upstream’ interventionsIncreasingly, public services will have to become more proactive in identifying and managing risk in older people, in order to mitigate them as much as possibleIt will be important to identify and support not just those at highest risk of these costs, but also those currently at lower risk who might become higher risk in futureCurrent methods of assessing risk largely rely on face to face assessments. In health care this approach has been shown to be less accurate at predicting predicting future events (hospitalisation) compared to statistical models.
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Predictive modelling
• BMJ in paper* in 2002 showed Kaiser Permanente in California seemed to provide higher-quality healthcare than the NHS at a lower cost. Kaiser identify high risk people in their population and manage them intensively to avoid admissions
• Modelling aims to identify people at risk of high costs in future• Use pseudonymous, person-level data• In health sector a number of predictive models are available
e.g. PARR++ and the combined model.*Getting more for their dollar: a comparison of the NHS with California's Kaiser Permanente BMJ 2002;324:135-143
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Predictive Models Identify who will be where on next year’s Kaiser Pyramid
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Patterns in routine data to identify high-risk people next year
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Patterns in routine data to identify high-risk people
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Patterns in routine data to identify high-risk people
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Distribution of Future Utilisation is Exponential
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Can we predict costly social care events in the same way?
• Do the data exist in local systems?• Can we extract individual, person-level data?• Can we link a wide range of health and social care data at the
person level?• Are the data accurate and complete enough to use?• Can we build a valid statistical model?
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Feasibility Project
• Funded by Department of Health Care Services Efficiency Delivery (CSED) programme over 18 months
• Worked with 5 sites to extract data sets• Extracted person-level health & social care data sets • Linked data from GP records (2 sites); GP register (all sites);
Hospital (all sites); Social Care (all sites)• Undertook exploratory analyses and ran a range of models• Tested the impacts of different data sets on the models
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Using health and social care data to predict health and social care use
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Using routine data
• Less labour intensive so they can stratify the population systematically and repeatedly
• Avoid “non-response bias”• Can identify people with lower, emerging, risk• Important issues of confidentiality and consent to consider • Linking data sources at individual level across health and
social care is problematic where there is no NHS number in social care
• The tools are never 100% accurate• Data may be missing from routine databases on certain
groups
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Predictive factors – examples
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Information flows
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Data collected
• From four ‘sites’ (~ PCT areas)• Total seven organisations: 3 PCTs, 1 Care trust, 3 LAs • Total 1.4M pop (range 100,000-700,000)
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Data linkage - approach
First instance: NHS number (encrypted) from LA
In absence of NHS number:• Central ‘batch tracing’?• Shared PCT/LA databases?
Ultimately: •construction of ‘alternative IDs’
97% of individuals in one site (population ~400,000) were found to have unique ‘alternative ID’.
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Data linkage
Groups of people in social care data – how many are we able to match to GP register list (of ages 75+)?
Varies, but better for those with > service use
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Data linkage Social & secondary care interface
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Predictive value, sensitivity and specificity of the model incorporating a £5,000 threshold
Predicting social care costs: feasibility study
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Information on Social Care Needs
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Simplifying and sorting
Accommodation Adaptation Adult Placement Advice Given Carer Support Counselling DA1 Day Care Direct Payments Domiciliary Employment/Training Equipment Fostering Group Activity Holiday Payment Housing Advice Given ILS Independent Living Individual Informal Projects Information & Advice Lifeline 400 Lifeline 400 + Smoke Alarm Mobility Training (DAY) Mobility Training (NIGHT) Nursing Nursing (Block Contract) Nursing Block Beds Old Codes OT Advice Given OT Rehab Sessions Other Other Carer Support Services Outreach Permanent Placements with Parents Professional Support R.O. Allowance Referral to Carers Contact Line Referral to Carers Information Service Referral to Crossroads Registration Rehabilitation Residential Residential Block Beds Respite Respite (Carer) Secure Accommodation Sheltered Social Development Special Sheltered Specialist Assessment and/or Treatment Specialist Communication Equipment Specific Approval Support Supported Living Talking Book Machine Time Allocation Vouchers Training Transport Washer
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Individual histories
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Transitions in care (75+ in one site)
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Individual health and social care event timeline over a three-year period
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Predictive Modelling
Attempting to predict: • For over 75s
a) Admission to care home (or receipt of high intensity home care) or £5,000 increase in social care costs in one year
b) Where person had no ‘significant’ costs in prior two years
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Original models predicting change at £5k threshold
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Diagram
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Distillation flask
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Trade off between PPV (blue line) and sensitivity (red line) according to different risk cut-offs
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Which variables are important in pooled £1k model?
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Models using lower £1k thresholds
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Impact of adding new datasets
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Iteration with Sites - Application
• Several sites said they would like to run the Combined Model and the social care model “side-by-side”
• Theographs thought to be very useful• Differences between sites: some sites preferred small numbers of
clients others wanted large numbers for mail-shots• Regarded concerns over invasion of privacy as “a non-issue if the
wording is right”• Keen to use predictive models for existing or planned re-ablement
services and multidisciplinary teams
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Personal observations on social care data
We could access user level social care data but...• Different systems • Issues of standardisation /coding etc and the absence of standard,
structured coding schemes• Some very detailed information collected but can be difficult to use
in models• We found it harder to obtain information on descriptions of needs • Local concerns about data quality• Note the history of comparative benchmarking
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study © Nuffield Trust
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
Headlines
• We have created linked health/social care data sets in five sites (overall populations about 2 million people aged 75+)
• We modelled intensive social care:• Models had satisfactory PPV but low sensitivity at a risk score
threshold of 50 (although can be traded-off against each other)• Mainly social care driven• Though accuracy worse than PARR model, accuracy was
comparable with SPARRA and the combined model• Discussion with sites – they were less concerned with model
performance than we were• We tried dozens of ways to improve the basic model (5K)• We modelled change in social care costs with lower thresholds:
1K models are much better but usefulness may be diminished
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study
And the data have so much more to offer.....
• Cost modelling • Iso-resource classification (individual budgets)• Trajectories and transitions of care• Evaluation (re-ablement)• Information for
- Commissioners and managers- Professionals- Service users and carers
© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility StudyFebruary 2011
www.nuffieldtrust.org.uk
Sign-up for our newsletterwww.nuffieldtrust.org.uk/newsletter
Follow us on Twitter(http://twitter.com/NuffieldTrust)
© Nuffield Trust