a guide to economic appraisal and evaluation
TRANSCRIPT
1
A guide to economic appraisal and evaluation:
in support of stronger business cases, more frequent evaluation and
better-informed decision-making.
Outline
1. Some questions for which economic appraisal can help find answers
2. Explanation of terms
3. The importance of “the margin”
4. Effectiveness, efficiency, equity and ethics (the 4 “E”s)
5. A pragmatic model of need, demand and supply
6. Which tool for which job?
7. Programme budgeting – organising investment to align with objectives
8. Benchmarking: how do we compare on investment and outcome?
9. Evaluation – accounting for investment and outcome, learning from experience
10. Further reading and resources
1 Some questions for which economic appraisal can help find answers
Does our investment as an organisation, directorate or team align with our
objectives? For example, can West Sussex County Council demonstrate how its
current budget aligns with its three strategic objectives, and will an understanding of
return on investment against those objectives inform future budgetary decisions?
Can we demonstrate to ourselves, our providers and the public that we provide value
for money, both overall and for any given service or age group?
How do we compare with our peers on spend, outcome and value for money
(effectiveness, efficiency) in any given service or age group?
How do we describe and demonstrate “fair shares” (equity) when it comes to tough
austerity decisions, and is there an explicit set of values (ethics) underpinning our
decisions?
When faced with a new business proposal or service transformation, how do we
decide whether to proceed? And when faced with budget cuts, how do we prioritise
which services to cut?
By investing in prevention and independence can we, and our local partners in health
and wellbeing, reduce the level of need and therefore budgets in the short, medium
or long term?
Is the payback period of a project short enough to justify up-front investment?
When an agency invests in a programme, to what extent are other agencies likely to
benefit?
Is the impact of a given proposal primarily concerned with money or public value?
2
2 Explanations of terms.
A decision is a choice that is made when the way forward is not self-evident.
Economic appraisal is the science behind the art of making tough choices. It supports
more open and robust decision making.
It is a systematic approach to setting out the inputs and outcomes from a list of
possible choices.
It is just as interested in effectiveness, efficiency, equity and ethics on the
outcomes side of the equation as it is on resources on the inputs side of the
equation. Resources are not just money but such things as buildings, people, time,
skills and motivation.
Economic appraisal is first and foremost a way of thinking, valuable to anyone in a
management role, and secondarily a set of tools to be deployed by those with
additional responsibility and some extra training. A few tasks are sufficiently
specialised to require the input of a qualified economist, perhaps from a local
university. West Sussex County Council would be well served if it set up such an
academic link, and the work of the Council would be an attractive test-bed for
research and development.
Economics recognises that resources are finite. It is concerned with opportunity
cost which is defined as the best alternative use of a quantity of resource.
It is not about “making economies” but about demonstrating the best possible return
on investment from the resources available.
3 The importance of “the margin”.
Economics is often more concerned with the margin than the average. The margin is the
next unit of resource or outcome as resources are added to, or removed from, a programme.
Marginal costs and benefits do not increase and decrease in a smooth linear fashion.
Where there is a sudden jump in cost it is known as a step cost. For example, a residential
nursing home might be able to cater for another two residents within existing staff and
buildings and at relatively low marginal cost, but a third patient might mean building a new
extension and taking on more staff, so the marginal cost of the third new resident would be a
substantial step cost.
It is a common phenomenon that as each unit of investment increases, marginal benefit
begins to tail off. This is commonly known as the “law of diminishing returns” or “the flat of
the curve”, or more correctly “diminishing marginal benefit.”
To illustrate the importance of average versus marginal costs, consider this imaginary
example (see boxes and graph).
3
A local area has an MMR (measles, mumps rubella) immunisation rate at 2 years of age of
85%. It would like to increase MMR immunisation rate to around 95-96% by employing more
immunisation nurses. It has established the following expected costs and outcomes,
reflecting the law of diminishing returns and the difficulties in immunising hard to reach
children. Each new nurse costs £50,000.
Investment-outcome curve for nurses and MMR immunisation uptake
Incremental investment (in nurses) and incremental outcomes (MMR uptake)
Cumulative MMR uptake
(% points)
incremental MMR uptake
(% points)
I nurse 89% 4%
2 nurses 92% 3%
3 nurses 94% 2%
4 nurses 95% 1%
5 nurses 95.7% 0.7%
6 nurses 96% 0.3%
1 2 3 4 5 6
Investment (nurses) 1 nurse = £50,000
Outcome
(Rise in
MMR
uptake)
(% points)
2
4
6
8
10
4
The local authority then constructed the following tables to inform a decision about value for
money, based on cumulative costs and cumulative outcomes for each successive nurse, and
marginal costs and outcomes at each step.
Seen in this way, and looking at the very high costs in the last two incremental steps to get
the last 1% uptake, the authority might decide to invest in just 4 new nurses, which would
deliver an uptake rate of 95%, and use the remaining £100,000 for other, better value, child
health purposes.
This illustrates the importance of marginal, rather than average, costs and benefits.
Average cost per outcome
Cumulative cost of each new
nurse (£)
Cumulative rise in MMR
uptake (% points)
Average cost per % point
increase in MMR uptake (£)
£50,000 4% £12,000
£100,000 7% £14,286
£150,000 9% £16,667
£200,000 10% £20,000
£250,000 10.7% £23,364
£300,000 11% £27,272
The authority was on the verge of agreeing to fund 6 new nurses, to reach their
upper target of 96% uptake, at a reasonable-looking average cost of £27,272 per
percentage point in uptake. But then they decided to look also at marginal costs,
and constructed the next table.
Marginal cost per outcome
Marginal cost of each new
nurse (£)
Marginal outcome for each
new nurse (MMR uptake %
points)
Marginal return on investment
[cost per % point increase in
MMR uptake] (£)
£50,000 4% £12,500
£50,000 3% £16,667
£50,000 2% £25,000
£50,000 1% £50,000
£50,000 0.7% £71,428
£50,000 0.3% £166,667
5
Optimal investment at the margin
In some circumstances harm can arise as well as benefit, for example chemotherapy or
radiotherapy in cancer care, or the institutionalisation that results from residential older
people’s care. Harm tends to rise in a continuous linear fashion. It can therefore be helpful
to plot incremental benefits and incremental harm on the same chart, against incremental
investment. The objective of this exercise is to stop investing when incremental benefits
start being overtaken by incremental harm. This point is referred to as optimal investment.
See the diagram below.
Optimal investment: where incremental benefits are overtaken by incremental harm
Benefit
Harm
Investment (£)
Widest gap between benefit and harm = optimal investment
Flat of the curve: diminishing marginal benefit
Outcome
6
4 Effectiveness, efficiency, equity and ethics (the 4 “E”s)
Effectiveness is a measurable change in health or wellbeing attributable to an intervention.
It is not about outputs but about outcomes.
Technical efficiency is concerned with delivering a given outcome at least cost. It is
particularly relevant to budget holders and providers of services for whom the objectives
have already been set and they can concentrate on the resources side of the equation. A
common example is the procurement process when a given specification is put out to tender
and the lowest cost bidder who meets that specification is awarded the contract.
Allocative efficiency is about delivering the maximum amount of health and wellbeing
across an entire budget and competing options. It is particularly relevant to commissioners
of services and policy makers, such as local authorities and clinical commissioning groups.
It is the commissioner’s role to decide where to set the limits to entitlements on offer, and the
range of services on offer, and thereby try to maximise population health and well-being from
the available budget. Another way of understanding the appraisal challenge here is that it is
the commissioner’s role to make sure that no-one outside the offer of services has a greater
need (ability to benefit) than someone who is inside the offer. This is known as “equity of
marginal met need”.
Technical efficiency can be thought of as “doing things right”.
Allocative efficiency can be thought of as “doing the right things”.
Equity has a number of connotations but in this context means “fair shares” as distinct from
“equal shares”. It could be argued that if there are two communities, one affluent and
healthy and the other less well-off and less healthy, rather than allocate equal shares of
money, staff and facilities to both, a greater share should be allocated to the latter on the
grounds that this would be fairer (more equitable).
Ethics in this context is a systematic and explicit approach to the values which underpin
decision making. Two key systems of ethics apply here. The first and generally
predominant value is utilitarianism – the principle of “the greatest good for the greatest
number”. This is often taken as read, but it can clash with other valid values such the
Hippocratic value, the clinician’s obligation to do the best for the patients in their care. That
in turn is closely aligned to the “rule of rescue” (which prioritises immediate threat to life and
the ability to rescue, irrespective of cost). The cost per life saved in some heroic
interventions under the rule of rescue can run into millions of pounds, to the potential
disadvantage of hundreds of other people whose opportunity to benefit has been foregone.
There are no “right answers” in ethics – just openness about the values that are being
brought into play when decisions are made. This point is illustrated in the next example.
The pursuit of efficiency is an ethical imperative for all who manage finite resources
in public services. At the end of the day it is the public who pay for any inefficiency,
and the currency in which they pay is not just taxes but avoidable distress, disability,
missed life-chances or even dying before their time.
7
Example: equity, efficiency and effectiveness in support of decision-making
In this example a commissioner of cancer care is faced with a choice between three cancer treatments: A, B and C. All have different costs and outcomes. There are 300 new cases of this cancer every year, and the budget for treatment is £1,500,000.
Treatment A costs £5,000 and yields 3 extra years of life. Treatment B costs £6,000 and yields 5 extra years of life. Treatment C costs £15,000 and yields 6 extra years of life. The quality of life is similar with all three treatments.
The commissioner constructs this table of costs, individual outcomes, affordability (how many people can be treated from the budget) and population outcomes, to inform its decision.
Treatment Cost per person (£)
Effect per person (added years of life)
Affordability (treated persons from budget)
Population health gain (total added years from budget)
A £5,000 3 300 900
B £6,000 5 250 1250
C £15,000 6 100 600
This table shows that:
A is the most equitable – all 300 people can be treated from the budget o (but it is not the most efficient in terms of population health gain, and it is the least
effective at individual level)
B is the most efficient – it yields most population health gain from the budget o (but it is not the most equitable since 50 people will go untreated, and it is not the
most effective at individual level)
C is the most effective – it yields the biggest individual outcome o (but it is the least equitable since 200 people will go untreated, and it is the least
efficient since it yields the least population health gain.) This simple analysis lays out the choices and trade-offs but does not spare the commissioner from value judgements nor tells the commissioner what to choose. In treatment options B and C a significant number of people go untreated and the commissioner must decide inclusion and exclusion criteria for receipt of treatment. Most commissioners would opt for A (on equity grounds) or B (on efficiency grounds) but would baulk at C since only one third of eligible people would be treated. A and B are examples of the utilitarian principle of greatest good for the greatest number. Individual cancer sufferers, their families and their clinicians are likely to press for treatment C because it prolongs life the most. They may well lobby their local member of parliament and the press. Laying out the trade-offs and the reasoning behind a decision on A or B in this way can help to engage with the public and deal with such pressure.
8
5 A pragmatic model of need, demand and supply.
The traditional economic market models of supply and demand do not apply in the context of
publicly funded health and social care, because the added consideration of “need” or
“entitlement” has to be added on. The definition of “need” is locally derived, in part from
evidence of ability to benefit from an intervention on offer and in part modified by political
considerations such as strategic priority, equity and affordability. “Need” for a service can be
elastic, becoming more or less inclusive as resources expand or contract, new technologies
arise or political priorities change.
The Venn diagram looks like this:
“Need” is the current public service offer, eg the current “need” for flu injections is not
universal but defined by age and disease criteria.
“Demand” is what people ask for.
“Supply” is what is funded, either by direct provision or via a contract.
1: A service is demanded, but does not meet criteria of need so is not supplied. Eg: inappropriate request for an antibiotic which is denied. To avoid clogging the system with demand that will not be met, the criteria of need to be explained, and alternative advice or support provided. 2: A service is demanded and supplied despite not meeting criteria of need. Eg. Inappropriate request for an antibiotic which is granted with a prescription. This is clinically inappropriate and a waste of scarce resources. There needs to be a system of monitoring and feedback for professional compliance with criteria of need. 3: A demanded service meets need criteria but has not been supplied. Eg someone on a waiting list for elective surgery or denied an intervention to which they were entitled. This matters because of the missed therapeutic opportunity and is another area for monitoring and feedback to professionals. 4: Need which has not been demanded or supplied. Eg a 50-year old male with diabetes who is unaware of his condition. This is an example of unmet need that might feature in a “joint strategic needs assessment”. It is a missed health or wellbeing opportunity. Note that “demand management” in this model can mean generating demand where it meets criteria of need and planned supply. 5: Needed service is provided without the individual initiating demand. Eg screening and immunisation programmes where people are called in to receive a service. It also includes services provided to vulnerable groups who cannot articulate their demand. 6. Waste and spare capacity. Eg unused medicines, underused staff or buildings. In times of austerity we need to bear down on this area to get a better overlap with need. 7. Needed services are demanded and supplied. This is the ideal overlap we strive for in commissioning.
Need
Supply
Demand
1
2
3 4
5
6
7 This is where
we want to be
9
6 Which tool for which job?
TOOL APPLICATION EXAMPLE
Cost-minimisation
appraisal
The outcome is given. The object of
appraisal is to deliver it at least cost.
Putting a service specification
out to tender and choosing the
lowest bid
Cost-effectiveness
appraisal
Choosing between two or more
alternatives with different costs and
outcomes and where the outcome is
measured in a natural unit (such as
blood pressure, tumour size or
independence). It is limited to
comparing like for like interventions
with the same units of outcome.
Note that nothing is absolutely cost-
effective – it is only more or less cost
effective than some alternative,
including “business as usual”. There
are several standard instruments
(questionnaires) that ascribe a
numerical value to aspects of health
and wellbeing.
Choosing between two or more
medicines to treat blood
pressure or cancer, or choosing
between two or more care
packages for frail older people
to preserve independence.
Cost-utility appraisal Similar to cost effectiveness, except
all outcomes are converted to the
same utility units, typically “quality-
adjusted life years” (QALYs). It has
the advantage of supporting choices
between options with different
natural units, eg hip replacements,
cancer treatments or heart
operations which can be converted to
QALYs. There are tables setting out
combinations of distress and
disability that quantify quality of life
on a scale of 0 to 1, hence someone
surviving for 2 years with 0.6 quality
of life would score 1.2 QALYs.
Allocative efficiency decisions
for planners, such as how to
divide up a large budget
between several competing
programmes and care groups.
A cost per QALY league table
would be a good place to start.
Cost-benefit
appraisal
In this, both resources and outcomes are ascribed monetary value. The difficulty arises in ascribing
monetary values to outcomes such as years of life, pain, mobility, or antisocial behaviour. Government departments use this technique and there are
tables and norms for ascribing certain values (see
http://www.pssru.ac.uk/project-pages/unit-costs/2015/index.php.)
Answers questions such as:
“Should we invest at all?” “Will
there be payback in a
reasonable time frame? Who
pays and who benefits? What
is the social return on
investment?
10
An example of cost-benefit appraisal: social return on investment (SROI)
SORI is an outcomes-based appraisal tool that helps organisations to understand and quantify the social, environmental and economic value they are creating. It is a participative approach that is able to capture in monetised form the value of a wide range of outcomes, whether these already have a financial value or not.
An SROI analysis produces a narrative of how an organisation creates and loses value in the course of its work, and a ratio that states how much social value (in £) is created for every £1 of investment.
SROI is informed by a set of principles that are designed to ensure that process is robust, transparent, and informed by stakeholders. These are the steps:
1. Conduct a needs assessment - establish the scope of the proposal by talking to key informants, looking at published evidence and studying local data.
2. In discussion with key informants, list the relevant inputs, outputs and outcomes.
3. Ascribe a total monetary value to the inputs.
4. For each benefit estimate how many potential beneficiaries (“people at risk”) there are, and estimate how many of those potential beneficiaries will actively engage and complete the programme (%).
5. Estimate the net benefit by subtracting the impact that would have occurred anyway (had there been business as usual).
6. Ascribe a total monetary value to the benefits. Some benefits, such as avoided transport costs, reduced hospital admissions or fewer court appearances are relatively easy to give a financial value. Others, such as jobs created, qualifications achieved or loneliness relieved are more difficult but should be included and the assumptions stated. There is a very useful database of typical values for a wide range of social outcomes here: http://www.pssru.ac.uk/project-pages/unit-costs/2015/index.php
7. Having arrived at a figure for inputs and for outcomes, the pace of engagement and timing of benefits and cash-release over the lifetime of the proposed intervention have to be factored in. Consider applying a “discount” (about 3.5% per annum is a norm in UK government practice) to future costs and future benefits.
8. Calculate the SROI by comparing input value to outcome value. This is usually expressed as a ratio, eg if £1 of investment yields £5 in benefits then the SROI is 1:5.
9. If the proposal is given the go-ahead, support the implementation and evaluation, and consider publishing the results if they are ground-breaking.
11
Forecasted SROIs predict how much social value will be created if activities meet their intended objectives. They are used in pitching business cases. A forecasted SROI can be followed with an evaluative SROI to verify the accuracy of the predictions.
Evaluative SROIs are conducted retrospectively and based on actual outcomes that have taken place over a given evaluation period. They are much easier to conduct if the objectives and data monitoring requirements were clearly stated at the outset; the method of evaluation should be an integral component of any prospective business plan.
SROI can be time-consuming and resource-intensive, especially when evaluative, but it offers additional benefits by promoting interest in social outcomes and opening dialogue with partners that can lead to fruitful collaboration.
SROI can be conducted by consultants brought in for the purpose, but that can detract from the development opportunities for core staff. The more that SROI is built into everyday planning and evaluation, the more robust the decision-making will be.
Economic appraisals involve judgements and assumptions, and frequently require a
sensitivity analysis to see if different assumptions tip the ratio of cost to outcome in
sufficiently to change a decision.
It is common for projects to span several years. Economists allow for the fact that costs and
benefits which occur in the future have less impact than present day ones, so they are
discounted – typically by 3.5% per year – to give them a present day value.
Financial return on investment is not the only consideration, hence the row to log other decision factors such as feasibility, strength of strategic fit with local and national priorities, and the results of any local consultation. The table below is a suggested framework for setting out competing business case proposals (for new investment or for disinvestment). It steers the assessors to constructing a “return on investment” row at the bottom, and the highest ratio should be the starting point for prioritisation. Where the numbers are close, it is worth doing a sensitivity analysis, changing some of the assumptions, to see what confidence there is of a true difference.
12
Generic decision-support checklist for prioritising competing business cases
(See appendix 2 for specific template in regard to “prevention and independence” bids)
Proposal 1 Proposal 2, etc
Title of Proposal
Net financial value of Inputs (£)
Year 1, Year 2, etc
Number and profile of beneficiaries
Allow for non-attenders
Age profile of beneficiaries [see section on programme budgeting]
Nature of outputs and outcomes
Year 1, Year 2, etc
Benefits to individuals
Benefits to society
Benefits to providers [eg time, efficiency]
Cashable benefits,
Other relevant decision factors, eg
Ease of implementation and evaluation
Strategic fit: local and national
Key informants’ views
Published evidence base
Net financial value of benefits (£)
[subtract cost of “business as usual”]
Return on investment: ROI
[Ratio of input (£) to outcome (£)]
13
7 Programme budgeting and marginal analysis (PBMA) – organising investment to align with objectives This is a technique used in many public sector organisations around the world including the NHS. It has an extensive evidence base. Its primary purpose is to aligning investment to outcome in a set of meaningful programmes. The essential steps in a PBMA approach are these:
• Align decisions into meaningful programmes, eg disease groups, age groups. • Agree the programmes’ objectives • Identify current programmes’ resources • Appraise incremental shifts in costs and benefits if programmes’ resources are
deployed in new ways – fist within programmes, then between programmes • Consult • Implement • Evaluate and share the learning
For each programme in turn it allows an authority to …
Simplify
Understand
Explain
Coordinate
Evaluate
Compare
Plan
Be accountable
An example of possible programme budget categories would be West Sussex County Council’s three strategic objectives relating to:
1. young people 2. economic development 3. older people
If the Council wanted to adopt an explicit programme budgeting approach along those lines it would align its expenditure to those three programmes and then seek to maximise outcomes in each. (Some functions, like back office support, might be allocated across all three programmes or, alternatively, given a programme of their own.) The Department of Health has been running a programme budget project for over a decade,
based on disease groups as programme budget categories. This was taken on by Public
Health England for CCGs, with data and benchmarking at CCG or upper tier local authority
level. It can be found here: http://www.yhpho.org.uk/default.aspx?RID=49488
Programme budgeting is frequently twinned with marginal analysis (PBMA), looking at
incremental changes in costs and outcomes when resources are moved within or between
programmes. It is a tool for maximising allocative efficiency (see above).
14
Age groups (stages of life) are a strong candidate for programme budget categories,
because:
It should be possible for all the partners in the Health and Wellbeing Board to map resource deployment to each age group and thereby facilitate alignment (or pooling) of budgets around common objectives. Here are two extant examples of age-based categories.
o Example 1. The NHS funding formula recognises eight different age weights (reflecting need for services) in the following age groups: births, 0 - 4 years, 5 - 14, 15 - 44, 45 - 64, 65 – 74, 75 - 84, and 85+. It is curious that the NHS never checks to see if those resources are deployed in those same age groups.
o Example 2. One of the West Sussex CCGs has expressed an interest in using five age groups: conception – 4years, 5 -19, 20 - 64, 65 – 84 and 85+. Objectives and measures of success have been drafted for each. (See appendix 1). An attempt may be made to estimate how much investment each partner on their integration group brings to each programme.
For West Sussex County Council, age breakdowns relate well to the three strategic objectives (early years, economic activity and older years).
Age is inclusive and precise – everyone in the population has just one age.
It gets around the problem of disease-based or care-based programme budgeting where multiple diseases and social care needs may occur in the same individual – that individual belongs in only one age group
We have reasonable data on age profiles of resident populations and GP-registered populations for planning purposes - not only current patterns but also future projections.
We know that the proportions of people in different age groups will rise or fall at different rates in future and we need to plan for that variance.
Age is a good predictor of pattern of both health and social need, for example many of the needs of pre-school children differ from those of school aged children, young working age (including peak reproductive age), later working age, early retirement and extreme old age respectively.
Programme budgeting works best where it is aligned to the business agenda of the participating partners in health and wellbeing boards, and better still when aligned to their directorate structures and business agenda. Imagine a CCG, local authority or NHS Trust adopting five stages of life as “programmes”, with programme objectives and programme budgets accordingly. Imagine appointing a programme director and finance support for each. The routine CCG, authority or Trust agenda would then follow the lines of: what is the current demand, activity, spend, user experience, outcome and trend in each programme? Each successive year’s programme budgets would be based on marginal analysis of cost and outcome between the programmes depending on what maximised population health and wellbeing from the total available budget. At the moment we often have cross-cutting responsibilities with no coherent overall picture, and a paucity of value-for money information within a given area or between areas. A PBMA approach would help sort out the muddle.
15
8 Benchmarking: how do we compare on cost and outcome?
There are a number of on-line resources that publish data on investment and outcome by
local authority and/or clinical commissioning group.
Most are “indicators”, and the purpose of an indicator is “a challenge to explain”.
The most relevant benchmarks are not the national or regional norm but the demographically
closest peers.
Beware of league tables and averages! In any league table or arithmetical average, half the
subjects will be below average. It is not the relative position that matters but the absolute
position: is this above an acceptable standard for outcome and value.
The chief value of peer comparisons is to act as stimulus to learn from and emulate the best.
Resources for benchmarking:
Local Government Inform
http://www.local.gov.uk/about-lginform
Spend and Outcome Tool (SPOT) for Local Authorities and Clinical
Commissioning Groups
http://www.yhpho.org.uk/default.aspx?RID=49488
CIPFA benchmarking club
Ask finance department for access to data. West Sussex County Council is a
participant in the club)
Public health outcomes framework
https://www.gov.uk/government/publications/healthy-lives-healthy-people-
improving-outcomes-and-supporting-transparency
16
9 Evaluation – accounting for investment and outcome, learning from
experience
Evaluation is: “the degree to which an intervention has met its objectives”.
It is much easier to evaluate a service whose objectives have been clearly stated.
Evaluation is best built in at the outset of a project or programme. It can be very difficult to
collect data retrospectively. It is a good discipline and good practice to make the method of
evaluation explicit before a project or programme is launched. Better still, draw up the
ghosts of tables that will be filled in when the evaluation is written up – are all the necessary
data being collated, including at the baseline?
Evaluation need not be expensive if built into day-to-day management and quality control,
but it does have a cost in staff time and other resources. A rule of thumb is that about 5% of
a project’s costs should be set aside for evaluation.
Generally, one of a project’s or service’s objectives will be to demonstrate value for money
against a benchmark, either against a prior business proposal, or a previous year’s
performance or a similar project elsewhere. It is therefore important that all relevant aspects
of cost (and other resources like time or volunteering) are identified and collected at the start
and conclusion.
Absence of evaluation makes it difficult to decide whether to stop, continue or expand a
service.
Most evaluations benefit from a comparator or control group, where the comparator may be
“business as usual” in another part of the service.
When does an evaluation become “research”? This is a grey area, but it matters when the
issues of research ethics and research governance come in. There are no hard and fast
rules, but generally it is research if you are doing something new that might have adverse
effects (such as new medicines); that might disadvantage others (like rehabilitating ex-
offenders in a sensitive neighbourhood) or is conducted primarily to advance knowledge and
be published. If in any doubt ask a senior member of the public health research unit or
secretary of the local NHS research ethics committee.
17
10 Further reading and resources
Unit costs of health and social care
http://www.pssru.ac.uk/project-pages/unit-costs/2015/index.php
Supporting public service transformation: cost-benefit guidance for local partnerships, April 2014, HM Treasury, Public Service Transformation Network, new economy (Manchester)
o http://neweconomymanchester.com/
Public health economics: a systematic review of guidance for the economic evaluation of public health interventions and discussion of key methodological issues. Edwards RT, et al, BMC Public Health 2013, 13: 1001
o http://www.biomedcentral.com/1471-2458/13/1001
Arts for health and wellbeing. An evaluation framework. Aesop (Arts enterprise with a social purpose) and University of Winchester, PHE, January 2016,
o www.gov.uk/phe
Oxford Handbook of Public Health Practice, 3rd edition, Guest C et al eds, Oxford University Press, 2013. Chapter on economic assessment, p64 – p72
A Social Return On Investment Primer can be accessed online at http://sroi.london.edu
SPOT tool for Local Authorities http://www.yhpho.org.uk/default.aspx?RID=49488
Dr Peter Brambleby. Interim consultant in public health. March 2016
18
APPENDIX 1: Age-based programme budgeting structure How it might work (illustrative example based on a real CCG)
Conception to 4 years: number of children now …. rising ….% by ….
Area of need
to be
addressed
Why does it
matter?
What can we
do?
[Add locality-
specific issues]
How much do
we invest?
Local Authority
NHS
Social capital
Where are we
now, and how
would we measure
progress?
Safe in the
womb
Many determinants
of health and
wellbeing, and
future inequalities
in health, are set in
the womb
Chiefly midwifery
and general practice
Maternal health and
wellbeing (including
nutrition, lifestyle,
finances, security,
and mental health)
Antenatal screening
Maternal smoking
Data from hospital
midwifery teams and
GP records.
Safe around
birth
Arrival into the world
and the first month of
life are a critical
period of risk and
opportunity
(Chiefly midwifery
and general
practice)
Obstetric
interventions
Breast feeding
Breast feeding rates
Low birth weight (% <
2.5kg)
Infant mortality rate per
1000 live births
Nurturing
environment
Although the genetic
endowment and
certain
developmental
features are set
before birth, the next
few years of life in
which the baby is
dependent for food,
safety, love and
stimulation, have a
large and mostly
irreversible impact on
future health and
wellbeing.
(Chiefly health
visiting, districts and
boroughs, social
services)
Family stability
Family finances
Housing
Safe from abuse
Use of hospital
services eg A&E
Admissions to hospital
in under 5’s due to
unintentional injury
Census data on 4
indicators of
deprivation:
employment,
education, health and
disability,
overcrowding.
Child protection
register
Setting the
foundations for
a healthy life
There are aspects of
wellbeing in the early
years where the
statutory agencies,
especially nursing,
(Chiefly health
visiting, general
practice, social
services, education)
National Child Measurement Programme: % of children overweight or obese at reception (5
19
general practice,
social services and
education can make
an impact.
A child’s educational
attainment is one of
the strongest
predictors of future
healthy life
expectancy, not only
for the child but also
for the next
generation.
Healthy weight
Healthy teeth
Immunisation uptake
Screening uptake
Early health
problems identified
and addressed, eg
genetic, congenital,
hearing, vision
Early learning
problems identified
and addressed
Use of hospitals
minimised
years):
Immunisation cover
Hospital activity data
(at practice level)
Prescribing data (at
practice level)
Dental activity data (at
dental practice level)
Indicators of morbidity
Use of hospitals, eg
A&E
Achieving
personal
potential and
developmental
milestones
This is a vital stage
at which to break a
cycle of missed
opportunity and
inequality.
Pre-school
educational
attainment
Speech and
language
Fit and active
Social skills
DfE index of “good level of development” at school entry:
5 – 19 years: numbers of people now …. rising by …. % by ….
Areas of
need to be
addressed
Why does it
matter?
What can we do?
[Add locality-
specific issues]
How much do
we invest?
Local Authority
NHS
Social capital
Where are we
now, and how
would we
measure
progress?
Laying
foundations
for healthy
adult life
This consolidates
the early years’
progress in
establishing
lifelong trends of
healthy mind,
body and lifestyle
habits
Healthy weight
Lifestyle:
smoking/drugs/alcohol/nu
trition/exercise
Unplanned pregnancy or
parenthood
Overweight or obese at 11 years (NCMP)
West Sussex Schools Survey 14-15 has good lifestyle data
15-17yrs pregnancy/1000 in 2013: (180 conceptions)
20
Emotional
wellbeing
Puberty and
adolescence are
when emotional
resilience is
developed,
aspirations are
shaped, moral
compass is
attuned and long-
term relationships
are forged.
80% of later adult
mental health
problems are
evident in teen
age
Relationships
Safe sex
Child and Adolescent
Mental Health Services
Wellbeing of children who
are carers
GP data at practice
level
NHS mental health
trust data
National “What about
Youth” study – some
data at district level
Safe from
harm
Independence
and choices are
part of growing up
but young people
need external
support
throughout this
period to keep
them safe.
Child sexual exploitation
Female genital mutilation
Bullying
“Prevent” agenda
(radicalisation)
Child protection data
Police records
School lifestyle
survey (bullying)
Nurturing
environment
As dependants,
young people
need to be
provided with
shelter, food,
education,
emotional support.
Housing
Family stability and
guardianship
Financial security
Census data (updated
ONS estimates)
Adoption and
fostering data
Achieving
potential and
personal
resilience
There are
important
measures at key
stages of physical
and educational
development that
give individual and
collective markers
of progress.
Special educational
needs catered for
Educational and skills
attainment
Physical activity
Citizenship and
engagement
5 or more GCSE grades A*-C including English and Maths, at school leaving;
21
20 – 64 years: number of people now …... rising by ….% by ….
Area of need
to be
addressed
Why does it
matter?
What can we
do?
[Add locality-
specific issues]
How much do
we invest?
Local Authority
NHS
Social capital
Where are we
now, and how
would we measure
progress?
Physical
health and
wellbeing
Physical activity is
probably the best
single intervention we
can encourage.
This is the age where
determinants of health
start to be expressed in
symptoms, illness and
use of health and
social care, but still at a
stage where
intervention can make
a difference.
Healthy workplaces:
statutory agencies
should aim to be
exemplary employers
Lifestyle: smoking,
diet, alcohol, drugs
Physical activity
Screening
programmes:
cancer, health
checks
Coastal CCG, top 5
annual causes of dying
before the age of 75
(3-year average):
Sport England
produces survey data
on adult activity over
16 years.
Cancer screening uptake within 6 months (CCG):
Reproductive
health and
wellbeing
This is the peak age for
reproduction – avoiding
it, planning it, or living
with it.
Family planning
Infertility
Childbirth
Parenting
PHE data on fertility
and childbirth.
Sexual health data
from local providers
Contraception data GP
and pharmacies
Emotional
health and
wellbeing
There is still time to
build emotional
resilience developed in
childhood, including
building healthy work-
life balance for a
sustainable career
Loneliness in later life
can have its origins
here
Consultations with
GP or referrals to
mental health
Family and
relationships
Gambling and
addictions
“Prevent” agenda
(radicalisation)
GP, social services anf
hospital activity data.
Police data
Social
wellbeing
This is the stage of
earning, saving and for
some being the
breadwinner for others.
Some adults will be
Housing
Employment
Finance and
DWP data
DfLG&C data
22
unpaid carers
Digital connectedness
is almost essential for
modern adult living
budgeting
Debt and fuel
poverty
Digital connectivity
Safe from domestic
violence
Achieving
potential and
personal
resilience
Acquisition of skills and
experience
Healthy work-life
balance
Skills and career
progression
Hobbies and
personal fulfilment
Social engagement
DfE data
DWP data
DfLG&C data
65-84 years: number of people now …. rising by ….% by ….
Area of need
to be
addressed
Why does it
matter?
What can we
do?
[Add locality-
specific issues]
How much do
we invest?
Local Authority
NHS
Social capital
Where are we now,
and how would we
measure progress?
Physical
health and
wellbeing
Remaining physically
active is one of the
most useful
interventions in older
age, especially if it also
leads to reduced
isolation
Active lifestyle
Health promotion
and protection
Screening
Immunisation (flu)
Care closer to
home, including
proactive care and
telemedicine
Self-care
Carer support
Further life expectancy at 65 years (west Sussex):
Carer support data (6 domains: occupation, control, personal care, safety, social participation, encouragement and support), West Sussex level, 2012/13
GP practice data
User feedback from Healthwatch
Emotional
health and
wellbeing
Loneliness is as high a
risk factor for ill health
in old age as obesity or
smoking.
Depression and
dementia are higher
risk in this age group
Loneliness
Digital connectivity
Depression
Dementia
GP dementia register
GP practice data –
prescribing
GP practice data –
referrals
Experience of care -
23
Healthwatch
Safety Frail and vulnerable
older people need
regular safety checks
for falls, fire, abuse,
and crime. Such an
assessment should be
within the scope of a
fire safety officer,
policeman, social
worker, home help or
community nurse.
Falls
Fire
Crime
Abuse
Hospital admissions
data
Fire service data
Police data
Social
wellbeing
Regular “wealth”
checks are as relevant
as regular “health
checks” for things like
maximising benefits,
savings and utility
contracts.
Finances and
benefits
Housing and down-
sizing
Home adaptations
Help in the home
Mobility outdoors
Social care caseload
data
Admissions to
residential and nursing
homes
District and borough
housing data
Attendance allowance
data
Disability allowance
data
Achieving
potential
Independent living
Engaged and making a
contribution
Volunteering
Contributing
Adult education and
interests
Data from districts and
boroughs,
Data from Education
85 years and older: number of people in …., rising by ….% by ….
Area of
need to be
addressed
Why does it
matter?
What can we
do?
[Add locality-
specific issues]
How much do
we invest?
Local Authority
NHS
Social capital
Where are we
now, and how
would we measure
progress?
Safety Protection of frail and
vulnerable older
people from avoidable
harm or threats to
independent living
Falls
Fire
Crime
Abuse
Standardised admission rate for falls aged 65+ per 100,000
24
Physical
health and
wellbeing
Maintaining physical
health in support of
activities of daily living
Physically active as
possible
Care closer to home
Reablement after
admissions
Medicines reviews
Flu immunisation
GP practice activity
data
GP prescribing data
Hospital and
community activity data
Experience of care -
Healthwatch
Emotional
health and
wellbeing
The avoidance of
loneliness.
Loneliness
Depression
Dementia
Social care data
GP practice and mental
health trust data
Social health
and wellbeing
– maintaining
independence
Respecting choice for
maximum
independence in
familiar surroundings.
Housing
Finance and benefits
Warmth
Help at home
Help with activities of
daily living
Support for carers
Fuel poverty %
households
Social care caseload
data
Legacy and
fulfilment
It is good medical and
social care practice to
recognise the
approaching end
stage of life and
manage it well, for the
sake of the individual
and those left behind.
Unplanned, avoidable
deaths in hospital can
cause unnecessary
distress and waste
resources.
Recording memories
Last wishes and will-
making
Saying goodbyes
and preparatory
bereavement
counselling for loved
ones
Place of death, 2013, Coastal CCG, all causes, over 85 years:
Place of death for those over 65 with dementia, Coastal CCG:
Bereavement counselling data
25
APPENDIX 2: CARE WELLBEING AND EDUCATION
BUSINESS CASE TEMPLATE FOR ADDITIONAL FUNDING
This template has been created to help you to submit applications for additional funding to the
Adults Leadership Team.
Guidance on completing the business case template
Here are some general points to consider before you start to complete it.
Think Big
The template is intended to capture the funding requirements for transformational-type ideas that
are likely to have a longer term pay back from the investment. It is difficult to put a value on this but
the types of ideas that are needed are likely to have funding requirements in excess of £100k.
Initiatives needing less funding will also be considered if it can be demonstrated that they will deliver
savings of a significant magnitude.
Be clear on financial benefits
The types of initiatives that are likely to attract additional funding need to lead to the County Council
spending less money in the future than it would otherwise need to do if we did nothing. So the
savings attributable to the scheme need to be clear, including where they will fall, and the
timescales for the delivery.
Ensure your idea is supported
Before submitting a business case, you will need to ensure that your idea has been fully developed
and is supported by the area of the business that will see budget reductions if the initiative is taken
forward. So, for instance, if Adults Social Care are putting forward an idea that will lead to savings
also being delivered in Children’s Social Care, the expectation is that both budget holders will sign
the business case. There is also an expectation that the costs and savings will have been ratified
with your Service Finance contact.
Measuring progress
26
Your business case will need to outline how progress and delivery of savings will be measured. For
instance if your initiative will lead to a reduction in residential placements, how will you measure
that and how frequently will that be reported?
Process for bid consideration
In order for your business case to be discussed by ALT, it will need to show that you have considered
the points above. Any cases which fail to do this are likely to be rejected.
Bids should be submitted Steph Baxter. They will then be considered by the next ALT and you will be
notified of the outcome via an email from Steph. Until you receive an email confirming that funding
has been approved, you should not make any commitments against the money.
27
Summary of change
Overview of the change (what is the service / model change)
Outline the change you are proposing
Why change?
Briefly outline why we need to change include any relevant local evidence or analysis
What are the arrangements (systems and processes) for governance, performance, quality and finance monitoring
Are there any dependencies?
28
Who will you need work with?
What providers and other partners are involved in delivery of this business case
Who will be the Lead Commissioning Officer?
Duration of Scheme and key Milestones Date
Implementation or go live date
Implementation complete and service change fully (100%) operational
Impact – benefits and risks
29
Please detail the impacts of the change (as appropriate) on the following outcomes
Reduction in Non-Elective Admissions
Reduction in permanent admissions to residential or nursing homes
Increase in the number of people still at home 91 days after discharge from hospital
Reduction in delayed transfers of care
Improvement in Social Care related Quality Of Life
Increased estimated diagnosis rate for people with dementia
Other non-financial benefits to customers, to WSCC, to health and to other beneficiaries?
Customers
30
WSCC
Health
Other beneficiaries
What risks are there to delivery
of this change?
Impact
5-Catastrophe
4-Significant
3-Moderate
2-Minor
1-Insignificant
Likelihoo
d
5-Certain
4-Likely
3-Possible
2-Unlikely
1-Rare
Risk
score
(Impact x
Likelihood)
Mitigation
How will the impact of the change be evaluated and when?
Measure Evaluation date Data source
31
Sustainability. If the change is successful how will it be sustained in the longer term?
Exit Strategy. If the change is not successful how will be project be closed down?
Costs and savings
COSTS (Non-cumulative in year effect in current prices) – please summarise below and show budget
source and add all assumptions and workings as an appendix
Costs - summary 15/16 16/17 17/18 18/19 Total
Total Costs
SAVINGS (Non-cumulative in year effect in current prices) – please summarise below but add all
assumptions and workings as an appendix
Savings - summary 15/16 16/17 17/18 18/19 Total
32
Total Savings
Total Net Cost/Saving of Proposal
Budget Holder name and agreement to proposal and
budget reductions as shown above
Name of Finance lead costs and savings discussed
with
Submission and approval
Submitted by (Name)
Title
Date
Business Case Bid Number -
Discussed at ALT meeting Date:
Decision of meeting Approved/Rejected
Applicant notified Date:
Further updates (where applicable)