TitleHow do you know if you have got it right? Evaluation and Indicators

Professor Vanessa Burholt

Outline

Evaluation of the Strategy for Older People in Wales

Selection of indicators to measure impact

Successes and shortcomings

Strategy for Older People in Wales: Aims

To tackle discrimination against older people wherever it occurs, promote positive images of ageing and give older people a stronger voice in society.

To promote and develop older people’s capacity to continue to work and learn for as long as they want and to make an active contribution once they retire.

To promote and improve the health and well-being of older people through integrated planning and delivery frameworks and more responsive diagnostic and support services.

To promote the provision of high quality services and support which enable older people to live as independently as possible in a suitable and safe environment and ensure services are organised around and responsive to their needs.

To implement the Strategy for Older People in Wales with support funding to ensure that it is a catalyst for change and innovation across all sectors, improves services for older people and provides the basis for effective planning for an ageing population.

Evaluation of policies: Challenges

“Policy is implemented within a social and political context of ideology, commercial interest, resources constraints, media reports, pressure groups, lobbyists and public expectations, all of which skew its effects”

(Harwood 2007, p.483)

Evaluation of policies: Framework

Process Evaluation:Examine early

development and implementation of the

work: how did this occur? What

happened? Who was involved?

Formative Evaluation:

Provide essential information to inform any adjustments to

original plans

Summative Evaluation:

Examination of outputs outcomes and impacts:

what has been achieved?

What have been the costs?

Strategy for Older People: Evaluative phases

Phase 1: Provide an independent overview of the implementation

of the Strategy for Older People in Wales

Phase 2: Identify baseline datasets consider the potential for

analysis and highlight the need for further developments for a

comprehensive set of performance indicators

Phase 3: Construct case studies in four different Local Authorities

in Wales

Phase 1: Research questions

What guidance has been provided by the Welsh Assembly Government to local authorities on how they should implement the Strategy?

How have local authorities responded to this guidance in terms of staffing and roles structures for consultation, engagement and decision-making their plans and reports?

Across Wales, which of the Strategy’s aims and objectives have been prioritised? Which have received least attention? Has the focus changed over the first three years of the Strategy?

In what dimensions has local implementation of the Strategy varied between local authority areas?

Phase 1: Research questions (continued)

What examples are there of innovative and interesting practice funded through the Strategy?

What has been the role of the four support organisations (Age Alliance Wales, Beth Johnson Foundation, Better Government for Older People (BGOP), Welsh Institute for Health and Social Care (WIHSC), in implementing the Strategy?

What are the views of local co-ordinators, Older People’s Champions, and members of older people’s forums on the way the Strategy has been implemented so far, and on the way forward?

Phase 2: Selected indicators

Strategic aim Indicators of change proposed in Strategy for Older People 2008

Valuing older people – maintaining and developing engagement

Fear of crime: % 50+ who report that their lives are greatly affected by fear of crime.

Contact with friends and family: If sufficient data can be found

Participation in sports or leisure activities: 50+ participation in a sport or activity in previous month

Access to transport: If sufficient data can be found

Changing society – the economic contribution of older people

Employment rate for: i) 16-59/64 (all-Wales and LA); ii) 50-59/64 (all-Wales) iii) 60/65+ (all Wales)

Older people in work-related education/training: % 50+ who have taken part in any education or training connected with their job, or any job that they might do in the future

Health, well-being and independence

Access to surgical procedures: 65+ rates per 1,000 for (i) hip replacement, (ii) knee replacement.

Disability free life expectancy 65+: number of years 65+ ca expect to live free from limiting long-standing illness

Receipt of community based services helping people to live at home: 65+ helped to live at home per 1,000 population aged 65+

Housing that is unfit or in a defective state: proportion 60/65+ living in household classed as (i) unfit (ii) defective.

Phase 3: Aims

To examine the structure and role of local strategic planning groups with responsibility for implementing the Strategy.

To examine the structure and role of 50+ forum(s) in each study area.

To explore the views of the members of 50+ forums and of other key stakeholders on the influence of older people on decision-making in relation to the Strategy.

To examine the role of the Older People’s Champions.

To describe the processes by which local needs have influenced implementation of the Strategy.

To gather the views of Chief Executives on the balance between self-contained projects under the Strategy and the ‘mainstreaming’ of older people’s issues.

Criticisms of evaluation

Selection of indicators are not comprehensive enough to cover all elements of the strategy

Consultation with older people focuses on those who are socially engaged

Conclusion

The Strategy for Older People in Wales has been a bold and broad move by the Welsh Assembly Government to improve the health and well-being of the ageing population….

BUT

A policy requires adequate resourcing for implementation and evaluation