none of the
Post on 15-Jan-2016
27 Views
Preview:
DESCRIPTION
TRANSCRIPT
None of the interventions are
evidence-based but think positive – you
get a new and untested
intervention
IMS APPROACH TO GOOD PRACTICE
Knut SundellJenny RehnmanMari Forslund
www.evidens.nu
Development & Evaluation
(assessment tools and interventions)
20 employees 30 projects lasting
3-4 years
Systematic reviews(assessment tools and
interventions)
10 employees 15 projects lasting 1-
2 yearsAdditional external
researchers
Support & Dissemination
(administrative support & dissemination of all
that is produced within IMS)
10 employees 15 projects lasting
less than 1 year
IMS director Board of IMS
(12 stake-holders)
Director-general NBHW
• Sweden is the third largest country in Western Europe with nine million inhabitants
• Social care service delivery involve agencies of the 290 municipalities, and 20 regional county councils. Median size of municipalities = 15.250 inhabitants
• National agencies and research councils, deals with research and development, supervision of service quality and safety, and the delivery of institutional care.
• The voluntary sector is considerable
• Social welfare states (e.g., low rate of unemployment, poverty, drug use, violence)
• General trust among Swedes in collective solutions and in government authorities
Sweden
Research on interventions for womenwith experience of partner violence(Anttila et al, 2007)
5 926 articles
Research on interventions for womenwith experience of partner violence(Anttila et al, 2007)
5 926 articles
Effect studies 99
Research on interventions for womenwith experience of partner violence(Anttila et al, 2007)
5 926 articles
Good evidence 6
0,1 per cent
Swedish outcome studies on social work
0
5
10
15
20
25
30
35
1990 1992 1994 1996 1998 2000 2002 2004 2006 2008
RCT Quasiexp
Two articles in Sweden’s largest morning paper by the Director-General of the Swedish National Board of Health and Welfare claiming that social work agency managers did not have any idea whether social care services made any difference to clients’ lives.
Important dates
National action plan for preventing alcohol related harm & National action plan on drugs (lasting to 2007). 100 million SEK on researchTraining of local prevention workers
IMS is inagurated
General election and a new government
The first national guidelines on social work (substance abuse treatment)
Official reports of the Swedish government – “Evidence-based social work practice – favouring clients”
Agreement between the government and the Swedish Association of Local Authorities and regions on implementing the national guidelines on substance abuse treatment
An inquiry initiated at the NBHW on how to support EBP byNational guidelines, research reviewsNational and regional knowledge transferDeveloping criteria for local-follow up and open comparisons
Initiative to clarify the concept of EBP among governmental authorities
Addiction Severity Index is introduced in Sweden
Looking for an evidence-based practice
Random sample of local authorities (2/3)
Managers responsible for budget & staff
Autumn 2007
Electronic questionnaire
Response rate 88% (n = 834)
INTEREST IN EVIDENCE-BASED METHODS (N = 834)
Definitely
Partly
No
PREVALENCE OF EVIDENCE-BASED METHODS (SOMETIMES)
0
20
40
60
80
100
Children Youth Substanceabuse
Disabled Socialassistance
Elderly
%
24 per cent
PREVALENCE OF EVIDENCE-BASED METHODS (MOSTLY)
0
20
40
60
80
100
Children Youth Substanceabuse
Disabled Socialassistance
Elderly
%
7 per cent
PREVALENCE OF EVIDENCE-BASED METHODS (SOMETIMES)
0
20
40
60
80
100
%
PREVALENCE OF STANDARDISED ASSESSMENT TOOLS (SOMETIMES)
0
20
40
60
80
100
Children Youth Substanceabuse
Disabled Socialassistance
Elderly
%
63 per cent
0
20
40
60
80
100
Children Youth Substanceabuse
Disabled Socialassistance
Elderly
%
14 per cent
PREVALENCE OF STANDARDISED ASSESSMENT TOOLS (REGUAL)
Sweden: What are the political issues that need to be addressed?
Keywords: Ethics – Evidence – Transparency
not the technical issues concerning internal / external validity
Knowledge transfer / dissemination of innovations need extensive support at a regional level
Bring forward good examples
Stockholm evidence-based Clearinghouse for social work
• ”Evidence-based medicine is the conscientious, explicit and judicious
use of current best evidence in making decisions about the care of
individual patients” (Sackett et al., 1996)
• The general aim of “the Stockholm evidence-based Clearinghouse for social work” is to build a bridge between professionals and research on “what works”
• It was launched
on 1st September, 2008
18
Stockholm evidence-based Clearinghouse for social work
• Web-based service (at IMS’ homepage): http://www.socialstyrelsen.se/Amnesord/socialt_arbete/IMS/Metodguiden_index.htm
• Provides information on interventions, assessment tools and general knowledge (e.g., mechanisms)
• Target group: Professionals, politicians and policy makers
• Not recommendations
• Arranged in a simple, straightforward format reducing the need to conduct literature searches
• Methodology of the systematic review (e.g., Higgins & Green, 2008)
Topics Currently Available on the Website
Target groups
• Child/family
• Social assistance (for welfare recipients)
• Disability
• Addiction
• Elderly
Subheadings
• Interventions (18 descriptions/5 final evaluations)
• Assessment tools (46 descriptions/13 final evaluations)
• General knowledge
• IMS projects
20
The evaluation process
• Selection of interventions to evaluate – starts with discussion in IMS research council: Which interventions should we evaluate? Which outcomes are important? What databases should be searched?
• All relevant studies with a RCT or QE (including data at baseline) are reviewed
• Use of a guide/protocol to assist the review process• Studies are evaluated for to their internal validity (i.e., can we trust
the result?)• Two independent reviewers evaluate each study. Any
disagreements are settled based on consensus with help from a coordinator
22
23
Judging of internal validity
• Selection bias e.g., Are there any important differences between the groups?
• Performance biase.g., Are researchers, participants and data collectors ”blind”?
• Attrition biase.g., Any differences in size and type of attrition between the groups?
• Detection biase.g., Are the outcome measures measured in the same way in all groups?
Potential bias is evaluated within and across each domain of bias• Low risk: plausible bias unlikely seriously alter the results• Unclear risk: plausible bias that raises some doubt about the results• High risk: plausible bias that seriously weakens confidence in the results
(from Higgins & Green, 2008)
Grading of evidence
• The scientific raiting scale is a modified version of a scale developed at California Evidence-Based Clearinghouse for Child Welfare (http://www.cachildwelfareclearinghouse.org/scientific-rating/scale)
• Based on statements in Flay et al., 2005 (Society for Prevention Research, SPR)
• The scale is devided into five grades - A lower score indicates effectiveness and a higher level of research support
• Grading of evidence is based on:– Number of studies with a certain degree of internal validity (i.e., risk of
bias)
– If the practice is evaluated in usual care
– If the effect are sustained over time
– If the practice may cause harm
– If the practice is replicable
24
1. Effective practice with well-supported research evidence
• At least two studies with low risk of bias, in different usual care or practice settings, have found the practice to be superior to treatment as usual (TAU).
• If multiple effectiveness studies have been conducted, the overall weight of the evidence supports the benefit of the practice.
• In at least one study with low risk of bias, the practice has shown to have a sustained effect at least one year beyond the end of treatment.
• There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it.
• The practice has a book, manual, and/or other available writings that specify components of the service and describes how to administer it.
2. Effective practice supported by research evidence
• At least one study with low risk of bias has found the practice to be superior to treatment as usual (TAU).
• If multiple outcome studies (at least with medium risk of bias) have been conducted, the overall weight of evidence supports the benefit of the practice.
• In at least one study with low risk of bias, the practice has shown to have a sustained effect of at least six months beyond the end of treatment.
• There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it.
• The practice has a book, manual, and/or other available writings that specifies the components of the practice protocol and describes how to administer it.
3. Practice with promising research evidence
• At least one study with medium risk of bias has established the practice's benefit over no intervention (or placebo or waiting list) or is found to be comparable to or better than treatment as usual (TAU).
• If multiple effectiveness studies with at least medium risk of bias have been conducted, the overall weight of evidence supports the benefit of the practice.
• There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it.
• The practice has a book, manual, and/or other available writings that specify the components of the practice protocol and describe how to administer it.
4. Practice where the evidence fails to demonstrate effect
• At least two studies with low risk of bias have found that the practice has not resulted in improved outcomes compared to no intervention (e.g., placebo or waiting list), or that the practice is shown to be less effective when compared to treatment as usual.
• If multiple effectiveness studies have been conducted, the overall weight of evidence does not support the benefit of the practice.
• There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it.
• The practice has a book, manual, and/or other available writings that specify the components of the practice protocol and describe how to administer it.
5. Concerning practice
• At least one study of low or medium risk of bias shows that the intervention can cause serious harm, and/or there is a reasonable theoretical basis suggesting that the practice constitutes a risk of harm to those receiving it.
• The practice has a book, manual, and/or other available writings that specify the components of the practice protocol and describe how to administer it.
Beyond the five grades the scientific scale includes an additional category
(not rated - no number is given).
Practice with unknown effect • There is lack of studies with a medium or low risk of bias. • There is no theoretical or empirical basis indicating that the practice
constitutes a substantial risk of harm to those receiving it.• The practice has a book, manual, and/or other available writings that
specifies the components of the practice protocol and describes how to administer it.
Conclusions and summary - Is there a sufficiently robust evidence base to identify good practice?
• What are the strength and weaknesses?
• Are there significant gaps that should be addressed?
• Is there an agreed approach to deciding what counts as evidence?
• Is there an agreed approach for judging the quality of the evidence base?
• Is there a scope for working internationally to strengthen the evidence-base?
IMS practical delivery mechanisms to promote the adoption of good practice
IMS want to reach:
• Social workers• Politicians and decision makers• Authorities• Universities and students • Researchers• Clients and others who are interested
Target groups
• Social work agency managers• Government officials
Parts of our communication strategy
• Value of direct, face-to-face communication
• Web-based services
• Tailored products
• Short courses for social work agency managers about EBP
• Start kit for municipalities for working with EBP
IMS STRATEGI
ETICS
INFORMATION
TOOLS FOR CHANGE
NOT COACHING SPECIFIC METHODS
Target groups
Social workers
Politicians and decision makers
Governments
- - - - - - - -
Universitys
Goals
Increase the interest for EBP
Provide support for decision-making
Implementation knowledge
Support to managers
Long-term partnership
Activities
Lectures
Clearinghouse
Readiness for change
Handbook in leading EBP
Pilot-project
Train the trainers
Ask a researcher
Start-up-package
IMS network for EBP
Organizational Organizational Readiness for Readiness for
ChangeChange
Dwayne SimpsonDwayne Simpson
Anonymous survey to co-workers, managers and clients
Pilot-project Södermalm - Trying different mechanisms to promote EBP
• A steering group and 24 IMS-coodinatiors with special drive• Seminars about EBP and implementation• ORC-survey• Focusing on the managers with seminars about leading EBP
– the importance of the managers– management research– tools from the handbook
• Local seminars about how to find research and assessment tools
Training trainers in ASIAddiction Severity Index
• Big difference when local support in the new method is given and when the managers get’s support and owns the implementation
• 2 persons at IMS, 25 local trainers, 12 county administrative boards and 194 municipalities'
• Combination of IMS knowledge in implementation research and the method ASI and the local knowledge about the practice and support to the managers
• Success factors:– National support– Local competence– Managers in focus– Networking
Start-up- package
Content: EBP – concepts and implications Evidence, ethics and affectivity Risk- and protective factors Assessment tools Research reviews Dissemination EBP and cost effectiveness
Including: DVD-films The book Evidence based
practice in social work The book To change social work PowerPoint material Glossary Information about IMS website
Focused on Managers
Study material as an introduction to EBP
Publications
Collaboration with a publishing house (Gothia förlag)
Adapted to our target group
Reading guidances
IMS-nytt
New layout + more pages
4 numbers annually
Special feature issues
Research and interviews with professionals and clients
Challenges
Challenges
1. Lack of interest among the Swedish social work academia to support an EBP, and train the future generations of social workers accordingly
2. Lack of (inter)national agreement on how to grade evidence
3. Questions on the transportability of evidence-based interventions between countries
4. Lack of reliable and valid measures of quality of services at a local level (in order to motivate change)
5. Lack of an infrastructure for diffusion of innovations, and knowledge transfer
AIDA – acronym used in marketing
A Attention
I Interest
D Desire
A Action
top related