adoption of a clinical innovation: “ best practices for ... · “best practices for concurrent...
TRANSCRIPT
Adoption of a Clinical Innovation
“Best practices for Concurrent Mental Health and Substance Use Disorders”
in Ontario, a One- year Follow Up.
by
Tamara Kennedy-MacDonald
A thesis submitted in conformity with the requirements
for the degree of Master of Science- Health Administration
Graduate Department of Health Policy, Management and Evaluation
University of Toronto
©Copyright by Tamara Kennedy-MacDonald, 2008
ii
Adoption of a Clinical Innovation “Best practices for Concurrent Mental Health and Substance Use Disorders”
in Ontario, a One- year Follow up
Tamara Kennedy-MacDonald
Master of Science- Health Administration
Graduate Department of Health Policy, Management and Evaluation
University of Toronto
2008
Abstract
Objectives: To determine the level of adoption and which characteristics are most strongly
associated with the adoption of a clinical innovation in Mental Health and Addiction Service
organizations in Ontario, one year after dissemination.
Methods: This cross-sectional study included a voluntary quantitative mail-out survey, using
a self administered questionnaire that was sent to 260 mental health and addiction service
organizations in Ontario. Linear regression analysis was conducted to identify significant
predictors of the overall adoption-decision of the best practice recommendations.
Results: Individuals‟ tenure within the organization and the provision of screening
(organizational variable) was identified to be predictors of adoption for the organization.
Conclusions: The results of this study demonstrated the majority of the organizations are on
the path towards a finale decision of adopting or rejecting the recommendations. The results
also demonstrate the importance of individual characteristics and organizational
characteristics in identifying predictors of adoption.
iii
Acknowledgements
I am indebted to Dr. Rhonda Cockerill and Dr. Jan Barnsley, who were my thesis advisor and
thesis committee member, respectively, during this journey through the graduate pro gram at
the University of Toronto. They gave me the clarity and focus to see what was important in
this process and how to set the right path and the steps I needed to do to accomplish it. I am
also grateful for their patience, when the bumps in the road came along the way with my
family and my priorities had to shift for a good part of the year in the final stretch.
I wish to thank Dale Butterill and Dr. Paula Goering, whose guidance, support and mentoring
sparked the fires within me on knowledge transfer theory and bringing research into practice.
They have been empathic colleagues as well as knowledgeable teachers. The groundwork that
they have done in mental health and addiction services research regarding knowledge transfer
has been an inspiration.
I would also like to thank Dr. Brian Rush- who gave me the confidence that I was on the right
path in evaluation science and provided deeper insight into evaluating best practices. His
honesty, professionalism, approachability, and common- sense approach set a definite
example for me to live by in the evaluation community.
iv
Christine Bois, Betty Dondertman, Laura Pinto and my colleagues, at the Centre for
Addiction and Mental Health (CAMH); without your support, sage advice and understanding,
I could not have completed this research.
Christine; for providing me access to do the follow-up study, supporting me during the survey
process and explaining the background information regarding the innovation. She made my
learning curve of mental health and substance use treatments a little less challenging.
Betty; for allowing those intangible resources to be available for the research during my time
at CAMH and in adjusting my workload during those crucial moments. No one could have
had a more supportive and understanding manager.
Laura; for her invaluable wisdom, input, guidance, grace, knowledge, support and overall
friendship. Without your shoulder to lean on, I do not think this would be possible or I would
have lost my sanity and my sense of humor.
To my family and friends: especially to my daughters Sarah and Julia. Thank you for your
patience and understanding how important education is to me. You have never known me not
to be working on a paper and have sacrificed much for me, over these past thirteen years and
especially during the past four years, in order for me to do this on a part time basis. From the
time that Sarah was born- I have always been in school, working full time, school part time
and many times- a part time mother. I have often wondered if I did do the right thing, early in
your life and looking back, I think I did, by providing you an example of what you can
achieve, if you believe in yourself.
v
To Alex Glover; my best friend, and my perpetual shoulder to lean on. You were my
emotional rock during this whole journey and more. I only hope that I gave back to you in
your own graduate studies, as much support as you have provided to me during these past few
years.
To my parents; thank you for their countless hours in helping with raising the girls. Though it
was tough at times for me to ask for assistance, I am very grateful you understood why this
was so important to me.
Finally, I am grateful to all of the individuals and organizations who participated in this
research and who provided encouraging feedback along the way.
vi
List of Tables
Table 1 Varimax Rotated Solution Factors
Table 2 Variance for Varimax Rotation
Table 3 Varimax Rotated Solution Factors- Presentation of the
recommendations/innovation
Table 4 Variance Varimax Rotation- Presentation of the recommendations/innovation
Table 5 Reliability of Dimensions Identified
Table 6 Descriptive Statistics of Independent Variables
Table 7 Descriptives of Best Practice recommendations
Table 8 Significant Correlations between Adoption and Independent Variables
Table 9 T-test Categorical Independent Variables Levene‟s Test of Equality of
Variances
Table 10 Stepwise Regression Model Summary
Table 11 Stepwise Regression Model Coefficients Dependent Variable Adoption
Table 12 Stepwise Regression Model F-test for Significance
vii
List of Figures
Figure 1 Rogers‟ S-curve
Figure 2 Adopter Categorization
Figure 3 Scree Plot of Varimax Rotation- Adoption
Figure 4 Scree Plot of Varimax Rotation- Peception of Research and Changing Clinical
Practices
Figure 5 Scree Plot of Varimax Rotation- Presentation of the
recommendations/innovation
Figure 6 Histogram of the Distribution of Adoption
viii
List of Appendices
Appendix 1 Questionnaire
Appendix 2 Variable Definitions and Measures
Appendix 3 Ethics Committee Approval
Appendix 4 Introduction-Cover Letter for Participants
Appendix 5 Information Sheet for Participants
Appendix 6 Questionnaire “Appendix A” of survey package
ix
Table of Contents
1.0 Introduction...........................................................................................................................1 1.1 Background ...............................................................................................................1
2.0 Literature Review .................................................................................................................8 2.1 Diffusion of Innovation and Adoption Theory Overview ........................................8 2.2 Innovation Adoption Variables...............................................................................13
2.2.1 Attributes of Innovation.......................................................................................13 2.2.2 Communication Channels....................................................................................19 2.2.3 Time .....................................................................................................................20
2.2.4 Social System.......................................................................................................31 2.2.5 Independent Characteristics .................................................................................34
2.3 Adoption Studies in Mental Health and Substance Use Treatment Settings ..........42 2.4 Criticisms of Rogers' Diffusion of Innovation........................................................44
3.0 Research Methodology .......................................................................................................46
3.1 Setting and Sample .................................................................................................47 3.2 Measures .................................................................................................................49
3.3 Instruments and Scales ...........................................................................................54 3.4 Quantitative Analysis..............................................................................................56
4.0 Results.................................................................................................................................67
5.0 Discussion ...........................................................................................................................89 5.1 Degree of Adoption within Organizations..............................................................90 5.2 Characteristics and Adoption..................................................................................91
5.3 Implications ............................................................................................................94 5.4 Recommendations for Future Research ..................................................................94
5.5 Ethical Issues and Considerations ..........................................................................96 5.6 Limitations of the Study .........................................................................................96
6.0 Conclusions.........................................................................................................................99
7.0 References.........................................................................................................................102 8.0 Appendices .......................................................................................................................107
Appendix 1: Questionnaire .........................................................................................107 Appendix 2: Variable Definitions and Measures........................................................120 Appendix 3: Ethics Committee Approval...................................................................130
Appendix 4: Introduction-Cover Letter for Participants ............................................131 Appendix 5: Information Sheet for Participants .........................................................132 Appendix 6: Questionnaire "Appendix A" for Survey Package .................................134
1
1.0 Introduction
The objective of this thesis is to examine the adoption of a clinical innovation in Mental
Health and Addiction Service organizations in Ontario, one year after dissemination.
The main research questions are:
1. To what degree did the organizations make a considered adoption decision
regarding the innovation?
2. What were the organizational, individual, environmental, and innovation
characteristics associated with the adoption decision?
This research will build upon previous work on the adoption of innovations in healthcare
organizations. The data produced by this research will identify variables likely to affect
adoption. Identification of these variables is important to understanding what contributes to
adoption capacity and what factors or predictors differentiate between good organizational
adoption and poor organizational adoption. The variables can explain the differences in the
effectiveness of the processes and mechanisms by which new knowledge is applied to
decision making (about innovation adoption) in health care organizations.
Though there is considerable research and literature on factors associated with
implementation of innovations in business settings (Frambach & Schillewaert, 2002), little
research has examined factors, and especially organizational factors, that may facilitate or
hinder the implementation of evidence based health innovations in mental health settings.
2
Furthermore, it is not often applied to mental health and substance abuse treatment settings
(Gotham, 2004).
The clinical innovation studied is the set of recommendations found in the document, “Best
practices for Concurrent Mental Health and Substance Use Disorders”. This document is a
synthesis of research published by Health Canada, intended to be a resource to managers and
staff of mental health, substance abuse and integrated mental health/substance abuse
services, as well as individual practitioners in the community (Health Canada 2002). The
reported research contained within the best practices document was developed by a
multidisciplinary team led by Rush and was the culmination of 12 months of data gathering,
research synthesis and consensus building. (Rush, 2003).
The purpose of the innovation was to identify best practices, defined as the most updated
synthesis and highest quality of research information that is available, combined with the
advice and input of experts and other key stakeholders, in the field of concurrent mental
health and substance use disorders. The project was initiated by Health Canada as a part of a
research agenda developed by the Federal/Provincial/Territorial Committee on Alcohol and
Other Drug Issues.
The report was developed by a multidisciplinary team led by the author (Rush) and was the
culmination of more than 12 months of data gathering, research synthesis and consensus
building. It concluded with “best practice” recommendations for improved service delivery
and better integration across the specialized sectors of addiction and mental health service
delivery (Rush, 2003). “Best practice” was recommendations that were defined on the basis
3
of scientific evidence and or expert consensus, where a combination of these methods was
employed (Health Canada, 2001). These recommendations represented a movement
towards the “no wrong door” approach, in which an organization based in either mental
health practice or substance use treatment practice would screen, assess and treat all clients,
regardless of whether the health concern was mental health or substance use.
The project was funded by Health Canada as a response to the emergence of concurrent
disorders as an important issue for mental health and addiction services programs over the
past few decades. High prevalence of co-morbidity and its implications are important for
the course, cost and outcome of treatment and other support services. This innovation
synthesized the research of the best practices of a provider population, which aimed to have
significant impact at the program, system and policy level in Ontario. As the knowledge in
concurrent disorders is rapidly expanding, the innovation integrates and complements
considerable research and development that has been done with respect to concurrent
disorders.
The best practice recommendations address five areas:
Screening for substance use and mental health disorders
Assessing people with concurrent disorders
Treatment and support of people with concurrent disorders
Service and system level delivery implications
Continued research in regards to attitudes of health care policy makers, planners
and providers and consumer health outcomes
4
The innovation provides an updated synthesis of the research and offers specific
recommendations for the screening, assessment and treatment/support of this in-need
population based on the highest quality research information available. The research
synthesis was combined with the advice and input of experts, consumers and other key
stakeholders in the field. The research for this thesis is not intended to critique the best
practice recommendations themselves but rather to examine the level of adoption and the
characteristics that led to the adoption.
The scope of the innovation document is limited to recommendations for best practice either
within or across specialized substance abuse and mental health programs and does not
include early identification or prevention of co-occurring substance use and mental health
disorders, neither does it detail how to implement or monitor such recommendations.
This study focuses on the process of making an adoption decision about this innovation
(Health Canada best practices recommendations) and the variables associated with that
decision.
Rogers‟s Diffusion of Innovations framework is used in this study in order to understand the
adoption of the innovation during the year after its introduction to the mental health and
addiction services field. The framework is helpful when determining the adoption of specific
clinical behaviors and when deciding which components will require additional effort if
diffusion or dissemination is to occur. The framework includes an innovation-adoption
(decision) process, which individuals and other decision makers go through as they gain
5
initial knowledge about an innovation, form their attitude about the innovation, make a
decision to adopt or reject the innovation, implement the new idea and finally confirm their
decision. Rogers (2003) suggests that this process addresses the uncertainty involved in
deciding whether a new idea, such as a new practice, and the level of adoption into clinical
practice.
As described by Rogers (2003), an innovation is an idea, practice or object perceived as new
by an individual or other unit of adoption. The newness of the innovation may be expressed
in terms of knowledge, persuasion or a decision to adopt. Though an organization may have
already been using certain recommendations (e.g. mental health organizations using Level
1/Level 2 screening tools for mental health and addiction treatment organizations using
mental health principles), the best practice recommendations as a whole would be
considered new and hence would be considered an “innovation”. This is because the
document represented the first of its kind in Canada, a synthesis of both mental health and
addiction services research in one document and with recommendations working towards a
“no wrong door” approach.
The research is important as though diffusion has been studied in other fields (Gotham,
2004), it is not often applied to mental health and substance use treatment settings. This is
because, according to Health Canada, within the last two decades, the co-occurrence of
addiction and mental health problems among people seeking treatment and support is an
important issue for those who plan and fund mental health and substance abuse programs as
well as for health care and social support providers who provide direct service. These
6
concerns are based upon the following rationale: 1) the prevalence of co-morbidity is high in
the general and treatment seeking populations and has largely been ignored in planning,
implementing and evaluating both mental health and substance abuse services 2) substance
abuse and mental health co-morbidity changes the course, cost and outcome of care and
presents significantly challenges for screening, assessment, treatment/support and outcome
monitoring and 3) substance abuse and mental health services in the community have
typically worked in isolation and often from competing perspectives (Health Canada, 2001).
Furthermore, it was identified within the best practice document, that there is a current high
interest in best practice guidelines, as it relates to larger trends in healthcare to improve
consumer outcomes and reduce variation in care and associated costs. Hence this research
will further add to the understanding of the adoption of an innovation for health care
organizations, especially for the mental health and substance use community, and provide
insight into organizational characteristics to be considered. The findings of such research on
diffusion of innovations can assist in bridging research-practice gaps that lead to patients
receiving services whose effectiveness have not been assessed or to patients not receiving
the most effective form of assessment and treatment. Better understanding of these
characteristics, will further the development of optimal implementation strategies that are
tailored to specific organizational and service contexts.
Though the literature is growing on how health care organizations adopt new
guidelines/innovations, more research is needed, especially to look at health care
organizations that specialize in mental health and addiction services.
7
The following chapter provides a literature review on diffusion theory and how it relates to
health care individuals and organizations and is followed by chapters on research
methodology, results, and discussion.
8
2.0 Literature Review
2.1 Diffusion of Innovation and Adoption Theory Overview
Rogers defines diffusion as the process in which an innovation is communicated through
certain channels over time among the members of a social system. Diffusion is also a type
of social change by which alteration occurs within the social system, such as when new
ideas are created, diffused, adopted or rejected.
According to Rogers , the innovation-adoption process begins with an individual or
organization becoming aware of an innovation and being interested in understanding how it
functions. This process can be contrasted with the diffusion process he examines. Diffusion
is a process in which an innovation is communicated through certain channels over time,
among the members of a social system. It is a special type of communication in which the
messages are about a new idea. Diffusion is a form of dissemination that is directed and
managed. Once an innovation is communicated to an individual or organization, the process
of innovation adoption can occur.
Rogers further identifies that the perceived newness of an innovation and the uncertainty
associated with it is a distinctive aspect of innovation decision-making in comparison with
other types of decision-making. The framework includes a consideration of characteristics of
the innovation, type of communication and other (individual, organization and environment)
variables. It is a linear model of diffusion and suggests a “stage like” approach to the
adoption process (Fitzgerald et al, 2003). This stage- like approach occurs over time and
9
consists of five stages, which can be applied to individuals and other decision-making units,
such as organizations.
The rate of adoption is defined as the relative speed with which members of a social system
adopt an innovation. It is a continuum from those who embrace the innovation at its
introduction to those who do not adopt the innovation at all. Adoption is one possible end
product of the innovation-decision process. Adoption is a decision regarding the use of an
innovation as the best course of action available. The alternate decision to adoption is
rejection, where the decision is made not to adopt the innovation. One reason for a decision
of not adopt the innovation may be due to the innovation not being a good innovation to the
potential adopters.
Several factors affect the diffusion process positively or negatively, and sometimes changes
do not occur because health professionals do not accept the innovation or insufficient
financial resources are available to implement the innovation (Bartholomew, KL et al,
2001). Most research on innovations in health care has been focused on individual doctors
working independently, within clinical practice guidelines (Flueren, 2004). Less is known
about the determinants of innovations in larger health care organizations, which may be
different from those of innovations for individual health care professionals. Although the
number of studies of innovation processes has greatly increased over the last 15 years (Grol,
2001), little is known about the factors or predictors of successful implementation to health
care organizations (Bartholomew, KL et al, 2001).
10
Most research on innovations in health has primarily focused on physicians and nurses
versus other health care professionals, especially in larger hea lth care organizations.
(Fleuren, M, 2004). Corrigan (2001) has discussed that research-guided mental health
practices that are widely adopted tend to be adopted at a slow pace, despite the amount of
resources invested into the dissemination and adoption of the practices (innovation).
Recognizing the need of such practices has been identified by Gotham, 2004, where she
states that as researchers continue to look for epidemiological data and clinical consensus
about new treatment needs, attention has recently been directed towards “co-occurring
disorders” (concurrent disorders). Studies indicate that up to 50% of mental health or
substance abuse clients have both types of disorders and that clinical evidence suggests that
clients with co-occurring disorders often fail when forced to seek separate treatment for their
interrelated problems. The report of “Providing integrated services for persons with co-
occurring mental health and substance abuse disorders”, authored by Dr. R.E. Drake et al in
2001, for the Arizona Department of Health Services, includes a developed integrated
treatment strategy for concurrent disorders which demonstrates new treatment needs that
continue to be recognized. Miller et al (2006) found that the standard of health professionals
to deliver treatment with the best current evidence of treatment has been slow to emerge in
substance abuse treatment.
According to Berta et al (2005) there is widespread acknowledgement of a gap between
research and practice that has contributed to the development of best practice guidelines and
the uptake and implementation of such guidelines.
11
This gap is emphasized by the following key observations:
• Knowledge generation and dissemination are distinct activities and not necessarily
carried out by the same individuals
• Knowledge must be actionable to be useful
• Challenges in knowledge transfer and innovation adoption are as much
organizational as they are individual
In Figure 1, Rogers‟ S-curve illustrates how innovations spread through a population. It
shows that any innovation is first adopted by a few people (or organizations). As more
people use it, others see it in use, and if the innovation is better than what was used before,
they begin to use it as well. Once the diffusion reaches a level of cr itical mass, it proceeds
rapidly. At some point the innovation reaches a part of the population that is less likely to
adopt it and diffusion slows to a point of saturation. A typical innovation per Rogers,
includes items such as technology, new products that are designed for instrumental action
that reduces the uncertainty in the cause-effect relationship involved in achieving a desired
outcome. Such examples would include VCRs, CDs and personal computers.
The bell-curve (shown within Figure 1- lower shape) shows the typical adoption of an
innovation over time when plotted on a frequency basis.
12
Figure 1; Bell-shaped frequency curve and S-shaped cumulative curve for adopter
distribution (Source: Rogers, 2003.)
Rogers used a normal distribution as the basis for categorizing the degree of
"innovativeness" for adopters. He identified five categories of adopters and the approximate
percentage of their numbers in any given population. These categories will be described
more fully in the literature review section of this paper.
It is important to note that terminology related to diffusion is varied and inconsistent (e.g.
dissemination can be confused with implementation or the actual use of a treatment).
Potential adopters use the terms “adoption”, “diffusion” and “knowledge utilization”
interchangeably. This can lead to confusion about how research is best put into practice and
about which activities match the use of the „research to practice‟ model. This confusion in
communication may impede adoption. An example of this confusion is seen in the research
by Gotham et al, who found that diffusion of innovations is also referred to as technology
13
transfer, which bridges the research/practice gap or knowledge utilization, and is sometimes
confused with translational research (moving from basic to applied science). Dobbins et al
(2001) examined definitions used in knowledge utilization; her position is that research
evidence is an innovation and that the knowledge gained from diffusion research is
applicable in understanding the process of research (knowledge) utilization. Backer (2000)
also identifies other terms that essentially mean the same to include: technology transfer,
dissemination and utilization, applied dissemination, effective dissemination, research
utilization, knowledge transfer and knowledge development and application.
Thus for the basis of this research, the recommendations within the best practices document
will represent as an innovation and that the framework of Diffusion of Innovation applies to
its use in the mental health and substance use treatment practice community.
2.2 Innovation Adoption Variables
In the diffusion of innovation framework, there are five main categories of variables that
affect the adoption of innovation: 1. Attributes of the innovation, 2. Communication
channels, 3. Time, 4. Social system, and 5. Independent variables (e.g., individual,
organizational and environmental variables).
2.2.1 Attributes of the innovation
An innovation is an idea, practice, or object that is perceived as new by an individual or
other unit of adoption. According to Rogers‟ model, the attributes of an innovation, as
perceived by the members of a social system, determine its rate of adoption. These attributes
can help explain why certain innovations spread more quickly than others. The attributes
14
that determine an innovation's rate of adoption are: (1) relative advantage, (2) compatibility,
(3) complexity, (4) trialability, (5) observability, and (6) innovation type.
(1) Relative advantage is the degree to which a new idea is perceived as superior to the
existing practice it replaces. This could be presented in economic terms or, in the case of co-
occurring mental health and substance use treatment, it could mean better outcomes,
improved client care or more services being offered. According to the theory, the relative
advantage of an innovation as perceived by members of a social system is positively related
to its rate of adoption. Current research shows that decisions about implementing best-
evidence practice are driven not only by patient welfare, but also by the interplay between
the interests of the patient, the clinician and the healthcare system (Sanson-Fisher, 2004).
The existing literature on diffusion of innovation has found relative advantage to be one of
the strongest predictors of an innovation‟s rate of adoption, especially if it is viewed as a
preventative innovation (e.g. screening) that will lower the probability of the health
condition to worsen and thus be less taxing on the health system and community overall
(Rogers, 2003). According to Henggeler et al (2002), the perceived advantages can vary by
stakeholder. Henggeler presents the example of a funder of an expensive mental health
service such as residential treatment, who might view cost-effective community-based
alternatives (innovation) as highly advantageous, whereas the providers, who prefer to work
in an institutional context, might view the innovation as disadvantageous. However, it does
not matter so much if an innovation has a great deal of objective advantage. What does
matter is whether an individual/adoption unit, perceives the innovation as advantageous. The
15
greater the perceived relative advantage of an innovation, the more rapid its rate of adoption
will be.
(2) Compatibility is the degree to which an individual perceives an innovation as similar to
previous experience, beliefs or values. If the idea is more compatible with the potential
adopter and fits more closely with the individual‟s situation, or the innovation is more
consistent with previous practice, it is more likely that the innovation will be adopted. This
can happen on three levels: a) socio-cultural values and beliefs b) previously introduced
ideas and c) the need for the innovation. According to Rogers, an innovation‟s
incompatibility with the individual or organization‟s “cultural” values can block its
adoption. This has been demonstrated in various anthropological and sociological settings
(Rogers, 2003). For a health care related setting, innovation must address an issue that
clinicians or others perceive to be a problem. (Sanson-Fisher, 2004)
Another aspect to consider is the compatibility of an innovation with a “preceding” idea,
which can either speed up or retard its adoption. The reason, Rogers explains, is that
individuals cannot deal with an innovation except on the basis of the familiar. Previous
practice provides a standard against which an innovation can be interpreted, thus decreasing
its uncertainty. Another challenge is the situation where the new idea is completely
congruent with the existing practice; there would be no innovation to the potential adopters.
The adoption of an incompatible innovation often requires the prior adoption of a new value
system, which is a relatively slow process, because the more compatible an innovation is,
16
the less of a change in behavior the adoption represents. Research suggests that
compatibility may be somewhat less important than other factors in predicting the rate of
adoption (Rogers, 2003). The degree to which members of the social systems perceive the
innovation as compatible is positively related to its rate of adoption (Rogers, 2003).
(3) Complexity is the degree to which a new idea is perceived as relatively difficult to
understand and use. Some innovations are clear in their meaning and some are not, and
simple innovations are more likely to be adopted than complex innovations. Though
complexity may not be as important as relative advantage or compatibility, it can be an
important barrier to adoption. Some innovations are readily understood by most members of
a social system; others are more complicated and will be adopted more slowly. New ideas
that are simpler to understand are adopted more rapidly than innovations that require the
adopter to develop new skills and understandings. This characteristic can be explained with
the following example: Most mental health services are committed to overcoming barriers to
service access for economically disadvantaged families. A simple and useful innovation to
address difficulties in service access would be for the mental health provider organization to
offer evening hours, or more evening hours, for working families. But an alternative service
could be a home-based model of service delivery (e.g. treatment provided in the home and
community settings, with 24-hour availability of clinicians). The latter would be an example
of a strategy for addressing barriers to service access that, although highly effective, is also
more complex, thus it would be expected that most clinicians would be more supportive of
the “evening hour” strategy. Therefore, the complexity of an innovation, as perceived by the
17
users or members of the “social system”, would have a negative relationship to the rate of its
adoption (Rogers, 2003).
(4) Trialability is the degree to which an innovation can be divided for experimental use on
a limited basis. New ideas that can be tried out in small increments are generally more
rapidly adopted than innovations that cannot be. However, not all innovations can be tried
out in small divisible increments. If an innovation can be designed so that it may be tried
more easily, it will have a greater rate of adoption. The ability to test a potential innovation
on a limited basis allows clinicians to explore the implementation of the innovation, its
acceptability to the patients and the potential outcomes. In addition, Rogers suggests that an
organization that undertakes a limited trial of the innovation promotes faith that the evidence
is correct and that implementation is possible. Thus triability of an innovation, as perceived
by the members of a social system, is positively related to its rate of adoption (Rogers,
2003).
(5) Observability is the degree to which others can easily see a new idea. When the
innovation is more readily observable by others, it will diffuse more rapidly than its
counterparts that are not. Some ideas are easily observed and communicated to others,
whereas others are not, and the challenge then is to further describe the innovation to others
in the social system. The visibility of an innovation can stimulate peer discussion, as
colleagues of a clinician adopting a new innovation will often request information about it.
(Sanson-Fisher, 2004). If a respected and influential adopter argues for the innovation and
can demonstrate the positive outcome of the innovation, it is likely to have a positive impact
18
upon adoption rates (Denis et al, 2002). Thus, observability of an innovation, as members of
a social system perceive it, is positively related to its rate of adoption.
A final attribute to consider is the Innovation type. The type of innovation may have
significant impact on the likelihood and rate of adoption or rejection of the innovation by the
adopting units. According to Rogers (2003) and Damanpour (1991), though there are several
types of innovations, two types of innovations will decrease the rate of adoption, resulting in
a moderating effect. These two are a) technological innovations and b) administrative
innovations. Technological innovations are defined as products, services, and production
processes that are directly related to the management of basic work activities (Damanpour,
1991). A technological innovation usually has at least some degree of benefit for its
potential adopters, but this advantage is not always clear to those intended adopters (Rogers,
2003). Examples of technological innovations in health care are equipment and clinical
procedures. Administrative innovations include organizational structures and processes that
are directly related to the management of basic work activities. Examples of administrative
innovations would be research evidence.
In summary, Rogers‟ model states that innovations perceived by individuals as having
greater relative advantage, compatibility, trialability, observability, and less complexity will
be adopted more rapidly than other innovations. Based on these attributes, the best type of
“innovation” for rapid adoption would be an evidence-based treatment that was simple,
similar with previous practice, had clear advantage, could be tried out temporarily and was
readily observable.
19
2.2.2 Communication channel
The second main category of variable in the diffusion of new ideas is the communication
channel. Communication is the process by which participants create and share information
with one another in order to reach a mutual understanding. A communication channel is the
means by which messages get from one individual to another. Mass media channels are
more effective in creating knowledge of innovations, whereas interpersonal channels are
more effective in forming and changing attitudes toward a new idea, and thus in influencing
the decision to adopt or reject a new idea. The literature furthermore suggests that o f the
interpersonal channels, the most effective communication strategy is the face-to-face
exchange (Sanson-Fisher, 2004), as it provides an opportunity to tailor information to
recipients, e.g., it allows the advocate of the change to explore the change and, if necessary,
tailor the message about why a shift in clinical behavior should occur. Interpersonal
communication is usually more effective when there is a high degree of professional
resemblance between the individual attempting to introduce the innovation and the recipient
(Rogers, 2003).
Dobbins et al suggests there is good evidence that successful dissemination strategies from
health researchers to health service users should involve a personal, one-to-one contact with
the intended audience, which will be more effective in facilitating research utilization and
hence adoption, versus using group-based strategies (e.g. continuing education
workshops/conferences) (Dobbins, 2002).
20
Healthcare professionals are bombarded with information from many sources includes
research publications, databases, mass media, forums, and attendance at
workshops/conferences. As interpersonal communication is usually more effective when
there is a high degree of professionalism between the messenger of the innovation and the
recipient, it is important to remember the means of the message when considering rates of
adoption (Sanson-Fisher, 2004).
Most individuals in a leadership position within an organization evaluate an innovation not
on the basis of scientific research by experts, but through the subjective evaluations of near-
peers who have adopted the innovation (Rogers, 2003). This dependence on the experience
of near-peers suggests that the center of the diffusion process consists of modeling and
imitation by potential adopters of their network partners who have previously adopted.
Diffusion is considered to be a very social process that involves interpersonal
communication relationships (Rogers, 2003).
2.2.3 Time
According to Rogers, the third main variable in the diffusion of new ideas is time. The time
dimension is involved in diffusion in three ways: a) the innovation-adoption process, b)
innovativeness and c) the rate of adoption.
a) Innovation-Adoption Process
The innovation-adoption process is the mental process through which an individual (or other
decision-making unit) passes from first knowledge of an innovation to forming an attitude
21
toward the innovation, to a decision to adopt or reject, to implementation of the new idea,
and to confirmation of this decision. An individual seeks information at various stages in the
innovation-adoption process in order to decrease uncertainty about the innovation's expected
consequences.
The five stages of the innovation-adoption process occur over time and include the
following: 1) knowledge 2) persuasion 3) decision 4) implementation and 5) confirmation
(Rogers, 2003), as described below.
Stage 1) Knowledge is gained when an individual or other decision making unit is
made aware of the existence of the innovation and develops an understanding of how
the innovation is to work. Knowledge is the first step for an individual or an
organization in becoming aware of an innovation and in being interested in
understanding how it functions (Rogers, 2003). Rogers describes three types of
knowledge: 1) awareness-knowledge, 2) how-to knowledge and 3) principles
knowledge. Awareness-knowledge is defined as simply knowing that an innovation
exists. Most change agents (e.g. Health Canada) concentrate their efforts on creating
this type of knowledge. How-to knowledge involves understanding how to use the
innovation correctly. In the case of best practices being recommended, a large
amount of how-to knowledge is required. Rogers found that when an adequate level
of how-to knowledge was not obtained before and during the adoption of an
innovation, rejection was the result. Principles-knowledge is the third type of
knowledge, which deals with an individual understanding the underlying principles
of how an innovation works. The example that Rogers uses is germ theory, which
22
supports the rationale of water boiling, vaccinations and latrines in sanitation and
health campaigns. However though it is possible to adopt an innovation without
principles-knowledge, there is the danger of misusing a new idea, which can result in
discontinuance. In the cases of where an innovation is adopted without principles-
knowledge, the understanding of the innovation facilitates the ability of an individual
to evaluate the effectiveness of an innovation. Rogers further suggests that change
agents could be most influential in the innovation-adoption process if they
concentrated on how-to knowledge. Unfortunately for most individuals and
organizations, this is one of the most challenging tasks when implementing practice
change, likely due to limited resources.
Stage 2) Persuasion is when an individual forms either a favorable or unfavorable
attitude towards the innovation. Attitudes are formed when the
individual/organization seeks to identify the consequences associated with adopting
or not adopting the innovation (Warner, 1975). According to Rogers, attitude is a
relative organization of an individual‟s beliefs about an innovation that predisposes
his or her actions. This is the state, along with the decision stages, where an
individual will seek innovation evaluation information, messages that reduce
uncertainty about an innovation‟s expected consequences. Rogers further states that
this type of information is usually sought from peers who have previously adopted
the innovation, and hence the motivation to adopt increases.
23
Stage 3) Decision stage occurs when an individual either adopts or rejects the
innovation. It is the step in the process where the individual/organization engages in
activities that lead to a choice to adopt or reject an innovation. Adoption would be
the decision to make full use of the innovation, whereas rejection would be the
decision not to adopt an innovation. Most individuals will not usually adopt an
innovation without first trying it for a short time period, to determine its usefulness
in their own situation. This trialability phase is an important part of the decision to
adopt. However for some innovations it is not feasible to trial and hence will be
adopted or rejected in total. The innovations that can be partitioned for trial are
generally adopted more rapidly (Rogers, 2003).
Stage 4) Implementation is the stage when the decision maker puts the innovation to
use. Until the implementation stage, the innovation-decision process has been only a
thought process. Implementation represents when the behavior change, as the
innovation, is put into practice. This includes activities in place to make the
adoption (i.e. practice change) occur. Implementation usually follows the decision
stage directly unless logistics or resources delay it. Though the individual has made
the decision to adopt, there is some degree of uncertainty about the expected
outcome of the innovation.
For organizations, the problems of implementation are usually more serious. In an
organizational setting, a number of individuals are usually involved in the
innovation-decision process and the implementers are often a different set of people
24
from the decision makers (Rogers, 2003). The organizational structure that provides
the stability and continuity to an organization may also resist the implementation of
an innovation (Rogers, 2003). As a result, the implementation stage may continue
for a lengthy period of time, depending on the nature of the innovation. However, a
tipping point is reached where the innovation becomes institutionalized as a regular
part of the adopter‟s ongoing operations (Rogers, 2003).
Stage 5) Confirmation stage occurs when an individual seeks reinforcement of an
innovation-decision already made, or reverses a previous decision to adopt or reject
the innovation if exposed to conflicting messages about the innovation (dissonance).
Each stage in the innovation-decision process is a potential rejection point. One can
gain awareness of an innovation in the knowledge stage, and then simply forget
about it. Dissonance is an uncomfortable state of mind that an individual seeks to
reduce or eliminate during the confirmation stage of the innovation-decision process.
Examples cited by Rogers include 1) when an individual becomes aware of a need
and seeks information about an innovation to meet this need; 2) when the individual
knows about a new idea and has favorable attitude towards it but has not adopted; 3)
after the innovation-decision to implement an innovation when the individual secures
further information that persuades themselves that he/she should have not adopted,
which would result in discontinuance. Discontinuance is the occurrence of rejection,
after a prior decision to adopt the innovation.
25
To summarize, the innovation-adoption process is considered to be an information seeking
and information processing activity, in which the decision maker obtains information in
order to gradually decrease uncertainty about the innovation. This process can either lead
the decision maker to adoption, where there is full use of the innovation, or rejection, where
the decision is made not to adopt the innovation. The process also proceeds in a time-
ordered sequence, although exceptions to the standard sequence of the five stages may occur
with some decision makers under some conditions (e.g. when a decision maker is ordered to
adopt by an authority figure) (Rogers, 2003). The length of time required to complete the
innovation-decision process can vary, with some individuals requiring many years to adopt
an innovation, while other people change more rapidly. However, when an innovation-
decision is made by a system, rather than by an individual, the decision process is more
complicated because of the number of individuals involved within the system or
organization.
b) Innovativeness
The second way in which time is involved in Rogers‟ model of diffusion is in the
innovativeness of an individual or other unit of adoption. Innovativeness is the degree to
which an individual or other unit of adoption is relatively earlier in adopting new ideas than
other members of a social system. According to Rogers and others, (Haider, 2004) (Rogers,
2003) (CHF, 2002), there are five adopter categories, or classifications of the members of a
social system on the basis on their innovativeness: (1) innovators, (2) early adopters, (3)
early majority, (4) late majority, and (5) laggards, as described below.
26
(1) Innovators (Venturesome) are the first members of a group to adopt a new
innovation and they represent 2.5% of adopters. The characteristics of innovators
include being more adventurous, cosmopolitan, educated and able to cope with a
high degree of uncertainty. Communication patterns and friendships among a clique
of innovators are common, even though the geographical distance between the
innovators may be considerable. The ability to understand and apply complex
technical knowledge is also needed. The innovator must be able to cope with a high
degree of uncertainty about an innovation at the time of adoption. Although an
innovator may not be respected by the other members of a social system, the
innovator plays an important role in the diffusion process: That of launching the new
idea in the system by importing the innovation from outside of the system's
boundaries. They serve as the gatekeepers of an innovation to the population that
they lead.
(2) Early Adopters (Respectable) represent 13.5% of adopters. These are individuals
who are educated but less cosmopolitan and less able to deal with uncertainty than
are innovators. These individuals are considered to be well-respected opinion
leaders and well integrated in the social system. As a result, they are the greatest
opinion leaders within most social systems, and will demonstrate success with the
innovation. Whereas innovators are cosmopolites; individuals who have had
exposures or experiences outside of their local setting, early adopters are „localites‟;
whose experiences remain at a local level, with little exposure to d ifferent
experiences. This adopter category, more than any other, has the greatest degree of
opinion leadership in most systems. Potential adopters look to early adopters for
27
advice and information about the innovation. Early adopters are generally sought by
change agents as local missionaries for speeding the diffusion process. Because early
adopters are not too far ahead of the average individual in innovativeness, they serve
as a role-model for many other members of a social system. The early adopter is
respected by his or her peers, and is the embodiment of successful, discrete use of
new ideas. The early adopter knows that to continue to earn the esteem of colleagues
and to maintain a central position in the communication networks of the system he or
she must make judicious innovation-decisions. The early adopter decreases
uncertainty about a new idea to others in the system by adopting it and then
conveying a subjective evaluation of the innovation to near-peers through
interpersonal networks.
(3) Early Majority (Deliberate) represents 34% of adopters. These individuals would
rather follow than lead, such that they interact frequently with peers, but are not
often found holding leadership positions. As the link between very early adopters
and people late to adopt, early majority adopters also play an important part in the
diffusion process. Early majority are deliberate, and are highly interconnected within
a peer system. The early majority are one of the two majority adopter categories,
making up one-third of the members of a system. The early majority may deliberate
for some time before completely adopting a new idea. "Be not the first by which the
new is tried, nor the last to lay the old aside," fits the thinking of the early majority
(Rogers, 2003). Rogers further states that adopters have more contact with non-local
professionals or organizations and are more information seeking. In a study of
28
psychologists, Beutler et al (1995) found that clinicians preferred to receive research
information through clinical newsletters, national conferences and books, whereas
researchers preferred to receive clinical information through research articles,
national conferences or colleagues.
(4) Late Majority Adopters (Skeptical) represents 34% of adopters. These
individuals will adopt new ideas just after the early majority. Late majority adopters
tend to be skeptical about new behaviors and require peer pressure to adopt them.
They are responsive to economic necessity and social norms. They have limited
economic resources and a low tolerance for uncertainty. The late majority adopt new
ideas just after the average member of a system. Like the early majority, the late
majority make up one-third of the members of a system. Adoption may be the result
of increasing network pressures from peers. Innovations are approached with a
skeptical and cautious air, and the late majority do not adopt until most others in
their system have done so. The weight of system norms must definitely favor an
innovation before the late majority will be convinced. The pressure of peers is
necessary to motivate adoption. Their relatively scarce resources mean that most of
the uncertainty about a new idea must be removed before the late majority feels that
it is safe to adopt.
(5) Final adopters/Laggards (Traditional) represent 16% and are the last adopters.
Laggards tend to be the least educated and least cosmopolitan. They are suspicious
of innovations and take a great deal of time to adopt a new innovation. They are
29
traditional in their roles, relatively isolated and usually are in a precarious economic
situation. They possess almost no opinion leadership. Laggards are the most
localized in their outlook of all adopter categories; many are near isolates in the
social networks of their system. The point of reference for the laggard is the past.
Decisions are often made in terms of what has been done previously. Laggards tend
to be suspicious of innovations and change agents. Resistance to innovations on the
part of laggards may be entirely rational from the laggard's viewpoint, as their
resources are limited and they must be certain that a new idea will not fail before
they can adopt.
Though the number of studies is increasing, Dobbins et al (2001) suggested there is
little understanding concerning the variation that exists amongst health-care
practitioners, decision-makers and organizations with respect to innovation adoption
behaviors.
Adopters versus non-adopters or early adopters versus late or non-adopters vary
according to individual, organizational, environmental, innovational and
communication characteristics (Rogers, 2003), which may also influence stages such
as awareness of the innovation and intention to adopt, even before the adoption
decision.
30
c) Rate of Adoption
The third way in which time is involved in diffusion is in the rate of adoption. The rate of
adoption is the relative speed with which members of a social system adopt an innovation.
The rate of adoption is usually measured as the number of members of the system that adopt
the innovation in a given time period (see Figure 2). The rate of adoption is a numerical
indicator of the steepness of the adoption curve for an innovation.
Figure 2: Adopter Categorization
Most of the variance in the rate of adoption of innovations, from 49-87%, is explained by
the five attributes (relative advantage, compatibility, complexity, trialability and
observability) of the innovation (Rogers, 2003) as described previously.
According to Gotham (2004) some medical innovations follow the traditional adoption S
curve based upon Rogers‟ model. However, in her research she found that Banta (1983)
described a “desperation-reaction model” of adoption where new medical innovations are
often diffused rapidly because physicians feel responsible for curing patients. However, that
model is criticized as being no longer applicable, and further investigation of this and other
31
models is needed. Studies on adoption rates for mental health innovations in the USA found
that programs in various parts of the Midwest were adopted simultaneously in clusters, but
this was most likely as a result of mandates of the state and federal funding agencies, which
supports the theory that external environmental pressures from funding bodies will cause
changes in adoption patterns for mental health treatments (Gotham, 2004). There are more
studies that examine changes in awareness, knowledge and intentions related to an
innovation in addition to the decision to adopt (Gotham, 2004).
2.2.4 Social System
The fourth main variable in Rogers‟ model of diffusion of innovation is the social system. A
social system is defined as a set of interrelated units that are engaged in joint problem
solving to accomplish a common goal. The members or units of a social system may be
individuals, informal groups, organizations, and/or subsystems. The social system
constitutes a boundary within which an innovation diffuses. According to researchers, the
systems that are able to respond most easily and quickly to innovations are ones that have an
established culture of creativity, a flat hierarchical system and a strong leadership committed
to effecting change. The healthcare system is a hierarchical model with separate
organizational structures for each professional group. According to Sanson-Fisher (2004),
this system tends to be bureaucratic with social norms that hinder rapid change; however, it
is possible for some clinicians to change their clinical activities relatively rapidly when there
are few restraints on determining choice of care. Changing to evidence-based, best practice
clinical behavior may mean modifying the system so that it adequately monitors the
frequency of the activity, outcomes and feedback to the clinician.
32
One area of research into social systems and innovation adoption involves the types of
innovation-decisions (whether individual adoption decisions or organizational decisions, and
whether they are made by an authority or by consensus). The authoritarian decision to adopt
an innovation leads to the fastest diffusion of the innovation, but effective adoption is often
bypassed during the implementation phase of the process. Alternatively, collective decisions
obtained through consensus building are slow, more complex and involve many individuals
in different roles, but the innovation is more likely to be sustained.
Another area of research into social systems and innovation adoption involves how norms
affect diffusion. Norms are the established behavior patterns for the members of a social
system. The culture of an organization influences decisions to adopt innovations. For
example, according to Henggeler et al (2002) a juvenile justice organization whose culture
emphasizes community protection is less likely to adopt a community-based alternative to
incarceration than is a mental health organization with a norm of rehabilitation. Mental
health innovations are consequently most likely to be adopted and sustained in organizations
in which opinion leaders support the innovation, staff at various levels has contributed to the
decision to adopt and the innovation is consistent with organizational culture. If there is
support at the consumer, community, academic, funding and legislative levels, evidence-
based research as an innovation has a greater likelihood to be adopted. However, mental
health innovations are not likely to be sustained when the decision to adopt has been made
autocratically, the opinion leaders do not concur with the decision and the innovation
conflicts with organizational culture.
33
A third area of research into social systems has to do with opinion leadership, the degree to
which an individual is able to informally influence other individuals' attitudes or overt
behavior in a desired way with relative frequency. These opinion leaders are described as
individuals within an organization whose influence has been earned and maintained through
high competence and behavior that is consistent with the system‟s norms. Opinion leaders
are generally more influential than innovators themselves and largely drive the dec isions to
adopt an innovation or not (Rogers, 2003).
Though opinion leaders are members of the social system in which they exert their
influence, the change agent is an individual who influences the individual‟s innovation
decision in a direction that is desired by the change agency (Rogers, 2003). Changes agents
seek to obtain the adoption of new ideas but may also result in slowing down the diffusion
and preventing the adoption if the innovation is undesirable. These change agents often use
opinion leaders in a social system to promote diffusion activities (Rogers, 2003).
A crucial concept in understanding the nature of the diffusion process is the critical mass, of
adopters, which occurs at the point at which enough individuals have adopted an innovatio n
that the innovation's further rate of adoption becomes self-sustaining. The concept of the
critical mass implies that outreach activities should be concentrated on getting use of the
innovation to the point of critical mass. These efforts should be focused on the early
adopters, the 13.5 percent of the individuals in the system who adopt an innovation after the
innovators have introduced the new idea into the system. Early adopters are often opinion
leaders, and serve as role models for many other members of the social system. Early
adopters are instrumental in getting an innovation to the point of critical mass, and hence, in
34
the successful diffusion of an innovation. This is important when examining the process of
how the innovation is made known to potential adopters. If the focus is not on the early
adopters, then enough critical mass may not be met for the rate of adoption to be sustainable.
2.2.5 Independent characteristics
Independent characteristics that affect the adoption of innovation include individual,
organizational and environmental factors. These characteristics are independent of the
actual innovation.
Individual characteristics of personnel in a health care organization (e.g. Executive
Directors, Managers) have received very little research but past studies have examined
individual variables that include demographics, tenure, level of education, level of
involvement with the social system/community and level of activity in professional
development (Glandon, 2004) . Additional individual characteristics that affect adoption
include sex, age, satisfaction with chosen profession, social origin, values, beliffs, view of
change, cosmopolitism and degree of professionalism.
Glandon (2004) reported on a study done by McFall et al (1996) that found physicians who
associated with a dissemination network were more likely to endorse treatment strategies
recommended by a consensus conference versus on their own. Furthermore, Glandon
stresses that many of the treatment staff are “para-professionals” unfamiliar with research, or
professionals trained when mental health and substance use treatment was less well
35
researched, and that clinical anecdotes suggest some mental health and substance use
treatment staff feel that one illness must be treated before the other can be addressed.
Organizational characteristics may be either structural or dynamic. Structural
characteristics include the size of the organization, in both brick and mortar terms and
number of staff or volunteers, complexity (levels of management), internal and external
communication channels as well as location and whether or not there is a drive for revenue
generation (Gotham, 2004). As an example, in HIV/AIDS service providers it was found
that organizational size can positively and negatively affect the dissemination and ultimately
the adoption of innovation.
According to Simpson (2002), organizational dynamics include the existence of leadership,
the type of organizational climate; defined as the process of quantifying the culture of an
organization, and readiness to change. These characteristics are becoming recognized as
essential to the adoption of innovation treatment models/recommendations. Futhermore, in a
study by Simpson and colleagues, a model of drug abuse treatment program change was
developed, positing that institutional and personal readiness to change affected the adoption
of treatments. This is supported by the transtheoretical model of change by J.O. Prochaska
and DiClemente (1983), which has been applied to organizations. In their research it was
found that readiness to change and beliefs regarding treatment effectiveness may hamper
dissemination.
The common assumption about how research is used in practice is that decision makers will
use knowledge of the research appropriately, given adequate dissemination, relevance and
36
timeliness (Berta et al, 2005). This assumption is challenged as theories of research
dissemination and use emphasize the importance of organizational interests and the
communication between research source and users. In Berta‟s (2005) review of research
utilization studies, it was found that these organizational factors accounted for the majority
of explained variance in regression analysis that simultaneously looked at the impact of
organization, environmental and individual characteristics. Studies done by Denis et al
(2002) used a multiple case study to show speed and pattern of adoption of an innovation in
health care have more to do with the fit of the innovation to the interests of the individua ls
involved (e.g. organizations) versus the strength of the evidence informing and improving
clinical practice and outcomes.
Many diffusion studies have examined characteristics of organizations in an attempt to
distinguish more innovative firms from less innovative ones (Damanpour, 1991).
Damanpour found that organization innovativeness is more accurately represented when
multiple innovations were considered rather than single innovations. In his research to
identify the relationships between organizational attributes and innovation, in the
organizations that were most widely studied, the following determinants were found;
specialization, functional differentiation, professionalism (of the members within the
organizations), formalization (following rules), centralization (locus of authority and
decision making), managerial attitude towards change, managerial tenure, external
communication (the ability to be in contact with and scan its task environment) and internal
communication. He also found that when multiple innovations are studied, the influence of
the innovation attributes decrease. Also, when all innovations adopted are considered, the
37
role of the organizational characteristics becomes more evident. Furthermore, determinants
of innovation and the strength of their influence depend on whether or not a comprehensive
group of innovations related to various parts of an organization is studied.
Other organizational characteristics, such as size of firms, product types, management
support and the existence of visionary leaders / innovation champions, are also known to be
attributes that have positive correlations to the rate of adoption of innovation in
organizations (McGowan and Madey, 1998).
Harting et al (2005) identify that at the organizational level, a linkage between a resource
system and a user system and the facilitating efforts of a change agent may be initially
important. The subsequent actual implementation requires the innovation to be modified to
fit the organization and the organizational structure to be altered to accommodate the
innovation. The influence of organizational characteristics varies considerably, although
larger organizations are rather consistently found to be more innovative. Camison-Zoromoza
et al (2004) completed a meta-analysis of the relationship between innovations and
organizations. In their findings, they identified studies that point to the existence of a
positive relationship with the suggestion that organizational size is the best predictor of
adopting an innovation. Conversely, their meta-analysis also identified that studies done by
Wade (1996), Aldrich and Auster (1986) and Hage (1980) defend the existence of a negative
relationship. The meta-analysis has thus led to a single conclusion, which is that the most
consistent result found in the organizational innovation literature is that it is inconsistent.
The meta-analysis also uncovered that both Kimberly and Evanisko (1981) and Meyer and
38
Goes (1988) found the characteristics of the innovation itself and the organizational
variables were the most important explanatory factors. They also found that studies exist
which see structural characteristics as being more closely related with the innovation than
that individual characteristic or attitudes within the organization.
Aarons (2005) found organizational culture and climate to be important characteristics that
influence attitudes toward adopting innovation both in general and specifically in research-
based innovations. Management may decide to adopt an innovation but individual
acceptance of an innovation is proposed to rely on both organizational and individual
characteristics (Rogers, 2003) and affect how well the innovation is adopted, as it relates to
accuracy and competency.
A number of organizational- level characteristics that affect clinician attitudes have been
studied in mental health service agencies and programs. More positive organizational
climate is associated with better organizational process, work attitudes and outcomes of
services. (Aarons and Sawitzky, 2006)
Another important organizational factor is the status of the organization in healthcare and in
the communities the organization supports. Some organizations pride themselves as the care
provider leader and therefore are more committed to making a difference with their
programs/services.
39
Organizational culture and climate have also been studied in human service organizations.
These studies showed that both culture and climate are contributing factors in the quality
and outcomes of mental health services. Glisson and James (2002) found that culture
influences work attitudes (e.g. job satisfaction, organizational commitment), service quality
and staff turnover. There is also the comparison of constructive cultures and defensive
cultures. Organizations with constructive “positive” cultures have characteristics such as
norms of achievement and motivation, individualism, self-actualization and supportive.
They encourage interactions with sources and approaches that will enable staff to meet their
higher-order need. Defensive “negative” cultures tend to seek approval via consensus and
will only interact with new ideas as long as it does not challenge or threaten the individual or
the organization‟s security. Carmazzi and Aarons (2003) found that providers working in
child and adolescent mental health agencies with more positive cultures had more positive
attitudes toward the adoption of innovation (new ideas of practice), whereas those with more
negative cultures embraced a more negative attitude towards adoption.
Aaron further comments that organizational culture and climate demonstrate that the
organizational process is related to staff attitudes, perceptions, behavior, service and
outcomes. It also demonstrates that culture and climate are tied to core organizational values
and perceptions; they must represent organizational processes that are likely to influence
mental health provider attitudes toward organizational change and adoption of innovations
in particular.
40
Environmental factors are defined as the socioeconomic infrastructure of a community and
can impact the organizational structure. The literature shows the importance of
environmental factors as they relate to diffusion and adoption of innovations. According to
Berta et al (2005), external factors can exert profound influence on the way knowledge
(research) is valued by an organization and on the decisions regarding opportunities to
implement (e.g. standards of care). Rogers states an organization‟s environment also
includes other organizations, from which leaders can potentially learn. The opportunity to
observe a new idea being applied by others is an important contribution to successful
adoption. In Berta‟s review, one report showed the importance of environmental factors to
include; the centrality of an organization‟s network location in terms of graduate medical
education, the reputation and visibility of the medical school with which the organization
(hospital) is affiliated. The report also showed the prior exchange of information concerning
the structure of the management system contributed significantly to predicting the adoption
of the same management system by other organizations. According to Berta, the effects of
the physical proximity is a significant predictor of adoption, as the cumulative number of
prior adopters location in the same geographic region will have a positive effect. Nutley et
al (2005) emphasized that the “workplace” environment characteristics such as commitment,
open discussion, personal anxiety and frustration will affect successful adoption of the
innovation. Commitment is important, as it demonstrates the senior management support in
implementing the innovation. Open discussion improves productivity and allows senior
management to better understand the challenges that are presented by the staff using the
innovation. Though personal anxiety and frustration may be classified as a reaction of the
individual, the source of the anxiety or frustration and how it is managed, is dictated by the
41
environment. When the environment is a contributing factor in this, it may result in staff
avoiding using the innovation or other innovations. Camison-Zornoza‟s meta-analysis of the
literature found that fast changes in the environment will “trigger off” innovation processes
within the organization. In addition, the meta-analysis states that some studies claim that the
relationship between organization and environment is reciprocal and that both of them
interact to bring about the adoption of innovations (Damanpour, 1991).
Anderson and King (1992) completed a joint analysis of internal and external factors of the
organization as elements that fostered organizational innovation. These factors included
leaders, structure, strategy, organizational culture and the environment. Additional
environmental factors associated with the adoption and diffusion of innovations include
collaboration among community networks, reporting relationships between top management
team and accountable boards, regulations and legislation, urbanization, peer pressure,
competition among institutions to attract specialized professionals and acquisition of
prestige (Dobbins et al, 2002). Greer (1977) further supports the importance of
environmental factors in regards to adoption behavior.
Mustonen-Ollila (1998), whose studies focus on technology and adoption, states the
environment is the most important source for an organization concerning the “knowledge,
the needs, the resources and the know-how”. He states that at the decision and
implementation stage, the three most affecting characteristics of environment are the norms
of the social environment, the economical restrictions and the resources in the functioning
environment.
42
2.3 Adoption Studies in Mental Health and Substance Use Treatment Settings
The original diffusion research was done as early as 1903 by the French sociologist Gabriel
Tarde, who plotted the original S-shaped diffusion curve. Tarde was a French sociologist,
criminologist and social psychologist who conceived sociology as based on small
psychological interactions among individuals, the fundamental forces being imitation and
innovation. Tarde‟s 1903 S-shaped curve is of current importance because "most
innovations have an S-shaped rate of adoption" (Rogers, 2003).
Since then, diffusion has been applied to different systems to include anthropology,
sociology, economics, marketing, technology, geology and education. Rogers (2003) states
that public health and medical sociology is one of the nine major diffusion research
traditions, constituting 10% of all diffusion publications. Previous research exists in
examining adoption in individuals, organizations, and healthcare organizations, but not
specifically in the mental health addiction field. (Gotham et al, 2004).
According to Gotham, research findings from diffusion-of- innovations, as it relates to
mental health and substance use treatment, can assist in bridging research-practice gaps that
lead to clients receiving treatments whose effectiveness has not been assessed or not
receiving the most effective treatments. Research regarding the diffusion of innovations
should be examined for application to treatment, including the transportability of treatments
into real-world settings, strategies to disseminate new treatment approaches, attr ibutes of
treatments that affect their adoption, and organizational change factors that affect
implementation.
43
Previous research comes from a broad rage of disciplines, many of which lie outside the
traditional areas of focus for healthcare research (Buchan et al, 2004). There are major
differences between the diffusion of mental health and substance developments and the
diffusion and adoption of many “tangible” medical innovations, such as the extent to which
definitive medical research findings are based upon accepted assessments and “gold”
referenced standards. (Glantz et al, 2004).
Glantz et al has found that even definitive medical research findings are often slow to be
adopted by the general medical community. In fields such as mental health and substance
use, where there are fewer consensuses about practice and treatment approaches, the factors
and barriers in adoption may be greater and other unidentified characteristics may be
involved.
The research described in this paper will focus on the fo llowing:
a) the degree of considered adoption decision the organizations made, regarding the
innovation of the Best Practice document
b) the organizational, individual, environmental and innovation characteristics
associated with the adoption decision
As Dobbins et al explains, “Diffusion of Innovation” theory has been around for the past 20
years, but only recently has it been used to explain the adoption of research evidence among
health organizations, rather than solely among individuals; this is the corners tone of this
research and how it relates to a unique segment of the health care sector.
44
2.4 Criticisms of Rogers Diffusion of Innovation
Critics, including Rogers himself, have critiqued the diffusion model, despite its success and
its effectiveness in improving adoption of many types of innovations across a wide variety
of settings. Rogers identifies four major challenges with the diffusion of innovation model.
These challenges are: 1) pro innovation bias, 2) individual blame bias, 3) recall problems
and 4) equality.
Pro innovation bias- the implication is that the innovation should be diffused and adopted
by all members of a social system that it should be diffused more rapidly and the innovation
should be neither rejected nor re-invented. This bias is usually assumed or implied. Because
of this assumption, the bias can lead the research field to ignore the study of ignorance of
innovation, thus overlooking the re-invention and fail to study “anti-diffusion” programs
that are designed to prevent the spread of a bad innovation. Thus this limits what literature
exists regarding diffusion.
Individual blame bias- is where there is likelihood to hold an individual accountable for the
outcome, versus the system of which the individual is a part. This is where the root cause
should be examined, as it is human nature, especially when there is a social problem, to
change the individual and hence the individual factors. However, in many cases, the causes
of the social problem exist in the larger context of the system. This bias can limit the
understanding of the diffusion process, where predictor variables of individual
innovativeness could be considered to be system-blame factors, such that the channel of
innovations might be at fault for not providing adequate information, in order for the change
45
agents to maximize their potential in leading adoption. It is recommended that individual-
level variables should not be centered on as a complete explanation of the diffusion
outcome.
Recall problems- as time is one of the main variables that impact diffusion, it leads to the
weakness of dependence upon self-reported recall data from respondents, regarding the
timeline of their adoption of the new idea. There is a question of the degree of accuracy,
varying on the basis of the innovation‟s knowledge to the individual and the time frame to
which the recall was requested. Rogers states that if data about a diffusion process are only
gathered at one point in time, then the investigator can only measure time through
respondent‟s recall, a possibly weak read on which to base the measurement of such an
important variable. Furthermore, the research used in diffusion studies do not tell us much
about the process of diffusion over time, other than what can be reconstituted from the
respondent‟s recall data.
Equality- According to Rogers, diffusion researchers have not paid much attention to the
consequences of innovation. There has been little attention to the issue of how the
socioeconomic benefits of innovation are distributed among individuals in a social system.
The studies that focused on the issue of equality show that the diffusion of innovations often
widens the socioeconomic gap between the higher and lower socioeconomic status segments
in the system.
46
3.0 Research Methodology
This chapter presents the method of analysis used for the research. It includes a description
of the sample and what measures were used for the independent and dependent variables.
The instruments are then discussed, in terms of reliability, validity and the scales used. The
chapter also describes the final analysis, using factor analysis, correlation and multiple linear
regressions.
In February 2003, the Centre for Addiction and Mental Health hosted a videoconference as a
first wave of dissemination on the Best practices document (the innovation). At the time of
the initial dissemination, participants in the videoconference were asked to complete a
questionnaire on the quality and value of the presentation with two additional questions,
which were:
a) Have you read the Best practice document?
b) Do you intend to use the Best practice document
No definitions were given for the terms “use” or “read”, but further discussion with the
videoconference organizers revealed that the term “use” had an implicit definition which
included a change of practice and/or an impact on procedures and policies within the system,
and “read” was taken to mean that the individual/organization was aware of the existence of
the innovation and had reviewed its contents at least once.
The participants in the videoconference were members of the leadership teams of the various
organizations in Ontario that dealt with the populations in mental health programs, substance
47
use programs or both. These members were either Executive Directors or in senior
leadership roles that would be responsible for administratively implementing the best
practices within the organizations.
The one-year follow up to the original dissemination of the innovation involved a cross-
sectional study design, which included a voluntary quantitative mail-out survey using a self-
administered questionnaire. The questionnaire was sent out to 260 of the Ontario
organizations that participated in the February 2003 videoconference and baseline survey.
The questionnaire included a cover letter, information sheet and survey form, with a pre-
addressed & stamped return envelope, in addition to a coupon for a free beverage at Tim
Horton‟s. The cover letter stated that if the Executive Director was not available to respond
to the survey, then the respondent should be the individual responsible for implementing the
Best Practice recommendations. Pretest overall indicated that the total length of time to
complete the questionnaire was an average of eight minutes and none of the questio ns were
considered to be intrusive or embarrassing.
3.1 Setting and Sample
The study sample included individuals in a leadership position representing their
organizations that participated in the original videoconference and completed the survey in
2004. The individuals in the study included executive directors and senior managers of
mental health and substance use treatment agencies, individuals who are responsible for
making decisions regarding practice, adopting and implementing new recommendations
regarding concurrent disorders.
48
The executive director/senior manager was sent an introductory letter explaining the study
and asking for their participation. The letter also included contact information for the
primary investigator, to whom they could direct any questions regarding the research.
Consent was implied upon completing and returning the survey and all participants were
informed that the results would be confidential and reported only in aggregate format.
The questions in the voluntary self-administered questionnaire were related to individual and
organizational demographic information, such as educational background of the individual,
the size of the organization, types and number of programs offered as well as questions on
their beliefs, accessibility and understanding of research as it informs practice. The
questionnaire was also approved by the University of Toronto, Health Sciences Ethical
Review Committee.
Questionnaire Development
The final section of the questionnaire then asked the respondent about current status of
adoption for each type of recommendation (innovation) included in the Best practices
document. For the purposes of this research, adoption of an innovation was considered to be
the same as “use” but on a graduated scale. The rationale to consider adoption to be the
same as use is based upon the research of Dobbins et al (2001), who examined definitions
used in knowledge utilization and stated that research evidence is an innovation and that the
knowledge gained from diffusion research is applicable in understanding the process of
research (knowledge) utilization.
49
Each respondent was allowed 6 weeks to respond, with follow- up reminders via the postal
mail as well as email, for those who provided an email address for the organization, and
telephone calls for those who did not.
3.2 Measures
The independent variables of interest included characteristics of the individual, organization,
environment and innovation, as well as the perception of research and the perceived
assessment of the communication method. A complete list of survey questions is presented
in Appendix 1 and a full list of independent variables with definitions and methods for
measurement is summarized in Appendix 2
The dependent variable for this study was adoption and was measured in the final analysis as
overall adoption of the recommendations (innovation).
Independent Variables
Appendix 1 displays the specific survey questions used to collect data on
independent variables. To facilitate easier identification of the specific survey questions, the
number of the question will be provided in brackets, following a description of the variable.
The questionnaire used terms that are common and well-understood in the mental health and
addiction use treatment community, and either an explanation or an example was included
for all statements and terms in the questionnaire.
50
Individual variables
Data for seven individual characteristics of the organization‟s leaders who responded were
collected within the survey. These were:
Sex (2)
Year of Birth (3)
Highest level of education completed (4)
Current position (5)
Length of employment at current organization (tenure) (6)
Field of career to date; specialization (7)
Number of conferences attended in past year (cosmopolitism) (8)
Organizational variables
Data were collected on nine organizational characteristics. These included:
The number of staff within the organization (11)
The number of volunteers within the organization (12)
Organization‟s primary focus (9)
Types of services offered by the organization (13)
Urban/rural mix of population served (geography) (14)
Number of management levels (17)
Special populations served (10)
Yearly average number of programs offered (16)
Monthly average of the number of clients/patients served (18)
51
Though organizational climate and culture is important, it was not included in this scope of
research.
Environment
As explained in the literature, the environment and organizational characteristics are closely
related and can overlap, with environment characteristics influencing the organizational.
Hence the overlap of the following characteristics:
Type of area that the organization provided services to (urban/rural- geography)(14)
Number of other mental health/addiction organizations in same regional municipality
(15)
Number of management levels in the organization (17)
Attributes of the Innovation
Data on five innovation attributes were collected in the survey: relative advantage,
complexity, compatibility, triability, and observability. Respondents‟ perceptions of the
innovation were measured by using a 5-point Likert agreement scale (strongly agree, agree,
neutral, disagree, and strongly disagree). The questions focused on the concepts of the
attributes of the innovation, which included:
Clarity (complexity) (39)
Plain language (complexity) (40)
Ease of understanding (complexity) (41, 43)
Ease of use (triability) (44)
Customizable (compatibility) (48)
52
Fit with personal beliefs and organizational values (compatibility) (45,46)
Ease of seeing results (observability) (47)
Improvement over current practice (relative advantage) (42)
Though they are not considered to be characteristics of the innovation, the survey also
measured the awareness of the innovation; “have you heard about the Best practices
document?” (19) (37) and “have you read the Best practices document?” (20) (38).
Perception of Research, Communication and Dissemination
Since perception of the innovation and of how the communication is shared is key to
Rogers‟ model, data was collected to assess the general perception of the role of research in
clinical practice, the success of dissemination and communication of the research and the
individual‟s openness to changing practice based upon new research. These data were
collected using 16 questions with a five point Likert agreement scale (strongly disagree,
disagree, neutral, agree and strongly agree).
Appendix 1 lists all the questions that were used in the questionnaire.
Questions were directed to the individual‟s perception of the following:
Importance of research to inform clinical practice (21) (24) (25) (26) (27)
Accessibility of research that informs clinical practice (23), (28),(29)(30)
Understanding and application perceptions (31) (32)
Willingness to try (33)
Perception of the need and benefits of research (34)(35)(36)
53
Dependent Variable
The dependent variable measured in this study was the overall measure of adoption of the 27
item recommendations (innovation) (Appendix 1) that were presented in the Best practices
document. Overall adoption was measured as an ordinal variable.
To measure overall adoption, respondents were asked to rate their level of adoption using a
six point ordinal scale that was adapted from Larsen‟s Information Utilization Scale (1982).
The average of the adoption level of the 27 item recommendations was calculated to
represent overall adoption.
Each organization had the same amount of time to be aware of the innovation and participate
in the adoption process. The scale that was adapted from Larsen represented the level of
adoption and included the following:
1. Considered and rejected- this meant the recommendation as innovation was
considered, but was rejected by the respondent. A high response rate of this selection
would represent non-adopters. It means that in respect to Rogers‟ model, there is
awareness of the innovation but it has been rejected.
2. Nothing done- this meant no action took place on the recommendation. It was neither
considered, nor rejected. This is in contrast to the previous option in which the
recommendation was actively considered and acted upon via being rejected. A
response of “nothing done” would equate that no decision was made to reject or to
consider implementing and adopting.
3. Under consideration- this meant the recommendation was at the stage of
consideration during the time of the survey. No decision had been made to consider
54
or reject adopting the innovation but the organization was still in the adoption
process.
4. Step towards implementation- this meant the recommendation is being evaluated.
There is awareness and interest, with the next step being towards evaluation of the
recommendation.
5. Partially implemented- this meant the recommendation has been evaluated, and is in
the trial stage, moving towards full implementation and adoption.
6. Implemented and adopted- this meant the organization had completed the adoption
process and has fully adopted and implemented the recommendation.
3.3 Instruments (Reliability, Validity) and Scales
The data for this research were collected using one questionnaire, with four sections
representing three different instruments. The first instrument (Section A of the survey)
collected demographic and characteristic data on the individual and organizational variables.
The second instrument was a five-point Likert agreement scale (Section B and C) that
collected data on the perceived characteristics of the innovation, and perception of research
and communication. The third instrument (Section D) was an adaptation of the Information
Utilization Scale created by Larsen, 1982, and was used to measure the dependent variable.
55
The original information utilization scale, created by Larsen, contained seven ordinal ranked
categories;
1. Considered and rejected
2. Nothing done
3. Under consideration
4. Steps toward implementation
5. Partially implemented
6. Implemented as presented
7. Implemented and adapted
According to Dunn (1983) and a thorough literature review, Larsen does not report
reliability or validity data for this scale. The scale was adapted for this research by
collapsing the two categories “implemented as presented” and “implemented and adapted”;
within the context of the research and the innovation, these distinctions were irrelevant.
Each instrument was pre-tested for test-retest reliability and face validity at one organization
that had equal specializations in mental health and addiction use treatment. A total of eight
individuals participated in the testing of the test-retest reliability of each instrument. The
questionnaire was administered by mail and indicated it would take up to 8-10 minutes of
the respondent‟s time.
Reliability was tested by running a Cronbach's alpha. The overall Cronbach‟s alpha score
was 0.67, which was considered adequate for this study. In addition, four of the eight
individuals agreed to participate in a test-re-test of the questionnaire. This involved
56
completing the questionnaire twice, one week apart. The interclass correlation coefficient
was used to identify specific items in the questionnaire that produced unreliable results
between the pre-test and post-test. Items with coefficients below 0.5 would have been
removed from the final questionnaire. Using this criterion, no items were in fact removed
from the final questionnaire.
3.4 Quantitative Analysis
The analysis included simple descriptive summaries such as means, frequencies and where
applicable, standard deviations for all study variables. Further analysis included t-tests,
analysis of variance (one-way ANOVA) correlations, factor analysis and multiple linear
regression modeling.
The dependent variable scale (adoption) was recoded to reflect the body of adoption
literature, where “considered and rejected” had the same response value as “implemented
and adopted” as it relates to adoption (e.g. a final decisions and action is made). The
rationale is that both responses reflected the process of considering a recommendation and
completing an action. Responses that were originally “considered and rejected” were
recoded to the same value as “implemented and adopted”. The scale was adjusted further,
recoded that the lowest level of adoption decision making would be reflected as “nothing
done” (1), with the highest level of adoption decision making reflected as “considered and
rejected/implemented and adopted” (5).
In order to find the missing data for the dependent variable of adoption, frequencies were
run. When missing data were identified, it was replaced with the value of “nothing done”;
as no action was indicated, i.e., the recommendation was neither considered nor rejected. If
57
one variable had more than 50% of its data missing, then the variable was removed from the
analysis. This did not cause any variables to be removed. If any one case had more missing
data than valid data, then the case in question was removed. Additionally, any cases that did
not respond to the section regarding level of adoption were removed. This resulted in 19
cases being removed from the analysis.
Further cleaning of the data included identifying any items that needed to be reverse-coded.
Only two items, Q 43 “the recommendations in the Best practices are difficult to
understand” and Q 44 “the recommendations in the Best practices document will be difficult
to use” required adjustment for reverse coding.
An exploratory factor analysis was performed on the dependent individual adoption items
(Q 49-75) as well as for the independent items regarding perception of research (Q 21-36)
and clinical practice and perception of the characteristics of the innovation (Q 39-47). This
was done to reduce the large number of independent variables to a smaller number of factors
for modeling purposes. From the factor analysis, reliability analysis was performed on the
seven dimensions identified to ensure the measurement consistency of the scales.
Tests of significance were run. T-tests were used for the independent categorical variables
with two groups. One-way ANOVAs were performed for the independent categorical
variables with three or more groups. Chi-square was not used for these variables, due to
sample size limitations.
Correlations were run for the independent continuous variables or variables with underlying
scales. Independent variables that were significantly correlated with the adoption decision
58
were identified. From these results, a hierarchical multiple linear regression was performed,
using a stepwise approach with the five independent variables that had a significant
correlation with the adoption decision.
Exploratory Factor Analysis
Adoption
The dimensionality of the 27 recommendation items measure was analyzed using maximum
likelihood factor analysis. Three criteria were used to determine the number of factors:
testing that adoption is unidimensional and not multidimensional, the scree plot and the
interpretability of the factor solution. Based upon the scree plot (Figure 3), one factor was
identified in the unrotated solution. This was repeated with the varimax rotated solution and
there was no significant change in identifying the number of factors. The unrotated solution
yelded seven factors with Eigen values greater than 1, and one factor that accounted for
31.5% of the variance. The one factor that fell on the sharp decent of the slope of the scree
plot indicates one factor (adoption). As the other factors fell on the flat line of the scree plot,
it was decided that these 27 items loaded up on the one factor which represents overall
adoption.
59
272625242322212019181716151413121110987654321
Component Number
10
8
6
4
2
0
Eig
en
valu
e
Scree Plot
Figure 3: Scree Plot of Varimax Rotation- Adoption
60
Based upon this factor analysis, an overall scale (adoption) with an overall rating for each
respondent was calculated. The overall scale was calculated by replacing missing data with
the value of “nothing done” and then calculating the average of each case for each item.
Perception on Research and Changing Clinical Practices
The dimensionality of the 16 items from the perception of research and changing clinical
practices measure was analyzed using maximum likelihood factor analysis. Three criteria
were used to determine the number of factors to rotate: the hypothesis that the measure was
unidimensional, the scree test and the interpretability of the factor solution. The scree plot
indicated that our initial hypothesis of unidimensionality was incorrect. Based upon the plot,
four factors were rotated using a Varimax rotation procedure.
The rotated solution, as shown in Table 1, yielded four interpretable factors; importance of
research, research findings being understood, the benefits of research and the application of
research findings.
The importance of research had an Eigen value of 4.319 and accounted for 26.9% of the
item variance, research findings being understood had an Eigen value of 2.887 and
accounted for 18.0% of the item variance, the benefits of research had an Eigen value of
1.996 and accounted for 12.4% of the item variance and application of research findings had
an Eigen value of 1.175 and accounted for 7.3% of the item variance (Table 2).
61
16151413121110987654321
Component Number
4
2
0
Eig
en
valu
e
Scree Plot
Figure 4: Scree Plot of Varimax Rotation- Perception of Research and Changing
Clinical Practices
62
Table 1:- Varimax Rotated Solution Factors
Factors
1 2 3 4
Keeping on top of current research is essential to professional
performance
.603 -.022 .071 -.168
Most research on mental health & addictions by universities/research
institutions does not directly apply
-289 .151 -.080 .458
It is difficult for MH&A workers so access research in their communities -.143 .257 -.005 .642
I expect staff to keep on top of research .800 -.035 .027 -.136
I expect staff to read MH& A journals .755 .028 .085 -.154
I seek out recent research
.690 -.062 .133 .004
I distribute research information to my staff regularly .678 -.006 -.023 -.076
Most research is presented in a way that is no accessible to my staff -.186 .537 .108 .496
Research findings in MH&A are difficult to find out about -.021 .993 -.030 .106
Research findings in MH& are difficult to obtain .047 .837 -.063 .214
Research findings in MH&A are difficult to understand .007 .591 .010 .456
Research findings in MH&A are difficult to apply -.185 .354 -.055 .533
I am willing to try new ideas in mental health/addiction practice .128 -.106 .381 .267
I see a need to change practice in my field -.119 -.040 .705 -.060
I see the benefits of changing practice in my field .151 .041 .837 .068
I consider best practice documents important .166 .045 .516 -.260
Table 2: Variance for Varimax Rotation
Component Total Eigen value % of Variance Cumulative %
1 4.319 26.991 26.991
2 2.887 18.04 45.034
3 1.996 12.475 57.509
4 1.175 7.35 64.856
63
Presentation and content of the recommendations/innovation
The dimensionality of the 12 items from the presentation, content and recommendations was
analyzed using maximum likelihood factor analysis. Three criteria were used to determine
the number of factors to rotate; the hypothesis that the measure was unidimensional, the
scree test and the interpretability of the factor solution. The scree plot indicated that our
initial hypothesis of unidimensionality was incorrect. Based upon the plot, two factors were
rotated using a Varimax rotation procedure. The rotated solution, as shown in Table 3
yielded two interpretable factors; complexity of communicating research findings and
innovation characteristics.
Complexity of communicating research findings had an Eigen value of 4.454 and accounted
for 44.5% of the item variance, and innovation characteristics had an Eigen value of 1.820
and accounted for 18.2% of the item variance (Table 4)
10987654321
Component Number
4
2
0
Eig
en
valu
e
Scree Plot
Figure 5- Scree plot Varimax Rotation- Presentation of the
recommendations/innovation
64
Table 3 : Varimax Rotated Solution Factors- Presentation of the
recommendations/innovation
Factor
1 2
The recommendations in the BP document were clearly presented .730 .235
The recommendations in the BP document were in plain language .843 .301
The BP document was easy to understand .889 .306
The recommendations in the BP document will improve current practice .129 .574
The recommendations in the BP are difficult to understand -.711 -.008
The recommendations in the BP document will be difficult to use -.211 .512
The recommendations in the BP document fit with my personal beliefs .284 .508
The recommendations in the BP document fit with my organization‟s values .279 .579
The recommendations in the BP document will produce results that are easy to see .041 .800
The recommendations in the BP document are customizable to my organization .093 .806
Table 4:- Variance –Varimax Rotation- Presentation of the recommendations/innovation
Component Total Eigen value % of Variance Cumulative %
1 4.454 44.538 44.538
2 1.820 18.204 62.741
65
Reliability
Reliability was tested on the factors from the factor analysis to ensure overall consistency of
the items that are used to define the scale. For adoption, the scale showed the level of
adoption (nothing done to fully implemented-adoption or considered/rejected) while the
independent variables of perception of research and characteristics of the innovation had a 5
point Likert scale of strongly disagree (1) to strongly agree (5). The Cronbach‟s alpha on
each factor was greater than 0.70, which indicated sn acceptable level of internal reliability.
Mean scores were then calculated for all of the independent variables and for overall
adoption, as shown in Table 5. From these results, it can be seen that the mean for
importance of research, benefits of research, complexity of communication, and innovation
characteristics were high (agree) but research findings being understood, and the application
of the research were low (neutral/disagree). The mean for adoption indicates steps towards
implementation.
Table 5: Reliability of dimensions identified
Cronbach‟s
Alpha
Mean Standard
Deviation
Adoption .896 3.87 0.70
Importance of research .841 4.04 0.61
Research findings being understood .872 2.73 0.78
Benefits of research .701 4.06 0.58
Application of research findings .700 2.90 0.79
Complexity of communicating research
findings
.887 4.02 0.66
Innovation characteristics .814 3.58 0.60
66
Multiple Linear Regression Model
As the purpose of this study was to identify characteristics associated with the adoption of
an innovation, the appropriate statistical analysis to use was regression analysis. Regression
analysis would identify the variables that predicted the outcome of adoption. As there is
more than one variable that can be used to predict the outcome, a multiple linear regression
analysis was used to identify the relationships between the multiple independent variables
that had a significant correlation with the dependent variable of adoption to identify the
degree to which the significant independent variables would predict adoption.
Additional tests were required in order to determine which independent variables should be
included in the regression equation prediction model. To test for significance, t-tests were
performed for groups of two, whereas one- way analysis of variance (one-way ANOVA)
was performed for groups of 3 or greater. Chi-square was not used due to sample size.
Correlation coefficients were run for independent continuous variables, or variables with
under lying scales. Any variable that had significance less than 0.05 (two tailed) was
identified as being correlated.
67
4.0 Results
This chapter will discuss the independent variables that were considered and the results of
the multiple linear regressions, showing which independent variables were predictors of
adoption.
Response Rate
One hundred and twenty one of the 260 ( response rate of 47 %) organizations returned the
questionnaire. Of the 121 returned questionnaires, there was complete data on the
independent variable for only 102 cases. Any question related to the independent variables
or the dependent variable that had less than 50% response rate was excluded, which resulted
in four cases removed. Any case that had less than a 50% response rate for all of the
questions was also excluded, which resulted in fifteen cases removed. This resulted in a
total of 19 cases being removed, which was less than 10% of the total number of
respondents. Thus the final data set was 102 cases.
Data missing from the dependent variable were replaced with the response value of “nothing
done”. There were two rationales for replacing the dependent variable missing data. The first
rationale is that if there was no response selected for that component of the recommendation,
then no decision had been made. The second rationale was that the action required by the
recommendation was not understood by the organization, though it should have been, due to
the dissemination activities by the publisher- Health Canada.
68
Independent Variables
The theoretical framework discussed earlier in this research is based on the results of years
of research in the areas of diffusion of innovation and adoption theory, and identified five
main categories of characteristics associated with adopting an innovation: 1. Attributes of
the innovation, 2. Communication channels, 3. Time, 4. Social system, and 5. Independent
variables (e.g., individual, organizational and environmental variables). A number of
independent variables and the attributes of the innovation were examined in the study. The
means and the standard deviation (where applicable) and frequencies for independent
variables are summarized in Table 6. Significant results are discussed in detail below.
69
Table 6: Descriptive Statistics of Independent Variables
Variable Label Operational
Definition
Mean/% SD N
Individual Characteristics
Year of Birth Year of birth 1956 7.8 98
Sex Male Female
30% 70%
N/A 100
Highest Level of Education High School
College Bachelor‟s
Master‟s PhD
1.0%
12.9% 31.7%
49.5% 5.0%
N/A 99
Current Position Front Line Supervisor
Manager Exec.Director
20.8% 11.9%
34.7% 32.7%
N/A 101
Length at Organization <3 years
3-5 years 6-9 years 10-15 years
>15 years
10.9%
24.8% 13.9% 23.8%
26.7%
N/A 99
Field of Career Mental Health Addictions
Social Work Concurrent Disorders
34.7% 33.7%
13.9% 16.8%
N/A
# Conferences attended year Zero (none)
1-3 4-6
>6
9.1%
64.6% 23.2%
3.0%
N/A 99
70
Organizational Characteristics
# Staff <6 6-10
11-20 21-30
31-50 >50
7.1% 13.3%
19.4% 14.3%
12.2% 33.7%
N/A 98
#Volunteers <6 6-10
11-20 21-30
31-50 >50
40.5% 11.9%
13.1% 4.8%
4.8% 25.0%
N/A 84
Organization focus Mental Health Addictions
Concurrent Disorders
Other
35.3% 37.3%
17.6%
9.8%
N/A 101
Special populations Youth Women Seniors
Gay/Bi/Lesbian
64.4% 72.0% 66.0%
52.0%
N/A 102
Types of services provided Screening Assessment
Treatment Referral Case Management
80.4% 85.3%
86.3% 88.2% 84.2%
N/A 102
Urban/rural mix of population Rural
Urban Suburban
Combo
13.7%
27.5% 7.8%
50.0%
N/A 101
# Management Levels 1 Level 2 Levels 3 Levels
4 Levels 5 or more
8.8% 44.1% 22.5%
10.8% 10.8%
N/A 99
# Programs offered annually Zero (none)
1-3 4-6 7-10
>10
1.0%
25.5% 25.5% 19.6%
25.5%
N/A 99
# Clients monthly <10 11-30 31-50
>50
1.0% 11.8% 11.8%
75.5%
N/A 102
71
Environment Characteristics
Type of geographical area Rural Urban
Suburban Combo
13.7% 27.5%
7.8% 50.0%
# Other organizations in area Zero (none)
1-3 4-6 7-10
>10
1.0%
28.4% 26.5% 8.8%
25.5%
N/A 92
# Management Levels 1 Level 2 Levels
3 Levels 4 Levels 5 or more
8.8% 44.1%
22.5% 10.8% 10.8%
N/A 99
72
Attributes of the Innovation
Characteristics
Clarity of the document (Complexity)
Recommendations were clearly
presented
5 point Likert scale 1=strongly disagree 5=strongly agree
1) 0 2) 2
3) 15 4) 51
5) 34 Mean=4.15
0.74 102
Document written in plain
language (Complexity)
Recommendations
were written in plain language
5 point Likert scale 1=strongly disagree
5=strongly agree
1) 0
2) 5 3) 13
4) 54 5) 30 Mean=4.07
0.79 102
Ease of Understanding (Complexity)
The document (itself) was easy to
understand 5 point Likert scale
1=strongly disagree 5=strongly agree
1) 0 2) 3
3) 18 4) 52 5) 29
Mean=4.05
0.76 102
Improvement over current
practice (Relative advantage)
Recommendations
were an improvement over current practice
5 point Likert scale 1=strongly disagree
5=strongly agree
1) 0
2) 7 3) 21 4) 49
5) 25 Mean=3.90
0.85 102
Ease of understanding Recommendations were difficult to
understand 5 point Likert scale 1=strongly disagree
5=strongly agree
1) 16 2) 4
3) 15 4) 67 5) 0
Mean=3.30
1.12 102
Ease of Use (Triability) Recommendations
would be difficult to try out
5 point Likert scale 1=strongly disagree
5=strongly agree
1) 7
2) 22 3) 27 4) 46
5) 0 Mean=3.10
0.97 102
73
Compatibility- personal Recommendations fit with personal beliefs
5 point Likert scale
1=strongly disagree 5=strongly agree
1) 1 2) 1 3) 22
4) 56 5) 21
Mean=3.94
0.75 102
Compatibility- organization Recommendations fit w/organization‟s
values
5 point Likert scale 1=strongly disagree 5=strongly agree
1) 2 2) 1
3) 17 4) 67
5) 15 Mean=3.90
0.72 102
Ease of seeing results
(Observability)
Recommendations
would present results easy to see
5 point Likert scale 1=strongly disagree 5=strongly agree
1) 1
2) 17 3) 44
4) 35 5) 2 Mean=3.25
0.83 102
Customizable Recommendations
are customizable to my organization
5 point Likert scale 1=strongly disagree
5=strongly agree
1) 2
2) 12 3) 37
4) 46 5) 4 Mean=3.38
0.82 102
74
Perception of Research,
Communication and
Dissemination Characteristics
Staying current in research Importance of staying current in
research 5 point Likert scale
1=strongly disagree 5=strongly agree
1) 1 2) 0
3) 2 4) 38 5) 61
Mean=4.55
0.64 102
Application of research Research from
university/research institutions do not directly apply
5 point Likert scale 1=strongly disagree
5=strongly agree
1) 7
2) 51 3) 28 4) 13
5) 3 Mean=2.55
0.91 102
Accessing research Difficult for staff to access research in
their communities 5 point Likert scale
1=strongly disagree 5=strongly agree
1) 3 2) 43
3) 13 4) 36 5) 7
Mean=3.00
1.09 102
Expectation of staff to remain
current on research
Management
expects staff to keep on top of research
5 point Likert scale
1=strongly disagree 5=strongly agree
1) 1
2) 7 3) 18 4) 60
5) 16 Mean=3.81
0.82 102
Expectation of staff to read
journals
Expectation of staff
to read MH&A journals
5 point Likert scale 1=strongly disagree
5=strongly agree
1) 1
2) 6 3) 22
4) 57 5) 16 Mean=3.79
0.81 102
75
Respondent seeks out research Management seeks out research
5 point Likert scale 1=strongly disagree
5=strongly agree
1) 1 2) 4 3) 13
4) 55 5) 29
Mean=4.05
0.81 102
Respondent distributes research to staff
Management distribute research information to staff
regularly
5 point Likert scale 1=strongly disagree 5=strongly agree
1) 1 2) 4 3) 13
4) 59 5) 25
Mean=4.00
0.79 102
Accessibility of research format Most research is presented that is not
accessible by staff 5 point Likert scale
1=strongly disagree 5=strongly agree
1) 2 2) 43
3) 27 4) 23 5) 7
Mean=2.90
1.00 102
Awareness of research Research findings
in MH&A are difficult to find out about
5 point Likert scale 1=strongly disagree
5=strongly agree
1) 2
2) 52 3) 25 4) 23
5) 2 Mean=2.74
0.90 102
Obtain ability of research Research findings
in MH &A are difficult to obtain
5 point Likert scale 1=strongly disagree
5=strongly agree
1) 2
2) 52 3) 23 4) 23
5) 2 Mean=2.72
0.91 102
76
Understanding research Research findings
in MH&A are
difficult to understand
5 point Likert scale 1=strongly disagree
5=strongly agree
1) 4 2) 55
3) 27 4) 14
5) 2 Mean=2.56
0.85 102
Application of research Research findings
in MH&A are difficult to apply
5 point Likert scale 1=strongly disagree
5=strongly agree
1) 2
2) 30 3) 32
4) 30 5) 8 Mean=3.12
0.99 102
Willingness to try ideas from research
Willing to try new ideas
5 point Likert scale 1=strongly disagree
5=strongly agree
1) 0 2) 0
3) 0 4) 46 5) 56
Mean=4.55
0.50 102
Need to change practice See a need to
change practice 5 point Likert scale
1=strongly disagree 5=strongly agree
1) 1
2) 13 3) 19 4) 56
5) 13 Mean=3.66
0.90 102
Benefits of changing practice See the benefits of
changing practice 5 point Likert scale
1=strongly disagree 5=strongly agree
1) 0
2) 2 3) 11 4) 66
5) 23 Mean=4.08
0.64 102
Importance of best practice
documents
Best practices are
considered to be important
5 point Likert scale 1=strongly disagree 5=strongly agree
1) 1
2) 0 3) 2 4) 49
5) 50 Mean=4.44
0.64 102
77
Individual Characteristics
There were seven individual characteristics examined in this study such as sex, age, highest
level of education obtained, years in current position and other variables related to the
individual‟s experience and involvement in professional development. Respondents were
classified into four distinct positions, of which 20.8% were front line staff, 11.9% were
supervisors, 34.7% were managers and 32.7% were executive directors. Fifty percent of
respondents had been at their organization for 10 years or longer. Respondents had a mean
birth year of 1957 and for 49% a Master‟s degree was the highest level of education. The
field in which the respondents had focused their careers to date was mental health (only)
34.7%, addictions (only) 33.7%, social work 13.9% and both mental health and addictions
(field of concurrent disorders) 16.8%. The majority (64.6%) had attended between one and
three conferences in the past year.
Thirty percent of the respondents were males and 70% were female.
Appropriate tests were performed to identify which individual characteristics were
significantly associated with adoption. Significant relationships will be further discussed in
the regression modeling section.
Organizational Characteristics
The ten organizational characteristics included 1) organization‟s primary focus, 2) special
populations that the organization serves, 3) number of staff employed at the organization 4)
number of volunteers affiliated with the organization 5) organization‟s specific functions
(e.g. screening, assessment, treatment, referrals, case management) 6) whether the
organization serves a rural, urban or suburban catchments area 7) number of other mental
health/addiction organization in the area, 8) number of programs and services the
78
organization offers 9) number of management levels in the organization and 10) the average
number of clients the organization serves on a monthly basis.
With respect to the organizational context variables, 33.7% of the organizations employed
over 50 employees and only 7.1% of organizations employed less than 6 people. With
respect to management levels, (45.5%) of the organizations had two management levels;
only 9.1% had five or more levels. More levels of management represent greater complexity
of the organization for making decisions about changes in practices. With respect to number
of programs, there was approximately equal representation of organizations in each
category, with 26.3% of organizations offering between 1-3 programs/services, 26.3%
offering 4-6 programs/services, 20.2% offering 7-10 programs and services and 26.3%
offering more than 10 programs and services. With respect to the number of clients seen on
a monthly basis, 75.5% of the organizations reported seeing over 50 clients each month and
1.0% of organizations reported seeing less than 10 clients per month. With respect to
number of volunteers, 40.5% of the organizations had less than 6 volunteers, and 4.8% of
the organizations had between 31-50 volunteers. With respect to services offered, most
organizations specialized in screening (80.4%), assessment (85.3%), referrals (88.2%),
treatment (86.3%) and case management (84.2%). Most organizations served urban
communities at 27.7% and only 7.9% served suburban areas. In terms of special populations
served, 64.4% served youth, 72.0% served women, 66.0% served seniors and over 50%
served gay, bisexual or lesbian clients.
Appropriate tests were performed to identify which individual characteristics had
significance with adoption. Significant relationships will be further discussed in the
regression modeling section.
79
Innovation Characteristics
Five innovation variables were used to assess the adoption of the recommendations
(innovation) in the Best Practice document. This included 1] relative advantage, 2]
compatibility, 3] complexity, 4] triability and 5] observability.
Relative advantage was measured by asking respondents if the best practice
recommendations, would improve current practice. Just over 72% of respondents indicated
that the recommendations would improve current practice, hence a perception that the
innovation as being better than the current idea/practice. Compatibility was measured by
asking respondents whether the recommendations fit with the personal beliefs of the
respondents, and whether they fit with the values of the organization. Just over 75% of
respondents indicated that the recommendations fit with their personal beliefs and 80.4%
indicated that they fit with the organization‟s values. Complexity was measured by asking
whether the recommendations were clearly presented in the document, if the
recommendations were in plain language, and whether the document was easy to understand
overall. The respondents were very positive; 83.3% considered the recommendations to be
clearly presented and 82.3% perceived the document to be written in plain language,
although 79.4% indicated that the recommendations were difficult to understand. Triability
was measured by asking respondents whether the recommendations were difficult to use,
and whether they could be customized to their specific organization. Fewer than half of
respondents or 45.1% agreed that it would be difficult to use the recommendations and
49.0% indicated that the recommendations could be customized. Observability was
measured by asking the respondents whether the recommendation results would be easy to
see. Only 38.9% said the recommendations in the Best Practice document would produce
80
results that were easy to see. These results suggest that although the innovation was
considered to be written in plain language, it was not easy to understand. Furthermore, there
were concerns on how results would be observed and if the recommendations could be
customized for the organization to adopt.
Perception of the Innovation
The sixteen questions based upon the belief and perception of research and changing clinical
practice assessed the respondents‟ perception of the innovation and included the importance
of research, understanding research findings, the benefits of research, and the application of
research findings. Respondents identified that keeping on top of current research is essential
to professional performance (97.1%) and they actively seek research out (82.3%). Fifty
seven percent found research findings applicable to their work and 42.2% found research
findings to be difficult to access, while the perception of difficulty in obtaining research was
indicated by 24.5% of the respondents. Strikingly, all respondents are willing to try new
ideas found in research (100%) and 67.6% agreed that change is needed in clinical practice
for mental health and addictions to move forward; 87.2% of the respondents perceive the
benefits of changing practice in the field, and an overwhelming 97% consider best practice
documents important. This would indicate the respondents perceive and appreciate research
and changing practice, however, they acknowledge that there are challenges in accessing the
research itself.
81
Presentation-communication of the innovations
The two characteristics based upon how the innovation is a) presented and b) communicated
within the documents are important to identify separately due to the ability of the
recommendations (the innovation) to be translated in to terms that could be easily
understood and adopted by the respondents, as this could be a barrier in the adoption of the
innovation. This was addressed by asking if the recommendations in the document were
clearly presented. A response rate of 83.3% indicated that they were, while 82.3% agreed
that the document was in plain language and 79.4% agreed that the recommendations were
easy to understand.
Dependent Variable
Adoption Decision
Respondents‟ adoption was initially assessed by a variable that provided different levels of
adoption, from “considered & rejected” to “implemented & adapted” on a six dimensional
scale. Each of the items in the adoption decision section is based upon the actual
recommendation in the Best practices document. Descriptives of the 27 items that were
collapsed into the adoption decision are found in Table 7.
82
Table 7: Descriptives of Best Practice Recommendations
Best practices Recommendations N Mean SD
Q49- Organization routinely screens for substance use 102 4.25 1.10
Q50- Organization routinely screens for mental health 102 4.43 1.05 Q51- Organization routinely screens for both substance use and mental health
102 4.10 1.06
Q52- Organization routinely does a Level 1 screening for substance use
102 4.18 1.30
Q53- Organization routinely does a Level 2 screening for
substance use 102 3.60 1.56
Q54- Organizational routinely does a Level 1 screen for mental
health 102 4.14 1.35
Q55- Organizational routinely does a Level 2 screen for mental health
102 3.50 1.69
Q56- Organization has access to psychiatric consultation 102 3.75 1.55 Q57- Organization has a policy for assessing CD 102 3.10 1.54
Q58- Organization uses SCID-IV 102 2.16 1.52 Q59- Organization uses clinical rating scales 102 3.19 1.61 Q60- Organization uses addictions severity index 102 3.00 1.68
Q61- Organization uses motivational interviewing 102 4.02 1.31 Q62- Organization uses stages of change 102 4.16 1.23 Q63- Organization uses psychosocial functioning 102 4.36 1.06
Q64- Organization is involved with integrated treatment 102 4.09 1.19 Q65- Organization is involved in program integration 102 3.78 1.39
Q66- Organization is involved in system integration 102 3.89 1.27 Q67- Organization is involved in provincial policy initiatives 102 3.29 1.67 Q68- Organization refers to other services in the community 102 4.67 0.82
Q69- Organization belongs to a formal network 102 4.42 1.12 Q70- Organization is involved with blended/integrated service
delivery teams from members of MH and SU agencies 102 3.77 1.44
Q71- Organization has CD treatment as part of its mandate 102 3.52 1.57 Q72- Organization formally collaborates amongst community
networks 102 4.63 0.77
Q73- Organization is aware of others using the recommendations 102 3.53 1.53 Q74- Organization has formal, internal communication paths 102 4.21 1.18
Q75- Organization has formal, external communication paths 102 3.88 1.27
A factor analysis was done to determine whether the research resulted in any possible
dependent variables other than the one hypothesized. The overall adoption decision was the
mean calculation with missing values replaced with the value of “nothing done”. The mean
83
adoption decision was 3.84 with a standard deviation of 0.70 which indicates that, on
average, respondents are between “taking a step toward innovation implementation” and
“innovation has been partially implemented”. Distribution of the adoption decision is shown
in Figure 6.
4.002.00
ADOPTION
20
15
10
5
0
Fre
qu
en
cy
Mean =3.837Std. Dev. =0.69983
N =102
Figure 6: Histogram of the Distribution of Adoption
Multiple Linear Regression
A multiple linear regression analysis was conducted to see which variables were predictors
of the adoption of the innovation- the recommendations in the Best practices document.
In order to identify which variables would be included in the multiple linear regressions, a
factor analysis was performed to see if any of the continual variables would collapse into
common dimensions. For the categorical variables, independent t-tests and one-way
84
ANOVAs were performed to further identify which variables would be included in the
model.
Individual Characteristics
An independent samples t-test was conducted to evaluate sex, and an analysis of variances
(one-way ANOVA) was performed on career field. Correlation was performed on year of
birth, highest level of education, current position, length of employment at current
organization (tenure) and number of conferences (cosmopolitism) attended in the past year.
Any variable that had significance less than 0.05 (two tailed) was identified as being
correlated. No significance was found within the independent t-tests or ANOVAs
performed. Significance was found with length of organization (tenure) (0.010) and number
of conferences (cosmopolitism) (0.032), as shown in Table 8. Thus these independent
individual characteristic variables were included in the regression model.
Organizational Characteristics
An independent samples t-test was conducted to evaluate the special populations that the
organization served. This included youth, women, seniors and lesbian, gay, bisexual or
transgender. Analysis of variances (one-way ANOVAs) was performed on the following
independent variables; organization‟s primary focus, the type of services the organization
offered and the geography mix of the population served. It was found that screening
services; t(102)=3.680, p=0.006, assessment services; t(102)=3.995, p=0.013 and case
management; t(102)=3.017, p=0.004 were significantly associated with adoption decision.
(Table 9) and were included in the model. Correlation was performed on the number of staff
85
within the organization, the number of volunteers within the organization, the number of
management levels, yearly average of the number of programs offered, and the monthly
average of the number of clients/patient served. The correlation coefficients with these
organizational variables were not significantly correlated with adoption decision. Hence, the
only organizational characteristics that were included in the regression model were whether
the organization provides screening, assessment or case management.
Environment
Correlation analysis was performed against adoption, with the independent variables that
represented the organizational. This included the number of other mental health/addiction
organizations in the same regional municipality and the number of management levels in the
respondent‟s organization. Neither variable had a significant correlation with adoption
decision. Georgraphical area was previously run using a one-way analysis of variance
(ANOVA) and was found not to have significance with adoption. No environment
characteristics were included in the regression model.
Attributes of the Innovation
Correlations were run between the perceived innovation attributes scores and the adoption
decision. These included the variables that were identified in the factor analysis, i.e.,
complexity of communicating research findings and innovation characteristics. One of these
variables was significant. Thus, no attributes of the innovation were included in the
regression model.
86
Perception of Research, Communication and Dissemination
Correlation coefficients were calculated between the innovation attribute independent
variables and the adoption decision score. These variables included the items that were
identified in the factor analysis as: Importance of Research, Research findings being
understood, Benefits of Research, and Application of Research findings. None of the
correlations were significant. Thus, no perceived characteristics of the innovation were
included in the regression model.
Table 8: Significant Correlations between Adoption and Independent Variables
Pearson Correlation (with Adoption)
Sig. 2 Tailed (with Adoption)
Individual Characteristics
Year of Birth 0.102 0.320
Highest Level of Education 0.075 0.458
Current Position -0.032 0.751
Length at Organization (Tenure) 0.256** 0.010
Number of Conferences attended
(Cosmopolitism)
0.126* 0.032
Organizational Characteristics
Number of staff 0.111 0.275
Number of volunteers -0.195 0.075
Number of management levels -.002 0.983
Number of program 0.175 0.083
Number of clients (monthly) 0.181 0.069
Environment
Number of other similar organizations in same
region
0.053 0.619
Number of management levels -0.002 0.983
Attributes of the innovation
Communication Complexity 0.166 0.095
Innovation Characteristics 0.193 0.054
Perception/Beliefs of Research
Importance of research 0.056 0.579
Access to research -0.140 0.161
Benefits of research 0.048 0.631
Application of research -0.165 0.097
*. Correlation is significant at the 0.05 level (2-tailed)
**. Correlation is significant at the 0.01 level (2-tailed)
87
Table 9: T-test of Categorical Independent Variables- Levene’s Test for Equality of
Variances
F Sig. t Sig.(2-tailed)
Individual Characteristics
Sex 0.125 0.724 0.959 0.340
Organizational Characteristics
Youth served 0.141 0.708 1.392 .167
Seniors served 0.149 0.700 -0.074 0.941
Women served 0.002 0.967 -0.152 0.879
LGBT served 2.450 0.121 -0.045 0.964
Screening 7.785 0.006 3.680 0.000
Assessment 6.432 0.013 3.995 0.000
Referrals 1.034 0.312 0.847 0.399
Treatment 0.794 0.375 3.762 0.000
Case Management 8.778 0.004 3.017 0.003
Based on these analyses, the following independent variables were included in the multiple
regression analysis; tenure, cosmopolitism, screening, assessment, and case management.
The final regression was done using a stepwise method, with all significant independent
variables, which included tenure, cosmopolitism, screening, assessment and case
management and the dependent variable of overall adoption. The first step (Table 10) had
an R square change of 0.062 with the significance F change of 0.014. Step 2 had a R square
change of 0.096 with a significant F change of 0.001. This meant that the final step in the
regression analysis explained 9.6% of the variation in the reported adoption of the
recommendations found in the Best Practice document. The variables that showed
significance in the final model were the individual‟s tenure, with a standardized Beta of
0.203 and a p value of 0.036 (Table 11), and the provision of screening (organizational
variable), with a standardized Beta -0.313 and a p value of 0.001 (Table 11). This means
that, as the individual‟s tenure increases, so does the level of the adoption decision. With
88
respect to services offered, it means that, if the organization provides screening services, the
adoption decision will be less advanced (i.e.,the final decision to adopt or reject the
recommendations will not have been made).
Table 10: Stepwise Regression Model Summary
Step R Square df1 df2 Sig .F Change
1 .062 1 95 .014
2 .096 1 94 .001
Table 11: Stepwise Regression Model- Coefficients- Dependent Variable: Adoption
Step Variables Standardized β Unstandardized β t Sig.
1 Length at organization .249 .129 2.504 .014
2 Length at organization
Screening
.203
-.313
.106
-.547
2.127
-3.274
.036
.001
Table 12- Stepwise Regression Model F test for significance
Regression df Total df F Sig.
Step 1 1 96 6.271 0.014
Step 2 2 96 8.815 0.000
89
5.0 DISCUSSION
This paper has presented the results of a one-year follow-up of the adoption of
recommendations that stemmed from a synthesis of research. The research was a
publication, a first of its kind to focus on Concurrent Disorders, a combined focus on best
practices for mental health and substance use disorders within Canada. The publication was
a synthesis of research that included 27 recommendations regarding the clinical and
administrative management of Concurrent Disorders.
The level of adoption was measured at the level of the organizational leader, which
encompassed the characteristics of the leader as an individual, the characteristics of the
organization, the environment within which the organization operates, the perceptions the
organization has about research, and the attributes of the innovation.
The selection and definition of the independent variables and the dependent variable of
adoption were based on Roger‟s Diffusion of Innovations theory.
In this study, the innovation was the best practice guide, Best practices for Concurrent
Mental Health and Substance Use Disorders, which comprised 27 recommendations. These
recommendations were based upon a synthesis of research to support the “no wrong door”
philosophy, such that regardless of the entry point an individual would have for the
treatment of their mental health issue or substance use , the individual would be screened,
assessed and treated using the same methodology.
This study was the first of its kind in Ontario and Canada to assess the influence of a variety
of variables on the adoption of best practices, as they relate to both mental health and
substance use disorders, in the context of organizations and organizational leaders.
90
5.1 Degree of Adoption within Organizations
Research Question #1
To what degree did the organizations make a considered adoption decision regarding
the innovation?
From the results, we see that the mean average of organizations is in between „having steps
towards implementation‟ (3) and „being partially implemented‟ (4), with a mean of 3.84
(Table 5). This indicates that one year after the dissemination of the recommendations, on
average, the organizations were on their way to adopting the recommendations, but were not
quite „there‟ yet. Examining the distribution of the adoption levels within the organizations,
the histogram (Figure 6) shows the shift to the right, demonstrating the majority of the
organizations were on the path towards a final decision of adopting or rejecting the
recommendations. Thus, the degree to which organizations adopted the innovation into
practice could be characterized as partial, i.e., most organizations had steps towards
implementation, and most have not fully implemented the innovation.
The level of adoption, according to the literature, is based upon various factors such as
individual, organizational, and environmental characteristics. Other factors include the
perceptions of the innovation (as research) and the characteristics of the innovation itself.
91
5.2 Characteristics and Adoption
Research Question #2
What were the organizational, individual, environmental, and innovation
characteristics associated with the adoption decision?
Research has identified a number of characteristics that are associated with this degree of
adoption, including individual, organizational and environmental characteristics, the
characteristics of the innovation itself, people‟s perception of research and the way the
innovation is communicated.
Individual characteristics, i.e., basic characteristics such as age, sex, level of education,
values, beliefs, tenure and cosmopolitism, have a relationship with the level of adoption. In
this study, tenure and cosmopolitism were shown to be significant factors with the degree of
adoption. This is consistent with the literature as well as with Rogers‟ model, where
innovators and early adopters are characterized as individuals with experience and with
exposure to other ideas and experiences (cosmopolitanism). Furthermore, in this study, it
was found that tenure was another positive predictor for adoption, i.e., 9.6% of adoption was
predicted by tenure. This fits with the literature and other studies, as tenure represents
experience that would demonstrate a high level of integration into the social system of
mental health and substance use disorders and hence someone with tenure would be
considered as an opinion leader and change agent.
The organizational characteristics that were considered in this study were both structural
and dynamic. Previous research found that organizational size can influence adoption.
McGowan and Madey found positive correlation between the level of management support
92
and the size of the organization to affect adoption. In this study, size and managerial support
was not found to be significant with adoption; however, the type of services the organization
offers was significant. The provision of screening, assessment and case management was
found to have significance with adoption. This significance aligns well with the intent of the
innovation (namely the 27 recommendations), since case management methodology is used
in both mental health and substance use disorder organizational settings. Screening and
assessment are also key elements within the organization to support the “no wrong door”
philosophy, which holds that, regardless of where the client enters the health care system,
the organization, at a minimum, would provide screening and assessment for both mental
health and substance use.
Though screening was demonstrated to have significance with adoption, in the regression
model to determine prediction, it was found that the Beta coefficient was negative. This
means that, as the organization increases its level of screening activities, the level of
adoption would decrease, due to an inverse relationship. Further research would be needed
to explore this phenomenon, but it may be a result of over-saturation of activities the
innovation represents. If an organization is fully focused on screening, it may not have the
resources to pursue the other recommendations within the innovation.
Environmental characteristics, which include the levels of management, location and the
awareness of other similar organizations in the area, were not significant with the degree of
adoption and none were identified as predictors. This may be a result of the uniqueness of
how mental health and substance use disorder organizations interact with their respective
environments.
93
Attributes of the innovation included complexity, triability, compatability, observability and
relative advantage. Though the literature and Rogers model demonstrates a strong positive
relationship between characteristics of the innovation and adoption, in this study, none of the
characteristics of the innovation demonstrated significance with, nor were they a predictor
of, adoption. This may be explained by similar research done by Damanpour, where it was
found that, as the number of multiple innovations that are introduced increases , the
influence that the attributes of the innovation have on adoption decreases (though in this
case, the 27 recommendations within the Best Practice document were considered to be one
innovation). Further research would be required to examine the 27 recommendations as
separate innovations.
Perceptions of research includes how the research from which the innovation is developed
is viewed, from the perspective of importance, accessibility, understanding and the need
and/or benefit of research. Although the literature that examines the relationship between
values/beliefs and adoption, i.e., how research is valued and perceived (with general
attitudes at both the individual and organizational level) shows a relationship with adoption,
in this study, a relationship does not exist.
94
5.3 Implications
Rogers‟ model states that innovations perceived by individuals as having greater relative
advantage, compatibility, trialability, observability, and less complexity will be adopted
more rapidly than other innovations. Based on these attributes, the best “innovation” for
rapid adoption would be an evidence-based treatment that was simple, similar with previous
practice, had clear advantage, could be tried out temporarily and was readily observable. In
this study, most organizations had, to some degree, adopted the innovation and most had
fairly positive perceptions of the innovation‟s advantage, compatibility and complexity.
The study found that the most significant predictor of adoption of the innovation was not the
attributes of the innovation, but rather the characteristics of the individuals with leadership
roles in the organizations. Specifically, individual characteristics such as tenure and
cosmopolitanism in an organization‟s leaders were key predictors for a positive degree of
adoption.
5.4 Recommendations for Future Research
As adoption studies continue to grow in the health care sector in Canada; there is very little
that specifically examines adoption predictors, accelerators and, specifically, barriers for
mental health and substance use treatment. Further research should be done on how mental
health and addiction organizations adopt innovation in these two very different sectors, so
that it may inform future diffusion of research and possibly change clinical practice further.
This is especially important, not just to health communities, but also to those generating the
research.
95
In this study, the innovation was contained within a national “best practices” guide; further
investigation into the adoption of other published “best practices” documents in other health
fields is warranted, to examine the degree of adoption, the characteristics that lead to
adoption and the outcomes of investing resources in producing such publications by Health
Canada. It would be interesting to further follow up on the innovation-adoption process to
examine the rate of adoption versus which organizations have fully progressed to a complete
adoption, after more time has passed. In addition, it would be recommended to do a further
follow up (e.g. 3, 5 year post publication) of the adoption of the recommendations with the
same organizations, as well as to investigate other provincial adoption and national
adoption. Further research could also include examining the level of readiness for the
organization to adopt the recommendations; the clinician vs. administrator perspective, as
many organizations may have their respective organizational leaders in both roles; a
comprehensive assessment of resources available to the organization; the organizational
climate and culture; flexibility of the organization; leadership turnover to include any
possible mergers of organizations; further research into the characteristics of the non-adopter
organizations and identifying the differences between adopting and non-adopting
organizations; political climate; level of sustainability and a comparison of the self-report
done within this research vs. additional research using trained independent observers to
assess level of adoption.
Further research would be required to identify how pre- implementation beliefs would affect
the level of adoption.
It would also be beneficial to measure the level of adoption with a qualitative approach,
where respondents would indicate what they have done with the recommendations, as the
96
adoption scale may have not captured the activity. Identifying barriers for non-
implementation (e.g. resources available, political climate) would also be an addition to the
growing literature. Even within the best practice document, it was identified as a key point
that more research is needed regarding the impact of best practice guidelines on the attitudes
and behaviour of health care policy makers, planners and providers on consumer health
outcomes, as they relate to mental health and addictions (Health Canada, 2001).
5.5 Ethical Issues and Considerations
The potential to cause harm to respondents was relatively small in this study. However, to
minimize/reduce risk, two steps were taken. First, in order to minimize any perceived
pressure that respondents might have experienced to participate in the research (e.g.
perceived pressure from the Centre for Addiction and Mental Health, the Ministry of Health
and Long Term Care- Mental Health and Addictions Branch or Health Canada), the
introductory letter stressed that participation was voluntary. Second was an issue of
confidentiality. It would have been unethical to report individual results that would identify
specific organizations, when the participation was confidential. This was resolved by
informing all participants that results would be reported only in aggregate form.
5.6 Limitations of the Study
There were several limitations of this study, which include missing respondents in the
sampling and small sample sizes. Though many of the respondents completed the first three
sections, a few did not complete the section that pertained directly to adoption and the
degree to which the organization adopted the recommendations. This may be as a result of a
97
perception in the community that the results of the study would impact the organization‟s
programs and services as it relates to funding provided by provincial and national health
funding bodies. In addition, as the non-response rate was high (53%) this may be an
indication of the individual/organizations that did not respond, were non adopters; where the
lack of participation was a characteristic of being a non-adopter. Based upon this hypothesis,
the sample then would have only included adopters, which would provide a bias for ratings
to be high on the adoption scale. This would impact the results as having a pre-existing
positive bias for adoption. Further research would be required to investigate this hypothesis.
As data was calculated on a limited number of individual, organizational, innovation and
environmental characteristics, there are gaps in identifying additional predictors, as the final
model of tenure and screening explained approximately 10% of adoption.
The research is also limited to the existence of pre- implementation beliefs of the
organizations and the differences of those pre-implementation beliefs.
Another limitation is the individual-blame bias (Rogers, 2003). This bias indicates when
something does not go as expected, it is the fault of the system vs. the individual. By
conceptualizing full implementation as the final step of the diffusion of the best practices, it
would be acknowledged that if Health Canada did not contribute to the adoption process,
then the health care system is responsible for the non-adoption.
The definition of screening may also be considered as a limitation. The term “screening” can
have a different definition to the mental health community, where standards are more based
in clinical standards versus the substance abuse treatment community, where basic general
98
questions, could be considered as a screening process. The best practice document highlights
it is not possible to recommend one approach or screening tool, however what it does
recommend is a two-tier screening process, where the Level 1 screening can include tools
with psychometric properties or may be a few simple questions. Level 2 screening tools are
clearly identified, as they require more time, expertise and have been tested for
psychometric properties. The best practice document does list what it would consider to be
a general accepted list of questions that would represent Level 1 screening. These examples
were included in the survey but were not the complete exhaustive list. Hence, depending on
the respondent, there may have not been full clarity regarding what activities would be
considered Level 1 versus Level 2 screening.
99
6.0 CONCLUSIONS
According to a study done by Marinelli-Casey et al (2002), the critical factor contributing to
the gap between research and practice is the lack of communication and cooperation
between researchers and practitioners/clinicians. That though the two groups do interact at
times, constructive communication has been absent. Researchers and practitioners/clinicians
have made little effort to understand or accept the relevance of each other‟s knowledge.
This can be said as well for the mental health and substance abuse treatment community, in
relation to the history and traditional background of how each field developed (Health
Canada, 2001).
In the case of the development of the research contained within the Best Practices for Co-
occurring Mental Health and Substance Use Disorders, there was recognition for the two
fields working together, as planners, policy makers and service providers in the fie lds of
mental health and substance abuse treatment were finding themselves on common meeting
ground at a departmental, regional and provincial level. The creation of the best practices
document is evidence that there was cooperation and communication between researchers
and practitioners/clinicians, with a demonstrated effort of understanding and accepting each
others knowledge. However, even with this bridge between the two communities, there was
a question of whether or not the best practices were adopted into each others respective
communities.
This research focused specifically on the follow up of the adoption, at a one-year post
dissemination study of the population of both mental health and substance use treatment
100
organizations that participated in a videoconference in 2003 in Ontario. This study asked if
the organizations are adopting and how far along they are in adopting the recommendations
found within a synthesis of research, presented as a “best practice” publication. This was an
important study in Ontario for mental health and substance use treatment, as it examined the
characteristics of the individual, the organization, the innovation, and the communication,
and the relationships between these variables and adoption. The most important finding to
emerge from this research was the importance of individual (tenure) and organizational
characteristics (organization offering the service of screening). However, as Glanz has
already suggested, in fields such as mental health and substance use where there is less
consensus about practice and treatment approaches, the factors and barriers in adoption may
be greater and other unidentified characteristics may be involved. These characteristics may
include such items as ease of implementation, the fit of what is currently being done at each
organization, cost effectives, and the response to the expressed clinical need. Barriers exist
and need to be further examined to identify the specific practices that require a greater effort
to adoption and be integrated with an approach that is acceptable to the organization. In
addition, though the best practices document provided recommendations of practice, it did
not provide a “blue-print” on how organizations, with varying characteristics and resources,
could implement and sustain the recommendations. Rush (2003) states that though the
implementation of the best practice document is essentially a provincial responsibility, the
dissemination process could benefit from more focus and leadership at a national level.
Backer et al (2000) identifies that there are two kinds of repetitive mistakes in dissemination
practice, both in substance abuse and other fields to include 1) decision level mistake;
spending scarce resources on publications that are not user friendly and therefore are
101
unlikely to results in actual innovation by their intended target audiences and 2)
implementation level mistakes: where in order to address the complex human dimensions of
changes that are required to implement innovations, there needs to be sufficient resources or
omit attention to issues that funders consider to be blind spots. Backer further suggests that
four science-based principles of effective dissemination to used; user- friendly
communication of the innovation, user-friendly evaluation of the innovation- such that it
works better and does not have significant side effects, resource adequacy and addressing
the complex human dynamics of change- where change activities are rewarded and there is
assistance for implementation.
Hence, the findings of this research will provide direction and guidance for future
dissemination of best practices and adoption with respect to the uniqueness of the mental
health and addiction use treatment community. Though the research did not explore all
possible factors and predictors it did identify individual characteristics such as tenure and
cosmopolitanism in an organization‟s leaders being key predictors for a positive degree of
adoption. An important conclusion therefore, is that a leader or champion is critical to
having a positive rate of innovation adoption within an organization.
102
7.0 REFERENCES
1. Aarons, G.A. (2005). Measuring provider attitudes toward evidence-based practice: Organizational context and individual differences. Child and Adolescent Psychiatric
Clinics of North America, 14, pages 255-271. 2. Aarons, G.A., Sawitzky, A.C.(2006) Organization Culture and Climate and Mental
Health Provider Attitudes Toward Evidence-Based Practice. Psychological Services, 2006, Vol.3, No.1, pages 61-72. American Psychological Association.
3. Aldrich, H.E. & Auster, E.(1986). Even dwarfs started small: Liabilities of age and size and their strategic implications. Research in Organizational Behavior. L.L.
Cummings and B.B. Staw (eds). Greenwich, CT: JAI Press. 4. Anderson, N., & King, N. (1992) Innovation in organizations. International Review
of Industrial and Organizational Psychology. Cl. Cooper and I.T. Robertson (eds), pages 1-34. New York; Wiley
5. Backer, T.E. (2000). The failure of success: Challenges of d isseminating effective
substance abuse prevention programs. Journal of Community Psychology, 28, 363-
373.
6. Banta, H.D. (1983). Social science research on medical technology: Utility and
limitations. Social Science of Medicine, 17, pages 1363-1369.
7. Bartholomew, K.L., Parcel, G.S., Kok, G. & Gottlieb, N.H.(2001). Intervention Mapping: Designing Theory and Evidence-based Health Promotion Programs. New York: McGraw-Hill Companies, 2001.
8. Bero, L.A.,Grilli,R., Grimshaw, J.M., Harvey, E., Oxman,A.D., Thomson,
M.A.(1998). Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. British Medical Journal, 1998:3 17, pages 465-468.
9. Berta,W., Teare.G.F., Gilbart,E., Soberman-Ginsburg, L., Lemieux-Charles, L.,
Davis, D., Rappolt, S.(2005). The Contingencies of Organization Learning in Long-
Term Care: Factors That Affect Innovation Adoption. Health Care Manage Review, 2005, 30(4), pages 282-292.
10. Beutler, L E., Williams, R.E., Wakefield, P.J., Entwistle, S.R.(1995). Bridging
scientist and practitioner perspectives in clinical psychology. American Psychologist,
50(12), Dec 1995, pages 984-994.
11. Buchan, H., Sewell, J.R., Sweet, M.(2004). Translating evidence into practice. Medical Journal of Australia, Vol. 180, pages 43-44.
103
12. California Healthcare Foundation, (2002). Diffusion of Innovation in Health Care. Institute for the Future, May 2002.
13. Camison-Zoromcza, C., Lapiedra-Alcami, R., Segarra-Cipres, M., Boronat-Navarro,
M.(2004). A Meta-analysis of Innovation and Organizational Size. Organization Studies 25(3) pages 331-361.
14. Carmazzi, A., & Aarons, G. A. (2003, February). Organizational culture and attitudes toward adoption of evidence-based practice. Presented at the MASMHPD Research Institute‟s 2003 Conference on State Mental Health Agency Services
Research, Program Evaluation and Policy, Baltimore, USA.
15. Corrigan, P.W., Steiner, L., McCracken, S.G., Blaser, B., Barr, M. (2001). Strategies for Disseminating Evidence-Based Practices to Staff Who Treat People With Serious Mental Illness. Psychiatric Services Volume 52: pages 1598-1606, December.
American Psychiatric Association.
16. Cockburn, J. (2004). Adoption of evidence into practice: can change be sustainable? Medical Journal of Australia, Volume 18, pages S66-S67.
17. Damanpour, F., (1991). Organizational Innovation: A Meta-analysis of Effects of Determinants and Moderators. Academy of Management Journal: September 1991;34,3, pages 555-590
18. Denis, J., Hebert, Y., Langley, A., Lozeau, D., Trottier, L. (2002). Explaining
Diffusion Patterns for Complex Health Care Innovations. Health Care Management Review, Volume 27(3), July 2002, pages 60-73.
19. Dobbins, M., Cockerill, R., Barnsley, J.(2001) Factors Affecting the Utilization of Systematic Reviews. International Journal of Technology Assessment in Health Care
17:2 (2001), pages 203-214. Cambridge University Press
20. Dobbins, M., Ciliska, D., Cockerill, R., Barnsley, J., DiCenso, A. (2002). A
Framework for the Dissemination and Utilization of Research for Health-Care Policy and Practice. The Online Journal of Knowledge Synthesis for Nursing, Volume 9, Document Number 7.
21. Drake, R.E., Essock, S.M. , Shaner, A., Carey, K.B., Minkoff, K., Kola, L., et al.
(2001). Implementing dual diagnosis services for clients with severe mental illness. Psychiatric Services, 52, 469-476.
22. Dunn, W.N. (1983). Measuring Knowledge Use. Knowledge: Creation, Diffusion and Utilization, Vol.5 No.1, September 1983, pages 120-133.
104
23. Frambach, R.T. and N. Schillewaert (2002), "Organizational innovation adoption: a multi- level framwork of determinants and opportunities for future research," Journal
of Business Research, 55 (2), pages 163-176.
24. Fitzgerald, L., Ferlie, E., Hawkins,C. (2003). Innovation in healthcare; how des credible evidence influence professionals? Health Social Care Community May;11(3): pages 219-228.
25. Fleuren, M., Wiefferink, K., Paulussen, T. (2004). Determinants of innovations
within health care organizations. International Journal for Quality in Health Care
Vol.16, No.2, pages 107-123.
26. Glandon, G.L. & Counte, M.A.(1995). An Analysis of the Adoption of Managerial Innovation: Cost Accounting Systems in Hospitals. Health Services Manager Research, Vol 8 No 4, pages 243-251.
27. Glantz, M.D., Compton, W.M. (2004). Mental Health and Substance Abuse
Innovations: Issues of Diffusion and Adoption. Clinical Psychology: Science and Practice, V11 N2, pages 183-185. American Psychological Association.
28. Glisson, C. & James. L.R. (2002). The cross- level effects of culture and climate in human service teams. Journal of Organizational Behavior, 23, pages 767-794.
29. Gotham, H.J. (2004). Diffusion of Mental Health and Substance Abuse Treatments: Development, Dissemination, and Implementation. Clinical Psychology: Science and
Practice, V11 N2. pages 160-176. American Psychological Association.
30. Greer A.L. (1977). Advances in the study of diffusion of innovation in health care
organizations. Milbank Memorial Fund Quarterly/Health and Society, 55(4), pages 505-532.[MEDLINE]
31. Grol, R., & Wensing, M. (2004). What drives change? Barriers to and incentives for
achieving evidence-based practice. Medical Journal of Australia 2004;180(6 Suppl):
S57-S60
32. Hage, J. (1980). Theories of organizations. New York: Wiley.
33. Haider, M. & Kreps,G.L. (2004). Forty Years of Diffusion of Innovations: Utility
and Value in Public Health. Journal of Health Communication, Volume 9;3-11.
34. Health Canada. (2002). Best practices: Concurrent Mental Health and Substance Use
Disorders. Publications- Health Canada, Ottawa Ontario.
35. Henggeler, S.W., & Lee, T.(2002) What Happens After the Innovation is Identified? Clinical Psychology: Science and Practice. Volume 9 Issue 2, 191.
105
36. Hutchinson, A.M., & Johnston, L. (2006). Beyond the BARRIERS Scale:
Commonly Reported Barriers to Research Use. Journal of Nursing Administration, Volumn 36(4), April 2006, pages 189-199.
37. Kimberely, J.R. & Evanisko, M.J.(1981). Organizational innovation: The influence
of individual, organization and contextual factors on hospital adoption of
technological and administrative innovations. Academy of Management Journal, 24(4), pages 689-713.
38. Larsen, J.K. (1982). Information Utilization and Non Utilization. Paolo Alto, CA. American Institutes for Research in Behavioral Science.
39. Marinelli-Casey, P., Domier, C.P., Rawson, R.A.(2002) The gap between research
and practice in substance abuse treatment. Psychiatric Services 53, 984-987.
40. McFall, S.L., Warnecke, R.B., Kaluzny, A.D. & Ford, L.(1996). Practice setting and
physician influences on judgements of colon cancer treatment by community physicians. Health Services Research, 31, pages 5-19.
41. McGowan M.K. & Madey, G.R. (1998). Adoption and implementation of electronic data interchange. Information systems innovation and diffusion: issues and directions. pages 116-178. Idea Group Publishing, Hershey, PA, USA.
42. Meyer A.D. & Goes, J.B. (1988). Organizational assimilation of innovations: A
multilevel contextual analysis‟. Academy of Management Journal 31: pages 897-923.
43. Miller,W.R., Sorensen, J. L., Selzer, J.A., Brigham, G.S. (2006) Disseminating
evidence-based practices in substance abuse treatment: A review with suggestions. Journal of Substance Abuse Treatment 31, 25-39.
44. Mustonone-Ollila, E. (1998). Methodologies Choice and Adoption: Using Diffusion
of Innovations as the Theoretical Framework. Paper presented to the Lappeenranta
University of Technology. Lappeenranta, Finland.
45. Nutley, S., Davies, H., & Walter, I. (2002). Learning from the Diffusion of
Innovations. Research Unit for Research Utilization, Department of Management, University of St. Andrews, The Scores, UK.
46. Prochaska,J.O. & DiClemente, C.C. (1984). The transtheoretical approach.
Homeward III: Dow Jones- Irwin.
47. Rogers, E.M. (2003). Diffusion of innovations (5th ed). New York: Free Press.
106
48. Rush, B.(2003). On the Development and Implementation of Best Practice
Guidelines for the Treatment and Support of People with Concurrent Substance Use and Mental Disorders. Draft document for journal submission. Center for Addiction
and Mental Health.
49. Sanson-Fisher, R.W. (2004). Diffusion of innovation theory for clinical change.
Medical Journal of Australia. 2004 March 15;180(6 Suppl.):S55-6 50. Simpson, D.D. (2002). A conceptual framework for transferring research to practice.
Journal of Substance Abuse Treatment 22, pages 171-182.
51. Wade, J. (1996). A communication –level analysis of sources and rates of technological variation in the microprocessors market. Academy of Management Journal 39: pages 1218-1244.
52. Warner, K.E (1975). A “desperation-reaction” model of medical diffusion. Health
Services Resources, Winter;10(4) pages 369-83
107
8.0 Appendices
Appendix 1: Questionnaire
Best Practices for Concurrent Mental Health and Substance Use Disorders
Adoption of a Clinical Innovation—A one year follow up
(Data collection for M.Sc. thesis)
Thank you for taking the time to complete this questionnaire.
Your involvement is necessary to ensure complete and accurate information on if and how the recommendations from the document: “Best Practices fo r Concurrent Mental Health and Substance Use Disorders” were used/adopted. All information fathered will be kept strictly
confidential. The organizational identification number on the top right hand side of this form will help us monitor returns and will not be used to identify individuals‟ or
organizations‟ responses.
All results will be presented in aggregate form only.
Completing this questionnaire will take approximately 5-10 minutes of your time.
Even if your organization is not currently using the recommendations,
please complete and return the questionnaire by Sept 10, 2004.
For further information about the purpose of this research, please refer to cover letter
and information sheet that were enclosed.
Questions or comments? Please contact Tamara MacDonald at 416-535-8501 ext.6503 or email [email protected]
This questionnaire is to be filled out by the individual who attended or viewed the February 2003 videoconference on the Best Practices of Concurrent Mental Health and Substance Use
Disorders (hosted by the Centre for Addiction and Mental Health).
Your views are important and thank you again for your assistance .
108
Thank you very much for taking the time
to complete this questionnaire.
Your contribution to this survey is very important.
Please return this questionnaire by September 10, 2004 in the self-addressed, pre-stamped envelope enclosed.
Adoption of a Clinical Innovation—Research Project
C/O Tamara MacDonald
762 Upper James, #127
Hamilton, ON L93 3A2
109
Best Practices of Concurrent Mental Health and
Substance Use Disorders: Terms of Reference
The Best Practices document was intended to be a resource to many individuals involved in mental health, substance abuse and integrated mental health/substance abuse services. For this survey, the following terms have been defined: Executive Director: a person responsible for the administration of an organization/agency Conferences: a meeting for consultation, deliberation, discussion, or interchange of opinions Regional municipality: a particular region or geographic area Programs: a system of services, opportunities, or projects, usually designed to meet a social need MH: short form for mental health SU: short form for substance use CD: short form for concurrent disorders A: short form for addictions Level 1 Screening for Substance Use: basic screening tools e.g. Index of Suspicion, CAGE/CAGE-AID
Level 2 Screening for Substance Use: screening tools e.g. DALI, MAST, DAST, AUDIT Level 1 Screening for Mental Health: basic screening tools e.g. Index of Suspicion, ABC Level 2 Screening for Mental Health: other tools Integrated treatment: mental health and substance use being treated via no wrong door Program integration: mental health and substance use being treated by a multi-disciplined team in the same program/organization
System integration: mental health and substance use being treated by two or more professionals, that are working for different service providers
Steps towards implementation: the decision to do so has been made and initial planning steps have been taken, but the innovation/information has not been used yet
Partially implemented: certain features of the innovation have been used while others have been discarded
110
Section A: Background information about yourself and your organization
This section is concerned with information about yourself and
the organization at which you work.
For each question, please circle the appropriate response, unless otherwise indicated.
1. Format of viewing the Best Practices videoconference: …Live[1] Taped[2] Not
viewed[3]
2. Sex…………………………Male[1] Female[2]
3. Year of Birth [fill in blank- 4 digit year]……………___________
4. Highest Level of Education completed………Less than High School[1] High School[2]
College[3] Bachelor‟s Degree[4] Master‟s Degree[5] PhD.[6] 5. Current position……………Front-line[1] Supervisor[2] Manager[3]
Executive Director[4]
6. Length at current organization…….<3 yrs[1] 3-5yrs[2] 6-9yrs[3]
10-15yrs[4] >15yrs[5]
7. Field of career to date……………Mental health[1] Addictions[2]
Health Promotion[3] Social Work[4]
Other:___________[5]
8. Number of conferences attended in the past year….0[1] 1-3[2]
4-6[3] >6[4]
9. Your organization‟s primary focus………..Mental Health[1] Addictions[2]
Health Promotion[3] Policy[4] Housing[5] Other:___________[6]
10. Special populations that your organization serves (circle all that apply):
Youth[1] Women[2] Seniors[3] Gay/Bi/Lesbian[4]
111
11. Number of staff employed at your organization…..<6[1] 6-10[2] 11-20[3]
21-30[4] 31-50[5] >50[6] Don‟t know[7]
12. Number of volunteers affiliated with your organization…<6[1] 6-10[2] 11-20[3]
21-30[4] 31-50[5] >50[6] Don‟t know[7]
13. Your organization‟s functions. Circle all that apply:
Screening[1] Assessment[2] Referrals[3] Treatment[3] Case Management/Support[4] Other:_________[5] 14. Type of area your organization serves:
Rural[1] Urban[2] Suburban[3] Other:___________[4] 15. Number of other mental health/addiction organizations in your regional municipality:
0[1] 1-3[2] 4-6[3] 7-10[4] >10[5] Don‟t know[6] 16. Number of programs/services at your organization(average):
0[1] 1-3[2] 4-6[3] 7-10[4] >10[5] Don‟t know[6] 17. Number of management levels in your organization: 1[1] 2[2] 3[3]
4[4] >5[5] Don‟t know[6]
18. Average number of clients your organization serves on a monthly basis:
<10[1] 11-30[2] 31-50[3] >50[4]
19. I have heard about the Best Practices document………………No [1] Yes[2]
20. I have read the Best Practices document……………………….No [1] Yes[2]
Please continue on to the next page……
112
Section B: Your opinion on research and changing clinical practices
This section is interested in YOUR perception of the role of research in clinical practice.
For each statement, check off the response that most clearly represents your opinion.
There are no right or wrong answers.
Strongly
Disagree
[1]
Disagree
[2]
Neutral
[3]
Agree
[4]
Strongly
Agree
[5]
21. Keeping on top of current research is essential to professional performance
□ □ □ □ □
22. Most research on mental health & addictions by universities/research institutions does not directly apply
□ □ □ □ □
23. It is difficult for MH & A workers to access research in their communities
□ □ □ □ □
24. I expect staff to keep on top of research
□ □ □ □ □
25. I expect staff to read MH & A journals
□ □ □ □ □
26. I seek out recent research
□ □ □ □ □
27. I distribute research information to my staff regularly
□ □ □ □ □
28. Most research is presented in a way that is not accessible to my staff
□ □ □ □ □
29. Research findings in MH & A are difficult to find out about
□ □ □ □ □
30. Research findings in MH & A are difficult to obtain
□ □ □ □ □
Please continue on to the next page……
113
Strongly
Disagree
[1]
Disagree
[2]
Neutral
[3]
Agree
[4]
Strongly
Agree
[5]
31. Research findings in MH & A are difficult to understand
□ □ □ □ □
32. Research findings in MH & A are difficult to apply
□ □ □ □ □
33. I am willing to try new ideas in mental health/addiction practice
□ □ □ □ □
34. I see a need to change practice in my field
□ □ □ □ □
35. I see the benefits of changing practice in my field
□ □ □ □ □
36. I consider best practice documents important
□ □ □ □ □
Please continue on to the next page……
114
Section C: Presentation & content of the recommendations
Best Practices of Concurrent Mental Health and Substance Use Disorders
This section is concerned about YOUR opinion of the Best Practices of Concurrent Mental
Health and Substance Use Disorders document.
For each statement, please check off the appropriate response.
Strongly
Disagree
[1]
Disagree
[2]
Neutral
[3]
Agree
[4]
Strongly
Agree
[5]
37. I have heard about the Best Practices
document
□ □ □ □ □
38. I have read the Best Practices document
□ □ □ □ □
39. The recommendations in the Best Practices
document were clearly presented
□ □ □ □ □
40. The recommendations in the Best Practices
document were in plain language
□ □ □ □ □
41. The Best Practices document was easy to
understand
□ □ □ □ □
42. The recommendations in the Best Practices
document will improve current practice
□ □ □ □ □
43. The recommendations in the Best Practices
document are difficult to understand
□ □ □ □ □
Please continue on to the next page……
115
Strongly
Disagree
[1]
Disagree
[2]
Neutral
[3]
Agree
[4]
Strongly
Agree
[5]
44. The recommendations in the Best Practices
document will be difficult to use
□ □ □ □ □
45. The recommendations in the Best Practices
document fit with my personal beliefs
□ □ □ □ □
46. The recommendations in the Best Practices
document fit with my organization’s values
□ □ □ □ □
47. The recommendations in the Best Practices
document will produce results that are easy
to see
□ □ □ □ □
48. The recommendations in the Best Practices
document are customizable to my
organization
□ □ □ □ □
Please continue on to the next page……
116
Section D: Use/adoption of the recommendations
Best Practices of Concurrent Mental Health and Substance Use Disorders
This section is concerned with the recommendations from the Best Practices of Concurrent
Mental Health and Substance Use Disorders document and the level of utilization and adoption
in YOUR organization. For each statement, please check off the appropriate response.
Recommendations Considered
& Rejected
[1]
Nothing
Done
[2]
Under
Consideration
[3]
Step Towards
Implementation
[4]
Partially
Implemented
[5]
Implemented
& Adapted
[6]
49. Organization
routinely screens
for substance use
□ □ □ □ □ □
50. Organization
routinely screens
for mental health
□ □ □ □ □ □
51. Organization
routinely screens
for both substance
use and mental
health
□ □ □ □ □ □
52. Organization
routinely does a
Level 1 screening
for substance use
□ □ □ □ □ □
53. Organization
routinely does a
Level 2 screening
for substance use
□ □ □ □ □ □
Please continue on to the next page……
117
Recommendations Considered
& Rejected
[1]
Nothing
Done
[2]
Under
Consideration
[3]
Step Towards
Implementation
[4]
Partially
Implemented
[5]
Implemented
& Adapted
[6]
54. Organization
routinely does a
Level 1 screening
for mental health
□ □ □ □ □ □
55. Organization
routinely does a
Level 2 screening
for mental health
□ □ □ □ □ □
56. Organization has
access to
psychiatric
consultation
□ □ □ □ □ □
57. Organization has a
policy for assessing
CD
□ □ □ □ □ □
58. Organization uses
SCID-IV
□ □ □ □ □ □
59. Organization uses
clinical rating
scales
□ □ □ □ □ □
60. Organization uses
addictions severity
index
□ □ □ □ □ □
61. Organization uses
motivational
interviewing
□ □ □ □ □ □
Please continue on to the next page…
118
Recommendations Considered
& Rejected
[1]
Nothing
Done
[2]
Under
Consideration
[3]
Step Towards
Implementation
[4]
Partially
Implemented
[5]
Implemented
& Adapted
[6]
62. Organization uses
stages of change
□ □ □ □ □ □
63. Organization uses
psychosocial
functioning
□ □ □ □ □ □
64. Organization is
involved in
integrated
treatment
□ □ □ □ □ □
65. Organization is
involved in
program
integration
□ □ □ □ □ □
66. Organization is
involved in system
integration
□ □ □ □ □ □
67. Organization is
involved in
provincial policy
initiatives (e.g.
Task Force)
□ □ □ □ □ □
68. Organization
refers to other
services in the
community
□ □ □ □ □ □
69. Organization
belongs to a formal
network
□ □ □ □ □ □
Please continue on to the next page……
119
Recommendations Considered
& Rejected
[1]
Nothing
Done
[2]
Under
Consideration
[3]
Step Towards
Implementation
[4]
Partially
Implemented
[5]
Implemented
& Adapted
[6]
70. Organization is
involved with
blended/integrated
service delivery
teams from
members o f MH &
SU agencies
□ □ □ □ □ □
71. Organization has
CD treatment as
part of its mandate
□ □ □ □ □ □
72. Organization
formally
collaborates
amongst
community
networks
□ □ □ □ □ □
73. Organization is
aware of others
using the
recommendations
□ □ □ □ □ □
74. Organization has
formal, internal
communication
paths
□ □ □ □ □ □
75. Organization has
formal, external
communication
paths
□ □ □ □ □ □
Thank you!
120
Appendix 2:
Variable Definition and Measurement-
Variable Label Definition Data Source
Individual Characteristics
Sex Male or female Questionnaire
Year of birth Year respondent was born in Questionnaire
Education Highest level of education
completed
Questionnaire
Current position Leadership position Questionnaire
Length at organzation Tenure Questionnaire
Field of career to date Mental health, addictions or both
Questionnaire
Conferences attended Cosmopolitism Questionnaire
Organizational Characteristics Questionnaire
Organization‟s primary focus Focus on Mental health, addictions or both
Questionnaire
Youth served Special population served-
youth
Questionnaire
Women served Special population served-women
Questionnaire
Seniors served Special population served-
seniors
Questionnaire
LGBT served Special population served- lesbian, gay, bisexual,
transgendered
Questionnaire
# staff employed Complexity of the organization
Questionnaire
# volunteers Complexity of the organization
Questionnaire
Screening Complexity of the
organization- type of service offered; screening
Questionnaire
Assess Complexity of the
organization- type of service offered; assessment
Questionnaire
Referrals Complexity of the organization- type of service
offered; referrals
Questionnaire
Treatment Complexity of the organization- type of service
Questionnaire
121
Variable Label Definition Data Source
offered; treatment
Case Management Complexity of the organization- type of service
offered; case management
Questionnaire
Other Complexity of the organization- type of service
offered; other services
Questionnaire
Urbanity Rural , urban or suburban Questionnaire
# of other organizations in area Competition Questionnaire
# of programs/services Complexity of the organization
Questionnaire
# of clients on monthly basis Complexity of the organization
Questionnaire
Perception of
research/communication
Questionnaire
Q21-Keep current Importance of current research
Questionnaire
Q22-University research Applicability of research
from universities
Questionnaire
Q23-Research accessible Difficulty for employees to access research in their communities
Questionnaire
Q24- Expect staff Expectation for staff to stay
current with research
Questionnaire
Q25-Expect staff read Expecation for staff to read journals
Questionnaire
Q26- Seek research Respondent seeks out
research
Questionnaire
Q27-Distribute research Distribution of research Questionnaire
Q28-Presentation Presentation of research being accessible
Questionnaire
Q29-R hard to find out Finding out about research Questionnaire
Q30-R hard to obtain Research being difficult to obtain
Questionnaire
Q31-R hard to understand Research difficult to
understand
Questionnaire
Q32-R hard to apply Research to difficult to apply Questionnaire
Q33-Try new ideas Willingness to try new ideas in practice
Questionnaire
Q34-See need Seeing the need to change
practice
Questionnaire
122
Variable Label Definition Data Source
Q35-Benefits Seeing the benefits of changing practice in the field
Questionnaire
Q36-BP important Considering best practice
documents important
Questionnaire
Innovation characteristics
Q39-Rec Clear Recommendations were clearly presented
Questionnaire
Q40-BP plain language Recommendations were in
plain language
Questionnaire
Q41-BPD easy understand The document as a whole, was easy to understand
Questionnaire
Q42-BP improve Practice Recommendations will
improve current practice
Questionnaire
Q43-BPR hard to understand Recommendations are difficult to understand
Questionnaire
Q44-BPR hard to use Recommendations will be
difficult to use
Questionnaire
Q45-BPR personal beliefs Recommendations fit with personal beliefs
Questionnaire
Q46-BPR org values Recommendations fit with organization‟s values
Questionnaire
Q47-BPR produce results Recommendations will produce results that are easy to see
Questionnaire
Q48-BPR customizable Recommendations are
customizable
Questionnaire
123
Adoption-Decision
Dependent Variable
Q49- Organization screens SU
Level of adoption, Likert scale
(1) Considered and rejected (2) Nothing done (3) Under consideration
(4) Step towards implementation (5) Partially implemented
(6) Implemented and adopted
Questionnaire
Q50-Organization screens MH
Level of adoption, Likert scale
(1) Considered and rejected (2) Nothing done (3) Under consideration
(4) Step towards implementation
(5) Partially implemented (6) Implemented and adopted
Questionnaire
Q51-Organization screens SU and
MH
Level of adoption, Likert scale
(1) Considered and rejected (2) Nothing done
(3) Under consideration (4) Step towards implementation
(5) Partially implemented (6) Implemented and adopted
Questionnaire
Q52-Organization does Level 1
screening for SU
Level of adoption, Likert
scale (1) Considered and rejected (2) Nothing done
(3) Under consideration (4) Step towards implementation
(5) Partially implemented (6) Implemented and adopted
Questionnaire
124
Q53-Organziaton does Level 2
screening for SU
Level of adoption, Likert scale (1) Considered and rejected
(2) Nothing done (3) Under consideration
(4) Step towards implementation (5) Partially implemented
(6) Implemented and adopted
Questionnaire
Q54- Organization does Level 1
screening for MH
Level of adoption, Likert scale
(1) Considered and rejected (2) Nothing done (3) Under consideration
(4) Step towards implementation (5) Partially implemented
(6) Implemented and adopted
Questionnaire
Q55- Organization does Level 2
screening for MH
Level of adoption, Likert scale
(1) Considered and rejected (2) Nothing done (3) Under consideration
(4) Step towards implementation
(5) Partially implemented (6) Implemented and adopted
Questionnaire
Q56- Organization has access to a
psychiatric consult
Level of adoption, Likert scale
(1) Considered and rejected (2) Nothing done
(3) Under consideration (4) Step towards implementation
(5) Partially implemented (6) Implemented and adopted
Questionnaire
Q57-Organization has a policy on
CD
Level of adoption, Likert
scale (1) Considered and rejected (2) Nothing done
(3) Under consideration (4) Step towards implementation
(5) Partially implemented (6) Implemented and adopted
Questionnaire
125
Q58- Organization uses SCID-IV
Level of adoption, Likert scale (1) Considered and rejected
(2) Nothing done (3) Under consideration
(4) Step towards implementation (5) Partially implemented
(6) Implemented and adopted
Questionnaire
Q59-Organization uses clinical
rating scales
Level of adoption, Likert scale
(1) Considered and rejected (2) Nothing done (3) Under consideration
(4) Step towards implementation (5) Partially implemented
(6) Implemented and adopted
Questionnaire
Q60-Organization uses addictions
severity index
Level of adoption, Likert scale
(1) Considered and rejected (2) Nothing done
(3) Under consideration (4) Step towards implementation
(5) Partially implemented (6) Implemented and adopted
Questionnaire
Q61-Organization uses
motivational interviewing
Level of adoption, Likert
scale (1) Considered and rejected
(2) Nothing done (3) Under consideration (4) Step towards
implementation (5) Partially implemented (6) Implemented and adopted
Questionnaire
126
Q62-Organization uses stages of
change
Level of adoption, Likert scale (1) Considered and rejected
(2) Nothing done (3) Under consideration
(4) Step towards implementation (5) Partially implemented
(6) Implemented and adopted
Questionnaire
Q63-Organziation uses
psychosocial functioning
Level of adoption, Likert scale
(1) Considered and rejected (2) Nothing done
(3) Under consideration (4) Step towards implementation
(5) Partially implemented (6) Implemented and adopted
Questionnaire
Q64-Organizations is involved in
integrated treatment
Level of adoption, Likert
scale (1) Considered and rejected (2) Nothing done
(3) Under consideration (4) Step towards
implementation (5) Partially implemented (6) Implemented and adopted
Questionnaire
Q65-Organization is involved in
program integration
Level of adoption, Likert scale (1) Considered and rejected
(2) Nothing done (3) Under consideration
(4) Step towards implementation (5) Partially implemented
(6) Implemented and adopted
Questionnaire
127
Q66-Organization is involved in
program integration
Level of adoption, Likert scale (1) Considered and rejected
(2) Nothing done (3) Under consideration
(4) Step towards implementation (5) Partially implemented
(6) Implemented and adopted
Questionnaire
Q67-Organization is involved in a
provincial policy initiative
Level of adoption, Likert scale
(1) Considered and rejected (2) Nothing done
(3) Under consideration (4) Step towards implementation
(5) Partially implemented (6) Implemented and adopted
Questionnaire
Q68-Organization refers to other
services
Level of adoption, Likert
scale (1) Considered and rejected (2) Nothing done
(3) Under consideration (4) Step towards
implementation (5) Partially implemented (6) Implemented and adopted
Questionnaire
Q69-Organization belongs to a
formal network
Level of adoption, Likert scale (1) Considered and rejected
(2) Nothing done (3) Under consideration
(4) Step towards implementation (5) Partially implemented
(6) Implemented and adopted
Questionnaire
128
Q70-Organzation is involved in
integrated service delivery teams
Level of adoption, Likert scale (1) Considered and rejected
(2) Nothing done (3) Under consideration
(4) Step towards implementation (5) Partially implemented
(6) Implemented and adopted
Questionnaire
Q71-Organization has CD treatment
mandate
Level of adoption, Likert scale
(1) Considered and rejected (2) Nothing done
(3) Under consideration (4) Step towards implementation
(5) Partially implemented (6) Implemented and adopted
Questionnaire
Q72-Organization formally
collaborates
Level of adoption, Likert
scale (1) Considered and rejected (2) Nothing done
(3) Under consideration (4) Step towards
implementation (5) Partially implemented (6) Implemented and adopted
Questionnaire
Q73-Organization is aware of
others using recommendations
Level of adoption, Likert scale (1) Considered and rejected
(2) Nothing done (3) Under consideration
(4) Step towards implementation (5) Partially implemented
(6) Implemented and adopted
Questionnaire
129
Q74-Organization has formal
internal communication
Level of adoption, Likert scale (1) Considered and rejected
(2) Nothing done (3) Under consideration
(4) Step towards implementation (5) Partially implemented
(6) Implemented and adopted
Questionnaire
Q75-Organization has formal
external communication
Level of adoption, Likert scale
(1) Considered and rejected (2) Nothing done
(3) Under consideration (4) Step towards implementation
(5) Partially implemented (6) Implemented and adopted
Questionnaire
130
Appendix 3: Ethics Committee Approval Letter
l
131
Appendix 4: Introduction- Cover Letter for Participants
July 30, 2004
Dear Potential Participant:
You are invited to participate in a voluntary self-administered confidential questionnaire that
will be used for research in a one-year follow-up, after the videoconference on the Best Practices of Concurring Mental Health and Substance Use Disorders for Ontario.
This work is being completed to fulfill the requirements of a Masters degree in the Department of Health Policy, Management and Evaluation, Faculty of Med icine, University
of Toronto. Dr. Rhonda Cockerill is the supervisor of this research and can be contacted at 416-978-7721.
Christine Bois, your host for the 2003 videoconference from the Centre for Addiction and Mental Health (CAMH) is supportive of this research. Your level of participation, however,
will not reflect on your relationship with CAMH.
Also enclosed is an information sheet with directions.
Completing this questionnaire will only take 5 minutes of your time.
Even if your organization is not currently using the recommendations within the best
practice document, please complete and return the questionnaire.
All information gathered will be kept strictly confidential. The organizational identification number on the top right hand side of this form will help us to monitor returns and will not be used to identify individual organizations‟ responses. All results will be presented in
aggregate form only.
As a way to show appreciation for completing and returning the questionnaire, I have enclosed a coupon for a free beverage of your choice at Tim Horton‟s/Starbucks. In addition, any organization that returns the questionnaire will be able to request a copy of the
final report. If you have any questions, please feel free to contact me.
Thank you,
Tamara MacDonald
Day: 416-535-8501 ext.6503 Evening: 905-574-5846
Email: [email protected]
132
Appendix 5: Information Sheet for Participants
Thank you for completing and returning the enclosed questionnaire.
Please find below further information and instructions about the research.
Title of Research Project
Adoption of a clinical innovation: “ Best Practices for Concurrent Mental Health and
Substance Use Disorders” in Ontario, a one- year follow up.
Background and Purpose of Research
The objective of this research is to examine the adoption of a clinical innovation in Mental Health and Addiction Service organizations in Ontario, one year after dissemination. This
research is being done as part of my Master‟s thesis in Health Policy, Management and Evaluation at the University of Toronto. Currently, I am a Community Health and Education Specialist at the Centre for Addiction and Mental Health (CAMH), specializing in
concurrent/co-occurring disorders. The clinical innovation studied is the “Best Practices for Concurrent Mental Health and
Substance Use Disorders”. This document is a synthesis of research published by Health Canada that was intended to be a resource for managers and staff of mental health, substance
abuse and integrated mental health/substance abuse services, as well as individual practitioners in the community.
Your involvement is important to ensure complete and accurate information for Ontario. Even if your organization is not currently using the recommendations within the Be st
Practices document, please complete and return the questionnaire. All information gathered will be kept strictly confidential. The organizational identification
number on the top right hand side of the questionnaire will help me to monitor returns and will not be used to identify individual‟s or organization‟s responses. All results will be presented in aggregate form only.
Christine Bois, the Priority Manager- Concurrent Disorders at CAMH and your host for the
February 2003 videoconference, has fully supported this research, though it is stressed that participation is voluntary and confidential and will not reflect upon your relationship with CAMH.
133
Who is invited to participate?
Representatives of the organizations who were part of the videoconference on the Best Practice document, which was hosted by CAMH in February 2003.
Instructions
1. As the Executive Director/Senior manager (whom attended the videoconference), please read and review the cover letter and information sheet.
2. If you are not the person who attended the videoconference but are in the position of Executive Director/Senior manager for this organization, please complete the questionnaire.
3. Even if your organization is not currently using the recommendations of the Best Practices document, please complete and return the questionnaire.
4. If you have any questions, please contact me at the numbers provided.
5. Complete the questionnaire to the best of your ability. Choose the responses that best reflects your answer.
6. Questionnaire will only take five (5) minutes to complete.
7. Return completed questionnaire via the self-addressed/stamped envelope within 15 days of receipt (target date: August 30, 2004) of the questionnaire.
8. Thank you! Enjoy your free coffee/tea!
Tamara Kennedy-MacDonald
Day: 416-535-8501 ext.6503 Evening: 905-574-5846 Email:[email protected]
134
Appendix 6: Appendix “A” for Questionnaire package
List of Recommendations of the Best Practices for Concurrent Mental Health and
Substance Use Disorders.
Definition of „screening‟: checking for the presence of a disease/disorder Definition of „assessment‟: determining the degree/classification of a disease/disorder
Routine screening for substance use (no wrong door policy). Routine screening for mental health (no wrong door policy).
Routine screening for both mental health and substance use (no wrong door policy). Level 1 screening tools for Substance Use.
o Example screening tools include Index of Suspicion, CAGE/CAGE-AID. o Asking straight forward questions concerning alcohol and other drug use
Level 2 screening tools for Substance Use.
o Example screening tools include: Dartmouth Assessment of Lifestyle Instrument (DALI)
Michigan Alcoholism Screening Test (MAST)
Drug Abuse Screening Test (DAST) Alcohol Use Disorders Identification Test (AUDIT)
Level 1 screening tools for Mental Health. Example screening tools include Index of Suspicion and ABC
Checklist.
Level 2 screening for Mental Health, however there are no examples of standardized screening tools to be used.
Access to psychiatric resources for consultation purposes. Policy in place for accessing concurrent disorders. Using the following tools for assessing concurrent disorders:
o SCID-IV (Structured Clinical Interview for Axis DSM-IV Disorders) o Clinical rating scales (e.g. Alcohol Use Scale, Drug Use Scale)
o Addictions Severity Index o Motivation (e.g. Motivational Interviewing technique) o Stages of Change (an actual technique)
o Psychosocial functioning assessment tools (e.g. Person in Environment- PIE & Global Assessment for Functioning Scale –GAF)
For the organization to be involved in integrated treatment
o Mental health and substance use being treated in an integrated way For the organization to be involved in program integration
o Mental health and substance use being treated by a cross discipline team in the same program
For the organization to be involved in system integration
o Mental health and substance use being treated by two or more professionals that are working for a different service provider
For the organization to be involved in planning the stages of any type of integration o A committee, shared data systems or involved in provincial policy initiatives
135
For the organization to be involved in the service delivery stage of any type of
integration o Blended service delivery teams from members of mental health and addiction
agencies For the organization to be involved in the training stage of any type of integration
o Cross-training across mental health and addiction agencies and programs
For the organization to be involved in treatment interventions For the organization to be involved in support services for the following:
o Substance Use and Mood/Anxiety Disorders
o Substance Use and Severe Persistent Mental Disorders o Substance Use and Personality Disorders
o Substance Use and Eating Disorders For the organization to have other services/service providers in the community, to
which it can refer clients to.
For the organization to be active in a concurrent disorder/mental health/addiction services network
For the organization to use Cognitive Behavioral Therapy (CBT) for Mood and Anxiety Disorders and substance use.