Transforming Mental Health Services Through the Use of Evidence-Based and Emerging Best Practices

Columbia River Doubletree June 3-4, 2004

Peer Support & Peer-Run Programs

Jean Campbell, Ph.D.

Program in Consumer Studies & TrainingMissouri Institute of Mental Health

New Studies & Tools

"Science itself is just a tool for achieving human ends; what the political community decides are appropriate ends are not ultimately scientific questions."

--Francis Fukuyama, Our Post Human Future, 2002. NY: Farrar, Straus. p 186

Peer Support Outcomes Peer Support Outcomes ProtocolProtocol Project (POPP)Project (POPP)

Development of an Evaluation Protocol for Community-Based Peer Support Programs

(1996-2001)

Case for ActionCase for Action A Need for Measurement

– To survive in an era of evidence-based funding, peer support programs need to measure:• cost, • effectiveness, • quality, • utilization, and • appropriateness of the services they

provide.

Case for Action (cont.)Case for Action (cont.) A need for accountability

– To build partnerships between members, program administrators, and external agencies

– To improve current programs and tailoring future efforts to

– To demonstrate utility and effectiveness to funding partners

– To advocate for programmatic efforts and to guide policy

– To recruit members and develop community support

Case for Action (cont.)Case for Action (cont.) Why a Peer Developed Protocol? As peer support programs continue to grow, so does

the need for an outcomes protocol with measures derived from mental health consumers’ experiences and points of view

– The POP: • was developed by consumers• can be administered by consumers• embodies consumer values• is consistent with peer support philosophy• recognizes and utilizes proven consumer

abilities to conduct survey and outcome studies

Overview of the POPPOverview of the POPP Purpose:

– To develop, field-test, and distribute an evaluation protocol that measures outcomes and satisfaction of community-based peer support programs that are operated by mental health consumers/survivors.

– Four Phase Project (1996-2001) Conducted by:

– Protocol and support materials developed and piloted by the Program in Consumer Studies and Training at the Missouri Institute of Mental Health in St. Louis.

Funded by:– The National Research and Training Center on Psychiatric

Disability at the University of Illinois-Chicago

Utility of the POPPUtility of the POPP Assess program outcomes for consumer self-

help field Present service outcomes to public funding

authorities and manage-care organizations Provide feedback to consumer-run

organizations, enabling consumers with information to improve the organization and delivery of peer support programs

POPP Outcome Domains

Demographics Service Use

Employment Housing/Community Life

Quality of Life

Well-Being (Recovery, Empowerment & Personhood)

Program Satisfaction

Specific outcome domains organized into individual modules:

Characteristics of Characteristics of ProtocolProtocol Measures 7 domains Designed to be done face-to-face Independent domains can be separated

– Avoids burdening respondents– Flexibility to tailor to program needs and goals

Criterion-related validity (POPP & Criterion Scales) Criterion Social Acceptance Scale with Recovery

(.55) Criterion (Rosenburg) Self-Esteem Scale with

Personhood (.76) Criterion Recovery Scale with Recovery (.63) Criterion Empowerment Scale with Empowerment

(.46) Criterion (QS-8) Satisfaction Scale with Program

Satisfaction (.55)

Psychometric RefinementPsychometric Refinement

Final ProtocolFinal Protocol

Seven Modules (with 14 scales)– Demographics

– Service Use

– Employment

– Community Life• Final Factor Structure accounted for 60% of the variance

Test-Retest – ranged from .46-.82

– Quality of Life– Well-Being– Program Satisfaction

Barriers to Using the POPBarriers to Using the POP

Requires information system to effectively

manage and utilize information

Requires support and openness to feedback from

members of Peer Support Programs

Potential lack of experience and/or training

resources to collect, analyze and feedback

information

POPP Supporting Resources

– Interviewer Training Manual

– Question by Question Guide for Interviewers

– Tool Kit

– Report on POP Psychometrics

Overcoming BarriersOvercoming Barriers

Consumer-OperatedService Program (COSP)Multisite Research Initiative 1998-2004

COSP Baseline Findings: Participant Characteristics

State of the Evidence

Prior studies of consumer-run programs suggest that they improve symptoms, promote larger social networks, and enhance quality of life. However, the evidence is limited:

-uncontrolled studies-demonstrations of feasibility-preliminary findings

Importance of Multisite Study Determine cost-effectiveness of consumer-

operated programs-What works for whom at what cost?

Such evidence is necessary for consumer-run programs that seek to be partners in the community continuum of care

-Funding & employment opportunities Study results can promote new programs,

improve quality of existing programs , expand services for people not easily engaged in traditional services, & reduce costs

Goals of the Study

• Establish the extent to which consumer-operated services when offered as an adjunct to traditional mental health services are effective in improving selected outcomes for people with severe mental illness

• Create strong and productive partnerships among consumers, service providers and service researchers

• Disseminate the knowledge gained

Program Models

»Drop-in Centers

»Educational & Advocacy Training

Programs

»Peer or Mutual Support Services

Participating Study Sites

Connecticut

Florida/California

Illinois

Maine

Missouri

Pennsylvania

Tennessee

Research DesignFour Year StudyRigorous Methodology

Experimental Multisite DesignRandomizationBaseline, 4, 8 & 12 month follow-upsCommon Protocol

Intervention: Consumer-Operated Program + Traditional Mental Health Services

Control: Traditional Mental Health Services Only

Selected Outcomes

Employment Empowerment Housing Service

Satisfaction Social Inclusion Costs Well-being

1,827 Study Participants!

Largest Study of Consumer Programs

in History

Demographics

There were more females (60%) than males (40%) among multisite participants.

Slightly less than half (43%) were minorities or individuals who described themselves using two or more race categories.

The average participant age was 43 years old.

Demographics

Only 13% of participants were married at baseline with another 23% having a “significant other” to whom they were not married.

On the other hand, 53% reported having children, averaging one child per parent.

Approximately half of the parents indicated their children were under the age of 18 years.

Education & Employment

More than half of the participants had achieved at least a high school diploma, with 42% going beyond high school.

Although nearly all participants (97%) had been employed at some point during their life, and 77% said that having a paying job was important to them, only about one-third (29%) were working either for pay or as a volunteer at the time of the baseline interview.

Employment & Benefits

Whereas 19% received income from paid employment (including a sheltered workshop), a substantial proportion of participants received income from non-employment sources, including:– Social Security (84%) income – Other social welfare benefits (40%), – Rent supplements (24%).

Benefits

– Most participants (79%) were receiving benefits that covered their psychiatric care, although only 59% reported that their benefits covered all the services they needed.

Housing

Although about half of the participants had been homeless at some time in their lives (51%), most participants’ living situations at baseline were fairly stable – 85% reporting that there was no time limit on

how long they could stay at their current place of residence. Note that this percentage does not indicate how many individuals may have wished to move from their current housing situation but were unable to do so; data were not collected on desire to change current housing.

Housing

More than half of the participants lived in their own residences at the time of the baseline interview (58%),

16% lived in someone else’s residence,

19% lived in temporary housing; and only 2% were currently homeless.

About one-third of participants were living alone (36%).

Housing

Of the two-thirds of participants who lived with someone else, –41% lived with another mental health consumer– 29% lived with a spouse or other live-in partner– 15% lived with their parents–28% lived with their children –14% lived with other family members

–27% lived with a non-related person

Housing

Overall, the housing situation of participants was positive, especially when compared to the fairly high rate of past homelessness reported by these same individuals.

Diagnosis

Nearly half of the participants were diagnosed with Schizophrenia and Schizoaffective Disorder (47%)– 31% Schizophrenia, 16% Schizoaffective Disorder

Depression was diagnosed for 25% of the participants

Bipolar Disorder for 18%. Other major diagnostic categories represented

among COSP participants included Anxiety Disorders, Dysthymia, and Psychotic Disorders other than Schizophrenia.

Diagnosis

Secondary diagnoses on Axis I were found in a small number (11%) of participants with 76% of those with more than one diagnosis having substance-related disorders.

The majority of these substance-related, secondary diagnoses were reported for participants in one study site that specifically provides services to a dually-diagnosed population.

Psychiatric Treatment History

Most participants had been hospitalized for psychiatric/emotional problems at some point in their lives (82%) with 25.8 years old being the average age at first psychiatric hospitalization.

Although 85% of these participants had been hospitalized more than once, and 62% had from 2-10 hospitalizations, relatively few participants reported any hospitalization for only a psychiatric reason within the four months prior to baseline (16%).

Psychiatric Treatment History

The average first age of any psychiatric contact – whether that was hospitalization or outpatient treatment – was 23.2.

Almost all participants reported that they had been taking prescribed psychiatric medications within the past four months (96%), and/or had seen a psychiatrist in the past four months (89%), with 91% engaging in both treatment activities.

95% reported experiencing side effects from psychiatric medications.

Psychiatric Treatment History

These percentages reflect a high level of participant involvement in the traditional mental health service delivery system.

Program Model Differences

Most of the characteristics of participants were significantly different across the program models. – The percentage of men in the study was lower for

the education/advocacy programs than for the drop-in centers and mutual support programs.

– The percentage of white participants was lower for the mutual support programs, as was the average age.

– The percentage of study participants who were married was greater for those in the education/advocacy programs.

Program Model Differences

– A higher percentage of drop-in center participants had received some Social Security income in the 30 days previous to the baseline interview. However, these participants did not appear to have the most severe illness as indicated by age at first psychiatric contact and lifetime history of hospitalization.

– The study participants in the mutual support programs had the youngest age of first psychiatric contact, and were more likely than participants at other sites to report 5 or more lifetime hospitalizations for psychiatric/emotional problems.

Program Model Differences

– However, the Hopkins Symptom Checklist (HSCL) indicated that participants of drop-in centers reported the lowest degree of symptoms, as did the Colorado Symptom Index Psychosis Subscale (CSIP).

– Study participants in the education/advocacy programs were more likely to live in their own residence at baseline, and less likely ever to have been homeless.

– On the other hand, study participants in the mutual support programs felt they were more likely to have to move from their current housing within the near future.

Program Model Differences

– Diagnoses of participants in both the mutual support and education/advocacy clusters were nearly evenly divided between schizophrenia and psychotic disorders and mood and anxiety disorders.

– At the drop-in center sites, on the other hand, more study participants were diagnosed with Psychotic Disorders and fewer with Mood Disorders.