The Most Significant Change 2012-2013

A special thanks to the United Way for providing the funding that made this

project possible.

Most Significant Change Project Statement of Confidentiality

The original stories have been edited and summarized to omit confidential and identifying in-

formation. Only the edited and summarized versions appear in this report.

Glossary

CHANGE Change witnessed or contributed to at Discovery House—great or small, positive or negative, individual or

situational

SIGNIFICANT Open to interpretation by each storyteller i.e., “what YOU feel is significant” CHANGE

STORY A single account of significant change written from staff perspective

STORYTELLER A staff member that has written a significant change story

NARRATIVE A story and a summary of its discussion

COLLECTION All of the narratives collected during a project cycle

OUTCOMES The following are central to Program Evaluation at Discovery House and are used throughout analysis:

HOUSING Clients will obtain safe, stable housing

SAFETY Clients will be safe from domestic violence

HEALTH & WELLBEING Primary areas: i) Clients will engage in active treatment for identified health issues,

ii) Clients will demonstrate improved emotional well-being,

ii) Clients will engage in a developmental assessment and intervention

Secondary areas: iv) The client and staff member will demonstrate a positive work-

ing alliance, v) The client will demonstrate a readiness for change

SOCIOECONOMIC Clients will demonstrate a reduction in socioeconomic related disparities, including

income/family resource, employment, education, citizenship, and legal involve-

ment

FAMILY Primary Areas: i) Clients will be supported in their interactions with child welfare, ii) Clients will have access to child care when needed

Secondary Areas: iii) Clients will demonstrate positive family dynamics

Table of Contents

Project Overview

Most Significant Change Journey 2012-2013 P1-2

Project in Review P3-4

Our Most Significant Change

Learning through Example P5-6

Narratives for Final Selection

Overcoming Communication Barriers P7

Specialized Roles P8

Christmas Past, Present and Future P9

Transitioning Tom P10

The Journey P11-12

Short Stories

Nineteen Stories P12

Short Stories P13-16

Baby Steps Guiding Change

Change for the Entire Family A Family’s Future

Moving Forward Breaking the Shell

Full Circle Power in Numbers

Transforming Anger Learning Together

Family Meetings Work Regain Confidence after Domestic Violence

Getting to Day Treatment Intake Tools of the Trade

Queen’s Cupcakes Resiliency Becomes Reality

New Home Honouring the Courage to be Real

A Delightful Mother-Daughter Duo

Analysis

Reading for Outcomes P17

Talking about Change P18

Spheres of Influence P19

Complexity of our Clients P20

Reflections

The Last Word P21-22

Most Significant Change

Journey

2012-2013

Our MSC journey began in October of 2011 when first intro-

duced to the technique at an Outcomes Mapping workshop.

While we were not searching for a qualitative evaluation tech-

nique at that time, the approach did catch our attention as

one that might have a lot of utility and value for the agency.

Narrative is a very rich source of qualitative data that we often

use informally and we believed that MSC could help us to ex-

plore the richness of these narratives in a more rigorous way.

From our perspective, the value of MSC for this agency lies in

the ability to expand our program evaluation activities to col-

lect rich, individual descriptions of client change. In this way,

MSC has the potential to both shed greater light on changes

that we document through our traditional program evaluation

procedures, at the same time that it also has the potential to

articulate changes that have previously gone unmeasured.

Further, story collection, analysis, and selection activities are

conducted by the entire agency and this participatory aspect

of the approach lends itself to widespread learning and use.

It took some time, but with the help of the United Way of Cal-

gary, we were able to initiate a pilot run of the MSC approach

in September 2012. This pilot was driven by three broad re-

search questions that were:

1. What kinds of change does this agency consider to be

significant?

2. What do the stories and selection process tell us about

the underlying values and assumptions within the

agency?

3. What does an analysis of the process itself tell us about

how to best mold the MSC approach to the environ-

ment at Discovery House?

The report that follows seeks to highlight both the steps that

we all took to get to this point and to also share some of the

learning that has taken place. This report is the next step in

the MSC journey and we hope that you enjoy a look back at

where we have been to get a better sense of where we might

be going in the future.

The Most Significant Change (MSC) technique is a participatory evaluation technique that

uses storytelling to explore significant changes within an agency, along with the meanings

and values attached to those changes.

1 -

Pro

ject

Ove

rvie

w

Pro

ject O

ve

rvie

w

Selection Process

FINAL SELECTION

COMMITTEE

OUR MOST SIGNIFICANT CHANGE

Final Selection - January 22

INITIAL SELECTION

COMMITTEE

↑↑↑

November 15

Selections

↑↑

December 7

Selections

↑

January 16

Selection

STORYTELLERS

↑↑↑↑↑

↑↑↑↑↑

November 2

Submissions

↑↑↑↑

↑↑↑↑↑

November 30

Submissions

↑↑↑

↑↑↑

January 4

Submissions

Report back to participants

Alison Casey

Allison Mclauchlan

Alyssa Geiger

Brigitte Baradoy

Carissa O’Connor

Cindy Holmes

Deborah Phillips

Diane Altwasser (United Way)

Doug Burden

Ed Rogers

Eileen Bell

Eliana Clay

Faiza Ahmad

Gladys Richards

Heidi Bragg

Hilda Lam

Janet Groom

Jannay Douville

Jaye Russell (Calgary Homeless

Foundation)

José Guillén

Leslie Hill

Lisa Sheldon

Mariam Mohamed

Mayra Jarillo Alvarado

Melissa Foster

Nikki Croft

Paula Jones

Raman Basi

Renee Wassick

Robin Peel

Sheena Furze

Tanya Rossetti

Tim Veitch (Calgary Homeless

Foundation)

Yvonne Chen

Participants

- 2

Pro

ject

Ove

rvie

w

Project in Review The Most Significant Change technique focuses on learning and reflection and, here, we share

some of the more important moments, decisions, and learnings from our MSC journey.

The original idea: 15 story submissions/month.

“What were we thinking? “

To include funders or not to include funder?

“We think it worked out.”

The first conversation in the first planning session started with Confidentiality.

“This remains the biggest chal-lenge for this project.”

Client stories vs. Case Manag-er stories about Clients.

“We ultimately got the latter.”

The content found in the sto-ries was amazing!

“We truly appreciate how invest-ed the storytellers were.”

Story length was a big issue.

“Did we want moments of change or more sweeping narra-

tives? “

Ground Rules!!!!

The intensity of the first meeting.

“Wow - that got emotional!”

Domains of change did not

work.

Lessons learned did not really

work.

No food makes participants

unhappy.

Commitment to the process.

“I was impressed by the selec-tion committees’ investment.”

First learned of MSC at a Workshop in October 2011.

“This is something we really wanted to explore.”

Planning Story Submission Selection Meetings

3 -

The Ebb and Flow of MSC

Pro

ject O

ve

rvie

w

Funders: “You do good work.”

Validation.

“..of the hard work and commit-ment of our staff.”

MSC Burnout.

Adding the frontline perspec-tive.

“We wanted to add their voices to the table.”

Change in the voting process.

“The two-stage concept was much better than the original.”

The story behind the story.

“It was a constant challenge to stick with what was written – especially when you knew the

cases.”

Learning for the administra-tive and operations staff.

“Learning was at the core of this entire project.”

The client was so proud and excited to hear the news that her story was selected!!!

Funders: “Thanks for trying

something new.”

Anti-Climactic.

“The process is more important than the story…and it does seem

to just tail off.”

The story sharing dilemma.

“A lesson was learned.”

Funders: “Thank you for in-cluding us in this project.”

Staff: “Thank you for provid-

ing a space to reflect on our

work.”

Staff: “Thank you for the chance to participate in this eval-uation process.”

Q: Future of the project?

A: “...it depends. What are you going to do with this?”

- 4

Feedback Final Selection Selection Meetings

When this client first came to Discovery

House, her self-worth was impaired due

to abuse and she had difficulty making

decisions for herself. She shared her

views with me about women in her cul-

ture. She said that women are not viewed

as being strong or independent and that

many women are abused by their hus-

bands. There were times that she

thought of returning to her abuser, and

she had pressure from her parents. But,

when her parents came to visit, they

learned from visiting Discovery House

that their daughter was okay, that she

should not return to her husband, and

that women should have more rights. The

client is very thankful for all of our sup-

port, for showing her respect, and for

helping her to believe in her ideas. She is

now more independent. Her parents saw

this independence as well, and she is

much happier with their support.

I always supported Ditka to make her own decision and re-

spected her. I didn’t know that respect could be so empow-

ering to someone that had not experienced that.”

Learning through Example

FAMILY

HEALTH & WELLBEING

SAFETY

Ou

r M

ost

Sig

nific

an

t C

ha

nge

Outcomes identified in analysis

Stars represent Final

Selection ranking

5 -

Ou

r Mo

st S

ign

ifican

t Ch

an

ge

Our Most Significant Change

Selection Talk

Initial Voting: Unanimous Initial Selection:

The ripple effect that can take place when we work with moms

Changes in this mom’s self-perception and perception of women had an intergenerational impact

Changes were to systemic and cultural beliefs Final Selection:

Discovery House promotes strong values of safety

With the right support, we can contribute to altering norms within a family

We have skilled, culturally-competent staff

We make a difference—across the world!

Final Selection Voting

INITIAL VOTE FINAL VOTE

Rank Score Rank Score

The Journey 4 85 3 130

Specialized Roles 6 40 6 10

Transitioning Tom 5 70 5 40

Christmas Past, Present & Future 3 105 2 150

Learning Through Example 1 290 1 270

Overcoming Communication Barriers 2 110 4 100

- 6

I have been working with this client and her

family for the past nine months. She immi-

grated to Canada and is currently taking Lan-

guage Instruction for Newcomers to Canada

(LINC) 1 even though she is illiterate in her

own language. At the beginning, it was diffi-

cult to communicate with each other and she

appeared very frustrated and overwhelmed. I

met with her twice a week sometimes for

two hours each visit, helping her to solve

problems with daily living. We went from me

helping her to understand what needed to be

solved and me making the phone calls, to

role playing and her making the phone calls

on speakerphone with me present, to role

playing and her making the phone calls on

her own. She has expressed that she is not

afraid anymore of solving daily problems,

that she feels more in control of her life, and

that she is proud to be paying her bills and

rent without external support.

This is a huge step for this immigrant woman who

was illiterate in her own language and is now trying

to learn a new system with new rules...”

Overcoming Communication Barriers

HEALTH & WELLBEING

SOCIOECONOMIC

HOUSING

Narr

ative

s f

or

Fin

al S

ele

ctio

n

Selection Talk

Initial Voting: Not unanimous Initial Selection:

Tasks that may seem trivial (making a phone call, paying bills) can be places for truly powerful and meaningful change (overcoming learned helplessness, building confidence and life skills, mom learning to stand on her own)

Discovery House as well-positioned to work with immigrant women; in this case, assisting client with culturally-rooted questions of identity (moving from “wife” to something new)

Final Selection:

We work with real people and the reality of their lives

We communicate through barriers and are well-positioned to do so

Commitment to clients and process

7 -

Specialized Roles One of the most significant changes I have seen in the past few

months at Discovery House is the addition of the Housing Liaison

role to the Community Housing Program. The Housing Liaison role

allows us to work more seamlessly as a team with each client, sup-

porting us to intervene early in tenancy issues and to more effective-

ly prevent evictions for our clients. The role has also been a signifi-

cant learning for the program, in that it has been filled with an indi-

vidual whose background is not in human services. As such, it has

required us to clearly delineate roles within the team, and to ensure

that the individual in the position has adequate training to effective-

ly deal with the situations and clients in the program.

The role has also been

a significant learning for

the program.”

HOUSING

Narra

tive

s fo

r Fin

al S

ele

ctio

n

Selection Talk

Initial Voting: Unanimous Initial Selection:

Complexity of working with our clients and necessity for foundation/orientation in human service

Importance of role delineation Final Selection:

We are an experienced agency that understands the sector and that seeks out specific ex-pertise

- 8

Last winter, this client had just moved from another

province, had no job, had just left an emergency shelter,

and was really struggling to pay bills and rent. At Christ-

mastime, I told her about the Adopt-a-Family program,

and her and her three boys were able to make a wish list

for Santa. This year, she expressed how thankful she

was that Adopt-A-Family was there for her last year

when she was in great need and that she no longer

needed to access the program. She also expressed that

next year she would like her and her children to adopt a

family, to give back to the community all that the com-

munity had given to them.

Christmas Past, Present, and Future Change takes time, and

families sometimes need a

little assistance in order to

realign after facing home-

lessness. The little things

can be very meaningful.”

SOCIOECONOMIC

Narr

ative

s f

or

Fin

al S

ele

ctio

n

Selection Talk

Initial Voting: Not unanimous Initial Selection:

This is a big-picture story, showing how small steps add up

Is this story really about presents or about maintaining the role of a parent while in crisis? Final Selection:

We provide skills to become self-sustainable, tools for healing, and ability to look forward Client’s desire to give back as a positive reflection of Discovery House

9 -

Over the past three months, I have worked

with a six-year old boy who has seen many

positive changes. I have spent time rela-

tionship building with mom and him, so he

would know and trust me. We also spent

time doing things that he likes in many

different environments (inside, outdoors,

at home, at school, in the community),

which has helped me to see his strengths.

When we started our sessions, he dis-

played many aggressive behaviours. He

would call me names and would cry be-

cause he did not want me to leave. He now

seems to be able regulate his emotions

when things are put away and when ses-

sions come to close. He still has difficulties

when I leave, but there is no longer crying

or name calling. Instead, he asks if I can

stay longer and tells me how much he likes

playing together.

The change in his ability to imagine, play, laugh, engage and

respond was incredible from when he is inside than when he is outdoors.”

HEALTH & WELLBEING

FAMILY

Narra

tive

s fo

r Fin

al S

ele

ctio

n

Selection Talk

Initial Voting: Not unanimous Initial Selection:

Stability and consistency as precursors to relationship-building and trust; it takes time

Going to the client instead of making the client come to us (at home, the park, coffee shops, school, outdoors, doctor’s office, public)

Final Selection:

We understand the resources we need and hire the right, competent people

We are a strength-based, learning organization

Transitioning

- 10

When I first began to work with this client, she was

very shy and withdrawn. It was often hard for her to

engage and she kept her experience close to heart.

She often stated that when she was in Calgary, she

wished to be back in her home community and when

she was in her home community, she wanted to be

back in Calgary. With time, the client began to attend

more sessions and to contact me when she wanted

to process out situations she found herself in. Later,

she went to her home community to visit family, lost

contact, and then reached out about a month later.

She was contemplating moving back to live with her

mom and dad so her youngest child could begin pre-

school there. We talked about the pros and cons and

how she would keep herself safe. The client stated

that working with her CHP team was the first time

that she experienced trust with a system, like she

could speak her truth and feel supported. The client

has returned home. However, it is my belief that we

planted seeds of change.

It isn’t about solving the problems our clients face but walking

alongside them and giving them a chance to develop and experi-

ence decision making in a safe, trusting and supportive environ-

ment.”

SAFETY

HEALTH & WELLBEING

Narr

ative

s f

or

Fin

al S

ele

ctio

n

11 -

The Journey

Selection Talk

Initial Voting: Not unanimous Initial Selection:

Significance of an Aboriginal woman trusting an agency represented by a young, white woman

Case Manager supporting her in understanding her options and in staying safe, regardless of choice

“This is what it’s all about.” Members familiar with the frontline felt that this story is representative of what they go up against every day

Final Selection:

Discovery House provides ways of thinking about being safe, not just shelter

Discovery House as culturally-sensitive/aware

Staff skilled in letting clients self-pace, relationship-building

Sh

ort S

torie

s Nineteen Stories

Note on Voting Process

We experimented with two different styles of voting to see which would be most appropri-

ate for Discovery House. The first method was open and fluid, where committee members

placed checkmarks on flipchart paper that could be changed throughout discussion. The

second method was more structured, using two rounds of secret ballot scorecards to sand-

wich discussion. In both cases, votes were changed after engaging in discussion about why

committee members thought specific stories should go forward. In this way, voting not only

personally subjective; it was also influenced by committee composition.

The 19 stories that follow are summarized versions of their originals, adapted with the aim of making them more accessible.

(A report with 25 full narratives would be a long read!) Please keep in mind that although these stories did not make it to Fi-

nal Selection, they were a meaningful part of the process and often focal points for discussion. Perhaps in another MSC uni-

verse with differently balanced committees, one of these short stories would be our Most Significant Change.

- 12

Sh

ort

Sto

rie

s

Short

Stories

Baby Steps

This client has been struggling a lot since being in our pro-

gram, and even though our support was there, she wasn’t

utilizing it. When she finally realized that there wasn’t any-

thing that she could do that was going to scare us away, she

started to trust us and was open to our support. Her beginning

to connect with us is the beginning of her making steps to-

wards the goals she has for herself.

Change for the Entire Family

When this family entered the program, mom’s primary focus

was on one of her children who had severe behavioural issues

and required a lot of time and energy. However, as months

passed and the child’s behaviour improved, mom was able to

see other things and to set other goals. By taking it one step at

a time and working on the present issue, a shift took place

that created room for the family to continue growing in other

areas.

Moving Forward

This client had a good childhood upbringing and was unfamil-

iar with abuse, so when her partner abused her and her

daughter, she didn’t know what signs to look for. What is

amazing about her story is that she wanted to change this by

educating herself about abuse, its effects, signs to look for,

how to protect herself and her daughter, and the resources

that she can access. What an amazing and courageous young

lady!

Full Circle

Not that long ago, this client was picking up the pieces of a

broken home and setting new roots in the community. She

approached life day by day, only being able to handle the de-

mands of her life and her children on a 24-hour snapshot.

Today, her world is much different. She is reinventing her nar-

rative— no longer identifying as a victim, but as a survivor;

seeking out resources; and engaging in community activism.

Transforming Anger

This client felt very depressed when she arrived at Discovery

House because, after only one month of being in Canada, her

marriage had broken down—something that conflicted with

her cultural beliefs. However, she turned her anger and self-

blame into self-determination and, with the help of her Case

Manager, was able to access community resources. She found

a daycare, applied for a parenting order, sought Legal Aid

counsel, got her driver’s license, found a part-time job, and is

now volunteering in research related to her profession.

When she finally realized that

there wasn’t anything she could

do that was going to scare us

away, she started to trust us

and was open to our support.”

However, she turned her anger

and self-blame into self determi-

nation and, with the help of her

Case Manager, was able to ac-

cess community resources.”

13 -

MSC 2013—FINAL SELECTION

Sh

ort S

torie

s

Family Meetings Work

This family had been here for five months and still did not feel

comfortable enough to disclose what they were feeling. When

separate from mom, the children reported that everything

was fine. However, when the writer met with the whole fami-

ly, they began to open up about their true feelings on how

certain aspects of their lives were creating stress, which in

turn had been expressed inappropriately. Together, we came

up with a plan to address some of the concerns. Since this

meeting, the family reports having better relations, having

made more time for each other, less verbal altercations, and

no physical altercations.

Getting to

Day Treatment Intake

This client has experienced all kinds of abuse throughout her

life, which has led to her struggles with mental health and her

use of substances to cope. Approximately one month ago,

Child and Family Services mandated this client to attend day

treatment due to increased drug use. After a number of weeks

of cancelled attempts to attend the intake appointment, we

went together this morning. This change is incredibly signifi-

cant because the client has struggled for years with this area

of her life and because—even though it is being “forced” on

her—she is excited about day treatment, which shows a hope-

ful shift. The road forward may not be straightforward or easy,

but she is beginning to make steps forward and to make fewer

steps backward.

Queen’s Cupcakes

After an upsetting incident with her father, this client became

quite subdued and not her usual sparkly self. However, in one

of our weekly sessions, she told me that she wanted to start

meeting twice a week so that we could spend one session

baking together. In our first baking session, we made cup-

cakes. When we were done, the client carefully carried them

out to the elevator, as if she were carrying a plate of precious

jewels. I asked her to give me one word to describe how she

was feeling and she said, “Proud!”

New Home

This client moved to the city from a reserve for safety and for

a new start; however, she had concerns about her housing

situation. After six months, she began taking steps to address

these concerns. Even though she was very anxious and need-

ed support in the process, she was able to advocate for her-

self and to ask for her needs.

A Delightful Mother-Daughter Duo

This child was quiet and shy in our sessions for the first six

months or so. Then, one day she just opened up and we had

some amazing conversations where we were able to process

her thoughts and feelings about the violence that took place

between her parents. This opened the way to a healthier rela-

tionship and more open communication between her and her

mother. Now, her and her mom find joy and excitement in

their relationship with one another.

I asked her to give me

one word to describe

how she was feeling,

and she said, ‘Proud!’”

Now, Nichole and her mom find

joy and excitement in their rela-

tionship with one another.”

When the writer met with

the whole family, they

began to open up about

their true feelings…”

- 14

Sh

ort

Sto

rie

s

Breaking the Shell

Since her move to Discovery House, this client has broken the

shell that she had encased herself in and that she had used to

cope. She has learned that she is a survivor, and not a victim,

and has made steps towards healthy relationships and safety

for her children.

Power in Numbers

This team has come to a place where I, as a practitioner and

an individual, feel safe to be me. Ultimately, we need to be

the change, and it needs to start with our working relation-

ships. To do this, people need to feel safe, empowered, have a

sense of belonging, and have an opportunity to reach our aspi-

rations—all great lessons we can bring forth to our clients.

Learning Together

This client and I were drawing a picture together and he

reached over to grab my marker without saying please. I cor-

rected him and asked him to try again. Thirty minutes passed

and we were still drawing and—without recognizing it—I

reached over and grabbed his marker and started to use it. He

stopped me and said, “HEY! You didn’t ask OR say please, Ms.

M!” He didn’t forget to say please or thank you after that!

Guiding Change

This client was an extremely angry child. He regularly had

screaming temper tantrums that were heard all over the

building and he was rude and bossy with his mom—

reproducing the attitudes and behaviours he had seen his dad

model. Through twice-weekly sessions, he has been able to

release a lot of anger and has learned other ways of express-

ing his feelings that are not as explosive or out of control. He

has even gotten to the point where he successfully attended a

week-long overnight camp this summer, which would not

have been possible last year.

A Family’s Future

This client is an immigrant woman with four children. While at

Discovery House, staff nominated her for a community bursa-

ry fund, which she received. Through this funding, she was

able to provide her children with gifts that will help with their

educational needs and emotional well-being, including a com-

puter, educational tools for a son with autism, a keyboard and

guitar for music lessons, clothing and Christmas gifts. I have

never seen her so happy as when she received the approved

items for her children. Because of the funding, we are also

able to continue to work with this family for three years—

something that will really benefit this family that faces so

many barriers.

This client was an

extremely angry child.”

He stopped me and said,

“HEY, You didn’t ask OR

say please Ms. M!”

She has learned that she is

a survivor, and not a victim,

and has made steps towards

healthy relationships and

safety for her children.”

15 -

Regain Confidence after Domestic

Violence

When this client dropped her children off at the Children’s

Centre, she trusted me and talked a lot about her life. When

she talked, I noticed that she looked hopeless and lacked con-

fidence. I could tell that she felt it was a great shame to come

to Canada and need government help even though she is ca-

pable. With the help of staff, she finally found a part-time,

professional job. Her face shone and she looked happier than

before. With the help of Discovery House, this client regained

her confidence.

Tools of the Trade

There has been a shift of focus in the program from theoreti-

cal to practical. Two months ago, the program was able to

order a variety of art supplies to use with clients while com-

pleting outreach work, and even more exciting was the ap-

proval to order two portable play therapy kits just last week.

Now, the Child and Youth Support Specialists can do the same

therapeutic work they could do in a playroom setting, but in-

stead bringing it to families in an outreach capacity.

Resiliency Becomes Reality

This client is a single mom who has encountered many barri-

ers in her lifetime, including an abusive partner. After her stay

at an emergency shelter, she found refuge at Discovery House

and spent the next year working on her personal issues and

learning more about domestic violence. She was also referred

to a career counselor and, as a result, decided to pursue a

career in social work. She and her children have now moved

back to their previous city where she is attending a Social

Work program. The change in this client during her stay at

Discovery House was remarkable. The strength of one woman

who endured so much!

Honouring the Courage to Be Real

She said that she wanted to lie to me and pretend that she

has her life together, to pretend that things were different

this time, but they weren’t. She knew she needed to hold her-

self accountable to her behaviours. She needed to be honest

and upfront with her support system in order to get the help

she truly needed. After a lengthy and emotional chat, she

agreed to a more intensive support plan.

She said she wanted to

lie to me and pretend

that she has her life to-

gether…”

With the help of Discovery

House, this client regained her

confidence.”

Sh

ort S

torie

s

- 16

An

aly

sis

Although participants were not asked to consider evaluation outcomes when choosing stories, these outcomes were apparent

throughout the collection. The illustration below depicts the frequency that each of our key outcomes—Housing, Socioeco-

nomic, Health & Wellbeing, Family, and Safety—appeared in the stories, telling us about the types of changes that staff valued

as significant.

HEALTH &

WELLBEING

33%

SAFETY

19%

SOCIO-

ECONOMIC

16%

FAMILY

25%

HOUSING

7%

Note on Analysis

Analysis is central to the MSC technique, and within this report, we focus on two streams of analysis:

Analysis within the MSC process

Participatory, discussion-based analysis within committees

Highlighted in this report in “Selection Talk” text boxes throughout stories/narratives, and in “Talking about

Change” on the following page

Analysis outside of the MSC process

Analysis undertaken by Program Evaluators outside of formal meetings

Includes, for example, content analysis highlighted in “Reading for Outcomes”

17-

Reading for Outcomes

An

aly

sis

Discussion is essential in the Most Significant Change technique, and we did a lot of discussing. One might say that the Initial

Selection Committee spent an entire workday (close to 8 hours over three meetings) discussing change stories—and without a

lunch break! From those meetings, several predominant themes emerged, which we have called Clients: Power of Self, Build-

ing Blocks for Change, Change Interpretation, and Staff: Role in Contributing to Change (see below). Together, the themes

build on each other, overlap, and are otherwise connected.

Talking about Change

The above themes were displayed during the Final Selection meeting to provide context for discussion and voting. Key discus-

sion points from the Final Selection meeting may be summarized as follows:

Discovery House…

Quality staff that does good social work

A learning organization

Culturally competent and well-positioned

Making a difference

- 18

Change Interpretation

Change as process; more steps forward than back Success might not be easily apparent The “trivial” can be powerful change

What the media might love to hear, staff might not want to put forward

CHANGE

Building Blocks for Change Time Trust

Respect Honouring the complexity of moms’ lives:

The good, the bad, and the ugly Flexibility

Meeting clients where they’re at

Clients: Power of Self Self-confidence and Self-worth

Sense of purpose Pressures to “be a good mom”

Why we focus on client-driven change “Our clients are pretty damn smart and resourceful!”

Staff: Role in Contributing to Change Provide tools

Connect Facilitate Empower

An

aly

sis

The Spheres The sphere of influence is an important concept that underlies all program evaluation at Discovery House. The sphere of influ-

ence is a model that forces us to see client changes/outcomes in terms of those that we can contribute to and those that we

cannot. Ultimately, we want our clients to achieve those long-term changes in condition and well-being that make up our vi-

sion, but recognize that we have very little control over that process.

From a program evaluation perspective, the goal then is to structure evaluation activities such that we monitor clients progress

toward changes that we actively contribute to. As you can see from the diagram below, our standard evaluation procedure

envisions this process as involving three stages from working alliance to achieving a standard set of agency outcomes. The

goal here is to structure outcome measurement such that we capture changes on the threshold of the sphere of influence and

that of the sphere of concern.

Working

Alliance

Our Vision for

the Clients

Sphere of Concern Sphere of Influence

Agency

Outcomes

Readiness for

Change

The challenge for us is that most evaluation activities assume that clients are working toward similar goals and that reaching

these goals has similar value for each client. The danger is that this assumption is somewhat flawed and does not lend itself

well to measuring significant changes that happen on the journey to a particular outcome (e.g., going to a walk-through vs.

permanent housing).

It is within these limitations that the value of MSC becomes most apparent. MSC not only captures these missed client chang-

es, but also layers these changes with the importance/significance to the individual. In this way, we can colour, layer, and add

to our standard evaluation activities so that we can give a better description of the changes that we are contributing to as an

agency.

It is not a surprise then that so many of the stories that were submitted described changes below the outcome threshold. The

stories were about the building blocks that lead to changes in our standard agency outcomes. In closing, this analysis leads to

three important questions:

1. Our standard agency outcomes were often absent or implied in the stories. What does this say about evaluation at the

agency? Should we explore the focus of our work?

2. Does the “theory of change” and associated values articulated in the stories match our official agency truth?

3. What changes would clients have written about? What is significant for them?

19 -

An

aly

sis

The task to evaluate change is not an easy one. Change, after

all, is subjective. One change may be considered significant in

one situation or by one person, and not in another situation or

by another person. Thus, questions of context and value are

central in evaluating change, as it is in MSC. In the case of our

MSC 2012-2013 project, analysis of the narratives gives us

meaningful insight into how staff members view clients,

change, and their roles as related to client change.

First, the analysis of narratives indicated a consistent view of

our clients as occupying complex social locations where gen-

der, culture, class, and country of origin (for example) interact

in specific ways to form complex challenges in systems that

are not always friendly or helpful. Perhaps most notably, we

see that many of our stories about immigrant women touch

on the challenges in navigating an unfamiliar system, some-

times with limited language ability and with limited financial

resources, while simultaneously facing family or cultural pres-

sures to return to an abuser. Similarly, stories about Aboriginal

families touch on the possible implications of choosing to

leave or return to a reserve, speaking to questions of culture

shock, trust, and social support systems. Thus, throughout

stories, it is evident that clients are not simply leaving domes-

tic violence—they are doing so in context, while navigating the

complexity of their lives and of the systems that they partici-

pate in. Moreover, it is evident that staff members appreciate

and adapt to client context, no matter how complex.

Perhaps it is no surprise, then, that much of the value taken

from the stories is not directly tied to outcomes (such as hous-

ing and safety), but to change process. Consider, for example,

the ability and confidence to make a phone call. Out of con-

text, this change might not appear to be meaningfully related

to our mission; yet, not only was this change considered sig-

nificant by the storyteller, it was also considered significant by

the committee members who voted the story forward for final

selection. We value this change because it represents a mean-

ingful step in this client’s process to live safely in the commu-

nity. This message repeats itself throughout the collection

(from portable play therapy kits and art supplies to the repeat-

ed focus on flexibility and readiness for change): We under-

stand change as individual, non-linear, and non-formulaic; we

value change as a journey.

Lastly, this project tells us about how we view our role in this

journey. There is no story in which a staff member claims to

be the sole source of client change. Instead, we see a con-

sistent emphasis on working alliance, talking again and again

about relationship-building and trust (from the skill to know

when to call a meeting short to the client who experienced

trust with a system for the first time). We also see a consistent

emphasis on conveying knowledge and connecting clients with

resources (from the client who wanted to learn more about

domestic violence and its effects to the client who has con-

nected her family with more resources than most of us could

name). What all of this tells us is that we view our role in cli-

ent change as one of contribution, not attribution, where we

facilitate, rather than effect, change.

This first iteration of MSC at Discovery House has been a valu-

able tool to examine agency values around change. Overall,

what we have learned is that when it comes to change, we

address the whole picture of domestic violence by engaging

context, valuing change as journey, and contributing to

change as facilitators. As one storyteller expresses, “It isn’t

about solving the problems our clients face, but walking along-

side them” in their journeys of change.

Change as Journey

- 20

Reflection

s

21 -

The Last Word As in much qualitative research, reflection is a powerful tool in the MSC technique. Throughout this pilot project, we have tried

to provide spaces for reflection to participants and to engage in active reflection ourselves. Here, we share some of these re-

flections with you.

First Selection Meeting

First, I was surprised by the rawness of some of the discussion.

To me, rawness reflects sincerity, and I think that this is an

excellent way to begin an evaluation process that is so de-

pendent on sincere discussion. I am thankful that group mem-

bers were willing to share this.

Second, upon reflection, I am more keenly aware of how or-

ganic this technique can be. There is no single formula to de-

termine which significant changes are most significant for Dis-

covery House. Rather, this is a unique opportunity to listen to

the diverse voices at the table, behind the stories, and within

our programs. Learning from this process could prove as valu-

able as the ‘final product.’

Second Selection Meeting

Creating spaces for discussion is not easy. Participants need to

be open, interested, and committed. (And, I’m thankful for the

group we have!) Beyond that, scheduling, structure, and “the

right organizational moment” are important. I do think this is

that moment, and even as we work out the details, I have al-

ready heard how valuable this process has been to some.

As for content, what stands out most to me is that moms’ and

children’s struggles are highlighted without using words of

judgment; instead, clients are written and talked about as deci-

sion makers.

Third Selection Meeting

As I receive informal feedback from participants, the MSC pro-

cess is validated as meaningful and useful. At the same time, I

have heard that we need to be careful of “MSC burnout”—

something that resonates with me. Reflective processes need

to be balanced, happening often enough to be connected, and

not so often as to be repetitive. Looking forward, this will be

an important design consideration if we want to maintain the

same level of engagement.

Final Selection Meeting

[MSC Burned Out]

Facilitator Flashbacks

Eliana Clay

Program Evaluator

We’ve spoken with almost everyone that participated in our

pilot cycle. Your feedback is in, and it’s a little bit ambivalent.

Do you think that we should prescribe domains of change

(themes)? Yes and No. Do you think that we should switch to

clients as storytellers? Yes and No. Do you think that we should

find a way to share the stories externally? Yes and No.

So, what do you agree on? Almost without exception, you agree

that this has been a valuable process, that you have appreciat-

ed the opportunity to share with and/or learn from your col-

leagues, that meetings were too long, that committees should

be more balanced, and that you want to see the results and

what we do with them.

Until now, you have probably only seen or experienced MSC in

the pieces you were directly involved in. We hope that this

report begins to address the question of results, providing you

with a more complete MSC picture. And, as always, know that

this is a work in progress.

Participant Feedback

Refle

ctio

ns

Best Laid Plans

Edwin Rogers Senior Program Evaluator

I would like to start with a thank you. This project would not have been possible without the efforts of all the project

participants and I wanted to extend my gratitude to all of you. I would also like to personally thank Eliana Clay for her

work on this project. It may have started with my vision but Eliana was gracious enough to take on the challenge and to

make the project her own. This report is a culmination of her passion and hard work and we would never have reached

this point without her. So, Eliana, thank you!

As I reflect on this project, I keep coming back to one word – Unexpected. I had a vision for how MSC would unfold and I

have to admit that much of what transpired did not meet that vision. This was true right from the beginning when we

first posted the part-time contract program evaluator position that would make this project possible. The quality of the

applicants far exceeded those of what I thought we would attract and the knowledge and experience of the successful

candidate changed the entire approach that I wanted to take. I had envisioned leading the process myself but instead

found someone who could champion the project for me.

From that point on, I was often pleasantly surprised at the unexpected events that transpired as we moved through the

process. I had not expected to include funders in the process, but they became an interesting and important part of the

selection committees. I had expected client stories, but instead found that we had case manager stories about cli-

ents. This was an important learning for me and completely changed how I felt about the project. I had expected narra-

tives about singular moments of change, but instead received much more sweeping large scale narratives about the suc-

cesses of our clients. I had expected people to be interested in the process, but was not ready for the intensity and pas-

sion of those writing and selecting the stories. I had expected the meetings to be smooth and comfortable for partici-

pants, but instead found them quite intense. We certainly had bitten a lot off with this project. I had assumed that the

stories would illuminate and highlight our work, but I did not expect how sweeping and powerful that process could

be. Many people left the table with a very different feeling about us and our work (the funders especially). Finally, the

story that was selected as most significant was not what I expected it to be.

For me, all the unexpected events speak to the success of this project. MSC is a participatory approach that’s value lies in

the shared exploration and learning of all those who participate. That this project took on a life of its own and was

shaped so clearly by all those who participated is the greatest compliment I could ever give it.

So, thank you, I am so glad that nothing turned out as I expected!

- 22

T H A N K S .