Programme evaluation of an adolescent psycho-educational

group for eating disorders entitled “New Directions”

Authors: Mary Kaye Lucier MSW, RSW, Executive Director, Bulimia Anorexia Nervosa Association (BANA), Rosanne Menna PhD, C Psych. , University of Windsor; Department of Psychology

Date: May 18, 2007

Executive Summary

• New Directions is a ten week-1.5 hour per session, psycho-educational group invention programme for adolescents aged 12 – 18 years, who have been diagnosed with an eating disorder.

• The goal of the New Directions programme is to increase insight and awareness of four related factors contributing to eating disorders: (i) eating disorder symptoms; (ii) factors contributing to the development of eating disorders; (iii) the physical and mental effects of eating disorders; and (iv) developing a plan for recovery.

• The programme was designed to interrupt symptoms related to eating disorders such as extreme dieting, body dissatisfaction and unhealthy weight control methods.

• Referral into New Directions is based on the results of a specialized structured eating disorder diagnostic assessment.

• Following the specialized assessment, each referral is presented at the weekly rounds, for a diagnosis.

• The diagnosis is based on the DSM IV criteria for eating disorders. Each person is provided with a treatment plan.

• Those with the diagnosis of an eating disorder are invited to attend the New Directions group.

• Following the completion of New Directions, the participants are reassessed and presented for a re-diagnosis and re-evaluation of the treatment plan.

Evaluation questions and purpose for the evaluation:

• The evaluation questions were:• (1) What are the characteristics of the adolescent clients, aged 12 – 18

years, who present for assessment at a community based treatment centre for eating disorders?

• (2) What are the changes in eating disorder symptoms from the pre-treatment assessment to the post treatment assessment at the end of the New Directions programme?

• (3) What is the average length of stay in the programme?• The purpose for the evaluation was to: • To create a snapshot of the participants in the New Directions programme

by analysing archival data collected on participants of the New Directions programme from September 1, 03 - March 31, 07;

• To enhance the agency’s capacity for more complex research on client outcomes;

• To disseminate project findings;• To create an evidence-based programme for adolescents with an eating

disorder that is accessible through a training manual suitable for delivery in other agencies.

Description of the Methods Used

• The specialized eating disorder assessment consists of a diagnostic interview, followed by the administration of psychometric tests that measure eating disorder psychopathology (i.e. symptoms, body dissatisfaction, and size acceptance), related psychological functioning (i.e., self-esteem, depression) and demographic information.

• The diagnosis is based on the DSM IV criteria for eating disorders, and is under the supervision of either a clinical psychologist or physician.

• Two assessments were done, each comprising of the above measures. Assessments were done before the groups began (at initial intake; pre-test) and following completion of the New Directions programme (post-test). Ninety-eight participants presented for treatment at the agency.

• Of those 74 were assessed for the New Directions programme for the period 2003 - 2007. • Twenty-four adolescents completed the programme.• The diagnostic interview and questionnaires were administered on all agency clientele and were

subject to agency policy and procedures. • The data collected for programme evaluation was explained at the time of the initial visit and was

presented on the agency "Consent to Treatment and Research Form". • Subjects had the option to withdraw their consent at any time. • All of the diagnostic assessments were conducted by trained clinicians in the field of eating

disorders. • Policies and procedures exist within the agency that reflect the most current and up to date

legislation on the collection, retrieval and storage of personal and health information. • The data entry personnel were required to sign confidentiality statements, prior to data entry.

Steps Involved in the Research Project

• 1. Archival data was entered into agency database. • 2. Data was collected from a specialized diagnostic

assessment interview of eating disorder behaviours, and self report questionnaire that measure attitudes, feelings, thoughts and behaviours related to the symptomatology of eating disorders, and related psychopathology.

• 3. Participants’ participation in agency research component for the purposes of data collection and data analysis, is voluntary.

• 4. Participants were reminded that participation in no way affects their ability to participate in the programmes offered at the agency.

Summary of Main Findings

Demographic Description

• The majority of the participants in the New Directions programme were female, from divorced or single parent families.

• The average age for the adolescents in the programme was 15. 3 years (SD= 1.8 years) years.

• The majority were Caucasian, and all spoke English.

• As a group, the sample tended to be middle class.

Changes in Psychological Functioning

• The majority of the participants of New Directions reported suicide attempts, suicide thoughts and plans for suicide at intake.

• Although most of the participants of New Directions reported no form of drug or alcohol use, one third reported using alcohol or cannabis socially, or referred to themselves as an alcoholic and/or a binge drinker at intake.

• Self-harming behaviours, sexual and emotional abuse were indicated by nearly one-half of the participants at intake as well.

• Most participants reported that the New Directions group was the first attempt at any form of psychiatric treatment.

• With the exception of cutting which showed a slight increase in the behaviour for those who completed the group, there was a general indication of some change in self-harm behaviours reported.

• Self-esteem scores indicated a small increase from pre to post New Directions.

• Depression scores showed little change from pre to post New Directions.

Changes in Eating Disorder Symptoms

• Overall, the participants who completed New Directions reported a general reduction in eating disorder behaviours.

• Feelings and reactions to possible weight gain did not change from pre to post New Directions.

• At post assessment, those who completed five or more sessions of the program, reported fewer binge episodes, vomit days, laxative days, and diet pill days than those who completed less than five sessions of New Directions.

• Participants reported less dieting and fewer attempts to lose weight after the programme

Changes in Diagnosis:

• For the purposes of this study, there were three possible diagnostic categories: – Anorexia Nervosa (AN), those with a Body Mass Index (BMI) of

less than 85% of BMI 20, severe dieting behaviours and body image disturbance;

– Bulimia Nervosa (BN), those who exhibited binge and/or purge behaviours and body image disturbance;

– and those with not otherwise specified eating disorders (EDNOS).

• Few were AN (7 percent), one third were BN, one third were EDNOS, and nearly one quarter could not be diagnosed due to lack of information or did not have an eating disorder.

• At post assessment approximately 70 percent of the categories of diagnoses did not change.

Average length of stay in New Directions

• Seventy-four adolescents were referred to the New Directions programme.

• Twenty-eight percent did not begin the programme. These individuals either lost contact with the agency (e.g., moved away) or chose not to begin the programme.

• The completion of the New Directions programme was determined by two factors: – those who completed less than five of the New Directions group– and those who completed five or more of New Directions group.

• Nearly one-half of the adolescents referred, completed the programme.

Summary of Implications of Findings

• The drop out rate from eating disorder treatment programmes is generally 50% for AN, and 30% for BN (Vandereyeken, & Pierloot, 1983; Mahon, 2000).

• This study indicated an overall drop out rate from New Directions of 28%.

• Those who completed more than five New Directions, reported less abuse, alcohol/drug use, and suicidal thoughts, plans and attempts at pre-treatment assessment.

• This finding implies that a history of alcohol/drug use, abuse, and suicidality may be a factor affecting the completion of New Directions programme.

• The high incidence of reported self-harm, abuse and suicidality implies that posttraumatic stress disorder and other psychiatric conditions may be present and affect the completion of New Directions programme.

• The high incidence of reported self harm, abuse and suicidality, also implies that the expected increase in self-esteem, lowered depression scores and reduction in eating disorder symptoms, were less than expected.

Summary of Recommendations

• Five recommendations resulted from the project:• The New Directions programme does show promise. • The evaluation should include additional screening tools to assist

with a better treatment fit, and a 6-month follow-up.• The Beck Depression Inventory-II should be included to assess

severity of depression• The Pre-treatment assessment should include a readiness to

change measurement• The assessment should screen for posttraumatic stress disorder. • Develop a pre-contemplation group for those who are not ready to

change and/or are experiencing symptoms of posttraumatic stress. • The pre-contemplation group would focus on symptom management

and harm reduction.

Next Steps

• Presentations and workshops about the programme and research findings will be developed for presentation at agencies across Ontario and Canada.

• An executive summary of recommendations to the Ontario Network of Eating Disorder Service Providers regarding treatment outcomes and programme evaluation will be prepared.

• Study findings will also be available on the BANA website, University of Windsor REB website, and Provincial Centre of Excellence for Children and Youth Mental Health at CHEO website.

• BANA will also deliver presentations at professional meetings and conventions, and publish findings in newsletters, and professional journals.

Methodology

Design• 98 adolescents presented for treatment at the agency. • Of those 74 were offered the New Directions

programme. • Twelve percent of the 74 adolescents were referred to

another treatment. • Twenty-eight percent did not begin the programme. • These individuals either lost contact with the agency

(e.g., moved away) or chose not to begin the programme at the initial assessment (pre-test).

• Thirty-eight adolescents began the programme. • Fourteen participants attended five or less sessions. • In total 24 adolescents completed the programme.

• Referral into New Directions was based on the results of a specialized structured eating disorder diagnostic assessment.

• The specialized eating disorder diagnostic assessment consists of a diagnostic interview, followed by the administration of psychometric tests that measure both eating disorder psychopathology, and related psychological functioning.

• Following the specialized structured eating disorder diagnostic assessment, each referral was presented at the weekly rounds, for a diagnosis.

• The diagnosis is based on the DSM IV criteria for eating disorders, and is under the supervision of either the clinical psychologist for persons aged 16 and older, or the physician for persons under the age of 16 years.

• Following the diagnosis, each person was provided a treatment plan. Those with the diagnosis of an eating disorder were invited to attend the New Directions group.

• Following the completion of New Directions, the participants were reassessed and presented for a re-diagnosis and re-evaluation of the treatment plan.

Data Collection

Demographic Questionnaire. • Adolescents were asked to provide their age, height,

weight, language spoken, ethnicity, and parent’s occupation.

• Weight and height were used to determine body mass index (BMI; calculated using the following formula: weight (kg)/height (m)2).

Diagnostic Interview. • Adolescents were interviewed about their typical day of

eating, binge, vomiting, fasting episodes, laxative, diuretics, and diet pill use, exercise, body image, body satisfaction, abuse history, alcohol and drug use, suicidal thoughts and behaviour and self-harm behaviour.

Eating Disorders Inventory-2 (EDI: Garner, 1991).

• The EDI-2 is a 91 item self-report measure of eating disorder symptoms.

• In completing the measure participants answer a number of questions about their shape, weight, and eating on a 6-point scale ranging from never to always.

• The EDI Drive for Thinness, Body Dissatisfaction, and Bulimia subscales were used in this research to describe the severity of participants’ eating disorder pathology.

• Extensive psychometric support for this instrument and norms for adolescents are available in the treatment manual (Garner, 1991).

Child Depression Inventory (CDI; Kovacs, 1983).

• The CDI is a 27 item self-report measure of depression.

• Each item consists of three statements graded in order of increasing severity from 0 to 2.

• For each item participants respond by marking the sentence that best describes them during the past 2 weeks.

• The measure has high levels of internal consistency, re-test reliability and convergent validity.

Rosenberg Self-Esteem Questionnaire (Rosenberg; 1979).

• This measure consists of 10-items addressing global positive or negative attitudes toward the self.

• Items are of the following type; I feel that I have a number of good qualities.

• All in all, I am inclined to feel that I am a failure. • Participants rated on a five-point scale (1=not at all

accurate, 5=completely accurate) the extent to which each statement is felt or experienced.

• A total is computed by summing the ratings on all items. • The possible scores on self-esteem range from 1 (low

self-esteem) to 50 (high self-esteem). • The instrument has been used widely and has good

reliability and validity.

Psychometric Testing• The forms and questionnaires were administered on all agency clientele and

were subject to agency policy and procedures. • The data collected for programme evaluation was explained at the time of

the initial visit and was presented on the agency "Consent to Treatment and Research Form".

• Opportunities for debriefing occurred at the time the consent was given. • At the time consent was given, participants were provided with copies of the

consent form, and opportunity to ask questions about the research component of the agency programme.

• At the beginning of the specialized diagnostic assessment interview, each participant was informed that the data collected for their file would be used for the purposes of programme evaluation.

• Each participant was also informed that they have a right to withdraw from the programme evaluation project at any time.

• Each participant was informed that the data collected for the purposes of programme evaluation would be kept confidential, and would not be used for other purposes without their prior consent.

• Participants’ participation in the agency research component for the purposes of data collection and data analysis, is voluntary.

• Participants were reminded that participation in no way affects their ability to participate in the programming offered at the agency.

• Each participant who agreed to participate in the programme evaluation, was provided with a research data entry number, and thereby, the identifying information in the database is unknown to the researcher.

Sources of information and data report

• Adolescents completed self-report questionnaires and structured interviews. • The questionnaires and diagnostic interviews were administered by trained qualified clinicians in

the field of eating disorders. • Each therapist has access to the hard copy file of the adolescents’ interviewed. • Each clinician is required to sign a confidentiality agreement, as a condition of employment. • Agency policies and procedures are in place, which reflect the Personal Health Information

Privacy and Protection Act (2005) to safeguard the collection, storage, and disposal of the data that is being collected.

• Undergraduate and graduate students in social work and psychology programmes entered data.• The data entry personnel were required to sign a confidentiality agreement and an offer to hire

stating that they will abide by agency policy and procedures relating to data collection and storage, as a condition of employment and prior to entering data.

• All hard copy data is stored in a locked file cabinet. All data is stored on the agency data base, protected by password.

• The principal investigator has access to the hard copy and the research database for the purposes of checking the data for accuracy.

• The co-investigator also has access to the hard copy of the data collected and the research database for the purposes of checking accuracy.

Evaluation limitations

• The sample size was small, making the findings and implications for the drop and completion rates from New Directions difficult to interpret.

• Generally the drop out rate for eating disorder treatment programmes is 50% for AN, and 30% for BN. Twenty-eight percent of the adolescents participated in the pre-assessment but did not attend programme.

• Forty-one percent did not complete the program (attended five or less sessions).

• The research on treatment outcomes for eating disorders indicates that substance abuse, sexual abuse, psychiatric co-morbidities and suicidality negatively affect outcome.

• This evaluation project revealed that those who completed more than five sessions of New Directions, reported less abuse, alcohol/drug use, and suicidal thoughts, plans and attempts at pre- treatment assessment.

• The evaluation did not include a follow-up assessment component (e.g. 6-month post-testing), therefore any positive or negative changes over time are unknown.

Results

• Research question #1: What are the characteristics of the adolescent clients, aged 12 – 18 years, who present for assessment at a community based treatment centre for eating disorders?

• The adolescent clients presenting for assessment at BANA reported typical patterns of eating and compensatory behaviours to persons with eating disorders.

• At intake, the participants mean height and weight was 5’3” (SD=.39), and 132 lb. (SD=52.33). The mean body mass index (BMI) was 24.1 (SD=7.76).

• There were four main classifications of diagnoses: Anorexia Nervosa (AN), Bulimia Nervosa (BN), Binge Eating Disorder (BED), and Eating Disorder Not otherwise Specified (EDNOS).

• Approximately 7% were diagnosed with AN; 30% with BN; 30% with EDNOS, and 9% diagnosed with BED.

• The majority of the clients were female (94 percent).

• Fifty-seven percent were from divorced or single parent families.

• The average age for the adolescents was 15. 3 years (SD= 1.8 years).

• The majority of the adolescents were Caucasian (97%) and the remaining adolescents were aboriginal.

• All participants spoke English. • As a group, the sample tended to be middle

class.

Table 1: Height, weight, and Body Mass Index of population

N Mean Std. Dev. Range

Height at Intake (feet and inches)

92 5.3 .39 4.05-5.98

Weight at Intake (pounds)

93 132 52.33 45-368

Body Mass Index

86 24.14 7.76 14.6-50

Suicide

• History of suicide attempts, current and past thoughts or plans for suicide were reported for many (58%) of the participants.

Table 2 illustrates a description of the suicidality for participants of New Directions at intake.

No Suicide History

Current

Suicidal Thoughts

Current Suicidal Plan

Past Suicide Thoughts

Past Suicide Attempt

42% 17% 3% 27% 11%

Alcohol/Drug Use

• The majority of the participants indicated no form of drug use, (80%), or alcohol use (73%).

• Of those who reported using drugs, 15% indicated cannabis.

• Of those who reported using alcohol, 18% indicated being a social drinker, and 6% referred to themselves as an alcoholic and/or binge drinker.

• Although the majority of the participants of New Directions reported no form of drug or alcohol use, one third reported using alcohol or cannabis socially, or referred to themselves as an alcoholic and/or a binge drinker at intake.

Self-Harm

• Approximately sixty-four percent of the participants reported no self-harm behaviours, and the remaining one third of the participants (36%) reported some form of self-harming behaviours.

• Most participants reported that the New Directions group was the first attempt at any form of psychiatric treatment.

• Most participants reported no previous psychiatric treatment. Less than 1% had previous inpatient psychiatric treatment and 2% reported outpatient treatment.

Abuse

• The majority reported no form of abuse (60%).

• Abuse was indicated by nearly forty percent of the participants at intake.

• Of those reporting abuse the following table indicates the type of abuse reported.

Table 3: Reported abuse at intake

Physical Sexual Emotional Physical & Emotional

Physical & Sexual

Emotional & Sexual

11% 26% 43% 14% 3% 3%

Research question #2:

• Research question #2: What are the significant changes in eating disorder symptoms from the pre-treatment assessment to the post treatment assessment at the end of psycho-education?

Changes in Eating Disorder Symptoms:

• Paired-sample t tests were conducted to evaluate the effects of the group on eating disorder symptoms.

• Overall, the findings were not significant.

• The pre and post mean scores indicate a general reduction in eating disorder behaviours.

Table 4 shows the means scores pre and post the New

Directions programme.

Variable/Behaviour Pre Post t-test

Mean SD n Mean SD n

Eating Disorder Symptoms

Dieting days in past 28 days 21.28 12.20 25 16.80 14.00 25 t(24) = 1.69, p = 0.10

Dieting days in past 3 months 67.25 37.92 24 57.50 42.35 24 t(23) = 1.35, p = 0.19

Binge episodes in past 28 days 6.54 11.10 24 4.58 6.68 24 t(23) = 1.19, p = 0.25

Binge episodes in past 3 months 25.21 42.73 24 15.83 21.01 24 t(23) = 1.12, p = 0.27

Binge days in past 28 days 7.33 10.57 21 4.10 5.91 21 t(20) = 1.80, p = 0.09

Binge days in past 3 months 22.19 28.29 21 14.48 17.90 21 t(20) = 1.38, p = 0.18

Vomit days in past 28 days 9.12 10.87 25 7.84 9.78 25 t(24) = 0.64, p = 0.53

Vomit days in past 3 months 30.64 40.29 25 32.08 43.11 25 t(24) = -0.20, p = 0.84

Fasting days in past 28 days 8.48 10.42 25 7.52 9.39 25 t(24) = 0.52, p = 0.61

Fasting days in past 3 months 26.52 32.46 25 27.482 28.45 25 t(24) = -0.20, p = 0.84

Laxative days in past 28 days 0.04 0.20 25 0.00 0.00 25 t(24) = 1.00, p = 0.33

Laxative days in past 3 months 0.24 0.83 25 0.12 0.44 25 t(24) = 0.62, p = 0.54

Exercise days in past 28 days 10.08 10.98 25 9.08 10.27 25 t(24) = 0.41, p = 0.69

Exercise days in past 3 months 28.64 32.50 25 23.24 29.75 25 t(24) = 0.91, p = 0.37

• Table 4: Change in eating disorder symptoms: pre - post New Directions

Changes in Diagnosis

• Table 5 reports the frequencies and percentage for categories of diagnosis pre and post programme.

• From pre to post New Directions, there were no changes in seventy percent of the clients’ categories of diagnoses.

Table 5: Change in client diagnoses from pre to post New Directions

Presenting Problem N % of Total

No change 14 70.00%

BED to EDNOS BN-P 1 5.00%

From an ED diagnosis to no diagnosis 1 5.00%

BN-P to EDNOS BN-P 1 5.00%

BED to BN-NP 1 5.00%

BN-P to pending diagnosis 1 5.00%

EDNOS AN-R to BN-P 1 5.00%

Table 6 shows the mean pre- and post- assessment scores on eating disorders symptoms, self-esteem and depression

inventories.

• Significant differences between the scores were not found, however the sample size limits data analyses and interpretation.

• Self-esteem mean scores indicated a small increase from pre to post New Directions programme.

• Eating disorder symptoms and depression scores showed little change from pre to post intervention.

Variable/Behaviour

Pre Post t-test I

EDI

Mean SD n Mean SD n

EDI Drive for Thinness

12.46 5.90 24 12.08 6.53 24 t(23) = 0.28,p =.78

EDI Bulimia 4.67 6.13 24 4.88 5.41 24 t(23) = -0.38,p= 71

EDI Body Dissatisfaction

17.96 7.98 24 19.13 8.64 24 t(23) = -0.72, =.48

Rosenberg

Rosenberg Self-Esteem

22.40 5.68 15 23.13 6.79 15 t(14) = -0.13,p=.90

CDI

CDI Total 17.67 10.99 6 18.17 11.27 6 t(5) = -0.28, p= .79

Research question # 3: What is the average length of stay in the programme?

• The average number of sessions attended for the entire sample was 5.60 (SD =2.2; range 2-10 sessions).

• Table 7 shows the percentages for sample drop out and completion rates.

• The drop out rate was created by tabulating those who did not attend any New Directions sessions.

• There were two types of dropouts: those who chose to drop after the pre-treatment assessment and those who lost contact with the agency after the pre-treatment assessment.

• There were two types of treatment outcomes: – (1) those that completed five or fewer sessions of the New Directions

group and– (2) those that completed five or more of the treatment sessions.

Table 7: Sample drop out and completion rates

Drop out rate (N=74) Completion rate (N=34)

17% after pre-treatment assessment

11% Lost contact with agency

41% Completed < 5 sessions

59% Completed > 5 sessions

Table 8: Characteristics of clients who completed group and clients who attended five

or less sessionsVariable/Behaviour Pre Post t-test

Mean SD n Mean SD n

Eating Disorder Symptoms

BMI 26.36 10.78 14 23.05 6.31 20 t(32) = 1.13, p = 0.27

Dieting days in past 28 days 20.53 12.82 14 23.33 10.74 18 t(31) = -0.68, p = 0.50

Dieting days in past 3 months 64.29 42.19 14 78.11 29.52 18 t(22.31) = -1.04, p = 0.28

Dieting days in past 6 months 128.57 84.38 14 147.22 59.59 18 t(22.46) = -0.70, p = 0.49

Binge episodes in past 28 days 14.36 16.89 14 12.74 21.15 19 t(31) = 0.22, p = 0.89

Binge episodes in past 3 months 40.00 49.27 14 45.21 70.98 19 t(31) = -0.24, p = 0.82

Binge episodes in past 6 months 80.14 95.90 14 82.68 129.7 19 t(31) = -0.06, p = 0.95 9.3

Binge days in past 28 days 9.33 11.00 12 10.31 11.77 13 t(23) = -0.21, p = 0.83

Binge days in past 3 months 31.50 33.04 12 30.77 30.94 13 t(23) = 0.06, p = 0.96

Binge days in past 6 months 66.33 69.58 12 62.46 59.88 13 t(23) = 0.15, p = 0.88

Vomit days in past 28 days 16.57 26.25 14 12.00 15.57 20 t(32) = 0.64, p = 0.53

Vomit days in past 3 months 45.71 63.13 14 36.90 42.60 20 t(21.14) = 0.45, p = 0.63

Vomit days in past 6 months 91.86 137.94 14 58.21 76.21 19 t(18.82) = 0.82, p = 0.42

Laxative days in past 28 days 2.00 5.44 14 0.40 1.79 20 t(14.99) = 1.06, p = 0.30

Laxative days in past 3 months 3.64 9.94 14 1.20 4.51 20 t(32) = 0.97, p = 0.34

• No significant differences were found between the non-completers and completers of the programme on eating disorder symptoms.

• Although not significant those who completed five or more sessions of New Directions, reported fewer binge episodes, vomit days, laxative days, and diet pill days than those who completed less than five sessions of New Directions.

Conclusion & Recommendations/Next Steps

• a) Discussion and interpretation of findings• The expected outcomes of the programme were increased self-esteem, reduction in

eating disorder symptoms, depression, and normalized eating behaviour. Interpretation of the findings is limited by a small sample size and lack of follow-up data.

• Overall, the participants who completed New Directions reported a general reduction in eating disorder symptoms.

• The high incidence of reported self-harm, abuse and suicidality implies that posttraumatic stress disorder and other psychiatric conditions are present and possibly having an affect on the completion of the New Directions programme.

• The seemingly high incidence of reported self-harm, abuse and suicidality, points to the need for assessment of posttraumatic stress disorder and other psychiatric conditions.

• This may explain why approximately 41 percent (N=14/34) of the participants completed less than half of New Directions programme.

• Overall, these characteristics may also explain why the increase in self-esteem, lowered depression scores and reduction in eating disorder symptoms, were less than expected (Berkman, Lohr, & Bulik, 2007).

• b) Conclusion reflects the findings• BANA collects a considerable amount of information on

each client. • Each variable was collected at pre and post treatment,

utilizing a standardized structured interview and administration of psychometric tests.

• The total number of adolescent client files for the calendar years from 2002 – 2006 was 98.

• Thirty-eight clients attended the New Directions programme.

• The programme evaluation indicated that psycho-education slightly reduced eating disorder symptoms, but did not lead to a state of recovery from the eating disorder for most of the participants who completed New Directions.

• A limitation of this conclusion is the sample size and lack of follow-up data.

• A system for consistent data entry and analysis by trained and skilled data entry personnel was developed and put into place.

• A tracking system for client data over the course of the New Directions programme through pre- post and follow-up assessments was created and put into practice.

• BANA’s capacity to conduct outcome research, and collaborate with other outpatient clinics is also greatly enhanced.

• The agency’s capacity to successfully apply for research grants is also increased.

• Agency Benchmarks were identified around drop out rate, the need for assessing readiness to change, posttraumatic stress, and the need for following up with persons who drop out before and after the assessment.

• Participants need more information regarding the management of self-harm behaviours during the course of the treatment.

• The literature on the drop out rate for eating disorder treatment specifies 50% for persons with Bulimia and 35% for persons with Anorexia Nervosa.

• This project pointed out that the drop out rate (before the programme began) for the New Directions programme is 28%, less than what is expected for either Anorexia Nervosa or Bulimia Nervosa (Bulik, Berkman, Brownley, Sedway, & Lohr, (2007). Approximately 41 percent did not complete the programme (attended five or less sessions).

Recommendations

• There were a number of adolescents with eating disorders who presented for treatment who could have benefited from a more intensive and specific psychiatric assessment.

• An additional referral for a more comprehensive assessment could be incorporated into the treatment plan.

• It is also suggested from these results that completion rate, was made more difficult due to the increased level of psychological distress experienced by some of the participants as evidenced by the self harm and suicidalality self reports (Shapiro, Berkman, Brownley, Sedway, J, Lohr, K, Bulik, 2007).

• Readiness to change and posttraumatic stress could be a factor in the outcome analysis.

• The next steps in the multi stage programme evaluation project would be to review the pre and post assessment protocol, include a follow-up component to the assessment and measure outcomes of the New Directions Group by creating a wait list control group and compare outcomes between the two groups.

• The agency and the University of Windsor, Psychology Department have agreed to partner again to continue to pursue funding from CHEO towards that end.

Lessons learned from evaluation activities

• The agency was well equipped to collect, enter and analyze the data.

• Not enough time was allotted to analyze the date, and write the report.

e) Impact of having done this evaluation on clients served, staff, and the organization as a whole

• Because the project was analyzing archival data, the impact on the clients was negligible.

• The impact on staff was important because the project allowed the opportunity to develop and appreciate the unique set of skills suitable for conducting research.

• The organization has increased its capacity for conducting research.

Next steps

• This proposal is the first step in a multi-step approach to programme evaluation.

• Step 1 provided a snapshot of the New Directions programme.

• It identified demographic information, examined clients’ level of functioning pre and post treatment group and generated an understanding of the drop out rate for New Directions.

• The outcomes of the programme i.e. increased self-esteem, reduction in eating disorder symptoms, depression, body dissatisfaction, and normalized eating behaviour were less than expected.

• The second step will be to expand on the methodology by including a waitlist control group, which will be compared to the treatment group pre and post evaluation.

• The third step in the programme evaluation approach will be to include follow-up assessment approximately three months after the intervention is complete in order to examine long-term effectiveness (e.g., identify rates of recovery, relapse, and remission).

• Presentations and a possible publication about the programme and research findings will be developed. Study findings will also be available on the BANA website, University of Windsor REB website, and CHEO website.

• We will also deliver presentations at professional meetings and conventions, and publish findings in newsletters, and professional journals.

Knowledge Exchange Plan• Overview of knowledge exchange activities relating to this programme/project• To post the results of the project on BANA website by June 30, 2007• To publish the results of the project in the BANA newsletter by June 30, 2007• To submit an article suitable for publication in a professional journal by September

30, 2007• Post the results of the project on the CHEO website by September 30/07;• Post the results on the University of Windsor Research Ethics Board website• Submit applications for poster presentations to Ontario Association of Social Workers

Annual Conference in 2008; Association of Psychologists Annual Conference in 2008; Academy of Eating Disorders Annual Conference in 2008.

• Knowledge exchange activities accomplished to date• Report to University of Windsor Research Ethics Board, April 6, 2007.• Report to BANA board of directors, April 30, 2007;• Application to Donner Foundation for Excellence Award in Research, May 15, 2007;

• Further plans regarding knowledge exchange activities

• To generate an article and a report suitable for publication in a professional journal.

• To identify the areas for more research on programme evaluation.

• To make recommendations to the Ontario Network of Eating Disorder Service Providers regarding treatment outcomes and programme evaluation.

• To publish the results on BANA website, University of Windsor REB website, and Centre of Excellence for Child and Youth Mental Health Services of CHEO website.

Acknowledgments

• Provincial Centre of Excellence for Child and Youth Mental Health Service of CHEO;

• University of Windsor Department of Psychology;

• Bulimia Anorexia Nervosa Association

End

Thank you!