Thesis DefenseThesis Defense

A Meta-Analysis of Adventure A Meta-Analysis of Adventure Therapy Program OutcomesTherapy Program Outcomes

Submitted in partial fulfillment of the degreeSubmitted in partial fulfillment of the degree

Master of Science: Kinesiology- Master of Science: Kinesiology- Outdoor EducationOutdoor Education

ByBy

NormanNorman StauntonStaunton

Committee MembersCommittee Members

• Chair: – Dr. Michael A. Gass, Ph.D.

• Members: – Dr. Deborah Sugerman, Ph.D.– Dr. Jason Bocarro, Ph.D.

• Advisor:– Mr. James Neill

MANY THANKS TO THEM ALL!

Gillis (1992)Gillis (1992)

“Someone needs to conduct a meta-analysis on [the] therapeutic aspect[s] of adventure-challenge-outdoor-wilderness that includes the criteria of clinical significance along with traditional methods of effect size”

Gillis (2000)Gillis (2000)

“To the best of my knowledge such a study has not been done.”

As of December of 2002 no such study had been completed, and it represented a significant gap in the body of AT research. This study fills that gap by conducting such a meta-analysis.

Reasons for conducting this study:Reasons for conducting this study:

• Not a duplication of previous research- it provides new information to the field

• Represents a significant contribution to the field- it fills a significant gap of needed information

• Provides documentation of collective efficacy- which is necessary for future funding, program design, research design

Question I:Question I:

• What are the effects of Adventure Therapy?– How do the effects of Adventure Therapy

compare to Adventure Education?– How do the effects of Adventure Therapy

compare to other forms of therapy?

Question II:Question II:

• What are the major correlates of Adventure Therapy program outcomes? Such as for:

–Programming (e.g. length, type, status)–Group (e.g. social support, cohesion)–Client (e.g. gender, age, diagnosis)–Outcome (e.g. clinical, affective, attitude)

DefinitionsDefinitions

• Adventure Therapy-– Is programming aimed at changing [specified]

dysfunctional behavior patterns, using adventure experiences as forms of habilitation and rehabilitation." (Priest and Gass 1997). For our purposes, this includes any adventure experience with diagnosed clients or reporting a specified therapeutic outcome.

DefinitionsDefinitions

• Meta-analysis-– A statistical technique for accumulating and

representing the research results reported in multiple studies. Meta-analyses pool the findings about a single research question from many different sources and analyze the overall effects.

DefinitionsDefinitions

• Effect Size-– Is a statistical standardized coefficient that

allows for comparisons of changes in means over time across studies. ES is calculated by the equation:

(M1-M2)/SDpooled

Practical Basis of MethodologyPractical Basis of Methodology

• Based on the previous OE meta-analyses• Cason (1993)- AE outcomes on adolescents• Hattie, Marsh, Neill & Richards (1997)- AE/ OB

outcomes for all populations • Marsh (1999)- Outcomes of camp experiences• Hans (2000)- AE effects on locus of control

Practical Basis of MethodologyPractical Basis of Methodology

This study departed from earlier OE meta-analyses by:

• Focusing specifically on AT and therapeutic outcomes

• Looks outside of OE for applicable research

• Being up to date

MethodologyMethodology

• Study Methodology is divided into four sections:

• Search Methods• Selection Criteria• Coding Methods• Statistical Methods

Search MethodsSearch Methods

• Searches were made in six categories:• Electronic Databases• Bibliographies• Top AT/OE Web Sites• Commonly Known Research• Email Listserv Requests• Existing Data

These methods returned over 2000 hits with over 600 unique articles

Selection CriteriaSelection Criteria

Studies were included based on the following criteria:

• Empirically-based and likely reporting descriptive statistics

• Adventure-based intervention with diagnosed populations

Studies meeting these criteria were actively sought by the researcher

Selection CriteriaSelection Criteria

Studies were further included based on:

• Further investigation revealed the first criteria were correct

• 75 articles passed the third criteria. 22 were unavailable, taking the effective number of studies to 53.

• Analysis during coding eliminated 36 studies. 17 studies remained.

Coding MethodologyCoding Methodology

Primary Variable Codes:Year of Study Quality of Measure

Authors Quality of Study

Institution Pre M and SD

Country 1st day M & SD

Publication Source Last day M & SD

Type of Program Follow up M & SDOutcomes Measure (M & SD calculated as needed)

Secondary Variable Codes Secondary Variable Codes HierarchyHierarchy

\PROGRAM/Type, Modality, Delivery Format, Duration, Daily Duration

\GROUP/Staff, Retention, Length of Program, Age, Gender

\INDIVIDUAL/

Age, Gender, Category, Diagnosis\OUTCOME/

Behavioral, Attitudinal, Affective, Physical, Cognitive

Statistical MethodsStatistical Methods

• Effect Sizes (ES)• ESs were calculated, then corrected to account for

their relative strength.

• Correlations• Relationship between effect sizes and primary and

secondary variables were analyzed using univariate analysis. 95% confidence intervals were used to compare significant differences from zero and between variables.

Statistical MethodsStatistical Methods

• Regression Analysis– Regression analysis was conducted until all

returned correlates of variance were significant at the p=.05 level

• File Drawer Effect– Rosenthal’s File Drawer Effect was calculated

to estimate the number of negative or non-significant effects needed to negate the findings.

ResultsResults

• 17 Studies were included

• 95 Effect sizes were calculated (91/4)

• No negative effect sizes were found

• Overall WES of .53 found

• .53 for immediate effects, .35 for follow-up

ResultsResultsID Author(s) Year ES W*ES Total

ESsTotal N Diagnosis Publication Type Program Type

10 Anderson 1995 .14 .14 12 33 Behavioral Disertation Wilderness

50 Bandoroff, & Scherer 1994 .74 .74 7 43 Behavioral Journal Wilderness

90 Castellano & Soderstrom, 1992 .49 .49 1 22 Behavioral Journal Wilderness

80 Chakravorty, et al. 1995 .36 .36 1 25 Psychological Journal ABC

540 Dahlen 1998 .72 .72 13 199 Behavioral Unpublished ABC

130 Fashimpar& Harris 1987 1.10 1.10 5 20 Psychological Journal ABC

150 Gillis, et al. 1995 .24 .24 6 134 Behavioral Unpublished Mixed

180 Herbert, J. 1998 1.19 1.19 4 36 Combined Journal Wilderness

200 Kelly & Baer 1969 .27 .27 20 60 Behavioral Journal Wilderness

390 Maizell 1988 .45 .45 4 19 Behavioral Disertation ABC

250 McPeake, et al. 1991 .64 .64 6 46 Behavioral Journal Mixed

470 Mossman 1997 .20 .20 2 12 Behavioral Disertation Mixed

600 Neill 2003 .49 .49 1 61 Behavioral Unpublished Wilderness

290 Pawlowski, et al. 1993 .41 .41 8 15 Psychological Journal Wilderness

480 Price & DeBever 1997 .46 .46 1 6 Behavioral Unpublished Wilderness

530 Russel 2002 1.11 1.12 4 144 Mixed Unpublished Wilderness

510 Witman 1987 .77 .77 1 5 Psychological Journal ABC

ResultsResults

• Univariate Analysis of Possible Predictors:

• Found significant positive results for:– Dissertations (.17); Medium Quality (.74);

High Quality (.44); ABC Programs (.72); Continuous Programs (.59); Intermittent Programs (.37); Residential Programs (.55); Single Sex Groups (.47); Coed Groups (.53); Adolescents (.49); Adults (1.11); Self Constructs (.25)

ResultsResults

• Univariate Analysis of Possible Predictors:

• All other variables were found to be significantly different from zero, but not between variables.

• Only Outpatient Programs had a non-significant ES (.19)

ResultsResults

• File Drawer Effect: 19 studies/ 648 ES would be needed to negate the findings of this study

• Regression returned Dissertations (-.28); ABC Programs (.38); Mixed Diagnoses (.74) as the three predictors significant at p=.05

• Regressed Predictors account for 35.9% of variance in ES

DiscussionDiscussion

• Sample Size: small and some sampling error likely to exist. However, Quality of Studies and File Drawer Effect suggest some strength to the sample.

• Dahlen (2002) & Russell (2000) deserve special comment. ES drops to .35 without these studies.

DiscussionDiscussionAuthor Studies ESs ES

Hattie et al (1997) 91 1728 .34

Hans (2000) 24 30 .36

Marsh (1999) 22 37 .25

Cason (1993) 43 147 .31

Bunting & Donley (2003) 10 n/a .75

Staunton (2003) 17 95 .53

Hattie (1992) Innovative Classroom Interv.

.28

Lipsey & Wilson (1993) Psych. Training .47

Smith et al (1980) Psychotherapy adults .68

Casey & Berman (1985) Psychotherapy child .71

In other words...In other words...

00.10.2

0.30.40.50.6

0.7

ES

ES Comparison

Innovative Interv.

Outdoor Education

PsychologicalTraining

AT

Psychotherapy

DiscussionDiscussion

• Why ABC Programs may be more effective:– More Developed Theory; Clinical

Associations; More Educated Practitioners; Flexibility

• Why Mixed Diagnoses may be more effective: – Holistic Nature of AT Programs; Role of

Processing and Behavior

DiscussionDiscussion

• Why Dissertations may be least effective:– Rigors of Publishing; Self-designed Measures;

Least Experienced Researchers

ConclusionsConclusions

• AT is effective (.53) and has lasting (increasing?) affects (.37).

• As a whole, the body of research is overwhelmingly positive.

• However, the overall quality of research and reporting is low

• AT compares favorably to other treatment modalities.

ConclusionsConclusions

• ABC Methodologies and Mixed Diagnoses appear to be correlates of large ES.

• Further research is needed to test the causes and correlates of AT program effects.

• Further Meta-analysis is recommended.

• AT should become a respected Tx Modality.

THE ENDTHE END

Thanks For Coming.Thanks For Coming.Any Questions?Any Questions?