Evidence Review: An evaluation of the impact

of adventure learning and leadership

programmes on young people’s self-esteem,

self-confidence and resilience

A Pro Bono Economics Report

Eugenia Caizzi, Matthew Parish, Samuel Mackley,

Nidhi Arun, Dylan West, Opeyemi Otunuga

August 2018

Contents Executive Summary ................................................................................................................................. 2

Key Findings ......................................................................................................................................... 2

1. Introduction ......................................................................................................................................... 3

Methodology ....................................................................................................................................... 3

2. Programme Differences ...................................................................................................................... 4

3. Characteristics of Participants ............................................................................................................. 6

4. Impact on targeted outcomes ............................................................................................................. 9

5. Methodology Assessment: Data Collection, Measurement and Results ........................................... 12

6. Conclusion ......................................................................................................................................... 16

Annex 1: Research log ........................................................................................................................... 17

Bibliography .......................................................................................................................................... 20

Executive Summary The Dulverton Trust funds a number of charities that offer adventure learning and leadership

programmes for young people to improve behavioural and emotional wellbeing. Pro Bono Economics

are partnering with The Dulverton Trust to create a better understanding of the impact and value of

this funding.

This report provides an overview of the existing evidence base relating to whether such adventure

learning and leadership programmes have a positive impact on young people’s self-esteem, self-

confidence and resilience.

Key Findings • There is a range of empirical evidence that attributes improvements in social and emotional

skills to participation in adventure learning programmes. Furthermore, it is widely recognised

that enhancing these skills results in improved development and wellbeing of young people1.

• The term “adventure learning” covers a wide range of programmes that vary substantially in

their structure, length and type of activity. Some common aspects amongst programmes that

are proven to be effective include: having a variety of activities (as opposed to a single one) and

the involvement of participants as part of a wider group.

• Arguments are made in the literature for why outdoor activities are likely to affect participants

differently depending on characteristics such as age, gender and risk factors related to poor life

outcomes. However, although there is evidence that interventions have led to positive results

for a range of demographic groups, there is insufficient robust empirical evidence to

differentiate conclusions between them and drive programme design.

• There is no consensus as to which of the three targeted outcomes (self-esteem, self-confidence

and resilience) are most significantly impacted by adventure learning programmes. However,

there is significant evidence that such programmes provide improvements in young people’s

wellbeing, including self-esteem, self-confidence and resilience2.

• There are methodological limitations and disparities in measurement techniques that are well

recognised in the existing evidence base. These limitations do not allow definitive conclusions

to be made for many of the issues explored in this report as the results between different

studies become difficult to compare.

• Further research would be strengthened by the development of a clearly defined theory of

change for these types of programmes as well as the adoption of consistent, externally

validated measures of the psychological traits of interest and greater consideration of how to

match participants against suitable control groups.

1 OECD (2018) 2 CUREE (2012)

1. Introduction The Dulverton Trust is a grant-making charity that supports UK charities and CIOs (Charitable

Incorporated Organisations) tackling a range of social issues. They understand the value of

strengthening young people’s wellbeing and in their 2016-2017 Annual Report recognised that they

award most grants (38.67%) under the theme of “youth opportunities”, where they detected the

biggest increase in need following cuts in public funding. Through this, they support a wide range of

activities including adventure training and character building, employability skills and opportunities,

youth social action, improving educational attainment, mentoring and coaching and more.

Pro Bono Economics have partnered with The Dulverton Trust to provide a coherent picture of the

effectiveness of adventure learning and leadership programmes for young people. This report reviews

existing literature to assess the impact that such programmes have on young people in relation to three

psychological traits, self-esteem, self-confidence and resilience, focusing on the importance of

developing young people’s wellbeing. We identify and explore emerging themes that appear to affect

the extent of the effectiveness of adventure learning and leadership programmes in the existing

evidence base:

• Section 2 reviews evidence by different types of adventure learning and leadership

programmes.

• Section 3 reviews the evidence by the different characteristics of participants of adventure

learning and leadership programmes.

• Section 4 evaluates how the three psychological traits are affected by adventure learning and

leadership programmes.

• Section 5 discusses the data collection and research methodologies used in the evidence base.

Finally, section 6 of the report provides some concluding comments, including where evidence needs

to be built, to identify general lessons on data collection and attempts to conclude what has been

identified to be most successful and what least, in terms of intervention.

Methodology The review team searched Google, Google Scholar, EBSCO’s Discovery Service and Taylor & Francis

Online databases utilising a combination of the following keywords to identify relevant studies:

adventure learning programmes, outdoor learning, self-esteem, self-confidence, resilience, young

people.

Papers were assessed for relevance using the following factors:

• Post 2015 studies and systematic reviews were prioritised to provide a more recent update to

the comprehensive study completed by Fiennes et al.3

• Those focused on participants younger than 25 were selected to align with the United Nations

definition of youth unemployment.

• UK studies were prioritised, although the scope was broadened to include other countries such

as New Zealand, Australia and USA where UK-based evidence was limited.

3 Fiennes et al. (2015)

2. Programme Differences

There is no universal definition of adventure learning and leadership programmes. This paper defines

them as “activities which offer challenging or new experiences, often due to being unconventional and

predominantly outdoor”.

This section of the paper explains how these programmes can differ from one another and how they

can affect young people’s self-esteem, self-confidence and resilience. When reviewing the literature, it

is clear that adventure learning and leadership programmes differ for two main reasons. Firstly, the

structure, type or nature of activity; and secondly the length of the activity, with length referring to the

duration of the programme.

Our definition therefore captures a wide range of programmes such as: outdoor education and learning

programmes, residential or day trips to outdoor activity centres, field studies and field work, and

community projects. These programmes differ substantially from one another, which ultimately makes

comparisons across the different programmes extremely difficult.

Each of these types of programme can be further divided into multiple activities. For example, Becker4

studies the effects of one type of adventure learning, outdoor education, on students’ learning, social

and health dimensions, and states that “an all-encompassing definition of outdoor education is scarcely

possible due to different meanings, understandings and practices within various research areas,

countries and cultures”.

There is evidence linking different types of adventure learning programmes to positive outcomes for

self-confidence, self-esteem and resilience. For example, two programmes that lead to increased self-

esteem consisted of participants aged 15 to 19 completing sailing training during a ten-day voyage5.

The programme was conducted on sailing vessels and challenged the participants mentally and

physically through challenges such as living in confined spaces, cooperating with strangers, climbing up

tall masts and completing on-board duties regardless of tiredness, sea sickness or adverse weather.

4 Becker (2017) 5 Scarf et al. (2017)

Key findings:

• There is no universal definition of “adventure learning” with programmes differing by type,

structure or nature of activity and length of activity.

• There is some evidence linking adventure learning programmes to positive effects on

young people’s self-confidence, self-esteem and resilience. However, the evidence around

effectiveness of the programmes is thinly spread and varies significantly.

• Some common aspects amongst programmes that are proven to be effective include

having a variety of activities (as opposed to a single one) and the involvement of

participants as part of a wider group.

• Nevertheless, the variation in programme types as well as the thin spread of literature

hinders comparisons between programmes, meaning definitive conclusions cannot be

drawn on what type of programme is most effective.

Another study6 analyses multiple adventure learning programmes and summarises that a seven-day

rock climbing programme ultimately promoted resilient outcomes for the 4-18 year old participants.

However, to draw more specific conclusions from the literature is especially challenging due to a thin

spread of studies, particularly on the effects of adventure learning or leadership programmes on young

people’s self-esteem, self-confidence and resilience. Even where there are studies covering similar

activities there can be differing conclusions. For example, one particular study7 looks into the effects of

a one-year school gardening project leading to increased self-esteem in primary and secondary school

students. The gardening programme took place over the course of one year with an average of four

hours each week. Another intervention8 again looks into the effects of a one-year gardening

programme on primary school students, also with an average of four hours per week. Whilst being

similar in approach, this intervention lead to growth in self-confidence rather than self-esteem.

Due to these limitations, conclusions around which type of programmes are most effective is not

possible with the literature reviewed. Whilst individual studies may conclude that programmes have

beneficial outcomes for young people’s self-esteem, self-confidence and resilience, comparisons

between these programmes are hindered by the inconsistency of programme types and structures,

along with the differences in programme length.

Despite this, there are common aspects between programmes which emerge throughout the literature.

For example, Gonzalez9 states that increased resilience and self-esteem occurs through a wide range

of adventure-based activities. The paper also concludes that having a range of activities, and the

increased length of activities may lead to these programmes being more effective.

Another common aspect contributing to the effectiveness programmes could be the grouping of

participants in a programme. Scarf et al10 states;

“Study 1 revealed that participants who completed the voyage showed an increase

in self-esteem from the first to the last day of the voyage. Partial correlation revealed

that group belonging made a unique contribution to this change. Study 2 replicated

Study 1 and, further, demonstrated that the relationship between group belonging

and self-esteem was not a function of self-efficacy or group esteem.”

The study attributes factors such as group relationships, group norms, group efficacy and sense of

community for the success of these outcomes.

To summarise, whilst not being able to conclude on which adventure learning or leadership

programmes work best for benefiting young people’s self-confidence, self-esteem or resilience, there

may be common aspects of programmes which increase their effectiveness including having a variety

of activities (as opposed to a single one) and/or the involvement of participants as part of a wider group.

6 Gonzales (2014) 7 Bowker & Tearle (2007) cited in Becker 92017) 8 Sharpe (2014) 9 Gonzalez (2014) 10 Scarf et al. (2017)

3. Characteristics of Participants

The underlying motive of adventure learning programmes is to put students in an environment where

they may encounter problems different to their daily lives. Such exercises test their natural disposition

and help develop their character. However, those belonging to disparate groups gain different things

from the same experience11. It is therefore crucial to consider varying outcomes for different groups

while designing outdoor learning programmes. Research in this area is thinly spread and studies are

narrowly focussed. For example, most papers are based on residential programmes for 11-14 year olds.

There is very little insight into effects on youth of other age groups, disabled or post-trauma. Fewer UK

studies were found on the three impact areas this literature review prioritises (self-esteem, self-

confidence and resilience), which allows us to make scant definitive conclusions.

Age

Evidence from a range of studies suggest that adventure programmes affect different age groups

differently, however there is lack of consensus on scale or even direction of these effects12.

Behavioural issues among 2-11 year olds that disrupt learning might be addressed better by matching

learning preferences in an environment that allows more flexible pedagogies13. Additionally, it has been

identified that positive outdoor learning emerged as a consequence of mixed age grouping during early

childhood years14. Despite teacher concerns regarding safety, the mixed age approach fostered

empathy and responsibility in older children as they mentored their juniors, while younger children

engaged in risk-taking and developed resilience. Secondary school students also have the ability to

develop leadership skills in an outdoor environment by being encouraged to make decisions, build

relationships, self-organise and be self-aware15.

A meta-analysis of self-esteem studies, mostly conducted in Western industrialised nations found that

women’s self-esteem is significantly lower than men’s, with a pronounced progressive drop seen

between 12 to 17 years of age16. Diminished confidence and self-worth among women have been linked

11 Overholt & Ewert (2015) 12 Barry et al. (2017) 13 Waite (2010) 14 Rouse (2015) 15 Karagianni and Montgomery (2018) 16 Clay et al. (2005)

Key findings:

• Arguments are made in the literature for why outdoor activities will affect participants

differently depending on characteristics such as age, gender and risk factors related to

poor life outcomes.

• This means that programmes should ideally be designed according to specific target groups

to achieve successful results. For example, different genders may develop resilience via

different types of activity.

• However, although there is evidence that interventions have led to positive results for a

range of demographic groups, there is insufficient robust empirical evidence to drive

programme design.

to body dissatisfaction during adolescence. Several reports have confirmed that adventure experiences

have a positive effect on teenage girls’ perceptions of themselves. A paper that studied the effects a

coeducational course had on undergraduate women between 20-24 years of age found that they begin

to develop appreciation for their bodies and value for its capabilities. Hence, outdoor programmes that

focus on promoting the concept of ‘self’ among adolescents have knock-on effects on the self-esteem

and self-confidence of its participants17.

It is important to flag that the conclusions drawn thus far cannot be generalised and must only be used

as preliminary analysis. There is a need for further research on the themes highlighted and potential

replication of studies with a larger participant pool.

Gender

Evidence shows that participation in adventure learning programmes can increase resilience. However,

it was found that the extent to which programmatic elements enhance resilience differs by gender.

Overholt and Ewert18 use a two-phase approach to evaluate this. Resilience scores for treatment and

control groups were first compared and questions raised were subsequently explored in an interview

process. Both males and females cited stressful events, uncertainty, physical difficulty and peer

relationships as contributors to resilience development and self-confidence. However, girls were

especially concerned with the physical challenges of expedition, whereas boys emphasised relational

difficulties, such as the ongoing need to debrief and discuss interpersonal issues with their group.

People who are gender-typed will suppress behaviours that they perceive as violating gender standards,

thus limiting their behavioural repertoire in given situations. This dominant discourse may be

challenged by opportunities for females to demonstrate physical strength and technical competency

and for males to express emotion and communicate with fellow group members. Participants who were

able to demonstrate non-gender-typed qualities perceived themselves to be more resilient after the

programme. However, because participation in an outdoor adventure course may be considered

outside the norm of typical feminine gender roles, higher resilience among female participants cannot

be directly attributed to the adventure learning programme itself. Additionally, it has been found that

older girls participate in outdoor activities more frequently than their younger counterparts, which

supports the possibility of a biased participant pool19. While the strength of this conclusion is that it is

based on multiple layers of assessment that value qualitative methods, it is also based on a small sample

size with self-reported differences.

Pre-adolescent girls are usually confident and active in outdoor environments however, as they reach

mid-teenage years, their participation often declines. The results of one20 suggest that several

constraints affect this trend especially lack of opportunity, peer relationships, self-concept and access

problems. Evidence accumulated in this study strongly suggests that outdoor programming builds skills

and confidence in girls, thereby making the outdoor environment less intimidating. Adventure learning

programmes should thus provide girls with an opportunity to learn outdoor skills, include unstructured

time to form positive peer relationships, provide female role models and encourage girls to take up

leadership roles among themselves.

There is a diverse range of opinions with regards to whether single-gender or co-educational adventure

activities are more effective in self-esteem and resilience development. Some argue that co-

educational outdoor programmes are better as they encourage resilience through cross-gender

17 Hovey et al. (2016) 18 Overholt & Ewert (2015) 19 Tsikalas et al. (2015) 20 Culp (1998)

learning21. While on the other hand, others find participation in outdoor programming in girl scouts to

be causally linked to self-esteem22. One study concerning adolescent girls has supporting evidence for

both co-educational and single gender programmes23. Co-educational programmes provide the

opportunity for social mingling and athletic competition while single gender programmes give girls a

chance to learn skills that push them outside of their comfort zone. On balance, the paper supports

single gender programmes more strongly.

‘At-Risk’ Youths

A systematic review24 identified 15 studies reporting the effect of physical activity programmes on the

social and emotional well-being of ‘at-risk’ youths. The activities examined were one of three types:

Outdoor adventure, sport and skill-based or physical fitness programmes. Many of the interventions

resulted in significant positive effects displaying the potential to improve social and emotional well-

being in ‘at-risk’ youths. However, the risk of bias was high in all included studies and more rigorous

trials would be needed to evaluate the effectiveness of these programmes. It is also difficult to identify

the individual effects of the three different types of outdoor programmes.

Wilson and Burnor25 studied the association between a compulsory adventure learning programme for

‘at-risk’ youths and their resilience development. The semester-long programme used adventure

activities as the primary education medium and was an urban continuation of school for students who

had previously dropped out, been expelled or were referred for behavioural or truancy issues. The

results suggest that participants perceived changes in their ability to speak out, form connections with

adults at school and form positive relationships with peers. All three of these are related to protective

factors that could help youths be more resilient against the risk factors that had previously led to their

academic failures. The design of this study does not allow for the determination of causal links. The

methods are qualitative and cannot isolate the influence of the challenge programme apprenticeship

versus other aspects of the adventure education semester and other circumstances in the youths’ lives.

There is much scope for future studies to discern specific types of outdoor education activities that

explicitly impact different target groups. Another area of research worth exploring would be the effect

on students from different socio-economic backgrounds and races. Despite being an area of significant

impact, very few UK-based papers were found on these.

21 Overholt and Ewert (2015) 22 Tsikalas et al. (2015) 23 Culp (1998) 24 Lubans et al, (2011) 25 Wilson & Burnor (2011)

4. Impact on targeted outcomes

This section of the report aims to explore the impact of outdoor education programmes on the three

targeted outcomes: self-esteem, self-confidence and resilience. It focuses on the importance of

enhancing youth’s emotional, social and behavioural skills, aiming to evaluate whether the evidence

base supports the research question of this report. Furthermore, it explores the indirect effects

through which this can be achieved, to attempt a final conclusion on whether adventure learning

programmes drive improved self-esteem, self-confidence and resilience.

It is widely recognised in a vast range of existing empirical evidence that enhancing social and emotional

skills, together with cognitive skills, is proven to result in improvements of the development and

wellbeing of young people. Evidence from the Organization for Economic Cooperation and

Development26 shows that it is internationally acknowledged that such skills are important drivers of

young people’s lifetime success, as well as more generally of important life outcomes, such as better

mental health and wellbeing, maximisation of participation in education, training and employment and

reduction in social and behavioural inequities. Deficits in such a critical stage of personal development

may push young people towards a trajectory whereby they are more likely to experience unwanted

social and behavioural problems, as well as detrimental effects on important academic, work and life

outcomes27.

Self-esteem

Self-esteem is defined as positive or negative evaluations of the self28. Scarf et al.29 conclude that,

following a review of a meta-analysis30, comprising 96 studies, self-esteem was the most consistent

benefit of outdoor adventure learning programmes, which they argue to have a greater impact when

compared to classroom-based approaches. In particular, they explore the indirect effects that mediate

the benefits to self-esteem. In their sail-training programme intervention, for instance, participants are

put into groups and they believe that the increased sense of belonging (or group esteem) in turn

26 OECD (2018) 27 See for example: DuBois, Burk-Braxton, & Tevendale (2002), Orth, Robins, Widaman, & Conger (2014) or Trzesniewski et al. (2006) cited in Scarf et al. (2017). 28 Smith & Mackie (2007) 29 Scarf et al. (2017) 30 Hattie et al. (1997)

Key findings:

• There is no consensus as to which of the three targeted outcomes (self-esteem, self-

confidence and resilience) are most significantly impacted by adventure learning

programmes.

• However, there is significant evidence that they provide improvements in young people’s

wellbeing, of which self-esteem, self-confidence and resilience are part of.

• It is also evident that this is achieved through enhancements in other character

development-type outcomes.

• Further research is required specifically targeting such outcomes in order to draw stronger

conclusions.

enhances young participants’ self-esteem. Similarly, Hovey et al.31 explore the relationship between

adventure learning programmes and body image in females as an indirect effect. In fact, they argue

that greater valuation of a young woman’s body and its capabilities, when being challenged physically,

drives higher self-esteem and a sense of empowerment.

Self-confidence

The focus of self-esteem is on own value and worth, as opposed to self-confidence which is more

specifically defined as trust in one’s ability to achieve some goal or activity32. Evidence from Centre for

the Use of Research and Evidence in Education33 identified self-confidence as one of the most

frequently reported student outcomes following their investigated outdoor learning programme. Self-

confidence, however, was associated again with other factors such as overcoming fears and worries

about specific activities, or simply about being apprehensive of going away from home. This study

identifies that offering different levels of participation, for example from surfing to just paddling, fosters

a greater sense of self-confidence in terms of being able to successfully complete an activity.

Furthermore, it also pushes young people to set themselves more difficult goals, and consequently

achieving them enhances their confidence further.

Resilience

Although defined in a number of ways, resilience, is an individual’s ability to successfully cope with

adversity34. It is also widely defined as the quality of being able to return quickly to a previous good

condition after resisting stressful situations. Ewert and Yoshino35 recognise that there are several

studies they reviewed36 that do not recognise any impact on levels of resilience following outdoor

programmes. However, the result from their own study suggest that their examined intervention

(adventure-based expedition) was effective in enhancing levels of resilience. Furthermore, they identify

six specific components associated with the adventure education that facilitate the improvement of

levels of resilience:

• perseverance,

• self-awareness,

• social support,

• confidence,

• responsibility to others,

• achievement

This is somewhat confirmed by CUREE37 as it reports children emphasising on perseverance and “not

giving up”, when asked if they felt more resilient after the adventure programme. Lastly, Richardson’s38

resiliency model posits that individuals begin with a state of biopsychospiritual homeostasis — in other

words, any state where an individual has adapted to his or her situation in life. From here, a disruption

is experienced, which is followed by a phase of reintegration. Most research papers in the area view

self-confidence and self-esteem to be a consequence of resilience developed in an unfamiliar situation.

31 Hovey et al. (2016) 32 Judge et al. (2002) 33 CUREE (2012) 34 Pecillo, Malgorzata (2016) 35 Ewert & Yoshino (2016) 36 For example Skehill (2001) 37 CUREE (2012) 38 Richardson (2002) cited by Overholt & Ewert (2015)

However, there is a major gap in literature documenting the process of reintegration for different

character groups post an adventure learning intervention.

Combining existing literature and the evidence base in this research area, there is no consensus as to

which of the three targeted outcomes (self-esteem, self-confidence and resilience) are most

significantly impacted by such programmes. It also is evident that the choice of outcomes measured is

thinly spread. Fiennes et al.39 outline that the outcomes measured are mainly ‘character development-

type’ (examples include teamwork, communication skills etc.), as opposed to educational ones.

However, there is little consistency in the methods of analysis of such outcomes and in measuring the

extent to which they are affected. As a consequence, this poses substantial limitations for the purpose

of comparison, as well as when attempting to evaluate the impact on targeted and specific

psychological outcomes.

However, a general conclusion can be drawn: there is substantial evidence across a variety of studies

that suggests that adventure learning programmes offer consistent positive effects on young people’s

wellbeing, under which the three traits analysed in this report fall40. It is also clear that all the studies

identify that this is often achieved indirectly – the end-result of increased self-esteem, self-confidence

and resilience is achieved through strengthening of other traits such as group belonging, perseverance

or self-awareness.

Finally, this conclusion supports the scope for a more comprehensive and consistent evidence base and

evaluation of the development of specifically targeted social, psychological and emotional skills for

young people, as a result of an adventure learning programme. More high-quality assessments of the

character-development outcomes, with further exploration of key indirect effects are needed to

attempt to find a definite answer to the research question and to ensure the full potential of the

programmes can be achieved. Perhaps introducing a standardised unit of measurements of such

outcomes could lead to more reliable and valid conclusion on what traits benefit the most.

39 Fiennes et al. (2015) 40 See, for example, CUREE (2012)

5. Methodology Assessment: Data Collection, Measurement and

Results

When reviewing the existing evidence base of adventure learning and leadership programmes on young

people’s self-esteem, self-confidence and resilience, a number of limitations surrounding data

collection and measurement were clear to see. The recurrence of these limitations throughout the

studies and systematic reviews result in a lack of robustness and a largely incomparable evidence base.

Whilst this can be seen in a negative light, it is important to acknowledge these limitations in order to

move forward in improving the evidence base.

Some of these limitations in the evidence base could be made less limiting by improving methodology,

others are inherent in the type of data required to measure the effectiveness of adventure learning and

leadership programmes on young people’s self-confidence, self-esteem and resilience. Nevertheless,

both remain useful to be aware of.

Becker et al.41 highlight a number of methodological limitations of the qualitative studies looked at in

their systematic review. They note a number of reasons for giving the qualitative studies they looked at

low ratings including;

“Insufficient information about the influence of the researcher on the observed or

interviewed participants, and vice-versa; … and an inappropriate connection

between the conclusions and the analyses”

Despite Becker’s review including a wide range of studies in terms of country of origin, type of

programme and outcome measured, the limitations look to be consistent throughout the field. In

addition to the aforementioned limitations, others included selection bias, over-reliance on self-report

measures, small sample sizes, inconsistency in measurement, absence of a Theory of Change and failure

to measure long term impacts.

Selection bias

There was often poor level of similarity between control and treatment groups in studies, which

subjects results to selection bias. This is likely to be characteristic of programmes that require

volunteers to serve as a treatment group, as one could reasonably expect the treatment group to differ

from the control in personality or attitude. Differences in attitude or personality might be captured in

measures of self-esteem, self-confidence and resilience. For this reason, observed positive effects

41 Becker et al. (2017)

Key Findings:

• There is inconsistency in the way data is measured in the existing evidence base.

• This inconsistency results in an inability to compare studies and therefore draw meaningful

conclusions.

• The existing evidence base also suffers from a number of recurring limitations in data

collection, measurement technique and methodology resulting in a lack of robustness.

• Ultimately, the current evidence base is not strong enough to base recommendations for

favourable adventure learning and leadership programmes – further research is required.

might not be as a result of the adventure learning and leadership programme. Instead, it could be

caused by differences between treatment and control groups.

A lot of the studies looked at are aware of this limitation42. Showing an awareness to the problem, but

with no attempt or ability to change the sampling methodology, possibly caused by a lack of resource

or another viable solution. An unequal distribution of gender between control and treatment groups

could also result in selection bias43. With preliminary analysis in that supports the idea that males and

females are affected differently by adventure learning programmes as explored previously in this

review44.

Some studies do address this issue and mention the matching process in their method45, but the

evidence is thinly spread. Studies should seek to match control and treatment groups as similar as

reasonably possible if they hope to report an accurate impact of the programme in question.

Over-reliance on self-report measures

Methods of measuring the effect of adventure learning on self-esteem, self-confidence and resilience

often place an over-reliance on self-report measures, namely a questionnaire given to participants

before and after the programme46. Whilst self-report measures are fundamental in understanding

psychological changes i.e. effects on self-esteem, self-confidence and resilience, there are many

limitations to be aware of – some of which can be made less limiting. However, many studies over-rely

on self-report measures and do not detail if/how they have attempted to make self-report less limiting,

and therefore make it hard to assess robustness.

Most of the limitations centre on participants having to assess themselves, which creates subjectivity.

Barry et al.47 touch on this, stating “[self-report measures] do not rely on standardized outcome

measures and are not subject to external verification”. Similar conclusions are drawn by Overholt and

Ewert’s48 study which states “when using a pre–post survey instrument researchers must take into

account the ways in which participants assess themselves”. Hoskin49 outlines some other limitations of

self-report measures, such as people interpreting questions differently and interpreting the scales on

which they answer differently too. He also mentions the presence of response bias, meaning some

participants give consistently positively or negatively bias responses throughout their questionnaire. As

previously mentioned self-report measures are fundamental in understanding the effect of adventure

learning and leadership programmes on young people’s self-esteem, self-confidence and resilience.

Nonetheless, it is important to consider whether the adoption of standardised, validated measures of

different psychological traits would improve the robustness of the evidence base.

Small sample sizes

A number of the studies reviewed relied on small samples; often fewer than one hundred individuals

are tracked in treatment and control groups collectively. In Hovey’s50 study only 28 students took part

42 See for example Scarf et al. (2017) 43 Scarf et al. (2017) 44 See for example Overholt and Ewert (2014) 45 Ewert and Yoshino (2011) 46 See for example Overholt and Ewert (2014), Scarf et al. (2017) and Furness et al. (2017) 47 Barry et al. (2017) 48 Overholt and Ewert (2014) 49 Hoskin (2012) 50 Hovey (2016)

in the programme. Whether for cost reasons or operational reasons it seems that studies often lack the

resource to be able to collect a large sample.

Small sample sizes can undermine the findings and conclusions from research. As a large limitation it

can decrease the statistical significance of the results and the level of confidence at which you can claim

the effect on the sample is an accurate estimate of what the effect would be on the population. This is

recognised in Overholt and Ewert’s study which demotes itself to “preliminary analysis”51 for this

reason.

The problem of small sample size can also be exacerbated by attrition. Participants have a propensity

to drop out of adventure learning programmes, as those dissatisfied with the experience have no

obligation to partake in programmes for the entire duration. This can not only introduce bias into the

results but would reduce an already meagre sample size. An example of this can be found in Ewert and

Yoshino’s study, from an initial sample of 85, “after screening the data for incomplete questionnaires,

out-of-range scores, or response sets, two unmatched responses were removed, resulting in a usable

sample size of 55’52.

Inconsistency in measurement

Due to the qualitative nature of the research, there are a lot of differences in the way that data is

measured. This inconsistency makes it very difficult to compare studies’ results with one another. Barry

et al53 pick up on this, reporting “due to the heterogeneity of the evaluation studies identified, a

quantitative synthesis of the findings from the intervention studies reviewed was not possible”. This

inconsistency in data measurement means that even in rare situations where studies have had the exact

same focus, the data is incomparable due to difference in measuring technique.

Absence of a Theory of Change

Outlining a theory of change is well accepted best practice in impact evaluation and has been

recognised in the specific context of adventure learning by Fiennes et al.54 It appears from the existing

evidence base for the effects of adventure learning that logic models or theories of change are

unutilized and rarely mentioned explicitly, if at all. It may be possible that a logic model is used but is

simply not mentioned in published reports, however, the lack of such a framework may be a factor

contributing to the weakness of the evidence base and the lack of robustness in the conclusions/results

drawn from studies in this area.

Failure to measure long term impacts

The studies reviewed in this paper often only considered the effects measured from adventure learning

programmes in the short term. Given that impacts from adventure learning may not be sustained in

the long run, a report only tracking short run changes to self-esteem, self-confidence and resilience

would fail to reflect what might be a depreciating effect. This supports the conclusion of Lubans et al.55:

“as none of the studies included long-term follow-ups, it remains untested whether the benefits

associated with participation in physical activity programmes are sustained once youth return to their

daily routines”.

51 Overholt and Ewert (2014) 52 Ewert and Yoshino (2015) 53 Barry et al. (2017) 54 Fiennes et al. (2015) 55 Lubans et al. (2015)

Whilst this is highlighted as a limitation, it is recognised that there are a number of difficulties in

measuring long term impacts, for example in defining a sufficient length of time after the participation

in a given programme which could be used to categorise a “long term” measurement.

Clearly, a significant number of limitations are present throughout the existing evidence base. As

previously mentioned, whilst this may seem negative, acknowledging these limitations will prove useful.

It is clear that improvements need to be made, of which most studies are aware. The main

recommendation in terms of this section is to move towards creating consistency in the way data is

measured to facilitate comparability, of which a Theory of Change may be useful to develop. In addition

to this some general lessons should be taken surrounding selection bias, over-reliance on self-report

measures and the existence of small sample sizes. A lot of our recommendations for future research

align with systematic reviews already in the existing evidence base, which further adds to a call for the

evidence base to be more robust as well as being consistent in measurement. In doing this it will allow

meaningful comparison between studies.

6. Conclusion There is substantial evidence across a variety of different studies that suggests adventure learning

programmes offer positive effects on young people’s wellbeing. However, a significant number of

limitations in methodology are present throughout this evidence base. Consequently, it is not possible

to draw a robust conclusion on what works best in terms of the effectiveness of such programmes as

an intervention on improving the three targeted outcomes: young people’s self-esteem, self-

confidence and resilience.

Inconsistency in the evidence base represents a constraint that also does not allow comparisons to be

made. In fact, different studies explore different types of programmes, of different length and with

different groups of participants. It is evident that such differences have varying impacts on the

effectiveness of the programmes. Therefore, only common characteristics of the programmes can be

identified to be fruitful.

Furthermore, the evidence base does not provide a clear consensus as to which of the three targeted

outcomes (self-esteem, self-confidence and resilience) are most significantly impacted by such

programmes. However, there is significant evidence that adventure learning programmes provide

improvements in young people’s wellbeing, of which self-esteem, self-confidence and resilience are

part of. This is achieved through indirect effects on other character development-type outcomes.

In order to draw clearer conclusions on whether adventure learning and leadership programmes

improve young people’s self-esteem, self-confidence and resilience, further research needs to be

conducted targeting the three outcomes specifically. Consistency in measurement and a method that

allows controlled outcomes to be tested against variables such as participants, type of the programme

and participants would thus improve comparability. This would provide a better understanding of what

exactly, within adventure learning and leadership programmes, works best.

Annex 1: Research log Ref. No

Title of Paper Authors Year of Publication

Relevance

1 The Existing Evidence-Base about the Effectiveness of Outdoor Learning

C. Fiennes, E. Oliver, K. Dickson, D. Escobar, A. Romans, S. Oliver

2015

2 Social Psychological Benefits of a Wilderness Adventure Program

T. Paxton, L. McAvoy

2000

3 The effects of an adventure based program: Mechanisms promoting resilient outcomes among youth in transition from childhood to adulthood

A.Gonzalez 2014

4 Evaluation of Learning Away Hypothesis 9: Resilience, self-Confidence and wellbeing

Centre for the Use of Research and Evidence in Education

2012

5 Review of Research on Outdoor Learning M. Rickinson 2004

6 The Influence of short term adventure-based experiences on levels of resilience

A.Ewert, A.Yoshino

2011

7 Gender Matters: Exploring the process of developing resilience through Outdoor Adventure

J.Overholt, A.Ewert

2015

8 Review: A systematic review of the impact of physical activity programmes on social and emotional well-being in at-risk youth

D. Lubans, R. Plotnikoff, N. Lubans

2011

9 A Review of the Evidence on the Effects of Community-based Programs on Young People’s Social and Emotional Skills Development

M. Barry, A. Clarke, S. Morreale, C. Field

2017

10 Effects of Regular Classes in Outdoor Education Settings: A Systematic Review on Students’ Learning, Social and Health Dimensions

C. Becker, G. lauterbach, S. Spengler, U. Dettweiler, F. Mess

2017

11 Predictors of Change in Body Image in Female Participants of an Outdoor Education Program

K. Hovey, J. Foland, J. Foley, M. Kniffin, J. Bailey

2016

12 Putting research into practice; or putting practice into research in sail training

H. Prince, E.Fletcher

2018

13 Satisfying psychological needs on the high seas: explaining increases self-esteem following an Adventure Education Programme

D. Scarf, S. Kafka, J. Hayhurst, K. Jang, M. Boyes, R. Thomson, J. Hunter

2017

14 Maximising Potential: The Psychological Effects of the Youth Development Programme Project K

K. Furness, M. Williams, J. Veale, D. Gardner

2017

15 The impact of outdoor youth programs on positive adolescent development: Study protocol for a controlled crossover trial

I.Williams, L. Rosec, C. Olssona, G. Patton, N. Allen

2018

16 Promoting Intrapersonal Qualities in Adolescents Evaluation of Rapport's Teen Leadership Breakthrough Program

Y. Hindes, K. Thorne, V. Schwean, A. Mckeough

2008

17 Children and Youth Services Review A. Parkhill, C. Deans, L. Chapin

2017 Moderately

18 Leadership and Adolescent Girls: A Qualitative Study of Leadership Development

M. Hoyt, C. Kennedy

2018 Moderately

19 The Impact of Leadership Programme on Self-Esteem and Self-Efficacy in School: A Randomized Controlled Trial

M. Wong, T. Lau, A. Lee

2012

20 Body image and self-esteem among adolescent girls: Testing the influence of sociocultural factors

D. Clay, L. Vignoles, H. Dittmar

2005 Moderately

21 Losing our way? The downward path for outdoor learning for children aged 2–11 years

S. Waite 2010

22 Adolescent girls and outdoor recreation: A case study examining constraints and effective programming

R. Culp 1998

23 Association between an adventure education apprenticeship and at-risk urban youths’ resilience

J. Wilson, A.Burnor

2011

24 Developing leadership skills among adolescents and young adults: a review of leadership programmes

D. Karagianni, J. Montgomery

2018

25 Mixed-age grouping in early childhood–creating the outdoor learning environment

E. Rouse 2015 Moderately

26 Girls’ Challenge Seeking: How Outdoor Exposure Can Support Girls in Taking Positive Risks

K. Tsikalas, K. Martyn

2015

27 A 10 –day developmental voyage: converging evidence from the studies showing that self-esteem may be elevated and maintained without negative outcomes

S. Kafka, J. Hunter, J. Hayhurst, M. Boyes, R. Thomson, H. Clarke, A. Grocott, M. Stringer, K. O’Brien

2012 ×

28 Bandura goes outdoors: Role of Self-efficacy in the outdoor leadership development process

D. Propst, R.Koesler

1998 ×

29 Diverse aims, challenges and opportunities for assessing outdoor learning: a critical examination of three cases from practice

S. Waite, O. Rutter, A. Fowle, A. Edward-Jones

2017 ×

30 Outdoor adventure education (OAE) in higher education: characteristics of successful university degree programmes

T. Potter, T. Socha, T. O’Connell

2012

31 Outdoor Leadership –The Last Male Domain?

N. Saunders, B.Sharp

2002

32 Independent thinkers and learners: a critical evaluation of the ‘Growing Together Schools Programme

D. Sharpe 2014

33 Gardening as a learning environment: A study of children’s perceptions and understanding of school gardens as a part of an international project

R. Bowker, P. Tearle

2007

34 The dangers of self-report R. Hoskin 2012

35 Social and Emotional Skills – Well-being, connectedness and success.

OECD 2018

36 Evaluation of Learning Away: Hypothesis 9: Resilience, self-confidence and wellbeing.

CUREE (Centre for the Use of Research and Evidence in Education)

2012

37 What is Subjective Well-Being? Understanding and Measuring Subjective Well-Being

Albuquerque, B. 2010

38 "The concept of resilience in OSH management: A review of approaches"

Pecillo, Małgorzata

2016

39 Smith, E. R.; Mackie, D. M. Social Psychology

2007

Bibliography • Albuquerque, B. (2010). What is Subjective Well-Being? Understanding and Measuring

Subjective Well-Being. Available: http://positivepsychology.org.uk/subjective-well-being/. Last

accessed 30th April 2018.

• Barry, M.M., Clarke, A.M., Morreale, S.E. and Field, C.A., 2018. A Review of the Evidence on

the Effects of Community-based Programs on Young People’s Social and Emotional Skills

Development. (Date Accessed: 22nd March 2018). Available from:

https://link.springer.com/article/10.1007/s40894-017-0055-2

• Becker, C. 2017. Effects of Regular Classes in Outdoor Education Settings: A Systematic

Review on Students’ Learning, Social and Health Dimensions. International Journal of

Environmental Research and Public Health. [Online].14(5), p.485. (Date accessed: February

2018). Available from: http://www.mdpi.com/1660-4601/14/5/485/htm

• Bowker, R; Tearle, P. 2007. Gardening as a learning environment: A study of children’s

perceptions and understanding of school gardens as a part of an international project.

Learning Environments Research. [Online]. 10(2), p.83-100. [Date accessed: February 2018].

Available from: https://link.springer.com/article/10.1007%2Fs10984-007-9025-0

• Clay, D., Vignoles, V. L., & Dittmar, H. (2005). Body image and self-esteem among adolescent

girls: Testing the influence of sociocultural factors. Journal of research on adolescence, 15(4),

451-477.

• Culp, R. H. (1998). Adolescent girls and outdoor recreation: A case study examining

constraints and effective programming. Journal of Leisure Research, 30(3), 356-379.

• CUREE (2012) Evaluation of Learning Away: Hypothesis 9: Resilience, self-confidence and

wellbeing. London: Paul Hamlyn Foundation. Available from: http://learningaway.org.uk/wp-

content/uploads/Hypothesis-9-Resilience-self-confidence-and-wellbeing-final2.pdf

• Dulverton Trust (2017): Annual Report and Accounts 2016/17 Available at:

https://www.dulverton.org/wp-content/themes/html5blank-stable/pdf/Annual-Report-2016-

17.pdf

• Ewert, A., Yoshino, A. (2011) The influence of short-term adventure-based experience on

levels of resilience, Journal of Adventure Education and Outdoor Learning, 11:1, pp. 35-50

Available at: http://dx.doi.org/10.1080/14729679.2010.532986

• Fiennes, C., Oliver, E., Dickson, K., Escobar, D., Romans, A. and Oliver, S., 2015. The existing

evidence-base about the effectiveness of outdoor learning. Institute of Outdoor Learning,

Blagrave Trust, UCL & Giving Evidence Report. (Date Accessed 19th March 2018) Available

from: https://www.outdoor-learning.org/Portals/0/IOL%20Documents/Research/outdoor-

learning-giving-evidence-revised-final-report-nov-2015-etc-v21.pdf?ver=2017-03-16-110244-

937

• Gonzalez, A. 2014. The effects of an adventure based program: Mechanisms promoting

resilient outcomes among youth in transition from childhood to adulthood. Undergraduate

Honors Theses. [Online]. 106. Date accessed: February 2018]. Available from:

https://scholar.colorado.edu/honr_theses/106/?utm_source=scholar.colorado.edu%2Fhonr_

theses%2F106&utm_medium=PDF&utm_campaign=PDFCoverPages

• Hovey, K,. Foland, J., Foley, J. T, Kniffin M., Bailey J.: (2016) Predictors of Change in Body

Image in Female Participants of an Outdoor Education Program. Journal of Outdoor

Recreation, Education and Leadership Vol 8 No 2, pp 200-208

• Judge, Timothy A.; Erez, Amir; Bono, Joyce E.; Thoresen, Carl J. (2002-09-01). "Are measures

of self-esteem, neuroticism, locus of control, and generalized self-efficacy indicators of a

common core construct?". Journal of Personality and Social Psychology. 83 (3): 693–710.

doi:10.1037/0022-3514.83.3.693. ISSN 1939-1315.

• Karagianni, D., & Jude Montgomery, A. (2018). Developing leadership skills among

adolescents and young adults: a review of leadership programmes. International Journal of

Adolescence and Youth, 23(1), 86-98.

• Kafka, S., Hunter, J.A., Hayhurst, J., Boyes, M., Thomson, R.L., Clarke, H., Grocott, A.M.,

Stringer, M. and O’Brien, K.S., 2012. A 10-day developmental voyage: Converging evidence

from three studies showing that self-esteem may be elevated and maintained without

negative outcomes. Social Psychology of Education, 15(4), pp.571-601.

• Lubans, D.R., Plotnikoff, R.C. and Lubans, N.J., 2012. A systematic review of the impact of

physical activity programmes on social and emotional well-being in at-risk youth. (Date

Accessed: 22nd March 2018). Available from:

https://onlinelibrary.wiley.com/doi/full/10.1111/j.1475-3588.2011.00623.x

• OECD. (2018). Social and Emotional Skills – Well-being, connectedness and success. Available:

http://www.oecd.org/education/school/UPDATED%20Social%20and%20Emotional%20Skills%

20-%20Well-being,%20connectedness%20and%20success.pdf%20(website).pdf Last

accessed: 30th April 2018

• Overholt, J.R. and Ewert, A., 2015. Gender matters: Exploring the process of developing

resilience through outdoor adventure. Journal of Experiential Education, 38(1), pp.41-55.

• Pecillo, Małgorzata (2016). "The concept of resilience in OSH management: A review of

approaches". International Journal of Occupational Safety and Ergonomics. 22 (2): 291–300.

doi:10.1080/10803548.2015.1126142. PMC 4867880. PMID 26652938.

• Propst, D.B. and Koesler, R.A., 1998. Bandura goes outdoors: Role of self-efficacy in the

outdoor leadership development process. Leisure Sciences, 20(4), pp.319-344.

• Rickinson, Dillon, Teamey, Morris, Young Choi, Sanders, Benefield, 2004: A review of Research

on Outdoor Learning. National Foundation for Educational Research and King’s College

London. Available at: https://www.field-studies-

council.org/media/268859/2004_a_review_of_research_on_outdoor_learning.pdf

• Rouse, E. (2015). Mixed-age grouping in early childhood–creating the outdoor learning

environment. Early Child Development and Care, 185(5), 742-751.

• Smith, E. R.; Mackie, D. M. (2007). Social Psychology (Third ed.). Hove: Psychology Press. ISBN

978-1-84169-408-5.

• Sharpe, D. 2014. Independent thinkers and learners: a critical evaluation of the ‘Growing

Together Schools Programme’. Pastoral Care in Education. [Online]. 32(3), p.197-207. [Date

accessed: February 2018]. Available from:

https://www.tandfonline.com/doi/abs/10.1080/02643944.2014.940551

• Scarf, D; Kafka, S; Hayhurst, J; Jang, K; Boyes, M; Thompson, R; Hunter, J. 2017. Satisfying

psychological needs on the high seas: explaining increases self-esteem following an

Adventure Education Programme. Journal of Adventure Education and Outdoor Learning.

[Online]. 18(2), p.165-175. [Date accessed: February 2018]. Available from:

https://www.tandfonline.com/doi/pdf/10.1080/14729679.2017.1385496?needAccess=true&

• Tsikalas, K. and Martin, K.L., 2015. Girls' Challenge Seeking: How Outdoor Exposure Can

Support Girls in Taking Positive Risks. Afterschool Matters, 21, pp.1-10.

• Waite, S. (2010). Losing our way? The downward path for outdoor learning for children aged

2–11 years. Journal of Adventure Education & Outdoor Learning, 10(2), 111-126.

• Wilson, J. D., & Burnor, A. C. (2011). Association between an adventure education

apprenticeship and at-risk urban youths’ resilience. World leisure journal, 53(4), 255-268.