out of school time (ost) stem activities impact on middle

Out of School Time (OST) STEM Activities Impact on Middle School Students’ STEM Persistence: A Convergent Mixed Methods Study

by

David Christopher Taylor, B.S., M.Ed.

A Dissertation

In

Curriculum and Instruction

Submitted to the Graduate Faculty of Texas Tech University in

Partial Fulfillment of the Requirements for

the Degree of

DOCTOR OF PHILOSOPHY

Approved

Jerry Dwyer, Ph.D. Co-Chair of Committee

Rebecca Hite, Ph.D.

Co-Chair of Committee

Warren DiBiase, Ed.D.

Mark Sheridan, Ph.D. Dean of the Graduate School

May, 2019

Texas Tech University, David Taylor, May 2019

ii

ACKNOWLEDGEMENTS

This process has helped me grow as a researcher, an educator, and a person. I

have learned so much about myself and gained a better understanding of the world.

Without my family, friends, and colleagues, I would not have been able to complete this

journey.

I want to thank my wife, Bri, for all her support, love, and understanding

throughout this process. Without her, I would have been lost. Her constant support has

provided me the strength I need when times were tough. You are my everything and I

love you!

I am truly grateful for my committee’s support, feedback, guidance, and help.

Thank you, Dr. Jerry Dwyer, Dr. Rebecca Hite, and Dr. Warren DiBiase. You all are

amazing educators!

Finally, I hope my children, David and Ryan, are proud of my work as an

educator and proud to know that I have earned my PhD. I want them to always know that

education opens doors. This lesson was taught to me by my parents, David and DeLila. I

am gratefully for everything that they have done for me and I hope I have made them

proud.


iii

TABLE OF CONTENTS

ACKNOWLEDGEMENTS ............................................................................................. ii

ABSTRACT ..................................................................................................................... vii

LIST OF TABLES ........................................................................................................... ix

LIST OF FIGURES .......................................................................................................... x

I. INTRODUCTION ......................................................................................................... 1

Need for the Study ....................................................................................................... 3

Background of the Problem ......................................................................................... 4

Growing a Global STEM Workforce .......................................................................... 5

Needs to Grow a Global STEM Workforce ...........................................................6

OST as a Strategy to Grow a Global STEM Workforce ........................................7

Problem Statement ....................................................................................................... 8

Conceptual Framework ................................................................................................ 9

Purpose of the Study .................................................................................................. 11

Research Questions .................................................................................................... 11

Overview of Research Design ................................................................................... 12

Significance of the Study ........................................................................................... 14

Audience ..............................................................................................................15

Assumptions .............................................................................................................. 15

Positionality ............................................................................................................... 16

Delimitations of the Study ......................................................................................... 16

Limitations of the Study ............................................................................................ 17

Definitions of Terms .................................................................................................. 18

II. LITERATURE REVIEW ......................................................................................... 24

Conceptual Framework .............................................................................................. 25

Motivation ............................................................................................................26

Interest .................................................................................................................31


iv

Persistence ...........................................................................................................37

21st Century Skills ...............................................................................................45

Summary .................................................................................................................... 49

II. METHODOLOGY .................................................................................................... 51

Mixed Methods Convergent Parallel Research Design ............................................. 51

Research Paradigm .................................................................................................... 56

Research Questions .................................................................................................... 57

Context of the Participants......................................................................................... 58

Data Collection .......................................................................................................... 62

General Data Collection Procedures ..................................................................62

Qualitative Data Collection ................................................................................62

Quantitative Data Collection ..............................................................................65

Data Analysis ............................................................................................................. 69

Qualitative Data Analysis ....................................................................................69

Quantitative Data Analyses .................................................................................73

Mixed Methods Data Analysis .............................................................................76

Potential Ethical Issues .............................................................................................. 78

Protection of Research Participants ...................................................................78

Researcher’s Resources and Skills ............................................................................ 81

Context of the Researcher ...................................................................................81

Summary .................................................................................................................... 82

IV. RESEARCH RESULTS ........................................................................................... 84

Quantitative Results ................................................................................................... 85

Paired-Means t-Test ............................................................................................86

Summary of the Quantitative Findings ................................................................96

Qualitative Findings ................................................................................................... 97

Supporting Student’s STEM Persistence ...........................................................100


v

Developing STEM Skills and Content ...............................................................119

Experience Levels ..............................................................................................129

Not Sure About a STEM Future .........................................................................138

Sources of Motivation ........................................................................................141

Summary of the Qualitative Findings ................................................................162

Mixed Method Analysis .......................................................................................... 163

Chapter Summary .................................................................................................... 165

V. DISCUSSION, IMPLICATIONS, LIMITATIONS, AND RECOMMENDATIONS FOR FUTURE RESEARCH...................................... 167

Discussion of the Results ......................................................................................... 168

Research Question #1: Change in Perceptions of and Actions Toward STEM Persistence ..............................................................................................168

Research Question #2: Alter 21st Century Learning Skills, Motivation, and Interest In STEM Careers ...........................................................................174

Limitations and Recommendations for Future Research ......................................... 189

Conclusion ............................................................................................................... 196

APPENDICES ............................................................................................................... 220

A. Recruitment Letter .............................................................................................. 220

B. Consent to Participate ......................................................................................... 222

C. Student Assent Form ........................................................................................... 226

D. Information Sheet ............................................................................................... 227

E. Observation Tool ................................................................................................. 229

F. Interview Tool ..................................................................................................... 231

G. STEM Extracurricular Activity Questionnaire (Descriptive Statistics) ............. 235

H. Student Attitudes Toward STEM (S-STEM) Survey ......................................... 238

I. Email From MISO ................................................................................................ 239

J. Friday Institute Permission .................................................................................. 240

K. Methodology Outline .......................................................................................... 242


vi

L. Audit Trail for Chapter IV .................................................................................. 243

M. IRB Letter of Approval ...................................................................................... 250

N. Institutional Approval Form ............................................................................... 253

O. Distribution of Forms.......................................................................................... 254

P. S-STEM Survey Statistical Results ..................................................................... 256

Q. Reliability Statistical Results .............................................................................. 277


vii

ABSTRACT

There is a continuous need to develop workers in the fields of Science, Technology,

Engineering and Math (STEM) who are knowledgeable (STEM content), engaged

(interested, motivated), and prepared (21st Century skills) workers for the growing Global

STEM workforce; conversely, there is a lack of students persisting to graduation with

STEM degrees. As such, it is important to understand why middle school students are

persisting with STEM through their education and into adulthood. The purpose of this

study was to investigate middle school students participating in Outside of School Time

(OST) STEM activities to understand their aptitude for 21st century skills, motivations,

interests, and plans to persist with STEM. Additionally, the study sought to underscore

the importance of OST STEM activities to support middle school students in developing

a STEM identity, thus encouraging them to pursue a STEM career path in the future.

This mixed-methods study examined 37 middle school students who participated

in different OST STEM activities at an independent school. The study analyzed

qualitative data from observations and interviews) and statistical data from the Student

Attitudes Toward STEM (S-STEM) Survey for Middle and High School Students (FI,

2012) in a pretest-posttest model. The results of this study suggest that OST STEM

activities can offer students the opportunity to pursue their STEM interests and develop

their 21st Century learning skills. Furthermore, OST STEM activities may positively

influence students’ perceived STEM persistence, particularly in the areas of future

careers in science and doing engineering to improve peoples’ lives. Lastly, this study


viii

highlighted the importance of intrinsic motivation and STEM family habitus for

independent school students.

Questionnaire Data Keywords: OST STEM activities, STEM Education, Middle

School, Independent Schools, Intrinsic Motivation, STEM Family Habitus


ix

LIST OF TABLES

3.1 Length and Duration of OST STEM Activities and Data Collection .................. 60

3.2 Amount of Weeks Between the Pretest and Posttest ........................................... 69

4.1 Qualitative Themes and Subthemes Breakdown ................................................. 99

4.2 Observed Data Related to the Themes and Subthemes Theme ......................... 109

4.3 Students Self-Reported OST STEM Activity Participation............................... 117

4.4 Questionnaire Data............................................................................................. 135

4.5 Interview Data Topics Related to the Subthemes .............................................. 143

A.1 All Subjects Paired Means t Test Data .............................................................. 256

A.2 Girls Paired Means t Test Data ......................................................................... 256

A.3 Boys Paired Means t Test Data ......................................................................... 257

A.4 6th Grade Paired Means t Test Data .................................................................. 257



A.7 All Subjects Wilcoxon Signed-Rank Test Data ................................................ 259

A.8 Girls Wilcoxon Signed-Rank Test Data............................................................ 262

A.9 Boys Wilcoxon Signed-Rank Test Data ........................................................... 265

A.10 6th Grade Wilcoxon Signed-Rank Test Data ..................................................... 268

A.11 7th Grade Wilcoxon Signed-Rank Test Data ..................................................... 271

A.12 8th Grade Wilcoxon signed-Rank Test Data ..................................................... 274

A.13 Reliability Statistics: Construct Level.............................................................. 277


x

LIST OF FIGURES

1. Inputs and Outputs of the Study............................................................................ 10

4.1. Themes and Subthemes Developed From Qualitative Analysis. ........................ 100

4.2. Additional Notes on Students' Overcoming Frustration. .................................... 111

4.3. Question 6 Shows Students’ Body Language Being Serious and Focused. ....... 113

4.5. Question 2 Shows the Students’ Projects............................................................ 124

A.1. Methodology Outline. ........................................................................................ 242

Texas Tech University, David C. Taylor, May 2019

1

CHAPTER I

INTRODUCTION

Each year across the United States, seven million middle school-aged students

participate in Out of School Time (OST) (i.e. informal or outside of formal instruction)

Science, Technology, Engineering, and Mathematics (STEM) activities (Afterschool

Alliance, 2015). STEM OST activities are important for middle-grade aged students and

foster learning of STEM content (Brown, 2016; Holmquist, 2014) and social, academic,

physical, moral, and physiological development (California Department of Education

Publication, 2017; Dickinson & Butler, 2001; Sahin, 2013). Furthermore, the introduction

of OST STEM activities to young adolescents in the middle grades is an opportunity to

support the development of their STEM identity (Archer et al., 2010; Hazari, Sonnert,

Sadler, & Shanahan, 2010), which is “their ability to see themselves as the kind of

people who could be legitimate participants in STEM through their interest, abilities,

race, gender, and culture” (Hughes, Nzekwe, & Molyneaux, 2013, p.1980); contributing

to their interests towards and possible future career in STEM (Afterschool Alliance,

2015; Archer et al., 2010; Brown, 2016; Sahin, 2013).

Considering issues of underrepresentation in STEM (National Science Foundation

[NSF], 2014), developing a STEM identity is extremely important for middle school

females (Barton, Kang, Tan, O’Neill, Bautista-Guerra, & Brecklin, 2012) and minority

students (Espinosa, 2011; Hite, Midobuche, Benavides, & Dwyer, 2018) to support their

interests in STEM learning. Yet, few students, especially students from gender, racial,

and ethnic minorities persist, or continue on a STEM pathway throughout high school


2

and college and becoming a part of the STEM workforce (Andersen & Ward, 2014;

Espinosa, 2011; Maltese & Tai, 2011;Tai, Qi Liu, Maltese, & Fan, 2006).

Prior research has shown that student access to quality OST STEM experiences is

a key factor in enhancing STEM persistence for students (Afterschool Alliance, 2015;

National Research Council [NRC], 2015; NRC, 2009). These activities offer authentic,

hands-on learning with STEM tools (technology) and practices (21st Century skills, the

engineering design process) which supports students’ learning of STEM content and

identity (International Technology and Engineering Education Association [ITEEA],

2016; Holmquist, 2014; Mohr-Schroeder et al., 2014; Nugent Barker, Grandgenett, &

Adamchuk, 2010). Middle school students’ desires to understand STEM content and

forge a STEM identity have been empirically connected to an individual’s motivation or

intrinsic desire to learn. When students are given choices, they tend to be more motivated

than learners who are compelled to comply (Deci, Vallerand, Pelletier, & Ryan, 1991;

Rigby, Deci, Patrick, & Ryan, 1992), which is why understanding student participation in

OST STEM activities may provide some insight into STEM persistence. This study

focused on middle school-aged students within an urban independent, private school

located in the Southeastern United States who participate in one of four of OST STEM

activities: robotics (SeaPerch (2013), sumo-bots, and drones), Science Olympiad (2017),

Girls Who Code (2017), and eCYBERMISSION (2016). Though specific requirements

vary by activity, these OST STEM activities all task students with tackling community-

based problems, challenge them in competitions, and require them to create working


3

prototypes. These activities require design thinking, hands-on learning, and problem-

solving using engineering skills, tools, and technology.

The study focused on middle school-aged students for two reasons. One, middle

school students are forming their STEM identities at this time and, thus, the factors

related to their interest and motivation in STEM may facilitate persistence in future

STEM studies. As such, this study sought to develop an understanding of the affordances

of STEM-based OST activities on middle school students’ interest and motivation in

STEM, as well as persistence (continued participation) in STEM-based OST courses.

Two, there has been little research on the impacts of OST STEM activities in middle-

school aged children and this study sought to address this research gap. Research

supports the notion that OST STEM activities in high school influenced participating

students’ STEM learning and persistence as measured by college STEM course

enrollment (Afterschool Alliance, 2015; Brown, 2016; NRC, 2015), but there is a need

for adding clarity to the understanding of it and how middle school OST STEM activities

can impact middle school students’ STEM persistence. This research and its findings can

help support the development of and changes to educational practices in OST settings to

better support students’ interest, motivation, and persistence in STEM.

Need for the Study

The relevance of informal STEM activities is illustrated by the financial support

of the U.S. government in the Every Student Succeeds Act (2015) and other federal

programs, such as the Teacher Incentive Fund (2018) and Upward Bound Math-Science

(2018) (President’s Council of Advisors on Science and Technology [PCAST], 2010;


4

Weber, 2012). Furthermore, the National Academy of Sciences (NAS) has been writing

about this topic at the national level extensively since 2009 (NRC, 2009). This robust

federal support for informal STEM reflects prior research suggesting that informal

settings are an excellent way for students to develop collaboration and communication

skills, while gaining an understanding of STEM concepts, materials, and topics (Mohr-

Schroeder et al., 2014; Weber, 2012). Most recently, national interest has increased in

informal STEM activities as a means to boost students’ interest in and motivation for

learning STEM (Holmquist, 2014; Nugent et al., 2010).

Though there is focus on the importance of OST STEM activities, there is a gap in

understanding of how informal STEM experiences affect middle school students’

perceived STEM persistence which is a strong mediator of students’ deciding to enter the

American STEM workforce (Fayer, Lacey, & Watson, 2017). Hence, this research

addresses the correlation between participation in OST STEM activities and motivation,

interest and, ultimately, STEM persistence. This is important as affect relates to

belonging, a fundamental component of identity; suggesting positive experiences (affect)

in STEM can support the construction of a robust STEM identity. The Afterschool

Alliance (2015) has suggested that standard academic measures are insufficient and do

not truly capture the OST STEM activity, especially OST STEM activities that focus on

engineering and technology to engage the different populations of students.

Background of the Problem

The STEM global economy, defined as the global economy fueled by innovation

and a highly skilled STEM workforce situated across the globe (Atkinson & Mayo, 2010;


5

Carnevale, Smith, & Melton, 2011), has continued to grow (Palmer, Davis, Moore, &

Hilton, 2010) since the 1980s, especially in the areas of engineering and computer

science (Brazell, 2013). As the STEM global economy has grown, so has the demand for

skilled STEM workers (Basham & Marino, 2013; Hossain & Robinson, 2012; Palmer et

al., 2010). In 2011, the U.S. Department of Commerce predicted growth in STEM-related

positions of 17% by 2018. A more recent prediction made by the U.S. Bureau of Labor

and Statistics (2017) was that millions of STEM jobs would be made available by 2024.

More specifically, jobs in the mathematical sciences are expected to grow by 28.2%, and

the number of computer occupations should see an increase of 12.5%. The European

Union expects the demand for STEM employees to grow 8% from 2015 to 2025, while

other fields are only expected to grow 3% (Caprile, Palmén, Sanz, & Dente, 2015). The

Organisation for Economic Co-operation and Development (OECD, 2016) states as one

of its central missions are to endorse policies that will advance the STEM global

economy and society.

Growing a Global STEM Workforce

Both the EU and the USA are both trying to increase the number of STEM

workers, particularly those from underrepresented groups (e.g., women, minorities),

going into STEM careers to support the demand for STEM work (Brazell, 2013; Caprile

et al., 2015). To address this need, innovative instructional practices such as culturally

relevant curriculum are being utilized to create spaces for underrepresented groups to

access and build affinity (identity) with STEM (Espinosa, 2011; Hite et al., 2018) and

reduce negative influences such as stereotype threat (Shapiro & Williams, 2012). Other


6

research-based strategies include same-sex instruction (Ahmed, 2016), like specialized

STEM schools for girls in Egypt (Ahmed, 2016) and same-sex informal STEM activities

(Hite et al., 2018; Hughes et al., 2013). These novel pedagogies are showing success in

Native American tribes (Stevens, Andrade, & Page, 2016), in the US more broadly, and

in Latin America countries (Hite et al, 2018; Zimmerman, Johnson, Wambsgans, &

Fuentes, 2011; Wang, 2013).

Needs to Grow a Global STEM Workforce

The global STEM economy coupled with the advancement of telecommunication

technology has created a world that needs 21st Century skilled workers prepared to

innovate across countries’ borders (Palmer et al., 2010; Wagner, 2014) for sustainability

of the global market and economy (Palmer et al., 2010). These 21st century soft skills,

such communication, collaboration, critical thinking, and problem-solving (P21, 2015;

Ahmed, 2016) are the foundation for students’ success in innovating a nation’s STEM

economy on a global scale (Ahmed, 2016; Capraro, Capraro, & Morgan, 2013). Further,

the Partnership of 21st Century Learning (P21, 2015) explained that students must have

skills in a variety of literacy areas, innovation, communication, information, media, and

technology to be successful in the future global economy and society. STEM education

can help students learn 21st Century skills by preparing them to collaborate,

communicate, and be globally aware by interconnecting the world (Brazell, 2013;

DeJamette, 2012; Peters, 2009; Vilorio, 2014). One example of a strategy for developing

21st century skills is the use of globally collaborative projects in which students can

create, communicate, and develop new products or viewpoints with their international

peers (Lindsay & Davis, 2012). As the domestic and international STEM economies


7

continue to grow, it is imperative that STEM educators prepare students with a

comprehensive education to become a part of the global STEM workforce 21st Century

(Brazell, 2013; Caprile et al., 2015; PCAST, 2010).

A lack of STEM skills and inadequate soft skill preparation for the workplace is a

concern for both national and international employers (Ahmed, 2016; Brazell, 2013;

Caprile et al., 2015). To support the STEM careers, K-12 and higher education

institutions need to develop students who have “both technical and non-technical skills

and dispositions” (Hossain & Robinson, 2012, p. 450). Students need to be able to think

logically and creatively to solve problems, as well as develop communication and

teamwork skills (Hossain & Robinson, 2012). These STEM-specific soft skills are such

an asset to the economies of countries like Australia, Argentina, and China that these

countries are making changes to their science education and science teacher preparation

programs (Cofré et al., 2015; Liu, Liu, & Wang, 2015; Treagust, Won, Petersen, &

Wynne, 2015).

OST as a Strategy to Grow a Global STEM Workforce

As mentioned previously, organizations are examining the use of informal STEM

opportunities to enhance STEM learning and the STEM workforce (NRC, 2015).

Research has signaled that these activities may help drive STEM persistence which will

help to fill the STEM economy gap. OST STEM activities are an example of informal

learning which encourages students to gain STEM experience (Afterschool Alliance,

2015; NRC, 2015; NRC, 2009) by enriching and deepening STEM learning outside of

classroom instructional time (NRC, 2015; Sahin, Ayar, & Adiguzel, 2014). There have

been retrospective research surrounding the use of the Persistence Research in Science


8

and Engineering (PRiSE, 2007) survey instrument, “designed to identify and characterize

the fundamental factors that influence students’ intentions to pursue an engineering

degree over the course of their undergraduate career, and upon graduation, to practice

engineering as a profession” (Eris et al., 2005, p. 10.476.1). Dabney et al. (2012)

analyzed the data from the PRiSE survey (N=6882) given to US university and college

students enrolled in introductory English courses and found that OST STEM activities in

middle school, along with other factors (i.e. gender, and middle school math and science

interest), had a part in these college students’ interest in pursuing STEM at the college

level. Lastly, research suggests that OST STEM activities are not only important for

developing students’ interest and motivation in learning STEM (Hite et al., 2018; NRC,

2015), but also may be important to continue to gain a greater understanding of middle

school students’ STEM persistence as it relates to OST STEM activities.

Problem Statement

It is important for future growth and sustainability of the global STEM economy

to develop knowledgeable (STEM content), engaged (interested, motivated), and

prepared (21st Century skills) workers for the future (Atkinson & Mayo, 2010; Carnevale

et al., 2011; Palmer et al., 2010). The majority of American K-12 students who are

engaging in STEM learning are not following through to STEM careers because they are

neither entering nor persisting through the STEM pipeline. Previous research indicates

there is a dearth of students who start college as a STEM major and persist to earn a

STEM college degree; furthermore, there exist factors that hinder their STEM persistence

(Espinosa, 2011; Graham, Frederick, Byars-Winston, Hunter, & Handelsman, 2013;


9

Griffith, 2010; Ost, 2010; Palmer, Maramba, & Dancy, 2011; Price, 2010). Also, research

has shown the importance of students ’success in engaging high school STEM course

work to STEM persistence (Andersen & Ward, 2014; Maltese & Tai, 2011), but greater

clarity is needed to understand the drivers of middle school students’ persistence in

STEM.

In regard to middle school-aged students learning STEM, the bulk of this research

has been conducted in the informal science settings—summer camps and afterschool

programs—and not focused on STEM persistence (Krishnamurthi, Ballard, & Noam,

2014; Mohr-Schroeder et al., 2014; Nugent et al., 2010). This research provides greater

clarity on OST STEM activities (those that occur at school, but outside of instructional

time) and the STEM persistence of the students who participate in the activities.

Conceptual Framework

The conceptual framework is rooted in the STEM persistence of middle school

students (see Figure 1 below). The input constructs were STEM interest, motivation to

participate in STEM activities, perceived persistence in STEM activities, and 21st

Century Skills. Each of these constructs were self-reported by the students. Students then

participated in OST STEM activities, participated in interviews, and were observed. The

inputs were then measured after the students participated in the OST STEM activities.

These measured input constructs in the model were transformed into the output constructs

rooted in the middle school students’ experiences in OST STEM activities. Each output

in the model was measured: STEM interest (i.e. S-STEM survey, interviews, and

descriptive statistics), persistence in STEM (i.e. STEM survey, interviews, descriptive


10

statistics, and observations), aptitude for 21st century skills (i.e. STEM survey,

interviews, descriptive statistics, and observations), and motivation in STEM (i.e.

interviews, descriptive statistics, and observations), The model in Figure 1 was used for

conceptualizing the students’ experience from their participation in the OST STEM

activities and how the activities affected their motivation, interests, and persistence for

STEM learning, as well as 21st century learning skills.

Figure 1. Inputs and outputs of the study.

The conceptual framework supported the design of the study to explore middle

school-aged students’ experiences and thoughts about STEM before, during, and after

participating in OST STEM activities. The students’ experiences in the OST STEM

activities were conceptualized using four constructs or inputs: STEM interest, motivation

in STEM, persistence in STEM, and aptitude for 21st century skills. The input and output

constraints are intrinsic motivation (Deci et al., 1991; Rigby, Deci, Patrick, & Ryan,

1992), interest in STEM (Dewey, 1913; Holmquist, 2014; Wang, 2013), persistence in

STEM (Andersen & Ward, 2014; Espinosa, 2011; Maltese & Tai, 2011)., and 21st century

skill learning (P21, 2015, 2016; Kay, 2009).


11

Purpose of the Study

The purpose of the study was to identify middle school students’ aptitude for 21st

century skills, as well as motivations, interest, and perceived persistence in STEM from

their OST STEM learning. The study explored these affective and influential factors of

four OST STEM activities and how those experiences played a role in participating

students’ reported motivations, interest, persistence in STEM using proxy measures such

as pre- and post-surveys, one-on-one interviews, observations, and inventorying which

and in how many STEM courses middle school students chose to enroll. This research

sought to underscore the importance of OST STEM activities to support middle school

students in developing a STEM identity so they may persist through STEM high school

courses, college majors, and/or careers. The knowledge gained from this research can

help to inform best practice in OST STEM activities and education.

Research Questions

Below are the research questions that guided this study.

Upon participation (before to after) in a program for OST STEM activities, how did this

intervention:

1. change middle school students’ perceptions (descriptions) of and actions

(enrollment) toward STEM persistence?

a. Type and number of current middle STEM courses in their formal

schooling?

b. Type and number of future STEM courses in their formal schooling?


12

2. alter middle school students’ 21st century learning skills, motivation, and

interest in STEM careers?

Overview of Research Design

Based on the researcher’s pragmatic worldview, this research study used a

convergent parallel mixed method design which provided the researcher choices of

philosophy, methods, techniques, and procedures to address the problem (Creswell,

2013). This design also offered the best method for answering the research questions and

enabled the researcher to combine and triangulate the data gathered in connection to the

OST STEM activities (Creswell & Plano Clark, 2011).

The qualitative research portion of the study was conducted as phenomenological

research, which was used to describe the students’ perceived experiences in the OST

STEM activities and their own interests, motivation, and perceptions of if they would

persist in pursuing STEM activities (Williams-Watson, 2017; Somerville-Midgette,

2015). The researcher collected data through surveys, observations, and interviews in

order to explain the phenomena of interest, attitude, and motivation for the students’

perceptions of STEM learning and persistence. The quantitative portion utilized a pre-

post survey design to determine if there was a significant difference in middle school

students’ motivation and interest to pursue STEM courses and activities after

participating in an OST STEM activity. This allowed the researcher to triangulate

understanding of if and how the OST STEM altered the middle school students’

motivation, interest, persistence in STEM, STEM careers, and 21st century learning skills.


13

This research study used a mixed methods design as the best means to answer the

research questions. The S-STEM survey (FI, 2012) was used to quantify the changes in

the students’ students’ thoughts towards STEM content areas, 21st century learning

skills, and interests in STEM careers after participating in their OST STEM activities, but

the qualitative research was able to provide insight into the students’ own experiences

through their own words, as well as gather background information related to the

students’ STEM learning. This is due to the limitations of the S-STEM survey (FI, 2012)

not being able to provide insight into the students’ prior STEM learning experiences,

other STEM-related influences, and the students’ own words related to the students’ own

experiences and thoughts related to their OST STEM activity experiences. Furthermore,

by using a mixed method design the S-STEM survey (FI, 2012) was not altered and kept

the reliability and validity of the survey intact, as well as provided the means to gain an

understanding of the impact of the OST STEM activities on the students’ perceived

STEM persistence, as well as other STEM learning factors influencing these students. A

single method only could not have been able to achieve these outcomes. The mixed

methods data analysis used a side-by-side comparison approach, which provides a mixed

methods interpretation of the qualitative and quantitative research to be compared in a

convergent parallel method to develop a discussion of the findings (Creswell, 2013;

Creswell & Clark, 2011). The pre-post survey responses were analyzed using a paired-

samples t-test along with a Wilcoxon signed-rank test for each individual topic of the

survey. Interviews and observations were coded and analyzed. The choice of using a

mixed method study that used qualitative and quantitative methods supported the deeper


14

and new understanding of a topic that the two methodologies independently could not

provide (Creswell, 2013), which led to more findings to answer the research questions

than through a single methodology (Creswell & Plano Clark, 2011). In following this

paradigm, this mixed method study reported the quantitative statistical results first and

then the qualitative result seconds, which confirmed or disconfirmed the quantitative

finding and lead to the qualitative results giving way to the overall findings of the study

(Creswell, 2013). Overall, the convergent parallel design afforded the researcher an

opportunity to use the qualitative and quantitative data equally by blending them through

concurrent timing.

Significance of the Study

This study explored middle school students’ reasoning for pursuing OST STEM

activities and their perceptions of how these activities influenced their 21st century skills,

motivation, interest, and persistence in both present and future STEM courses. The data

collected offered insight into middle school-aged students’ perceptions of 21st century

skills, motivation, and interest in STEM, as well as the impact of the OST STEM

activities on their reported STEM persistence. Furthermore, this research supports

informal STEM education by providing greater awareness of middle school students’

affect with regard to their perceptions of STEM learning. A strong understanding of why

middle school students pursue STEM (e.g. subjects, careers, etc.) and the impact OST

STEM activities have on their persistence in STEM knowledge and skills is important

information to support changes in formal and informal educational settings. Insight into

how informal programs, like OST STEM activities, may support students’ persistence in


15

STEM activities is important for supporting the global STEM pipeline. This study can

provide an understanding into middle school students’ perceptions of STEM persistence,

motivation, and interest for STEM learning, as well as what STEM curriculum and

instructional strategies support positive STEM affect like persistence, motivation, and

interest. This is especially important concerning OST and informal STEM activities due

to the impact they have on students’ forging STEM identities and continuing with their

STEM learning (Brown, 2016; Hazari et al., 2010; PCAST, 2010).

Audience

The findings of this study will help inform STEM educators about how OST

STEM activities foster students’ interest, motivation, 21st century skills and persistence in

STEM disciplines. Moreover, the research study can provide all stakeholders—parents,

educators, and policymakers—information needed to make informed decisions about

creating OST STEM programs. STEM education researchers can use the studies’ findings

to support OST STEM education research, specifically to how OST STEM activities

foster positive STEM affect and insight into the growing body of literature on adolescent

students’ perceptions of STEM persistence.

Assumptions

It is assumed that participants responded honestly to prompts and questions asked.

The presence of the researcher in the room during classroom activities was expected to

have no or minimal impact on the behavior of the OST STEM activity teacher and the

participating students.


16

Positionality

The researcher is a middle school engineering teacher with over 11 years of

experience in STEM education instruction at the middle school level. The researcher has

instructed a variety of OST STEM activities including afterschool programs and summer

camps for students ranging from 9 to 18 years old. This information influences the

researcher’s reflexivity due to assumptions of meanings and understanding of technical

skills, terms, and STEM activities that students were a part of during this study. The

researcher’s pragmatic worldview, which provided the researcher choices of philosophy,

methods, techniques, and procedures to solve the problem, guided the data collection and

selection of analysis procedures (Creswell, 2013). This pragmatic worldview of the

researcher allows for a focus on understanding the problem of practice as described in

this chapter (Creswell, 2013).

Delimitations of the Study

The scope of this mixed method study focused on the change in middle school

students’ perceptions (descriptions) of STEM persistence, before and after OST STEM

participation using individual interviews and group observations. Furthermore, the scope

of this study involved how the OST STEM activities alter middle school students’

motivation, interest, 21st century learning skills, and awareness in STEM careers

measured by students’ pretest and posttest responses on the Student Attitudes

toward STEM Survey or S-STEM survey (FI, 2012). The factors of motivation and

interest were selected due to the importance these factors have students’ STEM

persistence to continue in the STEM pipeline (Hite et al., 2018; NRC, 2015, as well as


17

their STEM identity (Afterschool Alliance, 2015; Archer et al., 2010; Brown, 2016;

Hughes et al., 2013; Sadler, Sonnert, Hazari, & Tai, 2012). The construct of 21st century

learning skills was selected due to the importance these skills have on students’ STEM

learning (Brazell, 2013; P21, 2015) and success in future careers in STEM (Atkinson &

Mayo, 2010; Palmer et al., 2010).

The focus on middle school students’ in OST STEM activities is due to the need

for increased clarity on this topic and the fact that the researcher is a middle school

STEM educator. This research study only focused on middle school students participating

in OST STEM courses and activities offered by their school, which is located in a

metropolitan city in the Southeastern United States. The researcher explored STEM

interests, and motivation to pursue STEM learning of only middle school students

participating in OST STEM activities; middle school students who did not participate in

OST STEM activities were excluded from the study, as they would not have recent

experiences under exploration.

Limitations of the Study

Creswell (2014) typically advises researchers not to collect data at their own

workplace in light of the potential of collecting inaccurate data or jeopardizing the

research; however, he explained that the researcher could provide a plan for not

compromising the research. The researcher teaches at the school where the study took

place and took steps to reduce researcher bias. The researcher resolved to be purposeful

with his interviewing process and accurately capture his participants’ stated responses.

Additionally, using a school administrator at the school to distribute and collect consent


18

and assent forms, surveys and questionnaire data minimized interaction and therefore

potential bias.

Creswell (2014) and Lincoln and Guba (1985) have argued that the trust or

relationship the researcher has built with prospective participants is likely to ensure the

capture of authentic data within the classroom. Stakes (1995) further explained that it is

perfectly normal and somewhat desirable for doctoral students that have a full-time job to

do the research in their own work settings. To mitigate researcher bias and increase

trustworthiness, the methodology was carefully constructed to include the use of extant

theory to model reality and collect multiple types of data (Erlandson, Harris, Skipper, &

Allen, 1993; Kincheloe, 2001). The qualitative portion of the mixed methods approach

limits the transferability to other middle school students and OST STEM programs, but

provides greater visualization to the experiences of these students in the studied

programs. The quantitative portion of the study used a statistical pre-post-test to analyze

the change in students’ interest in and motivation for STEM, which can be generalizable

to larger contexts.

Definitions of Terms

The following definitions will help give a clear understanding of important terms

that will be used in this study. The definitions below have been provided to offer clarity

to their meaning:

1. STEM/STEM education—STEM stands for science, technology, engineering,

and mathematics. STEM education is an interdisciplinary approach that

combines rigorous academic disciplines to prepare students to solve real-


19

world problems (Gerlach, 2012; Nkhata, 2013). It integrates problem-solving

and critical thinking, along with the scientific method and engineering design

processes, as well as 21st Century skills (Basham, Israel, & Maynard, 2010;

Brazell, 2013). Gerlach (2012) stated that STEM is “about moving forward,

solving problems, learning, and pushing innovation to the next level” (p. 2).

2. Interest—John Dewey (1913) stated, “Genuine interest is the accompaniment

of the identification, through action, of the self with some object or idea,

because of the necessity of that object or idea for the maintenance of a self-

initiated activity” (p. 14). Interest is a personal and objective matter that has

an individual actively concerned with it (Dewey, 1913). Interest influences

learners of all ages throughout all disciplines, in and out of school, by making

the connections that lead to learning (Renninger & Hidi, 2011).

3. Motivation—Motivation is the drive for doing something. Dewey (1937)

explained that learning is best supported when individuals are internally

motivated. He claimed this leads to accomplishments, excitement, and

satisfaction in work and learning and such an environment guides learning and

creates a sense of enjoyment for learning supports individual growth. Pink

(2011) explained that intrinsic motivation supports individual development

that is built upon autonomy, mastery, and purpose. Intrinsic motivation,

guided by personal enjoyment for doing an act, is crucial in cognitive

development and drives learning and exploration (Oudeyer & Kaplan, 2008).


20

Motivation is affected by beliefs, persistence, effort, and choice and is an

indicator of academic success (Freeman, Alston, & Winborne, 2008)

4. STEM persistence—STEM persistence is the ability and reasoning of a student

to continue with STEM learning and following a STEM pathway. STEM

persistence is influenced by a variety of factors, including academic

achievement, prior experiences, early STEM access, curriculum, teacher

impact and more (Andersen & Ward, 2014; Maltese & Tai, 2011). STEM

persistence for STEM college majors and degrees has been shown to be lower

for women and minorities (Andersen & Ward, 2014; Espinosa, 2011; Maltese

& Tai, 2011).

5. Constructivism—Constructivism involves an individual in an active process of

constructing knowledge through active, social, and contextual mediums (Hein,

1991). The social interaction promotes strong feelings of belonging and

satisfaction (Feldman & Matjasko, 2005; Ivaniushina & Alexandrov, 2014;

Sullenger, 2006). Students need social interactions with peers and cultural

products to construct knowledge, which leads to a richer development in

cognition and learning (Ernest, 1998; Leach & Scott, 2003).

6. 21st Century skills/learning—The Partnership for 21st Century Learning

(2015) defines 21st Century learning as learning activities that allow students

to be creative and innovative. Furthermore, they define 21st Century skills as

those that empower students to think critically, solve problems, communicate,

and collaborate. These skills support student development and learning, which


21

will support the changing global economy and society (Ananiadou & Claro,

2009).

7. Out of School Time (OST) STEM activities/programs—OST STEM

activities/programs are science, technology, engineering and/or mathematics

programs that provide hands-on, inquiry-based learning that is conducted in

informal educational settings outside of the formal classroom setting (Eshach,

2007). Informal science education (ISE) is comprised of after-school

programs that are facilitated by K-12 schools, non-profit education centers,

and organizations, and universities, which foster learning outside of the

traditional, formal school setting (Ayar, 2015; Brisson et al., 2010). This study

will focus specifically on OST activities and programs led by the school.

8. Robotics – The schools’ robotics program participates in three categories of

robotics throughout the school year: SeaPerch, Sumo-bots, and Drones. The

SeaPerch underwater robotics is a national competition involving teams of

students building Remotely Operated Vehicles (ROV) to complete underwater

obstacles while learning engineering and science concepts with marine

engineering theme in small teams (SeaPerch, 2013). Sumo-bots is a sport in

which students build and code autonomous robots using a specific robotics

platform, such as LEGO Mindstorms (LEGO, 2018), to push another

opposing robot out of a circular ring before being pushed out. Drones

involved teams of students flying prebuilt mini-drones to complete obstacles


22

and challenges against other local schools through autonomous coding and

remote controls.

9. Science Olympiad – Science Olympiad is a national science competition

focused on a wide range of science content areas that has a team-focused

approach surrounding active, hands-on involvement for students. Each year,

teams of 15 students prepare and participate in topics from the fields of

“nature of genetics, earth science, chemistry, anatomy, physics, geology,

mechanical engineering and technology (Science Olympiad, 2017a)”. These

teams cross-train for a variety of hands-on and content knowledge events.

Science Olympiads mission is “dedicated to improving the quality of K-12

science education, increasing male, female and minority interest in science,

creating a technologically-literate workforce and providing recognition for

outstanding achievement by both students and teachers (Science Olympiad,

2017b)”.For more information about Science Olympiad, please visit the

following: https://www.soinc.org/.

10. Girls Who Code – Girls Who Code is an OST program with the mission to

close the gender gap in technology. The girls participating in this program

learn coding concepts and skills in a safe and supportive environment with

peers and positive female role models to learn how to become computer

scientists (Girls Who Code, 2017). Furthermore, the girls’ complete real-

world impact projects. For more information, please visit the following site:

https://girlswhocode.com/.

https://www.soinc.org/

https://girlswhocode.com/


23

11. eCYBMERMISSION– eCYBERMISSION is a national STEM competition

for 6th, 7th, 8th, and 9th-grade students. The program focuses on students with

the support of an advisor to a solving real-world problem through proposing a

solution to a specific problem in their community and competing for state,

regional, and national awards (eCYBERMISSION, 2016). There are specific

scientific categories for the teams of students for the competition. For more

information, please visit the following link: https://www.ecybermission.com/.

12. Self-determination theory – Learners have an intrinsic desire to learn and

when given choices they tend to be more motivated than learners who are

heavily regulated to comply, such as choosing to participate in an OST STEM

activity (Deci et al., 1991; Rigby, Deci, Patrick, & Ryan, 1992).

https://www.ecybermission.com/


24

CHAPTER II

LITERATURE REVIEW

The following chapter describes the prior research and studies conducted that are

relevant to this study’s work which is changing middle school students’ perceptions

(descriptions) of motivation, interest, 21st century skill growth, and STEM persistence

through participation in 1 of 4 OST STEM activities. It is intended to illuminate the

connection between OST STEM activities and middle school students’ motivation,

interest, 21st century learning skills, and persistence in STEM. Specifically, the literature

review in this chapter provides ideas, theoretical perspectives, and previous research

findings that framed the exploration of OST STEM activities as a means to nurture

positive affect (interest, motivation) and development of 21st century skills for STEM

persistence. OST STEM activities are intended as engaging, hands-on learning activities

that have students using tools, technology, 21st Century skills, and the engineering design

process and can support students’ learnings of STEM content (Holmquist, 2014; ITEEA,

2016; Mohr-Schroeder et al., 2014; Nugent et al., 2010). OST STEM activities also

support the development of middle school students STEM identity and contribute to their

interests towards and possible future career in STEM (Afterschool Alliance, 2015; Archer

et al., 2010; Brown, 2016; Hazari et al., 2010; Hite et al., 2018; NRC, 2015; Sahin, 2013;

Wyss, Heulskamp, & Siebert, 2012). Hence, quality OST STEM activities may be a key

factor in enhancing STEM persistence (Afterschool Alliance, 2015; NRC, 2015, 2009).


25

Conceptual Framework

This study explored the change of middle school students’ perceptions

(descriptions) of STEM persistence as an output of participation in one of four OST

STEM activities: Girls Who Code, eCYBERMISSION, Science Olympiad, and robotics

(sumo-bots, and drones). It also explored the connection between OST STEM activities

and middle school students’ motivation, interests, 21st century learning skills, and

persistence in STEM careers,

The OST STEM activities in this study (e.g. Girls Who Code, eCYBERMISSION,

Science Olympiad, and robotics [sumo-bots and drones]) provide students’ engaging,

hands-on learning activities that have them using tools (i.e. soldering irons) and

technology (i.e. building robots and using 3D printers), utilize 21st Century skills, and

follow the engineering design process to develop their learning of STEM content and

careers (ITEEA, 2016; Holmquist, 2014; Mohr-Schroeder et al., 2014; Nugent et al.,

2010). Prior research suggests OST STEM activities, like the activities in this study, are

key factors in enhancing STEM motivation (Holmquist, 2014; Wang, 2013), interests

(Mohr-Schroeder et al., 2014; Nugent et al., 2010), and persistence (Afterschool Alliance,

2015; NRC, 2015; NRC, 2009), independently, but not as a collective. The conceptual

framework of this study used the theories and research on the development of middle

school students’ STEM identities and interests towards a possible future career in STEM

through OST STEM activities (Afterschool Alliance, 2015; Archer et al., 2010; Brown,

2016; Sahin, 2013). Therefore, this study set out to understand how these four outputs


26

(i.e. motivation, interest, persistence, and 21st century learning) influence students’

perceptions of STEM from their participation in OST STEM activities.

OST STEM activities were analyzed with the constructs of interest, motivation,

persistence in STEM, and 21st century skill learning as inputs. However, the novelty of

this work focused on the students’ perceptions of their learning in these four constructs

rather than leveraging other types of proxy measurements. The uniqueness of this study is

derived from the work of obtaining an understanding of the influence of the OST STEM

activities on the participating middle school students’ interest, motivation, persistence in

STEM, and 21st century skill learning from the students’ own responses (i.e. interviews

and survey responses).

Motivation

Motivation is a key factor in students’ pursuit of STEM knowledge and skills

because motivation is influenced by beliefs, persistence, effort, and choice, and is an

indicator of academic success (Freeman et al., 2008). Motivation is highly important

learning and academic success (Lirmenbrink & Pintrich, 2002). When a student is

interested in STEM, he or she is motivated to learn it, which increases their learning

(Boy, 2013). Dewey (1916) explained that interest is a major factor that affects

motivation, which consequently influences learning. Various factors of one’s learning

environment can affect motivation, which guides the “duration, intensity, and direction of

academic behavior” (Freeman et al., 2008, p. 228) and their future goals.

Motivation specific to STEM. Factors affecting student motivation for STEM,

such as social variables, grades, and challenges, impact students’ intrinsic and extrinsic

motivations (Freeman et al., 2008). Dewey (1937) explained that learning is best


27

supported when individuals are internally motivated; which leads to accomplishments,

excitement, and satisfaction in their work and learning. Intrinsic motivation guides

students’ interests and enjoyment in learning (Ryan & Deci, 2000). Intrinsic motivation

has been shown to have an impact on students’ motivation to pursue STEM due to the

“exposure to math and science courses, and math self-efficacy beliefs” early in their

academic pursuits (Wang, 2013). Extrinsically motivated reward systems have been

shown to negatively affect students (Adams, 2006). This, coupled with the importance of

developing students’ STEM identity (Hazari et al., 2010; Hite et al., 2018; NRC, 2015;

Sahin, 2013), demonstrates a need to develop intrinsic motivation in students to pursue

STEM. This can be done by providing learning support and making learning relevant.

Motivation is supported through goal-based activities that encourage students

(Schunk, Pintrich, & Meece, 2008), especially activities that challenge learners and

develop personal autonomy (Deci et al., 1981). The motivation for learning is

continuously impacted by students’ perceptions of school, relationships with peers and

adults, experiences with success, and engagement in their school work, all of which

impact STEM motivation related to their choices, effort, and persistence (Schunk et al.,

2008). Engaging, fun, and interesting STEM learning experiences create student

motivation and interest in learning STEM (Mohr-Schroeder et al., 2014). According to

research by Young, Fraser, and Woolnough (1997) student motivation is shaped by their

home life (e.g. family, and friends) as well as the influence of the school, (e.g. teachers’

enthusiasm, student-centered pedagogies, and access to career advice); these factors can

make a positive or negative impact on students’ career choices. This information is


28

important since students self-regulate their internal values and behaviors based on their

motivation and social world (Deci, Ryan, & Williams, 1996).

Motivation specific to middle-grade students. Students need to experience

teaching approaches that create excitement and passion and promote collaboration, all

which increase students’ drive for learning STEM (Dewey, 1913, 1916; Gasiewski et al.,

2012; Holmquist, 2014; Jensen & Sjaastad, 2014). This is important to self-determination

due to the social implications that influence intrinsic motivation, which provide students

the opportunity to self-regulate their extrinsic motivation (Ryan & Deci, 2000). They

need to practice relevant real-world skills and not simply collect knowledge to simply

pass an examination (Regassa & Morrison-Shetlar, 2009), which can have a negative

effect on students’ motivation (Rigby et al., 1992). It is important for educators to foster

students’ independent learning (Ayar, 2014) by creating learning environments that build

students’ self-determination due to the impact it has on their learning and self-esteem

(Deci, Schwartz, Sheinman, & Ryan, 1981).

Social constructivism, or learning environments fueled by peer to peer interaction

(Vygotsky, 1978), can help to enhance experiential learning, which facilitates learning by

encouraging students to connect new knowledge with existing knowledge (Allison &

Rehm, 2006; Barker & Ansorge, 2007; Dewey, 1913). One’s social development, as well

as cognitive and physical development, is governed by the individual’s natural motivation

(Ryan & Deci, 2000). Motivation to learn math and science can be impacted by students’

prior learning experiences (Mohr-Schroeder et al., 2014; Nugent et al., 2010; Wang,

2013).


29

Motivation specific to underrepresented groups. STEM education should be

accessible to everyone regardless of race, gender or ethnic background (Duderstadt,

2007). Engaging approaches to STEM learning help motivate students to learn, and this

process is improving STEM education through rich educational experiences, active and

collaborative learning, and challenging academics (Smith, Sheppard, Johnson, &

Johnson, 2005). This is especially important for underrepresented groups in STEM

(Espinosa, 2011), like Latinas (Hite et al., 2018).

Stereotypes affect long-term motivation to continue with STEM learning and can

create distractions for those who have a higher motivation for STEM (Shapiro &

Williams, 2012). OST activities can support motivation for STEM learning for

underrepresented students (Espinosa, 2011; Hite et al., 2018). This motivation for STEM

learning can also support their STEM identity if cultural connections are developed. It is

important to create a STEM pathway for females, minorities, and students of low

socioeconomic status to increase their STEM exposure, motivation, and learning, as well

as shape their STEM self-image due to their underrepresentation in these STEM areas

(Brisson et al., 2010; Hughes et al., 2013; PCAST, 2010; Sadler et al., 2012; Zimmerman

et al., 2011). STEM lessons that allow time for student collaboration and peer interaction

that promote as a sense of learning have shown to enhance minority students’ motivation

for STEM learning (Anderson & Ward, 2014; Freeman et al, 2008).

Motivation specific to formal versus informal learning. According to

Stocklmayer, Rennie, and Gilbert (2010), formal science learning has been established to

introduce students to a variety of scientific disciplines and scientific thinking (Bull et al.,


30

2008), to prepare all students to be scientifically literate citizens, and to prepare some

students to pursue a career in STEM. Stocklmayer et al. continued that formal science has

a structured schedule, follows specific content criteria during the school day and follows

a structured, assessed, and detailed plan to support students learning of STEM content

and motivate students learning (Leblebicioglu et al., 2017). They also noted that formal

STEM education that involves lack of student-centered learning can negatively affect

students’ motivation for STEM as it can hinder their understanding of the content and

abstract topics. Formal science learning has successfully motivated students when the

content is relevant to the students and can be the platform for creating motivation to

pursue further learning of a specific STEM topic (Sladek, 1998).

Informal learning is focused on students’ interest and motivations, as it is a

voluntary, open-ended learning environment with less structure (Leblebicioglu et al.,

2017; Stocklmayer et al., 2010). Eshach (2007) explained that informal learning is

learning that can take place anywhere, and an individual’s informal learning is influenced

by their experiences in different environments and situations throughout their life

(Eshach, 2007; Sladek, 1998). Informal learning, such as an OST STEM program, affords

instructors the opportunity to answer questions, support the interests of students, and

build students’ motivation for learning STEM topics; this may be limited in the formal

classroom setting (Brisson et al., 2010; Dierking, Falk, Rennie, Anderson & Ellenbogen,

2003; PCAST, 2010; Zimmerman et al., 2011). By supporting students’ interests for

science learning through informal learning, informal science may provide information

about science and the natural world, demonstrate the use of scientific inquiry, and


31

possibly inspire students to become future scientists (Brisson et al., 2010). Informal

learning provides students the opportunity to become self-directed in their learning.

Motivation specific to OST STEM activities. Informal learning provides

students the opportunity to develop new skills, both academic and social, that promote

strong feelings of belonging and satisfaction (Feldman & Matjasko, 2005; Ivaniushina &

Alexandrov, 2014) which are key constructs to identity formation (Hughes et al., 2013;

NRC, 2015; Sahin, 2013). Informal learning, such as an OST engineering program, has

been demonstrated to be a dynamic process for shaping an individual with regard to their

knowledge, productivity, and learning (Eshach, 2007; Sullenger, 2006) which may

facilitate one’s intrinsic motivation (Mohr-Schroder et al., 2014; Ryan & Deci, 2000).

OST activities learning experiences, especially at an earlier age like middle grades, create

student motivation for STEM learning and interest (Braund & Reiss, 2006; Freeman et

al., 2008; Krishnamurthi et al., 2014; Modi, Schoenberg, & Salmond, 2012; Tai, Liu,

Maltese, & Fan, 2006). OST STEM activities, such as Girls Who Code,

eCYBERMISSION, Science Olympiad, and robotics (sumo-bots and drones), have had

success with increasing students’ interest and motivation for STEM in informal settings

(Abernathy, & Vineyard, 2001; Brown, 2016). OST STEM activities (i.e. Girls Who

Code, Science Olympiad) are also now used to increase minority access to STEM

learning, motivating students to continue learning STEM (Dabney et al., 2012; Girls Who

Code, 2017; Science Olympiad, 2016).

Interest

Dewey (1913) explained that interest has individuals actively concerned with

personal and objective concepts. Instructional practices, content topics, access to high-


32

quality instruction and enrichment opportunities can affect students’ interest for learning

STEM (Holmquist, 2014; PCAST, 2010; Shapiro & Williams, 2012; Wang, 2013).

Furthermore, enriching experiences and access to new opportunities can affect learners’

interests in specific topics (Hite et al., 2018; Mohr-Schroder et al., 2014; Rigby et al.,

1992).

Interest specific to STEM. Providing curriculum and instruction that is engaging

and relevant to students can increase their interest in pursuing STEM fields of study

(Whalen & Shelley, 2010). Analyzing students’ learning experiences and interests, as

well as instructional methods and practices, has brought to light information which

supports the development of students entering a variety of STEM-based educational

opportunities, including 4-year college, 2-year college, or trade school, to pursue a STEM

career (Andersen & Ward, 2014; Brazwell, 2010; Maltese & Tai, 2011; PCAST, 2010).

Interest can impact students’ intrinsic motivation and internalization of extrinsic

motivation for building confidence and STEM identity (Ryan & Deci, 2000; Rigby et al.,

1992). Constructivist OST STEM learning experiences can support the development of

student motivation and interest for STEM, thereby increasing the likelihood that they will

persist in STEM education and eventually move into the STEM pipeline (Holmquist,

2014).

Teachers’ role in facilitating students’ STEM interests. Teachers can facilitate

student interest by providing a hands-on (Nugent et al., 2010), engaging and collaborative

learning environment (Rigby et al., 1992). A teacher showing enthusiasm for their subject

(Young et al., 1997) can be significant in building students’ interest in STEM (Ejiwale,


33

2012; PCAST, 2010). However, it is important to note that lack of teacher training,

deficits in understanding the content, and the absence of general rapport with students,

coupled with outdated pedagogical practices, can negatively impact students’ decisions to

pursue STEM (Jensen & Sjausted, 2014). This is particularly important for

underrepresented groups as environments that promote stereotypes or stereotype threat

(Shapiro & Williams, 2012) can negatively affect students (Gasiewski et al., 2012;

Gonzalez & Kuenzi, 2012; Makarova, Aeschlimann, & Herzog, 2016).

Fostering interest in the classroom can be facilitated through the curriculum

(Dewey, 1913, 1916, 1937). Using a curriculum that creates interest and motivation to

pursue STEM is an important factor (Dewey, 1913, 1916, 1937; Newbill, Drape,

Schnittka, Baum, & Evans, 2015). This guides students to grow and to reflect on their

learning which can then lead to increased understanding and interest for STEM (Capraro

et al., 2013). A curriculum that promotes student engagement and provides opportunities

for collaborative hands-on learning is likely to enhance student learning experiences and

increase student understanding of a STEM topic (Gasiewski et al., 2012; Holmquist,

2014; Jensen & Sjaastad, 2014; Mohr-Schroder et al., 2014). These impactful

developmental experiences can affect an individual’s future choices and influence his or

her personal desire for learning (Dewey, 1913).

Academic facilitators work individually with students to supporting them by

guiding them in their learning, so they can achieve their academic goals independently

(Ejiwale, 2012). Teachers may benefit from assuming the role of an academic facilitator,

due to its positive effects on student learning and interest for choosing STEM careers


34

(Gasiewski et al., 2012; Holmquist, 2014; Jensen & Sjausted, 2013; Mohr-Schroeder et

al., 2014; Young et al., 1997). When educators are the primary source of knowledge

(‘sage on the stage’) instead of being an academic facilitator (‘guide on the side’), it can

negatively affect students’ engagement and interest for STEM (Jensen & Sjaastad, 2014;

Woolnough, 1994a, 1994b).

Interest specific to underrepresented groups. OST STEM activities can support

STEM interest for minority and female students (Hite et al., 2018; NRC, 2015). This is

important due to the interest and achievement gap with minorities and females in STEM

careers (PCAST, 2010). Mohr-Schroeder et al. (2014) found that an OST STEM activity

can support interest in female middle school students, and challenges and competitions

related to learning engineering content have been evidenced to support interest in STEM

learning for minorities and female students (Brophy, Klein, Portsmore, & Rogers, 2008).

Female students’ interests in math and science can be negatively affected by

stereotypes (Shapiro & Williams, 2012). Stereotypes negatively affect longer-term

persistence in women, as well as minorities, in the STEM pipeline (Beasley & Fischer,

2012; Shapiro & Williams, 2012). The background of an instructor (i.e. race and gender)

and its similarity to the students’ background (i.e. race and gender) can positively or

negatively affect students’ long-term interest in STEM (Griffith, 2010; Price, 2010;

Shapiro & Williams, 2012). Role models can support girls’ interest in STEM learning

and content, as well as STEM-related leadership programs and hands-on STEM learning

have shown to increase girls’ interests in STEM (Mosatche, Matloff-Nieves, Kekelis, &

Lawner, 2013).


35

Interest specific to middle-grade students. When middle school students’

STEM experiences use technology, science and engineering practices, and collaboration,

there is a positive increase in student engagement, interest, and attitude towards STEM

(Hayden, Ouyang, Scinski, Olszewski, & Bielefeldt, 2011; Mohr-Schroeder et al., 2014;

Nugent et al., 2010). Middle school students are more interested in STEM when learning

experiences are engaging, fun, and hands-on (Hayden et al., 2011; Mohr-Schroeder et al.,

2014; Nugent et al., 2010). Furthermore, modification of curriculum to support middle

school students’ interests has been found to support STEM learning (Ruby, 2006).

Interest specific to formal versus informal learning. Formal STEM learning

provides students the opportunity to develop an interest in a specific STEM topic through

a structured curriculum and standardized process to a variety of STEM content

(Leblebicioglu et al., 2017; Stocklmayer et al., 2010). Sadler, Sonnert, Hazari, and Thi

(2014) found that exposure to advanced STEM courses in high school has a direct effect

on increasing students’ STEM interest, as well as increasing their likelihood of increasing

their STEM career interest:

that students who take one or two years of calculus, a second year of chemistry,

and one or two years of physics in high school exhibit a significantly higher

STEM career interest, as a group, than do students who do not take these courses.

Informal STEM learning like OST STEM activities provides students the

opportunity to learn STEM content not offered in the required STEM curricula (Ayar,

2015; Matterson & Holman, 2012). OST STEM activities, such as eCYBERMISSION

(2016), have been used to attract students and increase their interest and motivation for


36

STEM, with the hope that they will pursue a STEM career (Brown, 2016). Informal OST

STEM activities can also develop an interest in STEM by supporting content and

scientific thinking that has been introduced in formal class settings (Newbill et al., 2015)

and providing more time for hands-on learning (Paulsen, 2013).

Interest specific to OST STEM activities. OST STEM activities can have a

large positive impact on students’ interest as they provide opportunities for students to

pursue their own desires for learning. When educators understand the importance of

middle school OST education and technical skills they can positively impact student

learning and interest for STEM careers by informing them of new practices and real-

world outlooks to better prepare students with 21st Century skills for the global STEM

workforce (Ayar, 2014). OST STEM activities increase positive interest in STEM

learning and science understanding (Hite et al., 2018; McNally, 2012). Overall, OST

activities can have an efficacious impact on students’ STEM interests, persistence, and

identity (Hugh et al., 2013). This affective factor is critical as students’ STEM interest is

importance factor for students to continue in the STEM pipeline (Hite et al., 2018; NRC,

2015), as well as their STEM identity (Afterschool Alliance, 2015; Archer et al., 2010;

Brown, 2016; Sahin, 2013).

By engaging students in informal STEM afterschool activities, educators can

provide students opportunities to pursue their STEM interests (PCAST, 2010). Teachers

can leverage informal experiences to provide students the opportunity to dig deeper into

STEM as well as be an extension of the classroom (Peters, 2009; Sahin et al., 2014). In

conclusion, constructivist OST STEM learning experiences can support the development


37

of student motivation, interest, and aptitude for STEM, thereby increasing the likelihood

that they will persist in STEM education and eventually move into the STEM pipeline

(Holmquist, 2014).

OST STEM activities are a direct way to improve STEM education (Brown,

2016) by developing students’ interests in STEM. Prior research has suggested that OST

program, science enrichment programs, can create a dynamic process for shaping an

individual with regard to their knowledge, productivity, and learning (Eshach, 2007;

Sullenger, 2006). Furthermore, OST STEM activities have shown to support STEM

career interests for minorities (Dabney et al., 2012). OST STEM activities, such as Girls

Who Code, eCYBERMISSION, Science Olympiad, and robotics (sumo-bots and drones),

provides students the opportunity to pursue their specific STEM interests and content

(Brown, 2016; eCYBERMISSION, 2017; FIRST LEGO League, 2018; Girls Who Code,

2017; Science Olympiad, 2016).

Persistence

STEM persistence is defined as a student’s ability to continue STEM learning and

follow along a coherent (i.e. K-20) STEM-based pathway (Sithole et al., 2017). STEM

persistence is important to the global STEM economy and future workforce (BLS, 2017;

Sithole, et al., 2017). Student STEM persistence is influenced by quality teaching, STEM

interest, prior experiences, early access to STEM learning, stereotypes, and academic

success in STEM courses (DeJemette, 2012, Sithole, et al., 2017, Sadler et al., 2014;

Wang, 2013; Anderson & Ward, 2014).

Persistence specific to STEM subjects. The majority of student STEM

persistence research has focused college students who enter college as a STEM major and


38

the factors that influence their decisions to complete a degree (in any discipline). This can

be viewed in the way Price (2010) defined persistence as “entering college with the intent

of majoring in a STEM field and remaining in a STEM field major in subsequent

semesters” (p. 3). STEM persistence is impacted by a variety of factors, including

academic achievement, prior experiences, early STEM access, curriculum, teacher

quality and more (Andersen & Ward, 2014; Graham et al., 2013; Maltese & Tai, 2011).

Yet, research on the topic of STEM persistence has been conducted from a retroactive

perspective, by examining STEM college students’ academic preparation in high school,

especially their scores on math and science tests and in advanced placement courses

(Sadler et al., 2014; Wang, 2013).

Long-term STEM persistence. Soldner, Rowan-Kenyon, Inkelas, Garvey, and

Robbins (2012) stated that one in seven students in the United States earns a degree in

science or engineering, compared to one out of every two students in China and two out

of every three students in Singapore. In the US, this lower persistence is due to a variety

of factors, such as negative stereotypes towards women and minority groups (Beasley &

Fischer, 2012), lack of success in quality math and science courses (Sadler et Al., 2014;

Wang, 2013), and lack of early access to STEM learning (DeJanette, 2012).

The lack of long-term STEM persistence towards a STEM degree in the US is

becoming an issue due to the increase in demand for STEM jobs (Carnevale et al., 2011).

Data from the U.S. Bureau of Labor Statistics (BLS) shows that STEM-based jobs are

“projected to grow to more than 9 million between 2012 and 2022. That’s an increase of

about 1 million jobs over 2012 employment levels” (Vilorio, 2014, p. 3). As of January


39

of 2017, the BLS predicts that STEM fields will continue to cultivate even more jobs by

2024, especially in the areas of computers and mathematical sciences (Fayer et al., 2017).

This increase in jobs can be seen outside of the United States, too. By 2025, the European

Union (EU) has forecasted that there will be 7 million STEM job openings (Caprile,

Palmén, Sanz, & Dente, 2015).

Persistence specific to STEM careers. Educational systems (i.e. K-12 and

higher education) need to develop students’ technical and soft skills (i.e. 21st century

skills) to support student persistence towards STEM careers (Houssain & Robinson,

2012; P21, 2015). Future employees need to be logical thinkers who are able to problem

solve creatively, using communication skills in a collaborative environment. Giving

students the opportunity to gain insight and a passion for STEM fields while developing

skills is necessary to foster positive learning experiences for students in the areas of math

and science is critical for students’ STEM persistence (Maltese & Tai, 2011).

There have been five major approaches to encouraging STEM persistence in the

field of engineering: OST programs, in-school engineering design enrichment programs,

formal K-12 engineering curriculum, engineering guest speakers, and formal engineering

teacher professional development (Reynolds, Mehalik, Lovell, & Schunn, 2009). These

activities help support students’ cognition in and affect towards STEM while improving

students’ learning of content and skills in STEM (Brown, 2016).

Persistence in STEM influenced by teachers. Teachers and curriculum can

influence students’ decisions to pursue a STEM pathway (Gasiewski et al., 2012;

Holmquist, 2014; Jensen & Sjaastad, 2014; Makhmasi, Zaki, Barada, & Al-Hammadi,


40

2012; Woolnough, 1994a, 1994b). Specifically, teachers’ pedagogical approaches and

methods, such as being a facilitator rather than a gatekeeper of knowledge, can impact

students’ engagement, interest, and more, regarding STEM (Jensen & Sjaastad, 2014;

Woolnough, 1994a, 1994b) which leads to STEM persistence. For example, teachers who

use open-ended projects in STEM subjects are more likely to produce students with a

more positive attitude towards STEM who go to college for engineering or physical

science courses (Woolnough, 2000).

Increasing STEM persistence, motivation, and interest were made apparent

through pre- and post-survey data results in a study by Reynolds et al.’s (2009) where

high school students’ interests in engineering increased after participating in multiple

classroom engineering units. Curricular changes to support students such as personally

relevant content may increase the number of students, specifically women, in STEM

courses. Also, a focus on the high-quality math and science classes can support student

STEM persistence (Bottia, Stearns, Mickelson, Moller, & Parker, 2015) as can

participation in advanced formal high school math and science courses (Sadler et al.,

2014).

STEM careers and awareness. Providing an opportunity for students to learn

and become aware of STEM careers is as central as assisting them in learning content

related to these areas (Wyss et al., 2012). One such strategy for building awareness is

activities with explicit career connections which can help to bridge STEM learning and

possible career options in STEM (Christensen, Knezek, & Tyler-Wood, 2015). Wyss et

al. (2012) found that making students aware and informed of different STEM careers


41

increased their interest and attitude towards STEM fields. In addition, Reynolds et al.

(2009) found that high school students became more interested in engineering and

associated careers after participating in engineering content and career awareness units.

Persistence specific to STEM for middle-grade students. Access to quality

instruction and performance in math courses as early as middle school can affect

students’ persistence in STEM college paths (San Pedro et al., 2014). Eighth graders who

show interest in STEM are three times more likely to pursue a STEM career than their

peers who show no interest in STEM (PCAST, 2010), which alludes to the importance of

early STEM access and engaging instruction (Mohr-Schroeder et al., 2014; Nugent et al.,

2010; San Pedro et al., 2014). Engaging STEM content, such as the e-textiles, robotics,

and maker-based projects, can increase student’s pursuit for future STEM learning

(Mohr-Schroeder et al., 2014; Nugent et al., 2010; Tofel-Grehl et al. 2017). Parent and

teacher factors also play a part in the middle school students’ persistence towards science

or other STEM areas (Bandura, Barbaranelli, Caprara, & Pastorelli, 2001; Gallagher,

1994; Wyss et al., 2012). Conversely, negative learning experiences in science and math

negatively affect middle school students’ future decision in considering a STEM pathway

(Gasiewski et al., 2012; Jensen & Sjaastad, 2014; Navarro, Flores, & Worthington,

2007).

Persistence specific to underrepresented groups. Lack of diversity in STEM

fields adds criticality to attracting students from underrepresented groups to STEM

careers, specifically females and minority racial and ethnic groups (non-white) (Anderson

& Ward, 2014). In 2010, females made up only 13% of engineering and 30% of the


42

physical science workforces; only 30% of the general science and engineering

occupations are filled by underrepresented minorities (e.g. women, blacks, and

Hispanics) (NSF, 2014). Soldner et al. (2012) surveyed 5,240 first-year college, second-

semester students (2,098 men and 3,142 women) from 46 universities in the United States

who entered their institutions with an intention to major in a STEM field and were

enrolled in a STEM discipline at the time of the data collection. They found that women

had less confidence in their STEM high school preparation than did their male

counterparts, even though they earned higher grades in high school. Women in this study

did report greater confidence that they would graduate with their specific STEM degree

and find future success in their careers. Men reported having more interactions with their

professors outside of class in non-academic situations, whereas women reported having

more academic conversations with peers outside of class. Studies have also shown that

black STEM college students have a lower likelihood of persisting in STEM major than

non-black students (Sadler et al., 2014).

STEM persistence for STEM college majors and degrees has been shown to be

lower for women and minorities due to stereotypes, lack of culturally relevant

connections, and early access to quality STEM education (Andersen & Ward, 2014;

Maltese & Tai, 2011). Furthermore, this stereotype affect along with lack of cultural

connections to the learning of the STEM content is affecting minority students in the K-

12 setting (Espinosa, 2011; Hite et al., 2018). To increase STEM persistence at the

college level, a series of high school interventions are being implemented where students

begin taking courses in math and science their freshman year; later enrolling in Advanced


43

Placement (AP) courses while taught by encouraging teachers (DeJamette, 2012; Maltese

& Tai, 2011; Palmer et al., 2011).

Although these studies are in higher education, peer support, early access to

STEM in elementary and middle school grades, through involvement in STEM-related

activities, along with culturally relevant pedagogy and bilingual education demonstrated

increases in STEM persistence for students of color and minorities (Gonzalez & Kuenzi,

2012; Palmer et al., 2011). STEM content can support students’ persistence for future

STEM learning (Mohr-Schroeder et al., 2014; Nugent et al., 2010). For example, the

introduction of e-textile content and maker-based projects in a middle science has had a

positive effect on Indian American students’ persistence and developing an interest for

STEM learning (Tofel-Grehl et al. 2017). Anderson and Ward (2014) found in their study

on the comparison of high-ability ninth grade Hispanic, Black, and White students in

STEM learning that had a higher attainment value for science had a higher likelihood of

persisting in STEM. Furthermore, Anderson and Ward (2014) showed that Hispanic

students with a higher STEM utility value, or how students feel about the usefulness of

STEM courses towards their future college or career plans, was a predictor for STEM

persistence, and higher math achievement was a predictor for STEM persistence with

black students.

Persistence specific to formal versus informal learning. The importance of

formal and informal learning activities can be found in the Every Student Succeeds Act

(ESSA) (2015), which has provided funding for local education agencies to develop

programs and activities to improve instruction and increase student engagement in STEM


44

including computer science. ESSA (2015) invests in the long-term support of STEM

education by supporting the persistence of student achievement in formal STEM courses

and providing funding and legislation for highly trained and qualified STEM educators.

Furthermore, ESSA (2015) is providing resources for informal STEM activities (i.e.

robotics) for students; early access to these types of OST STEM activities have been

shown to increase students’ preserved interest in pursuing future STEM learning (Mohr-

Schroeder et al., 2014; Nugent et al., 2010).

Extra experiences in OST STEM activities can increase motivation and interest in

STEM as a whole to support student persistence in STEM (Nugent et al., 2010). Nugent

et al. (2010) studied a group of 147 mostly male (75%) middle schoolers of diverse

ethnicities who participated in STEM summer robotics camps across six locations in rural

and urban settings in Nebraska and found that the control and treatment groups of

students both had an increase in STEM aptitude and interest, but that only the treatment

group (who met for 40 hours) had an increase in learning of content. In another case,

Mohr-Schroeder et al. (2014) discovered that 99% of their middle school students

(N=144) who participated in a summer STEM camp wanted to attend a future STEM

camp.

Persistence specific to OST STEM activities. OST STEM activities, along with

formal STEM courses, made available to middle and high school students may help

encourage students to pursue and persist at majoring in STEM degrees in college (Bottia

et al., 2015). OST STEM activities are an influential factor in students’ educational

achievement in science (McNally, 2012) and such achievement impacts STEM


45

persistence. Falk and Dierking (2010) determined that OST activities are one of the most

influential factors during students’ decision to pursue a science career. Lastly, OST

STEM activities (i.e. Girls Who Code, eCYBERMISSION, Science Olympiad, and

robotics [e.g. sumo-bots and drones]) have been shown to increase STEM career

awareness and support a STEM pathway for persistence by allowing opportunity for

interaction with STEM professionals, competing in competitions, and exposure to STEM

professional practices (Abermathy & Vineyard, 2001; Brown, 2016; eCYBERMISSION,

2017; FIRST LEGO League, 2018; Girls Who Code, 2017; Science Olympiad, 2016).

21st Century Skills

The Partnership for 21st Century Learning (2015) defines 21st Century learning as

learning activities that allow students to be creative and innovative. 21st Century skills

empower students to think critically, solve problems, communicate, and collaborate (P21,

2015). Providing students, the opportunity to learn 21st Century skills will support them

in the global economy and society for the future (Ananiadou & Claro, 2009). The Bureau

for Labor Statistics (BLS) reports explained that STEM skill sets, such as 21st century

skills (P21, 2015), align them with what is needed of STEM workers in the economy

(Vilorio, 2014). Students equipped with the skills to create and innovate in is the outcome

of a quality education that is innovative and inspiring, one that will have them ready to

compete on a global scale (Duderstadt, 2007).

21st century skills specific to STEM. Twenty-first century learning skills have an

important role in students learning STEM as STEM work requires mastery of literacy,

innovation, communication, information, media, and technological skills to be successful

in the future global economy and society (Brazell, 2013; P21, 2015). STEM education


46

uses the 21st century skills of problem-solving and critical thinking to facilitate their

understanding of the scientific method and engineering design processes (Basham et al.,

2010). Gerlach (2012) stated that STEM is “about moving forward, solving problems,

learning, and pushing innovation to the next level” (p. 2). Students interested in becoming

STEM professionals should be able to process “science, technology, engineering, or math

to try to understand how the world works and to solve problems” (Vilorio, 2014, p. 3)

and be prepared to collaborate, communicate, and be globally aware in an interconnected

world (Brazell, 2013; DeJamette, 2012; Peters, 2009; Vilorio, 2014).

Basham and Marino (2013) stated, “To be successful during STEM learning

experiences, students need to be able to move beyond low-level cognitive tasks (e.g.,

recalling facts in isolation) and gain a foundational understanding of the content, which

enables high-order thinking skills” (p. 9). Twenty-first century skills are a subset of such

higher order skills as quality STEM content learning experiences along with early access

is extremely critical to students’ outlook on and success in these STEM fields

(DeJamette, 2012).

21st century skills specific to middle-grade students. Teaching 21st century

skills is highly important in middle school due to the academic, social, and psychological

development of students at this age (Kay, 2009; Vygotsky, 1978). Students need to

develop skills to exchange global information, understand economics, and solve high-

tech problems to be ready to meet the challenges of the future (Marzano & Heflebower,

2012; P21, 2015). Project-based learning (PBL) has been used successfully to increase

21st century skills including collaboration, critical thinking, and communication in middle


47

school (Bell, 2010). 21st century skills support student engagement and learning

(Marzano & Heflebower, 2012). Learning content that provides middle school students

an opportunity to practice problem solving and teamwork supports future learning (Bell,

2010; P21, 2015).

Twenty-first century skills are essential to the functioning and succeeding in the

global STEM economy (Brazwell, 2013; DeJamette, 2012; Vilario, 2014), and early

access to these 21st century skills in elementary and middle school provides students with

the opportunity to implement and approve on them for future success (Bell, 2010;

Dejamette, 2012).

21st century skills specific to underrepresented groups. The development of

21st century skills is even more important for underrepresented groups in STEM

(Rothwell, 2013). STEM learning that involves collaboration supports cultural minority

students to learn STEM content through the development of social networking with their

peers (Anderson & Ward, 2014). Underrepresented groups, such as girls, need to access

to quality STEM learning that involves engaging 21st century learning (ESSA, 2015;

Girls Who Code, 2017) including teamwork skills, such as communication and

collaboration, and access to 21st century technology that involves critical thinking and

problem solving (Brown, 2016; Girls Who Code, 2017; P21, 2015; Science Olympiad,

2016).

21st century skills specific to formal versus informal learning. There can be a

disconnect between formal and informal learning when there is a poorly planned

connection between the two learning environments, (Scardamalia, Bransford, Kozma, &


48

Quellmalz, 2012). Furthermore, there is can be a disconnect how to effectively

implement 21st century skills with assignment between formal and informal settings, such

as technology integration and usage by students (Scardamalia et al., 2012). Students in

the formal class setting can have their 21st century learning be supported by educators

connecting their prior knowledge from their informal experiences to support future

learning in the formal classroom setting (Scardamalia et al., 2012).

The aforementioned research suggests that informal learning can provide students

the opportunity for self-directed learning and can provide students with a deeper

understanding of formal class instruction. Informal education allows students to more

deeply engage with curriculum and STEM learning through a personal approach which

increases collaboration and communication through group problem-solving projects

(Mohr-Schroeder et al., 2014; PCAST, 2010). Ayar (2015) found OST STEM activities,

such as a robotics summer camp, were different from regular science classrooms in the

areas of goals, practical work, and social structure and concluded that during STEM

learning activities, students gain 21st Century skills and knowledge, such as skills in

communication and collaboration. Furthermore, informal learning provides an

opportunity to provide students with recognition to promote their learning and skills to

the formal classroom.

21st century skills specific to OST STEM activities. OST STEM activities are

an excellent way to develop student interest in STEM and develop 21st Century skills

(PCAST, 2010; Sahinet et al., 2014). After-school STEM activities can serve a large

number of students and provide them with hands-on, inquiry-based learning (Brisson et


49

al., 2010) and can provide learners with more time for deeper learning, an extension of

their formal education, and/or remediation (PCAST, 2010). Such activities provide

students the opportunity to engage in new experiences, make mistakes while trying to

solve a problem, and address dynamic and complex problems while getting satisfaction

through constructivist activities (Zimmerman et al., 2011). These activities can promote

hands-on, challenging learning that develops 21st Century skills, knowledge, and interest

(Matterson & Holman, 2012; PCAST, 2010). Furthermore, Mohr-Schroeder et al. (2014)

found that OST STEM activities (i.e. summer robotics camp) supported an atmosphere

for teamwork through collaboration and communication.

OST activities, clubs, and competitions provide chances for teachers to share

autonomy with their students and provide learning platforms in which students can gain

teamwork skills, heighten their STEM career awareness, engage in authentic research,

hone problem-solving skills with pertinent resources, and interact with STEM

professionals (Ayar, 2015; Hughes et al., 2013; Sahin et al., 2014). Students learn

communication, collaboration, global awareness, and scientific reasoning through

engaging activities (Sahin et al., 2014). Lastly, OST STEM activities can support

students’ 21st century learning skills and possible interests in STEM careers (Hite et al.,

2018; NRC, 2015; Wyss et al., 2012).

Summary

Curriculum, instruction, and informal opportunities for learning experiences play

key roles in developing student interest and motivation in STEM learning and persistence

to pursue a STEM pathway. Understanding how students’ persistence in STEM is


50

influenced by their own motivation and participation in OST STEM learning experiences

is important for developing the STEM pipeline.

Students make a choice to pursue careers in STEM fields at a young age; for this

reason, it is important to be aware of the factors that impact their choices (Makhmasi et

al., 2012; Maltese & Tai, 2011). Teachers need to be cognizant of the negative affects

they could have on students’ STEM persistence and their science identity (Gasiewski et

al., 2012; Jensen & Sjaastad, 2014; Makarova et al., 2016; Wang, 2013; Watters &

Ginns, 2000; Sahin, 2013). OST STEM activities support the development of middle

school students STEM identity and support student interests in a future STEM pathway

(Afterschool Alliance, 2015; Archer et al., 2010; Brown, 2016; Sahin, 2013). Supporting

students’ desires for STEM learning provides them with the opportunity to develop their

motivation, interests, and persistence in STEM (Rigby et al., 1992; Ryan & Deci, 2000).


51

CHAPTER III

METHODOLOGY

This chapter will explain the research design, describe the participants of the

study, and detail the data collection process related to the previously presented

conceptual framework. Additionally, the data analysis process and the context of the

researcher can be found in this chapter.

Mixed Methods Convergent Parallel Research Design

According to Creswell (2013), mixed methods research, or using both qualitative

and quantitative research methodologies, supports neutralizing the weaknesses and

limitations of each methodology. This was completed to provide a more complete

understanding of the influence of the OST STEM activities on the students through the S-

STEM survey (FI, 2012) results through merging them with the inferences and

conclusions derived from the qualitative data. This study used a mixed methods

methodology design as the best way to answer the research questions due to quantitative

S-STEM survey (FI, 2012) not being able to capture everything needed in the research

process. Together the qualitative and quantitative methods were able to accomplish this

research need due to the limitations of the quantitative portion (i.e. S-STEM survey) not

being able to provide an understanding of the students’ prior STEM influences and

insight into their own STEM learning experiences. The qualitative research gained insight

into the students’ own experiences from the OST STEM activities through their own

words, as well as gather background information related to the students’ STEM learning.

The S-STEM survey (FI, 2012) provided statistical data with regards to the changes in the


52

students’ students’ thoughts towards STEM content areas, 21st century learning skills,

and interests in STEM careers. The quantitative study was missing the ability to provide

an understanding of the students’ prior STEM learning factors and their explanations of

their lived STEM experiences. By using qualitative and quantitative methods together,

the researcher was able to gain a deeper understanding of the impact of the OST STEM

activities on the students’ perceived STEM persistence, as well as other motivational

STEM learning factors influencing these students. By using a mixed method study, the

researcher was able negate the limitations of using only quantitative research by being

able to gather more holistic qualitative data. Furthermore, by using both methods, the

reliability and validity of the S-STEM survey (FI, 2012) was left intact by not altering the

design of the survey by attempting to add to it.

Creswell (2013) states that mixed methods research provides the opportunity to

triangulate the qualitative and quantitative data to gain a deeper and new understanding

of the phenomena that the two methodologies independently could not provide. More

evidence and therefore answers to research questions can be gathered through a mixed

methods approach compared to a single methodology to gain a deeper understanding of

the influence of the OST STEM activities on the students STEM interests, motivations,

and persistence (Creswell & Plano Clark, 2011). Furthermore, Creswell and Plano Clark

stated that mixed methods research builds a bridge between different worldviews and

research methodologies , which allowed for the researcher to obtain quantitative data on

the students’ pre and post experiences in the OST STEM activities related to their

interests, motivations, 21st century learning skills, and future STEM career interests using


53

the S-STEM survey (FI, 2012) and merged it with the coded observed experiences, self-

reported descriptive statistics, and interview data. The quantitative data provide insight

into the students’ experiences in the OST STEM activities related to their interests,

motivations, 21st century learning skills, and persistence for STEM learning. By

conducting both methodologies and merging their results, the researcher was able to

provide more perspectives of the students’ experiences in the OST STEM activities and

comprehensive understandings of the changes in middle school students’ aptitude for 21st

century skills as well as motivations, interest, and perceived persistence in STEM. The

mixed method methodology provided the researcher the ability to statistically measure

the influence of the OST STEM activities and compare it to the gained insight into the

students’ lived experiences through the researcher’s observation and the students’ own

self-reported accounts and words.

In addition, mixed methods research can be used for a variety of research

problems, especially when a need exists to explain initial results, to generalize

exploratory results, to enhance a study with a second method, to best employ a theoretical

stance, and to understand a research objective through multiple research phases. In this

study, the mixed methodology used two research methods to enhance the study and to

gain a better understanding of the influential nature of OST STEM activities on middle

school students’ interests, motivations, 21st century learning skills, and persistence for

STEM learning. A convergent parallel design allows the researcher to conduct qualitative

and quantitative methods during the same phase of the research process, which show

importance to both methods equally, while keeping each method independent during


54

analysis and then mixing the findings during the overall breakdown of the data (p. 70–

71). Lastly, the mixed methods methodology supported the understanding of the studied

phenomena. This could not have been achieved with a single methodology due to the

understanding gained surrounding students’ STEM motivation, interests, 21st century

skills, and identity. The qualitative findings helped to explain the statistical results from

the S-STEM survey (FI, 2012) findings.

This study used a convergent parallel mixed methods design based on the

researcher’s pragmatic worldview, which provided the researcher choices of philosophy,

methods, techniques, and procedures to best understand an abstract problem (Creswell,

2013), students’ perceptions of their own affect. The convergent parallel design was used

because it was the most efficient means to answer the research questions (Creswell &

Plano Clark, 2011) and it enabled the researcher to combine and triangulate the data

gathered within the semester of a school year in which the OST STEM activities were

offered (Creswell & Plano Clark, 2011). This allowed the researcher to collect data on

participants to provide an understanding of the connection between OST STEM activities

and students’ STEM persistence. The approach allowed the researcher to collect a

combination of numerical data and qualitative data on participants’ reality to gain a

deeper understanding of the students’ interests, motivation, and persistence for STEM

learning, as well as how these factors relate to the development of their STEM identity.

The qualitative portion of the study was conducted as phenomenological research,

which was used to describe the students’ experiences in the OST STEM activities

including their own interests, motivations, and factors that affect their persistence in


55

pursuing STEM activities (Williams-Watson et al., 2017; Somerville-Midgette et al.,

2015). The researcher collected data through observations and interviews in order to

explain the phenomena of interest, to better gain an understanding of the students’ self-

determination, and to pursue STEM learning through the participants’ lived experiences.

The quantitative portion of the study utilized a pre-post design to determine if there was a

significant difference in middle school students’ attitudes towards and interest in

pursuing STEM courses and activities after participating in an OST STEM activity.

This study followed the recommendations outlined by Creswell and Plano Clark

(2011), as the researcher gave equal attention to qualitative and quantitative data. The

quantitative portion of this study used a pre-post design by which students first completed

a survey on their attitudes towards pursuing STEM learning, and then participated in

STEM activities for a period of 13–15 weeks, before completing the same survey at the

end of each specific STEM activity. This pre-post design allowed measurement of

changes in participants’ attitudes and interest towards pursuing STEM learning. The

qualitative data offered insight into the students’ experiences, interest, and motivation;

observations and interviews were coded and analyzed to the constructs within the

conceptual framework (see Figure 1.1). Quantitative data were aggregated and analyzed

using paired-samples t-tests, and Wilcoxon signed-rank tests were used to analyze

individual topics within the survey. This provided an understanding of the middle school

students’ self-determination to pursue future STEM learning. Overall, the convergent

parallel design afforded the researcher an opportunity to use the qualitative and

quantitative data equally by blending them through concurrent timing, which allowed for


56

an understanding of the observations and the students’ self-reported experiences from the

interviews and descriptive statistics with a comparison to the results from the S-STEM

survey (FI, 2102).

Research Paradigm

The qualitative portion of the research was designed to gain an understanding of

the affective reasons why students choose to participate in a STEM experience and how

the STEM experience impacts students’ decisions for the future, especially their interest

in STEM fields. The qualitative methodology was appropriate for gaining insight into

students’ motivation and how their personal experiences and insights influenced their

motivation. The data collection tools, which included surveys (i.e. S-STEM survey and

demographic questionnaire), interviews, and observations, permitted the researcher to

take a triangulated approach to gain knowledge and understanding of this topic as well as

laid the groundwork for future replication. In the qualitative paradigm, truth, value,

applicability, consistency, and neutrality were applied to the study to ensure

trustworthiness (Lincoln & Guba, 1985).

The qualitative portion of the convergent design process was used to answer the

qualitative research question and its sub-questions (Creswell, 2013). The

phenomenological research was most appropriate because of the participants’ shared

experiences in the OST STEM activities, the nature of gathering information through

multiple data sources to gain understanding, and an explanation of these specific OST

STEM courses through the participants’ perceptions (Creswell, 2013; Marshall &

Rossman, 2016). Using a variety of data sources and viewing the data through a variety


57

of lenses allows for multiple facets of the unique situation to be revealed and understood,

therefore the qualitative research was conducted to explore these specific OST STEM

activities and explain the influence they had on students’ expressed interest in furthering

their STEM knowledge in future courses (Baxter & Jack, 2008). The researcher was a

participant observer at the school, as he was teaching a portion of the students in the

study. The demographic questionnaire, interviews, and observational data collected were

coded to identify themes. The qualitative data collection instruments can be found in

Appendices E, F, and G.

Challenges. Creswell and Plano Clark (2011) explain that there are three major

challenges facing researchers conducting mixed methods studies: experience and skills,

time and resources, and convincing others of the importance of mixed methods research

(Creswell & Plano Clark, 2011). In this study, the schedule of the activities was the

greatest challenge, as the specific OST courses at the school only met for a limited time

on specific days. Prior planning and maintaining regular communication with the

administration, educators, and leaders of the OST activities to keep abreast of any

changes in time or dates helped to address this challenge.

Research Questions

Below are the research questions that guided this study. Upon participation

(before to after) in a program for OST STEM activities, how did this intervention:

1. change middle school students’ perceptions (descriptions) of and actions



58

a. Type and number of current middle school STEM courses in their formal

schooling?


2. alter middle school students’ 21st century learning skills, motivation, and


Context of the Participants

The participants for the quantitative portion were 37 middle school students (16

females and 21 males) in sixth (N=5), seventh (N=18), and eighth (N=14) grades, all of

whom participated in OST STEM activities and courses at an independent private school

in a metropolitan city in the Southeastern United States. This college prep school serves a

student population from kindergarten through 12th grade. In order to attend, students

must go through a formal application process, get accepted to the school, pay tuition, and

maintain the schools behavioral and academic expectations.

The school’s mission is to provide its students with engaging, growth-promoting

learning experiences through academics, sports, and more to foster holistic development.

Additionally, the school strives for students to be servant leaders in one of the various

service-learning programs sponsored by the school. The school is a National Blue Ribbon

school in their elementary, middle, and high school settings. Lastly, the school is member

of the National Association of Independent Schools (NAIS).

Thirty-seven students who attend the aforementioned school participated in the

study. Some students participated in multiple OST STEM activities. For example, one

student participated in Science Olympiad and Girls Who Code, while another two


59

students participated in Science Olympiad and eCYBERMISSION. Fifteen of the 18

students who participated in robotics participated in more than one of the offered robotics

activities, SeaPerch, sumobots, and drones. The SeaPerch program was offered before the

study began and was not observed during the study but was referenced by the robotics

students in the interviews and demographic questionnaire (descriptive statistics).

A nonrandomized process was used to identify a purposeful sample of

participants who were selected based on their participation in OST STEM activities.

Creswell (2013) recommends that qualitative phenomenological studies should have five

to 25 participants. Fifteen middle school students selected of the 37 student participants

(i.e. 11 from Science Olympiad, 6 from eCYBERMISSION, 2 from Girls Who Code, and

18 from Robotics activities) also participated in the quantitative portion of the study. The

larger number of students for the quantitative research, as compared to the number of

students for the qualitative aspect of the research, supports the validity of the statistics

(Creswell & Plano Clark, 2011). A purposefully selected sample was invited to complete

the surveys. Purposeful sampling was used to select an equal representation of girl and

boy students from different OST STEM groups and grade levels for the interviews to

provide insight into each STEM activity’s focus area and students’ perceptions of the

activity. Table 3.1 shows the nature of each studied STEM OST intervention with the

researcher’s data collection and participant involvement in the research.


60

Table 3.1

Length and Duration of OST STEM Activities and Data Collection

OST STEM Activity

Regularity of OST STEM Activities

Duration of Intervention (in weeks)

Observations (N = 78)

Student Participation in OST STEM

Activities (N = 101)

Survey (N = 37)

Interview (N = 15)

Science Olympiad

1 mandatory meeting a week, and 5 optional meetings during lunch

13 25 11

eCYBERMISSION

1 mandatory meeting a week, and 5 optional meetings during lunch

16 13 6

Girls Who Code

2 times a week before school

15 8 2

Robotics 2 mandatory meetings a week, and 5 optional meetings during lunch

15 32 18

Sumobots

27 18

Drones 5 10

The demographic questionnaire used to develop descriptive statistics gathered

general background information about the participants in the study. These descriptive


61

statistics were essential in gaining background information (i.e. age, grade, gender,

family members in STEM fields, involvement in STEM activities) and provided insight

into the students’ STEM interests and motivations through student-written descriptions.

For the interviews, a total of 15 students, seven girls, and eight boys, were interviewed:

eight from Robotics OST activities; three from Science Olympiad (2017) and

eCYBERMISSION (2016), three from Science Olympiad (2017), one from

eCYBERMISSION (2016) and one from Girls Who Code (2017). Furthermore, two

students interviewed participated in both Science Olympiad (2017) and

eCYBERMISSION (2016), as well as four students participated in sumobots and drones,

and four students only participated in sumobots. Also, four students from the interviews

participated in the prior robotics activity, SeaPerch, before the study and referenced this

activity in the interviews; 12 of the 18 students from the robotics group participated in

SeaPerch prior to the study. Of the 15 students interviewed, two were in sixth grade,

seven were in seventh grade, and six were in eighth grade. Seven students of the 15

interviewed were in more than one OST STEM activity. Fewer six graders (N=5) were

interviewed because there were fewer overall sixth graders participating in the study.

Maximal variation sampling was used to select participants who had unique

characteristics to provide diverse perspectives of the experience (Creswell & Plano Clark,

2011).


62

Data Collection

The following section discusses the data collection process. This section includes

the qualitative data, quantitative data, and mixed methods data collection in addition to

the mixed methods interpretation processes.

General Data Collection Procedures

Researching middle school students (ages 12–15) in the formal operational stage

of their cognitive development participating in OST STEM activities can provide insight

into students’ reasoning, logical independence, and functioning in their STEM learning

(Piaget, 1972). Furthermore, the evolution of the participants’ abstracting thinking and

their attitude towards and formal reasoning for science was important for this research.

The data were collected in a systematic process by having the participants take the pretest

(i.e., quantitative S-STEM survey), as well as a qualitative demographic questionnaire

(descriptive statistics), at the beginning of the data collection process in January of 2017.

The qualitative interviews and observations then were conducted during the activities.

Lastly, the quantitative post-test S-STEM survey was administered at the completion of

the OST STEM activities at the beginning of May.

Qualitative Data Collection

Qualitative data was collected over the course of about a semester (i.e. January-

May 2017). The schedule for the OST activities varied, but the majority of activities took

place after and before school. The qualitative data was collected to gain insight into the

students’ experience and thoughts with regards to STEM learning and their OST STEM

activities through their responses to interview and questionnaire questions, as well as be

able to observe the students’ experiences in action during the OST STEM activities.


63

Observations. The researcher took field notes of the observed student

engagement in the OST STEM activities on the Observation Tool (see Table H), which

was used as a guide for various aspects of the observed activities. The observation

protocol was developed using the research questions as a guide and considered the

conceptual framework. Each question considered 21st century skills, motivation, and

interests of the students in STEM learning and persistence through the OST STEM

activity. The observations provided the researcher the opportunity to view the students’ in

action during their OST STEM activities and provide insight into their experiences.

Furthermore, it allowed the researcher to be able to compare the observed data with other

forms of data collected to provide a holistic understanding of the phenomenon being

studied. The observation protocol tool was the best choice for the researcher to be able to

see the students’ participation in their OST STEM activities.

Interviews. Interviews were conducted with student participants and lasted no

longer than 15 minutes. The interviews were conducted in a separate room during the

time the OST STEM activity was being implemented. These interviews were guided by

the Interview Tool (Appendices G). The interview questions were developed using the

research questions as a guide and considered the conceptual framework. Each question

was used to look at 21st century skills, motivation, interest, and persistence in future

STEM learning through the students’ eyes. The interviews provided the researcher the

students’ own thoughts, ideas, and words related to the influence of the OST STEM

activities and their prior STEM learning experiences. Furthermore, it allowed the

researcher to be able to answer the research questions with a better understanding of the


64

students’ thoughts and opinions with regards to 21st century skills, motivation, interest,

and persistence in future STEM learning. The interview protocol was the best tool for

gaining insightful data from the students’ perspective in their own words.

The researcher’s personal iPhone (password-protected) was used to record the

interviews so that all student responses would be gathered accurately. The interview

timeline was determined after scheduling a time to interview students with the Assistant

Head of Middle School. The interviews did not take place during teacher whole-class

instruction, nor did they interfere with one-on-one tutoring of the participants. The

researcher conducted the interviews with minimal disturbance to the participants’

environment to protect their identities. Interviews were either conducted during

transitions between activities, at a specifically scheduled time, or when the student could

be approached privately for an interview. During the interviews, the research member

checked the responses of the interviewed participants by restating the interviewee's

responses for clarification and by asking follow-up questions.

The researcher transcribed the audio recordings within one week of the

interviews. This time frame helped the researcher recall expressions, tone, body

language, and other elements that were not captured audibly. The transcribed interviews

were saved using the students’ school identification number, as a way of de-identifying

the data.

Hard copy written notes from the interviews were shredded and digital files were

deleted, including the audio recordings of the interviews captured on the password-

protected iPhone, once the Texas Tech University’s Institutional Review Board (IRB)


65

timeline ended at the completion of the study. Notes on the Interview Tool (Appendix F)

which were used to help guide the interview used to capture data throughout the

interview were also deleted, as were the interview and observation tools were used during

the interviews (N=15) and observations (N=78) of students engaged in OST STEM

activities.

Demographic questionnaire. The demographic questionnaire (see Appendix G)

was used to gather general background information about the participants at the

beginning of the study. The demographic questionnaire provided descriptive statistics

including age, grade, gender, family members in STEM fields, current STEM activities,

prior STEM activities, motivation for joining their OST STEM activity, interests

connected to their OST STEM activity, about each of the participants so the researcher

could gain an understanding of the students’ prior experiences and background. The data

from the demographic questionnaire was used apart from the quantitative research

methodology and was as coded. The demographic questionnaire was used to gain

information about the students’ STEM experiences, influential background information,

possible motivational factors, and STEM interests to help answer the research questions.

It was the best protocol for gathering this type of information. Lastly, the questionnaire

provided the researcher with another resource to better understand the influence of OST

STEM activities on the middle school students’ overall persistence towards STEM

learning and their STEM identity.

Quantitative Data Collection

Quantitative data were collected using the Student Attitudes Toward STEM (S-

STEM) Survey for Middle and High School Students (FI, 2012) from the Maximizing the


66

Impact of STEM Outreach (MISO) project (see Appendix P). This instrument was

developed by the Friday Institute for Educational Innovation (2012), an educational

research center at North Carolina State University through a National Science Foundation

grant “to measure changes in students’ confidence and efficacy in STEM subjects, 21st

century learning skills, and interest in STEM careers” (MISO, 2011, para. 5). This survey

was selected because it provided the researcher the ability to compare students’ pre and

posttest results, including students’ perceived interest and motivation for STEM,

development of the students’ 21st century skills, and self-described future persistence in

STEM careers based on the intervention of the OST STEM activity. This instrument

supports the conceptual framework of STEM interest, motivation in STEM, persistence

in STEM, and aptitude for 21st century skills to gain an understanding of the influence of

the OST STEM activity on the students. The S-STEM Survey provided the researcher

with the ability to statistically measure the influential impact of the OST STEM activities

on the students’ persistence, motivation, and interest for STEM learning. This statistical

instrument was used to be able to show the possible influence of the OST STEM

activities on the middle school students through changes in their students’ perceived

interest and motivation for STEM.

The MISO site (2011) explains that the survey’s three major sections were

developed based on other surveys and national information. The STEM constructs were

developed by adapting them from a “survey created by evaluators of a program at the

engineering schools of Northeastern University, Tufts University, Worcester Polytechnic

Institute, and Boston University” (para 3). The survey’s 21st century skills section was


67

developed from the North Carolina Student Learning Conditions Survey (2010). Lastly,

the STEM career section was derived from national STEM agencies including the

National Academy of Engineering (MISO, 2011).

The second revision of the S-STEM Survey was administered to approximately

9,000 middle and high school students. After conducting validity and reliability testing of

the survey, the factor analysis showed the instrument to be highly reliable and capable of

measuring the intended constructs clearly. The construct reliability levels for the S-STEM

survey, measured with a Cronbach’s Alpha, are the following: Math Attitudes 0.9,

Science Attitudes 0.89 Engineering and Technology Attitudes 0.89, and 21st Century

Learning Attitudes 0.89 (FI, 2012). The FI explained that further research on the survey

showed the survey to be of “appropriate length and at appropriate reading-levels” (MISO,

2011, Appropriate Uses section). The MISO recommends a minimum of 30 student

subjects for the use of the S-STEM survey for purpose of validity (see Appendix N).

Permission was granted from the FI for the researcher to use the survey (Appendix H).

The quantitative data used in this study were reported by student participants and

recorded as an ordinal rank between 1 and 5. The S-STEM Survey asks respondents to

indicate the degree to which they agree or disagree with a series of statements; these data

were used to “measure changes in students’ confidence and efficacy in STEM subjects,

21st century learning skills, and interest in STEM careers” (FI, 2012). The ordinal values

for the content sections, which measure self-reported perceptions of knowledge of STEM

subjects and acquisition of 21st century learning skills, range from 1 to 5: 1 (Strongly

Disagree), 2 (Disagree), 3 (Neither Agree nor Disagree), 4 (Agree), and 5 (Strongly


68

Agree). The items in the Your Future section of the survey measure self-reported interest

in STEM careers and use ordinal values as well: 1 (Not at all Interested), 2 (Not so

Interested), 3 (Interested), and 4 (Very Interested). Lastly, the About Yourself section asks

questions about students’ future academic school endeavors. The first question asks,

“How well do you expect to do this year in your (English/Language Arts Class? Math

Class? Science Class?)” offering the following ordinal options: 1 (Not Very Well), 2 (OK

/ Pretty Well), and 3 (Very Well). Other related items ask, “In the future, do you plan to

take advanced classes in (Mathematics? Science?)” with the ordinal options of 3 (Yes), 2

(Not Sure), and 1 (No). Finally, the survey concludes by asking, “Do you plan to go to

college?” The response choices for this question are: 3 (Yes), 2 (Not Sure), and 1 (No).

This section, following the same format, asks questions regarding STEM-based career

aspirations.

The survey was administered in a pre-post fashion to determine the effect of the

OST STEM activities on the students’ STEM persistence. The researcher administered

the pretest at the beginning of the semester to each student involved in an OST STEM

activity. The posttest was administered at the end of each OST STEM activity’s meeting

cycle. The pretest was administered in February, 2017 with each of the OST STEM

groups and the posttest was completed by the first week of May, 2017. Approximately

four months transpired between the administrations of the pretest and posttest, depending

on the duration of each OST STEM activity (please see Table 3.2).


69

Table 3.2

Amount of Weeks Between the Pretest and Posttest

OST STEM Activity Pretest Posttest

Approximate Time Between (weeks)

Science Olympiad 2nd week of Feb. 1st week of May 13 weeks

Girls Who Code 2nd week of Feb. 3rd week of May 15 weeks

Robotics 2nd week of Feb. 3rd week of May 15 weeks

eCYBERMISSION 2nd week of Feb. 1st week of May 13 weeks

The survey was recreated using Google Forms and a link to the survey was

distributed to participants via email. This enabled them to complete the pretest and

posttest using an internet connected computer. The students posted their school

identification codes in the survey so the researcher could match their pretests with their

posttests.

Data Analysis

The following section discusses the data analysis process. This section includes

qualitative data, quantitative data, and mixed methods data analysis. The mixed methods

interpretation processes are also discussed.

Qualitative Data Analysis

The focus of the analysis of the qualitative data collected is to identify specific

trends and recurring themes. The analysis of the observations and interview data helped

the researcher better understand middle school students’ prior learning experiences,

motivations, and perceptions of how instructional/curricular practices affect their pursuit


70

of STEM learning. The demographic questionnaire provided insight into the students’

backgrounds. From this analysis, the researcher gained insight about how the sampled

OST STEM activities augmented students’ interest, attitudes, and motivations in STEM

as well as engaged with their prior experiences in STEM and what factors influenced

students’ perceived interest in pursuing future STEM courses. Each piece of qualitative

data collected provided the researcher the ability to compare the data from the observed

viewpoint of the researcher to the students’ own words from the interviews to important

background information on the middle school students through the questionnaire that

could be impacting their STEM learning pathway. The analysis of these three different

pieces of qualitative data provided a holistic understanding of the students’ experience

and was the best protocol to gain an understanding of this phenomena. This could not

have been done with each portion on its own.

Data from the demographic questionnaire, interviews, and observations were

coded for emerging themes using QSR International’s NVivo 11.1.1 Software (2018),

which allowed for advanced coding and reorganization of codes (Rossman & Rallies,

2003; Swanborn, 2010). This facilitated a systematic process for triangulating the data

collected for analysis (Creswell, 2013). The researcher used the conventional pattern

analysis coding approach. The conceptual framework of OST STEM factors of

motivation, interest, and engagement, related to self-determination theory, concerning

middle school students along with identified STEM persistence factors was used as a

guide for the coding process. This supported prioritization for the categories and themes

created during the coding process. The researcher used in QSR International’s NVivo


71

11.1.1 Software (2018) to develop emerging codes and themes through multiple iterations

of analysis by open coding themes in the data then developing patterns to support the

emergence of codes and themes. Throughout the inductive open coding processes,

categories evolved as they were combined and separated (Saldaña, 2016). The coding

process included coding the student participants’ descriptions and the researcher’s

recorded observations.

The researcher followed Creswell’s (2013) 6-step process for qualitative analysis

of the data and reviewing the data multiple times for thoroughness:

1. Organize and prepare the data for analysis.

2. Read or look at all of the data.

3. Start the coding of the data.

4. Use the coding process to generate a description of the setting or people as

well as the categories or themes for analysis.

5. Advance how the description and themes are represented in the qualitative

narrative.

6. Interpret the findings or results. (p. 197–200).

The coding allowed for the organization and capture of the meaning from the data

(Swanborn, 2010). Each code was organized by mutual relationship (Swanborn, 2010).

During this process of assigning codes and identifying themes, the research questions

were taken into consideration to guide the analysis (Rossman & Rallies, 2003). The data

analysis of the interviews, descriptive statistics, and observations led to five major

themes: Supporting Student’s STEM Persistence (N=203), Developing STEM Skills and


72

Content (N=111), Experience Levels (N=59), Not Sure About a STEM Future (N=52),

and Sources of Motivation (N=428).

Trustworthiness. Trustworthiness was supported through the collection of

multiple types of qualitative data (Erlandson et al., 1993; Lincoln & Guba, 1985;

Kincheloe, 2001). The researcher established a process and followed it for data collection

and analysis through credibility, transferability, dependability, and confirmability, to

establish trustworthiness (Kincheloe, 2001; Lincoln & Guba, 1985).

Credibility. Triangulation of the data sources and methods supported the

credibility of and confidence in the findings (Lincoln & Guba, 1985) of the study. Other

means of ensuring credibility include persistent observations as a data source and

clarifying students’ responses during the interview process by clarifying what was

recorded with the interviewee.

Transferability. Providing clear and precise steps for conducting the research

supported transferability, or applicability of findings in other contexts (Lincoln & Guba,

1985). Furthermore, describing the context of the research and the assumptions that were

made with regard to the research supported the transferability (Trochim, 2000). The

researcher also gave clear and thick descriptions of the interviews and observations to

support transferability rather than making broad generalizations, allowing connections to

be made in other contexts by the readers (Lincoln & Guba, 1985; Erlandson et al., 1993,

p. 33).

Dependability. Dependability, or the consistency of findings, was established by

having participants complete web-based questionnaires (i.e. S-STEM survey and


73

descriptive statistics) which limited researcher analysis error during data collection.

Furthermore, the researcher focused on documenting observations and detailed interviews

accurately. Protocols such as the Interview Tool (Appendix F) and the Observation Tool

(Appendix E) were used to record data and document evidence of students’ attitudes,

interest, and motivation demonstrated during their engagement in the OST STEM

activities (Trochim, 2000).

Confirmability. Through the use of clear research practices, including following a

systematic mixed methods methodology ascribed by Creswell (2013) and Creswell and

Plano Clark (2011) for the data collection and conducting an audit trail for data analysis

the researcher, established confirmability (Lincoln & Guba; 1985). The researcher

documented the procedures for checking and rechecking the data throughout the study

(Trochim, 2000). To that end, the researcher conducted a data audit of his work after the

data analysis to make sure that all data had been represented and analyzed accurately

(Lincoln & Guba; 1985); this audit is in Appendix L.

Quantitative Data Analyses

All responses to grouped Likert-scale items were transformed into numerical data,

at the construct level, to compare pretest-posttest data. The quantitative analysis used

descriptive statistics with the measures of central tendency and dispersion. The paired

means t-test and the Wilcoxon test were used to determine if the OST STEM activities

measurably influenced the students’ STEM persistence as a group and on the individual

level. By using these two statistical tests, the researcher was able to compare the students’

pre and posttest findings to measure the possible influence of the OST STEM activities

by the whole group and each individual, as well as for each grade level and gender. A


74

paired means t-test was used to determine the impact of the OST activity on the students’

STEM persistence and attitudes. The survey was validated at the construct-level, not at

the item-level; since the comparisons were made at the construct-level, the paired-means

t-test is appropriate. However, at the item-level where data is ordinal, the Wilcoxon

signed-rank test was used for comparisons of the pretest and posttest data for individual

comparisons. These tests were chosen as the best protocols for measuring the

comparisons of the pre and posttests at the item and construct levels to answer the

research questions. The Wilcoxon signed-rank test was selected to measure students’

STEM persistence, which is a complex construct, through the use of the factors of

changes in students’ confidence and efficacy in STEM subjects, 21st century learning

skills, and interest in STEM careers based on their responses on the S-STEM survey (FI,

2012). The following three assumptions were met when using a Wilcoxon signed-rank

test (conducted using IBM SPSS [24]):

Assumption #1: The dependent variable is on an ordinal scale.

Assumption #2: The independent variable of related groups indicates that the

same subjects were present in both groups.

Assumption #3: The distribution of the differences between the two related

groups is symmetrical in shape.

Validity and reliability. Steps were taken throughout the study to confirm

validity in the quantitative methodology for this mixed methods study. The quantitative

research’s validity was supported by the fact that the dependent variable was continuous

and by the pretest-posttest comparison of the independent variable of STEM persistence.


75

The FI has reported having high reliability with regard to factor analysis having clear,

high constructs (MISO, 2011, Development Uses section). The reliability results

conducted by the researcher to determine the internal consistency of the survey concluded

that the construct reliability levels, measured with Cronbach’s alpha, were 1.0 for Math

Attitudes, 0.78 for Science Attitudes, 0.8 for Engineering and Technology Attitudes, 0.88

for 21st Century Learning Attitudes for the pre-survey results. Next, the researcher

measured with Cronbach’s alpha to determine the item reliability levels, which were -

1.01 for Math Attitudes, 0.73 for Science Attitudes, 0.79 for Engineering and Technology

Attitudes, 0.88 for 21st Century Learning Attitudes for the pre-survey results.

Furthermore, the item reliability levels, measured with Cronbach’s alpha for the post-

survey results, were -0.83 for Math Attitudes, 0.8 for Science Attitudes, 0.93 for

Engineering and Technology Attitudes, 0.91 for 21st Century Learning. The math item

results concluded that the math items to be unreliable due to the negative correlation of

the questions, but questions six and seven in the math section did meet the reliability

expectation. All of the questions on the item-level met the requirement of >0.7 reliability

value for the Cronbach’s alpha measure, except for the Math Attitudes (see Appendix Q).

Lastly, the quantitative survey data met the three assumptions that are required for the

Wilcoxon signed-ranked test to give valid results. The researcher spoke with a

representative from MISO through multiple emails (see Appendix I), about the validity of

the number of student participants with regard to the use of the S-STEM survey; the

representative explained that a minimum of the 30 subjects is recommended based on

their testing (Taylor, personal communication, May 11, 2017).


76

Mixed Methods Data Analysis

The mixed methods data analysis used a side-by-side comparison approach where

the researcher first reported the quantitative statistical results and then discussed the

qualitative result to help confirm or refute quantitative findings (Creswell, 2013; Creswell

& Clark, 2011). This allowed for both sets of the research data to be collected and

analyzed separately and then merged for a final interpretation (Creswell, 2013). By

merging the data and comparing the quantitative results and the qualitative findings, the

goal was to gain a more robust understanding of the findings (Creswell & Clark, 2011).

This provided the researcher the ability to compare the statistical findings of the form the

S-STEM survey with the qualitative findings of the students’ own words and self-reported

information along with the observed experiences of the students, which gave the

researcher the ability to better understand the data holistically.

First, the researcher cleaned the statistical data before entering it into SPSS. Next,

the researcher completed the Wilcoxon statistical analysis test for the item level data that

is appropriate for ordinal data. A t-test was completed for the each of the S-STEM survey

categories after aggregating items into numerical sets, which is appropriate for interval

data with regards to the pretest-posttest comparison by averaging the data for a validated

construct. The quantitative results were then reported.

The qualitative data was entered into NVivo and open coded using to develop

themes in the data as described in the Qualitative Data Analysis section. The codes were

analyzed and regroup to create overachieving groups with subgroups. The qualitative

results were then reported.


77

Lastly, quantitative and qualitative results were merged to provide a final

explanation of the overall findings, which are discussed below. This information was

used to answer the research questions.

Mixed methods interpretation. The mixed methods interpretation of the

qualitative and quantitative research was compared in a convergent parallel method to

develop a discussion of the findings through “congruent and discrepant evidence between

the databases” (Creswell & Clark, 2011, p. 232). This allowed the researcher to be able to

understand the outcomes of the quantitative analysis of the S-STEM survey data using the

paired means t-test and the Wilcoxon test when compared to the qualitative interview,

questionnaire, and observational coded data. The researcher was able to explain the

results through comparison of each type of data. The researcher was able to interpret the

findings from the S-STEM survey (FI, 2012) using the themes discovered from the

qualitative results, which lead to answering the research questions and gaining an

understanding of the studied phenomenon. The qualitative and quantitative data were

allowed to tell their stories, which enable the researcher to identify similarities and

differences between them (Creswell & Clark, 2011).

The mixed methods data was analyzed using a side-by-side comparison approach

in which the quantitative statistical results were first reported and then the qualitative

findings were discussed (Creswell, 2013; Creswell & Clark, 2011), then the results were

interpreted by using the research questions and constructs as a guide to determine the

outcomes of the findings. This process allowed the researcher to gain an understanding of

the statistical results of the S-STEM survey (FI, 2012) through the students’ lens using the


78

interviews and descriptive statistics along with the researcher’s observations. Qualitative

data helped explain the why and how of the large survey datasets and provide a deeper

interpretation of the quantitative findings (Crede & Borrego, 2010). This confirmed and

disconfirmed some of the qualitative results through the use of comparing the qualitative

findings, giving way a deeper explanation of the overall findings of the phenomena

studied (Creswell, 2013; Creswell & Clark, 2011). Finally, the parallel-databases variant

brought the two parallel strands, which were conducted independently, together during

the interpretation to synthesize the results to examine the studied phenomenon to provide

a complete understanding of it (Creswell & Clark, 2011). These steps provided the

researcher the ability to understand the influence of the OST STEM activities and the

students’ STEM identities by comparing the S-STEM survey findings to the students’

self-reported experiences and the researcher’s observations.

Potential Ethical Issues

When working with human subjects, particularly protected populations like

children, it is important to conduct research with the utmost respect and integrity. In

accordance with federal guidelines on human subjects research, the following precautions

and steps were taken to adhere to these standards of human subject research.

Protection of Research Participants

The Texas Tech University Intuitional Review Board (IRB) approved this

research, along with supporting materials, on September 22, 2017. The approved IRB

(IRB2017-131) may be found in Appendices A-L. Upon approval, the researcher gained

permission from the student participants as well as their legal guardians/parents.

Permission was attained through the acquisition of signatures on the consent and assent


79

forms, in which the purpose of the study and the role of the participants in the study were

explained.

Participants’ trust. The researcher built trust with the student participants by

openly explaining each step of the research study process and explaining why the

researcher was doing the study. The students were asked to take the surveys, to be

interviewed, and it was explained to them why they were being observed during their

activities. Furthermore, trust was built by asking permission politely at each step of the

study and by gaining personal consent from the students and their parents for

participation in the study.

Research integrity. The research was conducted with integrity. The researcher

secured permission from the study school to conduct the research study prior to receiving

approval from the IRB at Texas Tech University. The researcher also received permission

from the school’s headmaster to conduct the study prior to recruiting the student

participants. All students who participated in the OST STEM activities at the independent

school were provided with a recruitment letter (Appendix A), consent form (Appendix

B), student assent form (Appendix C), and an information sheet (Appendix D). All

materials were distributed to the parents and students through the middle school. The

Assistant Head of the Middle School took the lead in distributing the forms in a sealable

envelope. This Assistant Head of the Middle School was asked to read the information

sheet to the middle school students, to describe the study and to recruit participants. The

information sheet was read to the students at their activity meeting, after which the

Assistant Head of the Middle School distributed the student assent forms, parent consent


80

forms, information sheets, and return envelopes. A drop box was placed in a public area

for parents to drop their sealed envelopes containing the signed consent and assent forms.

The information sheet informed the parents of the location of the drop box and instructed

them to place the signed assent and consent forms into the drop box. The researcher

collected the drop box twice a week to gather the documents. The letter and forms were

distributed a month before the study began. This allowed for ample time for parents to

sign and submit the forms as well as to give apply time to collect the forms. These steps

were taken so as to not compromise the identity of the participants.

All consent and assent forms were locked in a filing cabinet behind a locked door

of the researcher’s office. The Assistant Head of Middle School distributed the URL of

the digital survey (S-STEM survey) and questionnaire (descriptive statistics), created

with Google Forms, to the students via email. The researcher provided this administrator

the students’ names from the consent and assent forms. This step was taken to keep the

students’ identities confidential, unknown to their instructors and peers.

The researcher obtained approval from the IRB before collecting any data and

followed the procedures as they were presented on the assent and consent forms. During

the study, the students used their school identification codes to mark the pre and post

surveys. These codes were then used to match the pre-post survey responses. Data was

stored on the researcher’s personal computer and personal iPhone, both of which are

password-protected. Identities of all participants and the of the school were kept

confidential. The school was not identified by name or location, nor will it be disclosed in

any publication on the study. In fieldwork and data analysis, each student participant was


81

referred to by letter code. The researcher coded the data and used pseudonyms for

everyone involved in the study. The students’ school identification descriptor was used

for the pre-post survey data collection, but will not be disclosed in any publication on the

study. Additionally, a peer review of the research findings was conducted. The researcher

disclosed the research process that took place.

Issues of personal privacy. The researcher coded the data and used pseudonyms

for everyone involved in the study. All documents, notes, and recordings of interviews

with participants were either locked in a file cabinet in the researcher’s office or stored on

the researcher’s password-protected computer or personal iPhone. The researcher will

protect the identity of the participants, instructors, and the school by not referring to them

by name or disclosing identifiable information. Findings will not be presented in such a

way that any reader would be able to identify the study’s participants.

Researcher’s Resources and Skills

The resources the researcher used are derived from the knowledge and skills

gained over 4 years of doctoral course work, research, and experience. The researcher

conducted statistical analysis on quantitative data, using IBM SPSS (Version 24). The

qualitative coding was carried out using QSR International’s NVivo 11.1 Software.

Google Forms was used throughout the study.

Context of the Researcher

At the start of the study, the researcher had just begun his 13th year of teaching.

At the time of writing, the researcher was an engineering teacher at an independent-

college preparatory school, the robotics coach, a member of the technology committee,

and was developing collaborative projects with other teachers to create cross-curricular


82

engineering units in various content areas. The researcher had gone through a variety of

engineering education training, including Project Lead the Way and Fab Academy

certifications. Furthermore, the researcher has been a middle school STEM teacher for his

whole career and has taught in public and independent schools. This information is

important to the study due to the technical understanding and prior experiences with

STEM learning and content, as well as leading OST STEM activities. The researcher’s

prior experiences with teaching middle school students and leading OST STEM activities

provided him with a strong understanding of what the middle school students were doing

in their OST STEM activities.

Lastly, the researcher is a believer in constructivist learning theory, specifically

social constructivism, and uses social constructivist learning theory to direct his own

teaching. The researcher greatly values the learning that occurs out of human interaction

the impact such interaction can have on a child’s academic success. Since the OST

STEM activities in this study have a constructivist learning approach to them, they

provided the researcher with a stronger understanding of the activities.

Summary

To gain an understanding of the impact of OST STEM activities on student

STEM persistence, this study used a mixed methods convergent parallel design. Both

qualitative and quantitative data were collected at the same time, yet independently; the

analyses and interpretation were conducted for each set of data (Creswell & Plano Clark,

2011). The qualitative data was analyzed to identify codes and themes, and the

quantitative data analysis was carried out using a paired-samples t-test and a Wilcoxon


83

signed-rank test. Quantitative data includes the responses to the S-STEM Survey for

Middle and High School Students (FI, 2012), collected before and after engagement in

OST STEM activities, which were statistically analyzed using a paired-samples t-test at

the construct-level and a Wilcoxon rank-test for the item-level after coding the Likert-

scale items. The qualitative data included observations, interviews, and demographic

questionnaire information (descriptive statistics). The researcher used QSR

International’s NVivo 11.1.1 Software (2018) to develop the emerging codes and themes

through open coding and multiple iterations of analysis. The coding process used open

coding and was followed by the development of patterns to support the emergence of

codes and themes. The qualitative and quantitative data were then merged to provide

insight into affective factors that influenced students’ persistence and 21st century skill

growth based on participation in OST STEM activities (Creswell & Plano Clark, 2011).


84

CHAPTER IV

RESEARCH RESULTS

This research study sought to describe changes in middle school students’

aptitude for 21st century skills as well as motivations, interest, and perceived persistence

in STEM after 13-16 week participating in an OST STEM learning activity. The purpose

of this study was to explore how participation in OST STEM activities influenced

students’ affect, specifically how the students’ experiences played a role in their self-

reported motivations, interest, and STEM persistence. Data was collected and analyzed to

answer the research questions of the study, which focused on how participation in OST

STEM activity(ies):

1. changed middle school students’ perceptions (descriptions) of and actions


a. Type and number of current middle school STEM courses in their formal

schooling?


2. altered middle school students’ 21st century learning skills, motivation, and


Qualitative data were collected using observations, interviews, and a general

background questionnaire (descriptive statistics) to answer research questions.

Quantitative data were collected, using the FI’s (2012) S-STEM Survey for Middle and

High School (6-12th grade) Students. Pre and post survey data were calculated and

analyzed, using a paired-means t-test at the construct level and the Wilcoxon signed-rank


85

test at the item level. The qualitative and quantitative findings are presented in this

section. The following data collection instruments are located in the appendices:

Observation Tool (Appendix E), Interview Tool (Appendix F), STEM Extracurricular

Activity Questionnaire (descriptive statistics; Appendix G), and selected items from the

S-STEM Survey (Appendix H).

The mixed methods data were also analyzed using a side-by-side comparison

approach by which the quantitative statistical results were first reported and then the

qualitative findings were discussed (Creswell, 2013; Creswell & Clark, 2011). Next, the

quantitative results and qualitative findings were merged for a final interpretation; by

merging of the data and comparing the qualitative findings and quantitative results, the

researcher grasped a more robust understanding of the findings (Creswell, 2013; Creswell

& Clark, 2011).

Pseudonyms have replaced the names of the student participants throughout this

chapter; the pseudonyms were used to substantiate the findings (via the audit trail), while

protecting the participants’ privacy with regard to confidentiality.

Quantitative Results

The quantitative results were gathered using a pre-post survey model. First,

responses to Likert-scale items were transformed into numerical data for parametric

analysis. Then the pretest and posttest data results were compared. A paired-means t-test

was used to determine if participating in the OST STEM activity(ies) made a statistically

significant difference in the students’ attitudes and interests to pursue STEM studies. A

Wilcoxon signed-rank test was used to compare pretest and posttest data at the item level


86

The results of the findings were reported at the construct level by STEM subject areas

(i.e., Math, Science, Engineering and Technology, respectively), 2lst Century Learning,

About Yourself and Your Future. The data tables (A.1-A.13) for each category for the

paired-means t-test, the Wilcoxon signed-rank test, and the reliability statistical results

can be found in Appendix P.

Paired-Means t-Test

The paired-means t-test was used to determine if participation in OST STEM

activity(ies) made a statistically significant difference in the students’ attitudes and

interests to pursue STEM studies by comparing means of the pretest and posttest scores

of the entire survey. Among the 61 total questions on the survey, 59 questions were

analyzed using the paired-means t-test; two of the questions were open-ended. The data

were then analyzed by subject (Table A.1), gender (Tables A.2-A.3), and by grade level

(Tables A.4-A6). Results of the quantitative paired-means t-test for all students showed

no significant change in the students’ STEM persistence with respect to the two points in

time (before participation in the OST activity and after the OST activity).

All subjects paired-means t-test. A second paired-means t-test was used to

determine if participation in OST STEM activities made a statistically significant

difference in the students’ attitudes and interests to pursue STEM studies by the

construct. The results from Math, Engineering and Technology, 21st Century Learning,

and Your Future categories of the S-STEM Survey (FI, 2012) indicated no statistically

significant difference (i.e. no p values less than 0.05, see Table A.1). However, the

Science and About Yourself sections each were statistically significant with p values less


87

than 0.05 (see Table A.1). The Math section could not be computed; the standard error of

the difference was zero.

The Science section had a statistically significant difference (t[36] = -2.697, p <

0.011) between the pretest mean (M = 31.05, SD = 4.31) and posttest mean (M = 32.72,

SD = 4.57). The 95% confidence interval for the difference was [-2.94, -0.42] (see Table

A.1). The OST STEM activities influenced the students’ affect towards science.

The About Yourself section had a statistically significant difference (t[36] = -

2.057, p < 0.047) between the pretest mean (M = 20.83, SD = 4.85) and posttest mean (M

= 21.48, SD = 4.72). The 95% confidence interval for the difference was [-1.28, -0.09]

(see Table A.1). The OST STEM activities influenced the students’ awareness of their

academic performance in their formal courses and their knowledge of STEM

professionals that they know personally.

All subjects paired-means t test conclusion. Overall, the results suggest that the

OST STEM activities did not have a statistically significant effect on the participating

middle school students’ STEM persistence. The results suggest that the collection of

middle school participants’ STEM persistence was not affected, except for an increase in

their science attitude, and their awareness of their academic performance in their classes

and who they knew in their lives that are STEM professionals.

Gender paired-means t-test. The boys’ (N= 21) and girls’ (N = 16) mean

differences between pre and post survey administrations were compared. The boys’

survey responses that were analyzed using the paired t-test at the construct level and only

were statistically significant in the About Yourself section (t[20] = -2.359, p < 0.029) (see


88

Table A.2). The girls’ survey responses were analyzed and were only statistically

significant in the Science section (t[15] = -2.578, p < 0.021) (see Table A.3). This implies

that the OST STEM activities influenced the girls’ attitudes towards science, while the

boys’ became more aware of their academic performance in their formal courses and

STEM professionals they know in their lives after participation in an OST STEM

activity.

Girls’ paired-means t-test. The Science section was statistically significant at the

construct level. There was a statistically significant difference in the scores from pretest

(M = 32.125, SD = 4.44) to posttest (M = 34.5, SD = 3.88) for a p value less than 0.05

(t[15] = -2.578, p < 0.021) (see Table 8). The 95% confidence interval for the difference

is [-4.338, -0.411]. These OST STEM activities influenced the middle school girl

participates’ attitude towards science.

Boys’ paired-means t-test. The About Yourself section was statistically significant

at the construct level. The results in this section showed a p-value less than 0.05 (t[20] = -

2.359, p < 0.029) and a statistically significant difference in the scores from pretest (M =

23.71, SD = 3.87) to posttest (M = 24.86, SD = 2.48). The 95% confidence interval for

the difference is [-2.15, -0.1323] (see Table 9). The OST STEM activities influenced the

boys’ awareness of their academic performance in their formal courses and their

knowledge of STEM professionals that they know personally.

Gender paired-means t-test conclusion. In conclusion, data for the girls and boys

showed that there was very little significance at all between the OST STEM activities on

the middle school students’ STEM persistence at the construct level. The boys’ data


89

suggested positive statistically significant changes in the About Yourself section,

demonstrating increased awareness of their academic performance in their classes and

knowledge of STEM professionals. The girls reported statistically significant changes in

their attitudes toward science after participating in their OST STEM activities.

Grade 6, 7, and 8 paired-means t test data. The paired-means t test data were

separated into grade levels for analysis to be able to compare the sixth-, seventh-, and

eighth-grade levels. The scores provide insight into each grade level. The eighth-grade

students were the only grade-level to show any statically significant results at the

construct level.

6th grade. There was no statistically significant change between the pretest and

posttest scores at the construct level for the sixth-grade students (see Table A.4). This

may be due in part to the small number of sixth graders (n = 5) in this study.

7th grade. The seventh-grade student data (n = 18) had no statistically significant

changes to any of the sections at the construct level (see Table A.5).

8th grade. The paired-means t-test for the eighth-grade students’ data (n = 14)

showed statistical significance from pretest to the posttest for the Science section at the

construct level (see Table A.6). The paired-means t-test had a p-value less than 0.05

(t[13] = -3.13, p < 0.008). There was a statistically significant difference in the scores

from pretest (M = 32.21, SD = 3.59) to posttest (M = 34.5, SD = 4.24) as seen in Table

A.6. The 95% confidence interval for the difference is [0.729, -0.709]. Based on these

results, the eighth-grade students’ attitude towards science had changed positively from

participation in an OST STEM activity.


90

Conclusion for 6th, 7th, and 8th grade paired-means t-test data. In review, there

was no statistically significant change between the pretest and posttest scores for the

sixth-grade or seven-grade students. The eighth-grade students’ data showed a significant

positive change in students’ attitudes toward science after participating in an OST STEM

activity.

Wilcoxon Signed-Rank Test

The Wilcoxon signed-rank test was used to compare the pretests’ and posttests’

data at the item level. The Wilcoxon signed-rank test is a nonparametric test, which does

not assume normality and may be used for ordinal (e.g. Likert) type data. The Wilcoxon

signed-rank test was conducted to measure students’ STEM persistence, which is a

complex construct, through the use of the factors of changes in students’ confidence and

efficacy in STEM subjects, 21st Century learning skills, and interest in STEM careers on,

based on their responses on the S-STEM survey (FI, 2012). The data was analyzed by

looking at all of the subjects as one large group, at individual grade level breakdowns,

and at gender breakdowns. Furthermore, the test was used to analyze each of the

participants individually. There are 61 total question items on the survey, but only 59

questions were analyzed; two of the questions were open-ended.

All students’ Wilcoxon signed-rank test. Results of the Wilcoxon sign-rank test

at the aggregate level suggests that there is no statistically significant change between the

OST activity and the students’ STEM persistence (see Table A.7). At the item level, only

five questions demonstrated statistical significance.


91

The Wilcoxon signed-rank test did suggest a statistically significant change in

students’ attitude towards the question “Math is hard for me” (Z = -2.399, p = 0.016) (see

Table A.7). The median student response rating was 3.0 (Neither Agree or Disagree) for

both pre- and post-tests; two of 37 middle school students reported an increase in attitude,

19 students reported a decrease in attitude, and 22 students reported a consistent attitude.

The second reference that the Wilcoxon signed-rank test showed a statistically

significant change in students’ attitude towards the question of, “If I learn engineering,

then I can improve things that people use every day” (Z = -2.121, p = 0.034) (see Table

A.7). The median student response rating was 4.0 (Agree) for both pretests and posttests;

one of 37 middle school students reported an increase in attitude, seven students reported

a decrease in attitude, and 29 students reported a consistent attitude.

The third reference that the Wilcoxon signed-rank test showed a statistically

significant change in students’ attitude towards the question “I am confident I can work

well with students from different backgrounds” (Z = -2.500, p = 0.012) (see Table A.7).

The median student response rating was 4.0 (Agree) for both pre- and post-tests. Eleven

of the 37 middle school students reported increases in their attitude towards their

confidence that they can work with other students of different backgrounds, whereas two

participants reported decreases, and 24 participants’ viewpoints remained the same.

The final two items that the Wilcoxon signed-rank test showed that the OST

STEM activities did elicit a statistically significant change in students came from the

About Yourself section. The question, “How well do you expect to do this year in your:

Science Class?”, had a statistically significant change, Z = -2.111, p = 0.035 (see Table


92

A.7). The median student response rating was 3.0 (Very Well) for both pre- and post-

tests; five of the 37 middle school OST STEM students reported a positive change in

viewpoint, whereas four participants reported a negative change to their viewpoint and 28

participants reported no change in their viewpoint. Lastly, the question, “Do you know

any adults who work as mathematicians?”, had a statistically significant change, (Z = -

2.299, p = 0.022) (see Table A.7). The median student response rating was 3.0 (Not Sure)

for both pre- and post-tests. The OST STEM activities affected nine of the 37 middle

school students’ perception towards knowing adults that are mathematicians (increased),

whereas two participants decreased and 26 participants remained unchanged.

All subjects paired-means Wilcoxon signed-rank test conclusion. Overall, the

OST STEM activities had a minimal statistically significant effect on the students’ STEM

persistence between the pre and posttests. Furthermore, only students’ attitudes towards

learning engineering to improve things to people use every day and their confidence that

they can work with other students of different backgrounds have a statistically significant

positive affect. Lastly, the students’ perception of math being hard for them became

negative, as well as student perceived they were doing very well in science class and

knew more adult mathematicians (see Table A.7).

Gender Wilcoxon signed-rank test. Pre and posttest data were analyzed based on

the participant’s gender (16 girls and 21 boys). The male students’ data showed no

statistically significant changes (see Table A.8-A.9) whereas the female students’ data

had a total of two questions show significance, both of which were from the Science

section (see Table 14).


93

The Wilcoxon signed-rank test showed that the OST STEM activities did elicit a

statistically significant change in girl students’ perception towards the tenth question

from question ten in the Science section that stated, “I would consider a career in science”

(Z = -2.126, p = 0.033) (see Table A.9). Seven of the 16 girl students responded with an

increase in their perception towards considering a career in science, one participant

responded with decreased perception, and eight participants remained constant.

Additionally, the Wilcoxon signed-rank test showed that the OST STEM

activities did elicit a statistically significant change in female students’ perception

towards the question, “Science will be important to me in my life’s work.” (Z = -

2.121, p = 0.034) (see Table A.9). Five of the 16 female students reported an increase in

attitude and the remaining eleven students reported no change.

Gender Wilcoxon signed-rank test conclusion. In conclusion, Wilcoxon signed-

rank test data for gender (girls compared to boys) showed that there was very little

significance at all between the OST STEM activities on the middle school students’

STEM persistence. The boys’ data showed no statistically significant changes between

the pre- and posttests, and the girls only showed statistically significant changes towards

their perceptions on science with regards to considering a career in science and science

being important in their future work.

6th, 7th, and 8th grade Wilcoxon signed-rank data. The Wilcoxon signed-rank test

data were separated into grade levels to compare participants by grade. The majority of

the data showed that there was no statistically significant change between the pre- and


94

post-tests form the OST STEM activities (see Tables 16-18). However, there was some

item-level significance for each grade level.

6th grade Wilcoxon signed-rank data. The Wilcoxon signed-rank test showed that

there were no statistically significant changes between the pretest and posttest scores at

the item level for the sixth-grade students (see Table A.10). Once again, this may be

related to the small number of sixth graders students (n = 5) in this study.

7th grade Wilcoxon signed-rank data. The seventh grade students’ data showed

significant differences for four items. The first significant question was, “I would

consider choosing a career that uses math” (Z = -2.179, p = 0.029) (see Table A.11). Nine

of the 19 seventh grade students reported an increase in attitude after participating in an

OST STEM activity, two of the seventh grade participants reported a decrease in attitude,

and eight of the seventh grade participants reported no change in their attitude. The next

significant question was in the Science section that stated, “I would consider a career in

science” (Z = -2.070, p = 0.038) (see Table A.11). Seven of the 19 seventh grade students

reported an increase in consideration after participating in an OST STEM activity, one of

the seventh-grade participants reported a decrease in consideration, and 11 of the

seventh-grade participants reported no change in their consideration. The third significant

question was question 11 (question number 37 of the survey) in the 21st Century Learning

section that stated, “I am confident I can work well with students from different

backgrounds” (Z = -2.111, p = 0.035) (see Table A.11). Seven of the 19 seventh grade

students reported an increase in confidence after participating in an OST STEM activity,

one of the seventh-grade participants reported a decrease in attitude, and 11 of the


95

seventh-grade participants reported no change in their attitude. The fourth and last

question with statistical significance was, “Do you know any adults who work as

mathematicians?” from, the section About Yourself (Z = -1.994, p = 0.046) (see Table

A.11). Five of the 19 seventh grade students reported an increase in awareness after

participating in an OST STEM activity, four of the seventh-grade participants reported a

decrease in awareness, and 10 of the seventh-grade participants reported no change in

their awareness.

The data suggested that the seventh-grade students’ perceptions changed with

regards to considering a science-based career, a positive attitude towards working with

others with different backgrounds, and becoming aware of adults who are

mathematicians. Wilcoxon signed-rank test showed there was no other significance on

the seventh-grade students’ STEM interests and persistence.

8th grade Wilcoxon signed-rank data. The eighth-grade students’ data indicated

statistical significance for two items only. The first question to demonstrate statistical

significance was, “I am sure I could do advanced work in science” (Z = -2.000, p =

0.046) (see Table A.12). Four of the 14 eighth grade students reported an increase in their

attitude after participating in an OST STEM activity whereas ten participants reported no

change in attitude. The second question that demonstrated a statistically significant

change was, “If I learn engineering, then I can improve things that people use every day”

(Z = -2.000, p = 0.046) (see Table A.12). Four of the eighth-grade student participants

reported a negative shift in perception and 10 participants reported no change in

perception.


96

Overall, only two questions were significant; the rest of the questions showed that

the activities were not drivers of statistically significant change.

Conclusion for 6th, 7th, and 8th grade Wilcoxon signed-rank data. The eighth-

grade students both had statistically significant affects towards the question “If I learn

engineering, then I can improve things that people use every day”. This means 20 of the

students from the study experienced a shifted perception in regards to engineering’s

ability to improve things people use every day. Furthermore, the seventh and eighth

graders had similar statistically significant change towards considering science as a

career option, meaning that over half of the students (19 seventh and 14 eighth graders)

are considering possible work related to science.

Summary of the Quantitative Findings

At the construct level, the OST STEM activities had a statistically significant

impact on the students’ attitudes toward science and their awareness of the academic

performance in their class, as well as their awareness of people they know who are STEM

professionals. The OST STEM activities’ impact of the students’ attitudes toward science

were significant for the eighth-grade students and the girls. Furthermore, the OST STEM

activities impact of the students’ awareness towards their academic performance in their

classes and their increased awareness of STEM professionals they knew were found in

the results of the boys.

The analysis of the item-level survey data evidenced there were few areas of

significance, including students’ consideration of science as a future career option and

perception that learning engineering can help the students improve items people use

every day all changed positively. The difficulty of advanced math changed negatively


97

between the pretests and posttests. Lastly, the boys’ item-level data showed no

statistically significant changes between the pre- and posttests, and the girls’ item-level

data only showed statistically significant changes towards their perceptions of science

with regards to considering a career in science and science being important in their future

work.

Qualitative Findings

Data from interviews and observations completed during the middle school OST

STEM activities were coded into categories. This coding informed the creation of

specific themes with coded instances (see Chapter 3, methodology section for qualitative

data). This analysis led to the identification of five major themes: Supporting Student’s

STEM Persistence (N=203), Developing STEM Skills and Content (N=111), Experience

Levels (N=59), Not Sure About a STEM Future (N=52), and Sources of Motivation

(N=428). Each of the five major themes from the findings was built from related

categories (N=16) from the coded data, including common interests, experiences,

concepts, and outlooks.

The theme of Supporting Student’s STEM Persistence was developed by the

following subthemes: Promoting STEM Persistence in Middle School (N=130);

Enjoyment, Engagement, and Focus (N=44); and Involved in Multiple STEM Activities

(N=29). The theme of Developing STEM Skills and Content was developed by the

subthemes of Soft Skills (N=46) and Technical Skills (N=65). The theme of Experience

Levels was developed by the subthemes of Prior Experiences and Skills (N=56), and No

Prior Experience (N=3). The theme of Not Sure About a STEM Future was derived from


98

the subthemes of Lack of Interest or Source of Frustration in STEM (N=24), and

Indecisive About Choosing a Pathway (N=48). Lastly, Sources of Motivation was

developed form the following subthemes: Friends (N=41), Family (N=48), Teacher

(N=81), Supporting Others (N=3), STEM Activities and Content (N=75), Outside of

School Organization or People (N=8), and Self-Motivation and Internal Interest (N=134).

All themes are included in Table 4.1 along with how each theme connects to the research

questions, data source (i.e. interviews, descriptive statistics, and observations), and the

constructs (i.e. interest, persistence, 21st century skills, and motivation) from the

conceptual framework (see page 10).


99

Table 4.1

Qualitative Themes and Subthemes Breakdown

Theme(s)

Open Coding

Count by Theme(s)

Subtheme(s) and Open Coding Count by

Subtheme(s)

Research Questions (RQ)

Addressed

Data Sources: Interviews (I), Descriptive Statistics

(DS), & Observations (O)

Related to the Constructs

Supporting Student’s STEM Persistence

203 Promoting STEM Persistence in Middle School (N=130) Enjoyment, Engagement, and Focus (N=44) Involved in Multiple STEM Activities (N=29)

R1 Sub. 1 R1 Sub. 2 R2

I, DS, & O Persistence, Motivation, 21st Century Skills, & Interest

Developing STEM Skills and Content

111 Soft Skills (N=46) Technical Skills (N=65)

R1 R2

I, DS, & O Interest & 21st Century Skills

Experience Levels

59 Prior Experiences and Skills (N=56) No Prior Experience (N=3)

R1 Sub. 1 R2

I, DS, & O Interest, Motivation

Not Sure About a STEM Future

52 Lack of Interest or Source of Frustration in STEM (N=24) Indecisive About Choosing a Pathway (N=48)

R1 Sub. 2 R2

I, DS, & O Persistence & Interest

Sources of Motivation

428 Friends (N=41), Family (N=48), Teacher (N=81), Supporting Others (N=3), STEM Activities and Content (N=75), Outside of School Organization or People (N=8), Self-Motivation and Internal Interest (N=134).

R2 I, DS, & O Motivation, Interest, & Persistence


100

The concept map in Figure 4.1 shows the five major themes and the subthemes.

The table models the breakdown of the subthemes that create the five major

themes. The table functions as the model for the discussion of each theme and their

subthemes.

Figure 4.1. Themes and subthemes developed from qualitative analysis.


The theme of Supporting Student STEM Persistence is comprised of three

subcategories: Promoting STEM Persistence in Middle School; Enjoyment, Engagement,

and Focus; and Involved in Multiple STEM Activities. Supporting Student STEM

persistence had the second most instances among the five identified themes (N=203).


101

Data showed that the STEM activities were providing students a source for engagement

and enjoyment in STEM learning as well as a platform for promoting STEM persistence.

Multiple students reported being involved in various informal STEM activities at the

school which provided students with an outlet and opportunity for their STEM learning.

The interviews, observations, and questionnaire (descriptive statistics) data suggested

students attributed OST STEM activities with impacting their engagement and enjoyment

for learning STEM as well as supporting their learning and persistence in STEM.

Promoting STEM persistence in middle school. The subtheme of Promoting

STEM Persistence in Middle School was rooted in the observation notes (N=78) and

interview responses (N=15). The data implied that the individual OST activities (e.g.

Sumo-bots, eCYBERMISSION, and Girls Who Code) supported students’ STEM

persistence. Students reported that participation in OST STEM activities led to

enjoyment, exposure, and general learning of STEM content. Students described that the

environment of these informal STEM activities offered new learning opportunities and

enriched their classroom learning. The interview data supports this claim. For example,

Paul (a seventh-grade male) stated during his interview, “It’s a challenge but it’s good to

be able to learn because there’s so much you can learn from it. So, if you do something

wrong, you can just do it better next time” (Sumo.I1.2). This indicates that the STEM

activity (i.e. robotics) promoted STEM learning by challenging him to grow and improve.

Furthermore, Paul spoke about how his STEM activity (i.e. robotics) was supporting his

future, when he explained,


102

I do! I think with the new technologies, as I said before, there are so many career

opportunities with this kind of stuff, that having experience with this through

school is going to open up a lot of job opportunities. (Sumo.I1.17)

During the interviews, the 15 interviewees were asked, “Do you see yourself continuing

with these types of things, classes, activities in high school and in the future?” Helen

(eighth-grade female) responded to this question by stating, “Actually, I do. I signed up

for engineering in high school and plan on it in college...It’s fun for me. It interests me.

And I like to do it—would like to do it as a career” (ScOl.I2.111). Emmitt (sixth-grade

male) shared a similar thought, “Yes, I do,” and went on to explain, “I’m just really

interested in this. . . . It’s just that we’re coding robots and we can tinker with the code”

(Sumo.I3.174). Lastly, the Emmitt stated, “It’s got me really interested because it’s like

the first real-world competition coding thing” (Sumo.I3.184). For these students,

participation Science Olympiad and Sumo-bots promoted students’ STEM persistence by

providing exposure and creating interest.

The student subjects were aware of the need to think about their own future,

success, and educational paths. Their awareness was seen when future possibilities for

pathways, college, and careers were explained to them. This was evident in a statement

by Christopher (eighth-grade male) when doing robotics: “I think it’s, well first off, it

always looks good on your college resume when colleges look at what activities you do.

They’re also fun to do” (Sumo.I4.244). Furthermore, students were asked about if and

how the activity affected their decision to continue to pursue STEM activities in the

future. Their responses indicated that a majority of the interviewed students wanted to


103

continue their STEM learning in middle and high school. For example, Jennifer (seventh-

grade female) who participates in Science Olympiad stated, “I am taking engineering

next year. I definitely like STEM, and I like math and engineering” (ScOl.I5.318).

Christopher explained why he has continued to pursue STEM learning when he stated, “I

realized how much fun it was in sixth grade and what it’s like to work on a team and go

to a competition where you had to put what you worked on against other people’s

projects” (Sumo.I4.215).… I think it made me more open to doing more activities with

STEM” (Sumo.I4.250). Another, Price (seventh-grade male), in his third year of doing

robotics, explained how his continued participation in robotics created persistence: “I

think the more I do robotics, the more I like it. So I will do more STEM activities”

(Sumo.I7.463).

The continued support for and the promotion of STEM learning and future career

insights in STEM through the informal STEM activities, along with formal middle school

engineering classes during the school day, were cited by students as a pathway to

continue their STEM learning and to give them insight into career connections. Amy

(seventh-grade female), who was participating in the Science Olympiad, explained how

she wanted to continue with STEM learning activities in the future. She also explained

that her learning was making connections to career paths when she stated, “I’m

particularly interested in fashion design, and so I feel like that comes along with

engineering, now. I think it’s cool to figure out how things are made and sort of create

stuff” (ScOl.I8.516). Harry (sixth-grade male) explained that he wanted to go to college

for engineering (Sumo.I9.598). While Gina, (eighth-grade female) Science Olympiad


104

student, explained how she had already planned her upcoming freshman year STEM

activities: “Yes, I’m taking Design 1 and Design 2 next year, and then I’d like to go to

college for engineering. . . . Next year, I want to come back and be on the middle school

Science Olympiad team” (ScOl.I10.654). This is an example of how her Science

Olympiad have influenced her decision to take future courses to persist with STEM

activities.

In general, the students who were interviewed made statements that signified their

future persistence in STEM learning as well as even going to college to major in a STEM

discipline. For example, Gina stated, “[I’m] probably going to college for it. I think it’s a

good path to be on because like the future has a lot to do with technology and engineering

in general” (ScOl.I10.668). Furthermore, Hamilton, (eighth-grade male) in Robotics,

stated, “I’m interested in it and seem to be pretty good at it, and I really just love working

with things and especially like physical civil engineering “(Sumo.I13.856). Lastly,

pursuing future STEM learning opportunities was found when Sarah (seventh-grade

female in Science Olympiad) stated,

I always want to try different new activities involving engineering. . . . There’s

just so many people sort of closed off to engineering because they think it’s

tough, but I kind of like the challenge because it’s a new activity to try. And also,

it isn’t really that many females in that sort of section. (SciOl.I14.931)

These statements evidence the influence of OST STEM activities; in addition to exposing

students to STEM careers, these experiences (e.g. Science Olympiad, etc.) were cited as

helping the students’ persistence in STEM learning. Simon, (eighth-grade male) in


105

robotics explained how his STEM activity had supported his persistence and helped him

narrow his engineering career path when he stated,

I just really enjoy it, and it’s just something I'd like to pursue and continue to do. .

. . It’s exposing me to more stuff in engineering, and I can slowly decide what

type of engineering I want to continue with. (Sumo.I11.723)

All 15 students interviewed discussed this connection between the activity and supporting

their learning of STEM, and the descriptive statistics showed that 17 students were

interested in possibly pursuing a career field in STEM.

Progressing towards STEM. Students who were unsure about pursuing STEM

discussed considering or beginning to consider a pursuit of STEM learning opportunities

in the near future. Kimmy (an eighth-grade female student in eCYBERMISSION),

explained her uncertainty in pursuing STEM: “I don’t know. I just think, if I take it in

high school, I’ll probably pursue it in college and in my career. So, I’m just not sure yet”

(ScOl.I15.1004). Kimmy did express that her activity had a positive impact on her desire

to pursue STEM: “It has just opened my eyes and just leading to all of the possibilities in

engineering” ScOl.I15.1014). Furthermore, Sarah who participated in Science Olympiad,

echoed this same sentiment when she stated, “Yeah, in engineering. So it’s just like, yeah,

I can take off and see how it goes” (ScOl.I14.938). Lastly, Stewart (seventh-grade male),

who participated in sumobots, asserted that he was not talented in math or science, yet

continued with STEM learning because his involvement in his STEM activity made him

realize he is talented at technology and engineering. He explained,


106

Okay, then, “Yes” I’m going to continue with this one because I liked the

activities we did, and I thought they were fun. I’m not like really good at science

and math so, but like technology and engineering and other stuff like this.

(Sumo.I12.776)

Promotion. Some (n=5) students expressed they wanted to pursue a STEM

pathway even before participating in these OST STEM activities, but that the activities

further promoted their STEM persistence. For example, Simon (a male, 8th grader)

stated, “I’ve always wanted to do it. This just made me want to do it more”

(Sumo.I11.726). There were students, such as eighth-grade students Hamilton, Helen,

Kimmy, and Stewart, who came into the study with multiple years of participating in the

same informal activity, such as eCYBERMISSION or robotics, because the activity

provided them enjoyment or support for their STEM learning and persistence.

Holistically, the interview data suggests that the OST STEM activities promoted

student STEM learning and persistence in pursuing future STEM learning opportunities.

When the students were asked, “At this point in time, do you plan on pursuing a future

STEM class or OST activity?” 24 out of 37 students stated they wanted to participate in a

future STEM activity (formal or informal) in middle or high school. Furthermore, 17

students reported they were planning on or wanted to attend college as a STEM major or

pursue a career in STEM when asked the question, “At this point in time, do you plan on

pursuing a STEM college major and/or career?” These data indicate that the STEM

learning students’ experienced positively influenced the decisions of many to continue to

pursue future STEM opportunities and persist on a STEM pathway. Students discussed


107

during their interviews that the more STEM subjects and content they learned and the

more they participated in STEM learning the more they wanted to continue learning; this

was evidenced by students in more than half the activities. The subtheme of Promoting

STEM Persistence in Middle School stemmed from the positive impact engagement in

the OST STEM activities, specifically the enjoyment, exposure, and general learning of

STEM content, had on students’ STEM persistence.

Enjoyment, engagement, and focus. The OST STEM activities provided students

with an environment in which they could focus solely on STEM content, thereby

providing them with an enjoyable and engaging experience. Overall, the interview,

observations, and questionnaire (descriptive statistics) data showed that a majority of the

students enjoyed their informal STEM activity, which supported their STEM persistence.

Interview data. During the interviews, the students spoke to their enjoyment of

OST STEM activities. For example, Paul stated, “It’s been really fun!” (Sumo.I1.3). Paul

went onto explain in detail the process of joining, participating, learning, and continuing

with his OST activity:

So when I came into middle school, I had the opportunity to sign up for an engineering class. And when I learned that robotics was a thing, I thought this might be a cool thing to try and I might like it. So when I did, I learned more than I thought there would be to the program. So, I decided to keep going with it. (Sumo.I1.7)

Harry, a sixth-grade male student who participated in SumoBots and Drones, explained

how he enjoyed working with his robotics group: “It’s just a place where I feel happy,

and its’ something I enjoy doing” (Sumo.I9.601). Simon expressed a similar feeling when

he stated, “Well, I really like robotics, and I enjoy it. And I just enjoy engineering in

general. . . . I just really enjoy it, and it’s just something I'd like to pursue and continue to


108

do” (Sumo.I11.686). These comments illustrated students’ enjoyment in STEM learning

through their OST STEM activities.

Observations. The 78 observations that occurred during the meetings of the

STEM activities between January and May suggested students engaged in, focused on,

and enjoyed the OST STEM activities. The Observation Tool had two questions that lead

to observing the students being engaging in their activities and the students’ general

demeanor during the activities: “If students are engaged in a learning activity, what are

they doing?” and “What is the overall demeanor of the students during the course?”

Throughout the different informal STEM clubs, the students were observed with smiling

faces that were accompanied by serious and focused body language (Table 4.2). The

students were typically engaged in their specific project with their peers in small groups.

The peer interactions, generated by positive group dynamics, were demonstrations of

happiness and fun. The autonomy to pick their own groups may have enhanced their

enjoyment in the activities.


109

Table 4.2

Observed Data Related to the Themes and Subthemes Theme

Themes and subthemes Sub-theme Days sub-theme was observed (n) Example


Promoting STEM Persistence in Middle School 0 -

Enjoyment, Engagement, and Focus 47 Smiles and focused

body language

Involved in Multiple STEM Activities 0 -


Soft skills (i.e. 21st Century Skills 47

Teams communicating plans and collaborating on their projects

Technical skills (i.e. CAD, laser cutting, 3D Printing, soldering, etc.)

45 Use of the 3D printers and laser cutters

Experience Levels Prior experience and skills 44 Programming the

sumobots

No prior experience 25 Practicing soldering


Lack of interest or source of frustration in STEM 20

Frustration from projects not working

Indecisive about choosing a pathway 0 -


Friends 4 Positive peer interaction

Family 0 -

Teacher 18 Positive teacher feedback

Supporting Others 0 0

STEM Activities and content 40 Students using fabrication equipment

Outside of school organization or people 0 -

Self-motivation and internal interest 9

Students work during non-practice times, such as lunch


110

The overall excitement to work on their projects was evidenced by large number

of students who, by their own volition, chose to engage in the STEM activities during

their lunch break and during recess. Students in the Science Olympiad, Robotics, and

eCYBERMISSION groups exhibited a great deal of excitement. At times, the students’

body language signaled concern or frustration with the problems associated with their

project or struggles with a task, but they maintained their focus, as demonstrated by their

persistence to overcome their struggles with the support of their teachers (see Figure 4.2).

For example, a female student, after realizing her hoverboard did not meet the

requirements, had to redesign it, while a group of male students had a similar redesign

with their SumoBot in robotics. All of these students exhibited frustration through stiff

body language and angry facial expressions, yet persevered through these obstacles to

experience the joy of successfully completing the task, signified by smiling faces and

corresponding body language.


111

Figure 4.2. Additional notes on students' overcoming frustration.

On three different occasions times during the observations of the

eCYBERMISSION, Science Olympiad, and robotics groups, students were drawn off

task or did not have a sense of urgency to complete their tasks or projects, even when due


112

dates or competition times drew near. For example, a pair of girls Science Olympiad girls

were observed working on a Rube Goldberg machine were not focused on completing

their task as the due date for the project was near. Contrarily, students were observed as

focused to accomplish their projects. For example, a majority of the Science Olympiad

participants attended on a teacher work day because they were feeling the pressure to get

their projects completed. The students’ prior knowledge manifested during observations,

and the instructors supported their students by making connections to prior formal and

informal STEM learning and skill sets as a resource, such as prior experience with

fabrication tools (3-D printer and laser cutter), coding knowledge (Scratch, 2015) and

LEGO Mindstorm EV3 (2018). Teachers extrinsically motivated their students through

encouragement, one-on-one support, positive reinforcement, and the competition

deadlines. Students demonstrated intrinsic motivation to complete a quality project which

kept students extremely focused as evidenced by their intense body language (see Figure

4.3). The intense focus due to extrinsic and intrinsic pressure caused some students to

rush, resulting in mistakes, which suggests that too much pressure may have affected

students negatively. These mistakes affected the outcomes of their final projects, as well

as extended their overall amount of time to complete their projects.


113

Figure 4.3. Question 6 shows students’ body language being serious and focused.

Throughout the study of the different informal STEM activities, there were only

15 cases of students not appearing focused on their work and projects. In all other cases,

even when the work was frustrating, their focus helped them overcome their challenges


114

and accomplish their projects. Students’ focus increased when the due date for their

projects neared or an upcoming tournament was approaching. Examples of this

heightened focus could be seen the days before the sumobot robotics tournament, when

the eCYBERMISSION projects were due for submission, and the week before the

Science Olympiad competition. This suggests students’ focus on the STEM content and

projects fostered their STEM persistence by providing these students with a learning

outlet.

Questionnaire. The responses from the questionnaire (descriptive statistics)

provided even more insight into the students’ engagement and enjoyment of their OST

STEM activities (see Table 4.2). Students’ recorded comments about their activities

reflected their feelings about their informal STEM activity: “It seemed fun and interesting

and new” (eCYB.Q24.K), “I thought it would be fun” (Sumo.Q9.K), and “Fun. I like

engineering” (Sumo.Q13.K). Other students agreed—they found their STEM activity to

be fun too: “Because it’s fun and you learn a lot” (Sumo.Q25.K), and “Because it’s fun”

(ScOl.Q2.K). Sarah even described the importance of her informal STEM activity when

she stated, “Science Olympiad is very important to me because I am a bit on the

competitive side. And it is fun to work with my partner for our Science Olympiad

project” (ScOl.Q29.J).

In conclusion, the qualitative data suggests that the OST STEM activities (e.g.

Science Olympiad) promoted student STEM learning and persistence in pursuing future

STEM learning opportunities by providing STEM learning that was enjoyable and

engaging. Students’ body language indicated they were focused and happy. Students


115

were observed coming to the lab spaces to work on their project during their lunchtime

which could be considered evidence of students’ persistence in pursuing STEM learning.

Furthermore, the students were observed having a high engagement in their tasks,

projects, and challenges, which suggests how engaging the STEM activities were to the

students (see Tables 4.2 and 4.3). A majority (N=25) of the students participating in the

STEM activities, especially the seventh- and eighth-grade students, had more than one

year of experience in these activities. The subtheme of Enjoyment, Engagement, and

Focus suggests that students’ participation in STEM-focused OST activities supported

their perceptions of STEM persistence through the self-reported data.

Involvement in multiple STEM activities. The subtheme of Involved in Multiple

STEM Activities arose due to multiple students participating in more than one OST

STEM activity offered at the middle school (see Table 4.3). The STEM activities were

offered at different times, before and after school as well as different days of the week,

which allowed students to participate in more than one informal STEM activity

sponsored by the school. The multiple OST STEM activities provided students with

exposure to different STEM content and possible career fields. The interview and

questionnaire data showed that nearly half of the students were participating in multiple

STEM activities and described as well as how the specific activities are engaging

students in their STEM interests. Specifically, 22 of the 37 participants were involved in

more than one OST STEM activity. Ten of these 22 participants were female students.

The 22 students who were participating in more than one activity were in either the

seventh or eighth grade. This information was self-reported by the students and included


116

OST STEM activities not a part of this study. A pattern was noted that students who had

participated in the OST STEM activities the previous year were more likely to participate

in more than one activity.


117

Table 4.3

Students Self-Reported OST STEM Activity Participation

Students eCYBERMISSION Girls Who

Code Robotics Sumobots

Robotics Drones

Science Olympiad SeaPerch*

Verizon App Challenge*

Student 1

X X

Student 2 X X

Student 3

X X

Student 4

X X

Student 5

X

Student 6 X X

Student 7 X X

Student 8 X X

Student 9 X X

Student 10 X X

Student 11 X X

Student 12 X X

Student 13 X X

Student 14 X X

Student 15 X X

Student 16 X X

Student 17 X X

Student 18 X X

Student 19 X X

Student 20 X X

Student 21 X X

Student 22 X X


118

*This OST STEM activity was not a part of this study, and students self-reported their

participation.

The students expressed during the interviews and on the questionnaire what

activity combinations they were a part of at school. For example, Jennifer, stated, “I

participated in eCYBERMISSION and I am currently doing Science Olympiad”

ScOl.Q18.S). Other students went on to identify all of the formal and informal STEM-

related activities they were currently participating in at their school. For example, Mark

(eighth-grade male) stated,

I recently participated in SeaPerch underwater robots, and am currently participating in SumoBots. I also take my engineering class, which consists of multiple different activities such as electricity, 3D Design, Architecture, and aviation. I also take an Honors Geometry and Advanced Conceptual Physics class. (Sumo.Q10.S)

The data suggests that the OST STEM activities promoted student STEM learning

and persistence by providing multiple STEM learning options for the students. These

students chose to be a part of more than one of the informal STEM activities. Over half of

the students (N=22) participated in more than one of the OST STEM activities, which

they claimed supported their STEM learning and persistence. Furthermore, the student

participation in multiple activities demonstrates how these middle school OST STEM

activities are engaging students in the promotion of STEM learning and pathways. The

subtheme of Involved in Multiple STEM Activities is connected to a large number of

students participating in more than one STEM informal activity. This is directly

supporting students’ STEM persistence due to providing students with options that can

engage their interests, as well as provide access to a variety of STEM content.


119

Conclusion for the theme of supporting student STEM persistence. The theme

of Supporting Student STEM Persistence was the most prominent because the largest

amount of references in the qualitative data (N =230) pointed to this benefit. the OST

STEM activities provided students an environment in which to engage in and enjoy

learning of STEM content as well as a platform for the students to focus on STEM-

related interests.


The theme of Developing STEM Skills and content is made up of two subthemes:

Soft Skills and Technical Skills. All of the qualitative data collected revealed that the

subjects were gaining and developing a variety of STEM skills and content from their

informal OST STEM activities. Furthermore, the STEM activities were providing

students the opportunities to put prior skills into practice. These skills demonstrated,

discussed, and observed were soft skills, such as communication and collaboration, and

technical skills such as soldering and using Computer Aided Design (CAD).

The informal STEM activities provided students the opportunity to learn and

develop new skill sets, which included soft and technical skills. During the study, the

students (N = 36) discussed and demonstrated how they were learning new skills,

including how to use different equipment, such as power tools and laser cutters, software

tools, such as computer-based coding languages and design tools, and other skill sets

depending on the OST STEM activities the students were a part of during the study.

These same students explained how the specific activities provided them the opportunity

to go deeper into they had learned in prior years and to further develop the skills gained

from their school courses such as engineering. This insight came from questions asking


120

the students about their learning. For example, Mark listed, “I enjoy the activity of coding

and building with Lego EV3s” (Sumo.Q10.L), and Hamilton explained, “It [SeaPerch]

taught me basic electrical engineering and structural engineering” (Sumo.Q7.K).

Hamilton referenced his SeaPerch underwater robotics group meetings that were prior to

the study, where the students created their robots, using polyvinyl chloride pipe, DC

motors, category 5 cables, and soldering electronic components to a printed circuit board

using hand drill, and hand saws for creating the SeaPerch robot frames in teams as it is

described on the SeaPerch’s website. Finally, Harry stated, “It [Sumo-bot] challenges

your brain more to think of your own designs instead of following directions to build it”

(Sumo.Q33.K). Five of the interviewed students explained how they enjoyed working

together with others on their projects, which is demonstrative of developing

communication, collaboration, and shared creativity and problem-solving. For example,

Samantha (seventh-grade female) student said: “Science Olympiad is very important to

me because I am a bit on the competitive side, and it is fun to work with my partner for

our Science Olympiad project” (ScOl.Q29.K). The informal STEM activities are

providing students an opportunity to interact with their peers as explained by Jennifer,

“eCYBERMISSION was probably the most prominent because it was a great intro into

STEM and helped me get closer with my peers” (eCYB.Q18.K). These examples show

the importance of collaboration, a soft skill, and the positive impact it had on the

students.

Soft skills. The subtheme of Soft skills illustrates that a majority of the students

demonstrated skill sets related to communication, collaboration, problem-solving and


121

more (see P21, 2007). These skills were being developed and put into practice in each of

the informal STEM activities. The students overcame struggles and documented their

progress for communicating to the competition judges and specialists. All of the informal

STEM activities were carried out in small teams between two and four students, which

provided the opportunity for students to collaborate and communicate with one another.

Observations. During the observations (see Figure 4.4), the students in each of the

OST STEM activities demonstrated research skills. For example, a group of students

working on an eCYBERMISSION project discussed how to cut acrylic with a saw and

were researching the process using the Internet on their iPads. The students demonstrated

not just their research capabilities, but they also modeled problem-solving and


122

independence in decision-making (see Figure 4.4).

Figure 4.4. Question 3 shows students researching.


123

By far, the largest soft skill observed (n=78) was problem-solving and decision-

making by design during each observation. The observation tool’s questions, “If students

are engaged in oral engineering/STEM discussion, what is the content and topic of

discussion?” and “If students are engaged in a learning activity, what are they doing?”,

supported these observations. Each informal activity provided students different projects,

which led to different possible problems to overcome. The robotics students discussed,

designed, and figured out how to construct their robots, where to put sensors on the robot,

how to code their robot, and how to keep their robot within the constraints of the

competition. The eCYBERMISSION and Science Olympiad students demonstrated

decision-making when determining how to design and create their specific projects.

Through Science Olympiad, students built rockets, learned the meaning of food science

terms, and built windmill fan blades (see Figure 4.5). A group of girls who were working

on a mousetrap car that needed a custom controlled braking system worked with an

engineer to design a solution to this eCYBERMISSION problem. These students all

showed creative designing and problem-solving skills to complete their projects and

overcome challenges while developing solutions to provided problems.


124

Figure 4.5. Question 2 shows the students’ projects.

Communication and collaboration skills were demonstrated by the students in

each of the informal STEM activities, primarily due to the design and nature of the

activities, the team-driven focus and the lead teachers encouraging communication and


125

collaboration between the individual groups and partnerships. There are many examples

in the data of students communicating with each other to solve problems and plan their

projects. This planning and documentation were done through pictures, videos,

reflections, and written explanations through the use of Google sites in

eCYBERMISSION and Science Olympiad. Furthermore, the poster presentation boards

made by the SeaPerch students to communicate the building of their underwater robots

were seen by the researcher in the robotics groups meeting area, as well as the posters are

described on the SeaPerch website. Robotics students discussed how to design and

program their sumobots and the Girls Who Code group communicated their website

design and code. In the observations carried out in a single day, student communication

targeted: how to build their rocket, what they needed to get done before the competition,

laser cutting fan blades and building them, lamenting balsawood for tower parts, taping

bottle rocket parts, 3D printing and making adjustments, reiterating how to build their

windmill fans, and building components for the Rube Goldberg machine such as ramps

and cars.

In spite of the positive skill development surrounding creativity, problem-solving,

collaboration, and communication, there was evidence of student frustration and being

overwhelmed during the activities. There were instances in which students became

frustrated when their projects did not work or there was disagreement between teammates

on the direction of their project (see Table 4). Teachers typically intervened when

students were struggling and frustration was mounting due to misunderstandings.


126

Furthermore, teammates’ failure to follow through on commitments caused frustration

that required teachers to speak to the struggling groups of students.

Technical skills. The subtheme of Technical Skills is rooted in the fact that the

students demonstrated skill sets including using fabrication and design equipment,

developing specific science, engineering, and technical skill sets, and learning STEM

content. Technical skills were being developed and put into practice throughout each

informal STEM activity. The school has an engineering fabrication lab which provides

students access to laser cutters, 3D printers, CNC machines, digital fabrication tools, and

traditional woodshop equipment. The students practiced these technical skills by creating

solutions to challenges and problems through unique approaches to each of their projects

in their specific informal STEM activity and created finished products with a variety of

tools, software, and hardware.

In the robotics, eCYBERMISSION, Girls Who Code, and some of the Science

Olympiad activities, students learned about electricity, electronics, and computer

programming. This could be seen when Harry described the technical skills he gained

from his OST STEM activity when he stated, I’ve learned how to program deeper, put

things in loops, and how to use the sensors” (Sumo.I9.592). The students demonstrated

an understanding of circuits through projects that involved circuit design, using wiring,

direct current (DC) motors, light emitting diodes (LEDs), and resistors. Students also

learned and put into practice soldering skills for their projects. For example, a group of

eighth-grade girls soldered electronic components, including a DC fan, wires, and a

switch, as part of a hoverboard craft project. Katie explained her OST STEM activity


127

taught her engineering skills when she stated, “It obviously taught me engineering skills”

(eCYB.I15.1002). While Helen (a female eighth-grader) stated, “I’ve learned how to

laser cut. I’ve also learned how to work as a group and basically how to build towers".

Peter explained the technical skills he learned when he stated, “Soldering is also a really

cool skill and considering that I’ve done sea-perch and sumo bots I know how to build

robots (Sumo.I1.38).

Computer programming skill sets were primarily seen in the Girls Who Code and

the sumo-bot robotics groups, as well as small groups in the eCYBERMISSION and

Science Olympiad based on the nature of the particular project or challenge. The students

gained an understanding of computer programming terms and processes, such as looping

behaviors, if-else statements, and variables, by putting them into practice to create

finished outcomes. For example, the Girls Who Code group worked on developing a

website through the use of HTML5 (HyperText Markup Language 5), which is a markup

language for designing, structuring, and displaying websites on the World Wide Web.

Amy stated, "I learned python and I am learning HTML[5]. Right now, we’re building a

website, and then [learning] JavaScript and CSS (Cascading Style Sheets)”

(GWC.I8.490). Sumo-bot robotics teams also demonstrated an understanding of the

interaction between the hardware and software when they coded the wrestling robots,

using the LEGO Mindstorm robot. For example, groups of boys in the robotics group

programmed their design and custom built sumo-bots to use servomotors, color, and

ultrasonic sensors to find an opponent robot and push it out of the ring (see Table 4.5).

Lastly, a small group of students in eCYBERMISSION demonstrated coding skills using


128

an Arduino as a part of their project. These coding skills were seen throughout the

observations during the study (see Table 4.3).

Other fabrication and engineering skills were demonstrated at varying levels

depending on the informal STEM activity. For example, students in eCYBERMISSION

were using traditional shop tools in addition to more advanced technological tools to

build their prototypes. The Science Olympiad group used digital fabrication tools (i.e.

laser cutters and 3D printers) for their projects, too.

The researcher observed students learning and implementing computer-aided

design (CAD) software, such as Inkscape and TinkerCAD, into their projects (see Table

4). The eCYBERMISSION and Science Olympiad teams used CAD software with

specific projects and challenges. For example, Helen designed a tower for holding weight

for a Science Olympiad challenge using Inkscape, and she cut the parts out of balsawood

using a laser cutter. Other girls used CAD to design 3D printed custom rocket parts, such

as fins, for another project. Some students demonstrated a culmination of learning in their

final projects, which included, for example, CAD software design and laser cutting.

Throughout the study, students were learning and implementing technical skills

that involved a variety of hand tools, power tools, programming languages, and

fabrication equipment. Design software and physical tools provided students resources

for creating solutions to problems and answering the specific challenges given to them by

their OST STEM activity. The students integrated the tools in various ways, depending

on their project. For example, a group of eighth-grade girls in Science Olympiad used

Inkscape and the laser cutter for their windmill blades, while another group used the


129

Internet to study food science. Furthermore, a group of seventh-grade boys in robotics, in

coding their LEGO Mindstorm EV3, used an icon-based language after building their

robot with servomotors, an ultrasonic sensor, and a color sensor. These examples (see

Tables 4.2-4.5) show the various technical skills and tools used in different ways by the

middle school students.

Conclusion for developing STEM skills and content. The theme of Developing

STEM Skills and Content is directly connected to students’ learning and practicing soft

and technical skills. The qualitative data collected, suggest that the students were gaining

a variety of STEM skills and content from their informal OST STEM activities. The

informal STEM activities were providing students the opportunity to work on soft skills

that are 21st Century skills as such collaboration, creativity, problem-solving, and

communication. This was seen through the completion of collaborative projects that

provided students the opportunity creates solutions to different challenges and problems

(see Tables 4.2-4.5). Students also developed technical skill sets for using cutting-edge

fabrication equipment to build components, CAD software to design products, and

computer programming languages to accomplish unique tasks. Overall, the different

informal STEM activities provided the students’ knowledge and practice, to reinforce the

skills and content; the technical and soft skills went hand-in-hand in the STEM learning

experiences.

Experience Levels

The theme of Experience Levels is made up of two subthemes: Prior Experience

and Skills and No Prior Experience. All of the qualitative data collected from the subjects

of the study showed that a majority of the students had prior experiences learning STEM


130

or participating in formal OST STEM activities; there were only three students who

stated they had no prior experience in STEM learning.

Prior experience and skills. The subtheme of Prior Experience and Skills exists

because a majority of the students had previous experience within STEM learning.

During the interviews, the students spoke of experiences in formal classes learning

computer programming, building, and designing in addition to prior participation in

informal OST activities and personal learning on their own at home (see Tables 4.5).

Participation in prior OST activities included participation in LEGO robotics,

eCYBERMISSION, Boy Scouts of America (specific STEM-related badges), and

Technology Student Association competitions. Furthermore, some students spoke of

participating in summer camps and in online learning such as Hour of Code (2018). All

but three students had prior experiences in a specific STEM activity and had STEM

interests that were fueling their STEM persistence. All of the eighth graders had

participated in a specific OST STEM program in the past, all of the seventh graders

referenced participating in an OST activity or an elective course in school, and two of the

sixth graders stated they had engaged in afterschool engineering activities and attended

summer camps, some participating as early as third grade.

During the interviews, many of the students referenced their previous experiences

and interests in STEM (see Table 4.5). Six of the students spoke of working with LEGOs

and other STEM-related materials, such as Khan Academy’s learning (GWC.I8.486) to

code platform (Sumo.I3.159), at home in their free time (see Table 4.5). One student

described building shooting devices from clothespins, rubber bands, and other household


131

items (Sumo.I7.450). Multiple students referenced the skills and content they had learned

in their OST STEM activities in the past such as soldering, coding, 3D printing, and laser

cutting. During observations, the researcher noted a group of girls put their prior learned

skills surrounding 3D printing into practice by using TinkerCAD to design a custom 3D

printing part for their eCYBERMISSION project (see Table 4.5).

The STEM activities provided students an opportunity to put their prior technical

skills as well as their 21st Century skills into practice and it was apparent that their prior

knowledge and experiences influenced their behavior in the OST STEM activities

observed. During the observations, the students demonstrated prior skills in collaboration

and communication in working in small groups or partnerships in the majority of the

STEM activities to accomplish goals and tasks related to their specific activities, such as

collaboratively designing, building and coding a LEGO Mindstorm EV3 (LEGO, 2018)

sumo-bot robot. Gina stated, “Engineering, in general, is just collaborative learning. I like

working with other people to share ideas and share your knowledge with other people and

teaching them how to do things and learning how to do things” when she was asked about

what she has learned from her STEM activity (ScOl.I10.649). Furthermore, the students

throughout the OST STEM activities demonstrated researching skills and general

problem-solving techniques in developing their robots to provide a functional prototype

to solve a problem. This problem-solving skill set was confirmed in a statement from

Sarah when she was asked if the activity affected her decision to continue with future

STEM activities she stated,


132

The first time where the first machine didn’t work, I was just extremely frustrated.

. . . I continued just with trial and error. It made me want to do it again to know

that I could face the challenge and I could just do whatever I put my mind too.

(ScOl.I14.941)

These prior experiences evidence students finding outlets to bolster their STEM

persistence, learn new skills and improve their prior knowledge. Many students expressed

that their reasons for joining in the OST STEM activities related to enjoying learning

STEM skills and content wanting to continue this process. The questionnaire (descriptive

statistics) provided insight into this developed theme by way of student quotes such as “I

decided to join these competitions because I love to build and engineer,” “I enjoy

building and programming LEGO EV3,” and “I have always been interested in

engineering and building” (see Table 5).

The OST activities provided the students a way to continue nurturing their STEM

persistence as well as their learning. This can be seen when Hamilton was asked to

explain why he choose to participate in his robotics activity he explained, “I showed up

for robotics club, and I just had a knack for it. And I really loved it!” (Sumo.I13.828).

Furthermore, Gina stated, “I did junior solar sprint last year, and just the whole

atmosphere is very similar to the TSA [Technology Student Association] program”

(ScOl.I10.630), when she was explaining her reasoning for joining the Science

Olympiad. Prior experiences and skills were satisfying supporting the students’ interests

in looking for more STEM learning opportunities.


133

No prior experience. The subtheme of No Prior Experience is rooted in the fact

that only one student indicated not having any informal STEM learning experiences as

well as only having STEM learning experiences in formal classroom settings. The

questionnaire (descriptive statistics) data and interviews, suggested that a very limited

number of students had no prior experience in STEM learning (see Table 5).

Interviews. During the interviews, only three students out of 15 reported not have

a prior STEM-focused learning experience in the past that was STEM learning related or

similar to their current OST activity. This was evidenced by a quote from Paul, when

asked about his activity being like anything else he has done previously he stated, “I

didn’t do a ton with engineering and robotics until I heard about it through middle school.

It was kind of a new thing” (Sumo.I1.24). Furthermore, Christopher in robotics,

responded to the same interview question, “No, not really” (Sumo.I4. 224); Gina echoed

Christopher’s response when she said, “Not that I can remember” (ScOl.I10.641). These

three individuals had not been a part of prior OST STEM learning.

Qualitative questionnaire (descriptive statistics). The responses in the

questionnaire (descriptive statistics) data showed that only six students out of the 37 had

only formal classroom experiences in STEM, such as math, science, and engineering

courses (see Table 4.3) and had not participated in a STEM OST activity before. These

six students referenced required courses (e.g., math and science), elective courses (e.g.,

engineering and technology), and advanced school courses (e.g., advanced school courses

(e.g., advanced conceptual physics and honors geometry). On the questionnaire

(descriptive statistics), the remaining 31 students referenced prior experiences: summer


134

camp, after-school activities, and activities in their personal free time. These students

specifically referenced engineering summer camps, OST STEM activities (e.g., Science

Quiz Bowl and for-profit afterschool STEM programs), and tinkering with personal

projects at home (e.g., LEGO building and online coding tutorials). Overall, the

qualitative data showed that a majority of the students had prior experiences, while only a

limited number of students did not have any prior experience in STEM-related learning

activities.


135

Table 4.4

Questionnaire Data

Grade 6 7 8 Gender Male Male Female Male Female Question Responses (n) 3 8 8 6 8 Are your parents now, or were they ever, STEM professionals?

Yes, two (or more) of my parents

- 3 2 - 4

Yes, one of my parents 1 3 1 2 1 No 2 3 3 4 3

Who encouraged you to participate in this activity?

Parent 2 2 4 2 6 Teacher 1 2 7 4 7 Coach 1 - - - 1 Mentor - - - 1 - Sibling - - 1 - 1 Friend - 1 - - 1 Classmate 1 - 1 - 1 Self 1 4 6 4 6

Of these people, whose opinion do you value the most and why?

Parents 1 - 5 - 5 Teacher - 3 2 2 2 Friend 1 1 Self - 4 1 3 2

What STEM activities do you participate in?

Required school classes 1 3 4 2 3 Elective school classes 1 8 7 6 8 Advanced school classes 1 3 3 1 After School Clubs 1 5 5 3 2 Summer Camps 2 1 1 2 2


136

Table 4.4 Continued

Grade 6 7 8 Gender Male Male Female Male Female Question Responses (n) 3 8 8 6 8 What STEM Activities are you currently participating in at your school?

Sumobots 2 2 - 1 - SeaPerch 1 1 - 1 - Drones - - - - - Robotics 3 8 - - - Science Olympiad - - 2 - 6 eCYBERMISSION - - - - 1 Engineering 4 5 2 6 Girls Who Code - - 2 - - MIT Grant - - - - 1 Science - - 1 - 1 Math - 2 1 - 1 Verzion App Challenge - - 1 - 1 Duke TIP - - - - 1 Honors Math - - - - 1 Honors Science - - - - 1 Sumobots - - - - 1 Drones - - 1 1 - SeaPerch - - - - Girls Who Code - - 2 - -

Which of these activities are most important to you and why?

Science Olympiad 2 1 1 1 1 Verzion App Challenge - 1 1 eCYBERMISSION - 1 1 1 1 Robotics - 1 - - Advanced classes - - 2 - 1 Engineering class - - 1 4 Required classes - - 2 - Math class - 3 - 2 Friends - 3 2 2 - I enjoy STEM - 1 - - Parents - 1 - 1 -


137

Table 4.4 Continued

Grade 6 7 8 Gender Male Male Female Male Female Question Responses (n) 3 8 8 6 8 Why did you decide to participate in this activity?.

Personal interest in STEM 1 2 - - 2 Teacher - 1 - - - Elective school classes - 1 - - 2 Teacher influence 2 4 4 4 - OST STEM activity - 1 1 2 Friend - - 1 - 1 Teacher 2 - 1 - - Do not remember - - - 1 -

Thinking back, what event, class, or conversation sparked interest for you in STEM fields/activities? Please feel free to include more than one answer.

Do not remember 1 1 1 1 - Personal interest in STEM 1 - - - - Summer camp - 1 1 - - Science - 2 - - - Engineering class - 1 2 2 1 Sibling - 2 - - - Parents - - 1 - -

At this point in time, do you plan on pursuing a future STEM class or extracurricular activity?

Yes, definitely 1 4 2 - 4 Yes, tentatively - 3 2 - 1 Unsure 2 - 2 - 3 No, tentatively - - 1 - - No, definitely - - - - -

At this point in time, do you plan on pursuing a STEM college major and/or career?

Yes, definitely - 1 - 4 - Yes, tentatively - 4 2 1 3 Unsure - 3 3 1 3 No, tentatively - - 1 - - No, definitely - - 2 - -

Conclusion for experience level. The study revealed that 34 of the students had

prior experiences in learning STEM. These prior experiences varied from formal classes

to OST STEM activities. There were only three students who stated they had no prior

experience in STEM learning.


138


The theme of Not Sure About a STEM Future is made up of two subthemes: Lack

of Interest or Source of Frustration in STEM and Indecisive About Choosing a Future

Pathway. All of the qualitative data collected from the subjects of the study revealed that

a small number of students were not sure about persisting in a STEM pathway or unsure

of their decision on continuing a focus in STEM learning (see Tables 4.2-4.5). Reasons

given by this subset included lack of interest, frustration, and indecision.

Lack of interest or source of frustration in STEM. The subtheme of Lack of

Interest or Source of Frustration in STEM was developed from data showing that a small

group of students attributed their lack of interest was specific aspects of each of the OST

STEM activities and to frustration with those aspects. During the interviews and

observations, these students highlighted and explained specific frustrations, as well as

dislikes, about their OST STEM activities. Some students even assessed their interest

levels with certain aspects surrounding their STEM learning. During the interviews,

Jennifer expressed her frustration with the nature of her Science Olympiad activity when

she stated, “I preferred working on my individual project and learning different ideas

instead of specifically going to one topic” (ScOl.I4.334). Christopher explained, “If the

STEM options had giant teams, I am not sure I’d like to do that because there are too

many activities going on around me. It’s kind of like [when I was in] sixth grade with

FLL [FIRST LEGO League]. There were too many people on the same team”

(Sumo.I4.267). Both of these students referenced in their comments disliking large group

activities and the lack of autonomy with their STEM learning projects and activities. The


139

explanations specifically affected STEM choices; Christopher never participated in the

FIRST LEGO League again after his sixth-grade experience.

Lack of interest and source of frustration were also found in the questionnaire

(descriptive statistics) data (see Table 4.4). These were reported to affect a small number

of student decisions to persist in pursuing STEM learning or activities. Two seventh-

grade students, one male and one female, stated they tentatively did not want to continue

engaging in a future STEM course or activity. Furthermore, Penny (seventh-grade

female) reported that she tentatively did not want to pursue a possible future college

degree or career path in STEM.

This subtheme showed that a majority of the students are interested in pursuing

future STEM learning and that very few students were losing interest due to frustration in

some aspect of their particular STEM activity. During the observations of the activities,

students did lose focus or become frustrated when their projects did not work, such as

when a group of boys’ robotic coding failed and when an eighth-grade girls’ Science

Olympiad windmill blades fell off and broke. However, these students continued to

persist and improve their projects, ultimately overcoming their challenges and developing

working products (see figure). The frustration of their projects not working was seen in

each of the OST STEM activity, but students learned from their mistakes and continued

to improve their work. The majority of the students overcame their frustration through

perseverance, but for a limited amount of students, this reported frustration led in part to

a lack of interest in moving forward with STEM learning.


140

Students’ reported a lack of interest or frustration generated by their STEM

informal learning activities were at times attributed to the number of students assigned to

a group, team, or project as well as the amount of autonomy they had been afforded with

a given project within their OST activity (see Table 4.4).

Indecisive about choosing a future pathway. The subtheme Indecisive about

choosing a future pathway was developed from the data generated by students who

reported that they were unsure about their future choices, including in the short and in the

long term. This portion of the students was still learning and deciding on the pathway

they would choose in their near and distant futures.

Qualitative questionnaire (descriptive statistics). The questionnaire (descriptive

statistics) data showed that seven the 37 students were unsure about wanting to pursue a

future STEM activity or course (see Table 4.4). These seven students included four

eighth graders (three female students and one male student), two seventh graders (both

female students), and one-sixth grader (a male student). In the longer-term,15 of the 37

students were unsure at this time about pursuing college or a career in STEM. These 15

students included 6 eighth graders (three female students and three male students), 7

seventh graders (four female students and three male students), and 2 sixth-grade boys.

Five of these 15 students stated they were unsure about pursuing a future STEM activity,

class, college major, or a career. Of the five students, three were female (2 eighth graders

and 1 seventh grader) and two were male (1eighth grader and 1 sixth grader).

Interviews. The interview data showed uncertainty about future decisions in

STEM learning when students were asked, “Do you see yourself continuing with


141

activities like your STEM activity the rest of middle school, high school, or college?”

Kimmy, a female eighth-grade eCYBERMISSION student, expressed that she was not

sure about pursuing any future STEM activities beyond middle school when she stated,

“Well probably through middle school. I’m not sure about high school”

(eCYB.I15.1002). This same student stated that her current OST STEM activity,

eCYBERMISSION, had opened her eyes to different possibilities when she stated, “I

don’t know after just seeing all the other eCYBERMISSION competitions and

participating in them myself. It has just opened my eyes” (eCYB.I15.1012). Overall, this

student was not sure about her decisions, but has become more aware of her possible

STEM options in the future.

Conclusion for not sure about a STEM future. The study revealed some

students were frustrated by the activities they participated. This frustration for some was

a reason to not persist with the activity they were participating in. Only seven students

were unsure about continuing with STEM in the short-term whereas 15 students were

uncertain about a STEM major or career.


The theme of Sources of Motivation is made up of seven subthemes: Self-

Motivation and Internal Interest; Friends; Family; Teachers; Supporting Others; Outside

of School Organization or People; and STEM Activities and Content. This was the largest

theme (N=428). All of the qualitative data collected from the participants of the study

revealed the presence of a range of influential factors that were motivating students as

well as engaging their learning and participation. The reasons the students were


142

motivated and engaged in their OST STEM activity led to the variety of subthemes for

this theme of Sources of Motivation.

The interviews, observations, and questionnaire (descriptive statistics) data

showed that what inspired and motivated students to pursue STEM the most was their

self-motivation; the motivation they received from family, teachers, and STEM activities

followed (see Table 4.2-4.5). Students were also motivated by others, motivated by the

activity, or motivated by pop culture, such as the STEM movie titled Hidden Figures. A

majority of the students had prior experience with STEM-related activities that had been

inspirational. These activities ranged from school-related, formal classes and formal

community groups such as Boy and Girl Scouts.


143

Table 4.5

Interview Data Topics Related to the Subthemes

Topics Sub-Theme

Number of Students (n) Referencing the

Topic

OST STEM activity supporting student learning and persistence

Promoting STEM Persistence in Middle School

15

Enjoyed their OST STEM activity Enjoyment, Engagement, and Focus 15

Participation in multiple OST STEM activity

Involved in Multiple STEM Activities

8

Working with peers Soft skills (i.e. 21st Century Skills 5

Description of a learned technical skill (i.e. laser cutting, 3D printing, soldering, coding)

Technical skills (i.e. CAD, laser cutting, 3D Printing, soldering, etc.)

11

Description of prior STEM learning (i.e. summer camps, OST STEM activities, personal learning)

Prior experience and skills 12

No reference to prior STEM learning No prior experience 3

Preferred working alone or in smaller groups

Lack of interest or source of frustration in STEM

4

Unsure about participating in future STEM learning opportunities

Indecisive about choosing a pathway 1

Influential friends Friends 6

Supporting and inspiring family members

Family 8

A teacher being a source of motivation and inspiration

Teacher 7

Possibly helping others with their projects

Supporting Others 1

Discussed that STEM learning was important to them

STEM Activities and content 14

None school and family-related sources of motivation (i.e. Boy scouts, Hidden Figures movie

Outside of school organization or people

3

Self-driven and motivated for STEM learning

Self-motivation and internal interest 15


144

Each of the subthemes in the Inspiration theme influenced students’ motivation to

pursue learning STEM content and, in some cases, pursue a possible career in a STEM

field.

Self-motivation and internal interest. The subtheme of self-motivation and

internal interest is the largest subtheme in the theme of Motivation, Inspiration, and

Engagement. This category had, by far, the most frequent (n=134), subtheme. The

students demonstrated and expressed their internal interests for STEM learning and their

self-motivation for pursuing their specific STEM activities in observations, the

questionnaire (descriptive statistics) and in their interviews.

The data collected showed that some students had a strong internal drive, personal

interest, and self-motivation for their inspiration to learn STEM content. Twenty-three

participants listed themselves as a source of inspiration and encouragement for joining

their specific OST STEM activity (see Tables 4.2-4.5). Six students expressed that they

valued their own opinions and thoughts the most when asked whose opinion they valued

most and why. Hamilton explained the importance of following one’s own ideas when he

stated, “It's also important to satisfy yourself and your own desires”. For example, a

seventh-grade female student stated, “I like listening to my own ideas,” and another

student named Paige (seventh-grade female) went on to explain how she valued her own

decision-making: “Myself, because I make the decisions. I choose what I am interested

in” (ScOl.Q24. H). Lastly, Otis, a seventh-grade male student in robotics, assessed his

trust in himself: “I have a lot of trust in me” (Sumo.Q30.H).


145

Qualitative questionnaire (descriptive statistics). The qualitative questionnaire

(descriptive statistics) data confirmed that some students were self-driven and had a high

internal motivation for pursuing STEM learning and activities. 23 of the 37 students

listed themselves as a reason for joining their STEM activity; 13 were girls, and 10 were

boys (see Table 4.4). Five of these 23 students went on to describe how they valued their

own opinions in making decisions and found their own interests with regard to STEM

learning to be important. Examples of these students’ self-values could be seen in the

response to the question, “Of these people, whose opinion do you value the most and

why?” which was a follow-up to the question, “Who encouraged you to participate in the

activity?” Two female students, Kimmy (eighth grader) and Paige (seventh grader)

claimed, “My own because it looked like a good field to be involved in” (ScOl.Q4.QH),

and “I value my own opinion over others' opinions because I am the one to eventually

make the decision” (eCYB.Q24.H).

Interviews. During the interviews, the students explained how their self-

motivation was supporting the drive for engagement in STEM activities. When students

were asked about why they joined their OST activity, Helen stated, “We thought it would

be fun, and we thought it would be good to excel in engineering and grow as engineers”

(ScOl.I2.75). Emmitt, a sixth-grade male student, explained, “I just really like robotics,

and umm I just am really into robots, and I really like coding and yeah. It just really

interests me, yeah, and I just feel great while I’m doing it” (Sumo.I3.152). Price, a

seventh-grade male student, stated, “I’m just kind of interested in engineering altogether

and experimenting with stuff” (Sumo.I7.433). These students expressed how important


146

these OST activities were to them and how their own internal interests motivated them to

become better STEM students.

Other students mentioned similar internal interests for learning STEM. When

asked about having any prior STEM experiences, Price discussed, “I kind of build stuff at

home” (Sumo.I7.441); this reference to tinkering and building small devices on his own

shows his own interest driving participation. Price went on to explain that his choice of

being a part of the robotics groups has pushed him further: “I just kind of started the

building with stuff, and robotics got me more in-depth with engineering; so I just kind of

learned more and started to explore more” (Sumo.I7.451). Amy in Girls Who Code

explained how she was using her activity to explore more and advance her learning when

she stated, “I’m particularly interested in fashion design, and so I feel like that comes

along with engineering now. And I think it’s cool to figure out how things are made and

sort of create stuff” (GWC.I8.516). Amy went on to explain, “I’ll probably do some stuff

online, keep practicing” (GWC.I8.551). She was referencing her prior experiences of

completing online coding tutorials on www.code.org at home, guided by her own internal

interests.

When the students were asked about why they chose to participate in their OST

STEM activities, they expressed their personal interests and self-confidence in making

their own decisions. Hamilton explained his decision to participate in the robotics

program when he stated, “Well when I came to middle school, I’ve always been

interested in engineering, and I just wanted to try something new. And so I showed up for

robotics club, and I just had a knack for it. And I really loved it” (Sumo.I13.828).


147

Another student named Harry (a male sixth grader) explained that he joined the robotics

group due to his enjoyment of programming when he stated, “I enjoy doing it, and I enjoy

programming and inventing and doing it my own way” (Sumo.I9.569). Furthermore,

when asked what it was about programming and inventing that inspired him, he

explained, “I think, it’s just the way, I just enjoy it” (Sumo.I9.570). His statement

suggested that he focused on his own enjoyment and personal connection to the activity.

Harry went on to state that he wants to continue pursuing engineering in the future

through college because “It’s just a place where I feel happy, and it’s something I enjoy

doing” (Sumo.I9.601). Gina (eighth-grade female) discussed that she joined Science

Olympiad to satisfy her competitive nature. Gina stated, “My personal motivation

because I really like being able to create things like how I want them to be”

(ScOl.I10.669). Simon discussed how his parent had explained to him that he had always

had an interest in engineering since he was young: “According to my parents, when I was

little, I liked to watch this engineering show” (Sumo.I11.710).

Observations. Throughout the study, a majority of the students demonstrated

internal drive and expressed an interest in pursuing STEM learning. All of the

interviewed students expressed their interests and motivation for learning STEM content.

These students (15) also demonstrated confidence through body language and spoke of

their personal drive for pursuing STEM. In the observations, students showed the same

motivation for their STEM activities by coming to practices before and after school,

during lunch and recess time (see Table 4.4). A large number of students across all of the

different OST STEM activities were coming in during their free time consistently. This


148

internal motivation was evident in students participating in multiple OST STEM

activities, such as a small group of girls (n=3) were doing eCYBERMISSION, Girls Who

Code, or Robotics, and some boys were doing robotics and eCYBERMISSION or

multiple types of robotics platforms.

Conclusion for self-motivation and internal interest. Overall, the subtheme of

self-motivation and internal interest was the largest subthemes in the theme of

Motivation, Inspiration, and Engagement. A majority of the students modeled and

demonstrated, in the observations, their internal interest for learning STEM content, as

well as their self-motivation to join their OST STEM activity and pursue their projects to

the fullest. Over 60% of the students listed themselves as their source of motivation or

explained how their own interests and internal drive provided them the motivation for

pursuing STEM learning and their specific informal STEM learning program, in

responses to the interviews and questionnaire (descriptive statistics). In conclusion, the

data suggested that the students drove themselves to pursue STEM learning opportunities.

Friends. The subtheme of friends derives from students’ references to their

friends being an influential factor in their participation, motivation, and engagement in

their OST STEM activity. This subtheme was evidenced during the observations of the

STEM activities based on the students’ friendly interactions. The team selection in each

of the STEM activities was also student-driven (see Table 4.5).

Interviews. During the interviews, students made comments about their

connections to their friends and referenced them as a source of their motivation for

joining their OST activity through camaraderie and companionship. Six of the 15


149

students interviewed stated that their friends were influential in their decision to

participate in an OST STEM activity. Paul (seventh-grade male) explained, “I

encouraged some other friends who had done summer camps and other things along those

lines” (Sumo.I1.31). Helen stated, “We thought it would be fun. We thought it would be

good to excel in engineering and grow as engineers” (ScOl.I7.76). Both of the statements

speak to friends wanting to do the activities together.

Students referenced their collaboration with their peers as a source of inspiration

for choosing to do the OST STEM activities. Christopher was asked a follow-up question

regarding why he kept doing robotics since sixth grade. He stated,

I like the team aspect, especially with sumo bots’ three or four-person teams. It

was fun to work with my friends, and I think if it was a project with you by

yourself, it wouldn’t be as fun or as satisfying. (Sumo.I4.219)

Another student explained that his friends decided to join the OST STEM activity as a

group: “We decided this seemed fun, so we started doing it, and then we liked it. So we

continued” (Sumo.I4.219).

Two questions primarily provided insight into students’ characterization of

friends as an influential factor: “Who encouraged you to participate in this activity?” and

“Why did you decide to participate in this activity?” Friendship provided inspiration and

influenced the joining of activities, which was seen when a student stated, “A lot of my

friends are doing it, and they thought it was cool. So, I thought I could do it too. And if I

liked it, I could continue” (Sumo.I7.465).


150

The students reported having friends who were involved in STEM clubs and

activities that had influenced them in a positive manner and inspired them to join a

STEM club (see Table 4.2-4.5). Furthermore, students were finding enjoyment in

completing STEM projects associated with an individual activity, such as Science

Olympiad’s mousetrap car challenge, with peers and friends. Steven (seventh-grade male)

explained that his friends and his enjoyment for STEM led him to join his robotics group

“because my friends do it, and I like STEM”. John (sixth-grade male) explained that his

friends influenced him to join his STEM activity when he stated, “My friend was doing

it,” and he went on to say, “Me and my friend were talking about it, and then I decided”

(Sumo.Q32.K). Otis stated, “my friend said it was fun” (Sumo.Q30.K). Lastly, Ryan

(sixth-grade male) explained that because he valued his friends’ opinion and

encouragement, he decided to pursue his STEM activity.

Questionnaire. The responses from the questionnaire (descriptive statistics) data

included 11 students noting that they joined their OST STEM activity due to a friend’s

influence (see Table 4.4). For example, Kevin, a seventh-grade student, stated that he

joined the STEM activity “because my friends do it,” when asked why he decided to

participate in his activity (Sumo.Q16.K) while Otis stated, “Because my friends said it

was fun” (Sumo.Q30.K), and Jeffery (a sixth-grade male student) stated, “My friend was

doing it” (Sumo.Q32.K). Jeffery went on to say, “Me and my friend were talking about it,

and then I decided” (Sumo.Q32.L). A majority of the comments that pertained to the role

friends’ encouragement and influence played in a student joining an informal STEM

activity came from male students (see Table 4.4-4.5).


151

Conclusion for friends. The subtheme of Friends was the fifth largest subtheme

in the theme Motivation, Inspiration, and Engagement. There were multiple references

(10 from the questionnaire (descriptive statistics) and six from the interviews) by students

about joining an OST STEM or enjoying the teamwork aspects due to friends, such as

small team sizes in eCYBERMISSION and Sumo-bot robotics. Friends had inspired and

motivated middle school students to participate in OST STEM programming.

Family. The subtheme of family as a source of motivation and inspiration comes

from the frequency of references to family members including parents, siblings, and

grandparents (n=48). Through interviews and descriptive statistics, students reported that

family members support their experiences and learning in STEM in addition to their day-

to-day influence (see Table 4.4-4.5).

Parents. A majority of students stated their parents had been an influential or

inspirational factor on their STEM motivation. Furthermore, multiple students spoke

about their parents being in STEM professions. Four questions (two from the

questionnaire (descriptive statistics) and one from the interviews) primarily led to

responses identifying family members as a source of motivation (see Tables 4.4-4.5).

When students were asked, “Who encouraged you to participate in this activity?”,

“Thinking back, what event, class, or conversation sparked interest for you in STEM

fields/activities?”, “Why did you decide to participate in this activity?”, and “Has anyone

helped or inspired you to continue to learn more about STEM concepts?”, 51 responses

referenced family members.


152

Many of the students referenced their parents as a reason why they are pursuing

OST STEM activities. Seventeen out of the 37 students had a parent in a STEM field,

which may be having a positive influence on students’ motivation to pursue STEM (see

Table 4.4-4.5). Seventeen of the 37 students reported that one of their parents was in a

STEM career such as a computer programmer, medical doctor, math or science teacher,

and engineer; Nine of the 17 students reported that both of their parents were in a STEM

field. 16 of the 37 students listed their parents as an encouraging influence for joining

their OST STEM activity (see Table 4.4.-4.5). In the interviews, many students

referenced their parents as a positive influence for them pursuing STEM learning. For

example, Susan (seventh-grade female) in eCYBERMISSION, stated, “I remember

talking with my parents about STEM. They told me what they do in engineering and

computer programming, and that sparked my interest” (eCYB.Q29.L). Furthermore,

Sarah expressed a similar thought when she stated, “My dad was an engineer and always

taught me how to build things, take things apart, and put things back together”

(ScOl.Q19.L).

During the interviews and on the questionnaire (descriptive statistics), students

discussed how their parents were supporting their pursuit of STEM learning. This

positive support from parents engaged and motivated some students to pursue their

STEM learning OST activities. On the questionnaire (descriptive statistics), students were

asked about whose opinion they value; Vern (female 8th grade) explained, “I value my

parents’ opinion the most because they are people who I can talk about STEM to deeply.

They explain to me some about computer programming and engineering” (ScOl.Q15.H).


153

Christopher stated, “I value my parents’ opinion because of their success in their fields. I

have a high appreciation for STEM activities because of them” (Sumo.Q10.H).

Students also reported that they had joined activities because their parents wanted

them to join. Some students explained that they joined their STEM activity at their

parents’ request. Other students took this concept further such as Irene who stated, “they

know what's best for me” (ScOl.Q15.H2) and Emmitt who claimed “My parents

encouraged me to do so” (Sumo.Q34.L). During the interview, Christopher explained,

“My parents introduced me to the idea [OST STEM activities], and I thought it would be

fun. When I was in sixth grade and first did it, I stuck with it through middle school”

(Sumo.I4.215). Christopher’s parent introduced the idea of participating in a STEM

activity, as well as encouraged him to participate. The positive reinforcement supported

Christopher’s engagement in STEM learning. When asked about who had helped or

inspired her to continue learning about STEM, Kimmy explained, “My parents are proud

of me for doing it. So, it’s sort of like good, and it influences me” (ScOl.I15.1021).

Parents are supporting their children and introducing them to STEM opportunities. Two

students (Simon and Amy) explained that their parents looked explicitly for STEM-based

activities.

Five students referenced their older siblings as providing motivation and

inspiration for pursuing STEM learning. Helen explained that her older brother’s

influence was important because he too was participating in STEM learning; her brother

is currently majoring in engineering in college. She stated, “I think what inspired me

personally is my brother, who is in college right now studying engineering and doing an


154

internship” (ScOl.I2.81). Two other students referenced an older sister’s STEM

experiences. Penny (seventh-grade female) stated, “My sister created many fun projects

that I wanted to do, too” (eCYB.Q24.L) and Jennifer said, “Again, my sister did

eCYBERMISSION 3 years in a row and won regionals twice. I was hoping to follow in

her footsteps and was not disappointed” (ScOl.Q18.L). Older siblings were reported as

exposing younger siblings to STEM activities and inspiring them to purse OST STEM

interests.

Siblings. Having an older sibling involved in STEM learning has impacted

students’ engagement in pursuing STEM learning. When asked about how he got

involved with learning computer programming languages, Emmitt, a sixth-grade male

student, referenced his older brother as a source of motivation for wanting to pursue

STEM learning; “My brother influenced me actually on my old computer”

(Sumo.I3.157). Jennifer said, “I know my sister is really into engineering, and she really

wants to be an engineer. And I really want to follow in that path a little bit because she

does a lot of projects that do some of the similar things” (ScOl.I5.342). Penny echoed a

similar idea sentiment, “I also saw some of the projects my sister was doing, and I

thought it would be cool to do them” (eCYB.I6.420). Lastly, Jennifer explained in the

interview how her dad, a computer programmer, had been working with her sister to

support her coding skills (ScOl.I5.349). This influenced her to want to pursue what her

dad and sister were accomplishing.

Grandparents. Grandparents were also cited as inspiring and influencing

students’ choices for pursing OST STEM learning. During the interviews, two students


155

(Amy and Simon) revealed their grandfathers were engineers. Amy explained how her

grandfather’s occupation persuaded her mother to encourage Amy to pursue STEM,

“Well my grandfather is an engineer, so my mom has always been really big in me taking

engineering” (GWC.I8.530). Simon spoke of his grandfather and father both being

engineers and how he wanted to follow in their footsteps (Sumo.I11.691). In all, three

students referenced their grandfathers, particularly their grandfathers’ engineering

backgrounds.

Conclusion for family. The subtheme of the theme Motivation, Inspiration, and

Engagement with the third most references were related to family. Family members were

cited influences on students’ decisions to pursue STEM learning and reported as sources

of inspiration for their wanting to learn and pursue STEM activities. Family members

motivated students to continue learning STEM, provided a positive influence, and

supported students’ engagement in STEM concepts. Many students (n=20) referenced

their family or a specific family member as a reason why they themselves are inspired to

pursue OST STEM activities.

Teachers. The subtheme of teachers includes the observed interactions of the

teacher with their students during the OST activities (see Table 4.2) and the comments

made by the students about their teachers inspiring them to join the STEM activities and

motivating their learning for STEM (see Table 4.4-4.5). Teachers were cited by students

as encouraging them to participate in the OST activities by speaking to students in class

settings, one-on-one, and by promoting OST STEM activities school-wide. Teachers


156

were also cited as supporting students’ motivation for STEM and increasing their

engagement.

Questionnaire. The questionnaire (descriptive statistics) data showed that

teachers inspired some students to join an informal STEM activity and served as a source

of encouragement. When students were asked on the questionnaire (descriptive statistics)

about who encouraged them to participate (see Table 4.4), why did they decide to

participate, and what sparked their interest for STEM activities, 28 references by students

from the questionnaire noted the schools’ engineering teachers and the OST STEM

teachers. Over half, 24 of the 37, of the students responded that a teacher or teachers

encouraged them to join their OST STEM activity (see Table 4.4). Additionally, eight

students found that the STEM teachers’ inspirational attitudes were engaging their STEM

learning.

Interviews. During the interviews, students discussed their STEM teachers being

inspirational and motivating. Helen referenced her engineering teachers at her previous

schools and her current school where the study took place. Amy discussed how her

science teacher who promoted Girls Who Code drove her to join the group. Simon, who

participated in robotics activities, explained that “a lot of people related to engineering”

(Sumo.I11.734), influenced his pursuit of STEM learning. Furthermore, Kimmy

explained how her teacher pulled her aside to recommend her for the Science Olympiad

competition. This student went on to state, “It excited me about it, so I entered and then

continued on this year” (SciOl.I15.981). Other students, such as Helen, Gina, and Sarah

had similar positive experiences with this teacher.


157

Both male and female teachers were cited as an influence on the number of

students joining the STEM activities and their corresponding gender to join a STEM

activity. A number of male and female students were participating in an informal STEM

activity lead by a teacher of their same gender (see Table 4.4). Seven female students

referenced the female teacher who ran the Science Olympiad and eCYBERMISSION

activities during the interviews as being a motivating influence (see Table 4.5). Sarah

stated that she “pushed her and didn’t baby her” (SciOl.I14.946) while Kimmy and Gina

described how she recommended them for the STEM activity of they chose to be apart

for that school year. Because they found enjoyment with their chosen STEM activity,

they continued with the same activity the following year. The female teacher was

referenced by all of the girl participants in Science Olympiad and Girls Who Code. The

boys similarly referenced being influenced by male teachers, as participants of the

robotics groups were primarily boys. For example, the five males referenced the robotics

teacher who ran the robotics group.

Conclusion for teachers. Overall, a large percentage of students spoke about how

their teachers’ teaching styles and creative motivational techniques helped them to be

engaged in their STEM activities and classes. During the observations of the activities,

many of the students were noticeably motivated by teacher comments and feedback. The

positive working environment created by the teachers leading a trusting relationship

between the teacher and the students. As one student stated, “I value my teacher's opinion

because he wanted me to try something, and I liked it after I tried it” (Sumo.Q7.H).


158

Thirteen students stated their teachers were a direct inspiration for their joining their OST

STEM activity (see Table 4.4).

Supporting others and outside of school organization or people. The

combined subthemes of supporting others and outside of school organization or people

had the least number of references in the theme of Sources of Motivation. These

subthemes were developed based on consistent responses from a small number of

students evidenced by in the questionnaire (descriptive statistics) (see Table 4.4) and

interview data (see Table 4.5).

Supporting others. The subtheme of supporting others can be seen in a response

by Robin (a female, 7th grader), who was involved in eCYBERMISSION, to a question

about which of the STEM activities was the most important: “The Verizon App

Challenge, because I got to learn about and try to help people with Down syndrome”

(eCYB.Q23.J). Harper (a female, 8th grader), who had been working with classmates

from a former class on a grant supported by Massachusetts Institute of Technology,

stated, “I really enjoy our project for the Massachusetts Institute of Technology grant

because it has the potential to help people” (eCYB.Q3.J). These two examples evidence

how students reported being motivated and inspired to learn STEM concepts by their

humanitarian desire to help other people. This was also demonstrated when Emily (a

female 7th grader) from eCYBERMISSION stated, “I wanted to be able to make

something that would help other people” (eCYB.Q21.K). Veronica, a Science Olympiad

student who was asked to think back on what event, class, or conversation sparked her


159

interest in STEM learning, explained her work in supporting others in her engineering

class this way,

I was hesitant to take the class and completely ready to drop it for a study hall. But when we were given the opportunity to work in small groups to brainstorm, design and build a prototype, and figure out how to implement an idea/invention that would help our community, I realized how interested I was in the class. I was excited for that period, and making a breakthrough felt so gratifying. I do believe though, that the experience would not have been as fulfilling if we were not granted the freedom that we had been. (ScOl.Q15.L)

Supporting others was cited as a source of inspiration and motivation by a small

group of female students (n=4) when pursuing STEM concepts; there were no references

made by male students about being inspired, engaged, or motivated to pursue STEM

learning activities because of a desire to support other people. Supporting other people

helped motivate some female students to learn STEM concepts with the hopes of

improving other people’s lives and society as a whole.

Outside of school organizations or people. The other low-frequency subtheme

(n=8) for the theme of Source of Motivation is outside of school organizations or people

(see Table 3). This subtheme was derived from the middle school students’ reference of

non-school related people, groups, topics, and organizations which had inspired

motivation for STEM learning. Only a small group of students mentioned this as a source

of inspiration.

There were references in the questionnaire (descriptive statistics) responses and

interview data of specific people who had inspired some students to engage in STEM

learning. For example, there were references to mentors such as Boy Scout troop leaders

(Sumo.I11.734). Sarah referenced one of the female character leads in the movie Hidden

Figures, in response to a question about who has inspired you, when she explained, “The


160

women in Hidden Figures. She is 93 right now. She was a big inspiration because

everyone was telling her she couldn’t do it” (ScOl.I14.954). Sarah went on to describe

how that woman from Hidden Figures overcame obstacles, which is a root source for the

student’s inspiration for learning STEM.

Lastly, students referenced outside organizations (of school or OST STEM

activities) that were inspiring them to pursue STEM. For example, Boy Scouts of

America’s engineering- and science-related merit badges (Sumo.I11.703) and Code.org’s

Hour of Code each served for certain students as a catalyst for wanting to learn more

STEM content (GWC.Q31.L). When asked to explain when and how Harry had first been

inspired to take part in OST STEM activities, he answered, “In third grade, when I did

afterschool with Young Engineers” (Sumo.Q33.L).

Conclusion for supporting others and outside of school organization or people.

These combined subthemes were derived from a small proportion of students. The desire

to help others and individuals outside of the school setting that are connected to STEM

student learning each motivate some students to participate in STEM activities.

STEM activities and content. The subtheme of STEM activities and content had

a relatively low frequency of references (N=75) in the theme of Sources of Motivation.

This subtheme was developed based on consistent responses from students’ referencing

informal STEM activities or STEM-related content being a source of motivation or

inspiration for learning. These responses were primarily seen in the questionnaire

(descriptive statistics) and interview data. Hamilton, a robotics student, explained, “I

would say it’s just that it’s just really interesting to me and every time I do something I


161

look back at it and is ah, wow, that’s really cool” (Sumo.I13.800). Science Olympiad

student Gina explained that she wanted to do more than just her engineering class when

she stated, “I wanted to do something extra for engineering and last year solar sprint was

the thing that appealed to me the most and they needed people [for Science Olympiad]”

(ScOl.I14.611). Gina went on to explain, “It [Science Olympiad] was part of what they

[peer girls] wanted us doing in engineering and I like competing a lot I like comparing

my knowledge to others and seeing what I can do” (ScOl.I14.604). Sarah, when

participating in Science Olympiad, was asked why she chose to participate in her specific

OST STEM activity. She explained her Rube Goldberg project from her activity,

I like engineering as a whole and I also like just building things and trying new

sort of different activities out. I’d always had an interest in Rube Goldberg for

example and I thought it would just be a fun experience to try and build one for

like myself and see what it took and where it went to (SciOl.I14.861)

Some students spoke of how enjoyment of the subject matter led to them being motivated

to learn STEM. Peter (seventh-grade robotics student) explained, “I just like it. I just like

learning and making robots and other stuff” (Sumo.I2.446). Furthermore, he went on to

state, “I think the more I do robotics the more I like it so I will do more STEM activities”

(Sumo.I2.450). Fourteen out of the 15 students interviewed discussed how their

engineering course or informal STEM activity’s content was important to them (see

Table 4). The observations showed students being involved in their activities, self-

selecting activities and having autonomy.


162

STEM activities and content conclusion. Overall, the STEM Activities and

Content subtheme demonstrates motivation for some students based on the specific

activity in which they were participating.

Conclusion for motivation. The theme of Motivation explains student inspiration

and motivation for pursuing STEM. The interviews, observation, and survey data suggest

that the students’ Self-motivation and internal interest was the largest subtheme followed

by family (N=48), teachers (N=81), STEM activities and content (N=75), all being very

close in coding frequency. Friends had a lower frequency than the above subthemes, but

not as low as Outside of School Organizations or Supporting Others. A majority of

students reported a high level of self-motivation and internal drive for wanting to learn

STEM content. A majority of the students had participated in camps, clubs, and other

STEM-related activities that had been inspirational factors. These activities ranged from

school-related, such as clubs and informal and formal classes, to formal community

groups such as the Boy Scouts of America. Each of the subthemes in the theme of

Motivation, Inspiration, and Engagement represent the sources of motivation and

inspiration that provoked students’ engagement in and the pursuit of learning STEM

content.

Summary of the Qualitative Findings

The analysis of the interviews, questionnaire (descriptive statistics), and

observations led to five major themes: Supporting Student STEM Persistence,

Developing STEM Skills and Content, Experience Levels, Not Sure About a STEM

Future, and Sources of Motivation. These themes illuminate key aspects that impacted

students’ interest in STEM and their planned pursuit of it in the future. Subthemes under


163

each theme laid the foundation for creating an understanding of the factors, influences,

interests, and experiences that impacted these students’ decisions to continue learning

STEM content and pursuing activities in these areas. The findings of influential factors

that arose in the analysis of the data may give greater insight into students’ STEM

persistence.

Mixed Method Analysis

The mixed methods data analysis used a side-by-side comparison approach where

the researcher merged the data and compared the quantitative results and the qualitative

findings to gain a more robust understanding of the findings (Creswell & Clark, 2011).

The quantitative findings were reported at the construct level, and the Science and About

Yourself sections demonstrated that the OST STEM activities had a statistically

significant impact on the students’ attitudes toward science, their awareness of the

academic performance in their class, and their awareness of the people they know who

are STEM professionals. The students' significant change towards their science attitude

can be compared to the themes of Supporting Student’s STEM Persistence (N=203) and

Experience Levels (N=59). These themes support that STEM activities provided students

an environment in which to engage in and enjoy learning about STEM content and served

as a source for engagement and enjoyment in STEM learning. Similar results are rooted

in the quantitative analysis when the participants referenced family members, teachers,

and people from outside school activities (i.e. Boys Scout trooper leaders

[Sumo.I11.734]) that are STEM professionals. Seventeen students had a parent in a

STEM field, and another four students referenced other influential relatives in STEM


164

fields. 24 of the 37 students reported that their teachers encouraged them to join their

OST STEM activity, which could be a potential driver of increased student awareness of

teachers being STEM professionals.

The analysis of the item-level survey data showed that there were a few

statistically significant questions. A students’ perspective on considering science as a

future career option and the belief that learning to engineer can help the students improve

items people use every day each changed positively; conversely, the feeling that doing

advanced math is difficult changed negatively between the pretests and posttests. More

than half of the students’ (n=20) perception that learning engineering can improve things

people use every day increased, which was likely supported by the observations showing

that the students were learning technical and soft skill sets (n=78) and the students’ self-

reporting during the interviews (n=15) of the different STEM-related skills they were

learning. The increased access to the subject matter may have influenced their perception

that engineering helps things create things people use every day. Additionally, a large

number of students (n=22) participating in multiple OST STEM activities, supporting

their awareness of different STEM topics and fields. The qualitative themes of

Developing STEM Skills and Content (N=203) and Supporting Students STEM

Perspectives (N=111) supported the quantitative findings by explaining the skills students

were learning and content students were engaged in which motivated their learning of

STEM.

Overall, the quantitative findings showed that the OST activities did not impact

student perceptions of STEM subject areas, 21st century learning or their future career


165

decisions as a whole. However, the qualitative data found that 28 students planned on

participating in a future STEM activity and a majority (n=13) of the students interviewed

stated they would consider going to college for a STEM career. Furthermore, the

qualitative data contradicted the quantitative findings on the item-level for 21st century

learning through the subtheme of Soft Skills (N=46) in which the data showed the

teamwork and problem-solving aspects of the OST STEM activities. The largest

qualitative theme of Sources of Motivation (N=428) and its subthemes support the lack of

significance found in the quantitative analysis towards the STEM content attitudes not

changing due to variety of motivational sources influencing the students: Friends (N=41),

Family (N=48), Teacher (N=81), Supporting Others (N=3), STEM Activities and Content

(N=75), Outside of School Organization or People (N=8), and Self-Motivation and

Internal Interest (N=134).

Chapter Summary

The research findings from the qualitative and quantitative research have

enhanced understanding of the influence of the OST STEM activities on the students’

persistence for STEM learning. The qualitative and quantitative findings provide insight

into this phenomenon. The five major themes of Supporting Student STEM Persistence,

Developing STEM Skills and Content, Experience Levels, Not Sure About a STEM

Future, and Sources of Motivation illuminated key aspects of how students’ interest and

motivation were impacted along with how the activities aided students in learning 21st-

century skills and STEM content. The qualitative data also provided an understanding of

the factors influencing the middle school students’ interests and motivations for STEM


166

learning, as well as their STEM learning pursuit. The S-STEM Survey showed that the

OST activity had no statistically significant impact on the students’ STEM persistence.

The analysis of the survey data did show that students’ perception of considering science

as a future career option and understanding that engineering can help the students

improve items people use every day all changed positively. However, the perception that

doing advanced math is difficult changed negatively between the pretests and posttests.

The mixing of the data has shown that the OST STEM activities are influencing students’

motivation and interests for STEM learning and persistence towards a possible career in

science, as well as the students learning 21st Century skills.


167

CHAPTER V

DISCUSSION, IMPLICATIONS, LIMITATIONS, AND RECOMMENDATIONS FOR FUTURE RESEARCH

This research study sought to describe changes in middle school students’

aptitude for 21st century skills and their motivation for, interest in, and perceived

persistence in STEM after a 13-16-week participation in at least one of the following

OST STEM learning: eCYBERMISSION (2016), Science Olympiad (2017), Girls Who

Code (2017), and a robotics group (involving sumobots and drones). The study focused

on 37 middle school students (16 females and 21 males) in sixth (5), seventh (18), and

eighth (14) grades, all of whom participated in one or more OST STEM activities at an

independent, private school in a metropolitan city in the Southeastern United States. The

researcher studied the affective and influential factors of eCYBERMISSION (2016),

Science Olympiad (2017), Girls Who Code (2017), and a robotics group (sumo-bots and

drones). The researcher investigated the role that students’ experiences in the OST STEM

activities played in the students’ reported motivations, interests, and persistence in STEM

using proxy measures such as pre- and post-surveys, one-on-one interviews,

observations, and an inventory of in which and how many STEM courses middle school

students chose to enroll. The study aimed to highlight the importance of OST STEM

activities and their role in supporting middle school students in developing a STEM

identity, leading to the student pursuing STEM high school courses, college majors,

and/or careers. Furthermore, the knowledge gained from this study may inform best

practices in OST STEM activities and education.


168

Discussion of the Results

In this chapter, the results are grounded in the extant research literature to confirm

or refute previous findings. These findings are discussed with their potential implications

for research and practices. The interpretations, conclusions, and recommendations in this

chapter are based on major (significant) mixed methods research findings related to each

stated research question, respectively.

Research Question #1: Change in Perceptions of and Actions Toward STEM Persistence

Research question #1 focused on the change in middle school students’

perceptions (descriptions) of and actions (enrollment) toward STEM persistence. This

research question had two sub-questions that focused on the type and a number of current

middle school STEM courses in their formal schooling and future STEM courses in their

formal schooling. The questionnaire (descriptive statistics), interview questions,

observations, and the S-STEM Survey (FI, 2012), results were used to determine the

change in the middle school students’ perceptions of and actions toward STEM

persistence. The results from the study indicated there was a significant change in the

students’ views towards STEM persistence after participating in the OST STEM activity.

The results suggest that the OST STEM activities supported student STEM

learning and persistence in pursuing future STEM learning experiences. Twenty-four of

the 37 students stated they wanted to participate in a future STEM activity (formal or

informal) in middle or high school. Furthermore, 17 students reported they were planning

on or interested in attending college as a STEM major with the intent to pursue a career

that had a STEM focus. This confirms prior research, as Mohr-Schroeder et al. (2014)


169

found similar results with 99% of their middle school participants (N = 144), who

participated in a summer STEM camp, wanting to attend a future STEM activity (i.e.,

STEM summer camp). Like this middle school OST STEM study, previous studies

related to OST STEM activities have shown that these activities provide support for

students’ motivation for STEM learning (Bull et al., 2008; Dabney et al., 2012;

Leblebicioglu et al., 2017; Stocklmayer et al., 2010). This early exposure to STEM

learning also supports students’ motivation for future STEM learning (Wang, 2013).

The OST STEM activities provided students with a resource for pursuing STEM

learning. A majority of the students (N = 25) who participated in the OST STEM

activities had more than one year of experience participating in the OST STEM activity

for which they were being studied, particularly the seventh- and eighth-grade students. 21

of the 37 students reported participating in more than one of the STEM OST activities as

a proxy for STEM persistence, which is an example of how these middle school OST

STEM activities are supporting and promoting students’ STEM learning and persistence;

the STEM learning experiences are shaping the middle school students’ interest towards

STEM because the OST STEM activities are engaging, fun, and hands-on (Hayden et al.,

2011; Mohr-Schroeder et al., 2014; Nugent et al., 2010; Paulsen, 2013). The study also

revealed that a majority of the students participating in the OST STEM activities had

prior experiences in learning STEM or participating in other OST STEM activities—only

three students stated they had no prior experience in STEM learning. This draws

important corollaries between the participation in OST STEM activities and STEM

persistence as the continued participation in these activities grows the STEM pipeline


170

while impacting the students’ STEM interests, motivation, and identity (Hugh et al.,

2013; Hite et al., 2018; NRC, 2015).

The findings from the S-STEM Survey (FI, 2012), showed that the OST activity

did not have a statistically significant influence on the students’ views towards their

learning of their individual formal STEM content area classes, 21st Century learning, or

their future career decisions. However, analysis of the S-STEM Survey data showed that

after participating in the OST STEM activities, students’ attitudes towards science were

positively influenced, as were perceptions of how well they would perform in science and

the perception that learning engineering would enable them to improve things people use

every day. Conversely, after the OST STEM activities, the students’ views on their own

ability to do advanced work in math declined.

Subquestion #1. The first sub-question for the first research question was: Type

and number of current middle STEM courses in their formal schooling? There were no

significant changes between the students’ self-reported data on the S-STEM survey (FI,

2012) related to performance in their formal STEM classes which could be due to the

students’ high level of interest and self-motivation for STEM learning that already

existed for these particular students. All of the students in the study participated in formal

math and science courses for their respective sixth, seventh, and eighth grades that took

place daily for 50 minutes each. Furthermore, all of the seventh-grade (18) and eighth-

grade (14) students participated in a formal engineering middle school elective course,

while the sixth-grade students (6) participated in a 2-week Scratch (2017), a

programming unit that introduces students to the fundamentals of computer


171

programming. The middle school engineering courses are sorted by grade level (seventh-

or eighth-grade) and gender. Students (i.e., seventh- and eighth-graders) attend an

engineering course designed for their grade level and gender, and the gender of the

course’s educator matches that of the students. A majority of these students had a high

level of self-motivation for STEM learning, which could also help to explain why there

was no significant self-reported changes in student’s performance in their formal math

and science course work.

The engineering courses take place in a Fab Lab, where students learn about the

principles of engineering design and different fields of engineering and engage in project-

based learning using digital fabrication tools such as laser cutters, 3-D printers, and vinyl

cutters. It was apparent that the students’ skills and knowledge gained in the formal

engineering courses were applied in the OST STEM activities. For example, a Science

Olympiad group of girls used the laser cutter to fabricate windmill blades, and the

eCYBERMISSION teams used the 3-D printing techniques with their projects. Without

prior knowledge and exposure, students would not have had the same level of technical

aptitude when participating in the OST STEM activities. Furthermore, all of the teachers

of the OST STEM activities were also math, science, or engineering teachers at the

middle school, which could support the content aspects and possible student

encouragement for participation outside of the OST STEM activities.

In addition to engineering, students referenced other courses and OST activities.

Students referenced their formal science, math, and engineering courses as STEM classes

that they were participating in when asked, “What are the specific names of the STEM


172

activities that you participate in?” Two students reported working on a Massachusetts

Institute for Technology grant, and another referenced participation in the Duke

University Talent Identification Program that supports academically talented students in

Grades 4–12 with additional learning resources. Finally, another student referenced the

Version Innovation Learning App Challenge, which involves the brainstorming of app

ideas by teams of middle- and high-school-age students to help solve real-world

community problems with the chance of winning money and working with professional

app developers. Prior research has found that students’ interest for STEM learning is

positively impacted from active participation in engaging OST STEM activities, such as

afterschool programming (Krishnamurthi et al., 2014) and summer camps (Mohr-

Schroeder et al., 2014; Nugent et al., 2010), which could explain why the students in this

study are choosing to pursue other STEM learning opportunities.

Subquestion #2. The second subquestion for the first research question was: Type

and number of future STEM courses in their formal schooling? This subquestion focused

on the type and number of future STEM courses in the students’ formal schooling. After

participating in the OST STEM activities, all of the eighth-grade students (14) registered

for a formal high school engineering course for their freshman year of high school.

Furthermore, four of the five 6th-grade students registered for a seventh-grade

engineering course, and 17 of the 18 seventh-grade students registered for an eighth-

grade engineering course. This data alone demonstrates that majority (N = 35) of the

students sought future participation in a formal elective STEM course, in addition to their

formal science and math courses for the following school year. Of the two students who


173

did not seek future STEM learning, only one discussed not wanting to participate in an

OST STEM activity or a formal STEM-elective course in the future; the other student

transferred to a different school where the same electives weren’t offered. Similar results

were found in a prior study conducted after students participated in an OST science club

where 80% of the subjects wanting to take future formal STEM courses at school

(Krishnamurthi et al., 2014).

The OST STEM activities in this study did influence the students' self-reported

STEM persistence for wanting to participate in future STEM courses in the formal

setting, in which the majority of the students registering for an engineering class as well

as formal math and science courses for the next school year. A majority of the middle

school students participated in an OST STEM activity the following school year, except

for two students (one female seventh grader and one male seventh-grade student). These

results indicate that the OST STEM activity influenced the students’ STEM persistence

for participating in future STEM learning.

Summary for question #1. The results indicated that there was a significant

change in the students’ views towards STEM persistence. This finding showed that some

students (N = 24) expressed a desire to participate in future STEM activities and learning

opportunities. Furthermore, 17 of the 37 students reported that they were currently

planning on or wanted to pursue a STEM pathway long-term. Lastly, only one student

indicated during the study that she did not want to participate in an OST STEM activity

in the future; only seven students indicated they were unsure about future participation.

Though they did not all report continuation in the data set, students (N = 36) wanted to


174

continue with STEM courses and OST STEM activities, as a majority of the students

registered for future engineering courses and five sixth-graders and 16 of the 18 seventh-

graders indicated they wanted to participate in their same OST STEM activity the

following school year – this question did not apply to eighth-graders as the OST STEM

activities are middle-school specific. Furthermore, all but two of the 37 students

participated in a formal elective course or an OST STEM activity the following school

year, which supports previous research suggesting OST STEM activities, like the

activities in this study, are key factors in enhancing STEM motivation (Holmquist, 2014;

Wang, 2013), interests (Mohr-Schroeder et al., 2014; Nugent et al., 2010), and

persistence (Afterschool Alliance, 2015; NRC, 2015; NRC, 2009).

These results are significant due to findings suggesting that the OST STEM

activities supported student STEM learning and persistence in pursuing future STEM

learning experiences. Furthermore, the large amount of students participating in formal

elective courses and OST STEM activities the following school year suggests that the

OST STEM activities did impact the students’ STEM persistence positively, as well as

provided the self-motivated students an outlet for learning STEM. Lastly, the self-

reporting of data by students provides a new perspective to the overall body of research

on STEM persistence with middle school students and provides OST STEM activities a

deeper understanding of the topic.

Research Question #2: Alter 21st Century Learning Skills, Motivation, and Interest In STEM Careers

Research question #2 pertained to the alteration of middle school students’ 21st

Century learning skills, motivations, and interests in STEM careers after participating in


175

their OST STEM activity. The results indicated there was a change in the students’ 21st

Century learning skills, motivations, and interests in STEM careers. The questionnaire

(descriptive statistics), interview questions, and the S-STEM Survey results were used to

determine the change in the middle school students’ attitudes toward STEM. The

quantitative findings showed that the STEM activity had not impacted the students’

interests, motivations, 21st Century learning, or their future career decisions as a whole,

though this could be partially attributable to the small sample size. The qualitative

findings, though, did indicate a shift in students’ interests, self-motivation, and

participation in 21st Century learning.

21st century skills. Previous research has ascribed that 21st Century skills are

highly important for the future workforce (Atkinson & Mayo, 2010; Carnevale et al.,

2011; Palmer et al., 2010), are important for students’ STEM learning (Brazell, 2013;

P21, 2015) and are successful in STEM learning and careers (Atkinson & Mayo, 2010;

Palmer et al., 2010). All of the qualitative data collected suggest that the students gained

a variety of STEM skills rooted in 21st Century learning through participation in their

OST STEM activities. The OST STEM activities provided students with opportunities to

develop and practice 21st Century skills including collaboration, creativity, problem-

solving, and communication. These 21st Century skills were utilized frequently by the

students in each of the OST STEM activities, as the nature of the activities were team

focused and the teachers of the activities supported and encouraged communication and

collaboration between the individual groups. The collaborative projects and specific

outcomes of each OST STEM activity provided students with opportunities to derive


176

solutions to different challenges and to solve problems. Furthermore, the OST STEM

activities developed an interest in technical skills including the use of digital fabrication

equipment, CAD software, and computer programming languages to accomplish unique

tasks for defined problems; this has been recommended but not studied (Hossain &

Robinson, 2012). Prior research about STEM summer camp activities has shown

informal hands-on learning experiences in STEM, exposure to STEM technology and

team collaboration increased student desire to pursue STEM (Ayar, 2015; Mohr-

Schroeder et al., 2014; Nugent et al., 2010). Research has similarly shown that college

students benefit from using hands-on learning experience involving STEM content as it

supports a sustained interest in STEM (VanMeter-Adams, Frankenfeld, Bases, Espina, &

Liotta, 2014). The variety of OST STEM activities provided students’ knowledge and

practice in a learning environment to reinforce the skills and content learned in formal

classes, as well as those introduced in the OST STEM activities. This supports similar

findings with regards to OST STEM activities reinforcing students’ skills and content

learned in formal classes, as well as providing students the opportunity to dig deeper in

STEM content as extension of the formal classroom (Newbill et al., 2015; PCAST, 2010;

Peters, 2009; Sahin et al., 2014).

The OST STEM activities also offered students opportunities to practice and

develop 21st Century skills in a safe environment, one that allowed them to struggle and

learn from their experiences. The researcher observed frustration and signs of students

being overwhelmed during the projects they were completed during the OST STEM

activities. This frustration was seen in students of all grade levels, both genders, and in all


177

OST STEM activities. Examples of student frustration include instances when their

projects did not work or there was disagreement between teammates on the direction of

their project. Teachers intervened when students were struggling and frustrations

mounted, which supported their development and attainment of these 21st Century skills.

Overall, this frustration allowed for real-world learning, problem-solving, and teamwork.

This perseverance through frustration is an example of the authentic learning that is

necessary to prepare students for the future workforce (Holmquist, 2014; Mohr-

Schroeder et al., 2014; Nugent et al., 2010; P21, 2015). Furthermore, it confirms the

importance of providing students with opportunities to gain teamwork skills, heighten

their STEM career awareness, engage in authentic research, and hone problem-solving

skills with pertinent resources (Ayar, 2015; Hughes et al., 2013; Sahin et al., 2014).

All of the OST STEM activities in this study used project-based learning, which

has been shown to successfully develop 21st Century skills in middle school students

(Bell, 2010; P21, 2015). The students participated in hands-on, inquiry-based learning in

a collaborative environment where communication, problem-solving, and creativity, all

important factors that allowed them to develop 21st Century skills while gaining an

understanding of STEM concepts (Brisson et al., 2010; P21, 2015; PCAST, 2010). The

project-based learning facilitated the application of 21st century learning skills and

scientific reasoning and the gaining of an understanding of STEM content topics and

possible STEM career fields (Hite et al., 2018; NRC, 2015; Sahin et al., 2014; PCAST,

2010; Wyss et al., 2012; Weber, 2012).


178

The OST STEM activities provided the students with a platform for collaborating

and working in teams to solve problems. Students referenced their friends and the

collaborative team environment as factors contributing to their enjoyment of their OST

STEM activity. Students made comments about their connections to their friends and

referenced them as a driver for joining their OST STEM, along with the potential

opportunities for teamwork, collaboration, camaraderie, and companionship. Six of the

15 students interviewed stated that collaborating with their friends during the OST STEM

activities was influential in their decision to participate in their specific OST STEM

activity, which seems to indicate the importance of the collaborative environment of the

OST STEM activities. This supports prior research, which shows that collaborative

STEM learning is essential for preparing students for a global economy (Marzano &

Heflebower, 2011) and STEM workforce (Ayar, 2014; BLS, 2017; Sithole, et al., 2017)

and can also support students’ interests in STEM (Abermathy & Vineyard, 2001; Brown,

2016; Modi et al., 2012; Mohr-Schroeder et al., 2014; PCAST, 2010; Weber, 2012).

Furthermore, positive peer relationships in a STEM learning environment that fosters a

sense of community has been shown to enhance student success and learning of STEM

(Smith, Douglas, & Cox, 2009). This research resulted in similar findings, as the OST

STEM activities provided students the opportunity to collaborate with peers in a positive

way, which supported the development of 21st Century skills.

Overall, the 21st Century skills were modeled, taught, and practiced during these

informal STEM activities. These activities even provided students with the opportunity to

make mistakes while learning and, thus, grow from them. As seen in this study,


179

specifically the qualitative portion of the research, and in prior research, OST STEM

activities can support students’ 21st Century learning skills and assist them in developing

skills necessary to be successful in a STEM career (Hite et al., 2018; NRC, 2015; Wyss et

al., 2012). The OST STEM activities’ projects had the students engaged in collaboration,

creativity, critical thinking, and communication. All of the OST STEM activities had the

students collaborating in teams that involved important communication with regard to

planning, decision-making, and the logistics of completing their projects on time. For

example, the eCYBERMISSION teams worked in small groups and discussed their

approaches to overcome their specific project solutions collaboratively. Each OST STEM

activity offered students projects that involved challenges and different problems that had

the students problem-solving with critical thinking and creativity. For example, the sumo-

bot groups had to fix code and hardware issues that were presented to them while

designing their robots. The OST STEM activities had the students practicing these skills

repeatedly. In conclusion, the OST STEM activities provided the students the opportunity

to develop 21st Century skills, as well as practice them.

Motivation. All of the qualitative data collected suggests that some students

pursued STEM learning because of an internal drive to learn the content the OST STEM

activity was teaching. Fourteen out of the 15 students interviewed discussed how the

content of their engineering course or OST STEM activities was important to them. In

addition to being motivated by the content, students were also motivated by themselves

and their internal interests. This self-motivation was reported by a majority of students;

the OST STEM activities, with self-directed learning for students pursuing their STEM


180

interests, supports student motivation and persistence for STEM as seen in prior research

(Ayar, 2015; Mohr-Schroeder et al., 2014; Nugent et al., 2010; Sladek, 1998). Also, since

OST STEM activities are self-selected, students are able to pursue subjects by which they

are motivated (Deci et al., 1981; Rigby et al., 1992). Twenty-three of the 37 students

listed themselves as a reason for joining their STEM activity. This self-motivation was

listed by all grade levels and both genders. For these students, the enjoyment of the

activity was motivating them to continue pursuing STEM learning.

In addition to themselves, the students identified teachers as sources of motivation

for STEM learning. Twenty-four of the 37 students responded that a teacher or teachers

encouraged them to join in their OST STEM activity and this encouragement supported

their self-motivation for participating in OST STEM learning. Students found that the

STEM teachers’ motivational styles of teaching and welcoming attitudes shaped an

engaging learning environment. Furthermore, all of the teachers of the OST STEM

activities were STEM content teachers of middle school courses at the school, which

provided the opportunity for the teachers and students to develop a positive rapport as

well as allowed the teachers to encourage students to participate in an OST STEM

activity. These STEM teachers could possibly be viewed as recruiters for their OST

STEM activities through the relationships they develop through their formal STEM

courses. This study had similar findings to those of prior research on the topic of STEM

persistence, as the researcher of the present study discovered that STEM persistence was

influenced by teaching approaches, positive learning environments created by teachers,

the use of interesting curriculum, and rapport between teachers and students (Gasoewoski


181

et al., 2013; Holmquist, 2014; Jensen & Sjausted, 2013; Mohr-Schroeder et al., 2014;

Rigby et al., 1992). There were several references (10 from the descriptive statistics and

six from the interviews) made by students about the engagement of their STEM OST or

enjoying the collaborative environment aspects, which possibly can be related to the

learning environment created by the teachers.

The students also reported family support, including that of parents, siblings or

extended family, driving motivation. Additionally, many of the students had family

members in STEM fields influencing them. Seventeen out of the 37 students had a parent

in a STEM field, and other students referenced influential siblings, grandparents, and

relatives, also connected to STEM fields who were providing motivation for these

students. The finding that parents influenced their children’s motivation to pursue STEM

learning is similar to that of prior research on the influence of family and socioeconomics

on STEM learning and STEM career awareness (Archer et al., 2012; Archer et al., 2010).

Students referenced their parents as a generally positive influence for them pursuing

STEM learning opportunities.

Lastly, a small number of students discussed that their friends were a motivational

factor for being in an OST STEM activity. This suggests that the OST STEM activities

could be an outlet for students’ external motivational influence.

Interest. A majority of the students had previous STEM-related informal and

formal learning experiences through participation in camps, clubs, and other STEM-

related activities. These previous experiences were cited as factors which inspired certain

students to develop STEM persistence through the progression of interest in STEM


182

learning. These activities ranged from school-related learning, both formal and informal,

to participation in community groups such as the Boy Scouts of America; despite the

avenue, the activity influenced these students’ pursuit of learning STEM and provided

inspiration to continue that pursuit. Previous research has found that the Boy Scouts of

America’s model provides students a positive and supporting OST learning environment

that is impactful due to the influential role models they provide students (Hersberg et al.,

2015) which could possibly explain why it influenced the participant of this study,

Simon.

These OST STEM learning experiences influenced these students’ pursuit of

STEM learning by providing an inspirational and sustainable outlet for learning,

developing personal interest, and building self-motivation for STEM content. The data

from this study also suggest that the OST STEM activities provided a resource for

developing an interest in and gaining 21st Century skills. However, the S-STEM Survey

results indicated no significant change in the students’ responses between the pretest and

posttest after participating in their OST STEM activity.

Gender and grade-level outcomes. Overall, the OST STEM activities influenced

the students’ attitudes toward science, their awareness of their academic performance in

school and who they knew in their daily lives that are STEM professionals. The boys’

data indicated statistically significant growth in their awareness of their own academic

performance in their formal classes after participation in the OST STEM activities, as

well as their awareness of STEM professionals they know. The girls’ data showed

statistically significant changes in their attitudes toward science, which suggests that the


183

OST STEM activities positively impacted their perception of science. This is an

important change for the girls’ development of their STEM identity (Barton et al., 2012).

The sixth-grade and seventh-grade students showed no statistically significant

change between the pretest and posttest scores, indicating that the OST STEM activities

did not influence them on any of the S-STEM categories. The eighth-grade students did

show increased awareness of their own academic performance in their formal classes

after participation in the OST STEM activities. Eighth-grade students also reported

increased awareness of STEM professionals they know, which possibly be due to the

student population being introduced to a variety of STEM professions from the teachers

and OST STEM activities. Eighth-grade students also reported an increase in perception

of science after participation in the OST STEM activity. It is possible the multiple years

of OST STEM activity experiences reported by eighth-graders may explain the eighth-

graders’ change in their science attitude were affected by the OST STEM activities, as

they had more than one year of experiences and exposure to STEM learning. This may

suggest that multiple years of OST STEM activity experience could increase middle

school students’ perceived STEM persistence, interest, and motivation for learning

STEM content; this confirms prior studies which have found that early access to STEM

learning (DeJamette, 2012) and student participation in multiple learning activities

increase students’ interest in STEM (DeJamette, 2012; Reynolds et al., 2009).

The breakdown of the boys’ data at the construct level showed an increase in their

awareness of the academic performance and of STEM professionals in their lives, which

may be attributable to the OST STEM activities involving them in a variety of STEM


184

content learning and careers. The girls’ data at the construct level show an improvement

in attitudes toward science. This change could be attributed to the OST STEM activities

providing a positive female role model in a STEM education position in charge of their

OST STEM activities, confirming findings from prior research (Dubetz, & Wilson, 2013;

Weber, 2011). This change is also potentially explained by increased awareness of

different science careers due to participation in the OST STEM activity, which also

confirms prior research which has shown exposure to potential careers can increase

students’ interests and attitude towards STEM career fields (Christensen et al., 2015;

Wyss et al. (2012).

Paired means t-test. The quantitative results indicated that OST STEM activities

changed the students’ perspectives on the following questions: considering science as a

future career option and learning engineering can help the students improve items people

use every day. The quantitative results also showed that students’ perceptions of the

difficulty of doing advanced math changed negatively from pretest to posttest, indicating

that the OST STEM activities made students less confident in their ability to do advanced

math. Overall, the quantitative findings from the statistical analysis suggested that the

OST STEM activities had specific areas of significant impact on the middle school

students’ perceived persistence for STEM, but a majority of the item-level results showed

no significant affect. Prior research has refuted this finding (Ayar, 2015; Mohr-Schroeder

et al., 2014; Nugent et al., 2010). The study also discovered that the student population

had a high level self-motivated with large amounts of prior experience in STEM learning,


185

which may have led to a sample of students who had more strongly developed their

STEM identity; as such, the change in STEM attitudes would be slighter.

Wilcoxon signed-rank test. As discussed in the gender and grade-level sections,

the Wilcoxon signed rank test did find construct-level significance overall, by grade

level, and by gender for a few questions on the S-STEM. The data indicated that the

students’ perceptions had changed with regard to considering a science-based career,

awareness of STEM professionals, and the ability of engineering to improve peoples’

lives well as they had become more aware of adults who are mathematicians. This

confirms prior research, which has shown that students increase their interests and

attitude towards STEM career fields after becoming aware of STEM careers and

participating in engineering content and career awareness units (Christensen et al., 2015;

Reynolds et al., 2009; Wyss et al. (2012).

STEM identity. These OST STEM activities were possibly influential in the

support of middle school students’ development of their STEM identity (Archer et al.,

2010; Hughes et al., 2013). Prior research suggests that middle school students begin to

form their STEM identities in middle school (Hughes et al., 2013) and also develop their

interests towards and possible future career in STEM (Afterschool Alliance, 2015; Archer

et al., 2010; Brown, 2016; Sahin, 2013). It has also shown that the development of a

STEM identity is extremely important for middle school females due to the impact on

their interest and motivation for STEM (Barton et al., 2012). The data suggest that the

OST STEM activities supported students’ development of a STEM identity development

by providing them the opportunity learn STEM concepts; furthermore, the OST STEM


186

activities afforded them opportunities to be encouraged by their teachers to pursue STEM

learning in a motivating environment, as reported by the students.

Twenty-four of the 37 students reported their teachers encouraged them to join

their OST STEM activity, which suggests the importance of an OST STEM activity

teacher’s role in supporting students’ interest and motivation for STEM learning. Prior

research has shown that teachers and their decisions are highly influential on students’

learning, interest in and motivation for STEM as well as their overall STEM persistence

(Gasiewski et al., 2012; Holmquist, 2014; Jensen & Sjaastad, 2014; Makhmasi et al.,

2012; Woolnough, 1994a, 1994b). This impact of teachers was especially significant for

the girl participants, as seven female students referenced the positive impact of their

female teacher who ran the Science Olympiad, Girls Who Code, and eCYBERMISSION

activities when asked about motivating factors in their interviews.

The enjoyment of the OST STEM activities’ content, reported by 14 of 15

students interviewed, also supported the development of students’ STEM identities. This

enjoyment of STEM content was also evidenced by participant enrollment in OST STEM

activities, as 21 of the 37 overall participants were involved in multiple of the OST

STEM activities studied. Furthermore, 25 students had more than one year of experience

participating in their OST STEM activity, suggesting that the OST STEM activities were

increasing the students’ interests in and motivation for STEM learning and ultimately

supporting their STEM identity development. Prior research has found that this is

important, as prior experiences, early STEM access, engaging curriculum, positive

teacher role model, and quality instruction have all been shown to improve long-term


187

STEM persistence (Andersen & Ward, 2014; Graham et al., 2013; Maltese & Tai, 2011).

The OST STEM activities’ support the development of middle school students STEM

identity also confirms prior research which has shown that this which supports the

students having higher persistence towards a future STEM pathway (Afterschool

Alliance, 2015; Archer et al., 2010; Brown, 2016; Sahin, 2013).

In conclusion, the data suggest that the OST STEM activities supported the

students’ STEM identity development in a positive way. This may be partially

attributable to the structure of the OST STEM learning experiences, as prior research has

shown that innovate instructional practices (Espinosa, 2011; Hite et al., 2018), such as the

student-driven projects offered by the OST STEM activities in this study and the positive

learning environments they created free of negative influences, such as stereotype threat

(Shapiro & Williams, 2012), could have supported the development of students’ STEM

identities. For example, the Girls Who Code activity and the female instructor attracted

high numbers of female participants, confirming prior research findings surrounding

same-sex instruction (Ahmed, 2016) and same-sex informal STEM activities (Hite et al.,

2018; Hughes et al., 2013; Sadler et al., 2012).

Summary of research question #2. The data suggest that these OST STEM

activities offered students the opportunity to pursue their STEM learning, as well as to

indulge in their STEM interests and motivations. Thirty-five of the 37 student

participants from the study participated in an elective formal STEM course the following

school year after participating in the OST STEM activities. Thirty-five of the 37 OST


188

STEM students also elected to participate in an OST STEM activity the following school

year after participation in the OST STEM activities in this study.

The S-STEM Survey (FI, 2012) results showed only a few construct level

significant changes from pretest to posttest: 21st Century learning skills, motivations, and

interests in STEM careers. as discussed in prior sections. However, the observations and

descriptive statistics from qualitative data results did show that the middle school

students who participated in the OST STEM activities were highly self-motivated for

STEM learning, which could be a possible explanation for a large number of students

who planned to participate in future formal and informal STEM courses and OST

activities at this school, as that self-motivation had already helped them to form STEM

identities.

Data also found that students were motivated by a collection of factors ranging

from self to parents to teachers to the content of the activity itself. This confirms prior

research that hands-on STEM learning can support interest in and motivation for STEM

learning (Hayden et al., 2011; Mohr-Schroeder et al., 2014; Nugent et al., 2010). The

female STEM instructor who leads the Science Olympiad, Girls Who Code, and

eCYBERMISSION was also identified by her female students as being particularly

motivating, which confirms prior research which has shown that female students who are

able to work with STEM individuals who reflect their background can increase their

interests in STEM learning (Mosatche et al., 2013) and support the development of their

STEM identities (Hughes et al., 2013).


189

The students’ 21st century skills were put into practice and developed in the OST

STEM activities due to the requirements of the activities to be creative, collaborative, and

technically apt. The OST STEM activities provided students with opportunities to pursue

their STEM interests while building on their own intrinsic motivation through self-

directed learning projects. A majority of the students had previous STEM-related learning

experiences, and these previous experiences were important in supporting students’ self-

perceived STEM persistence, developing their interest in STEM learning, and supporting

their high level of internal motivation for STEM. The data is also significant since it

suggests that multiple years of OST STEM activity experiences may influence change in

students’ science attitude due to the development of content knowledge through the

multiple experiences and long-term exposure to STEM learning.

The mixed methods methodology provided a broad understanding of the topic that

a single methodology could not due in this situation. Had the study been purely

quantitative, the impact of the activities demonstrated by the qualitative data would have

been overlooked. For example, the quantitative findings showed that the STEM activities

had not impacted the students’ interests; however, interview data and observations

indicated increased student interest after participation in the OST STEM activities.

Specific motivating factors, including those of parents and teachers, were also

illuminated through the qualitative research.

Limitations and Recommendations for Future Research

There are multiple limitations and recommendations for future research from this

study. The participants in this study were only from one middle school, primarily from


190

middle- or upper-class economic backgrounds, with parents or guardians who placed a

high value on education. Furthermore, the type of school was an independent, college

prep school. As such, findings from this study are particularly applicable to the

independent school community, as the school in this study taking being an independent

school that is a member of the National Association of Independent Schools (NAIS). This

relatively homogeneous sample is consistent with approaches taken by prior studies

which have focused on specific background areas, such as gifted and talented students

(Coxon, 2012) and OST STEM programs (Varnado, 2005). The researcher chose to study

these students at this school due to accessibility and accessibility to multiple OST STEM

activities in a single location.

The researcher completed the study in the school in which he works. The

researcher did formally teach some (N = 15) of the students, but did not teach a majority

of the students (N = 22); the researcher taught only seventh- and eighth-grade boys based

on how the school structured its electives into gender separate courses with gender-

matched instructors. This could have possibly affected students’ responses during the

interviews, as well as the students’ responses on the descriptive statistics and S-STEM

Survey (FI, 2012) due to the prior relationship the researcher may have with his students

that he teaches in the formal classroom setting (Creswell, 2014; Creswell & Plano-Clark,

2011). On the other hand, Creswell (2014) and Lincoln and Guba (1985) have argued that

the trust or relationship the researcher has built with participants could ensure the capture

of authentic data in the study. The methodological design of collecting multiple types of

data and the relationship the researcher had built with some of the participants supported


191

the capture of authentic data within the OST STEM activities by the constructivist

researcher (Creswell, 2014; Lincoln & Guba, 1985). Stakes (1995) also explained that it

is perfectly normal for doctoral students that have a full-time job to do the research in

their own work settings as this researcher choice to do in this study due to accessibility.

The students’ backgrounds and findings in this study are similar to those of prior

research on family habitus outcomes for supporting students’ STEM learning (Archer et

al., 2012). While the middle school student sample had nearly an equal representation of

middle school girls and boys (16 females and 21 males), a majority of the student

population was White (32) with only five students of another race (one Asian American,

one student of Asian Indian descent, and three African Americans). Though the findings

of this study are transferrable, they may not necessarily be generalizable to all school

sites or to all student populations with respect to socioeconomic status, culture, or

ethnicity. Other studies have had similar populations with regards similar high levels (i.e.

75% and higher) of White students (Mohr-Schroeder et al., 2014; Nugent et al., 2010)

and male to female comparison (Vanard, 2005). The data from this study represented a

Western viewpoint from a metropolitan region in the Southeastern United States. For this

reason, it is unknown if the findings of this study may be transferable to different areas of

the world. Prior research suggests that culture may affect students’ perceptions of and

attitudes toward learning, which could affect the results or produce different outcomes

when replicating this study in other schools in the United States or in other countries

(Espinosa, 2011; Gonzalez & Kuenzi, 2012; Hite et al., 2018; Mosatche et al., 2013;

Palmer et al., 2011).


192

In future research, investigating a more diverse population of middle school

students or conducting the study in another country or region of the United States would

provide a richer understanding of the phenomena by evaluating a variety of perspectives

and experiences and gathering a robust data set so that there can be a greater

understanding of the changes in middle school students’ aptitude for 21st century skills,

motivations, interest, and perceived persistence in STEM. This would support the

understanding of how differences in middle school students’ cultural backgrounds may

influence the impact of participation in an OST STEM activity.

Prior research has shown the importance of supporting STEM learning and

persistence of minority and female students due to their underrepresentation in STEM

fields (Anderson & Ward, 2014; NSF, 2014; Soldner et al., 2012). Through replication of

the study, possibly different information could be found to support STEM learning, as a

whole, if minority middle school students were studied. With a more diverse population

of students (i.e., cultural and ethnic backgrounds), a better understanding of these

students’ STEM identities could possibly be gained, along with a strengthened

understanding of their intention to persist in future STEM learning (Barton et al., 2012;

Espinosa, 2011; Hite et al., 2018).

Replicating the study with a larger and diverse population, in other parts of the

United States and/or countries could support a deeper understanding of how OST STEM

activities influence middle school students’ perceived STEM persistence, as well as

motivation and interest for STEM learning and development of 21st Century skills. For

example, completing this study in a European country could possibly provide information


193

to support the increasing STEM workforce of the future in the European Union (Caprile,

Palmén, Sanz, & Dente, 2015). Furthermore, completing the study in a Latin American

country or in a Hispanic American community could possibly provide a better

understanding of cultural aspects related to middle school students’ perceived persistence

for STEM learning, which could support an increase of Hispanic people in STEM careers

(Hite et al., 2018; NSF, 2014; NRC, 2015).

This study focused on students who already had interest for STEM learning, as

these students chose to participate in their OST STEM activity; this could be considered

selection bias. The mixed method design reduced this limitation through pre and posttest

qualitative analysis of the students’ self-reported perceptions and attitudes for 21st

century skills as well as motivations, interest, and perceived persistence in STEM. Prior

research on OST STEM activities (i.e. STEM summer camps) have used similar student

selection process (Mohr-Schroeder et al., 2014; Nugent et al., 2010) and have focused on

students who had prior experiences in OST STEM activities (i.e. FIRST LEGO League)

(Varnado, 2005).

Another limitation of the study was the schedule by which the OST STEM

activities met, including the amount of time each OST STEM activity met during the

research period. The four different OST STEM activities (i.e. Girls Who Code,

eCYBERMISSION, Science Olympiad, and robotics [sumo-bots and drones]) met at

different times during the week, and the amount of time each of these groups met

differed. For example, the robotics groups met once per week formally for one hour but

also met informally during the students’ lunchtime whereas the Girls Who Code (2017)


194

group met twice per week for one hour and the eCYBERMISSION (2017) students met

during lunch each day for about 40 minutes. Some individual students and groups worked

on their OST STEM activity’s project during nonscheduled times, such as afterschool or

before school, which supports students’ independent learning (Ayar, 2015) and more time

exposed to STEM learning is better than less time in general (Nugent et al., 2010). The

researcher attempted to reduce this limitation by interviewing students from the different

OST STEM activities to gain a balanced view of each OST STEM activity reported by

the students. Furthermore, the researcher attempted to have equal research timelines for

studying each of the activities for 13-16 weeks. The researcher believes that the variation

in time that the different OST STEM activities met did not influence the overall outcomes

due to the students’ having a high-level self-motivation. Additionally, 22 of the students

participated in multiple OST STEM activities, limiting the impact of the varying

schedules. However, the varied amounts of time each group of students participated in

their OST STEM activity could have affected the results of the students’ perceptions

towards 21st Century learning, interests, motivations, and persistence for STEM, as prior

research has shown that OST STEM activities which allow students more time to pursue

personal STEM interests enables more time to learn STEM content that may not be

taught in the formal classroom (Ayar, 2015; (Leblebicioglu et al., 2017; Matterson &

Holman, 2012; Stocklmayer et al., 2010). Furthermore, the more time spent doing

STEM-related activities has been shown to increase interest for STEM (Reynolds et al.,

2009).


195

The study was also completed during the spring semester. A future study could be

carried out over the entire academic year, allowing for a longer period between the

pretest and posttest. Alternatively, tracking the formal meeting times of each OST STEM

activity and the amount of informal time (time that is not required of the OST STEM

activity participants) the students met could provide a richer understanding of the

students’ motivation and interest for STEM learning. This information could be

compared to determine if there is a correlation between time spent on an activity and

STEM persistence or if mandatory versus optional meetings influence individual

persistence for learning STEM.

This study attempted to bridge an age-specific research gap, as prior studies that

have examined STEM persistence of students have primarily focused on college STEM

students (Andersen & Ward, 2014; Brazwell, 2010; Maltese & Tai, 2011). Studies with

middle school students have specifically focused on the impact of STEM summer camps

on students’ interests, motivations, and perceived persistence for future STEM learning

(Holmquist, 2014; Mohr-Schroeder et al., 2014; Nugent et al., 2010) as opposed to the

school-year OST STEM activities researched in this study. The OST STEM activities

were selected for this study based on their availability, project-based structure, and focus

on competition. Considering the wide range of OST STEM activities offered nationally

and globally, future research could be focused on OST STEM activities that differ from

those in this study. Studying lesser known OST STEM activities could support the overall

body of research on OST STEM activities. This could possibly shed light on how

individual OST STEM activities are changing students’ perceptions towards STEM


196

persistence, as well as how motivations, interests, and 21st Century skills are influenced

by a specific activity. Furthermore, it could provide information about the students who

join a specific OST STEM activity versus students who join another to obtain more

information about the motivational aspects of the activities (Young et al., 1997).

Conclusion

The STEM economy continues to grow nationally and internationally; it is

imperative that students have a comprehensive education to become a part of the global

STEM workforce that develops STEM skills and soft skills which prepare them

workplace (Ahmed, 2016; Brazell, 2013; Caprile et al., 2015; PCAST, 2010). Previous

research on OST STEM activities and formal STEM courses have focused primarily on

how high school STEM learning has influenced participating students’ STEM learning

and persistence as measured by college STEM course enrollment (Afterschool Alliance,

2015; Brown, 2016; NRC, 2015). This study has added clarity to the understanding of

how middle school OST STEM activities have impacted middle school students’ STEM

persistence. Furthermore, the study has provided findings on the development of

students’ STEM identities, especially those of middle school girls (Archer et al., 2010;

Barton et al., 2012; Hughes et al., 2013). Lastly, the importance of STEM learning held

by the students’ parents, as well as the large percentage of students’ parents in a STEM

career, influenced the students’ STEM capital through family habitus.

Prior research has shown the importance of providing students extra STEM

learning opportunities due to the influence on students learning like these OST STEM

activities in this study (Marginson, Tytler, Freeman, & Roberts, 2013; NRC, 2011; NRC,


197

2013). Furthermore, authentic, interactive hands-on learning that was reported and

observed by the students in this study has been shown to have positive effects on

students’ future STEM learning in prior research (NRC, 2013). This study confirms prior

research which has shown that STEM learning that is relevant and personal can increase

STEM literacy, interest and motivation (NRC, 2011) as demonstrated by the self-

selection of the OST STEM activities and the student-driven selection of specific

projects. The OST STEM activities studied, along with the formal STEM courses in

which students participated, were emphasizing technical and 21st century skill

development to address real-world learning by providing students a STEM focus and uses

of the school’s digital fabrication lab. Furthermore, the teachers and the school

community have made STEM important part of the school by offering multiple STEM

learning experiences (Scott, 2012; White, 2014).

The results of this study suggest that OST STEM activities can support middle

school students’ perceived views towards STEM persistence. Furthermore, the OST

STEM activities offered students the opportunity to pursue their STEM learning, as well

as to pursue in their STEM interests and motivations. This indicates that OST STEM

activities may support middle school students’ interest and motivation for STEM

learning, as well as develop their 21st Century learning skills. Lastly, the data suggest

that the OST STEM activities may positively influence students’ perceived STEM

persistence, especially for possible future careers in science and doing engineering to

improve peoples’ lives. These OST STEM activities are resources providing middle

school STEM students with pathways to pursue their interests and motivation for STEM


198

and develop a long-term possible goal for STEM learning (Dweck, 2008). This study has

shown how the positive support of middle-schoolers’ STEM learning and persistence by

teachers and parents can positively impact the STEM persistence and continuation of

self-directed learning of STEM for those students (Duckworth, Peterson, Matthews, &

Kelly, 2007). This study was significant due to determination that OST STEM activities

support middle school students’ long-term STEM persistence by providing them the

opportunity to engage in their STEM interests and motivation (Von Culin, Tsukayama, &

Duckworth, 2014), as well as positively support their STEM identity (Hite et al., 2018).

This study illuminated the importance of intrinsic motivation for independent school

students. The students in this study had high levels of intrinsic motivation which is

important for students’ STEM learning and perceived STEM persistence (DeJamette,

2012; Nugent et al., 2010). Evidence of this intrinsic motivation includes the fact that the

majority of the students had more than one year of experience in OST STEM activities.

As prior research has shown, motivation to learn math and science can be impacted by

students’ prior learning experiences and support their STEM persistence (Andersen &

Ward, 2014; Graham et al., 2013; Mohr-Schroeder et al., 2014; Nugent et al., 2010;

Wang, 2013). Additionally, over half of the students participated in more than one of the

OST STEM activities in this study, demonstrating that the OST STEM activities were a

STEM learning modality for the majority of the students who used these activities as a

means to nurture their STEM persistence (Hall et. al., 2011). Finally, the largest

subtheme under the qualitative theme of Sources of Motivation (N = 428) was Self-


199

motivation and Internal Interest (N = 134) and over half of the students listed themselves

as a reason for joining their OST STEM activity.

This study highlighted the importance of STEM family habitus for independent

school students. Parents and family played an extremely important and influential role in

the development of these independent school students’ motivation, interest, and

persistence for STEM learning and impacted the development of their STEM identities

(Afterschool Alliance, 2015; Archer et al., 2010; Brown, 2016; Sabin, 2013). The

qualitative subtheme of Family (N = 48) under the theme of Sources of Motivation

supported this concept. Nearly half of the students listed their parents as an encouraging

influence for joining their OST STEM activity in addition to other students referencing

other influential relatives such as siblings or grandparents. These references are evidence

of students’ who are highly motivated for STEM learning being encouraged to continue

pursuing a STEM passion by their parents. Furthermore, nearly half of the students in this

study had at least one parent whose occupation was in a STEM field; this supports the

development of the students’ STEM identity and motivation for STEM learning through a

positive STEM family habitus (Bandura et al., 2001; Gallagher, 1994; Hall et. al., 2011;

Wyss et. al., 2012). This indicates that students with parents in STEM careers possibly

could be influencing the students’ motivation and persistence for STEM due to their

STEM family habitus.

The information discovered in this study could be important to independent

schools with a population of students with backgrounds similar to those in this study that

are developing STEM courses (i.e. formal and informal) and OST STEM activities. The


200

study can help the independent schools to support their students that are intrinsically

motivated for STEM learning and who want to continue their pursuit of STEM.

Furthermore, independent schools with students’ whose parents encourage STEM could

possibly support their schools’ STEM pipeline development by applying an

understanding of the importance of intrinsic motivation and STEM family habitus for

independent school students. Overall, this study provides enhanced insight into the

importance of intrinsic motivation and STEM family habitus for independent school

students’ motivation, interest, and persistence for STEM learning.


201

REFERENCES

Abernathy, T. V., & Vineyard, R. N. (2001). Academic competitions in science: What are the rewards for students? The Clearing House, 74(5), 269-276.

Adams, P. (2006). Exploring social constructivism: Theories and practicalities. Education, 34(3), 243-257.

Afterschool Alliance (2015). Full STEM ahead: Afterschool programs step up as key partners in STEM education. Retrieved from http://www.afterschoolalliance.org/aa3pm/STEM.pdf

Ahmed, H. O. K. (2016). Strategic Future Directions for Developing STEM Education in Higher Education in Egypt as a Driver of Innovation Economy. Journal of Education and Practice, 7(8), 127-145.

Allison, B. N., & Rehm, M. L. (2006). Meeting the needs of culturally diverse learners in family and consumer sciences middle school classrooms. Journal of Family and Consumer Sciences Education, 24(1), 50-63.

Ananiadou, K., & Claro, M. (2009). 21st century skills and competencies for new millennium learners in OECD countries. Paris: Organization for Economic Cooperation and Development. Retrieved from: http://www.oecd-ilibrary.org/education/ 21st Century-skills- and-competencies-for-new-millennium-learners-in-oecd-countries_218525261154 [March, 2017].

Anderhag, P., Hamza, K. M., & Wickman, P. O. (2015). What can a teacher do to support students’ interest in science? A study of the constitution of taste in a science classroom. Research in Science Education, 45(5), 749-784.

Andersen, L., & Ward, T. J. (2014). Expectancy‐value models for the STEM persistence plans of ninth‐grade, high‐ability Students: A comparison between black, hispanic, and white students. Science Education, 98(2), 216-242.

Archer, L., DeWitt, J., Osborne, J., Dillon, J., Willis, B., & Wong, B. (2010). “Doing” science versus “being” a scientist: Examining 10/11-year-old school childrens constructions of science through the lens of identity. Science Education, 94(4), 617-639.

Archer, L., DeWitt, J., Osborne, J., Dillon, J., Willis, B., & Wong, B. (2012). Science aspirations, capital, and family habitus: How families shape children’s engagement and identification with science. American Educational Research Journal, 49(5), 881-908.


202

Arikan, S. (2014). A Regression Model with a New Tool: IDB Analyzer for Identifying Factors Predicting Mathematics Performance Using PISA 2012 Indices. Online Submission, 4(10), 716-727.

Asunda, P. A. (2011). Open courseware and STEM initiatives in career and technical education. Journal of STEM Education, 48(2), 6-37.

Atkinson, R. D., & Mayo, M. (2011). Refueling the U.S. innovation economy: Fresh approaches to science, technology, engineering and mathematics (STEM) education. Washington, DC: The Information Technology and Innovation Foundation.

Ayar, M. C. (2015). First-hand experience with engineering design and career interest in engineering: An informal STEM education case study. Educational Sciences: Theory & Practice, 15(6), 1655-1675.

Bächtold, M. (2013). What do students “construct” according to constructivism in science education? Research in science education, 43(6), 2477-2496.

Bandura, A., Barbaranelli, C., Caprara, G. V., & Pastorelli, C. (2001). Self‐efficacy beliefs as shapers of children's aspirations and career trajectories. Child development, 72(1), 187-206.

Barker, B. S., & Ansorge, J. (2007). Robotics as means to increase achievement scores in an informal learning environment. Journal of Research on Technology in Education, 39(3), 229-243.

Basham, J. D., & Marino, M. T. (2013). Understanding STEM education and supporting students through universal design for learning. Teaching Exceptional Children, 45(4), 8-15.

Basham, J. D., Israel, M., & Maynard, K. (2010). An ecological model of STEM education: Operationalizing STEM for all. Journal of Special Education Technology, 25(3), 9-19.

Beasley, M. A., & Fischer, M. J. (2012). Why they leave: The impact of stereotype threat on the attrition of women and minorities from science, math and engineering majors. Social Psychology of Education, 15(4), 427-448.

Bell, S. (2010). Project-based learning for the 21st century: Skills for the future. The Clearing House, 83(2), 39-43.

Bellanca, J. A. (2010). Enriched Learning Projects: a practical pathway to 21st century skills. Bloomington, IN: Solution Tree Press.


203

Bottia, M. C., Stearns, E., Mickelson, R. A., Moller, S., & Parker, A. D. (2015). The Relationships among High School STEM Learning Experiences and Students' Intent to Declare and Declaration of a STEM Major in College. Teachers College Record, 117(3), n3.

Bourn, D., & Neal, I. (2008). The global engineer: Incorporating global skills within UK higher education of engineers. Report for the DFID Development Awareness Fund project on: “Promoting Development Awareness through dialogue and partnership exploration: UK Engineering Higher Education”.

Boy, G. A. (2013, August). From STEM to STEAM: toward a human-centred education, creativity & learning thinking. In Proceedings of the 31st European Conference on Cognitive Ergonomics (p. 3). ACM.

Braund, M., & Reiss, M. (2006). Towards a more authentic science curriculum: The contribution of out‐of‐school learning. International Journal of Science Education, 28(12), 1373-1388.

Brisson, L., Eisenkraft, A., Flatow, I., Friedman, A., Kirsch, J., Macdonald, M., & Witte, J. (2010). Informal science education policy: Issues and opportunities.

Brazell, J. (2013). STEM 2.0: Transformational Thinking about STEM for Education and Career Practitioners. Career Planning and Adult Development Journal, 29(2), 20-33.

Brophy, S., Klein, S., Portsmore, M., & Rogers, C. (2008). Advancing engineering education in P‐12 classrooms. Journal of Engineering Education, 97(3), 369-387.

Brown, J. (2016). The Case for Investing in Out-of-School Learning as a Core Strategy in Improving Science, Technology, Engineering, and Mathematics (STEM) Education. Retrieved from June 8, 2016: http://www.stemedcoalition.org/2016/04/27/the-case-for-investing-in-out-of-school-learning-as-a-core-strategy-in-improving-stem-education/

Bull, G., Thompson, A., Searson, M., Garofalo, J., Park, J., Young, C., & Lee, J. (2008). Connecting informal and formal learning: Experiences in the age of participatory media. Contemporary Issues in Technology and Teacher Education, 8(2), 100-107.

California Department of Education Publication. (2017). Characteristics of Middle Grade Students. Retrieved from http://pubs.cde.ca.gov/tcsii/documentlibrary/characteristicsmg.aspx#top

Cantrell, P., & Ewing‐Taylor, J. (2009). Exploring STEM career options through collaborative high school seminars. Journal of Engineering Education, 98(3), 295-303.

http://pubs.cde.ca.gov/tcsii/documentlibrary/characteristicsmg.aspx#top


204

Capraro, R., Capraro, M. M., & Morgan, J. R. (2013). STEM Project-based Learning: An Integrated Science, Technology, Engineering, and Mathematics (STEM) Approach. Retrieved from https://ebookcentral.proquest.com

Caprile, M., Palmén, R., Sanz, P., & Dente, G. (2015). Encouraging STEM studies for the labour market. European Parliament. Retrieved from http://www.europarl.europa.eu/RegData/etudes/STUD/2015/542199/IPOL_STU(2015)542199_EN.pdf

Carnevale, A. P., Smith, N., & Melton, M. (2011). STEM: Science Technology Engineering Mathematics. Washington, DC: Georgetown University Center on Education and the Workforce.

Chen, X. (2009). Students Who Study Science, Technology, Engineering, and Mathematics (STEM) in Postsecondary Education. Stats in Brief. NCES 2009-161. Retrieved from http://files.eric.ed.gov/fulltext/ED533548.pdf

Chow, C. (2012). Learning from our global competitors: A comparative analysis of science, technology, engineering and mathematics (STEM) education pipelines in the United States, Mainland China and Taiwan. World Economic Forum.

Chris, C. (2013). Learning with FIRST LEGO League. In Society for Information Technology & Teacher Education International Conference (pp. 5118-5124). Association for the Advancement of Computing in Education (AACE).

Crede, E., & Borrego, M. (2010). A content analysis of the use of mixed methods studies in engineering education. American Society for Engineering Education. Retrieved from https://peer.asee.org/a-content-analysis-of-the-use-of-mixed-methods-studies-in-engineering-education.pdf.

Christensen, R., Knezek, G., & Tyler-Wood, T. (2015). A retrospective analysis of STEM career Interest among mathematics and science academy students. International Journal of Learning, Teaching and Educational Research, 10(1), 45-58.

Cofré, H., González-Weil, C., Vergara, C., Santibáñez, D., Ahumada, G., Furman, M., ... & Pérez, R. (2015). Science teacher education in South America: The case of Argentina, Colombia and Chile. Journal of Science Teacher Education, 26(1), 45-63.

Covington, M. V. (2000). Goal theory, motivation, and school achievement: An integrative review. Annual review of psychology, 51(1), 171-200.

Coxon, S. V. (2012). The malleability of spatial ability under treatment of a FIRST LEGO League-based robotics simulation. Journal for the Education of the Gifted, 35(3), 291-316.

https://ebookcentral.proquest.com/

http://files.eric.ed.gov/fulltext/ED533548.pdf

https://peer.asee.org/a-content-analysis-of-the-use-of-mixed-methods-studies-in-engineering-education.pdf

https://peer.asee.org/a-content-analysis-of-the-use-of-mixed-methods-studies-in-engineering-education.pdf


205

Creswell, J. W. (2013). Qualitative inquiry & research design: Choosing among five approaches. Los Angeles: Sage Publishing.

Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications.

Creswell, J. W., & Plano Clark, V. C. (2011). Designing and conducting mixed methods research (2nded.). Thousand Oaks, CA: Sage.

Davis, D., & Veenstra, C. (2014). Community Involvement in STEM Learning. The Journal for Quality and Participation, 37(1), 30-33.

Deci, E. L., Vallerand, R. J., Pelletier, L. G., & Ryan, R. M. (1991). Motivation and education: The self-determination perspective. Educational psychologist, 26(3-4), 325-346.

de Winter, J. C. (2013). Using the Student’s t-test with extremely small sample sizes. Practical Assessment, Research & Evaluation, 18(10), 1-12.

DeJamette, N. (2012). America's children: Providing early exposure to STEM (science, technology, engineering and math) initiatives. Education, 133(1), 77-84.

Dewey, J. (1913). Democracy and education: an introduction to the philosophy of education. London, England: Macmillian.

Dewey, J. (1916). Interest and effort in education. Boston, MA: Houghton Mifflin Company.

Dewey, J. (1937). What is learning. John Dewey: The Later Works, 11, 1935-1937.

Dickinson, T. S., & Butler, D. A. (2001). Reinventing the Middle School. Middle School Journal, 33(1), 7-13.

Dierking, L. D., Falk, J. H., Rennie, L., Anderson, D., & Ellenbogen, K. (2003). Policy statement of the “informal science education” ad hoc committee. Journal of research in science teaching, 40(2), 108-111.

Dubetz, T. A., & Wilson, J. A. (2013). Girls in Engineering, Mathematics and Science, GEMS: A science outreach program for middle-school female students. Journal of STEM Education: Innovations and Research, 14(3).

Duderstadt, J. J. (2007). Engineering for a Changing Road, A Roadmap to the Future of Engineering Practice, Research, and Education. Ann Arbor, MI: The Millennium Project, University of Michigan.


206

Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: perseverance and passion for long-term goals. Journal of personality and social psychology, 92(6), 1087.Durlak, J. A., & Weissberg, R. P. (2007). The Impact of After-School Programs that Promote Personal and Social Skills. Chicago, IL: CASEL.

Dweck, C. S. (2008). Mindsets and math/science achievement. New York, NY: Carnegie Corp. of New York–Institute for Advanced Study Commission on Mathematics and Science Education.

eCYBERMISSION. (2018, March). About. Retrieved from eCYBERMISSION website https://www.ecybermission.com/about

Ejiwale J. A. (2012). Facilitating Teaching and Learning Across STEM Fields. Journal Of STEM Education: Innovations & Research, 13(3), 87-94.

Erlandson, D. A., Harris, E. L., Skipper, B. L., & Allen, S. D. (1993). Doing naturalistic inquiry: A guide to methods. Newbury Park, CA: Sage.

Eris, O., Chen, H., Bailey, T., Engerman, K., Loshbaugh, H., Griffin, A., & Cole, A. (2005). Development of the Persistence in Engineering (PIE) survey instrument. American Society for Engineering Education Annual Conference & Exposition. Retrieved from file:///C:/Users/david.taylor/Downloads/development-of-the-persistence-in-engineering-pie-survey-instrument.pdf.

Ernest, P. (1998). Social constructivism as a philosophy of mathematics. Suny Press.

Eshach, H. (2007). Bridging in-school and out-of-school learning: Formal, non-formal, and informal education. Journal of science education and technology, 16(2), 171-190.

Espinosa, L. (2011). Pipelines and pathways: Women of color in undergraduate STEM majors and the college experiences that contribute to persistence. Harvard Educational Review, 81(2), 209-241.

ESSA. (2015). Every Student Succeeds Act of 2015, Pub. L. No. 114-95 § 114 Stat. 1177 (2015-2016).

Fayer, S., Lacey, A., & Watson, A. (2017). STEM Occupations: Past, Present, And Future. Retrieved from: https://www.bls.gov/spotlight/2017/science-technology-engineering-and-mathematics-stem-occupations-past-present-and-future/pdf/science-technology-engineering-and-mathematics-stem-occupations-past-present-and-future.pdff

Falk, J., & Dierking, L. (2010). The 95% solution. American Scientist, 98, 486-493.

https://www.ecybermission.com/

http://www.bls.gov/careeroutlook/2014/spring/art01.pdf





207

Friday Institute for Educational Innovation (2010). NC Student Learning Conditions Survey, Raleigh, NC: Author.

Friday Institute for Educational Innovation (2012). Student Attitudes toward STEM Survey – Middle and High School Students, Raleigh, NC: Author.

Feldman, A. F., & Matjasko, J. L. (2005). The role of school-based OST activities in adolescent development: A comprehensive review and future directions. Review of educational research, 75(2), 159-210.

Freeman, K. E., Alston, S. T., & Winborne, D. G. (2008). Do Learning Communities Enhance the Quality of Students' Learning and Motivation in STEM?. The Journal of Negro Education, 77(3), 227-240.

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410-8415.

Furlong A., & Biggart, A. (1999). Framing ‘choices’: A longitudinal study of occupational aspirations among 13-17 year-olds. Journal of Education and Work, 12(1), 21-35.

Gagné, M., & Deci, E. L. (2005). Self‐determination theory and work motivation. Journal of Organizational behavior, 26(4), 331-362.

Gallagher, S. A. (1994). Middle school classroom predictors of science persistence. Journal of Research in Science Teaching, 31(7), 721-734.

Gallego, M. D., Bueno, S., & Noyes, J. (2016). Second Life adoption in education: A motivational model based on Uses and Gratifications theory. Computers & Education, 100, 81-93.

Gardner, H. (2006). Five minds for the future (Leadership for the common good). Harvard, MA: Harvard Business School Press.

Gasiewski, J. A., Eagan, M. K., Garcia, G. A., Hurtado, S., & Chang, M. J. (2012). From gatekeeping to engagement: A multicontextual, mixed method study of student academic engagement in introductory STEM courses. Research in higher education, 53(2), 229-261.

Gerlach, J. (2012). “NSTA Report, STEM: Defying a Simple Definition,” NSTA WebNews Digest, April 11, 2012. Retrieved from http://www.nsta.org/publications/news/story.aspx?id=59305

http://www.nsta.org/publications/news/story.aspx?id=59305


208

Green, M. (2007). Science and Engineering Degrees: 1966−2004 (NSF 07-307). Arlington, VA: National Science Foundation.

Gonzalez, H. B., & Kuenzi, J. J. (2012, August). Science, technology, engineering, and mathematics (STEM) education: A primer. Washington, DC: Congressional Research Service, Library of Congress.

Graham, M. J., Frederick, J., Byars-Winston, A., Hunter, A. B., & Handelsman, J. (2013). Increasing persistence of college students in STEM. Science, 341(6153), 1455-1456.

Griffith, A. L. (2010). Persistence of women and minorities in STEM field majors: Is it the school that matters?. Economics of Education Review, 29(6), 911-922.

Fitzgerald, A., & Gunstone, R. F. (2013). Embedding assessment within primary school science: a case study. In D. Corrigan, R. Gunstone, & A. Jones (Eds.), Valuing Assessment in Science Education: Pedagogy, Curriculum, Policy (pp. 307 - 324). The Netherlands: Springer.

Hayden, K., Ouyang, Y., Scinski, L., Olszewski, B., & Bielefeldt, T. (2011). Increasing student interest and attitudes in STEM: Professional development and activities to engage and inspire learners. Contemporary Issues in Technology and Teacher Education, 11(1), 47-69.

Hazari, Z., Sonnert, G., Sadler, P. M., & Shanahan, M. C. (2010). Connecting high school physics experiences, outcome expectations, physics identity, and physics career choice: A gender study. Journal of research in science teaching, 47(8), 978-1003.

Hein, G. (1991). Constructivist learning theory. Institute for Inquiry. Retrieved from http://www.exploratorium.edu/ifi/resources/constructivistlearning.html

Hemmo, V. (2008). Encouraging student interest in science and technology studies. Paris: OECD.

Hite, R., Midobuche, E., Benavides, A. H., & Dwyer, J. (2018). Third Space Theory: A Theoretical Model for Designing Informal STEM Experiences for Rural Latina Youth. In T. T. Yuen, E. Bonner, & M. G. Arreguin-Anderson (Eds.), (Under)Represented Latin@s in STEM: Increasing Participation Throughout Education and the Workplace. New York, NY: Peter Lang Publishing.

Holmquist, S. (2014). A multi-case study of student interactions with educational robots and impact on Science, Technology, Engineering, and Math (STEM) learning and attitudes. Retrieved from University of South Florida Graduate Theses and Dissertations. http://scholarcommons.usf.edu/etd/5043

http://www.exploratorium.edu/ifi/resources/constructivistlearning.html

http://scholarcommons.usf.edu/etd/5043


209

Hossain, M. M., & G. Robinson, M. (2012). How to Motivate US Students to Pursue STEM (Science, Technology, Engineering and Mathematics) Careers. Online Submission.

Hour of Code. (2018, September). Us. Retrieved from Hour of Code website https://hourofcode.com/us

Hughes, R. M., Nzekwe, B., & Molyneaux, K. J. (2013). The single sex debate for girls in science: A comparison between two informal science programs on middle school students’ STEM identity formation. Research in Science Education, 43(5), 1979-2007.

Hypothesis Test: Difference Between Paired-means. (n.d.). Retrieved from http://stattrek.com/hypothesis-test/paired-means.aspx?Tutorial=AP

Implications for Future Research, Policy, and Practice in STEM Education. (2011). ASHE Higher Education Report, 36(6), 87-126.

IBM Corp. Released 2016. IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp.

International Technology and Engineering Education Association (ITEEA) (2016). Technology and Engineering: Bring STEM to Life. Retrieved from: https://www.iteea.org/File.aspx?id=90060&v=5e1d6c4c

Ivaniushina, V., & Alexandrov, D. (2014). Socialization Through Informal Education: OST Activities of Russian School Students. Educational Studies, 3, 174-196.

Jensen, F., & Sjaastad, J. (2013). A Norwegian Out-Of-School Mathematics Project’s Influence on Secondary Students’ STEM Motivation. International Journal of Science and Mathematics Education, 11(6), 1437-1461.

Jinks, J., & Morgan, V. (1999). Children's perceived academic self-efficacy: An inventory scale. The Clearing House, 72(4), 224-230.

Kay, K. (2009). Middle schools preparing young people for 21st century life and work. Middle School Journal, 40(5), 41-45.

Ketelhut, D. J. (2007). The impact of student self-efficacy on scientific inquiry skills: An exploratory investigation in River City, a multi-user virtual environment. Journal of Science Education and Technology, 16(1), 99-111.

Kincheloe, J. L. (2001). Describing the bricolage: Conceptualizing a new rigor in qualitative research. Qualitative Inquiry, 7(6), 679-692.

Kirkwood-Tucker, T. F. (2009). Visions in global education. New York, NY: Peter Lang.

http://stattrek.com/hypothesis-test/paired-means.aspx?Tutorial=AP

https://www.iteea.org/File.aspx?id=90060&v=5e1d6c4c


210

Krapp, A. (1999). Interest, motivation and learning: An educational-psychological perspective. European journal of psychology of education, 14(1), 23-40.

Krishnamurthi, A., Ballard, M., & Noam, G. G. (2014). Examining the Impact of Afterschool STEM Programs. Washington DC: Afterschool Alliance.

Lawshe, C. H. (1975). A quantitative approach to content validity 1. Personnel psychology, 28(4), 563–575.

Lazarowitz, R., & Hertz-Lazarowitz, R. (1998). Cooperative learning in the science curriculum. International handbook of science education, 1, 449-469.

Leach, J., & Scott, P. (2003). Individual and sociocultural views of learning in science education. Science & Education, 12(1), 91-113.

Leblebicioglu, G., Abik, N. M., Capkinoglu, E., Metin, D., Dogan, E. E., Cetin, P. S., & Schwartz, R. (2017). Science camps for introducing nature of scientific inquiry through student inquiries in nature: Two applications with retention study. Research in Science Education, 1-25. DOI 10.1007/s11165-017-9652-0

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.

Lindsay, J., & Davis, V. (2012). Flattening classrooms, engaging minds: Move to global collaboration one step at a time. Boston, MA: Pearson Higher Ed.

Lirmenbrink, E., & Pintrich, P. (2002). Motivation as an enabler for academic success. School Psychology Review, 31, 313-327.

Liu, E., Liu, C., & Wang, J. (2015). Pre-service science teacher preparation in China: Challenges and promises. Journal of Science Teacher Education, 26(1), 29-44.

Mahoney, J. L., & Cairns, R. B. (1997). Do OST activities protect against early school dropout? Developmental psychology, 33(2), 241-53.

Maltese, A., & Tai, R. (2011). Pipeline persistence: Examining the association of educational experiences with earned degrees in STEM among U.S. students. Science Education, 95(5), 877-907.

Makarova, E., Aeschlimann, B., & Herzog, W. (2016). Why is the pipeline leaking? Experiences of young women in STEM vocational education and training and their adjustment strategies. Empirical Research in Vocational Education and Training, 8(1), 1-18.

Makhmasi, S., Zaki, R., Barada, H., & Al-Hammadi, Y. (2012). Students' interest in STEM education. In Global Engineering Education Conference (EDUCON), 2012 IEEE (pp. 1-3).


211

Marginson, S., Tytler, R., Freeman, B., & Roberts, K. (2013). STEM: country comparisons: international comparisons of science, technology, engineering and mathematics (STEM) education. Final report.

Marshall, C., & Rossman, G. (2016). Designing qualitative research (Sixth ed.). Thousand Oaks, CA: Sage.

Marzano, R. J., & Heflebower, T. (2012). Teaching & assessing 21st century skills. Retrieved from https://ebookcentral.proquest.com

Matterson, C., & Holman, J. (2012). Informal science learning review: Reflections from the Wellcome Trust. Retrieved from: http://www.wellcome.ac.uk/stellent/groups/corporatesite/@msh_peda/documents/web_document/wtp040859.pdf.

Maximizing the Impact of STEM Outreach [MISO]. (2011). S-STEM surveys. Retrieved from http://miso.ncsu.edu/articles/s-stem-survey

McCann, F. F. (2013). Engineers' self-perceptions and a strategy for fostering authentic images of engineers and scientists among elementary school students. (Unpublished doctoral dissertation). University of Oklahoma, Norman.

McNally, T. (2012). Innovative Teaching and Technology in the Service of Science: Recruiting the Next Generation of STEM Students. Journal of the Scholarship of Teaching and Learning, 12(1), 49-58.

McNeal, R. B. Jr. (1995). OST activities and high school dropouts. Sociology of education, 68(1), 62-80.

Metcalf, H. (2010). Stuck in the Pipeline: A Critical Review of STEM Workforce Literature. InterActions. UCLA Journal of Education and Information Studies, 6(2), Article 4. Retrieved from: https://escholarship.org/uc/item/6zf09176

Modi, K., Schoenberg, J., & Salmond, K. (2012). Generation STEM: What girls say about science, technology, engineering, and math. New York, NY: Girl Scouts of the USA.

Mohr‐Schroeder, M. J., Jackson, C., Miller, M., Walcott, B., Little, D. L., Speler, L., ... & Schroeder, D. C. (2014). Developing middle school students' interests in STEM via summer learning experiences: See Blue STEM camp. School Science and Mathematics, 114(6), 291-301.

Morrison, J. (2006). TIES STEM education monograph series, attributes of STEM education. The Student The School The Classroom. Retrieved from http://www.tiesteach.org/documents/Jans%20pdf%20Attributes_of_STEM_Education-1.pdf

https://ebookcentral.proquest.com/

http://www.wellcome.ac.uk/stellent/groups/corporatesite/@msh_peda/documents/web_document/wtp040859.pdf

http://www.wellcome.ac.uk/stellent/groups/corporatesite/@msh_peda/documents/web_document/wtp040859.pdf

http://miso.ncsu.edu/articles/s-stem-survey

https://escholarship.org/uc/item/6zf09176

http://www.tiesteach.org/documents/Jans%20pdf%20Attributes_of_STEM_Education-1.pdf

http://www.tiesteach.org/documents/Jans%20pdf%20Attributes_of_STEM_Education-1.pdf


212

Mosatche, H. S., Matloff-Nieves, S., Kekelis, L., & Lawner, E. K. (2013). Effective STEM programs for adolescent girls: Three approaches and many lessons learned. Afterschool matters, 17, 17-25.

Navarro, R. L., Flores, L. Y., & Worthington, R. L. (2007). Mexican American middle school students' goal intentions in mathematics and science: A test of social cognitive career theory. Journal of Counseling Psychology, 54(3), 320.

National Science Foundation-. (2014). National Science Board: Science and engineering indicators. Arlington, VA: Author.

National Research Council. (2015). Identifying and supporting productive STEM programs in out-of-school settings. Washington DC: National Academies Press.

National Research Council. (2013). Monitoring progress toward successful K-12 STEM education: A nation advancing?. National Academies Press.

National Research Council. (2010). Surrounded by Science: Learning Science in Informal Environments. Washington, DC: The National Academies Press.

National Research Council. (2009). Learning science in informal environments: People, places, and pursuits. Washington, DC: National Academies Press.

Newbill, P. L., Drape, T. A., Schnittka, C., Baum, L., & Evans, M. A. (2015). Learning across Space Instead of over Time: Redesigning a School-Based STEM Curriculum for OST. Afterschool Matters, 22, 4-12.

Nkhata, B. (2013). Career and Technical Education (CTE) Directors' Experiences with CTE's Contributions to Science, Technology, Engineering, and Math (STEM) Education Implementation. Retrieved from Virginia Tech ProQuest Dissertations and Theses. https://vtechworks.lib.vt.edu/bitstream/handle/10919/24203/Nkhata_B_D_2013.pdf?sequence=1

Nugent, G., Barker, B., Grandgenett, N., & Adamchuk, V. I. (2010). Impact of robotics and geospatial technology interventions on youth STEM learning and attitudes. Journal of Research on Technology in Education, 42(4), 391-408.

OECD (2016). Skills Matter: Further Results from the Survey of Adult Skills. Paris, France: OECD Publishing. Retrieved from: http://dx.doi.org/10.1787/9789264258051-en

Ost, B. (2010). The role of peers and grades in determining major persistence in the sciences. Economics of Education Review, 29(6), 923-934.

https://vtechworks.lib.vt.edu/bitstream/handle/10919/24203/Nkhata_B_D_2013.pdf?sequence=1

https://vtechworks.lib.vt.edu/bitstream/handle/10919/24203/Nkhata_B_D_2013.pdf?sequence=1

http://dx.doi.org/10.1787/9789264258051-en


213

Oudeyer, P., & Kaplan, F. (2008). How can we define intrinsic motivation? 8th

International Conference on Epigenetic Robotics (Epirob08). Retrieved from https://hal.inria.fr/file/index/docid/420175/filename/epirob08OudeyerKaplan.pdf.

Pajares, F., & Graham, L. (1999). Self-efficacy, motivation constructs, and mathematics performance of entering middle school students. Contemporary educational psychology, 24(2), 124-139.

Palmer, R. T., Davis, R. J., Moore, J. L., & Hilton, A. A. (2010). A nation at risk: Increasing college participation and persistence among African American males to stimulate US global competitiveness. Journal of African American Males in Education (JAAME), 1(2), 105-124.

Palmer, R. T., Maramba, D. C., & Dancy, T. E. (2011). A qualitative investigation of factors promoting the retention and persistence of students of color in STEM. The Journal of Negro Education, 80(4), 491-504.

Paulsen, C. A. (2013). Implementing Out-of-School Time STEM Resources: Best Practices from Public Television. Afterschool Matters, 17, 27-35.

Peters, L. (2009). Global education: Using technology to bring the world to your students. Washington, DC: International Society for Technology in Education.

Piaget, J. (1972). Intellectual evolution from adolescence to adulthood. Human development, 15(1), 1-12.

Pink, D. H. (2011). Drive: The surprising truth about what motivates us. New York, NY: Riverhead Books.

Pitt, J. (2009). Blurring the boundaries–STEM education and education for sustainable development. Design and Technology Education: An International Journal, 14(1), 37-48.

Popescu, C., & Diaconu, L. (2009). Informal Education and Productivity. SSRN. http://dx.doi.org/10.2139/ssrn.1328239

Powell, K. C., & Kalina, C. J. (2009). Cognitive and social constructivism: Developing tools for an effective classroom. Education, 130(2), 241-250.

President’s Council of Advisors on Science and Technology (PCAST) (September 2010). Report to the President, Prepare and Inspire: K-12 Education in Science, Technology, Engineering, and Math (STEM) for America’s Future. Retrieved from https://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-stem-ed-final.pdf

https://dx.doi.org/10.2139/ssrn.1328239

https://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-stem-ed-final.pdf

https://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-stem-ed-final.pdf


214

Price, J. (2010). The effect of instructor race and gender on student persistence in STEM fields. Economics of Education Review, 29, 901–910.

Regassa, L. B., & Morrison-Shetlar, A. I. (2009). Student Learning in a Project-Based Molecular Biology Course. Journal of College Science Teaching, 48, 58-67.

QSR International (2018). NVivo Software 11.1.1. Retrieved from http://www.qsrinternational.com/product/nvivo-mac

Renninger, K. A., & Hidi, S. (2011). Revisiting the conceptualization, measurement, and generation of interest. Educational Psychologist, 46(3), 168-184.

Reynolds, B., Mehalik, M. M., Lovell, M. R., & Schunn, C. D. (2009). Increasing student awareness of and interest in engineering as a career option through design-based learning. International Journal of Engineering Education, 25(4), 788-798.

Rittmayer, A. D., & Beier, M. E. (2008). Overview: Self-efficacy in STEM. SWE-AWE CASEE Overviews, 1-12.

Rivoli, G. J., & Ralston, P. A. S. (2009). Elementary and middle school engineering outreach: Building a STEM pipeline. In B. Bernal, (Ed.), Proceedings of the 2009 ASEE Southeastern Section Conference. Retrieved from http://www.softwareeducationsupport.com/ASEE%20SE%20Conference%20Proceedings/Conference%20Files/ASEE2009/papers/P2009035RAL.PDF.

Rossman, G., & Rallis, S. (2003). Learning in the field: An introduction to qualitative research (2nd ed.). Thousand Oaks, CA: Sage Publications.

Rothwell, J. (2013). The hidden STEM economy. Washington, DC: Brookings.

Ruby, A. (2006). Improving science achievement at high-poverty urban middle schools. Science Education, 90(6), 1005-1027.

Sadler, P. M., Sonnert, G., Hazari, Z., & Thi, R. (2014). The role of advanced high school coursework in increasing STEM career interest. Science Educator, 23(1), 1-13.

Sadler, P. M., Sonnert, G., Hazari, Z., & Tai, R. (2012). Stability and volatility of STEM career interest in high school: A gender study. Science education, 96(3), 411-427.

Sahin, A. (2013). STEM clubs and science fair competitions: Effects on post-secondary matriculation. Journal of STEM Education: Innovations and Research, 14(1), 5-11.

http://www.softwareeducationsupport.com/ASEE%20SE%20Conference%20Proceedings/Conference%20Files/ASEE2009/papers/P2009035RAL.PDF

http://www.softwareeducationsupport.com/ASEE%20SE%20Conference%20Proceedings/Conference%20Files/ASEE2009/papers/P2009035RAL.PDF


215

Sahin, A., Ayar, M. C., & Adiguzel, T. (2014). STEM Related After-School Program Activities and Associated Outcomes on Student Learning. Educational Sciences: Theory and Practice, 14(1), 309-322.Saldaña, J. (2016). The coding manual for qualitative researchers (3rd ed.). Thousand Oaks, CA: Sage.

Sanders, M. (2009). STEM, STEM education, STEM mania. Technology Teacher, 68(4), 20–26.

Savery, J. R. (2006). Overview of problem-based learning: Definitions and distinctions. The interdisciplinary Journal of Problem-based Learning, 1(1), 9-20.

Scardamalia, M., Bransford, J., Kozma, B., & Quellmalz, E. (2012). New assessments and environments for knowledge building. In Assessment and teaching of 21st century skills (pp. 231-300). Netherlands: Springer.

Schunk, D. H. (1991). Self-efficacy and academic motivation. Educational psychologist, 26(3-4), 207-231.

Schunk, D., Pintrich, P., & Meece, J. (2008). Motivation in education: Theory, research, and applications (3rd ed.). Upper Saddle River, NJ: Pearson Education.

Scott, C. E. (2012). An investigation of science, technology, engineering and mathematics (STEM) focused high schools in the US. Journal of STEM Education: Innovations and Research, 13(5).

Shapiro, J. R., & Williams, A. M. (2012). The role of stereotype threats in undermining girls’ and women’s performance and interest in STEM fields. Sex Roles, 66(3-4), 175-183.

Sithole, A., Chiyaka, E. T., McCarthy, P., Mupinga, D. M., Bucklein, B. K., & Kibirige, J. (2017). Student Attraction, Persistence and Retention in STEM Programs: Successes and Continuing Challenges. Higher Education Studies, 7(1), 46-59.

Sladek, M. (1998). A report on the evaluation of the national science foundation’s informal science education program. Washington, DC: National Science Foundation.

Smith, K. A., Douglas, T. C., & Cox, M. F. (2009). Supportive teaching and learning strategies in STEM education. New Directions for Teaching and Learning, 2009(117), 19-32.

Smith, K. A., Sheppard, S. D., Johnson, D. W., & Johnson, R. T. (2005). Pedagogies of engagement: Classroom‐based practices. Journal of engineering education, 94(1), 87-101.


216

Soldner, M., Rowan-Kenyon, H., Inkelas, K. K., Garvey, J., & Robbins, C. (2012). Supporting students' intentions to persist in STEM disciplines: The role of living-learning programs among other social-cognitive factors. The Journal of Higher Education, 83(3), 311-336.

Somerville-Midgette, K. N. (2014). An Engineering Journey: A Transcendental Phenomenological Study of African-American Female Engineers' Persistence. Retrieved from Liberty University ProQuest Dissertations and Theses. http://digitalcommons.liberty.edu/cgi/viewcontent.cgi?article=2007&context=doctoral

Spradley, J. P. (1979). The ethnographic interview. New York, NY: Holt, Rinehart & Winston.

Stake, R. E. (1995). The art of case study research. Thousands Oaks, CA: Sage.

Stevens, S., Andrade, R., & Page, M. (2016). Motivating young native American students to pursue STEM learning through a culturally relevant science program. Journal of Science Education and Technology, 25(6), 947-960.

Stocklmayer, S. M., Rennie, L. J., & Gilbert, J. K. (2010). The roles of the formal and informal sectors in the provision of effective science education. Studies in Science Education, 46(1), 1-44.

Sullenger, K. (2006). Beyond school walls: Informal education and the culture of science. Education Canada, 46(3), 15–18.

Swanborn, P. (2010). Case study research: What, why and how? Thousand Oaks, CA: Sage.

Tai, R. H., Qi Liu, C., Qi, Maltese A. V., & Fan, X. (2006). Planning early for careers in science. Science, 312(5777), 1143-1144.

Teddlie, C., & Yu, F. (2007). Mixed methods sampling a typology with examples. Journal of Mixed Methods Research, 1(1), 77-100.

The Partnership for 21st Century Learning (P21). (2016). Empowering Students For The Global Stage. Retrieved from http://www.p21.org/news-events/p21blog/1959-empowering-students-for-the-global-stage

The Partnership for 21st Century Learning (P21) (2015). P21 Framework Definitions. Retrieved from http://www.p21.org/storage/documents/docs/P21_Framework_Definitions_New_Logo_2015.pdf

http://digitalcommons.liberty.edu/cgi/viewcontent.cgi?article=2007&context=doctoral

http://digitalcommons.liberty.edu/cgi/viewcontent.cgi?article=2007&context=doctoral

http://www.p21.org/news-events/p21blog/1959-empowering-students-for-the-global-stage

http://www.p21.org/news-events/p21blog/1959-empowering-students-for-the-global-stage

http://www.p21.org/storage/documents/docs/P21_Framework_Definitions_New_Logo_2015.pdf

http://www.p21.org/storage/documents/docs/P21_Framework_Definitions_New_Logo_2015.pdf


217

The Partnership for 21st Century Learning (P21) (2015b). State Framework on Global Education. Retrieved from http://www.p21.org/storage/documents/Global_Education/P21_State_Framework_on_Global_Education_New_Logo.pdf

Thomas, B., & Watters, J. J. (2015). Perspectives on Australian, Indian and Malaysian approaches to STEM education. International Journal of Educational Development, 45, 42-53.

Thompson, R. A., & Zamboanga, B. L. (2004). Academic Aptitude and Prior Knowledge as Predictors of Student Achievement in Introduction to Psychology. Journal of Educational Psychology, 96(4), 778-784.

Tofel-Grehl, C., Fields, D., Searle, K., Maahs-Fladung, C., Feldon, D., Gu, G., & Sun, C. (2017). Electrifying engagement in middle school science class: improving student interest through e-textiles. Journal of Science Education and Technology, 26(4), 406-417.

Treagust, D. F., Won, M., Petersen, J., & Wynne, G. (2015). Science teacher education in Australia: Initiatives and challenges to improve the quality of teaching. Journal of Science Teacher Education, 26(1), 81-98.

Trochim, W. M. (2000). The research methods knowledge base (2nd ed.). Cincinnati, OH: Atomic Dog.

Usher, E. L. (2009). Sources of middle school students’ self-efficacy in mathematics: A qualitative investigation. American Educational Research Journal, 46(1), 275-314.

U.S. Bureau of Labor and Statistics (BLS). (2017). STEM Occupation: Past, Present, and Future. Retrieved from https://www.bls.gov/spotlight/2017/science-technology-engineering-and-mathematics-stem-occupations-past-present-and-future/pdf/science-technology-engineering-and-mathematics-stem-occupations-past-present-and-future.pdf

Uttal, D. H., & Cohen, C. A. (2012). Spatial thinking and STEM education: When, why and how. Psychology of learning and motivation, 57(2), 147-181.

Varnado, T. E. (2005). The effects of a technological problem solving activity on first lego league participants' problem solving style and performance (Doctoral dissertation, Virginia Tech).

VanMeter-Adams, A., Frankenfeld, C. L., Bases, J., Espina, V., & Liotta, L. A. (2014). Students who demonstrate strong talent and interest in STEM are initially attracted to STEM through extracurricular experiences. CBE—Life Sciences Education, 13(4), 687-697.

http://www.p21.org/storage/documents/Global_Education/P21_State_Framework_on_Global_Education_New_Logo.pdf

http://www.p21.org/storage/documents/Global_Education/P21_State_Framework_on_Global_Education_New_Logo.pdf

https://www.bls.gov/spotlight/2017/science-technology-engineering-and-mathematics-stem-occupations-past-present-and-future/pdf/science-technology-engineering-and-mathematics-stem-occupations-past-present-and-future.pdf





218

Vianna, E., & Stetsenko, A. (2006). Embracing History through Transforming It Contrasting Piagetian versus Vygotskian (Activity) Theories of Learning and Development to Expand Constructivism within a Dialectical View of History. Theory & psychology, 16(1), 81-108.Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.

Vilorio, D. (2014). STEM 101: Intro to tomorrow’s jobs. Retrieved from: http://www.bls.gov/careeroutlook/2014/spring/art01.pdf

Von Culin, K. R., Tsukayama, E., & Duckworth, A. L. (2014). Unpacking grit: Motivational correlates of perseverance and passion for long-term goals. The Journal of Positive Psychology, 9(4), 306-312.

Wagner, T., (2008). The Global Achievement Gap: Why Even Our Best Schools Don't Teach the New Survival Skills Our Children Need- And What We Can Do about It. New York, NY: Basic Books.

Watters, J., & Ginns, I. (2000). Developing Motivation to Teach Elementary Science: Effect of Collaborative and Authentic Learning Practices in Preservice Education. Journal of Science Teacher Education, 11(4), 301-321.

Wang, X. (2013). Why students choose STEM majors: Motivation, high school learning, and postsecondary context of support. American Educational Research Journal, 50, 1081–1121. http://dx.doi.org/10.3102/ 0002831213488622

Weber, K. (2011). Role Models and Informal STEM-Related Activities Positively Impact Female Interest in STEM. Technology and Engineering Teacher, 71(3), 18-21.

Weber, K. (2012). Gender differences in interest, perceived personal capacity, and participation in STEM-related activities. Journal of Technology Education, 24(1), 18-33.

Weiler, A. (2005). Information-seeking behavior in generation Y students: Motivation, critical thinking, and learning theory. The Journal of Academic Librarianship, 31(1), 46-53.

Whalen, D. F., & Shelley II, M. C. (2010). Academic success for STEM and non-STEM majors. Journal of STEM Education: Innovations and research, 11(1/2), 45-60.

White, D. W. (2014). What is STEM education and why is it important. Florida Association of Teacher Educators Journal, 1(14), 1-9.


http://dx.doi.org/10.3102/%200002831213488622


219

Williams-Watson, S. (2017). Persistence among Minority STEM Majors: A Phenomenological Study. Retrieved from the University of Phoenix ProQuest Dissertations and Theses. https://search.proquest.com/docview/1916889983?pq-origsite=gscholar

Woolnough, B. E. (2000). Authentic science in schools? An evidence-based rationale. Physics Education, 35(4), 293-300.

Woolnough B. (1994). Factors affecting students’ choice of science and engineering. International Journal of Science Education, 16(6),659-676.

Woolnough, B. E. (1994). Why students choose physics, or reject it. Physics Education, 29(6), 368-374.

Wyss, V. L., Heulskamp, D., & Siebert, C. J. (2012). Increasing middle school student interest in STEM careers with videos of scientists. International Journal of Environmental and Science Education, 7(4), 501-522.Xiang, P., McBride, R., & Solmon, M.A. (2003). Motivational climates in ten teachers’ elementary physical education classes: An achievement goal theory approach. The Elementary School Journal, 104, 71–92.

Young, D. J., Fraser, B. J., & Woolnough, B. E. (1997). Factors affecting student career choice in science: An Australian study of rural and urban schools. Research in Science Education, 27(2), 195-214.

Zimmerman, T. G., Johnson, D., Wambsgans, C., & Fuentes, A. (2011). Why Latino high school students select computer science as a major: Analysis of a success story. ACM Transactions on Computing Education (TOCE), 11(2), 1-17. Retrieved from https://dl.acm.org/citation.cfm?id=1993074

Zollman, A. (2012). Learning for STEM literacy: STEM literacy for learning. School Science and Mathematics, 112(1), 12-19.


220

APPENDICES

Appendix A

Recruitment Letter

Dear Parent(s)/Guardian(s):

The STEM-based career fields are to grow by 17.0% by 2018 as reported by the U.S. Department of

Commerce in 2011. To help support the STEM pipeline, which had over a 4.5% lower employment rate

between 1994-2010 then non-STEM occupations in the United States, it is important to understand why

students who are interested in these career areas and what is motivating them. We are trying to learn how

participating in a STEM extracurricular activity affects their STEM persistence.

To gain this insight, your child would be asked to complete two anonymous, online surveys/questionnaire

and participate in a short interview as well as observations will be recorded during the activity sessions. No

information will be gathered that could personally identify your child, and we would ask that you not put

your name on the survey. The interviews will take place in a separate room during the students’ individual

working time. The interviews will not take place during teacher instruction or interrupt student support

from the instructor. The interviews will take no more than 15 minutes of your child’s time during the

course. Furthermore, the total time allocated for the surveys and interviews will be less than 45 minutes. By

your child participating in this study, they may help us better understand how we support student interest in

the STEM fields.

The research is not part of the course and while the instructor is allowing it to take place, it is not part of the

expectations and the instructor will not know who participates and who doesn’t. Participation is

completely voluntary. The participant is free to leave the study at any time they wish. Thank you for

your time and consideration in helping us answer this important question. If you have any questions, please


221

do not hesitate to call David C. Taylor at 724-601-5650. More information is provided on the back of this

paper about this study.

Sincerely,

David C. Taylor

Texas Tech University Doctoral Student


222

Appendix B

Consent to Participate

What is this project studying?

This study will help us learn about the factors that inspire students to pursue STEM

(Science, Technology, Engineering, and Math) fields. As a result of this study, we will

help develop a better understanding of why students are interested in STEM education.

From this knowledge, we will create best practices for inspiring students to enter STEM

fields. Informal STEM learning lessons are an extra activity; from the student’s

experiences in these lessons, we can learn why they choose this course of learning.

What would I do if I participate?

In this study, students will be asked to share their experiences, thoughts and opinions.

These will be shared in two (2) ways: 1) two confidential, on-line surveys/questionnaires

and 2) an interview. Some of the questions from the surveys and interview will be about

his or her experiences related to engineering and STEM (Science, Technology,

Engineering, and Math) as a whole. Some questions will be about his or her thoughts.

Some will be about his or her interests, attitude and aptitude towards STEM. The

interviews will be audio recorded in order for us to obtain accurate information. The

interviews will take place during activities meetings in a separate room during the

student’s individual working time. The interviews will not take place during teacher

instruction or interrupt student support from the instructor. The interviews will take no

more than 15 minutes of your child’s time during the course. Furthermore, the total time


223

allocated for the surveys and interviews will be less than 30 minutes. A total of 45

minutes of your child’s time will be used throughout the course. Additionally, the

researchers may conduct observations of participants during activities directly related to

the study (i.e. during activity time). The researchers may take field-notes during this time

in order for us to obtain accurate information.

How will my child or me benefit from participating?

While no compensation, money or favoritism will be provided, your child will provide

the project with valuable information.

Can my child or I quit if I become uncomfortable?

Yes, absolutely. Your child’s involvement is completely voluntary. He or she may skip

any survey or interview questions he or she does not feel comfortable answering. He or

she can also stop answering questions at any time. He or she is free to leave the study at

any time. Participating is your choice. However, we do value any help you and your child

are able to provide. The research is not part of the extracurricular activity and while the

teacher is allowing it to take place, it is not part of the expectations and the teacher will

not know who participates and who doesn’t.

How long will participation take?

We are asking for a total of about 45 minutes of your student’s time of the time of the

course meetings.


224

How are you protecting privacy?

Your name and child’s name will not be connected to any documentation and any use of

this material in reports, publications or presentations will never be connected with your

child in this study without permission. No one other than the researchers associated with

this project will have access to the raw data. All related documentation will be stored

either in a locked file cabinet in the researcher’s office or on a password protected

computer. You and your child’s name and information will be kept confidential to the

research team. Teachers and other students will not be aware of your child’s

participation; only the school’s administrator in charge of extracurricular activities will

be aware of your child’s involvement. Mr. Morrow will be in charge of distributing and

collecting the documents at your school. Mr. Morrow will be the only person at school

with direct information of you and your child’s participation. By limiting access and

taking care not to identify your child during the study through limiting access to one

person at your school being aware of child’s participation, and the research team not

exposing your child to direct interactions with them that are obvious to peers and

teachers.

If my child or I have some questions about this study, whom can I ask?

David C. Taylor, a doctoral student at Texas Tech University, and Dr. Dan Carpenter, an

Assistant Professor of Education in Science Education within the Department of

Curriculum and Instruction at Texas Tech University, are running the study as research

related to doctoral studies at Texas Tech University. If you have any questions, you can


225

contact David C. Taylor at 724-601-5650 and [email protected], or Dr. Dan Carpenter

at 806-834-6660. TTU also has a Board that protects the rights of people who participate

in research. You can ask them questions at 806-742-2064. You can also mail your

questions to the Human Research Protection Program, Office of the Vice President for

Research, Texas Tech University, Lubbock, Texas 79409 or email them at [email protected].

_______________________________________________________________________

Signature Date

_______________________________________________________________________

Printed Name of Child

This consent form is not valid after 9/30/2017.

(Remember, even if you do say, “Yes,” now, you can change your mind later.)

mailto:[email protected]


226

Appendix C

Student Assent Form

Hello,

I am here today because I want to learn more about what past experiences are growing

students’ interests in STEM (Science, Technology, Engineering, and Math). This

information can help us learn about the factors that are exciting students to go into STEM

fields. I hope that you can help through your extracurricular STEM activity. I’m going to

ask a series of questions during an interview, as well as ask you to complete a survey and

interview. I’m going to record the interviews using an app. Lastly, I will be writing down

notes during your extracurricular STEM activities. All of the information collected will

be private, and I won’t share any of it with anyone else. Helping me during the

extracurricular STEM activity is up to you. If you decide you don’t want to participate in

the interview and surveys, that’s okay. If you want to help me, I’m going to ask you to

sign your name on the line below. Thank you.

_____________________________________________________________________

Student’s Signature Date

________________________________________________________

Printed Name

This assent form is not valid after 9/30/2017.

(Remember, even if you do say, “Yes,” now, you can change your mind later.)


227

Appendix D

Information Sheet

What is this project studying?

This study will help us learn about the factors that inspire students to pursue STEM

fields.

What would I do if I participate?

In this study, your son or daughter will be asked to share their experiences, thoughts and

opinions. These will be shared in two (2) ways: 1) two on-line questionnaires and 2) an

interview. The interviews will be audio recorded in order for us to obtain accurate

information. The researcher will also be doing observations.

How will my child or I benefit from being a part of the study?

While no payment, money or favoritism will be provided; your child will provide the

project with valuable information.

Is taking part in the study voluntary?

Yes! Your child’s involvement is completely voluntary. He or she may skip any survey

or interview questions he or she do not feel comfortable answering. He or she can also

stop answering questions at any time. He or she is free to leave the study at any time.

Joining is your choice. However, we do value any help you and your child are able to

provide. While the school is allowing the research to take place, it is not a part of the

extracurricular activity.

How long will my child be a part of the study?


228

We are asking for a total of about 45 minutes of your student’s time throughout the

course meetings.

How are you protecting privacy?

Your name and child’s name will not be connected to any part of the study and any use of

this material in reports, journals or presentations. No one other than the researchers will

have access to the raw data. All related data will be stored either in a locked file cabinet

in the researcher’s office or on a password protected computer. You and your child’s

name and information will be kept confidential to the research team. Teachers and other

students will not be aware of your child’s being a part of the study. Mr. Morrow will be

in charge of collecting the supplying and gathering the forms at your school. Mr. Morrow

will be the only person at school with information about who is in the study. Only the

researchers and one person at your school will be aware of your child’s participation. The

research team will not expose your child to direct interactions with them that will expose

that they are a part of the study. All forms and info will be destroyed after a year.

If my child or I have some questions about this study, whom can I ask?

You can contact David C. Taylor, a doctoral student at Texas Tech University, and Dr.

Dan Carpenter, an Assistant Professor of Education in Science Education within the

Department of Curriculum and Instruction at Texas Tech University. For questions about

your child’s rights as a subject, contact the Texas Tech University Human Research

Protection Program, Office of the Vice President for Research, Texas Tech University,

Lubbock, Texas 79409. Or, you can call (806) 742-2064 or email them at [email protected].


229

Appendix E

Observation Tool

During the course or activity, the observer will make notes regarding the following:

Class Observed: _____________________________________________________________

Observer: _________________________________________________________________

Date and Time of Observation: _________________________________________________

Support Questions for Observations:

1. What prior skills do the students have with regards to engineering, software and hardware?

2. What project is the student chosen to working on?

3. If students are engaged in oral engineering/STEM discussion, what is the content and topic of

discussion?

4. If students are engaged in a learning activity, what are they doing?


230

5. What is the over all demeanor of the students during the course?

6. What body language is taking place?

Additional notes and observations:


231

Appendix F

Interview Tool

While the class is working on their projects, the researcher will use the following guided questions and

format to interview each participant. The interview may not be limited to only these questions depending

on the interviewee’s response and the natural direction of the interview.

Introduction:

Thank you for speaking with me. I would like to write some of this down as you speak and record it, so I

can go over it later. Is that okay? I will review my notes with you to make sure I am accurately recording

your answers. The recording will be transcribing exactly the way you said at a later time. If you do not feel

comfortable answering a question, you do not have to answer it. Lastly, everything will remain anonymous

about you with regards to this study.

In this study, I am interested in finding out your thoughts about your experiences with this current

extracurricular activity, your prior experiences, and with STEM (Science, Technology, Engineering, and

Math) learning as a whole. This will help me gain an understanding of what is impacting students’

reasoning for choosing to continue with STEM learning. The results of this study will help guide future

students and teachers.

Interviewee School I.D.: _____________________________________________________________

Interviewer: _________________________________________________________________

Date and Time of Interview: _________________________________________________

1. How is the course or activate going for you?


232

2. Why did you choose to participate in this STEM activity or course? If any, what about them inspired

you?

3. Is it like anything you have done before?

4. If yes, what types of things?

5. How did you get involved in these things?

6. What new things have you learned in this course?

7. Do you see yourself continuing with these things/classes/activities throughout the rest of high school?

College? Etc.? Please tell me why?

8. If no, do you want to keep doing things like this? Do you know of anything like this at your school?

Would you want to learn similar tools, skills, or softwares?


233

9. Has this activity affected your decision to continue (or not continue) with STEM activities in the future?

If so, how?

10. Has anyone helped or inspired you to continue learning more STEM concepts? If anyone, why are they

inspirational or supportive?

11. Are there any factors are influencing your decision to continue (or not continue) with STEM activities

in the future?

Additional notes and observations:


234

Ending:

Do you have any other comments or questions? I appreciate you taking the time to answer my questions.

Thank you.


235

Appendix G

STEM Extracurricular Activity Questionnaire (Descriptive Statistics)


236


237


238

Appendix H

Student Attitudes Toward STEM (S-STEM) Survey


239

Appendix I

Email From MISO


240

Appendix J

Friday Institute Permission


241


242

Appendix K

Methodology Outline

Figure A.1. Methodology outline.


243

Appendix L

Audit Trail for Chapter IV

Note to the reader: As evidential referencing took place for this study, an endnote was

assigned to each piece of data being cited in order to establish a clear and consistent audit

trail. The endnotes are separated by section as well as point to the exact location of the

data in the document. Each endnote citation includes (1) OST STEM Activity-the OST

STEM activity that the participants were a part of during the study; (2) Source-the name

of the original data source, e.g. interview, and descriptive statistics; (3) Location-the

specific line in the document or location in the spreadsheet where the data source can be

found; and (4) Date- the date the original data source was collected.

Citation # Participants’ OST STEM Activity Source Location Date

1 Sumobots & Drones Interview 1 Line 2 4/26/17 2 Sumobots & Drones Interview 1 Line 17 4/26/17 3 Science Olympiad &

eCYMBERMISSION

Interview 2 Line 11 4/11/17

4 Sumobots & Drones Interview 3 Line 174 4/11/17 5 Sumobots & Drones Interview 3 Line 184 4/11/17 6 Sumobots Interview 4 Line 244 3/31/17

7 Science Olympiad & eCYMBERMISSIO

N


8 Sumobots Interview 4 Line 215 3/31/17 9 Sumobots Interview 4 Line 250 3/31/17 10 Sumobots Interview 7 Line 463 3/20/17 11 Girls Who Code Interview 8 Line 516 3/28/17 12 Sumobots & Drones Interview 9 Line 598 3/20/17 13 Science Olympiad Interview 10 Line 654 3/13/17 14 Science Olympiad Interview 10 Line 668 3/17/17 15 Sumobots Interview 13 Line 856 2/23/17 16 Science Olympiad Interview 14 Line 931 2/21/17


244

17 Sumobots Interview 11 Line 723 3/1/3/17 18 Science Olympiad &

eCYMBERMISSION



N


20 Science Olympiad Interview 14 Line 938 2/21/17 21 Sumobots Interview 12 Line 776 2/28/17 22 Science Olympiad &

eCYMBERMISSION

Interview 15 Line1013 2/21/17

15 Sumobots Interview 11 Line 726 3/13/17 16 Sumobots & Drones Interview 1 Line 3 4/26/17 17 Sumobots & Drones Interview 1 Line 7 4/26/17 18 Sumobots & Drones Interview 9 Line 601 2/9/17 19 Sumobots Interview 11 Line 686 3/30/17 20 eCYBERMISSION Descriptive Statistics

(demographic questionnaire)

Row 24, Column

K Response

to Q9

2/8/17

19 Sumobots Descriptive Statistics (demographic questionnaire)

Row 2, Column

K Response

to Q9

2/9/17

20 Science Olympiad & Girls Who Code

Descriptive Statistics (demographic questionnaire)

Row 29, Column J Response

to Q8

2/27/17

21 Science Olympiad & eCYBERMISSION


Row 18, Column S Response

to Q17

2/8/17

22 Sumobots & Drones Descriptive Statistics (demographic questionnaire)

Row 10, Column S Response

to Q17

2/817


Row 10, Column L Response

to Q10

2/8/17


245


Row 7, Column

K Response

to Q9

2/12/17


Row 33, Column

K Response

to Q9

2/8/17

24 Science Olympiad & Girls Who Code


Row 29, Column

K Response

to Q9

2/27/17

25 eCYMBERMISSION


Row 18, Column

K Response

to Q9

2/8/17

26 Sumobots & Drones Interview 9 Line 592 2/9/17 27 Science Olympiad &

eCYMBERMISSION


28 Girls Who Code Interview 8 Line 486 3/28/17 29 Sumobots & Drones Interview 1 Line 38 4/26/17 30 Girls Who Code Interview 8 Line 486 3/28/17 31 Sumobots & Drones Interview 3 Line 159 4/11/17 32 Science Olympiad Interview 10 Line 649 3/13/17 33 Science Olympiad &

eCYBERMISSION Interview 14 Line 941 2/21/14


Row 33, Column

K Response

to Q9

2/13/17


Row 34, Column

K Response

to Q9

2/8/17


Row 25, Column L

2/28/17


246

Response to Q10

37 Science Olympiad Interview 10 Line 630 3/13/17 38 Sumobots Interview 1 Line 24 4/26/17 39 Sumobots Interview 4 Line 224 3/31/17 40 Science Olympiad Interview 10 Line 641 3/13/17 41 Science Olympiad Interview 4 Line 334 3/31/17 42 Sumobots Interview 4 Line 267 3/31/17 43 Science Olympiad &


44 eCYBERMISSION Interview 15 Line 1012 2/21/17 45 eCYBERMISSION Descriptive Statistics


Row 24, Column

H Response

to Q6

2/8/17


Row 30, Column

H Response

to Q6

3/30/17

47 Science Olympiad Descriptive Statistics (demographic questionnaire)

Row 4, Column

H Response

to Q6

2/8/17

48 eCYMBERMISSION


Row 24, Column

H Response

to Q6

2/8/17


N


50 Sumobots Interview 3 Line 152 3/31/17 51 Sumobots & Drones Interview 7 Line 441 3/28/17 52 Sumobots & Drones Interview 7 Line 451 3/28/17 53 Girls Who Code Interview 8 Line 516 3/28/17 54 Girls Who Code Interview 8 Line 551 3/28/17 55 Sumobots Interview 13 Line 828 2/23/17 56 Sumobots & Drones Interview 9 Line 569 3/28/17 57 Sumobots & Drones Interview 9 Line 570 3/28/17 58 Sumobots & Drones Interview 9 Line 601 3/28/17 59 Science Olympiad Interview 10 Line 669 3/13/17


247

60 Sumobots & Drones Interview 9 Line 710 3/13/17 61 Sumobots & Drones Interview 1 Line 31 4/26/17 562 Science Olympiad &


63 Sumobots Interview 4 Line 219 3/31/17 64 Sumobots & Drones Interview 7 Line 465 3/28/17 65 Sumobots & Drones Descriptive Statistics


Row 32, Column

K Response

to Q9

2/28/17


Row16, Column

K Response

to Q9

2/10/17


Row 32, Column

K Response

to Q9

2/28/17



to Q10

2/28/17

69 eCYBERMISSION Descriptive Statistics (demographic questionnaire)


to Q10

2/27/17



to Q10

2/8/17


Row 15, Column

H Response

to Q6

2/24/17


Row 10, Column

H Response

to Q6

3/30/17


248


Row 15, Column

H Response

to Q6

2/24/17


Row 2, Column

H Response

to Q6

2/9/17



to Q10

2/8/17

76 Sumobots Interview 4 Line 215 3/31/17 77 Science Olympiad &






to Q10

2/8/17



to Q10

2/8/17



82 eCYBERMISSION Interview 6 Line 420 3/28/17 83 Science Olympiad &

eCYBERMISSION Interview Line 349 3/29/17

84 Girls Who Code Interview 8 Line 530 3/28/17 85 Science Olympiad Descriptive Statistics


Row 14, Column

H Response

to Q6

2/9/17

86 Sumobots Interview 11 Lines 734 3/13/17 87 Science Olympiad &


88 Science Olympiad Interview 14 Line 946 2/21/17


249


Row 7, Column

H Response

to Q6

2/9/17



to Q8

2/8/17


Row 21, Column

K Response

to Q9

3/30/17



to Q8

2/9/17



to Q10

2/24/17

94 Science Olympiad Interview 14 Line 954 2/21/17 95 Girls Who Code Descriptive Statistics



to Q10

3/31/17



to Q10

2/13/17

97 Sumbots Interview 13 Line 800 2/23/17 98 Science Olympiad Interview 14 Line 611 2/21/17 99 Science Olympiad Interview 14 Line 604 2/21/17 100 Science Olympiad Interview 14 Line 861 2/21/17 101 Sumobots & Drones Interview 1 Line 446 4/26/17 102 Sumobots & Drones Interview 1 Line 450 4/26/17


250

Appendix M

IRB Letter of Approval

Sep 22, 2017 11:12 AM CDT

Jerry Dwyer

CISER

Re: IRB2017-731 The Correlation and Influence of STEM Extracurricular Activities on

STEM Persistence in Middle School: A Convergent Mixed Methods Design Study

Findings: This study is approved.

Expiration Date: Aug 31, 2018

Dear Dr. Jerry Dwyer, David Taylor:

A Texas Tech University IRB reviewer has approved the proposal referenced above

within the expedited category of:

6. Collection of data from voice, video, digital, or image recordings made for research

purposes.


251

7. Research on individual or group characteristics or behavior (including, but not limited

to, research on perception, cognition, motivation, identity, language, communication,

cultural beliefs or practices, and social behavior) or research employing survey,

interview, oral history, focus group, program evaluation, human factors evaluation, or

quality assurance methodologies.

The approval is effective from Sep 22, 2017 to Aug 31, 2018. The expiration date must

appear on your consent document(s).

Expedited research requires continuing IRB review. You will receive an automated email

approximately 30 days before Aug 31, 2018. At this time, should you wish to continue

your protocol, a Renewal Submission will be necessary. Any change to your protocol

requires a Modification Submission for review and approval before implementation.

Your study may be selected for a Post-Approval Review (PAR). A PAR investigator may

contact you to observe your data collection procedures, including the consent process.

You will be notified if your study has been chosen for a PAR.

Should a subject be harmed or a deviation occur from either the approved protocol or

federal regulations (45 CFR 46), please complete an Incident Submission form.

When your research is complete and no identifiable data remains, please use a Closure


252

Submission to terminate this protocol.

Sincerely,

Kelly C. Cukrowicz, Ph.D.

Chair, Texas Tech University Institutional Review Board

Associate Professor, Department of Psychological Sciences

357 Administration Building. Box 41075

Lubbock, Texas 79409-1075

T 806.742.2064 F 806.742.3947

www.hrpp.ttu.edu

http://www.hrpp.ttu.edu/


253

Appendix N

Institutional Approval Form


254

Appendix O

Distribution of Forms

Please distribute the Parent Consent Form and the Information Sheet to students and/or

parents to be signed by the parents. Please collect the documents from the students by

having them put them in the sealed envelopes in a drop box that has been provided, so

that the teachers running each club, as well as you, are not aware of their participation in

the study. Lastly, please have the students sign the Student Assent Form. Please have the

students’ return the documents in a sealed envelope without their names on it by putting

in the drop box provided, and lock the envelops in a desk or filing cabinet. I will directly

collect them from you.

I will pick up the collected documents from you.

Distribution Script

Please take these documents home for you and your parents to sign. These documents are

for you to participate in a study connected to your STEM activity that you are

participating here at school. The study is interested in learning more about what past

experiences and motivations are growing students’ interests in STEM (Science,

Technology, Engineering, and Math). This information can help people, such as teachers,

learn about the important factors that are exciting students to go into STEM fields. I hope


255

that you can help throughout your extracurricular STEM activity. You will be asked a

series of questions during an interview, as well as ask you to complete a survey and

questionnaire. Please bring these documents back me after they are signed in a sealed

envelope without your name on it and put it in this drop box. Thanks!

If you have any questions, please do not hesitate to call David C. Taylor at 724-601-5650

and [email protected].

Thanks you,

David C. Taylor


256

Appendix P

S-STEM Survey Statistical Results

Table A.1

All Subjects Paired Means t Test Data

All Students Paired Means t test Scores

Section Mean SD SE T-Stat DF P-Value

Math - - - - 36 -

Science -1.7 3.77 0.62 -2.697 36 0.11

Engineering and Technology -0.54 4.23 0.7 -0.77 36 0.44

21st Century Learning -0.57 3.66 0.6 -0.94 36 0.35

Your Future -0.22 4.02 0.66 -0.33 36 0.746

About Yourself -0.64865 1.91799 0.31532 -2.057 36 0.047 Table A.2

Girls Paired Means t Test Data

Girls Paired Means t test Scores


Math -0.312 2.35850 0.589 -0.530 15 0.604

Science -2.375 3.68556 0.92139 -2.578 15 0.021

Engineering and Technology

-0.187 3.31097 0.827 -0.227 15 0.824


Your Future -0.812 3.42965 0.857 -0.948 15 0.358

About Yourself -0.0 1.21106 0.303 -0.0 15 1.0


257

Table A.3

Boys Paired Means t Test Data

Boys Paired Means t test Scores


Math -0.38 2.9 0.63 -0.604 20 0.552

Science -1.14286 3.9 0.84 -1.360 20 0.189


-1.09524 4.83 1.05 1.040 20 0.311


Your Future -0.23810 4.45 0.97 0.245 20 0.809

About Yourself -1.143 2.22 0.48 -2.359 20 0.029

Table A.4

6th Grade Paired Means t Test Data

6th Grade Paired Means t test Scores


Math -0.01389 0.88470 0.20853 -0.054 17 0.384

Science -0.16667 0.815951 0.19232 -0.80733 17 0.391667

Engineering and Technology -0.104939 0.81684 0.192532 -0.720333 17 0.386222


Your Future -0.00463 0.821606 0.193653 -0.001167 17 0.595583

About Yourself -0.04444 0.611821 0.144209 -0.3613 17 0.4796


258

Table A.5




Math 0.111 1.778 0.419 0.265 17 0.794

Science -1.388 4.461 1.051 -1.321 17 0.204

Engineering and Technology 1.277 5.062 1.193 1.071 17 0.299


Your Future 0.555 4.091 0.964 0.576 17 0.572

About Yourself -1.055 2.235 0.527 -2.003 17 0.061

Table A.6




Math -1.0 3.762 1.005 -0.995 13 0.338

Science -2.285 2.729 0.729 -3.133 13 0.008


-0.071 3.771 1.008 0.071 13 0.945

21st Century Learning -3.674 0.982 0.509 13 0.619 3.674

Your Future -0.643 4.199 1.122 -0.573 13 0.577

About Yourself -0.071 1.384 0.37 0.193 13 0.85


259

Table A.7

All Subjects Wilcoxon Signed-Rank Test Data

Question Z Asymp. Sig. (2-tailed)

Math

1. -.500b 0.617

2. -1.908c 0.056

3. -2.399b 0.016

4. -.263c 0.793

5. -.644b 0.519

6. -1.756c 0.079

7. -1.048c 0.295

8. -.909c 0.364

Science

9. -.329b 0.742

10. -1.162b 0.245

11. -.983b 0.326

12. -.803b 0.422

13. -.565b 0.572

14. -.341b 0.733

15. -.680b 0.497

16. -.538c 0.591

17. -.925b 0.355


260



18. -.471b 0.637

19. -2.121c 0.034

20. -1.291c 0.197

21. -.991c 0.322

22. -.593b 0.553

23. -.870c 0.384

24. -.994c 0.32

25. -.420b 0.675

26. .000d 1

21st Century Learning

27. -1.761b 0.078

28. -.258b 0.796

29 -.258b 0.796

30. -.832c 0.405

31 -.258b 0.796

32. -.243c 0.808

33 -.404b 0.686

34. -.655c 0.513

35. -.842b 0.4

36. -.225b 0.822

37. -2.500b 0.012


261


Your Future

1. -1.213b 0.225

2. -.836b 0.403

3. -.167c 0.867

4. -1.279c 0.201

5. .000d 1

6. -.188b 0.851

7. -.276b 0.783

8. -.440b 0.66

9. -.500b 0.617

10. -.607c 0.544

11. -.959b 0.337

12. -.688c 0.491

About Yourself

1. -.632b 0.527

2. -2.111c 0.035

3. -.333b 0.739

4. -1.184c 0.236

5. -.632c 0.527

6. .000d 1

7. -.294c 0.768

8. -.513c 0.608

9. -2.299c 0.022

10. -.660c 0.509 a. Wilcoxon Signed Ranks Test; b. Based on negative ranks c. Based on positive ranks; d. The sum of negative ranks equals the sum of positive ranks


262

Table A.8

Girls Wilcoxon Signed-Rank Test Data


Math

1. .000b 1

2. -.816c 0.414

3. -1.890d 0.059

4. .000b 1

5. .000b 1

6. -1.134c 0.257

7. .000b 1

8. -1.342c 0.18

Science

9. -1.890c 0.059

10. -2.126c 0.033

11. -.749c 0.454

12. -.632c 0.527

13. -1.508c 0.132

14. -2.121c 0.034

15. -1.508c 0.132

16. -1.000d 0.317

17. -1.841c 0.066


263



18. .000b 1

19. -1.732d 0.083

20. -.577c 0.564

21. -.832c 0.405

22. -1.508c 0.132

23. -.302d 0.763

24. -.333c 0.739

25. -.378d 0.705

26. .000b 1


27. -.632c 0.527

28. -.816c 0.414

29 -0.5b 0.617

30. -.816d 0.414

31. .000c 1

32. -1.189c 0.235

33 -.378c 0.705

34. .000b 1

35. -.447c 0.655

36. -1.890c 0.059

37. -.302b 0.763


264


Your Future

1. -1.508c 0.132

2. -0.98b 0.922

3. -.921c 0.357

4. -.378d 0.705

5. -.816d 0.414

6. -1.414c 0.157

7. -.816c 0.414

8. -.333d 0.739

9. -1.342c 0.18

10. .000b 1

11. -1.518c 0.129

12. -.905d 0.366

About Yourself

1. -1.414d 0.157

2. -1.414c 0.157

3. -.447d 0.655

4. -.378c 0.705

5. -.577c 0.564

6. .000b 1

7. -.322d 0.748

8. -.378c 0.705

9. -1.414c 0.157

10. .000b 1 a. Wilcoxon Signed Ranks Test; b. Based on negative ranks; c. Based on positive ranks; d. The sum of negative ranks equals the sum of positive ranks


265

Table A.9

Boys Wilcoxon Signed-Rank Test Data


Math

1. -.212b 0.832

2. -.344c 0.731

3. -.192b 0.848

4. -.351c 0.726

5. -.231b 0.817

6. -.024b 0.981

7. -.036b 0.972

8. -.159c 0.873

Science

9. -.369c 0.712

10. -.194b 0.846

11. -.423b 0.672

12. -.072c 0.942

13. -.037b 0.971

14. .000d 1

15. -.074c 0.941

16. -.122c 0.903

17. -.217b 0.828


266



18. -.378b 0.705

19. -.333c 0.739

20. -.037b 0.971

21. .000d 1

22. -.179b 0.858

23. -.332b 0.74

24. .000d 1

25. .000d 1

26. -.258c 0.796


27. -.189b 0.85

28. .000d 1

29 -.302c 0.763

30. -.302c 0.763

31 -.500c 0.617

32. 0.000b 1

33 -.225b 0.822

34. -.351c 0.726

35. -.538c 0.591

36. -.034b 0.973

37. .000d 1


267


Your Future

1. .000d 1

2. -.098b 0.922

3. -.355c 0.723

4. -.216b 0.829

5. -.421b 0.674

6. -.081b 0.935

7. .000d 1

8. -.226c 0.821

9. -.096c 0.923

10. -.535c 0.593

11. -.233b 0.816

12. -.504c 0.614

About Yourself

1. .000d 1

2. -.500c 0.617

3. .000d 1

4. -.462c 0.644

5. -.277b 0.782

6. .000d 1

7. .000d 1

8. -.513c 0.608

9. -.249c 0.803

10. -.655b 0.512 a. Wilcoxon Signed Ranks Test; b. Based on negative ranks; c. Based on positive ranks; d. The sum of negative ranks equals the sum of positive ranks


268

Table A.10

6th Grade Wilcoxon Signed-Rank Test Data


Math

1. .000b 1

2. -.577c 0.564

3. -1.414c 0.157

4. .000b 1

5. -1.000d 0.317

6. -1.000d 0.317

7. -1.000d 0.317

8. .000b 1

Science

9. .000b 1

10. -.577c 0.564

11. .000b 1

12. .000b 1

13. .000b 1

14. -1.000c 0.317

15. -.577c 0.564

16. -1.000c 0.317

17. -.577d 0.564


269



18. .000b 1

19. -1.000d 0.317

20. -1.000c 0.317

21. -1.000c 0.317

22. -1.414d 0.157

23. -1.000d 0.317

24. .000b 1

25. -1.414d 0.157

26. .000b 1


27. -1.414d 0.157

28. -1.000d 0.317

29 -1.000b 0.317

30. .000b 1

31 -1.000c 0.317

32. 0.000d 1

33 .000b 1

34. .000b 1

35. -1.000d 0.317

36. .000b 1

37. .000b 1


270


Your Future

1. .000b 1

2. -1.342c 1.8

3. .000b 1

4. -.816c 0.414

5. .000b 1

6. -.577d 0.564

7. -1.732d 0.083

8. -1.000c 0.317

9. .000b 1

10. -1.342c 0.18

11. .000b 1

12. .000b 1

About Yourself

1. -1.000c 0.317

2. -1.000d 0.317

3. -1.000c 0.317

4. .000b 1

5. .000b 1

6. .000b 1

7. -.447d 0.655

8. .000b 1

9. .000b 1

10. .000b 1 a. Wilcoxon Signed Ranks Test; b. Based on negative ranks; c. Based on positive ranks; d. The sum of negative ranks equals the sum of positive ranks


271

Table A.11

7th Grade Wilcoxon Signed-Rank Test Data


Math

1. -.707b 0.48

2. -2.179c 0.029

3. -1.811b 0.07

4. -.333b 0.739

5. -1.291b 0.197

6. -1.155c 0.248

7. .000d 1

8. -.378c 0.705

Science

9. .000d 1

10. -2.070c 0.038

11. -1.645c 0.1

12. -1.299c 0.194

13. -.649c 0.516

14. .000d 1

15. -1.473c 0.141

16. -.333b 0.739

17. -1.150c 0.25


272



18. -.535c 0.593

19. -1.732b 0.083

20. -.333b 0.739

21. -1.604b 0.109

22. -.525b 0.599

23. -1.265b 0.206

24. -1.732b 0.083

25. -.333b 0.739

26. -.378b 0.705


27. -1.732c 0.083

28. -.302c 0.763

29 0.000b 1

30. -.378b 0.705

31 -0.816c 0.317

32. -1.000c 0.444

33 -.707c 0.48

34. -.378b 0.705

35. -1.459c 0.145

36. -.264b 0.792

37. -2.111c 0.035


273


Your Future

1. -1.387c 0.166

2. -0.302d 0.763

3. .000d 1

4. -.632b 0.527

5. -.302b 0.763

6. -.351b 0.725

7. -.849b 0.396

8. -1.667c 0.096

9. -.302c 0.763

10. -.816b 0.414

11. -.277c 0.782

12. -.707b 0.48

About Yourself

1. -.447c 0.655

2. -1.633c 0.102

3. .000d 1

4. -.302c 0.763

5. -1.633c 0.102

6. .000d 1

7. -.368c 0.713

8. .000d 1

9. -1.994c 0.046

10. -.577c 0.564 a. Wilcoxon Signed Ranks Test; b. Based on negative ranks; c. Based on positive ranks; d. The sum of negative ranks equals the sum of positive ranks


274

Table A.12

8th Grade Wilcoxon signed-Rank Test Data


Math

1. .000b 1

2. -.707c 0.48

3. -1.000d 0.317

4. -.707c 0.48

5. -.378c 0.705

6. -1.155c 0.248

7. -1.186c 0.236

8. -.776c 0.438

Science

9. -1.134c 0.257

10. -1.633c 0.102

11. -1.081c 0.279

12. -1.633c 0.102

13. -1.732c 0.083

14. -1.667c 0.096

15. -.707c 0.48

16. .000b 1

17. -2.000c 0.046


275



18. .000b 1

19. -2.000d 0.046

20. -1.342d 0.18

21. -.632c 0.527

22. -1.265c 0.206

23. -.302d 0.763

24. -.264c 0.792

25. -.378c 0.705

26. -.333c 0.739


27. .000b 1

28. -.577d 0.564

29 -1.000b 0.317

30. -.816d 0.414

31 -.707c 0.48

32. -1.633c 0.102

33 .000b 1

34. -.577d 0.564

35. -1.342d 0.18

36. -.816c 0.414

37. -1.342c 0.18


276


Your Future

1. .000b 1

2. -0.632b 0.527

3. -.144d 0.885

4. -.816d 0.414

5. -.447c 0.655

6. -.577c 0.564

7. -.828c 0.408

8. -.632d 0.527

9. -.577c 0.564

10. -.513c 0.608

11. -1.414c 0.157

12. -.302d 0.763

About Yourself

1. -1.000d 0.317

2. -1.000c 0.317

3. .000b 1

4. -1.633c 0.102

5. -1.414d 0.157

6. .000b 1

7. -.276d 0.783

8. -.557c 0.577

9. -1.342c 0.18

10. -.272c 0.785 a. Wilcoxon Signed Ranks Test; b. Based on negative ranks; c. Based on positive ranks; d. The sum of negative ranks equals the sum of positive ranks


277

Appendix Q

Reliability Statistical Results

Table A.13

Reliability Statistics: Construct Level

S-STEM Survey Sections Cronbach’s

Alpha

Cronbach’s Alpha Based on

Standardized Items

Number of Items

Math Attitudes 1.0 1.0 2

Science Attitudes 0.78 0.78 2

Engineering and Technology Attitudes

0.8 0.85 2

21st Century Learning Skills Attitudes

0.88 0.87 2

Reliability Statistics: Item Level

S-STEM Survey Sections Cronbach’s

Alpha

Cronbach’s Alpha Based on

Standardized Items

Number of Items

Math Attitudes Pre-survey -1.01 -0.77 8

Math Attitudes Post-survey -0.83 -0.50 8

Science Attitudes Pre-survey 0.73 0.72 9

Science Attitudes Post-survey 0.80 0.80 9

Engineering and Technology Attitudes Pre-survey

0.79 0.78 9

Engineering and Technology Attitudes Post-survey

0.93 0.93 9

21st Century Learning Skills Attitudes Pre-survey

0.88 0.88 11

21st Century Learning Skills Attitudes Pre-survey

0.90 0.90 11

out of school time (ost) stem activities impact on middle

Documents