the university of tennessee – 1 · outlining 4-h youth activities and lockheed martin employee...

The University of Tennessee – 1

4‐HScience:Buildinga4‐HCareerPathwayInitiativeYearTwoProcessEvaluationReport

Joseph L. Donaldson, Ph.D.

Karen L. Franck, Ph.D.

June 2017


ContentsExecutive Summary ........................................................................................................................ 5 Introduction ..................................................................................................................................... 7 Purpose and Evaluation Questions .................................................................................................. 8 Methods........................................................................................................................................... 8 Findings......................................................................................................................................... 10

Question One. To what extent is the initiative providing youth with positive outcomes related to science and engineering? ...................................................................................................... 10

Science and Engineering Successes ...................................................................................... 11

Science and Engineering Challenges .................................................................................... 12

Question Two. To what extent is the initiative engaging its targeted youth population of girls and minorities? .......................................................................................................................... 15

Engaging Girls and Minorities Successes ............................................................................. 15

Engaging Girls and Minorities Challenges ........................................................................... 21

Question Three. To what extent is the initiative realizing the 4-H Career Pathway? ............... 22 Career Pathway Successes .................................................................................................... 22

Career Pathway Challenges .................................................................................................. 24

Question Four. To what extent is the initiative involving Lockheed Martin employees in 4-H volunteerism? ............................................................................................................................ 27

Employee Engagement Successes ........................................................................................ 27

Employee Engagement Challenges ....................................................................................... 30

Potential Initiative Exemplars ................................................................................................... 31 Adventures in Science (Maryland) ....................................................................................... 31

State 4-H Mission Make-It (Georgia) ................................................................................... 31

Paso Robles 4-H STEM Club (California) ........................................................................... 32

Cockeysville 4-H Robotics Club (Maryland) ....................................................................... 32

Video Conference with STEM Professionals (Texas) .......................................................... 32

Conclusions ................................................................................................................................... 32 Recommendations and Discussion ............................................................................................... 34

Year Three Programming Priorities .......................................................................................... 34 Professional and Volunteer Development ................................................................................ 35 Employee Engagement ............................................................................................................. 36 Realizing the 4-H Career Pathway ............................................................................................ 36

References ..................................................................................................................................... 36 Acknowledgements ....................................................................................................................... 37 Contact Information ...................................................................................................................... 37 Appendix ....................................................................................................................................... 38

4-H Community Volunteers Engaged Per State (Year Two) .................................................... 38


ListofTablesTable 1. 4-H Career Pathway and Evaluation Approaches ............................................................. 8 Table 2. Out of School Time Observation Instrument/STEM Plug-in Examples .......................... 9 Table 3. Overview of Science and Engineering Successes and Challenges ................................. 10 Table 4. Out of School Time Observation Instrument with STEM Plug-In Scores ..................... 12 Table 5. Overview of Successes and Challenges Engaging Girls and Minorities ........................ 15 Table 6. Cohort One Youth Reach and Benchmarks (Year Two) ................................................ 16 Table 7. Cohort One Youth Benchmark Status (Year Two) ......................................................... 16 Table 8. Cohort One Youth Reach and Benchmarks (Year One) ................................................. 17 Table 9. Cohort One Youth Benchmark Status (Year One) ......................................................... 17 Table 10. Cohort Two Youth Reach and Benchmarks (Year Two) ............................................. 18 Table 11. Cohort Two Youth Benchmark Status (Year Two) ...................................................... 18 Table 12. Cohort Two Youth Reach and Benchmarks (Year One) .............................................. 19 Table 13. Cohort Two Youth Benchmark Status (Year One) ....................................................... 19 Table 14. Overview of Successes and Challenges for the 4-H Career Pathway ........................... 22 Table 15. Career Pathway Definitions and Activities ................................................................... 25 Table 16. Overview of Successes and Challenges Involving Lockheed Martin Employees ........ 27 Table 17. Cohort One Lockheed Martin Employee Engagement ................................................. 28 Table 18. Cohort Two Lockheed Martin Employee Engagement ................................................ 28 Table 19. Potential Initiative Exemplars along the 4-H Career Pathway ..................................... 31


ListofFiguresFigure 1. 4-H STEM Career Readiness Pathway ............................................................................ 7


ExecutiveSummary Overview – The 4-H Science: Building A 4-H Career Pathway Initiative is a collaboration between National 4-H Council and Lockheed Martin to help youth develop STEM and workforce skills necessary for success, to immerse youth in the field of STEM work, and to engage youth with the science and engineering career pathway. The project seeks to build more youth for science professions. This three-year initiative is aimed at underserved youth, particularly girls and minorities. National 4-H Council and Lockheed Martin aim to involve 30,000 youth, up to 60% girls and minorities, through the efforts of Extension 4-H professionals partnered with 500 Lockheed Martin employees and 1,000 community volunteers in 13 states. Purposes – In Year 2, the overall purposes of the process evaluation were to track the initiative’s implementation and to describe the extent to which the initiative goals were being met. To what extent is the initiative –

1. Providing youth with positive outcomes related to science and engineering? 2. Engaging its targeted youth population of girls and minorities? 3. Realizing the 4-H Career Pathway? 4. Involving Lockheed Martin employees in 4-H volunteerism?

Methodology – A process evaluation was designed by researchers at the University of Tennessee. The process evaluation used a number of methods to measure performance. State grantees completed monthly activity reports to track youth reached. Focus groups were conducted with Lockheed Martin Employees, Extension 4-H Professionals, and 4-H youth and parents. Programs were observed and project documents were reviewed. Findings/Conclusions – The overall project benchmarks are being exceeded:

47,751 youth reached in Years One and Two o 30,870 in Cohort One States o 16,881 in Cohort Two States o 56% girls o 50% minority

492 Lockheed Martin employees engaged o 333 in Cohort One States o 159 in Cohort Two States

1,791 4-H Community Volunteers engaged o 946 in Cohort One States o 845 in Cohort Two States

The Career Pathway shows promise for empowering youth to achieve the attitudes, aspirations, and skills needed for successful STEM careers. In fact, youth survey results from learn and experience phases of the pathway demonstrated that:

98.8% report they now like science. 96.8% report they are now good at science. 95.3% think science will be important in their future. 94.9% taught others about science.


94.1% can now solve problems at school. 86.5% can now weigh the pros and cons of their future college options. 79.5% have helped with a community service project that relates to science.

The initiative has produced a number of best practices. States that exceeded their benchmarks for overall youth reached and involvement of girls and minorities located programs in under-served communities. They tended to recruit female and minority role models for youth specifically from Lockheed Martin, Society of Women Engineers, and/or National Society of Black Engineers. Lockheed Martin employees were successfully engaged in the initiative through informal networking with the Extension 4-H professionals in their local area. Texas 4-H professionals have successfully linked Lockheed Martin employees to 4-H youth through video conferences. Major barriers to employee engagement were the scheduling conflicts between work duties and 4-H events. States encountered common challenges related to professional development, STEM Curriculum, and Career Pathway development. 4-H professionals perceive that their STEM educational background is inadequate for the needs of today’s youth. 4-H professionals described the need for more curricula resources relative to STEM careers. 4-H professionals perceive that the 4-H Career Pathway as a viable resource for programming, but needs clear definitions and further development to be fully used across the 4-H System. Recommendations – National 4-H Council identified the need to strengthen the initiative’s outreach to girls. With state partners, National 4-H Council is currently piloting a new science program specifically for girls. National 4-H Council has also responded to the need for improved curriculum related to science careers with a Click2Science curriculum pilot test. If the pilots are successful, it is recommended that these programs be implemented during the final months of the current initiative. It is recommended that National 4-H Council implement the best practices identified over the first two years of this project during Year Three. A key strategy is to share the initiative exemplars with all state sub grantees, potentially via video conferences to emphasize practical approaches for involving girls and minorities, implementing the Career Pathway, achieving science and engineering outcomes, and engaging Lockheed Martin employees. Substantive professional and volunteer development is recommended to expand programs to under-served audiences through enhanced cultural competencies; to provide both basic and “level-up” science and engineering competencies; and to enhance pedagogy and youth development practices. It is recommended that Lockheed Martin promote 4-H volunteer opportunities from within the organization. Specifically, employees described the need to promote the initiative through internal corporate newsletters. State grantees should explore employee engagement through virtual opportunities to mediate difficulties related to travel, time, and work demands. Finally, it is recommended that National 4-H Council convene a national working group to improve the 4-H Career Pathway during Year Three in light of youth development research and science curriculum.


Introduction The 4-H Science: Building A 4-H Career Pathway Initiative is a collaboration between National 4-H Council and Lockheed Martin to help youth develop STEM and workforce skills necessary for success, to immerse youth in the field of STEM work, and to engage with the science and engineering career pathway. A key goal is involving more girls and monitories in STEM and STEM career fields. Ultimately, the project contributes to a robust pipeline for science professions in high demand such as computer scientists and engineers. Key goals for this three-year (2016-2018) initiative are:

30,000 youth involved, up to 60% girls and minorities 500 Lockheed Martin employees engaged 1,000 community volunteers engaged 13 states implementing the initiative

The 4-H STEM Career Readiness Pathway (Figure 1) provides an overall visual of the process, outlining 4-H youth activities and Lockheed Martin employee contributions. Figure 1. 4-H STEM Career Readiness Pathway

The process evaluation spans the entire three-year project, and this document is the Year Two Report. Process evaluations are conducted to document the extent to which program activities are implemented according to plan (CDC, 2017) with a focus on inputs/resources, activities, participation, and reactions (Radhakrishna & Bowen, 2010). As a process evaluation, this Year Two Report focuses on direct programming activities and audiences involved as well as successes, best practices, and recommendations to inform Year Three of project implementation.


Additionally, this process evaluation includes an outcome evaluation component to document the outcomes achieved by participants, for example, a greater interest in science. This Year Two Report should be read with an important caveat: All of the conclusions and recommendations are preliminary; they represent a snapshot of the project. The final report to be submitted after Year Three will include a comprehensive view of the project reflective of the entire evaluation effort and recommendations.

PurposeandEvaluationQuestions In Year 2, the overall purposes of the process evaluation were to track the initiative’s implementation and to describe the extent to which the initiative goals were being met. Specific evaluation questions follow. To what extent is the initiative –

1. Providing youth with positive outcomes related to science and engineering? 2. Engaging its targeted youth population of girls and minorities? 3. Realizing the 4-H Career Pathway? 4. Involving Lockheed Martin employees in 4-H volunteerism?

Methods This study was approved by the University of Tennessee Institutional Review Board (UTK IRB-15-02714-XP). The process evaluation for the 4-H Science: Building a 4-H Career Pathway Initiative has been aligned with the project goals, and it is a mixed-methods approach drawing on both the objectives-oriented and consumer-oriented evaluation traditions (Fitzpatrick, Sanders and Worthen, 2004). The central focus is to determine the extent to which the 4-H Science: Building a 4-H Career Pathway goals have been achieved, an objectives-oriented evaluation. The methods were specifically developed and implemented to describe science and engineering outcomes, the extent to which the initiative is reaching under-served youth, effectiveness of the 4-H Career Pathway, and extent of employee engagement. An overview of the 4-H Career Pathway and evaluation methods is shown in Table 1. Table 1. 4-H Career Pathway and Evaluation Approaches

Evaluation Approaches Explore Learn Practice Experience Monthly Activity Reports • • • • Focus Groups with Lockheed Martin Employees

• • • •

Focus Groups with Extension 4-H Professionals • • • • Focus Groups with 4-H Youth and Parents • • • • Focus Groups with 4-H Community Volunteers • • • • Surveys of 4-H Youth • • • Observation of Selected Programs Using OST Instrument with STEM Plug-In

• • • •

Document Review • • • •


Focus group interviews were conducted to describe opinions and perceptions of project participants regarding the project’s strengths and areas for improvement. To date, we have had 76 participants in 11 focus group interviews in four states and the STEM Futures Conference. Surveys of youth participants were used with Cohort One states to understand the outcomes achieved as the project progresses. These National 4-H Common Measures surveys focused on positive attitudes and aspirations, development of science skills and abilities, applying learning/making a contribution through science, career decision making, and college readiness. Cohort One states have submitted 242 surveys to date. States submitted monthly activity reports to track key project metrics such as the youth participants served by gender, race, and ethnicity. Extensive notes were taken during the monthly calls that National 4-H Council hosts with state sub- grantees. These notes, along with any other project documents produced, were considered as part of the ongoing document review for the project. Observations were made of actual initiative programs. To provide consistency and uniformity in observation, the Out of School Time (OST) Observation Instrument was used. The OST uses five domains:

Youth Relationship Building Youth Participation Staff Relationship Building Staff Instructional Strategies Activity Content and Structure

Each domain includes indicators and descriptions, and the total instrument has 28 indicators. A companion instrument for STEM program, the STEM Plug-in includes 14 indicators. Indicators are scored on a scale of 1 to 7 where 1 indicates the indicator is not evident and 7 indicates the indicator is highly evident and consistent. Examples are shown in Table 2. Table 2. Out of School Time Observation Instrument/STEM Plug-in Examples

Domain Indicator Description Youth

Relationship Building

Are friendly and relaxed with one

another

Youth socialize informally. They are relaxed in their interactions with each other. They appear to enjoy one

another’s company. Staff

Instructional Strategies

Ask youth to expand upon their answers and ideas

Staff encourage youth to explain their answers, to give evidence, or suggest conclusions. They ask youth “why,”

“how,” and “if” questions to get youth to expand, explore, better clarify, articulate, or concretize their thoughts/ideas. This item goes beyond basic Q&A.

STEM Youth Skill

Development

Draw connections to real-world concepts and

situations

Youth connect content to real-world situations. Youth may describe how STEM content appears in their

everyday lives or how STEM can be used to solve a problem in their community.


Findings For each question, successes and challenge have been described. A separate section has been included to detail program exemplars. These exemplars have been chosen for two purposes: to strengthen implementation in Year Three and to highlight the best practices with potential for replication throughout the 4-H system. QuestionOne.Towhatextentistheinitiativeprovidingyouthwithpositiveoutcomesrelatedtoscienceandengineering?

Overall successes and challenges are shown in Table 3 followed by a discussion that highlights findings from multiple data sources. Table 3. Overview of Science and Engineering Successes and Challenges Successes Challenges

Youth engagement and enthusiasm Youth receive support and guidance

Limited STEM knowledge for 4-H professionals and community volunteers

Limited youth development knowledge for volunteers

Confusion about science activities specifically for this initiative


ScienceandEngineeringSuccesses The 4-H STEM activities that have been provided as part of this grant received positive comments from youth, parents, volunteers and 4-H professionals. 4-H STEM activities provided through this project were engaging, successful, and provided hands-on learning that complemented and reinforced knowledge and content from school science classes. Observed clubs and events were similar in that they provided hands-on activities that were youth-driven with guidance from professionals and volunteers. One key difference was in the observed Maryland robotics club most youth had limited 4-H experiences but in the other states youth were much more involved with 4-H activities and other projects. Two key components contributed to the success of 4-H STEM activities: youth engagement and enthusiasm for the programs and support and guidance for youth provided by 4-H professionals and volunteers throughout the activities. Youth engagement and enthusiasm. Comments from youth, parents and volunteers indicated that youth were excited about these programs and wanted to participate. Several parents commented that this was the first out-of-school activity that their children wanted to attend and did not want to miss. One parent commented that her children sometimes would not eat supper because they wanted to get to the 4-H club. Parents and youth also appreciated the hands-on aspect of activities. Parents, especially in Maryland, identified the way 4-H STEM activities included all youth who wanted to participate—not just the top students. They also appreciated how 4-H was focused more on cooperation and helping others rather than winning competitions. Representative comments included:

“The alternatives we have in the area are mostly run through schools, and they tend to be

a lot more academic and competitive. . . if a child is looking for an extra-curricular activity, I think the 4-H club is a better fit. The school teams around here tend to be extremely competitive and very driven. . . so it might not be all that much fun if you don’t have a very competitive spirit or if you’re really not very motivated to win, win, win and things like that. So this is a very nice environment for them to explore what they want to explore. (MD parent)

“The kids get to do things on their own. It’s not adult-driven. It’s child-driven. The

children get to explore different things and figure out what works and figure out what doesn’t work. They’re working out their problems on their own.” (MD parent)

“There’s not really a right or wrong answer. They’re there to kind of discover. . . cause they’re not given that opportunity as much in the normal classroom.” (GA 4-H professional)

“I think schools, they don’t do as many like hands-on teaching. So I think it’s like very good, and it teaches, It’s just a better feeling and it really makes them engaged, like be engaged. It’s something that most schools don’t really do.” (CA youth)

“You really make it fun and interesting for them because there have been other programs that they have gone to once but they didn’t like it. But this, they want to come back. They like it.” (CA parents)


Youth receive support and guidance. Another success was the adult support and guidance that youth received in 4-H STEM activities contrasted with school science classes. Several youth commented on the importance of having this additional support.

“In school, since there’s more students, you don’t really get one-on-one help and then you don’t really understand what you’re doing. When you’re here, since we have mentors, we get more help.” (CA youth)

“At school, we are beginning to do science, but they don’t explain that much. But when I get here, they explain more.” (CA youth)

Evaluators observed actual programs during site visits. The OST instrument with the STEM Plug-In was used, and the indicators were scored on a scale of 1 to 7 where 1 indicates the indicator is not evident and 7 indicates the indicator is highly evident and consistent. For the Year 2 observations, scores ranged from 3.2 to 7.0 across the seven domains. The instrument showed positive results regarding all domains. However, indicators in STEM Youth Skill Development and STEM Staff Instructional Practices tended to be lower than the other domains (see Table 4). Specific areas for improvement are:

Draw connection to real-world concepts and situations. Discuss STEM careers and their educational pathways. Connect content to the real-world. Discuss how youth could pursue STEM content through their education and/or in a

career. It should be noted that staff is all inclusive of Extension 4-H professionals, para-professionals, and/or volunteers conducting the programming. Table 4. Out of School Time Observation Instrument with STEM Plug-In Scores

OST Domains CA-1 CA-2 CA-3 MD-1 GA-1 TX-1 Youth Relationship Building 7.0 7.0 7.0 5.6 7.0 6.4 Youth Participation 5.6 7.0 7.0 6.2 6.0 6.2 Staff Relationship Building 5.8 6.7 6.7 6.4 6.4 6.2 Staff Instructional Strategies 4.8 6.1 6.2 6.1 4.8 6.2 Activity Content and Structure 4.2 6.0 5.5 5.0 6.2 6.2 STEM Youth Skill Development 3.2 5.5 6.7 3.2 5.8 6.2 STEM Staff Instructional Practices 4.0 4.0 6.8 4.0 4.2 4.2 ScienceandEngineeringChallenges In addition to successes, participants also talked about challenges. Major challenges included: limited knowledge related to STEM subject matter for 4-H professionals and community volunteers and limited skills related to working and teaching youth for volunteers. Limited STEM knowledge for 4-H professionals and community volunteers. Several 4-H professionals identified their limited knowledge and skills related to STEM subject matter. They also felt that many potential community volunteers lacked these skills and abilities. Youth


ambassadors talked about the limitations of working with agents and club volunteers who did not have science backgrounds. This finding is a clear indication of the important need filled by this partnership with Lockheed Martin—the need for trained volunteers with passion and expertise in STEM. The 4-H professionals expressed the need to explore the potential for more Lockheed Martin involvement in their local programming, to increase STEM knowledge among youth, volunteers, professionals, and communities. Representative comments included:

“A lot of us don’t have a STEM background . . .” (CA 4-H professional)

“I’ll use robotics as an example. We hit a turnover with our volunteers, and we’ve had a very strong program, but literally nobody wants to take the training necessary to continue on with the program. And we’ve run an ad, we’ve offered possibly a stipend, and people really just did not want to take that level of training. They feel very uncomfortable with teaching that type of thing. And I had one parent even tell me, you know, when I said we had a person here that is willing to train you to make sure you understand everything, she told me when I bring my kids to something like that I don’t want to feel like the person is just learning how to do it. I want to know they have a level of expertise with doing it.” (MD 4-H professional)

“All our employees are adaptable, but it would also be nice to have a training on the foundation of physics or mechanics or basic engineering concepts, so that whenever I go out to teach robotics or whenever they go out to teach aerodynamics or whatever the latest, greatest lesson is, that we have a leg to stand on that makes us more credible.” (TX 4-H professional)

One suggestion for overcoming limited science skills was having prepackaged programs that were readily accessible for 4-H professionals and community volunteers. Comments included:

“If we can come up with age-specific material that teach certain elements of science that we can almost have ready kits or be able to put together simple kits to go use. That would kind of empower us to just run with it.” (CA Lockheed Martin volunteer)

“That’s good from our angle, from the educator’s angle, that it’s prepackaged. It’s ready to go. Here’s step one, step two, step three. Here’s a kit and you go and deliver it, and you know, it can be done in x amount of time. The teachers like that because they like those short, sweet, simple activities and lessons that kind of get them moving. . .” (TX 4-H professional)

“The National Youth Science Day curriculum is really nice. It comes as a kit. It’s got a facilitator manual. It’s really hands-on.” (CA 4-H professional)

Limited youth development knowledge for volunteers. Both professionals and volunteers discussed the need to improve volunteer skills in youth development practice, particularly among corporate volunteers. Representative comments included:


“When we find that expert, they can’t talk to the kids. They want to pile on all the information at once, and that’s when the kids get overwhelmed . . . they’re no longer having fun, so they’re going to trickle away. So finding that balance is difficult.” (MD 4-H professional)

“You get a lot of scientists there are just dying to teach but they don’t know how to teach

those kids.” (MD 4-H professional)

“I remember the first time I went into [agency]. I’d put a PowerPoint up and lost half of them about slide two or something. So I had the best intentions, but you have to know how to work with every different audience, and I think that’s training is a very event-specific thing because it could be a different audience one time versus the next time.” (CA Lockheed Martin volunteer)

“Break it down by age group like for the Science Day. We didn’t know the age group; it was like such a wide range. And it would have been helpful if there’s about five in this group and ten in this group, and then maybe some common behaviors for that group. Where like studies show their attention span is three minutes. That would have been helpful because in those two minutes you’ll lose them if you’re just saying hi how are you.” (CA Lockheed Martin volunteer)

Confusion about science activities specifically for this initiative. One source of concern is the confusion about what science curricula and activities “count” for this project. Several 4-H professionals made comments expressing concern about what types of science activities would be considered part of this project. For example, 4-H professionals in Georgia talked about working on ecology and environmental projects with youth but were not certain if this would be considered an activity that could be included for this project.


QuestionTwo.Towhatextentistheinitiativeengagingitstargetedyouthpopulationofgirlsandminorities?

Overall successes and challenges are shown in Table 5 followed by a discussion that highlights findings from multiple data sources. Table 5. Overview of Successes and Challenges Engaging Girls and Minorities Successes Challenges

Reaching girls Reaching minority youth (Cohort One

states) Identifying and serving underserved

communities Providing role models Developing local partnerships

Reaching minority youth (Cohort Two states)

Cultural competence professional development

EngagingGirlsandMinoritiesSuccessesThe Cohort One states have successfully reached girls and minorities at all stages of the 4-H Career Pathway. All three states have exceeded project benchmarks for Year Two for total youth reached, girls and minorities. The states were categorized based on comparing Year One and Year Two performance reports to the total benchmarks as follows: exceeded, met, significant


progress, limited progress, or no progress. At first glance, it may appear as if all benchmarks were not reached. At the explore phase, the Texas benchmark was 1200 total youth with 600 (50%) girls. Texas served 2782 youth at the explore phase with 806 (29%) being girls; however, this represents 67% of the benchmark (806 of 1200); see Tables 6 and 7. Table 6. Cohort One Youth Reach and Benchmarks (Year Two)

State Explore Learn Practice Experience

California 4476 total youth 59% girls 88% minority

476 total youth 51% girls 84% minority



CA Benchmark





Maryland 9676 total youth 52% girls 53% minority




Texas 2782 total youth 29% girls 79% minority




MD/TX Benchmark





Table 7. Cohort One Youth Benchmark Status (Year Two)


California Total – E1 Girls – E Minority – E

Total – E Girls – E Minority – E



Maryland Total – E Girls – E Minority – E




Texas Total – E Girls – E Minority – E




1The benchmark statuses were abbreviated as follows: E – Exceeded; M – Met; SP – Significant Progress; LP – Limited Progress; NP – No Progress A success for both Texas and Maryland was exceeding benchmarks in the practice and experience phases during Year Two as they had limited to no progress in these phases during Year One. Texas and Maryland were also leaders in involvement of 4-H community volunteers (see Appendix) with more than 200 volunteers per state involved in the initiative. See Tables 8 and 9.


Table 8. Cohort One Youth Reach and Benchmarks (Year One)


Maryland 6545 total youth 49% girls 59% minority




Texas 1256 total youth 50% girls 61% minority




MD/TX Benchmark





Table 9. Cohort One Youth Benchmark Status (Year One)


Maryland Total – E1 Girls – E Minority – E


Total – LP Girls – SP Minority – SP

Total – NP Girls – NP Minority – NP

Texas Total – E Girls – E Minority – E




1The benchmark status were abbreviated as follows: E – Exceeded; M – Met; SP – Significant Progress; LP – Limited Progress; NP – No Progress Notable performance issues for Cohort Two for Year Two include:

Florida and Kentucky were the only states to exceed benchmarks for total youth, girls, and minorities.

Five states exceeded benchmarks for girls: Alabama, Florida, Georgia, Kentucky, and Pennsylvania.

Four states showed no progress in reaching minority youth: New Jersey, New York, Pennsylvania, and Washington.

Performance regarding total reach, girls and minorities of both New Jersey and Washington was rated as no progress.

For Year Two, the total reach, girls, and minorities are shown for each Cohort Two state in Table 10 and benchmark statuses are shown is Table 11.


Table 10. Cohort Two Youth Reach and Benchmarks (Year Two)

State Total Girls Minorities

n % n % Alabama 628 313 50 36 5 Arkansas 113 58 51 67 59 Colorado 159 72 45 110 69 Florida 1709 844 49 916 54 Georgia 887 576 65 167 18 Kentucky 2777 1489 54 933 34 New Jersey 34 15 44 24 71 New York 204 105 51 28 14 Pennsylvania 2565 1209 47 96 4 Washington 66 31 47 37 56 Benchmark 350 175 50 175 50

Table 11. Cohort Two Youth Benchmark Status (Year Two)

State Total Girls Minorities Alabama E1 E NP Arkansas LP LP LP Colorado LP LP SP Florida E E E Georgia E E SP Kentucky E E E New Jersey NP NP NP New York SP SP NP Pennsylvania E E NP Washington NP NP NP

1The benchmark status were abbreviated as follows: E – Exceeded; M – Met; SP – Significant Progress; LP – Limited Progress; NP – No Progress


Notable performance for Cohort Two for Year One include: Florida, Georgia, Kentucky, and New Jersey exceeded benchmarks for total youth, girls,

and minorities. Alabama, Arkansas, and Pennsylvania made no progress and Colorado made limited

progress reaching minorities. Arkansas made no progress and Alabama made limited progress reaching girls. Washington showed no progress as zero contacts were reported.

For Year Two, the total reach, girls, and minorities are shown for each Cohort Two state in Table 12 and benchmark statuses are shown is Table 13. Table 12. Cohort Two Youth Reach and Benchmarks (Year One)

State Total Girls Minorities

n % n % Alabama 112 45 40 0 0 Arkansas 52 13 25 22 42 Colorado 170 94 55 75 44 Florida 268 102 38 158 59 Georgia 1304 860 66 221 17 Kentucky 4214 2191 52 379 9 New Jersey 374 733 51 228 61 New York 234 115 49 131 56 Pennsylvania 1011 525 52 10 1 Washington 0 0 0 0 0 Benchmark 250 125 50 125 50

Table 13. Cohort Two Youth Benchmark Status (Year One)

State Total Girls Minorities Alabama LP LP NP Arkansas LP NP NP Colorado SP SP LP Florida E E E Georgia E E E Kentucky E E E New Jersey E E E New York M SP E Pennsylvania E E NP Washington NP NP NP

1The benchmark status were abbreviated as follows: E – Exceeded; M – Met; SP – Significant Progress; LP – Limited Progress; NP – No Progress


It is possible that states have been successful engaging girls because 4-H is provided equally to girls and boys, consistent with Federal legislation (Civil Rights Act, 1964). As a California 4-H professional stated: “I don’t think we’ve done specific recruitment for this project to increase female participation because they’re already there participating.” Other specific successful practices for engaging underserved youth were identifying and serving schools and areas with minority populations, providing role models and developing partnerships. Identifying and serving underserved communities. Several 4-H professionals attributed their success to specifically identifying and including schools and areas with minority populations in this project. Representative comments included:

“I think knowing ahead of time that the target was 50% minority and 50% female, there was a concerted effort on all of our parts to make sure that those are the schools that could help us meet those. And so whereas we might have been doing something different, it really made us focus in on what we should be doing and recruiting at certain schools that had those audiences ready-made for us.” (TX 4-H professional)

“We targeted our locations to be providing programming in communities that are predominantly Latino.” (CA 4-H professional)

“And most of our programming, as far as Science Adventures, I would say we have an equal amount, if not more, girls that participate in that program. As a robotics program, it may be more boys, but we have several minorities in regards to African American and Hispanic. . . So the efforts to specifically reach out to minorities; we don’t necessarily have to do that because that’s pretty much who we service. But there has been a bit of a different push to reach out specifically to girls, especially within robotics.” (MD 4-H professional)

Providing role models. Several respondents identified their success through the engagement of role models in the program for both girls and minority youth. Several 4-H professionals also felt that it was important that volunteers be young so that youth could identify with them. Youth ambassadors talked about being successful engaging younger students because they served as role models and were excited about science. In Maryland, a 4-H professional talked about the importance of including girls in STEM marketing materials in an effort to help boost their interest. Typical comments included:

“So just a role model. Not that I’m like their role model, but just them seeing another black woman that’s into science.” (TX 4-H professional)

“So what I’m battling out is trying to get rid of the preconception of what a smart person looks like. So this was young black men and women, you know, wearing nice shoes. Wearing this and that, but they were so intelligent. These kids don’t see that. And so when we introduce ourselves, you know, they went around and spoke about each college they went to, and they spoke about the sororities and fraternities they went to, then they went to say ‘I designed rockets’ or ‘I do that’, and the kids were like ‘Oh!’” (TX 4-H professional)


“As far as the college kids, it actually helps them to also expose the youth to what it’s

like to be on a college campus. It makes the youth excited about that because they get to work to somebody closer to their age.” (MD 4-H professional)

“And if they’re [Lockheed Martin volunteers] young, and if they look like the kids do, that makes it all the better.” (TX 4-H professional)

Developing local partnerships. The key partnership identified was Lockheed Martin and this partnership is summarized in Objective 4. The findings discussed here relate to additional local partnerships that were formed to engage girls and minority youth. Examples include working with a women’s basketball team to sponsor a 4-H Stem-based Awareness Night as well as reaching out to professional scientific organizations for minorities and women. Typical comments included:

“We are trying to reach elementary to middle school to high school kids and women, we are also extending that into building relationships with college and college-bound students. So what we do is we partnered with Girls Inc., Society of Women Engineers, and 4-H, hopefully to build this collaboration to continue to grow and evolve into the offerings that we [Lockheed Martin volunteers] can provide.” (CA Lockheed Martin volunteer)

“When I recruit volunteers, I also made an effort to have women in the field of engineering interacting with the youth. And we had a partnership with the Society of Women Engineers at Cal Poly who came and led a lesson at Youth Works.” (CA 4-H professional)

“Once I made that contact, they’re [National Society of Black Engineers] calling me because they’re required to do things. So just by them needing us and we need their volunteers, we have two big programs coming up that they will be there. “ (TX 4-H professional)

EngagingGirlsandMinoritiesChallenges Cultural competence professional development. The major challenge identified and described by participants was the need for cultural competence when working with different ethnic groups. The 4-H professionals who attended the initial STEM Futures meeting at National Council in April 2016 expressed the importance and relevance of the cultural competency training component of this meeting. However, there is a need for ongoing professional development in this area as respondents talked about the importance of understanding about different cultures including how those cultures could include different reactions and attitudes towards 4-H and STEM. This was described by a California 4-H professional:

“Parents were happy for their girls to come and participate in our programming after school to learn about whatever they were interested in, but when it came time to talk about college applications, when it came time to talk about career paths, the conversation ended. That wasn’t an option for a lot of the girls who were participating in our programs.


So it was fine for them to be interested in science or engineering, but it wasn’t okay for them to become a scientist or to become an engineer. That’s not something that we tackled, but it’s something that I think about.”

QuestionThree.Towhatextentistheinitiativerealizingthe4‐HCareerPathway?

Overall successes and challenges are shown in Table 14 followed by a discussion that triangulates findings from multiple data sources. Table 14. Overview of Successes and Challenges for the 4-H Career Pathway Successes Challenges

Connecting 4-H STEM to careers

Measured outcomes demonstrating positive changes

Definition of concepts and expectations

Need to develop resources related to careers

CareerPathwaySuccessesCohort One and Two states are expected to recruit and engage youth in the Explore Stage. This stage has been defined as “introductory, short-term STEM projects”. Most states have been successful in providing large group STEM events for youth from across the state. However, 5 Cohort Two states have recruited significantly fewer than the 350 total youth benchmark with 2 states reporting under 100 youth engaged (New Jersey with 34 youth and Washington with 66 youth) and 2 states under 200 youth (Arkansas with 113 youth and Colorado with 159 youth). New York reported 204 youth in Year Two. In addition to the explore phase, Cohort One states


are expected to reach and engage youth in learn, practice and experience stages. The three states have aligned their STEM activities with these stages. See Tables 6, 7, 8, and 9 on pages 16-17. Connecting 4-H STEM to careers. One successful practice is tying 4-H STEM activities to future careers and educational goals. 4-H professionals have worked to connect STEM activities to careers for youth as evidenced in their comments and the comments of youth and parents.

“The school we’re going today, this teacher is very self-sufficient, but she was good at teaching the kids how to do robotics, but they couldn’t make the connection of why is this important. So I pretty much had to take a step back and start over. Like step 1, we need this because you are represented in this kind of career pathway.” (TX 4-H professional)

“We took the youth there [USDA] to work in their garden, and [female 4-H youth] realized this is a STEM job, and I can do this and get paid to work in a garden all day and learn how to create technology dealing with goldfish that provides manure and nutrients to plants. I mean, it just opened up a whole new realm for her, as well as one of her other friends that took the class. And they talk all the time about, you know, I would be interested in a career that would allow for me to do that.” (MD 4-H professional)

“I would [recommend this program to other youth] because you can like, it can help them get more interested, and then maybe in the future we can get a job in it.” (CA youth)

“It’s an activity that they can use with their heads and they’re thinking and science skills and maybe go to college for it.” (CA parent)

Measured outcomes demonstrating positive changes. Cohort One states randomly selected a sample of youth to complete 4-H Common Measures surveys related to the 4-H Career Pathway goals and outcomes. Completed surveys from youth participants demonstrated positive changes as described below. A total of 195 surveys were collected to measure outcomes at the learn phase (150 surveys from youth in grades 4-7 and 45 from youth in grades 8-12). Since these youth were randomly selected for survey participation, the findings can be generalized to the entire population served. Among 3339 youth served at the learn phase of the 4-H Career Pathway:

Attitudes and Aspirations 98.8% report they now like science. 96.8% report they are now good at science. 70.1% would like to have a job related to science. 62.7% do science activities that are not for school.

Attitudes and Aspirations, 8-12 graders

95.3% think science will be important in their future. 95.2% think science is useful for solving everyday problems.


At both learn and experience phases, 8-12th graders were surveyed regarding how the project may have influenced their decision making and problem-solving skills in the context of school and college. Among 3339 youth served at the learn phase and 137 youth served at the experience phase of the 4-H Career Pathway:

Decision Making and Problem-Solving, School 94.1% can now solve problems at school. 91.8% can now compare each possible solution with others to find the best one at

school. 80.2% report that once they have solved a problem at school, they consider how

their solution worked. Decision Making and Problem-Solving, College Readiness

86.5% can now weigh the pros and cons of their future college options. 75.9% now know where to look for information to help make college decisions. 78.3% can now identify information that is most important when making a decision

about their college options. At the Experience level, 47 youth in grades 8-12 were surveyed about applying learning and making a contribution through science, consistent with the Experience definition. As a result of this 4-H initiative, 137 youth at the experience phase realized the following:

Apply Learning and Make a Contribution through Science 94.9% taught others about science. 79.5% have helped with a community service project that relates to science. 66.7% organized or lead a science-related event. 51.3% used science tools to help in the community.

Among those in grades 4-7, 92.4% reported that they were in their first year of 4-H participation, and 46% of those in grades 8-12 reported that they were in their first year of 4-H participation. This suggests that the initiative is successfully reaching new audiences who might not have otherwise been involved in 4-H. CareerPathwayChallengesChallenges included the need to fully develop concepts, theory and expectations and to explore and develop more resources related to careers. Definition of concepts and expectations. The three Cohort One states have defined STEM activities and events for each of the stages but comments were made about the need for common definitions and expectations. Table 15 shows how definitions from the project application align with definitions described by states and activities used by states across the Career Pathway. In addition, one 4-H professional identified the need for promotional materials related specifically to the pathway that could be used to describe the program when working with Lockheed Martin volunteers.


Table 15. Career Pathway Definitions and Activities

Explore Learn Practice Experience Definition (Project Application)

Youth explore STEM

Introductory Short-term

STEM projects focused in underrepresented communities

Youth develop skills necessary for success

Long-term experiences to engage more girls and multicultural youth in engineering

Youth immerse themselves in the field of work

Long-term, rigorous projects to build both STEM and leadership skills in preparation for college major

Youth experience these skills in real world settings

Collaboration with LM businesses to explore careers and gain marketable experience

Definitions (Reported by States)

Brief experiences; one hour or less

1-6 hours of educational contact

Generally greater than 6 hours contact over a series of months or year-round

Ongoing experience with a STEM professional who engages youth through STEM and/or career development

Activities (Reported by States)

National Youth Science Day

STEM festivals Fairs Outreach events

After-school Day camps Workshops Special

interest programs

Clubs Ongoing

after-school Robotics

teams Engineering

programs

Mentoring Job

shadowing Internship Career

exploration

Representative comments included:

“I would love to see the pathway a little more developed and articulated. We spent a lot of time talking about what’s the difference between learn and practice. And at what point are we bumping somebody into experience? It’s like we’ve kind of come up with our own criteria for what that means, which, you know, I think more details would be helpful. . . I feel like the theory and the potential of the pathway is not articulated well enough.” (CA 4-H professional)


“You have the possibility of using this pathway to really create a culture where families are thinking for the first time maybe it is okay for my daughter to be a scientist or an engineer, and that’s not articulated in the pathway that I’ve seen, and I feel like it would help us all.” (CA 4-H professional)

“I think semantics . . . or definitions need to be clearer.” (TX 4-H professional) Resources to support careers. 4-H professionals identified several ways that National Council could help support and expand the Career Pathway for youth. Several ideas related to increasing resources to support careers. Additional ideas were to incorporate career development ideas throughout all 4-H curricula as described by a 4-H professional: “So if we can incorporate the career piece in all of our curriculum (sic) so that it just becomes second nature, then I think it’s easier for our volunteers when we pass on those curriculums (sic) to then just incorporate it.” Additional typical comments included:

“I think it would be awesome if 4-H could leverage the relationship that they have with people like Lockheed Martin, so that in addition to doing this programming that we’re providing more concrete opportunities to go beyond. Something that motivates a youth from sixth grade through high school to stick with it. Like there’s going to be x number of internship positions or college university grants or jobs. We’re saving x number of jobs to hire from youth that went through this program.” (CA 4-H professional)

“I think we’re all pretty good at that explore and learn. It’s getting those set up; that’s

what we do every day. The practice part, you know, we’re doing a lot of that already as well. But we get to the experience part, I think that’s someplace that not only is it something that we’re not typically doing, but I think that someplace National Council could really help us with, not just Lockheed, but all of their partners, reaching out and saying, you know, we have 4-Hers that want to do job shadowing.” (MD 4-H professional)

“Maybe like a college info session with some of the parents of 4-H. My child is really

interested in science and was thinking engineering or medical or whatever. We’d just have a little roundtable of like how did you get to do what you do?” (CA Lockheed Martin volunteer)


QuestionFour.TowhatextentistheinitiativeinvolvingLockheedMartinemployeesin4‐Hvolunteerism?

Overall successes and challenges are shown in Table 16 followed by a discussion that highlights findings from multiple data sources. Table 16. Overview of Successes and Challenges Involving Lockheed Martin Employees Successes Challenges

Identifying win-win situations Scheduling conflicts with work duties and volunteer opportunities

EmployeeEngagementSuccessesAs evidenced by 4-H professional input and the summarized monthly activity reports, Lockheed Martin employee engagement continues to be a challenge for most states with a few exceptions. Per project application, Cohort One states were expected to have 135 Lockheed Martin employees engaged over the three year project as follows: 80 in explore phase, 30 in learn phase, 20 in practice phase, and 5 in experience phase. The states were categorized based on comparing year one and Year Two performance reports to the total benchmarks as follows: exceeded, met, significant progress, limited progress, or no progress. See Table 17.


Table 17. Cohort One Lockheed Martin Employee Engagement

State Number of Lockheed

Martin Employees (Year Two)

Number of Lockheed Martin Employees

(Year One)

Benchmark Status1

Texas 230 total 107 Explore 99 Learn 1 Practice 23 Experience

72 total 52 Explore 20 Learn

Total – E Explore – E Learn – E Practice – NP Experience – E

California2 23 total 3 Explore 14 Learn 2 Practice 0 Experience

n/a Total – SP Explore – LP Learn – SP Practice – LP Experience – NP

Maryland 6 total 0 Explore 6 Learn 0 Practice 0 Experience

2 total 1 Explore 1 Learn 0 Practice 0 Experience

Total – NP Explore – NP Learn – NP Practice – NP Experience – NP

1The benchmark statuses were abbreviated as follows: E – Exceeded; M – Met; SP – Significant Progress; LP – Limited Progress; NP – No Progress 2California’s initiative began May 1, 2016 while all other state initiatives began a year earlier. Cohort Two states were expected to have 10 engaged in the explore phase over the three year project. The states were categorized based on comparing year one and Year Two performance reports to the total benchmarks as follows: exceeded, met, significant progress, limited progress, or no progress (Table 18). Table 18. Cohort Two Lockheed Martin Employee Engagement

State Number of Lockheed

Martin Employees (Year Two)

Number of Lockheed Martin Employees

(Year One) Benchmark Status

Arkansas 38 24 Exceeded Kentucky 16 30 Exceeded New York 11 15 Exceeded Washington 7 4 Exceeded Colorado 2 4 Significant Progress Pennsylvania 1 5 Significant Progress Alabama 1 0 No Progress Florida 1 0 No Progress Georgia 0 0 No Progress New Jersey 0 0 No Progress


The project request for applications discussed the different volunteer roles that Lockheed Martin would fulfill during this three-year project, specifically: STEM Program manager, Community Relations Lead, 4-H Champion, Leadership Development Clubs, Employee Affinity Groups, and Volunteer Clubs. 4-H professionals described how these roles did not exist, or they had difficulty maintaining relationships with their Lockheed Martin contact due to personnel changes in Lockheed Martin. Evaluators noted that of the five states visited, only California 4-H professionals and Lockheed Martin volunteers identified one of the Lockheed Martin employees as their “4-H Champion”. Consistent with the request for applications, this Lockheed Martin employee was the lead contact with the local 4-H professional for coordinating Lockheed Martin involvement in bi-annual 4-H Science events. Identifying win-win situations. While the formal volunteer roles have not worked, informal networking has been successful in engaging Lockheed Martin volunteers at the local level. One TX 4-H professional described their journey: “They [Lockheed Martin] didn’t know us. We didn’t know them. . . I don’t think that they probably felt comfortable going to a foreign environment to go volunteer for an organization that I don’t really know. . . So I think it took about a year to try to finally get our foot solidly in the door before we really established that relationship.” Texas has been successful engaging Lockheed Martin employees through virtual lecture series that allows for volunteers to contribute in a unique way that can work with their schedules and locations. Both Lockheed Martin and Extension 4-H professionals have reported that a major strategy has been to identify win-win situations for 4-H and Lockheed Martin volunteers to work together. Representative comments included:

“The State Fair had, you know, open arms to letting us come in and have one of the National Youth Science Day experiments there and . . . we could bring school groups to the State Fair. And not only did they get the fair experience, but they got this science experiment, experience, as well. This year. . . we’ve had an outpouring from Lockheed Martin, and I think having it at the State Fair probably helped get Lockheed Martin employees there because one they were done . . .they could spend time with their family at the fair, so it was kind of a win-win situation for them and for us.” (TX 4-H professional)

“We’ve had to adapt our programming to know they’re [Lockheed Martin] not going to come on a Saturday. I mean, some of them will, but you know we’ve had to figure out, okay how do we work with the employees best? We work with being able to have events at their facilities. We’ve learned to have it to where they can come into a school that’s close by them.” (TX 4-H professional)

“It’s a win-win because now people are starting to recognize 4-H is in Dallas and we do something other than agriculture, but we’re also started to reach out to the demographics that we said we wanted to reach.” (TX 4-H professional)

“We really benefit as a company doing these things. I mean, the employees, all of us have the desire and the give the employees the chance to do this and fit it into their work schedule and get support for it is very rewarding.” (CA Lockheed Martin volunteer)


“I just want to underline the importance of just having financial supports for 4-H STEM and if the volunteer part works out that’s better yet because we’ve all discussed the need for high-tech volunteer support.” (MD 4-H professional)

Lockheed Martin volunteers in California discussed the need for internal communication about 4-H volunteer opportunities. One commented: “We could also use, I think, some more publicity to let people know what’s happening. I would say definitely both locally and also within Lockheed. I thought that within Lockheed we could have done more advertising, get a little more leadership attention on it.”

Another idea to engage Lockheed Martin volunteers was to build out the national relationship between 4-H and Lockheed Martin down to the state level through a career day or other activity as described by a MD 4-H professional: “I think it will be very helpful if we had on a national level some kind of way that it was set up with Lockheed Martin to possibly have like a career day or something like that. Because they [National Council] do some much with Lockheed Martin.”

EmployeeEngagementChallengesIt is clear from the low number of Lockheed Martin volunteers that many states have faced challenges meeting these expectations. One state 4-H professional described how difficult it has been to connect 4-H youth with Lockheed Martin volunteers because volunteers are only willing to travel a short distance from the facility and youth are not permitted in the plant. As another 4-H professional commented: “I think that was part of our frustration too is that the grant required internships; however, the culture within and the security level of Lockheed Martin is what’s going to be the prohibitive part of that whole.” Scheduling conflicts with work duties and volunteer opportunities. The major challenge identified was the difficulty of balancing work duties and schedules with volunteer opportunities. In Texas, Lockheed Martin employees are encouraged to volunteer and can count a limited number of hours during work time. However, work duties must come first and often interfere with scheduled volunteer activities. This also means that volunteers do not want to help out in the evening or on the weekends.

“9 times out of 10, it’s been sorry, we [Lockheed Martin] can’t come. You know it’s been at the last minute. So I think that that’s one of the challenges.” (TX 4-H professional)

“He [Lockheed Martin representative] said, ‘You know we don’t just have our bosses to

deal with. . . if the military walks in and say okay this component’s not working . . . we’re going to have to drop what we’re doing, our commitment, and go and try to figure out how to get an airplane landed or something.” (TX 4-H professional)

“I still think Lockheed Martin should incentivize the volunteerism so that they really do participate because they have families, they have other, maybe they have other occupations. . . I know that they want to volunteer, but they just don’t feel that they have the time to do it.” (CA 4-H professional)


“The concept of having the volunteers from Lockheed Martin hasn’t really worked. We have some overlaps with them, and we had some already existing relationships, but to date they have not been able to accommodate the request that we made.” (MD 4-H professional)

PotentialInitiativeExemplars

Programs were selected as potential exemplars because of their progress in producing positive science and engineering outcomes, involving girls and minorities, realizing the 4-H career pathway, and engaging community volunteers. The potential exemplars are shown by the associated phase of the 4-H Career Pathway in Table 19. Three of these (State 4-H Mission Make-It in Georgia, Paso Robles 4-H STEM Club in California, and Cockeysville 4-H Robotics Club in Maryland) represent model maker movement programs (see Hill, Francis & Peterson, 2015). Table 19. Potential Initiative Exemplars along the 4-H Career Pathway

Explore Learn Practice Experience Adventures in

Science (Maryland)

State 4-H Mission Make-It (Georgia)

Paso Robles 4-H STEM Club (California)

Cockeysville 4-H Robotics Club (Maryland)

Video Conferences Connecting Youth with STEM Professionals (Texas)

Potential initiative exemplars are described below. AdventuresinScience(Maryland) Adventures in Science is a program of Maryland 4-H that was previously identified as one of the most promising 4-H science programs in the nation (Riley and Butler, 2012). The program provides short-term, hands-on experiences to help youth gain awareness of science and science careers. For the current initiative, the program successfully achieves the explore phase of the 4-H Career Pathway. State4‐HMissionMake‐It(Georgia) Georgia 4-H Mission Make-It is a statewide 4-H event where middle school students work together in teams to create solutions for science-related problems such as designing an environmentally friendly soda container. This non-competitive event has successfully involved girls and minorities from urban, suburban, and rural locations for the past two years. It relies heavily on teen leadership, further providing opportunities for teens to extend skills in both employability and science skills. Another aspect was youth science-related exhibits where youth displayed different topics and activities that they were working on or had learned about. The exhibits allowed the youth to share information with other 4-H youth. This program shows great potential for both explore and learn phases of the 4-H Career Pathway.


PasoRobles4‐HSTEMClub(California) The Paso Robles 4-H STEM Club meets in the Paso Robles Public Housing Authority weekly during the school year. The club is predominately Hispanic, and the majority of the members are female. The club is focused on the practice and experience levels of the 4-H Career Pathway. Purposeful volunteer recruitment and involvement has been key to the club’s success, and the volunteers represent both the Housing Authority and the California Polytechnic State University engineering faculty. In interviews, volunteers emphasized that the program would not have been possible without the Lockheed Martin funding. As an illustration, supplies for a single lesson, a previous National 4-H Youth Science Day experiment, for 40 youth cost $200 (more than $6,000 in supplies annually). Cockeysville4‐HRoboticsClub(Maryland) The Cockeysville 4-H Robotics Clubhas successfully engaged girls and minorities in the practice and experience levels of the 4-H Career Pathway. Youth build and program robots as part of the statewide 4-H robotics programming that culminates annually at the Maryland State Fair. The lead community volunteer is an engineering faculty member at Morgan State University, and the group meets weekly at the Baltimore County Extension Office. VideoConferencewithSTEMProfessionals(Texas) Texas conducts video conferences that link youth to science professionals. The professional contacts were made through Lockheed Martin and minority professional associations such as National Society of Black Engineers. The program is successful at the experience phase of the 4-H Career Pathway demonstrating that virtual mentoring is a best practice for connecting youth to real world settings.

Conclusions

The overall project goals were to reach 30,000 youth in STEM and career development programs with up to 60% representing girls and 50% representing involved. The goal was to support these youth with 500 Lockheed Martin employees and 1,000 4-H community volunteers in 13 states. The overall project goals are being exceeded:

47,751 youth reached in Years One and Two o 30,870 in Cohort One States o 16,881 in Cohort Two States o 56% girls o 50% minority

492 Lockheed Martin employees engaged o 333 in Cohort One States o 159 in Cohort Two States

1,791 4-H Community Volunteers engaged o 946 in Cohort One States o 845 in Cohort Two States

The Career Pathway shows great promise for empowering youth to achieve the attitudes, aspirations, and skills needed for successful STEM careers. Among the 195 youth completing


surveys, 92.4% of 4th-7th graders and 46% of 8th-12th graders reported they were in their first year of 4-H participation. This suggests that the initiative is successfully reaching new audiences who might not have otherwise been involved in 4-H. Key outcomes demonstrated that younger youth had gained positive attitudes and aspirations and developed decision making skills for school success. Older youth gained positive attitudes and aspirations, developed decision making skills college readiness, and could apply their learning to make a contribution through science. Youth surveys results showed positive results for:

Attitudes and Aspirations Decision Making and Problem-Solving, School Decision Making and Problem-Solving, College Readiness Apply Learning and Make a Contribution through Science

Purposeful outreach to under-served communities is essential. A best practice for engaging girls and minorities was to locate programs in under-served communities. Another best practice was to recruit female and minority role models for youth specifically with Lockheed Martin, Society of Women Engineers, and/or National Society of Black Engineers. Lockheed Martin employees were successfully engaged through informal networking at the local level. The formal, structured employee engagement approach, such as Community Relations Lead and Employee Affinity Groups, has not been successfully by any of the 13 states. States that have successfully engaged Lockheed Martin employees have involved the employees in local 4-H STEM programs. Also, Texas 4-H professionals have successfully linked Lockheed Martin employees to 4-H youth through video conferences. Major barriers to employee engagement were the scheduling conflicts between work duties and 4-H events. States that were unsuccessful in involving Lockheed Martin seemed to rely heavily on face-to-face volunteer opportunities, and they did not consistently offer initiative programs near the Lockheed Martin facilities. States encountered common challenges:

Professional Development – 4-H professionals reported several needs related to professional development. First, they perceived that their lack of cultural competencies is a barrier to involving more girls and minorities in 4-H STEM. Second, 4-H professionals feel that their STEM educational background is inadequate for today’s youth. Finally, they report a need to improve the training opportunities they offer to volunteers to focus on youth development and pedagogy.

STEM Skills and STEM Careers – The OST instrument identified high program quality in relationship building among youth and adults, youth participation, youth activities, and instructional strategies. Areas identified for improvement were in STEM youth skill development and STEM instructional practices. 4-H professionals described the need for more resources relative to STEM careers.

Career Pathway Development – States reported the need to further develop the 4-H Career Pathway and alignment with youth development and 4-H Science curriculum. A common question was related to what science activities “count” for this initiative. 4-H professionals also expressed difficulty at times understanding definitions and concepts distinguishing different phases of the Career Pathway.


Differences were found among states related to performance as measured against the stated benchmarks. Six states exceeded benchmarks for total youth reached and involvement of girls and minorities: California, Maryland, Texas, Florida, Georgia, and Kentucky. States that reported limited or no progress in these areas were: Arkansas, Colorado, New Jersey, and Washington. Also, Alabama showed no progress in reaching minority youth. Regarding engagement of Lockheed Martin employees, four states exceeded benchmarks: Arkansas, Kentucky, New York, and Washington. Five states showed no progress in employee engagement: Maryland, Alabama, Florida, Georgia, and New Jersey.

The initiative potential exemplars were identified. While these programs show great promise, continued effort through the practice and experience phases of the 4-H Career Pathway is needed.

RecommendationsandDiscussion

The following recommendations were prepared after carefully reviewing the project findings and conclusions. The recommendations were prepared to:

Delineate the next steps for the overall project to be more successful in Year Three. Assess needs for making the 4-H Career Pathway fully functional and ready for

replication throughout the 4-H system at the end of Year Three. Describe implications for adoption of the initiative’s best practices throughout the 4-H

system. National 4-H Council identified the need to strengthen outreach to girls, and with state partners, is piloting a special project in August 2017 in Maryland and Texas. The organization has also been responsive to the need for improved curriculum, particularly as it relates to science careers. The Click2Science curriculum is also in the pilot phase. The results of both pilots should be carefully monitored and if successful, it is recommended that programs be implemented by both Cohort One and Cohort Two states during the last half of Year Three. Other recommendations follow.

YearThreeProgrammingPrioritiesIf National 4-H Council provides Year Three funding for states that have not met Year Two benchmarks, it is recommended that those states be required to adopt the best practices identified over the first two years of this project. It is further recommended that the potential exemplars be shared with all states via video conferences. Potentially, these video conferences would help 4-H professionals appreciate how the programs are working toward excellence in the core components of the initiative: serving under-served youth audiences, involving Lockheed Martin, realizing the 4-H Career Pathway, and producing positive outcomes in science and engineering. In Year Three, the following recommendations and key questions are made to strengthen the potential exemplars and test the initiative for broader implementation:

All sub grantees should consider involving Lockheed Martin employees in professional and volunteer development as these were identified as major needs. The professional


relationships developed could potentially leverage direct employee engagement with youth.

Maryland – Adventures in Science has utilized Lockheed Martin employee engagement in the past (Riley & Butler, 2012). In Year Three, it is suggested that Lockheed Martin be specifically recruited to engage in the practice and experience levels.

o Key question: Two Lockheed Martin facilities (Greenbelt and Rockville) are one hour from Cockeysville. How could employees at those sites contribute to the Cockeysville Robotics Club and/or start clubs in their communities?

Georgia – In Year Three, it is suggested that Lockheed Martin be specifically recruited for professional and volunteer development. Observations, activity reports, and interviews with professionals, volunteers, and youth confirmed that STEM programming, particularly in exploring STEM concepts, was a strength of Georgia 4-H. However, 4-H professionals expressed significant needs for STEM professional and volunteer development.

o Key question: How could Lockheed Martin employees in Marietta, Georgia address 4-H professional development needs either face-to-face in Marietta or virtually?

Texas – In Year Three, it is suggested that the video conferences continue to link STEM professionals with 4-H youth.

o Key question: How could Lockheed Martin employee engagement at the Texas State Fair be leveraged for additional volunteer engagement, particularly in the practice and experience levels for internships, job shadowing, and career exploration?

California – In Year Three, it is suggested that programming be provided to help older members of the Paso Robles 4-H STEM Club to experience STEM skills in real work settings.

o Key question: How could the Lockheed Martin employees in Santa Cruz contribute to the Paso Robles 4-H STEM Club, either face-to-face or virtually?

ProfessionalandVolunteerDevelopmentThe following professional and volunteer development recommendations are not specific to Year Three, but speak to broader implementation throughout the 4-H system. It is recommended that National 4-H Council and the Cooperative Extension System collaborate to provide substantive professional and volunteer development in these areas:

Expanding programs to under-served audiences through enhanced cultural competencies Providing both basic and “level-up” science and engineering competencies Enhancing volunteer effectiveness though greater youth development competencies

Greater cultural competence is clearly a need given the number of states struggling to expand programming to underserved audiences. The states that have had the best overall performance in serving girls and minorities are the same states represented at the National 4-H STEM Futures Conference in 2016 which included interactive sessions on cultural competencies. Professionals have specifically mentioned the need for a basic review of science and engineering and the need to “level-up” their STEM skills which was the same finding described by Worker et al., 2017 in a comprehensive California 4-H Science needs assessment. This professional development should include the use of hands-on, easy-to-use science curricula. It has been suggested that


science curricula delivered through educational kits would strengthen programming. Additionally, it is recommended that professional development focus on resources in pedagogy and youth development practices (consistent with Worker et al., 2017). EmployeeEngagementIt is recommended that Lockheed Martin promote 4-H volunteer opportunities from within the organization. Specifically, employees mentioned the need to promote the initiative through internal corporate newsletters. Lockheed Martin employees are viewed as altruistic by Extension 4-H professionals once they met and conduct a successful programs. However, getting started is the hard part. State grantees should explore employee engagement through virtual opportunities to mediate difficulties related to travel, time, and work demands. This model has been successful in Texas and broader implementation is expected to yield greater engagement and more robust programming at the practice and experience phases of the 4-H Career Pathway. Realizingthe4‐HCareerPathwayIt is recommended that National 4-H Council convene a national working group to improve the 4-H Career Pathway. The current leadership team for the initiative at National 4-H Council (Bender, Golden, and Gyau-Moyer) and the current University of Tennessee evaluation team (Donaldson and Franck) should be joined by selected experts in career development, youth development, and 4-H professionals who have most successfully implemented this initiative. This group should create stronger definitions across the 4-H Career Pathway phases; align the 4-H Career Pathway with youth development research, 4-H career development, and 4-H Science activities and curricula; and establish a research agenda.

References Centers for Disease Control and Prevention (2017). Types of Evaluation. Available: https://www.cdc.gov/std/Program/pupestd/Types%20of%20Evaluation.pdf Civil Rights Act of 1964, Pub.L. 88-352, 78 Stat. 241 (1964). Fitzpatrick, J.L., Sanders, J.R. and Worthen, B.R. (2004). Program evaluation: Alternative approaches and practical guidelines (3rd ed.). Boston: Pearson Education, Inc. Pechman, E.M., Mielke, M.B., Russell, C.A., White, R.N., Cooc, N. (2008). Out-of-School time (OST) observation instrument: Report of the Validation Study. Washington, DC: Policy Studies Associates. Radhakrishna, R. & Bowen, C.F. (2010). Viewing Bennett’s Hierarchy from a Different Lens: Implications for Extension Program Evaluation. Journal of Extension (48), 6. Article 6TOT1. Available: https://www.joe.org/joe/2010december/tt1.php Riley, D. & Butler, A. (2012). Priming the pipeline: Lessons from promising 4-H science programs. Washington, DC: Policy Studies Associates, Inc.


Worker, S.M., Schmitt-McQuitty, L., Ambrose, A., Brian, K., Schoenfelder, E., Smith, M.H. (2017). Multiple-Methods Needs Assessment of California 4-H Science Education Programming. Journal of Extension (55), 2. Article 2RIB4. Available: https://www.joe.org/joe/2017april/rb4.php

Acknowledgements We are grateful to Betsy Brown, Adam Rosenberg, Allen O’Hara, Amy Gyau-Moyer, Sara Huber, Janet Golden, and Ed Bender. We would like to express our appreciation to National 4-H Council and Lockheed Martin for the opportunity to contribute to this project. Photo credits: National 4-H Council and University of Georgia.

ContactInformation Principal Investigator

Joseph L. Donaldson, Ph.D., Assistant Professor, Program Development and Evaluation Dept. of 4-H Youth Development, Agricultural Leadership, Education and Communications 2621 Morgan Circle 212-D Morgan Hall Knoxville, TN 37996-4525 Phone 865-974-7371; Fax: 865-974-1628; Email: [email protected]

Co-Principal Investigator

Karen L. Franck, Ph.D., Extension Assistant Professor Department of Family and Consumer Sciences 2621 Morgan Circle 119 Morgan Hall Knoxville, TN 37996-4525 Phone 865-974-8714; Email: [email protected]

Administrative Assistant

Rebekah Ownby, Administrative Assistant Dept. of 4-H Youth Development, Agricultural Leadership, Education and Communications 2621 Morgan Circle 320 Morgan Hall Knoxville, TN 37996-4525 Phone 865-974-7371; Fax: 865-974-1628; Email: [email protected]


Appendix 4‐HCommunityVolunteersEngagedPerState(YearTwo)

1,037 4-H community volunteers were engaged among the states. States reporting 4-H community volunteers ranked from highest number of volunteers to lowest:

1. Maryland 240 2. Texas 217 3. Kentucky 164 4. Pennsylvania 136 5. Alabama 69 6. Georgia 47 7. California 42 8. Florida 32 9. New Jersey 22 10. New York 22 11. Colorado 21 12. Washington 18 13. Arkansas 7

the university of tennessee – 1 · outlining 4-h youth activities and lockheed martin employee...

Documents