final needs assessment report
TRANSCRIPT
-
8/22/2019 FINAL Needs Assessment Report
1/33
Year ThreeNational 4-H Science E-Academy
Pre-Program Needs Assessment ReportSeptember 30, 2012
Mary E. Arnold, Ph.D.
Project Evaluator
Oregon State University
With assistance from
Courtney Archibeque, MpH
Graduate Research Assistant
Oregon State University
National 4-H ScienceLeadership Academy
-
8/22/2019 FINAL Needs Assessment Report
2/33
Table of Contents
Evaluators Statement .. iiAcknowledgements .. iii
Executive Summary .. iv
Overview .. 1
Respondent Demographics .. 3
Personal Interest in 4-H Science .. 3
Knowledge and Use of 4-H Science Tools ... 5
Increasing the Use of 4-H Science Tools ... 7
Content of the E-Academy ........ 9
E-Academy Format ..... 16
Time Commitment ..... 18
Pre and Post Academy Assignments ..... 18
Format of E-Academy Sessions ..... 18
Technological Capabilities ..... 20
Summary and Recommendations ... 21
Appendix ONE: Needs Assessment Respondent Demographics ..... 23
-
8/22/2019 FINAL Needs Assessment Report
3/33
ii
Evaluators Statement
This document serves as the final evaluation report for the National 4-H Science Academy: Year
Three E-Academy Needs Assessmentsponsored by National 4-H Council with funding from the
Noyce Foundation. The academies were held in five locations across four regions of the country
between January and April, 2012. All academy participants were invited to participate in theprogram evaluation.
All data for the evaluation were entered by participants directly into an on-line data collection
system. Access to the system was provided by the evaluator to the participants for data entry,
but only the evaluator and her research assistants had access to the actual dataset. The
integrity and accuracy of the raw data rests with the individual participants. The integrity and
accuracy of the analysis and interpretation rests solely with me as the project evaluator. To this
end, I certify that the analysis and results presented in this document are complete and
accurate insofar as the data entered by the participants were as well. Any questions or
concerns about this report should be addressed to me.
Mary E. Arnold, Ph.D.
Project Evaluator, Oregon State University
September 30, 2012
-
8/22/2019 FINAL Needs Assessment Report
4/33
iii
Acknowledgements
This needs assessment could not have taken place without the dedicated help and support of many
individuals.
First and foremost, I would like to thank National 4-H Council and the Noyce Foundation for theopportunity to conduct the Year Three Virtual Academy evaluation, which includes this formal needs
assessment. It continues to be exciting to witness the movement of the 4-H Science professional
development program from the national to the regional, and now to the local level, and to document
the important work that is taking place as a result of the Academy.
I would like to express my thanks and gratitude to Edward Bender, Janet Golden, Jo Turner, Beth
Bernstihl, and Maila Oliveria at National 4-H Council for their support and help with various aspects of
the needs assessment process.
Thank you, also, to the members of the Virtual Academy Leadership Team (VAULT) for your thoughtful
contributions to developing the needs assessment instrument.
Debbie Nistler, President of NA4-HA was very gracious in allowing us to send e-mail invitations through
their e-mail contacts to reach as many front line 4-H educators as possible. A special thank you to
Debbie and the NAE4-HA staff for making this happen and for sending out the invitations and reminders.
Similarly, I would like to extend my thanks and gratitude to all of the 4-H Science Liaisons across the
country who forwarded invitations to participate in the survey to their local 4-H contacts.
A very special thank you to my co-authors and graduate research assistant Ms. Courtney Archibeque.
Your focused assistance with the data analysis and report preparation could not be replaced. Thank you
especially for your cheerful willingness to concentrate your work time on the report so we could meet
the expected deadline.
I would like to thank each and every 4-H educator who contributed data. The sincerity with which you
approached the evaluation was evident in the data and information you provided. Your responses will
guide the work of the VALT as they plan the E-Academy based on the interests and needs you shared
through this process.
Finally, thank you to the Noyce Foundation for the generous support of the National 4-H Science
Leadership Academy. The funding provided by the foundation made this important program possible. As
a result, 4-H programs across the country are more prepared to develop and sustain programs for youth
in science, technology, engineering, and math.
-
8/22/2019 FINAL Needs Assessment Report
5/33
iv
Executive Summary
Respondent Demographics
We received 504 responses from 48 states, with the big 4-H science states showing the greatest
participation (e.g. Illinois, Kentucky, Minnesota, and Pennsylvania). The demographic data indicate that
the intended audience (i.e. frontline county agents and educators) was reached through the needs
assessment process.
Additional demographics show that
81% of respondents were agents or educators at the county level 92% were female 88% were NOT in a science specific role 9.7% attended the National Science Academy in 2010 32 % attended a regional academy in 2012
Personal interest in 4-H Science
Over 80% indicated a high or very high personal interest in professional developmentfor 4-H Science
Over 82% indicated a commitment to learn and develop skills for 4-H Scienceprogramming
Almost 80% indicated positive support in their work environment for implementing 4-HScience programs
Use of 4-H Science Professional Development Resources
Most respondents have NOT HEARD of the national tools available to support 4-HScience.
The exception is for specific training guides, such as Junk Drawer Robotics, and ThePower of Wind; 76.8% had HEARD about these resources
Even less have USED these resources, with only 56.5% reporting using specific trainingguides
70% of respondents reported that knowing more about the resources would increasethe likelihood that they would use them
Proposed Content of the E-Academy
The topics respondents were most interested in learning about are**:
Tools and resources available to support 4-H Science programs (Doing Science) Teaching others to inquiry-based science experiences (Doing Science)
-
8/22/2019 FINAL Needs Assessment Report
6/33
v
How to engage volunteers in 4-H Science as coaches or mentors (VolunteerDevelopment)
How to help traditional volunteers facilitate 4-H Science programs (VolunteerDevelopment)
How to effectively train volunteers in 4-H Science AND PYD principles (VolunteerDevelopment)
______________________________________________
** 75% or more are moderately to very interested in this topic
The topics respondents were next most interested in learning about are***:
Facilitating inquiry-based science experiences (Doing Science) Scientific engineering practices framework for K-12 Science (Doing Science) Facilitating science programs to limited resource and/or non-traditional audiences
(Doing Science)
How to recruit science-rich volunteers (Volunteer Development) How to include inquiry in all science programs (Curriculum) An overview of current 4-H Science curriculum resources (Curriculum) Strategies for engaging teens as teachers in 4-H Science programs(Curriculum)______________________________________________
*** 60% or more are moderately to very interested in this topic
E-Academy Format
Respondents were asked to indicate their preference for the format of the E-academy. This is
the only place where the needs assessment data differ in important ways when therespondents are separated out by group. For this analysis we separated out those who are
county educators from all other respondents, and in doing so, found an important difference.
Interactive e-learning sessions consisting of presentations with interactive componentsreceived the highest overall average rating (4.0/5.0)
Facilitated interactive learning: consisting of presentations viewed as a group at a localsite with a facilitator trained to conduct breakout sessions received the greatest number
of high ratings
BUT- if you consider only the responses from county-level educators the FacilitatedInteractive Learning is the most preferred (4.8/5.0 mean rating)
-
8/22/2019 FINAL Needs Assessment Report
7/33
vi
Time Commitment for an E-Academy Work
Respondents were asked to indicate the amount of time they are willing to commit to participating in
the E-Academy:
69.8% for ONE part day e-academy (up to four hours total) 61.5% for ONE full-day academy (5+ hours) 59.7% for an academy series over 3-5 months with part days of up to 4 hours 14.9% for three or more full consecutive days 26.8% for Three or more part consecutive days
Delivery of E-Academy Sessions
Respondents were asked their preference for the delivery of the E-Academy session. Only two
types of delivery received moderate positive ratings:
71.2% webinar presentations (both live and recorded) 64.9% video presentations
Technological Capacity
It is important to note that while 70% indicated they have attended webinarssuccessfully, this also means that almost 30% have not
In addition, only 58% indicated they have adequate technological support locally in theevent of problems with participation that are caused by local equipment
And one out of four people (25%) report they do not have an appropriate setting, suchas a quiet room for participating in the e-academy
Summary and Recommendations
The results of the needs assessment are, overall, promising for the success of the E-Academy. In
particular, the respondents seem interested in the proposed content of the E-Academy, and
indicate they are willing to commit time to attending the academy. There are a few important
things to note that should be taken into consideration by the E-Academy planners, however.
1. Respondents indicated the greatest interest in content related to science programimplementation (Doing Science) and Volunteer Development. Particular aspects of
Curriculum also were rated moderately high. There are several items that stand out
clearly as interest areas for this audience and should be kept in mind when the content
of the academy is planned.
2. The proposed content for the Evaluation and Fund Development areas received low tomoderate ratings. The planners should plan content in these areas with the audience in
-
8/22/2019 FINAL Needs Assessment Report
8/33
vii
mind and not plan a program that is not a match with the skills and needs of county-
based educators.
3. County educators indicated a very strong preference for the format of the E-Academy tobe a facilitated session. While creating this type of session is beyond the purview of the
E-Academy per se, some attention should be paid to providing information to states andcounties about how to plan for and host the E-Academy at the local level, including a
recommendation that learning be done in a facilitated session if possible.
4. Care should be taken to be sure the technology used for the delivery of the E-Academyis tested for potential problems that may occur when the educator participates in the
academy. The results of the needs assessment revealed just a fair to moderate
technological capacity of the educators. Especially noteworthy is that almost of
respondents reported having no on-site technical assistance to help them. Advanced
training, clear instructions for participation, and technical support will all be key to the
success of the E-Academy.
-
8/22/2019 FINAL Needs Assessment Report
9/33
1
Year Three: National 4-H Science E-Academy
Pre-Program Needs Assessment
Overview
The aim of year three of the National 4-H Science Academy is to provide a professionaldevelopment program for county-based 4-H educators. The program design for year three
follows a natural progression from the first academy, held in December, 2010 that reached
teams of four from LGUs, to last years regional academies that reached a broader audience, to
the E-Academy that is intended to reach as many front-line educators as possible. This
progression marks the overall development, scope, and reach of the National 4-H Science
Academy program.
A development team entitled the Virtual Academy Leadership Team (VALT) was formed in the
early summer of 2012, under the leadership of Edward Bender. Team members are:
Edward Bender, National 4-H Council Janet Golden, National 4-H Council Jim Kahler, NIFA Suzanne Le Menestrel, NIFA Doug Swanson, NIFA Misty Blue-Terry, North Carolina A & T University (1890s) Trudy Dunham, University of Minnesota (North Central) Debbie Fajans, University of Vermont (Northeast Heather Kent, University of Florida (South) Steven Worker, University of California, Davis (West) Mary Arnold, Oregon State University (Evaluator)
A face to face meeting of the VALT was held in July at the National 4-H Conference. At the
meeting the team was charged with the development and delivery of the E-Academy, and
began the initial planning phase. Part of the meeting was devoted to a review of the
evaluations of the Year One and Two academies, which informed some of the initial planning.
One of the end products of the July VALT meeting was the development of the instrument for
the pre-program needs assessment. Following the meeting the evaluator drafted the initial
instrument based on discussions and planning at the meeting, and worked via e-mail with VALT
members to refine and finalize the needs assessment instrument. The needs assessment
focused on the following distinct areas:
Respondent Demographics Personal interest in 4-H Science
-
8/22/2019 FINAL Needs Assessment Report
10/33
2
Current use of 4-H Science Professional Development Resources Proposed Content of the E-Academy
o 4-H Science Program Development and Deliveryo Volunteer Developmento Curriculum and Program Developmento Evaluationo Resource Development
Structure and Participation in the E-Academyo The Focus of the E-Academyo Time Commitment and E-Academy Worko Types of E-Academy Sessionso Technological Capacity
Respondents to the needs assessment were recruited through LGU Science Liaisons and
through direct e-mails from NAE4-HA to its members. E-mail invitations were sent at least two
times through each method, with some liaisons following up more than one time with areminder.
Data were collected on-line through a commercial survey company, and no problems were
reported with the data collection process. Data were collected between August 1 and August
20, 2012. Responses were downloaded from the on-line collector into an Excel file and finally
into SPSS for analysis.
-
8/22/2019 FINAL Needs Assessment Report
11/33
3
Respondent Demographics
In all, we received 504 responses from 48 states, with some of the bigger 4-H science states
showing the greatest participation (e.g. Illinois, Kentucky, Minnesota, Pennsylvania). Additional
demographics show that
81% of respondents were agents or educators at the county level 92% were female 88% were NOT in a science specific role 9.7% attended the National Science Academy in 2010 32 % attended a regional academy in 2012
The demographic data indicate that the intended audience (i.e. frontline county agents and
educators) was reached through the needs assessment process. A detailed analysis of the
respondent demographics can be found in Appendix One.
It is important to note that while the respondent demographics did match those of the
intended audience, the N of 504 is small relative to the estimated population size of over 3,000.
The demographic results, however, do indicate that the responses came from a fair
representation of educators across the country, with one exception: Because of the relatively
small N, and the percentage of the respondents who indicate a moderate to strong interest in
science, it may be safe to assume that these results represent those who are most interested in
promoting 4-H Science, and not the overall body of 4-H agents/educators in general.
Nonetheless, the E-Academy is being developed as a train-the-trainer model, thus the
development of the E-Academy should be geared toward those on the local level who show aninterest in 4-H Science, as they will most likely become the trainers of others at the local level.
To this end, the responses provided in the needs assessment appear to be valuable and
meaningful.
Personal Interest in 4-H Science
Over 80% indicated a high or very high personal interest in professional developmentfor 4-H Science
Over 82% indicated a commitment to learn and develop skills for 4-H Scienceprogramming
Almost 80% indicated positive support in their work environment for implementing 4-HScience programs
Table 1.0 represents respondents personal interest in 4-H Science. Participants rated desire
to learn and develop skills for developing 4-H Science programs as the area in which there is
-
8/22/2019 FINAL Needs Assessment Report
12/33
4
the most personal interest. The least interest is in level of positive support in your work
environment for implementing 4-H science programs. Figure 1.0 shows the percentage of
respondents who rated their interest in each item high or very high.
Table 1.0 Personal Interest in 4-H Science
Very Low Low High
Very
High
High and
Very
High Missing
Desire to learn and develop skills for
developing 4-H Science programs3
0.6%
48
9.5%
234
46.4%
193
38.3%
427
84.7%
26
5.2%
Commitment to learn and develop skills for
developing 4-H Science programs0
62
12.3%
229
45.4%
187
37.1%
416
82.5%
26
5.2%
Ability to implement what you learn in order
to enhance 4-H Science programming
2
0.4%
45
8.9%
279
55.4%
149
29.6%
428
85.0%
29
5.8%
Ability to use what you learn in order to
enhance 4-H Science programming0
30
6.0%
277
55.0%
167
33.1%
394
88.1%
30
6.0%
Level of positive support in your work
environment for implementing 4-H Science
programs
9
1.8%
68
13.5%
266
52.8%
134
26.6%
400
79.4%
27
5.4%
Figure 1.0
46.4% 45.4%
55.4% 55.0% 52.8%
38.3% 37.1% 29.6%33.1%
26.6%
0%
20%
40%
60%
80%
100%
Desire to learn
and develop skills
for developing 4-
H Science
programs
Commitment to
learn and develop
skills for
developing 4-H
Science programs
Ability to
implement what
you learn in order
to enhance 4-H
Science
programming
Ability to use
what you learn in
order to enhance
4-H Science
programming
Level of positive
support in your
work
environment for
implementing 4-H
Science programs
Very High
High
-
8/22/2019 FINAL Needs Assessment Report
13/33
5
Knowledge and Use of 4-H Science Tools
Most respondents have NOT HEARD of the national tools available to support 4-HScience.
The exception is for specific training guides, such as Junk Drawer Robotics, and ThePower of Wind; 76.8% had HEARD about these resources
Even less have USED these resources, with only 56.5% reporting using specific trainingguides
Table 2.0 presents the awareness of participants of various tools used for 4-H ScienceTable 2.1 shows those who have used them. The most heard of tool was the Specific
Project Training Guides (Junk Drawer Robotics, Power of Wind) and the least heard of
tool was the 4-H Science Smart Competency Training Guide. Respectively, these two
tools were also the most and the least used by respondents.
Table 2.0 Knowledge of 4-H Science Tools
Heard Of: Yes No Missing
4-H Science Smart Competency Training Guide 71
14.1%
391
77.6%
42
8.3%
Using inquiry-based learning to support 4-H science on-line course 156
31.0%
306
60.7%
42
8.3%
4-H Science checklist 177
35.1%
283
56.2%
44
8.7%
4-H Science 101 Training Guide 147
29.2%
314
62.3%
43
8.5%
4-H Science Competencies 237
47.0%
223
44.2%
44
8.7%Inquiry-based learning online learning modules 174
34.5%
286
56.7%
44
8.7%
Fund Development Toolkit 165
32.7%
295
58.5%
44
8.7%
4-H Science Logic Model 260
51.6%
202
40.1%
42
8.3%
Promising Practices 135
26.8%
324
64.3%
45
8.9%
Specific Project Training Guides (Junk Drawer Robotics, Power of Wind) 387
76.8%
77
15.3%
40
7.9%
Recruiting and Developing Volunteers 287
56.9%
173
34.3%
44
8.7%
Archived 4-H Science Academy Webinars 144
28.6%
316
62.7%
44
8.7%
Curriculum Rubrics 143
28.4%
312
61.9%
49
9.7%
Youth Engagement, Attitudes and Knowledge (YEAK) Survey 152
30.2%
306
60.7%
41
9.1%
-
8/22/2019 FINAL Needs Assessment Report
14/33
6
Table 2.1 Use of 4-H Science Tools
Used: Yes No Missing
4-H Science Smart Competency Training Guide 26
5.2%
429
85.1%
49
9.7%
Using Inquiry-Based Learning to Support 4-H Science on-line Course 60
11.9%
397
78.8%
47
9.3%
4-H Science Checklist 122
24.2%
335
66.5%
47
9.3%
4-H Science 101 Training Guide 89
17.7%
369
73.2%
46
9.1%
4-H Science Competencies 140
27.8%
318
63.1%
46
9.1%
Inquiry-Based Learning Online Learning Modules 65
12.9%
393
78.0%
46
9.1%
Fund Development Toolkit 37
7.3%
418
82.9%
49
9.7%
4-H Science Logic Model 139
27.6%
318
63.1%
47
9.3%
Promising Practices 68
13.5%
387
76.8%
49
9.7%
Specific Project Training Guides (Junk Drawer Robotics, Power of Wind) 285
56.5%
175
34.7%
44
8.7%
Recruiting and Developing Volunteers 153
30.4%
303
60.1%
48
9.5%
Archived 4-H Science Academy Webinars 51
10.1%
405
80.4%
48
9.5%
Curriculum Rubrics 56
11.1%
394
78.2%
54
10.7%
Youth Engagement, Attitudes and Knowledge (YEAK) Survey 72
14.3%
379
75.2%
53
10.5%
-
8/22/2019 FINAL Needs Assessment Report
15/33
7
Increasing Use of 4-H Science Tools
Table 3.0 displays ways participants would increase their use of the 4-H Science Tools. Most participants
(70.6%) reported that knowing more about them would increase their usage the most. Figure 3.0
presents the results in a graphic format.
Table 3.0
Increasing Usage of 4-H Science Tools
What would increase your likelihood of using tools?
Frequency
% Missing
Knowing more about them356
70.6%
148
29.4%
Easier access to the tools 212
42.1%
292
57.9%
Training on how to use the tools 285
56.5%
219
43.5%A need for them based on my clientele 173
34.3%
331
65.7%
Other Responses Include the Following:
Thirty (30) respondents mentioned that some aspect of time, i.e. more time in the day, moretime to learn about tools, more time to use the tools, and more time to implement them, is
needed in order for them to better use the tools.
Nine (9) respondents stated that they would use the tools more if they knew where to findthem and how to access them. Some complaints about accessing the tools include: the URLs
seem to change too frequently, searching/browsing is not effective in locating the tools, State
Liaisons not sharing any of the resources with County Liaisons, enrolling in webinars is
difficult, finding archived webinars is difficult, finding the tools is difficult when not all in one
spot.
Four (4) respondents mentioned needing more resources and funds for developingeducational kits, travel to trainings or for materials.
Three (3) respondents mentioned having more staff especially administrative staff to helpwith day to day tasks so more time can be spent in learning about and using the tools.
Two (2) respondents would like the focus of team meetings to be on the tools and trainingresources and to have clearer communication between state and county staff who have
already learned about using these tools.
One (1) respondent each reported that they would like: to see how others use/implementthese tools and resources, venues to use them (partnership opportunities, project days,
community events, etc.), and that more support is needed from supervisors to use science
programming.
-
8/22/2019 FINAL Needs Assessment Report
16/33
8
70.6%
42.1%
56.5%
34.3%
0%
20%
40%
60%
80%
100%
Knowing more about
them
Easier access to the tools Training on how to use
the tools
A need for them based on
my clientele
Figure 3.0
Increasing Usage of 4-H Science Tools
-
8/22/2019 FINAL Needs Assessment Report
17/33
9
85.1%
78.4% 78.4%72.2%
66.9%
0%
20%
40%
60%
80%
100%
Doing 4-H Science Volunteer
Development
Curriculum and
Program
Development
Evaluation Resource
Development
Content of E-Academy
Respondents were asked about to rate their interest in the proposed content of the E-
Academy, which is arranged around five content areas: 1) Doing 4-H Science; 2) Volunteer
Development; 3) Curriculum and Program Development; 4) Evaluation; and 5) Resource
Development. The first question asked if respondents would attend an e-learning session oneach topic. The percentages of those who replied Yes are presented in graphic form in Figure
4.0
Figure 4.0
Percentage of Respondents indicating they would attend an E-Learning Session on this Topic
When asked about specific topics of interest in each of the areas, the primary interest is in the
areas of Doing Science and Volunteer Development. A second tier of interest is in curriculum.
The topics respondents were most interested in learning about are**:
Tools and resources available to support 4-H Science programs (Doing Science) Teaching others to inquiry-based science experiences (Doing Science) How to engage volunteers in 4-H Science as coaches or mentors (Volunteer
Development)
How to help traditional volunteers facilitate 4-H Science programs (VolunteerDevelopment)
-
8/22/2019 FINAL Needs Assessment Report
18/33
10
How to effectively train volunteers in 4-H Science AND PYD principles (VolunteerDevelopment)
______________________________________________
** 75% or more are moderately to very interested in this topic
The topics respondents were next most interested in learning about are***:
Facilitating inquiry-based science experiences (Doing Science) Scientific engineering practices framework for K-12 Science (Doing Science) Facilitating science programs to limited resource and/or non-traditional audiences
(Doing Science)
How to recruit science-rich volunteers (Volunteer Development) How to include inquiry in all science programs (Curriculum) An overview of current 4-H Science curriculum resources (Curriculum) Strategies for engaging teens as teachers in 4-H Science programs(Curriculum)______________________________________________
*** 60% or more are moderately to very interested in this topic
Detailed results for each proposed topic are presented in Tables 4.0 - 4.4. Items that
respondents were 1) least interested in; 2) most interested in; and 3) had the highest combined
moderately and very interested are highlighted in each table. Also included are summaries of
the open-ended responses.
In most cases the same item was the one that received both the most very interested ratingsand the most combined moderately and very interested ratings, there were two
exceptions: 1) Learning about strategies for engaging teens as teachers in 4-H Science
programming emerged as the item with the most interest in the areas of curriculum and
program development; and 2) Building partnerships for 4-H Science rose to the top in the fund
development area, which is the only item that over 60% of respondents indicated interest.
-
8/22/2019 FINAL Needs Assessment Report
19/33
11
Table 4.0
Interest in Doing 4-H Science Topics
Not
interested
Somewhat
interested
Moderately
interested
Very
interested
Moderately
and Very
Interested MissingTools and resources available to
support 4-H Science programs5
1.0%
50
9.9%
142
28.2%
263
52.2%
405
80.4%44
8.7%
Why inquiry is the basis for 4-H
Science
44
8.7%
141
28.0%
161
31.9%
114
22.6%
275
54.5%
44
8.7%
Connection between science
inquiry and experiential learning
32
6.3%
114
22.6%
166
32.9%
148
29.4%
314
62.3%
44
8.7%
Scientific engineering practices
framework for K-12 science
17
3.4%
92
18.3%
171
33.9%
178
35.3%
349
69.2%
46
9.1%
Facilitating inquiry-based science
experiences
15
3.0%
77
15.3%
183
36.3%
185
36.7%
368
73.0%
44
8.7%
Facilitating science programs tolimited resource or non-traditional
audiences
13
2.6%
85
16.9%
155
30.8%
206
40.9%
36171.7%
45
8.9%
Teaching others to facilitate
inquiry-based science experiences
21
4.2%
68
13.5%
157
31.2%
212
42.1%
369
73.3%
45
9.1%
Asking high quality science
questions
19
3.8%
99
19.6%
179
35.5%
162
32.1%
341
67.6%
45
8.9%
Developing science argumentation
skills
45
8.9%
142
28.2%
149
29.6%
118
23.4%
267
53.0%
50
9.9%
Other:
Six (6) respondents mentioned that they have extensive training or knowledge in science and scienceeducation so more in-depth learning and moving past the basics is needed for them.
Three (3) respondents stated that they would like to see some sessions designed with volunteers inmind with specific science activities that can be taught to and used by volunteers.
Three (3) respondents mentioned that having more specific science resources such as curricula andactivities that can be used would be helpful.
One (1) respondent mentioned that having a thorough academy on the tools, practical applications andwhat has been successful would be helpful.
One (1) respondent mentioned that they would be interested in teaching most of these sessions. Other responses included: developing science tool kits, evaluation of science content, how animal
science can fit into the new 4-H Science/STEM push, informal vs. formal science environment, and
making the case for 4-H science.
-
8/22/2019 FINAL Needs Assessment Report
20/33
12
Table 4.1
Interest in Volunteer Development Topics
Not
interested
Somewhat
interested
Moderately
interested
Very
interested
Moderately
and Very
Interested MissingHow to recruit science-rich
volunteers
10
2.0%
80
15.9%
142
28.2%
223
44.2%
365
72.4%
49
9.7%
How to engage volunteers in 4-H
Science as coaches or mentors
11
2.2%
65
12.9%
150
29.8%
229
45.4%
379
75.2%
49
9.7%
How to help traditional volunteers
facilitate 4-H Science programs
8
1.6%
64
12.7%
131
26.0%
252
50.0%
383
76.0%
49
9.7%
Reframing the volunteer role to
match the needs of 4-H Science
programs
23
4.6%
93
18.5%
172
34.1%
167
33.1%
339
67.2%
49
9.7%
Implementing the new research-
based national 4-H campaign to
effectively identify, recruit and
retain 4-H volunteers
20
4.0%
89
17.7%
160
31.7%
182
36.1%
342
67.8%
53
10.5%
Strategies for recruiting corporate
volunteers
24
4.8%
95
18.8%
163
32.3%
172
34.1%
335
66.4%
50
9.9%
How to effectively train volunteers
in 4-H Science and PYD
7
1.4%
73
14.5%
149
29.6%
226
44.8%
375
74.4%
49
9.7%
Working with limited resource and
multi-cultural audiences in 4-H
science programs
19
3.8%
94
18.7%
142
28.2%
200
39.7%
342
67.9%
49
9.7%
Other:
Most respondents mentioned that they have had adequate training in volunteer development andrecruitment but need help with more specific issues such as finding volunteers in rural areas, specificsabout recruiting, training and supporting volunteers, real world help at the local level not theory,
One (1) respondent mentioned they need help with retention of volunteers and one other respondentreported that volunteers and parents complain that 4-H is becoming more like school.
-
8/22/2019 FINAL Needs Assessment Report
21/33
13
Table 4.2
Interest in Curriculum and Program Development Topics
Not
interested
Somewhat
interested
Moderately
interested
Very
interested
Moderately
and Very
Interested Missing
Using the 4-H science checklist
to determine science-ready
programs
23
4.6%
120
23.8%
162
32.1%
136
27.0%
298
59.1%
63
12.5%
Using the national 4-H
Curriculum rubrics to determine
high quality curriculum
33
6.5%
128
25.4%
157
31.2%
124
24.6%
281
55.8%
62
12.3%
Revising current curriculum to
include 4-H Science programs
31
6.2%
111
22.0%
141
28.0%
159
31.5%
300
59.5%
62
12.3%
How to include inquiry in all
science programs
24
4.8%
87
17.3%
165
32.7%
166
32.9%
331
65.6%
62
12.3%
Adapting or revising acurriculum to include science
inquiry
30
6.0%
91
18.1%
164
32.5%
152
30.2%
316
62.7%
67
13.3%
An overview of current 4-H
Science curriculum resources
18
3.6%
77
15.3%
146
29.0%
201
39.9%
347
68.9%
62
12.3%
How to participate in the NYSD 57
11.3%
115
22.8%
134
26.6%
135
26.8%
269
53.4%
63
12.5%
Strategies for creating youth-
adult partnerships in 4-H science
programs
16
3.2%
116
23.0%
164
32.5%
143
28.4%
307
60.9%
65
12.9%
Strategies for engaging teens as
teachers in 4-H Scienceprogramming
15
3.0%
77
15.3%
158
31.3%
191
37.9%
349
69.2%
63
12.5%
Designing 4-H Science programs
that integrate the mission
mandates
22
4.4%
116
23.0%
167
33.1%
136
27.0%
303
60.1%
63
12.5%
Other:
Five (5) respondents mentioned that they are less interested in sessions about things they already dosuch as the NYSD.
Two (2) respondents mentioned they need help in writing curricula and finding lesson plans.
-
8/22/2019 FINAL Needs Assessment Report
22/33
14
Table 4.3
Interest in Evaluation Topics
Not
interested
Somewhat
interested
Moderately
interested
Very
interested
Moderately
and Very
Interested MissingOverview of evaluation basics 52
10.3%
113
22.4%
148
29.4%
123
24.4%
271
53.8%
68
13.5%
Planning and conducting an
evaluation of 4-H Science
programs
33
6.5%
97
19.2%
146
29.0%
159
31.5%
305
60.5%
69
13.7%
Overview of existing evaluation
instruments and tools
24
4.8%
91
18.1%
147
29.2%
173
34.3%
320
63.5%
69
13.7%
Accessing the 4-H online
evaluation basics course
29
5.8%
102
20.2%
152
30.2%
152
30.2%
304
60.4%
69
13.7%
Overview of recent 4-H research
that documents the impact of 4-
H Science programs
38
7.5%
110
21.8%
152
30.2%
134
26.6%
286
56.8%
70
13.9%
Overview of current
informal/non-formal science
learning research and how it
documents the impact of 4-H
Science programs
38
7.5%
119
23.6%
152
30.2%
127
25.2%
279
55.4%
68
13.5%
Preparing effective 4-H Science
Program success stories
31
6.2%
96
19.0%
151
30.0%
159
31.5%
310
61.5%
67
13.3%
Using photos/images to
document program success
34
6.7%
85
16.9%
153
30.4%
163
32.3%
316
62.7%
69
13.7%
Other:
Four (4) respondents mentioned that they want new ideas on evaluation which can include:developing/using tools that allow agents to use record books and demonstrations as evaluation
methods, animal science and environmental science evaluation, new ideas and programs to evaluate
and what else is out there besides YEAK.
One (1) respondent mentioned that they are not interested in the nationally available evaluation toolssince their state does not use them.
One (1) respondent mentioned that they need help in conveying the program stories throughevaluation.
-
8/22/2019 FINAL Needs Assessment Report
23/33
15
Table 4.4
Interest in Resource Development Topics
Not
interested
Somewhat
interested
Moderately
interested
Very
interested
Moderately
and Very
Interested Missing
Mission to Market; The End of
Fund Raising
42
8.3%
125
24.8%
114
22.6%
140
27.8%
254
50.4%
83
16.5%
Implementing the 4-H Science
Fund Development Toolkit
39
7.7%
124
24.6%
136
27.0%
120
23.8%
256
50.8%
85
16.9%
The top 10 funding strategies for
4-H Science
32
6.3%
89
17.7%
138
27.4%
162
32.1%
300
59.5%
83
16.5%
Developing a 4-H Science
Advocacy card to use with
potential donors
59
11.7%
119
23.6%
126
25.0%
117
23.2%
243
48.2%
83
16.5%
Building partnerships for 4-H
Science
23
4.6%
91
18.1%
157
31.2%
151
30.0%
308
61.2%
82
16.3%
Using the NYSD to market your
4-H Science program
41
8.1%
101
20.0%
139
27.6%
139
27.6%
278
55.2%
84
16.7%
Volunteer and alumni funding
support
56
11.1%
130
25.8%
129
25.6%
104
20.6%
233
46.2%
85
16.9%
Examples of current successful
partnerships for 4-H Science
35
6.9%
115
22.8%
149
29.6%
121
24.0%
270
53.6%
84
16.7%
Developing an elevator speech
about your 4-H science program
to use with potential donors
56
11.1%
117
23.3%
115
22.8%
132
26.2%
247
49.0%
84
16.7%
Using 4-H Science impact
research to support your case
for potential funders
42
8.3%
126
25.0%
126
25.0%
125
24.8%
251
49.8%
85
16.9%
Creating a public valuestatement for your 4-H Science
program
38
7.5%
109
21.6%
138
27.4%
133
26.4%
271
53.8%
86
17.1%
Strategies for making a
successful ask of potential
donors
41
8.1%
124
24.6%
131
26.0%
121
24.0%
252
50.0%
87
17.3%
Other:
Two (2) respondents reported that training on fund development might be more helpful if it was tailored todifferent job positions so that each person knows their role in fund development based on their position. This
could help communication between co-workers regarding fund development.
Two (2) respondents stated that finding funding is an issue and would like more resources for finding funding thatarent competitive.
One (1) respondent would like to have these sessions available after the academy for references when they areneeded.
One (1) respondent would like more focus to be placed on other science topics such as animal and environmentalscience.
One (1) respondent would like help in convincing other extension staff that 4-H science is important and thatfundraising is needed.
One (1) respondent mentioned that they need more time to receive all of the training topics and implement them.
-
8/22/2019 FINAL Needs Assessment Report
24/33
16
E-Academy Format
Respondents were asked to indicate their preference for the format of the E-academy. This is
the only place where the needs assessment data differ in important ways when the
respondents are separated out by group. For this analysis we separated out those who arecounty educators from all other respondents, and in doing so, found an important difference.
Interactive e-learning sessions consisting of presentations with interactive componentsreceived the highest overall average rating (4.0/5.0)
Facilitated interactive learning: consisting of presentations viewed as a group at a localsite with a facilitator trained to conduct breakout sessions received the greatest number
of high ratings
BUT- if you consider only the responses from county-level educators the FacilitatedInteractive Learning is the most preferred (4.8/5.0 mean rating)
Table 5.0 represents the preference of all respondents for the format of the E-Academy.Table 5.1 shows the preferences for respondents who are county-based educators.
County educators indicated a strong preference for an facilitated interactive e-learning
session as the format of the academy. All participants reported that they would prefer
to participate via distance technologies.
-
8/22/2019 FINAL Needs Assessment Report
25/33
17
Table 5.0
Preference in Format of E-AcademyAll Respondents
Format TypesAverage
1=low agreement
5=high agreement
I would not
participate in this
format
AllResponses
On-line e-learning session: Presentation-based learning,
non-interactive presentations about each topic3.3
26
5.2%
Interactive e-learning session: Presentations with some
interactive component4.0
20
4.0%
Enhanced interactive e-learning: Interactive
presentations with required additional on-line
participation such as assignments, field trips, breakout
discussions
3.443
8.5%
Facilitated interactive learning: Presentations viewed as
a group at a local site with a facilitator trained to
conduct breakout sessions
3.929
5.8%
I would NOT participate if only offered via distance
technologies1.7
30
6.0%
Other:
Nine (9) respondents mentioned that they would prefer to have face to face training ratherthan the E-Academy but realized that logistics and costs are inhibitors. Also, E-learning has
advantages that are not available with a traditional conference format.
Two (2) respondents reported that the E-learning that blends live interaction is the bestformat.
One (1) respondent stated that they want information to be archived so that the informationcan be accessed when it is needed.
Table 5.0
Preference in Format of E-AcademyCounty Educators
CountyE
ducators
On-line e-learning session: Presentation-based learning,
non-interactive presentations about each topic3.3
24
5.9%
Interactive e-learning session: Presentations with some
interactive component3.9
18
4.4%
Enhanced interactive e-learning: Interactive
presentations with required additional on-line
participation such as assignments, field trips, breakoutdiscussions
3.434
8.3%
Facilitated interactive learning: Presentations viewed as
a group at a local site with a facilitator trained to
conduct breakout sessions
4.822
5.4%
I would NOT participate if only offered via distance
technologies1.8
27
6.6%
-
8/22/2019 FINAL Needs Assessment Report
26/33
18
Time Commitment
Respondents were asked to indicate the amount of time they are willing to commit to participating in
the E-Academy. The following percentages indicated a yes for the indicated time:
69.8% for ONE part day e-academy (up to four hours total) 61.5% for ONE full-day academy (5+ hours) 59.7% for an academy series over 3-5 months with part days of up to 4 hours 14.9% for three or more full consecutive days 26.8% for Three or more part consecutive days
Pre and Post Academy Assignments
Respondents were asked the likelihood that they would participate in pre and post academy
assignments. Figure 5.0 presents the percentage of respondents who indicated they were likely or
very likely to complete these assignments.
Figure 5.0
Likelihood of Completing Pre and Post Academy Assignments
Format of E-Academy Sessions
Respondents were asked to rate their interest in participating and presenting at the E-Academy.
The top two topics respondents were interested in participating in are: video (64.9%) and
webinar (71.2%) presentations. Participants have the least interest in participating in the
Pechakucha sessions (39.7%). All respondents reported that they would be the most interested
48.4%
54.8%
48.0%
59.1% 56.5%10.7%
21.4%
9.7%
12.7% 12.9%
0%
20%
40%
60%
80%
100%
Complete
assignments
BEFORE the e-
academy
Complete
assignments
during the e-
academy
Complete any
post-academy
assignments
Attend any post
academy e-
learning
opportunities
Complete online
learning modules
pre or post e-
academy
Very Likely
Likely
-
8/22/2019 FINAL Needs Assessment Report
27/33
19
in presenting lightning sessions.1 (22.8%) Respondents would be least interested in
presenting Pechakucha2
sessions (69.6%). Figure 6.0 presents the percentage of respondents
who indicated they were interested in a particular format. Figure 6.1 shows the percentage of
respondents who are interested in presenting a particular type of session at the E-Academy.
Figure 6.0
Percentage of Respondents Indicating Interest in Types of E-Learning Sessions
1A Lightning session is a short presentation given at a conference or similar forum that last only a few minutes;
several will usually be delivered in a single period by different speakers.2
Pechakucha is a simple presentation format where 20 images are shown, each for 20 seconds. The images
advance automatically and presenter talks along to the images.
47.243.7
59.964.9
59.7
71.2
0
10
20
30
40
50
60
70
80
90
100
On-line poster
presentations
Pechakucha
session
Lightning
sessions
Video
presentation
Voice-over
power point
presentation
(recorded)
Webinar
presentations
(live and
archived)
http://en.wikipedia.org/wiki/Presentationhttp://en.wikipedia.org/wiki/Academic_conferencehttp://en.wikipedia.org/wiki/Academic_conferencehttp://en.wikipedia.org/wiki/Presentation -
8/22/2019 FINAL Needs Assessment Report
28/33
20
Figure 6.1
Percentage of Respondents Indicating Interest in Presenting Types of E-Learning Sessions
Technological Capabilities
Finally, respondents were asked rate their technological capability to attend the e-academy successfully.
The most inhibitive piece of technology to the respondents attending an E-Academy is having a
functional voice over internetprotocol (VOIP) microphone system. (27.4%) Other inhibitive aspectsinclude having a webcam (25.0%) and having adequate technical support in the event of a
technological problem on your end. (26.6%).
It is important to note that while 70% indicated they have attended webinars successfully, thisalso means that almost 30% have not
In addition, only 58% indicated they have adequate technological support locally in the event ofproblems with participation that are caused by local equipment
And one out of four people (25%) report they do not have an appropriate setting, such as aquiet room for participating in the e-academy
Figure 7.0 presents the percentage of respondents who indicated they had the technological capability
to attend the e-academy. .
18.70%13.90%
22.80%
14.90%20.00% 18.50%
0%
10%
20%
30%
40%
50%
60%
70%80%
90%
100%
On-line poster
presentations
Pechakucha
session
Lightning
sessions
Video
presentation
Voice-over
power point
presentation
(recorded)
Webinar
presentations
(live and
archived)
-
8/22/2019 FINAL Needs Assessment Report
29/33
21
Figure 7.0
Percentage of Respondents Indicating Sufficient Capability to attend the E-Academy Successfully
Summary and Recommendations
The results of the needs assessment are, overall, promising for the success of the E-Academy. In
particular, the respondents seem interested in the proposed content of the E-Academy, and
indicate they are willing to commit time to attending the academy. There are a few importantthings to note that should be taken into consideration by the E-Academy planners, however.
1. Respondents indicated the greatest interest in content related to science programimplementation (Doing Science) and Volunteer Development. Particular aspects of
Curriculum also were rated moderately high. There are several items that stand out
clearly as interest areas for this audience and should be kept in mind when the content
of the academy is planned.
2. The proposed content for the Evaluation and Fund Development areas received low tomoderate ratings. The planners should plan content in these areas with the audience inmind and not plan a program that is not a match with the skills and needs of county-
based educators.
3. County educators indicated a very strong preference for the format of the E-Academy tobe a facilitated session. While creating this type of session is beyond the purview of the
E-Academy per se, some attention should be paid to providing information to states and
70.2
58.3
84.3
56.259.9
75.2
0
10
20
30
40
50
60
70
8090
100
Past webinar
participation
without technical
difficulties on
your end
Adequate tech
support in the
event of a tech
problem on your
end
Access to a
computer for
online viewing
A functional
voice over
internet protocol
(VOIP)
microphone
system
Webcam Appropriate
settings, such as
a quiet room
with no
interruptions
-
8/22/2019 FINAL Needs Assessment Report
30/33
22
counties about how to plan for and host the E-Academy at the local level, including a
recommendation that learning be done in a facilitated session if possible.
4. Care should be taken to be sure the technology used for the delivery of the E-Academyis tested for potential problems that may occur when the educator participates in the
academy. The results of the needs assessment revealed just a fair to moderatetechnological capacity of the educators. Especially noteworthy is that almost of
respondents reported having no on-site technical assistance to help them. Advanced
training, clear instructions for participation, and technical support will all be key to the
success of the E-Academy.
-
8/22/2019 FINAL Needs Assessment Report
31/33
23
Appendix One
Needs Assessment Respondent Demographics
Frequency Percentage
Gender MaleFemale
78390
15.5%92.5%
State
Alabama
Arizona
Arkansas
California
Colorado
Connecticut
Delaware
Florida
Georgia
HawaiiIdaho
Illinois
Indiana
Iowa
Kansas
Kentucky
Louisiana
Maine
Maryland
Massachusetts
Michigan
MinnesotaMississippi
Missouri
Montana
Nebraska
Nevada
New Hampshire
New Jersey
New Mexico
New York
North Carolina
North DakotaOhio
Oklahoma
Oregon
Pennsylvania
Rhode Island
South Carolina
2
2
5
14
6
2
1
16
16
24
35
11
5
25
32
9
8
8
4
9
365
22
3
8
7
1
8
1
16
25
324
11
12
26
1
6
0.4%
0.4%
1.0%
2.8%
1.2%
0.4%
0.2%
3.2%
3.2%
0.4%0.8%
6.9%
2.2%
1.0%
5.0%
6.3%
1.8%
1.6%
1.6%
0.8%
1.8%
7.1%1.0%
4.4%
0.6%
1.6%
1.4%
0.2%
1.6%
0.2%
3.2%
5.0%
0.6%4.8%
2.2%
2.4%
5.2%
0.2%
1.2%
-
8/22/2019 FINAL Needs Assessment Report
32/33
24
Tennessee
Texas
Utah
Vermont
Virginia
Washington
West Virginia
Wisconsin
Wyoming
Missing
7
16
10
7
5
9
4
11
1
3
1.4%
3.2%
2.0%
1.4%
1.0%
1.8%
0.8%
2.2%
0.2%
0.6%
Region
Northeast
North Central
South
West
1890s
Missing
88
180
139
78
10
9
17.5%
35.7%
27.6%
15.5%
2.0%
1.8%
Number of years
in 4-H
First year
2-5 years6-10 years
11-15 years
16-20 years
21+years
Missing
52
123112
79
47
88
3
10.3%
24.4%22.4%
15.8%
9.4%
17.6%
0.6%
State Science
Liaison
Yes
No
Missing
36
444
24
7.1%
88.1%
4.8%
Official Role
Volunteer
Partner
State 4-H Science LiaisonState 4-H Science Specialist
State 4-H Specialist (not science)
County 4-H Educator (agent role)
County 4-H Educator (non-agent role)
State 4-H Program Leader
State 4-H Foundation Staff
State 4-H Foundation Director
Missing
4
3
614
35
345
64
14
1
2
16
0.8%
0.6%
1.2%2.8%
6.9%
68.5%
12.7%
2.8%
0.2%
0.4%
3.2%
Science Specific
Role
Yes
No
Missing
53
443
8
10.5%
87.9%
1.6%
Period Hired,
Science Specific
Role Only
December 2009 or earlier
January to June 2010
July to December 2010
January to June 2011
July to December 2011
January to June 2012
41
7
2
10
11
11
48.8%
8.3%
2.4%
11.9%
13.1%
13.1%
-
8/22/2019 FINAL Needs Assessment Report
33/33
25
July to December 2012 2 2.4%
Period Hired,
Science Specific
Role Only
Pre December 2009 to December 2010
January 2011 to December 2012
50
34
59.5%
40.5%
Where is time
spent?
Direct Programming to youth
Teaching and preparing volunteers and partners
Teaching and preparing 4-H Staff
Developing curricula
Adapting current curricula
Creating science lessons and activities
Fund development
Evaluation
Developing programs
Developing partnerships
Missing
204
87
25
6
22
22
7
2
53
29
47
40.5%
17.3%
5.0%
1.2%
4.4%
4.4%
1.4%
0.4%
10.5%
5.8%
9.3%
Where SHOULD
time be spent?
Direct Programming to youth
Teaching and preparing volunteers and partners
Teaching and preparing 4-H StaffDeveloping curricula
Adapting current curricula
Creating science lessons and activities
Fund development
Evaluation
Developing programs
Developing partnerships
Missing
100
235
405
7
5
5
6
40
29
32
19.8%
46.6%
7.9%1.0%
1.4%
1.0%
1.0%
1.2%
7.9%
5.8%
6.3%
Attended National
Academy
Yes
No
Missing
49
431
24
9.7%
85.5%
4.8%
If yes, which track
did you attend?
Curriculum
Evaluation
Professional Development
Fund Development
8
18
14
6
17.4%
39.1%
30.4%
13.0%
If yes, what was
your role?
Attendee (non-presenter)
Presenter
Both
44
0
2
95.6%
0
4.4%
Attended
REGIONAL
Academy 2012
Yes
No
Missing
161
315
28
31.9%
62.5%
5.6%
If yes, what wasyour role?
Attendee (non-presenter)
Presenter
Both
115
1
44
71.9%
0.6%
27.5%