energizing career opportunities through local education ... · of the energizing career...
TRANSCRIPT
Energizing Career Opportunities through Local Education and Development (ECO LED)
Evaluation Final Report
California State University Consortium
Funding Provided by
Southern California Edison
Submitted by
Cobblestone Applied Research & Evaluation, Inc.
Rebecca M. Eddy, Ph.D., H. Todd Ruitman, M.A., & Nancy Hankel, M.A.
July 17, 2012
CSU ECO LED Program Final Evaluation Report 2 Cobblestone Applied Research & Evaluation, Inc.
Table of Contents
Table of Contents .......................................................................................................................................... 2
Executive Summary ....................................................................................................................................... 4
Introduction .................................................................................................................................................. 6
Energizing Career Opportunities through Local Education and Development Program .............................. 8
Evaluation of the ECO LED Program ............................................................................................................. 8
Evaluation Methods .................................................................................................................................. 9
Evaluation Instruments ............................................................................................................................. 9
Student Survey ........................................................................................................................................... 9
Faculty Written Reports ........................................................................................................................... 10
Community Partners Survey .................................................................................................................... 10
Key Evaluation Questions ....................................................................................................................... 11
Participating Sample ............................................................................................................................... 11
Service-Learning Course Descriptions ......................................................................................................... 12
Course 1: Aquatic Ecology (BIOL 172S) at Fresno State .......................................................................... 13
Course 2: Mine Reclamation (EES 250T) at Fresno State ........................................................................ 13
Course 3: Biogeography of Southern California (GEO 380) at CSU Dominguez Hills .............................. 14
Course 4: Water and Conflict in the West (UNIV 298) at CSU Channel Islands ...................................... 14
Evaluation Results ....................................................................................................................................... 15
Section 1: Education in STEM.................................................................................................................. 15
STEM Career Interest ............................................................................................................................... 16
Hands-on Learning ................................................................................................................................... 18
Education Summary ................................................................................................................................. 19
Section 2: Environment ........................................................................................................................... 19
Environment Summary ........................................................................................................................... 22
Section 3: Community Building ............................................................................................................... 22
Community Education ............................................................................................................................. 22
Student Community Engagement............................................................................................................ 23
Developing Community Partnerships ...................................................................................................... 25
Community Building Summary ............................................................................................................... 27
Successes and Challenges of the ECO LED Program ................................................................................... 27
Successes of the ECO LED Program ........................................................................................................ 27
CSU ECO LED Program Final Evaluation Report 3 Cobblestone Applied Research & Evaluation, Inc.
Challenges of the ECO LED Program ....................................................................................................... 28
Conclusions ............................................................................................................................................. 29
References .................................................................................................................................................. 31
Appendix A: Student Survey Factor Analysis .............................................................................................. 32
Appendix B: Community Partner Survey Items .......................................................................................... 33
Tables
Table 1. Key Evaluation Questions and Corresponding Data Sources ........................................................ 11
Table 2. Course Influence on Degree and Career Posttest Individual Item Means ..................................... 18
Table 3. Influence of Service Learning on Student Learning Items Posttest Mean Scores ......................... 19
Table 4. Summary of ECO LED Program Activities ...................................................................................... 20
Table 5. Challenges and Possible Solutions for the ECO LED Program ....................................................... 29
Figures Figure 1. Student Posttest Mean Ratings of Interest in Pursuing a STEM Career ....................................... 17
Figure 2. Student Posttest Mean Ratings of Community Engagement ...................................................... 24
Figure 3. Posttest Student Mean Rating of Course Influence on Community Engagement........................ 25
Figure 4. Community Partners Satisfaction, Frequency of Survey Responses ........................................... 25
CSU ECO LED Program Final Evaluation Report 4 Cobblestone Applied Research & Evaluation, Inc.
Executive Summary
Cobblestone Applied Research & Evaluation, Inc. was hired to evaluate the effectiveness of the Energizing Career Opportunities through Local Education and Development (ECO LED) Program. The evaluation was designed to capture anecdotal evidence on how service-learning STEM courses in the California State University (CSU) system might create unique opportunities for students, faculty, and their local communities. Data collected in the evaluation provides some evidence of the potential for impact on education, environment, and community building through service-learning courses. Evaluation Question 1: How were ECO LED Program activities implemented? A total of four service-learning courses were part of the ECO LED Program at three southern California CSU campuses. These courses included the Mine Reclamation and Aquatic Ecology courses at Fresno State, Biogeography of Southern California at CSU Dominguez Hills, and Water Conflict in the West at CSU Channel Islands. These courses included service-learning activities that reinforced knowledge obtained in the classroom as well as benefited the community. For example, students at Fresno State developed an erosion control plan report after studying the abandoned Tip Top Mine in San Luis Obispo County. Other projects included a community garden, outreach and education for middle school students, and restoration and protection of an endangered species habitat. Evaluation Question 2: What was the effect of the ECO LED Program on students’ education in STEM? An analysis of student survey data revealed two main themes that emerged with regard to students’ experiences in the service-learning course related to their education in STEM; specifically, STEM career interest and the possible impact of hands-on learning. Results indicated that the service-learning activities provided students with a positive, hands-on learning experience that, according to students, influenced their degree and career goals. Evaluation Question 3: What was the effect of the ECO LED Program on the environment? Results showed that the four courses participated in fifteen separate activities that made a positive impact on the environment. Some examples of activities include: students monitored, analyzed, and reported on water quality to the community partner leading to identified areas of concern where future efforts might be focused; students monitored habitat for a river salmon restoration project; and students conducted an erosion control project on the campus through the installation of erosion control matting, wattles, and silt fencing. Evaluation Question 4: What was the effect of the ECO LED Program on promoting community building?
We investigated how the efforts of faculty, students, and community partners combined to address community needs and create opportunities for the future. The analysis revealed that the ECO LED Program had an effect on: community education, student community engagement, and the development of community partnerships. Specifically, multiple
CSU ECO LED Program Final Evaluation Report 5 Cobblestone Applied Research & Evaluation, Inc.
opportunities were found to educate the community regarding course content. Also, students, already willing to help the community, were given the information and skills needed to engage in community service on their own. Lastly, the community partners reported their satisfaction of working with the faculty and students. All community partners indicated that they would be willing to partner with faculty and students for a future service-learning course. Evaluation Question 5: What were the major successes and challenges of the ECO LED Program? Success of ECO LED
It was clear that faculty, community partners, and most students felt that the ECO LED Program was a positive experience. Beyond the programs’ impacts to education, environment, and community, another success was that students’ showed interest in pursuing careers that were directly related to the service-learning projects. These courses demonstrated that service learning can be used in higher education to provide students with experiences that both reinforce classroom learning and theory and provide students with opportunities to utilize what they have learned in possible workplace environments. Challenges of ECO LED The challenges that were encountered in the courses were logistical in nature, and most could be avoided in future course offerings with adequate planning and coordination. Lessons from these challenges can be used to aid in the creation of new service-learning courses or adding the service-learning component to existing courses. Conclusions Faculty, students, and community partners were able to accomplish much during the scope of the four service-learning courses that comprised the ECO LED Program in spring 2012. The positive effects of these courses on students’ education, the environment, and community building were evident given the analysis of student surveys, faculty reports, photographic and video evidence, and community partners’ surveys. Additional time and study is needed to better understand how service-learning courses such as these have longer-terms impacts such as increasing interest in STEM disciplines, graduation in STEM degrees, and desire for STEM careers.
CSU ECO LED Program Final Evaluation Report 6 Cobblestone Applied Research & Evaluation, Inc.
Introduction
Academic, public, and private institutions have a vested interest in promoting the
development of students’ knowledge and skills that are necessary in the current global
economy. Often referred to as 21st century skills, these include a combination of academic or
core content areas skills; learning and innovation; information, media, and technology; and life
and career skills (Partnership for 21st Century Skills). Topics such as global awareness,
environmental literacy, flexibility and adaptability, leadership, and responsibility have joined
core content areas such as language arts, mathematics, and science as key learning outcomes
for all students. In addition, “students must also learn the essential skills for success in today’s
world, such as critical thinking, problem solving, communication, and collaboration.”
(Partnership for 21st Century Skills).
One critical area in the development of these core skills relies on educating students
about the importance of conservation and the environment as a way of promoting the health
and life of future generations. Research suggests that high levels of environmental knowledge
(i.e., knowledge of current environmental issues) predict students’ concern, attitudes, and
feelings of personal responsibility toward environmental protection (Teksoz, Sahin, & Tekkaya-
Oztekin, 2012). In other words, environmental education helps to promote - “environmental
literacy” – awareness, sensitivity, knowledge, concern, responsibility, and action related to
environmental conservation (Roth, 1992). Providing the opportunity for young people to both
participate in and become educated about environmental needs can serve as an effective
platform for building these types of 21st century skills. Further, providing opportunities for
them to interact with their communities may help to decrease civic disengagement that is
prevalent.
One strategy that can address the problem of civic disengagement is community
engagement or community-based education. Community engagement refers to the
“collaboration between educational institutions and their larger communities for the mutually
beneficial exchange of knowledge and resources in a context of partnership and reciprocity”1
(Carnegie Foundation for the Advancement of Teaching and Learning). Community engagement
1 http://www.calstate.edu/cce/about_us/vision.shtml
CSU ECO LED Program Final Evaluation Report 7 Cobblestone Applied Research & Evaluation, Inc.
includes a number of activities such as service learning, voter registration drives, community-
based research and community service.
Service learning is one type of community engagement activity that specifically focuses
on civic involvement, active learning, and application of knowledge. Service learning is an
educational methodology that combines community service with classroom learning to engage
students in the educational process, teach civic responsibility, and strengthen communities
(Bringle & Hatcher, 1995). Service learning helps students gain socially responsive knowledge,
that is, education obtained through direct academic-based problem solving of social issues
(Altman, 1996). Students participating in service learning apply the skills and knowledge they
learn in their academic coursework to identify and solve real-world community problems and
also access the expertise of community partners in addressing these problems. Additionally,
students become contributing citizens and active community members through the service they
perform.
Research indicates that service learning has an overall positive impact on students’
social, personal, and cognitive outcomes (Giles & Eyler, 1994). Students who participated in
service-learning education had greater gains in perspective-taking, complex problem solving,
and critical thinking than students who did not participate in service-learning education
(Batchelder & Root, 1994; Markus, Howard, & King, 1993). Additionally, service learning
increases students’ awareness of contemporary social issues (Driscoll, Holland, Gelmon &
Kerrigan, 1996), enhances students’ self-efficacy relating to community service (Reeb,
Katsuyama, Sammon, & Yoder, 1998), and increases positive attitudes about civic engagement
and social responsibility (Markus et al., 1993).
Given the need to inform the next generation of critical issues in the environment and
conservation, service learning can serve as a functional strategy to promote active learning and
advocacy while providing students with a rewarding educational opportunity. This opportunity
can have reciprocal, positive impacts on students and their local community. Service-learning
experiences are often developed through a collaboration of community organizations and
higher education institutions through academic departments or programs. The Energizing
Career Opportunities through Local Education and Development (ECO LED) Program was
CSU ECO LED Program Final Evaluation Report 8 Cobblestone Applied Research & Evaluation, Inc.
developed to address educational needs of southern California students to promote 21st
century skills while serving the local community related to specific environmental issues.
Energizing Career Opportunities through Local Education and Development Program
The California State University (CSU) system is the largest system of higher education in
the United States and recognizes the responsibility to provide its students with knowledge and
opportunity to fully participate in civil society. For more than a decade, the CSU system has
been committed to increasing opportunities for student community engagement, and has been
expanding partnerships to do so. Southern California Edison (SCE) is committed to supporting
the development and education of the workforce for the 21st century as well. In partnership
with the CSU, SCE is committed to ensuring that students in southern California gain both
academic and technical skills to succeed in the workforce while supporting efforts to sustain a
quality environment. Therefore, SCE has provided financial support for CSU’s ECO LED Program.
All eight CSU campuses based in southern California were eligible to submit proposals for
serving-learning courses during spring 2012. Courses were focused on the disciplines of science,
technology, engineering, and mathematics (STEM) with a particular emphasis on environmental
support. Faculty formally applied to participate in the program and received funding sub-grants
to design and implement a service-learning STEM course in one of four priority areas. This
report summarizes all ECO LED Program activities and describes the evaluation of the ECO LED
Program.
Evaluation of the ECO LED Program
Cobblestone Applied Research & Evaluation, Inc. (Cobblestone) was hired to evaluate
the effectiveness of the ECO LED Program and to determine the effectiveness of program
activities on students, faculty and the community. The small number of courses and relatively
few faculty and students involved in the overall program was considered in designing the
evaluation. A rigorous, experimental study was not possible given the limited participation in
the program; however, the evaluation was designed to capture anecdotal evidence on how
service-learning STEM courses in the CSU system might create unique opportunities for
students, faculty, and their local communities in the future. The ultimate goal of these courses
CSU ECO LED Program Final Evaluation Report 9 Cobblestone Applied Research & Evaluation, Inc.
in the CSU is to increase the number of CSU students interested in, and successfully graduating
from STEM disciplines thus developing and educating the workforce of the 21st century through
service learning. While the current evaluation was not intended to address this longer-term
vision, data collected in the evaluation provides some evidence of the potential for impact on
education, environment, and community building through service-learning courses.
Evaluation Methods
The evaluation relied on data collection activities such as administration of student
pretest and posttest surveys and community partner surveys. Course faculty also provided
reports that included written descriptions of course activities, successes, and challenges of each
project. In addition, photographic evidence and videos were provided as evidence of program
implementation and the effect on the environment. The evaluation team at Cobblestone
designed the evaluation to facilitate minimally intrusive data collection, which relied heavily on
course faculty to facilitate.
Evaluation Instruments
Evaluators from Cobblestone designed the instruments such as the surveys and faculty
reporting protocols in cooperation with the Project Director of the ECO LED Program at the CSU
Center for Community Engagement. The following section describes these evaluation
instruments in greater detail, with references to appendices, where appropriate.
Student Survey
Student surveys were designed to capture participating students’ attitudes and behavior
on topics such as community engagement, interest in STEM, and the like. Students were
administered a pretest survey in class by the faculty member at the beginning of their
respective courses. Survey items had a response set of a 5 point scale that ranged from 1 =
Strong Disagree to 5 = Strongly Agree and also included a Not Applicable or Don’t Know option.
The pretest survey contained 11 survey items to measure students’ community engagement (6
items) and interest in pursuing a STEM career (5 items). At the end of the course, students were
asked to complete the posttest survey containing the original 11 items in addition to questions
related to their experience in the course. Additional posttest items inquired about topics such
as specific attitudes and behaviors around service learning, how the course influenced their
CSU ECO LED Program Final Evaluation Report 10 Cobblestone Applied Research & Evaluation, Inc.
community involvement, and influence of the course on reaffirming or clarifying their future
major and/or career. Items for the student survey were modified from Gelmon, et al. (2001)
and additional items were created specifically to evaluate the impact and success of the ECO
LED Program.
Only responses from three of the four courses were able to be matched between the
student pretest and posttest data.2 In addition, overall descriptive data were obtained by
calculating means on the entire sample of non-matched pretest and posttest data. To analyze
these data, a factor analysis was conducted to test items for reliability and develop scales for
future studies (Fabrigar, Wegener, MacCallum & Strahan, 1999). The results of the factor
analysis indicate good, reliable survey items that can serve as the basis for future student
surveys in service-learning courses. In total, four factors were obtained from the student survey
data, including the following categories: Community Engagement; Interest in STEM Careers;
Course Influence on Community Engagement; and Course Influence on Degree and Career
Selection. Appendix A contains the results of the reliability analyses of the survey factors.
Faculty Written Reports
As part of the ECO LED Program sub-grants to faculty, recipients were required to
submit a final report describing the service-learning course activities. The guidelines for the
final report included a description of the course, identification of community partners, and
implementation of service-learning projects. In addition, faculty members were asked to
provide a written reflection of the successes, challenges, and impacts on the community and
students’ learning. Finally, the faculty members provided digital photos and video that
documented the interaction of their projects with their local community and participating
students.
Community Partners Survey
An online survey was created to ascertain feedback from community partners involved
with the ECO LED Program. Course faculty were asked to email the survey link to participating
community partners and responses were captured and analyzed by the evaluation team. The
survey measured the general satisfaction of the community partners in working with students
2 One course required students to complete the posttest survey online. While individual student responses were
obtained, matching with pretest scores was not possible.
CSU ECO LED Program Final Evaluation Report 11 Cobblestone Applied Research & Evaluation, Inc.
and the faculty, the influence of the partnership on the capacity of the community partners to
fulfill their organizations’ mission, any challenges that were experienced, any economic effects
that extended above and beyond what the organization would have normally done, and any
negative impacts to the organization. In total, four community partners responded to the
survey requests and hence these data were considered a limited view of community partners’
experiences. The community partners’ survey items can be found in Appendix B.
Key Evaluation Questions
The purpose of the evaluation was to answer key evaluation questions developed by the
ECO LED Program and the evaluation team to understand the effect of service-learning courses
on students, faculty and the community. Evaluation questions were related to each major
program area (i.e., Education, Environment and Community Building). Key evaluation questions
are listed in Table 1. Data sources that addressed each question are also listed in the table.
Table 1. Key Evaluation Questions and Corresponding Data Sources
Evaluation Question Data Sources
1. How were ECO LED Program activities implemented?
Faculty written reports Photographs Video
2. What was the effect of the ECO LED Program on students’ education in STEM?
Student surveys Faculty written reports
3. What was the effect of the ECO LED Program on the environment?
Faculty written reports Community partner surveys
4. What was the effect of the ECO LED Program on promoting community building?
Faculty written reports Student surveys Community partner surveys
5. What were the major successes and challenges of the ECO LED Program?
Faculty written reports Student surveys Community partner surveys
Participating Sample
A total of four service-learning courses were part of the ECO LED Program at three
southern California CSU campuses. Five faculty members (two co-taught one course) led the
courses. While only four separate community partners (from two courses) responded to the
online survey, a total of 15 community partners participated in one of the four courses. A total
of 54 students completed posttest surveys, while 35 of these surveys were matched with
pretest responses. A number of K-12 students and their teachers also participated in aspects of
CSU ECO LED Program Final Evaluation Report 12 Cobblestone Applied Research & Evaluation, Inc.
the program; however, no data were collected from these individuals as part of the ECO LED
Program evaluation.
Service-Learning Course Descriptions
The ECO LED Program was developed to support southern California CSU campuses in
service-learning courses. Courses were designed to promote SCE’s priority in “sustaining a
quality environment” in one of four areas:
Protect and restore the habitats of endangered and threatened plants and animals;
Provide environmental education to various communities, especially to youth and
families;
Address water quality and storm water management in ways that use electricity wisely;
and/or
Renewable energies.
In all, the funding provided support for four separate service-learning courses delivered
at three CSU campuses (Fresno, Dominguez Hills, and Channel Islands) in spring 2012. While the
original intent was to support eight separate service-learning courses throughout the CSU
system, the planning time required to develop courses conflicted with the funding award
timeline and therefore only four of eight possible projects were supported by program funds.
The following section provides more detailed information about the courses that were
supported in the ECO LED Program.
To answer the first evaluation question, the next section includes a description of the
ECO LED-supported service-learning courses from spring 2012. The majority of course
descriptions were generated from applications to the ECO LED Program as well as faculty final
reports, photographic evidence and videos.
Evaluation Question 1: How were ECO LED Program activities implemented?
CSU ECO LED Program Final Evaluation Report 13 Cobblestone Applied Research & Evaluation, Inc.
Course 1: Aquatic Ecology (BIOL 172S) at Fresno State
Aquatic Ecology at Fresno State was offered in
spring 2012 and taken primarily by graduating seniors.
The course focused on physical and chemical features
of inland waters as well as other elements such as
community structure and function, ecological
interactions, adaptations, and identification of aquatic
organisms. The course was also intended to foster
students’ career development in aquatic ecology.
Students worked with multiple community partners in
the course and therefore had a variety of projects
as part of their experience. For example, one group
worked with a community partner, Friends of Lost
Lake Park, to develop inventories of flora and fauna
in the park. Another group worked on Cottonwood
Creek with the San Joaquin River Stewardship
Program to discover how cattle-grazing affects the
quality of downstream recreational waters. Through the course, service-learning projects were
documented with videos and a Facebook page was established to highlight the projects.
Course 2: Mine Reclamation (EES 250T) at Fresno State
The Mine Reclamation course at Fresno
State was developed to educate students about
abandoned mines in the region and to
separately investigate erosion control
methods—a service-learning project that was
completed on the Fresno State campus. Course
activities included the installation of erosion
control matting, wattles and silt fencing in a
storm water retention basin on campus that was
CSU ECO LED Program Final Evaluation Report 14 Cobblestone Applied Research & Evaluation, Inc.
subject to erosion. This project provided the added ecological and water quality benefit of
minimizing suspended sediment through work on the storm drain. A second course component
was working with the community partner, the US Forest Service, to develop an erosion control
plan report after studying the abandoned Tip Top Mine in San Luis Obispo County. This course
was the first service-learning based course in the Earth & Environmental Science Department
on campus and included participation of twelve graduate students.
Course 3: Biogeography of Southern California (GEO 380) at CSU Dominguez Hills
The course offered at CSU Dominguez Hills,
Biogeography of Southern California, was designed
to educate students about regional Mediterranean
ecosystems. The Earth Science club on campus
designed a service-learning project to build a native
plant garden on the Dominguez Hills campus. Also,
students in the course assisted in restoring a coastal
sage scrub habitat with their community partner,
the Palos Verdes Land Conservancy. In addition to classroom components that included
research and presentations, the project also focused on community education through
participation in Earth Day events to provide information on topics such as water use,
sustainable gardening, native plants and insects through information signs, handouts and
conversations. The garden established through the course will continue as a permanent part of
the campus and additional projects with the community partner are already underway,
including efforts to assist the endangered Blue Butterfly at the Chandler Preserve.
Course 4: Water and Conflict in the West (UNIV 298) at CSU Channel Islands
Water Conflict in the West was offered as an interdisciplinary course at CSU Channel
Islands to provide students with a research-based and community-based experience related to
water restoration, watershed management and environmental education. Undergraduate
students mentored local middle school students in a variety of activities including data
collection, analysis and interpretation of water quality data related to pollution as well as the
facilitation of a water symposium for these students. Service-learning course students
CSU ECO LED Program Final Evaluation Report 15 Cobblestone Applied Research & Evaluation, Inc.
monitored water quality in the Callegues Creek watershed and worked with the community
partner Ventura Coastkeeper. Undergraduate students also learned key science concepts
related to water quality monitoring protocols. The course focused on critical thinking in real
world problems where field lessons for middle school students were reinforced by classroom
activities such as:
Building water impoundments on watershed models
Taste testing water from various sources
Water jeopardy (quiz show format)
Role playing to resolve water conflicts
In addition to these course activities, students also took a three day tour of the Colorado
River Aqueduct. Students in the course were also engaged in stream restoration by replanting
native habitat near the campus to improve nesting conditions for an endangered bird species.
Evaluation Results
The following section addresses the influence of ECO LED on education, environment,
and community building. The information used to report on the evaluation results was obtained
through student surveys, faculty written reports, and community partner surveys. Each of these
data sources were used to answer the remaining key evaluation questions.
Section 1: Education in STEM
An analysis of student survey data revealed two main themes that emerged related to
students’ experiences in the service-learning course related to their education in STEM:
specifically, STEM career interest and the impact of hands-on learning. These themes will be
discussed in further detail in the following section in which we describe how the ECO LED
Program influenced students’ learning and the potential for this learning to impact their future
education and career choices.
Evaluation Question 2: What was the effect of the ECO LED Program on students’
education in STEM?
CSU ECO LED Program Final Evaluation Report 16 Cobblestone Applied Research & Evaluation, Inc.
STEM Career Interest
Students’ STEM careers aspiration and influence was specifically used to measure the
success of the ECO LED Program given the long-term goal of increasing the number of CSU
students interested in, and successfully graduating from STEM disciplines. Of course, due to the
limited scope and timing of the program, this goal was not fully measured; however, many of
the evaluation results indicated that continued and increased use of service-learning courses
may lead to the achievement of this goal in the CSU system. Our findings indicate that students
who participate in these service-learning STEM courses are likely to pursue STEM degrees and
thus pursue STEM careers.
Interest in pursuing a career in the STEM disciplines was captured by two composite
items developed in the analysis of student survey results. The two composite factors include a)
Interest in STEM Careers; and b) Course Influence on Degree and Career (posttest only).
Students were administered a pretest and posttest survey containing several items measuring
their attitudes towards STEM careers (see Appendix A). The pretest-posttest survey method
was utilized by the evaluation team to measure if students’ attitudes changed over the course
of the service-learning experience. As a reminder, students responded on a scale of 1 = Strongly
Disagree to 5 = Strongly Agree, indicating that higher scores are generally more positive or have
higher agreement. Negative items were reverse-coded to correspond to the same scale.
Survey results indicated that for the first composite, Interest in STEM Careers, students
entered the course already highly interested in a STEM career (M = 4.1, SD = .87, n = 64) so
there was no overall growth from pretest to posttest (t(32) = -.282, ns). Considering students
who are initially interested in course topics such as these, this finding is not surprising. That is,
these courses were created for upper-level undergraduate students that were primarily STEM
majors with an established strong interest in STEM careers.
One group that might have demonstrated growth during the course, unfortunately, was
the one course for which matched pretest-posttest data were not available. For these students,
we were unable to measure the change in attitude scores from the beginning to the end of the
course. This course was a graduate-level course that included primarily political-science
students (non-STEM majors). However, given the available data, the pretest survey showed that
CSU ECO LED Program Final Evaluation Report 17 Cobblestone Applied Research & Evaluation, Inc.
the graduate students were neutral in the interest for a STEM career at pretest (M = 3.2, SD =
.66, n = 15) but made slight gains at the posttest survey (M = 3.4, SD = .68, n = 12) regarding
STEM career interest. Although this growth was minimal, with larger samples and matched data
it might have been possible to observe attitude changes over the course of the program. Figure
1 shows posttest mean ratings for individual questions included in the Interest in Pursuing a
STEM Career composite factor.
Figure 1. Student Posttest Mean Ratings of Interest in Pursuing a STEM Career
(R) = Reverse-coded item
Despite the lack of evidence to support the growth of interest in STEM careers during
the service-learning experience, students and faculty reported on the success of service
learning in confirming students’ interests in pursuing a STEM career. For example, one faculty
member mentioned, “It is interesting to see this degree of influence among the students.
Some are already set on a career path, others are not and are influenced by the experience.
But positive or negative, it’s rewarding to see that the [service-learning] experience at least
provided them with an idea of the workplace and tasks within this sub-discipline of Biology.”
Additionally, most students responded that they Agreed or Strongly Agreed with statements
that inquired about the degree to which the community work influenced or reaffirmed the
students’ choice of degree and profession. The composite variable, Course Influence on Degree
and Career yielded fairly positive results on the posttest (M = 4.1, SD = .64, n = 52). The
4.3 4.5
4.3
3.9 4.0
1
1.5
2
2.5
3
3.5
4
4.5
5
A career in the STEMfields would be
enjoyable.
STEM fields areinteresting to me.
I am confident that Ican succeed in a
STEM-based career.
I have alwayswanted a career in
the STEM fields.
I have no interest inpursuing a career inthe STEM fields.(R)
CSU ECO LED Program Final Evaluation Report 18 Cobblestone Applied Research & Evaluation, Inc.
following Table 2 indicates the overall mean scores for the three items that comprised this
score.
Table 2. Course Influence on Degree and Career Posttest Individual Item Means
Survey Item Mean (SD)
Performing work in the community helped me clarify or reaffirm which major I will pursue.
4.2 (.72)
The community work in this course assisted me in defining which profession I want to enter.
4.0 (.77)
The work I accomplished in this course has made me more marketable in my chosen profession when I graduate.
4.2 (.72)
Given the student survey results, the service-learning activities provided students with a
positive, hands-on learning experience that, according to students and faculty, influenced their
degree and career goals.
Hands-on Learning
While influencing students toward a STEM career may be a long-term impact of ECO
LED, more immediate impacts were suggested by the results of the student survey. Specifically,
the survey contains items that addressed attitudes of students regarding, for example, learning
the course material. Table 3 shows these specific survey items and their results. Most students
reported that the community service activities of the courses helped with applying course work
to everyday life and to better understand the lectures and course readings. Students had the
most favorable agreement to the idea that service learning be instituted more widely on their
campuses. This also speaks to how much students liked their service-learning courses
considering that this was the first time these courses had the service-learning component and
will, in all likelihood, improve given what faculty learned during the program. The faculty
members also reported their views on how service learning impacted student learning. One
faculty member wrote, “The activities of the course, were very much enhanced by the ECO LED
grant. The opportunity to conduct the erosion control project, in the words of one student,
‘made sure that I remembered how to design these sorts of things’… Students have
mentioned how they notice erosion control measures all around them now along highways
and in construction sites. The experience opened their eyes. When the time came for students
to design their own erosion control measures for the Tip Top Mine project, students reported
CSU ECO LED Program Final Evaluation Report 19 Cobblestone Applied Research & Evaluation, Inc.
that it was straightforward because they had already designed and installed one project
already.” This quote reinforces the view held by these faculty members regarding the
importance of students being able to work on location to put theory into practice.
Table 3. Influence of Service Learning on Student Learning Items Posttest Mean Scores
Survey Item Number of Students
Mean (SD)
The community participation aspect of this course helped me to see how the subject matter I learned can be used in everyday life.
53 4.3 (.61)
The community work I did through this course helped me to better understand the lectures and readings in this course.
53 4.3 (.78)
The idea of combining work in the community with university coursework should be practiced in more classes in the university.
53 4.5 (.58)
I feel I would have learned more from this course if more time was spent in the classroom instead of doing community work. (R)
52 3.65 (1.10)
(R) = Reverse-coded item
Education Summary
Both faculty and student data indicated that the ECO LED program had a positive impact
on student education. Students in these courses had, on average, a high interest in a STEM
career both before and following the course. Further, these service-learning courses influenced
students’ decision to continue working toward a STEM degree and eventually a STEM career.
Students and faculty also agreed on the benefit of service learning on helping students’ learning
through community involvement and hands-on projects and activities. Faculty reported that
students showed positive behaviors toward learning that were not found as often in traditional
courses.
Section 2: Environment
The second focus area of the ECO LED Program was to provide specific improvement in
the environment through service-learning students’ hands-on projects. The following section
Evaluation Question 3: What was the effect of the ECO LED Program on the environment?
CSU ECO LED Program Final Evaluation Report 20 Cobblestone Applied Research & Evaluation, Inc.
relates to Evaluation Question 3. Table 4 shows all of the environment-focused activities
students engaged in through their service-learning courses on each campus.
Table 4. Summary of ECO LED Program Activities
Campus Course Environment Activities
Channel Islands
Water and Conflict in the West
Engaged middle school students with environmental education possibly influencing the future academic and careers paths of students
Restored and protected the habitat of a local California endangered species, the Least Bell’s Vireo
Monitored, analyzed, and reported on water quality to the community partner leading to identified areas of concern where future efforts might be focused
Dominguez Hills
Biogeography of Southern California
Planned, prepared, and installed a 20’ by 20’ native plant garden on campus which continues to serve as an environmental education showplace for the community
Worked with the Palos Verdes Land Conservancy - students individually volunteered at restoration days, both clearing land and planting as well as volunteering in the Conservancy’s greenhouse
Fresno Aquatic Ecology
Developed inventories of flora and fauna in a park
Studied how the Western Pond Turtles’ upland migration could be linked to how weather and climate affect the turtles’ pond habitats
Investigated the effect of land use, specifically cattle grazing, on water quality downstream of recreational waters
Helped with a middle school class and wildlife club
Determined fish species that survived following the draining of a lake for dam repairs
Monitored habitat for a river salmon restoration project
Studied effects of net pens on salmonid survivorship and growth
Studied rainbow trout natural spawning habitat
Fresno Mine Reclamation
Developed a mine reclamation report for the US Forest Service to improve wildlife habitat, reduce environmental and safety hazards, and improve water quality in the Morro Creek, CA watershed
Conducted erosion control project on the campus through the installation of erosion control matting, wattles, and silt fencing
Students engaged in various projects through their service learning courses, but all
projects focused on improving the environment in some capacity. Examples of project goals
CSU ECO LED Program Final Evaluation Report 21 Cobblestone Applied Research & Evaluation, Inc.
included: habitat restoration and protection of endangered and threatened animals;
environmental education for youth; improving water quality; and developing a campus garden.
The local environments have already been impacted by these courses and their accompanying
service-learning projects.
Students at Fresno in the Mine Reclamation course reduced the need for repairs to the
campus storm water retention basin facility due to their installation of erosion control
matting, wattles, and silt fencing. This project showed an immediate impact by reducing
erosion of sediment into the storm water retention basins caused by winter storms in
2012. Additionally, students in the Mine Reclamation course produced a mine
reclamation plan for a local abandoned mine. This plan has the potential impact of
improving wildlife habitat, reducing environmental and safety hazards, and improving
water quality in the Morro Creek, CA watershed.
Students in the Water and Conflict in the West course at Channel Islands learned how to
properly test for water quality. They were able to analyze results and report findings to
local agencies as well as present possible solutions for water quality problem areas.
Additionally, these same students improved nesting habitat conditions for the Least
Bell’s Vireo.
Students in the Aquatic Ecology course, also at Fresno, worked with multiple community
partners and engaged in multiple activities that impacted their local environment. They
reported water quality levels in a nearby creek, determined if a local pond could serve
as a habitat for a particular species of turtle, and determined densities of juvenile trout,
which can lead to management decisions that will manage the habitat to foster
additional natural spawning of trout in a local river.
Students in the Biogeography of Southern California course planted a native, sustainable
garden that will be a permanent part of the campus beyond the time of the service-
learning course. A “mailbox” installed will hold information sheets about the garden’s
plants, the insects and animals they attract as a continued support for community
education. In addition, through work started with the community partner, Palos Verdes
Land Conservancy, additional projects started include a 7-month commitment to the
CSU ECO LED Program Final Evaluation Report 22 Cobblestone Applied Research & Evaluation, Inc.
PVPLC’s Palos Verdes Blue Butterfly restoration project. This project will allow students
to have additional impact on the environment by clearing and replanting a parcel of land
at the Chandler Preserve, to assist with the endangered Blue Butterfly in the area.
Environment Summary
The long-term impact on the environment is yet to be determined and will only be
known through future tracking and research. However, even with the small number of projects
supported by the current funding, these service-learning courses have already provided the
opportunity to have a positive impact on southern California communities to work toward
greater conservation and restoration efforts.
Section 3: Community Building
The next area of interest for the evaluation team addresses Evaluation Question 4.
Specifically, the team investigated how the efforts of faculty, students, and community partners
combined to address community needs and create opportunities for the future. Three general
themes of community education, student community engagement, and developing community
partnerships resulted from analyzing the faculty reports, student surveys, and community
partner surveys.
Community Education
The faculty report regarding the campus garden project at Dominguez Hills stated the
garden “is an environmental showcase for the campus community and also for the community
at large.” Each plant in the garden was labeled, providing an informative experience for
everyone who visited or will visit the garden. Additionally, the faculty report stated the
students “cannot wait to begin the restoration involved in re-introducing the endangered PV
Blue Butterfly to [Palos Verdes Land Conservancy].”
Students participating at Channel Islands engaged with middle school students at a
nearby school. They taught the young students water testing protocols and the reasons behind
Evaluation Question 4: What was the effect of the ECO LED Program on promoting
community building?
CSU ECO LED Program Final Evaluation Report 23 Cobblestone Applied Research & Evaluation, Inc.
systematic investigation of water quality condition. Following their meetings with students, a
math and science teacher at the middle school wrote:
“… Thank you so so so so so much for an AMAZING experience with you and
your students at CSUCI this afternoon. They had beyond a good time, and I am so
impressed at the level in which they engaged themselves with the activities and
learning. I truly appreciate the opportunity and greatly enjoy being a part of this with
the students.”
Those in the Aquatic Ecology course at Fresno engaged in multiple types of service-
learning projects. One student worked with a middle school class and wildlife club. According to
the faculty report, through many encounters with them she “had clear mutual benefits… this
greatly reinforced her love of science teaching, especially in the area of wildlife and natural
resources.” All of the projects at Fresno, however, benefitted not only the students’ learning,
but the community partners as well. A good example of this was the project with a local citizen
group called Friends of Lost Lake Park. The Fresno students were able to develop inventories of
the flora and fauna in the park, thereby providing a service for the park that could not be met
by the community in general.
The faculty member who taught the Mine Reclamation course at Fresno noted about his
students:
“In addition, students informally transmitted their newfound awareness, skills, and
knowledge to the wider community. Some talked about how excited their co-workers were to
discuss project ideas. Others discussed career plans with partners and friends. In the process,
awareness and understanding has increased in the wider community.”
As demonstrated by the above comments and data, the service-learning courses not
only had an impact on the environment but on the community partners and local communities
as well. This aligns to the mission of service learning, which is utilized to not only enrich the
learning experience but ultimately strengthen local communities.
Student Community Engagement
Additionally, the ECO LED Program aimed to affect student community engagement.
The results of the student survey showed students’ attitudes of community engagement
CSU ECO LED Program Final Evaluation Report 24 Cobblestone Applied Research & Evaluation, Inc.
effectively remained the same from pretest to posttest (t(34) = -.567, ns). Further analysis
showed that growth in students’ community engagement was unlikely considering, as a group,
students started out with a high sense of community responsibility. The mean pretest score for
the composite variable measuring Community Engagement was 4.3 (SD = .52, n = 68). These
results also suggest that students with high community engagement may have chosen these
service-learning courses over more traditional courses to begin with. Figure 2 shows the
posttest results of each of the items that make up the Community Engagement composite
variable.
Figure 2. Student Posttest Mean Ratings of Community Engagement
While student attitudes about community engagement remained high over the course,
the students reported that the service learning allowed them to gain additional knowledge on
how to increase community involvement and awareness. Student survey results regarding
students’ agreement to these survey items (i.e., Course Influence on Community Engagement)
were a mean score of 4.0 (SD = .64, n = 52). Thus, results indicate that service learning was able
to link the students’ desires to help with the resources to do so. Figure 3 shows mean ratings of
the individual items of the Course Influence on Community Engagement composite variable.
4.7 4.5
4.2 4.4 4.5
4.2
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Students my agecan do things tomake the world
better.
I can make adifference in myneighborhood or
town.
I feel responsiblefor helping
others.
I often thinkabout the needs
of others.
Helping to solvecommunityproblems issomething
everyone shoulddo.
I intend tovolunteer
throughout mywhole life.
CSU ECO LED Program Final Evaluation Report 25 Cobblestone Applied Research & Evaluation, Inc.
Figure 3. Posttest Student Mean Rating of Course Influence on Community Engagement
Developing Community Partnerships
Though only four partners responded to the community partner survey, the results were
very complimentary toward both the students and faculty with whom they worked. Three
respondents worked with students in the Aquatic Ecology class offered at Fresno. The
remaining respondent worked with students in the Mine Reclamation course, also offered at
Fresno.
Figure 4. Community Partners Satisfaction, Frequency of Survey Responses
3.8 4.0 4.0
4.2
1.0
2.0
3.0
4.0
5.0
The communityparticipation aspect ofthis course showed mehow I can become more
involved in mycommunity.
I feel that the communitywork I did through thiscourse benefited the
community.
The community workinvolved in this coursehelped me to become
more aware of the needsof my community.
I have a responsibility toserve my community.
0 1 2 3 4
Quality of interaction with students
Level of trust with students
Overall communication with students
Quality of student work
Input into planning of student service-learning activities
Quality of interaction with faculty
Level of trust with faculty
Overall communication with faculty
Very Satisfied Satisfied Neutral Dissatisfied Very Dissatisfied Not Applicable
CSU ECO LED Program Final Evaluation Report 26 Cobblestone Applied Research & Evaluation, Inc.
The first part of the survey asked the community partners to rate their level of
satisfaction with various components of their experience with students and faculty. Figure 4
shows the frequencies of responses to these items. Overall, responses tended to be either Very
Satisfied or Satisfied with only three Neutral responses.
The survey also addressed the best aspects of the overall experience. Community
partners’ responses included, “The student I was able to work with was very open to taking in
as many experiences as she could while assisting me in the classroom as well as on field trips.
Her excitement around my students was infectious.” and, “The best aspect for us was
establishing a working relationship between our volunteer organization at Lost Lake Park and
the staff and students at Fresno State University. By partnering with the university, we are
better able to advance one of our goals: protecting sensitive habitat in the park. And
students are able to get practical field experience.”
Specific questions were also asked regarding the partners’ experiences with the
university. Again, only one partner provided feedback in these specific areas, but this partner
indicated that working with the students from Fresno provided the partner with access to
university technology and expertise and it also identified additional volunteers. Furthermore,
working with this group provided the community partner with new connections/networks with
other community groups and increased its geographical area served.
Every community partner indicated they were either Likely (n = 1) or Highly Likely (n = 3)
to serve as a community partner for another service-learning course in the future. Overall,
community partner responses indicated a high level of satisfaction with their experience with
students and faculty in service-learning courses.
Additionally, faculty indicated on their final reports that these service-learning courses
supported the goal of developing community partnerships. For example, students in the
Dominguez Hills class showed continued enthusiasm for planting native plants at home and
informing family, friends, and other community members about the benefits of a native plants
garden. One faculty member from Fresno provided the evaluation team with an extensive list of
options for community partners to potentially work with on future service-learning projects.
CSU ECO LED Program Final Evaluation Report 27 Cobblestone Applied Research & Evaluation, Inc.
Consequently, it is evident that these types of courses are developing local community
partnerships.
Community Building Summary
The ECO LED Program was reported to have contributed to developing relationships and
attitudes that support community involvement. Specifically, multiple opportunities were
provided to students to educate the community regarding environmental issues and proactive
activities. Also, students who were already willing to help the community were given the
information and skills needed to engage in community service on their own. Lastly, the
community partners reported their satisfaction in working with the faculty and students and all
indicated that they would be willing to partner with faculty and students for a future service-
learning course.
Successes and Challenges of the ECO LED Program
In this last section of the report, we answer the final evaluation question related to the
successes and challenges of the ECO LED Program. Through the successes, the positive impacts
on student learning and influence on students’ degrees and future careers is explored. With an
examination of challenges encountered during the courses, faculty can learn from these and
work to create optimal experiences for students and the communities they serve in future
service-learning courses.
Successes of the ECO LED Program
It was clear that faculty, community partners, and most students felt that the ECO LED
Program was a positive experience. Beyond the programs’ impacts on education, environment,
and community that have been discussed in detail, other possible benefits and successes were
mentioned by faculty and community partners.
Unfortunately, due to the duration of the ECO LED Program and scope of the evaluation,
we will be unable to assess the long-range impact of this program on students graduating from
Evaluation Question 5: What were the major successes and challenges of the ECO LED
Program?
CSU ECO LED Program Final Evaluation Report 28 Cobblestone Applied Research & Evaluation, Inc.
STEM disciplines. However, the evidence that was gathered suggests that the STEM students
intend to move toward graduation in their chosen degrees and pursue STEM careers. In fact,
faculty reports suggested that some students showed interest in pursuing careers that were
directly related to the service-learning projects in which they participated. This suggests that
service learning can be used in higher education to provide students with experiences that both
reinforce classroom learning and theory and provide students with opportunities to utilize what
they have learned in possible workplace environments.
The faculty also reported that their students were benefited by service learning in the
following ways:
Students appreciated the chance to work on a professional level.
Students were interested in the course content.
Students showed high levels of motivation.
Students showed excitement with course content and activities.
Students showed high levels of creativity.
Students collaborated well with each other and the community partners.
Students voluntarily took on leadership roles.
Students worked with each other despite being from different academic backgrounds.
Students had improved problem-solving skills.
These added benefits to students’ education and course experiences detail the
additional benefits of service learning in general and the ECO LED Program specifically.
Challenges of the ECO LED Program
The challenges that were encountered were logistical in nature, and while challenges
did occur, they could be mostly avoidable for future course offerings. Also, as previously stated,
we hope that other faculty will be able to learn from these challenges to aid in the creation of
new service-learning courses or adding the service-learning component to existing courses.
Project challenges are provided in Table 5 along with possible solutions. Some solutions were
identified by faculty themselves in written reports of their projects, while other possible
solutions are offered by the evaluation team. While the list of possible solutions is not
CSU ECO LED Program Final Evaluation Report 29 Cobblestone Applied Research & Evaluation, Inc.
exhaustive, they demonstrate that additional planning and repeated execution of these courses
will allow smoother execution of future projects.
Table 5. Challenges and Possible Solutions for the ECO LED Program
Challenges Possible Solutions
Travelling long distances to the project worksite
Find a closer project site to allow for multiple visits or laboratory exercises
Keeping students’ workload manageable More and smaller projects; field trips earlier in the course
Time of course; e.g., an evening course limits the amount of daylight for outdoor projects
Determine if time of the course is consistent with the proposed projects
Difficulty coordinating a project with a community partner
Coordinate with partners before course begins; students should begin the process of finding an SL partner early in the course
Obtaining permission for the project if multiple agencies are involved
Anticipate and request permissions prior to course
Coordinating schedule with middle school teachers, obtaining parental consent, and ensuring curriculum continuity
Hire an education coordinator; create a checklist of school/ district requirements ahead of time
The topic covered in the course was too extensive to be covered in a one semester course
Create two related courses; limit activities for the course
Two community partners responded with areas they would have changed. These
included, “We would recommend setting up a protocol between Fresno State University and
the 4 agencies governing the park so that there is easier access to and greater use of the park
by university students.” , and “The only drawback is that the period of the work is short and
not timed for maximum efficiency. But that is only a shortcoming of the schedule and the
timing of the semester.” Again, these challenges were logistical and may be avoided in the
future.
Conclusions
Faculty, students, and community partners were able to accomplish much during the
scope of the four service-learning courses that comprised the ECO LED Program in spring 2012.
The positive effects of these courses on students’ education, the environment, and community
building were evident given the analysis of student surveys, faculty reports, photographic and
video evidence, and community partners’ surveys. Additional information is needed to better
CSU ECO LED Program Final Evaluation Report 30 Cobblestone Applied Research & Evaluation, Inc.
understand the relationship of how service-learning courses such as these might have longer-
term impacts such as increasing interest in STEM disciplines, graduation in STEM degrees, and
selection of STEM careers. Given that the courses that resulted from the faculty sub-grants
were designed for junior, senior, and graduate students, there was little evidence to support
that STEM service-learning courses created from this program could influence students to
pursue STEM degrees if they were not already interested in these topics or careers. We suggest
that lower level courses be added in the ECO LED Program model to determine if students may
be influenced to pursue a STEM discipline instead of a different path. Also, the results
presented in this report and apparent success of the ECO LED Program warrant the future
expansion and evaluation of the program to investigate further the relationship of service
learning to graduation in a STEM degree and pursuit of a STEM career.
CSU ECO LED Program Final Evaluation Report 31 Cobblestone Applied Research & Evaluation, Inc.
References
Fabrigar, L. R., Wegener, D. T., MacCallum, R.C., & Strahan, E.J. (1999). Evaluating the use of exploratory
factor analysis in psychological research. Psychological Methods, 4, 272-299.
Gelmon, S.B., Holland, B.A., Driscoll, A., Spring, A., & Kerrigan, S. (2001). Assessing service-learning and
civic engagement: Principles and techniques. Boston, MA: Campus Compact.
Partnership for 21st Century Skills (2012). Retrieved from http://www.p21.org/overview/skills-
framework.
Altman, I. (1996). Higher education and psychology in the millennium. American Psychologist, 51(4),
371–378.
Batchelder, T. H., & Root, S. (1994). Effects of an undergraduate program to integrate academic learning
and service: Cognitive, prosocial cognitive, and identity outcomes. Journal of Adolescence, 17,
341–355.
Bringle, R. G., & Hatcher, J. A. (1995). A service-learning curriculum for faculty. Michigan Journal of
Community Service Learning, 2, 112–122.
Driscoll, A., Holland, B., Gelmon, S., & Kerrigan, S. (1996). An assessment model for service-learning:
Comprehensive case studies of impact on faculty, students, community, and institutions.
Michigan Journal of Community Service Learning, 3, 66–71.
Giles, D., & Eyler, J. (1994). The impact of a college community service laboratory on students' personal,
social and cognitive outcomes. Journal of Adolescence 17, 327-339.
Markus, G. B., Howard, J. P. F., & King, D. C. (1993). Integrating community service and classroom
instruction enhances learning: Results from an experiment. Educational Evaluation and Policy
Analysis, 15, 410–419.
Reeb, R. N., Katsuyama, R. M., Sammon, J. A., & Yoder, D. S. (1998). The Community Service Self-Efficacy
Scale: Evidence of reliability, construct validity, and pragmatic utility. Michigan Journal of
Community Service Learning, 5, 48–57.
Roth, C. (1992). Environmental literacy: Its roots, evolution, and directions in the 1990s. ERIC
Clearinghouse for Science, Mathematics, and Environmental Education, Columbus, Ohio.
Teksoz, G., Elvan, S., & Tekkaya-Oztekin, C. (2012). Modeling environmental literacy of university
students. Journal of Science Education and Technology, 21, 157-166.
CSU ECO LED Program Final Evaluation Report 32 Cobblestone Applied Research & Evaluation, Inc.
Appendix A: Student Survey Factor Analysis
Extraction Method: Principal Axis Factoring Rotation Method: Oblimin with Kaiser Normalization Four Factors Extracted Factor 1: Community Engagement (Cronbach’s alpha = .807)
1. Students my age can do things to make the world better. (.696) 2. I can make a difference in my neighborhood or town. (.550) 3. I feel responsible for helping others. (.775) 4. I often think about the needs of others. (.707) 5. Helping to solve community problems is something everyone should do. (.607) 6. I intend to volunteer throughout my whole life. (.466)
Factor 2: Interest in STEM Careers (Cronbach’s alpha = .929)
1. A career in the STEM fields would be enjoyable. (.938) 2. STEM fields are interesting to me. (.870) 3. I am confident that I can succeed in a STEM-based career. (.580) 4. I have always wanted a career in the STEM fields. (.856) 5. Reversed- I have no interest in pursuing a career in the STEM fields. (.471)
Factor 3: Course Influence on Community Engagement (Cronbach’s alpha = .776)
1. The community participation aspect of this course showed me how I can become more involved in my community. (.685)
2. I feel that the community work I did through this course benefited the community. (.670)
3. The community work involved in this course helped me to become more aware of the needs of my community. (.581)
4. I have a responsibility to serve my community. (.255) Factor 4: Course Influence on Degree and Career (Cronbach’s alpha = .854)
1. Performing work in the community helped me clarify or reaffirm which major I will pursue. (.795)
2. The community work in this course assisted me in defining which profession I want to enter. (.844)
3. The work I accomplished in this course has made me more marketable in my chosen profession when I graduate. (.663)
CSU ECO LED Program Final Evaluation Report 33 Cobblestone Applied Research & Evaluation, Inc.
Appendix B: Community Partner Survey Items
Please rate your level of satisfaction in working with [campus] in the [course name] course in the following areas. Scale: 1 = very dissatisfied to 5 = very satisfied with option of not applicable
Quality of interaction with students
Level of trust with students
Overall communication with students
Quality of student work
Input into planning of student service-learning activities
Quality of interaction with faculty
Level of trust with faculty
Overall communication with faculty How did working with faculty and students influence your capacity to fulfill the mission of your organization? If you did not experience any notable influence, please select "No influence". However, if working with faculty and students did influence your capacity, please select all that apply from the following list.
No influence
New insights about the organization's operations
Increase in number of clients served
Enhanced offerings of services
Increased leverage of financial/other resources
Increase in geographical area served
New connections/networks with other community groups
Increase in environmental impact
Change in organizational direction
Increase in number of services offered
Other (please specify)
We would like to know if you experienced any major challenges while working with students and faculty during the course. If you did not experience any major challenges, please select "No major challenges encountered". However, if you did experience any major challenges, please select all that apply from the following list.
No major challenges encountered
Demands upon staff time
Project time period insufficient
Students not well prepared
Number of students inappropriate for the size of my organization
Mismatch between course goals and my organization
Little contact/interaction with faculty
Students did not perform as expected
Other (please specify)
CSU ECO LED Program Final Evaluation Report 34 Cobblestone Applied Research & Evaluation, Inc.
What were some of the economic effects of your work with [campus] above and beyond what your organization normally could have done? Please select all that apply.
No economic effects
Increased value of services
Increased organizational resources
Completion of project(s)
Access to university technology and expertise
New products, services, materials generated
Increased funding opportunities
Identification of new staff
Identification of additional volunteers
Other (please specify)
Were there any significant negative impacts to your organization in working with [campus]?
No negative impacts
Excessive use of staff time
Excessive use of resources
Other (please specify) What was the best aspect of this experience for you?
What aspects of this experience would you change? How likely would you be to serve as a community partner for this or another service-learning course in the future? Scale: 1 = highly unlikely to 5 = highly likely with option of not sure/don’t know