use assesstoolstogaugesl outcomes.1.scsu
DESCRIPTION
Using Standardized & Locally Created Assessment Tools to Gauge Student Service-Learning Outcomes. This session was presented at the "Creating Engaged Communities: The Role of Service-Learning" conference at the St. Cloud State University Welcome Center on May 23, 2011.TRANSCRIPT
Using Standardized and Locally Created
Assessment Tools to Gauge Student Service-Learning Outcomes
St. Cloud State University Presenters:
Beth Knutson-Kolodzne–Coordinator-Volunteer Connection
Prof. Isolde Mueller – Faculty Liaison for Service-Learning
Kim Oren – Director of Institutional Research
Brief History of Service-Learning & Community Engagement at SCSU…
• 2003 AmeriCorps Grant (4 yrs.)SL Coord. engages faculty in using SL in courses.
• 2005 Faculty use of SL in courses = up 2x since 2003 ; number of students engaged = up 5x; number of hours reported = up 10x.
• 2006 Began surveying service- learning students about their experience.
• 2007 Grant ends • Faculty Liaison for SL begins 25% reassigned time• New President of SCSU• Secured 10 hr./wk. GA for SL
Brief History of Service-Learning & Community Engagement at SCSU…
• 2008 Faculty Liaison begins 50% reassigned time• Began assessment work with Dir. of IR • Adapted student survey instrument
• 2009 Measured variety of forms of community-based learning among students in courses.
• Increased University focus on overall community engagement within and outside of courses
• 2010 Engagement continues to remain stable or grow slightly in some areas.
• 2011 University reorganization plans underway.• Position for a Faculty Scholar in the new University College• Position for Assoc. Director for Civic Engagement–who is a
liaison with the Faculty Scholar.
Outcomes of Service-LearningTaken from: Where’s the Learning in Service-Learning?
Eyler & Giles, 1999
• Tolerance• Personal Development• Interpersonal Development• Understanding & Applying Knowledge• Problem Solving/Critical Thinking• Perspective Transformation• Citizenship
Two data sources
SCSU Student Community-Based Learning Questionnaire Locally developed. Administered Spring to students after their community engagement experiences (2009, Fall 2009, Spring 2010, and Fall 2010). Copy of survey instrument is in handout.
National Survey of Student Engagement (NSSE). Administered at SCSU in spring semesters of 2001, 2003, 2005, 2007, and 2009. Copy of survey instrument is in handout.
National Survey of Student Engagement (NSSE or “Nessie.”)
Derived from vast research literature on college student outcomes.
National Survey of Student Engagement
Measures: x
• Students’ involvement (or engagement) in activities xxknown to facilitate their growth and development. x
• Institutional conditions that are conducive to such xxengagement.
Numbers of SCSU students who completed the NSSE, by year
2001 2003 2005 2007 2009 Total
First year 199 234 161 490 458 1542
Seniors 217 240 199 730 651 2037
Total 416 474 360 1220 1109 3579
Started with NSSE question 1k:During the current school year, about how often have you participated in a community-based project (e.g., service learning) as part of a regular course?
Very often, Often, Sometimes, or Never?
NSSE item 1k: During the current school year, about how often have you participated in a community-based project (e.g., service learning) as part of a regular course?
First year students
N %Never 917 62 Sometimes 390 27 Often 125 8 Very often 39 3 Total 1471 100
All combinedSeniors
N %Never 926 47 Sometimes 703 35 Often 245 12 Very often 107 5 Total 1981 100
All combined
NSSE item 1k: During the current school year, about how often have you participated in a community-based project (e.g., service learning) as part of a regular course?
First year students
N %
Never 917 62
Sometimes 390 27
Often 125 8
Very often 39 3
Total 1471 100
All 5 years
First year students
N %
Did not participate 917 62
Participated 554 38
Total 1471 100
All 5 years
NSSE item 1k: During the current school year, about how often have you participated in a community-based project (e.g., service learning) as part of a regular course?
Seniors
N %
Never 926 47
Sometimes 703 35
Often 245 12
Very often 107 5
Total 1981 100
All 5 years
Seniors
N %
Did not participate 926 47
Participated 1055 53
Total 1981 100
All 5 years
72 79
53
62
52
62
28 21
47
38
48
38
0
20
40
60
80
100
Sp 2001 Sp 2003 Sp 2005 Sp 2007 Sp 2009 Total
First year students: Participated in a community-based project (e.g., service learning) as part of a regular course.
Did not participate
Particpated
61 58
41 42 45 47
39 42
59 58 55 53
0
20
40
60
80
Sp 2001 Sp 2003 Sp 2005 Sp 2007 Sp 2009 Total
Seniors: Participated in a community-based project (e.g., service learning) as part of a regular course.
Did not participate
Particpated
First year students
N %
Did not participate 917 62
Participated 554 38
Total 1471 100
All 5 years
Seniors
N %
Did not participate 926 47
Participated 1055 53
Total 1981 100
All 5 years
15 NSSE breakout questions. Compared Participants versus Non-participants (from question 1k) for each.
6e How often? Tried to better understand someone else's views by imagining how an issue looks from his or her perspective
1t How often? Discussed ideas from your readings or classes with others outside of class (students, family members, co-workers, etc.).
1u How often? Had serious conversations with students of a different race or ethnicity than your own.
1v How often? Had serious conversations with students who are very different from you in terms of their religious beliefs, political opinions, or personal values.
7b Community service or volunteer work. Done or plan to do?
11b SCSU contribution: Acquiring job or work-related knowledge and skills
11e SCSU contribution: Thinking critically and analytically
11h SCSU contribution: Working effectively with others
11i SCSU contribution: Voting in local, state (provincial), or national (federal) elections
11k SCSU contribution: Understanding yourself
11l SCSU contribution: Understanding people of other racial and ethnic backgrounds
11m SCSU contribution: Solving complex real-world problems
11o SCSU contribution: Contributing to the welfare of your community
8a Quality of relationships with other students
8b Quality of relationships with faculty members
Tried to better understand someone else's views by imagining how an issue looks from his or her perspective (Sps 01, 03, 05, 07, & 09 combined).
Total
Comm proj in course N % Total
Did not participate 272 48 566
Participated 254 59 430
All respondents 526 53 996
Total
Comm proj in course N % N
Did not participate 324 51 638
Participated 581 69 845
All respondents 905 61 1483
* Chi-square significant at p < 0.01** Chi-square significant at p < 0.001
Response options: Very often, Often, Sometimes, or Never
Often or Very often
Often or Very often
Seniors**
First year**
Summary of comparisons on 15 NSSE breakout questions
1t Discussed ideas outside of class X X
1u Serious conversations with different races or ethnicities X X
1v Serious conversations with different religions, politics, or values X X
6e Tried to understand someone else's perspective X X
7b Community service or volunteer work X X
11b SCSU contribution: Job or work-related knowledge and skills X X
11e SCSU contribution: Thinking critically and analytically X
11h SCSU contribution: Working effectively with others X X
11i SCSU contribution: Voting in elections X X
11k SCSU contribution: Understanding yourself X X
11l SCSU contribution: Understanding other racial and ethnic backgrounds
X X
11m SCSU contribution: Solving complex real-world problems X X
11o SCSU contribution: Contributing to welfare of community X X
8a Quality of relationships with other students X
8b Quality of relationships with faculty members X
Xs = Statistically significant difference between Participants and Non-participants in the predicted direction (27 of 30 comparisons). That is, Participants were more highly engaged than Non-participants.
Job or work-related knowledge/skills. Understanding yourself.Thinking critically and analytically. Understanding other races/ethnicities.Working effectively with others Solving complex real-world problems.Voting in local/state/national elections. Contributing to welfare of community.
First year students: SCSU contribution composite
Job or work-related knowledge/skills. Understanding yourself.Thinking critically and analytically. Understanding other races/ethnicities.Working effectively with others Solving complex real-world problems.Voting in local/state/national elections. Contributing to welfare of community.
Seniors: SCSU contribution composite
First year students: Frequency of effective behaviors compositeTried to better understand someone else's views.
Discussed ideas from your readings or classes with others outside class.
Serious conversations with students of different races/ethnicities.
Serious conversations with different religions/politics/values.
Seniors: Frequency of effective behaviors compositeTried to better understand someone else's views.
Discussed ideas from your readings or classes with others outside class.
Serious conversations with students of different races/ethnicities.
Serious conversations with different religions/politics/values.
Outcomes of Service-LearningTaken from: Where’s the Learning in Service-Learning?
Eyler & Giles, 1999
• Tolerance• Personal Development• Interpersonal Development• Understanding & Applying Knowledge• Problem Solving/Critical Thinking• Perspective Transformation• Citizenship
Results from Local Survey Data
Student Community-Based Learning Questionnaire (2009-2010)
• Demographics• Form of Community–Based Learning (CBL)• Mandatory or optional?• Number of hours.• Value.• Learning.• Connection to community.• Awareness of community needs.
Local Survey results (page 1 of 5)
Class status.
N %
Senior 671 31
Junior 502 23
Freshman 445 20
Sophomore 444 20
Other 113 5
Total 2175 100 Miss ing 16
Form of Community-Based Learning (CBL).
N %
Service learning 1115 60
Volunteering 421 23
Field experience 130 7
Community-based research 105 6
Other 87 5
Total 1858 100 Miss ing 333
Local Survey results (page 2 of 5)
Was CBL mandatory for the class?
N %
Mandatory 1876 87
Optional 291 13
Total 2167 100 Miss ing 24
Had you done prior CBL?
N %
No 674 52
Yes 507 39
Uncertain 107 8
Total 1288 100 Miss ing 2
Note: Asked only in Fa l l 2009 and Fa l l 2010.
Local Survey results (page 3 of 5)
# of hours engaged in CBL for class.
N %
11 to 20 710 33
1 to 10 695 32
21 to 30 460 21
More than 40 180 8
31 to 40 135 6
Total 2180 100 Miss ing 11
How CE affected my values and ideas.
N %
1094 51
836 39
202 9
13 1
Other 11 1
Total 2156 100
Miss ing 35
Continued to see value in CE I already possessedSaw value in CE I did not possess beforeNo effect on my ideas about CENegatively affected my ideas about CE
Local Survey results (page 4 of 5)
Plan to engage in subsequent Community Service.
N %
Yes 1497 69
Undecided 550 25
No 125 6
Total 2172 100 Miss ing 19
How CE affected my learning.
N %
1802 83
297 14
55 3
Other 18 1
Total 2172 100
Miss ing 19
Learned less than I thought I would
Helped me learn more than I thought I would
Did not affect my learning
Local Survey results (page 5 of 5)
Feel a greater connection to community due to CBL in class.
N %
Yes 1574 72
No 349 16
Undecided 249 11
Total 2172 100 Miss ing 19
More aware of community needs due to CBL in class.
N %
Yes 1785 83
Undecided 192 9
No 178 8
Total 2155 100 Miss ing 36
Effect on learning by hours.
N % N % N % N % N % N %
546 79 576 81 400 88 116 87 162 91 1800 83
129 19 106 15 38 8 13 10 10 6 296 14
14 2 21 3 13 3 2 2 4 2 54 2
3 0 5 1 5 1 2 2 2 1 17 1
692 100 708 100 456 100 133 100 178 100 2167 100
Other.
Total
Did not affect my learning.
Helped me learn more than I thought I would.
1 to 10 Total21 to 30 31 to 40 More than 40
Learned less than I thought I would.
11 to 20
65
10
13
16
7
10
13
11
13
8
10
13
12
10 10
13
0
5
10
15
20
2001 2003 2005 2007 2009
% "
Oft
en"
or "
Ver
y of
ten"
1k. Participated in a community-based project (e.g., service learning) as part of a regular course.
SCSU
Peers
Carn
Nat
Region
MnSCU
First year students13
Response options: Very often, Often, Sometimes, or Never.
Response options: Very often, Often, Sometimes, or Never.
89
20 20
21
18
13
14
2018 19
12
14
19
1717
15
20
5
10
15
20
25
2001 2003 2005 2007 2009
% "
Oft
en"
or "
Ver
y of
ten"
1k. Participated in a community-based project (e.g., service learning) as part of a regular course.
SCSU
Peers
Carn
Nat
Region
MnSCU
Seniors
Response options: Done, Plan to do, Do not plan to do, or Have not decided.
64
69
73
74
79
69
74
79
75
79
71
75
80
77
73
75
60
65
70
75
80
85
2001 2003 2005 2007 2009
% "
Don
e" o
r "P
lan
to d
o."
7b. Community service or volunteer work.
SCSU
Peers
Carn
Nat
Region
MnSCU
First year students79
78
Response options: Done, Plan to do, Do not plan to do, or Have not decided.
64
62
69
75
77
73
61
63
75
70
72
63
66
76
73
7171
55
60
65
70
75
80
2001 2003 2005 2007 2009
% "
Don
e" o
r "P
lan
to d
o."
7b. Community service or volunteer work.
SCSU
Peers
Carn
Nat
Region
MnSCU
Seniors
73
Q & A…
• History
• Data Sources
• Outcomes
• Etc…..