kathy o’byrne, ph.d. ucla center for community learning may 5, 2010 assessment of service-learning...
TRANSCRIPT
Kathy O’Byrne, Ph.D.UCLA Center for Community
Learning May 5, 2010
Assessment of Service-learning Courses in Higher Education:
Understanding the Rationale and Techniques
Agenda
Welcome and IntroductionsDefinitions of assessment and service
learningLearning outcomesResearch options: unit of analysisIntegration into your syllabus &
examplesResourcesQuestions and Discussion
Your campus and your experience
History of your service learning program?
Institutional resources?
Disciplines?Campus
leadership?
History with assessment?
Service learning courses or others?
Research and publications?
Faculty development experience?
Key components of assessment
Assessment is linked to learning goals/outcomes.
Meaningful work in service learning courses is designed to elicit higher order thinking
Multiple forms of evidence
Assessment activities should draw on the affective and cognitive domains of learning
Self-assessment is often part of the overall design
Student involvement and feedback is important data to guide future learning
Consider an examination of processes as well as outcomes
Pre-planning and integrated with instruction
Expectations apparent to students
Definitions and Common LanguageAssessment practices are a reflection of your values.
Assessment is a means of understanding the learning process, and improving the quality of learning that occurs.
Assessment is the systematic application of tools and techniques to better understand and improve the process of learning by everyone involved.
Assessment is a mind set that asks questions– good questions, hard questions, about what and how students are learning.
Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our
expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically
gathering, analyzing and interpreting evidence to determine how well performance matches those expectations and
standards; and using the resulting information to document, explain, and improve performance.
Assessment…can help us focus our collective attention, examine our assumptions, and create a shared academic
culture dedicated to…improving the quality of higher education.
What is service learning?
Equal parts “service” and “learning”Reflection is keyEmbedded in academic, credit-bearing
coursesConnection to graded assignmentsReciprocal benefitCollaborative approach to projects
Examples of strong service learning programs
Tulane UniversityUniversity of San
DiegoKapiolani
Community College
University of Washington
UCLA
Options for assessment: unit of analysis
Students: learning outcomes, knowledge, skills, attitudes, behaviors
Faculty: changes in teaching and instruction
Community: impact of the project, qualities of the partnership
Potential service-learning outcomes for studentsCommunication Skills (written and oral)
Interpersonal Skills
Cultural awareness/Appreciation of diversity
Improved use of technology
Logical reasoning
Application of theory
Self-understanding
Critical thinking Investigation ofPower relationships
Analysis of social institutions
Explication of values
Identify and frame problems
Active listening Negotiation skills Public dialogue on shared issues
Creative conflict to produce growth
Political imagination: reimaging futures
Mentoring Commitment to service/service ethic
Career Development
Values clarification
Character Education
Civic Engagement
Perspective Transformation
Other?
Student learning and assessment issues
1. Define student learning outcomes: knowledge, skills, attitudes or values?
2. Systemic issues: what is the role of the student, faculty, sites? How will students demonstrate learning?
3. What data will you use? Graded assignments? Pre-post instruments? Other?
Service-learning outcomes for assessment
1. Understand and apply knowledge2. Develop identity and values3. Develop community awareness and
practice community engagement4. Demonstrate a commitment to lifelong
learning and action5. Awareness of social justice issues
Summarize overall plan
1. Expectations/assumptions about students2. Learning outcomes: what they should
know and/or know how to do3. Teaching strategies in support of
outcomes4. Assessment Strategies: who? when?
how? where? what?
Five Assessment techniques: Strategy #1: Critical Incidents
Describe an incident that was “critical” to you.
It can be surprising, frightening, enlightening or something that made you proud
What made it significant? What did you learn or how did it change
your worldview/assumptions?
Strategy #2: Concept mapping
One main concept from the courseAdditional cards or post-its that students
arrange in a patterRelated concepts connected or close to one
anotherPolicy, diversity, social justice, the role of
government, economic factors, etc.
Strategy #3: Problem based learning
Based on training from medical schoolsStudents write about an issue related to
their service learning work.Have to provide a potential solution for
change in the community, that includes classroom/curriculum learning and connections to the service learning site/staff/meaningful work they performed.
Strategy #4: Focus Groups
Small groups of students who all worked at the same site answer structured, pre-planned questions about the organization, the community, the work they accomplished, and the future of the organization.
If change is needed, who would have to get involved to make that happen? Elected officials? Residents? Others?
Strategy #5: Pre-post instruments
Can focus on knowledge, attitudes, beliefs, values related to the work of the organization.
Can have students complete the pre-test and return it after they complete the post-test to compare their answers.
Can break down differences according to gender, majors, other characteristics including grades.
Other options for assessing service learning: faculty or academic departments
Departmental views for integrating service learning into a major or minor;
Faculty transformation after teaching a service learning course
Research and publication on discipline-based service learning
Assessment with Community Partners
SatisfactionImpactVisibilityBuilding the
organization’s capacity
Outcomes for community residents or agency clients
Connection of projects to public policy
Research that informs decision-making
Example: Sample questions for service learning community partners
Rate student learning: cultural awareness, appreciation of diversity, communication, critical thinking, negotiation, career exploration, understanding of non-profits and public policy issues, etc.
General: understanding of their responsibilities, how well prepared were students?
What was the meaningful work that was performed?
Your syllabusDefinition of service
learningLearning outcomesCommunity partners
and criteriaJob descriptions for
each siteConnection to graded
assignmentsLogistics
Some learning outcomes Course Competencies and Learning Outcomes: Students in this course will learn: Definitions of civic engagement and service learning; Current and historical national trends in higher education to promote
democratic action and civic engagement; The diverse traditions of service and the history of service movements; How civic engagement and participation in public life contribute to overall
quality of life in the community; An overview of the issues raised by theories and empirical studies of links
between individual motivation and success, access to resources and distribution of wealth;
A description of the national movement for reform of undergraduate education;
Trends in research and scholarship that are impacted by this movement; Definitions and research on sustainable environments, including (but not
limited to) issues of water usage, endangered species and native habitats, transportation, and pollution.
The role of community-based organizations that support a variety of sustainable environment programs, non-profits and governmental agencies;
The potential range of personal choices that individuals can make to promote sustainable environments in Los Angeles.
Principles of university/community partnerships (CCPH)
Principles of good community-campus partnerships: Partners have agreed upon mission, values, goals, and measurable
outcomes for the partnership. The relationship between partners is characterized by mutual trust,
respect, genuineness, and commitment. The partnership builds upon identified strengths and assets, but
also addresses areas that need improvement. The partnership balances power among partners and enables
resources among partners to be shared. There is clear, open and accessible communication between
partners, making it an ongoing priority to listen to each need, develop a common language, and validate/clarify the meaning of terms.
Roles, norms, and processes for the partnership are established with the input and agreement of all partners.
There is feedback to, among, and from all stakeholders in the partnership, with the goal of continuously improving the partnership and its outcomes.
Partners share the credit for the partnership's accomplishments. Partnerships take time to develop and evolve over time.
A “menu” of service learning options: Based on time available, resources, and
experience
An “Appetizer”: Pre-post research on knowledge, attitudes, etc.
An “Entrée”: Ongoing collection of data throughout the academic term or a comparison of service learners to non-service learners
A “Dessert”: small scale study at the end of a course.
Examples
Attitude toward Korean immigrant neighborhoods before and after the service learning experience
Compare the learning outcomes of two sections in a large general education course (service learning and non)
In what way would you say you have changed your mind—if only a little bit—about one issue, as a result of the service learning work?
Resources: Publications and Conferences
Journal of Applied Learning in Higher Education): 2009 Ash and Clayton
Generating, Deepening, and Documenting Learning: The Power of Critical Reflection in Applied Learning
This article will consider the meaning of critical reflection and principles of good practice for designing it effectively and will present a research-grounded, flexible model for integrating critical reflection and assessment.
http://www.missouriwestern.edu/appliedlearning/issues.asp
1999: Where’s the Learning in Service Learning?
Janet Eyler and Dwight Giles.
National Service Learning Clearinghouse
Bonner Foundation Campus Compact
Tenth International Research Conference on Service-learning and Community EngagementInternational Perspectives: Crossing Boundaries through ResearchOctober 28-30, 2010Indianapolis, IN .
The Indiana University-Purdue University Indianapolis (IUPUI) Center for Service and Learning, Indiana Campus Compact, and the International Association for Research on Service-learning and Community Engagement.
October 28-30, 2010 at the Crowne Plaza at Historic Union Station in Indianapolis, IN.
The featured theme for the 10th annual conference is “International Perspectives: Crossing Boundaries through Research.” Service-learning is valued as an active learning strategy across the globe; however, little is known about the ways that service-learning is similar or different in varied contexts. Understanding service-learning and community engagement from diverse cultural perspectives will add insight necessary for comparative research and to improve practice.
.
Questions
Reactions DiscussionRelevanceNeed for further informationNext steps for your service-learning course!
?
