susan agre-kippenhan, portland state university professor, art department
DESCRIPTION
Susan Agre-Kippenhan, Portland State University Professor, Art Department. Evaluating the Effectiveness of Service Learning. Why evaluate?. Assessment as a continual Service Learning improvement tool - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/1.jpg)
Susan Agre-Kippenhan,Portland State University Professor, Art Department
Evaluating the Effectiveness of
Service Learning
![Page 2: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/2.jpg)
Why evaluate?Assessment as a continual Service
Learning improvement toolClarification of critical learning
objectives, ability to focus on Service Learning specifics
Yields useful information for programs, faculty, students
Evidence and support claims
![Page 3: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/3.jpg)
SL Outcomes: Service Learning Claims:
Understanding of course contentTheory to practice Understanding of Community Community related skills Civic Engagement Diversity MORE?
![Page 4: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/4.jpg)
Student Learning FocusValue of authentic work samples
Moving from self-report Moving from less integrated forms of assessment (i.e. observation, focus group)
![Page 5: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/5.jpg)
Continual Improvement Feedback to actual courses, programs Applicable/actionable assessment Can be course/assignment based Can serve other programmatic goals
![Page 6: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/6.jpg)
How do we know what works? A working model for assessment based on
the premise that there are:
Key questions to ask Logical steps to follow Usable data Transforms student learning and community experiences
![Page 7: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/7.jpg)
1Assessment Cycle Model
4Develop Learning Objectives
Identify Work Samples
Locate Critical Junctures
DetermineAssessment Criteria/Quality
TestValidity /Reliability
DisseminateRevise/Revisit
56
23
![Page 8: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/8.jpg)
Assessment Cycle Model
4Develop Learning Objectives
Identify Work Samples
Locate Critical Junctures Determine
Assessment Criteria/Quality
TestValidity /Reliability
DisseminateRevise/Revisit 561
2 3
![Page 9: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/9.jpg)
Assessment Cycle Model What am I trying to assess?
Action: Define Student Learning Outcome
Student Learning Outcomes in the context of:
Mission Priorities Program Goals
1
![Page 10: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/10.jpg)
What are appropriate work samples?
Action: Identify Student Work Samples
Qualitative vs. Quantitative Examples: Portfolios, Papers (writing samples, essays, reflective papers, editorials, journals…) Performances, Artwork, Lab reports
Assessment Cycle Model
2
![Page 11: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/11.jpg)
When should I assess?
Action: Identify critical junctures
Critical Junctures Entering students Passages Culminating
Assessment Cycle Model
3
![Page 12: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/12.jpg)
What am I looking for in student work?
Action: Define scoring criteria
Rubric Development: Scoring Criteria What indicators will I use to demonstrate desired student learning?
Examples: Bloom's taxonomy
4Assessment Cycle Model
![Page 13: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/13.jpg)
How do I evaluate the range of student work?
Action: Define performance levels
Rubric Development: Performance standards How many levels will my rubric have and how will I define performance at each level?
Specific clear descriptors at each levelQualitative differencesCommon errors
4Assessment Cycle Model
![Page 14: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/14.jpg)
Sample RubricWork Sample
Scoring Criteria 1
Scoring Criteria 2
Scoring Criteria 3
Performance Level 1- Low
Performance Level 2- Medium
Performance Level 3- High
![Page 15: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/15.jpg)
Once I have a rubric how do I know it's valid?
How do I know if the rubric is reliable?
Evaluate the rubric against the learning objectives, learning opportunities etc… Is there clear alignment?
Pilot test with multiple evaluatorsDiscuss how well the rubric worksCompare scores for consistencyDetermine procedures for inconsistent scores
5Assessment Cycle Model
![Page 16: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/16.jpg)
What's next?
Action: Back to the cycle
Revise and revisit - learning outcomes, learning opportunities, assignments, critical junctures, rubrics, program, course offerings/order, curriculum, pedagogy…
6Assessment Cycle Model
![Page 17: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/17.jpg)
SL Case Study University Studies - General Education
4 Year integrated program Interdisciplinary program
Critical Features Clearly defined goals
Communication, Critical thinking, Diversity, Civic Responsibility
Common Pedagogy Required Courses
![Page 18: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/18.jpg)
SL Case Study Student Learning Outcomes
Action: Define Student Learning OutcomeAppreciation and Understanding of Civic Responsibility
PSU Mission - Let Knowledge Serve the City University Studies Goal - Civic Responsibility
1
![Page 19: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/19.jpg)
SL Case StudyWork samples
Action: Identify Student Work SamplesCommon assignment given in each class
Paper2
![Page 20: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/20.jpg)
Critical Junctures
Action: Identify critical junctures Pilot in Senior Capstone Classes
Senior Capstone required course Culminating group experience for major and general education Application of expertise to a real project in the community
SL Case Study
3
![Page 21: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/21.jpg)
Common Assignment “Please write a 3-5-page reflective analysis
summarizing the ways you have gained experience with the University Studies goal of Civic Responsibility. Use specific examples from your discussions and course readings, or in completing your project.”
SL Case Study
![Page 22: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/22.jpg)
Scoring Criteria and Performance Levels
Action: Define scoring criteria/levelsCivic Responsibility Stages
Based on Bloom's taxonomy
SL Case Study
4
![Page 23: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/23.jpg)
Valid Rubric
How do I know if the rubric is reliable?
PSU Mission - Let Knowledge Serve the City University Studies Goal - Civic Responsibility
Pilot test with multiple evaluators Compare scores for consistency. Discuss how well the rubric works.
SL Case Study
5
![Page 24: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/24.jpg)
Revisiting ProcessSuggestions for improvements Revise assignment for clarity Revise rubrics for specificity Clarify pedagogy
Confirmed Program goals addressed in class Basis of assignment is valid Keep assessment in Capstone
SL Case Study
6
![Page 25: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/25.jpg)
Focus on student learning objectives What are appropriate work samples? Are there existing critical junctures? How do we evaluate quality? How can this improve our practice?
SL Assessment in practice
![Page 26: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/26.jpg)
Connect to goals and mission Knowledge of the process that is
transferable Assessment as a tool for partnerships Applicable/actionable assessment
Lessons Learned
![Page 27: Susan Agre-Kippenhan, Portland State University Professor, Art Department](https://reader036.vdocuments.us/reader036/viewer/2022062520/56815aca550346895dc89ba7/html5/thumbnails/27.jpg)
For More InformationSusan [email protected]