exploring good practices in outcomes-based assessment program review marilee j. bresciani, ph.d....
TRANSCRIPT
Exploring Good Practices in Outcomes-Based
Assessment Program Review
Marilee J. Bresciani, Ph.D.Associate Professor, Postsecondary Education and
Co-Director of the Center for Educational Leadership, Innovation, and Policy
San Diego State University3590 Camino Del Rio NorthSan Diego, California, U.S.A.
Bresciani, M.J.
Overview of Book Describes “good practices” in outcomes-
based assessment program review that were generated from multiple case studies.
Outlines basic components of outcomes-based assessment program review.
Introduces criteria for identifying good practices within outcomes-based assessment program review.
Outlines steps for implementing sustainable outcomes-based assessment program review.
Bresciani, M.J.
Context Public demand for accountability in
higher education Commission on the Future of
Higher Education Professor Bensimon’s highlight of
our innate intellectual inquiry, “Why is that what I am doing doesn’t work?”
Bresciani, M.J.
Purpose The purpose of this study is to use
multiple institutional case studies to explore effective outcomes-based assessment program review practices in order to establish criteria that could be adapted at other institutions for implementing pervasive and sustainable outcomes-based assessment program review.
Bresciani, M.J.
Framework and Epistemology
Astin’s I-E-O Astin, A. W. (1993). Assessment for
Excellence: The Philosophy and Practice of Assessment and Evaluation in Higher Education. American Council on Education; Oryx Press.
Papert’s Situating Constructionism Papert, S. (1991). Situating Constructionism.
In Constructionism. Edited by Papert and Harel. Cambridge, MA:MIT Press.
Bresciani, M.J.
Research Questions What are the components of outcomes-
based assessment program review at institutions where this practice is pervasive, effective, efficient, and enduring ?
What are the criteria for pervasive, effective, efficient, and enduring outcomes-based assessment program review processes?
What are the key steps in implementing pervasive, effective, efficient, and enduring outcomes-based assessment program review processes?
Bresciani, M.J.
Methodology The methodology for this study was
multiple case studies (Stake, 1995; Merriam, 1988)
Multiple sources of information were utilized in this analysis: surveys, case study submissions, document analysis, and clarification sought through email and over phone.
3 Year Study Open, axial, and selective coding Member checks
Bresciani, M.J.
Sample Criterion sampling A list of 43 institutions generated
based on criteria and circulated to 25 assessment scholars for further comment. No additions or deletions were made.
18 institutions completed all phases and were included in the study
Bresciani, M.J.
Good Practice Components
1. Program Name and Description2. Program Mission Statement3. Program Goals4. Program Outcomes5. Student Learning Outcomes6. Evaluation Methods for Each
Outcome
Bresciani, M.J.
Good Practice Components (cont’d)
7. Results by outcomes or interpretation of results as they relate to outcomes
8. Decisions & recommendations made about the program for each outcome
9. Level of involvement of program faculty/staff in process
10. Time-frame of evaluation process
Examine the application of the template
See program review guidelines from John F
Kennedy University
Bresciani, M.J.
Criteria for Good Practices of Outcomes-Based Assessment
Program Review
Criterion 1: Clear Understanding of Goals and Expectations for Program Review
Criterion 2: CollaborationCriterion 3: Use of ResultsCriterion 4: Awards and RecognitionCriterion 5: Resources to Support
Program Review
Bresciani, M.J.
Criteria for Good Practices (cont’d)
Criterion 6: Coordination of the Process
Criterion 7: Flexibility Criterion 8: Addressing
Barriers Criterion 9: An Evaluation of
the Program Review Process
Examine a sample matrix for considering effective, efficient, and enduring OBAPR
Bresciani, M.J.
Suggestions for Implementing Outcomes-Based Assessment Program Review (OBAPR)
Create a Well-Represented, Well-Respected OBAPR Committee
Organize the Committee’s Role and Responsibilities
Articulate Expectations for Outcomes-Based Assessment Program Review
Plan Short-Range and Long-Range Goals
Bresciani, M.J.
Suggestions for Implementing Outcomes-Based Assessment Program Review (OBAPR), Cont.
Identify Existing Resources and Processes and Identifying New Resources
Establish a Communication Plan Discuss Implementation Barriers
and Strategies to Overcome Them Move Forward with Flexibility
Examine sample questions to consider when implementing OBAPR
Bresciani, M.J.
Criterion Sample Selection References
American Association of Higher Education (1994). Nine principles of good practice for assessing student learning. Retrieved March 17, 2006, from http://www.aahe.org/assessment/principl.html.
Eckel, P., Green, M., and Hill, B. (2001). On change V—Riding the waves of change: Insights from transforming institutions. Washington DC: American Council of Education.
Lopez, C. (1997). Opportunities for improvement: Advice from consultant-evaluators on assessing student learning. Evidence of strong institutional support for assessing student learning. Retrieved March 16, 2006, from http://www.ncahigherlearningcommission.org/resources/assessment/index.html.
Bresciani, M.J.
Citation Bresciani, M.J. (2006). Outcomes-
Based Academic and Co-Curricular Program Review: A Compilation of Institutional Good Practices. Sterling, VA: Stylus Publishing