identifying your role in outcomes-based assessment program review marilee j. bresciani, ph.d....
TRANSCRIPT
Identifying Your Role in Outcomes-Based
Assessment Program Review
Marilee J. Bresciani, Ph.D.Professor, Postsecondary Education and
Co-Director of the Center for Educational Leadership, Innovation, and Policy
San Diego State University3590 Camino Del Rio NorthSan Diego, California, U.S.A.
Bresciani, M.J.
Presentation Overview Overview of Outcomes-Based Assessment
(OBA) Program Review Importance of OBAPR Importance of Assessing Student Learning Astin I-E-O Questions to Consider Questions
Bresciani, M.J.
Ask Yourself These Questions What decision did you make about
your program last year? What evidence did you use to
inform that decision? What was it that you were trying to
influence about your program when making that decision with the stated evidence?
Bresciani, M.J.
That is Outcomes-Based Assessment (Bresciani, 2006)
Most people do capitalize on their innate intellectually curiosity to find out what works
Most people just don’t articulate their intended end results (e.g., outcomes) ahead of time
Most people don’t document the decisions made based on their results
Most people don’t follow up later to see if their decisions made the intended improvement
Bresciani, M.J.
The Assessment Cycle (Bresciani, 2006)
The key questions…• What are we trying to do and why? or• What is my program supposed to accomplish? or• What do I want students to be able to do and/or
know as a result of my course/workshop/orientation/program?
• How well are we doing it?• How do we know?• How do we use the information to improve or
celebrate successes?• Do the improvements we make contribute to our
intended end results?
The IterativeSystematicOBPR Cycle(Bresciani, M.J.)
Mission/Purposes
Goals
Outcomes
Implement Methods to
Deliver Outcomes
(Action Planning)
and Methods to Gather Data
Gather Data
Interpret Evidence
Document decisions to improve programs; enhance
student learning and development;
inform institutional decision-making, planning,
budgeting, policy, public accountability
External Review
Strategic Planning/ Inputs/Capacity
-Values influence every aspect of this cycle.
Bresciani, M.J.
Bresciani, M.J.
The Purpose
Outcomes-Based assessment does not exist for assessment’s sake
It is taking what most of us already do, and making it systematic
Its purpose is to reflect on the end result of doing - - how well are we accomplishing that which we say we are?
Bresciani, M.J.
The Purpose, Cont. Strategic and Action Planning
are incorporated into it It is intended to inform decisions
for improvement and resource re-allocation
It helps you link what you do to institutional strategic initiatives and performance indicators
Bresciani, M.J.
The Purpose, Cont. Makes the purpose of our programs
and services clear to students, faculty, parents and other constituents Promotes student responsibility for
learning Promotes opportunities for
collaboration Outcomes-Based Assessment is
not research
Importance of Assessing Student Learning Demonstrates contributions to
institutional mission and goals And contributions to
institutional priorities Assists in informing
prioritization of your time as well as other resources
Outcomes-Based Assessment can be Implemented at Multiple Levels
Decisions are made at multiple levels - some decisions reside only at one level
Bresciani, M.J.
Bresciani, M.J.
Given what OBAPR is, how do you perceive your role in the process?
Why the increased emphasis in OBAPR?
Bresciani, M.J.
Bresciani, M.J.
Drivers of Outcomes-Based Assessment (Ewell, 2003)
To improve the underperforming student
Competency Movement in Business and Industry International Trade Agreements
Bologna Declaration of 1999
Bresciani, M.J.
Drivers, Cont. Government Conversation, 1985
The Higher Education Re-authorization Act Testimonies in USA, 2002 and 2006
Response to NCLB Legislation Regional Accreditation – flexibility
CRAC – 2003, 2004 Both documents focus on student
learning
Bresciani, M.J.
Drivers, Cont.
National Commission on the Future of Higher Education
• Demand for Public Information about Performance• Transparency of outcomes and
results• Comparable measures of quality• Demonstration of value-added of
the entire educational experience
Bresciani, M.J.
Drivers, Cont. Accountability requirements
handed back to states Performance indicators
Discipline Standards could be designed by disciplines
AACU Essential Outcomes/Degree Frameworks and Expected Levels of Learning
Economic State of the Country
Astin’s I-E-O
Inputs - Pre-Assessment
Environment –Formative Assessment
Outcomes – Summative Assessment
Bresciani, M.J.
The Focus on Assessing Student Learning
“The concepts of learning, personal development, and student development are
inextricably intertwined and inseparable.”
– The Student Learning Imperative
Some Questions about Student Learning and Development(Bresciani, Moore Gardner, & Hickmott, 2009)
What do you expect your students to know and be able to do by the end of their education at your institution? And how is your program designed to contribute to that expected learning?
What do you do in your programs to promote the kinds of learning and development that your institution seeks? Bresciani, M.J.
Some More Questions (Bresciani, Moore Gardner, & Hickmott, 2009)
How do I influence student learning? Is it through the education of my
colleagues? Of faculty? Of parents? Of community participants?
Bresciani, M.J.
Reflection Questions (Bresciani, Moore Gardner, & Hickmott, 2009)
How are you directly or indirectly contributing to student learning?
How are you directly or indirectly supporting student learning?
How are you directly or indirectly interfering with student learning?
Bresciani, M.J.
The Ideal for Student Learning
Establish collaborations between academic and student affairs to• facilitate student learning• facilitate student engagement
and socio-academic integration• evaluate professional
development for faculty and staff about effective learning environments
Bresciani, Zelna, & Anderson, 2004Bresciani, M.J.
Now, how do you perceive your role in OBAPR?
Ideas for Your Involvement Pre-assessment design
Cognitive Developmental
Testing the efficacy of placement scores Incorporating the developmental
aspects into design of placement Working with advisors to see how
placement scores are usedBresciani, M.J.
Ideas, Cont. Working with Academic and
Student Affairs to evaluate the efficacy of all assessment instruments and determine what other measures may be utilized to evaluate student learning and development.
Bresciani, M.J.
Ideas, Cont. Determining which comparable
measures of learning are appropriate for your organization to use
Assisting with the development of new comparable measures of learning and testing their efficacy Rubrics
Bresciani, M.J.
What Else?
Bresciani, M.J.
Questions to Consider What is your vision for your role in
OBAPR? With whom do you need to
communicate your vision? How might you need to re-organize
your office to meet your emerging role in OBAPR?
Bresciani, M.J.
Questions, Cont. What resources do you need in
order to carry out your vision? Who do you need to collaborate
with within your Institution State Nation?
Bresciani, M.J.
Questions, Cont.
In order to make your vision a reality,
What do you need to do immediately when you return to your campus?
What do you need to do three months from now?
What about a year from now?Bresciani, M.J.
Bresciani, M.J.
Resources
Each Other NASPA, ACPA, and AACU University Planning and Analysis
(UPA) Assessment website http://www2.acs.ncsu.edu/UPA/assmt/
Questions?
Bresciani, M.J.
One Minute Evaluation
What is the most valuable lesson that you learned from this workshop?
What is one question that you still have?
What do you think is the next step that your division/program needs to take in order to implement systematic program assessment?
References Astin, A. W. (1993). What matters in
college: Four critical years revisited. San Francisco: Jossey-Bass.
Bresciani, M.J. (September, 2002). The relationship between outcomes, measurement. and decisions for continuous improvement. National Association for Student Personnel Administrators, Inc NetResults E-Zine. http://www.naspa.org/netresults/index.cfm
Bresciani, M.J., Zelna, C.L., and Anderson, J.A. (2004). Techniques for Assessing Student Learning and Development in Academic and Student Support Services. Washington D.C.:NASPA.
References, Cont. Bresciani, MJ.(2006). Outcomes-Based
Undergraduate Academic Program Review: A Compilation of Institutional Good Practices. Sterling, VA: Stylus Publishing.
Bresciani, M. J., Gardner, M. M., & Hickmott, J. (2010). Demonstrating student success in student affairs. Sterling, VA: Stylus Publishing.
Palomba, C.A. and Banta, T.W. (1999). Assessment essentials: Planning, implementing and improving assessment in Higher Education. San Francisco: Jossey-Bass.