![Page 1: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/1.jpg)
Connecting Assessment of Student Learning to
Program Review
Marilee J. Bresciani, Ph.D.Professor, Postsecondary Education and
Co-Director of the Center for Educational Leadership, Innovation, and Policy
San Diego State University3590 Camino Del Rio NorthSan Diego, California, U.S.A.
![Page 2: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/2.jpg)
Bresciani, M.J.
Presentation Overview Overview of Outcomes-Based
Assessment (OBA) Ways in Which Results can Be Used Elements of Outcomes-Based Program
review (OBPR) Working on the Assessment of
Citizenship Questions and Discussion
![Page 3: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/3.jpg)
Bresciani, M.J.
Ask Yourself These Questions How would you describe the
purpose of assessment to your colleagues?
How would you describe the purpose of program review to your colleagues?
![Page 4: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/4.jpg)
Bresciani, M.J.
The Assessment Cycle (Bresciani, 2006)
The key questions…• What are we trying to do and why? or• What is my program supposed to accomplish? or• What do I want students to be able to do and/or
know as a result of my course/workshop/orientation/program?
• How well are we doing it?• How do we know?• How do we use the information to improve or
celebrate successes?• Do the improvements we make contribute to our
intended end results?
![Page 5: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/5.jpg)
The IterativeSystematicOBPR CycleAdapted from Peggy Maki, Ph.D. by Marilee J. Bresciani, Ph.D.
Mission/Purposes
Goals
Outcomes
Implement Methods to
Deliver Outcomes
(Action Planning)
and Methods to Gather Data
Gather Data
Interpret Evidence
Documented decisions to improve programs; enhance student learning and development;
inform institutional decision-making, planning,
budgeting, policy, public accountability
External Review
Strategic Planning/ Inputs/Capacity
![Page 6: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/6.jpg)
Frame Shift (Jenefsky et al, 2009)
From traditional input-based model to outcomes-based model
Heightened attention to improving the quality of student learning
From description & advocacy to evidence-based analyses and planning
From audit to collective inquiry & reflection
From focus on conducting effective program review to using the results effectively
6
![Page 7: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/7.jpg)
What are you already doing that could be considered outcomes-based assessment (OBA)?
How well is your OBA process informing program decisions?
How could you readily incorporate what is working well
into your program review process?
![Page 8: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/8.jpg)
Report Out
How does your intended purpose for OBPR and current
process support your intended use of the data generated from OBPR?
Bresciani, M.J.
![Page 9: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/9.jpg)
It is all about the Use of the Data! How do you want to use the
data generated from program review?
Does the intended use of the data align with your intended purpose for program review?
Bresciani, M.J.
![Page 10: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/10.jpg)
Uses of Assessment Results(WASC Program Review Guidelines, 2009)
Developing program learning outcomes and identifying appropriate means for assessing their achievement
Better aligning department, college and institutional goals
Refining departmental access, and other interventions to improve retention/attrition, and graduation rates
Bresciani, M.J.
![Page 11: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/11.jpg)
Uses, Cont.(WASC Program Review Guidelines, 2009)
Designing needed professional development programs, especially for faculty and staff to learn how to develop and assess learning outcomes
Reorganizing or refocusing resources to advance specific research or community outreach agendas
Re-assigning faculty/staff or requesting new lines
Bresciani, M.J.
![Page 12: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/12.jpg)
Uses, Cont.(WASC Program Review Guidelines, 2009)
Illuminating potential intra-institutional synergies
Developing specific plans for modifications and improvements
Informing decision making, planning and budgeting, including resource re/allocation
Linking and, as appropriate, aggregating program review results to the institution’s broader quality assurance/improvement efforts
Bresciani, M.J.
![Page 13: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/13.jpg)
In order for these Uses to Occur, An Institution Needs…(Bresciani, 2006)
Set priorities around institutional values Communicate a shared conceptual
framework and common language Systematically gather data that
actually evaluates outcomes Document how information gets used
to actually inform decisions
Bresciani, M.J.
![Page 14: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/14.jpg)
In order for these Uses to Occur, An Institution Needs…(Bresciani, 2006)
Provide professional development and support for faculty and staff and administrative decision makers
Demonstrate leadership commitment to support the process and use the data to improve programs, re-allocate resources, and reinforce institutional priorities
Bresciani, M.J.
![Page 15: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/15.jpg)
In order for these Uses to Occur, An Institution Needs…(Bresciani, 2006)
Commit to re-allocate time to intentional reflection and its systematic documentation of student learning as well as research
Centralize coordination of data/report management
Manage a way to systematically engage in documentation
Conduct a meta-assessment of the process
Bresciani, M.J.
![Page 16: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/16.jpg)
How do you see using the results of Outcomes-Based Program Review (OBPR)?
What do you need to do differently with your process in order to utilize the results?
Bresciani, M.J.
![Page 17: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/17.jpg)
Report Out
Design the Program Review Process so that you can use the Results to achieve your
Purpose
Bresciani, M.J.
![Page 18: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/18.jpg)
So, what do we need to document?
Well…(insert technical disclaimer)
![Page 19: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/19.jpg)
Bresciani, M.J.
Typical Components of OBA(Bresciani, 2006)
Program Name Program Mission or Purpose Goals
Align with your strategic plan, strategic initiatives, institutional goals, division goals, or department goals
Outcomes Student Learning and Program
Planning for Delivery of Outcomes Concept Mapping/Curriculum Alignment Matrix Course/Workshop Design (e.g., syllabus for the
workshop)
![Page 20: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/20.jpg)
Bresciani, M.J.
Typical Components of An OBA (Bresciani, 2006)
Evaluation Methods/Tools Link the method/tool directly to the
outcome Include criteria for each method as it
relates to each outcome Add Limitations, if necessary Include Division, Institutional, or State
Indicators Determine acceptable level of
performance and why
![Page 21: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/21.jpg)
Bresciani, M.J.
Typical Components of OBA Implementation of Assessment Process
Identify who is responsible for doing each step in the evaluation process (list all of the people involved in the assessment process at each step of the process)
Outline the timeline for implementation Identify who will be evaluated Identify other programs who are assisting with the
evaluation Identify who will be participating in interpreting the
data and making recommendations and decisions
![Page 22: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/22.jpg)
Bresciani, M.J.
Typical Components of OBA
Program Name Outcomes Results
Summarize the results for each outcome Summarize the process to verify/validate
the results Summarize how the results link with
performance indicators
![Page 23: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/23.jpg)
Bresciani, M.J.
Typical Components of OBA Decisions and Recommendations
Summarize the decisions/recommendations made for each outcome
Identify the groups who participated in the discussion of the evidence that led to the recommendations and decisions
Summarize how the decisions /recommendations may improve performance indicators
Identify how intended improvements enhance strategic initiatives, if applicable
![Page 24: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/24.jpg)
Bresciani, M.J.
Typical Components of OBA Decisions and Recommendations, Cont.
Summarize the suggestions for improving the assessment process
Identify when each outcome will be evaluated again (if the outcome is to be retained)
Identify those responsible for implementing the recommended changes
Identify the resources needed to make the necessary improvements, if applicable
![Page 25: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/25.jpg)
In addition… Link to professional accreditation when possible Organize an External Review
Can be external to department if not able to do external to institution
Explain level of expected performance (student learning and research) and how it was derived/decided
Document decisions made and resources re-allocated, if applicable
Bresciani, M.J.
![Page 26: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/26.jpg)
Differentiate the Program Process from the Institutional Process
What do you need to document as a program in order to provide the institution with its required information?
Avoid annoying faculty and staff to document items at an institutional level, if you don’t use that information for anything.
Bresciani, M.J.
![Page 27: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/27.jpg)
Differentiate the Program Process from the Institutional Process
What are roles and responsibilities of program personnel verses institutional personnel?
How will the data be examined and used at varying levels?
What are appropriate guiding questions for program officials, external reviewers and higher level administrators to inform the use of the results?
Bresciani, M.J.
![Page 28: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/28.jpg)
For Example - Program Program student learning outcomes
and results Program enrollment data and program
specific data and contribution to understanding whether program goals are met
Program decisions, resource re-allocations, practice and policy changes
Bresciani, M.J.
![Page 29: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/29.jpg)
For Example - Institution
Verses Whether program is meeting or not meeting
institutional goals/priorities Required review of evidence-based
recommendations that affect other parts of the institution
Re-allocation of resources Articulating priorities and alignment with
institutional valuesBresciani, M.J.
![Page 30: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/30.jpg)
Examine your Institutional Guidelines and Templates…
Which portions of your guidelines and templates help you use the data to achieve the purpose of OBPR?
What portions align with your professional accreditation process?
What templates can be combined/aligned to decrease documentation efforts?
![Page 31: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/31.jpg)
Report Out
Bresciani, M.J.
![Page 32: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/32.jpg)
Prioritize Institutional learning outcomes and
strategic initiatives Resources to improve those values Time allocated to the data
collection, reflection, and improvements you desire
Bresciani, M.J.
![Page 33: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/33.jpg)
Process for Reviewing and Using Data Be sure the process for reviewing
and using the data is clear Specify roles and responsibilities of
everyone involved Articulate how decisions will be
documented and approved Provide guiding questions to those
reviewing the reports including guidelines for external reviewers
Bresciani, M.J.
![Page 34: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/34.jpg)
Who do you want looking at these reports in order to make the best informed decisions?
In other words, who should see these reports
(differentiate between content and process) and on what criteria should they be
reviewed?
![Page 35: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/35.jpg)
Report Out
Bresciani, M.J.
![Page 36: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/36.jpg)
Drafting Outcomes and Evaluation Tools for Citizenship
Bresciani, M.J.
![Page 37: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/37.jpg)
Report Out
Bresciani, M.J.
![Page 38: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/38.jpg)
Bresciani, M.J.
Outcomes/Objectives
Outcomes are more detailed and specific statements derived from the goals.
These are specifically about what you want the end result of your efforts to be. In other words, what do you expect the student to know and do as a result of your one hour workshop; 1 hour individual meeting; website instructions; etc.
It is not what you are going to do to the student, but rather it describes how you want the student to demonstrate what he or she knows or can do.
![Page 39: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/39.jpg)
Constructing Learning Outcomes/Objectives
Outcomes use active verbs such as articulate, illustrate, conduct, synthesize, analyze, construct, etc.
Depending on what level of learning you expect from your learning delivery method. http://www.coun.uvic.ca/learn/program
/hndouts/bloom.html
![Page 40: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/40.jpg)
Bresciani, M.J.
Outcomes/Objectives You may want to start with articulating
outcomes that are more manageable. For instance, articulate outcomes for
your outreach programs first; then later, move to your individual
consultations; than your information pieces, if at all.
![Page 41: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/41.jpg)
Bresciani, M.J.
Another Take on Bloom
1. Knowledge = workshops2. Skills = opportunities to apply3. Attitudes/Values Clarification
= facilitated reflection4. Behavior Change = facilitated
interventions
![Page 42: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/42.jpg)
Bresciani, M.J.
Outcomes/Objectives, Cont.
Make a conscious decision to articulate outcomes that infer pre- and post-tests
Make a conscious decision to be held responsible for behavior
Remember that your outcomes may look different for your various constituents - - you may want to start with your more manageable population first, such as your Para-professionals
![Page 43: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/43.jpg)
Bresciani, M.J.
Outcomes/Objectives, Cont. Regardless of whether your goals
are top down – the outcome is where you operationalize the goal.
Therefore, the outcome or end result of the doing allows you to “personalize” the goal to your own program.
![Page 44: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/44.jpg)
Example Outcomes Adapted from TAMU
identify one’s role within the organization
identify one’s own behavior that is congruent with organizational mission
translate one’s role within organization to larger community
explain a global perspective on issues related to organization Bresciani, M.J.
![Page 45: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/45.jpg)
Bresciani, M.J.
Questions to Ask Yourself About
Outcomes/Objectives Is it measurable/identifiable? Is it meaningful? Is it manageable? Who is the target audience of my outcome? Who would know if my outcome has been
met? How will I know if it has been met? Will it provide me with evidence that will lead
me to make a decision for continuous improvement?
![Page 46: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/46.jpg)
Assignment:
Draft or Refine your program outcomes
for citizenship
![Page 47: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/47.jpg)
Ensuring that you have provided the student with the opportunity to learn that which you expect of them
Bresciani, M.J.
![Page 48: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/48.jpg)
ExampleOutcomes
5 minute presentation
in classroom workshop
one-on-one counseling
students will be able to identify one reason to do an internship X X X
define internships X X
explain how career services can help
them obtain internships X X X
![Page 49: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/49.jpg)
ExampleOutcomes
posters, emails, bulletin boards one-on-one
workshop at heath fair
describe the relationship
of hand hygiene to well-being x oral exam
demonstrate or apply
correct hand washing
techniques x observation observation
be able to explain the relationship
of hand washing to the number
of times he/she is sick oral exam
one minute question
![Page 50: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/50.jpg)
Key Things to Remember(King, 2003; Komives & Assoc., 2003; Mentkowski & Assoc, 2000, Kuh et al., 2005; Astin, 1996; Bresciani et. al., 2009)
Student learning must be intentionally designed
Activities to support intentional student learning must be planned and made systematic
Learning must be facilitated
![Page 51: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/51.jpg)
Key Things to Remember, Cont.(King, 2003; Komives & Assoc., 2003; Mentkowski & Assoc, 2000, Kuh et al., 2005; Astin, 1996; Bresciani et. al., 2009)
Learning must be evaluated at the point of the facilitation prior to evaluating the transferability of learning Evaluate the learning when you expect it to occur first; then evaluate how well it transferred
![Page 52: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/52.jpg)
Key Things to Remember, Cont.(King, 2003; Komives & Assoc., 2003; Mentkowski & Assoc, 2000, Kuh et al., 2005; Astin, 1996; Bresciani et. al., 2009)
In order to systematically improve learning, we must systematically design and evaluate the opportunities to improve student learning
Outcomes-based assessment is not research
![Page 53: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/53.jpg)
Work on Outcome Delivery Map
![Page 54: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/54.jpg)
Bresciani, M.J.
Before Choosing an Assessment Method…
Think about what meeting the outcome looks like Be sure to describe the end result of the
outcome by using active verbs This helps articulate the criteria for identifying
when the outcome has been met Describe how your program is delivering the
outcome There may be clues in the delivery of the
outcome that help you determine how to evaluate it
![Page 55: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/55.jpg)
Bresciani, M.J.
Before Choosing an Assessment Method, Cont.
Think about collecting data from different sources to make more meaningful
and informed decisions for continuous improvement (e.g., surveys, observations, self-assessment) and for triangulation/verification of data
that you believe will be useful in answering the important questions you have raised
that will appeal to your primary constituents or to those with whom you are trying to influence
![Page 56: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/56.jpg)
Bresciani, M.J.
Measurement Methods(Palomba and Banta, 1999)
Evidence of learning- basically two types Direct-methods of collecting
information that require the students to display their knowledge and skills
Indirect- methods that ask students or some one else to reflect on the student learning rather than to demonstrate it
![Page 57: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/57.jpg)
Bresciani, M.J.
Another Way to Look at It(Ewell, 2003) There are naturally occurring
assessment techniques (e.g. project-embedded assessment methods such as essays, observed behavior, student interactions, student debates)
There are those designed as a means to evaluate (e.g., surveys)
![Page 58: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/58.jpg)
Bresciani, M.J.
Your Choices are Which method(s) – optional to skip and focus on tools
Which tool(s) – by what means will you gather the data?
Which criteria?
![Page 59: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/59.jpg)
Bresciani, M.J.
Choosing A Tool It is important to choose tools based
on what you are trying to assess, not on what tool is most appealing to you
Consider what will influence your constituents
Consider what will provide you with information to make decisions
Be able to justify your choice of tool and method
![Page 60: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/60.jpg)
Bresciani, M.J.
Things to Consider When Choosing an Instrument
What outcome(s) are you measuring? What criteria will determine if the
outcome is met? Who is being assessed? How often do I
have access to them? Do I know who they are?
What is my budget? What is my timeline? What type of data is most meaningful to
me: direct/indirect and words/numbers
![Page 61: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/61.jpg)
Bresciani, M.J.
Things to Consider, Cont.
Who will analyze the data and how? Who needs to see this data? How easily can I fit this method into my
regular responsibilities? (every day, week, semester, year)
Who needs to make decisions with this data?
How will I document the evidence and the decisions made from that evidence?
![Page 62: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/62.jpg)
Bresciani, M.J.
Common Tools for Identifying Learning and Development
Interviews Focus Groups Observations Surveys Criteria and Rubrics Case Studies Portfolios
![Page 63: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/63.jpg)
Bresciani, M.J.
Possible Assessment Tools, Cont.
Quiz Essay Journal One-Minute Question Peer Evaluation with criteria or rubric Professional Evaluation with criteria or
rubric
![Page 64: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/64.jpg)
Developing Criteria
Criteria checklists or rubrics
![Page 65: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/65.jpg)
Bresciani, M.J.
Uses of Rubrics Provide evaluators and those whose work is
being evaluated with rich and detailed descriptions of what is being learned and what is not
Combats accusations that evaluator does not know what he/she is looking for in learning and development
Can be used as a teaching tool – students and staff begin to understand what it is they are or are not learning or are or are not able to demonstrate what they know
![Page 66: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/66.jpg)
Bresciani, M.J.
For example - Use of Journal Rubric You can use a rubric to
Norm staff’s expectations Inform students of what you are looking for Give students an opportunity to see how
they have improved Make grades more meaningful Help students identify their own learning or
absence thereof Assess a student, course, workshop, or a
program
![Page 67: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/67.jpg)
Bresciani, M.J.
Some Types of Rubrics Checklist - A simple list of criteria and
possibly a rating scale Advanced Checklist – Full descriptions of
the list of criteria and a rating scale Simple Model - Full descriptions of the list
of criteria and simple descriptions of levels
Full Model - Full descriptions of the list of criteria and full descriptions of levels
![Page 68: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/68.jpg)
Bresciani, M.J.
Some Types of Rubrics
Checklist - A simple list of criteria and possibly a rating scale
1. 2-minute description of ethical dilemma ____2. Explanation of reason for ethical dilemma ____3. Explanation of ethical dilemma ____4. Depth of awareness of potential barriers to resolving ethical dilemma ____5. Illustration of expected results in resolving dilemma ____
Y = Yes N = No or
4 = Excellent 1 = Poor
![Page 69: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/69.jpg)
Bresciani, M.J.
Excerpt for Oral Presentation OutcomeBresciani, M.J.
Distinguished Intermediate Novice
Volume Presenter is easy to hear. Audience is able to hear as a w hole, but there are times w hen volume is not quite adequate.
Presenter is diff icult to hear.
10 10 5 0Rates Rates of speech are appropriate. Speaker may at times seem like s/he
is rushing or exaggerating pauses.The rates of speaking are too slow or too fast.
10 10 5 0Mannerisms Speaker makes eye contact w ith
everyone and has no nervous habits. Speaker has excellent posture.
Eye contact may focus on only one member of the audience or a select few members. Mildly distracting nervous habits are present but do not override the content.
Very little eye contact is made w ith the audience. It may sound like the speaker is reading the presentation. Nervous habits that distract the audience are present.
10 10 5 0Engagement Presentation involves audience, allow ing
time for audience to think and respond.Audience is involved but inadequate processing or response time is provided.
Speaker does not involve audience.
10 10 5 0
Oral Presentation RubricPresenter's Name: __________________________________________________
![Page 70: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/70.jpg)
Bresciani, M.J.
Steps to Creating a Rubric Articulate the outcome Decide what meeting the outcome looks like
– “How do you know the outcome has been met? What does it look like?” Articulate exactly what you are looking for and
how you will know it has been met List the aforementioned as criteria or a
detailed description Choose a model for a rubric that bests fits
your project
![Page 71: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/71.jpg)
Bresciani, M.J.
Steps to Create a Rubric, Cont. Articulate the levels you would
expect that criteria to be demonstrated
If you choose, define those levels in great detail
Norm the group using the rubric Pilot the rubric Revise the rubric
![Page 72: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/72.jpg)
Bresciani, M.J.
Basic Agreements Agree on an outcome Agree on method/tool of data collection Agree on the meaning for the outcome
and definition – in other words agree on how you know the outcome is met and what it will look like when you see it met
Agree on the systematic implementation of the assignments and the rubric
![Page 73: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/73.jpg)
Refer to sample rubrics and handouts
Bresciani, M.J.
![Page 74: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/74.jpg)
Select one of your outcomes, identify an evaluation tool
and draft a criteria checklist or a rubric
![Page 75: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/75.jpg)
You do not have to assess everything you do every year.
You don’t have to do everything at once-start with 2 or 3 learning outcomes
Prioritize your goals/outcomes Think baby steps and be flexible Acknowledge and use what you have already done. Assessment expertise is available to help you - - not
to do the work for you Borrow examples from other institutions to modify
as appropriate Time for this must be re-allocated We allocate time according to our priorities
Take-Home Messages
![Page 76: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/76.jpg)
Bresciani, M.J.
Resources
Each Other AAC&U, WASC, NASPA, and ACPA University Planning and Analysis
(UPA) Assessment website http://www2.acs.ncsu.edu/UPA/assmt/
![Page 77: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/77.jpg)
Bresciani, M.J.
On-Line Rubric Resources http://school.discovery.com/schrockguide/a
ssess.html http://www.odyssey.on.ca/~elaine.coxon/ru
brics.htm http://rubistar.4teachers.org/ http://intranet.cps.k12.il.us/Assessments/Id
eas_and_Rubrics/ideas_and_rubrics.html http://teachers.teach‑nology.com/web_tools
/rubrics/
![Page 78: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/78.jpg)
Questions?
![Page 79: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/79.jpg)
Bresciani, M.J.
One Minute Evaluation
What is the most valuable lesson that you learned from this workshop?
What is one question that you still have?
What do you think is the next step that your division/program needs to take in order to implement systematic program assessment?
![Page 80: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/80.jpg)
References Jenefsky, C. Bresciani, M.J., Buckley,
L., Farris, D., Kasimatis, M. (2009). WASC Guidelines for Program Review. Oakland, CA: WASC
Bresciani, M.J., Zelna, C.L., and Anderson, J.A. (2004). Techniques for Assessing Student Learning and Development in Academic and Student Support Services. Washington D.C.:NASPA.
![Page 81: Connecting Assessment of Student Learning to Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649f4e5503460f94c7031e/html5/thumbnails/81.jpg)
References, Cont. Bresciani, MJ.(2006). Outcomes-
Based Undergraduate Academic Program Review: A Compilation of Institutional Good Practices. Sterling, VA: Stylus Publishing.
Bresciani, M. J., Gardner, M. M., & Hickmott, J. (2010). Demonstrating student success in student affairs. Sterling, VA: Stylus Publishing.