the arl assessment framework in practice at montana state ...€¦ · gap analysis 10 what are peer...
TRANSCRIPT
![Page 1: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/1.jpg)
Meta-Assessment: The ARL Assessment Framework in Practice at Montana State University
Scott Young, Montana State UniversityDavid Swedman, Montana State University Martha Kyrillidou, QualityMetrics LLC
![Page 2: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/2.jpg)
Outline
▫ Motivations▫ Methods▫ Results▫ Reflections ▫ Recommendations
2
![Page 3: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/3.jpg)
▫ Motivations_▫ Methods▫ Results▫ Reflections ▫ Recommendations
3
![Page 4: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/4.jpg)
4
![Page 5: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/5.jpg)
How can we critically reflect on our own evolving assessment practice?
5
![Page 6: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/6.jpg)
6
![Page 7: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/7.jpg)
_Audience Question_
How many participated in the data gathering for ARL’s rethinking of the assessment program supported by Atheneum21?
7
![Page 8: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/8.jpg)
_Audience Question_
Have you tried to use this framework for your own assessment work?
8
![Page 9: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/9.jpg)
▫ Motivations▫ Methods_ ▫ Results▫ Reflections ▫ Recommendations
9
![Page 10: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/10.jpg)
Methods▫ Case Study
▪ What are we doing at MSU?
▫ Comparative Analysis▪ What are we doing in relation to
the ARL Framework?
▫ Gap Analysis▪ What are peer programs doing?
10
![Page 11: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/11.jpg)
CaseStudy
11
UX and Assessment at Montana State University
▫ Strategic Planning process (via BSC)
▫ Mission, Vision, Values
▫ Strategy Map
▫ Measures
▫ UX & Assessment program
![Page 12: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/12.jpg)
Comparative Analysis
12
We conducted a comparative analysis of the MSU UX&A program vis-à-vis the assessment framework described in the ARL Assessment Ecosystem.
![Page 13: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/13.jpg)
13
![Page 14: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/14.jpg)
Evaluating MSU practice via the ARL Assessment Ecosystem Framework
ARL Framework Component
Self-assessment prompts Established Emerging Non-existent
WCAG 2.0 / Section 508 Accessibility Standards
Has the institution conducted an accessibility audit in the last three years?
X
Parent Entity's User Data
Student data andFaculty data
X
Parent Entity Technology Infrastructure
The university has a technology plan that supports its strategy and assessment goals.
X
14
![Page 15: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/15.jpg)
15
Framework components established at the MSU Library
![Page 16: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/16.jpg)
16
Framework components emerging at the MSU Library
![Page 17: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/17.jpg)
17
Framework components non-existent at the MSU Library
![Page 18: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/18.jpg)
GapAnalysis
18
We conducted a gap analysis comparing the ARL recommendations with established and emerging user experience and assessment programs in place at other research libraries.
![Page 19: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/19.jpg)
▫ Motivations▫ Methods ▫ Results_▫ Reflections ▫ Recommendations
19
![Page 20: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/20.jpg)
Results- What helps success happen?
- Leadership support
- Access to data
- Communication
- Integration of assessment into units throughout the organization
20
![Page 21: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/21.jpg)
▫ Motivations▫ Methods ▫ Results▫ Reflections_ ▫ Recommendations
21
![Page 22: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/22.jpg)
Reflections- Identify areas of strength and growth
- Dialogue around the challenges and opportunities of our particular program
- Recognize and react to our specific, local, contextual needs and goals
22
![Page 23: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/23.jpg)
▫ Motivations▫ Methods ▫ Results▫ Reflections ▫ Recommendations_
23
![Page 24: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/24.jpg)
24
![Page 25: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/25.jpg)
Recommendations
25
![Page 26: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/26.jpg)
Recommendations
26
![Page 27: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/27.jpg)
Recommendations
27
![Page 28: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/28.jpg)
Recommendations
28
![Page 29: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/29.jpg)
Recommendations
29
![Page 30: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/30.jpg)
30
![Page 31: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/31.jpg)
How can we critically reflect on our own evolving assessment practice?
31
![Page 32: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/32.jpg)
...by comparing our practice to the ARL Assessment Framework and to other peer programs.
32
![Page 34: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/34.jpg)
34
![Page 35: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/35.jpg)
35
Appendix
![Page 36: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/36.jpg)
36
![Page 37: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/37.jpg)
37
![Page 38: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/38.jpg)
38
![Page 39: The ARL Assessment Framework in Practice at Montana State ...€¦ · Gap Analysis 10 What are peer programs doing? Case Study 11 UX and Assessment at Montana State University Strategic](https://reader033.vdocuments.us/reader033/viewer/2022043002/5f7b611a3686a076516daed5/html5/thumbnails/39.jpg)
39