mia alexander-snow, phd planning & institutional effectiveness tara rose, mpa assessment roger...
TRANSCRIPT
![Page 1: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/1.jpg)
UK’s Program Review Orientation:The Role of Planning, Institutional
Research & Assessment
Mia Alexander-Snow, PhD
Planning & Institutional EffectivenessTara Rose, MPA
AssessmentRoger Sugarman, PhDInstitutional Research
![Page 2: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/2.jpg)
2
Program Review Orientation Overview
Institutional Effectiveness & AssessmentProgram review overviewProgram review schedule & current processEducational units participating in 2012-2013
cycleProgram review components: self study,
external review, and implementation planHow does the office of the Vice President for
Institutional Effectiveness inform Periodic Program Review?
![Page 3: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/3.jpg)
3
What is institutional effectiveness?
“The institution engages in ongoing, integrated, and institution-wide research-based planning and evaluation processes that incorporate a systematic review of programs and services that (a) results in continuing improvement and (b) demonstrates that the institution is effectively accomplishing its mission” (SACS Core Requirement 2.5).
![Page 4: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/4.jpg)
4
What is assessment at UK?
“Assessment is used to monitor the University’s effectiveness in achieving its mission, vision, and goals. The University and its units shall demonstrate an explicit use of assessment results to facilitate resource allocation and budgeting decisions in support of their strategic plans and to ensure quality enhancement” (UK AR 1:4)
![Page 5: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/5.jpg)
What assessment activities monitor UK’s Institutional Effectiveness efforts?
Strategic Planning
• Identifies and prioritizes the actions the University and its units can take to help it best accomplish the University’s goals and fulfill its mission (AR 1:4)
5
![Page 6: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/6.jpg)
6
What assessment activities monitor UK’s Institutional Effectiveness efforts?
Annual Progress Reporting
Dynamic process for reviewing, updating and revising strategic planning efforts over a 3-5 year period
Answers the following questions in a systematic and thorough way :“How are we doing? ---Actual Results“What things are working? ---Reflection and Analysis“What needs to happen next?” ---Improvement Action
![Page 7: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/7.jpg)
What assessment activities monitor UK’s Institutional Effectiveness efforts?
6 yr Periodic Program Review : the primary vehicle for assessment of educational and administrative units and for documentation of institutional effectiveness (AR 1:4). • 254 Units participate in Program Review• 113 Administrative and Educational Support units• 18 Colleges and Schools• 95 Academic departments ( 365 Academic degree &
certificate programs)• 28 Research Centers & Institutes
o As of 7/24/12—reported by UK registrar’s 7
![Page 8: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/8.jpg)
8
What are the purposes of program review?
Program Review Purposes Planning Assessment Budget Other
University •Governing Regulations I-X•Administrative Regulations•University Senate Rules
√ √ √ IE
State (Legislature, CPE) √ √
SACS IE
External Constituencies (e.g., Accrediting Agencies)
√
![Page 9: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/9.jpg)
9
The Role of the Office of Planning in Program Review
Planning
Student Learning
Institution
al Research
![Page 10: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/10.jpg)
10
What is UK’s Program Review Schedule?Schedule: http://www.uky.edu/IRPE/ie/program_review.html
Purpose: communicate to organizational entities the full 6-yr
review cycle and when units can expect to undergo program review
Goals: provide the transparent and accurate maintenance of
the review schedule for the university’s educational and administrative units; and
monitor unit progress
![Page 11: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/11.jpg)
11
What units are scheduled for the 2012-2013 Program Review cycle?College of Agriculture
2 departments (4 degree programs)/ college-wide reviews
College of Arts & Sciences3 Departments (10 degree programs)
College of DentistryCollege-wide/2 degree programs
College of Public Health1 Research Center
College of Medicine 1 Center/12 Clinical Units & Departments
![Page 12: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/12.jpg)
12
What units are scheduled for the 2012-2013 Program Review cycle?
College of AgricultureCommunity & Leadership Development
(delayed to 2013-2014)Merchandising, Apparel & Textiles (deferred to
2013-14)College-wideHorticulture
![Page 13: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/13.jpg)
13
What units are scheduled for the 2012-2013 Program Review cycle?
College of Arts & SciencesEarth & Environmental SciencesMathematicsSociologyModern & Classical Language,
Literature ,Culture (Delayed to 2014-2015)
![Page 14: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/14.jpg)
14
What units are scheduled for the 2012-2013 Program Review cycle?
College of Medicine Basic Science Departments:
Medical Sciences Microbiology, Immunology, & Molecular Genetics (request deferral)) Toxicology
Clinical Departments: Internal Medicine (request deferral) Neurology Neurosurgery Obstetrics & Gynecology Ophthalmology & Visual Sciences Pathology & Laboratory Medicine (request deferral) Physical Medicine and Rehabilitation Radiology (request deferral)
Center: Center on Drug & Alcohol Research
![Page 15: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/15.jpg)
15
What units are scheduled for the 2012-2013 Program Review cycle?
College of Public HealthKY Injury Prevention & Research Center
![Page 16: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/16.jpg)
What are the components of UK’s program review process?
16
• Unit/Program Self –Study• (internal Review)
SWOT Analysis
• External Review• within UK• outside UK
Recommendations
• Implementation Plan
Quality Enhancement
Agenda
• Annual Progress Report
Documentation of enhancement
results & Informs Strategic
Planning & Budget
![Page 17: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/17.jpg)
17
Unit Self-Study
Part 1: CPE Reporting
CPE SACS
UK Accredited Units
A. Centrality to University Mission √ √ √ √
B. Program quality and Student Success √ √ √ √
C. Program Demand/Unnecessary Duplication
√ √ √ √
D. Resources/ Cost and Funding √ √ √ √
E. Program Review and Assessment √ √ √ √
![Page 18: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/18.jpg)
18
Unit Self-Study
Part 2: University Reporting
SACS UK Accredited Units
Executive Summary √
Description of Academic Program
√
Input from Affected Constituents
√ √ √
Adherence to Policies & Procedures
√ √ √
Evaluation of Quality and Productivity √ √ √
Service Extension & Extension Programs √ √ √Program Delivery √ √ √
Other Areas: QEP (Quality Enhancement Plan)
√ √ √
![Page 19: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/19.jpg)
What are the components of UK’s program review process?
19
• Unit/Program Self –Study• (internal Review)
SWOT Analysis
• External Review• within UK• outside UK
Recommendations
• Implementation Plan
Quality Enhancement
Agenda
• Annual Progress Report
Documentation of enhancement
results & Informs Strategic
Planning & Budget
![Page 20: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/20.jpg)
20
What is the focus of the External Review?
Examination of the self-study/accreditation report
Use of appropriate data collection techniques to assure objectivity;
Assessment of the validity of conclusions reached in self-study; and
Identification of additional strengths and recommendations for quality enhancement.
![Page 21: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/21.jpg)
21
How is the External Review conducted?
External Review Committee (comprised of members outside the unit and UK) 6-8 Appointed Members
Appointed by Dean (educational departments and degrees)
Appointed by Provost (College-wide) Majority Faculty (Educational Units)
2-faculty in same discipline or college and who are external to University
1-2 ex-officio members, appointed to support external review committee
Conducts external review, meets with unit and its leadership to discuss preliminary findings and writes report.
![Page 22: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/22.jpg)
22
How does Accreditation fit into the External Review Process?Fully Accredited Programs: Accreditation
Self-Study, Accreditation Review and Accreditation ReportMay Substitute for:
UK’s self-study UK’s External Review, and UK’s External Review Committee Report
Will NOT Substitute for: Program Review Implementation Plan
Partially Accredited Programs : Accreditation Review and Recommendation Report : May ONLY Substitute for:
External Reviewer (s) for the UK External Review
![Page 23: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/23.jpg)
What are the components of UK’s program review process?
23
• Unit/Program Self –Study• (internal Review)
SWOT Analysis
![Page 24: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/24.jpg)
24
What is the purpose of the Implementation Plan?
sets agenda for change and quality enhancement over the next 5-7 year cycle.
unit faculty, staff, and/or students under the leadership of unit head define unit agenda based on self-study and external review report/recommendations.
must be approved by unit Chief Administrative Officer (CAO).
used by unit to document future plans and resource needs for consideration in budgetary decision-making.
folded into strategic plan and annual progress reports.
![Page 25: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/25.jpg)
25
The Role of Assessment in Program Review
Planning
StudentLearnin
g
Institutional Research
![Page 26: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/26.jpg)
26
What is assessment?Assessment is the process by which …
“the institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on analysis of those results” (SACS Comprehensive Standard 3.3.1).
![Page 27: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/27.jpg)
Student Learning Outcomes Assessment Process
Student learning outcomes
Assessmentmethods
Datacollection
Programimprovement
Timeline/schedule
Faculty achieve consensus on learning outcomes to be assessed, and map outcomes to curriculum
Faculty choose assessment methods appropriate to the outcomes, purpose of assessment, and student population
Faculty develop timeline for gathering assessment data, both formative and
summative
Faculty and staff work together to gather
formative and summative
assessment data and compile for review
Faculty and staff collaborate to interpret
assessment data and develop strategies to
improve student learning outcomes
Continuous improvemen
t cycle
![Page 28: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/28.jpg)
28
How does the Office of Assessment inform program review?
Unit Self-Study? CPE SACS UK Accredited Units
E. Program Review and Assessment √ √ √ √
![Page 29: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/29.jpg)
1. Mission Statement2. Assessment Oversight3. Program-level Student learning outcomes
• Opportunity to revise student learning outcomes• Ensure the use of Overt Verbs: Bloom’s Taxonomy• Review and re-align outcomes in accordance to
university academic learning• Map Courses to Outcomes
4. Assessment Methods and Measures at Entry and Exit
Must include direct and indirect measures for each outcome Which components of the outcome will be assessed and when Be sure to include your current Program Review information
Designing a program-level Assessment Plan
![Page 30: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/30.jpg)
Designing a program-level Assessment Plan…continued
5. Data Collections When, how, what, and who
6. Data Analysis Assessment cycle and annual date for analyzing data
7. Teaching Effectiveness – NEW Identify measures of teaching effectiveness Efforts to improve Plans to evaluate students post-graduate success
8. Using Assessment data for continuous Improvement
Annual reporting and reflections
9. Appendices Include report forms, curriculum maps, rubrics, other important
materials
![Page 31: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/31.jpg)
31
The Role of Institutional Research in Program Review
Planning
Student Learning
Institutional Resear
ch
![Page 32: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/32.jpg)
32
How does IR inform Program Review? What are the must
haves?CPE SACS UK Accredite
d Units
B. Program quality and Student Success √ √ √ √
C. Program Demand/Unnecessary Duplication
√ √ √ √
D. Resources/ Cost and Funding √ √ √ √
![Page 33: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/33.jpg)
33
What IR sources inform Program Review?Student-Related Data Freshman Profile (ACT Scores & High School
GPAs)Headcount Enrollments (10 years)Degrees Conferred (10 years) Retention and Graduation Rates Surveys
SeniorAlumniNSSE Campus Climate
![Page 34: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/34.jpg)
34
What IR sources inform Program Review?Faculty-Related Data
Course History Information (total enrolled, # of sections)
Student Credit Hours (by semester and course level)
Teacher/Course Evaluations
Instructional Faculty Salaries (by CIP code, rank and gender)
![Page 35: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/35.jpg)
35
What other institutional data sources inform Program Review?
Other SourcesThe College Profile Reports (on IR website)
The Graduate School
Annual Statistical Report
Doctoral Placement Report
GRE Report
National Research Council Report
Office of Sponsored Projects Administration
www.research.uky.edu/ospa/info/spifi.htmlSources
![Page 36: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/36.jpg)
36
Institutional Data Sources Institutional data compiled by the UK Office of
Institutional Research can be found at:
www.uky.edu/IRPE/ir.html
www.uky.edu/IRPE/ie/supportdata.htmlwww.uky.edu/IRPE/colleges.html
![Page 37: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/37.jpg)
37
Questions?
What is the Program Review Calendar?
*Calendar:
Purpose:
communicates steps and timeline for completing program review; and
ensures timely completion
*Refer to educational unit calendar for 2012-2013 cycle: http://www.uky.edu/IRPE/ie/program_review.html
![Page 38: Mia Alexander-Snow, PhD Planning & Institutional Effectiveness Tara Rose, MPA Assessment Roger Sugarman, PhD Institutional Research](https://reader035.vdocuments.us/reader035/viewer/2022062314/56649dc75503460f94abb8af/html5/thumbnails/38.jpg)
38
Contact InformationMia Alexander-SnowOffice for Planning & Institutional Effectiveness Website: http://www.uky.edu/IRPE/ie.html
Tara RoseOffice of AssessmentWebsite: http://www.uky.edu/IRPE/assessment.html
Roger SugarmanOffice of Institutional ResearchWebsite: http://www.uky.edu/IRPE/welcome.html