institutional effectiveness: bite-s jargon free … · about faculty, staff, and most importantly...

61
I NSTITUTIONAL E FFECTIVENESS : B ITE - S IZED AND J ARGON F REE (M OSTLY ) CUNY MSCHE C OUNCIL W ORKSHOP M ARCH 1, 2019 Christopher Shults, Ph.D Dean of Institutional Effectiveness and Strategic Planning MSCHE Accreditation Liaison

Upload: others

Post on 19-Jul-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

INSTITUTIONAL EFFECTIVENESS: BITE-SIZED AND JARGON FREE (MOSTLY)

CUNY MSCHE COUNCIL WORKSHOP

MARCH 1, 2019

Christopher Shults, Ph.D

Dean of Institutional Effectiveness and Strategic Planning

MSCHE Accreditation Liaison

Page 2: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULE

9:00 – 9:10 Introduction and Self-Study Lessons

9:10 – 10:10 Guide to Institutional Effectiveness/IE and Standards

10:10 – 10:15 Break

10:15 – 12:00 Central State College/IE Compliant Planning

12:00 – 12:45 Lunch and Examples

12:45 – 2:00 IE Compliant Assessment

2:00 – 2:05 Break

2:05 – 2:20 IE Compliant Resource Allocation

2:20 – 2:45 Develop a Draft IE Framework

2:45 – 3:00 Discussion and Q&A

Page 3: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULE

Ready!

Set!

Here we Go!

Page 4: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

LESSONS LEARNED

Criticality of Evidence

Expectations Regarding Assessment

Programmatic

AES

Expectations Regarding Planning

Strategic

Operational

Institutional

Expectations Regarding Institutional Effectiveness

System

Integrated

Evaluated

Institutional Effectiveness Impacts ALL Standards

Expectation of Mature, Evaluated Systems

Accountability and Responsibility

Page 5: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

INSTITUTIONAL EFFECTIVENESS JARGON

Eschew Obfuscation, Articulate Coherently

Basically – Avoid Jargon

Page 6: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

INSTITUTIONAL EFFECTIVENESS JARGON BINGO

Page 7: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULEINTRODUCTION TO INSTITUTIONAL EFFECTIVNESS (IE)

Achievement of Mission

What do we Do?

Who do we Serve?

What do we Provide?

Proxies for Achievement

Missions are not Measurable

Institutional Goals/Strategic Goals

S.M.A.R.T.

How do we Get to Institutional?

Academic Programs

AES Units

Planning

Resource Allocation

Why are we Doing this?

Student Learning and Success

Prioritization/Resource Allocation

Sense making/Direction Setting

Accreditation

Page 8: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

What IE IsMission Oriented

About Faculty, Staff, and most importantly Students

Measurable by Proxy

Mandate

Contextual and Multidimensional (Jargon Alert!)

What IE is notAssessment

Evaluation

A Metric

A KPI

An Isolated Construct

Limited to a Report, Office, Function, or Mandate

Information about DefinitionsImportance Emerged after Spellings Commission

Literature is Primarily in Community Colleges

Regional Bodies (MSCHE) don’t offer Definitions

INTRODUCTION TO INSTITUTIONAL EFFECTIVNESS (IE)

DEFINITION AND IMPLEMENTATION OF IE AT THE DISCRETION OF

INDIVIDUAL COLLEGES

Page 9: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

King University (TN)

The institutional effectiveness program at King University is one of demonstrating and documenting the

extent to which the College fulfills its mission and achieves its goals as stated in the strategic plan [and]

promotes institutional quality through a comprehensive, institution-wide, data-driven continuous cycle of

planning, implementation, assessment, and improvement.

Carteret Community College (NC)

Institutional effectiveness (IE) at Carteret Community College is a systematic, integrated, and ongoing

process of planning and assessing institutional performance through regular review of the college’s mission,

goals, and outcomes; evaluation of programs and services; assessment of outcomes across all institutional

units; and use of data and assessment results to inform decision-making.

IE DEFINITIONS

Page 10: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE DEFINITIONS

Ogeechee Technical College (GA)

Institutional Effectiveness at Ogeechee Technical College is a continuous, comprehensive, and integrated

system of analysis, planning, implementation, assessment, and application of the results, designed to

demonstrate the progress of the College in fulfilling its stated mission. This continuous, comprehensive and

integrated approach to improving quality also involves monitoring local, state, and federal budgeting.

Institutional Effectiveness employs a system that ensures plans are implemented and evaluated with emphasis

on decision-making and changes as a result of the evaluation findings. In addition, research is conducted and

results are utilized in the planning and evaluation process.

Central Piedmont Community College (NC)

Institutional effectiveness at Central Piedmont Community College consists of a set of ongoing and

systematic, institutional processes and practices that include planning, the evaluation of programs and

services (including administration and student services), the identification and measurement of outcomes

across all institutional units (including learning and program outcomes in instructional programs), and the

use of data and assessment results to inform decision-making (culture of evidence). All of these activities are

accomplished with the purpose of improving programs and services and increasing student success and

institutional quality.

Page 11: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

INTRODUCTION TO INSTITUTIONAL EFFECTIVNESS (IE)

INSTITUTIONAL EFFECTIVENESS, AT ITS CORE, IS ABOUT DOCUMENTING EVIDENCE THAT THE COLLEGE IS

PROGRESSING TOWARDS ACHIEVEMENT OF ITS MISSION. MAKING THE CASE FOR ACHIEVEMENT OF THE MISSION

REQUIRES NOT ONLY A PLANNING-BASED MISSION STATEMENT, BUT ALSO GOALS THAT ARE DERIVED FROM THE

STATEMENT. THESE INSTITUTIONAL GOALS ACT AS PROXIES FOR ACHIEVEMENT OF THE MISSION.

THE DEGREE TO WHICH BMCC IS ACHIEVING ITS MISSION AS EVIDENCED BY YEARLY PROGRESS MADE

TOWARDS THE ACHIEVEMENT OF ITS INSTITUTIONAL GOALS, STRATEGIC PLANNING OUTCOMES, AND STRATEGIC

OBJECTIVES. THE INSTITUTIONAL EFFECTIVENESS SYSTEM, WHICH ALLOWS FOR THE TRACKING OF PROGRESS, IS

AN INTEGRATED PLANNING MODEL THAT INCORPORATES COMPREHENSIVE AND SYSTEMATIC ASSESSMENT,

PERIODIC EVALUATION, EFFECTIVE PLANNING PROCESSES, AND DATA INFLUENCED RESOURCE ALLOCATION.

Page 12: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

INTRODUCTION TO INSTITUTIONAL EFFECTIVNESS (IE)

Page 13: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

INTRODUCTION TO INSTITUTIONAL EFFECTIVNESS (IE)

Page 14: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

INTRODUCTION TO INSTITUTIONAL EFFECTIVNESS (IE)

Page 15: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

INTRODUCTION TO INSTITUTIONAL EFFECTIVNESS (IE)

Page 16: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULEINTRODUCTION TO INSTITUTIONAL EFFECTIVNESS (IE)

Page 17: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

DEVELOP A DEFINITION OF INSTITUTIONAL EFFECTIVENESS

Based on what you have heard so far in regards to institutional

effectiveness, its time to craft your own, unique description of IE at your

college. Consider the following while crafting it:

Institutional Mission

Assessment of Programs and Services

Planning Efforts

Resource Allocation

Continuous Improvement

System and Support

Page 18: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE AND THE MIDDLE STATES STANDARDS

History of Regional Accreditors and IE

Recent Changes to Accrediting StandardsACJCC - 2014

HLC – 2018 (Alpha Version)

NEASC - 2016

NWCCU – (N.D.)

SACS - 2017

WASC - 2014

Middle States Commission on Higher Education2004 – 14 Standards

Standard 2 – Planning, Resource Allocation, and Institutional Renewal (Planning, Resources, Improvement)

Standard 3 – Institutional Resources (Resources)

Standard 7 – Institutional Assessment (Systematic Assessment/Evaluation, Planning, Data and Reporting)

Standard 8 – Student Admissions and Retention (AES Assessment/Evaluation)

Standard 9 – Student Support Services (AES Assessment/Evaluation)

Standard 12 – General Education (Gen. Ed. Assessment)

Standard 13 – Related Educational Activities (Programmatic Assessment)

Standard 14 – Assessment of Student Learning (Course Assessment, Program Assessment)

Institutional Effectiveness was there, but not Reviewed as a Whole. Standard 7 was the Assessment of

Institutional Effectiveness, but not Necessarily the Expectation of an Explicit System, Plan, or Report

Page 19: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE AND THE MIDDLE STATES STANDARDS

Middle States Now7 Standards – 5 Explicitly Connected with IE, 1 Indirectly

Distinction between Institutional Context and Effectiveness gone

No longer an Institutional Assessment Standard

Assessment integrated into all Standards

Effectiveness mentioned 11 times

Old Requirement of Affiliation #7

Institutional planning integrates plans for academic, personnel, information resources and

technologies, learning resources, and financial development

Requirement of Affiliation #10

Institutional planning integrates goals for academic and institutional effectiveness and

improvement, student achievement of educational goals, student learning, and the results of

academic and institutional assessments. (NOTE the Expectation for an IE Plan)

Institutional Effectiveness is now being Viewed as a Foundation for Accreditation. All of the

Elements are Required in the Standards and Alignment is no Longer Sufficient. Demonstration

of a Planning System that is Reviewed and Modified is Expected. Expectations for IE have

Evolved.

Page 20: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE AND MSCHE STANDARD I

The Foundational Standard for IE

About Institutional Mission and Goals

Focus on the Uniqueness of the Institution

Institutional Goals are EssentialExplicit Connection to Mission?

Alignment of Strategic Goals?

Development, Evaluation, and Revision of Mission and Goals

What Documents are Essential to Compliance with the Standard,

from an IE Perspective? What Processes and Procedures need to be

Addressed?

Page 21: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE AND MSCHE STANDARD III

Focus on the Learning Experiences

Education Effectiveness

Scope of the Mission

Academic Programs/General Education

AES Units

Planning ProcessesStrategic Plan?

Academic Plan?

Resource Allocation/Support

Assessment of Effectiveness

What Documents are Essential to Compliance with the Standard,

from an IE Perspective? What Processes and Procedures need to be

Addressed?

Page 22: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE AND MSCHE STANDARD IV

Focus on Supporting StudentsEducational Support Units

Student Support Units

Administrative Support Units (IR/IE)

Missions, Goals, and Outcomes

Alignment and Incorporation of PlansEnrollment Management Plans

Retention and Completion Plans

AES Assessment Plans

Alignment and Incorporation of EffortsEnrollment Management Committee

AES Assessment/Planning Committee

What Documents are Essential to Compliance with the Standard,

from an IE Perspective? What Processes and Procedures need to be

Addressed?

Page 23: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE AND MSCHE STANDARD V

Focus on Educational GoalsInstitutional Level

Department Level (if appropriate)

Program Level

Linkage to Institutional Mission and Goals

Focus on Student LearningCourse Learning Outcomes

Program Learning Outcomes

General Education Learning Outcomes

Cohort Learning Outcomes

Focus on Student Learning Outcomes SystemAssessment Processes

Assessment Support

Use of Assessment Results

Integration of Efforts

What Documents are Essential to Compliance with the Standard,

from an IE Perspective? What Processes and Procedures need to be

Addressed?

Page 24: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE AND MSCHE STANDARD VI

Focus on Resource AllocationBudget Structure and Sources (all revenue streams)

Budget Processes and Procedures

Budget Alignment to Operational Priorities

Alignment between Budget and Planning

Focus on PlanningStrategic Planning Process, Plans, and Reporting

Alignment of Institutional Planning Efforts (Strategic, Operational, Long-Range)

Academic Program and AES Unit Planning Efforts

Use of data in Planning

What Documents are Essential to Compliance with the Standard,

from an IE Perspective? What Processes and Procedures need to be

Addressed?

Page 25: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE AND MSCHE STANDARD VII

Focus on Accountability for IE EffortsResponsibility for Assessment

Responsibility for Planning

Responsibility for Resource Allocation

Assessment, Planning, and Resource Allocation within Affiliated Entities

Appropriate Support for IE EffortsSupport Structure in Place (Assessment, Planning, and Resource Allocation)

Appropriateness of those Accountable

Demonstration of Institutional Commitment to IE (Funding Sources, Org. Chart)

Evidence of Support in Planning Documentation

What Documents are Essential to Compliance with the Standard,

from an IE Perspective? What Processes and Procedures need to be

Addressed?

Page 26: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)
Page 27: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

CENTRAL STATE COLLEGE

“THE EDUCATIONAL AND ECONOMIC HEART OF THE CENTRAL VALLEY”

Hypothetical Construct College

INTRODUCTION TO CENTRAL STATE

Page 28: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

Mission and GoalsPlanning-Based, not Marketing

Unique to the College or University

Institutional Goals that are Developed from the Mission

Institutional Goals that can Operate as Proxy Measures for Progress

Both Developed with Significant Internal and External input

Strategic PlanningAppropriate to Higher Education and the Institution

Significant Involvement from the College Community and Stakeholders

Strategic Goals Aligned with Institutional Goals

Metrics (Objectives/Outcomes) Grouped within Goals (i.e. KPIs)

Annual Comprehensive Reporting on Progress

Operational PlanningEvidence of Activities Guided by the Strategic Plan

Involvement by Academic Programs and AES Units

Annual Reporting Process

Long-Range PlanningCreate Continuity and Relevance for Divisions

IT, SEM, Development, PR, Student Affairs, Academic Plan, etc.

IE COMPLIANT PLANNING

Page 29: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULEIE COMPLIANT PLANNING

Planning’s Reliance on AssessmentWhat Direction/Decisions Make Sense?

Modification? Continuation? Evolution?

Planning’s Impact on AssessmentDevelopment of the Appropriate Benchmarks/Metrics

Help Prioritize what is being Assessed

Planning’s Impact on Resource AllocationsStrategic Planning Process, Plans, and Reporting

Alignment of Institutional Planning Efforts (Strategic, Operational, Long-Range)

Academic Program and AES Unit Planning Efforts

Use of data Gathered through Institutional Planning

Planning’s Reliance on Resource AllocationEnsure Resources are Available to Pursue Use of Results

When Considered Broadly, Provides Strategic Options

Page 30: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT PLANNING

Standard IMaintain Processes and Procedures Relevant to Mission and Goals

Documented, systematic approach to evaluation (Plan to Plan)

Collaborative process to review and revise

Evidence of usage for decision-making related to planning

Ensure that the Mission and Goals are Reflective of the Institution (Alignment)

Ensure Mission and Institutional Goals are Focused on Student Learning/Supported by AES Units

Alignment of academic program and AES missions to institution

Alignment of academic program and AES goals to institution

Standard IIIMaintain Processes and Procedures Relevant to Design of the Student Learning Experience

Institutional structure for utilizing results for planning in academic programs/AES units

Documentation of evidence that planning activities guide student learning activities

Standard IVMaintain Processes and Procedures Relevant to Support of the Student Experience

Assessment and evaluation of relevant institutional plans (i.e. Enrollment Management)

Institutional structure for utilizing results for planning in academic programs/AES units

Documentation of evidence that planning activities guide student learning activities

Page 31: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT PLANNING

Standard VMaintain Processes and Procedures Relevant to Educational Effectiveness Assessment

Institutional structure for utilizing assessment results for planning in academic programs/AES units

Documentation of evidence that planning occurs based on assessment results

Documentation that planning impacts academic programs and AES unit

Evidence that planned changes are evaluated

Standard IVDocumentation of an Implemented, Systematic and Sustained process that links Planning, Assessment,

and Resource Allocation

Institutional effectiveness plan

Documentation of an integrated planning approach

Institutional, program, and unit objectives

Maintain Processes and Procedures Relevant to Planning, Resources, and Institutional Improvement

Documentation of planning structures, processes, results, and use of results

Institutional structure for utilizing results for planning in academic programs/AES units

Financial and budget planning processes aligned with mission and strategic planning

Backbone/administrative planning processes aligned with mission and strategic planning

Page 32: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT PLANNING

Standard VIIMaintain Process and Procedures Relevant to Governance, Leadership, and Administration

Document and demonstrate accountability/responsibility for planning

Governance documents and processes

Clarity of structure and process in institutional documents

Page 33: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT PLANNING

Note: IE Compliant Planning Begins with the Strategic Plan

To Ensure there is a Framework for Institutional Effectiveness, you need to have

Something to Measure Against. Remember, Institutional or Strategic Goals are not, in

and of Themselves, Measurements. They are Proxies for Mission Achievement. A

College Needs Objectives to Strive Towards. At this Point, Please use the Planning

Template to Come up with new Objectives as well as Key Performance Indicators to

Ensure Measurability. We will be using these Objectives and Indicators for Assessment

in the Afternoon.

Page 34: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

ENJOY!

+

Page 35: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

Detail the Role of AssessmentInstitutional Philosophy

Connection to Institutional Improvement

Definitions

Locations and Levels of Assessment

Detail the Responsibilities in AssessmentRole of the Students

Role of Faculty

Role of Staff

Role of Institutional and Departmental Administration

Role of Governance

IE COMPLIANT ASSESSMENT

Page 36: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

Assessment is a recurring process of inquiry and improvement in which clearly articulated

SLOs and SOs, aligned with appropriate institutional, program, and unit missions and goals,

are measured against pre-established performance criteria. Assessment results may meet or

exceed expectations, fall short in some way, or uncover unanticipated learning or unexpected

outcomes. Disparities between performance expectations and actual assessment results form

the basis for dialogue and possible action. It is important to note that missed targets don’t

equate to failure. The only way to fail at assessment is to fail to conduct the assessment or

utilize the results. When the results of assessment activities are used to bring about

improvement in teaching or learning or in support of the learning environment, it reflects the

appropriate use of results for improvement.

BMCC Institutional Effectiveness Plan

IE COMPLIANT ASSESSMENT

Page 37: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

Impact of Assessment on PlanningThe Use of Results

Addresses the Findings with Action

Impact of Planning on AssessmentProvides an Opportunity for Follow-Up

Positive Results vs. Negative Results

Impact of Assessment on Resource AllocationHelps Determine if Resources were Deployed Properly

Can Provide Insight into Appropriateness of Resource Allocation Processes

Impact of Resource Allocation on AssessmentEnsure Resources are Available to Pursue Use of Results

When Considered Broadly, Provides Strategic Options

IE COMPLIANT ASSESSMENT

Page 38: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT ASSESSMENT

Standard IMaintain Process and Procedures Relevant to Mission and Goals

Use of results need to be guided by the institutional goals

Use of results need to be guided by the outcomes/objectives

Ensure that the Institutional Goals Focus on Student Learning

Ensure that the Institutional Goals are Supported by AES Units

Periodic Assessment of Mission and Goals

Standard IIIMaintain Process and Procedures Relevant to Design of the Student Learning Experience

Academic experts in charge of direct student learning

Content experts in charge of indirect student learning

Maintain and Evaluate the General Education Program

Maintain an Appropriate Process for Program/Couse Creation

Page 39: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT ASSESSMENT

Standard IVMaintain Process and Procedures Relevant to Support of the Student Experience

How Determine the Effectiveness of Admissions Processes

How Determine the Effectiveness of Retention Processes

How Determine the Effectiveness of Facilitating Student Success

What Units are Directly Associated with Co and Extra-Curricular Activities

Standard VDocumentation of Systematic, Sustained Process to Assess Student Learning at all Levels

Systematic, sustained (not periodic) assessment of ILOs, PLOs, and CLOs

Systematic, sustained (not periodic) assessment of ILO/PLO

Systematic, sustained (not periodic) assessment of PLO/CLO

Appropriate, structured communication of results to appropriate parties

Documentation of Systematic, Sustained Process to Utilize Results of Assessments

Documentation that educational effectiveness has been improved

Documentation that SLOs at all levels have been impacted by assessments

Documentation of changes to curriculum and/or pedagogy

Documentation of Use of Results in the APR/Unit Review Process

Evidence that academic programs/AES units have been reviewed and revised

Evidence that professional development has been addressed adequately

Resource availability has been addressed

Page 40: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT ASSESSMENT

Standard VIIMaintain Process and Procedures Relevant to Governance, Leadership, and Administration

Document and demonstrate accountability/responsibility for assessment

Governance documents and processes

Clarity of structure and process in institutional documents

Page 41: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT ASSESSMENT – DEVELOPING GOALS

WHAT UNIT GOALS ARE

• Statements about the Day to Day Functions

• Based on the Consensus of Faculty and Staff within the Units

• Clearly Associated with Institutional/Strategic Goals

• Firmly Rooted in the Unit Mission

• Guidelines for Individuals Located Outside the Unit

• Primary Indicator of Effectiveness

WHAT UNIT GOALS ARE NOT

• Annual Objectives

• Outcomes

• Substitutes for the Unit Mission

• The Primary Unit for Measurability

• Proxies for Individual Performance

• Substitutes for Individual Performance Goals

Page 42: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT ASSESSMENT – DEVELOPING GOALS

Institutional

Mission

Institutional

Goals

Unit Mission

Unit Goals

= Derived from

= Aligned with

Student Learning

Outcomes

Specific to the unit

Derived from the goals

Measure of how a goal will be achieved

Details expectations of students

More detailed than goals

Support

Outcomes

Specific to the unit

Derived from the goals

Measure of how a goal will be achieved

Details expectations of the support provided

More detailed than goals

Specific to the unit

Anchored in the mission

Overarching achievement tied to the purpose

Frames the functions of the unit

It bridges the mission and support outcomes

Page 43: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULEIE COMPLIANT ASSESSMENT - GOALS

Now that we have Discussed AES Unit Goals, its time to Develop a Few of your

Own. You are now in the Position of Director of an AES Unit at CSC. It is time

to take some time to Develop some Goals that are Informed by the College’s

Mission, Aligned with the Institutional Goals, and Derived from the Unit

Mission.

Page 44: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT ASSESSMENT - OUTCOMES

OUTCOMES ARE

• Unique to the Unit

• Derived from the Goals

• A Measure of how a Goal will be achieved

• More Detailed than the Goals

• Must be Measurable

STUDENT LEARNING OUTCOMES

• Direct Indicators of Student Learning

• Locus of Exploration is the Student

SUPPORT OUTCOMES

• Indirect Indicators of Student Learning

• Direct Indicators of Support for the Learning Environment

• Locus of Exploration is the Unit

Page 45: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT ASSESSMENT - OUTCOMES

STUDENT LEARNING OUTCOMES ARE:

• Detailed Expectations of Students

• Knowledge

• Skills

• Disposition

• About the Impact of the Unit on Students

SUPPORT OUTCOMES ARE:

• Detailed Expectations of Support

Provided by Units

• Focused on the Delivery of Services,

Processes, Activities, or Functions to

Students, Faculty, or Staff

• About the Nature of the Unit

Page 46: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT ASSESSMENT - OUTCOMES

Institutional

Mission

Institutional

Goals

Unit Mission

Unit Goals

= Derived from

= Aligned with

Specific to the unit

Based on mandates

Driven by best practices

Answers the major questions

Student Learning

Outcomes

Support

Outcomes

• Specific to the unit

• Anchored in the mission

• Overarching achievement tied to the purpose

• Frames the functions of the unit

• It bridges the mission and support outcomes

• Specific to the unit

• Derived from the goals

• Measure of how a goal will be achieved

• Details expectations of students

• More detailed than goals

• Specific to the unit

• Derived from the goals

• Measure of how a goal will be achieved

• Details expectations of the support provided

• More detailed than goals

Page 47: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT ASSESSMENT

[Student Learning] outcomes must be expressed in terms of demonstrable characteristics. Since

the learner’s performance ought to be observable and measurable, the verb chosen for each

outcome statement should result in overt behavior that can then be observed and measured (see

Bloom’s Taxonomy ). Outcomes focus on what students [impacted by the division] can

demonstrate rather than on what [the staff of the division or department delivers].

[Support] outcomes must be expressed in terms of demonstrable characteristics. [In other words,

the goal speaks to the functions carried out within the division while the outcomes describes how

the function is delivered or what the effect of the delivery will be].

Adapted from the Indiana University Southeast Assessment Manual

Page 48: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT ASSESSMENT

Now that you have Developed some AES Unit Goals, it is Time to Move onto

the Development of the Outcomes that will be Used for Assessment. Please use

the Bloom’s Taxonomy and Verb Wheel to Develop SLOs and the SDW

Taxonomy and Verb Wheel to Develop SOs. Make sure that the Outcomes are

Informed by the Unit Mission, Aligned, where possible, with the Institutional

Goals, and Derived from the Goals.

Page 49: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)
Page 50: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULEIE COMPLIANT RESOURCE ALLOCATION

Budgeting, the Budget Process, and Resource Allocation

Conscientious Use of Results to Allocate Resources

Strategic Redeployment/New Resources

Planning should Guide Budgeting, Where Appropriate

Documentation of Efficient Use of Resources

Document Resources to Achieve the Mission

Page 51: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULEIE COMPLIANT RESOURCE ALLOCATION

While the budget process is central to the resource allocation process, it is not inclusive. In

fact, resource allocation is as much about redeployment of existing resources to ensure

greater student success. Resources can and often do reflect money; however, people, time,

and systems are important resources whose impact should not be underappreciated. As a

result of conducting assessments or evaluating the impact of strategic activities, units may

determine during the planning process that the results necessitate a need for either

redeployment of existing or the creation of new resources. The assessment and planning

cycle has been aligned with the institutional budget cycle so that department, unit, and

division leaders can utilize the information to make a data-influenced request.

BMCC Institutional Effectiveness Plan

Page 52: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULEIE COMPLIANT RESOURCE ALLOCATION

Interaction between Resource Allocation and AssessmentEnsure Resources are Available to Pursue Use of Results

When Considered Broadly, Provides Strategic Options

Interaction between Resource Allocation and PlanningBudget Alignment to Operational Priorities

Alignment between Budget and Planning

Page 53: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

IE COMPLIANT RESOURCE ALLOCATION

Standard IMaintain Process and Procedures Relevant to Mission and Goals

Mission that guides faculty staff, administration, and governance in resource allocation

Goals that guide faculty, staff, administration, and governance in resource allocation

Ensure that the Institutional Goals Focus on Student Learning

Ensure that the Institutional Goals are Supported by AES Units

Periodic Assessment of Mission and Goals

Standard IIIMaintain Process and Procedures Relevant to Design of the Student Learning Experience

Ensure resource allocation process that provides sufficient resources to academic programs

Ensure resource allocation process that provides sufficient resources to AES units

Standard VMaintain Process and Procedures Relevant to Support of the Student Experience

Ensure resource allocation process that provides sufficient resources to academic programs

Ensure resource allocation process that provides sufficient resources to AES units

Page 54: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULEIE COMPLIANT RESOURCE ALLOCATION

Standard IVDocumentation of an Implemented, Systematic and Sustained process that links Planning, Assessment,

and Resource Allocation

Institutional effectiveness plan

Documentation of an integrated planning approach

Institutional, program, and unit objectives

Maintain Processes and Procedures Relevant to Planning, Resources, and Institutional Improvement

Documentation of comprehensive resource allocation processes and use of results

Institutional structure for utilizing results for resource allocation in academic programs/AES units

Comprehensive financial processes aligned with mission and strategic planning

Documentation of assessment and evaluation of the comprehensive financial process

Documentation of a process where fiscal and human resources are directed toward mission and

strategic plan achievement

Documentation of a process where physical and technological resources are directed toward

mission and strategic plan achievement

Standard VIIMaintain Process and Procedures Relevant to Governance, Leadership, and Administration

Document and demonstrate accountability/responsibility for resource allocation

Governance documents and processes

Clarity of structure and process in institutional documents

Page 55: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULEIE COMPLIANT RESOURCE ALLOCATION @

Page 56: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULEIE COMPLIANT RESOURCE ALLOCATION

Three Institutional Areas – Academic Affairs, Student Affairs, B&F

Information Provided in Budget Call Letters

Budget Request Forms (Alignment with Strategic Plan)

Continuing Allocations

New Allocations

Use of Assessment Results

Annual Assessments

Academic Program Reviews

AES Unit Reviews

Strategic Deployment of Resources

Grants

Research Foundation

Development Office

Auxiliary Funds

Continual Evolution

Page 57: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULEDIY IE MODEL DEVELOPMENT

You have Spent Most of Today Learning about IE. We have Covered Definitions,

MSCHE Criteria and Requirements, Elements of IE, and the Required Alignment. You

have also worked on CSC’s Strategic Plan and established AES Unit Goals and

Outcomes. Now it is time to put this Knowledge to Work. Please use the Sheet Provided

to Consider what an IE Model Could look like at your Institution. Make sure you

Account for your Unique Institutional Structure and Context.

Page 58: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULEWORKSHOP TAKEAWAYS

Institutional Effectiveness is about the Mission

All Proxies need to have Measurability

There is no MSCHE Definition, but there are Indicators

Colleges have the Elements – Its about Integration, Process and Structure

Assessment is an Element of IE, but it is not IE

Integration of Assessment, Planning, and Resource Allocation is Key

To Leverage IE, you need a Model, Plan, and System

To be Meaningful, IE must be Relevant to the College

Page 59: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULE

Page 60: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULE

Christopher Shults ([email protected])

LinkedIn: Christopher Shults

Twitter: @infamous_PhD

Association for Higher Education Effectiveness (AHEE) www.ahee.org

CONTACT INFORMATION

Page 61: INSTITUTIONAL EFFECTIVENESS: BITE-S JARGON FREE … · About Faculty, Staff, and most importantly Students Measurable by Proxy Mandate Contextual and Multidimensional (Jargon Alert!)

TODAY’S SCHEDULECUNY MSCHE COUNCIL THANKS YOU!