ASSESSMENT OF MISSION AT CHRISTIAN COLLEGES AND
UNIVERSITIES
HOW DO WE KNOW WE ARE ACHIEVING OUR MISSIONS?
ASBURY COLLEGE
GAY H. HOLCOMB, PH.D.Director of Institutional Research and Assessment
QUALITY ENHANCEMENT PLAN (QEP)
Recent requirement of SACS Focus on a learning outcome central to the
mission of the institution Become intentional about a critical
element of its mission. Broad-based involvement & support Commit significant resources, able to
sustain Topic that institution becomes known for
ASBURY’S QEP ON SPIRITUAL VITALITY
Academic Excellence & Spiritual Vitality
Cornerstone Project Holiness Scripture Stewardship Mission
STUDENT LEARNING OUTCOMES1. Understand Cornerstones2. Core Curriculum 3. Personally Value Cornerstones 4. Creation Care5. Service Learning6. Christian Mission & Stewardship7. Increase Student Initiative
MEASURES1. Capstone Course Assignments2. Furnishing the Soul Inventory by Alidade 3. Case Studies of Exemplars4. Chapel Survey5. Alumni Surveys6. Co-Curricular Involvement Survey &
Participation Logs7. Waste Audits8. Mix of qualitative/quantitative & direct/indirect
AT THE HEART OF WHO ASBURY IS Enthusiastic buy-in
Each Cornerstone tied to Mission Each Stu Learning Outcome, Measure,
Achievement Target tied explicitly to Strategic Vision
Impacts everything we do Cornerstone Project approved by SACS
in Dec Initial phase – no data yet
Fully re-accredited by SACS in December 2009
BETHEL UNIVERSITY
JOEL FREDERICKSON, PH.D.Professor and Chair, Psychology Department
MINING EXISTING DATA Institutions generally participate in
national surveys (indirect assessment) and standardized tests (direct assessment)
Use this existing data to assess your institution’s mission and core values
EXAMPLES OF SURVEYS National Survey of Student Engagement
(NSSE) Cooperating Institutions Research Project
(CIRP) First-year survey (from HERI @ UCLA) College Senior Survey (CSS) Follow-up to CIRP College Student Beliefs & Values Inventory
(CSBV) from HERI Student Satisfaction Inventory (SSI)
Additional CCCU items are added to this survey.
EXAMPLES OF DIRECT ASSESSMENTS OF LEARNING
Collegiate Learning Assessment (CLA) Collegiate Assessment of Academic
Proficiency (CAAP), from ACT Major Field Tests, from ETS Defining Issues Test (DIT): Measure of
Moral Thinking California Critical Thinking Skills Test
(CCTST)
LINKING ASSESSMENTS TO MISSION Examine your institution’s mission,
vision, and, if applicable, core values. Do an audit of the various assessments
your institution is engaged in. Link the two together.
NSSE items: Bethel’s Core Value of Reconciliation 1e. Included diverse perspectives (different races, religions,
genders, political beliefs, etc.) in class discussions or writing assignments
1u. Had serious conversations with students of a different race or ethnicity than your own
1v. Had serious conversations with students who are very different from you in terms of their religious beliefs, political opinions, or personal values
10c. Institutional Emphasis: Encouraging contact among students from different economic, social, and racial or ethnic backgrounds
11l. Educational & Personal Growth: Understanding people of other racial and ethnic backgrounds
HAD SERIOUS CONVERSATIONS WITH STUDENTS OF A DIFFERENT RACE OR ETHNICITY THAN YOUR OWN
2005 2006 2007 2008 20092.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
BethelCCCUCarnegie
ENCOURAGING CONTACT AMONG STUDENTS FROM DIFFERENT ECONOMIC, SOCIAL, AND RACIAL OR ETHNIC BACKGROUNDS
2005 2006 2007 2008 20092.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
3
BethelCCCUCarnegie
STUDENT SATISFACTION INVENTORY: BETHEL’S CORE VALUE OF CHRIST-FOLLOWERS
74. Being on this campus is contributing to my spiritual growth.
75.My understanding of God is being strengthened by classroom and/or campus experiences.
76.Faculty, administrators, and/or staff are helpful to me in processing issues related to my faith.
78.This campus provides adequate opportunities for involvement in ministry.
79.Given where I am spiritually right now, this campus is a good "fit" for me.
BEING ON THIS CAMPUS IS CONTRIBUTING TO MY SPIRITUAL GROWTH
Bethel CCCU5.2
5.4
5.6
5.8
6
6.2
6.4
6.6
ImportanceSatisfaction
CSS: ADDRESSING THE LEADERSHIP COMPONENT OF OUR MISSION
Students “very satisfied” or “satisfied” with their college in: Leadership opportunities
Since entering college you have: Participated in leadership training
Students reporting “much stronger” abilities and skills compared to when they first entered college: Leadership abilities
Compared with the average person of his/her age, student rated self “above average” or “highest 10%” in Leadership ability
Considerations noted as “essential” or “very important” in thinking about student’s career path: Leadership potential
Student objectives noted as “essential” or “very important”: Becoming a Community Leader
CSS ANALYSES Comparisons with other 4-year religious
colleges & all 4-year private colleges Longitudinal comparisons for:
Becoming a Community Leader Leadership ability
BECOMING A COMMUNITY LEADERLONGITUDINAL CHANGES
First-year Senior Change
Bethel 35.5 44.1 8.6
Religious Colleges 31.8 41.1 9.3
All Private Colleges 34.0 40.8 6.8
CORE VALUES: LEARNERS & TRUTH-SEEKERS
Combining data from direct and indirect assessments. CCTST indicates good growth in Critical
Thinking CLA results indicate poor growth in
integrative writing NSSE & CSS data indicate that students
perceive that they are growing in critical thinking
SUMMARY Don’t get caught up in precise
definitions (the “what really is critical thinking?” conversations)
Focus on using a variety of exisiting measures to assess mission.
Look for a convergence of evidence
CORNERSTONE UNIVERSITY
TIMOTHY DETWILER, Ph.D. Associate Provost for Traditional Programming
CORNERSTONE UNIVERSITY 2650 Students
1200 Traditional - 1000 Adults 250 Seminary - 200 Seminary (Asia)
Began in 1941 A faith-based institution Grand Rapids, St. Joseph, Kalamazoo,
Detroit, Holland Worldview, information literacy, civitas
and spiritual formation mark the programs
Mission Statement
University Assessment Plan
Campus-wide Learning Objectives
Core Curriculum Design
A Specific Core Course
CORNERSTONE UNIVERSITY
Mission Statement
To empower men and women to excel as influencers in our world for Christ by offering a student-focused learning community where Jesus Christ is central.
MISSION STATEMENTS One of the main purposes of a mission
statement is to guide decisions about organizational priorities in a systematic manner. Once an organization has identified what it wants to accomplish, it has to determine how to accomplish those things and how to measure the success of those activities.
(Nancy Kidd – NCA Director)
Effective organizations are the momentary results of constant reassessment and redesign. When the effectiveness of organizations is not tested, they rapidly tend to become unresponsive and inefficient.
Rodney Stark
ACADEMICS/SPIRITUAL FORMATION A public frame for the assessment
efforts Resonate deeply with the cultural
memory A stimulus to initiate change
Referring to a mason’s level A leveling instrument or a plumb line Placing our “stones” in order and
keeping them in line
MISHQELET PROJECT
“CIVITAS”DIRECT ASSESSMENT METHODS
Essays (pre/post experiences) Exams Experiential Aspects (self
reporting/observational) Electronic Discussion Board (rubric) Journal (rubric) Capstone Project Portfolio
CIVITASINDIRECT METHODS OF ASSESSMENT
Student Satisfaction Survey (EFS, SSI, CSS)
Experiential Reflective Journals Alumni Surveys Course Evaluation Forms Professional Evaluation of the Program
ENG 212 WRITING IN CULTURE
Link to mission Link to core curriculum learning
objectives Link to the Humanities Division ENG 212
THE KEY TO IT ALL
Institutional leaders have the major role in determining whether assessment results are important in making improvements. Unless top-level administrators believe that planning and assessment data are linked and important in strategic and logistic decisions, the assessment program in general will be brought into a dead-end. (IACBE)
MISSION DRIVEN ASSESSMENT
The end of virtually all assessment activities (except for some accountability initiatives) is the improvement of student learning, campus programming and institutional enhancement – in other words “organizational development”.
(Nichols & Nichols)
ORGANIZATIONAL DEVELOPMENT
Assessment is a strategy for organizational
Accountability Distinctiveness Accreditation Effectiveness Reflection Development
ORGANIZATIONAL DEVELOPMENT
MORE IMPORTANTLY, assessment is a strategy for understanding, confirming
and improving student learning and educational quality based upon an
institution’s mission.
Your Context
Your Mission, History, Culture,
Vision, Goals
Assessment is a matter of
institutional responsibility,
integrity,and mission.
MOUNT VERNON NAZARENE UNIVERSITY
DAVID W. KALE, Ph.D.Director of Assessment and Professor of
Communication
ASSESSING STUDENT LEADERSHIP MVNU Vision Statement: An academic
community of faith preparing Christ-like leaders for life long service and global impact.
The purpose of this assessment project was to help students assess their own leadership development.
In the process we wanted to see the degree to which institutional vision was reflected in student thinking about leadership.
ASSESSING STUDENT LEADERSHIP To achieve this, I worked with student
leaders to develop a rubric with which they could assess their own leadership ability.
I started by having all leaders list aspects of leadership they would like to use to assess themselves.
I then compiled a master list with all the leadership aspects included in their lists.
ASSESSING STUDENT LEADERSHIP I then presented the total list to them
and asked them to vote for five on the list they wanted to use to assess themselves.
Based on their voting, I then constructed a rubric using their top five dimensions.
You have a copy of the rubric I constructed in the materials which were distributed to you.
ASSESSING STUDENT LEADERSHIP As I have found in using this method of
rubric construction in other settings, the method itself is an important assessment procedure.
What aspects of leadership do our students consider so important that they want to use them in assessing their own leadership abilities?
ASSESSING STUDENT LEADERSHIP1. Compassion2. Empowering3. Communication – listening4. Honesty5. Empathy
ASSESSING STUDENT LEADERSHIP We were particularly interested and pleased
to see compassion as the #1 leadership ability chosen by our students.
This certainly ties in to the vision statement’s emphasis on developing Christ-like leaders.
One of our staff members who had done his doctoral studies in leadership indicated that he had never seen a list of leadership qualities that included compassion.
ASSESSING STUDENT LEADERSHIP While we were very pleased with the
orientation of the list toward empowering and meeting the needs of others, we also had some concerns with the list.
Leadership characteristics such as problem solving, ability to motivate others and creativity came in between 15th and 20th on the list. This gave our student development staff some information about the kinds of training student leaders needed.
ASSESSING STUDENT LEADERSHIP Another concern we had with the rubric
is that when the student leaders used it to assess themselves, they gave themselves very high marks right from the start.
We therefore had a ceiling effect which negated our plan to have students assess themselves at the beginning and end of the academic year to chart their development during that time.
ASSESSING STUDENT LEADERSHIP This was a surprise to me because in
using this approach to rubric construction for student self assessment in their moral and ethical reasoning ability, almost none of them gave themselves high marks.
ASSESSING STUDENT LEADERSHIP In summary of this strategy, we found the rubric
construction process to be very informative in terms of giving us feedback on the degree to which institutional vision was reflected in the leadership qualities students selected to assess their leadership abilities.
It was also helpful in providing information to student development as to areas of leadership development they might focus on in training student leaders.
We did not find this strategy helpful in developing a method of student self assessment that would permit them to identify areas in which they needed to improve or to track their development over an academic year.
GENERAL EDUCATION OBJECTIVE We have a general education objective
associated with our mission statement which addresses developing students’ abilities in moral and ethical reasoning.
SENIOR REQUIRED GENERAL EDUCATION COURSE IN APPLIED CHRISTIAN ETHICS
Section sizes kept to 24 students Debate national health care, drilling for
oil in ANWR, affirmative action, waterboarding, etc.
Course text: Stassen and Gushee, Kingdom Ethics, Intervarsity Press
DEFINING ISSUES TEST Based on the concepts of Lawrence
Kohlberg Essentially a critical thinking test in the
area of ethics Students get scores in Personal
Interest, Maintaining Norms, Post conventional or Principled Reasoning , etc.
We administer the test to first year students, juniors and seniors.
Mean S. D. N Sig.MVNU Seniors
23.06 10.62 76
MVNU Juniors
26.94 11.43 60 T= -2.04P= <.05D = .35
Nat’l Sample All Seniors
24.8 12.53 2441 ns
Nat’l Sample All Politically conserv.Seniors
23.48 12.01 881 ns
DEFINING ISSUES TEST – 2PERSONAL INTEREST SCORE
MVNU SENIORS 2006
MODEL FOR ANALYZING MORAL AND ETHICAL ARGUMENTS
Powers and Authorities
Social change Threats
Openness
Rules and Practices
Principles and Virtues
LoyaltiesInterestsPassionsModels
BASIC CONVICTIONSGod and Human
NatureForgiveness
Christlikeness and Justice
Mission of the Church
USE OF THE MODEL IN THE TEXT Students use the model in class in
groups to analyze a case of euthanasia from 20/20 on television.
Students use the model individually on an outside of class assignment to analyze a case provided by the instructor.
For mid-term exam, students use the model on a case they have not seen before exam period.
Mean S. D. N Sig.MVNU Seniors ( 2006)
23.06 10.62 76
MVNU Seniors (2009)
21.17 11.28 44
National Sample – All Seniors
24.8 12.9953 2441 D = .347T = -2.01P < .05
MVNU Juniors (2006)
26.56 11.82 164 D= .49T= -4.04P< .05
DEFINING ISSUES TEST – 2PERSONAL INTEREST SCORE
MVNU SENIORS, 2006 &2009
SUMMARY Now that there is evidence that I am
making progress in reducing students’ use of personal interest in making moral and ethical decisions, I would like to see even more progress on their parts in using internalized biblical principles as opposed to relying so much on external sources.