Council for the Advancement of Standards

in Higher Education

CAS Self-Assessment Guide for

Assessment Services

One Dupont Circle, NW, Suite 300, Washington, D.C. 20036-1188

August 2012

2

Assessment Services CAS Standards Contextual Statement

Assessment was an important element in higher education well before the founding of higher education in the United States; juried reviews were used to demonstrate student learning at the University of Bologna as early as 1063 CE (Bresciani, Moore Gardner, & Hickmott, 2009). Although the need for assessment and program evaluation in American higher education has long been acknowledged, it remains a pressing need today. With fiscal restraints and calls for accountability coming from multiple constituents (from parents to regional accrediting bodies), the ability to document and evaluate the impact of our programs and services is a critical responsibility. Early proponents of outcomes assessment included William Rainey Harper, President of the University of Chicago. In 1889, he called on colleges and universities to adopt a program of research with the college student as the subject "in order that the student may receive the assistance so essential to his [sic] highest success, another step in the onward evolution will take place. This step will be the scientific study of the student” (as cited in Rentz, 1996, p. 28). Responding to Harper’s vision, the Student Personnel Point of View (American Council of Education, 1937) challenged the field of student services to employ “studies designed to improve these functions and services” (p. 42). Later in the document, four specific kinds of studies were identified: student out-of-class life and its connection to the educational mission, faculty and student out-of-class relationships, financial aid to students, and after-college studies to ascertain the effects of college on careers and personal adjustment. The 1949 revision of the Student Personnel Point of View (American Council of Education) stated that the “principal responsibility of personnel workers lies in the area of progressive program development . . . this means that each worker must devote a large part of time to the formulation of new plans and to the continuous evaluation and improvement of current programs (p. 34). This document also stressed the importance of personnel workers being “thoroughly trained in research methods as a part of their professional preparation” (p. 35). Ultimately, the standard for student affairs programs, according to the 1949 document, is in “the difference it makes in the development of individual students” (p. 34). Historical and foundational documents in the field give clear evidence that the role of assessment and program evaluation in higher education and student affairs is important in the education of the “whole” student. Although most agree about the importance of conducting research on students and programs, still relatively few student affairs divisions have considered it a vital part of their operations. Of the 4100 colleges and universities in the United States, approximately 100 have student affairs assessment offices (G. Henning, personal communication). In the past decade, student affairs documents have continued the call for assessment and accountability of program effectiveness as it relates to student learning and development. The Student Learning Imperative (1996) charged student affairs staff to “participate in efforts to assess student learning . . . and periodically audit institutional environments to reinforce those factors that enhance, and eliminate those that inhibit student involvement in educationally-purposeful activities” (p. 6). The Principles of Good Practice for Student Affairs (1996) asserted the need to use systematic inquiry to improve student and institutional performance. Specifically, “student affairs educators who are skilled in using assessment methods acquire high-quality information; effective application of this information to practice results in programs

3

and change strategies . . . [that] improve student achievement” (p. 3). In Learning Reconsidered (2004), the language of assessment and student learning was more comprehensive. “Student Affairs must lead broad, collaborative institutional efforts to assess overall student learning and to track, document, and evaluate the role of diverse learning experiences . . . assessment should be a way of life—part of the institutional culture” (p. 26). According to Assessment Reconsidered, the companion to Learning Reconsidered, assessment is a collaborative exercise influenced by external forces but "more importantly emerges from the desire of faculty members, student affairs professionals, parents, students, and institutional administrators to know, and improve, the quality and effectiveness of higher education (Keeling, Wall, Underhile, & Dung, 2008, p. 1). Assessment is a process with dual purposes: accountability and continuous improvement. According to Bresciani (2006),

. . . in assessment faculty and staff articulate what the program intends to accomplish in regard to its services, research, student learning, and faculty/staff development programs. The faculty and/or professionals then purposefully plan the program so that the intended results (e.g., outcomes) can be achieved; implement methods to systematically–over time–identify whether end results have been achieved; and, finally, use the results to plan improvements or make recommendations for policy considerations, recruitment, retention, resource allocation, or new resource requests. (p. 14)

The terms assessment, research, and evaluation are often used interchangeably, but there are distinctions.

Briefly, research concerns theory: forming it, confirming it, and disconfirming it. Research assumes broader implications than one institution or program. Assessment, on the other hand, is more focused on the outcomes of participant programs, though this can be very broad to include an entire institution. It does not infer individual student outcomes. The purpose of assessment is to guide practice rather than to relate practice to theory. Evaluation is even more particular to a specific program and is concerned with the satisfaction, organization, and attendance of a program. (Jones, Torres, & Arminio, 2006, p. 30)

Recently, student affairs literature focused on enabling practitioners to implement assessment. These “how-to” guides provide information about the assessment planning process, writing outcomes, designing instruments, and performing qualitative as well as quantitative assessments (Bresciani, Zelna, & Anderson, 2004). These resources include the self-assessment guides published by CAS, Schuh and Associates' Assessment Methods for Student Affairs (2009), Suskie's Assessing Student Learning: A Common Sense Guide (2009), and Demonstrating Student Success (2009) by Bresciani, Moore Gardner, and Hickmott. In addition, professional associations have begun to provide their members with assessment tools, as have numerous for-profit companies. Campuses today should have a comprehensive assessment plan that includes consideration of learning outcomes, student needs and inputs, campus environments, student motivation, student use and satisfaction, and cost effectiveness (Schuh & Upcraft, 2001). Bresciani, Zelna, & Anderson (2004) offered reasons and uses for conducting assessment in student affairs, based on a compilation of other works, including those by the American Association for Higher Education (1994), Bresciani (2003), Ewell (1997a), Maki (2004), Palomba and Banta (1999), and Upcraft and Schuh (1996). These sources noted the multiple reasons for

4

carrying out assessment: reinforcing or emphasizing unit missions; improving a program’s quality or performance; comparing a program’s quality or value to the program’s previously defined principles; informing planning, decision-making, and policy discussions at the local, state, regional, and national levels; evaluating programs and personnel; assisting in the request for additional funds from the college or university and external community; assisting in the reallocation of resources; assisting in meeting accreditation requirements; identifying models of best practices and national benchmarks; celebrating successes while reflecting on the attitudes and approaches taken in improving learning and development; and creating a culture of continuous improvement—a culture of accountability, learning, and improvement. Assessment efforts may take many forms. Assessment can employ both qualitative and quantitative data collection methods such as interviews, focus groups, observations, rubrics, portfolios, surveys, and questionnaires. On-line survey software is used frequently in creating customized assessments. Assessment is a process that can be used to discover an institution’s best practices and to bring about continual improvement within the unique context of each institution. Student affairs divisions undertaking assessment efforts should not be discouraged by the seeming enormity of the task. The important consideration is to be purposeful and systematic and to use sound assessment methods that improve operations incrementally. The assessment process can be disaggregated into its component parts for easier implementation. Assessment Services (AS) as a functional area is sometimes organized as a single unit, while at other institutions it is a collective of institutional or departmental assessment bodies. In any case, collaboration and consultation is imperative between and within the AS and various individuals and departments as well as with institutional leadership. For a discussion of specific skills necessary to work in the AS in addition to those discussed in the attached CAS standards, the ACPA Assessment Skills and Knowledge (ASK) standards and the ACPA/NASPA Assessment, Evaluation, and Research professional competencies may be informative. The standards articulated here offer principles for assessment that are valuable for those working in or directing an assessment office as well as for those conducting assessments as a part of their position responsibilities in another functional area. References, Readings, and Resources American Association of Higher Education (1994). Nine principles of good practice for assessing

student learning [On-line]. Retrieved from http://www.iuk.edu/%7Ekoctla/assessment/9principles.shtml

American College Personnel Association (1996). The student learning imperative: Implications for student affairs [On-line]. Retrieved from http://www.acpa.nche.edu/sli/sli.htm

American College Personnel Association. (2006). ASK standards: Assessment skills and knowledge content standards for student affairs practitioners and scholars. Washington, DC: Author.

American College Personnel Association & National Association of Student Personnel Administrators. (2010). Professional competency areas for student affairs practitioners. Washington, D.C.: Authors.

American Council of Education. (1937). The student personnel point of view: A report of a conference on the philosophy and development of student personnel work in colleges and universities [On-line]. Retrieved from http://www.myacpa.org/pub/documents/1937.pdf

5

American Council of Education. (1949). The student personnel point of view: A report of a conference on the philosophy and development of student personnel work in colleges and universities [On-line]. Retrieved from http://www.myacpa.org/pub/documents/1949.pdf

Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers (2nd Ed). San Francisco, CA: Jossey-Bass.

Banta, T. W., Lund, J. P., Black, K. E., & Oblander, F. W. (Eds.). (1996). Assessment in practice: Putting principles to work on college campuses. San Francisco, CA: Jossey-Bass.

Banta, T. W., & Kuh, G. D. (1998, March/April). A missing link in assessment: Collaboration between academic and student affairs professionals. Change, 40-48.

Blimling, G. S., & Whitt, E. J. (1999). Good practices in student affairs: Principles to foster student learning. San Francisco, CA: Jossey-Bass.

Bresciani, M. J. (2003). An updated outline for assessment plans. NetResults [On-line]. Retrieved from http://www.naspa.org/

Bresciani, M. J., Zelna, C. L., & Anderson, J. A. (2004). Assessing student learning and development. Washington, DC: NASPA.

Bresciani, M. J. (2006). Outcomes-based academic and co-curricular program review: A compilation of institutional good practices. Sterling, VA: Stylus.

Bresciani, M. J., Moore Gardner, M., & Hickmott, J. (2009). Demonstrating student success: A practical guide to outcomes-based assessment of learning and development in student affairs. Sterling, VA: Stylus.

Driscoll, A., & Wood, S. (2007). Developing outcomes-based assessment for learner-centered education. Sterling, VA: Stylus.

Ewell, P. T. (1997a). From the states: Putting it all on the line—South Carolina’s performance funding initiative. Assessment Update, 9(1), 9-11.

Huba, M. E., & Freed, J. E. (2000). Learner-centered assessment on college campuses: Shifting the focus from teaching to learning. Boston: Allyn and Bacon.

Jones, W. R., Torres, W., & Arminio, J. (2006). Negotiating the complexities of qualitative research. New York: Routledge.

Keeling, R. P. (Ed.). (2004). Learning reconsidered. A campus-wide focus on the student experience. Washington, D.C.: National Association of Student Personnel Administrators & American College Personnel Association.

Keeling, R. P. (2006). Learning reconsidered 2: Implementing a campus-wide focus on the student experience. American College Personnel Association, Association of College and University Housing Officers-International, Association of College Unions-International, National Academic Advising Association, National Association for Campus Activities, National Association of Student Personnel Administrators, National Intramural-Recreational Sports Association.

Keeling, R. P., Wall, A. F., Underhile, R., & Dungy, G. J. (2008). Assessment reconsidered: Institutional effectiveness for student success. International Center for Student Success and Institutional Accountability.

Maki, P. (2010). Assessing for learning (2nd Ed). Sterling, VA: Stylus. National Association of Student Personnel Administrators & American College Personnel

Association. (2004) Learning reconsidered: A campus-wide focus on the student experience. Washington, DC: NASPA & ACPA. Retrieved from http://www.myacpa.org/pub/documents/LearningReconsidered.doc

Palomba, C. A., & Banta, T. W. (1999). Assessment essentials: Planning, implementing and improving assessment in higher education. San Francisco, CA: Jossey-Bass.

Rentz, A. L. (1996). A history of student affairs. In A. L. Rentz (Ed.), Student affairs practice in higher education (pp. 28-55). Springfield, IL: Thomas.

Shuch, J. & Associates. (2009). Assessment methods in student affairs. San Francisco, CA: Jossey-Bass.

6

Schuh, J., & Upcraft, M. L. (1998, Nov/Dec). Facts and myths about assessment in student affairs. About Campus, 2-8.

Schuh, J. H., Upcraft, M. L., & Associates, (2001). Assessment practice in student affairs: An application manual. San Francisco: Jossey-Bass.

Suskie, L. (2009). Assessing student learning: A common sense guide (2nd Ed.). Bolton, MA: Anker.

Strayhorn, T. L. (2006). Frameworks for assessing learning development outcomes. Washington, DC: Council for the Advancement of Standards in Higher Education.

U.S. Department of Education. (2006). A test of leadership:Charting the future of U.S. Higher Education. Jessup, MD: Ed Pubs.

Upcraft, M. L., & Schuh, J. H. (1996). Assessment in student affairs: A guide for practitioners. San Francisco, CA: Jossey-Bass.

Walvoord, B. E. (2004). Assessment clear and simple: A practical guide for institutions, departments, and general education. San Francisco, CA: Jossey-Bass.

Wehlburg, C. M. (2008). Promoting integrated and transformative assessment: A deeper focus on student learning. San Francisco, CA: Jossey-Bass.

Williamson, E. G., & Biggs, D. A. (1975). Student personnel work: A program of development relationships. New York, NY: John Wiley & Sons.

Contextual Statement Contributors Gavin Henning, Dartmouth College Jan Arminio, Shippensburg University

Previous Contributors Jan Arminio, Shippensburg University (2009) Joel H. Scott, University of Georgia (Baylor University) (2006) Cara Skeat Ray, Gainesville State College/University of Georgia (2006) Roger B. Winston, Jr., University of Georgia

7

Office of Institutional Research, Planning, and Assessment (OIRPA)

CAS Functional Area Review Executive Summary

The Office of Institutional Research, Planning, and Assessment (OIRPA) conducted its Council for the Advancement of Standards in Higher Education (CAS) Functional Area Review (FAR) during the Spring 2016 semester. The team consisted of Angelina Medeiros (Director of Institutional Research, Planning, and Assessment, Chair), Joanne Carroll-Connor (Associate Registrar), Aaron Reis (Staff Associate ITS), Denise Dimarzio (Instructor in English, Division of Humanities and Education), and Jennifer Dekkers (Assistant Professor, Division of Business and Information Management). The team held biweekly meetings throughout the Spring 2016 semester, as needed, and also conducted a great deal of work electronically by utilizing email and the department share drive to organize and review CAS materials. The evidence that was reviewed and discussed throughout this process included: annual and monthly reports, OIRPA annual goals, BCC Fact Sheets, the OIRPA webpage, the IRB webpage, job descriptions, and process and procedure documents. A few limitations arose throughout this review process. First, since there is no CAS Self-Assessment Guide (SAG) specifically designed for Institutional Research, the team needed to utilize the Assessment Services SAG for its review. While there were many parallels to OIRPA policies and procedures in Assessment Services, the document mostly ignored the two other major functions of the Office - research and planning. The Director and team were cognizant of this fact throughout the review and strived to incorporate all three office functions during the review process. The Office’s workflow document, which appears at the end of this summary, served as a guide to the variety of functions that are carried out by the Office. Three themes arose from the IR CAS/FAR review process:

compile a comprehensive “policies and procedures” manual increase the public face of IR increase transparency and accessibility of data

Regarding the first theme, it became apparent through this review that there is a need for the Office to organize and compose an all-inclusive “policies and procedures” manual in order to improve and enhance office productivity. While there are some policies that have been composed, such as the BCC Institutional Review Board (IRB) Policy, there are other guidelines that were identified through this review that need to be composed, such as a specific Ethics policy for OIRPA. The Office also has some written procedures but there is a significant need for guiding documents for all projects due to the small number of OIRPA staff and heavy reliance on cross-training. During Academic Year 16-17, the Director was cognizant of the need for written procedures and therefore produced draft procedures for some of the major projects of the office (e.g. ARGOS, HEIRS, and Gainful Employment). Composing all procedures is a major project that will be conducted in AY17-18. As each project is completed throughout the year, the instructions will be recorded. All instructions will be compiled in a manual for future use and updated continuously. Discussion of important policies that are needed in OIRPA will also take place in AY17-18. A finalized manual with all necessary documentation will be completed by Spring 2019. Regarding the second theme, it became clear that those who consistently work with OIRPA (Registrar, ITS, and Grants) were knowledgeable about Office functions and the information it has to offer. However, faculty members who tend to have limited interactions were less aware of Office tasks and offerings. Consequently, it is important for OIRPA to compose and publish a Mission Statement to specify all resources and services available to the College Community. The

8

Office also plans to work with Communications to promote OIRPA and publish important and interesting facts about the College in the BCC Weekly. On a related note, the third theme addresses the need to assist others with accessing and owning their own data. With the College's adoption of ARGOS, access and ease of use to obtain data has increased exponentially. Hence, in AY17-18, the Office will assist those in need of data to help them obtain their own information for independent use. It will also provide support to interpret and utilize information for planning and decision-making purposes. OIRPA strengths were identified in the areas of modeling ethical behavior and leadership as well as being able to use various research and assessment methodologies and methods. We continuously use these strengths to carry out the focus of OIRPA. Due to consistent preparedness training at the College, an additional strength was identified in the area of being knowledgeable about and trained in emergency procedures, crisis response, and prevention efforts. The Vice President is on the preparedness team for A Building in which the Office resides. The CAS process was abundantly helpful to OIRPA as it has made evident several areas that could be strengthened and improved. While there were some limitations to this process, including the focus mostly on Assessment, the themes that arose from this process will guide the work of OIRPA over the next five years.

9

CAS

Self-Assessment Guide Assessment Services

August 2012

Part 1. MISSION

The mission of Assessment Services (AS) is to develop a comprehensive assessment program to increase the institution’s knowledge about students, the educational environment, and institutional effectiveness to continuously improve student programs and services and to enhance student learning. AS must develop, disseminate, implement, and regularly review their missions. The mission must be consistent with the mission of the institution and with professional standards. The mission must be appropriate for the institution's student populations and community settings. Mission statements must reference student learning and development.

ND 0 1 2 3 4 5

Does Not Apply Insufficient Evidence/

Unable to Rate

Does Not Meet

Partly Meets Meets Exceeds Exemplary

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

1.1

The Assessment Service (AS) mission is to develop a comprehensive assessment program to increase the institution’s knowledge about students, the educational environment, and institutional effectiveness to continuously improve student programs and services and enhance student learning

3 4 3 2 3 X 3

1.2 The AS

1.2.1 develops, disseminates, and implements its mission

3 3 3 3 3 X 3

1.2.2 regularly reviews its mission

2 3 3 2 2 X 2.4

1.3 The AS mission statement

1.3.1 is consistent with that of

the institution 3 4 3 3 3 X 3.2

1.3.2 is consistent with professional standards

3 3 3 3 3 X 3

1.3.3 is appropriate for student populations and community settings

3 3 3 3 3 X 3

1.3.4 references learning and development

1 2 3 1 3 X 2

10

Part 1. Mission Overview Questions The primary focus of OIRPA is available on the Bristol Community College (BCC) website (http://www.bristolcc.edu/facultystaff/institutionalresearchplanningandassessment/) and reads as follows:

Institutional Research, Planning, and Assessment is the official source of institutional data at Bristol Community College. Our primary focus is to collect, analyze, and interpret information about the College that can then be used for strategic planning, assessment, and continuous quality improvement.

This focus is accomplished through the following essential duties: providing data on a routine basis to the Massachusetts Department of Higher Education’s data warehouse (HEIRS) and to the National Center for Education Statistics’ Integrated Postsecondary Education Data System (IPEDS); developing statistical and analytical reports to support decision-making, budgeting, and strategic planning at the College; designing and maintaining a comprehensive, annual Fact Sheet; designing, conducting, analyzing, and assessing results of web-based surveys of students, faculty, and/or staff; guiding faculty, staff, and students on evaluation and assessment projects; overseeing requests from external organizations, including guidebook publishers, and assignments to appropriate staff; administering college-wide standardized surveys and assessing results of these surveys (e.g. CCSSE); collecting and preparing materials for Academic Program Review and Functional Area Review; responding to requests for data from senior college officers; submitting the New England Association of Schools & Colleges’ (NEASC) Annual Report as well as implementing data collection methods to assist in the College’s self-study endeavors. An official Mission Statement needs to be composed that encompasses the wide range of services and functions of OIRPA. It is evident from the aforementioned essential duties that the published “primary focus” provides a very limited overview of all the Office is able to accomplish. Further, it does not provide an accurate depiction of everything OIRPA has to offer to the College Community. B. How does the mission embrace student learning and development? The Office goals statement does not directly address student learning and development. However, OIRPA provides evidence that aids in the assessment and continuous improvement of student learning and development at the College. For instance, the Community College Survey of Student Engagement (CCSSE) is administered regularly (2007, 2009, 2011, 2013, 2015, 2016, and 2017) to examine student engagement which is a key component to student learning and retention. In 2017 a Survey of Online Student Engagement was administered to study the engagement of the online student population. The Noel-Levitz Student Satisfaction Inventory has also been administered (2008, 2012) to aid in the improvement of the quality of the student experience. In Spring 2016, the Office facilitated the administration of the Noel-Levitz Adult Learner Inventory (ALI) and the Director participated in the completion of the Adult Learner Focused Self-Assessment. Student priorities and satisfaction are examined regularly from these surveys in order to identify issues that are important to students and assist with planning and decision-making processes. More intentional inclusion of the manner in which OIRPA supports student learning and development should be added to the Mission Statement to identify strategies that are utilized to support these measures.

11

C. In what ways does the program mission complement the mission of the institution? The Bristol Community College Mission (http://www.bristolcc.edu/aboutbcc/missioncorevalues/bristolcommunitycollegemissioncorevalues/) reads as follows: As the leading resource for education and workforce development in southeastern Massachusetts, Bristol Community College provides programs that promote individual opportunity and the region's economic health. Our programs offer a strong foundation in liberal arts and sciences; career-ready education in health sciences, engineering and technology, and business; and comprehensive developmental education and adult literacy services; delivered in a learner- centered, supportive community. The College also develops active partnerships with business and industry, preK-12, colleges and universities, and social service agencies to maintain relevance and effectiveness of all credit and noncredit programming. We value and respect diversity within the College and the world. Bristol Community College prepares well-rounded learners for employment and for life. This statement clearly identifies the importance of providing students with the necessary skills and opportunities to succeed in higher education and beyond. The OIRPA Mission will directly align with the College’s mission by specifically identifying ways in which the Office provides evidence of student learning for strategic planning, assessment, and continuous quality improvement which ties into the larger Mission of the College to prepare students for their educational career, employment, and life. Through the duties mentioned above and assessment initiatives such as CCSSE, OIRPA is able to support the College Mission through providing the evidence that is needed in order to monitor College initiatives and how well we are accomplishing our goals.

Part 2. PROGRAM

The formal education of students, consisting of the curriculum and the co-curriculum, must promote student learning and development outcomes that are purposeful, contribute to students' realization of their potential, and prepare students for satisfying and productive lives. Assessment Services (AS) must collaborate with colleagues and departments across the institution to promote student learning and development, persistence, and success. Consistent with the institutional mission, AS must identify relevant and desirable student learning and development outcomes from among the six domains and related dimensions:

Domain: knowledge acquisition, integration, construction, and application Dimensions: understanding knowledge from a range of disciplines;

connecting knowledge to other knowledge, ideas, and experiences; constructing knowledge; and relating knowledge to daily life

Domain: cognitive complexity

12

Dimensions: critical thinking, reflective thinking, effective reasoning, and

creativity

Domain: intrapersonal development Dimensions: realistic self-appraisal, self-understanding, and self-respect;

identity development; commitment to ethics and integrity; and spiritual awareness

Domain: interpersonal competence Dimensions: meaningful relationships, interdependence, collaboration, and

effective leadership.

Domain: humanitarianism and civic engagement Dimensions: understanding and appreciation of cultural and human

differences, social responsibility, global perspective, and sense of civic responsibility

Domain: practical competence Dimensions: pursuing goals, communicating effectively, technical

competence, managing personal affairs, managing career development, demonstrating professionalism, maintaining health and wellness, and living a purposeful and satisfying life

[LD Outcomes: See The Council for the Advancement of Standards Learning and Developmental Outcomes statement for examples of outcomes related to these domains and dimensions.] AS must

assess relevant and desirable student learning and development provide evidence of impact on outcomes articulate contributions to or support of student learning and development in

the domains not specifically assessed articulate contributions to or support of student persistence and success use evidence gathered through this process to create strategies for

improvement of programs and services AS must be

intentionally designed guided by theories and knowledge of learning and development integrated into the life of the institution reflective of developmental and demographic profiles of the student

population responsive to needs of individuals, populations with distinct needs, and

relevant constituencies delivered using multiple formats, strategies, and contexts

13

Where institutions provide distance education, AS must assist distance learners to achieve their educational goals by providing access to information about programs and services, to staff members who can address questions and concerns, and to counseling, advising, or other forms of assistance. Regardless of its structure, AS must collaborate and consult with institutional research or various departments to ensure that assessment efforts address institutional needs. AS may be organized as a functional area or be a collective body of assessment initiatives across an institution. Whenever there are both AS and an institutional research function in an institution, there should be clear delineation of responsibilities. In institutions that do not have AS, a senior officer must be an advocate for assessment and program evaluation and must collaborate and consult with, and otherwise provide support to, institutional assessment efforts. AS must include activities that assess student needs and student learning and development outcomes, assess whether goals are being achieved, describe student characteristics, determine whether professional standards are being met, and determine effectiveness of programs and services for students. Results of these studies must be disseminated to appropriate personnel and students in the institution. AS must describe the demographics, personal characteristics, and behaviors of students conduct regular assessments of student needs review and use available literature about the characteristics and developmental

changes of post-secondary students assess whether the work of AS is consistent with and achieves stated objectives use appropriate professional standards, tools, and instruments for assessment study the extent to which students, programs, departments, and institutions

meet their overall educational goals and determine conditions that enhance or hamper goal achievement

study the impact of the college experience on students and alumni examine retention and graduation rates compare institutional practices against professional standards investigate the impact of campus culture assist others in the effective use of assessment to inform decisions and guide the

development and improvement of services, programs, and policies

AS should examine cost effectiveness and the level of student satisfaction with programs and services. AS practices must be conducted in the context of existing and developing strategic initiatives. AS should provide assessment and evaluation support for other institutional offices and institutional decision makers.

14

AS should collect and analyze student data beginning with pre-enrollment characteristics and

continuing through follow-up studies of graduates and other former students plan, coordinate, or conduct regular studies of various programs, facilities, services, classes,

and student sub-groups describe students’ intellectual, emotional, social, moral, spiritual, vocational, and physical

development and behavior; such data should be regularly collected, updated, and disseminated

analyze data to identify trends in student behavior and attitudes to consider the implications for institutional policies and practices

analyze data to identify retention trends to consider the implications for institutional policies and practices

collect and analyze data, including cost effectiveness data, to be used for making decisions about the continuation, modification, or termination of programs and services

compare institutional practices against benchmarks coordinate assessment plans across units and act as a resource to faculty and staff

regarding assessment and evaluation efforts disseminate information about assessment and evaluation findings to members of the

campus community guide and evaluate research and assessment efforts conducted by students track campus studies, for example in theses and dissertations, in which campus students

are participants

ND 0 1 2 3 4 5

Does Not Apply Insufficient Evidence/

Unable to Rate

Does Not Meet

Partly Meets Meets Exceeds Exemplary

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

2.1

The Assessment Service (AS) promotes student learning and development outcomes that

2.1.1 are purposeful 2 0 3 3 0 X 1.6

2.1.2 contribute to students’ realization of their potential

3 2 3 3 3 X 2.8

2.1.3 prepare students for satisfying and productive lives

3 2 3 3 3 X 2.8

2.2

The AS collaborates with colleagues and departments across the institution to promote student learning, development, persistence, and success

4 3 4 4 4 X 3.8

2.3 The AS

15

Criterion Measures Rater

1

Rater

2

Rater

3

Rater

4

Rater

5 Consistent Discrepant

Average

Rating

2.3.1

assesses relevant and desirable student learning and development

3 2 3 3 3 X 2.8

2.3.2 provides evidence of impact on outcomes

3 2 3 3 3 X 2.8

2.3.3

articulates contributions to or support of student learning and development in the domains not specifically assessed

3 2 3 3 3 X 2.8

2.3.4

articulates

contributions to or support of student persistence and success

3 2 3 3 3 X 2.8

2.7.5

uses evidence gathered through assessment to create strategies for improvement

3 2 3 3 3 X 2.8

2.4 The AS is

2.4.1 intentionally designed 4 4 4 3 4 X 3.8

2.4.2 guided by theories and knowledge of learning and development

3 3 3 3 0 X 2.4

2.4.3 integrated into the life of the institution

4 3 4 3 4 X 3.6

2.4.4

reflective of developmental and demographic profiles of the student population

4 3 4 3 4 X 3.6

2.4.5

responsive to needs of individuals, populations with distinct needs, and relevant constituencies

3 2 3 2 3 X 2.6

2.4.6 delivered using multiple formats, strategies, and contexts

3 3 3 3 3 X 3

2.5

When distance education is provided, the AS assists learners in achieving their education goals by providing access to

2.5.1 information about programs and services

ND ND ND ND ND X ND

2.5.2 staff members who can address questions and concerns

ND ND ND ND ND X ND

2.5.3 counseling, advising, or other forms of assistance

ND ND ND ND ND X ND

16

Criterion Measures Rater

1

Rater

2

Rater

3

Rater

4

Rater

5 Consistent Discrepant

Average

Rating

2.6

The AS collaborates and consults with institutional research or various departments to ensure that assessment efforts address institutional needs

3 3 3 3 3 X 3

2.7

In institutions without an AS, a senior officer advocates for assessment and program evaluation and collaborates, consults with, and

supports institutional assessment efforts

ND ND ND ND ND X ND

2.8 The AS includes activities that

2.8.1

assess student needs and student learning and development outcomes

3 2 3 3 3 X 2.8

2.8.2 assess whether goals are being achieved

3 3 3 3 3 X 3

2.8.3 describe student characteristics

3 4 3 3 3 X 3.2

2.8.4 determine whether professional standards are being met

3 3 3 3 3 X 3

2.8.5 determine effectiveness of programs and services for students

3 3 3 3 3 X 3

2.8.6

disseminate results of studies to appropriate institutional personnel and students

3 4 3 3 3 X 3.2

2.9 The AS

2.9.1

describes the demographics, personal characteristics, and behaviors of students

3 4 3 3 3 X 3.2

2.9.2 conducts regular assessments of student needs

3 3 3 2 3 X 2.8

2.9.3

reviews and uses available literature about the characteristics and developmental changes of postsecondary students

3 0 3 0 0 X 1.2

2.9.4

assesses whether the work of AS is consistent with and achieves stated objectives

3 3 3 3 3 X 3

17

Criterion Measures Rater

1

Rater

2

Rater

3

Rater

4

Rater

5 Consistent Discrepant

Average

Rating

2.9.5

uses appropriate professional standards, tools, and instruments for assessment

3 3 3 3 3 X 3

2.9.6

studies the extent to which students, programs, departments, and institutions meet their overall educational goals and determine conditions that enhance or hamper goal achievement

3 3 3 3 3 X 3

2.9.7 studies the impact of the college experience on students and alumni

3 3 3 3 3 X 0

2.9.8 examines retention and graduation rates

3 3 3 3 3 X 3

2.9.9 compares institutional practices against professional standards

3 3 3 3 3 X 3

2.9.10 investigates the impact of campus culture

3 0 3 0 0 X 1.2

2.9.11

Assists others in the effective use of assessment to inform decisions and guide development and improvement of services, programs, and policies

3 4 3 3 3 X 3.2

2.1

AS practices are conducted in the context of existing and developing strategic initiatives

4 4 4 3 4 X 3.8

Part 2. Program Overview Questions A. What are the primary elements of the program? As the official source of institutional data at the College, OIRPA strives to collect, analyze, and interpret information about BCC that can be used for continuous quality improvement through strategic planning, assessment, program evaluation, accreditation, etc. The Office provides student demographics through its Fact Sheet (http://www.bristolcc.edu/aboutbcc/presidentsoffice/factsheets/) that is published annually and contains a summary of college information such as enrollment, composition of the student population, graduation statistics, tuition information, and transfer schools. The OIRPA annual report also provides longitudinal data on enrollment, affordability, success, and quality indicators as well as trends in student success.

18

The OIRPA Vice President and Director serve on several committees and workgroups that promote student learning and development. The Vice President and Director have served on the former Enrollment-Retention Council to promote student retention and persistence through gathering, disseminating, and discussing student data with committee members. Through the Multi-State Collaborative (MSC) that is being led by the Massachusetts Board of Higher Education, the Director collaborates with the Center for Teaching and Learning (CTL) to gather evidence of student learning outcomes for students nearing graduation. There is also consistent participation at statewide Advancing a Massachusetts Culture of Assessment (AMCOA) meetings by the Director and Vice President. The Vice President is the administrator of the new Educational Effectiveness Committee (formerly Institutional Effectiveness and Outcomes Assessment Committee) which aligns with the new NEASC Standard 8:

The institution demonstrates its effectiveness by ensuring satisfactory levels of student achievement on mission-appropriate student outcomes. Based on verifiable information, the institution understands what its students have gained as a result of their education and has useful evidence about the success of its recent graduates. This information is used for planning and improvement, resource allocation, and to inform the public about the institution. Student achievement is at a level appropriate for the degree awarded (https://cihe.neasc.org/standards-policies/standards-accreditation/standards-effective-july-1-2016#standard_eight).

The Vice President collaborated with the co-chairs of the former committees to establish this new committee to ensure that the College meets the criteria for the new standard. Both the Vice President and Director serve on working groups (Academic Program Review and CAS/FAR, respectively) that support the charge of Educational Effectiveness. B. What evidence exists that confirms the contributions of the program to student learning and development? Student behavior is examined through the collection of data via institution-wide surveys such as the CCSSE and Noel-Levitz. These are valid and reliable instruments that are conducted nationally, therefore allowing comparisons with state and national peers. Various internal surveys are also conducted for departments and areas of the College upon request in order to assess student needs and collect feedback on various initiatives and grants. While a wealth of information is collected by OIRPA, the Office also strives to disseminate this information to others through reports, individualized meetings, presentations, and published articles in the former Bristol Buzz and current BCC Weekly. The Office continues to focus its attention on disseminating data to ensure that information is used in a timely and effective manner. OIRPA collaborates with President’s Cabinet, the Educational Effectiveness Committee, Strategic Planning Committee, Pathways Leadership Team, Adult Learner Taskforce, and various working groups to aid in this important task. A great wealth of information on student learning and development is also collected through Academic Program Review and CAS Functional Area Review - assessment processes that are administered by the Office. The BCC Institutional Review Board (IRB) is also managed through OIRPA with the Vice President as the Chair and the Director as the Administrator. The Board strives to protect human subjects who are participating in research and encourages fair and just research

19

practices at the College. The Vice President and Director assist honors students with their Culminating Honors Project (CHP), teaching them about survey design and assessment strategies. The Director presents IRB information at a CHP course each semester to introduce and education students about research ethics and the role of the IRB. C. What evidence is available to confirm achievement of program goals? The following evidence is used to support the assessment and planning of programs and services offered at the College:

2014 NEASC Accreditation Materials

National and state reporting: IPEDS and HEIRS Grant evaluation and assessment: Performance Incentive Grant Evaluation Report, Title

III Internal Evaluation Reports and Presentations, TAACCCT4 Annual Reports, GPSTEM Outcomes Summary, etc.

Internal Institutional Reports: Monthly and Annual Reports Internal and External Data Requests: Data requested by students, faculty, staff, and

community members Policies and procedures

Part 3. ORGANIZATION AND LEADERSHIP

To achieve student and program outcomes, Assessment Services (AS) must be structured purposefully and organized effectively. AS must have

clearly stated goals current and accessible policies and procedures written performance expectations for employees functional work flow graphics or organizational charts demonstrating clear

channels of authority

Leaders with organizational authority for the programs and services must provide strategic planning, supervision, and management; advance the organization; and maintain integrity through the following functions: Strategic Planning

articulate a vision and mission that drive short- and long-term planning set goals and objectives based on the needs of the population served and

desired student learning or development and program outcomes facilitate continuous development, implementation, and assessment of goal

attainment congruent with institutional mission and strategic plans promote environments that provide meaningful opportunities for student

learning, development, and engagement develop and continuously improve programs and services in response to the

changing needs of students served and evolving institutional priorities intentionally include diverse perspectives to inform decision making

Supervising

manage human resource processes including recruitment, selection, development, supervision, performance planning, evaluation, recognition, and reward

20

influence others to contribute to the effectiveness and success of the unit empower professional, support, and student staff to accept leadership

opportunities offer appropriate feedback to colleagues and students on skills needed to

become more effective leaders encourage and support professional development, collaboration with

colleagues and departments across the institution, and scholarly contribution to the profession

Managing

identify and address individual, organizational, and environmental conditions that foster or inhibit mission achievement

plan, allocate, and monitor the use of fiscal, physical, human, intellectual, and technological resources

use current and valid evidence to inform decisions incorporate sustainability practices in the management and design of

programs, services, and facilities understand appropriate technologies and integrate them into programs and

services be knowledgeable about codes and laws relevant to programs and services

and ensure that staff members understand their responsibilities through appropriate training

assess potential risks and take action to mitigate them Advancing the Organization

communicate effectively in writing, speaking, and electronic venues advocate for programs and services advocate for representation in strategic planning initiatives at appropriate

divisional and institutional levels initiate collaborative interactions with internal and external stakeholders who

have legitimate concerns about and interests in the functional area facilitate processes to reach consensus where wide support is needed inform other areas within the institution about issues affecting practice

Maintaining Integrity

model ethical behavior and institutional citizenship share data used to inform key decisions in transparent and accessible ways monitor media used for distributing information about programs and services

to ensure the content is current, accurate, appropriately referenced, and accessible

AS leaders must

understand the foundations of higher education understand the educational value and objectives of programs and services interpret assessment results to guide educational practice advocate for institutional response to assessment findings serve as an expert in administering effective and efficient assessment

programs stay current about trends in assessment work to ensure that students are not over-assessed

21

understand and be able to use various research and assessment methodologies and methods

Because outcomes assessment and program evaluation efforts are conducted on most campuses in cooperation with other institutional research and evaluation efforts, a staff member must be designated to manage specific assessment activities, priorities, and timelines. Assessment goals should result from a collaborative effort between leaders of AS, those responsible for the various programs and services being assessed, and others responsible for institutional research efforts.

ND 0 1 2 3 4 5

Does Not Apply

Insufficient Evidence/

Unable to Rate

Does Not Meet

Partly Meets Meets Exceeds Exemplary

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

3.1 The Assessment Service (AS) has

3.1.1 clearly stated goals 3 3 3 3 3 X 3

3.1.2 current and accessible policies and procedures

2 3 3 3 3 X 2.8

3.1.3

written performance expectations for employees

3 3 3 3 3 X 3

3.1.4

functional work flow graphics or organizational charts demonstrating clear channels of authority

2 2 3 2 2 X 2.2

3.2 In providing strategic planning, AS leaders

3.2.1

articulate a vision and mission that drive short- and long-term planning

3 4 4 3 3 X 3.4

3.2.2

set goals and objectives based on the needs of the population served and desired student learning or development and program outcomes

3 4 4 3 4 X 3.6

22

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

3.2.3

facilitate continuous development, implementation, and assessment of goal attainment congruent with institutional mission and strategic plans

4 3 4 3 3 X 3.4

3.2.4

promote environments that provide meaningful opportunities for student learning, development, and engagement

2 2 0 0 0 X 0.8

3.2.5

develop and continuously improve programs and services in response to the changing needs of students served and evolving institutional priorities

3 4 3 0 0 X 2

3.2.6

intentionally include diverse perspectives to inform decision making

4 4 4 3 3 X 3.6

3.3 In providing supervision, AS leaders

3.3.1

manage human resource processes including recruitment, selection, development, supervision, performance planning, evaluation, recognition, and reward

3 3 3 3 3 X 3

3.3.2

influence others to contribute to the effectiveness and success of the unit

4 3 4 0 0 X 2.2

3.3.3

empower professional, support, and student staff to accept leadership opportunities

4 4 4 0 0 X 2.4

23

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

3.3.4

offer appropriate feedback to colleagues and students on skills needed to become more effective leaders

3 3 3 0 0 X 1.8

3.3.5

encourage and support professional development, collaboration with colleagues and departments across the institution, and scholarly contribution to the profession

4 3 3 3 3 X 3.2

3.4 In providing management, AS leaders

3.4.1

identify and address individual, organizational, and environmental conditions that foster or inhibit mission achievement

3 3 3 0 0 X 1.8

3.4.2

plan, allocate, and monitor the use of fiscal, physical, human, intellectual, and technological resources

3 2 3 2 3 X 2.6

3.4.3 use current and valid evidence to inform decisions

3 4 4 0 0 X 2.2

3.4.4

incorporate sustainability practices in the management and design of programs, services, and facilities

3 4 4 0 0 X 2.2

3.4.5

understand appropriate technologies and integrate them into programs and services

3 4 4 3 3 X 3.4

3.4.6

are knowledgeable about codes and laws relevant to programs and services and ensure that staff members understand their responsibilities

3 3 3 3 0 X 2.4

24

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

through appropriate training

3.4.7

assess potential risks and take action to mitigate them

3 3 3 0 0 X 1.8

3.5 In advancing the organization, AS leaders

3.5.1

communicate effectively in writing, speaking, and electronic venues

3 3 3 3 4 X 3.2

3.5.2 advocate for programs and

services

2 4 4 3 0 X 2.6

3.5.3

advocate for representation in strategic planning initiatives at appropriate divisional and institutional levels

4 4 4 3 3 X 3.6

3.5.4

initiate collaborative interactions with internal and external stakeholders who have legitimate concerns about and interests in the functional area

2 3 3 2 3 X 2.6

3.5.5

facilitate processes to reach consensus where wide support is needed

3 3 3 3 0 X 2.4

3.5.6

inform other areas within the institution about issues affecting practice

3 2 3 3 4 X 3

3.6 In maintaining integrity, AS leaders

3.6.1

model ethical behavior and institutional citizenship

5 5 4 4 5 X 4.2

3.6.2

share data used to inform key decisions in transparent and accessible ways

2 3 3 2 2 X 2.4

25

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

3.6.3

monitor media used for distributing information about programs and services to ensure the content is current, accurate, appropriately referenced, and accessible

ND ND ND ND ND X ND

3.7 AS leaders must

3.7.1 understand the foundations of higher education

4 4 4 4 5 X 4.2

3.7.2

understand the

educational value and objectives of programs and services

3 4 4 3 3 X 3.4

3.7.3

interpret assessment results to guide educational practice

3 3 3 3 0 X 2.4

3.7.4

advocate for institutional response to assessment findings

3 3 3 3 3 X 3

3.7.5

serve as an expert in administering effective and efficient assessment programs

4 3 4 4 4 X 3.8

3.7.6 stay current with trends in assessment

4 3 4 0 0 X 2.2

3.7.7 work to ensure that students are not over-assessed

3 2 3 3 0 X 2.2

3.7.8

understand and be able to use various research and assessment methodologies and methods

4 3 4 4 3 X 3.6

3.8

The AS designates a staff member to manage specific assessment activities, priorities, and timelines in cooperation with other institutional research and evaluation efforts

4 4 4 3 3 X 3.6

26

Part 3. Organization and Leadership Overview Questions A. In what ways are program leaders positioned and empowered to accomplish the program

mission? The Office is staffed with a full-time Vice President, Director, and Administrative Assistant. Leadership positions are held by the Vice President and Director. Two work-study positions are vacant at this time. OIRPA consistently collaborates with the Cooperative Education program to obtain interns who usually work 10 hours per week to assist with research projects. B. How do program leaders advance the organization? The Vice President is a member of President's Cabinet which consists of all Vice Presidents at the College and President Douglas. The Cabinet meets weekly in order to discuss, plan, and make decisions on various institutional goals and objectives. The Vice President is highly involved in many of the leadership projects for the cabinet as data, assessment, and evaluation are used to inform continuous planning and decision-making. This is particularly true for NEASC Accreditation, Strategic Planning and major grant initiatives that are discussed with President's Cabinet. The Vice President also serves in various leadership capacities for committees and workgroups. She is the Administrator of the Educational Effectives Committee and Strategic Planning Committee. The Vice President is also the Chair of the IRB. Further, Rhonda serves on the Board for Stepping Stone, a residential center treating substance abuse in Fall River. The Director serves on important leadership committees including the Leading for Change Team at BCC and for the Consortium led by Bridgewater State, former Operating From A Culture Of Data Subgroup for the Enrollment-Retention Council, Educational Effectiveness Committee, the NEASC Outcomes Assessment Workgroup, the MSC Team, and the Adult Learner Taskforce. She is the Administrator for the IRB and Co-chair of the CAS Functional Area Subgroup for Educational Effectiveness. Through their involvement in various committees and work groups, the Vice President and Director are able to carry out the goals of OIRPA by being knowledgeable about various college initiatives and therefore be able to predict the types of data and information that are needed to evaluate and assess the progress of these initiatives. They are also able to indicate areas of improvement and strength in order to lead decision-making processes. C. How are program leaders accountable for their performance? The Vice President, Director, and Administrative Assistant are non-unit professionals and therefore undergo an annual evaluation of their performance at the end of the fiscal year. This process is conducted through the Office of Human Resources where the employee being evaluated conducts a self-assessment, the supervisor conducts a parallel process of evaluation, and then a meeting is held to discuss results. Once finalized, the evaluation is sent to the President. D. What leadership practices best describe program leaders? The Vice President and Director use collaboration as their major leadership practice. OIRPA supports and contributes to many areas of the College and therefore this leadership approach assists the evaluation of various college-wide practices.

27

Part 4. HUMAN RESOURCES

Assessment Services (AS) must be staffed adequately by individuals qualified to accomplish mission and goals. Within institutional guidelines, AS must

establish procedures for staff recruitment and selection, training, performance planning, and evaluation

set expectations for supervision and performance assess the performance of employees individually and as a team provide access to continuing and advanced education and appropriate

professional development opportunities to improve the leadership ability, competence, and skills of all employees.

AS must maintain position descriptions for all staff members. To create a diverse staff, AS must institute recruitment and hiring strategies that encourage individuals from under-represented populations to apply for positions. AS must develop promotion practices that are fair, inclusive, proactive, and non-discriminatory. To further the recruitment and retention of staff, AS must consider work life initiatives, such as compressed work schedules, flextime, job sharing, remote work, or telework. AS professional staff members must hold an earned graduate or professional degree in a field relevant to the position they hold or must possess an appropriate combination of educational credentials and related work experience. AS professional staff members must engage in continuing professional development activities to keep abreast of the research, theories, legislation, policies, and developments that affect their programs and services. AS must have technical and support staff members adequate to accomplish their mission. All members of the staff must be technologically proficient and qualified to perform their job functions, be knowledgeable about ethical and legal uses of technology, and have access to training and resources to support the performance of their assigned responsibilities. Degree- or credential-seeking interns or graduate assistants must be qualified by enrollment in an appropriate field of study and by relevant experience. These individuals must be trained and supervised adequately by professional staff members holding educational credentials and related work experience appropriate for supervision. Supervisors must be cognizant of the roles of interns and graduate assistants as both student and employee and closely adhere to all parameters of their job descriptions, work hours, and schedules. Supervisors and the interns or graduate assistants must agree to compensatory time or other appropriate compensation if circumstances necessitate additional hours

28

Student employees and volunteers must be carefully selected, trained, supervised, and evaluated. They must be educated on how and when to refer those in need of additional assistance to qualified staff members and must have access to a supervisor for assistance in making these judgments. Student employees and volunteers must be provided clear job descriptions, pre-service training based on assessed needs, and continuing development. All AS staff members, including student employees and volunteers, must receive specific training on institutional policies pertaining to functions or activities they support and to privacy and confidentiality policies and laws regarding access to student records and other sensitive institutional information. All AS staff members must receive training on policies and procedures related to the use of technology to store or access student records and institutional data. AS must ensure that staff members are knowledgeable about and trained in emergency procedures, crisis response, and prevention efforts. Prevention efforts must address identification of threatening conduct or behavior of students, faculty and staff members, and others and must incorporate a system for responding and reporting. AS must ensure that staff members are knowledgeable of and trained in safety and emergency procedures for securing and vacating facilities. Within the institution, a qualified professional AS staff member must be designated to coordinate the assessment efforts and must work closely with or be responsive to leaders of programs and services. The number of staff members assigned to assessment efforts should be a function of the size, complexity, and purpose of the institution. Institutions unable to assign a full-time professional staff member should devote a portion of their institutional research program's resources to this effort. Staff assigned responsibility for assessment should possess effective communication and consultation skills and have an appropriate combination of coursework, training, and experience in the following areas: research methodology, design, and analysis; computer literacy; program planning implementation and evaluation; and human development theory, including the study of student sub-group cultures. When staff members lack adequate knowledge in any of these critical areas, they should seek expertise from appropriate resources.

29

ND 0 1 2 3 4 5

Does Not

Apply

Insufficient

Evidence/ Unable to Rate

Does Not

Meet Partly Meets Meets Exceeds Exemplary

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

4.1

The Assessment Service (AS) is staffed adequately to accomplish mission and goals

1 2 2 1 1 X

1.4

4.2 Within institutional guidelines, the AS

4.2.1

establishes procedures for staff recruitment and

selection, training, performance planning, and evaluation

3 3 3 3 3 X

3

4.2.2 sets expectations for supervision and performance

3 2 3 3 3 X

2.8

4.2.3

assesses the performance of employees individually and as a team

2 3 3 2 2 X

2.4

4.2.4

provides access to continuing and advanced education and appropriate professional development opportunities to improve the leadership ability, competence, and skills of all employees

3 3 3 3 3 X

3

4.3 The AS

4.3.1 maintains position descriptions for all staff members

3 4 4 3 3 X

3.4

4.3.2

institutes recruitment and hiring strategies that encourage applications from under-represented populations

3 2 3 2 3 X

2.6

4.3.3

develops promotion practices that are fair, inclusive, proactive, and non-discriminatory

3 4 3 3 3 X

3.2

30

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

4.3.4

considers work life initiatives, such as compressed work schedules, flextime, job sharing, remote work, or telework

ND ND ND ND ND X ND

4.3.5

has technical and support staff members adequate to accomplish the mission

2 2 2 2 2 X

2

4.4 AS professional staff members

4.4.1

hold earned graduate or professional degrees in fields relevant to the position or possess an appropriate combination of educational credentials and related work experience

3 4 4 3 3 X

3.4

4.4.2

engage in continuing professional development activities

3 3 3 3 3 X

3

4.5

Degree- or

credential-seeking interns or graduate assistants in the AS

4.5.1

are qualified by enrollment in an appropriate field of study and by relevant experience

3 3 3 3 3 X

3

4.5.2

are trained and supervised adequately by professional staff members

3 4 4 3 3 X

3.4

4.6

Supervisors of AS interns or graduate students adhere to all parameters of job descriptions, work hours, and schedules

3 3 3 0 0

X 1.8

4.7 Student employees and volunteers

4.7.1

are carefully selected, trained, supervised, and evaluated

3 4 4 3 0

X 2.8

31

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

4.7.2

are educated on how and when to refer those in need of additional assistance to qualified staff members and have access to a supervisor for assistance in making these judgments

3 2 3 0 0

X 1.6

4.7.3

are provided clear job descriptions, pre-service training based on assessed needs, and continuing development

2 3 3 3 2 X

2.6

4.8

All AS staff members, including student employees and volunteers,

4.8.1

receive specific training on institutional policies pertaining to functions or activities they support

3 3 3 3 0

X 2.4

4.8.2

receive specific

training on privacy and confidentiality policies and laws regarding access to student records and other sensitive institutional information

3 5 3 3 0

X 2.8

4.8.3

receive training on policies and procedures related to the use of technology to store or access student records and institutional data

3 3 3 3 3 X

3

4.8.4

are knowledgeable about and trained in emergency procedures, crisis response, and prevention efforts and in safety and emergency procedures for securing and vacating facilities

4 4 4 4 4 X

4

32

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

4.9

The AS designates a qualified professional AS staff member to coordinate assessment efforts and work closely with or be responsive to leaders of programs and services

3 3 3 3 3 X

3

Part 4. Human Resources Overview Questions

A. In what ways are staff members’ qualifications examined?

Upon being hired by Bristol, applicants are required to provide information on educational and professional qualifications. These are discussed in detail during the application and interview processes. Human Resources requires that all staff provide educational records.

B. In what ways are staff members’ performance judged?

As previously mentioned, staff members’ qualifications and performance are discussed through annual performance evaluations. At this time, staff efforts and achievements are noted as well as areas of growth and improvement.

Part 5. ETHICS

Assessment Services (AS) must review relevant professional ethical standards and must adopt or develop and implement appropriate statements of ethical practice. AS must publish and adhere to statements of ethical practice and ensure their periodic review by relevant constituencies. AS must orient new staff members to relevant ethical standards and statements of ethical practice and related institutional policies. Statements of ethical standards must specify that staff members respect privacy and maintain confidentiality in all communications and records to the extent that such communications and records are protected under relevant privacy laws. Statements of ethical standards must specify limits on disclosure of information contained in students' educational records as well as requirements to disclose to appropriate authorities. AS must seek from the institutional review board approval to study human subjects for studies whose findings will be published beyond internal review of the institution. AS must know and adhere to the human subjects policies and procedures of the institution.

33

AS should seek approval to conduct assessment studies through the institution’s human subjects review process. Statements of ethical standards must address personal and economic conflicts of interest, or appearance thereof, by staff members in the performance of their work. Statements of ethical standards must reflect the responsibility of staff members to be fair, objective, and impartial in their interactions with others. Statements of ethical standards must reference management of institutional funds. Statements of ethical standards must reference appropriate behavior regarding research and assessment with human participants, confidentiality of research and assessment data, and students’ rights and responsibilities. Statements of ethical standards must include the expectation that AS staff members confront and hold accountable other staff members who exhibit unethical behavior. Statements of ethical standards must address issues surrounding scholarly integrity. As appropriate, AS staff members must inform users of programs and services of ethical obligations and limitations emanating from codes and laws or from licensure requirements. AS staff members must recognize and avoid conflicts of interest that could adversely influence their judgment or objectivity and, when unavoidable, recuse themselves from the situation. AS staff members must perform their duties within the limits of their position, training, expertise, and competence. When these limits are exceeded, individuals in need of further assistance must be referred to persons possessing appropriate qualifications. Promotional and descriptive information must be accurate and free of deception. AS must adhere to institutional policies regarding ethical and legal use of software and technology. AS must ensure that the privacy or anonymity of study participants and the confidential nature of data are not breached. AS must regularly purge identifiable information collected about students and other participants to protect their privacy, consistent with institutional policies and federal guidelines. AS should maintain raw data for a number of years, based on applicable policy, after the study’s written report is completed to respond to subsequent questions. Instances involving research on sensitive topics may require protections such as a Certificate of Confidentiality.

34

AS must acknowledge methodological limitations of assessment studies. These limitations could include unrepresentative samples, low response rate, or errors in trying to make decisions from ungeneralizable qualitative findings.

ND 0 1 2 3 4 5

Does Not Apply

Insufficient Evidence/

Unable to Rate

Does Not Meet

Partly Meets Meets Exceeds Exemplary

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

5.1

The Assessment Service (AS) reviews relevant professional ethical standards and implements appropriate statements of ethical practice

4 3 4 3 3 X

3.4

5.2

The AS publishes and adheres to statements of ethical practice and ensures periodic review by relevant constituents

4 3 4 3 3 X

3.4

5.3

The AS orients new staff members to relevant statements of ethical practice

and related institutional policies

3 3 3 3 3 X

3

5.4 The AS’s statement of ethical standards

5.4.1

specifies that staff members respect privacy and maintain confidentiality as appropriate

0 0 0 0 0 X

0

5.4.2

specifies limits on disclosure of student records as well as requirements to

disclose to appropriate authorities

0 0 0 0 0 X

0

5.4.3

addresses personal and economic conflicts, or appearance thereof, by staff members in the performance of their work

0 0 0 0 0 X

0

35

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

5.4.4

reflects the responsibility of staff members to be fair, objective, and impartial in their interactions with others

0 0 0 0 0 X

0

5.4.5 references management of institutional funds

0 0 0 0 0 X

0

5.4.6

references appropriate behavior regarding research and assessment with human

participants, confidentiality of research and assessment data, and students’ rights and responsibilities

0 0 0 0 0 X

0

5.4.7

includes the expectation that staff members confront and hold accountable other staff members who exhibit unethical behavior

0 0 0 0 0 X

0

5.4.8

addresses issues

surrounding scholarly integrity

0 0 0 0 0 X

0

5.5 AS staff members

5.5.1

inform users of programs of ethical obligations and limitations emanating from codes and laws or from licensure requirements

3 3 3 3 3 X

3

5.5.2

recognize and avoid conflicts of interest that could influence their

judgment and objectivity

3 3 3 3 3 X

3

5.5.3

perform duties within the limits of the position, training, expertise, and competence, and when limits are exceeded make referrals to persons possessing

3 3 3 3 3 X

3

36

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

appropriate qualifications

5.6

Promotional and descriptive information are accurate and free of deception

3 3 3 3 3 X

3

5.7

The AS adheres to institutional policies regarding ethical and legal use of software and technology

3 4 4 3 3 X

3.4

5.8 The AS

5.8.1

seeks from the

institutional review board approval to study human subjects for studies whose findings will be published beyond internal review of the institution

3 3 3 3 3 X

3

5.8.2

knows and adheres to the human subjects policies and procedures of the institution

3 3 3 3 4 X

3.2

5.9

The AS ensures the privacy or

anonymity of study participants and confidentiality of data

3 3 3 3 4 X

3.2

5.1

The AS regularly purges identifiable information collected about students and other participants

3 3 3 3 3 X

3

5.11

The AS acknowledges the methodological limitations of assessment studies

3 3 3 3 3 X

3

Part 5. Ethics Overview Questions

A. What ethical principles, standards, statements, or codes guide the program and its staff members?

OIRPA staff abide by and adhere to various ethical principles, standards, and legal codes including the Family Education Right to Privacy Act (FERPA). Further, all staff abide by the ethical standards set by the state ethics commission as Bristol is a state institution within the

37

Commonwealth of Massachusetts. All staff regularly complete a training in ethics and send the certificate of completion to Human Resources.

The BCC IRB was established by the Vice President in 2008. Since then, the Board has been managed by OIRPA to ensure the ethical treatment of human subjects participating in research at the College. As stated on the BCC IRB webpage, “The goal of the Institutional Review Board is to uphold the principles of the Belmont Report and other established guidelines, and ensure the rights, welfare, and privacy of individuals involved in research” (http://www.bristol.mass.edu/facultystaff/institutionalresearchplanningandassessment/institutionalreviewboard/). This crucial board of ethics consists of College and community members who are qualified through their experience and expertise to review specific research activities and determine the acceptability of proposed research. The Board has scheduled monthly meetings and get together on an “as needed” basis.

B. What is the program’s strategy for managing student and staff member confidentiality and privacy issues?

To maintain and protect student and staff confidentiality and privacy, College computers are locked when staff are not at their computers. All computers are password-protected but data are not encrypted. Private and confidential hardcopy material are secured in locked cabinets and keys are located in separate and secure areas.

C. How are ethical dilemmas and conflicts of interest managed?

Through Bristol's Office of Human Resources, a formal structure for handling ethical dilemmas and conflicts exist. If ethical dilemmas and conflicts arise, such would be handled through Human Resources protocol.

D. In what ways are staff members informed and supervised regarding ethical conduct?

The College community is informed of ethical practices through Human Resources. Ethical conduct and practices are upheld in OIRPA under the supervision of the Vice President.

Part 6. LAW, POLICY, AND GOVERNANCE

Assessment Services (AS) must be in compliance with laws, regulations, and policies that relate to their respective responsibilities and that pose legal obligations, limitations, risks, and liabilities for the institution as a whole. Examples include constitutional, statutory, regulatory, and case law; relevant law and orders emanating from codes and laws; and the institution's policies. AS must inform staff members, appropriate officials, and users of programs and services about existing and changing legal obligations, risks and liabilities, and limitations. AS must have written policies on all relevant operations, transactions, or tasks that have legal implications. AS must regularly review policies. The revision and creation of policies must be informed by best practices, available evidence, and policy issues in higher education.

38

AS staff members must use reasonable and informed practices to limit the liability exposure of the institution and its officers, employees, and agents. AS staff members must be informed about institutional policies regarding risk management, personal liability, and related insurance coverage options and must be referred to external sources if the institution does not provide coverage. The institution must provide access to legal advice for staff members as needed to carry out assigned responsibilities. AS must have procedures and guidelines consistent with institutional policy for responding to threats, emergencies, and crisis situations. Systems and procedures must be in place to disseminate timely and accurate information to students, other members of the institutional community, and appropriate external organizations during emergency situations. AS staff members must neither participate in nor condone any form of harassment or activity that demeans persons or creates an intimidating, hostile, or offensive environment. AS must obtain permission to use copyrighted materials and instruments. AS must purchase the materials and instruments from legally compliant sources or seek alternative permission from the publisher or owner. References to copyrighted materials and instruments must include appropriate citations. AS staff members must be knowledgeable about internal and external governance systems that affect programs and services.

ND 0 1 2 3 4 5

Does Not Apply

Insufficient Evidence/

Unable to Rate

Does Not Meet

Partly Meets Meets Exceeds Exemplary

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

6.1 The Assessment Service (AS)

6.1.1

is in compliance with laws, regulations, and policies that relate to its respective responsibilities and that pose legal obligations, limitations, risks, and liabilities for the institution as a whole

3 3 3 3 3 X

3

39

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

6.1.2

informs staff members, appropriate officials, and users of programs and services about existing and changing legal obligations, risks and liabilities, and limitations

3 3 3 3 3 X

3

6.1.3

has written policies on all relevant operations, transactions, or tasks that have legal implications

1 0 2 0 0 X 0.6

6.1.4

regularly reviews policies to ensure that they reflect best practices, available evidence, and policy issues in higher education

2 3 2 0 2 X 1.8

6.1.5

has procedures and guidelines consistent with institutional policy for responding to threats, emergencies, and crisis situations

3 3 3 3 3 X

3

6.1.6

has systems and procedures to disseminate timely and accurate information to students, other members of the institutional community, and appropriate external organizations during emergency situations

ND ND ND ND ND X

ND

6.1.7

obtains permission

to use copyrighted materials and instruments

3 3 3 3 3 X

3

6.1.8

purchases the materials and instruments from legally compliant sources or seeks permission from the publisher or owner

3 3 3 3 3 X

3

40

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

6.1.9

references copyrighted materials and instruments with appropriate citations

3 3 3 3 3 X

3

6.2 AS staff members

6.2.1

use reasonable and informed practices to limit the liability exposure of the institution and its officers, employees, and agents

3 3 4 3 3 X

3.2

6.2.2

are informed about institutional policies regarding risk management, personal liability, and related insurance coverage options and are referred to external sources if coverage is not provided by the institution

3 3 3 3 3 X

3

6.2.3

neither participate in nor condone any form of harassment that demeans persons or creates an intimidating, hostile, or offensive environment

3 3 3 3 3 X

3

6.2.4

are knowledgeable about internal and external governance systems that affect programs

3 3 4 3 3 X

3.2

6.3

The institution provides access to legal advice for AS staff members as needed to carry out assigned responsibilities

3 3 3 3 3 X

3

Part 6. Law, Policy, and Governance Overview Questions

A. What are the crucial legal, policy and governance issues faced by the program?

Legal issues as they relate to the Office include concerns regarding FERPA regulations, privacy, and confidentiality. At times OIRPA works with the Office of Disability Services (ODS) to ensure accommodations are provided, as needed, to assist staff or student participation in a research studies, surveys, or focus groups conducted at the college. Through the IRB, legal concerns are

41

always of the utmost importance, particularly upholding the physical, psychological, and emotional safety of research participants.

C. How are staff members instructed, advised, or assisted with legal, policy and governance concerns?

At Bristol, campus legal counsel is accessible to the College community.

C. How are staff members informed about internal and external governance systems?

OIRPA staff members are made aware of internal and external governance through the Vice President who is knowledgeable about College services available for legal, policy, or governance concerns.

Part 7. DIVERSITY, EQUITY, AND ACCESS

Within the context of each institution's unique mission and in accordance with institutional polices and all applicable codes and laws, Assessment Services (AS) must create and maintain educational and work environments that are

welcoming, accessible, and inclusive to persons of diverse backgrounds equitable and non-discriminatory

free from harassment AS must not discriminate on the basis of ability; age; cultural identity; ethnicity; family educational history (e.g., first generation to attend college); gender identity and expression; nationality; political affiliation; race; religious affiliation; sex; sexual orientation; economic, marital, social, or veteran status; or any other basis included in institutional policies and codes and laws. AS must

advocate for greater sensitivity to multicultural and social justice concerns by the institution and its personnel

modify or remove policies, practices, facilities, structures, systems, and technologies that limit access, discriminate, or produce inequities

include diversity, equity, and access initiatives within their strategic plans foster communication that deepens understanding of identity, culture, self-

expression, and heritage promote respect about commonalities and differences among people within

their historical and cultural contexts address the characteristics and needs of a diverse population when

establishing and implementing culturally relevant and inclusive programs, services, policies, procedures, and practices

provide staff members with access to multicultural training and hold staff members accountable for integrating the training into their work

respond to the needs of all students and other populations served when establishing hours of operation and developing methods of delivering programs, services, and resources

ensure physical, program, and resource access for persons with disabilities recognize the needs of distance learning students by providing appropriate

and accessible services and resources or by assisting them in gaining access to other appropriate services and resources in their geographic region

42

AS must ensure inclusion of persons with disabilities in data collection efforts in compliance with local, state/provincial, and federal guidelines. AS must design studies and collect data so that potential differential outcomes based on diversity among sub-groups can be explored.

ND 0 1 2 3 4 5

Does Not Apply

Insufficient Evidence/

Unable to Rate

Does Not Meet

Partly Meets Meets Exceeds Exemplary

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

7.1

The Assessment

Service (AS) creates and maintains educational work environments that are

7.1.1

welcoming, accessible, and inclusive to persons of diverse backgrounds

3 4 4 3 3 X

3.4

7.1.2 equitable and non-discriminatory

3 3 4 3 3 X

3.2

7.1.3 free from harassment

3 3 4 3 3 X

3.2

7.2

The AS does not discriminate on the basis of ability; age; cultural identity; ethnicity; family educational history; gender identity and expression; nationality; political affiliation; race; religious affiliation; sex; sexual orientation; economic, marital, social, or veteran status; or any other basis included in institutional policies and codes and laws

3 4 4 3 3 X

3.4

7.3 The AS

43

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

7.3.1

advocates for sensitivity to multicultural and social justice concerns by the institution and its personnel

3 3 4 3 3 X

3.2

7.3.2

modifies or removes policies, practices, facilities, structures, systems, and technologies that limit access, discriminate, or produce inequities

3 3 4 3 3 X

3.2

7.3.3

includes diversity, equity, and access initiatives within their strategic plans

3 3 4 3 3 X

3.2

7.3.4

fosters communication that deepens understanding of identity, culture, self-expression, and heritage

3 3 4 3 3 X

3.2

7.3.5

promotes respect about commonalities and differences among people within their historical and cultural contexts

3 2 4 3 3 X 3

7.3.6

addresses the characteristics and needs of a diverse population when establishing and implementing culturally relevant and inclusive programs, services, policies, procedures, and practices

3 3 3 3 3 X

3

7.3.7

provides staff members with access to multicultural training and holds staff members accountable for integrating the training into their work

3 3 4 3 3 X

3.2

44

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

7.3.8

responds to the needs of all students and other populations served when establishing hours of operation and developing methods of delivering programs, services, and resources

ND ND ND ND ND X

ND

7.3.9

ensures physical, program, and resource access for persons with disabilities

ND ND ND ND ND X

ND

7.3.10

recognizes the needs of distance learning students by providing appropriate and accessible services and resources or by assisting them in gaining access to other appropriate services and resources in their geographic region

ND ND ND ND ND X

ND

7.4

The AS ensures inclusion of persons with disabilities in data collection efforts in compliance with local, state/provincial, and federal guidelines

3 4 4 3 3 X

3.4

7.5

The AS designs studies and collects data so as to permit exploration of potential differential outcomes based on diversity among subgroups

3 3 4 3 3 X

3.2

45

Part 7. Diversity, Equity, and Access Overview Questions

A. How does the program ensure non-discriminatory, fair, and equitable treatment to all constituents?

Office information and services are available for all members of the College community as well as external constituents upon request. OIRPA staff members take into consideration the various needs of individuals who are requesting data and/or research assistance and strive to meet the needs of those who contact the Office. If needed, ORIPA staff seek out services at the College if more resources or assistance are needed. For instance, ODS is contacted if accommodations are needed for faculty, staff, or students to participate in the CCSSE.

B. What policies and/or practices are in place to address imbalances in participation among selected categories of students and imbalances in staffing patterns among selected categories of program staff members?

Through the analysis of data and utilization of assessment strategies, the Office is aware of the imbalances that exist for Bristol's students and staff. Office staff are involved in various initiatives in order to reduce these imbalances. For example, the Director is involved in the Leading for Change Team at BCC as well as the Consortium Planning Team which promotes the use of benchmarking and data collection to reduce staffing inequalities and student achievement gaps. OIRPA strives to enhance the diversity of Office staff.

C. How does the program create and maintain the educational and work environment to comply with institutional policies and all applicable codes and laws?

The Office strives to have staff who are aware of the diverse needs of Bristol's students, faculty, and staff. Upon hiring staff and work-study students, OIRPA staff are cognizant of the need for more diversity in the Office. All staff have participated in BCC’s diversity and inclusion training and will continue to seek out related professional development opportunities.

Part 8. INSTITUTIONAL AND EXTERNAL RELATIONS

Assessment Services (AS) must reach out to relevant individuals, groups, communities, and organizations internal and external to the institution to

establish, maintain, and promote understanding and effective relations with those that have a significant interest in or potential effect on the students or other constituents served by the programs and services

garner support and resources for programs and services as defined by the mission statement

disseminate information about the programs and services collaborate, where appropriate, to assist in offering or improving programs

and services to meet the needs of students and other constituents and to achieve program and student outcomes

engage diverse individuals, groups, communities, and organizations to enrich the educational environment and experiences of students and other constituents

AS must have procedures and guidelines consistent with institutional policy for

communicating with the media contracting with external organizations for delivery of programs and services

46

cultivating, soliciting, and managing gifts applying to and managing funds from grants

AS must adhere to all institutional policies with respect to the communication of student data. Regular and effective communication systems for the dissemination of results must be established and maintained with academic and administrative offices, institutional governance bodies, and other appropriate constituencies.

ND 0 1 2 3 4 5

Does Not Apply

Insufficient Evidence/

Unable to Rate

Does Not Meet

Partly Meets Meets Exceeds Exemplary

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

8.1

The Assessment Service (AS) reaches out to internal and external populations to

8.1.1

establish, maintain, and promote understanding and effective relations with those that have a significant interest in or

potential effect on the students or other constituents served by the programs and services

4 4 4 3 3 X 3.6

8.1.2

garner support and resources for programs and services as defined by the mission statement

3 3 4 3 3 X 3.2

8.1.3

disseminate information about the programs and services

3 3 3 3 3 X 3

8.1.4

collaborate, where appropriate, to assist in offering or improving programs and services to meet the needs of students and other constituents and to achieve program and student outcomes

4 4 4 4 4 X 4

47

8.1.5

engage diverse individuals, groups,

communities, and organizations to enrich the educational environment and experiences of students and other constituents

4 4 4 4 4 X 4

8.2

The AS has procedures and guidelines consistent with institutional policy to

8.2.1 communicate with the media

ND ND ND ND ND X ND

8.2.2

contract with external organizations for delivery of programs and services

3 4 4 3 3 X 3.4

8.2.3 cultivate, solicit, and manage gifts

ND ND ND ND ND X ND

8.2.4 apply to and manage funds from grants

ND ND ND ND ND X ND

8.3

The AS adheres to all institutional policies on communicating student data

4 3 4 3 4 X 3.6

8.4

The AS establishes and maintains regular and effective communication systems for disseminating assessment results to academic and administrative offices, institutional governance bodies, and other constituencies

3 3 4 3 4 X 3.4

Part 8. Institutional and External Relations Overview Questions

A. With which relevant individuals, campus offices, and external agencies must the program maintain effective relations?

Due to the collaborative nature of the work of the Office, OIRPA staff interacts with all areas of the College in order to collect essential data for various projects as well as provide assistance and guidance in the utilization of information. OIRPA also provides data on a routine basis to state and national databases (HEIRS and IPEDS). Further, the NEASC Annual Report is

48

completed as well as data collection methods to assist in the College’s self-study activities. The Office also fulfills data requests by members of the community and must comply with the Freedom of Information Act.

B. What evidence confirms effective relationships with program constituents?

OIRPA staff have effective relationships at the College as evidenced by continuous and consistent participation on various committees and teams in which a variety of data is provided to guide the work of these groups. OIRPA always provides data to HEIRS and IPEDS in a timely manner and has been commended by IPEDS Program Director, Mr. Reeves, for completing all IPEDS surveys in two rather than three reporting periods over the past four years.

Part 9. FINANCIAL RESOURCES

Assessment Services (AS) must have funding to accomplish the mission and goals. In establishing funding priorities and making significant changes, a comprehensive analysis must be conducted to determine the following elements: unmet needs of the unit, relevant expenditures, external and internal resources, and impact on students and the institution. AS must demonstrate efficient and effective use and responsible stewardship of fiscal resources consistent with institutional protocols. Financial resources should be sufficient to support study conceptualization, data collection, data entry and analysis, and the dissemination of assessment and research findings, as well as methodological training for staff.

ND 0 1 2 3 4 5

Does Not Apply

Insufficient Evidence/

Unable to Rate

Does Not Meet

Partly Meets Meets Exceeds Exemplary

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

9.1

The Assessment Service (AS) has adequate funding to accomplish its mission and goals

2 2 2 2 2 X 2

9.2

The AS

demonstrates fiscal responsibility, responsible stewardship, and cost-effectiveness consistent with institutional protocols

3 3 3 3 3 X 3

49

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

9.3

An analysis of expenditures, external and internal resources, and impact on the campus community is completed before

9.3.1 establishing funding priorities

ND ND ND ND ND X ND

9.3.2 making significant changes

3 3 3 3 3 X 3

Part 9. Financial Resources Overview Questions

A. What is the funding strategy for the program?

The Office is funded by an institutional base budget. The Vice President is also allowed to submit a one-time budget request each year for additional initiatives or projects in which more funds are needed. This request goes to the Budget Committee and the Vice President attends a meeting with the budget committee each year to explain the request in more detail.

B. What evidence exists to confirm fiscal responsibility, responsible stewardship, and cost-effectiveness?

All expenditures must go through a process in which support for purchases is required. Monies

cannot be spent without approval for purchase. Since 2015, the Vice President has requested a

Staff Associate as well as software to improve Office efficiency. These requests have been

denied.

Part 10. TECHNOLOGY

Assessment Services (AS) must have adequate technology to support the achievement of their mission and goals. The technology and its use must comply with institutional policies and procedures and be evaluated for compliance with relevant codes and laws. AS must use current technology to provide updated information regarding mission, location, staffing, programs, services, and official contacts to students and designated clients. AS must explore the use of technology to enhance delivery of programs and services, especially for students at a distance or external constituencies. When technology is used to facilitate student learning and development, AS must select technology that reflects intended outcomes. AS must

50

maintain policies and procedures that address the security, confidentiality, and backup of data, as well as compliance with privacy laws

have clearly articulated plans in place for protecting confidentiality and security of information when using Internet-based technologies

develop plans for replacing and updating existing hardware and software as well as plans for integrating new technically-based or supported programs, including systems developed internally by the institution, systems available through professional associations, or private vendor-based systems

Technology, as well as workstations or computer labs maintained by programs and services for student use, must be accessible to all designated clients and must meet established technology standards for delivery to persons with disabilities. When providing student access to technology, AS must

have policies on the use of technology that are clear, easy to understand, and available to all students

provide assistance, information, or referral to appropriate support services to those needing help accessing or using technology

provide instruction or training on how to use the technology inform students on the legal and ethical implications of misuse as it pertains

to intellectual property, harassment, privacy, and social networks Student violations of technology policies must follow established institutional student disciplinary procedures. Students who experience negative emotional or psychological consequences from the use of technology must be referred to support services provided by the institution. AS must have access to sufficient data analysis software for efficient data collection, storage, retrieval, and analysis. Both statistical analysis software and qualitative analysis software should be available. AS must have timely access to appropriate institutional records. AS should advocate for integration of institutional databases.

ND 0 1 2 3 4 5

Does Not Apply

Insufficient Evidence/

Unable to Rate

Does Not Meet

Partly Meets Meets Exceeds Exemplary

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

10.1

The Assessment Service (AS) has adequate technology to support its mission and goals

1 3 2 2 3 X 2.2

10.2 Use of technology in the AS complies

4 3 4 3 3 X 3.4

51

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

with institutional policies and procedures and legal requirements

10.3

The AS uses current technology to provide updated information regarding mission, location, staffing, programs, services, and official contacts to students and designated clients

2 3 2 2 3 X 2.4

10.4

The AS explores use of technology to enhance delivery of programs and services, especially for students at a distance and external constituencies

ND ND ND ND ND X ND

10.5

The AS uses technology that facilitates learning and development and reflects intended outcomes

2 3 2 3 3 X 2.6

10.6 The AS

10.6.1

maintains policies and procedures that address the security, confidentiality, and backup of data, as well as compliance with privacy laws

3 3 4 4 3 X 3.4

10.6.2

has plans in place for protecting confidentiality and security of

information when using Internet-based technologies

3 4 4 3 3 X 3.4

10.6.3

develops plans for replacing and updating existing hardware and software as well as for integrating new technically-based or -

3 3 3 3 3 X 3

52

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

supported programs

10.7

Workstations and computer labs maintained by the AS for student use are accessible to all designated clients and meet standards for delivery to persons with disabilities

ND ND ND ND ND X ND

10.8 The AS provides

10.8.1

access to policies on technology use that are clear, easy to understand, and available to all students

ND ND ND ND ND X ND

10.8.2

assistance, information, or referral to appropriate support services to those needing help accessing or using technology

3 4 3 3 3 X 3.2

10.8.3 instruction or training on how to use technology

ND ND ND ND ND X ND

10.8.4

information on the legal and ethical implications of misuse as it pertains to intellectual property, harassment, privacy, and social networks

3 3 4 3 3 X 3.2

10.9

Student violations of technology are addressed in student disciplinary procedures

ND ND ND ND ND X ND

10.1

A referral support system is available for students who experience negative emotional or psychological consequences from the use of technology

ND ND ND ND ND X ND

53

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

10.11

The AS has access to sufficient data analysis software for efficient data collection, storage, retrieval, and analysis

2 2 2 2 2 X 2

10.12

The AS has timely access to appropriate institutional records

3 3 3 3 3 X 3

Part 10. Technology Overview Questions

A. How is technology inventoried, maintained, and updated?

OIRPA is not responsible for inventorying, maintaining, or updating technology. This falls within the purview of Information Technology Services (ITS).

B. What evidence exists to confirm that technology is available for all who are served by the program?

A major tool that is used by the Office to gather data for those served by OIRPA is ARGOS. This program provides various data reports based on information that is available in Banner. The Office utilize hundreds of reports in ARGOS to meet the needs of the College. Office staff are also striving to empower those who request data to use ARGOS in order to independently generate their own information. OIRPA staff are available to work with those who generate their own data to assist with the interpretation and use of information.

Part 11. FACILITIES AND EQUIPMENT

Assessment Services (AS) must have adequate, accessible, and suitably located facilities and equipment to support the mission and goals. If acquiring capital equipment as defined by the institution, AS must take into account expenses related to regular maintenance and life cycle costs. Facilities and equipment must be evaluated on an established cycle, including consideration of sustainability, and be in compliance with codes and laws to provide for access, health, safety, and security. AS staff members must have workspace that is well equipped, adequate in size, and designed to support their work and responsibilities. For conversations requiring privacy, staff members must have access to a private space. AS staff members who share workspace must be able to secure their own work. The design of the facilities must guarantee the security and privacy of records and ensure the confidentiality of sensitive information. AS must have sufficient secure storage facilities to maintain materials related to studies.

54

The location and layout of the facilities must be sensitive to the needs of persons with disabilities as well as the needs of other constituencies.

ND 0 1 2 3 4 5

Does Not Apply

Insufficient Evidence/

Unable to Rate

Does Not Meet

Partly Meets Meets Exceeds Exemplary

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

11.1 The Assessment Service (AS)

11.1.1

has adequate, accessible, and suitably

located facilities and equipment to support its mission and goals

2 3 3 2 2 X 2.4

11.1.2

takes into account expenses related to regular maintenance and life-cycle costs when purchasing capital

equipment

ND ND ND ND ND X ND

11.2 AS facilities and equipment

11.2.1 are evaluated regularly

3 3 3 3 3 X 3

11.2.2

are in compliance with relevant legal and institutional requirements that ensure access, health, safety, and security of students and other users

3 3 3 3 3 X 3

11.3 AS staff members have

11.3.1

workspace that is well equipped, adequate in size, and designed to support their work

2 2 2 2 2 X 2

55

Criterion Measures Rater

1 Rater

2 Rater

3 Rater

4 Rater

5 Consistent Discrepant

Average Rating

11.3.2

access to appropriate space for private conversations

2 3 2 2 2 X 2.2

11.3.3

the ability to adequately secure their work

3 3 3 3 3 X 3

11.4

The facilities guarantee security and privacy of records and ensure confidentiality of sensitive

information

3 3 3 3 3 X 3

11.5

The AS has sufficient secure storage facilities to maintain materials related to studies

3 3 3 3 3 X 3

11.6

The location and layout of the facilities are sensitive to the needs of persons with disabilities as well as with the needs of other constituencies

2 3 2 2 2 X 2.2

Part 11. Facilities and Equipment Overview Questions

A. How are facilities inventoried and maintained?

Facilities and equipment are inventoried and maintained by the respective areas from which they originate - Information Technology Services, Facilities, etc.

B. What evidence exists to confirm that access, health, safety, and security of facilities and equipment are available for all who are served by the program?

Access, health, safety, and security fall within the broader campus context of services offered by facilities and campus police. The campus building in which the Office resides currently meets the requirements and expectations of the Office.

Part 12. ASSESSMENT AND EVALUATION

Assessment Services (AS) must have a clearly articulated assessment plan to document achievement of stated goals and learning outcomes, demonstrate accountability, provide evidence of improvement, and describe resulting changes in programs and services.

56

AS must have adequate fiscal, human, professional development, and technological resources to develop and implement assessment plans. Assessments must include direct and indirect evaluation and use qualitative and quantitative methodologies and existing evidence, as appropriate, to determine whether and to what degree the stated mission, goals, and intended outcomes are being met as effectively and efficiently as possible. The process must employ sufficient and sound measures to ensure comprehensiveness. Data collected must include responses from students and other constituencies, and aggregated results must be shared with those groups. Results of assessments must be shared appropriately with multiple constituents. AS should model good assessment practices related to its own operations as well as when assisting other entities within the institution. Results of assessments and evaluations must be used to identify needs and interests in revising and improving programs and services, recognizing staff performance, maximizing resource efficiency and effectiveness, improving student achievement of learning and development outcomes, and improving student persistence and success. Changes resulting from the use of assessments and evaluation must be shared with stakeholders.

ND 0 1 2 3 4 5

Does Not Apply

Insufficient Evidence/

Unable to Rate

Does Not Meet

Partly Meets Meets Exceeds Exemplary

Criterion Measures

Rater 1

Rater 2

Rater 3

Rater 4

Rater 5

Consistent Discrepant Average Rating

12.1

The Assessment Service (AS) has a clearly-articulated assessment plan to

12.1.1

document achievement of its stated goals and learning outcomes

3 2 3 3 3 X 2.8

12.1.2 demonstrate accountability

3 3 3 3 3 X 3

12.1.3 provide evidence of improvement

3 2 3 3 3 X 2.8

12.1.4 describe resulting changes

3 3 3 3 3 X 3

12.2

The AS has adequate resources in the following dimensions to develop and implement assessment plans:

12.2.1 fiscal 2 2 2 2 2 X 2

12.2.2 human 2 2 2 2 2 X 2

12.2.3 professional development

3 3 3 3 3 X 3

57

Criterion Measures

Rater 1

Rater 2

Rater 3

Rater 4

Rater 5

Consistent Discrepant Average Rating

12.2.4 technology 2 3 2 2 3 X 2.4

12.3

AS employs direct and indirect evaluation and qualitative and quantitative methodologies to

12.3.1 determine achievement of mission and goals

3 3 3 3 3 X 3

12.3.2

determine achievement of learning and development outcomes and whether they are met effectively and

efficiently

2 2 2 2 2 X 2

12.3.3 ensure comprehensiveness

3 3 3 3 3 X 3

12.4 Data are collected from students and other constituencies

3 3 3 3 3 X 3

12.5

Assessments are shared appropriately with multiple constituencies

4 4 4 3 3 X 3.6

12.6 Assessment and evaluation results are used to

12.6.1

identify needs and interests in revising and improving programs and services

3 3 4 3 3 X 3.2

12.6.2 recognize staff performance

3 3 3 3 3 X 3

12.6.3 maximize resource efficiency and effectiveness

3 4 4 3 3 X 3.4

12.6.4

improve student learning and development outcomes

2 3 3 2 3 X 2.6

12.6.5 improve student persistence and success

3 4 4 3 3 X 3.4

12.7

Changes resulting from assessment and evaluation are shared with stakeholders

3 3 4 3 3 X 3.2

Part 12. Assessment and Evaluation Overview Questions

A. What is the comprehensive assessment strategy for the program?

The results of this CAS Functional Area Review will assist with the development of a larger College-wide Assessment Plan. The first step in developing a comprehensive plan is the revision

58

of the College’s General Education Outcomes to ensure the College has an updated set of outcomes for all students. A wealth of data are collected by this Office and efforts are made to continually distribute this information to close the feedback loop. The major vehicle through which this goal will be accomplished is through the recently developed Educational Effectiveness Committee.

B. How are tangible, measurable outcomes determined to ensure program achievement of mission and goals

Program achievement of mission and goals are most notably discussed in monthly and annual reports that are completed by the Office. More staff and technology are needed to continually support the continuous feedback loop for the assessment and evaluation of the Office and the Institution.

General Standards revised in 2011; AS (formerly Outcomes Assessment and Program Evaluation) content developed/revised in 1986, 1997, & 2008

59

CAS Work Form A

Assessment, Ratings, and Significant Items

INSTRUCTIONS: Step One: This work form should be completed following a review of the individual ratings of the team members. Examine the ratings of each criterion statement by the team members, and record the following in the form below:

Discrepancies: Item number(s) for which there is a substantial rating discrepancy (two or more ratings apart). These items will need to be discussed further by team members.

Strengths: Item number(s) for which all participants have given a rating of 4 & 5, indicating agreement that the criterion Exceeds the standard or is Exemplary.

Needed Improvements: Item numbers for which all participants have given a rating of 1 & 2, indicating agreement that the criterion Does Not Meet or Partly Meets the standard.

Items not listed in one of these categories represent consensus among the raters that practice in that area is satisfactory, having been rated a 3, which indicates Meets the standard. Items rated 0 because of Insufficient Evidence/Unable to Rate should be listed in Needed Improvements. Step Two (below): List the items needing follow-up action for improvement and indicate what requires attention. The team or coordinator should consider including any criterion measure rated as being not met by any reviewer, as well as those with significant discrepancies that are not resolved by team discussion.

Part Number

Part

Discrepancies Strengths

Needed Improvements

1 Mission

1.1 and 1.3.4

2 Program 2.1.1, 2.4.2, 2.9.3, and 2.9.10

3 Organization and Leadership

3.2.4, 3.2.5, 3.3.2, 3.3.3, 3.3.4, 3.4.1, 3.4.3, 3.4.4, 3.4.7, 3.5.2, 3.5.5, 3.7.3, 3.7.6, and 3.7.7

3.6.1 and 3.7.1

4 Human Resources 4.6, 4.7.1, 4.7.2, 4.8.1, and 4.8.2

4.8.4 4.1 and 4.3.5

5 Ethics

6

Law, Policy, and Governance

6.1.3 and 6.1.4

60

7

Diversity, Equity, and Access

7.3.5

8

Institutional and External Relations

9 Financial Resources 9.1

10 Technology 10.1 10.11

11 Facilities and Equipment

11.3.1

12 Assessment and Evaluation

12.2.1, 12.2.2, and 12.3.2

Step Two: List item number(s) for each Part determined to merit follow-up, and describe the practice weaknesses that require attention.

1.1 The Assessment Service (AS) mission is to develop a comprehensive assessment program to

increase the institution’s knowledge about students, the educational environment, and

institutional effectiveness to continuously improve student programs and services and enhance

student learning

OIRPA needs to compose a Mission Statement and ensure that it references the manner in which

OPIRA supports student learning and development.

2.1.1 The Assessment Service (AS) promotes student learning and development outcomes that

are purposeful

OIRPA needs to provide evidence regarding the manner in which it supports student learning and

development through its Mission Statement as well as its policies and procedures. It will also

assert the ways in which the Office is guided by theories of learning and development via current

and available literature.

3.2.4 In providing strategic planning, AS leaders promote environments that provide meaningful

opportunities for student learning, development, and engagement

While the Office is highly involved in the development of the Strategic Plan, policies and

procedures need to be composed to outline the role of Office staff and how Strategic Planning

activities promote the growth of student learning, development, and engagement.

61

3.3.2 In providing supervision, AS leaders influence others to contribute to the effectiveness and

success of the unit

Policies regarding supervision need to be established in order to outline the ways in which

supervision is effective in the success of the Office and how it supports the acceptance of

leadership opportunities for staff and the promotion of professional development in this area.

3.4.1 In providing management, AS leaders identify and address individual, organizational, and

environmental conditions that foster or inhibit mission achievement

Policies regarding management practices are needed regarding mission achievement, informed

decision-making, sustainability practices, and risk-management.

4.6. Supervisors of AS interns or graduate students adhere to all parameters of job descriptions,

work hours, and schedules

OIRPA needs to compose specific policies and procedures regarding the selection, training,

supervision, and evaluation of student interns and employees.

4.1 The Assessment Service (AS) is staffed adequately to accomplish mission and goals

While the Office is able to accomplish a great deal of activities and initiatives, more staff are

needed in order to conduct larger, college-wide assessment and research studies and to provide

oversight to close the loop on several assessment and research activities.

4.3.5 The AS has technical and support staff members adequate to accomplish the mission

In order to provide more resources and promote data transparency to the College Community,

the Office needs a comprehensive dashboard system. Predictive analytics software would also

provide a great deal of assistance for planning and decision-making at the College.

5. None

6.1.3 The Assessment Service (AS) has written policies on all relevant operations, transactions,

or tasks that have legal implications

A range of policies and procedures, as mentioned above, need to be composed by OIRPA and

reviewed regularly to ensure they reflect best practices, available evidence, and policy issues in

higher education

7. None

62

8. None

9.1. The Assessment Service (AS) has adequate funding to accomplish its mission and goals

More funding is needed by the Office in order to obtain adequate staff and software as previously

mentioned in 4.1. and 4.3.5.

10.1 The Assessment Service (AS) has adequate technology to support its mission and goals

See explanation for 4.3.5 above.

11.3.1 AS staff members have workspace that is well equipped, adequate in size, and designed

to support their work

If there is an increase in employees in the Office, in addition to student employees, more space

will be needed to support the work of the Office.

The AS has adequate resources in the following dimensions to develop and implement

assessment plans:

12.2.1 fiscal

12.2.2 human

12.2.3 professional development

To accomplish a comprehensive and active assessment plan, OIRPA will need more support in

terms of technology, staff, and professional development to support this major initiative.

63

CAS Work Form B

Follow-Up Actions

INSTRUCTIONS: The purpose of this work form is to begin the planning for action to be taken on practices judged to merit follow-up (Work Form A, Step Two). In the chart below, as Step Three, transfer short descriptions of the practices requiring follow-up and detail these items using the table format provided.

Step Three: Describe the current practice that requires change and actions to initiate the change

Practice Description

Corrective Action Sought Task Assigned To

Timeline Due Dates

Mission Statement

Compose Mission Statement and post on OIRPA webpage

Director Fall 2018

Policies and Procedures Manual

Compose a comprehensive policies and procedures manual including, but not limited to: all Office projects, Strategic Planning, ethics, supervision of staff and student employees, and management practices.

All Staff

Spring 2019

Increased Staff

One-time budget request will include a Staff Associate position. Requests for this position have been denied since 2015.

Vice President December 2017

Increased Technology

One-time budget request will include dashboard or predictive analytics software. Requests for this software have been denied since 2015.

Vice President December 2017

Promote OIRPA

While not evident in the ratings, through conversations with CAS team members it is clear that OIRPA needs to promote Office services and increase the transparency and accessibility of data. This will be done through regular BCC Weekly articles and an increased use of ARGOS for independent use by colleagues.

Director Spring 2019

64

CAS

Work Form C Summary Action Plan

Step Four: This form concludes the self-assessment process and calls for action to be taken as a consequence of study results. Write a brief action plan statement in the spaces below for each Part in which action is required. (Note: If using the electronic/CD version, text boxes will expand with typing.) Part 1: Mission

A comprehensive Mission Statement will be composed and posted on the OIRPA webpage by or before Fall 2018.

Part 2: Program

The OIRPA Mission Statement will include the manner in which the Office supports student learning and development using current research literature to guide its work.

Part 3: Organization and Leadership

Policies outlining OIRPA’s role in Strategic Planning will be composed by Spring 2019 along with policies regarding Office management and supervision.

Part 4: Human Resources

OIRPA will compose specific policies regarding student interns and employees by Spring 2019. Budget requests will be submitted by December 2017 for increased staff and technology.

Part 5: Ethics

None.

Part 6: Law, Policy, and Governance

Policies regarding best practices, available evidence, and policy issues in higher education will be composed to address the role of OIRPA at the College.

Part 7: Diversity, Equity, and Access

None.

Part 8. Institutional and External Relations

None.

65

Part 9: Financial Resources

See Part 4 above.

Part 10: Technology

See Part 4 above.

Part 11: Facilities and Equipment

If there is increased staff in the Office, in addition to student employees, more space will be requested to support the work of the Office.

Part 12: Assessment and Evaluation

To form a comprehensive and active assessment plan, OIRPA will strive to maintain more support in terms of technology, staff, and professional development.