outcomes) assessment) handbook€¦ · communication, oral communication, critical thinking,...

Outcomes Assessment Handbook

Spring 2012

Outcomes Assessment Committee (A. Helwig) South Suburban College

Spring 2012

South Suburban College | 15800 S. State Street |South Holland, IL 60473 Page 1

2012 Outcomes Assessment

Outcomes Assessment Handbook

I. SSC Mission Statement/OA Mission Statement II. OA Committee III. History of OA at SSC IV. Assessment at SSC

• General Education Assessment • Program Assessment • Departmental Assessment

Appendix

A. Glossary of terms B. College Wide Assessment Rubrics C. Livetext Instructions D. Annual Update Form E. Sample Annual Update F. Annual Update Review Checklist



I. SSC Mission Statement Our mission is to Serve our Students and the Community through lifelong learning. OA Mission Statement

At SSC, we take great pride in examining how well our students learn so that we can continually improve the learning process. Whether it is a course, program, or general education component, we utilize various assessment techniques to determine what works best for student success toward mastery of the learning objectives. Our faculty and administrators from academic and student services areas collaborate and modify instruction and services for optimal learning. We are an institution which is learning focused.



II. The Outcomes Assessment Committee (Excerpt from 2010-2012 Agreement Between South Suburban College and the South Suburban College Faculty Association; Sec 6.2, p.30.)

The Outcomes Assessment Committee will serve as a liaison to faculty and administration regarding outcomes assessment matters and will serve as representatives of the faculty and administration in providing advice to the Vice President for Academic Services on matters relating to outcomes assessment. Working with the Vice President of Academic Services, the Committee will coordinate the implementation and administration of the plan for assessing academic achievement filed with the Higher Learning Commission of the North Central Association (HLC). On an annual basis the committee will assist departments in reviewing and revising their plans for assessing student academic achievement and adjust the College’s assessment plan accordingly. The members will serve as resource persons on outcome assessment for the College community and will recommend overall policy and philosophy for outcomes assessment matters at the College. During faculty development days, prior to the beginning of Fall classes, each of the following faculty areas shall select a faculty member from their area to represent their area on the committee: Communications and Humanities, Mathematics, Science, English, Social and Behavioral Sciences, Developmental, Counseling, Engineering and Technology, Allied Health, Human Services, Nursing, Business, Office Automation Technology, and Computer Information Science. The Vice President for Academic Services will appoint five (5) administrators to serve on the committee. The Committee selects from among its membership a Chair who must be a faculty member.



III. History of Outcomes Assessment at SSC



IV. Outcomes Assessment Today General Education Assessment at SSC

*For General Education courses, only one of the two assessment activities per year may use the Oral or Written rubric. The other activity must use the Critical Thinking, Technology or Research Skills rubric.

General Educa,on Assessment:

"OA Rubrics with LiveText"

WHO

EVERY Faculty member

WHEN At least one class per semester; Data due by

Dec 31st (fall) May 31st (spring)

WHAT * Use one of the five OA rubrics to assess an acUvity in class and report the data using LiveText.

South Suburban College

Collegewide General Education Assessment



In an effort to ensure the measurability of the general education objectives, the department chairs have revised many of the objectives. In addition, departments have also aligned their course objectives with the overall general education objective of the college. The revised General Education Objectives included below.

COMMUNICATION C1 Students will compose written English in a grammatically correct, well-organized and

coherent manner for a variety of purposes. C2 Students will compose, develop and deliver effective, English, language-based oral

presentations for a variety of purposes. C3 Students will demonstrate an understanding of the various forms of rhetoric. C4 Students will integrate library/research skills into various areas of study.

MATHEMATICS M1 Students will demonstrate analytical, quantitative, and problem-solving skills. M2 Students will integrate mathematical relationships with other fields of study.

HUMANITIES H1 Students will identify and analyze ideas, movements, civilizations and cultures. H2 Students will critically assess and express ideas about cultural diversity and individual

roles in society.

FINE ARTS FA1 Students will apply aesthetic reasoning for the expression, analysis, and interpretation of

art forms. FA2 Students will express themselves by creating or performing works of verbal or non-verbal

Art.

SOCIAL AND BEHAVIORAL SCIENCES SB1 Students will examine human thought processes and behaviors in diverse populations,

cultures and settings. SB2 Students will use the principles of Social and Behavioral Science to analyze and interpret

individual and world issues.

PHYSICAL AND LIFE SCIENCE PS1 Students will demonstrate an ability to effectively use scientific terminology. PS2 Students will apply fundamental scientific conceptual frameworks for understanding and

evaluating natural phenomena and their causes and effects.

TECHNOLOGY T1 Students will demonstrate the use of a variety of forms of technology as fundamental

tools.



T2 Students will demonstrate the effective application of current computers and software to a particular field of study.

Many academic areas have chosen to assess the general education objectives at SSC using college-wide rubrics developed by the Outcomes Assessment Committee. During the 2008-09 academic year, areas of the college were surveyed with respect to the types of assessment being utilized and the concepts being assessed. After compiling all of the data, five major themes were obvious. Across the college, instructors are assessing students’ communication skills, both written and oral; critical thinking skills; research skills; and technology related skills. One concern was the varied methods used to assess these skills in our students.

In order to provide one consistent method of assessment, the Outcomes Assessment committee developed general rubrics to be used to assess activities in the areas of written communication, oral communication, critical thinking, research skills, and technology skills. The rubrics were developed to be interdisciplinary so they could be used across the college in any discipline. All faculty are asked to use one of the five Outcomes Assessment rubrics in at least one class each semester. This includes all full-time and part-time faculty. Instructors in general education courses are asked to only use a communication rubric once per year and select another rubric (critical thinking, research skills or technology) for the second semester. The results are then entered into LiveText (www.Livetext.com) according to the published deadlines. For access to Livetext please contact Zac Hughes at x2225.



Faculty Rubric Data Submission Deadlines Fall Semester Each faculty should use at least one OA rubric in at least one course during the semester. By midterm, create your activity in Livetext. By December 31st, enter the assessment scores from the fall semester. Spring Semester Each faculty should use at least one OA rubric in at least one course during the semester. By midterm, create your activity in Livetext. By May 31st, enter the assessment scores from the spring semester. Summer Semester-optional Each faculty should use at least one OA rubric in at least one course during the semester. By July 1st, create your activity in Livetext. By August 15th , enter the assessment score from the summer term. Suggestions: If an instructor is teaching multiple sections of the same course in a semester, it is encouraged to submit assessment scores for the same activity for all of the sections. The more data we have at our disposal the better the subsequent analysis. Within a department, rubric data across several sections of the same course could be utilized in the preparation of departmental annual updates.



LiveText Instructions (more detailed instructions are included in the appendix)

I. Login to Livetext – http://www.livetext.com Use the username/password sent by Livetext (or Contact Zac Hughes at x2225) II. Open a course Use the course tab to select from a list of available courses III. Assignments a. Create a NEW assignment i. Click on the Assignment tab, Select NEW ii. Give the assignment a title iii. Choose the posting date-NOW, due date (optional) iv. Attach rubric 1. Go to Assessment Method 2. Click on Insert Assessment Rubric(s) (BROWSE to find desired rubric) 3. Select appropriate rubric,

then click INSERT DOCUMENT to add the rubric. 4. SAVE assignment b. Scoring Students Assignments

i. From the Dashboard, click on the Student Progress bar for the desired assignment.

ii. Select a student to assess by clicking on the student’s name iii. Click the Assessment Rubrics button to open the rubric. iv. Click in the appropriate areas of the rubric to score the student work

iv. Click on the conversion bubble to add comments (optional) v. Click Submit Assessment

*To assess multiple students at one time, check the box next to each desired student, then select “Assess Selected Student(s)” and proceed as above

IV. Print rubric (OPTIONAL)

a. “Undock” the rubric (button in upper right hand corner of rubric window), turns rubric into an internet window.

b. In your internet browser application, go to FILE and PRINT to obtain a copy of the completed rubric to return to the student. This prints the rubric without the student name, it is suggested that the students initials are entered as part of the comment line in the rubric.



Program Assessment

New Program requirements

Together with the Curriculum Committee, an Outcomes Assessment plan is required with any proposals for new courses or programs of study. The department is asked to complete the first part of an annual update outlining the student objectives and their expected outcomes. These are then reviewed by the outcomes assessment committee and suggestions are returned to the department. For new programs this information may be covered across several courses over an extended period. Programmatic objectives can be clearly outlined through the use of a program map outlining where and when each objective will be addressed and what level of competency is expected, as well as how each objective is to be measured.

Career Program Rubric Collaboration Project

Several career programs from the College have collaborated on a rubric development project with LiveText and other surrounding community colleges. Faculty from the participating college programs worked together within their disciplines to develop rubrics to assess student objectives throughout their time in the program. To date, we have program wide rubrics for Radiologic Technology, Graphic Design, and Occupational Therapy Assistant. These rubrics are a very useful tool for assessing a student’s achievement as they progress through a program.

Program Review Cycle



Departmental Assessment

*If an assessment project was not completed in a particular academic year, explain why and propose a plan for the next year.

Annual Updates

The Annual Update was developed as a tool to gather data on the assessment of students, on a common set of objectives, across several courses/sections,. As the name suggests, it is completed on an annual basis. A department/program develops a plan to evaluate student achievement in regards to a set of objectives. The plan is implemented over the course of an academic year. The results are then analyzed and student success is reviewed. The plan is then adjusted to address the students’ success in terms of the department’s expected outcomes. New elements may be added to the plan to achieve greater success or new projects can be proposed as goals are met. Ideally, assessment of student objectives is achieved using direct measures which provide measurable outcomes. A final course grade or a score on an instructor’s unit exam may not be enough.

Departmental/Program Assessment:

"Annual Update"

WHO

Dept Chair Program Coordinator

or Designee

WHEN

The report is to be submiYed electronically by June 1st each

year to the Outcomes Assessment chair.

WHAT * Report on the departmental assessment

project results for the year and demonstrate how the results were used to design the plan for the next year. Plans are reviewed by the

commiYee and feedback provided.

South Suburban College

Departmental/Program Assessment



Annual Update Submission Guidelines Preparation - The Annual Update should be prepared during April/May of each academic year. Submission – The deadline for submission of the Annual Update is May 31st of each year. The report should be ELECTRONICALLY submitted to the academic administrator and the OA committee chair (currently Anna Helwig, [email protected]) Review - All Annual Updates will be reviewed by OA committee members based on the Annual Update Review Checklist. Feedback will be provided to the department/program for incorporation into the plan prior to implementation. Implementation - The plan proposed in the Annual Update should be put into action during the Fall and Spring terms of the academic year. Data Analysis - During the spring semester, fall data can be analyzed as well as the spring data as it becomes available. Evaluate student performance according to the expected outcomes. Then, summarize the results and update the action plan for the next year. Best Practices: 1. Data should be collected for multiple cycles. The department/program may find

it necessary to start collecting data on one set of objectives while still continuing to collect data on a previous set of objectives.

2. If the department/program intends on using data from the summer term, please notify the academic administrator and OA committee chair.

3. If the department/program plans on utilizing OA rubric data for report

preparation, please contact the Office of Institutional Research at x2225 for assistance in obtaining the necessary data.



APPENDIX



Appendix A. Glossary of Terms

Direct Evidence Students show their mastery of learning goals/objectives through the demonstration of knowledge, skills.

Examples: Licensure/certificate exam scores and pass rates, capstone experiences, pre/post test scores, Internship/clinical skill ratings.

Indirect Evidence Generally perception or comparison data that allows one to make inferences about learning, but does not demonstrate actual learning.

Examples: Suverys, course grades, placement data, graduate/professional school accepatance rates.

Assessment:

The systematic collection, examination, and interpretation of qualitative and quantitative data about student learning and the use of that information to document and to improve student learning.

Performance-‐Based Assessment

To accurately evaluate what a person has learned, an assessment method must examine his or her collective abilities. This is what is meant by authentic(performance-‐based) assessment. Authentic assessment presents students with real-‐world challenges that require them to apply their relevant skills and knowledge. Student performance on a task is typically scored on a rubric to determine how successfully the student has met specific standards.

Benchmark: A description or example of student or institutional performance that serves as a standard of comparison for evaluation and judging quality.

Class assessment:

A process of assessment techniques customized to the specific needs of the students and curriculum, and guided by the professional judgment and knowledge of the teacher.

Closing the Loop/Feedback Loop: The process by which assessment results are used in programmatic and campus-‐wide decisions to impact student learning.

Competency: Statements which describe skills or knowledge students are expected to possess as a result of completing the course successfully.

Competency-‐Based Assessment: An assessment of a student’s performance as compared to a specific learning objective or performance standard.

Developmental Outcome:

How are students developing as a result of an intervention delivered through your class, course, program or service? These outcomes illustrate affective dimensions or attitudes (i.e. sensitivity toward others, efficacy, civic responsibility).

Direct Measurement:

A measurement of student learning outcomes showing what they have learned. Examples of such measures include but are not limited to: licensure test results; capstone course portfolios; entry and exit test results.



Evaluate/Evaluation:

Definition 1: Evaluation uses assessment information to make an informed judgment on such things as: whether students have achieved the learning goals we’ve established for them; the relative strengths and weaknesses of our teaching/learning strategies; or what changes in our goals and teaching/learning strategies might be appropriate. Assessment results alone guide us; evaluation informs our decisions. Definition 2: Evaluation is used to investigate and judge the quality or worth of a program, project, or other entity, rather than student learning. Under this definition, evaluation is broader than assessment.

Formative Assessment:

Refers to assessments that provide information to students and educators that are used to improve teaching and learning. These are often informal and ongoing, though they need not be. Data from summative assessments can be used in a formative way. (see also Summative Assessment).

Grading: A process of faculty review and evaluation of student learning that is used as a basis for rating performance.

Indirect Measurement:

A measurement of program outputs using student performance information. Examples of such measures include but are not limited to: number of students successfully transferring; graduation rates; placement data; advisory committee evaluation; and feedback from students, graduates, or employers. Indirect measures include self assessments, such as student responses to surveys.

Institutional Effectiveness: The measure of what an institution actually achieves, done for the purpose of continual improvement.

Learning Outcome:

Statement that specifies what learners will know or be able to do as a result of a learning activity. Outcomes are usually expressed as knowledge, skills or attitudes. In addition, three distinguishing characteristics of learning outcomes are that the specified action of the learner must be observable, measurable, and done by the learner. What are students learning as a result of the class, course, program or service? Have students’ cognitive abilities changed the way they were supposed to change as a result of their participation in a class, course, program or service (i.e. theoretical knowledge)?

Norm-‐Referenced Assessment:

An assessment of a student’s performance or performances as compared to a larger group. Usually the larger group or “norm group” is a national sample representing a wide and diverse cross-‐section of students.

Portfolio:

A representative collection of a student's work, including evidence that the student has evaluated the quality of his or her own work. May be in a variety of formats, including electronic and hard copy.



Primary Trait Analysis (PTA):

The process of identifying major traits or characteristics that are expected in student work. After the primary traits are identified, specific criteria with performance standards, are defined for each trait.

Program assessment:

A combination of assessments techniques, data collection and analysis about student achievement for learning outcomes at the class and course levels, and leading to improvements of the academic program.

Program Outcome:

Has a class, course, program or service accomplished what it was designed to accomplish? Was the task completed or not? These may be primarily quantitative data sets (i.e. number of students served, number of programs, satisfaction ratings).

Rubric or Scoring Guide: A set of scoring guidelines that can be used to evaluate students' work.

Summative Assessment:

Refers to the cumulative assessments, usually occurring at the end of a unit or topic coverage, that intend to capture what a student has learned, or the quality of the learning, and judge performance against some standards. Although summative assessments are often thought of as traditional objective tests, this need not be the case. For example, summative assessments could follow from an accumulation of evidence collected over time, as in a collection of student work. (see also Formative Assessment).

Testing:

Testing is the systematic measurement of a mental trait such as an aptitude, achievement, skill, or attitude. Therefore, testing is included in the assessment process, however, testing is, in the broader sense, more than the traditional multiple-‐choice, essay, and true/false strategy.



Appendix B. College-Wide General Education Rubrics



Appendix C. Livetext Instructions



Appendix D. Annual Update Form



Appendix E. Sample Annual Update



Appendix F. Annual Update Review Checklist

outcomes) assessment) handbook€¦ · communication, oral communication, critical thinking,...

Documents