Simplifying UK’s Course Evaluation Process

Carol YuOffice of Institutional Research

Fall 2011

AgendaTeacher and Course Evaluation Process

Challenges for TCE Contacts and IRCourse List Revision ProcessSummary of Online TCE Committee Recommendations

Information and Resources on IR website

TCE Process in a nutshell…Identify courses that need to be

evaluated

Distribute paper evaluations

Send online invitations to students for evaluation

Collect and analyze data

Distribute and post TCE reports

Challenges for the TCE contactsToo much work to do

Unable to access SLCM or HR systems

Unable to enter updates into the system during the semester

Lack of communication between faculty and staff for instructor updates

Challenges you haven’t yet shared with us

Challenges for IR staffUp-to-date information on faculty or courses

is not readily available Mismatches between what is on the TCE

course lists and what is in SAPUnmet deadlines for revising informationWorking with mixed online and paper

evaluations for the same college/department Inconsistent updates from different units on

cross-listed or distance-learning coursesIR does not own specially designed software

to administer online evaluations

Revisions to Course Lists

Course Lists were created… after the last day for students to

Add/Drop coursesbased on HR data, Event Planning Data

and Student Enrollment Data

Basic Rules for Revising ListsCheck every single line in the Course

List file; don’t assume information on the list is accurate.

Don’t delete any columns or rows in the file.

If you need to insert a new row for either a second instructor or new section, please don’t copy and paste previous rows in the file

Revising and submitting material on time is critical to the success of the evaluation process

Color Scheme on the Updated Course ListInitial updates from the college/department

are highlighted in RED

Color Scheme on the Updated Course List

Initial edits from IR are highlighted in BLUE

Color Scheme on the Updated Course List

Final updates from the college/department are highlighted in GREEN

Color Scheme on the Updated Course List

Final edits from IR are highlighted in ORANGE

Updates to Instructor Information

Instructor Information Make sure the correct instructor is associated

with the course

The correct instructor is the person teaching the course, not the course coordinator

All instructors including TAs who teach the course need to be entered into SAP to match with the Course List

Leave no columns missing or blank

Indicate whether the instructor is full-time, part-time, or TA

Updates to Enrollment InformationThere are two enrollment columns

Section enrollments are used to generate online TCE surveys

Combined enrollments are used to plan the number of paper TCE forms by adding together sections taught in the same classroom

Updates to TCE ListsThere are several columns indicating the

evaluation mode (online, paper, distance learning) and timing (standard or early) of the TCE

Only one column will have “Yes” indicated for a given class. The others should be either “NO” or left blank

Updates to TCE ListsIf the same instructor teaches a Lecture and a

Lab, the Lab should not be evaluatedIf the same course section is listed twice in the

Course List, only one course section should be evaluated

Updates to TCE Lists for Instructors Teaching Multiple Sections in the Same Classroom

Enter YES in the “PaperTce” column for the first section and leave blanks in the column for all other sections meeting at this place and time

For paper course evaluations…

Updates to TCE Lists for Instructors Teaching Multiple Sections in the Same Classroom

Enter YES in the “Online TCE column” for each section

For online course evaluations…

Look at the column titled “Cross-list” and put a “YES” in column if the course is sponsored by the department associated with the prefix and course number

Put a “NO” in the column if the department associated with the prefix and course number does not sponsor the course

Online forms will be sent to students in both sections, but the results will be combined and sent only to the department sponsoring the course

Paper forms based on the number of combined enrollments will be sent to the department sponsoring the course

Updates to Cross-listed Courses

Updates for Instructors Wishing Combined Output for Different Courses

Put “Yes” in the “Combine Output” column on all courses the instructor wishes to combined. (There aren’t many cases now).

Add a brief description in the Note column of courses the instructor wants to combine

Use of Additional Notes The Note column explains any special

cases in a few words. (Please don’t put a “Yes” or “No” in this column.)

Revisions to Course Lists A Word about the Spreadsheet Tabs…

The tabs are records of what has been submitted at each step in the revision processThe original course list will be included in

updated versions of the list (e.g., fa10_en_org below)

Your initial edits and my initial edits will be captured in the second tab of the final file (e.g., fa10_en_edit below)

The final version of the course list appears along with the original and edited versions (e.g., fal10_en_final below)

Common Mistakes Using Hand-Slugged Forms

• Evaluation forms pre-slugged by hand have Course IDs that do not match the course list

• Course ID is not bubbled or is bubbled in- correctly.

• Instructor’s name is crossed out on the form for new instructor

Common Mistakes during the Submission of TCE Forms Evaluation forms for different instructors

are placed in separate envelopes. (Forms may be placed together in one box; there’s no need to sort them.)

TCE forms from different semesters are mixed together in one box. (Forms from previous semesters should never be included.)

Online TCE Committee: Recommendations

Recommended Organizational and Software Specifications:

Give college/departmental contacts the ability to record course evaluation preferences directly into SAP

Take information on departments, instructors, TAs and courses directly from the Event Planning module, eliminating the need for merging and updating multiple files.

Distinguish between the sponsors of ‘cross-listed’ courses and ‘also offered as’ courses so evaluations are collected and results are aggregated properly.

Online TCE Committee: Recommendations

Recommended Organizational and Software Specifications (cont’d):

Design the ‘basic‘ evaluation form using information provided about the course (e.g., lectures, seminars, labs)

Allow for additional customization of questions based on input from departments and colleges

Establish a bank of existing questions that could be used for additional items beyond those specified in the uniform evaluation form

Online TCE Committee: Next Steps

Conduct second iteration of Blackboard Pilot Study in Fall 2011

Investigate a university-wide CourseEval™ contract as a cost-effective, backup alternative if Blackboard is not found to be a suitable option

Implement the recommendation calling for deans to choose whether their colleges will adopt all online or all paper-based course evaluations beginning in Spring 2012

Available Information and Resources on the Institutional Research Website

www.uky.edu/IRPE/faculty/tce.html Overview of the course evaluation

process

Current schedule of events for TCE process

IR’s policy on course evaluations

Sample evaluation form

TCE results for previous semesters

Additional Questions

Carol YuSurvey Coordinator/Data AnalystOffice of Institutional Research 1080 Export Street, Room 214Lexington, KY 40504Phone: (859) 257-4110Fax: (859) 323-3999E-mail: cbyu2@email.uky.edu