EFFECT ON FACULTY RATINGS OF PAPER BASED VERSUS

WEB BASED STUDENT COURSE ASSESSMENTS

Dr. Juliana LancasterDirector of Institutional Effectiveness

Dr. Michael FurickAssistant Professor of Business

Introducing GGC The 35th member of the University

System of Georgia Opened its doors to an inaugural junior

class of 120 students in fall 2006 The first freshmen class of 337 was

admitted fall 2007 with total enrollment of 787.

First graduation was held on June 28, 2008

Enrollment has grown to over 3000 students

Accredited by SACS in June 2009

Introducing GGC GGC’s mission supports access to

baccalaureate degrees that meet the economic development needs of the growing and diverse population of the northeast Atlanta metropolitan region.

GGC offers majors in: Biology Education Mathematics Exercise Science English History Information Technology Political SciencePsychology Business Administration Criminal Justice/Criminology

Importance of Student Course Evaluations

Formative Assessment: Used by faculty to improve and shape the quality of our own teaching

Summative Assessment: Used to determine overall performance, may be used in personnel decisions

Programmatic Assessment: Used to evaluate the role of a course within a degree program

Summarize Lit Review (2 slides)

General Interest Questions Do you do paper or web? Who gets to see results? Who should see results?

Paper vs Web Systems

Paper WebExecution in class: Passive system for student

Delivery via email or portal link: Requires active participation by student

Requires short time window to complete data collection

Requires longer time window to complete data collection

Familiar to faculty and students Unfamiliar to faculty and studentsRequires extensive time and cost to prepare and process

Reduces time and cost to prepare and process

Results returned to faculty within weeks

Results returned to faculty within days

Is Web really better…..or worse?

Will most students actually do a web based evaluation?

Will the students who do a web based evaluation be different from the overall student population?

Will the evaluation ratings be affected by changes in which students complete evaluations?

Shifting Delivery at GGCSemester Modality Distribution of Form/

Access InformationFall 2007 Paper In-classSpring 2008 On-line Passwords distributed by

faculty in classSummer 2008- On-line Passwords

distributed by Summer 2009 system to student emailFall 2009 On-line Active links to course

surveys in student portal

Study Design Compared Fall 2007 (paper), Spring 2009

(web with email delivery) and Fall 2009 (web with portal delivery)

Sampled all evaluations of faculty who have been at GGC continuously since Fall 2007 (N=78)

Used only items that have remained (near) constant on evaluation instrument (N=11)

Slide or two with distributions

Questions

Did overall response rates change?

Did profile of respondents change?

Did ratings change in a consistent way?

Clicker responses to three questions

Did overall response rates change?

Response rates for web based semesters are lower than for paper based semester.

Response Rate

% Class sections with RR < 15%

Fall 2007 77.4% 0.0%Spring 2009 38.7% 9.2%Fall 2009 48.8% 1.9%

Did profile of respondents change?

Expected Grade by Semester

A B C D F0

5

10

15

20

25

30

35

40

45

50

F07S09F09

Did profile of respondents change?

Looked at expected grade as reported by respondent.

Overall pattern is highly similar, suggesting web did not attract more unhappy students.

Did ratings change in a consistent way?

SC SO TO PI IP IE DI IC FC CC3.1

3.2

3.3

3.4

3.5

3.6

3.7

3.8

F07S09F09

Did ratings change in a consistent way?

Overall, ratings have shown a steady upward trend for this set of faculty.

For 10 of 11 items analyzed, Fall 09 mean ratings were significantly higher than Fall 07 ratings.

For one item, ratings dropped for the Spring 09 term but rose again in Fall 09

General Conclusions Shifting from paper to online evaluations

does cause a reduction in response rates The reduction is spread across expected

grades and across the spectrum of ratings

Overall, our mean evaluations have risen over time.

Next step: Replicate study with some paper and some online surveys in Spring 2011