academic success center university tutoring...spring 2015 1451 2013-2014 academic year 3230 -5.6%...

Academic Success Center

University Tutoring

2014-2015 Impact Report

Michael A. Donzella, M.S.

Academic Program Coordinator

P a g e | 2 of 23

Executive Summary

Each section in this report provides insights regarding University Tutoring (UT) operations and demonstrates

six key facts about UT services.

I. Usage Data (pp. 3-4)

Tutoring usage marginally decreased for the academic year by 4%. While tutoring visits increased in

Fall 14, visits declined during Spring 15.

II. Spring 2015 Decrease

Tutoring visits decreased by 17% during Spring 15 due to reduced usage in scheduled tutoring and

Wright Hall Residence Hall tutoring. Diminished enrollment in math emporium courses also played a

crucial role.

III. Efficiency Data (pp. 6-11)

In light of the marginal decrease in usage, tutoring funds and hours were allocated appropriately.

IV. Grade Impact of Scheduled Tutoring (pp. 12-20)

Using scheduled tutoring regularly has a positive impact on grades and has an effect on the DFW rate.

V. Qualitative Assessments and Improvements (pp. 21-22)

UT has made improvements based on feedback and continues to do so.

VI. Survey Results (p. 23)

When surveyed, students who use UT services reported their experience as beneficial.

The beginning of each section illustrates the overall data of each reporting category for easy reference.

The aggregate results are then followed by more detailed analysis and supporting data.

P a g e | 3 of 23

Section I: Usage Data

Visits

During this academic year, UT experienced a marginal decrease in overall visits.

For the Fall 14 semester, UT experienced its largest gain for fall semester compared to the previous

year due a large influx of students enrolled in math courses requiring mandatory tutoring resulting in a 7.7%

increase. In contrast, UT visits decreased 17% during the Spring 15 semester for a variety of reasons (see

Section II) leading to an overall 4% decrease for the year. Given the decrease, the two year comparison

maintains the magnitude of the expansion in which the decrease constitutes a marginal flux.

F12 F13 F14 S13 S14 S15 F12-S13 F13-S14 F14-S15

6025 12105 13036 9199 11238 9353 15224 23343 22389

1 Year Difference 931 1 Year Difference -1885 1 Year Difference -954

2 Year Difference 7011 2 Year Difference 154 2 Year Difference 7165

1 year % Change 7.7% 1 year % Change -17% 1 year % Change -4%

2 year % Change 116.4% 2 year % Change 2% 2 year % Change 47%

P a g e | 4 of 23

Unique Students

This academic year resulted a small overall decrease in unique students. Even though Fall 14 unique

students decreased, visits increased implying more consistent participation. See section II for greater detail

regarding the Spring 15 decrease.

Unique Students % Change

Fall 2013 2102 -1.5%

Fall 2014 2070

Spring 2014 1799 -19.3%

Spring 2015 1451

2013-2014 Academic Year 3230 -5.6%

2014-2015 Academic Year 3046

eTutoring

Although traditional tutoring participation fluctuated this year, eTutoring participation surged in terms of

its largest increase and utilization of all-time, particularly in Spring 15. A significant factor in the recent increase

involves inclusion of a direct link to eTutoring services on the ASC landing page.

eTutoring

Fall 13 Spring 14 Fall 14 Spring 15

eQuestions 10 8 11 14

eChat 29 41 39 40

Writing Queue 51 59 96 217

Total: 90 108 146 271 Difference % Increase

Annual Total: 198 417 219 111%

P a g e | 5 of 23

Subject Distribution

During Fall 14, mathematics and mandatory tutoring sections accounted for a majority of tutoring

activity.

Even though, enrollment in mandatory tutor sections were reduced in Spring 15, math continued to

maintain the majority of all tutoring provided.

MATH42%

CHEM11%

PHY5%

BSCI2%

OTHER8%

Nursing10%

Mandatory Tutoring

22%

Fall 2014 Subject Distribution

MATH57%CHEM

16%

PHY5%

BSCI3%

OTHER12%

Nursing7%

Spring 2015 Subject Distribution

P a g e | 6 of 23

Section II: Spring 15 Decrease

Scheduled Tutoring

In researching appointment trends, we found that there exists a three week time period at the beginning

of the semester in which a majority sign-ups typically occur. Based on the data, snow days and inclement

weather during the first three weeks of the semester put scheduled tutoring at a particular disadvantage. As

historical data demonstrates, active appointments in scheduled tutoring do not experience the same magnitude

of increase as in the first three weeks of the semester.

As scheduled tutoring appointments occur weekly, a reduced number of signups at the beginning of the

semester leads to a cumulative loss for the semester.

Scheduled Tutoring Visits

Spring 14 Spring 15 Difference % Decrease

4936 3757 -1179 24%

0

100

200

300

400

500

600

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8

Act

ive

Sch

edu

led

Tu

tori

ng

Ap

pp

oin

tmen

ts

Spring 15 Active Scheduled Tutoring Appointments

Spring 13 Spring 14 Spring 15

P a g e | 7 of 23

Wright Hall Tutoring

Wright Hall serves as University Tutoring’s evening science outreach program, focusing primarily on

Chemistry, Biology, and pre-nursing courses. Even after increased marketing efforts to current UT students,

classroom announcements, and departmental outreach for this sub-center, visits declined dramatically

compared to the previous year. Further investigation revealed the meeting times for the aforementioned

science courses occured in the morning leading to students to take advantage of tutoring during daytime

hours. Responding appropriately, staffing was reduced and half of Wright Hall science tutors were scheduled

during the daytime hours.

. Wright Hall Visits Spring 14 Spring 15 Difference % Decrease

602 167 -435 260%

Math Emporium Tutoring

During the Spring 15 semester, Basic Algebra I-IV courses saw a significant reduction in enrollment.

Due to new placement polices, students registered for courses such as:

MATH10041 Intro to Statistics

MATH 20095 Algebra for Calculus Plus

MATH 20095 Modeling Algebra Plus

MATH 20095 Algebra for Calculus Stretch I/II

Math Emporium Enrollment

Course Spring 14 Spring 15

BASIC ALGEBRA I 170 121

BASIC ALGEBRA II 464 356

BASIC ALGEBRA III 1,032 223

BASIC ALGEBRA IV 614 126

Total: 2,280 826

As a result, emporium visits and tutor hours declined with enrollment.

Emporium Visits

Spring 14 Spring 15 Difference % Decrease

1174 390 -784 201%

Conclusion

Each location listed above suffered a net loss of 2398 visits; however, the overall loss for Spring 15

totaled 1885 visits. This implies 513 visits of new growth have emerged from other locations (CS Lab,

StatsLab, etc).

P a g e | 8 of 23

Section III: Efficiency Data

Introduction

Efficiency in learning centers provides a comparative description of tutorial services both offered and

used. In this context, adjustments have been made in terms of increasing or decreasing staffing as well as

marketing specific events and locations. To this end, the implementation of a quantitative device to

demonstrate the need for an increase in funding or staffing, to enhance systemic performance for learning

centers with multiple outreach locations, and to assess semesterly or yearly changes in student utilization

serves as an ideal reporting tool.

Quantifying overall efficiency stems from the notion that UT would like one tutoring hour supplied to

coincide with one contact hour of tutoring. Hence, the ratio of tutor hours to contact hours obtains the Overall

Efficiency Ratio (OER). If this ratio is less than or equal to one, then tutor hours are being utilized effectively. If

the ratio is greater than one, then more tutor hours are being provided than are being used by students.

In an effort to prevent inaccurate data, an additional figure is implemented as a system of checks and

balances to ensure the reliability of the OER. The Visit Ratio (VR), the ratio of visits per tutor hour, determines

whether or not the OER can be utilized. Essentially, to show an increase in the OER, both the OER and the VR

must decrease when compared to the prior year. For example, if students do not properly log out of our

tracking system, the OER would appear significantly lower than the VR rendering the OER unusable.

The OER is a more accurate indicator than the VR since students may use more than one hour of

tutoring in a drop-in setting. For example, if a student uses two hours of tutoring, then those two hours coincide

with two tutoring hours supplied. On the other hand, with the VR, only the student’s single visit coincides with

the two tutoring hours supplied.

An ideal OER lies between 0.70 and 1.00 indicating that a single tutor hour correlates to one or slightly

more than one contact hour used. An OER less than .5 correlates to at least two students utilizing each tutor

hour. Similarly, an OER of 0.33 or less implies that for every tutor hour provided, three students utilized it.

Smaller OERs are undesirable as more students receive less individualized attention and fewer tutors exist on

staff to handle the demand.

Overall Efficiency Ratio = Tutor Hours / Contact Hours

OER ≤ 1 Efficient use of tutor hours

OER > 1 Inefficient use of tutor hours

P a g e | 9 of 23

Overall Efficiency

The Fall 14 results demonstrate for each tutor hour provided, at least two students utilized it. Combined

with qualitative observation, University Tutoring received a high volume of demand.

Overall

Fall 2013

Overall Efficiency Ratio: 0.65 Visit Ratio: 0.88

Fall 2014


The Spring 15 results show that even with an overall 17% decrease in visits, tutor hours were still

effectively allocated with an OER less than 1.

Overall

Spring 2014


Spring 2015


P a g e | 10 of 23

Fall 2013/2014 Efficiency Comparisons

The Fall 14 semester can be categorized by an overwhelming surge in utilization for drop-in tutoring.

The data below shows that Library Drop-in Tutoring attained a significant increase in visits as well as contact

hours while using less tutor hours. The OER of .34 illustrates that each hour of tutoring supplied was utilized by

at least three students. In tandem with qualitative observation, most of the influx can be attributed to students

enrolled in math courses with a mandatory tutoring requirement. In addition to increased visits, numerous

students utilized this location for more than hour with.

Library Drop-In

Fall 2013 Total

# of Contact Hours: 6573.30 Overall Efficiency Ratio: 0.56

# of Visits: 4263 Visit Ratio: 0.87

# of Tutor Hours: 3696.07

Fall 2014 Total




Scheduled tutoring appointments accumulated quickly with the onset of the semester. Appointments

remained consistent, leading to a negligible decrease in visits and increase in contact hours. Combined with a

decrease in tutor hours, the OER increased by 11% compared to Fall 13.

Scheduled Tutoring

Fall 2013 Total




Fall 2014 Total




P a g e | 11 of 23

Fall 2013/2014 Efficiency Comparisons (Continued)

Fall 2014 Residence hall tutoring saw proportionately similar utilization and staffing as Fall 13 with

slight variation in each attribute.

Residence Halls

Fall 2013 Total




Fall 2014 Total




P a g e | 12 of 23

Spring 2014/2015 Efficiency Comparisons

Library drop-in tutoring results illustrate a modest increase in visits and contact hours as well as a slight

decrease in tutor hours due to snow delays. Hence, the location performed as efficiently as the previous year.

Library Drop-In

Spring 2014 Total


# of Visits: 3529 Tutor Hours/Visits: 1.05


Spring 2015 Total




Although scheduled tutoring admitted a 24% decrease in visits, unutilized tutor hours were appropriated

by library drop-in tutoring. An OER of less than 1.00 implies that each tutor hour supplied in scheduled tutoring

was matched with at least one contact hour of scheduled tutoring.

Scheduled Tutoring

Spring 2014 Total




Spring 2015 Total




P a g e | 13 of 23

Spring 2014/2015 Efficiency Comparisons (Continued)

Due to the reduction in popularity of Wright Hall, overall residence hall tutoring resulted in a significant

decrease in the OER relative to Spring 14. However, the OER illustrates that approximately every tutor hour

provided was utilized by at least one student.

Residence Halls

Spring 2014 Total




Spring 2015 Total


# of Visits: 1137.84 Tutor Hours/Visits: 0.63


P a g e | 14 of 23

Section IV: Grade Impact of Scheduled Tutoring

Methodology

To demonstrate the effectiveness of scheduled tutoring, students who utilized this service regularly

throughout of the semester were sampled. Students fitting the criteria registered for tutoring within the first two

weeks of the Fall 14 or Spring 15 semester and attended consistently until the end of the semester. Each

student had at least eight scheduled visits in the highest volume subject areas. The final grades from these

students in Math, Pre-Nursing, Chemistry, Biology, and Physics were compared to their peers in the same

courses who attended infrequently or not at all. The courses included in this survey are as follows:

Math

o MATH 10041 Introductory Statistics

o MATH 11008 Explorations in Modern

Math

o MATH 11009 Modeling Algebra

o MATH 11010 Algebra for Calculus

o MATH 11012 Intuitive Calculus

o MATH 11022 Trigonometry

o MATH 12001 Algebra & Trigonometry

o MATH 12002 Analytic Geometry &

Calculus I


Calculus II

o MATH 12021 Calculus for Life

Sciences

o MATH 14001 Basic Math Concepts I

o MATH 14002 Basic Math Concepts II

o MATH 20095 Modeling Algebra Plus

o MATH 20095 Algebra for Calculus Plus

o MATH 20095 Algebra for Calculus Stretch

o MATH 20095 Basic Math Concepts Plus

o MATH 21001 Linear Algebra


Calculus III

o MATH 30011 Basic Probability &

Statistics

Pre-Nursing

o BSCI 20020 Biological Structure and ______

___. Function

o BSCI 20021 Basic Microbiology

o CHEM 10050 Fundamentals of ___________\

Chemistry

o CHEM 10052 Intro to Organic Chemistry

Chemistry

o CHEM 10060 General Chemistry I

o CHEM 10061 General Chemistry II

o CHEM 10960 General Chemistry (Honors)

o CHEM 20481 Basic Organic Chemistry I

o CHEM 30481 Organic Chemistry I

Biology

o BSCI 10001 Human Biology

o BSCI 10110 Biological Diversity

o BSCI 10120 Biological Foundations

o BSCI 30156 Elements of Genetics

Physics

o PHY 13001 General College Physics I

o PHY 13002 General College Physics II

o PHY 13012 College Physics II

o PHY 23101 General University Physics I

In particular, the DFW rates, as well as pass rates, were examined to illustrate scheduled tutoring’s

effect on student grades. It should be noted that the overall Spring sample size is smaller as tutoring

participation is traditionally less than in Fall semester.

P a g e | 15 of 23

Overall Grade Impact

Combining all the data together, the results below detail the aggregate effect of utilizing scheduled

tutoring regularly on the DFW rate for the Fall 14 and Spring 15 semesters, respectively.

Overall Conclusions

Within each major facet of scheduled tutoring, students who attend regularly have a significantly higher

AB rate compared to irregular or non-attendees. Moreover, in every instance, the DFW rate is significantly

lower for students who do not attend regularly. By demonstrating the impact of tutoring on grades and DFW

rates, students are provided with an incentive to register for scheduled tutoring and maintain their participation.

32%24%

39%

25%

17%

21%

9%

9%

10%

11%

With UT, ABC Pass Rate 87% WithUT, DFW Rate 13%

(N=207)

Without UT, ABC Pass Rate 70%,Without UT, DFW Rate 30%

(N=8339)

Fall 2013 Overall Grade Comparisons With and Without University Tutoring

W

F

D

C

B

A

32%21%

30%

22%

24%

26%

10%

11%

10%

10%

With UT, ABC Pass Rate 86% WithUT, DFW Rate 14% (N=

184)

Without UT, ABC Pass Rate 69%Without UT, DFW Rate 31%

(N=6608)

Spring 2014 Overall Grade Comparisons With and Without University Tutoring

W

F

D

C

B

A

P a g e | 16 of 23

Individual Subject Breakdown

Mathematics

Mathematics serves as UT’s most utilized subject area, and thereby constituted the largest sample

size. This is especially true since in many Math courses, students must receive a grade of at least a “C” to

progress on to the next course. The courses compared in this study are as follows:

MATH 10041 Introductory Statistics

MATH 11008 Explorations in Modern Math

MATH 11009 Modeling Algebra

MATH 11010 Algebra for Calculus

MATH 11012 Intuitive Calculus

MATH 11022 Trigonometry

MATH 12001 Algebra & Trigonometry

MATH 12002 Analytic Geometry & Calculus I

MATH 12003 Analytic Geometry & Calculus II

MATH 12011 Calculus with Pre-Calculus I

MATH 12021 Calculus for Life Science

MATH 14001 Basic Math Concepts I

MATH 14002 Basic Math Concepts II

MATH 20095 Modeling Algebra Plus

MATH 20095 Algebra for Calculus Plus

MATH 20095 Algebra for Calculus Stretch

MATH 20095 Basic Math Concepts Plus

MATH 21001 Linear Algebra

MATH 22005 Analytic Geometry & Calculus III

MATH 30011 Basic Probability & Statistics

P a g e | 17 of 23

Mathematics (Continued)

The results revealed that students who utilized scheduled tutoring regularly achieved a higher GPA in

comparison to their peers. Moreover, students also exhibited a significantly lower DFW rate when utilizing

scheduled tutoring.

24% 22%

37%24%

20%

22%

12%

10%

5%

10%

2%12%

With UT, ABC Pass Rate 81%With UT, DFW Rate 19%

(N=93)


(N=3827)

Fall 2013 Math Grade Comparisons With and Without Univesity Tutoring

W

F

D

C

B

A

19% 18%

25% 24%

33%25%

16%

11%

6%

12%

11%


(N=67)


(N=3024)

Spring 2014 Math Grade Comparisons With and Without Univesity Tutoring

W

F

D

C

B

A

P a g e | 18 of 23

Pre-Nursing

The next category includes courses required prerequisite course for Pre-Nursing majors. To enter the

Nursing program students must perform well in four particular science classes, BSCI 20020 and 20021,

Biological Structure and Function, and Basic Microbiology as well as CHEM 10050 and AS 10095,

Fundamentals of Chemistry, and Molecules of Life. In order to be considered for the Nursing major, Pre-

Nursing students need a cumulative GPA of 2.75 in these courses. As a result, the AB pass rates and DFW

rates were reported. Note that students who utilized scheduled tutoring had an AB pass rate that was greater

than those who did not.

38%

23%

45%

21%

12%

19%

9%

17%

4%11%

With UT, AB Pass Rate 82% With UT,CDFW Rate 18% (N=85)

Without UT, AB Pass Rate 44% WithoutUT, CDFW Rate 56% (N=1154)

Fall 2014 Pre-Nursing Grade Comparisons With and Without University Tutoring

W

F

D

C

B

A

43%

24%

30%

27%

19%

20%

9%

9%

9%

12%

With UT, AB Pass Rate 73%,With UT, DFW Rate 11%

(N=72)

Without UT, AB Pass Rate 50%,Without UT, DFW Rate 30%

(N=1067)

Spring 2014 Pre-Nursing Grade Comparisons With and Without University Tutoring

W

F

D

C

B

A

P a g e | 19 of 23

Chemistry

With Chemistry tutoring, students who utilized scheduled tutoring throughout the semester performed

better than their peers. In particular, the ABC pass rate for scheduled tutoring students far surpasses that of

the general population who did not utilize tutoring throughout the whole of the semester. The chemistry

courses compared are as follows:

CHEM 10060 General Chemistry I

CHEM 10061 General Chemistry II

CHEM 10960 Honors General Chemistry

CHEM 20481 Basic Organic Chemistry I

CHEM 30481 Organic Chemistry I

27% 24%

45%

22%

16%

21%

11%

12%

12%

10%


(N=44)


(N=1182)

Fall 2013 Chemistry Grade Comparisons With and Without University Tutoring

W

F

D

C

B

A

29%21%

48%

24%

19%

23%

13%

9%

10%


(N=21)


(N=789)

Spring 2014 Chemistry Grade Comparisons With and Without University Tutoring

W

F

D

C

B

A

P a g e | 20 of 23

Biology

Although Biology had a relatively smaller sample size, the results demonstrate scheduled tutoring’s

impact on student grades. Once again, the ABC pass rate of students who regularly attend scheduled tutoring

exceeds that of those who do not. The biology courses compared are as follows:

BSCI 10001 Human Biology

BSCI 10110 Biological Diversity

BSCI 10120 Biological Foundations

BSCI 30156 Elements of Genetics

25% 23%

42%27%

25%

21%

8%

10%10%8%


(N=12)


(N=1614)

Fall 2013 Biology Grade Comparisons With and Without University Tutoring

W

F

D

C

B

A

14% 18%

43% 29%

36%

23%

7%

12%

11%7%


(N=14)


(N=1287)

Spring 2014 Biology Grade Comparisons With and Without University Tutoring

W

F

D

C

B

A

P a g e | 21 of 23

Summary of Results

For each semester, the DFW rates and ABC pass rates are summarized for scheduled tutoring below.

F14 Scheduled Tutoring ABC Rates

Subject ABC Rate with UT N with UT ABC Rate without UT N without UT Difference

Math 83% 160 68% 5662 15%

Pre-Nursing 94% 85 63% 1157 31%

Chemistry 85% 89 60% 1766 25%

Biology 98% 52 75% 2368 23%

S15 Scheduled Tutoring ABC Rates

Subject ABC Rate with UT N with UT ABC Rate without UT N without UT Difference

Math 76% 111 65% 4211 11%

Pre-Nursing 73%* 37 48%* 1137 25%

Chemistry 76% 49 63% 1645 13%

Biology 89% 46 76% 1588 13%

*Denotes AB pass rate

P a g e | 22 of 23

Section V: Qualitative Assessments and Improvements

Outcomes of Assessment

During the Fall 14 - Spring 15 academic year, students and tutors were surveyed to provide feedback

on UT programs. Based on their feedback, UT has made the following improvements over the past academic

year.

Marketing

Offered flashperks for students who attend tutoring regularly

Scheduled tutors to make classroom announcements to increase awareness

Targeted students at drop-in tutoring to sign up for scheduled tutoring

Planned Marketing Efforts

Tutoring Informational video produced by multimedia services (July 15)

Increased remote tutoring sign-up locations during the first week of the semester

Marketing to Distance Learning students via eTutoring.

Technology

Automated student and tutors surveys in order to effectively and efficiently analyze customer feedback.

Automated scheduled tutoring attendance.

Implemented Google Docs to streamline tutor specialties and call-off procedures

Premiered searchable Drop-in Tutoring App

Infrastructure

Achieved CRLA Level I certification

Developed Summer training course for statistics tutors

Staffed tutors in classrooms for Math 10041 Intro to Statistics courses

Administered tutor performance evaluations for merit-based raises.

Outsourced tutor training to Computer Science and Athletics Department

P a g e | 23 of 23

Section VI: Survey Results

Methodology

During the Spring 15 semester, 276 students were surveyed at 5 outreach locations over the course of

two weeks. Students were asked a series of 25 questions on a 4 point agree/disagree scale. Moreover,

students were given the opportunity to write in testimonials, feedback, and suggestions on our program. The

survey results are listed below.

Question Agree Disagree

I was aware of tutoring at the Academic Success Center (ASC) at the beginning of the semester.

91.3% 8.3%

I received a tutor in a timely manner. 98.2% 1.4%

The location of the tutoring services is convenient. 95.3% 4.3%

I was able to schedule an appointment with ease. 95.3% 1.8%

The ASC staff creates a friendly learning environment. 98.2% 0.4%

Staff at the ASC helped me to develop/maintain a connection with the university.

94.9% 3.6%

I felt comfortable about asking my tutor questions. 97.8% 0.7%

My tutor explained the subject matter so I could understand it. 96.7% 1.8%

My tutor listened carefully. 97.8% 0.7%

My tutor answered questions well and provided examples to clarify problems.

98.9% 1.1%

My tutor spoke clearly and distinctly. 100.0% 0.0%

My tutor was on time and familiar with the material. 99.6% 0.0%

My tutor has adequate knowledge to assist me with academic problems.

98.9% 0.4%

My tutor encouraged my participation in each session. 98.6% 1.4%

My tutor acted in a professional manner. 99.3% 0.4%

My tutor was patient and did not rush through the session. 98.9% 1.1%

If my tutor were qualified to tutor another course I was taking, I’d request him/her again.

98.6% 1.1%

If I take another course for which tutoring is offered, I would consider signing up for assistance at the ASC.

99.3% 0.7%

I would recommend the tutoring service at the ASC to my friends.

98.9% 0.7%

I would recommend the tutoring service at the ASC to my friends.

98.6% 1.4%

Tutoring has helped to improve my belief that I can

graduate from college.

97.5% 2.2%

Tutoring assistance plays an important role in my decision

to remain at KSU.

94.6% 4.7%

Overall, my tutoring experience was beneficial. 93.1% 6.2%

academic success center university tutoring...spring 2015 1451 2013-2014 academic year 3230 -5.6%...

Documents