2011using data analytics

Assessment & Analytics

Using Data to Inform & Improve Student Success Strategy

Introductions

Initial Data ReviewFindings

Fall 2005

85% of online students were completing courses.

77% of students who completed, passed course with grade of C or better.

Is this good enough?

What questions should we ask?

Initial Data ReviewFindings

Challenge identified in 2005:

77% of at-risk students completed courses

57% of at-risk students earned C or better

At-risk included:

New to online (40%)

Returning to school after 5 years (35%)

Low High School GPA (15%)

Desire to increase Student Success

By 2007:

82% of at-risk students completed

5% Improvement

66% of at-risk students earned C or better

9% Improvement

Program ImprovementIowa Results

Identify at-risk students early.

“The effectiveness of early (identification) is that it’s early” - Vincent Tinto, NISOD 2011

Intervene and support students in need.

Increase at-risk student success rate.

Goal in 2005Improve At-Risk Student Success

What is At-Risk?Before Course Start

First-time online student

Returning to school after 5+ years

Registered for three or more online classes simultaneously

Failed an online course the previous 2 years

What is At-Risk?After Course Start

Student enrolls after course start

Grade-to-date is below a ‘C’

Stops attending for 5 consecutive days

Logs-in fewer than 3 times per week

Activity less than 3 hours per week.

Week 1 Days Logged In:Completed vs. Dropped Student

3.12.9

3.4

2.7 2.8

3.33.6

3.2

1.71.5

1.91.7

1.41.6

1.91.8

0

1

2

3

4

5

Business CompScience

English HealthSciences

Math Psych Science Soc

CompletedAvg. = 3.1

DroppedAvg. = 1.7

Avg. Days Logged In/Section by Department

Completed Student at Add/Drop Dropped Student at Add/Drop

Week 1 Activity:Completed vs. Dropped Student

2.93.3

3.8

2.52.8

3.3

4.0

3.0

1.4 1.3

1.9

1.3 1.2 1.2 1.21.5

0

1

2

3

4

5

Business CompScience

English HealthSciences

Math Psych Science Soc

CompletedAvg. = 3.1

DroppedAvg. = 1.3

Avg. Student Activity/Section by Department (hrs)

Completed Student at Add/Drop Dropped Student at Add/Drop

Action Taken

Hire Student Services Manager

Leverage LMS Data Warehouse

Identify / Track At-Risk Students

Provide Manual Intervention

Intervention Strategy

Personalized, Automated Welcome Email

Semi-Automated Intervention Emails

Weekly Activity/Progress Reports

Instructor Reporting

Sharing of Better Practices

FirstName

Course Last

Login Date

Activity Minutes

Activity Submission

Count

Course Points

Earned

Course Points

Possible to Date

Course Average

Grade To Date

Rocky 4/28/11 722 11 133 470 28.30%Melissa 5/8/11 3,053 35 479 530 90.38%Megan 5/5/11 2,981 30 557 556 100.18%Anneliese 5/7/11 5,633 66 509 585 87.01%Bradley 5/8/11 2,613 57 508 820 70.37%Alyssa 5/8/11 3,290 116 625 680 91.91%Lynsey 5/8/11 2,536 72 860 1,000 88.02%Jeremy 5/8/11 1,209 37 62 190 32.63%Angelica 5/7/11 731 24 789 900 87.67%Carrie 4/5/11 103 2 31 145 21.38%

Progress ReportProvided to College Advisors Weekly

Instructor ReportConveniently Located Within Course

Instructor ReportProvided to Advisors

Student Name

Instructor Name

Reason for At Risk

Other Explanations At Risk Date

Heather Lori Other Did not complete Unit 1. I did get an e-mail doesn't understand how class works, needs some help.

1/23/2011

Ann Steve Not completing assignments

missed first quiz and first lab assignment

1/26/2011

Courtney Rose Other Did not participate in Week 1. 1/26/2011

Donna Mark Stopped Participating

She participated through week 6, has since dropped off the map and I have not received any work from her. She’s not responded to e-mail.

3/16/2011

Results

Course success rate for at-risk students increased 9% over 2-year period.

IntermissionYou Can Do This

Questions?

In 2010:

84% of at-risk students completed their courses

Additional 2% improvement in 3 years

68% of successful at-risk students completed courses with a C or higher

Additional 2% improvement in 3 years

Now What?Results Today

60%

80%

100%

0 500 1,000 1,500 2,000 2,500 3,000 3,500 4,000

Avg. SectionCompletionRate = 77%

Department Term Start Enrollments

I

IIIII

IV

Median DeptEnrollments = 767

Dep

artm

ent

Co

mp

leti

on

Rat

es

– T

erm

Sta

rt t

o T

erm

En

d

English

Mathematics

Now What?Completion Rates vs. Enrollments

What Now?

Further improve Success in English Composition and Math.

Enhance At-Risk Case Management

Provide simple, visual Dashboard for instructors

Publish Methodology

Faculty Dashboard

Faculty Dashboard

Student Dashboard

Enrollments (at course end) 12Withdrawals 7Course Average Grade 55%

Noticeable Trends:Top scoring students show strong edges to professor in both directions.

Secondary networks forming between multiple high performing students

Questions to Research (Predictive)1. Do networks between students relate to successful completion?

2. How does professor interaction with at risk or low performing students impact student success?

MAT100 – Section A

Thread Interaction

Enrollments (at course end) 17Withdrawals 4Course Average Grade

83%

Noticeable Trends:Multiple directional edges to and from the professor node.

Multiple secondary networks forming.

Lamsen is a curiosity – no target interaction to that node, yet multiple source edges

Joshua not part of the forming subnetworks

Question to Research (Predictive)How important is multi node, multi directional interaction to course success?

MAT100 – Section B

Thread Interaction

ConclusionYou can do this too.

Questions?

Contacts

Steve Rheinschmidt, Director ICCOC

stever@scciowa.edu

Steve Ast, Senior Account Exec. eCollege

steveast@ecollege.com