improved success in science course following active learning transformation

Improved Success in Science

Course Following Active

Learning Transformation

Presented by

Debra F. McLaughlin, PhDChair, Natural Sciences

Community College OLI

2

1

2

3

• Gateway courses account for more withdrawals and failures than other

courses

• Proficiency scores often lower in gateway courses than in other courses

• Redesign learning environment using powerful and innovative techniques

of Carnegie Mellon University’s Open Learning Initiative (OLI)

• OLI courses are scientifically grounded and focused on ongoing

improvement

• Project will build on existing learning research and contribute new

knowledge about how adults learn

Key Ideas from Grant Proposal

Executive Summary

3

Teaching with OLI

http://oli.cmu.ed

u/

4

The Course Design Triangle

Instructional Activities

Objectives

AssessmentsTasks that provide feedback on students’ knowledge and skills

Descriptions of what students should be able to do at the end of the course

Contexts and activities that foster students’ active engagement in learning

5

Alignment with Biology Literacy and

Big Picture Unifying Theme

6

Combining UMUC- and OLI-type activities

In unit 8, Molecular Genetics,module 41 covers “Translation: Making Proteins.”

Concepts are…

• explained via expository text and other passive content;

• reinforced by a rich, interactive learning object from UMUC’s original course; and then

• assessed through OLI-type learning objects.

7

Learn By Doing

8

Did I Get This?

9

What are the students saying about OLI?

• The strongest feature of the course is identified as OLI several times in each

OLI course section.

• “The use of OLI website with the learning activities made it fun and easy to

understand.”

• “I really enjoyed the OLI materials. I felt that they covered the material really

well, especially with all the activities and quizzes that are interspersed with the

reading.”

• “OLI was the best part of the course.”

• “The materials at OLI were easy to use and multimedia helped to reinforce the

concepts.”

Students First: UMUC’s Mission

10

Success Rate

This graph compares

the average

successful completion

rate (A, B, or C

grades) for Summer

2010 – Summer 2012

(before OLI) with Fall

2012 – Spring 201558

59

60

61

62

63

64

To

tal C

om

ple

tio

n %

Average Across Semesters

General Education Biology

Successful Completion

AVERAGE-Pre OLI

AVERAGE-Post OLI

p < 0.08

11

Failure and Withdrawals

This graph compares

the average

Withdrawal and

Failure percentages

for Summer 2010 –

Summer 2012 (before

OLI) with Fall 2012 –

Spring 201528

29

30

31

32

33

34

35

36

To

tal F

/W/F

N %

Average Across Semesters

General Education Biology

Total F/W/FN

AVERAGE-Pre OLI

AVERAGE-Post OLI

p < 0.09

12

Activity and GradesStudents

demonstrated

more active online

engagement

compared with

average for other

high enrollment

courses (NUTR,

BIOL 160)

General

Education

Biology

Nutrition and

Human Biology

(high

enrollment

sections)

13

OLI Learning Dashboard• allows instructors

to see if students are achieving the learning objectives

• tracks student progress

– class– group– individual

• allows instructors to email students directly from the OLI site

14

OLI Learning Dashboard• Estimated

Learning Level is an index of learned content (based on low-stakes self-assessments and quiz scores

• All activities aligned with objectives

15

OLI Learning Dashboard

Colors and histogram convey estimated learning level for each student for a given learning objective

16


For a given learning objective, can look at performance levels for performance group as well as for individual students

17


Can examine how many students participated in an activity or quiz and view their performance

18

Student Progress--Dashboard

19

Individual Student Progress--Dashboard

20

Individual Student Progress-Gradebook

21


22


23


24

Student and Faculty Activity Data

OLI technical and administrative

collaboration

Activity data extends the Learning

Dashboard

Data examples: Student content views,

student logins, and more

25

Grades and OLI Engagement

Examination of the

relationship between

engagement with OLI

activities and course

grades reveals that

grades of A and B are

significantly correlated

with participation in

Learn By Doing and Did

I Get This learning

objects.

Estimated

Coefficient

Standard Error p-value

Did I get this 0.3858 0.1822 0.0395

Learn by doing 0.3785 0.1834 0.0444

Lowstakes Activities 0.3833 0.1827 0.0412

All Activities 0.4171 0.1890 0.0321

26

Grades and OLI Engagement

Examination of the

relationship between

engagement with OLI

activities and course

grades reveals that quiz

grades are significantly

correlated with

participation in Learn

By Doing and Did I Get

This learning objects.

Estimated

Coefficient

Standard Error p-value

Did I get this 33.608 4.942 1.49e-08

Learn by doing 33.614 4.981 1.79e-08

Lowstakes Activities 33.642 4.957 1.56e-08

All Activities 37.832 4.707 1.93e-10

27

Quiz Grades and OLI

EngagementSeveral information items about student engagement in OLI

can be extracted.

Activity in each of the areas below were checked for

correlation with OLI quiz grades.

Coefficient

Estimate

Standard Error p- value

mean_dashboard_acti

ons

0.3810 0.9138 0.679

mean_total_logins 0.4716 0.1539 0.00358 **

mean_myscores_view

s

0.9379 0.2453 0.00038 **

mean_myscores_actio

ns

0.03744 0.01362 0.00841 **

mean_pages_accesse

d

0.15234 0.01475 8.69e-14 **

mean_total_pageview

s

0.039352 0.004342 5.74e-12 **

28

• For the results shown in this presentation, students have not been

incentivized for engagement with self-assessment activities Learn

By Doing and Did I Get This in the OLI classroom

• To date, student participation in the OLI activities has been

encouraged but there are no gains or losses for course grades for

OLI participation

• Findings indicate that success (earning A and B grades and

passing quizzes) are significantly correlated with engagement in

OLI activities

• Analytics results across several semesters indicate that the

percentage of successful completions (grades A, B or C) have

improved after OLI was adopted as primary content (not statistically

significant)

Reflection

29

Next Steps

• In collaboration with the Center for Innovation, Learning and Student

Success, we are conducting a pilot study this Fall for courses in 4th

session of Fall 2015.

• Students will be incentivized by extra credit.

• Spring 2016 research study will involve all sections. Half of the sections

will be associated with incentive points.

• The experimental design of the Spring 2016 study is in development. It

will likely involve OLI activity engagement as a higher stakes

contributor to course points.

• The Open Learning Initiative team at Carnegie Mellon University,

Stanford University and their corporate partners have recently

developed the Student Dashboard. We are hopeful for inclusion of

Student Dashboard in the Spring 2016 research study. This will permit

students track their performance on Learn by Doing and Did I Get This

self-checks.30

Wrap Up

• Questions

• Discussions

• OLI Demonstration (if interests)

31