student success orientation course & persistence of online students

THE RELATIONSHIP BETWEEN A MANDATORY STUDENT SUCCESS ORIENTATION COURSE AND PERSISTENCE OF ONLINE STUDENTS

DR. JEFF HALL PROFESSOR/LEAD FACULTY MORGAN JOHNSON DIRECTOR OF INSTRUCTIONAL SERVICES

ICEBREAKER

Three Minute Brainstorm:

How is orientation different for online vs. traditional students?

PRESENTATION OUTLINE Jeff Hall:

About Ashford University

Rationale, purpose, structure, components, and future of orientation

Morgan Johnson:

Assessments in Orientation

Persistence Data

Prediction Models

Quality Assurance

ABOUT ASHFORD UNIVERSITY •Mount St. Clare College (1918) - Sisters of St. Francis in Clinton, Iowa •Acquired by Bridgepoint Education in 2005 and renamed Ashford University

•Large proprietary institution, enrollment > 80,000; HLC Accreditation •Traditional campus in Iowa; 98% of students online • Open Admissions

•FT faculty in Iowa, Denver, and San Diego

RATIONALE FOR ORIENTATION •Open admissions = Greater risk of attrition •Provide students a no-cost, risk-free opportunity to explore online learning and experience the Ashford University classroom •Allows students to self-reflect on their readiness for online learning

•Gives students time to decide whether Ashford is the right university for them

•Could affect persistence rates in two ways: 1. Students can opt-out and not enroll in first course: EXP 105. 2. Filter out students who are not ready/committed

PURPOSE OF ORIENTATION • Provide a warm welcome – First faculty contact

• Help students become comfortable with the concept of online

learning and navigation of the online classroom

• Foster a sense of community that facilitates academic success

• Provide instructive, individualized, and personal feedback

• Challenge students to think, reflect, learn, and grow

• Continually remind students that “Graduation Begins Today!”

STRUCTURE OF ORIENTATION •11 day online experience

•Limited to students in one college with <24 transfer credits

•Closely mimics structure of typical Ashford course

•Faculty driven (part-time faculty cohort) •Interaction with peers (40 student cap)

•Non-graded (Pass/Not Pass) – Requires subjectivity from faculty

•Emphasis: Readiness & Reflection

COMPONENTS OF ORIENTATION Continually evolving course - Incremental Changes October 2011 •Introduction Discussion Board •SmarterMeasure Readiness Assessment & Reflection •English Proficiency Assessment •Time Management Activity & Discussion •13 question quiz – 100% required •Academic Dishonesty reflection & submission to Turnitin

January 2012 – Present •Introduction Discussion Board •SmarterMeasure Readiness Assessment & Reflection •Ashford Library/Writing Center Discussion •10 question quiz – 80% required •Self-reflection describing challenges & knowledge gained in Orientation

FUTURE OF ORIENTATION June 2012 •Introduction •SmarterMeasure Readiness Assessment & Reflection •Writing Readiness Assessment created by SmarterMeasure •Time Management discussion moved to EXP 105 •Ashford Library/Writing Center Discussion •10 question quiz – 80% required •Self-reflection describing challenges & knowledge gained in Orientation •Roll out to additional colleges (Liberal Arts: July, 2012; Ed. & Bus. after) •Continual assessment and tweaking

Orientation Assessment

& Research

ENGLISH PROFICIENCY EXAM • What did it predict? Orientation course metrics are indicative of students’ current and future performance. English Proficiency score is predictive of both Orientation completion and first course (EXP 105) grade.

• Why did we get rid of it? Assessment vs. reflection tool, no immediate intervention. • What could we be doing with the results? Will be reintroducing a SmarterMeasure writing readiness assessment that provides reflective exercises and immediate feedback.

Presenter

Presentation Notes

Why did we get rid of English Proficiency? More important than taking your child’s temperature is taking appropriate action based on his/her temperature. More important than measuring student readiness is taking appropriate action based on student scores.

ENGLISH PROFICIENCY EXAM Students who pass orientation

course have higher average score in English

Proficiency test.

Students who successfully complete first course have higher average scores on English test. English test score explains 2% of the total

variation of EXP105 grade points.

*Provided by BPE Data Analytics

SMARTERMEASURE: STUDENT READINESS

Student Readiness

Know which students are at

risk of not doing well at online

learning.

Know which students are a

good fit for online learning.

Have a dialogue with students about learning

online.

Provide resources to help online

students succeed.

Presenter

Presentation Notes

The SmarterMeasure Learning Readiness Indicator is a web-based tool which assesses a learner's likelihood for succeeding in an online and/or technology rich learning program. SmarterMeasure indicates the degree to which an individual student possesses attributes, skills, and knowledge that contribute to success.

SMARTERMEASURE IN ORIENTATION • A 124 item online skills test and attributes inventory that measures a student’s level of readiness for studying online • Key indicators:

•Individual Attributes – Motivation, Procrastination, Ask For Help •Learning Styles •Technical Skills & Competency •On-Screen Reading Rate & Recall •Typing Rate & Accuracy •Life Factors – Time, Place, Resources

Presenter

Presentation Notes

In September 2011,three hundred institutions which use the SmarterMeasure Learning Readiness Indicator were asked to participate in a survey to identify trends and good practices in the implementation of SmarterMeasure. 2011 SMARTERMEASURE INTEGRATION SURVEY REPORT: http://smartermeasure.com/smartermeasure/assets/File/Usage-Report-2011.pdf Report Highlights include: 61% of institutions report using SM in an Orientation course. 59% of institutions embed SmarterMeasure as an assignment in an online course. 68% of institutions have added SmarterMeasure to their school’s website

SMARTERMEASURE AND ORIENTATION COURSE OUTCOME Students who do not complete orientation (Final grade = unknown) have different distribution in Personal Attributes and Life Factor scores.


Presenter

Presentation Notes

Tech Competency and Life Factors are the top two indicators of student success in orientation class.

ORIENTATION RESULTS AND IMPACT ON PERSISTENCE • Orientation course metrics are indicative of students’ current and future performance:

English Proficiency score is predictive of both Orientation completion and EXP105 grade. Life Factors and Personal Attributes are most indicative of a student’s ability to progress through the Orientation course. Tech Competency, Reading, and Tech Knowledge are most correlated with EXP105 success. Life Factors and Personal Attributes are the only two measurements that correlate with EXP105 engagement.

FIRST COURSE COMPARISON Orientation cohort has a higher success rate than other cohorts.


Presenter

Presentation Notes

Do students graduate from Orientation behave differently than others in EXP 105? Yes. Orientation course has a positive impact on EXP105: � Orientation graduates have a significantly higher submission rate than other cohorts.� Orientation graduates have higher success rate and grades than other students.� Orientation cohort spend more time in classroom and login more frequently than other cohorts.

FIRST COURSE COMPARISON Orientation completers have higher grade point averages in EXP 105 and are more engaged (time spent online) than other students.


Presenter

Presentation Notes

Do students graduate from Orientation behave differently than others in EXP 105? Yes. Orientation course has a positive impact on EXP105: � Orientation graduates have a significantly higher submission rate than other cohorts.� Orientation graduates have higher success rate and grades than other students.� Orientation cohort spend more time in classroom and login more frequently than other cohorts.

PREDICTIVE MODELING

“REAL” Intervention:

Reading: Average number of times the student accesses digital course materials per day. Engagement: Average amount of minutes the student is accessing the online classroom per day. Assignments: Current course grade based on graded assignments. Logins: Number of days since the student last posted attendance by submitting a gradable assignment.

Presenter

Presentation Notes

Success: An overall predictive metric, based on the data in the other four metrics, which indicate the statistical likelihood that the student will continue. The various factors that influence student success were discovered by observing students’ behavior. These factors were then used to create a statistical model that predicts likelihood of continuation. The intervention report is a list of students in a course who are falling below average in certain areas. The report looks at student reading, engagement, assignments, and logins.

PREDICTIVE MODELING High Risk

Low Risk

Medium Risk

Target students that need additional support for academic success

Academic intervention has had most impact on students that fall into medium-risk

Presenter

Presentation Notes

Based on the first few months of the intervention pilot, it appears that we are having a positive impact for students that have been identified as "medium-risk." The impact is greater for students starting in EXP105 vs. PSY202. There are improvements for PSY202 students, but they are not statistically significant on their own with the current number of data points.

QUALITY CONTROL • A team of dedicated professionals support faculty in all courses, including Orientation

Instructional Specialist

Support Faculty

Facilitate Academic

Issues

Monitor Faculty

Performance

Presenter

Presentation Notes

To commensurate with Ashford University's mission to provide high quality learning opportunities and degree programs, multiple initiatives are underway to support faculty performance. A group of full-time staff support and monitor faculty to ensure a quality classroom experience for all students. Ashford’s online associate faculty are guided and supported by Ashford’s team of Instructional Specialists. This role includes monitoring and evaluating online courses to ensure that best practices in pedagogy are maintained, providing faculty with constructive feedback, and ensuring student learning needs are met within the online course environment. Additionally, the Instructional Specialist works closely with other departments to support faculty recruitment, professional development, scheduling, and the investigation of academic issues, including grade appeals and other grievances. The model Ashford University has had in place for several years, which utilizes Instructional Specialists to monitor and support online faculty, has proven to ensure active classroom participation and faculty-student engagement. The model has required all faculty to perform at high levels. It also ensures that students receive quick responses to their queries and timely, helpful feedback to their assignment posts; that grades are posted in a timely manner; and that faculty has a responsive point of contact with the University for their questions or concerns related to the classroom.

CONCLUSION • Does Orientation filter? Yes, given the evidence. • Does it better prepare students for success? Inconclusive thus far, but we’d like to think so!

student success orientation course & persistence of online students

Education

traditional students

online classroom

faculty contact help

ashford universityrationale

ashford universitymount

online learning andexperience

orientation different

persistence rates