using analytics for institutional transformation - dr. yvette mozie-ross - university of maryland...

Using Analytics for Institutional Transformation

Yvette Mozie-Ross, Ph.D.Blackboard Teaching and Learning Conference, Asia

October 6, 2016

On your mobile device, please browse to:

responseware.turningtechnologies.com

Enter Session ID: bblearn

50.umbc.edu | #UMBC50

Highlights

• About Us

• How We Got Here

• Analytics on the Analytics

• Data Driven Decision-Making

• Impact and Implications

• What’s Next?

50.umbc.edu | #UMBC50

About UMBC

50.umbc.edu | #UMBC50

UMBC: An Honors University in Maryland

Student Enrollment, Fall 2016Undergraduate: 11,142Graduate: 2,498Total: 13,640

Freshman Class Profile, Fall 2016SAT Average: 1217Grade Point Average: 3.7Living on Campus: 75%

15 NCAA Division I Sports (America East Conference)

Faculty:527 Full-time298 Part-time

Staff:1,288

UG Programs:62 Majors; 41 Minors; 22 Certificates

Grad Programs: 52 Masters; 29 Doctoral; 26 Certificates

50.umbc.edu | #UMBC50

President Freeman A. Hrabowski

Carnegie Classification – Doctoral Universities (Higher Research Activity)

What significant milestone is UMBC celebrating this year?

50.umbc.edu | #UMBC50

Recent AccoladesTop 5 Most Innovative Schools

US News and World Report

Top Universities for Undergraduate EducationPrinceton Review’s The Best 381 Colleges

Top 100 Public UniversitiesForbes Top Colleges

Leaders in Supporting Low Income StudentsUS Department of Education

Outstanding Academic Workplace (Honor Roll Distinction)Chronicle of Higher Education

President’s Cup Qualifier (Final Four of College Chess) - A Record 15 Times!National Collegiate Team Chess Championship

50.umbc.edu | #UMBC50

How We Got Here

50.umbc.edu | #UMBC50

Brief History of REX (Report Exchange)2006UMBC purchased a data warehouse product from a small local company, iStrategy. The product loads data from People Soft Student Administration (PSSA) into a set of Microsoft tables designed to facilitate reporting (came with many delivered reports).

2007Converted critical legacy data into a “virtual” People Soft database, thereby getting the warehouse and its reporting tool to work.

2009Went live with PS Student Administration (Recruitment, Admissions, Financial Aid, Registration and Student Billing)

2010Converted census (“frozen”) data from 1995 to present into data warehouse.

50.umbc.edu | #UMBC50

Brief History of REX (Report Exchange)

2011Blackboard acquired iStrategy and renamed the product line, Blackboard Analytics (BbA).UMBC begins beta testing the BbA modules for PeopleSoft Financial Aid and PeopleSoft Finance

2012UMBC migrates the Student Admin module to BbA version 4.0

2013REX web site is born and the data warehouse is upgraded to SQL Server 2012

50.umbc.edu | #UMBC50

5 Key Elements to Success

50.umbc.edu | #UMBC50

Key Element #1

The Right People

Champions (Campus and Division/Unit)Users (Inquisitive Nature, Data-Driven)

Technical (for Implementation)

50.umbc.edu | #UMBC50

Key Element #2

Spirit of Collaboration

Shared Data WarehouseOne Source

Transparency

50.umbc.edu | #UMBC50

Key Element #3

A Well-Defined Access and Training Strategy

Skill Levels of Report UsersLearning Styles and Modes of Delivery

Training Mechanism Sustainable Over Time

50.umbc.edu | #UMBC50

How many REX Guided Report Users?

50.umbc.edu | #UMBC50

Snapshot of Current REX Users

657 Guided Report Users (Parameter Driven MS Reporting Services Reports)President, Provost, Deans, Department Chairs, Back offices - Undergraduate Admissions, Undergraduate and Graduate Program Directors, Advising Coordinators

50 Exploratory Users (Pyramid)Enrollment Management Directors, Associate Deans, Academic ProgramSpecialists/Coordinators, Admissions Counselors

15 Report Builders (Directly Against Cube or Dimensional Model)Institutional Research Analysts, Division of Information Technology Analysts, Department IT Analysts

Key Element #4

A Comprehensive Communications Plan

Endorsed by LeadershipEnable/Enhance a Data Driven Culture

Include Outreach and FeedbackBranding

50.umbc.edu | #UMBC50

Key Element #5

An Effective Governance Structure

Fluid Not RigidOrganically Created

Team-based

50.umbc.edu | #UMBC50

Key Elements Summary

1. The Right People

2. Spirit of Collaboration

3. A Well-Defined Access and Training Strategy

4. A Comprehensive Communications Plan

5. An Effective Governance Structure

50.umbc.edu | #UMBC50

Analytics on the Analytics

50.umbc.edu | #UMBC50

Number of Distinct Users Since January 1, 2016?

A. 58B. 448C. 726

Number of Overall Report Runs Since January 1, 2016?

A. Less than 100,000B. 100,000 to 175,000C. Over 175,000

Peak Day of Week for REX Usage?

A. MondayB. WednesdayC. Friday

Peak Time of Day for Rex Usage?

A. 10:00 AMB. 12:00 PMC. 4:00 PM

Data Driven Decision-Making

50.umbc.edu | #UMBC50

UG Program Directors and Chairs Want to Know…

What are the admission and yield rates of students from XYZ High School?

How are students from XYZ High School performing in key gateway courses in the majors? Are there subject areas or courses of particular concern (high DFW rates)?

What are the graduation and retention rates of students from XYZ High School matriculating at UMBC?

50.umbc.edu | #UMBC50

Report Highlights:• Last 5 Years of Data• 5 Year Applicant and Enrollment Trends• Student Profile – GPA and Test Scores• Retention/Graduation Rates• Degrees Awarded• Segment Further by School Control and

County

Report Highlights:• DFW Counts and Rates• DFW Rate Comparison by

Institution, College, and Subject• Grade Distribution• Segment Further by School Control

and County

UG Admissions Wants to Know…

Which recruitment events appear to have the greatest impact on yield?

Do specific events appear to impact specific demographic populations differently?

What do we know about our event participants? How did they get to our doorsteps? (Prospects, Inquirers, Score Senders, Stealth Applicants, etc.)

50.umbc.edu | #UMBC50

Report Highlights:• Event Attendance Comparison• Yield by Initial Recruiting Status• Segment Further by

Race/Ethnicity, Gender, and Residency

Enrollment Management and Deans Want to Know…

Are we offering enough of the types of classes our students need to progress to degree completion in a timely fashion?

Where is there evidence of enrollment pressure and unmet demand?

Are we maximizing use of existing resources? Where is there evidence of underutilization of courses?

50.umbc.edu | #UMBC50

Report Highlights:• Current Year

Comparison to PY Current, Day 1, and Census

• Over/under utilized courses

• Waitlist Totals• Further Segment

by Course Attribute (i.e. Gen Eds)

Academic Advisors Want to Know…

How do students’ performance in course A relate to their performance in course B?

What are the outcomes (persistence, retention, graduation) based on student performance in a course?

50.umbc.edu | #UMBC50

Report Highlights:• Course Descriptive

Statistics• Comparison to

“Course Not Taken” Population

• Segment further by Course Section

Report Highlights:• Outcomes by Grade• Further Segment by

Course Section• Display by Admit Type,

Race/Ethnicity, Grades• Useful for assessing

effectiveness of Interventions

Faculty Want to Know…

How does my Bb LMS course activity (accesses, interactions, submissions) change over the course of the semester?

How does my Bb LMS course activity compare with others?

50.umbc.edu | #UMBC50

Report Highlights:• My Course Activity Average

v. All Course Activity Average

• Averages by Type of Activity

• Useful for Assessing Course Redesign Impact

Students Want to Know…

How am I doing in XYZ course relative to my peers?

How have students with comparable activity performed in XYZ course?

50.umbc.edu | #UMBC50

Report Highlights:

• Student’s Activity Level and Activity Rank

• Course Activity (Average Usage)

• Student Usage by Tool• Course Usage by Tool

Impact

• Changed the nature of the questions

• Allowed for more substantive conversations

• Improved data quality and confidence in data

• Enabled responsiveness to constituents

• Supported more evidence-based decisions

50.umbc.edu | #UMBC50

Examples of ImpactEnrollment Management/Course Demand Committee –

Made the Case for the Allocation of $250K - $300K Annually in Enrollment Pressure Funding to Departments to Support the Addition of Course Sections

Undergraduate Admissions – Confirmed Decision to Discontinue Hosting Regional Reception for Admitted Students Resulting in $40K

Saving In Recruitment Expenditures

Academic Advising and Financial Aid – Strategically Identified Students Not Yet Registered Due to Outstanding Balance; Leveraged Senior

Completion Fund ($50K) to Assist 26 Students w/ Outstanding Balances Resulting in Increased Enrollment, Increased Net Revenue and Increased Degrees Awarded

Departments/Colleges – Inspired Course Redesign in Gateway Chemistry and Mathematics Courses

50.umbc.edu | #UMBC50

Implications

• Demand – Experienced Significant Increase (”Success Breeds Success”)

• Staffing – Need More Technical Staff (Currently Supported by Staff of 5)

• Policy – Need to Establish Policy w/ Respect to Access and Use of Data

• Training – Need for Increased Training on FERPA and Acceptable Use

50.umbc.edu | #UMBC50

What’s Next?• Co-Curricular Analysis - Bringing Other Data Into the Warehouse ( i.e. Event Attendance, Co-

Curricular Involvement, Living-Learning Communities)

• Assessment - Using the Warehouse to Assess Effectiveness of Interventions (i.e. Supplemental Instruction, Tutoring)

• Predictive Analytics – Using the Warehouse to Feed and Receive Data Back from Predictive Tools (i.e. Civitas, Blackboard Predict)

• Administrative Reporting – Increasing Availability of Finance and HR Data and Reports

• Documentation/Training – Developing a More Comprehensive User Resources Library (i.e. Data Dictionary)

50.umbc.edu | #UMBC50

Questions?

50.umbc.edu | #UMBC50