Data Quality in an Emergency Notification System

UCCSCWednesday, June 17, 2009

1:30-2:30pm, 1003 Giedt Hall

1Rev 3

Welcome!

• Introductions– Marian Sherrin– Eric Goodman– Audience

• Campus• Alert System Status• Statistics Collection• Experience with Data Quality

2

Agenda

• Data Quality Fundamentals

• Building Data Quality Into the Project

• Data Flows & Design Decisions Required

• Results

• Comparison to other Campuses

• Questions and Answers

3

Data Quality FundamentalsBasic Term Definition

Record of Origin (ROO) The first electronic instance of the data.

Record of Reference (ROR) The authoritative database considered the most reliable.

Reuse The activity of utilizing data from a ROR instead of creating a new instance.

Continuous Improvement The activity of improving the data quality in the ROR by understanding and fixing root causes of data errors via process improvement.

Data Quality Attributes

Definition

Accuracy The quality of being free from errors.

Completeness The degree to which values are present where required/desired.

Consistency The measure of data across different records.

Timeliness The degree to which the data is up to date.

Uniqueness The state of being the only record of its kind, not duplicated.

Validity The quality of data falling within a system of classification and allowable values.

4Keep the Goal in Mind!

Building DQ into the Project

Chair: VC Vani

C:Marian SherrinB:Alma SifuentesT:Greg Gaither

C: Ed TitusT: Ed Titus/John Rudd

C: Naomi GuntherB:Cathy Schoenfeld/Nancy Degnan T: Bob Oberg

C:Chris GaylordB:Chris Gaylord

C: Jim BurnsChair:Biz Lead:Tech Lead:

5

Co Chairs: Chris Gaylord/ Doug HartlineProject Mgr: Marian Sherrin

We found it important to focus on Staff/Faculty and

Students separately

VC Student Affairs as team member helped with process

and communication.

Keep the Goal in Mind!

Team Structure

Type

(% Data Complete for Campus “Readiness”)

Source Comments ITS+Business Effort

(Requires Validation with IT App Solutions)

Very Low – by April (existing data+IDM Source)

Low - Modified data, existing IDM source (<3 months - July)

Med – Modified data, new IDM source (<6 months – 2008)

High – New data source/IDM direct (>6 months – 2009+)

MUST HAVE for System “GO”

Student Cell Text AIS Registration

1) Registrars office has made decision to make input of Local and/or Mobile mandatory starting with Winter Registration.

2) Need approval to use “Mobile phone” numbers for WARN (can we designate “Local” as a Landline).

Low

Student RA Room Phones (100%)

Student Housing Online (SHO) DB

RA Cell/Text included above

Need source to designate RA’s and associated dorm phones (they are in PPS as RA title and Type 4 Students; also SHO)

Med (if use SHO)

Staff/Faculty 1st Choice

(cell, office, home) (>80%)

OnLine Directory

Modification to Online Directory to designate choice of contact and type (Land/Cell) +New Source for IDM

EOC Members included.

Med

EOC Member Office phones (100%)

EOC Member Cell /Text included above.

Need EOC “Group” (could be managed in WARN).

Source TBD (online directory or Phone database (sapphire?)

Low

Must Haves

6Keep the Goal in Mind!

Set Data Quality Goals

Early

Desired

HIGHLY DESIREDStaff Faculty Office or 2nd Device (100% for office)

OnLine Directory

Source TBD (online directory or Phone database (Pinnacle).

Can do Office phone with existing capability.

Med

Staff/Faculty CruzMail (100%)

IDM Can do with existing capability Very Low

Student CruzMail (100%)

IDM Can do with existing capability Very Low

Student Local (Landline) phone

AIS Assumption is that many of these would be off-campus.

Need to modify user interface to see if landline or should we assume (low risk?)

Very Low

Emergency Contacts for Student

AIS Need to change screens and database to designate if contact device is Land/cell/email.

Low

NICE TO HAVEEmergency Contacts for Student/Staff Faculty

OnLine Directory

Need to change screens and database to designate if contact device is Land/cell/email.

Med

Phase 1

Phase 1

Phase 1

NOT Req for Phase 1

NOT Req for Phase 1

7Keep the Goal in Mind!

8

High Level Project Timeline

Charge Letter

T -2

T -1

T4

Project Proposal Approved

T0

Contract Start date

T2T

1

#1 Campus Communication includes request to update directory & CruzAlert Opt In Page for Staff & Faculty

#1 Test capability of office telephone systems

Modifications made to AIS (if needed)Intercept July AIS Release

Student and Non-University Contact Information collected with Fall Registration

T3

T6

Phase 1 data transmitted to WARN

Returning Student Communication Request for additional contact information

Student information collected

6/1/08 July’08 July/Aug’08

#2 Test of EOC and DOC Coverage

Jan’093/21/08 4/2/08

Campus Implementation Schedule

Project Charter Approved

7/1/08

New Student Orientation

#2 Campus Communication to Request for additional contact information

Sept’08

#3Test of capability of email and cell SMS

Required to improve data

quality attributes:Accuracy

TimelinessValidity

Great opportunity to get to new students and their parents. New student cell

phone completeness 85%!

(returning students 4%)

T6

Student data mandatory for Winter’09

Returning student completeness rate increases from 4% to 98% (Cell – 83%, Landline 63%)

(2% Opt Out)

May’09

Keep the Goal in Mind!

9

Students choose type of phone

Students must validate information

annually

Opt-out capability was new

Student Screens

Breaking into 3 boxes will help with validity, but format example

has helped

10

Staff/Faculty Screens

11

Pre-populated with Name, Cruzmail

address, and office phone

You can change your office phone (without impacting directory

info) or add a second one under Additional

Phone

Add your Cell phone for SMS/Text

messages

Add an email in case CruzMail is out of

service

Home number may be used for campus

closures

Staff/Faculty Screens

Data Flow

AIS

PPS

Academic HR

Online Directory

IdentityManager

(also CruzMail

ROO)

W.A.R.N.Command

Currently Existed:

Status & Data

New

Only used high quality data:

Employee Status

Student Housing

12

New: Opt Out Flag

New

: RA

Gro

up

Pinnacle Phone

W.A.R.N.Broadcast

New: Campus Phones &

Resident Hall Phones

New Extracts: - All Phones- All Work- All Home- All Cell

Data

Last, First UC email UC office phone Cell Home / Landline

Personal email

Employee or Student

Student or Employee ID

W.A.R.N. ID

CruzAlertOracle Table

New

: RA

D

uty

Pho

ne

RA Duty Phones

Major Design Decisions• Student Workers – Count as students

• Resident Advisors (RA) – Used SHO vs. PPS CTO Codes

• RA Duty Phones – Manually load annually

• Separations– From employment – Exclude via PPS

– Graduation – Exclude via AIS

– Dropping out, transferring – Exclude via AIS

• Leaves, Sabbaticals, Temporary Assignments – Exclude

• Summer Sessions/Non-degree – Include via AIS

• Emeritus – Include (some still live on campus)

• Sundry – Include (email only)

• Visitors/Conferences – Include in Phase 2 – New Process

• Parents – Not included in Phase 1 (students could use local phone)

13

Key Actions TakenData Attribute Addressed Action/Comments

Name ReuseCompletenessTimeliness

From PPS/AIS (ROO) via IDM (ROR, Reuse)

UC email Accuracy From IDM (ROO)

Cell Phone CompletenessTimelinessValidity

Student: From AISEmployee: From CruzAlert Opt InArea Code CheckWARN Validity/Block Check

Office Phone CompletenessValidity

Broadcast list from Pinnacle to capture all campus phones.

Home Phone Validity Area Code Check

Alternate email Validity After Action Follow-up (Continuous Improvement)

ID: Student or Employee

Uniqueness Default is Student

ID: W.A.R.N. Uniqueness Created foreign key in WARN

14

Completeness Actions

From W.A.R.N. Command nightly

Annual Load All Campus Phones

•Office•Reception•Hall•Classroom•Residence Halls•RA/CRE Duty Phones

15

UCSC ResultsMetric - Completeness Goal Launch

September 2008October 2008

May 2009

Campus Email 100% 100% 100% 100%

Student - Cell 75% 23% 54% 83%

Student – Local Landline InaccurateUntimelyInvalid

63%

Staff & Faculty – Office 80% 54% 54% 71%/100%*

Staff & Faculty - Cell 70% 14% 14% 25%

Staff & Faculty - Home 13% 13% 22%

May 2009 Campus Population: 19,850Active Students - 15,590

Employees - 4,260Staff - 3,240

Faculty - 1,020

*100% covered by WARN Broadcastlist covering ALL campus phones.

16

Emergency Notification System Status (UC campuses)

Campus Usage StatusUC Davis W.A.R.N. Deployed 3/6/08 (1st use in real situation)

UC Berkeley Campus W.A.R.N.; Lawrence Lab uses 3N

UC San Francisco – Med Center

W.A.R.N.

Also SF Alert

Deployed 4/09

SF Alert is SF city owned and we believe uses ROAM

UC Santa Cruz W.A.R.N.

Also City Watch

Deployed 9/08

1st use in real situation 5/21/09

UC Merced W.A.R.N.

UC Irvine None Yet: WARN and MIR3 evaluation

Unknown. W.A.R.N. demo’d in January’08

UC Santa Barbara ROAM Secure Deployed Fall’07.

Fall’08 – Included student emails and cell phone opt in.

Fall’09 – Student cell phones from registrar – opt out.

UC Los Angeles Alternate System Used Bruin Alert Deployed; Parents Notified

of Student Info Request

UC San Diego Uses MIR3

Med center uses Amcom

UC Riverside Uses 3N Deployed November 2007

17

Methods & Results

Methods• Mandatory with Opt Out• Opt In• Hybrid

UCSC Choice – Hybrid• Mandatory for UC “owned” accounts/devices

• UCSC Email• UCSC Networked Phones• Student Residence Phones

• Mandatory with Opt Out• Student phone contact via either:

• Cell/Mobile• We create to WARN SMS

• Local Landline

• Employee Personal Data - Opt In• Specific Office Phones • Personal Cell phone

• We create WARN SMS

• Home/Alternate Phone• Personal Email

18

Methods & Results Campus Population Method Results/Completeness

Santa Cruz(M. Sherrin)

S – 15,590E – 4, 260

Hybrid* Email – 100%Office – 71/100% (1)

Cell /Home – 60%

Davis(V. Lucas)

S – 38,960E – 23, 640

Hybrid Email – 100%Office – 25% (2)

Cell Phone – 26%(2)

San Francisco(C. Jones)

S – E – 18,600

Hybrid Email – 100%Cell Phone – 4% (4)

Riverside S – 18,000E –

Opt In Cell/Home – 60%

Merced S – 2,400

Berkeley S – 35,400F - 2,000

Santa Barbara

S – 20,700E – 3,880

Hybrid Email – 104%Cell Phone – 37%

Irving S – 27,631E – 14,000

Los Angeles S – 38,400F – 4,000

San Diego S – 28,500E – 26,000

3N Industry Data

N/A Opt In 30-40%

Methods• Mandatory with Opt Out• Opt In• Hybrid

UCSC Choice – Hybrid• Mandatory for UC “owned” accounts/devices

• UCSC Email• UCSC Networked Phones• Student Residence Phones

• Mandatory with Opt Out• Student phone contact via either:

• Cell/Mobile• We create to WARN SMS

• Local Landline

• Employee Personal Data - Opt In• Specific Office Phones • Personal Cell phone

• We create WARN SMS

• Home/Alternate Phone• Personal Email

(1) 71% of staff/faculty specific phones in W.A.R.N. Command100% of campus phones in W.A.R.N. Broadcast(2) Percent of total population, not just staff/faculty.

(3) Source: UCR Larry L. Sautter Award Application, 2008(4) Launch was April 2009. 19

Key Takeaways

20

• Understand Data Quality Fundamentals

• Build Data Quality Into the Project• Keep the End in Mind• Teams• Goals• Schedule

• Make the Right DQ Decisions

• Measure Results

• Continuously Improve

Q&A

21

Reference Sources

• Larry P. English, Improving Data Warehouse and Business Information Quality, John Wiley & Sons, Inc., New York, NY. 1999

• DOD Guidelines on Data Quality Management (Summary), ~1997http://www.tricare.mil/ocfo/_docs/DoDGuidelinesOnDataQualityManagement.pdf

• Larry McGrath & Joel Nylander, UC Riverside Application for 2008 Larry L. Sautter Award, 2008

22

Presenter/Contributor Biographies

• Marian Sherrin– New to the University of California, Santa

Cruz, in 2008. She was the Data Quality Manager for Intel’s Supply Chain Network from 1999-2005.

• Eric Goodman– Joined the University of California, Santa Cruz

in 1990. For the past several years has been the project manager of UCSC's Identity Management system.

23