dashboards as a data management tool
DESCRIPTION
Dashboards as a Data Management Tool. Dr. David C. Oehler Director of Assessment, Information and Analysis Northwest Missouri State University. Presentation Overview. Facilitating decision-making and driving improvement Measurement impact on focus of institutional activities and processes - PowerPoint PPT PresentationTRANSCRIPT
Dashboards as a Data Dashboards as a Data Management ToolManagement Tool
Dr. David C. OehlerDirector of Assessment, Information and Analysis
Northwest Missouri State University
06/21/2006 2
Presentation Overview Facilitating decision-making and
driving improvement Measurement impact on focus of
institutional activities and processes
How we developed our system and integrated it into information systems
Building a House of Quality
MANAGEMENT BY FACT
CONTINUOUSIMPROVEMENT
CU
ST
OM
ER
FO
CU
S
HIG
H E
XP
EC
TA
TIO
NS
INV
OL
VE
ME
NT
AS
SE
SS
ME
NT
&F
EE
DB
AC
K
TRUST
SHARED VALUES AND VISION
06/21/2006 4
Facilitating Decision-making and Driving Improvement
Importance of institutional culture Importance of internal
communication Alignment with mission and vision Alignment with strategic planning
06/21/2006 5
“All work is process.”
“Every process is perfectly designed to create the outcomes it produces.”
“If you can’t define it, you can’t measure it; if you don’t measure it,
you won’t improve it.”
Institutional CultureCQI Philosophy
06/21/2006 6
Communication – Northwest’s Planning Model
Since 1993, all units at Northwest use a Seven-Step Planning Process (the Hubbard Cycle) to focus and define their work.
This process is based on five underlying questions:
06/21/2006 7
CQI Underlying Questions
1. Who are your customers/clients? 2. What are their expectations?3. How do you “deliver” quality? 4. How do you know how well you
are doing? 5. What performance improvement
targets are reasonable?
06/21/2006 8
Key Quality Indicators “Key” - identifying a few very
important things to watch. “Quality” - a perception held by
students, stakeholders, and each other.
“Indicators” - evaluative tools that indirectly measure aspects of quality; multiple indicators are needed where direct measures are not available.
06/21/2006 9
1. What are your key decision areas? (as defined in your job description)
2. What kinds of decisions do you make?3. What information do you need to make
good decisions?4. What data do you need to create good
information?5. What is the best way to obtain that
data?
Focusing on Peoples’ Information Needs
06/21/2006 10
Communication and Systems Alignment
Annual Report Format Five-year Academic Department
Review (Coordinating Board for Higher Education)
Annual departmental visits (Deans’ executive summaries)
Phase 1:STRATEGICCONTEXT
Phase 2:STRATEGY
IDENTIFICATION& REFINEMENT
Phase 3:7-STEP TACTICAL
PLAN DEVELOPMENT &
DEPLOYMENT
Phase 4:BALDRIGE
SELF-ASSESSMENT
MISSION/VISION/VALUES
DECISIONDRIVERS
ENVIRON-MENTAL
SCAN
INTERNALANALYSIS
STRATEGICKQIs
TACTICAL KQIs
OBJECTIVES
ASSESSMENT
ACTION PLANS
RESULTS
PROCESSMANAGEMENT
FACULTY/STAFF FOCUS
PLANNING
STUDENTFOCUS
INFORMATION& ANALYSIS
LEADERSHIP
NORTHWEST QUALITY SYSTEMS MODEL
TRENDS
IMPROVEMENTS
STRETCH GOALS
STRATEGICINITIATIVES
SICHAMPIONS
DEPLOYMENTPLAN
MEASUREMENTPLAN
COMMUNI-CATIONS
PLAN
06/21/2006 13
Presentation Overview Facilitating decision-making and
driving improvement Measurement impact on focus of
institutional activities and processes
How we developed our system and integrated it into information systems
What is measured gets noticed
What is noticed gets acted on
What is acted on
gets improved
This and the following slide were adapted from a Dee W.
Hook presentation.
Phenomenon of Measurement
06/21/2006 15
What is a Dashboard? Definition/use:
Both a process and a tool Looking for unfavorable trends or patterns and
focusing energy on improving priority areas A (diagnostic) means for monitoring
performance to ascertain what is working well and where additional attention is needed
A few (4-6) sets of indicators, representing the most central areas related to high performance
06/21/2006 16
How Does a Dashboard Focus Activities and Processes?
Requires clear definition of outcomes Focuses on a manageable (small) set of key
outcomes (results) Encourages cross-functional communication Requires fact-based decision processes
Data reporting structures Process improvement orientation Layering of detail (summative vs. formative)
06/21/2006 17
Dashboards help you know what’s important
Dashboards focus on actions that make a difference
Collect data to create information you can use, then use it
Time is Increasingly a Precious Resource
06/21/2006 18
Assessment needs to answer questions
Systems to collect, analyze, and report information need to be developed to support the specific information requirements
Data for Decision-making
06/21/2006 19
Assessment System Design
1. Data Collection Centralized measures Decentralized
measures
3. Reporting Systems Summative
Information Formative Information
2. Data Processing Disaggregation system Aggregation system No transformation
4. Analysis/Decision-making
Cabinet, Deans, Directors,
Department Chairs Department Chairs,
Faculty
Data Collection
Reporting Systems
Analysis and Decision-making
Centralized
Decentralized
Disaggregation System
Aggregation System
Summative Information (Dashboards,
Profiles)
Formative Information (Operational)
Cabinet, Governing Board, External
Audiences
Directors, Department Chairs,
Faculty, Staff
No Transformation
Deans, Directors, Department Chairs
No Transformation
Directors, Department Chairs
Much Little
Detail Detail
Data Processing
06/21/2006 21
What Metrics are in a Dashboard?
The Dashboard includes a balanced view of an organization Learning and Growth (student academic progress;
faculty and staff development, scholarship and research
productivity) Customer Relations (students, faculty, staff, alumni,
parent satisfaction and involvement) Internal Processes (functional area performance –
accuracy, timeliness, friendliness) Financial Measures (fiscal health and viability)
Sample Traditional Balanced Scorecard
Measure/Index title 10 9 8 7 6 5 4 3 2 1 Wgt
Index 1 - Student Academic Progress 20%
Index 2 - Faculty & Staff Development 10%
Index 3 - Scholarship & Research Productivity 5%
Index 4 - Student Satisfaction & Involvement 12%
Index 5 - Faculty & Staff Satisfaction & Involvement 4%
Index 6 - Alumni & Parent Satisfaction & Involvement 4%
Index 7 - Internal offices process accuracy 10%
Index 8 - Internal offices process timeliness 5%
Index 9 - Internal offices process friendliness 5%
Index 10 - Fund balance 10%
Index 11 - Endowment 10%
Index 12 - Expenditures 5%
Lea
rnin
g &
G
row
thC
ust
om
er
Rel
atio
ns
Inte
rnal
P
roce
sses
Fin
anci
al
Mea
sure
s
06/21/2006 26
Northwest’s Dashboard Model
Our model includes several types of information/report presentations Dashboards – single screen current status Trend charts – key data element trends over
time Data tables –key data detail trends over time
Majors, minors, advisees, degrees, SCH, financials Special interest charts/tables
06/21/2006 27
Features of the Northwest Balanced Scorecard System
Dashboard “lights” to indicate current status
Hyperlinks to navigate through workbooks Hyperlinks to “drill down” to detail Comparative data links for setting targets Real-time data updates Accommodates various data sources Modular design to facilitate upgrading Automated updating of modules
President’s DashboardPresident’s Dashboard
Provost’s DashboardProvost’s Dashboard
VP Student Affairs’ DashboardVP Student Affairs’ Dashboard
Additional Detail
06/21/2006 32
Comparative Data In order to judge how good your
performance is, results should be put into some context Trends over time Comparisons to other internal units Comparisons with peer groups Comparisons outside of the education
sector
Academic Workload - SCH Production
50%
60%
70%
80%
90%
100%
110%
120%
130%
140%
150%
SCH Production (SU Total) % 100.0% 96.0% 97.9% 105.2% 103.9% 131.7% 142.9% 123.3% 122.5% 121.6%
SCH Production (FA Total) % 100.0% 100.0% 100.1% 101.6% 100.3% 102.3% 104.3% 106.2% 104.9% 103.3%
SCH Production (SP Total) % 100.0% 101.2% 101.7% 103.8% 102.6% 106.2% 104.3% 107.7% 104.2% 101.5%
SCH Production (Total) # 12,602 73,708 67,404 12,095 73,704 68,193 12,339 73,770 68,564 13,259 74,908 69,945 13,089 73,906 69,143 16,601 75,386 71,599 18,011 76,857 70,282 15,533 78,312 72,580 15,442 77,356 70,213 15,328 76,177 68,421
0 0 0 Su 94 F 94 Sp 95 Su 95 F 95 Sp 96 Su 96 F 96 Sp 97 Su 97 F 97 Sp 98 Su 98 F 98 Sp 99 Su 99 F 99 Sp 00 Su 00 F 00 Sp 01 Su 01 F 01 Sp 02 Su 02 F 02 Sp 03 Su 03 F 03 Sp 04
Student Satisfaction
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Instructor made objectives clear 1.5 1.5 1.5 1.5 1.5 1.5 1.4 1.4 1.5
Instructor helped me achieve them 1.8 1.7 1.7 1.7 1.7 1.6 1.6 1.6 1.6
Instructor provided regular feedback 1.8 1.8 1.8 1.8 1.8 1.7 1.6 1.6 1.6
Instructor accessible during office hrs 1.7 1.6 1.6 1.6 1.5 1.4 1.4 1.4 1.4
Alumni - overall satisfaction rating 1.6 1.8 1.8 1.7 1.4
Alumni 'N'
93-94 94-95 95-96 96-97 97-98 98-99 99-00 00-01 01-02 02-03 03-04
Graduation Rate (CBHE)
30%
40%
50%
60%
70%p
erce
nt
(0-1
00)
Northwest 44.0% 44.0% 43.0% 48.0% 53.0% 57.3% 59.0%
Best competitor 45.0% 53.0% 54.0% 55.2% 56.0%
sp 95 data sp 96 data sp 97 data sp 98 data sp 99 data sp 00 data sp 01 data sp 02 data sp 03 data sp 04 data sp 05 data
Positive Referrals - Alumni Outcomes Survey
70%
80%
90%
100%
Per
cen
t P
osi
tive
(0-
100)
choose national 77.8% 81.2% 81.7%
choose cohort 75.7% 80.3% 79.1%
recommend national 94.3% 97.0% 97.0%
recommend cohort 94.6% 98.0% 97.5%
if beginning again, choose this school NW 85.0% 92.9% 94.1%
recommend to others NW 97.0% 99.5% 99.4%
2000 2001 2002
06/21/2006 40
Interpreting Dashboard Indicators
To follow up on indicators of interest, use hyperlinks to access increasing levels of detail
Student satisfaction as an example President’s dashboard to Provost’s dashboard to Noel-Levitz Student Satisfaction
Inventory data trends
Praxis andC-BASEresultsmodule
MajorField test
resultsmodule
AcademicProfileresultsmodule
Undergrad majors and
minors, Graduate majors,
Degrees, Advisees module
Studentopinionnairesof teaching
module
GeneralEducation
localmodule
ProgramSCH
generationmodule
Financialdata
module
Placementdata
module
EMSASmodule
(freshman success)
Alumnisatisfaction
module
Major fieldlocal/senior
capstonemodule
Department ‘A’ Profile andDashboard
College ‘A’Profile andDashboard
Department ‘etc.’ Profile andDashboard
College ‘etc.’Profile andDashboard
Provost’sProfile andDashboard
Service unit ‘A’ Profile and Dashboard
Service unit ‘B’Profile and Dashboard
Service unit ‘C’Profile and Dashboard
Service unit ‘etc.’Profile and Dashboard
President’sDashboard
Provost’s Dashboard Metrics Architecture
July 28, 2002
Comparativedata for targets
Studentsatisfaction
module
Current Status
Satisfaction 1.160 highest 1.160Students 1.074 overall 1.074 Link to
0.756 lowest 0.756 run chart
Item Perf WgtRatio
Calculation annualFr. Noel-Levitz Academic Advising 0.350 1.8% 1.160 1.160 show me!Fr. Noel-Levitz Safety and Security 1.360 1.4% 0.756 0.756 show me!Fr. N-L Instructional Effectiveness 0.640 5.0% 1.044 1.044 show me!Fr. Noel-Levitz Student Centeredness 0.490 2.2% 1.104 1.104 show me!Fr. Noel-Levitz Campus Climate 0.530 6.1% 1.088 1.088 show me!Fr. Noel-Levitz Concern for Individual 0.580 2.2% 1.068 1.068 show me!Fr. N-L Registration Effectiveness 0.700 1.8% 1.020 1.020 show me!Fr. N-L Campus Support Services 0.420 2.5% 1.132 1.132 show me!Fr. Noel-Levitz Recruitment/Fin. Aid 0.690 2.2% 1.024 1.024 show me!Fr. Noel-Levitz Service Excellence 0.590 2.9% 1.064 1.064 show me!Fr. Noel-Levitz Campus Life 0.600 5.4% 1.060 1.060 show me!
06/21/2006 46
Presentation Overview Facilitating decision-making and
driving improvement Measurement impact on focus of
institutional activities and processes
How we developed our system and integrated it into information systems
Praxis andC-BASEresultsmodule
MajorField test
resultsmodule
AcademicProfileresultsmodule
Undergradmajors andminors,
Graduate majors,
Degrees, Advisees module
Studentopinionnairesof teaching
module
GeneralEducation
localmodule
ProgramSCH
generationmodule
Financialdata
module
Placementdata
module
EMSASmodule
(freshman success)
Alumnisatisfaction
module
Major fieldlocal/senior
capstonemodule
Department ‘A’ Profile andDashboard
College ‘A’Profile andDashboard
Department ‘etc.’ Profile andDashboard
College ‘etc.’Profile andDashboard
Provost’sProfile andDashboard
Service unit ‘A’Profile and Dashboard
Service unit ‘B’Profile and Dashboard
Service unit ‘C’Profile and Dashboard
Service unit ‘etc.’Profile and Dashboard
President’sDashboard
Provost’s Dashboard Metrics Architecture
July 28, 2002
Comparativedata for targets
Studentsatisfaction
modules
Alignment of Dashboards and Profiles
President’s Dashboard
Provost’s Dashboard
College Profiles (3) Department Profiles (11)
Department Profiles (5)
Department Profiles (5) VP Student Affairs Dashboard Unit Profiles (8)
Unit Profiles (8)
VP Information Systems Dashboard
VP Finance and Support Services Dashboard
VP Communication and Marketing Dashboard
VP Institutional Advancement Dashboard
Director of Athletics Dashboard
Exec. Assistant to the President Dashboard
Unit Profiles (6)
Unit Profiles (6)
Unit Profiles (3)
Unit Profiles (3)
Unit Profiles (4)
Primary Source Modules
Phase 1:STRATEGICCONTEXT
Phase 2:STRATEGY
IDENTIFICATION& REFINEMENT
Phase 3:7-STEP PLANNING
PROCESS DEVELOPMENT &
DEPLOYMENT
Phase 4:BALDRIGE
SELF-ASSESSMENT
MISSION/VALUES
DECISIONDRIVERS
ENVIRON-MENTAL
SCAN
INTERNALANALYSIS
STRATEGICKQIs
TACTICAL KQIs
GOALS ANDOBJECTIVES
ASSESSMENT
ACTION PLAN
RESULTS
PROCESSMANAGEMENT
FACULTY/STAFF FOCUS
PLANNING
STUDENTFOCUS
INFORMATION& ANALYSIS
LEADERSHIP
NORTHWEST QUALITY SYSTEMS MODEL
TRENDS
IMPROVEMENTS
STRETCH GOALS
STRATEGICINITIATIVES
CHAMPIONS
HRPLANS
MEASURE-MENT PLANS
DEPLOYMENTPLANVISION
COMMUNI-CATIONS
PLAN
06/21/2006 51
Northwest Dashboard Information and Sharing Days
Contact information: Dave Oehler Northwest Missouri State University 800 University Drive 660.562.1527 (voice) [email protected]