it metrics/dashboards at duke: curation , automation, aggregation?
DESCRIPTION
IT Metrics/Dashboards at Duke: Curation , Automation, Aggregation?. CSG January 11, 2012. Background. Initiated metrics effort in 2008 (1 FTE dedicated) Aligned with finance initially, now integrated with service management team - PowerPoint PPT PresentationTRANSCRIPT
IT Metrics/Dashboards at Duke:
Curation, Automation, Aggregation?
CSGJanuary 11, 2012
2
Background Initiated metrics effort in 2008 (1 FTE dedicated)
Aligned with finance initially, now integrated with service management team
Focused on availability, capacity, service usage/demand, internal resources
Consult with units on what metrics they should capture Help with collection and analysis of metrics Ensure consistent, universal reporting of data
3
Initial scope and progress Monthly reporting process for managers and
community Internal: Data collection, report development,
management review, publication of detailed report External: availability, usage, performance summaries
Limited range of services historically Email, network/voice, paging, HR, IT security,
telepresence
4
5
6
The challenges of our “curation” process Labor-intensive Monthly periodicity limits immediacy
Fine for long-term trending, but less immediate Reliance on local units to self-report Leveraging Duke’s post-incident review (PIR)
process, but remaining data still sui generis
Curation: work by content specialists immersed in a specialized discipline and imbued with analysis
7
Metrics and newsgathering: Classic curation processes?
8
Duke’s metrics curation challenges No one wants to read “yesterday’s news”
Hard to avoid with curation The stories are too long
New monthly exec-summary focused on trends has helped (like the WSJ’s “What’s News” box) to a point
Focus on putting out a daily paper takes away from the “longreads”
Time required to produce manual reports takes away time expected to be spent on consulting with units to help them be primary data-gatherers
9
Curation vs. automation
10
Automation: leveraging monitoring data
System/network monitoring produces thousands of data points a day – how can we use them?
Left: Daily low-level alert analysis (SPC methodology)
Not shown: Loss of redundancy reports
11
Automation: Leveraging monitoring data
12
Duke’s metrics automation challenges
Automation a great start, but plenty of “curation” still occurs!
Monitoring’s “service dashboard” useful, but not ready to be directly published
Monitoring events don’t/can’t catch everything Human adjustment still needed to raw data
Low-level alert reports are automated… and reviewed by daily operational staff
Ultimately, there’s infinite ways to leverage data, but what do you care about?
13
Curation, automation, aggregation
14
Duke’s aggregation questions What data mix should appear?
Curated data from monthly metrics, live report on high-priority user tickets, performance graphs, availability measures against targets on one screen
Broad enough to be watched; specific enough to be useful
What platform should we use? SharePoint? Improve collection, visualization ServiceNow? Leverage dashboards, APIs, data
structure; easy access to support tickets Javascript/JSON components for easily customized
dashboards? – node.js and d3.js