request tracker project review oliver thomas if you find yourself in a hole, the first thing to do...

Request Tracker Project Review


Oliver Thomas

If you find yourself in a hole,the first thing to do is stop

digging.

- Will Rogers

Wednesday, October 6th, 2004 Slide 2


Agenda

10m The colorful history of ticket tracking at MIT10m Status quo: Casetracker today30m Project overview

- Discovery and recommendations- Architecture review- Development and pilot

60m Issues and possible mitigations

10m Next actions



Context Setting: Levels

• The Request Tracker project• The future of Casetracker / RT• A central ticket tracking service in IS&T• A central ticket tracking service at MIT

– is this a business we want to be in (we’re in it now, but not “formally”)

– aside from the specific implementation how do we want to structure this?

– what revenue stream does it tie into, if any?



History of Ticket Tracking at MIT



Status Quo: Casetracker Today

• Usage inside of IS&T– Tightly linked queues for Help Desk teams and 3rd tier ops

teams– Some of the high-volume queues

• Help Desk• Network Security• Volume-Licensed Software• DITR

– Some of the time-critical queues• Network Security• Network Operations and Network repair• Administrative Server Services Team



Casetracker Today

• Usage outside of IS&T– Used by many departments for IT problem tracking; some of

the more prominent:• MIT Libraries IT Support• Student Services IT• Alumni Network Services• Economics IT Support

– Used by several groups for tracking non-IT issues:• Resource Development• Environmental Health & Safety• VIP Card support• MIT Press Journal Subscriptions• Tech Cash support



Casetracker Today

0

50000

100000

150000

200000

250000

1996 1997 1998 1999 2000 2001 2002 2003 2004

Year

Total Cases Across All Categories

Canceled Cases

Accepted Cases

Some system scale metrics: 245 active work queues across ~100 unique teams 847 active consultants / analysts 336737 e-mail messages received over last year



Casetracker Today

Standing Tooltime Team, mostly made up of “volunteers”, deals with:

– Modifications and on-going feature tweaks– Administration tasks (queue creation, consultant setup,

tech support, on-demand training)– Service operation (except the machine and database;

mostly Steve Turner and Oliver Thomas)– Handouts from Server Operations and Database

Services to maintain production hardware, OS, and Oracle Database

– Community engagement and outreach• CT-Lead council and list• Consulting services



Project Overview



The Discovery Project

• As customer base diversity grows, “one size fits all” increasingly becomes a problem

• High profile projects such as the Help Desk Benchmarking Project identify functionality gaps

• Eight key functionality areas identified in which Casetracker is currently lacking

1. Authorizations2. Time Tracking3. Referrals and Escalations4. Meta data / tagging5. Reporting6. Client information and History7. Template and Document integration8. Archiving



Discovery Roadmap



Discovery Project Highlights

• To determine the best way to proceed a variety of possibilities were looked at:

– in-house web-only replacement for Casetracker– commercial solutions – RT identified as a potential replacement for Casetracker

• Reminder: even though some end-of-year money was identified for this project, the on-going activity is still somewhat on the margin

• Folks were excited about an open source system in general use at the same time as Remedy was locking up the commercial market and raising their prices across the board

• There has always been a desire to deploy a tool that supports the way people naturally work, supporting diverse and flexible work methods, while providing an infrastructure facilitating hand-offs and knowledge sharing.



Discovery Project Highlights

• Coalesced around the recommendation to go with Request Tracker

– strictly web-based system that incorporates most of the functionality of the desktop client (risk for the Help Desk)

– brings a rich API, vendor-independence, and a vibrant developer community to the table

– examples of large instances of a scale similar to or greater than what we would need

– Additionally encouraged by commitment from ISST to run central RT instance (as well as smaller instances for others)

• ITAG review 10/29/2003 to talk about possible issues:– MySQL (decision: acceptable DB but option to swap out for

Oracle; ties to operations group)– Roles (decision: postpone but consider)– ITAG okay to move forward with implementation and pilot



Delivery Roadmap



Delivery Timeline Highlights

• Internal developer (Steve Turner) begins looking at and learning RT’s API and code base (September)

• Discovery council okay’s Discovery recommendations• Contract negotiations across MIT IPO, Procurement,

and Best Practical begin in October, conclude in late January

• Steve Turner begins development work on in-house modifications (November)

• Statements of work for initial set of modifications are signed and Best Practical begins work (early February)



Delivery Timeline Highlights

• Usability and feature testing with Casetracker users begins (3/22/2004)

• Operations discussion with with staff from OIS and CSS

– Results in purchase and deployment of high-end production server and test server

– Discussion around who in OIS will maintain what• Inviting pool of pilot testers• Migration scripts tested and working• Current pool of pilot testers consists of some migrants

(ANS, SSIT) and some new adopters (StopIt, AC)



Delivery Timeline Highlight

• Decision to postpone further migration to incorporate more pilot feedback and deploy major RT upgrade (on which new features are based) and appropriately resourcing the operation of the service

• Attempts to get upgrades and new releases deployed to pilot/production server begin June 30th/July 1; unable to schedule release/deployment dates due to resource constraints

• Development, training, documentation and some testing continue, but we are effectively stalled on deploying upgrades and fixes to the pilot and test servers, as well as adding definition to the service operations space



Project Budget

Developer at 40% time $XXXXXX

Production server (SunFire v480) $XXXXXX

Back-up software for MySQL $XXXXXX

Third-party enhancements to RT

Re-architect custom field system $XXXXXX

Extend client information display / search $XXXXXX

Extend reporting functionality $XXXXXX

Add topic hierarchy to knowledge base $XXXXXX

Branding and UI enhancements to KB $XXXXXX

Simple search interface to KB $XXXXXX

Logging / import enhancements to KB $XXXXXX



Issues and Possible Mitigations



Issues: Small, Medium, and Large

• Issue: additional development and documentation tasks identified during pilot

– Have been able to complete a fair number of these due to deployment delays on production and test servers

– Only a few tasks are critical to migrations • Issue: migration and upgrade to RT version 3.3 -

affects timeline• Switching from an existing, working system to a new

one and being able to run the two in parallel gives us flexibility in timeline and migration schedules

• Risk: some queues need to migrate together• Risk: some queues have black-out windows for

migration



Issues: Small, Medium, and Large

• Operations group that agreed to run the service is not currently resourced to do so at the required scale

– Have been unable to deploy fixes and upgrades to production and test servers since 7/1

– No resources to commitment to time table / schedule• No explicit service agreement for service operation

– Priorities– Service expectations– Escalation paths– No release model

• No real business model around a central ticket tracking service



Potential Issues

• Performance– Web-based system vs. local client application

• Of concern to the Help Desk call center• Unable to test in call center pilot because we’ve not been able to

deploy new feature sets (upgrades) to pilot and test servers– System performance under full load

• Database and tuning expertise currently exists in OIS for Oracle• Unclear how deep our MySQL knowledge is• Mitigations: purchased larger than needed server; consultant

option

• Fewer emergency “throttles”• E-mail currently enters Casetracker through a wrapper script• Non-local messages are replied to with a 5-minute delay (loop

mitigation)• Mitigations: more input into production server setup



Possible Mitigations

• We can decide we cannot run RT at the scale required to support all Casetracker customers

– We can migrate some of the pilot groups back to Casetracker and continue with it (for now?)

– Non-Casetracker groups on pilot server may be a small enough load that they can continue via current service

– Still leaves the question of Casetracker as an activity• We can figure out a new operational model (and

business model) for RT– Fee based? (Use richer feature set as springboard)– Offer freebies to pilot testers– Revise timeline– Beyond original project scope– Address any architecture concerns



Possible Mitigations

• Reset and look at a trouble ticket service / work management at a higher level

– Strategic direction– Ownership / scope– Business model– Executive support from C’s



Context Setting: Levels

• The Request Tracker project• The future of Casetracker / RT• A central ticket tracking service in IS&T• A central ticket tracking service at MIT

– is this a business we want to be in (we’re in it now, but not “formally”)

– aside from the specific implementation how do we want to structure this?

– what revenue stream does it tie into, if any?

request tracker project review oliver thomas if you find yourself in a hole, the first thing to do...

Documents

mit slide

project overview slide

year slide

rogers slide

discovery project

help desk benchmarking

time tracking

problem tracking