REQADM

Organizing the Chaos:

Managing RequestTickets in a Large

Environment

Steve WilloughbyIntel Corporation

REQADM

Introduction

• Every support group needs to deal with tracking of customer requests.– Different size groups have different needs.– If you have the resources, building your

own may be an ideal solution• Requires significant time and effort to design

and implement

– Or, you may prefer to purchase a system• Also requires significant time and effort to

configure, customize and manage

REQADM

Introduction

• But it’s not just about software.– How you design your system support

infrastructure is just as important.– Perhaps more important.– Unexpected advantage to building your

own:• Must thoroughly examine your work model

• Opportunity to build closer relationship with customers

REQADM

Introduction

• This is the story of our adventures…… creating tracking software

… creating a support infrastructure

… for a growing group of 50–1500.

• There’s more to the story, though...– REQADM BoF Thursday at 8:00– Paper in Conference Proceedings– Download REQADM and play with it

REQADM

50

Small Groups

1991• May be single administrator.

• Need: keep system administrator sane and organized.

• Requests often informal– “Hey, Steve! Got a second?”– E-mail directly to system administrator– Little or no tracking (status reports)

REQADM

50

Small Groups

1991• Problems:

– Accountability– Problem Analysis– Missed/Forgotten Commitments

• Solution:– I got to learn Perl :)– Single flat-file “database” of issues– Reports to management– Really helped prioritize and track issues

REQADM

501991 50

1001992

Small Groups• Same basic game, with more

players– Customers mail favorite sysadmin

directly.– Ask for help informally in hallway.

• Simple solutions– Mail alias to contact support team– Still informal handling of issues by

administrators

REQADM

501991 50

1001992 100

4001993

Larger Groups• More specialization

• More formal tracking needed– Requests “owned” by admins.– Customer feedback throughout

process and at resolution time.– Management reports of

“delinquent” tickets, follow up with owners.

REQADM

50

100

400

1991

1992

1993

Larger Groups

Customer Disp.

Steve Kathy Tim

Mailed request

Assignment

REQADM

50

100

400

1991

1992

1993

Larger Groups

Customer Disp.

Steve Kathy Tim

1-on-1 interaction,feedback on progress

REQADM

501991 50

1001992 100

4001993 400

8001996

Growing...• Growing pains (software)

– Perl scripts breaking, corrupting tickets

– Sysadmin mistakes: re-packing the request queue

– Difficult to get indicators to track computing issues proactively

– Web/GUI interface needed: can’t send mail in every situation

– Not scaling well

REQADM

501991 50

1001992 100

4001993 400

8001996

Growing...

• Growing pains (human)– Customer relations nightmare

• Miscommunicated urgency

• Tickets closed too soon

• Need more personal attention

– Frustrated system administrators• Overloaded, overwhelmed sysadmins

• Working hard but not always feeling appreciated

• Call volume increasing

REQADM

501991 50

1001992 100

4001993 400

8001996

Solutions...• Request Team formed

– Key customers identified to provide input

– Key support admins and managers– Identified customer requirements– Specified new help desk processes– Worked out specification for new

software tools to implement help desk changes

REQADM

501991 50

1001992 100

4001993 400

8001996

Solutions...• New procedures and policies

– Service Level Agreement• “You want what? When?”

– Computing Support Center (CSC)• Central place to go for help

• Phone and walk-in support

• Everyone close enough to cooperate

• Full-time “front line” staff

• Next tier of more experienced admins serving week-long CSC shift

REQADM

501991 50

1001992 100

4001993 400

8001996

Solutions...• New software tools

– Better “importance” rating system• Urgent, high, medium, low

• Specific “requested by” date

– Rotations• Automatic, weighted assignments

within group of owners

– Easier process• Mail, X, web, batch, CLI, NT (soon)

clients

REQADM

501991 50

1001992 100

4001993 400

8001996

Solutions...• New software tools

– Root-cause analysis (RAT)• Proactively fix problems (stop

requests from coming in to us)

– Bi-directional E-mail tracking• Make everything as easy to use as

possible

– Personal preference profiles– Customer approval of resolution

REQADM

501991 50

1001992 100

4001993 400

8001996 800

1500+1999

Huge Groups• Three years later…

– Customer groups almost doubled again, call volume heavy, more tickets…

– No massive changes to software or CSC; both scaling well to larger demands.

• Looking at supporting more distant sites– Ticket passing between sites

REQADM

501991 50

1001992 100

4001993 400

8001996 800

1500+1999

Huge Groups• Drive incoming tickets down

– More environment measurement– Better generation of request

metrics, analysis for trends

• What could we do better?– Tie-in to knowledge bases– Replicated backup servers– Better cross-referencing of tickets

REQADM

• Come to the BoF session– Thursday, 8–10pm, Douglas Room– Live demo of REQADM software– Discussion of issues involved in

creating request tracking systems.

• Look at REQADM yourself:– ftp://ftp.intel.com/pub/reqadm/

• Contact me:– steve@ichips.intel.com

501991 50

1001992 100

4001993 400

8001996 800

1500+1999

More...