problem mgmt 12-11-09

7/30/2019 Problem Mgmt 12-11-09

1/14

OAAISPROBLEM MANAGEMENT

PROCESS

VERSION 2.1, REV. 12/11/09

University of CaliforniaSan Francisco

Office of Academic and AdministrativeInformation Systems (OAAIS)

7/30/2019 Problem Mgmt 12-11-09

2/14

UCSF Internal Use Only 2 of 14 Rev 12/11/09Problem Mgmt 12-11-09.doc

UC FSOffice of Academic and Admin istrativeInformation Systems (OAAIS) Problem Management Process

TABLE OF CONTENTSTABLE OF CONTENTS ........................................................................................... 2

DOCUMENT VERSION CONTROL ............................... .................................... ........ 3

STAKEHOLDER TEAM ........................................................................................... 3

1. INTRODUCTION .................................... ....................................... ................ 4

1.1. PURPOSE .................................................................................................................................................................... 4 1.2. SCOPE ......................................................................................................................................................................... 4 1.3. DEFINITIONS ............................................................................................................................................................. 4

2. RESPONSIBILITIES ................................. ..................................... ............... 4

3. PROCESS DEFINITION ..................................... ....................................... ..... 5

3.1. PROCESS MAP ........................................................................................................................................................... 5

3.2. A CTIVITY D IAGRAMS .................................................................................................................................................. 6

4. RACI CHART ................................................................................................ 8

5. ENTRY CRITERIA .................................... ................................... .................. 9

6. PROCEDURE ................................................................................................ 9

7. EXIT CRITERIA ..................................... ..................................... ................ 13

8. PROBLEM MANAGEMENT SERVICE LEVELS ..................... ........................... 13

9. PROBLEM MANAGEMENT SERVICE METRICS .................................... .......... 14

7/30/2019 Problem Mgmt 12-11-09

3/14



DOCUMENT VERSION CONTROL

Document Name OAAIS Problem Management Process

Process Owner Mimi Sosa

Version Number

Issue Date Prepared By Reason for Change

1.0 5/11/07 Terrie Coleman First draft

2.0 7/09/09 Francine Sneddon Updated to reflect Remedy 7 changes.

2.1 12/11/09 Francine Sneddon Updated Stakeholder Team

STAKEHOLDER TEAM

Department Name

Customer Support Services (CSS) Rebecca Nguyen

Enterprise Information Security (EIS) Michael Kamerick (Interim)

Enterprise Network Services (ENS) Jeff Fritz

Information Technology Services (ITS) Heidi Schmidt

Academic Research Systems (ARS) Michael Kamerick

Application Services (AS) Jane Wong

Business & Resource Management (BRM) Shahla Raissi

This document contains confidential, proprietary information intended for internal use only and is not to be distributed outside the University of California, San Francisco (UCSF) without an appropriate non-disclosure agreement in force. Itscontents may be changed at any time and create neither obligations on UCSFs part nor rights in any third person.

7/30/2019 Problem Mgmt 12-11-09

4/14



1. INTRODUCTION1.1. PURPOSEThe objective of Problem Management is to resolve the underlying root cause of incidents and

consequently prevent them from recurring. Reactive Problem Management aims to identify theroot cause of past incidents and presents proposals for improvement or rectification. ProactiveProblem Management aims to prevent incidents from recurring by identifying weaknesses in theinfrastructure and making proposals to eliminate them.

1.2. SCOPEThe scope of the Problem Management process includes a standard set of processes, procedures,responsibilities and metrics that are utilized by all OAAIS services, applications, systems andnetwork support teams.

1.3. DEFINITIONSA p r o b l e m describes an undesirable situation, indicating the unknown root cause of one or more

existing or potential incidents. A k n o w n e r r o r is a problem for which the root is known and forwhich a temporary workaround has been identified. A R e q u e s t f o r Ch a n g e ( R FC) proposes achange to eliminate a known error and is addressed by the Change Management process.

The Problem Management process includes Problem Control, Error Control, Proactive ProblemManagement and Providing Management Reporting.

2. RESPONSIBILITIESProblem Management Team

Reactive Problem Management

Identifies and records problems by analyzing incident details Approves problem resolution recommendations and establishes resolution priority Generates Requests for Change (RFC)

Proactive Problem Management

Identifies trends and records problems Approves problem resolution recommendations and establishes resolution priority

Problem Owner Investigates and manages problems based on their priority Assigns (or obtains) resources and manages error control activities Schedules and facilitates major problem reviews Develops recommendations for problem resolution Monitors the progress of known errors Generates Requests for Change (RFC)

Problem Manager Coordinates and guides activities of the Problem Management Team and Problem Owner(s) Provides management information and uses it proactively to prevent the occurrence of

incidents and problems in both production and development environments Escalates the analysis and resolution of cross-functional problems to Unit and OAAIS levels Conducts post mortem or Post-Implementation Reviews (PIR) for continuous improvement Develops and improves Problem Control and Error Control procedures

7/30/2019 Problem Mgmt 12-11-09

5/14

UCSF Internal Use Only 5 of 14Problem Mgmt 12-11-09.doc

UC FSOffice of Academic and Admin istrativeInformation Systems (OAAIS)

3. PROCESS DEFINITION3.1. PROCESS MAP

7/30/2019 Problem Mgmt 12-11-09

6/14



3.2. ACTIVITY DIAGRAMS

OAAIS Problem Management Process v1.0

P r o

b l e m

M a n a g e m e n

t T e a m

P r o

b l e m

M a n a g e r

P r o

b l e m

O w n e r ( s

)

5Review

Problem(s)

6 AssignProblemOwner(s)

START

3Identify

Problem(s)

7 AcceptProblem

Assignment

4Match

Incidents& Create

Problem(s)

2Identify HighOccurrenceof Similar

Incidents

1CategorizeIncidents

Conduct Problem Control

12Document

PossibleSolution(s)

9Investigate

& DiagnoseProblem

10Determine

Root Cause

11Document

the KnownError

8 AssessProblem

Conduct Error Cont

PreSolRec

mend

No

FromStep 29

FromSteps23/24

7/30/2019 Problem Mgmt 12-11-09

7/14



7/30/2019 Problem Mgmt 12-11-09

8/14


UC FS Office of Academic and Admin istrativeInformation Systems Problem Management Process

4. RACI CHART

Step #Problem Management

Step Description P r o b

l e m M a n

a g e m e n

t T e a

m

P r o b

l e m M a n

a g e r

P r o b

l e m O w n

e r

OutputConduct Problem Control

1 Categorize Incidents R A2 Identify High Occurrence of Similar Incidents R A3 Identify Problems R A4 Match Incidents & Create Problems R A5 Review Problems R A6 Assign Problem Owners R A7 Accept Problem Assignment A R

8 Assess Problem A/R

9 Investigate & Diagnose Problem A/R10 Determine Root Cause A/R11 Document the Error A/R12 Document Possible Solutions A/R

13 Present Solution Recommendation I I A/R

14 Approve? R A/R15 Fix? R A/R16 Determine Priority R A/R17 Schedule R A/R18 Monitor the Error I A/R19 RFC Required? A/R20 Create Change Request A/R

21 Implement Solution A/R22 Monitor Resolution A/R23 Error Resolved? A/R24 Permanent Solution? A/R25 Update & Close Problem A/R26 Conduct Review A/R


27

Co ect, Review & Ana yze Data - inci ent, pro em,known errors, industry, performance management, security, and user data A/R

28

Identify Infrastructure Issues - weak, overloaded,

vulnerable components A/R29 Create Problem & Review Problem A/R

Provide Reporting30 Generate Updated Problem Report A/R31 Distribute Problem Reports, as required A/R

Conduct Error Control


Responsible People who do the work, facilitate it and/or organize it

Accountable The one who ensures that desired outcomes are reached and has yes/no decision making authority

Consulted People who have critical expertise to contribute before a decision is made

Informed People who are significantly affected by the activity/decision and must be informed to ensure successful implementation

Perform Change Management Process - Hand-off

7/30/2019 Problem Mgmt 12-11-09

9/14



5. ENTRY CRITERIA Incident details, including workarounds and RFCs Configuration details (from Configuration Management Database future) Product information including technical details and known errors Details about infrastructure behavior, capacity, performance and service levels

6. PROCEDURE

ID Step Responsibility


1 Categorize Incidents

Team receives an Incident Report from Remedy showing open and closed incidents.Incidents will be categorized by category, type and item. The team reviews the reportand assigns a problem category to each incident, such as:

Application Functional Geography

Database Type Version

Operating System Hardware Network

Problem Management Team

2 Identify High Occurrence of Similar Incidents

Using best judgment and experience, review the categorized incidents looking forsimilarities and/or high occurrence of the same incident.


3 Identify Problem(s)

Identify problem(s) based on high occurrence, critical issues, trends, threatenedservice levels and / or incidents not linked to an existing problem or known error.


4 Match Incidents & Create Problems

Link new incidents to a problem, by entering the incident # into the problem worklog.If an incident cannot be linked to an existing problem the team will create a newproblem in Remedy with a link to the incident. The following fields are required:

Assignment Group Summary Description Requester Create Date Assign To Planned End Date Status Case Type = Problem


5 Review Problem(s)

A Problem Report is generated by Remedy, reviewed and the team:

Consolidates similar problems Creates new problems, if needed Prioritizes problems Defines investigation scope Establishes target resolution dates


7/30/2019 Problem Mgmt 12-11-09

10/14

7/30/2019 Problem Mgmt 12-11-09

11/14




12 Document Possible Solution(s)

Analyze, compare and evaluate alternatives including permanent solutions, temporary

solutions or not to fix. Possible resources include: Vendor troubleshoot guides and documents Internet searches Vendor trouble tickets Identified workarounds and fixes User group recommendations

Draft the following documents based on the severity and type of problem:

Cost benefit analysis Risk analysis Impact analysis Criteria for success

Problem Owner(s)

13 Present Solution Recommendation

Problem Owner(s) presents recommendation(s) for solving the problem to the team.The following items may be included:

Cost benefit analysis Risk analysis Impact analysis Criteria for success

Problem Owner(s)

14 Approve?

Evaluate and approve the recommendation(s) using the following criteria:

Objective analysis of problem Strategic direction Severity/visibility of problem Cost benefit analysis

Yes go to Step 15 / No go to Step 12


15 Fix?

Yes go to Step 16/ No go to Step 25


16 Determine Priority

Set the priority based on:

Impact on the business Urgency of the problem Risk assessment Resource constraints


17 Schedule?

Estimate level of effort and schedule the work to be completed or determine that thework cannot be scheduled at this time but should be monitored.



7/30/2019 Problem Mgmt 12-11-09

12/14




18 Monitor the Error

Monitor the problem using the following tools:

Log files Scripts Performance tools

Alert the Problem Manager if the problem recurs or there is a change in the impact of the problem.

Go to Step 16

Problem Owner(s)

Problem Manager

19 RFC Required?

Yes go to step 20 / No go to step 21

Problem Owner(s)

20 Create Request for Change

Go to Perform Change Management Process

Problem Owner(s)

Perform Change Management Process Hand-off

21 Implement Solution Problem Owner(s)

22 Monitor Resolution

Monitor the resolution after implementation to make sure that problem does not recur.Validate that there is a reduction in occurrences and severity and that other problemsdo not occur as a result of the fix.

Problem Owner(s)

23 Error Resolved?


Problem Owner(s)

24 Permanent Solution?Yes go to Step 25 / No go to Step 8

Problem Owner(s)

25 Update and Close Problem

Update and close Problem, Known Error and associated Incident records in Remedy.

Communicate problem resolutions and known errors to support team incidentmanagers.

Problem Owner(s)

26 Conduct Review

Conduct Post-Implementation Review (PIR) to understand what was done well, whatwas done badly, how to do better next time, and how to prevent recurrence of thefailure.

Problem Manager

7/30/2019 Problem Mgmt 12-11-09

13/14




27 Collect, Review and Analyze Data

Possible data sources include:

Incident data Problem data Known errors Industry data Performance data System management data Security data User data

Analyze data for similarities, repeat occurrences and trends. Review performance andindustry data looking for potential problems and best practices solutions.


28 Identify IT Enterprise Issues: People, Process, Technology

Identify specific components in the enterprise that are causing problems such as:

Failing or outdated equipment Insufficient CPU, memory or storage Poorly written code Incorrect configuration Inadequate user training


29 Create and Review Problem

Create a Problem Report in Remedy.

Go to Step 6


Provide Management Reporting

30 Generate Updated Problem Report

Provide reports on open problems, resolved problem, and know errors.

Provide reports on problem management service levels and service metrics.

Problem Manager

31 Distribute Problem Reports, as required

Distribute reports on open problems, resolved problem, and know errors to the servicedesk and incident managers.

Distribute reports on problem management service levels and service metrics tomanagement.

Problem Manager

7. EXIT CRITERIA Known Error database updated Request for Change (RFC), if required Problem records updated with known errors, solutions and / or workarounds Closed problem records once root cause is eliminated Management information

8. PROBLEM MANAGEMENT SERVICE LEVELS An analysis of incidents and identification of problems will occur at least monthly Reporting of problem management activity and performance metrics will occur at least

monthly

7/30/2019 Problem Mgmt 12-11-09

14/14


9. PROBLEM MANAGEMENT SERVICE METRICSProblem management service metrics will be collected and calculated for each support group, unit,and for OAAIS overall.

Category Description Measure Target

Number of problems

Measures problem management workloadand effectiveness in resolving openproblems.

Number open and resolvedproblems.

Establish baseline of problemmanagement volume.Monitor trends over time.

Average time toclose a problem

Measures effectiveness of the problemmanagement process in resolvingproblems. Reductions indicate betterprocesses, tools and training are beingused. Increases indicate an ineffectiveprocess or that problem managementmay be under-resourced.

Number of days fromproblem creation to problemclose for problems closed inthe current period.

Establish problem closebaseline. Monitor trends overtime.

Number of incidentsresolved byKnown Errors

Indicator of the time and re-workeliminated by managing and resolvingproblems. An increase in the number of resolved incidents should improve servicelevels and customer satisfaction.

Number of incidents that areclosed by solutionsregistered in the KnownErrors database.

Establish incidents withKnown Errors baseline andmonitor trends over time.

problem mgmt 12-11-09

Documents