it risk management – a causal modeling approach for end ...€¦ · it risk management – a...

IT Risk Management –A Causal Modeling Approach forEnd-User Computing

Daniel J. Hinz

IWI Jour fixeFrankfurt/Main, February 14th, 2006

1

AGENDA

• Motivation and research history

• Theoretical foundation– IT infrastructure continues to be a hot topic for more than 15 years now– Surprisingly, research neglects the management of inherent infrastructure risks– Financial risk management theory suggests the use of causal modeling

approaches

• Study design– A single case study was used to create a design-oriented artifact– For desktop availability, experts identified and estimated a Bayesian Belief

Network (BBN)– The validation with historical loss data resulted in adaptions

• Key findings– Implications for decision making: user incidents are a sensitive figure– Findings can be efficiently communicated by a balanced scorecard

• Contributions, limitations, and further research

2

AGENDA



approaches





3

A PRACTITIONER'S PROBLEM …

Situation

• An international logistics company has an outsourced ITinfrastructure of end-user computers, servers, and network

• During contract renegotiation, end-user computers are hotly debated

– Major structural changes are the foundation to improve reliabilityand to reduce costs

– Corresponding tightened SLA figures and objectives werenegotiated

• Currently, transformation still in progress and SLA figures are notfully met

Who is responsible?Are fundamental assumptions of the

new contract correct?

4

?

… IS ADDRESSED BY MY RESEARCH QUEST(ION)

How can risks arisingfrom IT infrastructure beeffectively assessed andcommunicated?

Assessment

Main question

Sub questions

Communication

• How to identify IT risks in a structured way

• How to measure or predict the risk potential(e.g., downtime)

– In steady state (normal operation)

– In change scenarios (e.g., outsourcing,contract renegotiation)

• How to facilitate decision making bycommunicating IT risks and the potentialeffects of mitigitation measures effectively

• How to control risk mitigation strategiesthrough effective reporting

5

THESIS STRUCTURE AND PEER REVIEW PLAN

Introduction/Motivation

Application for IT managers• Communication and management

of IT risks with BSC

• Risk mitigation strategies

• Financial riskmanagementdomain

• IT manage-ment theory

• DSS

Causal modeling of IT risks• Development of classification

model for operational risk• Identification of key risk drivers

and dependencies• Modelling of Bayesian Belief

Network for PC desktop risks• Empirical validation (single case

study)

Outlook and further research

Thesis structure

Theoretical foundation

* Part of T-Systems agenda ** Best paper nomination

• HICSS-39: "Enhancing the Prognostic Power of IT BalancedScorecards with Bayesian Belief Networks" (with S. Blumenberg)**

• EFLQ 3/2005: "Management Communication of Complex RiskAssessment" (with S. Blumenberg)*

• ECIS 2006: "An Integrated Approach to Assess and Communicate ITRisks" (with Blumenberg, Weitzel)*

• IRMA 2006: "Mitigating Software Risk with Web Services"

Part ofthesisPeer review plan

• PACIS 2004: "A Framework for Classifying the Operational Risks ofOutsourcing" (with H. Gewald)

• WP: "IT risk assessment – methods and application" (withcluster 2: Pérez, Martinovic, Berbner, Steinmetz)

• IRMA 2006: "The Next Wave in IT Infrastructure Risk Management –A Causal Modeling Approach with BBNs" (with H. Gewald)

• (Journal): Empirical results

• HICSS-38: "High Severity IT Risks in Finance"• ECIS 2006: "Employing Bayesian Belief Networks for Measuring the

Operational Risk of Information Systems" (with H. Gewald)*

• GITMA 06: "IT Risks: Definition and Challenges" (part of roof paper)

• HICSS-39: "Assessing the Risks of IT Infrastructure – A PersonalNetwork Perspective" (with J. Malinowski)

�Accepted

�

�

�

�

02/06

( )� Submitted/completed

( )�

( )�

( )�

�

�

�

�

6

AGENDA



approaches





7

IT INFRASTRUCTURE CONTINUES TO BE A HOT TOPICFOR MORE THAN 15 YEARS NOW

1990 "IT infrastructure" appears forthe first time in the top 10issues identified by the survey

1995 "Building a responsiveinfrastructure" was rankednumber one challenge

2000

2001

"IT infrastructure management"ranked third in these twoinformal surveys

2003 "Infrastructure developments"ranked second in the categoryof top application andtechnology developments

• Survey to identify themost critical issues inIS management

• Conducted regularlyby the Society forInformationManagement (SIM),supported by the MISResearch Center(MISRC)

• Among theirmembers consistingof top executives aswell as ISresearchers

SIM survey

Source: [Niederman, Brancheau, Wetherbe 1991], Brancheau, Janz, Wetherbe 1996], [Luftmann, McLean 2004]

8

SURPRISINGLY, RESEARCH NEGLECTS THEMANAGEMENT OF INHERENT INFRASTRUCTURE RISKS

Main viewpointson IT infrastructure

• Flexibility• Standardization• Security• Web Services• SOA

IT architecture

• What are requirements andfeatures of a good ITinfrastructure?

• What is the enablingtechnology?

Main research questions Main answers

IT alignment

• Reflection of firm‘s strategicobjectives

• …

• How can IT and business bealigned?

• What is the economic impact?

IT infrastructureoutsourcing

• Risk assessment• SLA management

• What is a good managementinterface to the provider?

(Internal) ITmanagementprocesses

• Practical "handbooks" (ITIL)• Which internal supportprocesses are needed?

• How can IT (risks) internallybe managed?

Focus of my research

9

Measurement

FINANCIAL RISK MANAGEMENT THEORY SUGGESTSTHE USE OF CAUSAL MODELING APPROACHES

Identification Decision making ControlGeneric (risk)managementprocess

Classical sourceof information

• Expert judgment • Analysis ofhistorical lossdata

• Parallelconsideration ofprior analyses

Literature for operational riskmanagement suggests causalmodelling with Bayesian BeliefNetworks (BBN) [Alexander 2002]• To combine expert estimations

with loss data• To identify key risk indicators and

mitigation levers• To allow for upfront simulation

Balanced Scorecard(BSC) is a powerful andwell-established method tocommunicate causal de-pendencies to top manage-ment [Van der Zee, DeJong 1999]

Suggestionsfrom financialtheory

Main tasks • Systematicallyidentify mainsources of risk

• Estimateprobabilities andloss to quantifyrisk

• Decide onmitigationmeasures

• Monitor thesuccess ofmeasures andrisk change

• Monitoring ofkey figures

Assessment Communication

10

AGENDA



approaches





11

A SINGLE CASE STUDY WAS USED TO CREATE ADESIGN-ORIENTED ACTIFACT

Step 1: Model building andinitialization based on expertestimation

Step 2: Model adaption basedon historical data

Resulting model

• Two subject matter expertsfrom leading IT consultancies

– One with deep knowledge ofcompany's IT and contractintentions

– One with general knowledgeof IT infrastructure

• No prior input from logisticscompany

• Iterative approach according toEisenhardt and Yin

• Incident data from company'shelpdesk systems duringmigration phase

• Enriched with individualconfiguration data for eachsystem over time

• Key figures

– App. 30,000 computers– 4 months of observation

– Over 80,000 incidents

– App. 120,000 aggregateddata sets

Typical process forconstruction of Bayesian

Belief Networks

12

FOR DESKTOP AVAILABILITY, EXPERTS IDENTIFIEDAND ESTIMATED THE FOLLOWING BBN

Desktopoperational?

Serversavailable?

Softwareoperational?

Hardwareoperational?

Useroperational?

Hardwarecomplexity

Hardwarestandardized?

SW imagestandardized?

SW imagecomplexity

# of hardwareerrors

FieldserviceTTR

# of softwareerrors

Helpdesk TTR# user

incidents

Helpdesk load

Automationtool supportNetwork up?

Servers up?

Softwarematurity

User skill level

Data available

13

Helpdesk load

Automationtool supportNetwork up?

Servers up?

Softwarematurity

User skill level

Serversavailable?

THE VALIDATION WITH HISTORICAL LOSS DATARESULTED IN ADAPTIONS

Desktopoperational?

# of hardwareerrors

FieldserviceTTR

Hardwarecomplexity

Hardwarestandardized?

# of softwareerrors

Helpdesk TTR

SW imagestandardized?

SW imagecomplexity

# userincidents

Softwareoperational?

Hardwareoperational?

Useroperational?

? ��?� ?

• Edges connecting ostensiblyindependent nodes are kept in orderto reflect the experts‘ judgement

• New edges are added to incorporatenewly identified dependencies

Counter-intuitive findings Resulting adaptions

• Notebook computers are as reliable asdesktop computers

• Standardization has no effect on errorresolution times

• Software image standardization andcomplexity obviously also drivesnumber of user incidents

14

AGENDA



approaches





15

IMPLICATIONS FOR DECISION MAKING:USER INCIDENTS ARE A SENSITIVE FIGURE

52

30

61

12

172

8

% of total downtime

1 89

% of total incidents

Other*

Network*

Hardware

Software

User

Special attention has to be given to userincidents, as impact on desktop uptime isminor, but number of user incidents isenormous. Further analyses (e.g., bybenchmarking) may indicate, whetherfocus should be on reduction of incidentvolume instead of TTR

4,446Network*4,229Software

3,354Other*2,720Hardware

0,335User

Average TTR in days

Only 8% of total measured downtimeis due to user incidents and furtherreduction of TTR seems difficult …

… however, more than half of allincidents are user inquiries

* Not significant (e.g., network mass problems not included)

16

CLASSICAL APPROACHES FOCUS ON "OFFICIAL"INCIDENTS, BUT THEY ARE ONLY ONE PART

SituationClassical approaches measure mostly officialincidents (e.g., helpdesk calls) [Niessink and Van Vliet, 2000]

Complication• Not all incidents are reported but

instead solved by asking co-workers orknown experts

• How can those aspects be consideredto get a more realistic number ofincidents?

17

THE DENSITY OF THE SOCIAL NETWORK HAS ANINFLUENCE ON PROBLEM SOLVING

Based on expert interviews,two measures from SocialNetwork Analsis (SNA) werechosen to influence problemsolving

Socio-Centric Density (SCD)of the network of co-workers [Barnes 1974]

)1( −=

nn

lSCD

n

lECD =

Ego-Centric Density (ECD)of the helpdesk [Scott 2000]

18

SCD AND ECD CAN BE USED TO GET A MOREREALISTIC NUMBER OF USER INCIDENTS

+⋅=

ECD

SCDCHDUI α1

U3

U1

U2 HD0.9

0.8

0.8

0.6

1.0

0.8

Calculation of user incidents:

SCD = 0.68

ECD = 0.80

� UI = 185

UI: User incidentsCHD: Calls that reach the helpdeskα: Scaling factor

The ratio of both densities can be usedto predict the unkown number of totaluser incidents (UI) from the number ofknown incidents (CHD)

Example

19

THE INITIAL MODEL HAS TO BE EXTENDED TO REFLECTTHESE FINDINGS

Helpdesk qualityand social networkdensity nowdetermine the useraction upon anincident

20

BALANCED SCORECARDS CAN BE LAYERED UPON ABAYESIAN BELIEF NETWORK DUE TO SIMILARITIES

I

PC

F

Bayesian Belief NetworkF

PC

I

Balanced Scorecard

Balanced Scorecard (BSC)

Consists of entities (called figures),grouped within perspectives

Directed edges indicate causalrelationships

Loops are allowed, but should beomitted to be compatible with BBNs

Bayesian Belief Networks (BBN)

Consists of entities (called nodes), maybe grouped graphically

Directed edges describe causalrelationships and are used to calculateconditional probabilities

Loops are not allowed (graphs has to bedirected and acyclic)

21

FINDINGS CAN BE EFFICIENTLY COMMUNICATED BY ABALANCED SCORECARD

Desktop infrastructureavailability

Percentage ofdowntime

Fieldservice TTR

Avg. time per call

Helpdesk TTR

Avg. time per call

User incidents

Number of calls

Server availability

Percentage of downtime

SW standardization

Percentage ofstandardized systems

Tar

get

Inte

rnal

Bus

ines

sP

roce

sses

Cus

tom

er(U

ser)

Ext

erna

l

HW standardization

Percentage ofstandardized systems

SW image complexity

Percentage ofcomplex images

22

AGENDA



approaches





23

EX-ANTE SIMULATIONS ARE ONE OF THE MAJORSTRENGTHS OF THE APPROACH

Contributions to theory Contributions to practice

• Causal modelling techniques can beapplied to the assessment of ITinfrastructure risks

• Bayesian Belief Networks and BalancedScorecards can be combined to supporta seamless and fully integrated riskmanagement process

• Users seem to be a crucial pointconcerning risk, which are currentlyneglected by researchers andpractitioners

• Simulations of the causal model help toidentify most important risk mitigationlevers

• In change scenarios like outsourcingnegotiations, they may help to agreeon key figures in the SLA

• The process itself of building andtraining the model improves riskunderstanding

Limitations and further research

• Further research has to show, whether this approach is actually better than others whenapplied in real world scenarios

it risk management – a causal modeling approach for end ...€¦ · it risk management – a...

Documents