case study: it infrastructure benchmarking

- ITPMG Confidential -

ITPMG

© Copyright 2010 ITPMG, LLC

Case study: IT

Infrastructure

Benchmarking

Charlie Ambuhl

Michael Bitterman

Founders - ITPMG, LLC

ITPMG. All rights reserved.

Copyright 2010 - Confidential2

ITPMG © Copyright 2010 ITPMG, LLC

Overview

What are the challenges facing today’s CIO / CTO in understanding IT

infrastructure costs and efficiency? Are IT costs appropriate when

compared to peer organizations performing the same type of work? Are

costs appropriate to industry peers relative to efficiency?

IT benchmarking is a set of models and methodologies to provide

comparative measurements relative to peer organizations. Meaningful

metrics on hardware, software, infrastructure and personnel are generated

and reported. In-Depth metrics covering cost, asset, and personnel

comparisons and efficiency are examined as well, providing an overall view

of IT’s structure, costs and efficiency.

This presentation will illustrate the results of two enterprise level IT

infrastructure benchmarks. 1) A very large IT organization trying to

determine exactly where costs are excessive and 2) an organization wanting

to determine if their outsourced IT costs are within market norms.




Case Study 1: Insurance Company

• Situation:

– Major North American Insurance Company

– Wanting to cut costs and improve IT efficiency

– Large Portfolio of Legacy Software Applications

– CIO knows that he is ―leaking‖ costs but not sure where.




Case Study 1: Engagement

Objectives/Scope of Work

•The benchmarking Client wanted to understand how the efficiency of its IT

operations compares with other Insurance companies, and to identify

potential opportunities to reduce current IT costs and/or improve current

levels of IT efficiency

•The Life Insurance and Property & Casualty organizations were

benchmarked independently to understand their specific performance

compared to comparable industry companies.

•An enterprise level benchmark of the entire IT operation was also

conducted, incorporating the all the Client’s business and their associated IT

organizations.




Case Study 1: IT Functional Areas

Benchmarked

In-H

ou

se S

taff

ing

co

sts

So

ftw

are

Co

sts

Hard

ware

Co

sts

Dis

aste

r R

eco

very

Co

sts

Tra

nsm

issio

n C

osts

Facilit

ies C

osts

So

urc

ing

Co

sts

Co

mm

on

Metr

ics

Distributed Computing

Help Desk

Midrange Server

Wide Area Data

Mainframe

Wireline

$ / User

$ / Handled Contact

$ / Server

$ / Connected Device

$ / MIPS

Wireless

PBX

$ / Minute

$ / Wireless Device

$ / Extension

Application Development

Application Support

$ / Function Point

$ / Function Point

Life Company

In-H

ou

se S

taff

ing

co

sts

So

ftw

are

Co

sts

Hard

ware

Co

sts

Dis

aste

r R

eco

very

Co

sts

Tra

ns

mis

sio

n C

os

ts

Facilit

ies C

osts

So

urc

ing

Co

sts

Co

mm

on

Metr

ics


Help Desk

Midrange Server

Wide Area Data

Mainframe

Wireline

$ / User

$ / Handled Contact

$ / Server


$ / MIPS

Wireless

PBX

$ / Minute

$ / Wireless Device

$ / Extension


Application Support

$ / Function Point

$ / Function Point

Property & Casualty Company

In-H

ou

se S

taff

ing

co

sts

So

ftw

are

Co

sts

Hard

ware

Co

sts

Dis

aste

r R

eco

very

Co

sts

Tra

nsm

issio

n C

osts

Facilit

ies C

osts

So

urc

ing

Co

sts

Co

mm

on

Metr

ics


Help Desk

Midrange Server

Wide Area Data

Mainframe

Wireline

$ / User

$ / Handled Contact

$ / Server


$ / MIPS

Wireless

PBX

$ / Minute

$ / Wireless Device

$ / Extension


Application Support

$ / Function Point

$ / Function Point

Enterprise View




Help Desk

Midrange Server

Wide Area Data

Distributed Computing Desktops, laptops, local area networking, peripherals, infrastructure equipment and the staff that support

IT help desk staff, hardware, software, and network

All communications devices connected to the network that generate and /or terminate traffic across the

backbone, associated data transmission costs and the staff that support

All centrally managed server infrastructure including applications, enterprise application systems, database

management servers, SAN/NAS, Internet servers and the staff that support

MainframeThe hardware and software costs of all centrally managed mainframe computers and support infrastructure and

the staff that support


All development tools, staff and programming efforts that meet either of the following two criteria:

All new code developed within 12 months of this assessment for a new application

Enhancements and bug fixes to existing applications requiring more that 80 staffing hours

Application Support

Wireless

Wireline

PBX

All development tools, staff and programming efforts that meet any of the following three criteria:

Any bug fixes of any type

Maintenance of hard coded data or tables within a software application

Any minor functional enhancements that are too small to be tracked as new development

All hardware, software, network and staff costs for the internal phone system including desk phones, extensions

and voicemail

All wireless voice network, equipment and usage plans and the infrastructure and staff that support

All voice network and local and long distance charges, minutes used and the internal support dedicated to

providing this service to the enterprise

Case Study 1: Descriptions of

IT Functional Areas




Case Study 1: Project Timeline

• In many cases, costs and staffing were allocated to Life and Property &

Casualty organizations based on corporate charge back information.

• IT management and finance costs not directly related to a module were

allocated based on proportional usage

• Cost and staffing information for each module was assembled using Client’s

IT Finance and HR information

Analyze Clients' existing data

to prepare for data collection

requests

Assist Client in collecting and

validating IT system and

personnel metrics

Generate and analyze

benchmark reports with

automated tool

Step 1:

Planning

Step 2:

Collect and Validate Data

Step 3:

Generate and Analyze

Benchmarks

Step 4:

Develop

Recommendations

Utilize benchmark data and

root cause analysis to identify

improvement opportunities

Days 1-5 Days 6-31 Days 32-42 Days 43-53




Case Study 1: Benchmark Results

Key Results:

Application Development and Application

Support account for 64% of the Clients Total IT

costs

Application Development is 19% less efficient

than Peer companies

Application Support is 127% less efficient than

Peer companies




• Total IT costs are 28% higher than Peer companies and 14% higher than Insurance companies in the database

• Total staffing costs (full-time employees, contractors, and outsourced staff/consultants) are 52% higher than both Peer and Insurance companies

• Full-time and contractor staffing costs are 72% and 55% higher than bottom quartile Peer and Insurance companies

• Average salaries are 6% and 17% higher than Peer and Insurance companies; full time staffing counts are 92% and 64% higher

• Contractor average rates are 19% and 9% lower than Peer and Insurance companies; Contractor counts are 51% and 27% higher

• Outsourcing costs are 66% and 62% lower than Peer and Insurance companies

Case Study 1: Other Key Findings




Case Study 1: Benchmark Variances & Drivers

Sourcing and Compensation Client use of strategic sourcing is 67% and 44% less than Peer and Insurance companies

Client average salaries are 6% and 17% higher than Peer and Insurance companies

Client IT organization has 294% and 227% more managers than Peer and Insurance companies (11%

of total full time staff are managers)

• Manager costs are 435% and 337% higher than Peer and Insurance companies

Client IT organization has 742% and 675% more people focused on planning and process activities

than Peer and Insurance companies (19% of full time staff)

• Costs are approximately 10X higher than Peer and Insurance companies

Application Development and

Support

Client Application Development productivity is 19% and 25% lower than Peer and Insurance companies

Application Support is 127% and 130% lower than Peer and Insurance companies

Client application portfolio is complex, aging and in need of rationalization

IT Operating

Model

IT

Organization

IT Process

Efficiency

Business-IT

Alignment

Client IT discretionary spend is managed at the business unit level

• High likelihood of redundant expenditures / efforts

• High likelihood of sub-optimal prioritization of initiatives

High-level IT Benchmark Conclusions




Case Study 1: Potential IT Cost Reduction

Approaches

Application Development and

Support

Set aggressive project delivery

targets: Timeline, Budget

Sourcing and Compensation Outsource “non-strategic”

Application Dev / Support

IT Operating

Model

IT

OrganizationReduce management layers

IT Process

Efficiency

Streamline IT process, simplify

policies and procedures

Business-IT

Alignment

Implement comprehensive IT

project portfolio management

Rationalize application portfolio

Move portion of Application

Support off/near-shore

Implement “key skill” resourcing

strategy

Optimize Help Desk’s adoption of

best practices

Eliminate redundant or “marginal”

projects

Optimize SDLC

Reduce contractor headcount

Reduce non-core and low

performing staff

Improve demand / capacity

management and planning

IT Cost Reduction Approaches




Case Study 1: Potential Savings Based on

Best Practice Cost Reductions by IT Tower

Category IT Cost Reduction Approach Range Source

IT Organization Reduce management layers 10 – 20% Mgmt. staff costs

Implement “key skill” resourcing strategy 5 – 10% AD direct staff costs

Reduce non-core and low performing staff 5 – 15% Non-Mgmt. staff costs

IT Processes Streamline IT processes 7 – 12% Plan and process staff costs

Optimize Help Desk adoption of best practices 5 – 7 % Help Desk agent staff costs

Business-IT Alignment

Implement comprehensive IT project portfolio management 10 – 15% AD total costs

Eliminate redundant or “marginal” projects 2 - 5% AD total costs

Improve demand / capacity management 5 – 10% AD and AS Mgmt. staff costs

Application Development /

Support

Set aggressive project delivery targets 5 – 10% AD direct staff costs

Rationalize application portfolio 10 – 20% AS direct staff costs

Optimize SDLC 10 – 40% AD and AS direct staff costs

Data, Network, Infrastructure

Optimize printer utilization 10 – 20% Printer costs

Optimize data network utilization 40 – 60% WAD transmission costs

Sourcing and Compensation

Outsource non-strategic application development / support 5 – 10% AD and AS direct staff costs

Move portion of Application Support off/near-shore 10 – 30% AS direct staff costs

Reduce contractor headcount 20 – 40% Contractor costs




Case Study 2: Large Logistics/Services

Company

• Situation:

– Major North American Logistics and Delivery Company

– Wanting to validate that outsourced IT costs are competitively priced.

– Satisfied with services provided by outsourcer.

– CIO believes that organization/staffing of outsourcer is currently

inefficient and therefore more expensive than necessary.




Case Study 2: Engagement

Objectives/Scope of Work

•The benchmarking Client wanted to understand how the efficiency and cost

of its IT operations compared to ―word-class‖ IT organizations from an

efficiency and cost perspective.

•The Client organization retains ownership of most hardware and software

assets and outsources nearly all labor.

•Non critical infrastructure services such as storage, network bandwidth,

messaging, file print services, SAP are outsourced.

•All peer and industry costs must be adjusted and normalized to reflect the

workload and asset ownership mix of this unique outsourcing relationship.




Case Study 2: IT Functional Areas

Benchmarked

In-H

ou

se S

taff

ing

co

sts

So

ftw

are

Co

sts

Hard

ware

Co

sts

Dis

aste

r R

eco

very

Co

sts

Tra

nsm

issio

n C

osts

Facilit

ies C

osts

So

urc

ing

Co

sts

Co

mm

on

Metr

ics


Help Desk

Midrange Server

Wide Area Data

Mainframe

Wireline

$ / User

$ / Handled Contact

$ / Server


$ / MIPS

Wireless

PBX

$ / Minute

$ / Wireless Device

$ / Extension


Application Support

$ / Function Point

$ / Function Point

Enterprise View




Case Study 2: Project Timeline

Benchmark Study Approach

Plan/Baseline Benchmark Analysis Analyze Gaps

30 Days 15 Days 10 Days

Define

Performance

Measures

Perform

Benchmark

Analysis

Identify

Performance

Gaps

Collect /

Validate

Data

Determine objectives

Define scope

Identify peer group data

sources

Identify performance

measures

Collect baseline data

Baseline capability and

management process

best practices

Verify / validate data

Perform adjustments and

normalization

Select peer group for

efficiency / effectiveness

dimensions

Make peer group

adjustments

Quantify magnitude of

cost / benefit of the

opportunities

Identify and explain gaps

Develop working

hypothesis for roadmap




Case Study 2: Peer Selection and

Normalization

Variance

+/- Prime

Workload Drivers

Weighted

Complexity Score 1–100

Weighted

Complexity Score 1–100

User counts

Employee counts

Site counts

Server counts

Total MIPS

Network port counts

~100–320

Peers

User type

User skill level

Count of images

supported

Count of technology

platforms in production

MACS

HW/SW diversity

~40–100

Peers

End user dispersion

Multiple support

languages (2)

Hours of operation (24X7)

Ageing

Application portfolio mix

(e.g., Client server, web

based)

Support SLAs

Efficiency Peer Selection Approach (by Service Tower)

~1200

Peers12–17

Peers

Filter 2:

Complexity

(1 of 2)

Filter 3:

Complexity

(2 of 2)

Filter 1:

Workload Entire

Database

Adjustments

+ / -

HW/SW costs

Service level costs

Transition costs / financing

Cost of Savings Paid in

Advance (financial

engineering)

Contract terms and

conditions costs

Management costs

+ / - Adjustments to reflect

Client Unique Costs

Scale Adjusted

Data Set

Complexity Adjusted

Data Set

Final Reference

Peer Group




Case Study 2: In aggregate, there is no gap compared to the

TQ Peer Group — However, there are gaps in the mainframe

and the server towers

Client Efficiency Metric vs. Peer TQ

% Delta

Client Efficiency vs. Peer TQ

Distributed

PBX

WAD

-16%

-31%

-7%

Peer TQ

Client

HD-USD

HD-ASD

-56%

-65%

Wireline-79%

Unix LG(1)

Unix SM

Windows MD

Windows SM+61%

+131%

+75%

+60%

Mainframe+81%

Further Analysis

Warranted in

Mainframe and

Servers




Case Study 2: Apparent Reasons for

Gaps in Mainframe and Servers

• The midrange gap can be partially explained by:

– Very high storage requirements

– A large SAP implementation

– Job scheduling workload associated with SAP changes —significant and above benchmark level

– Another possibility (not evaluated):

– much of the activity associated with the SAP workload is not automated

• The mainframe gap can be partially explained by very low volume, as

well as the legacy systems required by the Client




Case Study 2: Key Findings

•Mainframe - lack of scale is a key driver behind mainframe cost per

MIPs compared to the TQ Peer Group

•Unix - SAP driven workload and high storage charges should be the

starting point for additional root cause analysis

•Windows -gaps may be caused by a combination of application

workload and a lack of automation




Case Study 2: Conclusion

• In aggregate, there is no gap in the costs to Client charged by the

outsourcer for a comparable level of performance as seen in Top Quartile peer

companies.

• Amortizing transition and savings costs paid in advance into the fees over the

contract term is an outdated practice — nevertheless, these must be applied to

enable like-to-like benchmark comparisons

• The study has identified a scope for improvement to World-Class performance

levels and the basis of action to underpin such plans:

• Increase leverage of Outsourcer’s existing, internal capabilities

• Revise the Client – Outsourcer operating model

• If future benchmarks are to be conducted, the parties should consider a 'true-

up payment' to remove embedded legacy costs in fees to minimize perceived

cost differences (e.g., transition costs, savings paid in advance)

• Additionally, future benchmarks should include the active participation of all

parties – the Client and outsourcer



ITPMG22

To receive additional materials and details on

Case Studies: Benchmarking IT Infrastructure,

contact ITPMG at:

by email:

[email protected]

or phone:

+1.843.377.8228

© Copyright 2010 ITPMG, LLC

case study: it infrastructure benchmarking

Documents