the looming complexity catastrophe in large (and ultra ... · customer repair tracking i35 early...

© Carliss Y. Baldwin and Kim B. Clark, 2005

The Looming ComplexityCatastrophe in Large (and Ultra-Large) Systems

Carliss Y. BaldwinHarvard Business School

August 15, 2005


IT system of a large retailer, 1999Boxes=Systems; Lines=Standards

E01-EDI

Data Warehouse(Interfaces to and from the

Data Warehouse are notdisplayed on this diagram)

DRAFT Best Buy - Application Diagram V4 DRAFTNovember 10, 1999

G02 - General

Ledger

A05 - AP

S01 - SalesCorrections

I01 PO

Receiving

I03 Return to

Vendor

I06 Warehouse

Management

M a inframe apps - Blue

PC/NT apps - G reen

Unix apps - Yellow3rd party interface - Orange

Lines: Colors have no special meaning.

They are to help make the diagram easier toread.For More Information: See the database

containing information about eachapplication: Application V4.mdb

S06 - Credit App

P15 EES EmployeeChange Notice

OTHER APPS - PC

AP - Collections/Credit

TM - Credit Card DB

ACCTS REC APPS - PC

990COR

Bad Debt

Benefical Fees

Beneficial Reconcil

JEAXF

JEBFA

JEBKA

JEDVA

JESOA

JEVSA

JEVSF

NSF

TeleCredit Fees

INVENTORY CONTROL APPS - PC

Code Alarm

Debit Receivings

Devo Sales

Display Inventory

In Home

Junkouts

Merchandise Withdrawl

Promo Credits

RTV Accrual

Shrink

AP Research - Inv Cntrl

AP Research-Addl Rpts

Book to Perpetual Inventory

Close Out Reporting

Computer Intelligence Data

Count Corrections

Cross Ref for VCB Dnlds

Damage Write Off

Debit Receivings

DFI Vendor Database

Display Inventory Reconcil

Display Inventory Reporting

INVENTORY CONTROL APPS - PC

DPI/CPI

IC Batching

Inventory Adj/Count Correct

Inventory Control Reports

Inventory Levels

Inventory Roll

Merchandise Withdrawl

Open Receivings

PI Count Results

PI Time Results from Inv

Price Protection

Sales Flash Reporting

Shrink Reporting

SKU Gross Margin

SKU Shrink Level Detail

USM

VCB Downloads

Journal Entry Tool Kit

Scorecard - HR

L02-ResourceScheduling

(Campbell)

P09 - P17Cyborg

M02 - Millennium

M03 - Millennuim 3.0

Banks - ACH and Pos toPay

Cobra

B01 - StockStatus

S03-Polling

P14 On-line NewHire Entry

CTS

Plan Administrators(401K, PCS, Life,

Unicare, Solomon

Smith Barney)

D01 Post LoadBilling

I04 HomeDeliveries

I02 -Transfers

Arthur Planning

I07 Purchase

Order

I12 Entertainment

Software

I05Inventory Info

E13E3 Interface

S04 - Sales Posting

V01-Price ManagementSystem

I10 Cycle PhysicalInventory

I55 SKUInformation

K02Customer Repair

TrackingI35 Early Warning

System

B02 MerchandiseAnalysis

I13- Auto

Replenishment

U18 - CTO

Intercept

I09 Cycle Counts

E02-EmployeePurchase

Texlon 3.5

ACH

Stock Options

I17 Customer PerceivedIn-Stock

U16-Texlon

SiteSeer

C02 - CapitalProjects

F06 - FixedAssets

US Bank Recon

File

Star Repair

EDICoordinator

Mesa Data

NEW Soundscan

NPD Group

AIG Warranty Guard

Resumix

Optika

Store Budget

Reporting

P16 - Tally Sheet

Cash Receipts/Credit

S05 - HouseCharges

Ad Expense

L01-PromoAnalysis

V02-PriceMarketing

Support

BMP - Busperformance Mngt

StoreScorecard

I11 PriceTesting

Valley Media

P09

Bonus/HR

I15 Hand ScanApps

Roadshow

POS

S08 - VertexSales

Tax

A04 - Cust

Refund Chks

Equifax

ICMS Credit

CellularRollover

S09 - Digital

SatelliteSystem

NPD,SoundScan

Sterling VANMailbox (Value)

I 18SKU Rep

X92-X96Host to AS400

Communication

S02 -Layaways

Washington,

RGIS,Ntl Bus Systems

V04-SignSystem

I14 Count CorrectionsNARM

P01-

EmployeeMasterfile

I06 - CustomerOrder

Frick

Co

UAR - Universal AccountReconcilliation

Depository

Banks

S07 - Cell

Phones

S11 - ISP

Tracking

AAS

Fringe PO

Cash Over/Short

L60 MDF

CoopSKU Selection

Tool

SKU

Performance

SupplierCompliance

1

I35 - CEI

ASIS

Misc Accounting/Finance Apps - PC/NT

COBA (Corp office Budget Assistant)PCBS(Profit Center Budget System)

Merchandising Budget

AIMS

Merch Mngr ApprovalBatch ForcastingAd Measurement

AIMS Admin

AIMSReportingAd

Launcher

V03- MktReactions

SpecSource

CTO2.Bestbuy.com

RebateTransfer

SignSystem

CopyWriter'sWorkspace

ELTPowerSuite

StoreMonitor

AIS Calendar

Stores & Mrkts

Due Dates

Smart Plus

Insertions

Orders

BudgetAnalysis Tool

Print CostingInvoice App

AIS Reports

Broadcast

Filter

Smart PlusLauncher

General

Maintenance

Printer PO

PrinterMaintenance

VendorMaintenance

Vendor Setup

Connect 3

Connect 3Reports

Connect 3

PDF Transfe

Spec SourceSKU Tracking

S20-SalesPolling

Prodigy

PSP

In-Home

Repair

WarrantyBilling

System

Process Servers(Imaging)

Prepared by Michelle Mills

Page 1 of 2


Another appalling hairball from banking—3-year plan to do merger integration

Universal VRU-ContactCtr/Einstein E2

Desktop

#3: NC/SC Deposits,WIA/CAP, CD's, IRA,

OD Protect. . .

Comml Credit CardsConv'n FDR to MBNA

Fixed Assets GEACFAE Series

SignageChange (finish)

Key dates which will cross Lines of Business. Lines indicate point in time.

Size of milestone indicates complexity of event - small, medium or large

Key dates - specifically conversion dates, which will cross Lines of Business. Lines indicate a point in time.

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

CorporateRisk

Commercial Credit & RiskProducts (CA Tools, Viper, MFAand Polar)

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

E-Commerce

Internet Merge tonew Wachovia.com

Internet online billpay -FUNB CheckFree toMetavante

Internet Online Banking& Bill Payment

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

Executive

Accounts Payable toOracle Financials

Systems DataMapping

LawPacCase Mngmt

TN - Close

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

Finance

Purchasing -CLARUS to OSCAR

GeneralBank

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

Divestiture DeconversionDep & Loans

ATM Pilot

SBA Loan Conv'n to Strategy/McCracken (AS400)

WB Mortg Loans via Mortg Mngr

Retail Loan Coll'n and Recovery

All Mortg LoanOrig'ns Close to

Alltel/CPI Credit Card Conv'n:First USA to MBNA

Retail Consmr Loans:ACLS Conv'n 4.x to 5.0

ATM-Base 24ATM MachineConv'n-HWMigration

Commercial Res EstateConstruct'n Loans CLCS PEL's-Home

Equity Linesof Credit

ATM Card andCheckCard

Retail ConsumerLoans: AHInterface to ACLS

Retail ConsumerLoans ExistingFTU Port'o

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

HR/Training

HR-Payroll/BenefitsConversion

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

Operations

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

Item Processing(Crossover Logic)

InformationTechnology

EBSA/DRCore Doc Mngmt System

(Image File Folder)

Commercial Loans(CLS to AFS, TES)

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

TreasuryServices

Reg#1 Unit test

#1: Simulation

#1: RegionalConversion Live

Reg #1-4: DepositEliminate Duplicate AcctNums for all States

NGN Stress Test: Virginia

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

WealthManagement

Kinnexus Aggreg'nSoftware

Personal TrustSEI to NCS

Trust PortfolioOnline via Internet

Insurance ADS toPivot Life/Health

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

MergerProjectOffice

CapitalManagement

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

Key Accounts toCap/Brokerage

BrokerageComplianceand Finance

Institutional Custody of4,500 accounts fromNCS (FTU) to SEI (FTU)

InstitutionalInvestments

OMPM

EB/401KCustody &

Transf. AgentConv'n: WB to

Evergreen StBk

M.F. WBFund toEvergreenSys.

WIA's to CAP (foreach region)

Int'l DOC toEXIMBILLS

Int'l LOC toEXIMBILLS

Leveraged Fin. Asset-Based Lending Conv'n

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

Corporate &Investment

Balance Sheet MngmtPortfolio

Fixed Income Salesand Trading

ForeignExchange Conv'nBSM Custodian Conv'n

REFS CPConduit Conv'n

Int'l GFI &Payments Investm't Banking Loan

Syndication Conversion

Int'l: ConsolidateLondon Branches Leveraged Fin.

Leasing Conv'n

International BrandingSafekeeping Conv'n

2/1/02Jan 1, 2002 Dec 31, 2003

3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03

RegionalDeposit

Conversions

Legal NameChange

MergerConsummationof Bank Charters

FirstNet/ETNVideo Sat. Dish

RolloutNorth Region PennDel

and Atlantic ExternalSignage Change

T&EExpenseReporting

Compas/OFSA Profitability

EIM - Data Warehousecombine WB & FTU

Network StressTest Florida

Network StressTest Georgia

NGN Network

EIM - Sold/ProConversion (lead

generation)

Network StressTest N/S Carolina

Treasury Svcs FileTransmissions w/Existing Customers

Data CenterConsolidation CIC-Silas

InterofficeMail CodeConversion

OCCIF - CISCustomer Conv'n ACH (Prior to first

regional dep conv'n)

Wire Transfer - MTS& IPX to Intranet

ARP

Lockbox - WholesaleWILIS to DMP

ACH - PEP + 5.0 to 5.0.3DB2 Major Release

Denovo Branch Opens: Stuart, FL

Interim Mortg Servc'gto Raleigh

Dealer Financial Services(DFS) Interface Review

PCs in FinancialCtrs - FL (545)

Common Prod. Conv'n

PCs in Financial Ctrs- GA (255)

New Retail Consmr LoansOriginat'n via AH

Teller Argo for Win. - FL

#1: ReadinessReview

#1:Go/No-GoDecision

Teller Argo for Win. - GA

PCs in Fin Ctrs -N/S Carolina (442)

PCs in Fin Ctrs -VA (70)

Teller Argo forWin. - NC/SC

Teller Argo for Win (foreach region - Legacy

FTU must be 90 daysprior)

#4: VA Deposits, WIA/CAP, CD's, IRA, ODProtect. . .

Platform (witheach regional)

Treasury ServcsCustomer SoftwareRollout

Account Analysis (witheach regional conv'n)

Corp Phone Single Directory withnew Org. (follows PeopleSoft)

OFFITBANK TrustSEI to Sungard/NCS

Year-end 2001ProcessingMoratorium

GAP/EnhancementsFinal Approval

Year-end 2002ProcessingMoratorium

Year-end 2003 - Allconversions complete

2002-2003 Conversion Timeline(based on 05/14/02 Project Plan)

SignageChange (Start)

Today

Charitable TrustSIEBEL ContactManagment

CommercialInsurance Agencies

Contact Center -Columbia Call

Center

Updated: 05/20/02

EIM - Base Warehouse InfoIntegration-Pre Legacy

Conversions/Joint Householding -Grouping all Relationships

EIM - Legacy/Source ApplicationConversions - Data Warehouse,

Marts, COMPAS

EIM - Consumer Risk Management InfoIntegration - Pre Legacy Conversions

CIS/OCCIF and Relationship(with each regional dep conv'n)

Teller Argo for Win. - VA

Platform - Argo Salesand Service (IncludesGUI for SOLD)

SOLD-Prospecting

Early AnnuitySales Rollout

#2: GA Deposits, WIA/CAP, CD's, IRA, OD

Protect. . .

DUC - Statement PrintMove: WEC to DUC

Wire Tr. Upgrade VAX toUNIX on Intranet Wires

Region #1-#4Deposit: Account

Number Generationfor all States

SOLD-Sales Tracking

Divestiture-Man'l ConvWIA to CAP

SOLD-Book ofBusiness

Community Banking-First Contact

Re-Issue LWB FL-ATMCards

Trust SEI Data Merge

Day 2 ExceptionProcessing-Vector 5to GEMS

1/2004 12/2004

Data Center Move/Consolidations

ImageDEO/PODPlatformConv'n

Business Requirements

File Retention ConversionLockdown

EnhancementLockdown

IntegratedTest

Integ.Testday2 proc.

Integ.Testday 3

Mock Conversion

Day 1 Mock

Start IntegrationTesting

Data CenterMove- Go/No-go

Data CenterMove Simulation

Integ.Testday1 proc.

Emerald-new version

Credit Card Conv'n:EDS to MBNA/TSIS

Data Center Move:Baymeadows toSilas

Lockbox-DallasWILIS to DMP

New WachoviaBrand/LogoAnnouncement

AcctsPayable toOracle Finan

SBLOriginat'n

CommonProductConv

RIGSelf-

DirectedIRA

Image Archive -SDC - Jacx toSilas

Image ArchiveRetrieval Mgr

Lockbox-Charlotte WILISto DMP

Lockbox-AtlantaWILIS to DMP

CommunityBanking-Firstcontact

Internet eMail Address Conversion

Internet Email Conversion(@firstunion.com to @wachovia.com

EIM - Consumer Risk Management InfoIntegration - Pre Legacy Conversions

Credit Card -Merchant


Goals for an Ultra-Large System

Complete interoperability of components(mix-and-match)

Seamless upgradability Unlimited extensibility

No First or Second Level Hairballs!Internet is good at the First Level but has

Second Level Problems (e.g. Windows)


To avoid complexity catastrophesin Ultra-Large Systems we need:

Tools/techniques to map actualinterdependencies in large, changingcodebases

Formal measures of design option value exante and ex post

“If you can’t measure it, you can’t manageit.”


Complexity = Interdependency There are ex ante architectectural representation

languages (UML)– Like architectural drawings

But very few code maps– Maps are ex post, updatable, show what’s really there– Like plumber’s/electrician’s diagrams– Example: Rusovan, Lawford, Parnas (2005) critique of

Linux was based on ONE sourcefile! Architecture of functionally similar codebases can

be very different


Mozilla Before Redesign Mozilla After Redesign

Browsers


Word ProcessorsAbiword Open Office Word


SpreadsheetsGnumeric Open Office Calc


These architectural differencesaffect design option value Design options have “technical potential”,

denoted σ Technical potential, σ, varies

– By system– By module– Over time


Global Design Rules v.1

Version 1.0Version 1.2

Version 1.5Version 1.8

σ = Low Medium Zero High

Evidence of Option Value Successive, improving versions are evidence of

option values being realized over time—after thefact

Designers “see” option values before the fact


Measuring Option Value Measure unpredicted residuals from

performance data Example: TPC-C is a benchmark for

transactions processors– 153 submissions from 1/1/01 - 1/25/05– Use regression to cull out predictable

performance:

TpmC = a0 + a1(time) + a2(CPUs) + a3(FrEnds) + ε


Measuring Option Value (cont)TpmC = a0 + a1(time) + a2(CPUs) + a3(FrEnds) + εStdDev(ε) is a proxy for σ—

02.0e

-06

4.0e

-06

6.0e

-06

Dens

ity

-500000 0 500000 1000000e

σ= ± 124814

σ= ± 69% ofMean TpmC


To avoid complexity catastrophesin ULSs

We need to track the evolution ofarchitectures in real systems as they grow.

For this we need good maps of real systemsat the level of code,

And better ways to measure option value exante and ex post.

the looming complexity catastrophe in large (and ultra ... · customer repair tracking i35 early...

Documents