a computer scientist looks at the year 2000 problem

1JMH Associates © 1998, All rights reserved

A Computer Scientist Looks at the Year 2000 Problem

The Keys to High-Quality Year 2000 System Renovation

Johnson (John) M. [email protected]

JMH Associates

“Give us the tools, and we will finish the job.”Winston Churchill to President Roosevelt, 1941


PretendPretendPretendPretend You are the CIO at a large corporation You are responsible for the Year 2000

readiness of 100s of millions of lines of COBOL code Terabytes of data

The future of the company is in your hands! And - Pretend it’s 1997 or earlier Can computer science and software

engineering help you?


Y2K Renovation – Getting StartedY2K Renovation – Getting StartedY2K Renovation – Getting StartedY2K Renovation – Getting Started

You are Y2K aware Have a budget Performed inventory and impact analysis The project team is organized You’ve decided on windowing or expansion

Of course you’ll test the renovated systems Now what?

How will you renovate your code and data?


Renovation: The Neglected StepRenovation: The Neglected StepRenovation: The Neglected StepRenovation: The Neglected Step

How will you renovate your systems? Better renovation: fewer errors to find in test Shift resources from testing to renovation Large systems require automation benefits

Two key steps Date-Sensitive Item Identification System Correction

Caveat: Not entirely automated Some manual intervention


Common Statements Demonstrate Common Statements Demonstrate Renovation NeglectRenovation Neglect

Common Statements Demonstrate Common Statements Demonstrate Renovation NeglectRenovation Neglect

“It will be good enough” “There’s no ROI in better renovation” “We’ll fix the mistakes in testing” “Not enough testing?

Decrease the renovation budget!” “We’ll test all the paths” “We only need to test what has changed”


Caution!Caution!Caution!Caution!

Testing misses serious & common errors “Change-based testing” doesn’t test what

should have been changed Good ID & correction: fewer errors to detect You can’t test quality in Do it right the first time

You’ll save time and money! More on this later


Date-Sensitive Item IdentificationDate-Sensitive Item IdentificationDate-Sensitive Item IdentificationDate-Sensitive Item Identification

Key requirement for quality Y2K renovation What are Date-Sensitive Items?

Variables, constants File & screen fields Anything containing year/century data

Required for ALL renovation strategies You can’t fix what you can’t find Faulty ID leads to defective systems


DSI ID Investment Pay-OffsDSI ID Investment Pay-OffsDSI ID Investment Pay-OffsDSI ID Investment Pay-Offs

Decreased testing time and expense Decreased risk to operation Decreased system repair time System documentation with long-term value


…

Faulty DSI ID Example: Original CodeFaulty DSI ID Example: Original CodeFaulty DSI ID Example: Original CodeFaulty DSI ID Example: Original Code

DATA DIVISION.WORKING-STORAGE-SECTION01 TODAYS-DATE PIC 9(6). Correctly identified - yymmdd01 PAYMENT-YEAR-TO-DATE PIC 9(6). False positive -

yymmdd01 PAYMENT-DUE PIC 9(6). Correctly identified - yymmdd01 NEW-PAYMENT PIC 9(6). False negative - is yymmdd01 MAX-YEAR-TO-DATE PIC 9(6) VALUE 100000.

False positive - yymmdd. . .

PROCEDURE DIVISION.IF TODAYS-DATE > PAYMENT-DUE

AND PAYMENT-YEAR-TO-DATE > MAX-YEAR-TO-DATETHEN

MOVE NEW-PAYMENT TO PAYMENT-DUE.


DATA DIVISION.WORKING-STORAGE SECTION01 TODAYS-DATE PIC 9(8). Correctly identified - yymmdd01 PAYMENT-YEAR-TO-DATE PIC 9(8). False positive - yymmdd01 PAYMENT-DUE PIC 9(8). Correctly identified - yymmdd01 NEW-PAYMENT PIC 9(6). False negative - is yymmdd01 MAX-YEAR-TO-DATE PIC 9(8) VALUE 19100000.

False positive - yymmdd. . .

PROCEDURE DIVISION.IF TODAYS-DATE > PAYMENT-DUE

AND PAYMENT-YEAR-TO-DATE > MAX-YEAR-TO-DATETHEN

MOVE NEW-PAYMENT TO PAYMENT-DUE.

Renovated (Expanded) CodeRenovated (Expanded) CodeRenovated (Expanded) CodeRenovated (Expanded) Code


Defective DSI ID Example: ResultsDefective DSI ID Example: ResultsDefective DSI ID Example: ResultsDefective DSI ID Example: Results

Defective code and corrupted data PAYMENT-YEAR-TO-DATE must exceed

$19M rather than just $100K PAYMENT-DUE appears to be a first

century date PAYMENT-YEAR-TO-DATE a file field?

Corrupted data! $500,000 becomes $19,500,000 or $20,500,000

Windowing logic correction is just as defective


Identification: MethodIdentification: MethodIdentification: MethodIdentification: Method

“Knowledge” propagation Start with some “truth” about the system

Initial knowledge seeds the propagation Propagate this knowledge through the code Get seed information from easy places

Files, reports, screens, pattern matching suggestions

Provide attribute information Format (YY, YYMMDD, YYDDD, etc.)

There are some hard cases


IF TODAYS-DATE > PAYMENT-DUE AND PAYMENT-YEAR-TO-DATE > MAX-YEAR-TO-DATE

THEN MOVE NEW-PAYMENT TO PAYMENT-DUE.

Knowledge PropagationKnowledge PropagationKnowledge PropagationKnowledge Propagation

SEED

Propagate knowledge by applying identification rules at each program statement


Propagating KnowledgePropagating KnowledgePropagating KnowledgePropagating Knowledge

Year 2000-sensitive I/O fields

Year 2000-sensitive variables and constants

Variables that are not Year 2000-sensitive

Linkage

Module C

Module B

File BFile A

Module A


DSI ID – Key PointsDSI ID – Key PointsDSI ID – Key PointsDSI ID – Key Points

DSI ID must be automated in large systems Pattern matching: inadequate

Can suggest seeds Cannot make decisions

The complete system must be examined DSI ID must also determine format

(YYMMDD, DDMMYY, YYDDD, ...)


Total Automation Not Possible Total Automation Not Possible Total Automation Not Possible Total Automation Not Possible

There are always some human decisions “No silver bullet”

Pattern matching has some value Can provide seeding & consistency checks Assessment Inventory Other preparatory phases But, no matter how refined, pattern (and

usage) matching can not decide the dates


These Techniques Work!These Techniques Work!These Techniques Work!These Techniques Work!

Tools from several vendors in use today Tools can be fast & are getting faster

Best techniques: based on compiler theory Hard work turns the theory into practice

Cost-effective, especially in large systems Why do anything else?

Human knowledge & decisions still necessary Some situations are inherently ambiguous


Quality DSI ID Can Prevent DefectsQuality DSI ID Can Prevent DefectsQuality DSI ID Can Prevent DefectsQuality DSI ID Can Prevent Defects

Before they get into the system! Faulty DSI ID causes costly defects

Corrupted data Incorrect business decisions

Defect detection through testing is expensive, risky, and will miss many defects We’ll see why later

Identification drives code and data correction


Automated Year/2000Automated Year/2000Correction is NecessaryCorrection is Necessary

Automated Year/2000Automated Year/2000Correction is NecessaryCorrection is Necessary

Correction must be accurate, repeatable Systems are large and complex Simplified testing and verification Intricate changes must be consistent, reliable Date identification drives correction


Example: Code Renovation Example: Code Renovation Windowing Logic CorrectionWindowing Logic CorrectionExample: Code Renovation Example: Code Renovation Windowing Logic CorrectionWindowing Logic Correction

(a/k/a Procedural Workaround) Procedure division logic changed

Interpret 2-digit dates in a century window Window range [ 1900+b and 2000+b–1 ] Example: [1950, 2049 ]

Existing files retain 2-digit year fields No changes to existing data files

Variations: Date Expansion, in-line, custom


Code fragment before correction

Logic Correction Example –Logic Correction Example –Windowing Logic CorrectionWindowing Logic CorrectionLogic Correction Example –Logic Correction Example –Windowing Logic CorrectionWindowing Logic Correction

IF DUE-DATE GREATER THAN START-DATE ANDDUE-DATE LESS THAN END-DATE ORSTART-DATE EQUAL FIRST-DATE

PERFORM SOMETHING.


Logic Correction Example –Logic Correction Example –Corrected CodeCorrected Code

Logic Correction Example –Logic Correction Example –Corrected CodeCorrected Code

CALL 'CVRT2TO4' USING DUE-DATE YYYYMMDD-1CALL 'CVRT2TO4' USING START-DATE YYYYMMDD-2CALL 'CVRT2TO4' USING DUE-DATE YYYYMMDD-3CALL 'CVRT2TO4' USING END-DATE YYYYMMDD-4IF YYYYMMDD-1 GREATER THAN YYYYMMDD-2 AND

YYYYMMDD-3 LESS THAN YYYYMMDD-4 ORSTART-DATE EQUAL FIRST-DATE

PERFORM SOMETHING.


ID & Correction: The Keys to QualityID & Correction: The Keys to QualityID & Correction: The Keys to QualityID & Correction: The Keys to Quality

Don’t rely on testing, no matter how thorough It’s expensive It will miss many defects It finds symptoms, not causes

Do it right the first time Sound software engineering Essential for “due diligence”


Testing Will MissTesting Will MissMost False NegativesMost False Negatives

Testing Will MissTesting Will MissMost False NegativesMost False Negatives

Despite extensive, full-coverage test cases Unknown dates will not be aged “Test what has changed”

Totally ignores false negatives The original code would pass the test!

Also misses many false positives, bad logic Testing finds symptoms, not root causes


The RealityThe RealityThe RealityThe Reality

Many renovations are poor quality Recent competitive comparison

6 vendors, 100 KLOC, 159 required changes Accuracy ratings:

89%, 75%, 75%, 70%, 60%, 56% 90% means 16 errors on 100 KLOC

1.6 per 10 KLOC


Answer: Independent Pre-Test AuditAnswer: Independent Pre-Test AuditAnswer: Independent Pre-Test AuditAnswer: Independent Pre-Test Audit

A high-quality tool reports: All detected false positives and negatives All detected bad fix insertions All missing fixes, faulty logic, mixes bases

Fix it now! Before test! You are fixing the root cause Testing only finds a symptom

Due diligence, IV&V


It Works! It’s Cheap!It Works! It’s Cheap!It Works! It’s Cheap!It Works! It’s Cheap!

Examples: 110 KLOC, tested. Found 33 errors 350 KLOC, tested. Found 200 errors 400 KLOC, tested. Found 70 errors 250 KLOC, carefully renovated.

Found one error High-quality renovation: high-quality result


A More Dramatic Case StudyA More Dramatic Case StudyA More Dramatic Case StudyA More Dramatic Case Study

Testing found 31 faulty DSI IDs 55 staff-days; 1.77 SD per defect found

Independent audit found 22 more faulty IDs 4 staff-days to find. 0.075 SD per defect

Nearly 25 : 1 cost ratio per defect detected Plus, additional defects were detected Detected root cause, not symptoms

Cost-to-fix ratio is even better No collateral damage from residual defects


BUDGET ASSUMPTIONS:

1,000,000 LINES OF CODE

$1,000,000 DOLLAR BUDGET

$1.00/LOC

40% PLANNING & ANALYSIS

50% TESTING & FIX

10% RENOVATION

Savings from Independent Savings from Independent Renovation InspectionRenovation Inspection

Savings from Independent Savings from Independent Renovation InspectionRenovation Inspection

Example: Original Budget


Reduced Budget with IRIReduced Budget with IRIReduced Budget with IRIReduced Budget with IRI

40% PLANNING & ANALYSIS

25% TESTING & FIX

10% RENOVATION

5% IRI

20% PROJECT SAVINGS

ASSUME: IRI FOR 1/2 THE COST OF RENOVATION

AN AVERAGE OF 1 ERROR PER 10,000 LINES OF CODE

ASSUME TESTING UNCOVERS ANOTHER 100 ERRORS


SummarySummarySummarySummary

Renovation is an exacting, repetitive task Accurate Date ID is essential Changes procedure, data divisions and files

Automated tools are fast and accurate Testing is necessary but not sufficient Independent audits are cost-effective Experience to date is excellent


More InformationMore InformationMore InformationMore Information

Date Sensitive Item IdentificationYear/2000 Journal, March/April, 1998http://www.y2kjournal.com (see “past issues”)

Automation Assisted Correction Year/2000 Journal, May/June, 1998

Testing Limitations and Auditinghttp://www.itpolicy.gsa.gov/mks/yr2000/y2kconf/paper59fp.htm

http://www.itpolicy.gsa.gov/mks/yr2000/y2kconf/paper60fp.htm

Year/2000 Journal, Sept/Oct, 1998

a computer scientist looks at the year 2000 problem

Documents