1

Risk Analysis

2Chapter 5

Assessing Risk

• The potential consequences of not assessing and managing risks can include the following:– Failure to attain expected benefits from the project,– Inaccurate project cost estimates,– Inaccurate project duration estimates,– Failure to achieve adequate system performance levels,

and– Failure to adequately integrate the new system with

existing hardware, software, or organizational procedures.

3Chapter 5

Project Risk Factors

• Project size– Team size, organizational departments, project duration,

programming effort• Project structure

– New vs. renovated system, resulting organizational changes, management commitment, user perceptions

• Development group– Familiarity with platform, software, development method, application

area, development of similar systems• User group

– Familiarity with IS development process, application area, use of similar systems

4Chapter 5

Assessing Risk(Cont.)

• Risk can be managed on a project by:– Changing the project plan to avoid risky factors.– Assigning project team members to carefully

manage the risky aspects.– Setting up monitoring methods to determine

whether or not potential risk is, in fact, materializing.

5Chapter 5

Assessing Risk(Cont.)

• The four primary factors associated with the amount of technical risk on a given project are:– Project size,– Project structure,– The development group’s experience with the application

and technology area, and– The user group’s experience with systems development

projects and the application area (see also Kirsch, 2000).

6Chapter 5

Assessing Risk(Cont.)

• Four general rules emerged as technical risk assessments:– Larger projects are riskier than smaller projects.– A system in which the requirements are easily

obtained and highly structured will be less risky than one in which requirements are messy, ill structured, ill defined, or subject to the judgment of an individual.

7Chapter 5

Assessing Risk (Cont.)

– The development of a system employing commonly used or standard technology will be less risky than one employing novel or nonstandard technology.

– A project is less risky when the user group is familiar with the systems development process and application area than if unfamiliar.

8Chapter 5

Assessing Risk(Cont.)

Effects of degree of project structure, project size, and familiarity withapplication area on project implementation risk (Source: Based on 7th Applegate, Austin, and McFarlan. 2007; Tech Republic, 2005.)

9

Risks and Their Implications in the Requirements Process

• Insufficient user involvement

• Creeping user requirements

• Ambiguous requirements• Gold-plating• Minimal requirements• Overlooking user classes• Incompletely defined

requirements

Unacceptable product

Overruns and degraded quality

Rework and wasted time

Unnecessary features

Missing features

Dissatisfied customers

All of the above

Controlling Risk

• Project Management• Application of knowledge, skills, tools, and

techniques to achieve targets within specified budget and time constraints

• Scope- Quality• Time• Cost• Risk

Kaiser Permanente Botches Its Kidney Transplant Center Project

• Read the case study and then discuss the following questions:

• Classify and describe the problems Kaiser faced in setting up the transplant center. What was the role of information systems and information management in these problems?

• What were the people, organization, and technology factors responsible for those problems?

• What steps would you have taken to increase the project’s chances for success?

• Were there any ethical problems created by this failed project? Explain your answer.

12

Quantifying Risk

• For each risk event we’d like to estimate two factors

• Probability– What are the chances the event will occur?

• Impact– What is the cost if it does occur?

• Together, these allow us to compute our exposure for the risk event

13

Risk Event Exposure

Exposure = Probability x Cost

Note that the exposure is never the actual cost incurred. If the risk event occurs, the total impact is absorbed; if it doesn’t occur, there is no impact.

The exposure is a measure of what to expect “on the average.” It is useful primarily for assessing the potential cost of an aggregate of risk events.

14

Risk Exposure: An Example• Suppose the risk event we’re considering is that we

miss a delivery deadline• We estimate the probability that this will occur to

be 25%• Suppose further that our assessment of what

missing the delivery deadline will cost us is $100,000

• The exposure is then = .25 X 100,000 = $25,000• Will this risk event ever cost us $25,000? No.

– the cost will be $0 if we deliver on time– the cost will be $100,000 if we miss the deadline

15

Risk Exposure: An Example (cont’d)

• Suppose now that we have identified 4 main risk events for our project, each with these same characteristics: 25% probability and $100,000 cost if the event occurs.

• So our exposure on each risk is $25,000.• We add these to get our total exposure on the

project which is $100,000.

16

Qualitative Risk Assessment

• Sometimes it is very difficult to make confident estimates of probabilities and impacts associated with risk events

• In these cases, a qualitative assessment of these two factors may make more sense

• Ranking each of the two factors (low-high, low-medium-high, or similar) in a two dimensional table will still provide a basis for prioritizing risks

• Prioritization is crucial to planning and creating risk management strategies

17

Qualitative Risk Assessment

Impact

Low High

Probability

High

Low

L H H H

L L H L

18

Question:Prioritizing Using Qualitative Assessments

How would you prioritize risk events based on assessments done using the chart on the previous slide?

HHHLLHLL

orHHLHHLLL

19

Some other sources of risk– Scope incorrectly defined– Incomplete or incorrect requirements– Poor estimates– Team turnover– Changing scope and requirements– Poor project management and planning– New technologies (already mentioned)

• Risks should be assessed and “managed” even though they are unpredictable

20

Risk Response Strategies

• Perform contingency planning, including:– a contingency budget– schedule alternatives, to include some built-in float– complete emergency responses designed to deal with major areas of risk

• Develop workarounds designed to avoid or minimize selected risks• Mitigate risks

– mitigation involves adding activities/deliverables to a project to offset the possible effect of a potential risk event

– mitigation occurs before the risk event materializes– therefore mitigation costs are incurred whether the risk event occurs or

not– a kind of insurance policy against selected risk events

• Evade risk (Hope for the Best)

21

Risk Avoidance and Mitigation• Risks should be identified early in the project• Stakeholders should be aware of potential risks and their

impacts• They should help devise strategies for either avoiding

them or minimizing their impact• Requires planning• Formulate contingency plans for mitigation• Track risk event potential and revise plan accordingly

22

1) What do you consider to be the risks in the BEC project as you currently understand it?.

2) Is this a low, medium, or high risk project?

3) How would you propose dealing with the risks?

Group Exercise #1

23

Software Risk: A Recent ExampleOn the following several slides, a recent highly publicized software failure is described. Read this case study and we will then discuss its implications for our study.

Errant Trades Reveal a Risk Few Expected

 Downloaded and adapted from NY Times Online, August 2, 2012

Errant trades from the Knight Capital Group began hitting the New York Stock Exchange almost as soon as the opening bell rang on Wednesday. The trading firm Knight Capital recently rushed to develop a computer program so it could take advantage of a new Wall Street venue for trading stocks. But the firm ran up against its deadline and failed to fully work out the kinks in its system, according to people briefed on the matter. In its debut Wednesday, the software went awry, swamping the stock market with errant trades and putting Knight’s future in jeopardy.

Knight, founded in 1995, is a leading matchmaker for buyers and sellers of stocks, handling 11 percent of all trading in the first half of 2012. Knight lost three-quarters of its market value in two days, in addition to losing $440 million from the errant trades, and was scrambling to find financing or a new owner.

24

Errant Trades Reveal a Risk Few Expected (cont’d)The fiasco, the third stock trading debacle in the last five months, revived calls for bolder changes to a computer-driven market that has been hobbled by its own complexity and speed. Among the proposals that gained momentum were stringent testing of computer trading programs and a transaction tax that could reduce trading.

In the industry, there was a widespread recognition that the markets had become more dangerous than even specialists realized. Some S.E.C. officials are pushing new measures that would force firms to fully test coding changes before their public debut, according to a government official who spoke on the condition of anonymity. While the idea has long been discussed at the agency, it gained traction after the Knight debacle.

The S.E.C. applied limited safeguards on trading after the “flash crash” of 2010 sent the broader market plummeting in a matter of minutes. But big investors like T. Rowe Price, members of Congress and former regulators said Thursday that the S.E.C. and the industry had been too complacent and needed to do more to understand and control the supercharged market.

25

Errant Trades Reveal a Risk Few Expected (cont’d)

Arthur Levitt Jr., a former chairman of the Securities and Exchange Commission, said that recent events “have scared the hell out of investors” and called for the agency to hold hearings. “I believe this latest event was handled better than the flash crash, but the larger question is whether our markets are adequate to deal with the technology that is out there,” Mr. Levitt said. “I don’t think they are.”

Regulators have made changes to the markets over the last two decades that have taken it out of the hands of a few New York institutions and allowed dozens of high-frequency trading firms and new trading venues to dominate the stock market.

The high-speed firms like Knight, which connect directly to the servers of the exchanges and are capable of executing thousands of trades a second, are responsible for more than half of all activity in American markets. Companies that have benefited from the fragmentation and computerization of the markets have largely managed to fend off tighter controls by pointing to the steady decline in the cost of trading stocks.

26

Errant Trades Reveal a Risk Few Expected (cont’d)

But even people who had previously defended the advances in trading technology said on Thursday that too many problems had been overlooked. In Knight’s breakdown on Wednesday, as well as in the botched initial public offerings of Facebook in May and BATS Global Markets in March, the problems were caused by new computer programs that had not been adequately tested. Currently regulators have no protocol for signing off on new software programs like the one Knight rolled out.

“When they put these things out in the world they are really being tried for the first time in a real-life test,” said David Leinweber, the head of the Center for Innovative Financial Technology at the Lawrence Berkeley National Laboratory. “For other complex systems we do offline simulation testing.”

Mr. Leinweber has suggested to the S.E.C. that it do this work with the help of the supercomputing facilities at his center. The S.E.C. has recently moved in this direction by contracting with a high-speed trading firm that will provide it with more up-to-date market information.

27

28

Group Exercise1. What factors seem to have led to the

Knight Capital Group disaster?2. Is lack of adequate testing the only

culprit here?3. Do you think these issues might relate

to requirements?4. Could good risk analysis and

management have mitigated this outcome? Explain your answer.

29

Requirements-Related Risks• During Requirements Elicitation

– Scope creep– Schedule pressures– Completeness and correctness– Defining non-functional requirements (performance, usability)– Unstated or implicit requirements– Customer understanding and agreement– Customer-presented “solutions” vs. actual needs

30

Requirements-Related Risks (cont’d)

• During Requirements Analysis– Requirements Prioritization– Technically difficult features– New environments (hardware, software, application area)

• During Requirements Specification– Gaps in understanding and expectation– Schedule pressure to proceed before specification is complete– Ambiguous terminology– Design embedded in requirements

31

Requirements-Related Risks (cont’d)

• During Requirements Validation– Unvalidated requirements– Poor inspection techniques for requirements

• During Change Management– Changing requirements– Poor change process– Unimplemented requirements– Scope creep

32

In this activity, you will work with the proposed new online system to automate seminar registration.

1) Identify several risks inherent in the development of the online seminar registration system project.

2) Consider the risks you’ve identified and decide which can potentially be avoided or eliminated, and which will require mitigation strategies.

3) More specifically, what particular avoidance and mitigation strategies would you suggest for the various risks you’ve identified?

Team Exercise #2

3333

ERD Activity: Seminar Registration System

1. Seminar registration is now handled by mail or by phone, based on seminar brochures sent out in the mail. The customer wishes to implement an online (web-based) enrollment system.

2. A potential seminar enrollee should be able to go the new web site, select a specific seminar and then pay for and enroll in it if space is available.

3. Payment would be made by an online credit card transaction.

4. Currently all seminar information (title, instructor, location, date, time, current enrollment, and maximum enrollment allowed) is held in a Excel spreadsheet.

5. A separate Excel spreadsheet contains some information (name, address, phone, email, credit card info) about previous seminar attendees.

6. The seminar manager requested a new daily report showing the current status of enrollment for all seminars.

Recall our work with the proposed new online system to automate seminar registration for a company that offers seminars at multiple sites and on multiple dates. Here are some features of the proposed system that were gathered at an initial one-hour meeting with the customer.

ProposedSystem

SeminarAttendee

registration_req_w_cc

registration_req_w_corp_pmt

cancellation

catalog_search_req

request_for_transcript

confirmation_of_seat

confirmation_of_cc_pmt

transcript

certificate

confirmation_of_cancellation

registration_declineseminar_cancelled

Email Sys

Class Admin

class_rosters

completion_rosters

Seminar Mgr

daily_report

seminar_cancellation

seminar_cancellation

CorporateFinancial System

payment_info

credit

action

Seminar Info*

new_semina

r

* Customer will convert current data

Attendee Info*

search_results

Context DFD