march 2007copyright 2007, rci1 top ten risks in net- centric systems donald j. reifer reifer...

March 2007 Copyright 2007, RCI 1

Top Ten Risks in Net-Centric Systems

Donald J. ReiferReifer Consultants, Inc.

P.O. Box 4046Torrance, CA 90510

Phone: (310) 530-4493Email: don@reifer.com

March 2007 Copyright 2007, RCI 2

Introduction• Identify the “top ten” risks• Discuss what you can do

about them• Highlight others’ experiences• Recommend that you tackle

risks head-on• Tell you that “even with all

the headaches that the move to net-centric systems is worth the pain”

March 2007 Copyright 2007, RCI 3

Net-Centric Defined • Net-centric - Participating as a part of a continuously-

evolving, complex community of people, devices, information and services interconnected by networks to achieve optimal benefit of resources and better synchronization of events and their consequences. – Involves more than interfaces and interoperability

– Views the network as an enabler and distribution channel

– Built on concepts like industry-wide standards and service-oriented architectures

– Facilitates collaboration and real-time access to knowledge

– Security, availability and quality of service directly impact the warfighter’s ability to succeed on the battlefield

Source: Wikipedia

March 2007 Copyright 2007, RCI 4

Net-Centric Creates New Challenges

Commands

Navigation DataTargeting Data

Tracking Info

CooperativeEngagement

Data

Navigation Data

Shipboard CIC

COTS-BASED

March 2007 Copyright 2007, RCI 5

Top Ten Risks (When Addressing these Challenges)

1. Poor system engineering

2. Inadequate security engineering

3. COTS incompatibilities

4. Impossible schedules

5. Malicious code nightmares

6. Incompatible/ immature processes

7. Unforeseen/unfunded requirements

8. Supply chain surprises

9. Unpredictable quality of service levels

10. High maintenance costs

Source: review of twelve major recent programs

March 2007 Copyright 2007, RCI 6

1. Poor Systems Engineering• Risks

– Poorly defined operational requirements/architecture

– Inadequate attention given to non-functional details

– Lack of early attention on test and integration needs

• Mitigation Actions– Develop system use cases

(sys-ML) and scenarios• Map to capability

requirements

March 2007 Copyright 2007, RCI 7

More Mitigation Actions• Embrace service-oriented

architectures to facilitate sharing and commonality

• Allocate non-functional requirements via end-to-end scenarios as quantitatively as possible using threads

• Consider elevating information operations to the subsystem level

• Tie threads to operational scenarios and then use them as the basis for regression test development

Source: Sun

March 2007 Copyright 2007, RCI 8

2. Inadequate Security Engineering• Risks

– Networks unreliable– Networks subject to

infections/outages• Mitigation Actions

– Elevate Information Operations (IO) to subsystem status early in life cycle

– Design security into the products

– Focus on high return activities

– Address vulnerabilities early in design stage

Many Security Threats• Poor engineering practices• Regulatory hell

– New requirements/no money• Supplier chain vulnerabilities

– Rootkits and other malicious code attacks

• Bad guys getting smarter• New types of attacks

– Use of bots as a Parasitic malware (malicious code)

• Mobile threats in shared networks

• Evolving convergence threats

March 2007 Copyright 2007, RCI 9

Vulnerabilities Everywhere You Look• Hardware vulnerabilities

– RFID used to track movement

– Physical security barriers– Protected links– Access controls/video

surveillance

• Software vulnerabilities– Authentication mechanisms– Principle of least privilege– Identity protection/

biometrics– Behavior patterns/alerts

• Network vulnerabilities– Authentication mechanisms– Traffic/usage patterns/alerts– Voice loggers/Voice-over-IP– Vulnerability scans/

penetration testing– Wireless stack protection– Properly configured firewalls

and intrusion detection devices

– Web filters/gateway restrictions

– Security settings updated as patches tracked and logged

March 2007 Copyright 2007, RCI 10

More Mitigation Actions• Automation vulnerabilities

– Controls isolated, monitored and authenticated

– Synchronization by variable time clocks

• Tampering vulnerabilities– Difficult to reverse engineer

– Critical Program Information (CPI) protected

– Difficult to tamper with

• Incident handling & response– Strategies to recognize attacks

and pro-actively deal with them

• Budget for security– Involves much more than

administrative functions

– Cost for equipment and software is substantial

– Focus is on keeping networks operational

• Staff for security– Requires skilled engineers

who understand tradeoffs

• Implement a security as part of your culture– Lives are at stake

March 2007 Copyright 2007, RCI 11

3. COTS Incompatibilities• Risks

– Suppliers unresponsive

– Plug-and-play becomes patch-and-pray

– Performance plagues

– High maintenance costs

– Licensing nightmares

• Mitigation Actions– Adopt and promote a modern,

standards-based architecture

– Make suppliers vested members of the team

March 2007 Copyright 2007, RCI 12

More Mitigation Actions• Embrace those standards, both

current and future, that best support your needs– Maintain watch on evolving standards;

try to influence their development

• Address dynamic interplay between COTS systems used across the network by focusing on services that implement plug-and-play

• Understand the games vendors play

• Team with those COTS vendors that you trust and can work with

March 2007 Copyright 2007, RCI 13

4. Impossible Schedules• Risks

– Scheduled operational need date impossible to meet

• Nobody has the guts to admit it

– Schedules do not focus on interim capabilities

• Mitigation Actions– Incrementally develop using

capability builds and spirals

– Conduct incremental demos to engage user community and maintain their confidence

March 2007 Copyright 2007, RCI 14

More Mitigation Actions• Run models to determine if

the schedule continues to remain feasible – If not, adjust capability and

build plan accordingly• Monitor progress; assess

status; conduct periodic assessments to determine if schedules remain feasible

• Remember, the easiest way to proceed according to your schedule is to lie to yourself– Avoid this by believing the

metrics and indicators

ISSUES• Interfacing/integrating

with legacy systems always takes longer than expected

• Reuse of legacy turns into a pipe-dream

• Networks require you to pay constant attention to testing and refactoring

• Testing networks forces you to upgrade your bench and develop regression test baselines

March 2007 Copyright 2007, RCI 15

5. Malicious Code Nightmares• Recognize that defense-in-depth

and at the perimeter still leaves holes in network defenses

– GOTS, COTS and ROTS often riddled with malware

• Much of this is unintentional

– New vulnerabilities occur hourly that must be addressed

• Mitigation Actions– Check for malicious code in

GOTS, COTS and ROTS

March 2007 Copyright 2007, RCI 16

More Mitigation Actions• Whenever possible, use components that

are on the certified products list (per common criteria)

• Keep all of your critical software up-to-date• Address false alarm rates by properly

configuring your intrusion detection and/or prevention devices

• Keep abreast of new and known vulnerabilities by monitoring the CVE

• Initiate alerts/alarms using a situation awareness display via the network operations center

• Design your systems to prevent insider as well as outsider attacks

March 2007 Copyright 2007, RCI 17

6. Incompatible/Immature Processes• Risks

– Current CMMI-compatible processes do not address many of the processes used for network-centric warfare

– C&A and DIACAP add time and effort to the mix

– Supply chain dynamics may not be in synch

• Mitigation Actions– Employ agile processes for

integration and test

March 2007 Copyright 2007, RCI 18

More Mitigation Actions• Focus on the processes that drive your

cost and schedule• Recognize that common processes for

the “net-centric” must be agreed upon by its many sources

• Try to manage the network as it evolves using a annual build and release process

• Use a demo-driven process to increase confidence in the releases and reduce risk

• Keep players involved via an ICWG• You cannot survive without common

CM/QC practices

March 2007 Copyright 2007, RCI 19

Size Drivers

Exponential Scale Factors

Systems ofSystems

Definition and

IntegrationEffort

Calibration

• Interface-related equivalent KSLOC

• Number of logical interfaces at SoS level

• Integration simplicity• Integration risk

resolution• Integration stability• Component readiness• Integration capability• Integration processes

COSOSIMO Operational Concept

COSOSIMO

Source: Jo Ann Lane, University of Southern California, 2006

March 2007 Copyright 2007, RCI 20

7. Unforeseen/Unfunded Requirements

• Risks– New opportunities/threats lead

to new requirements– Legacy and reuse shortfalls– More interfaces to mechanize

than anyone thought – Standards will change as will

interface specifications

• Mitigation Actions– Budget for a fixed level of

volatility and change

March 2007 Copyright 2007, RCI 21

More Mitigation Actions• Maintain backup plans to cope with

legacy and reuse shortfalls • Incorporate changes into your build

plans– Prioritize capabilities/incrementally deliver

• Seek additional funds when changes are needed to support development of needed capabilities

• Maintain “visible” reserves to address contingencies

• Do not be a good guy and do things for others for nothing

• Manage your resources tightly

March 2007 Copyright 2007, RCI 22

8. Supply Chain Surprises• Risks

– Other government agencies may not live up to their responsibilities

– Technology refresh

– Vendor surprises

• Mitigation Actions– Maintain technology and vendor

watch functions

– Develop Plan B’s (and C’s)

March 2007 Copyright 2007, RCI 23

More Mitigation Actions• No matter what you do to prevent it,

supplier issues will dominate– Vendors may go out of business – Government supplier may not live up to

their obligations• Negotiate a two-tier support agreement

with critical vendors• Look for fallback positions for critical

items• Maintain good relationships with your

suppliers• Be capable of maintaining government

software organically as a last resort

March 2007 Copyright 2007, RCI 24

9. Unpredictable Quality of Service Levels

• Risks– Poor performance (real or perceived)

– Unreliable service (real or perceived)

– High false alarm rates

• Mitigation Actions– Define quality of service expectations

– Define metrics and measures that quantify your expectations

– Develop benchmarks/assess usage

– Identify heavy usage profiles/patterns

March 2007 Copyright 2007, RCI 25

Metrics Important to Network Operations

Percent of population thinking metric is important

1 10 100

Accuracy

Availability

Delay

Latency

MTBF

MTTR

Response Time

Thruput

Other

Enterprise

Service

March 2007 Copyright 2007, RCI 26

10. High Maintenance Costs• Risks

– Lack of sufficient resources during maintenance

– Service degradation and lack of support

– Finger-pointing over who bears responsibility

• Mitigation Actions– Develop WBS for net-centric ops

– Adequately budget for WBS tasks

– Recognize COTS/GOTS is not free

March 2007 Copyright 2007, RCI 27

Why Worry about NCO Risks?

Rand, Net-Centric Ops Case Study, Stryker Brigade Combat Team, 2005

Light Infantry Brigade

Stryker

Brigade

Quality of individual and shared information

- 10% - 80%

Speed of command 48 hours 3 hours

Ability to control the speed of command

No Yes

Blue-Red Casualty Ratio 10:1 1:1

March 2007 Copyright 2007, RCI 28

Summary and ConclusionsSummary• We identified the “top ten”

net-centric risks and how to mitigate them

• In doing this, we highlighted past experiences and showed you how to capitalize on them to reduce risk

• We highlighted the fact that even though there was a lot of pain involved that the move was worthwhile

Conclusions• The path to network-centric

warfare is paved but has many potholes

• Most of the risks discussed are managerial– Sound management practices

should help eliminate them

• Many of the risks are inherent in any large system development– The mitigation actions

discussed however are not

March 2007 Copyright 2007, RCI 29

Today’s Net-Centric Trends Lead to Tomorrow’s Challenges

• Today’s Trends– Convergence

– Interoperability

– Legacy operations

– Service-oriented architecture-based

• Applications servers

• Publish and subscribe protocols

• Standards-based

– Spiral acquisition

• Tomorrow’s challenges– Security

– Operational architecture

– Layered frameworks

– Net-ready standards• Uniform measures of

performance

• Measures of effectiveness

• Measures of readiness

• Measures of compliance

– Spiral incentives

March 2007 Copyright 2007, RCI 30

Parting Thoughts• If development/fielding of

such networked-centric systems were easy, everyone would have them– Would not be a differentiator

in the battlefield

• The future battlefield raises the ante because getting enough bandwidth and performance remain major issues – Still lots of fun to have

solving these problems

• Many other challenges– Information overload

– Appropriate levels of automation

– Adaptive automation

– Distributed decision-making and team coordination

– Decision biases

– Mitigating complexity

– Security

– Trust and reliability

– Accountability

March 2007 Copyright 2007, RCI 31

Contact Information

Donald J. Reifer, PIReifer Consultants, Inc.Phone: (310)530-4493dreifer@earthlink.net