Independent Verification and Validation (IV&V) Techniques for Object Oriented Software Systems

SAS meeting July 2003

COMPANY PROPRIETARY2

APC0342

Outline

IV&V of OO artifacts

Identifying Risks

Proposed Techniques

GQM (Goal Question Metric) Application

BBN (Bayesian Belief Network) Application

COMPANY PROPRIETARY3

APC0342

Problem Statement

NASA has considerable experience applying IV&V to traditional function-based software systems

However the emerging use of OO brings unique challenges and risks

There is a need to develop IV&V techniques for addressing these risks

Verify these techniques

COMPANY PROPRIETARY4

APC0342

Objective

Understand risks unique to large OO systems

Establish a framework for identifying, evaluating, and implementing IV&V techniques for mitigating these risks

COMPANY PROPRIETARY5

APC0342

Multi -Year Goals

Identify Risks

Develop base set of IV&V Techniques

Apply & evaluate techniques

Focus additional research and match emerging techniques to prioritized needs

COMPANY PROPRIETARY6

APC0342

Approach

Phased Approach– Identify Risks, Develop Base set of Techniques, Evaluate Base set– Focused research to adapt techniques for a broader spectrum of OO

projects

Reasoning– Level of maturity of IV&V’s understanding of the problem, developers

processes, and V&V techniques specific to OO system development– Limited availability of defect data specific to OO development

COMPANY PROPRIETARY7

APC0342

Focused research to map, adapt, or develop techniques to prioritized IV&V activities based on experienced and perceived needs for improvement in efficiency, accuracy, and capacity.

1) Identify OO Development Risks

2) Develop Base IV&V Techniques

– Requirements– Analysis– Design

3) Evaluate Base IV&V Techniques

Continuous research &

evaluation of em

erging techniques

Current Phase Next Phase

COMPANY PROPRIETARY8

APC0342

Activities to-date

Evaluation of current & emerging OO IV&V techniques

Identification of Risks with OO development

Investigating framework for OO IV&V

Developing base set of IV&V techniques

COMPANY PROPRIETARY9

APC0342

State of IV&V Practice

Rudimentary manual techniques for Use cases or OO analysis

Techniques limited to static analysis

No automated tools or metrics being used for requirements, analysis & design

Manual techniques ensure • Specifications use UML compliant notation • Design is adequately documented• Design specifications will meet requirements

Techniques do not address the quality of OO design objectives• Encapsulation, coupling, cohesion, polymorphism etc.

COMPANY PROPRIETARY10

APC0342

Emerging Techniques

Some examples of emerging techniques– Dynamic Analysis Techniques e.g. Yacoub et. al. "Dynamic Metrics for

Object Oriented Designs"– Semantic Metrics e.g. Etzkorn et. al. “Towards a Semantic Metrics Suite

for OOD”– Application of BBN e.g. Fenton et.el. “Bayesian Belief Network Model for

the Safety Assessment of Nuclear Computer-Based Systems”

State of practice– These techniques offer exciting promise for improving IV&V products– Need further enhancement for practical IV&V application

COMPANY PROPRIETARY11

APC0342

Risks to OO Development

Risk to OO development fall into three major categories– Programmatic Risks

• Project termination, software component integration, utilization of tools, and project failure

– Technology Risks• Issues with technology used to build the product, such as: Operating

system, Compilers, Libraries, and middleware – Methodology Risks

• Type of OO methods being used, incorrect application of a method, and issues related to lifecycle phases

COMPANY PROPRIETARY12

APC0342

Programmatic Risks

Project termination – Sub par schedule and performance of product

Methods abandonment– Lack of progress or OO related benefits

High learning curves – Insufficient budget and schedule for training

Not meeting OO objectives– Autonomy between organizations, absence of short iterative development cycle

Dependencies on Tools– Cost and tool deployment requirements

COMPANY PROPRIETARY13

APC0342

Technology Risks

Object Model – Technology does not fully support the object model e.g. asynchronous

communication between objects in relational databases

Persistence– Poor support for object persistence e.g. OODB, relational DB, ad-hoc flat

files

COMPANY PROPRIETARY14

APC0342

Methodology Risks

Incorrect application of methodology, insufficient methodology, or no methodology– Design mixed with requirements/use cases , insufficient information in

design models

Software performance – Language/hardware combination fails to supports performance

High defect level – System usability, reliability, and safety

COMPANY PROPRIETARY15

APC0342

GQM Approach

Identify software product metrics that can support analysis goals

For Use Case diagrams– GOALS- completeness, traceability, correctness– Questions-How to achieve the goals– Metrics-use case metrics such as by Marchesi

Later plans for application to Class diagrams

COMPANY PROPRIETARY16

APC0342

BBN Approach

For Use Case diagrams– Use of BBN to determine readiness for project transition into the next

lifecycle phase– Actors, actions, communication association, as part of network topology– Work is on going for determining characteristics of nodes, e.g. weight,

child and parent levels– CPT tables

Assumption: Use Cases completely expanded

COMPANY PROPRIETARY17

APC0342

Near term Goals

Develop base set of IV&V techniques for– Requirements phase products assuming Use Case Artifacts.– Analysis phase products assuming class and sequence diagrams.– Design phase products assuming class, sequence, and deployment diagrams

and state charts.

Solidify feasibility of applying – GQM for Requirements and Analysis Phases– BBN to gauge readiness for lifecycle phase transition