validating requirements determining completeness and correctness of requirements using the system...
TRANSCRIPT
![Page 1: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/1.jpg)
Validating Requirements
Determining Completeness and Correctness of Requirements Using
the System Reference Model
IV&V Workshop16 September 2009
![Page 2: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/2.jpg)
Overview
• Validation Purpose and Definitions• A Correct and Complete SRM, and the Three
Questions• SRM Correlation Mapping• Analysis of Correlation Results• Correct, Complete, Incorrect, and Incomplete
Examples• Best practices, lessons learned, and challenges
![Page 3: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/3.jpg)
IV&V’s Validation Definition• The process of evaluating artifacts to ensure that the
right behaviors have been defined in the artifacts. • The right behaviors are those that adequately describe
– what the system is supposed to do– what the system is not supposed to do, and – what the system is supposed to do under adverse conditions.
• Validation ensures that the software system performs to the user’s needs under operational conditions.– Contains the desired capabilities to accomplish the mission
goals – Does not contain unspecified limitation that impedes the
capabilities
![Page 4: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/4.jpg)
Validation Goal• To ensure that
– The right behaviors have been defined• Adequately describe
– What the system is supposed to do– What the system is not supposed to do– What the system is supposed to do under adverse
conditions
• Correct and Complete
– The requirement specifications are of high quality• Unambiguous, Consistent, and Verifiable
![Page 5: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/5.jpg)
Correct (IVV 09-1)• Applicable requirement(s) meet all or part of the goals
and behaviors of the system– Note: not all requirements can be evaluated in isolation; it may require a
set of requirements to be evaluated together in order to determine that a particular goal or behavior is being met).
• The requirements are an accurate elaboration of the defined objectives or goals– e.g., the use of temporal modal operators like “next”, “until”, “always”,
and “eventually”, are appropriately used to reflect the desired behavior
• The requirements adequately refine the higher-level requirements
• Design or implementation-specific information is specified as constraints to the behaviors captured in the requirements
![Page 6: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/6.jpg)
Complete (IVV 09-1)• All the needed information to completely specify a
desired behavior is identified (i.e., all preconditions, postconditions, and invariants are specified for the described behavior).
• Threads of behavior are represented by more than one requirement, versus one compound requirement that attempts to capture the entire thread (i.e., that each requirement specifies only one “thing”).
• The use of conjunctions (e.g., “and”, “or”) are restricted to preconditions, postconditions, and invariants.
![Page 7: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/7.jpg)
How?
“This goal is achieved through the development and application of a system reference model (SRM) that will include a formal specification. The SRM can then be used to show (e.g., validate) that the right system behaviors are specified and the associated requirements are unambiguous, correct, complete, consistent, and verifiable. The SRM can also be used to validate (or develop) a test design that will demonstrate that the software products meet the specification and the operational need.” – IVV 09-1
![Page 8: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/8.jpg)
The System Reference Model• Includes sets of Modeling Artifacts
– Use cases– Activity Diagrams, Sequences Diagrams– Statecharts– Domain Models (Class Diagrams, Communication Diagrams)– Statechart Assertions– JUnit Test Cases
• A concise description of the IV&V team’s understanding of the problem – Analysis tool– Communication tool
• Captures expected system behaviors– 3 Questions
![Page 9: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/9.jpg)
What’s In the SRM?Validation and Verification involves answering the following
three questions:1. Will the System do what it is supposed to do?2. Will the System not do what it is not supposed to do?3. Will the System maintain operations under adverse conditions?
Note, in order to answer these questions, we must first have an independent understanding of:– What the system is supposed to do– What the system is not supposed to do– What the system is supposed to do to maintain operations
under adverse conditions
This information can be found within different areas of our model.
![Page 10: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/10.jpg)
SRM Product Dependencies
![Page 11: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/11.jpg)
![Page 12: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/12.jpg)
System Goals
![Page 13: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/13.jpg)
System Goals
![Page 14: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/14.jpg)
Constraints, Actors, and Environment
![Page 15: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/15.jpg)
Text Use Cases
![Page 16: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/16.jpg)
Use Case Example
What the system is supposed to do
All parts within the Main Success Scenarios
describe the actions that must take place to
accomplish the goal(s).
Adverse ConditionsExtension Scenarios show
how the system should react to adverse conditions to get back on the success
path or transition to safe mode.
![Page 17: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/17.jpg)
Activity Diagram Example
Flight System
Goal: Flight System precesses and damps nutation to point the High Gain Antenna at earth for communication and GRAV science
Precondition: Engineering Instruments are calibrated
Flight SystemTurn on IMUs
Turn on Precession catalyst bed heaters
Turn on X-band Transmitters
Passive Nutation Damping
Return Errors
<<Main Success Scenario>>Precess to Earth Point
<<Extension Scenarios>>Precess to Earth Point Fails
Select High Gain Antenna
Select Forward Low Gain Antenna
Pulse RCS Thrusters
Turn off IMU
Turn off cat bed heaters
Use Other Thrusters
[IMU Does not turn on or malfunctions]
[Cat bed heaters
do not turn on]
[turning to
earth point][not turning to earth point]
[thrusters do not fire]
What the system is supposed to
doAll parts within the
Main Success Scenarios describe
the actions that must take place to
accomplish the goal(s).
Adverse ConditionsExtension Scenarios show
how the system should react to adverse conditions to get back on the success
path or transition to safe mode.
![Page 18: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/18.jpg)
Sequence Diagrams Example
What the system is supposed to do
The interactions of the Sequence
Diagram describe the steps involved to accomplish the
goal(s).
18
Adverse ConditionsA sequence diagram
can also be developed to show
how the system should react to
adverse condition.
![Page 19: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/19.jpg)
SRM Validation Scenarios
getEstimatedStateVector()
estimateStateVector()
AssertTrue()
AssertFalse()
estimateStateVector()
AssertTrue()
getEstimatedStateVector()
getEstimatedStateVector()
public void testScenario2() {st.getEstimatedStateVector();st.estimateStateVector();assertTrue (st.isSuccess());st.estimateStateVector();assertTrue (st.isSuccess());st.getEstimatedStateVector();st.getEstimatedStateVector();assertFalse (st.isSuccess()); }
What the system is NOT supposed to do
This information can be found in our Assertions
and the Validation scenarios we create to test
against them.
![Page 20: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/20.jpg)
Validation WBS (IVV 09-1)
1.0 Validation1.1 Obtain/Develop a System Reference Model
(SRM)
1.2 Validate System Requirements
1.3 Validate Test Design
![Page 21: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/21.jpg)
Validate System Requirements• For each level of system decomposition, we need to
determine– Sufficiency of the requirements
• Is there a corresponding requirement for every SRM behavior and Statechart assertion at that level?
– Quality of the requirements• Assess the quality of each requirement that has a
corresponding SRM behavior or Statechart assertion at that level
– Sufficiency of the SRM• Is there any mission-critical, safety-critical requirement not
covered by an SRM behavior or Statechart assertion at that level?
• A “correlation map” is built to capture these relationships
![Page 22: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/22.jpg)
Validation Possibilities
SRM Behaviors Requirements
SRM Correct?Requirements Complete?
SRM Correct?Requirements In
Scope and Valid?
ValidatedRequirements
Unambiguous, Correct,
Consistent, & Verifiable
Requirements
![Page 23: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/23.jpg)
An Example Behavior
Extensions – Q2 & Q3
Preconditions
Main Success Scenario – Q1 & Q2
Constraints
Post-conditions
Goal
References
![Page 24: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/24.jpg)
Requirement Proxies
![Page 25: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/25.jpg)
Requirement Proxies
![Page 26: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/26.jpg)
Requirement Proxies
![Page 27: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/27.jpg)
Requirement Proxies
![Page 28: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/28.jpg)
Other Analysis Tools
Subject Requirement
Child Requirements
Parent Requirements
Validation Findings
Correlation Mapping
![Page 29: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/29.jpg)
Requirement Data
Subject Requirement
Child Requirements
Parent Requirements
Validation Findings
Correlation Mapping
Subject Requirement
![Page 30: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/30.jpg)
Parent Traces
Subject Requirement
Child Requirements
Parent Requirements
Validation Findings
Correlation Mapping
Parent Requirements
![Page 31: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/31.jpg)
Child Traces
Subject Requirement
Parent Requirements
Validation Findings
Correlation Mapping
Child Requirements
![Page 32: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/32.jpg)
Correlation Mapping
Subject Requirement
Child Requirements
Parent Requirements
Validation Findings
Correlation Mapping
![Page 33: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/33.jpg)
Validation Findings
Subject Requirement
Child Requirements
Parent Requirements
Correlation Mapping
Validation Findings
![Page 34: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/34.jpg)
Requirement Proxies
![Page 35: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/35.jpg)
Correlation Mapping & Requirement Evaluation
![Page 36: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/36.jpg)
Correlation Map
![Page 37: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/37.jpg)
Correlation Map
Mod
el E
lem
ents
Req
uire
men
t Map
ping
Find
ings
/Issu
es
![Page 38: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/38.jpg)
Correct (IVV 09-1)• Applicable requirement(s) meet all or part of the goals
and behaviors of the system– Note: not all requirements can be evaluated in isolation; it may require a
set of requirements to be evaluated together in order to determine that a particular goal or behavior is being met).
• The requirements are an accurate elaboration of the defined objectives or goals– e.g., the use of temporal modal operators like “next”, “until”, “always”,
and “eventually”, are appropriately used to reflect the desired behavior
• The requirements adequately refine the higher-level requirements
• Design or implementation-specific information is specified as constraints to the behaviors captured in the requirements
Coverage of Model
Consistency with Model
Separation of Information
![Page 39: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/39.jpg)
Complete (IVV 09-1)• All the needed information to completely specify a
desired behavior is identified (i.e., all preconditions, postconditions, and invariants are specified for the described behavior).
• Threads of behavior are represented by more than one requirement, versus one compound requirement that attempts to capture the entire thread (i.e., that each requirement specifies only one “thing”).
• The use of conjunctions (e.g., “and”, “or”) are restricted to preconditions, postconditions, and invariants.
Coverage of Model
![Page 40: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/40.jpg)
Correlation Mapping & Requirement Evaluation
![Page 41: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/41.jpg)
Correlation Mapping & Requirement Evaluation
![Page 42: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/42.jpg)
Identifying Issues
• L2 Rqmnt - The Project shall generate, route, transport, store and execute a sequence containing any of the following types of time-tagged commands: absolute time, time relative to a sequence-external time value stored on-board, time relative to the execution of the previous command in the sequence.
• L3 Rqmnt - The sequence time tags shall be either an absolute execution time, time relative to a sequence external time value, or a time relative to the execution of the previous command or command file.
• L4 Rqmnt - The flight software shall provide the means for running onboard relative and absolute time, relative to a sequence external time value, tagged sequences.
![Page 43: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/43.jpg)
Identifying Issues
Mod
el E
lem
ents
Req
uire
men
t Map
ping
Find
ings
/Issu
es
Model Requirements Findings
![Page 44: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/44.jpg)
Lessons Learned & Best Practices
![Page 45: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/45.jpg)
Challenges• Varying levels of detail between the SRM and
requirements being validated• What vs. How – “Requirements Model” vs. “Design
Model”• Terminology differences between SRM behaviors and
requirements• Lack of adequate tools, work instructions, product
descriptions/templates – particularly MKS Artifact Mapping capability
![Page 46: Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009](https://reader036.vdocuments.us/reader036/viewer/2022062423/56649f225503460f94c3acab/html5/thumbnails/46.jpg)
Questions?