obstacle driven development: extending requirements analysis
TRANSCRIPT
Obstacle Driven Development
Extending Requirements Analysis
©odd.enterprises
19/11/2014
Obstacle Driven Development
19/11/2014 ©odd.enterprises 2
ODD Process
19/11/2014 ©odd.enterprises 3
ODD Traffic Light Model
19/11/2014 ©odd.enterprises 4
Background
Ideas of Obstacle Driven Development (ODD) are based on numerous development processes including:
• ISO V-model
• Test Driven Development
• ISO specifications
• Requirements analysis
• Agile principles
19/11/2014 ©odd.enterprises 5
About Requirements Analysis
Requirements analysis encompasses tasks that determine the needs or conditions necessary for a new or altered product.
Tasks necessary for requirements analysis include:
• Analysing, documenting, validating and managing software or system requirements
• Identify and resolve conflicting requirements of stakeholders
• Identifying business needs or opportunities using testable and traceable processes
19/11/2014 ©odd.enterprises 6
Requirements Analysis 1
Requirements analysis is performed in numerous ways
• Spiral model
• Use case analysis
• Safety integrity levels
This presentation is used to show how a requirements analysis spiral and SILs were adapted to give ODD processes.
19/11/2014 ©odd.enterprises 7
Requirements Analysis 2
Comparing, mapping and adapting the processes from the Spiral model to ODD led to this model.
• Agreed Behaviours replaces Agreed Requirements due to extended Specification
• Quality Assurance equivalent to Testing
• Negotiation similar to Verification
19/11/2014 ©odd.enterprises 8
Requirements Analysis 3
Further adapting the model leads to an Obstacle Driven Development model.
• Verification replaces Negotiation
• Validation replaces Evaluation
• Testing replaces Quality Assurance
Consolidated Requirements and Documents can be thought of as checkpoints for development19/11/2014 ©odd.enterprises 9
Requirements Checkpoint
The Consolidated Requirements can be considered the checkpoint for the Analysis stage.
• Consolidation using SILs is used to ensure the most important requirements are identified
• Most important requirements for each situation must be dealt with for a successful product
19/11/2014 ©odd.enterprises 10
Documents Checkpoint
When using a product there should be sufficient documentation to describe all the behaviours expected of it.
• Documents should describe everything the product does
• Decomposed from high level behaviours into components
19/11/2014 ©odd.enterprises 11
ODD M-model Checkpoints 1
Adding solution and production stages of development results in an ODD M-model.
• ODD process is linked from start to finish, and beyond
• Verification and validation between stages is performed
• Tests are ran as additions and editions are made
19/11/2014 ©odd.enterprises 12
ODD M-model Checkpoints 2
For the solution and production stages there has also been Checkpoints determined.
• Prototype should be created from an integrated solution
• Product should be the result of production
• Linked to other checkpoints horizontally
19/11/2014 ©odd.enterprises 13
Prototype Checkpoint
Once a solution is fully integrated and tested then a prototype of the product should be achieved.
• Created from the integrated solutions at various levels
• Used to ensure the requirements are covered by the solution
• A working model or template of the product is achieved
19/11/2014 ©odd.enterprises 14
Product Checkpoint
Finally once production is complete then working products should be achieved.
• Decomposition is used to ensure all products are produced according to solution
• Product Assembly at high level to demonstrate decomposition process
19/11/2014 ©odd.enterprises 15
ODD M-model Checkpoints 3
Checkpoints allow linking and testing of the results of previous stages.
• Each checkpoint is used to link another
• Prototype should fulfil identified requirements
• Product should behave as described in documents
19/11/2014 ©odd.enterprises 16
ODD Analysis
The following slides are used to demonstrate how the requirements analysis is adapted to allow for ODD.
• Safety Integrity Levels used to measure and process hazards
• Decision tree approach used to create situations
• Verification and validation of specification
19/11/2014 ©odd.enterprises 17
Fire Triangle
A fire triangle is a well known educational tool for understanding and preventing fires.
• If the fire triangle is completed then a fire will occur
• Preventing one of the situations will prevent a fire
• Requirements will often regard preventing fires
19/11/2014 ©odd.enterprises 18
Fire Triangle 2
By reordering the fire triangle we can create a diagram which demonstrate the components interaction for a fire.
• Situations can be constructed using a tree diagram
• Constructing situations from components allow greater range to be considered
19/11/2014 ©odd.enterprises 19
Fire Triangle 3
Using the reordered fire triangle it can be seen that the components combine to create a hazard.
• Process is adaptable to all fire hazards and environments
• Can be extended to any number of situations
• Each of the components can be given SIL ratings for Probability, Severity and Controllability
19/11/2014 ©odd.enterprises 20
Fire Prevention 1
A simple fire prevention example to demonstrate how the requirements may be linked through the process.
• Fuel and oxygen are present
• Fire is prevented by preventing heat sources
• No smoking chosen as a solution, more are possible
Note this diagram has been simplified significantly.19/11/2014 ©odd.enterprises 21
Fire Prevention 2
The diagram shows how a behaviour has been identified which prevents fire.
• Fuel and oxygen are present so preventing fire through preventing heat is chosen
• Adaptable to different situations and fire prevention strategies.
19/11/2014 ©odd.enterprises 22
Probability Tree 1
Probability Trees are used to measure the probability of a situation occurring that is based on a number of interacting events.
• Common examples are the probabilities of situations occurring from a coin toss or a drawing from a deck of cards
• Can be used to model complex interactions
19/11/2014 ©odd.enterprises 23
Heads 50%
Heads 50%
Tails 50%
Tails 50%
Heads 50%
Tails 50%
Binomial Distribution 1
Binomial Distributions can be used to determine the probability for any number of events with 2 possible outcomes.
• Binomial process illustrates the concept of decision trees can be extended
• Can be used to model complex interactions
19/11/2014 ©odd.enterprises 24
Heads 50%
Heads 50%
Tails 50%
Tails 50%
Heads 50%
Tails 50%
Binomial Distribution 2
Binomial Distributions can be used to determine the probability for any number of events with 2 possible outcomes.
• Binomial process illustrates the concept of decision trees can be extended
• Can only be used to model yes or no experiments
𝑃 𝑋 = 𝑘 =𝑛𝑘
𝑝𝑘(1 − 𝑝)𝑛−𝑘
19/11/2014 ©odd.enterprises 25
Probability Tree 2
We can extend the coin toss example into engineering by replacing the coin tosses with system components.
• Working component replaces heads
• Failing component replaces tails
• Potential hazards of a series of failures can be determined
19/11/2014 ©odd.enterprises 26
Component 1Pass 99%
Component 2Pass 98%
Component 2Fail 2%
Component 1Fail 1%
Component 2Pass 98%
Component 2Tails 2%
Binomial Distribution 3
Binomial Distributions can be used to determine the probability for components failing when creating a product.
• Process is a common example of using probabilities in manufacturing
• Can be extended to model failure in the field
19/11/2014 ©odd.enterprises 27
Component 1Pass 99%
Component 2Pass 98%
Component 2Fail 2%
Component 1Fail 1%
Component 2Pass 98%
Component 2Tails 2%
Probability Tree 3
Probability Trees can be easily extended to other situations.
• Coins may also land on their side
• The event has been assigned a probability of 0.017 % (1 per 6000 tosses)
• Probability tree has exponentially more branches due to another outcome
19/11/2014 ©odd.enterprises 28
Head ≈ 50%
Head ≈ 50%
Tails ≈ 50%
Side ≈ 1 / 6000
Tails ≈ 50%
Head ≈ 50%
Tails ≈ 50%
Side ≈ 1 / 6000
Side ≈ 1 / 6000
Head ≈ 50%
Tails ≈ 50%
Side ≈ 1 / 6000
Probability Tree 4
Probability Trees can be used to ensure all possible situations are modelled.
• Systems may have unknown states between pass and fail
• Unknown states can include loss of communication or wear
• Unknown states then investigated for effects
19/11/2014 ©odd.enterprises 29
Component 1Pass 98%
Component 2Pass 96%
Component 2Fail 2%
Component 2Unknown 2%
Component 1Fail 1%
Component 2Pass 96%
Component 2Fail 2%
Component 2Unknown 2%
Component 1Unknown 1%
Component 2Pass 96%
Component 2Fail 2%
Component 2Unknown 2%
Safety Integrity Levels 1
Safety Integrity Levels (SILs) are used to provide a process for measuring the potential hazards of a situation.
• Situation is analysed and measures for Probability, Severity and Controllability determined
• From these measures for the risk of hazard occurring from the situation
𝑆𝐼𝐿 = 𝑃 𝐸 ∗ 𝑆 𝐸 ∗ 𝐶 𝐸
19/11/2014 ©odd.enterprises 30
Safety Integrity Levels 2
Safety Integrity Levels may be used for a wide range of safety critical analysis.
• Probability is measured by how likely the situation will occur
• Severity is measured by the potential damage of the situation
• Controllability is measured by the ability to effect change in the situation
19/11/2014 ©odd.enterprises 31
Component𝑆𝐼𝐿 = 𝑃 𝐸 ∗ 𝑆 𝐸 ∗ 𝐶 𝐸
Safety Integrity Levels 3
Coin toss example will be extended to provide example of SILs.
• Probability of outcome is slightly different for different coins
• Severity of an outcome can be measured by what is lost or gained
• Controllability of probability and severity determined by who flips the coin and the stakes
19/11/2014 ©odd.enterprises 32
Safety Integrity Levels 4
If a Probability Tree and SILs are processed in the same way then why not combine them in a decision tree?
• Measures are added for severity and controllability
• Each branch is a situation which can be processed to find SIL ratings and determine requirements
19/11/2014 ©odd.enterprises 33
Component 1Pass
P(P1)*S(P1)*C(P1)
Component 2Pass
P(P2)*S(P2)*C(P2)
Component 2Fail
P(F2)*S(F2)*C(F2)
Component 1Fail
P(F1)*S(F1)*C(F1)
Component 2Pass
P(P2)*S(P2)*C(P2)
Component 2Fail
P(F2)*S(F2)*C(F2)
ODD Decision Tree 1
It is theorised that a Decision Tree can be used to find requirements.
• Severity and Controllability are added to each event
• Requirements are found with SIL processes using the branches
• Facilitates a unit testing approach for situations
19/11/2014 ©odd.enterprises 34
Component 1Pass
P(P1)*S(P1)*C(P1)
Component 2Pass
P(P2)*S(P2)*C(P2)
Component 2Fail
P(F2)*S(F2)*C(F2)
Component 1Fail
P(F1)*S(F1)*C(F1)
Component 2Pass
P(P2)*S(P2)*C(P2)
Component 2Fail
P(F2)*S(F2)*C(F2)
ODD Decision Tree 2
Adding the SIL components to a Probability Tree allows requirements to be found directly from a decision tree.
• Structure is as a Probability Tree with processing including SILs
• SILs may be found by multiplying along the branches of the decision tree
19/11/2014 ©odd.enterprises 35
Component 1Pass
P(P1)*S(P1)*C(P1)
Component 2Pass
P(P2)*S(P2)*C(P2)
Component 2Fail
P(F2)*S(F2)*C(F2)
Component 1Fail
P(F1)*S(F1)*C(F1)
Component 2Pass
P(P2)*S(P2)*C(P2)
Component 2Fail
P(F2)*S(F2)*C(F2)
ODD Decision Tree 3
Processing the resulting Decision Tree should be similar to a Probability Tree.
• SIL ratings processed by multiplying probability, severity and controllability along branches
• SIL result rating between 1 and 4, with 4 being the best
19/11/2014 ©odd.enterprises 36
Component 1Pass
P(P1)*S(P1)*C(P1)
Component 2Pass
P(P2)*S(P2)*C(P2)
Component 2Fail
P(F2)*S(F2)*C(F2)
Component 1Fail
P(F1)*S(F1)*C(F1)
Component 2Pass
P(P2)*S(P2)*C(P2)
Component 2Fail
P(F2)*S(F2)*C(F2)
ODD Decision Tree 4
Using a Decision Tree and SILs gives numerous advantages.
• Branches can be used to ensure that situations are not missed
• Extendible and adaptable to new and differing situations
• Each branch gives a situation
• Situations can be found from combining their components
19/11/2014 ©odd.enterprises 37
Component 1Pass
Component 2Pass
Situation A
Component 2Fail
Situation B
Component 1Fail
Component 2Pass
Situation C
Component 2Fail
Situation D
Processing Decision Tree
Situation A is used to define the situation where both components have passes.
Using the decision tree we can find the SIL rating for the situation.
𝑆𝐼𝐿 𝐴= 𝑃 𝑃1 ∗ 𝑆 𝑃1 ∗ 𝐶 𝑃1 ∗𝑃 𝑃2 ∗ 𝑆 𝑃2 ∗ 𝐶(𝑃2)
Situation D is used to define the situation where both components have failed.
Using the decision tree we can find the SIL rating for the situation.
𝑆𝐼𝐿 𝐷= 𝑃 𝐹1 ∗ 𝑆 𝐹1 ∗ 𝐶 𝐹1 ∗𝑃 𝐹2 ∗ 𝑆 𝐹2 ∗ 𝐶(𝐹2)
19/11/2014 ©odd.enterprises 38
ODD Decision Tree 5
When a Decision Tree is used there is an infinite range of situations possible and most potential outcomes can be considered.
• Events can be comprehensively modelled and extended
• New situations are added as branches
• Models the complexity of the expected situations encountered
19/11/2014 ©odd.enterprises 39
ODD Decision Tree 6
Component 1 Pass
Component 2 Pass
Component 3 Pass
Component 3 Fail
Component 3 Unknown
Component 2 Fail
Component 3 Pass
Component 3 Fail
Component 3 Unknown
Component 3 Unknown
Component 3 Pass
Component 3 Fail
Component 3 Unknown
Component 1 Fail
Component 2 Pass
Component 3 Pass
Component 3 Fail
Component 3 Unknown
Component 2 Fail
Component 3 Pass
Component 3 Fail
Component 3 Unknown
Component 2 Unknown
Component 3 Pass
Component 3 Fail
Component 3 Unknown
Component 1 Unknown
Component 2 Pass
Component 3 Pass
Component 3 Fail
Component 3 Unknown
Component 2 Fail
Component 3 Pass
Component 3 Fail
Component 3 Unknown
Component 2 Unknown
Component 3 Pass
Component 3 Fail
Component 3 Unknown
19/11/2014 ©odd.enterprises 40
The following Decision Tree is used to illustrate the complexity of creating decision trees. This Decision Tree models just 3 components with 3 different states: Pass, Fail and Unknown.
Creating Unit Tests 1
ODD uses a unit testing approach throughout so tests must be created directly from situations to create behaviours.
• Each branch represents a situation and therefore a behaviour should cover each
• Unit tests should be created for each branch of the decision tree
19/11/2014 ©odd.enterprises 41
Verify Behaviour
DescribeBehaviour
Validate Behaviour
Situation
Creating Unit Tests 2
1. Situation is chosen from a branch on the decision tree.
2. Behaviour verification test is first created and linked directly from the situation.
3. Behaviour is described to cover a situation.
4. Behaviour is validated.
5. Repeat for all situations.
19/11/2014 ©odd.enterprises 42
Verify Behaviour
DescribeBehaviour
Validate Behaviour
Situation
Creating Unit Tests 3
Creating a specification from the analysis requires each situation will have a test created which is then passed.
• Specification used to describe all behaviours of the product
• Using an ODD Specification allows cross examination of the behaviours against situations
19/11/2014 ©odd.enterprises 43
Verify Specification
DescribeSpecification
Validate Specification
Situation
Creating Unit Tests 4
The diagram shows an alternative form of ODD with the traffic light testing process between analysis and specification.
• Demonstrates the linking of stages with testing processes
• Traffic lights on the right indicate processes used to link the specification and solution (Design is
assumed to be tested)
19/11/2014 ©odd.enterprises 44
odd.enterprises Service Model
19/11/2014 ©odd.enterprises 45
Legal Stuff
ReferencesRequirements Analysis
http://en.wikipedia.org/wiki/Requirements_analysis
Safety Integrity Level
http://en.wikipedia.org/wiki/Safety_Integrity_Level
Decision Tree
http://en.wikipedia.org/wiki/Decision_tree
Requirements Spiral Model
www.engineering.uiowa.edu/~kuhl/SoftEng/Slides4.pdf
Unit Testing
http://en.wikipedia.org/wiki/Unit_testing
DisclaimerThe ODD M-model and associated processes are provided by odd.enterprises and may be used for any purpose whatsoever.
The names odd.enterprises and associated logos should not be used in any representation, advertising, publicity or other manner whatsoever to endorse or promote any entity that adopts or uses the model and/or associated processes.
odd.enterprises does not guarantee to provide support, consulting, training or assistance of any kind with regards to the use of the model and/or processes including any updates.
You agree to indemnify odd.enterprises and its affiliates, officers, agents and employees against any claim or demand including reasonable solicitors fees, related to your use, reliance or adoption of the model and/or processes for any purpose whatsoever.
The model is provided by odd.enterprises “as is” and any express or implied warranties, included but not limited to the implied warranties of merchantability and fitness for a particular purpose are expressly disclaimed.
In no event shall odd.enterprises be liable for any damages whatsoever, including but not limited to claims associated with the loss of data or profits, which may result from any action in contract, negligence or other tortious claim that arises out of or in connection with the use or performance of the model.
19/11/2014 ©odd.enterprises 46