combining stpa and bdd for safety analysis and...
TRANSCRIPT
![Page 1: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/1.jpg)
Combining STPA and BDD for Safety Analysis and Verification
Yang Wang Joint work with Stefan Wagner STAMP Workshop MIT, March 29, 2018
Papers will be published in:the 19th International Conference on Agile Software Development, from May 21 to May 25, Porto, Portugal.the 40th International Conference on Software Engineering Companion, from May 27 to June 3, Gothenburg, Sweden.
![Page 2: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/2.jpg)
2
Concept of operations
Requirements and Architecture
Detailed Design
Implementation
Integration, Test and Verification
System verification and validation
Operation and Maintenance
STPA BDD
What we will talk about?
2
Copyright Yang Wang | [email protected]
![Page 3: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/3.jpg)
Agenda
1. Motivation
2. BDD
3. STPA-BDD
4. Evaluation
5. Conclusion & Future Work
3
Why do we
use BDD
What is BDD
How to use
BDD for STPA
How are the results
So the next
…
3
![Page 4: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/4.jpg)
Agenda
1. Motivation
2. BDD
3. STPA-BDD
4. Evaluation
5. Conclusion & Future Work
4
Why do we
use BDD
What is BDD
How to use
BDD for STPA
How are the results
So the next
…
4
![Page 5: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/5.jpg)
Existing safety verification
5
5
![Page 6: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/6.jpg)
6
6
In industries, the prevalent method for verifying safety is testing (i.e. UAT).
Usually, UAT happens in a conference or war room sort of a set up where the users, PM, QA team representatives all sit together for a day or two and work through all the acceptance test cases.
Copyright Yang Wang | [email protected]
![Page 7: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/7.jpg)
7
Integrated Safety Analysis Using Systems-Theoretic Process Analysis and Software Model Checking, Asim Abdulkhaleq and Stefan Wagner, 2016
STPA + Model Checking
7
Copyright Yang Wang | [email protected]
![Page 8: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/8.jpg)
Problem Statement
8
8
![Page 9: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/9.jpg)
9
“Weak communication between requirements engineers and test engineers often leads to confusing features.” - E. Bjarnason, P. Runeson, M. Borg et al.
The safety verification needs to support communication.9
Copyright Yang Wang | [email protected]
![Page 10: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/10.jpg)
Agenda
1. Motivation
2. BDD
3. STPA-BDD
4. Evaluation
5. Conclusion & Future Work
10
Why do we
use BDD
What is BDD
How to use
BDD for STPA
How are the results
So the next
…
10
![Page 11: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/11.jpg)
11
“ Behaviour-Driven Development (BDD) builds upon Test-Driven Development … The best practitioners work from the outside-in,
starting with a failing customer acceptance test that describes the behaviour of the system from the customer’s point of view …We make
a deliberate effort to develop a shared, ubiquitous language for talking about the system.”
- Matt Wynne et al.
11
![Page 12: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/12.jpg)
In the family of Test-Driven Development Relies on testing system behaviour
Implements a template for generating test scenarios and test cases
Has been used for verifying non-functional requirements
Behaviour-Driven Development (BDD)
12Copyright Yang Wang | [email protected]
![Page 13: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/13.jpg)
• Add a test• Run all tests and see if the new test fails• Write the code• Run tests• Refactor code• Repeat
- Kent Beck
Relies on testing system behaviour
Implements a template for generating test scenarios and test cases
Has been used for verifying non-functional requirements
13Copyright Yang Wang | [email protected]
Behaviour-Driven Development (BDD)
![Page 14: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/14.jpg)
14
Relies on testing system behaviour
Implements a template for generating test scenarios and test cases
Has been used for verifying non-functional requirements
Copyright Yang Wang | [email protected]
- Thomas Dohmke
Behaviour-Driven Development (BDD)
![Page 15: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/15.jpg)
15
TDD creates well-written unit of codeATDD emphasises on developer-tester-
business customer collaboration
Implements a template for generating test scenarios and test cases
Has been used for verifying non-functional requirements
Copyright Yang Wang | [email protected]
- Thomas Dohmke
Behaviour-Driven Development (BDD)
![Page 16: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/16.jpg)
16
Implements a template for generating test scenarios and test cases
Has been used for verifying non-functional requirements
TDD BDD ATDD
Copyright Yang Wang | [email protected]
- Thomas Dohmke
Behaviour-Driven Development (BDD)
![Page 17: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/17.jpg)
17
Given the initial contextWhen an event occurs
Then ensure some outcomes
Has been used for verifying non-functional requirements
TDD BDD ATDD
Copyright Yang Wang | [email protected]
- Thomas Dohmke
Behaviour-Driven Development (BDD)
![Page 18: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/18.jpg)
18
Given the initial contextWhen an event occurs
Then ensure some outcomes
- continuumsecurity
Scenario: Present the login form over an HTTPS connection Given a new browser instanceAnd the login page is displayed…When the HTTP request-response containing the login formThen the protocol should be HTTPS
TDD BDD ATDD
Copyright Yang Wang | [email protected]
- Thomas Dohmke
Behaviour-Driven Development (BDD)
![Page 19: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/19.jpg)
Agenda
1. Motivation
2. BDD
3. STPA-BDD
4. Evaluation
5. Conclusion & Future Work
Why do we
use BDD
What is BDD
How to use
BDD for STPA
How are the results
So the next
…
19
19
![Page 23: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/23.jpg)
Agenda
1. Motivation
2. BDD
3. STPA-BDD
4. Evaluation
5. Conclusion & Future Work
Why do we
use BDD
What is BDD
How to use
BDD for STPA
How are the results
So the next
…
23
23
![Page 25: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/25.jpg)
25
Productivity
We test how many safety requirements can be written into test cases within a limited time slot.
25
Copyright Yang Wang | [email protected]
![Page 26: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/26.jpg)
26
Quality
We test the quality through the automated test reports from Eclipse and PIT.
26
Copyright Yang Wang | [email protected]
![Page 27: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/27.jpg)
27
Communication
The participants portray as business analysts and developers to discuss the STPA-BDD test cases and test results.
27
From the developer’s perspective:
BDD has a clear documentation.
The developers could flush out functional gaps before development.
The developers have a good understanding of the business requirements.
BDD test cases have a good organisation and structure.
Realistic examples make the developers think harder.
There is an obvious glue between test cases and code.
Copyright Yang Wang | [email protected]
![Page 28: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/28.jpg)
28
Communication
The participants portray as business analysts and developers to discuss the STPA-BDD test cases and test results.
28
From the business analyst’s perspective:
The developers consider the safety requirements deeply and initiatively.
The business analysts are more confident about the test cases.
It becomes easier to identify conflicts in business rules and test cases.
The business analysts are clear about the status of acceptance testing.
The business analysts could spend less time on sprint-end acceptance tests.
Copyright Yang Wang | [email protected]
![Page 29: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/29.jpg)
Agenda
1. Motivation
2. BDD
3. STPA-BDD
4. Evaluation
5. Conclusion & Future Work
Why do we
use BDD
What is BDD
How to use
BDD for STPA
How are the results
So the next
…
29
29
![Page 30: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/30.jpg)
30
Conclusion
BDD seems to be a possible way for STPA to verify safety requirements.
It verifies system behaviours.
It can start at an early stage.
It supports communication.
30
Copyright Yang Wang | [email protected]
![Page 31: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/31.jpg)
Future Work
Combine BDD with STPA requirements specification.
Test automation of BDD.
Evaluation with professionals.
31
31
Copyright Yang Wang | [email protected]
![Page 32: Combining STPA and BDD for Safety Analysis and Verificationpsas.scripts.mit.edu/home/wp-content/uploads/2018/04/... · 2018-04-06 · Combining STPA and BDD for Safety Analysis and](https://reader034.vdocuments.us/reader034/viewer/2022042111/5e8c0a2310a32c564520472d/html5/thumbnails/32.jpg)
Thanks!
e-mail phone +49 (0) 711 685- www.
University of Stuttgart
Yang Wang, PhD candidate
88342iste.uni-stuttgart.de/en/se/people/yang-wang.html
Institute of Software Technology