playing the devil ’ s advocate: verifying real-time systems jan jürjens software & systems...
TRANSCRIPT
![Page 1: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/1.jpg)
Playing the Devil’s Advocate:
Verifying Real-Time SystemsJan Jürjens
Software & Systems Engineering TU Munich, Germany
http://www.jurjens.de/jan
![Page 2: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/2.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 2
Dependability
Ability to deliver service that can justifiably be trusted.
By definition issue supposed to include Reliability, Safety, Security attributes, the threats against them and the means to encounter the threats. (Although historically „security“ and „dependability“ communities have been largely disjoint.)
![Page 3: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/3.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 3
Safety
Safety: software execution without contributing to hazards.
Safety-critical systems: five failure condition categories: catastrophic, hazardous, major, minor, no effect.
Corresponding safety levels A - E (DO-178B standards in avionics).
![Page 4: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/4.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 4
Reliability
For safety need sufficient level of reliability: probability of failure-free functioning of software component for specified period in specified environment.
Reliability goals: via the maximum allowed failure rate. For high degree of reliability, testing not sufficient (1 failure per 100,000 years).
![Page 5: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/5.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 5
Embedded Systems
In particular, embedded software increasingly used in safety-critical systems (flexibility):
• Automotive• Avionics• Aeronautics• Robotics, Telemedicine• …Our treatment of dependable systems in
particular applies to embedded systems.
![Page 6: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/6.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 6
Fault-tolerance
Redundancy model determines which level of redundancy provided.
Goal: no hazards in presence of single-point failures.
In the following treatment:• focus on reliability, in particular for safety• focus on fault-tolerance aspects of
reliability
![Page 7: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/7.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 7
Faults vs. Failures
Failures: perceived deviation of output values from expected values.
Faults: possible cause of failures in hardware, code or other artefacts.
For example, a faulty communication line may result in a communication failure.
Failures relative to system requirements (real-time: inacceptable communication delay „failure“).
![Page 8: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/8.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 8
Faults vs. Failures II
Faults in component cause failures of component. Are faults of subsystem containing component. Leads to failures of subsystem …
Faults / failures relative distinction. Can be both.
![Page 9: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/9.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 9
From UMLsec to UMLdep
Reliability = „Security against stupid adversaries“
Security = „Reliability for paranoids“
Adversaries in security correspond to failures in reliability.
Replace adversary model in UMLsec by failure model to get UMLdep.
![Page 10: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/10.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 10
Fault Semantics Modeling
For redundancy model R, stereotypes {∊ ¿crash/performanceÀ, ¿valueÀ}, have set FaultsR(s) {delay(⊆ t), loss(p), corrupt(q)}, with interpretation:
• t: expected maximum time delay, • p: probability that value not delivered within t,• q: probability that value delivered in time
corrupted(in each case incorporating redundancy).
Or use ¿riskÀ stereotype with {fault} tag.
![Page 11: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/11.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 11
Example
Suppose redundancy model R uses controller with redundancy 3 and fastest result. Then can define:
• delay(t): t delay of fastest controller, • loss(p): p probability that fastest result not
delivered within t,• corrupt(q): q probability that fastest result is
corrupted(each wrt. given fault semantics).
![Page 12: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/12.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 12
¿guaranteeÀDescribe guarantees required from
communication dependencies resp. system components.
Tags: {goal} with value subset of {immediate(t), eventual(p), correct(q)}, where
• t: expected maximum time delay, • p: probability that value is delivered within t,• q: probability that value delivered in time not
corrupted.
![Page 13: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/13.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 13
Reliable Architecture
Is this a reliable architecture ?
![Page 14: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/14.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 14
¿reliable linksÀPhysical layer should meet reliability requirements
on communication given redundancy model R.Constraint: For dependency d stereotyped
¿guaranteeÀ and each corresponding communication link l with stereotype s:
• if {goal} has immediate(t) as value then delay(t‘) 2 FaultsR(s) implies t‘·t,
• if {goal} has eventual(p) as value then loss(p‘) 2 FaultsR(s) implies p‘·1-p, and
• if {goal} has correct(q) as value then corruption(q‘) 2 FaultsR(s) implies q‘·1-q.
![Page 15: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/15.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 15
Example ¿reliable linksÀ
Given redundancy model none, ¿reliable linksÀ fulfilled iff T ≥ t where delay(t) 2 Faultsnone(¿crash/performanceÀ).
![Page 16: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/16.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 16
Reliable Data Structure
Assuming immediate(t) goals(realtime), data structure reliable ?
![Page 17: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/17.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 17
¿reliable dependencyÀ
Communication dependencies should respect reliability requirements on ¿criticalÀ data.
For each reliability level {l} for ¿criticalÀ data, have goals(l){immediate(t), eventual(p), correct(q)}.
Constraint: for each dependency d from C to D stereotyped ¿guaranteeÀ:
• Goals on data in D same as those in C.• Goals on data in C that also appears in D met
by guarantees of d.
![Page 18: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/18.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 18
Example ¿reliable dependencyÀ
Assuming immediate(t) goals(realtime), violates ¿reliable dependencyÀ, since Sensor and dependency do not provide realtime goal immediate(t) for measure() required by Controller.
![Page 19: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/19.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 19
¿reliable behaviorÀEnsures that system behavior in presence of failure
model provides required reliability {goals}:For any execution trace h, any transmission of a value
along a communication dependency stereotyped ¿guaranteeÀ, following constraints should hold, given the reliability goal:
• eventual(p): With probability at least p, …• immediate(t): … every value is delivered after at
most t time steps.• correct(q): Probability that a delivered value is
corrupted during transmission is at most 1-q.
![Page 20: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/20.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 20
Dependable Interference
Acceptable interference between safe and unsafe data ?
![Page 21: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/21.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 21
¿containmentÀ
Prevent indirect corruption of data.
Constraint:
Value of any data element d may only be influenced by data whose requirements attached to ¿criticalÀ imply those of d.
Make precise by referring to execution semantics (view of history associated with dependability level).
![Page 22: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/22.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 22
Example ¿containmentÀ
Violates containment because a {safe} value depends on un{safe} value.
Can check this mechanically.
![Page 23: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/23.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 23
Tool Support: Fault Models
lqln: messages on link l delayed further n time units.
phn: probability of fault at nth iteration in history h.
For link l stereotyped s where loss(p)FaultsR(s),
• history may give lql0:=; then append p to ph
n,• or no change, then append 1-p.For link l stereotyped s where corruption(q)FaultsR(s),
• history may give lql0:={■}; then append q,
• or no change; append 1-q. For link l stereotyped s with delay(t)FaultsR(s), and lql
0,
history may give lqln:=lql
0 for n·t; append 1/t.
Then distribute lql0; for each n, lql
n:=lqln+1.
![Page 24: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/24.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 24
Other Checks
Have other consistency checks such as
• Is the software‘s response to out-of-range values specified for every input ?
• If input arrives when it shouldn't, is a response specified ?
…and other safety checks from the literature.
![Page 25: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/25.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 25
Any Questions ?
![Page 26: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/26.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 26
Testing Real-Time Systems
Very challenging.For high level of assurance, would need full
coverage (test every possible execution).Usually infeasible (especially reactive systems).Have heuristics for trade-off between
development effort and reliability.Need to ask yourself:• How complete is the heuristic ?• How can I validate it ?
![Page 27: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/27.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 27
Recent Trends in Academic Research
Model-based Testing (e.g. based on Real-time UML). Advantages:
• Precise measures for completeness.
• Can be formally validated.
Two complementary strategies:
• Conformance testing
• Testing for criticality requirements
![Page 28: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/28.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 28
Conformance Testing
Classical approach in model-based test-generation (much literature).
Can be superfluous when using code-generation [except to check your code-generator, but only once and for all].
Works independently of real-time requirements.
![Page 29: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/29.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 29
Conformance Testing: Caveats
• Complete test-coverage still infeasible (although can measure coverage).
• Can only test code against what is contained in model. Usually, model more abstract than code. May lead to „blind spots“.
For both reasons, may miss critical test-cases. Want: „criticality testing“.
![Page 30: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/30.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 30
Criticality Testing: Strategies
Internal: Ensure test-case selection from models does not miss critical cases: Select according to information on criticality.
External: Test code against possible environment interaction generated from parts of the model (e.g. deployment diagram with information on physical environment).
![Page 31: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/31.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 31
Criticality Testing
Shortcoming of classical model-based test-generation (conformance testing) motivates „criticality testing“.
Goal: model-based test-generation adequate for critical real-time systems.
![Page 32: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/32.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 32
Internal Criticality Testing
Need behavioral semantics of used specification language (precise enough to be understood by a tool).
Here: semantics for simplified fragment of UML in „pseudo-code“ (ASMs).
Select test-cases according to criticality annotations in the class diagrams.
Test-cases: critical selections of intended behavior of the system.
![Page 33: Playing the Devil ’ s Advocate: Verifying Real-Time Systems Jan Jürjens Software & Systems Engineering TU Munich, Germany juerjens@in.tum.de](https://reader030.vdocuments.us/reader030/viewer/2022032800/56649d215503460f949f67a8/html5/thumbnails/33.jpg)
Jan Jürjens, TU Munich: Critical Systems Development with UML 33
External Criticality Testing
Generate test-sequences representing the
environment behaviour from the
criticality information in the deployment
diagrams.