validation and verification - its-mobility.de · introduction − using models and (semi-)formal...
TRANSCRIPT
![Page 1: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/1.jpg)
Validation and
verification of
specification models
Test4Rail, Braunschweig
Dr. Oliver Lemke
V2.0
![Page 2: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/2.jpg)
Agenda
− Introduction
− Needs
− Process
− Conclusion
18.10.2017 2
![Page 3: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/3.jpg)
18.10.2017 3
SIGNON business activities
Planning Technical Consulting Engineering
Signalling systems
Telecommunications
Power supply
Systems
Software
Safety
Studies
Methodology
Processes
![Page 4: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/4.jpg)
Introduction
− Using models and (semi-)formal languages like SysML have become more common
over the last years for specifying railway systems:
▪ NeuPro – DB’s standardisation of interlocking architecture in Germany
▪ EULYNX – European counterpart
− EULYNX (https://www.eulynx.eu) is an initiative of 12 European infrastructure
managers (IMs) to harmonize interlocking architectures
− This is accomplished by creating unified operator’s specification documents for the
supply industry
18.10.2017 4
![Page 5: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/5.jpg)
NO_OPERATING_VOLTAGE
BOOTING FALLBACK_MODE
OPERATIONAL
INITIALISING
OPERATING_VOLTAGE_SUPPLIED
F_EST_SubS_TDS - Behaviour [SubSTDS STD1]
when( T1_P ower_On_detec ted )/when( T2_P ower_Off_detec ted )/
when( T5_S IL_not_fulfilled )/
when( T3_Res et )/
when( T5_SIL_not_fulfilled )/
T12_Interrupt_Safe_Communication_Protocol_Connection := true;
when( T3_Res et )/
when( T4_Booted )[D13_S tart_Tim e_s ync hronisation_is_initiated]/
when( T9_Proces s_Data_Interface_connec tion_established )/
when( T5_SIL_not_fulfilled )/
T12_Interrupt_Safe_Communication_Protocol_Connection := true;
when( T3_Res et )/
when( T7_Invalid_or_m iss ing_Configurat ion_Data_carrier )/
when( T10_Safe_Communication_Protocol_Connection_disconnected
)/
SysML state machine diagram Simulation interface
Specification document
18.10.2017 5
![Page 6: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/6.jpg)
Agenda
− Introduction
− Needs
− Process
− Conclusion
18.10.2017 6
![Page 7: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/7.jpg)
Needs
− The claim is, that model-based, (semi-)formal specifications improve the
specification quality by:
▪ Being correct
▪ Being consistent
▪ Being unambiguous
− But this is only true if, the underlying model is unambiguous, consistent and correct
− As models can be reused for system acceptance tests, integration tests and various
simulations, these requirements for model quality are even aggravated.
Hence assuring a high level of quality for the models becomes essential.
18.10.2017 7
![Page 8: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/8.jpg)
Agenda
− Introduction
− Needs
− Process
− Conclusion
18.10.2017 8
![Page 9: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/9.jpg)
Process - simplified CENELEC V-model
Implementation
phase
Validation
Verification
P1
P5
18.10.2017 9
P9
![Page 10: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/10.jpg)
Process - small V-model for specification phase
State machine (STM)
implementation
STM
acceptance
Validation
Verification
User
requirements
Formalised
requirements
18.10.2017 10
![Page 11: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/11.jpg)
User
Reqs. Formalised reqs.
State machine
implementation
State machine
acceptance
System Env.
Stimulus A
Response a
Response d
Stimulus B
Stimulus C
Scenarios as SysML
sequence diagrams (ca. 20 –
50 scenarios per subsystem)
NO_OPERATING_VOLTAGE
BOOTING FALLBACK_MODE
OPERATIONAL
INITIALISING
OPERATING_VOLTAGE_SUPPLIED
F_EST_SubS_TDS - Behaviour [SubSTDS STD1]
when( T1_P ower_On_detec ted )/when( T2_P ower_Off_detec ted )/
when( T5_S IL_not_fulfilled )/
when( T3_Res et )/
when( T5_SIL_not_fulfilled )/
T12_Interrupt_Safe_Communication_Protocol_Connection := true;
when( T3_Res et )/
when( T4_Booted )[D13_S tart_Tim e_s ync hronisation_is_initiated]/
when( T9_Proces s_Data_Interface_connec tion_established )/
when( T5_SIL_not_fulfilled )/
T12_Interrupt_Safe_Communication_Protocol_Connection := true;
when( T3_Res et )/
when( T7_Invalid_or_m iss ing_Configurat ion_Data_carrier )/
when( T10_Safe_Communication_Protocol_Connection_disconnected
)/
SysML state machines in
modelling tool Executable simulator
Know-
ledge
Informal
documents
Creation - Modeller
Verification - Tester
Validation - Stakeholder
18.10.2017 11
![Page 12: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/12.jpg)
Process - verification
Verification step 1 (Black-Box-Verification):
− Verify that state machines (STM) react as specified in sequence diagrams (SD)
− Implemented by stimulating the executable simulator according to the SDs and
reading back its responses, comparing them to the responses defined in the SDs
− Test execution is highly automated through GUI testing tools (e.g. Ranorex)
Verification step 2 (White-Box-Verification):
- Verify that STM does not add implicit behaviour not specified in sequence diagrams
- Checked by verifying that all SDs fully cover the STM according to defined coverage
criteria, e.g. full state and transition coverage
- Unmarked states and transitions are not covered by sequences and therefore describe
additional behaviour
- Generate SDs covering the missing elements in STM and discuss with stakeholder
18.10.2017 12
![Page 13: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/13.jpg)
NO_OPERATING_VOLTAGE
BOOTING FALLBACK_MODE
OPERATIONAL
INITIALISING
OPERATING_VOLTAGE_SUPPLIED
F_EST_SubS_TDS - Behaviour [SubSTDS STD1]
when( T1_P ower_On_detec ted )/when( T2_P ower_Off_detec ted )/
when( T5_S IL_not_fulfilled )/
when( T3_Res et )/
when( T5_SIL_not_fulfilled )/
T12_Interrupt_Safe_Communication_Protocol_Connection := true;
when( T3_Res et )/
when( T4_Booted )[D13_S tart_Tim e_s ync hronisation_is_initiated]/
when( T9_Proces s_Data_Interface_connec tion_established )/
when( T5_SIL_not_fulfilled )/
T12_Interrupt_Safe_Communication_Protocol_Connection := true;
when( T3_Res et )/
when( T7_Invalid_or_m iss ing_Configurat ion_Data_carrier )/
when( T10_Safe_Communication_Protocol_Connection_disconnected
)/
18.10.2017 13
![Page 14: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/14.jpg)
Process - validation
Know-
ledge
Informal
documents
Operational stakeholders validate the STM against the input documents by using their own
test cases. This assures diversity and coverage of domain knowledge.
18.10.2017 14
![Page 15: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/15.jpg)
Agenda
− Introduction
− Needs
− Process
− Conclusion
18.10.2017 15
![Page 16: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/16.jpg)
Experiences
Statistics for a SysML model of the interface between interlocking and axel counting
system.
Model
− Number of SDs as formalised requirements: 45
− Number of states in the STMs: 15
− Number of transitions in the STMs: 38
Detected errors (after multiple manual reviews and quality checks)
− Verification - functional errors (STM does not match SDs): 5
− Verification - implicit behaviour (STM contains behaviour not specified in SDs): 3
− Validation - functional errors: 3
18.10.2017 16
![Page 17: Validation and Verification - its-mobility.de · Introduction − Using models and (semi-)formal languages like SysML have become more common over the last years for specifying railway](https://reader031.vdocuments.us/reader031/viewer/2022011803/5b8a9ee77f8b9a655f8ec99e/html5/thumbnails/17.jpg)
Conclusion
− The process presented is able to improve the specification quality
− The quality of model-based specifications is typically higher than of text-based
specifications
− The additional benefit of using formal verification techniques must be evaluated, as
the efforts for applying them in the real world are still very high
Thank you!
18.10.2017 17