automated generation of domain-specific graph models the...

Automated Generation of Domain-Specific Graph Models

The CORE-DISC Challenge

Dániel VarróMcGill University, Canada

Dept. of Measurement and Information Systems, BME MTA-BME Lendület Cyber-Physical Systems Research Group

IncQuery Labs Ltd.

Outline and Main Contributors

Graph Generator (BME)

• O. Semeráth

• K. Marussy

• G. Szárnyas

• G. Bergmann

• R. Farkas

Graph Generator (McGill)

• A. Babikian

• S. Pilarski

• B. Chen

• L. Li

• A. Li

• M. Ding

• V. Gidla

Lendület

• G. Bergmann

• A. Vörös

• M. Búr

• Cs. Debreceni

• R. Farkas

• B. Graics

• Á. Hajdú

• D. Honfi

• V. Molnár

• A. Nagy

• O. Semeráth

• G. Szárnyas

VIATRA

• I. Ráth

• Á. Horváth

• Á. Hegedüs

• Z. Ujhelyi

• G. Bergmann

• T. Szabó

• P. Lunk

• D. Segesdi

• I. Dávid

• I. Papp

• A. Nagy

• B. Grill

• D. Harmath

Graphs in Software Tools for Safety-Critical Systems

The Need for Automated Model Generators

Automated Generation of Consistent Graph Models

Automated Synthesis of Diverse Graph Models

GRAPHS IN SOFTWARE TOOLS FOR SAFETY-CRITICAL SYSTEMS

Graphs in Software Tools for Safety-Critical Systems

The Need for Automated Model Generators

Automated Generation of Consistent Graph Models

Automated Synthesis of Diverse Graph Models

Model-Based Systems Engineering

▪ Models for engineering of modern cars, aircrafts...

o Identify problems + optimize the system • Early in the design process

• Without physically building it

• Multidisciplinary models

• By simulations + testing + optimizationE.g. virtual crash tests, virtual test scenarios

o „Model-in-the-loop”• Virtual software crashes

Image source: http://virtualperformance.esi-group.com/

Motivation: Early validation of design rules

SystemSignalGroup design rule (from AUTOSAR)

o A SystemSignal and its group must be in the same IPdu

o Challenge: find violations quickly in large models

o New difficulties

• reversenavigation

• complexmanualsolution

AUTOSAR: • standardized SW architecture of the automotive industry

• now supported by modern modeling toolsDesign Rule/Well-formedness constraint: • each valid car architecture needs to respect• designers are immediately notified if violatedChallenge: • >500 design rules in AUTOSAR tools• >1 million elements in AUTOSAR models• models constantly evolve by designers

Similar challenges at: Thales, NASA JPL, CEA, Ericsson, ThyssenKrupp ...

Metamodels and (Instance) Models

Meta-model

Aggregation(Edge Type)

Class/(Node Type)

Attribute(Type)

Generalization

Reference(Edge Type)

Multiplicity

DerivedFeature

Enumeration(Kind)

Metamodels and (Instance) Models

Meta-model

Model

s1:Semaphore s2:Switch

s3:SegmentS4:Sensor

:between

:next

:sensor

:betweenObject(Node)

Link(Edge)

Mathematically:• Many engineering models ➔

typed, attributed graphs ➔algebraic or logic specs

Validation of Well-formedness Constraints

Meta-model

Model

pattern switchWOSignal(sw) {Switch(sw);neg find switchHasSignal(sw);

}

pattern switchHasSignal(sw) {Switch(sw);Signal(sig);Signal.mountedTo(sig, sw);

}

Query

Modify

User

Result

Graph Pattern Matching for Queries

▪ Match:

o m: L → G (graph morphism)

o CSP:

• Variables: Nodes of L

• Constraints: Edges of L

• Domain values: G

o Complexity: |G|^|L|

L

Gstraight

left

route: Route sp: SwitchPosition

switch: Switchsensor: Sensor

follows

switch

sensor

routeDefinition

All sensors with a switch that belongs to a route must directly be linked to the same route.

left

route: Route sp: SwitchPosition

switch: Switchsensor: Sensor

switchPosition

switch

sensor

routeDefinition

Graph Pattern Matching (Local Search)

▪ Search Plan:

o Select the first nodeto be matched

o Define an ordering ongraph pattern edges

▪ Search is restarted fromscratch each time

12

0

3

4

straight

left

route: Route sp: SwitchPosition

switch: Switchsensor: Sensor

switchPosition

switch

sensor

routeDefinition

Incremental Graph Pattern Matching

▪ Main idea: More space to less timeo Cache matches of patterns

o Instantly retrieve match (if valid)

o Update caches upon model changes

o Notify about relevant changes

▪ Approaches: o TREAT, LEAPS, RETE, …

o Tools: VIATRA, GROOVE, MoTE, TCore

straight

left

route sp switch sensor

r1 sp1 sw1

VIATRA: Open Source Software Project @Eclipse.org

▪ Incremental graph query engineo Declarative language

o Incremental graph queries

o Highly scalable

▪ Easy integration into toolso On-the-fly validation

o Derived features

o Custom views

o Traceability

▪ Reactive model transformation frameworko Event-based + reactive execution

o Internal DSL over Xtend

o Scalable M2M & M2T

▪ High-level featureso Complex event processing

o Design space exploration

o Reactive transformations

VIATRA An Eclipse project

Official Eclipse project3 Project leads 10+ Eclipse committers

Tool integration with: Papyrus UML, Sirius, RMF, Capella, ARTOP, mbeddr

http://eclipse.org/viatra

How to improve scalability of modeling tools?

Industrial use at: Thales, CEA, ThyssenKrupp, Ericsson, Embraer, NASA, CERN, …

Industrial Applications of VIATRA

THE NEED FOR AUTOMATED GRAPH MODEL GENERATORS

Graphs in Software Tools for Safety-Critical Systems

The Need for Automated Model Generators

Automated Generation of Consistent Graph Models

Automated Synthesis of Diverse Graph Models

How to Validate Software Tools?

High-Level Requirements

Low-Level Requirements(System Spec)

System Architecture

Software Component

Design

Implementation (Source Code)

Software ComponentVerification

System Integration & Verification

System Validation

Final Acceptance Test

Verification tools:• fail to detect errors

Development tools:• input ➔ output deterministically• introduce new errors

Promises of Tool Qualification• reduce development + V&V cost • increase quality and productivity➔ reduce certification costs

Obstacles for Tool Qualification➔ extreme qualification costs• complex V&V for graph models? • graph models as test input?

How to systematically test development and verification tools used for critical systems?

Automated Model Generation of Test Context

▪ R3COP & R5COP EU projects:Testing of autonomous robots

o Generate diverse test context(with various obstacles)

o Test: Navigable by autonomous robot?

Related challenge: How to generate test scenarios for safety assurance of autonomous vehicles?

Towards Automated Graph Generators

• Correct: All (well-formedness) constraints are satisfied

• Complete: All (and only) consistent models are derivedCOnsistent

• Cannot be distinguished from a real graph model

• By removing text+values and evaluating graph metricsREalistic

• Structural diversity within a single graph

• Large distance between any pair of graph modelsDIverse

• In size: the size of the graph grows

• In quantity: generation time of next graph is stableSCalable

How to automatically synthesize graph models which are...

ICSE’18 ICSE’19

MODELS’16 ICSE’20

FASE’18 STTT

AUTOMATED GENERATION OF CONSISTENT GRAPH MODELS

Software Tools for Safety-Critical Systems

The Need for Automated Model Generators

Automated Generation of Consistent Graph Models

Automated Synthesis of Diverse Graph Models

Input of Model Generation: Domain Specification

21

Entry State

«abstract»

VertexTransitionsrc [1..1]

trg [1..1]

▪ Concepts + Relations

▪ Basic graph structure

▪ Statechart: Transitions, Entries, States, source, target

Metamodel

WF1: TrgToEntry WF2: NoSrcFromEntry WF3: MultipleSrc

e: Entry

t:Transition

trg

e: Entry

t:Transition

src

NEG

e: Entry

t1 t2

src src

t1 ≠ t2

▪ First order graph predicates (Graph Patterns / OCL)

▪ Entry is invalid if:▪ WF1: Has incoming transition

▪ WF2: Has no outgoing transition

▪ WF3: Multiple outgoing transitions

Well-formedness constraints

Domain Specification: Metamodel + Constraints

• Capella (Thales)• Artop (AUTOSAR)• Yakindu (Itemis)• MagicDraw (NoMagic)• Papyrus

Domain Specification

Output of Consistent Model Generation: Models

22

Metamodel

Entry State

«abstract»

VertexTransitionsrc [1..1]

trg [1..1]

Well-formedness constraintsWF1: TrgToEntry WF2: NoSrcFromEntry WF3: MultipleSrc

e: Entry

t:Transition

trg

e: Entry

t:Transition

src

NEG

e: Entry

t1 t2

src src

t1 ≠ t2

Instance Models

𝑴𝟏

s1:State

e1:Entry

t1:Transition

t2:Transition

src

trg

src trg

ModelGenerator

𝑴𝒏+𝟏

𝑴𝟐

s2:State

s1:State

t2:Transition

t3:Transition

src

trg

src

trg

𝑴𝟑

s2:State

s1:State

t1:Transition

t2:Transition

src

trg

src trg

SAT

UNSAT

Language Specification

Model Generation Setup: Logic Solver

23

Metamodel

Entry State

«abstract»

VertexTransitionsrc [1..1]

trg [1..1]

Well-formedness constraintsWF1: TrgToEntry WF2: NoSrcFromEntry WF3: MultipleSrc

e: Entry

t:Transition

trg

e: Entry

t:Transition

src

NEG

e: Entry

t1 t2

src src

t1 ≠ t2

Instance Models

𝑴𝟏

s1:State

e1:Entry

t1:Transition

t2:Transition

src

trg

src trg

ModelGenerator

𝑴𝒏+𝟏

𝑴𝟐

s2:State

s1:State

t2:Transition

t3:Transition

src

trg

src

trg

𝑴𝟑

s2:State

s1:State

t1:Transition

t2:Transition

src

trg

src trg

SAT

UNSA

T

Challenge 1:Existing solver-based techniques are unable to generate models for industrial modeling languages

• Alloy (MIT): 50-100 objects• Z3 (Microsoft): 10-12 objects

ModelGenerator

Logic SolverAlloy: SAT (MIT)Z3: SMT (Microsoft)

Challenge 2:Logic solvers tend to create similar, highly symmetric models (Copy-Paste)

• No diversity guarantee• Biased sampling

Graph solver

Approach: create a logic solver that operates natively over graphs

Challenges in Model Generation

▪ Representing graphs as predicates introduce a large number of variableso 1 variable for each object and type (Transition, Vertex Entry, State)

o 1 variable for each object pair and reference type (src, trg)

o For 100 objects: more than 20k Boolean variables

▪ Quantified well-formedness constraints areunfolded into complex FOL constraintso Each quantified variable is checked for each object

o ¬∃𝑒, 𝑡: Entry 𝑒 ∧ Transition 𝑡 ∧ src 𝑡, 𝑒 →

30k atomic expressions for 100 objects

▪ Existing solvers fail to generate graphs with more than 100 nodes

24

1 2

3 4

SAT Solver Overview: DPLL Algorithm

▪ DPLL: Well-known SAT-algorithm, basis of most modern solvers(Davis–Putnam–Logemann–Loveland)

▪ Refines partial variable binding

▪ Decision rules +Unit propagation

▪ Search Strategy:Backtracking +Backjumping +Random restarts

25

[1,_,_,_]

[1,1,0,_] [1,0,0,_] [1,0,1,1]

(A ∨ B ∨ C)∧(¬C ∨ B ∨ D)∧(¬A ∨ B ∨ C)∧(¬A ∨ ¬B ∨ ¬C)

[1,_,1,_][1,_,0,_] [1,_,1,1]

Our approach: Boolean variables→ Graphs

Overview: 3-Valued Partial Models as States

26

Entry Transitionsrc

«new»trg

srctrg

~

trg

Entry Transitionsrc

«new»

srctrg

~

trg

State

src,trg

~

Entry Transitionsrc

«new»srctrg

trg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

trg

Graph Solver:

1) Based on classic SAT-algorithm

2) Refinement of3-valued partial graph models

3-Valued Partial Models as States

27

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

trgEntry Transition

src

«new»

srctrg

~

trg

State

src,trg

~

Entry Transitionsrc

«new»srctrg

trg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

Entry Transitionsrc

«new»trg

srctrg

~

trgGraph Solver:

1) Based on classic SAT-algorithm

2) Refinement of3-valued partial graph models

▪ Uncertain properties areexplicitly represented:1 | 0 | ½: Unknown

½ edge

½ equivalence½ node existence

Partial Model Refinement

29

State

src,trg

~

Entry Transitionsrc

«new»srctrg

trg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

Entry Transitionsrc

«new»trg

srctrg

~

trgGraph Solver:

1) Based on classic SAT-algorithm

2) Refinement of3-valued partial graph models

▪ Uncertain properties areexplicitly represented:1 | 0 | ½: Unknown

▪ Generation as monotonous partial model refinement:½ → 1|0

▪ Decision +Unit prop. →Graph Transformation

Entry Transitionsrc

«new»

srctrg

~

trg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

trg

Uncertain edge refinement:trg(Entry, Transition) → 1

trg(Entry, new) → 0

new→new+StateUncertain equivalence refinement: Splittingnew → new + State

State«new»

Approximated Constraint Evaluation

30

Entry Transitionsrc

«new»trg

srctrg

~

trg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

trg

State

src,trg

~

Entry Transitionsrc

«new»srctrg

trg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

Graph Solver:

1) Based on classic SAT-algorithm

2) Refinement of3-valued partial graph models

3) Constraint evaluation by incremental graph query engine

▪ Constraint evaluated onpartial solutions

▪ Monotonousreasoning

▪ Incremental constraintreevaluation

Entry Transitionsrc

«new»

srctrg

~

trg

WF1: TrgToEntry

e: Entry

t:Transition

trg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

Constraint evaluation respects refinement:

WF1 is violated in P ⇒WF1 will be violated in all refinements of P

Approximated constraint evaluation is monotonous (despite the use of negation)

Overview of Refinement and Approximation

▪ Overview of refinement & constraint approximation

φ[P] φ[Q] (with P≻Q)

Action

1 1 Inconsistent, backtrack

0 0 Consistent

½ 1 Inconsistent, backtrack

½ 0 Consistent(corrected)

½ ½ Unknown

31

M0

½

P2

½

Q1

½

R1

½

Q1

1

P1

1

P3

0

Equivalence Partitioning

32

Entry Transitionsrc

«new»trg

srctrg

~

trg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

trgEntry Transition

src

«new»

srctrg

~

trg

State

src,trg

~

Entry Transitionsrc

«new»srctrg

trg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

Graph Solver:

1) Based on classic SAT-algorithm

2) Refinement of3-valued partial graph models

3) Constraint evaluation by incremental graph query engine

4) Equivalencedetectionby graphisomorphism

▪ State encoding

Equivalence Partitioning

33

Entry Transitionsrc

«new»trg

srctrg

~

trg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

trgEntry Transition

src

«new»

srctrg

~

trg

State

src,trg

~

Entry Transitionsrc

«new»srctrg

trg

State

src,trg

~

Entry Transitionsrc

trg

«new»srctrg

Graph Solver:

1) Based on classic SAT-algorithm

2) Refinement of3-valued partial graph models

3) Constraint evaluation by incremental graph query engine

4) Equivalencedetectionby graphisomorphism

▪ State encoding

▪ Partial order reduction

Different Solutions

VIATRA Solver: An Open Source Tool

▪ Standard EMF as input and output| Configuration language | Visualization

▪ Incremental Query Engine:o Constraint language: VIATRA Query

o Internally uses: Incremental constraint reevaluation, DPLL as VIATRA DSE

▪ Open source: github.com/viatra/VIATRA-Generator

36

Scalability Measurements

Maximal model size

37

FAM: Industrial, Avionics

FS: File System example of Alloy

5 min timeout

Example comparison (FAM)

Yakindu: Industrial, Statemachine

Ecore: Metamodelling language

Our solver generates two orders of magnitude larger models

GENERATION OF DIVERSE GRAPH MODELS

38

Graphs in Software Tools for Safety-Critical Systems

The Need for Automated Model Generators

Automated Generation of Consistent Graph Models

Automated Synthesis of Diverse Graph Models

Conclusions

THANKS FOR YOUR ATTENTION

Links to tools:

VIATRA: https://www.eclipse.org/viatra/

VIATRA Generator: https://github.com/viatra/VIATRA-Generator