mateus araújo borges - ufpe
TRANSCRIPT
Mateus Araújo Borges
TECHNIQUES TO FACILITATE PROBABILISTIC SOFTWARE
ANALYSIS IN REAL-WORLD PROGRAMS
M.Sc. Dissertation
Federal University of [email protected]
www.cin.ufpe.br/~posgraduacao
RECIFE2015
Mateus Araújo Borges
TECHNIQUES TO FACILITATE PROBABILISTIC SOFTWAREANALYSIS IN REAL-WORLD PROGRAMS
A M.Sc. Dissertation presented to the Center for Informatics
of Federal University of Pernambuco in partial fulfillment
of the requirements for the degree of Master of Science in
Computer Science.
Advisor: Marcelo Bezerra d’Amorim
RECIFE2015
Catalogação na fonteBibliotecária Joana D’Arc Leão Salvador CRB4-532
B732t Borges, Mateus Araújo. Techniques to facilitate probabilistic software analysis in real-world
programs / Mateus Araújo Borges. – Recife: O Autor, 2015. 63 f.: fig., tab.
Orientador: Marcelo Bezerra d’Amorim. Dissertação (Mestrado) – Universidade Federal de Pernambuco. CIN,
Ciência da Computação, 2015. Inclui referências.
1. Engenharia de software. 2. Monte Carlo – método de. I. Amorim, Marcelo Bezerra d' (Orientador). II. Titulo.
005.1 CDD (22. ed.) UFPE-MEI 2015-074
Dissertação de Mestrado apresentada por Mateus Araújo Borges à Pós Graduação em
Ciência da Computação do Centro de Informática da Universidade Federal de
Pernambuco, sob o título “Techniques to Facilitate Probabilistic Software Analysis of
Realworld Programs”, orientada pelo Prof. Marcelo Bezerra d'Amorim e aprovada
pela Banca Examinadora formada pelos professores:
______________________________________________ Prof. Alexandre Cabral Mota Centro de Informática/UFPE ______________________________________________ Prof. Fernando Magno Quintão Pereira Departamento de Ciência da Computação / UFMG _________________________________________________ Prof. Marcelo Bezerra d'Amorim Centro de Informática / UFPE Visto e permitida a impressão. Recife, 24 de abril de 2015. ___________________________________________________ Profa. Edna Natividade da Silva Barros Coordenadora da PósGraduação em Ciência da Computação do Centro de Informática da Universidade Federal de Pernambuco.
Acknowledgements
I would like to express my thanks to:
� My advisor, Marcelo d’Amorim, for supporting (and putting up with) me for the pastfive years. I hope he finds a better student in the future :)
� FACEPE, for funding my studies.
� My friends, for keeping me sane.
� My parents, José Carlos and Iolanda, for their unconditional love and support.
� Everyone else who helped me in this journey.
—A MAN MUST STAND ERECT, NOT BE KEPT ERECT BY OTHERS.(Marcus Aurelius)
List of Acronyms
HM-MC Hit-or-Miss Monte Carlo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
ICP Interval Constraint Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
PSA Probabilistic Software Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
SPF Symbolic PathFinder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
List of Figures
1.1 Flow of a generic PSA tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1 Code fragment and the corresponding symbolic execution tree. Labeled edgesrepresent branch decisions, and dashed boxes represent the end of the branch. . 18
2.2 Probabilistic software analysis chain with qCORAL. . . . . . . . . . . . . . . . 192.3 Example illustrating interval constraint propagation with stratified sampling. The
boxes returned by the Interval Constraint Propagation (ICP) are shown in red. . 22
4.1 Flap controller. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.1 Convergence rate of iterative sampling and the baseline approach on selectedsubjects (y-axis in logarithmic scale). . . . . . . . . . . . . . . . . . . . . . . 54
List of Tables
2.1 Variance reduction for the example case. . . . . . . . . . . . . . . . . . . . . . 22
3.1 Microbenchmarks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.2 Characterization of the VolComp benchmark. . . . . . . . . . . . . . . . . . . 323.3 Comparison of NIntegrate (default numerical integration method from Math-
ematica [2]), VolComp, and qCORAL with features STRAT and PARTCACHEenabled. Note that the comparison is restricted to linear constraints and thecomparison metrics are not the same. . . . . . . . . . . . . . . . . . . . . . . . 33
3.4 Comparison of different configurations of qCORAL on different sampling rates. 35
4.1 Characterization of the qCORAL Benchmark . . . . . . . . . . . . . . . . . . . 414.2 Distribution-aware sampling: comparison of NProbability and qCORAL. . 434.3 Comparison of different discretization methods. Discretization invokes
qCORAL [8] once for every (discretized) region within each constraint partition.For every subject, two input variables are normally distributed and the others areuniform. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5.1 Time to reach a target accuracy for incremental sampling techniques and baselineapproach for 1k and 10k sampling budget per iteration. Initial uniform samplingbootstrap was 50k. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Resumo
Análise Probabilística de Software (PSA) visa a quantificar a probabilidade de que umevento de interesse seja alcançado durante a execução de um programa, dada uma caracterizaçãoprobabilística do comportamento do programa ou do seu ambiente de execução. O eventode interesse pode ser, por exemplo, uma exceção não capturada, a invocação de um métodoespecífico, ou o acesso à informação confidencial. A técnica coleta restrições sobre as entradasque levam para os eventos de interesse e as analisa para quantificar o quão provável que umaentrada satisfaça essas restrições. Técnicas atuais ou suportam apenas restrições lineares, ousuportam distribuições contínuas utilizando uma "discretização" do domínio de entrada, levandoa resultados imprecisos e caros.
Este trabalho apresenta uma abordagem iterativa, composicional e sensível às dis-tribuições para suportar o uso de PSA em restrições com operações matemáticas arbitrariamentecomplexas e distribuições contínuas de entrada. Nossa abordagem composicional permite que asrestrições sejam decompostas em subproblemas que podem ser resolvidos independentemente.Em cada iteração a análise é reorientada automaticamente para a estimação dos subproblemas quemais afetam a precisão dos resultados, assim aumentando a taxa de convergência da computação.Esta reorientação é guiada por três diferentes estratégias de ranqueamento. Experimentos emprogramas publicamente disponíveis mostram que a técnica proposta é melhor do que abordagensexistentes em termos de escalabilidade e precisão.
Palavras-chave: Execução Simbólica. Amostragem de Monte Carlo. Análise Probabilística.Testes.
Abstract
Probabilistic software analysis aims at quantifying how likely a target event is to occur,given a probabilistic characterization of the behavior of a program or of its execution environment.Examples of target events may include an uncaught exception, the invocation of a certain method,or the access to confidential information. The technique collects constraints on the inputs thatlead to the target events and analyzes them to quantify how likely it is for an input to satisfy theconstraints. Current techniques either handle only linear constraints or only support continuousdistributions using a “discretization” of the input domain, leading to imprecise and costly results.
This work proposes an iterative distribution-aware sampling approach to support prob-abilistic symbolic execution for arbitrarily complex mathematical constraints and continuousinput distributions. We follow a compositional approach, where the symbolic constraints aredecomposed into sub-problems whose solution can be solved independently. At each iterationthe convergence rate of the computation is increased by automatically refocusing the analysison estimating the sub-problems that mostly affect the accuracy of the results, as guided bythree different ranking strategies. Experiments on publicly available benchmarks show that theproposed technique improves on previous approaches in terms of scalability and accuracy of theresults.
Keywords: Symbolic Execution. Monte Carlo Sampling. Probabilistic Analysis. Testing.
Contents
1 Introduction 131.1 Problem Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.2 Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.3 Solution overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151.4 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151.5 History of Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161.6 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2 Background 172.1 Symbolic Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.2 Interval Constraint Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . 182.3 Probabilistic Software Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.3.1 Computing Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . 202.3.2 Monte Carlo Simulation . . . . . . . . . . . . . . . . . . . . . . . . . 202.3.3 Interval Constraint Propagation and Stratified Sampling . . . . . . . . 21
3 Compositional Solution Space Quantification 233.1 Disjunction of Path Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . 233.2 Conjunction of Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.5 Algorithms and Implementation . . . . . . . . . . . . . . . . . . . . . . . . . 283.6 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.6.1 RQ1: What is the accuracy of qCORAL estimates? . . . . . . . . . . . 303.6.2 RQ2: How qCORAL compares with Mathematica and VolComp? . . . 313.6.3 RQ3: How the different features of qCORAL affect accuracy and time
when handling complex constraints? . . . . . . . . . . . . . . . . . . . 35
4 Distribution-Aware Sampling 374.1 Illustrative Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.2 Technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.2.1 Distribution-aware versus discretization-based approaches . . . . . . . 404.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.3.1 Comparison with NProbability . . . . . . . . . . . . . . . . . . . . . . 424.3.2 Comparison with Discretization . . . . . . . . . . . . . . . . . . . . . 44
5 Iterative Approaches for Optimal Sampling Allocation 465.1 Summary of the Technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465.2 Gradient-descent variance minimization . . . . . . . . . . . . . . . . . . . . . 47
5.2.1 Choosing the step size. . . . . . . . . . . . . . . . . . . . . . . . . . . 515.2.2 ICP-based stratified sampling. . . . . . . . . . . . . . . . . . . . . . . 51
5.3 Sensitivity analysis and computational overhead . . . . . . . . . . . . . . . . . 525.4 Local heuristic for sampling allocation . . . . . . . . . . . . . . . . . . . . . . 525.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.5.1 Convergence rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.5.2 Time to converge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
6 Conclusion 576.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 576.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
References 60
131313
1Introduction
The pervasiveness of software, the heterogeneity of its users, and the growing complexityof interactions with third-party components are introducing a new degree of uncertainty aboutthe execution conditions of a program. This raises the quest for verification techniques able todeal with and to quantify such uncertainty, both in the problem and in the verification results.
One approach that can be used to analyze such systems is Probabilistic Model Check-
ing [18, 32] (PMC), a formal verification technique that has been rising in importance. It consistsin creating a probabilistic model of the desired system (usually with Markov chains or somevariant of probabilistic automata), and then performing a mathematical analysis of this model toverify properties of the original system. PMC is able to determine quantitative properties, likeperformance and reliability, and has been used successfully applied to multiple case studies in alarge range of domains, such as communication protocols and power management of hardwarecomponents. However, the applications of PMC tend to be restricted to the early stages ofsoftware construction process. Models need to be kept consistent with the implementation, whichcan be hard and expensive.
Probabilistic Software Analysis (PSA) attempts to address this limitation with techniquesthat operate directly on code artifacts [8, 20, 22, 47]. PSA aims at quantifying the probabilityof a target event to occur, given a probabilistic characterization of the behavior of a programor of its execution environment. Examples of target events include an uncaught exception, theinvocation of a certain method, or the access to confidential information.
1.1 Problem Overview
Figure 1.1 shows the toolchain adopted by recent PSA approaches [8, 20, 22, 47]. First,the paths of the program leading to the occurrence of the target event are collected with symbolic
execution, a program analysis technique that execute programs with symbolic inputs to explorethe possible execution paths. The collected paths are stored as path conditions (PC), whichare constraints over the symbolic input that characterize the inputs that follow that path. ThePCs are then analyzed (in the “Probabilistic Analysis” component) to quantify how likely is an
1.2. CHALLENGE 14
input, distributed according to certain usage profiles [20], to satisfy any of them. In this context,satisfaction of one individual path constraint implies the occurrence of the event of interest. Theoutput is the probability of the occurrence of the target event and the accuracy.
Figure 1.1: Flow of a generic PSA tool
1.2 Challenge
The quantification of the solution space for a set of constraints is one of the main obstaclesto the applicability of probabilistic software analysis in practice. To calculate the probabilityof a path constraint being satisfied, we need to compute (or estimate) the number of solutionsfor that PC. In previous studies [20, 22], model counting techniques have been applied to countthe number of points of a bounded integer domain that satisfy given linear constraints. Thesecounts are then coupled with a probabilistic input usage profile, which establishes a probabilitydistribution over the input domain, to assess the probability for the target event to occur in thatusage profile. Sankaranarayanan et al. [47] proposed a different approach to quantify the solutionspace of linear constraints over bounded floating-point domains. They propose an iterativealgorithm to compute tight over-approximating bounds of the actual solution space suitable forefficient volume computation. All these approaches are thus quite limited as they can only handlelinear constraints.
When considering floating-point domains, the quantification of the solution space for aconstraint usually involves the computation of an integral. Symbolic and numerical integrationare usually impracticable in the case of complex mathematical constraints and multi-dimensionalinput domains due to the high computational time and memory demands of such methods [29].Furthermore, despite the fact they could be able to provide exact results, symbolic methodscannot deal with problems whose solution cannot be expressed in analytical form, which is thecase for many integration problems [27]. Statistical methods overcome the limitations in terms offeasibility and memory demand of both symbolic and numerical methods, allowing to deal witharbitrarily complex constraints. However, statistical methods are based on simulation and canonly provide approximate results. While the accuracy of the results can be arbitrarily improved,
1.3. SOLUTION OVERVIEW 15
the price to be paid is the increased number of simulations, which increases the analysis time,possibly making it unreasonably long.
1.3 Solution overview
This dissertation presents qCORAL, a distribution-aware, compositional statistical ap-
proach for the efficient quantification of solution spaces for arbitrarily complex (non-linear,containing transcendental functions, etc.) constraints over bounded floating-point domains.qCORAL takes as input a disjunction of path conditions plus the usage profile and, based onthat, it computes the probability of the event to occur. It focuses on the last processor in thefigure pipeline, the “Probabilistic Analysis” component. qCORAL is not dependent on anyprogramming language.
qCORAL follows a layered compositional strategy. It splits up the problem of estimatingthe solution space of a set of symbolic conditions (describing the program paths leading to theoccurrence of the target event) into the problem of estimating the solution space of individual pathconditions. qCORAL further decomposes each path condition into a set of independent clausesthat can be analyzed separately. It then composes the estimated results back together. Thisdivide-and-conquer strategy speeds up the analysis by reducing large problems into sub-problemsthat are easier to analyze, and also allows the reuse of partial results for clauses appearing inseveral constraints. For each independent clause, qCORAL further uses an off-the-shelf intervalconstraint solver [26] to break the solution spaces into even smaller regions, whose unionnecessarily bounds the solution space of the clause. qCORAL then uses stratified sampling [46],a well known technique for speeding up the convergence of Monte Carlo simulations, to analyzethe data from the independent regions, and to compose the results. qCORAL can handle arbitraryunivariate continuous input distributions and can iteratively re-focus the sampling effort on theconstraints that have higher impact on the accuracy of the result.
1.4 Contribution
The contributions of this work are summarized as follows:
1. Approach. qCORAL combines synergistically multiple individual techniques toincrease the accuracy of the results and speed up the quantification of constraints.When possible, we provide a formal assessment of those techniques.
2. Implementation. We developed an implementation of qCORAL in Java, which canbe downloaded at the homepage of the project 1. The source code is available uponrequest.
1http://pan.cin.ufpe.br/QCoral.html
1.5. HISTORY OF PUBLICATIONS 16
3. Evaluation. We evaluated qCORAL experimentally on publicly available bench-marks, including real world software taken from the medicine and the aerospacedomains.
1.5 History of Publications
Some of the ideas applied in qCORAL came from a previous work, CORAL2 [7, 48]: ameta-heuristic constraint solver for dealing with numerical constraints involving floating-pointvariables and mathematical functions. CORAL was developed in collaboration with MatheusSouza, Marcelo d’Amorim (Federal University of Pernambuco, Brazil), Corina S. Pasareanu(CMU SV/NASA Ames Research Center, USA), Saswat Anand (Stanford University, USA), andDavid Bushnell (NASA Ames Research Center).
qCORAL was developed in collaboration with Antonio Filieri (University of Stuttgart,Germany), Corina S. Pasareanu, Marcelo d’Amorim and Willem Visser (University of Stellen-bosch, South Africa). Together, we submitted a paper to the ACM SIGPLAN ProgrammingLanguage Design and Implementation (PLDI 2014).
1.6 Outline
The remainder of this document is organized as follows. Chapter 2 presents an overviewof the background knowledge used in the research and development of this work. Chapter 3describes the compositional approach used in qCORAL. The next two chapters present twoimprovements: a distribution-aware sampling procedure (Chapter 4) and an iterative approachto focus sampling efforts on the most important subproblems (Chapter 5). Finally, Chapter 6discusses related work and our final conclusions.
2http://pan.cin.ufpe.br/coral/
171717
2Background
In this chapter, we introduce the key ideas and technologies which upon our work isbuilt:
� Symbolic Execution (covered in Section 2.1).
� Interval Constraint Propagation (covered in Section 2.2).
� Probabilistic Software Analysis (covered in Section 2.3).
2.1 Symbolic Execution
Symbolic Execution [12, 13, 30] is a program analysis technique which executes pro-grams on unspecified inputs by using symbolic inputs instead of concrete data. The analysiskeeps a symbolic state (σ ), which maps variables to expressions over the symbolic inputs, anda path condition PC, which is a conjunction of boolean conditions characterizing the inputsthat follow that path. The symbolic state is updated every time the execution of a statementresults in a modification of a memory location. When a conditional statement with predicate q isexecuted, the symbolic execution forks and follow both branches. At this point, PC is updated toPC∧q, representing the branch where q is true, and a new PC′ is created with the value PC∧¬q,representing the branch where q is false. If the end of a branch is reached, or if the currentPC becomes unsatisfiable, the analysis backtracks. Satisfiability of the PCs is checked usingoff-the-shelf solvers.
The execution paths followed during the symbolic execution of a program are charac-terized by a symbolic execution tree. The nodes represent program (symbolic) states and thearcs represent transitions between states. Symbolic execution of looping programs may resultin an infinite symbolic execution tree. For this reason, symbolic execution is typically run witha (user-specified) bound on the search depth. Traditional applications of symbolic executioninclude test case generation and error detection, with many tools available [11, 24, 45, 49].Example. Consider the following code fragment, which checks if the sum of two distinct integersis bigger than 80, and the corresponding symbolic execution tree (Figure 2.1). Initially, the PC
2.2. INTERVAL CONSTRAINT PROPAGATION 18
is true and the symbolic state maps the variables x and y to the symbolic values $X and $Y ,respectively. When a branch decision is reached (labeled edges in the figure), the execution splitsand the PC is updated with new constraints for each corresponding path. For example, afterthe execution of the first condition statement, if (x == y), we have two PCs: $X = $Y and$X 6= $Y . Both constraints are satisfiable, as determined by a constraint solver, and thus bothpaths are feasible. The symbolic execution will then fork and follow both branches.
int x, y;//...if (x == y) {return false; //(1)
} else {x = x + y;y = 0;if (x > 80) {
return true; //(2)} else {
return false;//(3)}
}
[PC : $X != $Y] σ : x = $X + $Y, y = $Y
[PC : $X == $Y] σ : x = $X, y = $Y
return false (1)
x > 80 ?
[PC : $X != $Y ∧ $X+$Y > 80] σ : x = $X + $Y, y = 0
[PC : $X != $Y ∧ $X+$Y <= 80] σ : x = $X + $Y, y = 0
return false (3)return true (2)
[PC : true] σ : x = $X, y = $Y
x == y ?
[PC : $X != $Y] σ : x = $X + $Y, y = 0
Figure 2.1: Code fragment and the corresponding symbolic execution tree. Labelededges represent branch decisions, and dashed boxes represent the end of the branch.
2.2 Interval Constraint Propagation
Interval Constraint Propagation (ICP) [15] is an algorithmic approach to compute intervalsolutions to equality and inequality numerical constraints. The input is a list of n-variableconstraints and the output is a list of n-dimensional boxes. Each constraint is an equality or aninequality constraint, possibly involving non-linear analytical expressions and not necessarilydifferentiable. A box is a characterization of a subset of the Cartesian product of the domains ofinput variables.Example. Here is one of the boxes that RealPaver reports on output for the constraint (1.5− (x ·(1.0− y))) = 0.0:
x : [99.99925650834012, 100]
y : [0.9849998884754217, 0.9850000000000001]
Geometrically speaking, a box is an hyperplane, which generalizes a 2-dimensionalplane for higher dimensions. The approach guarantees that the union of all boxes reported on
2.3. PROBABILISTIC SOFTWARE ANALYSIS 19
output contains all solutions. Hence, a problem is unsatisfiable if the union of all boxes is empty.RealPaver [26] is a well-known implementation of the approach.
Even though the low and high values of these intervals are floating-point numbers,the box denotes a 2-dimensional region of real solutions. RealPaver uses two parameters todetermine how tight are the boxes reported on output: the decimal bound and the time budget.The bound is the number of decimal digits that limits the size of the smallest box that RealPavercan report and the time budget limits the time for computing boxes. The tightness of the boxesreported is proportional to the time budget, and inversely proportional to the decimal bound.budget the tighter are the boxes reported. Note that, irrespective of these parameters, the outputshould always include all real solutions. The approach is flexible to support a wide range ofmathematical functions.
2.3 Probabilistic Software Analysis
Probabilistic Software Analysis (PSA) is concerned with quantifying how likely softwareexecution is to satisfy a given property. PSA is relevant in contexts where the software is designedto exhibit uncertain or randomized behavior [47] or when the execution environment, includinginteractions with users, is characterized by a probabilistic profile [6]. In these situations, it isusually more relevant to quantify the probability of satisfying (/violating) a given property thanto just assess the possibility of such events to occur.
Figure 2.2 shows the main flow of the tool that we use to support probabilistic analysis.The tool takes on input a Java program and the usage profile for the program’s input variables. Theusage profile includes the domain of input variables and the probability distribution associatedwith each domain. The output of the tool is an estimate of the probability for satisfying (orviolating) a property of interest, e.g. an assertion in the code or a certain observable event ofinterest. Internally, the tool uses symbolic execution to produce a set of path constraints. Inparticular we use Symbolic PathFinder (SPF) [40] for the symbolic execution of Java bytecode,but other similar tools can be used.
Java
SPF qCORAL PCs
Probabilistic Software Analysis
Probability+ Accuracy
Inputprofile
TargetEvent
Figure 2.2: Probabilistic software analysis chain with qCORAL.
2.3. PROBABILISTIC SOFTWARE ANALYSIS 20
2.3.1 Computing Probabilities
Let’s classify the path constraints produced with symbolic execution in the sets PCT andPCF based on whether they lead to the occurrence of the target event (T) or not (F).
Under the assumption that symbolic execution terminates and only produces constraintsfor complete paths, the path constraints define disjoint input sets and they cover the wholeinput domain [30, 45]. However, in order to deal with possible non termination due to loopingconstructs in the code, SPF actually performs bounded symbolic execution. Hence, if an eventhas not occurred within the symbolic execution bound, the corresponding PC is not includedin PCT . On the other hand, since hitting the execution bound is an event observable thoughSPF, it is possible to introduce a third set of PCs containing those where the bound has beenhit and quantify the probability of such sets as well; this probability can give a measure for theconfidence in the results obtained within the bound (the lower the probability the higher theconfidence). This approach has been applied, for example in [20].
We define the probability of satisfying a given property, as the probability of an inputdistributed according to the usage profile to satisfy any of the path constraints in PCT . Formally,what we aim to compute is:
∫D1PCT (x) · p(x)
� �2.1
where D is the input domain defined as the Cartesian product of the domains of the inputvariables, p(x) is the probability of an input x to occur for the given usage profile, and 1PCT (x)is the indicator function on PCT , that is, a function that returns 1 when x satisfies any of the PCsin PCT , and 0 otherwise [42]. Equation
� �2.1 represents the expected probability for satisfyingthe target property.
2.3.2 Monte Carlo Simulation
In general, the constraints in PCT can be non-linear or can make the integral ill-conditioned for numerical solutions [29]. To retain generality, qCORAL builds on simulation-based methods. These methods have theoretically no limitations on the complexity of theconstraints they can handle. However, they can take a long time to converge, especially in thepresence of large, multidimensional domains. The simplest simulation-based method applicableto the problem at hand is the Hit-or-Miss Monte Carlo (HM-MC) method [46]. In practice,it consists in generating a sequence of n independent inputs (x0,x1, . . . ,xn−1), according tothe usage profile, and to count the number of hits, i.e., the number of samples that satisfy theconstraints under analysis. The ratio of hits (fraction of samples which are hits) provides anefficient, unbiased, and consistent estimator for the probability of satisfying the constraints [42].In particular such estimator X has a binomial distribution with expected values and variancegiven by the following equations [42]:
2.3. PROBABILISTIC SOFTWARE ANALYSIS 21
E[X]= x Var
[X]=
x · (1− x)n
� �2.2
where x = ∑n−1i=0 1PCT (xi)/n is the sample mean. Note that the more samples are collected, the
closer X gets to the integral of Equation� �2.1 . In particular, this convergence can be quantified
by the estimator variance Var: the closer the variance is to 0 the more accurate is the estimation.
2.3.3 Interval Constraint Propagation and Stratified Sampling
Despite their generality, HM-MC methods may suffer from a slow convergence rate [29],especially if the probability of the target event gets close to zero [51]. If additional informationis known about the problem under analysis, it can be exploited to improve the estimation perfor-mance, i.e. to reduce the estimator variance. A well established method for variance reduction isthe stratified sampling [31, 46]. This approach consists in partitioning the sample space, whichin our case corresponds to the input domain D, into disjoint subsets ({R0,R1, . . . ,Rm}), calledstrata. Each strata Ri can be analyzed separately via hit-or-miss Monte Carlo, obtaining thecorresponding estimator Xi. Since the sampling processes within each region is independent, andassuming we take the same number of samples on each strata [46], the strata estimators can becombined to obtain an estimator X over the original sample space, with the following expectedvalue and variance [46]:
E[X]= ∑
iwi ·E
[Xi]
Var[X]= ∑
iw2
i ·Var[Xi] � �2.3
where wi is defined as wi = size(Ri)/size(D), denoting the size of region Ri relative to the sizeof the domain D. Since the strata constitute a partition of D, ∑i wi = 1. The expected values andvariance of the estimators Xi are obtained with Equation
� �2.2 .It has been shown that the variance of an estimator obtained through stratified sampling
cannot be worse than the variance obtained running Hit-or-Miss Monte Carlo (HM-MC) in-tegration on the entire domain, although only a suitable choice of strata provides significantbenefits [46]. Optimal stratification criteria can be defined for each problem, although thisusually requires additional information about the problem, which is not always available or easyto estimate [31].
In our case, the set of constraints composing a PC are assumed to be formalized as theconjunction of mathematical inequalities, which may involve also non-linear analytical functions.By definition, the indicator function of Equation
� �2.1 evaluates to 1 if and only if the inputsatisfies the path constraint, that is, if it belongs to the locus of the solutions of the system ofmathematical inequalities composing it. The exact computation of these solutions is usuallyinfeasible, but interval constraint propagation can help to identify a set of sub-regions of thedomain containing them. For this dissertation, we use the tool RealPaver [26], which providesinterval constraint propagation for systems of non-linear inequalities over real domains. Given a
2.3. PROBABILISTIC SOFTWARE ANALYSIS 22
x y w E[X ] Var[X ]b1 [−1,−.5] [−1,−.5] .25/4 .5012 .2501b2 [−.5,−.5] [−1,−.5] .5/4 1 0b3 [.5,1] [−1,−.5] .25/4 .508 .25b4 [−.5, .5] [−.5,0] .5/4 .4904 .25
Table 2.1: Variance reduction for the example case.
set of inequalities and a bounded domain, RealPaver identifies a set of non-overlapping boxeswhose union contain all the solutions of the problem. The boxes may be tight, meaning they onlycontain solutions, or loose, containing both solutions and other points. Since no solution existsoutside those boxes, there is no need to analyze such region through HM-MC, since we alreadyknow that the estimator would converge to 0, with variance 0, since the integral of Equation
� �2.1evaluates to 0.Example. As an example of the use of ICP for variance reduction, let us consider the constraintx ≤−y ∧ y≤ x, where values of variables x and y are uniformly distributed over the domain[−1,1]. As illustrated in Figure 2.3, the probability of satisfying the constraints can be easily com-puted as the ratio between the area
0
−1 0
−1
1
b4
b3b2b1
Figure 2.3: Exampleillustrating interval constraint
propagation with stratifiedsampling. The boxes returnedby the ICP are shown in red.
of the triangle identified by the constraint and the size of the do-main, and it is exactly 1/4. By using HM-MC with 104 samples,we obtain mean (i.e., expected value) .2577 and variance .19131.Let’s consider that ICP produces four boxes (b1 . . .b4), as shownin Figure 2.3, containing all solutions. Let’s consider we take2,500 samples within each box, thus overall still 104 samplesas in the case of the regular (non-stratified) HM-MC approach.Table 2.1 shows the boxes coordinates for each of the boxes, size,and corresponding estimates. The part of the domain not coveredby the four boxes accounts for 3/4 of the domain, and in suchregion we already know that both the mean and the variance ofa hit-or-miss estimator would be 0, because there are no solutionsare in there. If we combine the estimators of the single boxes asper Equation
� �2.3 , we obtain mean .2494 and variance .00586,which is a significant improvement, despite the relatively small number of samples we tookwithin each region. Further evaluation of the accuracy achievable through this approach will beprovided in Section 3.6.1.
232323
3Compositional Solution Space Quantification
Simulation techniques are highly sensitive to the number of samples they take – seethe number of samples factor “n” in the denominator of Equation 2.2. Efficiency is thereforeessential for these techniques: improved efficiency means one can sample more within the sametime budget and therefore reduce the estimate variance.
We propose a compositional approach to efficiently estimate the solution space forcomplex mathematical constraints, encoding the disjoint path conditions obtained throughsymbolic execution. Our approach leverages the following two observations:
� Disjunction: From the semantics of symbolic execution, an input cannot satisfymore than one path constraint in PCT . Each path constraint in this set denotes oneequivalence class of the input space. We therefore decompose the estimation forPCT into the estimation of each individual PC in the set. Section 3.1 describes anapproximate composition rule for the disjunction of path constraints in PCT .
� Conjunction: Each individual path constraint PC is a composition of simpler inde-pendent predicates that can be estimated separately; furthermore these predicatesoccur across multiple path constraints in PCT , and therefore their estimates can beefficiently re-used. Section 3.2 provides a heuristic for dealing with the conjunctionof many constraints.
Section 3.3 summarizes the benefits of utilizing this approach, and an practical exampleof its use is shown in Section 3.4. Implementation details and the algorithms used in qCORAL arepresented in Section 3.5. We conclude the chapter with an evaluation (Section 3.6) of qCORALon a set of benchmarks taken from the literature and a set of real world software taken frommedicine and aerospatial domain.
3.1 Disjunction of Path Conditions
Consider the set PCT = {PCT1 ,PCT
2 , . . . ,PCTm} of the PCs leading to the occurrence of
the target event. The elements PCTi are pairwise disjoint by construction. Let XT
i be the estimator
3.2. CONJUNCTION OF CONSTRAINTS 24
of the probability, given an input profile, of satisfying the constraints in PCTi (i.e. an estimator of
the integral in� �2.1 ). Since there is no intersection between any two sets PCT
i and PCTj such that
i 6= j, we can define an estimator XT of the disjunction between PCTi and PCT
j as follows:
XT = XTi + XT
j
� �3.1
Since there is no intersection between any two sets PCTi and PCT
j , the expected value ofXT can be straightforwardly computed as [42]:
E[XT ]= E
[XT
i]+E
[XT
j] � �3.2
The following theorem gives a bound on the variance for the composed estimator XT ,based on the variance computed for the individual path conditions:
Theorem 1. Let PCTi , PCT
j ∈ PCT | i 6= j, XTi and XT
j be estimators of the probability ofsatisfying PCT
i and PCTj respectively. Let XT = XT
i + XTj , then:
Var[XT ]≤ Var
[XT
i]+Var
[XT
i] � �3.3
Proof. Since XT is defined as the sum of two random variables, the following general relationholds [42]:
Var[XT ]= Var
[XT
i]+Var
[XT
i]+2 ·Cov
[XT
i , XTj]
where the covariance Cov[XTi , XT
j ] = E[XTi · XT
j ]−E[XTi ] ·E[XT
j ]. We already know from Equa-tion
� �2.1 that
E[XTi · XT
j ] =∫
D1PCT
i(x) ·1PCT
j(x) · p(x)
but the product of the indicator functions in the equation above is alway zero because, for everyinput, at most one of the PCs can be satisfied. On the other hand, the probability of satisfying aPC cannot be negative, thus Cov(XT
i , XTj )≤ 0.
Theorem 1 allows us to process each PC separately (possibly also in parallel) and thencompose the partial results to estimate the probability of satisfying any of the PCs in PCT .The price for this approach is the need to provide an overestimation of the variance of thecomposition, although, as will be shown in Section 3.6, in practical applications the benefits ofthe variance reduction techniques on analyzing the single disjuncts often overcome the loss dueto the conservativeness we took for disjuncts composition.
3.2 Conjunction of Constraints
We now consider the problem of efficiently computing the probability of satisfying anindividual path condition PC. For a real application, a path condition could be very large and
3.2. CONJUNCTION OF CONSTRAINTS 25
may include many constraints relating many input variables. We propose to partition theseconstraints into smaller sets that have in common the input variables, and whose satisfiabilitytherefore can be determined independently from one another. A path condition PCi is defined bythe conjunction of several simpler constraints ci0∧ ci1∧ . . .cim. Each constraint cik can make anassertion about one or more input variables v j, or functions thereof. Let indicate it by A(cik,v j).
Let us introduce a dependency relation (Dep) among the variables appearing in a program:
Definition 1. Given the path conditions obtained by the symbolic execution of a program {PCi},such that PCi = ci0∧ ci1∧ ·· · ∧ cim, and let V be the set of the input variables of the program.The relation Dep⊆V ×V is recursively defined by the following statements:
� ∀v ∈V : Dep(v,v)
� ∀vi,v j ∈V : ∃ckl | A(ckl,vi)∧A(ckl,v j) =⇒ Dep(vi,v j)
� ∀vi,v j,vk ∈V : Dep(vi,vk)∧Dep(vk,v j) =⇒ Dep(vi,v j)
Intuitively, two input variables depend on each other if they appear together in at least oneconstraint in the path condition. If this is the case, to evaluate the satisfaction of such conditionsthe assignments to both variables has to be evaluated at the same time.
Notice that Dep is by construction an equivalence relation over V and for this reason itinduces a partition of V . Let us refer to the sets composing these partitions as {V0,V1, . . . ,Vl}.We can now extend the definition of A(·, ·) by stating that A(cik,Vj) holds if there exists a variablev j ∈ Vj such that A(cik,v j) holds.
Consider now a path condition PCi. Let us define the constraints Ci j = {cik | A(cik,Vj)} .That is, Ci j is the conjunction of all the constraints cik occurring in PCi and containing any ofthe variables from Vj ⊆ V . The probability of satisfying PCi is the same as the probability ofsatisfying all the constraints Ci j at the same time. But since two constraints Ci j and Cik do notshare any variables by construction (Vj and Vk are disjoint), the satisfaction of one of them isindependent from the satisfaction of the other1. This independence is straightforwardly inheritedby the hit-or-miss estimators Xi j and Xik of the probability of satisfying Ci j and Cik, respectively.Indeed, the Monte Carlo procedures will generate random assignments for the sets of variablesVj and Vk independently.
Thanks to the independence of the estimators Xi j and Xik of the probabilities of satisfyingthe constraints Ci j and Cik, the probability of Ci j ∧Cik can be estimated by Xi jk = Xi j · Xik
having [42]:
1As support of this statement, notice that Pr(Ci j|vk = vk1) = Pr(Ci j|vk = vk
2), where vk /∈ Vj and v1k and v2
k areany two valid assignments for vk. Thus, we can conclude that Pr(Ci j|vk = vk
1) = Pr(Ci j).
3.3. OBSERVATION 26
E[Xi jk]
= E[Xi j]·E[Xik] � �3.4
Var[Xi jk]
= E[Xi j]2 ·Var
[Xik]+E
[Xik]2 ·Var
[Xi j] � �3.5
+Var[Xi j]·Var
[Xik]
3.3 Observation
By applying the composition methods introduced in the previous sections, we may obtainthe following benefits. First, we can split the analysis of a large path condition into the analysisof several simpler constraints. Second, each of the simpler constraints will possibly involveonly a subset of the input variables, making the generation of the samples faster. Furthermore,reducing the input space to only the variables actually involved in the constraint would likelyresult in a better coverage of the input sub-domain, for a fixed number of samples. Finally,we remark that the simpler constraints can be analyzed efficiently using the interval constraintpropagation with Monte Carlo stratified sampling described in the previous section.
3.4 Example
The following code snippet mimics a safety monitor for an autopilot navigation system. Ifthe altitude is less than or equal to 9000 meters, the autopilot is allowed to control the position ofthe vehicle by manipulating the front and head flaps through floating point variables headFlapand tailFlap. These variables range in the interval [−10,10]. altitude is also a floatingpoint variable.
if (altitude <= 9000) { ...
if(Math.sin(headFlap*tailFlap)>0.25){
callSupervisor();
} ...
} else { callSupervisor(); }
Listing 3.1: Example of events and complex conditions in Java.
If during the flight the safety monitor realizes that the relative position of the flapsviolates the imposed safety conditions, it calls a human supervisor to take over the control of thevehicle. The same procedure is actuated if the vehicle gets over 9000 meters of altitude.
For this example, our target event is the call of a supervisor and the code substantiatesthis event with function callSupervisor. We want to quantify the probability of such event tooccur given a certain probabilistic profile for variables headFlap, tailFlap, and altitude.Such profile can be obtained by monitoring data of similar vehicles. For the sake of simplicity,let us consider a uniform profile for the three variables over their respective domains. We restrict
3.5. ALGORITHMS AND IMPLEMENTATION 27
Algorithm 1: qCORALInput: PCs,DOutput: mean,vardep← computeDependencyRelation(PCs)mean← 0; var← 0for pc ∈ PCs do〈pcMean, pcVar〉 ← analyzeConjunction(pc,D,dep)mean← mean+ pcMeanvar← var+ pcVar
return 〈mean,var〉;
altitude to the range [0,20000]. Symbolic execution produces the following path constraints thatreach our target event:
PCT1 : altitude > 9000
PCT2 : altitude≤ 9000 ∧ Math.sin(headFlap · tailFlap)> 0.25
We want to quantify the solution space over PCT ={PCT1 ,PCT
2 }. We illustrate how qCORAL
analyzes these constraints in the following.PCT
1 consists of a single atomic constraint predicating only on the value of variablealtitude, thus no further decomposition is possible. This constraint can be analyzed byapplying the techniques introduced in this chapter, obtaining for its statistical estimator XT
1 :E[XT
1]= 0.55 and Var
[XT
1]= 0. Note that the variance of the estimator is 0 thanks to the help
of the ICP solver, which is able to identify one tight box for this constraint.In contrast to PCT
1 , PCT2 is a conjunction of two boolean expressions. According to
Definition 1, we know that variables headFlap and tailFlap depend on one another, while altitude
does not depend on any other variables. For this reason, as discussed in Section 3.2, we cananalyze the two constraints separately and then merge results. For the atomic constraint altitude≤9000 we obtain the estimator XT
2,1 having E[XT
2,1]=0.45, with variance 0. For the constraint
sin(headFlap · tailFlap)> 0.25, although its solution space does not fit in a single box, the useof ICP-based stratified sampling does still help in exploiting the geometry of the solution space.For this constraint the estimator XT
2,2 has Var[XT
2,2]=8.103406 · 10−6 and E
[XT
2,2]=0.417975.
Equations� �3.4 and
� �3.5 enable qCORAL to compose the estimators of sub-formulas of PCT2 to
obtain the estimator XT2 , with E
[XT
2]=0.188089 and Var
[XT
2]=1.64094 ·10−6.
Finally, Equation� �3.2 enables qCORAL to obtain the estimator XT , with E
[XT ] =
E[XT
1]+E
[XT
2]= 0.738089, while Var
[XT ]≤ Var
[XT
1]+Var
[XT
2]= 1.64094 ·10−6 . In this
example, the exact probability, rounded to the 6th digit is 0.737848.
3.5. ALGORITHMS AND IMPLEMENTATION 28
Algorithm 2: analyzeConjunctionInput: pc,D,depOutput: mean,varmean← 1; var← Nil; cache← getResultCache()for varSet ∈ dep do
part← extractRelatedConstraints(pc,varSet)if !cache.contains(part) then〈partMean, partVar〉 ← stratSampling(part,varSet,D)cache.put(part, 〈partMean, partVar〉)
〈partMean, partVar〉 ←cache.get(part)mean← mean∗ partMeanif var = Nil then
var← partVarelse
var← mean2 ∗ partVar+ partMean2 ∗ var+ var ∗ partVar
return 〈mean,var〉;
3.5 Algorithms and Implementation
This section provides some implementation details of qCORAL. The input of qCORALincludes a set of disjoint constraints PCs, representing the path conditions leading to the occur-rence of the target event as obtained from the symbolic execution stage (see Section 2.3), and adescription of the input domain, i.e., a map from floating-point input variables to their domain.The domain of each variable is a closed interval. We used Symbolic PathFinder (SPF) [45] asour symbolic execution engine.
Algorithm 1 describes the main loop of qCORAL. It iterates over the input path constraints,processes each one, and combines the partial results as described in Section 3.1. Note thatqCORAL returns an upper bound for the variance of a disjunction of path constraints, as definedin Equation
� �3.3 . The procedure computeDependencyRelation returns the partition of the inputvariables according to Definition 1. To efficiently implement this procedure we rely on the JUNGgraph library[38]. We create an undirected graph, where a node corresponds to an input variableand and edge corresponds to a dependence between two variables. This procedure visits everypath condition and when two variables appear in the same constraint it adds an edge betweentheir corresponding nodes. The procedure then computes the weakly connected components ofthe graph (with complexity at most quadratic in the number of input variables).
Algorithm 2 computes results for one conjunctive clause. The procedureextractRelatedConstraints goes through all the conjuncts in the path constraint pc and projectsthose containing variables in varSet. Note that each varSet contains the variables in a parti-tion induced by the dependency relation as computed by computeDependencyRelation. Thepartial results for each set of independent constructs is computed by means of the procedurestratSampling, and then combined with the previous ones. As an optimization, the results
3.6. EVALUATION 29
Algorithm 3: stratSamplingInput: part,varSet,DOutput: mean,varmean← 0; var← 0boxes← icp(part,varSet,D)for box ∈ boxes do〈boxMean,boxVar〉 ← hitOrMiss(part,varSet,D)boxWeight← size(box)/size(D)mean← mean+boxWeight ∗boxMeanvar← var+boxWeight2 ∗boxVar
return 〈mean,var〉;
obtained for an independent constraint can be safely stored in a cache and reused thanks tothe global independence of their estimator with all the others. Conceptually caching makes atrade-off between accuracy and efficiency. It is possible that, when caching is active, the error inlocal sampling may be amplified as qCORAL samples a partition only once. However, since thesubproblems whose results are cached are simpler than the original one (especially in the caseswhen only a small subset of input variables are related to one another), the accuracy we canobtain is usually higher, especially considering that, thanks to caching, the time budget for eachsimulation can be increased. Section 3.6 evaluates the impact of caching in terms of precisionand time.
Algorithm 3 computes results for one partition using stratified sampling and HM-MC.Argument varSet defines the integration variables that needs to be considered to quantify part
through simulation. The procedure icp makes a call to our ICP solver. The estimate and varianceobtained with stratified sampling on each box are cumulatively added to return variables mean
and var.RealPaver. We used RealPaver [26] for Interval Constraint Propagation (ICP). Section 2.3.3describes input and output of an ICP solver. Our RealPaver configuration is constrained with thefollowing stop criteria: time budget per query of 2s, a bound on the number of boxes reportedper query of 10, and a lower bound on the size of the computed boxes of 3 decimal digits. Theseparameters are set empirically, based on previous experiences with the tool. All other parametersare fixed. We note that the time budget, in particular, enables one to calibrate the time spent withdomain stratification and time spent with simulation.
3.6 Evaluation
Our evaluation addresses the following research questions:
� RQ1. What is the accuracy of qCORAL estimates?
� RQ2. How qCORAL compares with the built-in numerical integration routines ofMathematica[50] and with VolComp [47] when handling linear constraints?
3.6. EVALUATION 30
subject Analytical 103 samples 104 samples 105 samples 106 samplesSolution estimate σ estimate σ estimate σ estimate σ
Convex PolyhedraTetrahedron 0.512682 0.5151 0.0623 0.5144 0.0210 0.5122 0.0050 0.5128 0.0012
Cube 8.0 8.0 0.0 8.0 0.0 8.0 0.0 8.0 0.0Icosahedron 2.181695 2.2043 0.1471 2.1948 0.0440 2.1843 0.0133 2.1829 0.0039
Rhombicuboctahedron 14.333333 14.4027 0.3111 14.3286 0.1403 14.3382 0.0416 14.3330 0.0114
Solids of RevolutionCone 1.047198 1.0577 0.0509 1.0495 0.0223 1.0462 0.0060 1.0471 0.0019
Conical frustrum 1.8326 1.8483 0.1291 1.8385 0.0451 1.8340 0.0115 1.8326 0.0034Cylinder π 3.1470 0.0451 3.1424 0.0189 3.1417 0.0045 3.1415 0.0017
Oblate spheroid 16.755161 16.7723 0.4242 16.7428 0.1479 16.7560 0.0586 16.7550 0.0122Sphere 4/3 ·π 4.1930 0.1060 4.1857 0.0370 4.1890 0.0146 4.1887 0.0031
Spherical segment 113.557882 113.5982 2.9401 113.4129 0.8739 113.5646 0.2628 113.5618 0.0887Torus 1.233701 1.2277 0.0467 1.2305 0.0127 1.2327 0.0042 1.2337 0.0013
IntersectionTwo spheres 56.5485 56.7837 1.9705 56.6357 0.6703 56.5498 0.2413 56.5480 0.0600
Cone-cylinder 2.7276 2.7315 0.1401 2.7276 0.0500 2.7289 0.0136 2.7285 0.0036
Table 3.1: Microbenchmarks.
� RQ3. How the different features of qCORAL affect accuracy and time when handlingcomplex constraints?
All experiments have been run on an Intel Core i7 920 64-bit machine, with 2.67Ghz and8GB, running Ubuntu 12.04.
Although the methodology proposed in this chapter is general for any usage profile, theresults reported in this section use uniform profiles only. We present an efficient approach tosupport the use of complex probability distributions in Chapter 4.
3.6.1 RQ1: What is the accuracy of qCORAL estimates?
This experiment evaluates how accurate qCORAL is for computing the volume of severalgeometric figures for which analytical solutions are widely known. We used the symbolicintegration routines of Mathematica to obtain the exact volumes of these solids, while forqCORAL we computed them as the fraction of a domain of known size.
Table 3.1 summarizes the results. We evaluated the accuracy of our approach for 103,104, 105, and 106 as maximum number of samples for the Monte Carlo procedure. We run30 times each configuration and reported the average value and standard deviation over thepopulation of estimated volumes.
We grouped our subjects in three groups: convex polyhedra, solids of revolution, andintersection of solids. Except for the convex polyhedra cases, all other subjects contain non-linear constraints and mathematical functions, namely exponentiation and square root. For thesesubjects the compositional approach does not provide benefits, since the three variables charac-terizing each solid are tightly dependent one another. We will come back to compositionalitylater in this section.
All the experiments completed within 2 seconds, with the exception of Icosahedronand Rhombicuboctahedron for 106 samples (4 and 7 seconds, respectively). Notably, thanks
3.6. EVALUATION 31
to Interval Constraint Propagation (ICP) and stratified sampling, even with a relatively smallnumber of samples, qCORAL provided a reasonably accurate result for most of the subjects.
Finally, consider the case of Cube: the standard deviation is 0 because, regardless thenumber of samples, qCORAL was always able to find the exact solution thanks to ICP. Indeed,the real subject a 3D box and RealPaver can exactly identify it, driving the estimation error tozero.
3.6.2 RQ2: How qCORAL compares with Mathematica and VolComp?
This experiment evaluates how qCORAL (with both stratified sampling and the compo-sitional analysis) compares with existing techniques for quantifying solution spaces of linearconstraints. As baseline for comparison, we use the built-in procedure for numerical integrationof the commercial tool Mathematica, NIntegrate. This procedure is available off-the-shelfand we run it with its default settings. We also include VolComp, a recently developed toolproducing as output a tight closed interval over the real numbers containing the requestedsolution.
The built-in numerical integration routine of Mathematica performs a recursive procedurecalled Global Adaptive Integration [34]. On each recursive call, it analyzes the behavior of theintegrand function within the integration domain and automatically selects an integration rule toapply. Mathematica supports a broad variety of integration rules, e.g. Trapezoidal, Simpson, orNewton [2]. After each iteration, the procedure analyzes the obtained results and if they do notmeet the accuracy goals, the integration domain is bisected and the integration routine is invokedon each of the two parts. Partial results are then combined, similarly to the case of stratifiedsampling. The procedure terminates when the default accuracy requirements are met or when therecursion depth limit has been reached. Since NIntegrate guarantees the accuracy of the result(or gives a notification when its requirements are not met), it provides a reference to evaluatehow tight are the intervals produced by VolComp and how precise are the estimates producedby qCORAL.
The techniques we evaluate in this experiment report solutions in different formats.Mathematica reports a single point solution, which is exact up to an informed bound, VolCompreports an interval solution that bounds the exact point solution, and qCORAL reports an estimateon the exact solution and a statistical variance for that estimate. In place of the variance, forqCORAL we report the standard deviation (square root of the variance), which is in the sameunit scale of the estimate. Furthermore, since qCORAL implements a randomized algorithm, theestimate and the standard deviation we report are averaged over 30 executions.
To compare the tools, we used the VolComp benchmark, which is publicly available [3].The subjects we selected from the benchmark are: the Framingham 2 atrial fibrillation risk calcu-lator (ATRIAL), a steering controller to deal with wind disturbances (CART), the Framingham
2http://www.framinghamheartstudy.org
3.6. EVALUATION 32
Subject Asrt. #Paths #Ands #Ar. Ops. 0-var./Parts.VolComp subjects [47]
ARTRIAL1 442 1,484 0 (0) 23/232 2,439 1,740 443 (3) 4/273 2,260 68,630 19,125 (3) 10/44
CART4 44 1,209 638 (3) 1/455 47 1,296 681 (3) 1/48
CORONARY6 320 195 62 (3) 8/157 274 31 8 (3) 5/8
EGFR EPI8 45 547 31 (3) 46/469 44 422 33 (2) 38/38
EGR EFI (SIMPLE)10 13 163 12 (3) 13/1811 14 101 9 (3) 14/14
INVPEND 12 1 54 229 (3) 0/1
PACK
13 1,103 16,414 0 (0) 40/4014 906 12,080 0 (0) 37/3715 924 12,293 0 (0) 38/3816 821 840 0 (0) 38/3817 954 14,850 6,948 (2) 815/95418 1,030 16,186 7,578 (2) 935/103019 1,132 17,972 8,420 (2) 1132/1132
VOL 20 24 13,824 882,508 (3) 0/1Table 3.2: Characterization of the VolComp benchmark.
coronary risk calculator (CORONARY), an eGFR 3 estimator of chronic kidney’s disease (EGFREPI and EGFR EPI SIMPLE), an inverted pendulum (INVPEND), a model of a robot that packsobjects of different weights in a carton with the goal to maximize total weight within a certainlimit (PACK), and a controller for filling up a tank with fluid (VOL).
Table 3.2 reports a characterization of these subjects. Column “Asrt” denotes a unique idwe used to identify the checked assertions during the discussion. Column “#Paths” indicatesthe number of symbolic paths leading to the violation of the assertion. Columns “#Ands” and“#Ar. Ops” denote respectively the number of conjuncts and the number of arithmetic operations.For column “#Ar. Ops”, in parenthesis the number of different operators (e.g., + or sin()). Avalue of 0 in this column means that all the constraints involve only comparisons (e.g., x < y
or x < 5). Finally, “Parts.” indicates the number of independent subproblems identified forcompositional analysis and “0-var” indicates the number of subproblems with exact solutionsafter applying ICP. Column “0-var./Parts.” indicates how many of the subproblems we obtainedvariance 0 on an execution of qCORAL with 100k samples. A subproblem can obtain 0 varianceafter sampling when either ICP! (ICP!) returned an exact solution or if all the samples returnedthe same truth value (cf. Equation 2.2).
We re-ran the benchmarks from [3] using their scripts. The PCs were obtained by
3http://nephron.com/epi_equation
3.6. EVALUATION 33
Subject Assertion NIntegrate VolComp qCORAL{STRAT,PARTCACHE} - 30ksolution time(s) bounds time(s) avg. estimate avg. σ avg. time(s)
ARTRIAL1 0.1343 5.05 [0.1343, 0.1343] 3.62 0.1343 0.00e+00 1.032 0.0005 89.26 [0.0005, 0.0005] 40.0 0.0005 1.00e-06 1.513 0.9350 4,179.36 [0.9340, 0.9364] 771.1 0.9352 1.63e-04 4.14
CART 4 0.9746 7.26 [0.9390, 1.0000] 32.29 0.9739 1.12e-02 4.185 0.9826 7.66 [0.9470, 1.0000] 33.74 0.9818 1.11e-02 4.39
CORONARY 6 0.0006 3.44 [0.0006, 0.0006] 3.93 0.0006 1.90e-06 0.927 0.0001 0.86 [0.0001, 0.0001] 1.99 0.0001 4.29e-07 0.57
EGFR EPI 8 0.1264 1.98 [0.1264, 0.1264] 0.60 0.1262 3.29e-04 1.619 0.0986 1.69 [0.0986, 0.0986] 0.50 0.0986 4.80e-05 1.42
EGFR EPI (SIMPLE) 10 0.5388 0.83 [0.5387, 0.5389] 0.46 0.5389 8.71e-04 0.8511 0.3012 0.65 [0.3012, 0.3012] 0.14 0.3012 0.00e+00 0.66
INVPEND 12 0.0507 1.15 [0.0000, 0.1225] 6.20 0.0515 7.82e-04 0.79
PACK
13 0.9546 57.44 [0.9546, 0.9546] 16.45 0.9546 0.00e+00 2.1814 0.3898 41.76 [0.3898, 0.3898] 12.59 0.3898 0.00e+00 1.9415 0.1428 45.48 [0.1427, 0.1427] 13.75 0.1428 0.00e+00 2.0516 0.0002 4.41 [0.0002, 0.0002] 5.24 0.0002 0.0000 1.2517 0.2462 5066.20 [0.2522, 0.2800] 104.8 0.2663 2.72e-05 68.7918 0.6771 70.16 [0.6369, 0.7155] 60.15 0.6772 1.67e-04 82.9819 0.9592 54.48 [0.9592, 0.9592] 16.93 0.9592 0.00e+00 92.33
VOL 20 1.0005 1245.30 [0.0000, 1.0000] 3.76 1.0001 5.18e-03 821.11
Table 3.3: Comparison of NIntegrate (default numerical integration method fromMathematica [2]), VolComp, and qCORAL with features STRAT and PARTCACHE
enabled. Note that the comparison is restricted to linear constraints and the comparisonmetrics are not the same.
translating PCs produced with their frontend. All experiments ran until completion. Table 4.2shows the comparison between qCORAL and VolComp [3]. Notice that the two techniquesprovide different types of results. VolComp uses iterative bounding to return an intervalcontaining the solution. qCORAL returns a statistical estimator characterized by its expectedvalue and its variance: the expected value is the most likely value as it comes out from qCORAL
analysis; the variance provides a measure of the uncertainty of the estimate; such uncertaintycould be used to quantify the probability the real value belongs to an interval, for example byusing Chebyshev’s inequality [42].
The first 2 columns of Table 3.3 state the name of the subject and the targeted assertion.For each tool we then show the estimated probability of satisfying the assertions specified in thefirst column. We make the following observations from these results:
� ARTRIAL. For one of the three assertions Mathematica takes very long to produceresults and VolComp takes more than 10m to finish. qCORAL produces an accurateresult very quickly.
� CART. This subject produces a highly skewed polynomial [47], which is knownto be a case where branch-and-bound techniques do not perform well. Indeed,results of VolComp are not good for this case. qCORAL also suffers as it relieson RealPaver which also uses branch-and-bound techniques. For this reason, weobserve a high standard deviation in our results relative to other subjects. Note thatdespite this limitation qCORAL can still report an estimate close to the exact solution.Mathematica can easily handle the subject in this case.
3.6. EVALUATION 34
� CORONARY and EGFR. These are the best cases for qCORAL. As compared toother tools, qCORAL reports results very quickly and precisely.
� PACK. qCORAL is slower for the last two cases of PACK. The reason for this highcost is the high number of paths and the high interdependence among the variables oneach path constraint, which reduces the impact of our divide and conquer strategy forconjuncts. We observed that for one case in this subject Mathematica actually missesthe interval; this is highlighted in grey color. This happened because the defaultsettings of NIntegrate does not allow to collect enough points for the numericalintegration to converge. This situation is reported by the tool. Note that on somecases rounding the result to four decimal digits made the results of VolComp bedifferent than those of Mathematica and qCORAL by the last digit. These have to beconsidered just rounding error.
� VOL is a case that stresses qCORAL. Mathematica finishes in more than 20m andVolComp returns the full range 0.0-1.0, i.e. it did not perform any pruning. qCORALreports the expected result of ∼1.0. Note that the estimate that qCORAL reports isactually slightly greater than 1, for this case. This is due to the propagation of errorsin the estimation which magnifies when the exact probability is close to the cornercase 1. Mathematica also reports a result > 1 due to the finite accuracy of numericalintegration, partially due to the use of off-the-shelf settings for such complex subject.
In summary, we observed that qCORAL reports estimates very close to the exact pointsolutions that Mathematica’s numerical integration method reports (apart from the case of PACKwhere Mathematica reported no convergence and a wrong). However, numerical integration ispotentially expensive when the number of variables grows [44]. This observation is confirmed forthe cases of ATRIAL and PACK, where Mathematica takes respectively more than 1h to complete.
In most of the experiments, especially those on complex subjects, qCORAL resultedfaster than both off-the-shelf use of Mathematica’s numerical integration function and VolComp.For specific problems advanced settings of Mathematica may improve its performance, though adeeper understanding of the mathematical nature of the problem might be required, which mightbe not straightforwardly derived from the code under analysis. We also notice that the results ofqCORAL have been consistent with those of VolComp: the qCORAL estimates fall within thecorresponding VolComp intervals almost always, up to the accuracy (last decimal digit might bedangling because of rounding); the only exception is 8, where the standard deviation is anywaylarge enough to account for the deviation (3.29 ·10−4). Concluding, we observe that qCORALhas provided a reasonable balance between time efficiency and precision when handling linearconstraints on this benchmark suite.
3.6. EVALUATION 35
Monte Carlo Hit-Or-Miss Apollo Conflict Turnlogic1k 10k 100k 1k 10k 100k 1k 10k 100k
Mathematicaest. 0.62521 0.62538 0.62451 0.49794 0.49992 0.50064 0.72254 0.72029 0.72209σ 0.02126 0.00896 0.00278 0.01308 0.00451 0.00526 0.02250 0.00777 0.00257
t.(s) 65.44 221.75 1723.45 0.66 1.10 5.12 2.18 5.97 41.85
qCORAL{}est. 0.62780 0.62574 0.62540 0.49927 0.49906 0.50011 0.71913 0.72208 0.72195σ 0.01342 0.00458 0.00160 0.01691 0.00509 0.00156 0.01338 0.00436 0.00116
t.(s) 18.17 130.75 1263.02 0.44 0.92 5.71 0.64 2.46 18.81
qCORAL{STRAT}est. 0.62211 0.62610 0.62550 0.50016 0.50015 0.50017 0.72282 0.72162 0.72238σ 0.01262 0.00438 0.00143 0.00027 0.00010 0.00002 0.01462 0.00490 0.00105
t.(s) 62.98 206.89 1583.02 1.53 2.41 10.14 1.22 2.83 16.77
qCORAL{STRAT,PARTCACHE}
est. 0.62461 0.62497 0.62524 0.50016 0.50015 0.50017 0.72282 0.72162 0.72237σ 0.00972 0.00315 0.00089 0.00027 0.00010 0.00002 0.01462 0.00490 0.00105
t.(s) 45.04 97.43 625.47 1.65 2.31 8.23 1.62 3.35 18.16
Table 3.4: Comparison of different configurations of qCORAL on different samplingrates.
3.6.3 RQ3: How the different features of qCORAL affect accuracy andtime when handling complex constraints?
This experiment evaluates how different configurations of qCORAL compare with respectto precision and time. We considered two implementations of HM-MC: one from Mathemat-ica (baseline) and one from qCORAL: qCORAL{}. The empty braces indicate that no featurefrom qCORAL has been enabled in this configuration. The configuration qCORAL{STRAT}incorporates stratified sampling in the analysis of individual path conditions. The configurationqCORAL{STRAT,PARTCACHE} cumulatively incorporates partitioning and caching. We consid-ered the following subjects, which we will refer to in the remainder of this dissertation as theqCORAL benchmark [4], from the aerospace domain in this experiment:
� Apollo. The Apollo Lunar Autopilot is a Simulink model that was automaticallytranslated to Java using the Vanderbilt tool-set [39]. The model is available fromMathWorks 4. It contains both Simulink blocks and Stateflow diagrams and makesuse of complex Math functions (e.g. Math.sqrt). The code contains 2.6KLOC,deployed in a single package with 54 classes. We analyzed 5,779 path constraints forthis subject.
� TSAFE. The Tactical Separation Assisted Flight Environment (TSAFE) is designedto prevent near misses and collisions of aircraft that are predicted to happen in the nearfuture (from 30s to 3m). The Conflict Probe module of TSAFE tests for conflictsbetween a pair of aircraft within a safe time horizon. The two aircraft may eitherbe flying level or engaged in turns of constant radius. The following math functionsappear in the path constraints of this subject: cos, pow, sin, sqrt, and tan. TheTurn Logic module of TSAFE computes the change in heading required once animpending loss of separation between two aircraft is detected. It assumes a constantturning radius for the aircraft making the maneuver. Path constraints for this subject
4http://www.mathworks.com/products/simulink/demos.html
3.6. EVALUATION 36
contain the atan2 function. Each of these modules is about 50 LOC. We analyzedrespectively 23 path constraints on Conflict and 225 path constraints on Turn
Logic.
These case studies contain complex constraints. They have 2.6KLOC for Apollo and50 each for the two TSAFE components analyzed. Since no properties were defined, insteadof fabricating a property, we generated all the PCs (using SPF with search bound equal to 50)and selected a percentage of PCs to quantify the path probabilities. We generated 5779 PCs forApollo and 225 for TSAFE; the latter is smaller but involves complex mathematical functionsand has high dependence among variables. We arbitrarily picked the first 70% of the PCs in abounded depth-first order. We picked 70% of the paths so to avoid obtaining a probability closeto 0 or 1, to not bias the evaluation for the Monte Carlo estimation towards its worst cases (someextreme cases have already been evaluated in Table 4.2). This selection mimics a property that issatisfied on some of the paths and not on the rest.
Table 3.4 shows experimental results. Note the reported estimations are not exactly 70%because different PCs have different solution spaces.
Different groups of lines show different configurations of qCORAL, each group ofcolumns denotes a subject, and each column in a group denotes a different maximum numberof samples allowed for simulation. Results show that the addition of feature STRAT results in asignificant reduction in the variance of the estimate at the expense of an overhead in time. Thisadditional cost is justified by the multiple invocations of the ICP solver; one call per disjunctiveclause of the input formula. Adding the feature PARTCACHE, i.e. performing compositionalanalysis, may further improve precision (see Section 3.2), however, considering the most time-consuming cases, it always reduces time. Increasing the maximum sample size from 1K to 100Kresults in higher time savings as the cost for analyzing each subproblem increases. While therelationship between execution time and maximum number of samples is approximatively linearfor the basic configuration, the impact of stratified sampling and compositionality on executiontime depends on the specific subject.
373737
4Distribution-Aware Sampling
This chapter extends the compositional sampling approach presented before with anefficient way to handle inputs that follow continuous distributions. This can be achieved byenhancing the Monte Carlo estimation methods with a distribution-aware sampling strategy,where the random samples can be generated according to a specific continuous distributioninstead of a uniform one. However, the use of interval constraint propagation and stratifiedsampling requires some care, since we need to restrict the sampling to specific sub-regions of theinput domains.
Section 4.1 presents a small code example that will help to guide the discussion in thischapter. The formal mathematical details and definitions on how to achieve distribution-awaresampling in qCORAL are shown in Section 4.2. This section also compares the benefits and costsof discretization against our approach. Section 4.3 compares the accuracy and performance ofour distribution-aware sampling procedure with the same analysis based on the discretization[20] of the usage profile on several case studies.
4.1 Illustrative Example
To illustrate our approach, we introduce a small code snippet modeling a safety checkfor a simplified flap controller of an aircraft modified from [20] (see Figure 4.1). The controlleris composed of a flap actuator and a safety check to avoid overrun of the flap. The variablesinfluencing the behavior of the flap are the goal position, the current position of the flap, and thewind effect. The actuator performs a move towards the goal but the actuation can be hindered bythe effect of the wind that can lead to an overrun of the flap.
The goal position can vary in the range [−10,10], while the current position of the flapwhen the next control step is activated is assumed to be within [−5,5]. For both these variables,any value in the domain is considered equally likely, i.e. the concrete inputs are assumed asrealizations of a Uniform distribution over each variable domain.
The wind effect is instead assumed to behave as a Normal distribution, with mean 0 anda standard deviation which depends on the strength of the wind. We will consider two different
4.2. TECHNIQUE 38
goal = new Uniform(-10, 10);flapPosition = new Uniform(-5, 5);
windEffect = new Normal(0, 0.5, -15, 15); /* weak wind *///windEffect =// new Normal(0, 7.25, -15, 15); /* strong wind */
actuatorEffect = 5;MAX_POSITION = 15;MIN_POSITION = -15;
// actuatorif(goal < 0){flapPosition = flapPosition - actuatorEffect + windEffect;
} else{flapPosition = flapPosition + actuatorEffect + windEffect;
}
// safety checkif (flapPosition > MAX_POSITION ||
flapPosition < MIN_POSITION){throw new OverrunException();
}
Figure 4.1: Flap controller.application scenarios: in case of weak wind, the standard deviation is assumed to be 2, meaningthat the effect is quite close to 0 most of the time; in case of strong wind, the standard deviationis assumed to be 7.25, so larger values of the the wind effect are more likely. In either case, theeffect of the wind is bounded by the interval [−15,15]. This simplified model exemplifies howthe uncertainty about this physical phenomena can be taken into account for the analysis.
In Figure 4.1, the random distributions for the input variables are characterized by thevalue of their parameters and the lower and upper bound of the domain (which are always thelast two arguments). The Uniform distribution does not need additional parameters besidesthe domain. The Normal distribution is characterized by its mean and its standard deviation,respectively, besides the domain.
The probabilistic distribution of the wind effect can be obtained systematically fromtelemetry mission data. It is possible, for example, to measure the frequency over time of valuesoccurring within certain ranges during a mission to obtain realistic usage profiles. Techniquesfor the automatic inference of probabilistic profiles are described elsewhere, see e.g. [23].
Analyzing the example program. As mentioned in Chapter 2.3, Probabilistic Software Anal-ysis computes the probability of a certain event to occur or not during the execution of theprogram, given a usage profile. The result of this kind of analysis is not a boolean (indicatingpresence of an error), but a quantitative figure, whose value depends on both the program and itsusage. In this example the probability of throwing an OverrunException is only 0.04% in thepresence of weak wind, but it grows up to 8.43% when the wind is strong.
4.2 Technique
Similarly to probabilistic programming [25], we assume that each input variable isdefined over a bounded continuous interval [a,b] and its values are distributed according to
4.2. TECHNIQUE 39
a known distribution Distributioni(θi), restricted, or truncated [14], to this interval. θi is a(possibly empty) constant vector of known parameters characterizing the distribution, e.g.,a Normal distribution is characterized by the values of its mean and its variance, while anExponential distribution only by its mean. We will use the notation Distributioni(θi)|a,b torepresent the distribution truncated to the interval [a,b]. The mapping between input variablesand their corresponding truncated probability distributions constitutes the usage profile (see thevariable declaration in Figure 4.1).
As mentioned in Chapter 2.2, the output of ICP is a set of boxes containing all the inputssatisfying PCT . Each box is defined by the conjunction of constraints of the form vi ∈ [ai,bi],where each variable is restricted to a particular interval within its domain. Since in general therange of Distributioni(θi) may fall outside [ai,bi], we need to restrict the sampling to [ai,bi]. Asimple approach would be to generate for each variable vi a set of samples x1
i ,x2i , . . .x
ni (drawn
according to the distribution) and then prune out all the x ji /∈ [ai,bi]. However this may be
inefficient, especially if the intersection between the range of Distributioni(θi) and [ai,bi] issmall.
We propose an efficient solution obtained by exploiting the results of probability theoryfor truncated distributions [14]. Consider a random variable vi with Distributioni(θi). Eachdistribution is associated to a unique, known, cumulative distribution function CDFi(t) definedas CDFi(t) = Pr(vi <= t), which we will use for the sampling. Consider also a non-emptyinterval [ai,bi]. Let the random variable rvi be the restriction of vi to the interval [ai,bi], then thefollowing result holds [14]:
CDFrvi(t) =CDFi(max(min(t,bi),ai)−CDFi(ai)
CDFi(bi)−CDFi(ai)
� �4.1
Furthermore, the cumulative distribution has inverse:
CDF−1rvi
(u) =CDF−1i (CDFi(ai)+u · (CDFi(bi)−CDFi(ai))
� �4.2
where 0≤ u≤ 1. For the most common continuous distributions both CDF(·) and its inverseCDF−1(·) can be computed efficiently using off-the-shelf tools or libraries (e.g., [1]). In thefollowing we show how to use these functions to obtain samples of any truncated distributionsfrom samples drawn from uniform distributions which can be easily obtained from many existingoff-the-shelf libraries.
Example. Recall from Figure 4.1 that variable goal follows an Uniform distribution in theinterval [−10,10]. According to the definition above:
CDFgoal(t) =
(t +10)/20 if −10≤ t ≤ 10
1 if t > 10
The interested reader can refer to [42] for the definition of the CDF function for the most popular
4.2. TECHNIQUE 40
continuous distributions. The ability to compute CDF−1rvi
(·) and its inverse allows us to implementa general sampling strategy for continuous distributions restricted to intervals of interest. Indeed,to take a sample from rvi, i.e. variable xi restricted to [ai,bi], it is sufficient to generate a sampleu from a Uniform distribution over [0,1] (by using any robust pseudo-random generator) and usethis sample u to generate the sample ¯rvi =CDF−1
rvi(u) from the restricted random variable rvi.
This approach allows to bring distribution-awareness to ICP-enabled stratified sampling,thus allowing to achieve both the precision and scalability of distribution-aware sampling andthe improved convergence rate due to stratified sampling.
4.2.1 Distribution-aware versus discretization-based approaches
The symbolic probabilistic analysis work from [8, 20, 33] assume that the probabilitydistribution over the input domain are specified by a usage profile (UP) defined as:
UP =
c1 : p1
c2 : p2
. . . . . .
� �4.3
where the ci are a partition of the input variables, i.e. ∪ici = D and ci∩ c j 6= /0 =⇒ i = j, and
∑i pi = 1. Each pair ci : pi is called usage scenario.This formalism for UPs allows to arbitrarily partition the input domain into a finite set
of regions, each with an assigned probability. The constraints ci can be arbitrarily complex,making the formalism expressive enough to predicate about non trivial relations among inputvariables. However, if the values of an input are distributed according to a continuous probabilitydistribution, casting this case into a finite UP requires a discretization procedure, partitioningthe domain of each variable into a finite number of intervals and assigning to each intervala probability computed from the original continuous distribution. Depending on the numberand the size of the intervals, discretization may be an arbitrarily precise approximation of thecontinuous distribution.
Nonetheless, the unavoidable loss of precision due to discretization may introduce a biasin the analysis results, when the approximation is not fine enough. On the other hand, a finerdiscretization requires to partition variables domain into a larger number of intervals. Assuming v
input variables are partitioned into m intervals each one, the total number of constraints to obtaina discretized version of the original UP would be vm. Though the complexity of the analysis islinear in the number of usage scenarios, the latter grows exponentially with the required precisionof discretization, limiting the scalability of the analysis.
4.3. EVALUATION 41
4.3 Evaluation
Distribution-aware sampling aims at providing direct support to input variables charac-terized by continuous probability distributions. This section reports on two different experimentswe conducted to evaluate our proposed technique. The first experiment compares the resultsof qCORAL and NProbability with respect to precision and efficiency. The second experimentcompares the precision, lack of bias, and scalability of distribution-aware sampling against theanalysis with discretized usage profiles.
For our evaluation, we used the publicly available benchmarks of VolComp [3] andqCORAL. The subjects from the VolComp benchmark only contain linear constraints (thatwe translated in the input format for our tool). These subjects are: a heart fibrillation riskcalculator (ARTRIAL), a steering controller to deal with wind disturbances (CART), a coronarydisease risk calculator (CORONARY), an estimator of chronic kidney’s disease (EGFR-EPIand EGFR-EPI-SIMPLE), an inverted pendulum (INVPEND), and a model of a robot decid-ing how to pack goods of different weights into envelopes with limited capacity (PACK). Acharacterization of these subjects can be found in Table 3.2.
The subjects from the qCORAL benchmark contain non-linear constraints and morecomplex mathematical functions (e.g., sine); they are a model of the Apollo lunar vehicleautopilot (APOLLO) and two core modules of an aircraft collision-detection monitor (CONFLICTand TURN LOGIC). These subjects were implemented in Java and were analyzed using SymbolicPathFinder (SPF) [40] to compute the PCs (of paths leading to assert violations). Like before(3.6.3), we selected 70% of the PCs arbitrarily to mimic a property that is satisfied on someof the paths. To reduce the analysis time and help with the realization of the experiments, allconstraints involving equality in those three subjects were removed. This “optimization” doesn’timpact the result of the estimation, because the probability of a continuous random variablebeing equal to a determined value or another continuous random variable is equal to 0 [17]. 1
Table 4.1 contains more information about those subjects.
Subject Asrt. #Paths #Ands #Ar. Ops. 0-var./Parts.qCORAL subjects [8]
APOLLO 21 866 10037 249655 (4) 216/253CONFLICT 22 14 70 693 (4) 8/14
TURN LOGIC 23 73 505 1718 (2) 65/73Table 4.1: Characterization of the qCORAL Benchmark
Reference results. To compare the accuracy of the different PSA approaches, we solved thequantification problem with the commercial tool Mathematica (version 10). We used the off-
1Although floating-point numbers have finite precision, thus it is formally incorrect to state that P(X = k) orP(X1 = X2) is 0, the probability of such equality is many orders of magnitudes smaller than the accuracy we canreasonably achieve with qCORAL. Thus the impact of the samples satisfying such an equality is negligible, even inthe extremely unlikely case one of those samples is actually drawn.
4.3. EVALUATION 42
the-shelf function NProbability with default arguments. This procedure is designed to providesolutions to a broad range of problems. Compared to PSA-tailored quantification methods, thisresults (in some cases) in both a slower performance and exceptions, reported by Mathematica,where NProbability fails to achieve precise results. Different configurations of NProbability mayavoid these exceptions, however they would require human expertise beyond the off-the-shelfuse of the tool.Execution environment. We run qCORAL on an Intel Core i7 920 (2.67GHz, 8M cache)machine, with 8GB of RAM. Considering the higher computational demand for NumericalIntegration with Mathematica, we performed this operation on an r3.large Amazon EC2 machine,running on an Intel Xeon E5-2670 v2 (2.50 GHz, 25M cache) with 16GB RAM2. Both themachines run Ubuntu server 64 bits.
4.3.1 Comparison with NProbability
Figure 4.2 reports the results of analyzing each subject with qCORAL and NProbability.qCORAL performs a single sampling round with 100k samples, thus no iterative sampling
allocation (which will be presented in Chapter 5) is used for this experiment. The cells shadowedin grey for NProbability highlight the cases for which Mathematica reported an exception andthe results might thus not meet the default prescribed accuracy of at least five decimal digits.A “-” is used to mark the cases where the analysis failed to return results within 40 hours ofprocessing.
To evaluate distribution-aware sampling, we experimented with different continuousinput distributions. In the first set of experiments we assigned to each variable a truncated Normaldistribution centered in the middle of the variable domain and with standard deviation equalto 1/6 of the domain length, and truncated by the bounds of the domain. We set the standarddeviation to 1/6 so that already the non-truncated distribution has probability 99% to generate asample within the domain. Truncation introduces a small correction to guarantee that all samplesfall within the bounds. In the second set of experiments we assigned to each variable a truncatedExponential distribution. The rate parameter of the distribution has been tuned again to make99% of the samples fall within the original variable domain. Since Exponential distributions aredefined over positive domains, we excluded the subjects whose inputs have negative domains.
For all the cases where NProbability terminated without exceptions the results of qCORALare consistent, at the reported result accuracy (σ ). In several cases the results of qCORAL areexact σ = 0 (up to Java double numbers accuracy) and match those of NProbability. Theexecution time of qCORAL is significantly shorter than the one required by NProbability for thesame subjects, especially on more complex subject where the difference is by several orders ofmagnitude. qCORAL also produces more robust results compared to NProbability, completingthe analysis for all the subjects without any exception. This is due to the intrinsic robustness of
2www.aws.amazon.com/ec2
4.3. EVALUATION 43
Subject Assertion NProbability qCORAL [100k]solution time(s) estimate avg. σ time(s)
Gaussian distributions
ARTRIAL1 0.031483 28.14 0.031483 0.00e+00 0.602 0.000118 37.08 0.000119 0.000001 0.763 0.968101 ≈ 26m 0.964187 0.001159 2.31
CART 4 0.999896 ≈ 12m 0.993029 0.014424 1.235 0.997998 ≈ 13m 1.005696 0.014901 1.30
CORONARY 6 0.000005 73.03 0.000005 5.26e-08 0.497 0.000000 12.41 0.000000 2.91e-09 0.40
EGFR EPI 8 0.008055 10.35 0.008055 0.00e+00 0.729 0.006240 7.97 0.006240 0.00e+00 0.63
EGFR EPI (SIMPLE) 10 0.592727 3.29 0.592715 0.000229 0.4811 0.171350 2.09 0.171350 0.00e+00 0.42
INVPEND 12 0.002235 ≈ 23m 0.002248 0.000030 2.35
PACK
13 0.999788 164.52 0.999788 0.00e+00 1.4214 0.065674 148.88 0.065674 0.00e+00 1.2215 0.008084 154.95 0.008084 0.00e+00 1.2516 0.000000 8.82 0.000000 0.00e+00 0.6617 0.036888 ≈ 31h 0.036883 0.000077 3.6718 0.525443 ≈ 32h 0.515872 0.023695 4.0119 0.986260 ≈ 36h 0.999885 0.00e+00 4.35
APOLLO 21 0.702961 ≈ 4h 0.622393 0.032056 3.02CONFLICT 22 0.000091 103.93 0.500701 0.000702 1.52
TURN LOGIC 23 0.176554 ≈ 10m 0.717674 0.029120 0.87Exponential distributions
ARTRIAL1 0.053505 29.07 0.053505 0.00e+00 0.642 0.00e+00 35.51 0.00e+00 0.00e+00 0.703 0.995979 ≈ 15m 0.996645 0.000219 2.26
CORONARY 6 0.000047 32.99 0.000050 0.000007 0.487 0.00e+00 5.62 0.00e+00 0.00e+00 0.34
EGFR EPI 8 0.731059 0.16 0.078329 0.00e+00 0.619 0.116840 15.36 0.116840 0.00e+00 0.59
EGFR EPI (SIMPLE) 10 0.698996 2.91 0.698996 0.00e+00 0.4011 0.032105 2.04 0.032105 0.00e+00 0.37
PACK
13 0.948792 329.84 0.948792 0.00e+00 1.4614 0.393585 242.62 0.393585 0.00e+00 1.2315 0.149316 254.56 0.149316 0.00e+00 1.2616 0.000214 18.77 0.000214 0.00e+00 0.6517 0.243622 ≈ 1.2h 0.243619 0.000377 3.5918 0.641769 578.15 0.640926 0.002324 3.8619 0.952801 303.06 0.952801 0.00e+00 4.21
APOLLO 21 - +40h 0.643080 0.0631531 2.68CONFLICT 22 0.00e+00 36.02 0.024934 0.00513 1.04
TURNLOGIC 23 0.031814 ≈ 2h 0.736356 0.059437 0.77Table 4.2: Distribution-aware sampling: comparison of NProbability and qCORAL.
simulation-based approaches.A final note concerns the result of qCORAL for assertion 5 with Normal usage profile.
This result is indeed larger than 1, which is clearly not a valid probability. This is due tothe accumulation of inaccuracies of the subproblems estimates, when the result is close to 1.However, the corresponding σ makes the result compatible with NProbability. Cutting it to 1would alter the confidence intervals computable with the estimate and σ , thus we report the
4.3. EVALUATION 44
estimate as is, including accumulated errors.
4.3.2 Comparison with Discretization
This section compares the accuracy vs scalability trade-off of distribution-aware samplingwith the same analyses performed on discretized usage profiles. For the experiments reportedin Figure 4.3 we assigned for each subject a truncated Normal distribution to two of the inputvariables, following the same parameterization procedure described for the previous experiments.All the other variables have the original Uniform distribution defined in [3, 4]. Only two variableshave been assigned a non-uniform profile for scalability reasons, since the size of discretizedusage profiles grows exponentially in the number of non-uniform variables (cf. Section 4.2.1).This results in a complexity already sufficient for comparing the approaches.
There are several possibilities for discretizing a continuous distribution. A simplesolution requiring no prior knowledge on the problem consists in dividing the domain into acertain number of equally large intervals and to assign each interval the probability it wouldhave according to the original distribution, i.e. the probability for an interval [a,b] wouldbe CDF(b)−CDF(a), where CDF(·) is the cumulative distribution function of the originalcontinuous distribution. A deeper knowledge of the constraints to be quantified may allowmore effective discretization where smaller intervals are used to increase the resolution of theapproximate distributions around the points mostly affecting the satisfaction of the constraints tobe quantified. However, this would require in general human expertise on the specific problem.
The results of our experiments are reported in Figure 4.3. We keep the result of NProba-bility as reference value. Notice that the results in this table differ from those in Figure 4.2 dueto the different usage profile. Furthermore, with the simplified usage profile we use for theseexperiments NProbability always terminates without exceptions.
We discretized the domain of the two non-uniform variables in 3 and 6 intervals. Thetotal number of usage scenarios composing the discretized profile is thus 9 and 36, respectively(cf. Section 4.2.1). A too coarse-grained discretization may introduce a bias in the result dueto the loss of information. Finer discretization improves the precision of the result, but doesnot scale (due to the exponential blowup in the number of usage profiles). These two situationsare visible in Figure 4.3, where the results for a three interval discretization deviate from thereference value more than those for the six intervals. Distribution-aware sampling prevents therisk of introducing such biases.
Finally, finer discretization requires a higher computation cost. Though this cost mightbe reduced leveraging the caching of partial results for independent subproblems shared by thedifferent usage scenarios, the worst case complexity remains exponential. Even with the relativelycoarse-grained discretization we applied on only two non-uniform variables, the analysis timefor the discretized profiles takes longer than with distribution-aware sampling. The latter growsinstead only linearly with the number of variables, thus scaling to significantly larger problems.
4.3. EVALUATION 45
Assertion NProbabilityDiscretization
qCORAL [100k]3 intervals 6 intervalssolution time(s) avg. est. time(s) avg. est. time(s) est. time(s)
ARTRIAL1 0.052928 15.83 0.067016 0.66 0.059714 0.73 0.052928 0.642 0.000284 15.05 0.000320 0.74 0.000279 0.83 0.000285 0.713 0.927292 648.34 0.923024 2.85 0.917593 2.87 0.924958 2.31
CART4 0.974609 11.10 0.973542 1.63 0.964577 2.29 0.991601 1.125 0.982561 11.87 0.984726 1.57 0.982017 2.28 0.992781 1.15
CORONARY6 0.000201 1.81 0.000310 0.55 0.000241 0.61 0.000202 0.537 0.000033 0.35 0.000033 0.43 0.000059 0.45 0.000033 0.42
EGFR EPI8 0.130741 5.73 0.128302 0.65 0.128934 0.71 0.130741 0.689 0.099552 4.30 0.100637 0.63 0.099036 0.65 0.099553 0.62
EGFR EPI (SIMPLE)10 0.597568 1.74 0.596731 0.56 0.595728 0.70 0.597526 0.4811 0.159820 1.05 0.198692 0.50 0.178238 0.56 0.159819 0.43
INVPEND12 0.051223 24.59 0.051119 13.54 0.051276 50.24 0.051096 2.14
PACK13 0.984252 186.83 0.964020 1.44 0.990221 1.47 0.984252 1.4314 0.284253 136.10 0.314348 1.21 0.303445 1.24 0.284253 1.2015 0.089651 140.28 1.039513 1.23 0.096642 1.27 0.089651 1.2216 0.000099 10.26 0.000136 0.65 0.000113 0.69 0.000099 0.6317 0.156305 ≈ 2h 0.201841 6.02 0.200773 10.84 0.186872 3.6018 0.623169 334.15 0.645441 6.72 0.631922 12.20 0.632369 3.9119 0.985956 300.59 0.945867 8.89 0.969026 14.08 0.985955 4.41
APOLLO21 0.600787 ≈ 2h 0.633724 3.15 0.618542 3.61 0.063177 3.51
CONFLICT22 0.049456 101.09 0.499715 5.77 0.500178 18.10 0.500100 1.87
TURNLOGIC23 0.370931 443.62 0.7317807 1.70 0.725658 3.94 0.727923 1.13
Table 4.3: Comparison of different discretization methods. Discretization invokesqCORAL [8] once for every (discretized) region within each constraint partition. For
every subject, two input variables are normally distributed and the others are uniform.
464646
5Iterative Approaches for Optimal Sam-pling Allocation
Statistical methods proved to be very effective in solving the integration problems behindprobabilistic analysis. Especially when the number of variables grows, statistical methodsoutperform symbolic and numerical methods [29]. Nonetheless, general statistical integrationmethods available off-the-shelf are not capable of exploiting all the information about a programbehavior obtained through symbolic execution. It is indeed evident that certain constraints havehigher impact in driving the program execution toward the occurrence of a target event. Similarly,certain constraints have higher impact on the convergence rate of statistical estimation, sincethey provide more information about the possibility for the target event to occur.
A systematic analysis of the path conditions leading to the occurrence of the targetevent allows for ranking the constraints according to their impact on the convergence of thestatistical analysis. Exploiting such ranking, we iteratively re-focus the sampling to gathermore information about the satisfaction of the most important constraints first, achieving higherestimation accuracy in a shorter time.To rank the impact of each constraint, we propose threedifferent strategies: two of them are based on gradient descent optimization [36] while thethird one uses a simple but efficient and often accurate heuristic giving more importance to theconstraints whose satisfaction probability estimates are farther from convergence. Experimentalresults show significant improvement over built-in routines of general purpose mathematicaltools, both in terms of accuracy of results and analysis time.
5.1 Summary of the Technique
The divide-and-conquer procedure reported in Chapter 3 solves the problem of quantify-ing the solution space of PCT in terms of simpler independent subproblems: each independentconstraint is sampled in order to estimate the probability of satisfying it, as well as the varianceof this estimator; these partial results are then composed according to the conjunction anddisjunction composition rules. It further caches and reuses the results for the subproblems to
5.2. GRADIENT-DESCENT VARIANCE MINIMIZATION 47
speed up the quantification. The estimates for the subproblems are composed according to thedisjunction and conjunction rules from Equations
� �3.2 /� �3.3 and
� �3.4 /� �3.5 , respectively. The
variance of such estimators is composed as well, providing an index of the accuracy of the finalresults. Although the asymptotic convergence of the compositional estimators is guaranteed [8],i.e., when the number of samples used to obtain the local estimators for each subproblem growsto ∞), the convergence rate of the procedure is hard to quantify and may be slow in practicethe approach we present cannot distinguish which constraint partitions contribute the most toimproving the estimate.
This chapter introduces an iterative sampling approach to speed-up the convergencerate of the quantification procedure. At each iteration, the sampling is focused on the parts ofthe input space that are likely to have the largest influence on the variance of the composedestimator obtained from the previous iteration, with the goal of minimizing the variance and thusincreasing the overall accuracy of the estimation. We explore three iterative approaches.
The first approach is based on gradient descent optimization [36] (Section 5.2) whichprovides a natural solution to the sampling allocation problem. It uses the composition rulespresented in Sections 3.1 – 3.2 to compute the gradient of the global variance with respect tothe number of samples allocated to each local estimator. The impact of each local estimator, asquantified by the gradient, is then used to decide how many new samples to allocate for eachsubproblem, aiming at minimizing the global variance.
The second approach overcomes the computational overhead related to bootstrappingnew sampling procedures for multiple subproblems. It uses a relaxed form of gradient descentoptimization based on a sensitivity analysis (Section 5.3), where, at each iteration, new samplesare allocated only for the single most influent subproblem, as identified by the gradient.
The third and last approach introduces a simple heuristic sampling allocation that, ateach iteration, allocates new samples only for the subproblem whose estimator has the largestvariance (Section 5.4). This heuristic is computationally cheaper than the other optimizationmethods because it does not require the computation of the gradient of the global variance withrespect to the number of samples allocated for each subproblem. However, our experimentsshow that this simple heuristic works well in practice.
The three different strategies discussed in the next sections will be evaluated on severalcase studies in Section 5.5.
5.2 Gradient-descent variance minimization
Gradient descent is a simple optimization method for finding the minimum of functionsfor which derivatives can be defined [36]. The gradient descent method starts with an initial(random) candidate solution and iteratively discovers new candidates closer to the optimum. Ateach step, a new candidate solution is produced following the direction of the negative gradientof the function. As a local search method, it can get stuck in local optima. However, in our
5.2. GRADIENT-DESCENT VARIANCE MINIMIZATION 48
context there is a unique minimum, as the variance is guaranteed to decrease with each newsample.
The dependency of the global variance on the number of samples allocated for eachsubproblem can be computed in analytical form combining Equations
� �2.2 to� �2.3 and
� �3.2to
� �3.5 . For the sake of simplicity we will for now ignore ICP-based stratified sampling,thus assuming each subproblem to be quantified by simple Hit-or-Miss Monte Carlo (HM-MC)(Equation 2.2). We will bring ICP-based stratified sampling later.
As an example of this computation, consider the following PC from the flap controllercode (Figure 4.1):
goal<0 && flapPosition + actuatorEffect + windEffect > 15
The quantification problem for this PC can be reduced to the quantification of the independentconstraints c0 : goal<0 and c1 : flapPosition + windEffect > 10 (where the constantactuatorEffect has been already evaluated), whose solution space is quantified by theestimators x0 and x1, respectively. If n0 samples are allocated for the estimation of x0 and n1 forthe estimation of x1, their respective variances would be (Equation
� �2.2 ):
Var[x0] =x0 · (1− x0)
n0Var[x1] =
x1 · (1− x1)
n1
� �5.1
The variance of the estimator for the PC as a function of n0 and n1 can then be computed applyingthe conjunction composition rule (Equation
� �3.5 ):
Var(n0,n1) = x20 ·
x1 · (1− x1)
n1+ x2
1 ·x0 · (1− x0)
n0+
x0 · (1− x0)
n0· x1 · (1− x1)
n1
� �5.2
An analogous procedure can be applied to deal with disjunctive forms.Our goal is now to minimize the function Var(n0,n1, . . . ,nm), which denotes the variance
of the global estimate as a function of the number of samples allocated for the estimation of eachof the m+1 independent constraints the quantification problem has been split in. We want tofind a sequence of sample allocations that brings global variance near 0 quickly. Note that 0 isthe unique global minimum for the variance and it is reachable when the number of samplesgrows to infinity.
The initial solution n0 = [n00,n
01, . . . ,n
0m] can be computed in a bootstrap stage where an
arbitrary number of samples is allocated uniformly to each estimation subproblem. This initialround of sampling provides also an initial estimate of the expected value and the variance of eachindependent constraint, which will be used to estimate the value of Var(n0) at the initial point.
Given the value of nk at step k, the value of nk+1 is computed according to the followingformula:
nk+1 = nk− γ ·∇Var(nk)� �5.3
where γ is the step size (we will get back to this later) and ∇Var(n) =
[∂Var/∂n0,∂Var/∂n1, . . . ,∂Var/∂nm] is the gradient, i.e., the vector of the partial derivatives
5.2. GRADIENT-DESCENT VARIANCE MINIMIZATION 49
of Var(·) with respect to the arguments ni. Intuitively, the gradient indicates at each step “howmuch” each of the independent constraints can contribute to minimize Var(·).
Recall that the quantification problem consists in estimating the probability that an inputsatisfies any of the path conditions in PCT , given a probability distribution on the input space. Inother words, we aim to estimate the probability of satisfying the disjunction of the PCs in PCT ,and each PC is the conjunction of independent constraints. Exploiting the compositional rulesof Chapter 3, the gradient of the global variance with respect to the number of samples to beallocated to each independent subproblem can be computed compositionally too.
Theorem 2. Derivative of disjunction compositions. The derivative of the variance of theestimator xc for the disjunction c = c0 ∨ c1 ∨ . . .ck with respect to the number of samples ni
allocated to the estimators xi for the constraints ci (i ∈ [0,k]) can be computed as:
∂Var[xc]
∂ni6 ∑
i∈[0,k]
∂Var[xi]
∂ni
� �5.4
Proof. The composition rule in Equation� �3.2 overestimates the variance of a disjunction as
the sum of the variances of the disjuncts. The proof follows from the linearity of the derivativeoperator.
Theorem 3. Derivative of conjunction compositions. The derivative of the variance of theestimator xc for the conjunction c = c0 ∧ c1 ∧ . . .ck with respect to the number of samplesni allocated to the estimators xi for the constraints ci (i ∈ [0,k]) can be computed, where theconstraints ci are independent, according to the definition introduced in Section 3.2, and theestimators xi use HM-MC, as:
∂Var[xc]
∂ni=−Var[xi]
ni∏
j∈[0,k], j 6=i
(E[x j]
2 +Var[x j]) � �5.5
Proof. Consider the computation of the derivative with respect to n0. The commutativity of theconjunction allows to generalize to any ni. From Equation
� �3.5 , the variance of the estimatorx0,1 for the conjunction c0,1 = c0∧ c1 is:
Var[x0,1]= E
[x0]2 ·Var
[x1]+E
[x1]2 ·Var
[x0]+Var
[x0]·Var
[x1] � �5.6
Since all the estimators are HM-MC, by applying Equation� �2.2 , we can rewrite
� �5.6 as:
Var[x0,1]= E
[x0]2 · E[x1
]· (1−E
[x1])
n1+
� �5.7
E[x1]2 · E[x0
]· (1−E
[x0])
n0+
E[x0]· (1−E
[x0])
n0·
E[x1]· (1−E
[x1])
n1
5.2. GRADIENT-DESCENT VARIANCE MINIMIZATION 50
whose derivative with respect to n0 is (simplified applying again Equation� �2.2 ):
∂Var[x0,1]
∂n0=−
Var[x0]
n0·(E[x1]2+Var
[x1]) � �5.8
Exploiting the associativity property of the conjunction operator, the variance of the estimatorx0,2 for the constraint c0,2 = (c0∧ c1)∧ c2 can be computed as in Equation
� �5.6 , where x0 isreplaced by x0,1 and x1 by x2:
Var[x0,2]= E
[x0,1]2 ·Var
[x2]+E
[x2]2 ·Var
[x0,1]+Var
[x0,1]·Var
[x2] � �5.9
The derivative of V [x0,2] with respect to n0 can be calculated as:
∂Var[x0,2]
∂n0=
∂(E[x0,1]2 ·Var
[x2])
∂n0
� �5.10
+∂(E[x2]2 ·Var
[x0,1])
∂n0+
∂(Var[x0,1]·Var
[x2])
∂n0
=∂Var
[x0,1]
∂n0·(E[x2]2+Var
[x2])
=−Var[x0]
n0·(E[x1]2+Var
[x1])·(E[x2]2+Var
[x2])
where the terms not directly dependent on n0 behaves as constant coefficients for the derivativeoperator. Repeating the same construction for x0,3, x0,4, . . . x0,n the proof follows.
The gradient descent method terminates when either the target variance is achieved orwhen the gradient gets close enough to 0. The latter indicates the optimum has been reached,usually within a finite accuracy. Notably, the convergence speed of gradient descent methods isproportional to the value of the gradient. This implies that the closer the gradient gets to 0 theslower the convergence is [36]. Nonetheless, for the problem at hand the improvement on theconvergence rate of the global estimator is still significant, as will be shown in Section 5.5.
As a final remark, notice that the expected values of the estimates obtained by thecomposition rules of Chapter 3 are not affected by the gradient descent procedure. The estimatorskeep their unbiasedness and their consistency, while only the rate of convergence of the varianceto 0 is possibly optimized.
Going back to our example, assume that, after allocating during the initial bootstrapphase 1000 samples for estimating each of the constraints c0 : goal<0 and c1 : flapPosition+ actuatorEffect + windEffect > 15, considering the weak wind profile we obtainedthe estimates x0 = 0.489 and x1 = 0.088. The gradient ∇Var(n0,n1) after the first iterationwould be the vector [−1.955 ·10−9,−1.921 ·10−8]. Thus, the sampling budget available for thenext iteration should be assigned to x0 and x1 proportionally to their corresponding derivatives,roughly 9.2% of the samples for c0 and 90.8% of the samples for c1.
5.2. GRADIENT-DESCENT VARIANCE MINIMIZATION 51
5.2.1 Choosing the step size.
The partial derivatives composing the gradient are used to decide how the sampling
budget for the next iteration will be distributed among the independent constraints. This budgetis quantified by the step size γ . In general, if γ is too small, then the algorithm will convergevery slowly. On the other hand, if γ is not chosen small enough, then the algorithm may usesampling time inefficiently. In our case, we have to take into account the overhead of starting anew sampling procedure for each independent constraint at each iteration. For a too small budget,the bootstrapping time for the sampling procedures might overcome the time for sampling,increasing the computational overhead. An optimal value for γ depends in general on the specificproblem at hand. Furthermore, instead of fixing the value of γ , we fix the total number of samplesto be allocated for each iteration and compute γ so to use it all, i.e. γ ·∑∂Var/∂ni has to beequal to the total number of samples allowed for each iteration. Since only an integer number ofsamples can be allocated for each subproblem, possible decimal results are rounded up to thesmallest larger integer.
For this work we empirically evaluated different values for the total number of samplesto be allocated during each iteration, and in turn γ (Section 5.5). Adaptive decisions for γ havebeen proposed too (e.g., [36, 43]), and we plan to evaluate them in the future.
5.2.2 ICP-based stratified sampling.
ICP-based stratified sampling (see Section 2.3.3) can be used to reduce the variance ofthe estimates for the single independent subproblems. ICP is used to partition the samplingspace into a set of disjoint boxes containing all the solutions for the constraint, pruning out theregions of the domain containing no solutions. With the same number of samples, this approachcannot perform worse than HM-MC, and usually performs better [8]. Therefore the varianceobtainable by HM-MC can be seen as an upper bound of the one of stratified sampling for thesame problem.
The gradient descent procedure we defined is based on HM-MC and decides how manysamples to allocate on each subproblem during each iteration. Enhancing the local estimatorswith stratified sampling can only produce better results (which will be also reflected by the moreaccurate values for the local estimates used to evaluate V (·) on the next iteration) but requires anadditional decision about how to distribute the samples allocated on an independent constraintamong the boxes containing its solutions.
Our decision strategy is to distribute 2/3 of the samples proportionally to the productbetween the variance of the local estimate within the box and the size of the box, and 1/3uniformly among all the boxes. Recalling Equation
� �2.3 , both the variance and the size of thebox directly affect the variance of the stratified sampling estimates. With this heuristic we takeinto account this dependency. The remaining third of samples is distributed uniformly to speedupthe convergence of the estimators for all the boxes, and thus a better assessment of their variance
5.3. SENSITIVITY ANALYSIS AND COMPUTATIONAL OVERHEAD 52
for the next iterations. This is especially relevant when the probability of satisfying the targetproperty within the box is close to 0; if no samples satisfy the constraint restricted within thebox, the variance is incorrectly estimated as 0 and the box would receive 0 samples on the nextiteration, preventing the local estimator to assess the actual variance.
5.3 Sensitivity analysis and computational overhead
Each sampling round requires to start a sampling procedure for each box of each inde-pendent constraint for which we allocate new samples. This operation introduces a significantoverhead, especially when the number of subproblems is large and the partial derivatives in thegradient are mostly in the same order of magnitude. This implies that the budget will be dis-tributed almost uniformly, requiring to add only a few samples for each independent subproblemat each iteration.
A sub-optimal strategy to trade convergence rate for lower computational overheadconsists in a relaxation of the gradient descent method based on sensitivity analysis. Thesensitivity of the global variance Var(·) with respect to the number of samples ni allocated to theconstraint ci is defined by the partial derivative ∂Var/∂ni. The single constraint mostly affectingthe global variance is the one having the larger sensitivity, in absolute value (note that all of thederivatives are negative since adding more samples can only decrease the global variance). Thestrategy thus consists in allocating the entire sampling budget for the next iteration to the singlemost important constraint.
Recalling our example, if ∇Var(n0,n1) = [−1.955 · 10−9,−1.921 · 10−8], the wholesampling budget for the next iteration will be allocated to increase n1 because the correspondingsubproblem has the highest expected impact on the global variance.
From a mathematical viewpoint, this means that instead of following the gradient, wefollow its projection on the single dimension providing the best improvement. This is in generalless effective than using all the information in the gradient, but may produce valuable results at alower computational cost, as we will show in Section 5.5.
5.4 Local heuristic for sampling allocation
The final strategy we included into our iterative approach is a low-overhead heuristic thatdoes not require to compute the gradient of Var(·). This heuristic simply prescribes to allocate ateach iteration all the samples to the single estimator having the highest variance.
Back again to our example, where after allocating 1000 samples for each of the subprob-lems c0 : goal<0 and c1 : flapPosition + windEffect > 10, we obtained the estimatesx0 = 0.489 and x1 = 0.088 with variance 2.498 ·10−4 and 8.025 ·10−5, respectively, this heuris-tics would require to allocate the entire sampling budget for the next iteration to increase n0 sincethe estimator x0 is the one showing the highest variance.
5.5. EVALUATION 53
Due to the nature of the problem, this heuristic intuitively guarantees the convergenceof the estimator, since the allocation of more samples to an estimator with positive variancealways strictly decreases its variance (Equation 2.2). This guarantees that all the estimators withnon-zero variance will eventually receive additional samples, following the convergence of theglobal estimator as well.
5.5 Evaluation
Iterative sampling allocation aims to improve the convergence rate of simulation-basedquantification by allocating samples to subproblems according to their predicted importance.Sections 5.2, 5.3 and 5.4 proposed three strategies to decide how to allocate samples to non-initial iterations of the quantification procedure. The first strategy – gradient – decides how manysamples to allocate to each subproblem pursuing a gradient descent minimization of the globalvariance. The second strategy – sensitivity – performs a gradient-based sensitivity analysis toidentify the single partial subproblem with highest impact on the global variance and allocatenew samples to improve the estimate. The third strategy – local – uses a low-overhead heuristicselecting the subproblem to improve only based on the variance of its local estimator, thus notrequiring a global impact analysis.
The goal of the three strategies is to increase the convergence rate of the compositionalsimulation-based quantification described previously. We take as baseline the distribution-awaresampling described and evaluated in the previous chapter. We evaluate the iterative techniqueswith a sampling budget per iteration of 1k and 10k samples (which determines the step size forthe gradient-based methods). We report the results of our experiments on selected subjects fromthe VolComp and qCORAL benchmarks in Figures 5.1 and 5.1. Each subject is identified by thename of the program and the id of the assertion (see Table 3.2 and 4.1 for more information).For the experiments in this section, we assigned each variable a truncated Normal distributionwith mean in the center of the variable domain (defined in the original papers of the benchmarks)and standard deviation equal to 1/6 of the domain length.Selection criteria. We excluded the subjects where the PCs were too simple to be split intoindependent conjuncts, i.e. where there is no choice on the subproblem to optimize. We alsoexcluded the cases where the ICP procedure perfectly partitioned the input domain so that 0variance has been obtained already after the initial bootstrap. Apart from these cases, we alsoexcluded EFGR-EPI(SIMPLE) because all approaches evaluated reached high accuracy valuesin a small time, which is explained by the low complexity of the subject.
5.5.1 Convergence rate
Figure 5.1 shows the convergence rate of the three approaches and the baseline throughplots having the wall-clock time on the x-axis and the average standard deviation of the global
5.5. EVALUATION 54
estimator (i.e., the square root of its variance) on the y-axis, in logarithmic scale. All theexperiments have been ran for 30 minutes. The initial bootstrapping has been performed bytaking 50k samples uniformly among all the subproblems.
10−7
10−6
10−5
0 1000
ARTRIAL (2) - 1k
10−4
10−3
0 1000
ARTRIAL (3) - 1k
10−3
10−2
10−1
0 1000
CART (4) - 1k
10−7
10−6
10−5
0 1000
ARTRIAL (2) - 10k
10−4
10−3
0 1000
ARTRIAL (3) - 10k
10−3
10−2
0 1000
CART (4) - 10k
10−3
10−2
0 1000
CART (5) - 1k
10−9
10−8
0 1000
CORONARY (7) - 1k
10−3
10−2
0 1000
APOLLO (21) - 1k
10−3
10−2
0 1000
CART (5) - 10k
10−9
10−8
0 1000
CORONARY (7) - 10k
baselinegradientsensitivitylocal10−3
10−2
10−1
0 1000
APOLLO (21) - 10k
Figure 5.1: Convergence rate of iterative sampling and the baseline approach on selectedsubjects (y-axis in logarithmic scale).
Gradient and sensitivity outperformed the baseline for all of the subjects and both 1kand 10k sampling budget per iteration. When more samples are allowed, these two approachesperforms almost equally. The best improvement is achieved for APOLLO (21), while the worst isfor ARTRIAL(3). These two subject are the most complex in terms of number of PCs and numberof conjuncts per PC. However, while in the case of APOLLO only a few subproblems have a
5.5. EVALUATION 55
high impact on the global variance, for ARTRIAL the subproblems to be analyzed have similarimpact on it. In particular, for APOLLO only a few subproblems have a large local variance anda high impact on the global result. This is an optimal condition for all three allocation strategies,which perform similarly. The baseline approach is instead unable to exploit this information andallocates samples on subproblems already close to convergence or with low impact on the globalresult. For ARTRIAL (3), there is not a big difference in the impact of the different subproblems,though a few of them have slightly larger effect on the global result. In this case the benefit ofgradient-based techniques is limited and also local does not provide a significant improvement.
The local strategy fails to improve the convergence rate, and actually slows it down,when the subproblems with highest variance have a low impact on the global result: ARTRIAL(2), and CORONARY(7). Indeed, the impact of a subproblem depends not only on its localvariance but also on the estimates and variance of the other subproblems that are in conjunctionwithin the PCs under analysis. In CORONARY (7) it is possible to observe a small spike for thelocal strategy. The initial sampling provides indeed only approximate estimates for the varioussubproblems. These estimates are improved through the subsequent iterations. The greediness oflocal makes the method to keep sampling from a single subproblem until its variance is reducedenough to move to another one. When more samples are allocated to a subproblem, not only thevariance of the corresponding estimator is decreased, but also the approximation of the globalresult is improved through the composition rules, possibly correcting a wrong initial assessment.For this reason it is possible to obtain small spikes when moving from one subproblem to another.
5.5.2 Time to converge
Figure 5.1 reports in tabular form the results of some of the experiments for a deeperinvestigation. All the analyses have been interrupted after 30 minutes. Target σ representsdifferent target accuracies in terms of standard deviation of the global result. We reported thecomputation time required by the three iterative allocation strategies and the baseline approachto achieve the target accuracy.
For lower accuracy (larger σ ), the gradient-based methods performs better with 1ksampling budget per iteration, while for higher accuracy 10k performs slightly better. In bothcases the convergence rate gets slower while moving toward higher accuracy. This observation isconsistent with the theory on gradient-descent optimization, since this optimization approachgets less efficient when the result approaches its optimum. Nonetheless, the two methodsoutperform baseline and local even for higher accuracies, since they are capable of exploitingmore information about the problem. Indeed, though all the derivatives tend to converge to 0when approaching the optimum, the small differences among them are still enough to improve onthe uninformed uniform allocation performed by baseline, while local may waste time samplingfrom low impact subproblems having high variance. For higher accuracy, when the differencesamong the derivatives get smaller, the expected impact of the different subproblems tends to
5.5. EVALUATION 56
Assertion Target σ
Time (s)Baseline Local Sensitivity Gradient1k 10k 1k 10k 1k 10k 1k 10k
APOLLO
2110−1 3.56 3.40 3.34 3.36 3.35 3.36 3.00 3.0510−2 191.49 64.13 5.97 6.11 4.73 5.20 6.13 4.4610−3 +30m +30m 253.03 235.47 130.79 124.79 297.78 138.54
CART
410−1 1.48 1.61 1.45 1.58 1.39 1.53 1.44 1.5610−2 83.40 28.08 4.10 3.63 3.25 3.29 6.99 4.3610−3 +30m +30m 527.37 526.14 217.71 221.28 641.56 310.85
510−1 1.53 1.58 1.47 1.52 1.48 1.55 1.50 1.5610−2 86.51 31.00 4.17 3.77 3.35 3.31 7.32 4.4910−3 +30m +30m 541.47 525.43 227.99 224.31 686.19 320.69
ARTRIAL
2
10−4 0.96 1.02 0.95 1.02 0.95 1.01 0.96 1.0510−5 7.10 4.44 0.95 1.02 1.11 1.08 1.57 1.4610−6 580.69 309.14 +30m +30m 17.25 15.53 40.28 24.0310−7 +30m +30m +30m +30m 1603.58 1475.79 +30m +30m
3
10−1 2.97 3.13 3.14 3.26 3.21 3.28 3.06 3.1910−2 4.93 3.72 3.14 3.26 3.21 3.28 3.62 3.3610−3 207.40 57.62 13.12 8.93 20.81 11.97 92.26 22.9410−4 +30m +30m +30m 1565.44 1730.84 972.12 +30m 1727.74
CORONARY
6
10−5 0.64 0.82 0.58 0.82 0.61 0.83 0.61 0.6410−6 2.99 2.33 0.58 0.82 0.65 0.84 0.74 0.6910−7 167.43 114.38 0.63 0.83 4.55 4.40 6.16 5.6410−8 +30m +30m +30m +30m 354.03 346.11 527.16 450.78
7
10−6 0.53 0.49 0.83 0.57 0.84 0.58 1.10 0.6010−7 1.08 0.81 0.83 0.57 0.85 0.58 1.11 0.6010−8 30.38 23.13 0.84 0.59 2.31 1.59 3.00 2.0510−9 +30m +30m +30m +30m 109.84 100.68 120.60 105.37
Table 5.1: Time to reach a target accuracy for incremental sampling techniques andbaseline approach for 1k and 10k sampling budget per iteration. Initial uniform sampling
bootstrap was 50k.
become similar. In this case, since there is no significantly better choice, it is more efficient totake more samples per iteration instead of consuming analysis time for updating the gradientand taking new decisions frequently. We plan to investigate in the future dynamic techniques tohandle this situation.
According to our experiments, sensitivity overall provides the best strategy for iterativesampling allocation, combining the effectiveness of gradient-based impact analysis with areduced overhead for decision making, eventually leading to more accurate quantification resultsin a shorter time.
575757
6Conclusion
This document presents qCORAL, an iterative, distribution-aware, compositional sta-tistical approach for the quantification of solution spaces for arbitrary complex mathematicalconstraints with application to probabilistic software analysis. Given a set of complex mathemati-cal constraints representing the path conditions collected with symbolic execution over a program,we use a statistical technique to estimate the probability of satisfying them assuming the valuesof the constrained inputs follow given continuous probability distributions. To speed-up theconvergence rate of the analysis, we proposed three strategies for iterative sampling allocation,which focus the sampling on the constraints that have the largest influence on the estimatedresults. Experimental evaluation of qCORAL shows promising results.
6.1 Future Work
In principle, our statistical approach can be applied to both integer and floating-pointdomains, but in practice it performs poorly for integer constraints if compared to counting-basedtechniques (particularly problematic are the equality constraints). One venue for future workis the handling of mixed integer and floating-point constraints, possibly through a combinationwith exact volume computations or guidance heuristics.
In the future we plan to handle multivariate input distributions (that relate multiple inputvariables). In theory our proposed approach works for such cases, however efficient samplingfrom multivariate distributions is an open problem in statistics, often involving more sophisticatedtechniques such as Gibbs sampling [31, 46]. We also plan to explore the automatic tuning of thesampling budget to allocate for each iteration for reducing the decision overhead by taking intoaccount the relative gain of each possible decision over the others.
6.2 Related Work
This work is related to probabilistic program analysis [25], probabilistic abstract inter-pretation [35], probabilistic model checking [28] and volume computations [16]. This section
6.2. RELATED WORK 58
discuss the work we consider the most closely related.qCORAL calculate the probability of a path, or more generally a set of paths, being
executed. Typically these paths lead to an event of interest, such as an assertion violation forexample. Techniques for doing such probabilistic analyses differ in the type of input distributionsthey consider, the language features supported, and the approach used to calculate the number ofsolutions.
One approach in this area was that of Geldenhuys et al.[22] that considered uniformdistributions for the inputs, linear integer arithmetic constraints, and used LattE Machiato [16]to count solutions of path conditions produced during symbolic execution. Their goal was tocalculate the probability of covering branches and assertion violations in the code. One of themain technical differences between this work and others based on symbolic execution (includingthe present dissertation) is that probabilities are calculated at each branching point rather thanafter symbolic execution is finished. Here we support complex constraints, including non-linearconstraints, and we use a statistical approach to count solutions. Our proposed approach iscompositional and hence we believe it is applicable in the incremental setting from [22] .
Sankaranarayanan et al. [47] and Filieri et al. [20] recently proposed similar techniquesto compute probabilities of violating state assertions. Both techniques remove the restrictionof uniform distributions, although in the latter case it is by discretizing the domain into smalluniform regions. As with [22] both approaches only consider linear constraints. Their techniquesalso build on symbolic execution to compute a relevant set of symbolic paths of a program thatleads execution to some assertion of interest. Both techniques estimate probability of exercisingthose paths and violating the corresponding assertion from such a set. Sankaranarayanan et al.
developed a customized algorithm for under and over-approximations of probabilities. Theyuse Linear Programming (LP) solvers to compute over-approximations and heuristics-based“ray-shooting” algorithms to compute under-approximations, which is applicable for convexpolyhedra. Filieri et al. used the LattE Machiato [16] tool to compute probabilities. Furthermorethe approach in [20] provides treatment of multi-threading and input data structures (it uses theKorat tool [10] for counting the input structures). Follow-on work provides thorough treatmentof nondeterminism [33] and describes alternative statistical exploration of symbolic paths [21].Our technique complements the works of Sankaranarayanan et al. and Filieri et al. by supportinga wider range of constraints. Experimental results show that although there is a potential loss inprecision by using our analysis, when that loss occurs it is small and can mitigated by the directhandling of non-uniform distributions and focused, iterative sampling.
Bouissou et al. [9] handles non-linear constraints with a combination of abstraction basedon affine and p-box arithmetic. The approach relies on the use of noise variables to represent theuncertainty of non-linear computations. However they cannot handle conditional branches as wecan here. More recently Adje et al. [5] extended this work to allow conditional statements aswell. The main difference with our approach lies in that they use an abstraction based approachwhereas we use a statistical approach. We can thus handle a wider set of non-linear constraints
6.2. RELATED WORK 59
such as complex mathematical functions (sine, cosine, etc.). In future work we would like to doan empirical comparison between these two approaches on the examples that both can handle.
Pavese et al. [41] propose the use Monte Carlo simulation and model inference [19] tooptimize probabilistic model checkers, such as PRISM [28]. Conceptually, the idea is to obtain asymbolic computational model from a set of traces obtained with sampling and feed that modelto the model checker. Experimental results show that their technique can obtain tighter lowerbounds for the mean time to first failure (MTTF) of network protocol models with rare failures.Our work is orthogonal to theirs. Our technique applies to imperative programs, such as thosefor controlling vehicles while their technique applies to probabilistic programs. Our techniqueexpects as input a summary of the behavior characterized with a set of complex constraints.
Another simulation-based approach for the analysis of probabilistic programs has beenproposed in [37]. In this paper, Markov Chain Monte Carlo estimation [46] is enhanced witha preliminary program analysis aiming at generating convenient verification conditions forall the operations involving probabilistic variables and operators. A violation of any of theseconditions implies the violation of the program assertions and, since these conditions may beprocessed before reaching the assertions, the simulations can terminate earlier reducing theoverall analysis time. The effectiveness of this approach relies on the ability of synthesizingprobabilistic verification conditions leading to early execution termination, which is in general anon trivial task, while the probabilistic analysis is not compositional itself.
606060
References
[1] Math - Commons-Math: The Apache Commons Mathematics Library, 2013.http://commons.apache.org/math/.
[2] Mathematica NIntegrate, 2013. http://reference.wolfram.com/mathematica/ref/NIntegrate.html.
[3] VolComp, 2013. http://systems.cs.colorado.edu/research/cyberphysical/probabilistic-program-analysis/.
[4] qCORAL, 2014. pan.cin.ufpe.br/qcoral.
[5] Assale Adje, Olivier Bouissou, Jean Goubault-Larrecq, Eric Goubault, and Sylvie Putot.Static analysis of programs with imprecise probabilistic inputs. In VSTTE, volume 8164,pages 22–47. Springer, 2014. ISBN 978-3-642-54107-0. doi:10.1007/978-3-642-54108-7_2. URLhttp://dx.doi.org/10.1007/978-3-642-54108-7_2.
[6] L. Baresi, E. Di Nitto, and C. Ghezzi. Toward open-world software: Issues and challenges.Computer, 39(10):36–43, 2006.
[7] Mateus Borges, Marcelo d’Amorim, Saswat Anand, David H. Bushnell, and Corina S.Pasareanu. Symbolic execution with interval solving and meta-heuristic search. In 2012IEEE Fifth International Conference on Software Testing, Verification and Validation,Montreal, QC, Canada, April 17-21, 2012, pages 111–120, 2012. doi:10.1109/ICST.2012.91. URL http://dx.doi.org/10.1109/ICST.2012.91.
[8] Mateus Borges, Antonio Filieri, Marcelo d’Amorim, Corina S. Pasareanu, and WillemVisser. Compositional solution space quantification for probabilistic software analysis. InPLDI, pages 123–132. ACM, 2014. doi: 10.1145/2594291.2594329.
[9] Olivier Bouissou, Eric Goubault, Jean Goubault-Larrecq, and Sylvie Putot. Ageneralization of p-boxes to affine arithmetic. Computing, 94:189–201, 2012.
[10] Chandrasekhar Boyapati, Sarfraz Khurshid, and Darko Marinov. Korat: automated testingbased on java predicates. In ISSTA, pages 123–133, 2002.
[11] Cristian Cadar, Vijay Ganesh, Peter M. Pawlowski, David L. Dill, and Dawson R. Engler.Exe: Automatically generating inputs of death. ACM Transactions on Information andSystem Security, 12(2):10:1–10:38, 2008.
[12] Cristian Cadar, Patrice Godefroid, Sarfraz Khurshid, Corina S Pasareanu, Koushik Sen,Nikolai Tillmann, and Willem Visser. Symbolic execution for software testing in practice:preliminary assessment. In Proceedings of the 33rd International Conference on SoftwareEngineering, pages 1066–1071. ACM, 2011.
[13] Lori A. Clarke. A system to generate test data and symbolically execute programs. IEEETSE, 2(3):215–222, 1976.
REFERENCES 61
[14] A.C. Cohen. Truncated and Censored Samples: Theory and Applications. Statistics: ASeries of Textbooks and Monographs. Taylor & Francis, 1991. ISBN 9781420052114.
[15] Ernest Davis. Constraint propagation with interval labels. Artificial Intelligence, 32(3):281–331, July 1987.
[16] J. A. De Loera, B. Dutra, M. Köppe, S. Moreinis, G. Pinto, and J. Wu. Software for ExactIntegration of Polynomials Over Polyhedra. ACM Communications in Computer Algebra,45(3/4):169–172, 2012.
[17] M.H. DeGroot and M.J. Schervish. Probability and Statistics. Addison-Wesley, 2012.ISBN 9780321500465. URLhttp://books.google.com.br/books?id=4TlEPgAACAAJ.
[18] Marie Duflot, Marta Kwiatkowska, Gethin Norman, David Parker, Sylvain Peyronnet,Claudine Picaronny, and Jeremy Sproston. Practical applications of probabilistic modelchecking to communication protocols. 2012.
[19] Michael D. Ernst, Jeff H. Perkins, Philip J. Guo, Stephen McCamant, Carlos Pacheco,Matthew S. Tschantz, and Chen Xiao. The Daikon System for Dynamic Detection ofLikely Invariants. SCP, 69(1-3):35–45, 2007.
[20] Antonio Filieri, Corina S. Pasareanu, and Willem Visser. Reliability analysis in symbolicpathfinder. In ICSE, pages 622–631, 2013.
[21] Antonio Filieri, Corina S. Pasareanu, Willem Visser, and Jaco Geldenhuys. Statisticalsymbolic execution with informed sampling. In FSE, pages 437–448, 2014. doi:10.1145/2635868.2635899. URLhttp://doi.acm.org/10.1145/2635868.2635899.
[22] Jaco Geldenhuys, Matthew B. Dwyer, and Willem Visser. Probabilistic symbolic execution.In ISSTA, pages 166–176, 2012.
[23] Carlo Ghezzi, Mauro Pezzè, Michele Sama, and Giordano Tamburrelli. Mining behaviormodels from user-intensive web applications. In Proceedings of the 36th InternationalConference on Software Engineering, ICSE ’14, pages 277–287. ACM, 2014. ISBN978-1-4503-2756-5. doi: 10.1145/2568225.2568234. URLhttp://doi.acm.org/10.1145/2568225.2568234.
[24] Patrice Godefroid, Nils Klarlund, and Koushik Sen. Dart: directed automated randomtesting. In PLDI, pages 213–223, 2005.
[25] Andrew D. Gordon, Thomas A. Henzinger, Aditya V. Nori, and Sriram K. Rajamani.Probabilistic programming. In ICSE FOSE, pages 167–181, 2014.
[26] Laurent Granvilliers and Frédéric Benhamou. Algorithm 852: Realpaver: an intervalsolver using constraint satisfaction techniques. ACM Transactions on MathematicalSoftware, 32:138–156, 2006.
[27] Rod Haggarty. Fundamentals of mathematical analysis. Addison-Wesley New York, 1989.
[28] Andrew Hinton, Marta Kwiatkowska, Gethin Norman, and David Parker. PRISM: A Toolfor Automatic Verification of Probabilistic Systems. In TACAS, pages 441–444. 2006.
REFERENCES 62
[29] F James. Monte carlo theory and practice. Reports on Progress in Physics, 43(9):1145,1980.
[30] James C. King. Symbolic execution and program testing. Communications of the ACM, 19(7):385—394, July 1976.
[31] Dirk P Kroese, Thomas Taimre, and Zdravko I Botev. Handbook of Monte Carlo Methods,volume 706. John Wiley & Sons, 2011.
[32] Marta Kwiatkowska. Quantitative verification: Models, techniques and tools. InESEC-FSE, pages 449–458, 2007.
[33] Kasper Luckow, Corina S. Pasareanu, Matthew B. Dwyer, Antonio Filieri, and WillemVisser. Exact and approximate probabilistic symbolic execution for nondeterministicprograms. In ASE, pages 575–586. ACM, 2014. doi: 10.1145/2642937.2643011.
[34] Michael A. Malcolm and R. Bruce Simpson. Local versus global strategies for adaptivequadrature. ACM Transactions on Mathematical Software, 1(2):129–146, June 1975.
[35] David Monniaux. An abstract monte-carlo method for the analysis of probabilisticprograms. In POPL, pages 93–101, 2001.
[36] J. Nocedal and S. Wright. Numerical Optimization. Springer Series in OperationsResearch and Financial Engineering. Springer, 2006. ISBN 9780387400655.
[37] Aditya V. Nori, Chung-Kil Hur, Sriram K. Rajamani, and Selva Samuel. R2: An efficientmcmc sampler for probabilistic programs. In AAAI Conference on Artificial Intelligence(AAAI). AAAI, July 2014. URL http://research.microsoft.com/apps/pubs/default.aspx?id=211941.
[38] J. O’Madadhain, D. Fisher, S. White, and Y. Boey. The JUNG (Java UniversalNetwork/Graph) Framework. Technical report, UCI-ICS, October 2003. URLhttp://www.datalab.uci.edu/papers/JUNG_tech_report.html.
[39] C. S. Pasareanu, J. Schumann, P. Mehlitz, M. Lowry, G. Karasai, H. Nine, and S. Neema.Model based analysis and test generation for flight software. In SMC-IT, pages 83–90,2009.
[40] Corina S. Pasareanu, Willem Visser, David H. Bushnell, Jaco Geldenhuys, Peter C.Mehlitz, and Neha Rungta. Symbolic pathfinder: integrating symbolic execution withmodel checking for java bytecode analysis. Autom. Softw. Eng., 20(3):391–425, 2013.
[41] Esteban Pavese, Víctor A. Braberman, and Sebastián Uchitel. Automated reliabilityestimation over partial systematic explorations. In ICSE, pages 602–611, 2013.
[42] W.R. Pestman. Mathematical Statistics. De Gruyter, 2009.
[43] W.H. Press. Numerical Recipes 3rd Edition: The Art of Scientific Computing. CambridgeUniversity Press, 2007. ISBN 9780521880688.
[44] William H. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery.Numerical Recipes: The Art of Scientific Computing. Cambridge University Press, 3edition, 2007.
REFERENCES 63
[45] Corina S. Pasareanu and Neha Rungta. Symbolic pathfinder: symbolic execution of javabytecode. ASE, pages 179–180. ACM, 2010.
[46] Christian P. Robert and George Casella. Monte Carlo Statistical Methods. Springer-VerlagNew York, Inc., 2005.
[47] Sriram Sankaranarayanan, Aleksandar Chakarov, and Sumit Gulwani. Static analysis forprobabilistic programs: inferring whole program properties from finitely many paths. InPLDI, pages 447–458, 2013.
[48] Matheus Souza, Mateus Borges, Marcelo d’Amorim, and Corina S. Pasareanu. CORAL:Solving Complex Constraints for Symbolic PathFinder. In NASA Formal Methods, pages359–374, 2011.
[49] Nikolai Tillmann and Jonathan de Halleux. Pex-white box test generation for .net. In TAP,pages 134–153, 2008.
[50] Wolfram Research, Inc. Advanced Numerical Integration in Mathematica, 2008.https://www.wolfram.com/learningcenter/tutorialcollection/AdvancedNumericalIntegrationInMathematica/.
[51] Paolo Zuliani, Christel Baier, and Edmund M. Clarke. Rare-event verification forstochastic hybrid systems. In HSCC, pages 217–226, 2012.