recovering configuration information for configurable systems via symbolic execution
DESCRIPTION
Recovering Configuration Information for Configurable Systems via Symbolic Execution. Justin Gorham, Xusheng Xiao, Tao Xie North Carolina State University jwgorham,xxiao,[email protected]. Modern Software Systems. Designed to be highly customizable Extensibility Flexibility Portability. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/1.jpg)
Recovering Configuration Information for Configurable Systems via Symbolic
Execution
Justin Gorham, Xusheng Xiao, Tao XieNorth Carolina State Universityjwgorham,xxiao,[email protected]
![Page 2: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/2.jpg)
Modern Software Systems
• Designed to be highly customizable– Extensibility– Flexibility– Portability
![Page 3: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/3.jpg)
Configurable Systems
• A software system with mechanisms in place for implementing preplanned variations in system behavior
• Characterized by configuration options• Each configuration option has a configuration
domain
![Page 4: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/4.jpg)
Formal Definition
• Configuration option – A label that represents a dimension of a system that can be set
• Configuration domain – The set of possible values that can be selected for a configuration option
• ‘configuration’: - a mapping of configuration options to a subset of their respective domains to create an instantiation of a configurable system
![Page 5: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/5.jpg)
Example: Microsoft Word
option
![Page 6: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/6.jpg)
Example: Microsoft Word
option domain
![Page 7: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/7.jpg)
Configuration Binding Time
• Early Feature Binding– Software Product Lines• Boeing: Bold Stroke Avionics Software Family• Nokia: Mobile Phones• Toshiba: Software Product Line for Electric Power
Generation Plant Monitoring and Control
• Dynamically Reconfigurable Systems– NASA’s Deep Space 1 Remote Agent software
• User Configurable Systems
![Page 8: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/8.jpg)
Runtime User Configurable System
• Configuration options are assigned at runtime• Obtain values typically through configuration
files and command lines• Options typically bind at startup time or at an
early stage of runtime
![Page 9: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/9.jpg)
Example: (win)grep
Command Line Options•/h|/H - printing a usage help explanatory text;•/c - print a count of matching lines for each input file;•/i - ignore case in pattern;•/l - print just files (scanning will stop on first match);•/n - prefix each line of output with line number;•/r - recursive search in subdirectories;•/E:reg_exp - the Regular Expression used as search pattern. •/F:files - the list of input files. The files can be separated by commas as in
![Page 10: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/10.jpg)
Example: JPF
• Uses a dictionary object to parse a hierarchical set of property files– Site Properties– Project Properties– Application Properties
• Primarily uses a <key> = <value> style of configuration
![Page 11: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/11.jpg)
JPF: Site Properties
• site.properties file is machine specific and not part of any JPF project
• Each project is listed as a <name>=<directory> pair
![Page 12: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/12.jpg)
JPF: Site Properties
• site.properties file is machine specific and not part of any JPF project
• Each project is listed as a <name>=<directory> pair
jpf-core = ${user.home}/projects/jpf/jpf-corejpf-shell = ${user.home}/projects/jpf/jpf-shell jpf-awt = ${user.home}/projects/jpf/jpf-awt ... extensions=${jpf-core},${jpf-shell}
![Page 13: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/13.jpg)
JPF: Project Properties
• Each JPF project contains a jpf.properties file in its root directory
• defines the paths that need to be set for the component to work properly
![Page 14: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/14.jpg)
JPF: Project Properties
1. <project-name>.native_classpath2. <project-name>.classpath3. <project-name>.test_classpath 4. <project-name>.sourcepath
![Page 15: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/15.jpg)
JPF: Project Properties
1. <project-name>.native_classpath2. <project-name>.classpath3. <project-name>.test_classpath 4. <project-name>.sourcepath
jpf-aprop = ${config_path}
#--- path specifications jpf-aprop.native_classpath = build/jpf-aprop.jar;lib/antlr-runtime-3.1.3.jarjpf-aprop.classpath = build/examples jpf-aprop.test_classpath = build/tests jpf-aprop.sourcepath = src/examples
#--- other project specific settingslistener.autoload=${listener.autoload},javax.annotation.Nonnull,...listener.javax.annotation.Nonnull=gov.nasa.jpf.aprop.listener.NonnullChecker ...
![Page 16: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/16.jpg)
JPF: Project Properties
1. <project-name>.native_classpath2. <project-name>.classpath3. <project-name>.test_classpath 4. <project-name>.sourcepath
jpf-aprop = ${config_path}
#--- path specifications jpf-aprop.native_classpath = build/jpf-aprop.jar;lib/antlr-runtime-3.1.3.jarjpf-aprop.classpath = build/examples jpf-aprop.test_classpath = build/tests jpf-aprop.sourcepath = src/examples
#--- other project specific settingslistener.autoload=${listener.autoload},javax.annotation.Nonnull,...listener.javax.annotation.Nonnull=gov.nasa.jpf.aprop.listener.NonnullChecker ...
![Page 17: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/17.jpg)
JPF: Application Properties
• Used to tell JPF what main class it should start to execute
• Stored in *.jpf properties files that are part of the test projects
• target setting defines the main class of SUT• Also define target_args along with other JPF
properties that define how the application is to be checked
![Page 18: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/18.jpg)
JPF: Application Properties
• Used to tell JPF what main class it should start to execute
• Stored in *.jpf properties files that are part of the test projects
• target setting defines the main class of SUT• Also define target_args along with other JPF
properties that define how the application is to be checked
#--- dependencies on other JPF projects@using = jpf-awt@using = jpf-shell
#--- what JPF should runtarget = RobotManager
#--- other stuff that defines how to run JPFlistener+=,.listener.OverlappingMethodAnalyzer
shell=.shell.basicshell.BasicShellawt.script=${config_path}/RobotManager-thread.escg.enumerate_random=true...
![Page 19: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/19.jpg)
Scenario
• Developer/Testers– QA to validate configurations– Software Configuration Space Explosion
• End Users
![Page 20: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/20.jpg)
Software Configuration Space Explosion
• Example– 20 configuration options– The domain for each option is either ‘true’ or
‘false’– 220 = 1,048,576 possible configurations– Add another option, configuration space doubles
• Infeasible to test a large configuration space
![Page 21: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/21.jpg)
Motivation
• Extract configuration information for a runtime user configurable system– How to extract available configuration options– How to extract constraints among options and
their values• domain of a configuration option• relationships among options and/or their values
– How to extract runtime behavior of configurations• Fault due to interaction
![Page 22: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/22.jpg)
Extracting Configuration Information
• Manually search through source code– Expensive, error prone…
• Static analysis– Cannot capture runtime behavior of configurtions
• Combinatorial Testing– Black Box, Scalability Issues
• Symbolic Execution
![Page 23: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/23.jpg)
Thanks to Dr. Tao Xie for reusing his slide
Symbolic Execution
Code to generate inputs for:
Constraints to solve
a!=null a!=null &&a.Length>0 a!=null &&a.Length>0 &&a[0]==1234567890
void CoverMe(int[] a){ if (a == null) return; if (a.Length > 0) if (a[0] == 1234567890) throw new Exception("bug");}
Observed constraints
a==nulla!=null &&!(a.Length>0)a!=null &&a.Length>0 &&a[0]!=1234567890
a!=null &&a.Length>0 &&a[0]==1234567890
Data
null
{}
{0}
{123…}a==null
a.Length>0
a[0]==123…T
TF
T
F
F
Execute&MonitorSolve
Choose next path
Done: There is no path left.
Negated condition
![Page 24: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/24.jpg)
SE with Configurable Systems
• Using Symbolic Evaluation to Understand Behavior in Configurable Software Systems
• Reisner, E; Song, C; Ma, K; Foster, J; Porter A
Figures on this page from “Using Symbolic Evaluation to Understand Behavior in
Configurable Software Systems“
![Page 25: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/25.jpg)
SE with Configurable Systems
Figures on this page from “Using Symbolic Evaluation to Understand Behavior in
Configurable Software Systems“
![Page 26: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/26.jpg)
SE with Configurable Systems
Figures on this page from “Using Symbolic Evaluation to Understand Behavior in
Configurable Software Systems“
![Page 27: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/27.jpg)
SE with Configurable Systems
• Required to know configuration options in advance
• Program inputs may interact with configuration options
![Page 28: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/28.jpg)
Parameterized Unit Test
![Page 29: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/29.jpg)
Parameterized Configuration Mechanism
![Page 30: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/30.jpg)
Proposed Solution
Unit TestsCode Under
Test
Configuration Mechanism
Inputs
Option Value
Output: Test Suite with concrete
inputs
![Page 31: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/31.jpg)
Proposed Solution
PUTsCode Under
Test
ParameterizedConfiguration Mechanism
Output: Test Suite with concrete inputs and
assumptions on configuration options
![Page 32: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/32.jpg)
Our Approach
![Page 33: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/33.jpg)
Implementation
![Page 34: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/34.jpg)
Challenges
• Object Creation and Data Transformation• Static Configuration Options
![Page 35: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/35.jpg)
Future Directions
![Page 36: Recovering Configuration Information for Configurable Systems via Symbolic Execution](https://reader035.vdocuments.us/reader035/viewer/2022070405/56813dab550346895da77073/html5/thumbnails/36.jpg)
END
• Questions