extent-2016: industry practices of advanced program analysis
TRANSCRIPT
Institute for System Programming of the Russian Academy of Sciences
Industry practices of advanced program analysis
Alexey [email protected]
ExTENT-2016London, 22 June 2016
The Goal of Verification
to make sure there is no any bugs in any possible execution of the target software
● with minimal cost
Verification
Fresh Software Verified Software
The Goal of Verification
to achieve maximal assurance that there is no any bugs in any possible execution of the target software
● within reasonable cost
Verification
Fresh Software Verified Software
Testing
Test
Test Suite
1 kindof bugs
all kindsof bugs
in all executionsin 1 execution
Inductive Reasoning● Conclusions are supported by its premises● If premises are true, it would be unlikely
impossible for the conclusions to be false● Example:
● Every time I’ve walked by that dog,he hasn’t tried to bite me.
● So, the next time I walk by that dog he won’t try to bite me.
Inductive Reasoning● Conclusions are supported by its premises● If premises are true, it would be unlikely
impossible for the conclusions to be false● Example:
● The component behaves correctly on all test data.
● So, the component always behaves correctly.
Inductive Reasoning on Program Correctness
● The component behaves correctly on all test data
● Tests are quite representative
=>● The component always behaves correctly
Deductive Reasoning● Conclusions certainly follow
from its premises● If premises are true, it is impossible for the
conclusions to be false● Example:
● All men are mortal.● Socrates is a man.● Therefore, Socrates is mortal.
Deductive Reasoning on Program Correctness
TargetComponent
Requirements
Review
Deductive Reasoning on Program Correctness
TargetComponent
Requirements
Deductive Reasoning on Program Correctness● If assumptions are held
● regarding compiler&linker● regarding environment behaviour● regarding input data
● The component always behaves correctly
Deductive Reasoning on Program Correctness● If assumptions are held
● regarding compiler&linker● regarding environment behaviour● regarding input data
● The component always behaves correctly
(i.e. terminates and its output satisfies to postcondition)
Deductive VerificationHistorical Perspective
1947
1970
● Lecture of Alan Turing to London Mathematical Society
● Methods of Floyd/Hoare
START:( y
1, y
2 ) ( 0, x
1 )
y2 x
2
FT
( y1, y
2 ) ( y
1+1, y
2- x
2 ) HALT:
( z1, z
2 ) ( y
1, y
2 )
A: Precondition(x
1 0) (x
2 0)
С: Postcondition (x
1 = z
1x
2 + z
2) (0 z
2 < x
2)
B: Invariant (x
1 = y
1x
2 + y
2) ∧ (y
2 0)
Deductive Verification
Prove program correctness by induction:
Let precondition to be held in A we pass A->B => Invariant is held in BLet invariant to be held in B we pass B-T->B => Invariant is held in BLet invariant to be held in B we pass B-F->C => Postcondition is held in С
Let precondition to be held in A we pass A->C => Postcondition is held in С
=>
Deductive VerificationHistorical Perspective
1947
1970
2000
● Lecture of Alan Turing to London Mathematical Society
● Methods of Floyd/Hoare
● Deductive verifcation tools for Ada, C, Java, С#● SunRise, ESC/Java, Frama-C, LOOP, Boogie/VCC
Deductive VerificationHistorical Perspective
1947
1970
2000
2010
● Lecture of Alan Turing to London Mathematical Society
● Methods of Floyd/Hoare
● Deductive verifcation tools for Ada, C, Java, С#● SunRise, ESC/Java, Frama-C, LOOP, Boogie/VCC
● Application in real-life projects for small-size components● Nuclear power (UK, France)● Avionics (Airbus, NASA, UK Air Traffic Control)● Components of operating systems
(seL4, Verisoft, Verisoft-XT)
Industry Applications● UK Air Traffic Management System
● 250 KLOC of logical lines of code (in Ada)● proof type safety, few functional correctness
code● 153K VCs, of which 98.76% are proven
automatically
(*) Angela Wallenburg “Safe and Secure Programming Using Spark”
Years Tools Target code Scope Size
Verisoft 2004-2008 Isabelledesigned for verification
hw/kernel/ compiler/
libraries/apps10 kLOC(kernel)
L4.verifiedseL4
2004-2009 Isabelle
designed for verification,
performance oriented
microkernelsecurity model
(no MMU)
7.5 kLOC(without asm and
boot)
Verisoft-XT small-hv
2007-2013 VCCdesigned for verification
separation property only
2.5 kLOC
Verisoft-XT Hyper-V
2007-2013 VCC industrialseparation
property only100 kLOC
Verisoft-XT PikeOS
2007-2013 VCCindustrial,
simplicity for performance
some system calls
10 KLOC
OS Deductive Verification
Linux Verification Center
founded in 2005● OLVER Program● Linux Standard Base Infrastructure Program● Linux Driver Verification Program● Linux File System Verification Program● Linux Deductive Verification Program
Toolset for Event-B models verification
Model of securityrequirements
Formalized securitymodel
Formalizedlow-level security
model
Manual
Automatedverification
Securityrequirements
Security model
AstraVer Toolchain (*)for deductive verification of C programs(based on Frama-C – Jessie – Why3)
Pre-/post-conditionsof LSM operations
Securityarcitecture
LSMsource code
Model of securityrequirements
Mathematicalnotation
LSMImplementation
implements
Specificatiion oflibrary functions
Linuxkernel
AstraVer Project
AstraLinux(*) The research on deductive verification tools development was carried out with funding from the Ministry of Education and Science of Russia (the project unique identifier is RFMEFI60414X0051
Deductive Verification Status
● Reasonable Tool Support● Ada, C, C#, Java● Functional specification as comments,
even natively supported in Ada-2012● Dedicated languages: Boogie, Why3
● Manual efforts still significant● up to 10x of development efforts● highly skilled team required
Testing and Deductive Verification
1 kindbugs
all kindsof bugs
in all executions
Deductive
1. Proof of complete correctnessunder some assumptions 2. Very labour intensive and time consuming
in 1 execution
Test
Test Suite
TestingDeductive
Verification
Kind of bugs almost all almost all
Executionsunder analysis small almost all
Development cost
linear huge
Executioncost
hw small(target hw)
hw small
Result analysiscost small big
Maintenancecost
small to big huge
Testing and Deductive Verification
Deductive Verification Status (2)
● reasonable to apply only for really important code
Static Analysis
● is a commodity● mature market of static analysis tools
SVACE by ISPRAS
● Static analysis of C/C++/Java code, Linux/Windows
● 150+ kinds of defects● Buffer overflows, NULL-pointer dereferences● Memory management, tainted input● Concurrency issues● Lightweight analysis of semantic patterns
● Eclipse plugin or WebUI
TestingStatic
AnalysisDeductive
Verification
Kind of bugs almost allsafetyonly almost all
Executionsunder analysis small almost all almost all
Development cost
linear 0 to small huge
Executioncost
hw small(target hw)
hw small hw small
Result analysiscost
smallmedium
(false alarms)big
Maintenancecost
small to big small huge
Testing and Program Analysis
TestingStatic
AnalysisDeductive
Verification
Kind of bugs almost allsafetyonly almost all
Executionsunder analysis small
bigalmost all almost all
Development cost
linear 0 to small huge
Executioncost
hw small(target hw)
hw small hw small
Result analysiscost
smallmedium
(false alarms)big
Maintenancecost
small to big small huge
Testing and Program Analysis
Testing and Program Analysis
1 kindbugs
all kindsof bugs
in all executions
Deductive
in 1 execution
Test
Test Suite
SVACE
1. Static analysis2. Quickly finds potential bugs3. No any guarantee
R...
Model Checking
R2
R1
R0
[Clarke/Emerson, Sifakis 1981]
● Iterative fixpoint post computation
Software Model Checking
entry point
error location
● Reachability problem
Error Location?int f(int y){struct urb *x;
x = usb_alloc_urb(0,GFP_KERNEL); ... usb_free_urb(x);
return y;}
Error Location?int f(int y){struct urb *x;
x = usb_alloc_urb(0,GFP_KERNEL); // allocate new URB ... usb_free_urb(x); // deallocate URB: assert(x is NULL or previously allocated URB)
return y;}
… // after module exit: assert( all allocated URBs are deallocated)
Instrumentation
int f(int y){struct urb *x;
x = usb_alloc_urb(0,GFP_KERNEL);
...
usb_free_urb(x);
return y;}
set URBS = empty;
int f(int y){struct urb *x;
x = usb_alloc_urb(); add(URBS, urb); ... assert(contains(URBS, x)); usb_free_urb(x); remove(URBS, urb);
return y;} … // after module exit assert(is_empty(URBS));
Software Model Checking
entry point
error location
● Reachability problem
Bounded Model Checking
● finite unfolding of transition relation
b1
a2 a2
b2 b2 b2 b2
b1
a2 a2
b2 b2 b2 b2
r
Counter-Example Guided Abstraction Refinement● Detailed model of a program is huge● Detailed model of a program is not needed to
check a particular property● Detailed model of a concrete path in a program
is suitable for analysis
=>● Build a model that is just enough to check the
particular property
SV-COMP-2013 Competition Results
DriversDrivers
Bit Bit operationsoperations
PointersPointers
CEGARCEGAR BMCBMC
Sh
ape-
Sh
ape-
an
alys
isa
nal
ysis
SVCOMP'14 Results
SVCOMP'15 Results
BLASTCPAchecker
15 tools in 201422 tools in 201535 tools in 2016
Industry Applications
Linux Verification Center
founded in 2005● OLVER Program● Linux Standard Base Infrastructure Program● Linux Driver Verification Program● Linux File System Verification Program● Linux Deductive Verification Program
Linux Driver Verification
http://linuxtesting.org/ldv
Bugs Foundhttp://linuxtesting.org/results/ldv
>230 patches already applied
SVACE vs. LDVSVACE LDV-CPAchecker
Time of analysis 2 hrs 111 hrs (4.5 days)
Warnings 35 328
True bugs 8 103
True positive rate 23% 31%
● Target code:● Linux kernel 3.17-rc1, allmodconfig, x86-64● 3 223 modules, 33 373 source files
● Target bugs:● double free, memory leaks
There is no a single common bugs!
Testing and Program Analysis
1 kindbugs
all kindsof bugs
in all executions
Deductive
in 1 execution
Test
Test Suite
SVACE LDV
1. Investigates all possible paths2. Is able to prove absence of bugs of particular kind3. Domain-specific (framework) adaptation (environment model, library model)4. Applicability limited to medium-sized components (up to 50 KLoC)5. Requires nonzero hardware resources - time: 15 minutes per rule per module - memory: 15 Gb
TestingStatic
AnalysisSoftware Model
CheckingDeductive
Verification
Kind of bugs almost allsafetyonly
safetyonly almost all
Executionsunder analysis small big almost all almost all
Development cost
linear 0 to small medium huge
Executioncost
hw small(target hw)
hw small hw big hw small
Result analysiscost small
medium(false alarms)
medium(false alarms) big
Maintenancecost
small to big small small huge
Testing and Program Analysis
Conclusions● There is no silver bullet● The key is in a competent combination of
available techniques
Conclusions (2)● Two advanced program analysis techniques:
● Deductive verification● + proof of complete correctness under some
assumptions ● – significant manual efforts● – highly skilled team required● => only for really important code
● Software model checking● + investigates almost all possible paths● + complimentary to static analysis● – per framework/domian adaptation required● – limited to medium-sized components (up to 50 KLoC)
Institute for System Programming of the Russian Academy of Sciences
Thank you!
Alexey [email protected]://linuxtesting.org/
Morris Kline. “Mathematics: The Loss of Certainty” Oxford Press, 1980