jfuzz – java based whitebox fuzzing david harvison adam kiezun
TRANSCRIPT
![Page 1: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/1.jpg)
jFuzz – Java based Whitebox Fuzzing
David HarvisonAdam Kiezun
![Page 2: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/2.jpg)
2
Summary
Problem Generating interesting test inputs for file
reading programs takes time.
Approach Create a smart fuzzer to generate inputs
that cause programs to crash.
Results jFuzz generates a many input files and
creates a base for others to expand upon.
![Page 3: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/3.jpg)
3
Problem
Bugs in a program may cause crashes for specific input files.
A compiler with buggy code, a media player with a corrupt file, etc.
Creating input files by hand takes time. Some of the files may exercise the same code.
Want a way to automatically generate input files that crash the program.
![Page 4: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/4.jpg)
4
Idea
Generate inputs that cause crashes by generating many inputs that exercise unique execution paths.
![Page 5: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/5.jpg)
5
Program Exampleif (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++;
}
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
} else {
val = car1;
}
File reading code.
![Page 6: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/6.jpg)
6
Program Example File reading code. Want to generate
files which exercise different program paths.
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++;
}
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
} else {
val = car1;
}
![Page 7: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/7.jpg)
7
Program Example File reading code. Want to generate
files which exercise different program paths.
car0 == '-'car1 == '5'
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++;
}
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
} else {
val = car1;
}
![Page 8: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/8.jpg)
8
Program Example File reading code. Want to generate
files which exercise different program paths.
car0 == '-'car1 == '7'
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++;
}
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
} else {
val = car1;
}
![Page 9: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/9.jpg)
9
Program Example File reading code. Want to generate
files which exercise different program paths.
car0 == '+'car1 == '3'
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++;
}
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
} else {
val = car1;
}
![Page 10: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/10.jpg)
10
Program Example File reading code. Want to generate
files which exercise different program paths.
car0 == '+'car1 == '9'
if (car0 == '-') {
neg = true;
} else {
neg = false;
cnt++;
}
if (car1 >= '0' && car1 <= '5') {
val = car1 - '0';
} else {
val = car1;
}
![Page 11: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/11.jpg)
11
Related Tools
Cute, EXE, SAGE, catchconv, Apollo Smart fuzzers - programs that generate
interesting new inputs for programs. Not for Java.
JCute Smart fuzzer for Java. Reinstruments code – Requires source files. Has problems with the JDK.
![Page 12: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/12.jpg)
12
Overall Idea
Input A compiled (into bytecode) Java program. A valid input file.
Output New input files which exercise unique control paths.
Run the subject program in a modified JVM.
A logic predicate, the Path Condition, is formed as the program executes.
Describes control flow of execution.
New inputs are created by manipulating the path condition.
![Page 13: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/13.jpg)
13
Example
public void top(char[] input) { int cnt = 0; if (input[0] == ‘b’) cnt++; if (input[1] == ‘a’) cnt++; if (input[2] == ‘d’) cnt++; if (input[3] == ‘!’) cnt++; if (cnt > 3) crash();}
good
![Page 14: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/14.jpg)
14
Example
public void top(char[] input) { int cnt = 0; if (input[0] == ‘b’) cnt++; if (input[1] == ‘a’) cnt++; if (input[2] == ‘d’) cnt++; if (input[3] == ‘!’) cnt++; if (cnt > 3) crash();}
good
I0 != 'b'
I1 != 'a'
I2 != 'd'I3 != '!'
path condition
Negate constraints in path condition.Solve the new path condition to create new inputs.
![Page 15: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/15.jpg)
15
Example
public void top(char[] input) { int cnt = 0; if (input[0] == ‘b’) cnt++; if (input[1] == ‘a’) cnt++; if (input[2] == ‘d’) cnt++; if (input[3] == ‘!’) cnt++; if (cnt > 3) crash();}
good
I0 != 'b'
I1 != 'a'
I2 != 'd'I3 == '!'
goo!
![Page 16: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/16.jpg)
16
Example
public void top(char[] input) { int cnt = 0; if (input[0] == ‘b’) cnt++; if (input[1] == ‘a’) cnt++; if (input[2] == ‘d’) cnt++; if (input[3] == ‘!’) cnt++; if (cnt > 3) crash();}
good
I0 != 'b'
I1 != 'a'
I2 == 'd'godd
![Page 17: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/17.jpg)
17
Example
public void top(char[] input) { int cnt = 0; if (input[0] == ‘b’) cnt++; if (input[1] == ‘a’) cnt++; if (input[2] == ‘d’) cnt++; if (input[3] == ‘!’) cnt++; if (cnt > 3) crash();}
good
godd
goo!
gaod
bood
All paths are explored systematically.
![Page 18: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/18.jpg)
18
Tool Used
NASA Java PathFinder Dynamic analysis framework for Java
implemented as a JVM. Allows backtracking including saving and restoring
the whole state of the VM. Can execute all thread interleavings. Can execute a program on all possible inputs.
![Page 19: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/19.jpg)
19
Attributes
Additional state-stored information. Associated with runtime values. JPF propagates attributes across
assignment, method calls, etc. Allows us to keep track of how the
variables relate to the input using symbolic expressions.
![Page 20: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/20.jpg)
20
Concrete v. Concolic
3 4
+
7
Normal
+
3 4
7
With Attributes
exp0 exp1
Sum(exp0, exp1)
Concolic execution is both concrete and symbolic.
Concrete
Symbolic
![Page 21: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/21.jpg)
21
Concrete v. Concolic
public class IADD extends Instruction {
public Instruction execute (... ThreadInfo th) {
[1] int v1 = th.pop();
[2] int v2 = th.pop();
[3] th.push(v1 + v2, ...);
[4] return getNext(th);
}
}
public class IADD extends ...bytecode.IADD {
public Instruction execute (... ThreadInfo th) {
[1] int v1 = th.pop();
[2] int v2 = th.pop();
[3] th.push(v1 + v2, ...);
[4] StackFrame sf = th.getTopFrame();
[5] IntExpr sym_v1 = sf.getOperandAttr();
[6] IntExpr sym_v2 = sf.getOperandAttr();
[7] if (sym_v1 == null && sym_v2 == null)
return getNext(th);
[7] IntExpr result = sym_v1._plus(sym_v2);
[8] sf.setOperandAttr(result);
[9] return getNext(th);
}
}
Concrete
Symbolic
![Page 22: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/22.jpg)
22
jFuzz Architecture
Runs JPF many times on the subject program and input files.
Each run: Collects the Path Condition
(PC). Negates each constraint,
reduces, and solves. Uses new PCs to generate new
input files.
Keeps track of inputs which caused exceptions to be thrown.
jFuzz
JPF
Subjectand Input
PC
Solver
NegatedPC
NewInput
Subjectand
OriginalInput
Inputswhichcause
crashes
![Page 23: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/23.jpg)
23
Creating New Inputs
For a given execution some parts of the input may not be read.
![Page 24: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/24.jpg)
24
Creating New Inputs
For a given execution some parts of the input may not be read.
![Page 25: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/25.jpg)
25
Creating New Inputs
For a given execution some parts of the input may not be read.
When the path condition is solved, only the read parts will have new values.
![Page 26: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/26.jpg)
26
Creating New Inputs
For a given execution some parts of the input may not be read.
When the path condition is solved, only the read parts will have new values.
The changes are written over the original input, preserving the unused parts.
![Page 27: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/27.jpg)
27
Reducing the Path Condition
Path Conditions can be very long.
Not all constraints are effected by negating the PC.
Constraints not effected can be removed from the PC.
jFuzz
JPF
Subjectand Input
PC
Solver
NegatedPC
NewInput
Subjectand
OriginalInput
Inputswhichcause
crashes
PCMinimiz
er
![Page 28: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/28.jpg)
28
Example Reduction
Path Condition:
[1] a + b < 10
[2] b > 6
[3] c < 15
[4] a < 3
[5] c + d > 7
[6] e != 1
[7] c – e = 5
[8] a == 2
![Page 29: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/29.jpg)
29
Example Reduction
Start with fuzzing the last constraint.
Path Condition:
[1] a + b < 10
[2] b > 6
[3] c < 15
[4] a < 3
[5] c + d > 7
[6] e != 1
[7] c – e = 5
[8] a != 2
![Page 30: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/30.jpg)
30
Example Reduction
Start with fuzzing the last constraint.
Select all constraints which contain variables in that constraint.
Path Condition:
[1] a + b < 10
[2] b > 6
[3] c < 15
[4] a < 3
[5] c + d > 7
[6] e != 1
[7] c – e = 5
[8] a != 2
![Page 31: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/31.jpg)
31
Example Reduction
Start with fuzzing the last constraint.
Select all constraints which contain variables in that constraint.
If one of the constraints contains multiple variables, select all constraints which contain those variables.
Path Condition:
[1] a + b < 10
[2] b > 6
[3] c < 15
[4] a < 3
[5] c + d > 7
[6] e != 1
[7] c – e = 5
[8] a != 2
![Page 32: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/32.jpg)
32
Example Reduction
Start with fuzzing the last constraint.
Select all constraints which contain variables in that constraint.
If one of the constraints contains multiple variables, select all constraints which contain those variables.
All other constraints can be removed.
Path Condition:
[1] a + b < 10
[2] b > 6
[4] a < 3
[8] a != 2
![Page 33: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/33.jpg)
33
Example Reduction
Start with fuzzing the last constraint.
Select all constraints which contain variables in that constraint.
If one of the constraints contains multiple variables, select all constraints which contain those variables.
All other constraints can be removed.
Variables not in the new PC are left unchanged.
Path Condition:
[1] a + b < 10
[2] b > 6
[3] a < 3
[4] a != 2
![Page 34: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/34.jpg)
34
Reducing the Path Condition
Reductions are performed for every constraint that is negated.
jFuzz uses a UnionFind data structure to find which variables are connected to each other.
In our case study, the average reduction was from around 250 constraints to about 5 constraints.
![Page 35: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/35.jpg)
35
Case Study
Subject: Sat4J SAT solver written in Java. Takes inputs in dimacs files. ~10 kloc.
Goals: Create inputs that crash
Sat4J. Create a set of good inputs.
test1.dimacs
c test 3 single clauses and 2
c binary clauses
p cnf 4 5
1 0
2 0
3 0
-2 4 0
-3 4 0
![Page 36: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/36.jpg)
36
Results
After 30 minutes of execution: 12,000 input files were created. 70 crashes where found.
The crashes are actually normal for SAT4J 38 Invalid DIMACS files. 27 Contradictions. 4 Assertion Errors.
A Java compiler would be more compelling. Any crash is due to a bug in the compiler. Much larger program.
![Page 37: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/37.jpg)
37
Performance
Sat4J was run 100 times in each VM. The times are average runtime. Simplifying the Path Condition reduces
the solving time by 30%.
Time (s) Ratio PC Solving (s)Sun JVM 0.21 1 - 0.15JPF 2.5 11 - 1.8
5.8 28 2.5 1.85.3 25 1.8 1.8
Init (s)
jFuzz--jFuzz
![Page 38: JFuzz – Java based Whitebox Fuzzing David Harvison Adam Kiezun](https://reader036.vdocuments.us/reader036/viewer/2022081519/56649f465503460f94c68874/html5/thumbnails/38.jpg)
38
Conclusions
jFuzz is the first concolic tester for Java which will work for any bytecode.
This opens the door for more advanced fuzzing techniques, such as grammar based fuzzing.