dynamic analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · dynamic binary...
TRANSCRIPT
![Page 2: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/2.jpg)
2
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
![Page 3: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/3.jpg)
3
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
– Did my program ever …?
![Page 4: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/4.jpg)
4
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
– Did my program ever …?– Why/how did … happen?
![Page 5: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/5.jpg)
5
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
– Did my program ever …?– Why/how did … happen?– Where am I spending time?
![Page 6: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/6.jpg)
6
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
– Did my program ever …?– Why/how did … happen?– Where am I spending time?– Where might I parallelize?
![Page 7: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/7.jpg)
7
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
– Did my program ever …?– Why/how did … happen?– Where am I spending time?– Where might I parallelize?– Tolerate errors
![Page 8: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/8.jpg)
8
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
– Did my program ever …?– Why/how did … happen?– Where am I spending time?– Where might I parallelize?– Tolerate errors– Manage memory / resources.
![Page 9: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/9.jpg)
9
e.g. Reverse Engineering
Static CFG (from e.g. Apple Fairplay):
This is the result of a control flow flattening obfuscaton.[http://tigress.cs.arizona.edu/transformPage/docs/flatten/]
![Page 10: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/10.jpg)
10
e.g. Reverse Engineering
Static CFG (from e.g. Apple Fairplay):
Dynamically Simplified CFG:
This is the result of a control flow flattening obfuscaton.[http://tigress.cs.arizona.edu/transformPage/docs/flatten/]
![Page 11: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/11.jpg)
11
How?
● Can record the execution
![Page 12: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/12.jpg)
12
How?
● Can record the execution– Record to a trace– Analyze post mortem / offline– Scalability issues: need enough space to store it
![Page 13: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/13.jpg)
13
How?
● Can record the execution– Record to a trace– Analyze post mortem / offline– Scalability issues: need enough space to store it
● Can perform analysis online
![Page 14: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/14.jpg)
14
How?
● Can record the execution– Record to a trace– Analyze post mortem / offline– Scalability issues: need enough space to store it
● Can perform analysis online– Instrument the program– Modified program invokes code to 'analyze' itself
![Page 15: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/15.jpg)
15
How?
● Can record the execution– Record to a trace– Analyze post mortem / offline– Scalability issues: need enough space to store it
● Can perform analysis online– Instrument the program– Modified program invokes code to 'analyze' itself
● Can do both!– Lightweight recording– Instrument a replayed instance of the execution
![Page 16: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/16.jpg)
16
How?
● Can record the execution– Record to a trace– Analyze post mortem / offline– Scalability issues: need enough space to store it
● Can perform analysis online– Instrument the program– Modified program invokes code to 'analyze' itself
● Can do both!– Lightweight recording– Instrument a replayed instance of the execution
Some analyses only make sense online.Why?
![Page 17: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/17.jpg)
17
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?
![Page 18: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/18.jpg)
18
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?
Profiling is a common dynamic analysis!
![Page 19: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/19.jpg)
19
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?● How can we modify our program to find this?
![Page 20: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/20.jpg)
20
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?● How can we modify our program to find this?
BB:0
BB:1 BB:2
BB:3
?
![Page 21: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/21.jpg)
21
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?● How can we modify our program to find this?
BB:0
BB:1 BB:2
BB:3
count[2] += 1
x = foo()y = bar()...
![Page 22: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/22.jpg)
22
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?● How can we modify our program to find this?
BB:0
BB:1 BB:2
BB:3
count[2] += 1
x = foo()y = bar()...
for i in BBs: count[i] = 0
for i in BBs: print(count[i])Start: End:
![Page 23: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/23.jpg)
23
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?● How can we modify our program to find this?
BB:0
BB:1 BB:2
BB:3
count[2] += 1
x = foo()y = bar()...
for i in BBs: count[i] = 0
for i in BBs: print(count[i])Start: End:
![Page 24: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/24.jpg)
24
Simple Idea: Basic Block Profiling
● Big concern: How efficient is it?– The more overhead added, the less practical the tool
![Page 25: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/25.jpg)
25
Simple Idea: Basic Block Profiling
● Big concern: How efficient is it?– The more overhead added, the less practical the tool
count[0] += 1…
count[1] += 1…
count[5] += 1…
count[6] += 1…
count[4] += 1…
count[2] += 1…
count[3] += 1…
![Page 26: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/26.jpg)
26
Simple Idea: Basic Block Profiling
● Big concern: How efficient is it?– The more overhead added, the less practical the tool
– Can we do better?
count[0] += 1…
count[1] += 1…
count[5] += 1…
count[6] += 1…
count[4] += 1…
count[2] += 1…
count[3] += 1…
![Page 27: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/27.jpg)
27
Simple Idea: Basic Block Profiling
● Big concern: How efficient is it?– The more overhead added, the less practical the tool
– Can we do better?
count[0] += 1…
count[1] += 1…
count[5] += 1…
count[6] += 1…
count[4] += 1…
count[2] += 1…
count[3] += 1…
count[1] = count[4] = count[2] + count[3]
![Page 28: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/28.jpg)
28
Simple Idea: Basic Block Profiling
● Big concern: How efficient is it?– The more overhead added, the less practical the tool
– Can we do better?
count[0] += 1…
count[1] += 1…
count[5] += 1…
count[6] += 1…
count[4] += 1…
count[2] += 1…
count[3] += 1…
count[1] = count[4] = count[2] + count[3]count[0] = count[6] = count[1] + count[5]
![Page 29: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/29.jpg)
29
Simple Idea: Basic Block Profiling
● Big concern: How efficient is it?– The more overhead added, the less practical the tool
– Can we do better?
count[0] += 1…
count[1] += 1…
count[5] += 1…
count[6] += 1…
count[4] += 1…
count[2] += 1…
count[3] += 1…
count[1] = count[4] = count[2] + count[3]count[0] = count[6] = count[1] + count[5]
![Page 30: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/30.jpg)
30
Efficiency Tactics
● Abstraction
![Page 31: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/31.jpg)
31
Efficiency Tactics
● Abstraction
● Identify & avoid redundant information
![Page 32: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/32.jpg)
32
Efficiency Tactics
● Abstraction
● Identify & avoid redundant information
● Sampling
![Page 33: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/33.jpg)
33
Efficiency Tactics
● Abstraction
● Identify & avoid redundant information
● Sampling
● Compression / encoding
![Page 34: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/34.jpg)
34
Efficiency Tactics
● Abstraction
● Identify & avoid redundant information
● Sampling
● Compression / encoding
● Profile guided instrumentation
![Page 35: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/35.jpg)
35
Efficiency Tactics
● Abstraction
● Identify & avoid redundant information
● Sampling
● Compression / encoding
● Profile guided instrumentation
● Thread local analysis
![Page 36: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/36.jpg)
36
Efficiency Tactics
● Abstraction
● Identify & avoid redundant information
● Sampling
● Compression / encoding
● Profile guided instrumentation
● Thread local analysis
● Inference
![Page 37: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/37.jpg)
37
How / When to Instrument
● Source / IR Instrumentation– LLVM, CIL, Soot, Wala– During (re)compilation– Requires an analysis dedicated build
![Page 38: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/38.jpg)
38
How / When to Instrument
● Source / IR Instrumentation– LLVM, CIL, Soot, Wala– During (re)compilation– Requires an analysis dedicated build
● Static Binary Rewriting– Diablo, DynamoRIO, SecondWrite, – Applies to arbitrary binaries– Imprecise IR info, but more complete binary behavior
![Page 39: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/39.jpg)
39
How / When to Instrument
● Source / IR Instrumentation– LLVM, CIL, Soot, Wala– During (re)compilation– Requires an analysis dedicated build
● Static Binary Rewriting– Diablo, DynamoRIO, SecondWrite, – Applies to arbitrary binaries– Imprecise IR info, but more complete binary behavior
● Dynamic Binary Instrumentation– Valgrind, Pin, Qemu (& other Vms)– Can adapt at runtime, but less info than IR
![Page 40: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/40.jpg)
40
Phases of Dynamic Analysis
In general, 2-3 phases occur:
1) Instrumentation– Add code to the program for data collection/analysis
![Page 41: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/41.jpg)
41
Phases of Dynamic Analysis
In general, 2-3 phases occur:
1) Instrumentation– Add code to the program for data collection/analysis
2) Execution– Run the program an analyze its actual behavior
![Page 42: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/42.jpg)
42
Phases of Dynamic Analysis
In general, 2-3 phases occur:
1) Instrumentation– Add code to the program for data collection/analysis
2) Execution– Run the program an analyze its actual behavior
3) (Optional) Postmortem Analysis– Perform any analysis that can be deferred after termination
![Page 43: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/43.jpg)
43
Phases of Dynamic Analysis
In general, 2-3 phases occur:
1) Instrumentation– Add code to the program for data collection/analysis
2) Execution– Run the program an analyze its actual behavior
3) (Optional) Postmortem Analysis– Perform any analysis that can be deferred after termination
Very, very common mistake to mix 1 & 2.
![Page 44: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/44.jpg)
44
Static Instrumentation
1) Compile whole program to IR
foo.cbar.cbaz.c prog.ll
![Page 45: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/45.jpg)
45
Static Instrumentation
1) Compile whole program to IR
2) Instrument / add code directly to the IR
foo.cbar.cbaz.c prog.ll prog’.ll
![Page 46: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/46.jpg)
46
Static Instrumentation
1) Compile whole program to IR
2) Instrument / add code directly to the IR
3) Generate new program that performs analysis
foo.cbar.cbaz.c prog.ll prog’.ll
![Page 47: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/47.jpg)
47
Static Instrumentation
1) Compile whole program to IR
2) Instrument / add code directly to the IR
3) Generate new program that performs analysis
4) Execute
foo.cbar.cbaz.c prog.ll prog’.ll
Test Cases
Results
![Page 48: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/48.jpg)
48
Dynamic Binary Instrumentation (DBI)
1) Compile program as usual
2) Run program under analysis framework
(Valgrind, PIN, Qemu, etc)
3) Instrument & execute in same command:– Fetch & instrument each basic block individually– Execute each basic block
valgrind --tool=memcheck ./myBuggyProgram
![Page 49: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/49.jpg)
Example: Test Case Reduction
![Page 50: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/50.jpg)
50
Testing and Dynamic Analysis
● In some cases, just running a program with different inputs is enough
![Page 51: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/51.jpg)
51
Testing and Dynamic Analysis
● In some cases, just running a program with different inputs is enough– Carefully selected inputs can target the analysis– The result of running the program reveals coarse information about its
behavior
![Page 52: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/52.jpg)
52
Testing and Dynamic Analysis
● In some cases, just running a program with different inputs is enough– Carefully selected inputs can target the analysis– The result of running the program reveals coarse information about its
behavior
● Intuitively, even just testing is a dynamic analysis– It requires no transformation– The result is just the success or failure of tests
![Page 53: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/53.jpg)
53
Testing and Dynamic Analysis
● In some cases, just running a program with different inputs is enough– Carefully selected inputs can target the analysis– The result of running the program reveals coarse information about its
behavior
● Intuitively, even just testing is a dynamic analysis– It requires no transformation– The result is just the success or failure of tests
● But even that is interesting to consider....
![Page 54: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/54.jpg)
54
Bug reports are problematic
● Failing inputs can be large and complex
a r h w l n y e u m g k o w h > ` p
MB? GB?
![Page 55: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/55.jpg)
55
Bug reports are problematic
● Failing inputs can be large and complex
a r h w l n y e u m g k o w h > ` p
MB? GB? What is relevant and essential to the bug?
![Page 56: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/56.jpg)
56
Bug reports are problematic
a r h w l n y e u m g k o w h > ` p
● Failing inputs can be large and complex
a b cBug 2
a b cBug 3
a b cBug 1
![Page 57: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/57.jpg)
57
Bug reports are problematic
a r h w l n y e u m g k o w h > ` p
● Failing inputs can be large and complex
a b cBug 2
a b cBug 3
a b cBug 1
1) Are these reports the same bug?2) Can we make it easier to reproduce?
![Page 58: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/58.jpg)
58
Bug reports are problematic
a r h w l n y e u m g k o w h > ` p
● Failing inputs can be large and complex
a b cBug 2
a b cBug 3
a b cBug 1
a b cBug
![Page 59: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/59.jpg)
59
Bug reports are problematic
a r h w l n y e u m g k o w h > ` p
● Failing inputs can be large and complex
a b cBug 2
a b cBug 3
a b cBug 1
a b cBug
1) Same? Yes!2) Easier? Yes! And easier to understand!
![Page 60: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/60.jpg)
60
Bug reports are problematic
a r h w l n y e u m g k o w h > ` p
● Failing inputs can be large and complex
a b cBug 2
a b cBug 3
a b cBug 1
a b cBug
Test Case Reduction: finding smaller test cases that reproduce a failure
![Page 61: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/61.jpg)
61
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>
http://en.wikipedia.org/wiki/File:Netscape_2_logo.gif
![Page 62: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/62.jpg)
62
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>
http://en.wikipedia.org/wiki/File:Netscape_2_logo.gif
![Page 63: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/63.jpg)
63
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error
![Page 64: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/64.jpg)
64
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7> = cIntuition: trial and error1) Start w/ a failing text configuration c
![Page 65: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/65.jpg)
65
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ}})
![Page 66: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/66.jpg)
66
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ}})3) Failure still exists → new input is “better”
![Page 67: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/67.jpg)
67
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ}})3) Failure still exists → new input is “better”4) Repeat on the new input
![Page 68: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/68.jpg)
68
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ}})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'
![Page 69: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/69.jpg)
69
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ}})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'
Smallest subset of the originalinput reproducing the failure
![Page 70: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/70.jpg)
70
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ}})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'
Smallest subset of the originalinput reproducing the failure
Completely impractical! Why?
![Page 71: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/71.jpg)
71
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ}})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'● Local Minimum: c : ∀ c'⊂c, c'
![Page 72: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/72.jpg)
72
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ}})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'● Local Minimum: c : ∀ c'⊂c, c'
No subset of the result canreproduce the failure.
![Page 73: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/73.jpg)
73
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ}})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'● Local Minimum: c : ∀ c'⊂c, c'
No subset of the result canreproduce the failure.
How does this differ from a global minimum?Is it still problematic?
![Page 74: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/74.jpg)
74
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ}})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'● Local Minimum: c : ∀ c'⊂c, c'● 1-Minimal: c: ∀ δ ∈ c, (c-{δ})
![Page 75: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/75.jpg)
75
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ}})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'● Local Minimum: c : ∀ c'⊂c, c'● 1-Minimal: c: ∀ δ ∈ c, (c-{δ})
No one element can be removedand still reproduce the failure
![Page 76: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/76.jpg)
76
Classically – Delta Debugging1 2 3 4 5 6 7 8
Does binary search work?
![Page 77: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/77.jpg)
77
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 78: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/78.jpg)
78
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
So what should we do?
![Page 79: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/79.jpg)
79
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
So what should we do?
We refine the granularity
![Page 80: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/80.jpg)
80
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 81: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/81.jpg)
81
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 82: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/82.jpg)
82
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 83: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/83.jpg)
83
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 84: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/84.jpg)
84
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
And now check complements
![Page 85: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/85.jpg)
85
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 86: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/86.jpg)
86
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 87: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/87.jpg)
87
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 8
![Page 88: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/88.jpg)
88
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 81 2 5 6 7 8
![Page 89: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/89.jpg)
89
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 8
What's clever about how we recurse?
![Page 90: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/90.jpg)
90
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 8
![Page 91: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/91.jpg)
91
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 7 81 2 7 8 So close! How many more?
![Page 92: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/92.jpg)
92
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 7 81 2 7 8
1 2 7 81 2 7 81 2 7 81 2 7 81 2 7 81 2 7 8
Done?
![Page 93: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/93.jpg)
93
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 7 81 2 7 8
1 2 7 81 2 7 81 2 7 81 2 7 81 2 7 81 2 7 81 7 81 7 81 7 81 7 81 7 81 7 8
Done?
![Page 94: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/94.jpg)
94
Classically – Delta Debugging
1) Test case to minimize
1 2 3 4 5 6 7 8c =
ddmin(c) = ddmin2(c, 2)
![Page 95: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/95.jpg)
95
Classically – Delta Debugging
1 2 3 4 5 6 7 8c =
1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin(c) = ddmin2(c, 2)
![Page 96: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/96.jpg)
96
Classically – Delta Debugging
1) Test case to minimize2) Granularity (|Δ|=|c|/n)
1 2 3 4 5 6 7 8c = Δ = 4
Δ1 Δ2 Δ3 Δ4
ddmin(c) = ddmin2(c, 2)
![Page 97: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/97.jpg)
97
Classically – Delta Debugging
1) Test case to minimize2) Granularity (|Δ|=|c|/n)
1 2 3 4 5 6 7 8c = Δ = 4
Δ1 Δ2 Δ3 Δ4
∇1
ddmin(c) = ddmin2(c, 2)
![Page 98: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/98.jpg)
98
Classically – Delta Debugging
1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin(c) = ddmin2(c, 2)
![Page 99: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/99.jpg)
99
Classically – Delta Debugging
ddmin(c) = ddmin2(c, 2) 1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin2(Δi, 2) If ... (a)
Δi = {3,4} (a) 1 2 3 4 5 6 7 8
![Page 100: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/100.jpg)
100
Classically – Delta Debugging
ddmin(c) = ddmin2(c, 2) 1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin2(Δi, 2)
ddmin2(∇i, max(n-1,2))
1 2 3 4 5 6 7 8
If ... (a)If ... (b)
Δi = {3,4} (a)(b)
1 2 3 4 5 6 7 8
![Page 101: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/101.jpg)
101
Classically – Delta Debugging
ddmin(c) = ddmin2(c, 2) 1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin2(Δi, 2)
ddmin2(∇i, max(n-1,2))
1 2 3 4 5 6 7 8
If ... (a)If ... (b)
Δi = {3,4} (a)(b)
1 2 3 4 5 6 7 8
![Page 102: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/102.jpg)
102
Classically – Delta Debugging
ddmin(c) = ddmin2(c, 2) 1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin2(Δi, 2)
ddmin2(∇i, max(n-1,2))ddmin2(c, min(|c|,2n))
1 2 3 4 5 6 7 8
If ... (a)If ... (b)If ... (c)
Δi = {3,4} (a)(b)(c) n < |c|
1 2 3 4 5 6 7 8
![Page 103: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/103.jpg)
103
Classically – Delta Debugging
ddmin(c) = ddmin2(c, 2) 1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin2(Δi, 2)
ddmin2(∇i, max(n-1,2))ddmin2(c, min(|c|,2n))
1 2 3 4 5 6 7 8
If ... (a)If ... (b)If ... (c)
Δi = {3,4} (a)(b)(c) n < |c|
1 2 3 4 5 6 7 8
![Page 104: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/104.jpg)
104
Classically – Delta Debugging
ddmin(c) = ddmin2(c, 2) 1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin2(Δi, 2)
ddmin2(∇i, max(n-1,2))ddmin2(c, min(|c|,2n))c
1 2 3 4 5 6 7 8
If ... (a)If ... (b)If ... (c)otherwise
Δi = {3,4} (a)(b)(c) n < |c|
1 2 3 4 5 6 7 8
![Page 105: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/105.jpg)
105
Classically – Delta Debugging
● Worst Case: |c|2 + 3|c| tests– All tests unresolved until maximum granularity– Testing complement succeeds
![Page 106: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/106.jpg)
106
Classically – Delta Debugging
● Worst Case: |c|2 + 3|c| tests– All tests unresolved until maximum granularity– Testing complement succeeds
● Best Case: # tests ≤ 2log2(|c|)– Falling back to binary search!
![Page 107: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/107.jpg)
107
Classically – Delta Debugging
● Worst Case: |c|2 + 3|c| tests– All tests unresolved until maximum granularity– Testing complement succeeds
● Best Case: # tests ≤ 2log2(|c|)– Falling back to binary search!
● Minimality– When will it only be locally minimal?– When will it only be 1-minimal?
![Page 108: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/108.jpg)
108
Classically – Delta Debugging
● Worst Case: |c|2 + 3|c| tests– All tests unresolved until maximum granularity– Testing complement succeeds
● Best Case: # tests ≤ 2log2(|c|)– Falling back to binary search!
● Minimality– When will it only be locally minimal?– When will it only be 1-minimal? – Does formal minimality even matter?
![Page 109: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/109.jpg)
109
Classically – Delta Debugging
● Observation:In practice DD may revisit elements in order to guarantee minimality
![Page 110: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/110.jpg)
110
Classically – Delta Debugging
● Observation:In practice DD may revisit elements in order to guarantee minimality
ddmin2(∇i, max(n-1,2))1 2 3 4 5 6 7 8
![Page 111: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/111.jpg)
111
Classically – Delta Debugging
● Observation:In practice DD may revisit elements in order to guarantee minimality
1 2 3 4 5 6 7 8
1 2 5 6 7 8
...ddmin2(∇i, max(n-1,2))
1 2 3 4 5 6 7 8
![Page 112: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/112.jpg)
112
Classically – Delta Debugging
● Observation:In practice DD may revisit elements in order to guarantee minimality
● If guaranteeing 1-minimality does not matter the algorithm can drop to linear time!– In practice this can be effective for what developers may care about
1 2 3 4 5 6 7 8
1 2 5 6 7 8
...ddmin2(∇i, max(n-1,2))
1 2 3 4 5 6 7 8
![Page 113: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/113.jpg)
113
Classically – Delta Debugging
● Observation:In practice DD may revisit elements in order to guarantee minimality
● If guaranteeing 1-minimality does not matter the algorithm can drop to linear time!– In practice this can be effective for what developers may care about
1 2 3 4 5 6 7 8
1 2 5 6 7 8
...ddmin2(∇i, max(n-1,2))
1 2 3 4 5 6 7 8
Don’t get bogged down by formalismwhen it doesn’t serve you!
![Page 114: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/114.jpg)
114
Test Case Reduction in Practice
● Most problems do not use DD directly for TCR.– It provides guidance, but frequently behaves poorly
![Page 115: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/115.jpg)
115
Test Case Reduction in Practice
● Most problems do not use DD directly for TCR.– It provides guidance, but frequently behaves poorly
● What are the possible causes of problems?
1 2 3 41 2 3 41 2 3 4
Monotonicitymatters
![Page 116: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/116.jpg)
116
Test Case Reduction in Practice
● Most problems do not use DD directly for TCR.– It provides guidance, but frequently behaves poorly
● What are the possible causes of problems?
1 2 3 41 2 3 41 2 3 4
Monotonicitymatters
1 2 3 41 2 3 4
Determinismmatters
![Page 117: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/117.jpg)
117
Test Case Reduction in Practice
● Most problems do not use DD directly for TCR.– It provides guidance, but frequently behaves poorly
● What are the possible causes of problems?
1 2 3 41 2 3 41 2 3 4
Monotonicitymatters
1 2 3 41 2 3 4
Determinismmatters Structure
matters
for
i range =
5 10 [] *
a i i 5
![Page 118: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/118.jpg)
118
Test Case Reduction for Compilers
● Programs are highly structured, so TCR for compilers faces challenges
![Page 119: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/119.jpg)
119
Test Case Reduction for Compilers
● Programs are highly structured, so TCR for compilers faces challenges
● What structures could we use to guide the process?
![Page 120: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/120.jpg)
120
Test Case Reduction for Compilers
● Programs are highly structured, so TCR for compilers faces challenges
● What structures could we use to guide the process?
for
i range =
5 10 [] *
a i i 5
![Page 121: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/121.jpg)
121
Test Case Reduction for Compilers
● Programs are highly structured, so TCR for compilers faces challenges
● What structures could we use to guide the process?
● What challenges still remain?
for
i range =
5 10 [] *
a i i 5
![Page 122: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/122.jpg)
Example: Memory Safety Bugs
![Page 123: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/123.jpg)
123
Example: Finding memory safety bugs
● Memory safety bugs are one of the most common sources ofsecurity vulnerabilities
![Page 124: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/124.jpg)
124
Example: Finding memory safety bugs
● Memory safety bugs are one of the most common sources ofsecurity vulnerabilities
● Effects may only be visible long after invalid behavior– This complicates comprehension & debugging
![Page 125: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/125.jpg)
125
Example: Finding memory safety bugs
● Memory safety bugs are one of the most common sources ofsecurity vulnerabilities
● Effects may only be visible long after invalid behavior– This complicates comprehension & debugging
● Two main types of issues:– Spatial – Out of bounds stack/heap/global accesses– Temporal – Use after free
![Page 126: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/126.jpg)
126
Example: Finding memory safety bugs
● Memory safety bugs are one of the most common sources ofsecurity vulnerabilities
● Effects may only be visible long after invalid behavior– This complicates comprehension & debugging
● Two main types of issues:– Spatial – Out of bounds stack/heap/global accesses– Temporal – Use after free
● We would like to automatically identify & provide assistancewith high precision and low overhead– Suitable for testing & sometimes maybe deployment!
![Page 127: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/127.jpg)
127
Example: Finding memory safety bugs
● Most common approach – track which regions of memory are valid– During execution!– Updated when new memory is allocated– Checked when pointers are accessed– With low overhead
![Page 128: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/128.jpg)
128
Example: Finding memory safety bugs
● Most common approach – track which regions of memory are valid– During execution!– Updated when new memory is allocated– Checked when pointers are accessed– With low overhead
● Common implementations– Valgrind – DBI based– AddressSanitizer – static instrumentation based
![Page 129: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/129.jpg)
129
Example: Finding memory safety bugs
● Most common approach – track which regions of memory are valid– During execution!– Updated when new memory is allocated– Checked when pointers are accessed– With low overhead
● Common implementations– Valgrind – DBI based– AddressSanitizer – static instrumentation based
Note, the implementation style affects which bugs can be recognized!
Why?
![Page 130: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/130.jpg)
130
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free
![Page 131: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/131.jpg)
131
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
![Page 132: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/132.jpg)
132
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks
ptr = malloc(sizeof(MyStruct));
ptr
![Page 133: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/133.jpg)
133
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed
free(ptr);
ptr
![Page 134: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/134.jpg)
134
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
void foo() { int buffer[5]; ...}
buffer[0]
buffer[6]
![Page 135: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/135.jpg)
135
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
![Page 136: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/136.jpg)
136
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
*address = ...;
instrumentation ?
![Page 137: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/137.jpg)
137
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
*address = ...; If (IsPoisoned(address, size)) { ReportError(address, size, isWrite);}*address = ...
instrumentation
![Page 138: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/138.jpg)
138
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
● The tricky part is tracking & efficiently checking redzones.
![Page 139: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/139.jpg)
139
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
● The tricky part is tracking & efficiently checking redzones.– Instrumenting every memory access is costly!
![Page 140: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/140.jpg)
140
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
● The tricky part is tracking & efficiently checking redzones.– Instrumenting every memory access is costly!– We must track all memory ... inside that same memory!
![Page 141: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/141.jpg)
141
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
● The tricky part is tracking & efficiently checking redzones.– Instrumenting every memory access is costly!– We must track all memory ... inside that same memory!
This kind of issue is common in dynamic analyses.
![Page 142: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/142.jpg)
142
AddressSanitizer – Shadow Memory
Application Memory
Need to know whether any byte of application memory is poisoned.
![Page 143: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/143.jpg)
143
AddressSanitizer – Shadow Memory
Application Memory Shadow Memory
● We maintain 2 views on memory
![Page 144: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/144.jpg)
144
AddressSanitizer – Shadow Memory
Application Memory Shadow Memory
● We maintain 2 views on memory
Shadow memory of theshadow memory!
![Page 145: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/145.jpg)
145
AddressSanitizer – Shadow Memory
Application Memory Shadow Memory
● We maintain 2 views on memory
● Shadow Memory is pervasive in dynamic analysis– Can be thought of as a map containing analysis data
![Page 146: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/146.jpg)
146
AddressSanitizer – Shadow Memory
Application Memory Shadow Memory
● We maintain 2 views on memory
● Shadow Memory is pervasive in dynamic analysis– Can be thought of as a map containing analysis data– For every bit/byte/word/chunk/allocation/page,
maintain information in a compact table
![Page 147: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/147.jpg)
147
AddressSanitizer – Shadow Memory
Application Memory Shadow Memory
● We maintain 2 views on memory
● Shadow Memory is pervasive in dynamic analysis– Can be thought of as a map containing analysis data– For every bit/byte/word/chunk/allocation/page,
maintain information in a compact table
Where have you encountered this before?
![Page 148: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/148.jpg)
148
AddressSanitizer – Shadow Memory
Application Memory Shadow Memory
● We maintain 2 views on memory
● Shadow Memory is pervasive in dynamic analysis– Can be thought of as a map containing analysis data– For every bit/byte/word/chunk/allocation/page,
maintain information in a compact table– Common in runtime support, e.g. page tables
![Page 149: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/149.jpg)
149
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
Encoding & abstraction efficiency strategies
![Page 150: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/150.jpg)
150
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)
![Page 151: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/151.jpg)
151
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable
k
![Page 152: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/152.jpg)
152
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states
![Page 153: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/153.jpg)
153
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
4
6
7
0
5
3
-1
1
2
![Page 154: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/154.jpg)
154
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
All good = 0 All bad = -1 Partly good = # good
4
6
7
0
5
3
-1
1
2
![Page 155: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/155.jpg)
155
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
● What does accessing shadow memory for an address look like? (N=8)
![Page 156: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/156.jpg)
156
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
● What does accessing shadow memory for an address look like? (N=8)– Preallocate a large table
![Page 157: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/157.jpg)
157
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
● What does accessing shadow memory for an address look like? (N=8)– Preallocate a large table– Shadow = (address >> 3) + Offset
![Page 158: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/158.jpg)
158
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
● What does accessing shadow memory for an address look like? (N=8)– Preallocate a large table– Shadow = (address >> 3) + Offset– With PIE, Shadow = (address >> 3)
![Page 159: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/159.jpg)
159
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
● What does accessing shadow memory for an address look like? (N=8)– Preallocate a large table– Shadow = (address >> 3) + Offset– With PIE, Shadow = (address >> 3)
if (*(address>>3)) { ReportError(...);}*address = ...
![Page 160: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/160.jpg)
160
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
● What does accessing shadow memory for an address look like? (N=8)– Preallocate a large table– Shadow = (address >> 3) + Offset– With PIE, Shadow = (address >> 3)
if (*(address>>3)) { ReportError(...);}*address = ...
Now you can also see the reason for the numerical encoding....
![Page 161: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/161.jpg)
161
AddressSanitizer – Shadow Memory
shadow = address >> 3state = *shadowif (state != 0 && (state < (address & 7) + size)) { ReportError(...);}*address = ...
● Handling accesses of size < N (N=8)
![Page 162: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/162.jpg)
162
AddressSanitizer – Shadow Memory
shadow = address >> 3state = *shadowif (state != 0 && (state < (address & 7) + size)) { ReportError(...);}*address = ...
● Handling accesses of size < N (N=8)
Careful construction of states can make runtime checks efficient
![Page 163: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/163.jpg)
163
AddressSanitizer - Evaluating
● In dynamic analyses, we care about both overheads & result quality
![Page 164: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/164.jpg)
164
AddressSanitizer - Evaluating
● In dynamic analyses, we care about both overheads & result quality
● Overheads– Need to determine what resources are being consumed
![Page 165: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/165.jpg)
165
AddressSanitizer - Evaluating
● In dynamic analyses, we care about both overheads & result quality
● Overheads– Need to determine what resources are being consumed– Memory –
Shadow memory capacity is cheap, but accessed shadows matter
![Page 166: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/166.jpg)
166
AddressSanitizer - Evaluating
● In dynamic analyses, we care about both overheads & result quality
● Overheads– Need to determine what resources are being consumed– Memory –
Shadow memory capacity is cheap, but accessed shadows matter– Running time –
Can effectively be free for I/O bound projectsUp to 25x overheads on some benchmarks
![Page 167: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/167.jpg)
167
AddressSanitizer - Evaluating
● In dynamic analyses, we care about both overheads & result quality
● Overheads– Need to determine what resources are being consumed– Memory –
Shadow memory capacity is cheap, but accessed shadows matter– Running time –
Can effectively be free for I/O bound projectsUp to 25x overheads on some benchmarks
● Quality– Precision & recall matter
Where will it miss bugs?Where will it raise false alarms?
![Page 168: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/168.jpg)
168
AddressSanitizer - Evaluating
● False negatives– Computed pointers that are accidentally in bounds
![Page 169: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/169.jpg)
169
AddressSanitizer - Evaluating
● False negatives– Computed pointers that are accidentally in bounds– Unaligned accesses that are partially out of bounds
![Page 170: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/170.jpg)
170
AddressSanitizer - Evaluating
● False negatives– Computed pointers that are accidentally in bounds– Unaligned accesses that are partially out of bounds– Use after frees with significant churn
![Page 171: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/171.jpg)
Example: Comparing Executions
![Page 172: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/172.jpg)
172
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
![Page 173: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/173.jpg)
173
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison
![Page 174: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/174.jpg)
174
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison– Debugging regressions – old vs new
![Page 175: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/175.jpg)
175
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison– Debugging regressions – old vs new– Validating patches – old vs new
![Page 176: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/176.jpg)
176
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison– Debugging regressions – old vs new– Validating patches – old vs new– Understanding automated repair – old vs new
![Page 177: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/177.jpg)
177
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison– Debugging regressions – old vs new– Validating patches – old vs new– Understanding automated repair – old vs new– Debugging with concurrency – buggy vs nonbuggy schedules
![Page 178: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/178.jpg)
178
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison– Debugging regressions – old vs new– Validating patches – old vs new– Understanding automated repair – old vs new– Debugging with concurrency – buggy vs nonbuggy schedules– Malware analysis – malicious vs nonmalicious run
![Page 179: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/179.jpg)
179
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison– Debugging regressions – old vs new– Validating patches – old vs new– Understanding automated repair – old vs new– Debugging with concurrency – buggy vs nonbuggy schedules– Malware analysis – malicious vs nonmalicious run– Reverse engineering – desired behavior vs undesirable
![Page 180: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/180.jpg)
180
How it might look
Correct Buggy
![Page 181: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/181.jpg)
181
How it might look
x was 5 instead of 3
Correct Buggy
![Page 182: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/182.jpg)
182
How it might look
x was 5 instead of 3
So y was 2 instead of 7
Correct Buggy
![Page 183: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/183.jpg)
183
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branch
Correct Buggy
![Page 184: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/184.jpg)
184
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
Correct Buggy
![Page 185: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/185.jpg)
185
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy
![Page 186: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/186.jpg)
186
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy What do we need?● locations● state● flow● causation
![Page 187: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/187.jpg)
187
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy What do we need?● locations● state● flow● causation
![Page 188: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/188.jpg)
188
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy What do we need?● locations● state● flow● causation
![Page 189: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/189.jpg)
189
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy What do we need?● locations● state● flow● causation
![Page 190: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/190.jpg)
190
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy What do we need?● locations● state● flow● causation
![Page 191: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/191.jpg)
191
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy What do we need?● locations● state● flow● causation
We can construct this backwardfrom a point of failure/difference
![Page 192: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/192.jpg)
192
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
![Page 193: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/193.jpg)
193
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
![Page 194: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/194.jpg)
194
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
baz()
foo()
![Page 195: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/195.jpg)
195
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
baz()
foo()
foo()
baz()
foo()
![Page 196: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/196.jpg)
196
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
baz()
foo()
foo()
baz()
foo()
What is marked as different?
![Page 197: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/197.jpg)
197
foo()
foo()
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
baz()
baz()
foo()
What is marked as different?
![Page 198: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/198.jpg)
198
baz()
baz()
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
foo()
foo()
foo()
What is marked as different?
What is intuitively different?
![Page 199: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/199.jpg)
199
baz()
baz()
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
foo()
foo()
foo()
What is marked as different?
What is intuitively different?
![Page 200: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/200.jpg)
200
baz()
baz()
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
foo()
foo()
foo()
What is marked as different?
What is intuitively different?
Execution comparison mustaccount for what a program
means and does!
![Page 201: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/201.jpg)
201
The big picture
● Fundamentally, execution comparison needs to account for
![Page 202: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/202.jpg)
202
The big picture
● Fundamentally, execution comparison needs to account for– Structure – How is a program organized?
![Page 203: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/203.jpg)
203
The big picture
● Fundamentally, execution comparison needs to account for– Structure – How is a program organized?– Value – What are the values in the different executions?
![Page 204: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/204.jpg)
204
The big picture
● Fundamentally, execution comparison needs to account for– Structure – How is a program organized?– Value – What are the values in the different executions?– Semantics – How is the meaning of the program affected by the differences?
![Page 205: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/205.jpg)
205
The big picture
● Fundamentally, execution comparison needs to account for– Structure– Value– Semantics
● We can attack these through
![Page 206: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/206.jpg)
206
The big picture
● Fundamentally, execution comparison needs to account for– Structure– Value– Semantics
● We can attack these through– Temporal alignment
● What parts of the trace correspond?
![Page 207: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/207.jpg)
207
The big picture
● Fundamentally, execution comparison needs to account for– Structure– Value– Semantics
● We can attack these through– Temporal alignment
● What parts of the trace correspond?
– Spatial alignment● What variables & values correspond across traces?
![Page 208: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/208.jpg)
208
The big picture
● Fundamentally, execution comparison needs to account for– Structure– Value– Semantics
● We can attack these through– Temporal alignment
● What parts of the trace correspond?
– Spatial alignment● What variables & values correspond across traces?
– Slicing● How do differences transitively flow through a program?
![Page 209: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/209.jpg)
209
The big picture
● Fundamentally, execution comparison needs to account for– Structure– Value– Semantics
● We can attack these through– Temporal alignment
● What parts of the trace correspond?
– Spatial alignment● What variables & values correspond across traces?
– Slicing● How do differences transitively flow through a program?
– Causality testing● Which differences actually induce difference behavior?
![Page 210: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/210.jpg)
210
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
i2
i1
?
![Page 211: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/211.jpg)
211
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
i2
i1
?
![Page 212: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/212.jpg)
212
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()
i2
i1
?
![Page 213: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/213.jpg)
213
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()
i2
i1
?
![Page 214: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/214.jpg)
214
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()
i2
i1
?
![Page 215: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/215.jpg)
215
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()
i2
i1
?
![Page 216: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/216.jpg)
216
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()
i2
i1
?
![Page 217: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/217.jpg)
217
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()Position along a path can be maintained via a counter
i2
i1
?
![Page 218: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/218.jpg)
218
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()Position along a path can be maintained via a counter
Only need to increment along1) back edges2) function callsi
2
i1
?
![Page 219: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/219.jpg)
219
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
i2
i1
?
![Page 220: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/220.jpg)
220
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
i2
i1
?
![Page 221: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/221.jpg)
221
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
The position in the DAG relates the paths
i2
i1
?
![Page 222: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/222.jpg)
222
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
i2
i1
?
![Page 223: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/223.jpg)
223
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
We can unwind the loop to make it logically acyclic
i2
i1
?
![Page 224: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/224.jpg)
224
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
i2
i1
?
![Page 225: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/225.jpg)
225
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
i2
i1
?
![Page 226: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/226.jpg)
226
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
...
i2
i1
?
![Page 227: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/227.jpg)
227
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
...
These are different iterations of one loop.A counter for each active loop suffices (mostly).
i2
i1
?
![Page 228: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/228.jpg)
228
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
...
1 counter per active loop+ the call stack disambiguates!
i2
i1
?
![Page 229: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/229.jpg)
229
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
– Can we efficiently represent this?
i2
i1
?
![Page 230: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/230.jpg)
230
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
– Can we efficiently represent this?
Call stack/context Iteration stack Instruction IDi2
i1
?
![Page 231: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/231.jpg)
231
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
– Can we efficiently represent this?
Call stack/context Iteration stack Instruction ID
Can be encoded/inferred Can be inferred
i2
i1
?
![Page 232: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/232.jpg)
232
Spatial Alignment
● We must also ask what it means to compare program state across executions
![Page 233: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/233.jpg)
233
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?
3 != 5
![Page 234: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/234.jpg)
234
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?
0xdeadbeef in T1 = 0xcafef00d in T2?
![Page 235: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/235.jpg)
235
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?
0xdeadbeef in T1 = 0xcafef00d in T2?
If you allocated other stuff in only one run,this can be true even without ASLR!
![Page 236: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/236.jpg)
236
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
![Page 237: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/237.jpg)
237
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
![Page 238: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/238.jpg)
238
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
![Page 239: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/239.jpg)
239
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?
![Page 240: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/240.jpg)
240
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?
1) list.append(value++)2) if c:3) list.append(value++)4) list.append(value++)
![Page 241: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/241.jpg)
241
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?
1) list.append(value++)2) if c:3) list.append(value++)4) list.append(value++)
1
1
![Page 242: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/242.jpg)
242
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?
1) list.append(value++)2) if c:3) list.append(value++)4) list.append(value++)
1
1 3
![Page 243: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/243.jpg)
243
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?
1) list.append(value++)2) if c:3) list.append(value++)4) list.append(value++)
1
1 3
4
4
![Page 244: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/244.jpg)
244
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?
1) list.append(value++)2) if c:3) list.append(value++)4) list.append(value++)
1
1 3
4
4
![Page 245: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/245.jpg)
245
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?
1) list.append(value++)2) if c:3) list.append(value++)4) list.append(value++)
1
1 3
4
4
![Page 246: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/246.jpg)
246
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
● Again, the semantics of the program dictate the solution– Identify heap allocations by the aligned time of allocation
A B C
A BT1
T2
1
1 3
4
4
![Page 247: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/247.jpg)
247
Dual Slicing
● Now we can– Identify corresponding times across executions– Identify & compare corresponding state at those times
![Page 248: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/248.jpg)
248
Dual Slicing
● Now we can– Identify corresponding times across executions– Identify & compare corresponding state at those times
● We can use these to enhance dynamic slicing by being aware of differences! (called dual slicing)
![Page 249: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/249.jpg)
249
Dual Slicing
● Now we can– Identify corresponding times across executions– Identify & compare corresponding state at those times
● We can use these to enhance dynamic slicing by being aware of differences! (called dual slicing)– Based on classic dynamic slicing– Include transitive dependencies that differ or exist in only 1 execution
![Page 250: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/250.jpg)
250
Dual Slicing
● Now we can– Identify corresponding times across executions– Identify & compare corresponding state at those times
● We can use these to enhance dynamic slicing by being aware of differences! (called dual slicing)– Based on classic dynamic slicing– Include transitive dependencies that differ or exist in only 1 execution
1)x 1← 1
2)y 1← 1
3)print(x+y)
1)x 0← 1
2)y 1← 1
3)print(x+y)3
2
1
3
2
1
0
1 1
1
![Page 251: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/251.jpg)
251
Dual Slicing
● Now we can– Identify corresponding times across executions– Identify & compare corresponding state at those times
● We can use these to enhance dynamic slicing by being aware of differences! (called dual slicing)– Based on classic dynamic slicing– Include transitive dependencies that differ or exist in only 1 execution
1)x 1← 1
2)y 1← 1
3)print(x+y)
1)x 0← 1
2)y 1← 1
3)print(x+y)3
2
1
3
2
1
0 1 1
1
![Page 252: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/252.jpg)
252
Dual Slicing
● Now we can– Identify corresponding times across executions– Identify & compare corresponding state at those times
● We can use these to enhance dynamic slicing by being aware of differences! (called dual slicing)– Based on classic dynamic slicing– Include transitive dependencies that differ or exist in only 1 execution
1)x 1← 1
2)y 1← 1
3)print(x+y)
1)x 0← 1
2)y 1← 1
3)print(x+y)3
2
1
3
2
1
3
2
1
0
1 1
10 1
![Page 253: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/253.jpg)
253
Dual Slicing
● The differences in dependencies capture multiple kinds of information
![Page 254: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/254.jpg)
254
Dual Slicing
● The differences in dependencies capture multiple kinds of information– Value-only differences
![Page 255: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/255.jpg)
255
Dual Slicing
● The differences in dependencies capture multiple kinds of information– Value-only differences– Provenance/Source differences
![Page 256: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/256.jpg)
256
Dual Slicing
● The differences in dependencies capture multiple kinds of information– Value-only differences– Provenance/Source differences– Missing/Extra behavior
![Page 257: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/257.jpg)
257
Dual Slicing
● The differences in dependencies capture multiple kinds of information– Value-only differences– Provenance/Source differences– Missing/Extra behavior
● Recall: Dynamic slicing could not handle execution omission,but dual slicing can!
![Page 258: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/258.jpg)
258
Dual Slicing
● The differences in dependencies capture multiple kinds of information– Value-only differences– Provenance/Source differences– Missing/Extra behavior
● Recall: Dynamic slicing could not handle execution omission,but dual slicing can!
● Dual slices can be effective for concurrent debugging & exploit analysis[Weeratunge, ISSTA 2010][Johnson, S&P 2011]
![Page 259: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/259.jpg)
259
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
Correct Buggy
![Page 260: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/260.jpg)
260
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
![Page 261: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/261.jpg)
261
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
![Page 262: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/262.jpg)
262
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
Correct
x = 10y = -1Truez = 0
“0”
![Page 263: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/263.jpg)
263
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 264: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/264.jpg)
264
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 265: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/265.jpg)
265
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 266: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/266.jpg)
266
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 267: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/267.jpg)
267
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 268: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/268.jpg)
268
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 269: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/269.jpg)
269
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
BuggyDual slicing captures differences, not causes.
What does that mean here?
![Page 270: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/270.jpg)
270
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 271: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/271.jpg)
271
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
The cost of these extra edges is high in practice!All transitive dependencies...
![Page 272: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/272.jpg)
272
Adding Causation
● Causation is often framed via “alternate worlds” & “what if” questions...– We can answer these causality questions by running experiments!
![Page 273: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/273.jpg)
What Should We Blame?
Recall:
?
Trial
![Page 274: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/274.jpg)
What Should We Blame?
Recall:
?
Trial
![Page 275: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/275.jpg)
What Should We Blame?
Recall: x = 5y = 4z = 3
x = 5y = 4z = 1
y = 4
?
Trial
![Page 276: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/276.jpg)
What Should We Blame?
Recall: x = 5y = 4z = 3
x = 5y = 4z = 1
y = 4
?
Trial
![Page 277: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/277.jpg)
What Should We Blame?
Recall: x = 5y = 4z = 3
x = 5y = 4z = 1
y = 4
?
Trial
What does this patched run even mean?
![Page 278: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/278.jpg)
x ← 1 1y ← 1 3z ← 1 6if False:
else: y ← 1 4print(4)
Example – Altered Meaning
1)x ← 1 input()2)y ← 1 input()3)z ← 1 input()4)if y+z > 10:5) y ← 1 56)else: y ← 1 y+17)print(y)
x ← 1 0y ← 1 7z ← 1 3if False:
else: y ← 1 8print(8)
CorrectBuggy
![Page 279: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/279.jpg)
Example – Altered Meaning
What should we blame here?
1)x ← 1 input()2)y ← 1 input()3)z ← 1 input()4)if y+z > 10:5) y ← 1 56)else: y ← 1 y+17)print(y)
x ← 1 0y ← 1 7z ← 1 3if False:
else: y ← 1 8print(8)
x ← 1 1y ← 1 3z ← 1 6if False:
else: y ← 1 4print(4)
CorrectBuggy
![Page 280: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/280.jpg)
x ← 1 0y ← 1 7z ← 1 3if False:
else: y ← 1 8print(8)
Example – Altered Meaning
1)x ← 1 input()2)y ← 1 input()3)z ← 1 input()4)if y+z > 10:5) y ← 1 56)else: y ← 1 y+17)print(y)
x ← 1 1y ← 1 3z ← 1 6if False:
else: y ← 1 4print(4)
CorrectBuggy Trial
![Page 281: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/281.jpg)
x ← 1 0y ← 1 7z ← 1 3if False:
else: y ← 1 8print(8)
y ← 1 7
Example – Altered Meaning
1)x ← 1 input()2)y ← 1 input()3)z ← 1 input()4)if y+z > 10:5) y ← 1 56)else: y ← 1 y+17)print(y)
x ← 1 1y ← 1 7z ← 1 6if False:
else: y ← 1 4print(4)
CorrectBuggy Trial
![Page 282: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/282.jpg)
x ← 1 0y ← 1 7z ← 1 3if False:
else: y ← 1 8print(8)
x ← 1 1y ← 1 7z ← 1 6
Example – Altered Meaning
1)x ← 1 input()2)y ← 1 input()3)z ← 1 input()4)if y+z > 10:5) y ← 1 56)else: y ← 1 y+17)print(y)
x ← 1 1y ← 1 7z ← 1 6if False:
else: y ← 1 4print(4)
CorrectBuggy Trial
![Page 283: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/283.jpg)
x ← 1 0y ← 1 7z ← 1 3if False:
else: y ← 1 8print(8)
x ← 1 1y ← 1 7z ← 1 6if True: y ← 1 5
print(5)
Example – Altered Meaning
1)x ← 1 input()2)y ← 1 input()3)z ← 1 input()4)if y+z > 10:5) y ← 1 56)else: y ← 1 y+17)print(y)
x ← 1 1y ← 1 7z ← 1 6if False:
else: y ← 1 4print(4)
CorrectBuggy Trial
![Page 284: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/284.jpg)
x ← 1 1y ← 1 7z ← 1 6if True: y ← 1 5
print(5)
Example – Altered Meaning
1)x ← 1 input()2)y ← 1 input()3)z ← 1 input()4)if y+z > 10:5) y ← 1 56)else: y ← 1 y+17)print(y)
● New control flow unlike original runs
● Occurs in large portion of real bugs
x ← 1 0y ← 1 7z ← 1 3if False:
else: y ← 1 8print(8)
x ← 1 1y ← 1 7z ← 1 6if False:
else: y ← 1 4print(4)
CorrectBuggy Trial
![Page 285: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/285.jpg)
Dual Slicing
1)x ← 1 input()2)y ← 1 input()3)z ← 1 input()4)if y+z > 10:5) y ← 1 56)else: y ← 1 y+17)print(y)
76
2
2)y ← 1 input()6)y ← 1 y+17)print(y)
Extract
x ← 1 0y ← 1 7z ← 1 3if False:
else: y ← 1 8print(8)
x ← 1 1y ← 1 3z ← 1 6if False:
else: y ← 1 4print(4)
CorrectBuggy
![Page 286: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/286.jpg)
y ← 1 3y ← 1 4print(4)
Example – Extracted Meaning
y ← 1 7y ← 1 8print(8)
2)y ← 1 input()6)y ← 1 y+17)print(y)
CorrectBuggy Trial
![Page 287: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/287.jpg)
y ← 1 7 y ← 1 7y ← 1 4print(4)
Example – Extracted Meaning
y ← 1 7y ← 1 8print(8)
2)y ← 1 input()6)y ← 1 y+17)print(y)
CorrectBuggy Trial
![Page 288: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/288.jpg)
y ← 1 8print(8)
y ← 1 7 y ← 1 7y ← 1 4print(4)
Example – Extracted Meaning
y ← 1 7y ← 1 8print(8)
2)y ← 1 input()6)y ← 1 y+17)print(y)
Trial can now correctly blame y
CorrectBuggy Trial
![Page 289: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/289.jpg)
289
Adding Causation
● Causation is often framed via “alternate worlds” & “what if” questions...– We can answer these causality questions by running experiments!
● We perform these causality tests in both directions in order to collect symmetric information
![Page 290: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/290.jpg)
Summing Up
![Page 291: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/291.jpg)
291
Key Challenges
● Identifying the information you care about– Dynamic dependence? Valid memory? Just the execution outcome?
![Page 292: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/292.jpg)
292
Key Challenges
● Identifying the information you care about– Dynamic dependence? Valid memory? Just the execution outcome?
● Collecting that information efficiently– abstraction, encoding, compression, sampling, ...
![Page 293: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/293.jpg)
293
Key Challenges
● Identifying the information you care about– Dynamic dependence? Valid memory? Just the execution outcome?
● Collecting that information efficiently– abstraction, encoding, compression, sampling, ...
● Selecting which executions to analyze– Existing test suite? Always on runtime? Directed test generation?– How does underapproximation affect your conclusions?– Can you still achieve your objective in spite of it?
![Page 294: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/294.jpg)
294
Key Challenges
● Identifying the information you care about– Dynamic dependence? Valid memory? Just the execution outcome?
● Collecting that information efficiently– abstraction, encoding, compression, sampling, ...
● Selecting which executions to analyze– Existing test suite? Always on runtime? Directed test generation?– How does underapproximation affect your conclusions?– Can you still achieve your objective in spite of it?
● Doing some of the work ahead of time– What can you precompute to improve all of the above?
![Page 295: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/295.jpg)
295
Summary
● Analyze the actual/observed behaviors of a program
![Page 296: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/296.jpg)
296
Summary
● Analyze the actual/observed behaviors of a program
● Modify or use the program’s behavior to collect information
![Page 297: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/297.jpg)
297
Summary
● Analyze the actual/observed behaviors of a program
● Modify or use the program’s behavior to collect information
● Analyze the information online or offline
![Page 298: Dynamic Analysis - cs.sfu.cawsumner/teaching/745/08-dynamic-analysis.pdf · Dynamic Binary Instrumentation (DBI) 1) Compile program as usual 2) Run program under analysis framework](https://reader030.vdocuments.us/reader030/viewer/2022040401/5e7937335afebb57ce79263a/html5/thumbnails/298.jpg)
298
Summary
● Analyze the actual/observed behaviors of a program
● Modify or use the program’s behavior to collect information
● Analyze the information online or offline
● The precise configuration must be tailored to the objectives & insights– Compiled vs DBI– Online vs Postmortem– Compressed, Encoded, Samples, ...– ...