dynamic analysis - simon fraser university
TRANSCRIPT
![Page 2: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/2.jpg)
2
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
![Page 3: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/3.jpg)
3
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
– Did my program ever …?
![Page 4: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/4.jpg)
4
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
– Did my program ever …?– Why/how did … happen?
![Page 5: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/5.jpg)
5
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
– Did my program ever …?– Why/how did … happen?– Where am I spending time?
![Page 6: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/6.jpg)
6
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
– Did my program ever …?– Why/how did … happen?– Where am I spending time?– Where might I parallelize?
![Page 7: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/7.jpg)
7
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
– Did my program ever …?– Why/how did … happen?– Where am I spending time?– Where might I parallelize?– Tolerate errors
![Page 8: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/8.jpg)
8
Dynamic Analysis
● Sometimes we want to study or adapt the behavior of executions of a program
– Did my program ever …?– Why/how did … happen?– Where am I spending time?– Where might I parallelize?– Tolerate errors– Manage memory / resources.
![Page 9: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/9.jpg)
9
e.g. Reverse Engineering
Static CFG (from e.g. Apple Fairplay):
This is the result of a control flow flattening obfuscaton.[http://tigress.cs.arizona.edu/transformPage/docs/flatten/]
![Page 10: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/10.jpg)
10
e.g. Reverse Engineering
Static CFG (from e.g. Apple Fairplay):
Dynamically Simplified CFG:
This is the result of a control flow flattening obfuscaton.[http://tigress.cs.arizona.edu/transformPage/docs/flatten/]
![Page 11: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/11.jpg)
11
How?
● Can record the execution
![Page 12: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/12.jpg)
12
How?
● Can record the execution– Record to a trace– Analyze post mortem / offline– Scalability issues: need enough space to store it
![Page 13: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/13.jpg)
13
How?
● Can record the execution– Record to a trace– Analyze post mortem / offline– Scalability issues: need enough space to store it
● Can perform analysis online
![Page 14: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/14.jpg)
14
How?
● Can record the execution– Record to a trace– Analyze post mortem / offline– Scalability issues: need enough space to store it
● Can perform analysis online– Instrument the program to collect useful facts– Modified program invokes code to 'analyze' itself
![Page 15: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/15.jpg)
15
How?
● Can record the execution– Record to a trace– Analyze post mortem / offline– Scalability issues: need enough space to store it
● Can perform analysis online– Instrument the program to collect useful facts– Modified program invokes code to 'analyze' itself
● Can do both!– Lightweight recording– Instrument a replayed instance of the execution
![Page 16: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/16.jpg)
16
How?
● Can record the execution– Record to a trace– Analyze post mortem / offline– Scalability issues: need enough space to store it
● Can perform analysis online– Instrument the program to collect useful facts– Modified program invokes code to 'analyze' itself
● Can do both!– Lightweight recording– Instrument a replayed instance of the execution
Some analyses only make sense online.Why?
![Page 17: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/17.jpg)
17
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?
![Page 18: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/18.jpg)
18
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?
Profiling is a common dynamic analysis!
![Page 19: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/19.jpg)
19
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?● How can we modify our program to find this?
![Page 20: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/20.jpg)
20
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?● How can we modify our program to find this?
BB:0
BB:1 BB:2
BB:3
?
![Page 21: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/21.jpg)
21
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?● How can we modify our program to find this?
BB:0
BB:1 BB:2
BB:3
count[2] += 1
x = foo()y = bar()...
![Page 22: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/22.jpg)
22
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?● How can we modify our program to find this?
BB:0
BB:1 BB:2
BB:3
count[2] += 1
x = foo()y = bar()...
for i in BBs: count[i] = 0
for i in BBs: print(count[i])Start: End:
![Page 23: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/23.jpg)
23
Simple Idea: Basic Block Profiling
Knowing where we are spending time is useful:
● Goal: Which basic blocks execute most frequently?● How can we modify our program to find this?
BB:0
BB:1 BB:2
BB:3
count[2] += 1
x = foo()y = bar()...
for i in BBs: count[i] = 0
for i in BBs: print(count[i])Start: End:
![Page 24: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/24.jpg)
24
Simple Idea: Basic Block Profiling
● Big concern: How efficient is it?– The more overhead added, the less practical the tool
![Page 25: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/25.jpg)
25
Simple Idea: Basic Block Profiling
● Big concern: How efficient is it?– The more overhead added, the less practical the tool
count[0] += 1…
count[1] += 1…
count[5] += 1…
count[6] += 1…
count[4] += 1…
count[2] += 1…
count[3] += 1…
![Page 26: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/26.jpg)
26
Simple Idea: Basic Block Profiling
● Big concern: How efficient is it?– The more overhead added, the less practical the tool
– Can we do better?
count[0] += 1…
count[1] += 1…
count[5] += 1…
count[6] += 1…
count[4] += 1…
count[2] += 1…
count[3] += 1…
![Page 27: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/27.jpg)
27
Simple Idea: Basic Block Profiling
● Big concern: How efficient is it?– The more overhead added, the less practical the tool
– Can we do better?
count[0] += 1…
count[1] += 1…
count[5] += 1…
count[6] += 1…
count[4] += 1…
count[2] += 1…
count[3] += 1…
count[1] = count[4] = count[2] + count[3]
![Page 28: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/28.jpg)
28
Simple Idea: Basic Block Profiling
● Big concern: How efficient is it?– The more overhead added, the less practical the tool
– Can we do better?
count[0] += 1…
count[1] += 1…
count[5] += 1…
count[6] += 1…
count[4] += 1…
count[2] += 1…
count[3] += 1…
count[1] = count[4] = count[2] + count[3]count[0] = count[6] = count[1] + count[5]
![Page 29: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/29.jpg)
29
Simple Idea: Basic Block Profiling
● Big concern: How efficient is it?– The more overhead added, the less practical the tool
– Can we do better?
count[0] += 1…
count[1] += 1…
count[5] += 1…
count[6] += 1…
count[4] += 1…
count[2] += 1…
count[3] += 1…
count[1] = count[4] = count[2] + count[3]count[0] = count[6] = count[1] + count[5]
![Page 30: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/30.jpg)
30
Efficiency Tactics
● Abstraction
![Page 31: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/31.jpg)
31
Efficiency Tactics
● Abstraction
● Identify & avoid redundant information
![Page 32: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/32.jpg)
32
Efficiency Tactics
● Abstraction
● Identify & avoid redundant information
● Sampling
![Page 33: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/33.jpg)
33
Efficiency Tactics
● Abstraction
● Identify & avoid redundant information
● Sampling
● Compression / encoding
![Page 34: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/34.jpg)
34
Efficiency Tactics
● Abstraction
● Identify & avoid redundant information
● Sampling
● Compression / encoding
● Profile guided instrumentation
![Page 35: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/35.jpg)
35
Efficiency Tactics
● Abstraction
● Identify & avoid redundant information
● Sampling
● Compression / encoding
● Profile guided instrumentation
● Thread local analysis
![Page 36: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/36.jpg)
36
Efficiency Tactics
● Abstraction
● Identify & avoid redundant information
● Sampling
● Compression / encoding
● Profile guided instrumentation
● Thread local analysis
● Inference
![Page 37: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/37.jpg)
37
How / When to Instrument
● Source / IR Instrumentation– LLVM, CIL, Soot, Wala– During (re)compilation– Requires an analysis dedicated build
![Page 38: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/38.jpg)
38
How / When to Instrument
● Source / IR Instrumentation– LLVM, CIL, Soot, Wala– During (re)compilation– Requires an analysis dedicated build
● Static Binary Rewriting– Diablo, DynamoRIO, SecondWrite, – Applies to arbitrary binaries– Imprecise IR info, but more complete binary behavior
![Page 39: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/39.jpg)
39
How / When to Instrument
● Source / IR Instrumentation– LLVM, CIL, Soot, Wala– During (re)compilation– Requires an analysis dedicated build
● Static Binary Rewriting– Diablo, DynamoRIO, SecondWrite, – Applies to arbitrary binaries– Imprecise IR info, but more complete binary behavior
● Dynamic Binary Instrumentation– Valgrind, Pin, Qemu (& other Vms)– Can adapt at runtime, but less info than IR
![Page 40: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/40.jpg)
40
How / When to Instrument
● Source / IR Instrumentation– LLVM, CIL, Soot, Wala– During (re)compilation– Requires an analysis dedicated build
● Static Binary Rewriting– Diablo, DynamoRIO, SecondWrite, – Applies to arbitrary binaries– Imprecise IR info, but more complete binary behavior
● Dynamic Binary Instrumentation– Valgrind, Pin, Qemu (& other Vms)– Can adapt at runtime, but less info than IR
● Black Box Dynamic Analysis
![Page 41: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/41.jpg)
41
Phases of Dynamic Analysis
In general, 2-3 phases occur:
1) Instrumentation– Add code to the program for data collection/analysis
![Page 42: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/42.jpg)
42
Phases of Dynamic Analysis
In general, 2-3 phases occur:
1) Instrumentation– Add code to the program for data collection/analysis
2) Execution– Run the program an analyze its actual behavior
![Page 43: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/43.jpg)
43
Phases of Dynamic Analysis
In general, 2-3 phases occur:
1) Instrumentation– Add code to the program for data collection/analysis
2) Execution– Run the program an analyze its actual behavior
3) (Optional) Postmortem Analysis– Perform any analysis that can be deferred after termination
![Page 44: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/44.jpg)
44
Phases of Dynamic Analysis
In general, 2-3 phases occur:
1) Instrumentation– Add code to the program for data collection/analysis
2) Execution– Run the program an analyze its actual behavior
3) (Optional) Postmortem Analysis– Perform any analysis that can be deferred after termination
Very, very common mistake to mix 1 & 2.
![Page 45: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/45.jpg)
45
Static Instrumentation
1) Compile whole program to IR
foo.cbar.c
baz.cprog.ll
![Page 46: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/46.jpg)
46
Static Instrumentation
1) Compile whole program to IR
2) Instrument / add code directly to the IR
foo.cbar.c
baz.cprog.ll prog’.ll
![Page 47: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/47.jpg)
47
Static Instrumentation
1) Compile whole program to IR
2) Instrument / add code directly to the IR
3) Generate new program that performs analysis
foo.cbar.c
baz.cprog.ll prog’.ll
![Page 48: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/48.jpg)
48
Static Instrumentation
1) Compile whole program to IR
2) Instrument / add code directly to the IR
3) Generate new program that performs analysis
4) Execute
foo.cbar.c
baz.cprog.ll prog’.ll
Test Cases
Results
![Page 49: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/49.jpg)
49
Dynamic Binary Instrumentation (DBI)
1) Compile program as usual
2) Run program under analysis framework
(Valgrind, PIN, Qemu, etc)
valgrind --tool=memcheck ./myBuggyProgram
![Page 50: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/50.jpg)
50
Dynamic Binary Instrumentation (DBI)
1) Compile program as usual
2) Run program under analysis framework
(Valgrind, PIN, Qemu, etc)
3) Instrument & execute in same command:– Fetch & instrument each basic block individually– Execute each basic block
valgrind --tool=memcheck ./myBuggyProgram
![Page 51: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/51.jpg)
Example: Test Case Reduction
![Page 52: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/52.jpg)
52
Testing and Dynamic Analysis
● In some cases, just running a program with different inputs is enough
![Page 53: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/53.jpg)
53
Testing and Dynamic Analysis
● In some cases, just running a program with different inputs is enough– Carefully selected inputs can target the analysis– The result of running the program reveals coarse information about its
behavior
![Page 54: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/54.jpg)
54
Testing and Dynamic Analysis
● In some cases, just running a program with different inputs is enough– Carefully selected inputs can target the analysis– The result of running the program reveals coarse information about its
behavior
● Intuitively, even just testing is a dynamic analysis– It requires no transformation– The result is just the success or failure of tests
![Page 55: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/55.jpg)
55
Testing and Dynamic Analysis
● In some cases, just running a program with different inputs is enough– Carefully selected inputs can target the analysis– The result of running the program reveals coarse information about its
behavior
● Intuitively, even just testing is a dynamic analysis– It requires no transformation– The result is just the success or failure of tests
● But even that is interesting to consider....
![Page 56: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/56.jpg)
56
Bug reports are problematic
● Failing inputs can be large and complex
a r h w l n y e u m g k o w h > ` p
MB? GB?
![Page 57: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/57.jpg)
57
Bug reports are problematic
● Failing inputs can be large and complex
a r h w l n y e u m g k o w h > ` p
MB? GB? What is relevant and essential to the bug?
![Page 58: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/58.jpg)
58
Bug reports are problematic
a r h w l n y e u m g k o w h > ` p
● Failing inputs can be large and complex
a b cBug 2
a b cBug 3
a b cBug 1
![Page 59: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/59.jpg)
59
Bug reports are problematic
a r h w l n y e u m g k o w h > ` p
● Failing inputs can be large and complex
a b cBug 2
a b cBug 3
a b cBug 1
1) Are these reports the same bug?2) Can we make it easier to reproduce?
![Page 60: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/60.jpg)
60
Bug reports are problematic
a r h w l n y e u m g k o w h > ` p
● Failing inputs can be large and complex
a b cBug 2
a b cBug 3
a b cBug 1
a b cBug
![Page 61: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/61.jpg)
61
Bug reports are problematic
a r h w l n y e u m g k o w h > ` p
● Failing inputs can be large and complex
a b cBug 2
a b cBug 3
a b cBug 1
a b cBug
1) Same? Yes!2) Easier? Yes! And easier to understand!
![Page 62: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/62.jpg)
62
Bug reports are problematic
a r h w l n y e u m g k o w h > ` p
● Failing inputs can be large and complex
a b cBug 2
a b cBug 3
a b cBug 1
a b cBug
Test Case Reduction: finding smaller test cases that reproduce a failure
![Page 63: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/63.jpg)
63
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>
http://en.wikipedia.org/wiki/File:Netscape_2_logo.gif
![Page 64: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/64.jpg)
64
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>
http://en.wikipedia.org/wiki/File:Netscape_2_logo.gif
![Page 65: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/65.jpg)
65
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error
![Page 66: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/66.jpg)
66
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7> = cIntuition: trial and error1) Start w/ a failing text configuration c
![Page 67: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/67.jpg)
67
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ})
![Page 68: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/68.jpg)
68
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ})3) Failure still exists → new input is “better”
![Page 69: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/69.jpg)
69
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ})3) Failure still exists → new input is “better”4) Repeat on the new input
![Page 70: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/70.jpg)
70
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'
![Page 71: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/71.jpg)
71
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'
Smallest subset of the originalinput reproducing the failure
![Page 72: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/72.jpg)
72
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'
Smallest subset of the originalinput reproducing the failure
Completely impractical! Why?
![Page 73: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/73.jpg)
73
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'● Local Minimum: c : ∀ c'⊂c, c'
![Page 74: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/74.jpg)
74
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'● Local Minimum: c : ∀ c'⊂c, c'
No subset of the result canreproduce the failure.
![Page 75: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/75.jpg)
75
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'● Local Minimum: c : ∀ c'⊂c, c'
No subset of the result canreproduce the failure.
How does this differ from a global minimum?Is it still problematic?
![Page 76: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/76.jpg)
76
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'● Local Minimum: c : ∀ c'⊂c, c'● 1-Minimal: c: ∀ δ ∈ c, (c-{δ})
![Page 77: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/77.jpg)
77
Classically – Delta Debugging
<SELECT NAME="priority" MULTIPLE SIZE=7>Intuition: trial and error1) Start w/ a failing text configuration c2) Try removing subsets (Δ) of input elements ({δ})3) Failure still exists → new input is “better”4) Repeat on the new input
When do we stop? / What is our goal?● Global Minimum: c : ∀ |c'|<|c|, c'● Local Minimum: c : ∀ c'⊂c, c'● 1-Minimal: c: ∀ δ ∈ c, (c-{δ})
No one element can be removedand still reproduce the failure
![Page 78: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/78.jpg)
78
Classically – Delta Debugging1 2 3 4 5 6 7 8
Does binary search work?
![Page 79: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/79.jpg)
79
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 80: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/80.jpg)
80
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
So what should we do?
![Page 81: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/81.jpg)
81
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
So what should we do?
We refine the granularity
![Page 82: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/82.jpg)
82
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 83: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/83.jpg)
83
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 84: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/84.jpg)
84
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 85: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/85.jpg)
85
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 86: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/86.jpg)
86
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
And now check complements
![Page 87: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/87.jpg)
87
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 88: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/88.jpg)
88
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 8
![Page 89: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/89.jpg)
89
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 8
![Page 90: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/90.jpg)
90
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 81 2 5 6 7 8
![Page 91: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/91.jpg)
91
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 8
What's clever about how we recurse?
![Page 92: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/92.jpg)
92
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 8
![Page 93: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/93.jpg)
93
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 7 81 2 7 8 So close! How many more?
![Page 94: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/94.jpg)
94
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 7 81 2 7 8
1 2 7 81 2 7 81 2 7 81 2 7 81 2 7 81 2 7 8
Done?
![Page 95: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/95.jpg)
95
Classically – Delta Debugging1 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 3 4 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 5 6 7 81 2 7 81 2 7 8
1 2 7 81 2 7 81 2 7 81 2 7 81 2 7 81 2 7 81 7 81 7 81 7 81 7 81 7 81 7 8
Done?
![Page 96: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/96.jpg)
96
Classically – Delta Debugging
1) Test case to minimize
1 2 3 4 5 6 7 8c =
ddmin(c) = ddmin2(c, 2)
![Page 97: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/97.jpg)
97
Classically – Delta Debugging
1 2 3 4 5 6 7 8c =
1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin(c) = ddmin2(c, 2)
![Page 98: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/98.jpg)
98
Classically – Delta Debugging
1) Test case to minimize2) Granularity (|Δ|=|c|/n)
1 2 3 4 5 6 7 8c = Δ = 4
Δ1 Δ2 Δ3 Δ4
ddmin(c) = ddmin2(c, 2)
![Page 99: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/99.jpg)
99
Classically – Delta Debugging
1) Test case to minimize2) Granularity (|Δ|=|c|/n)
1 2 3 4 5 6 7 8c = Δ = 4
Δ1 Δ2 Δ3 Δ4
∇1
ddmin(c) = ddmin2(c, 2)
![Page 100: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/100.jpg)
100
Classically – Delta Debugging
1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin(c) = ddmin2(c, 2)
![Page 101: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/101.jpg)
101
Classically – Delta Debugging
ddmin(c) = ddmin2(c, 2) 1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin2(Δi, 2) If ... (a)
Δi = {3,4} (a) 1 2 3 4 5 6 7 8
![Page 102: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/102.jpg)
102
Classically – Delta Debugging
ddmin(c) = ddmin2(c, 2) 1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin2(Δi, 2)
ddmin2(∇i, max(n-1,2))
1 2 3 4 5 6 7 8
If ... (a)If ... (b)
Δi = {3,4} (a)(b)
1 2 3 4 5 6 7 8
![Page 103: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/103.jpg)
103
Classically – Delta Debugging
ddmin(c) = ddmin2(c, 2) 1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin2(Δi, 2)
ddmin2(∇i, max(n-1,2))
1 2 3 4 5 6 7 8
If ... (a)If ... (b)
Δi = {3,4} (a)(b)
1 2 3 4 5 6 7 8
![Page 104: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/104.jpg)
104
Classically – Delta Debugging
ddmin(c) = ddmin2(c, 2) 1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin2(Δi, 2)
ddmin2(∇i, max(n-1,2))ddmin2(c, min(|c|,2n))
1 2 3 4 5 6 7 8
If ... (a)If ... (b)If ... (c)
Δi = {3,4} (a)(b)(c) n < |c|
1 2 3 4 5 6 7 8
![Page 105: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/105.jpg)
105
Classically – Delta Debugging
ddmin(c) = ddmin2(c, 2) 1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin2(Δi, 2)
ddmin2(∇i, max(n-1,2))ddmin2(c, min(|c|,2n))
1 2 3 4 5 6 7 8
If ... (a)If ... (b)If ... (c)
Δi = {3,4} (a)(b)(c) n < |c|
1 2 3 4 5 6 7 8
![Page 106: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/106.jpg)
106
Classically – Delta Debugging
ddmin(c) = ddmin2(c, 2) 1) Test case to minimize2) Granularity (|Δ|=|c|/n)
ddmin2(c, 2)=
ddmin2(Δi, 2)
ddmin2(∇i, max(n-1,2))ddmin2(c, min(|c|,2n))c
1 2 3 4 5 6 7 8
If ... (a)If ... (b)If ... (c)otherwise
Δi = {3,4} (a)(b)(c) n < |c|
1 2 3 4 5 6 7 8
![Page 107: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/107.jpg)
107
Classically – Delta Debugging
● Worst Case: |c|2 + 3|c| tests– All tests unresolved until maximum granularity– Testing complement succeeds
![Page 108: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/108.jpg)
108
Classically – Delta Debugging
● Worst Case: |c|2 + 3|c| tests– All tests unresolved until maximum granularity– Testing complement succeeds
● Best Case: # tests ≤ 2log2(|c|)– Falling back to binary search!
![Page 109: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/109.jpg)
109
Classically – Delta Debugging
● Worst Case: |c|2 + 3|c| tests– All tests unresolved until maximum granularity– Testing complement succeeds
● Best Case: # tests ≤ 2log2(|c|)– Falling back to binary search!
● Minimality– When will it only be locally minimal?– When will it only be 1-minimal?
![Page 110: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/110.jpg)
110
Classically – Delta Debugging
● Worst Case: |c|2 + 3|c| tests– All tests unresolved until maximum granularity– Testing complement succeeds
● Best Case: # tests ≤ 2log2(|c|)– Falling back to binary search!
● Minimality– When will it only be locally minimal?– When will it only be 1-minimal? – Does formal minimality even matter?
![Page 111: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/111.jpg)
111
Classically – Delta Debugging
● Observation:In practice DD may revisit elements in order to guarantee minimality
![Page 112: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/112.jpg)
112
Classically – Delta Debugging
● Observation:In practice DD may revisit elements in order to guarantee minimality
ddmin2(∇i, max(n-1,2))1 2 3 4 5 6 7 8
![Page 113: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/113.jpg)
113
Classically – Delta Debugging
● Observation:In practice DD may revisit elements in order to guarantee minimality
1 2 3 4 5 6 7 8
1 2 5 6 7 8
...ddmin2(∇i, max(n-1,2))
1 2 3 4 5 6 7 8
![Page 114: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/114.jpg)
114
Classically – Delta Debugging
● Observation:In practice DD may revisit elements in order to guarantee minimality
● If guaranteeing 1-minimality does not matter the algorithm can drop to linear time!– In practice this can be effective for what developers may care about
1 2 3 4 5 6 7 8
1 2 5 6 7 8
...ddmin2(∇i, max(n-1,2))
1 2 3 4 5 6 7 8
![Page 115: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/115.jpg)
115
Classically – Delta Debugging
● Observation:In practice DD may revisit elements in order to guarantee minimality
● If guaranteeing 1-minimality does not matter the algorithm can drop to linear time!– In practice this can be effective for what developers may care about
1 2 3 4 5 6 7 8
1 2 5 6 7 8
...ddmin2(∇i, max(n-1,2))
1 2 3 4 5 6 7 8
Don’t get bogged down by formalismwhen it doesn’t serve you!
![Page 116: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/116.jpg)
116
Test Case Reduction in Practice
● Most problems do not use DD directly for TCR.– It provides guidance, but frequently behaves poorly
![Page 117: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/117.jpg)
117
Test Case Reduction in Practice
● Most problems do not use DD directly for TCR.– It provides guidance, but frequently behaves poorly
● What are the possible causes of problems?
1 2 3 41 2 3 41 2 3 4
Monotonicitymatters
![Page 118: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/118.jpg)
118
Test Case Reduction in Practice
● Most problems do not use DD directly for TCR.– It provides guidance, but frequently behaves poorly
● What are the possible causes of problems?
1 2 3 41 2 3 41 2 3 4
Monotonicitymatters
1 2 3 41 2 3 4
Determinismmatters
![Page 119: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/119.jpg)
119
Test Case Reduction in Practice
● Most problems do not use DD directly for TCR.– It provides guidance, but frequently behaves poorly
● What are the possible causes of problems?
1 2 3 41 2 3 41 2 3 4
Monotonicitymatters
1 2 3 41 2 3 4
Determinismmatters Structure
matters
LOOP
i
RANGE
ASSIGN
5 10
INDEX EXPR
a i i 5for in [ ] = *
![Page 120: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/120.jpg)
120
Test Case Reduction for Compilers
● Programs are highly structured, so TCR for compilers faces challenges
![Page 121: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/121.jpg)
121
Test Case Reduction for Compilers
● Programs are highly structured, so TCR for compilers faces challenges
● What structures could we use to guide the process?
![Page 122: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/122.jpg)
122
Test Case Reduction for Compilers
● Programs are highly structured, so TCR for compilers faces challenges
● What structures could we use to guide the process?
LOOP
i
RANGE
ASSIGN
5 10
INDEX EXPR
a i i 5for in [ ] = *
![Page 123: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/123.jpg)
123
Test Case Reduction for Compilers
● Programs are highly structured, so TCR for compilers faces challenges
● What structures could we use to guide the process?
● What challenges still remain?
LOOP
i
RANGE
ASSIGN
5 10
INDEX EXPR
a i i 5for in [ ] = *
![Page 124: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/124.jpg)
124
Generalizing Further
● What else could we think of as test case reduction?
![Page 125: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/125.jpg)
125
Generalizing Further
● What else could we think of as test case reduction?– Failing traces of a program?– “ ” in a distributed system?– “ ” microservice application?
![Page 126: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/126.jpg)
126
Generalizing Further
● What else could we think of as test case reduction?– Failing traces of a program?– “ ” in a distributed system?– “ ” microservice application?– Automatically generated test cases?
![Page 127: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/127.jpg)
127
Generalizing Further
● What else could we think of as test case reduction?– Failing traces of a program?– “ ” in a distributed system?– “ ” microservice application?– Automatically generated test cases?– ...
![Page 128: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/128.jpg)
128
Generalizing Further
● What else could we think of as test case reduction?– Failing traces of a program?– “ ” in a distributed system?– “ ” microservice application?– Automatically generated test cases?
● The ability to treat the program as an oracle is also very powerful
![Page 129: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/129.jpg)
129
Generalizing Further
● What else could we think of as test case reduction?– Failing traces of a program?– “ ” in a distributed system?– “ ” microservice application?– Automatically generated test cases?
● The ability to treat the program as an oracle is also very powerful– We can get new data by running the program– This can be combined with reinforcement learning to accomplish hard tasks
![Page 130: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/130.jpg)
130
Generalizing Further
● What else could we think of as test case reduction?– Failing traces of a program?– “ ” in a distributed system?– “ ” microservice application?– Automatically generated test cases?
● The ability to treat the program as an oracle is also very powerful– We can get new data by running the program– This can be combined with reinforcement learning to accomplish hard tasks– We saw this before when discussing test suites!
![Page 131: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/131.jpg)
Example: Memory Safety Bugs
![Page 132: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/132.jpg)
132
Example: Finding memory safety bugs
● Memory safety bugs are one of the most common sources ofsecurity vulnerabilities
![Page 133: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/133.jpg)
133
Example: Finding memory safety bugs
● Memory safety bugs are one of the most common sources ofsecurity vulnerabilities
● Effects may only be visible long after invalid behavior– This complicates comprehension & debugging
![Page 134: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/134.jpg)
134
Example: Finding memory safety bugs
● Memory safety bugs are one of the most common sources ofsecurity vulnerabilities
● Effects may only be visible long after invalid behavior– This complicates comprehension & debugging
● Two main types of issues:– Spatial – Out of bounds stack/heap/global accesses– Temporal – Use after free
![Page 135: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/135.jpg)
135
Example: Finding memory safety bugs
● Memory safety bugs are one of the most common sources ofsecurity vulnerabilities
● Effects may only be visible long after invalid behavior– This complicates comprehension & debugging
● Two main types of issues:– Spatial – Out of bounds stack/heap/global accesses– Temporal – Use after free
● We would like to automatically identify & provide assistancewith high precision and low overhead– Suitable for testing & sometimes maybe deployment!
![Page 136: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/136.jpg)
136
Example: Finding memory safety bugs
● Most common approach – track which regions of memory are valid– During execution!– Updated when new memory is allocated– Checked when pointers are accessed– With low overhead
![Page 137: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/137.jpg)
137
Example: Finding memory safety bugs
● Most common approach – track which regions of memory are valid– During execution!– Updated when new memory is allocated– Checked when pointers are accessed– With low overhead
● Common implementations– Valgrind – DBI based– AddressSanitizer – static instrumentation based
![Page 138: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/138.jpg)
138
Example: Finding memory safety bugs
● Most common approach – track which regions of memory are valid– During execution!– Updated when new memory is allocated– Checked when pointers are accessed– With low overhead
● Common implementations– Valgrind – DBI based– AddressSanitizer – static instrumentation based
Note, the implementation style affects which bugs can be recognized!
Why?
![Page 139: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/139.jpg)
139
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free
![Page 140: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/140.jpg)
140
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
![Page 141: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/141.jpg)
141
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks
ptr = malloc(sizeof(MyStruct));
ptr
![Page 142: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/142.jpg)
142
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed
free(ptr);
ptr
![Page 143: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/143.jpg)
143
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
void foo() { int buffer[5]; ...}
buffer[0]
buffer[6]
![Page 144: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/144.jpg)
144
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
![Page 145: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/145.jpg)
145
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
*address = ...;
instrumentation ?
![Page 146: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/146.jpg)
146
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
*address = ...; If (IsPoisoned(address, size)) { ReportError(address, size, isWrite);}*address = ...
instrumentation
![Page 147: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/147.jpg)
147
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
● The tricky part is tracking & efficiently checking redzones.
![Page 148: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/148.jpg)
148
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
● The tricky part is tracking & efficiently checking redzones.– Instrumenting every memory access is costly!
![Page 149: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/149.jpg)
149
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
● The tricky part is tracking & efficiently checking redzones.– Instrumenting every memory access is costly!– We must track all memory ... inside that same memory!
![Page 150: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/150.jpg)
150
AddressSanitizer
● Need to track which memory is valid & check efficiently...
● Big Picture:– Replace calls to malloc & free– Poison memory: (create red zones)
1) around malloced chunks2) when it is freed3) around buffers and local variables
– Access of poisoned memory causes an error
● The tricky part is tracking & efficiently checking redzones.– Instrumenting every memory access is costly!– We must track all memory ... inside that same memory!
This kind of issue is common in dynamic analyses.
![Page 151: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/151.jpg)
151
AddressSanitizer – Shadow Memory
Application Memory
Need to know whether any byte of application memory is poisoned.
![Page 152: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/152.jpg)
152
AddressSanitizer – Shadow Memory
Application Memory Shadow Memory
● We maintain 2 views on memory
![Page 153: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/153.jpg)
153
AddressSanitizer – Shadow Memory
Application Memory Shadow Memory
● We maintain 2 views on memory
Shadow memory of theshadow memory!
![Page 154: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/154.jpg)
154
AddressSanitizer – Shadow Memory
Application Memory Shadow Memory
● We maintain 2 views on memory
● Shadow Memory is pervasive in dynamic analysis– Can be thought of as a map containing analysis data
![Page 155: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/155.jpg)
155
AddressSanitizer – Shadow Memory
Application Memory Shadow Memory
● We maintain 2 views on memory
● Shadow Memory is pervasive in dynamic analysis– Can be thought of as a map containing analysis data– For every bit/byte/word/chunk/allocation/page,
maintain information in a compact table
![Page 156: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/156.jpg)
156
AddressSanitizer – Shadow Memory
Application Memory Shadow Memory
● We maintain 2 views on memory
● Shadow Memory is pervasive in dynamic analysis– Can be thought of as a map containing analysis data– For every bit/byte/word/chunk/allocation/page,
maintain information in a compact table
Where have you encountered this before?
![Page 157: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/157.jpg)
157
AddressSanitizer – Shadow Memory
Application Memory Shadow Memory
● We maintain 2 views on memory
● Shadow Memory is pervasive in dynamic analysis– Can be thought of as a map containing analysis data– For every bit/byte/word/chunk/allocation/page,
maintain information in a compact table– Common in runtime support, e.g. page tables
![Page 158: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/158.jpg)
158
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
Encoding & abstraction efficiency strategies
![Page 159: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/159.jpg)
159
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)
![Page 160: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/160.jpg)
160
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable
k
![Page 161: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/161.jpg)
161
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states
![Page 162: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/162.jpg)
162
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
4
6
7
0
5
3
-1
1
2
![Page 163: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/163.jpg)
163
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
All good = 0 All bad = -1 Partly good = # good
4
6
7
0
5
3
-1
1
2
![Page 164: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/164.jpg)
164
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
● What does accessing shadow memory for an address look like? (N=8)
![Page 165: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/165.jpg)
165
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
● What does accessing shadow memory for an address look like? (N=8)– Preallocate a large table
![Page 166: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/166.jpg)
166
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
● What does accessing shadow memory for an address look like? (N=8)– Preallocate a large table– Shadow = (address >> 3) + Offset
![Page 167: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/167.jpg)
167
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
● What does accessing shadow memory for an address look like? (N=8)– Preallocate a large table– Shadow = (address >> 3) + Offset– With PIE, Shadow = (address >> 3)
![Page 168: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/168.jpg)
168
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
● What does accessing shadow memory for an address look like? (N=8)– Preallocate a large table– Shadow = (address >> 3) + Offset– With PIE, Shadow = (address >> 3)
if (*(address>>3)) { ReportError(...);}*address = ...
![Page 169: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/169.jpg)
169
AddressSanitizer – Shadow Memory
● Designing efficient analyses (& shadow memory) often requires a careful domain insight
● NOTE: Heap allocated regions are N byte aligned (N usually 8)– In an N byte region, only the first k may be addressable– Every N bytes has only N+1 possible states– Map every N bytes to 1 shadow byte encoding state as a number
● What does accessing shadow memory for an address look like? (N=8)– Preallocate a large table– Shadow = (address >> 3) + Offset– With PIE, Shadow = (address >> 3)
if (*(address>>3)) { ReportError(...);}*address = ...
Now you can also see the reason for the numerical encoding....
![Page 170: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/170.jpg)
170
AddressSanitizer – Shadow Memory
shadow = address >> 3state = *shadowif (state != 0 && (state < (address & 7) + size)) { ReportError(...);}*address = ...
● Handling accesses of size < N (N=8)
![Page 171: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/171.jpg)
171
AddressSanitizer – Shadow Memory
shadow = address >> 3state = *shadowif (state != 0 && (state < (address & 7) + size)) { ReportError(...);}*address = ...
● Handling accesses of size < N (N=8)
Careful construction of states can make runtime checks efficient
![Page 172: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/172.jpg)
172
AddressSanitizer - Evaluating
● In dynamic analyses, we care about both overheads & result quality
![Page 173: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/173.jpg)
173
AddressSanitizer - Evaluating
● In dynamic analyses, we care about both overheads & result quality
● Overheads– Need to determine what resources are being consumed
![Page 174: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/174.jpg)
174
AddressSanitizer - Evaluating
● In dynamic analyses, we care about both overheads & result quality
● Overheads– Need to determine what resources are being consumed– Memory –
Shadow memory capacity is cheap, but accessed shadows matter
![Page 175: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/175.jpg)
175
AddressSanitizer - Evaluating
● In dynamic analyses, we care about both overheads & result quality
● Overheads– Need to determine what resources are being consumed– Memory –
Shadow memory capacity is cheap, but accessed shadows matter– Running time –
Can effectively be free for I/O bound projectsUp to 25x overheads on some benchmarks
![Page 176: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/176.jpg)
176
AddressSanitizer - Evaluating
● In dynamic analyses, we care about both overheads & result quality
● Overheads– Need to determine what resources are being consumed– Memory –
Shadow memory capacity is cheap, but accessed shadows matter– Running time –
Can effectively be free for I/O bound projectsUp to 25x overheads on some benchmarks
● Quality– Precision & recall matter
Where will it miss bugs?Where will it raise false alarms?
![Page 177: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/177.jpg)
177
AddressSanitizer - Evaluating
● False negatives– Computed pointers that are accidentally in bounds
![Page 178: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/178.jpg)
178
AddressSanitizer - Evaluating
● False negatives– Computed pointers that are accidentally in bounds– Unaligned accesses that are partially out of bounds
![Page 179: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/179.jpg)
179
AddressSanitizer - Evaluating
● False negatives– Computed pointers that are accidentally in bounds– Unaligned accesses that are partially out of bounds– Use after frees with significant churn
![Page 180: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/180.jpg)
Example: Comparing Executions
![Page 181: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/181.jpg)
181
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
![Page 182: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/182.jpg)
182
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison
![Page 183: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/183.jpg)
183
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison– Debugging regressions – old vs new
![Page 184: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/184.jpg)
184
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison– Debugging regressions – old vs new– Validating patches – old vs new
![Page 185: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/185.jpg)
185
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison– Debugging regressions – old vs new– Validating patches – old vs new– Understanding automated repair – old vs new
![Page 186: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/186.jpg)
186
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison– Debugging regressions – old vs new– Validating patches – old vs new– Understanding automated repair – old vs new– Debugging with concurrency – buggy vs nonbuggy schedules
![Page 187: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/187.jpg)
187
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison– Debugging regressions – old vs new– Validating patches – old vs new– Understanding automated repair – old vs new– Debugging with concurrency – buggy vs nonbuggy schedules– Malware analysis – malicious vs nonmalicious run
![Page 188: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/188.jpg)
188
Why compare traces or executions?
● Understanding the differences between two executions(& how some differences cause others)can help explain program behavior
● Several tasks could be made simpler by trace comparison– Debugging regressions – old vs new– Validating patches – old vs new– Understanding automated repair – old vs new– Debugging with concurrency – buggy vs nonbuggy schedules– Malware analysis – malicious vs nonmalicious run– Reverse engineering – desired behavior vs undesirable
![Page 189: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/189.jpg)
189
How it might look
Correct Buggy
![Page 190: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/190.jpg)
190
How it might look
x was 5 instead of 3
Correct Buggy
![Page 191: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/191.jpg)
191
How it might look
x was 5 instead of 3
So y was 2 instead of 7
Correct Buggy
![Page 192: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/192.jpg)
192
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branch
Correct Buggy
![Page 193: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/193.jpg)
193
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
Correct Buggy
![Page 194: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/194.jpg)
194
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy
![Page 195: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/195.jpg)
195
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy What do we need?● locations● state● flow● causation
![Page 196: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/196.jpg)
196
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy What do we need?● locations● state● flow● causation
![Page 197: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/197.jpg)
197
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy What do we need?● locations● state● flow● causation
![Page 198: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/198.jpg)
198
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy What do we need?● locations● state● flow● causation
![Page 199: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/199.jpg)
199
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy What do we need?● locations● state● flow● causation
![Page 200: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/200.jpg)
200
How it might look
x was 5 instead of 3
So y was 2 instead of 7
So the TRUE branch executedinstead of the FALSE branchSo the update of z was skipped
So the incorrect value of z was printed
Correct Buggy What do we need?● locations● state● flow● causation
We can construct this backwardfrom a point of failure/difference
![Page 201: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/201.jpg)
201
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
![Page 202: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/202.jpg)
202
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
![Page 203: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/203.jpg)
203
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
baz()
foo()
![Page 204: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/204.jpg)
204
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
baz()
foo()
foo()
baz()
foo()
![Page 205: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/205.jpg)
205
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
baz()
foo()
foo()
baz()
foo()
What is marked as different?
![Page 206: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/206.jpg)
206
foo()
foo()
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
baz()
baz()
foo()
What is marked as different?
![Page 207: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/207.jpg)
207
baz()
baz()
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
foo()
foo()
foo()
What is marked as different?
What is intuitively different?
![Page 208: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/208.jpg)
208
baz()
baz()
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
foo()
foo()
foo()
What is marked as different?
What is intuitively different?
![Page 209: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/209.jpg)
209
baz()
baz()
So why not just...
● Traces can be viewed as sequences....– Why not just do LCS based sequence alignment?
def foo(int c): if c: while bar(): ...
foo(...)baz(...)foo(...)
foo()
foo()
foo()
foo()
What is marked as different?
What is intuitively different?
Execution comparison mustaccount for what a program
means and does!
![Page 210: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/210.jpg)
210
The big picture
● Fundamentally, execution comparison needs to account for
![Page 211: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/211.jpg)
211
The big picture
● Fundamentally, execution comparison needs to account for– Structure – How is a program organized?
![Page 212: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/212.jpg)
212
The big picture
● Fundamentally, execution comparison needs to account for– Structure – How is a program organized?– Value – What are the values in the different executions?
![Page 213: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/213.jpg)
213
The big picture
● Fundamentally, execution comparison needs to account for– Structure – How is a program organized?– Value – What are the values in the different executions?– Semantics – How is the meaning of the program affected by the differences?
![Page 214: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/214.jpg)
214
The big picture
● Fundamentally, execution comparison needs to account for– Structure– Value– Semantics
● We can attack these through
![Page 215: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/215.jpg)
215
The big picture
● Fundamentally, execution comparison needs to account for– Structure– Value– Semantics
● We can attack these through– Temporal alignment
● What parts of the trace correspond?
![Page 216: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/216.jpg)
216
The big picture
● Fundamentally, execution comparison needs to account for– Structure– Value– Semantics
● We can attack these through– Temporal alignment
● What parts of the trace correspond?
– Spatial alignment● What variables & values correspond across traces?
![Page 217: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/217.jpg)
217
The big picture
● Fundamentally, execution comparison needs to account for– Structure– Value– Semantics
● We can attack these through– Temporal alignment
● What parts of the trace correspond?
– Spatial alignment● What variables & values correspond across traces?
– Slicing● How do differences transitively flow through a program?
![Page 218: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/218.jpg)
218
The big picture
● Fundamentally, execution comparison needs to account for– Structure– Value– Semantics
● We can attack these through– Temporal alignment
● What parts of the trace correspond?
– Spatial alignment● What variables & values correspond across traces?
– Slicing● How do differences transitively flow through a program?
– Causality testing● Which differences actually induce difference behavior?
![Page 219: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/219.jpg)
219
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
i2
i1
?
![Page 220: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/220.jpg)
220
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
i2
i1
?
![Page 221: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/221.jpg)
221
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()
i2
i1
?
![Page 222: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/222.jpg)
222
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()
i2
i1
?
![Page 223: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/223.jpg)
223
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()
i2
i1
?
![Page 224: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/224.jpg)
224
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()
i2
i1
?
![Page 225: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/225.jpg)
225
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()
i2
i1
?
![Page 226: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/226.jpg)
226
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()
i2
i1
?
![Page 227: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/227.jpg)
227
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()
i2
i1
?
![Page 228: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/228.jpg)
228
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()
i2
i1
?
![Page 229: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/229.jpg)
229
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()Position along a path can be maintained via a counter
i2
i1
?
![Page 230: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/230.jpg)
230
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
foo()Position along a path can be maintained via a counter
Only need to increment along1) back edges2) function callsi
2
i1
?
![Page 231: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/231.jpg)
231
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
i2
i1
?
![Page 232: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/232.jpg)
232
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
i2
i1
?
![Page 233: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/233.jpg)
233
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
The position in the DAG relates the paths
i2
i1
?
![Page 234: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/234.jpg)
234
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
i2
i1
?
![Page 235: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/235.jpg)
235
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
We can unwind the loop to make it logically acyclic
i2
i1
?
![Page 236: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/236.jpg)
236
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
i2
i1
?
![Page 237: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/237.jpg)
237
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
i2
i1
?
![Page 238: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/238.jpg)
238
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
...
i2
i1
?
![Page 239: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/239.jpg)
239
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
...
These are different iterations of one loop.A counter for each active loop suffices (mostly).
i2
i1
?
![Page 240: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/240.jpg)
240
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
How can we extend the acyclic case?
...
1 counter per active loop+ the call stack disambiguates!
i2
i1
?
![Page 241: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/241.jpg)
241
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
– Can we efficiently represent this?
i2
i1
?
![Page 242: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/242.jpg)
242
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
– Can we efficiently represent this?
Call stack/context Iteration stack Instruction IDi2
i1
?
![Page 243: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/243.jpg)
243
Temporal Alignment
● Given i1 in T1 and i2 in T2,– when should we say that they correspond? [Xin, PLDI 2008][Sumner, ASE 2013]
– how can we compute such relations?
● In the simplest case T1 and T2 may follow the same path[Mellor-Crummey, ASPLOS 1989]
● Suppose that we know the programs are acyclic?
● Now consider the case where cycles can occur... [Sumner, ASE 2013]
– Can we efficiently represent this?
Call stack/context Iteration stack Instruction ID
Can be encoded/inferred Can be inferred
i2
i1
?
![Page 244: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/244.jpg)
244
Spatial Alignment
● We must also ask what it means to compare program state across executions
![Page 245: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/245.jpg)
245
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?
3 != 5
![Page 246: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/246.jpg)
246
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?
0xdeadbeef in T1 = 0xcafef00d in T2?
![Page 247: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/247.jpg)
247
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?
0xdeadbeef in T1 = 0xcafef00d in T2?
If you allocated other stuff in only one run,this can be true even without ASLR!
![Page 248: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/248.jpg)
248
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
![Page 249: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/249.jpg)
249
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
![Page 250: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/250.jpg)
250
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
![Page 251: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/251.jpg)
251
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?
![Page 252: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/252.jpg)
252
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?1) list.append(value++)2) if c:3) list.append(value++)4) list.append(value++)
![Page 253: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/253.jpg)
253
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?1) list.append(value++)2) if c:3) list.append(value++)4) list.append(value++)
1
1
![Page 254: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/254.jpg)
254
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?1) list.append(value++)2) if c:3) list.append(value++)4) list.append(value++)
1
1 3
![Page 255: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/255.jpg)
255
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?1) list.append(value++)2) if c:3) list.append(value++)4) list.append(value++)
1
1 3
4
4
![Page 256: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/256.jpg)
256
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?1) list.append(value++)2) if c:3) list.append(value++)4) list.append(value++)
1
1 3
4
4
![Page 257: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/257.jpg)
257
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
A B C
A BT1
T2
What are the differences?1) list.append(value++)2) if c:3) list.append(value++)4) list.append(value++)
1
1 3
4
4
![Page 258: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/258.jpg)
258
Spatial Alignment
● We must also ask what it means to compare program state across executions– How can we compare two integers X and Y?– How can we compare two pointers A and B?– How can we compare allocated regions on the heap?
Should they even be compared?!?
● In practice, comparing state across executions requires comparing memory graphs– We need a way to identify corresponding nodes (state elements)
● Again, the semantics of the program dictate the solution– Identify heap allocations by the aligned time of allocation
A B C
A BT1
T2
1
1 3
4
4
![Page 259: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/259.jpg)
259
Dual Slicing
● Now we can– Identify corresponding times across executions– Identify & compare corresponding state at those times
![Page 260: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/260.jpg)
260
Dual Slicing
● Now we can– Identify corresponding times across executions– Identify & compare corresponding state at those times
● We can use these to enhance dynamic slicing by being aware of differences! (called dual slicing)
![Page 261: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/261.jpg)
261
Dual Slicing
● Now we can– Identify corresponding times across executions– Identify & compare corresponding state at those times
● We can use these to enhance dynamic slicing by being aware of differences! (called dual slicing)– Based on classic dynamic slicing– Include transitive dependencies that differ or exist in only 1 execution
![Page 262: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/262.jpg)
262
Dual Slicing
● Now we can– Identify corresponding times across executions– Identify & compare corresponding state at those times
● We can use these to enhance dynamic slicing by being aware of differences! (called dual slicing)– Based on classic dynamic slicing– Include transitive dependencies that differ or exist in only 1 execution
1)x 1←
2)y 1←
3)print(x+y)
1)x 0←
2)y 1←
3)print(x+y)3
2
1
3
2
1
0
1 1
1
![Page 263: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/263.jpg)
263
Dual Slicing
● Now we can– Identify corresponding times across executions– Identify & compare corresponding state at those times
● We can use these to enhance dynamic slicing by being aware of differences! (called dual slicing)– Based on classic dynamic slicing– Include transitive dependencies that differ or exist in only 1 execution
1)x 1←
2)y 1←
3)print(x+y)
1)x 0←
2)y 1←
3)print(x+y)3
2
1
3
2
1
0 1 1
1
![Page 264: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/264.jpg)
264
Dual Slicing
● Now we can– Identify corresponding times across executions– Identify & compare corresponding state at those times
● We can use these to enhance dynamic slicing by being aware of differences! (called dual slicing)– Based on classic dynamic slicing– Include transitive dependencies that differ or exist in only 1 execution
1)x 1←
2)y 1←
3)print(x+y)
1)x 0←
2)y 1←
3)print(x+y)3
2
1
3
2
1
3
1
0
1 1
10 1
![Page 265: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/265.jpg)
265
Dual Slicing
● The differences in dependencies capture multiple kinds of information
![Page 266: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/266.jpg)
266
Dual Slicing
● The differences in dependencies capture multiple kinds of information– Value-only differences
1
0 2
1
1
3
2
3
![Page 267: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/267.jpg)
267
Dual Slicing
● The differences in dependencies capture multiple kinds of information– Value-only differences– Provenance/Source differences
1
0 2
1
13
2
3
![Page 268: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/268.jpg)
268
Dual Slicing
● The differences in dependencies capture multiple kinds of information– Value-only differences– Provenance/Source differences– Missing/Extra behavior
4
1
0
4
2
1
13
![Page 269: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/269.jpg)
269
Dual Slicing
● The differences in dependencies capture multiple kinds of information– Value-only differences– Provenance/Source differences– Missing/Extra behavior
● Recall: Dynamic slicing could not handle execution omission,but dual slicing can!
![Page 270: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/270.jpg)
270
Dual Slicing
● The differences in dependencies capture multiple kinds of information– Value-only differences– Provenance/Source differences– Missing/Extra behavior
● Recall: Dynamic slicing could not handle execution omission,but dual slicing can!
● Dual slices can be effective for concurrent debugging & exploit analysis[Weeratunge, ISSTA 2010][Johnson, S&P 2011]
![Page 271: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/271.jpg)
271
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
Correct Buggy
![Page 272: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/272.jpg)
272
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
![Page 273: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/273.jpg)
273
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
![Page 274: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/274.jpg)
274
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
Correct
x = 10y = -1Truez = 0
“0”
![Page 275: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/275.jpg)
275
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 276: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/276.jpg)
276
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 277: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/277.jpg)
277
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 278: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/278.jpg)
278
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 279: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/279.jpg)
279
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 280: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/280.jpg)
280
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 281: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/281.jpg)
281
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
BuggyDual slicing captures differences, not causes.
What does that mean here?
![Page 282: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/282.jpg)
282
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
![Page 283: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/283.jpg)
283
Adding Causation
● Now we can produce explanations exactly like our example!– Can answer “Why” and “Why not” questions about behavior & differences
[Ko, ICSE 2008]
– But they may still contain extra information/noise...
1) x = ...2) y = ...3) if x + y > 0:4) z = 05) else:6) z = 17) print(z)
11
33
77
4
Correct Buggy
6
2 2
Correct
x = 10 x = 0y = -1 y = -2True Falsez = 0
z = 1“0” “1”
Buggy
The cost of these extra edges is high in practice!All transitive dependencies...
![Page 284: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/284.jpg)
284
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research
![Page 285: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/285.jpg)
285
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
1
3
7
4
E1
2
1
3
7
4
E2
![Page 286: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/286.jpg)
286
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
● There are several things we could consider– In general, simpler explanations are preferred 1
3
7
4
E1
2
1
3
7
4
E2
![Page 287: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/287.jpg)
287
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
● There are several things we could consider– In general, simpler explanations are preferred– Minimize the “# steps”?
1
3
7
4
E1
2
1
3
7
4
E2
![Page 288: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/288.jpg)
288
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
● There are several things we could consider– In general, simpler explanations are preferred– Minimize the “# steps”?– Minimize the “# dependencies”?
1
3
7
4
E1
2
1
3
7
4
E2
![Page 289: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/289.jpg)
289
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
● There are several things we could consider– In general, simpler explanations are preferred– Minimize the “# steps”?– Minimize the “# dependencies”?
1
3
7
4
E1
2
1
3
7
4
E2
What big challenges do you see withthese 2 approaches?
![Page 290: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/290.jpg)
290
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
● There are several things we could consider– In general, simpler explanations are preferred– Minimize the “# steps”?– Minimize the “# dependencies”?– Minimize the “# local dependencies”?
1
3
7
4
E1
2
1
3
7
4
E2
![Page 291: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/291.jpg)
291
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
● There are several things we could consider– In general, simpler explanations are preferred– Minimize the “# steps”?– Minimize the “# dependencies”?– Minimize the “# local dependencies”?
1
3
7
4
E1
2
1
3
7
4
E2
argmins∈sd i|sd|
∧E1[ sd (E2)i ]→sd (E2)i+1
∧E2[ sd (E1)i ]→sd (E1)i+1
![Page 292: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/292.jpg)
292
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
● There are several things we could consider– In general, simpler explanations are preferred– Minimize the “# steps”?– Minimize the “# dependencies”?– Minimize the “# local dependencies”?
1
3
7
4
E1
2
1
3
7
4
E2
argmins∈sd i|sd|
∧E1[ sd (E2)i ]→sd (E2)i+1
∧E2[ sd (E1)i ]→sd (E1)i+1
![Page 293: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/293.jpg)
293
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
● There are several things we could consider– In general, simpler explanations are preferred– Minimize the “# steps”?– Minimize the “# dependencies”?– Minimize the “# local dependencies”?
1
3
7
4
E1
2
1
3
7
4
E2
![Page 294: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/294.jpg)
294
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
● There are several things we could consider– In general, simpler explanations are preferred– Minimize the “# steps”?– Minimize the “# dependencies”?– Minimize the “# local dependencies”?
1
7
E1
2
3
4
1
3
7
4
E2
![Page 295: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/295.jpg)
295
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
● There are several things we could consider– In general, simpler explanations are preferred– Minimize the “# steps”?– Minimize the “# dependencies”?– Minimize the “# local dependencies”?
1
7
E1
2
3
4
1
3
7
4
E2
![Page 296: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/296.jpg)
296
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
● There are several things we could consider– In general, simpler explanations are preferred– Minimize the “# steps”?– Minimize the “# dependencies”?– Minimize the “# local dependencies”?
1
3
7
4
E1
1
3
7
4
E2
![Page 297: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/297.jpg)
297
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
● There are several things we could consider– In general, simpler explanations are preferred– Minimize the “# steps”?– Minimize the “# dependencies”?– Minimize the “# local dependencies”?
● There are currently unknown trade offs betweentractability, intuitiveness, and correctness
1
3
7
4
E1
2
1
3
7
4
E2
![Page 298: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/298.jpg)
298
Adding Causation
● So what would we want an explanation to contain?– This is an area with unsolved problems & open research– What does it mean for one explanation to be better than another?
● There are several things we could consider– In general, simpler explanations are preferred– Minimize the “# steps”?– Minimize the “# dependencies”?– Minimize the “# local dependencies”?
● There are currently unknown trade offs betweentractability, intuitiveness, and correctness
1
3
7
4
E1
2
1
3
7
4
E2
Even local blame is actually challenging
![Page 299: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/299.jpg)
299
Adding Causation
● Causation is often framed via “alternate worlds” & “what if” questions...– We can answer these causality questions by running experiments!
![Page 300: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/300.jpg)
What Should We Blame?
?
Trial
![Page 301: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/301.jpg)
What Should We Blame?
?
Trial
![Page 302: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/302.jpg)
What Should We Blame?
x = 5y = 4z = 3
x = 5y = 4z = 1
y = 4
?
Trial
![Page 303: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/303.jpg)
What Should We Blame?
x = 5y = 4z = 3
x = 5y = 4z = 1
y = 4
?
Trial
![Page 304: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/304.jpg)
What Should We Blame?
x = 5y = 4z = 3
x = 5y = 4z = 1
y = 4
?
Trial
What does this patched run even mean?
![Page 305: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/305.jpg)
x ← 1y ← 3z ← 6if False:
else: y ← 4print(4)
Example – Altered Meaning
1)x ← input()2)y ← input()3)z ← input()4)if y+z > 10:5) y ← 56)else: y ← y+17)print(y)
x ← 0y ← 7z ← 3if False:
else: y ← 8print(8)
CorrectBuggy
![Page 306: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/306.jpg)
Example – Altered Meaning
What should we blame here?
1)x ← input()2)y ← input()3)z ← input()4)if y+z > 10:5) y ← 56)else: y ← y+17)print(y)
x ← 0y ← 7z ← 3if False:
else: y ← 8print(8)
x ← 1y ← 3z ← 6if False:
else: y ← 4print(4)
CorrectBuggy
![Page 307: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/307.jpg)
x ← 0y ← 7z ← 3if False:
else: y ← 8print(8)
Example – Altered Meaning
1)x ← input()2)y ← input()3)z ← input()4)if y+z > 10:5) y ← 56)else: y ← y+17)print(y)
x ← 1y ← 3z ← 6if False:
else: y ← 4print(4)
CorrectBuggy Trial
![Page 308: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/308.jpg)
x ← 0y ← 7z ← 3if False:
else: y ← 8print(8)
x ← 0y ← 7z ← 3
Example – Altered Meaning
1)x ← input()2)y ← input()3)z ← input()4)if y+z > 10:5) y ← 56)else: y ← y+17)print(y)
x ← 1y ← 3z ← 6if False:
else: y ← 4print(4)
CorrectBuggy Trial
![Page 309: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/309.jpg)
x ← 0y ← 7z ← 3if False:
else: y ← 8print(8)
x ← 0y ← 7z ← 3if False:
else: y←8print(8)
Example – Altered Meaning
1)x ← input()2)y ← input()3)z ← input()4)if y+z > 10:5) y ← 56)else: y ← y+17)print(y)
x ← 1y ← 3z ← 6if False:
else: y ← 4print(4)
CorrectBuggy Trial
![Page 310: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/310.jpg)
x ← 0y ← 7z ← 3if False:
else: y ← 8print(8)
y ← 7
Example – Altered Meaning
1)x ← input()2)y ← input()3)z ← input()4)if y+z > 10:5) y ← 56)else: y ← y+17)print(y)
x ← 1y ← 3z ← 6if False:
else: y ← 4print(4)
CorrectBuggy Trial
![Page 311: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/311.jpg)
x ← 0y ← 7z ← 3if False:
else: y ← 8print(8)
x ← 1y ← 7z ← 6
Example – Altered Meaning
1)x ← input()2)y ← input()3)z ← input()4)if y+z > 10:5) y ← 56)else: y ← y+17)print(y)
x ← 1y ← 3z ← 6if False:
else: y ← 4print(4)
CorrectBuggy Trial
![Page 312: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/312.jpg)
x ← 0y ← 7z ← 3if False:
else: y ← 8print(8)
x ← 1y ← 7z ← 6if True: y ← 5
print(5)
Example – Altered Meaning
1)x ← input()2)y ← input()3)z ← input()4)if y+z > 10:5) y ← 56)else: y ← y+17)print(y)
x ← 1y ← 3z ← 6if False:
else: y ← 4print(4)
CorrectBuggy Trial
![Page 313: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/313.jpg)
x ← 1y ← 7z ← 6if True: y ← 5
print(5)
Example – Altered Meaning
1)x ← input()2)y ← input()3)z ← input()4)if y+z > 10:5) y ← 56)else: y ← y+17)print(y)
● New control flow unlike original runs
● Occurs in large portion of real bugs
x ← 0y ← 7z ← 3if False:
else: y ← 8print(8)
x ← 1y ← 3z ← 6if False:
else: y ← 4print(4)
CorrectBuggy Trial
![Page 314: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/314.jpg)
Dual Slicing
1)x ← input()2)y ← input()3)z ← input()4)if y+z > 10:5) y ← 56)else: y ← y+17)print(y)
76
2
2)y ← input()6)y ← y+17)print(y)
Extract
x ← 0y ← 7z ← 3if False:
else: y ← 8print(8)
x ← 1y ← 3z ← 6if False:
else: y ← 4print(4)
CorrectBuggy
![Page 315: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/315.jpg)
y ← 3y ← 4print(4)
Example – Extracted Meaning
y ← 7y ← 8print(8)
2)y ← input()6)y ← y+17)print(y)
CorrectBuggy Trial
![Page 316: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/316.jpg)
y ← 7 y ← 7y ← 4print(4)
Example – Extracted Meaning
y ← 7y ← 8print(8)
2)y ← input()6)y ← y+17)print(y)
CorrectBuggy Trial
![Page 317: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/317.jpg)
y ← 8print(8)
y ← 7 y ← 7y ← 4print(4)
Example – Extracted Meaning
y ← 7y ← 8print(8)
2)y ← input()6)y ← y+17)print(y)
Trial can now correctly blame y
CorrectBuggy Trial
![Page 318: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/318.jpg)
318
Adding Causation
● Causation is often framed via “alternate worlds” & “what if” questions...– We can answer these causality questions by running experiments!
● We perform these causality tests in both directions in order to collect symmetric information
![Page 319: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/319.jpg)
319
Adding Causation
● Causation is often framed via “alternate worlds” & “what if” questions...– We can answer these causality questions by running experiments!
● We perform these causality tests in both directions in order to collect symmetric information– How did the buggy run behave differently than the correct one?– How did the correct run behave differently than the buggy one?– These questions do not have the same answer!
![Page 320: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/320.jpg)
320
Adding Causation
● Causation is often framed via “alternate worlds” & “what if” questions...– We can answer these causality questions by running experiments!
● We perform these causality tests in both directions in order to collect symmetric information– How did the buggy run behave differently than the correct one?– How did the correct run behave differently than the buggy one?– These questions do not have the same answer!
● In practice, there are additional issues,and even defining causation in this context needs further research.
![Page 321: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/321.jpg)
321
Adding Causation
● Causation is often framed via “alternate worlds” & “what if” questions...– We can answer these causality questions by running experiments!
● We perform these causality tests in both directions in order to collect symmetric information– How did the buggy run behave differently than the correct one?– How did the correct run behave differently than the buggy one?– These questions do not have the same answer!
● In practice, there are additional issues,and even defining causation in this context needs further research.– Did we want to blame only y in the example?– Pruning blame on y is necessary in many real cases, can they be refined?– Can it be done without execution? With a stronger statistical basis?
![Page 322: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/322.jpg)
Summing Up
![Page 323: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/323.jpg)
323
Key Challenges
● Identifying the information you care about– Dynamic dependence? Valid memory? Just the execution outcome?
![Page 324: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/324.jpg)
324
Key Challenges
● Identifying the information you care about– Dynamic dependence? Valid memory? Just the execution outcome?
● Collecting that information efficiently– abstraction, encoding, compression, sampling, ...
![Page 325: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/325.jpg)
325
Key Challenges
● Identifying the information you care about– Dynamic dependence? Valid memory? Just the execution outcome?
● Collecting that information efficiently– abstraction, encoding, compression, sampling, ...
● Selecting which executions to analyze– Existing test suite? Always on runtime? Directed test generation?– How does underapproximation affect your conclusions?– Can you still achieve your objective in spite of it?
![Page 326: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/326.jpg)
326
Key Challenges
● Identifying the information you care about– Dynamic dependence? Valid memory? Just the execution outcome?
● Collecting that information efficiently– abstraction, encoding, compression, sampling, ...
● Selecting which executions to analyze– Existing test suite? Always on runtime? Directed test generation?– How does underapproximation affect your conclusions?– Can you still achieve your objective in spite of it?
● Doing some of the work ahead of time– What can you precompute to improve all of the above?
![Page 327: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/327.jpg)
327
Summary
● Analyze the actual/observed behaviors of a program
![Page 328: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/328.jpg)
328
Summary
● Analyze the actual/observed behaviors of a program
● Modify or use the program’s behavior to collect information
![Page 329: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/329.jpg)
329
Summary
● Analyze the actual/observed behaviors of a program
● Modify or use the program’s behavior to collect information
● Analyze the information online or offline
![Page 330: Dynamic Analysis - Simon Fraser University](https://reader030.vdocuments.us/reader030/viewer/2022012719/61b2215394b1091333177ade/html5/thumbnails/330.jpg)
330
Summary
● Analyze the actual/observed behaviors of a program
● Modify or use the program’s behavior to collect information
● Analyze the information online or offline
● The precise configuration must be tailored to the objectives & insights– Compiled vs DBI– Online vs Postmortem– Compressed, Encoded, Samples, ...– ...