Software SecurityDynamic analysis and fuzz testing

Bhargava Shastry

Prof. Jean-Pierre SeifertSecurity in Telecommunications

TU Berlin

SoSe 2016

1 / 21

Introduction

What is this lecture about?How do we find bugs in programs?

... by executing an open-source program

... applies to binary code in principleLet’s recap

Last lecture, we discussed static analysis → Find bugs without programexecutionHow does it compare with Dynamic analysis? Pros and cons?

2 / 21

Motivation

Why do this?Find an input to the program that leads to undefined/insecurebehavior

Typically, we are looking for an input that crashes a programProgram crashes provide an entry point to craft exploits → Not allprogram crashes are exploitable

Technically, this is just another way to find bugs in programs

3 / 21

Outline

Fuzz testingBlack-box vs. White-box testingDynamic analysisLimitationsConclusion

4 / 21

Example 1Which input make this program crash?

1 #i n c l u d e <s t d i o . h>2 #i n c l u d e <u n i s t d . h>3 #i n c l u d e <s t d l i b . h>4 #i n c l u d e <s t r i n g . h>56 i n t main ( i n t argc , char ∗a rgv [ ] ) {78 char buf [ 2 0 ] ;9

10 w h i l e ( AFL LOOP (1000) ) {11 memset ( buf , 0 , 20) ;12 i f ( r ead (0 , buf , 19) < 0) {13 p e r r o r ( ” read ” ) ;14 r e t u r n 1 ;15 }1617 i f ( buf [ 0 ] != ’ p ’ )18 p r i n t f ( ” f i r s t l e t t e r i s not p\n” ) ;19 e l s e i f ( buf [ 1 ] != ’w ’ )20 p r i n t f ( ” second l e t t e r i s not w\n” ) ;21 e l s e i f ( buf [ 2 ] != ’ n ’ )22 p r i n t f ( ” t h i r d l e t t e r i s not n\n” ) ;23 e l s e24 a b o r t ( ) ;2526 p r i n t f ( ” buf c o n t a i n s %s\n” , buf ) ;27 }28 r e t u r n 0 ;29 }

5 / 21

1 #i n c l u d e <s t d i o . h>2 #i n c l u d e <u n i s t d . h>3 #i n c l u d e <s t d l i b . h>4 #i n c l u d e <s t r i n g . h>56 i n t main ( i n t argc , char ∗a rgv [ ] ) {78 char buf [ 2 0 ] ;9

10 w h i l e ( AFL LOOP (1000) ) {11 memset ( buf , 0 , 20) ;12 i f ( r ead (0 , buf , 19) < 0) {13 p e r r o r ( ” read ” ) ;14 r e t u r n 1 ;15 }1617 i f ( buf [ 0 ] != ’ p ’ )18 p r i n t f ( ” f i r s t l e t t e r i s not p\n” ) ;19 e l s e i f ( buf [ 1 ] != ’w ’ )20 p r i n t f ( ” second l e t t e r i s not w\n” ) ;21 e l s e i f ( buf [ 2 ] != ’ n ’ )22 p r i n t f ( ” t h i r d l e t t e r i s not n\n” ) ;23 e l s e24 a b o r t ( ) ;2526 p r i n t f ( ” buf c o n t a i n s %s\n” , buf ) ;27 }28 r e t u r n 0 ;29 }

pwn1

1Pwn is a leetspeak slang term derived from the verb own, as meaning to appropriate or to conquer to gainownership. - Wikipedia

6 / 21

Fuzz testing aka Fuzzing

Idea attributed to Prof.Barton Miller (Miller, Fredriksen, & So, 1990). . . i n the F a l l o f 1988 , t h e r e was a w i l d midwest thunde r s to rm . . . With the heavy

r a i n , t h e r e was n o i s e on the ( d i a l −up ) l i n e and t h a t n o i s e was i n t e r f e r i n gwi th my a b i l i t y to type s e n s i b l e commands to the s h e l l . − Pro f . M i l l e r

Feed random inputs to a program → No knowledge of programinternalsIdea bore very good initial results → crash or hang between 25-33%of the Unix utility programs

7 / 21

Let’s fuzz!

8 / 21

Black-box testing

Black-box testing treats the program under test as a black-box(opaque to tester)This means the testing framework feeds random inputs to the programHowever, random inputs alone are unlikely to trigger bugs

x = 1000/(input + 100000), how long will it take the fuzzer to feedinput = −100000?

Fuzzing, in its initial form, was conceived as a black-box testingtechnique

Ever since, it has evolved into a white box testing techniqueState of the art fuzzers like afl (Zalewski, n.d.) “learn” from programbehavior

9 / 21

White-box testing

While-box testing treats the PUT as a white-box (transparent totester)Program internals can be used to reduce input search space

Techniques: Program coverage guided fuzzing, Concolic execution etc.However, these need a model that abstracts program behavior forgenerating inputsUses program instrumentation to realize the model =⇒ Access toprogram binary or source codeInstrumentation may be static (compile-time) or dynamic (run-time)

10 / 21

Coverage guided fuzzing

Fuzz inputs based on program coverage informationFor instance, retain only those input mutations that lead to newcoverage

11 / 21

Example

Each colour indicates a different program path1 #i n c l u d e <s t d i o . h>2 #i n c l u d e <u n i s t d . h>3 #i n c l u d e <s t d l i b . h>4 #i n c l u d e <s t r i n g . h>56 i n t main ( i n t argc , char ∗a rgv [ ] ) {78 . . .9 . . .

1011 i f ( buf [ 0 ] != ’ p ’ )12 // buf [ 0 ] != ’ p ’13 p r i n t f ( ” f i r s t l e t t e r i s not p\n” ) ;14 e l s e i f ( buf [ 1 ] != ’w ’ )15 // buf [ 0 ] == ’ p ’ && buf [ 1 ] != ’w ’16 p r i n t f ( ” second l e t t e r i s not w\n” ) ;17 e l s e i f ( buf [ 2 ] != ’ n ’ )18 // buf [ 0 ] == ’ p ’ && buf [ 1 ] == ’w ’ && buf [ 2 ] != ’ n ’19 p r i n t f ( ” t h i r d l e t t e r i s not n\n” ) ;20 e l s e21 // buf [ 0 ] == ’ p ’ && buf [ 1 ] == ’w ’ && buf [ 2 ] == ’ n ’22 a b o r t ( ) ;2324 . . .25 . . .26 }

12 / 21

Instrumentation

But how does the fuzzer know which program path was taken?Answer: Instrument the programWhat does the instrumentation look like?

cur_location = random-number;shared_mem[cur_location ˆ prev_location]++;prev_location = cur_location >> 1;

Each branch point indexes a byte in global stateState captures number of times branch point takenComparing state changes per input, it is possible to filter seeminglyredundant inputs

13 / 21

Thought experiment

Is the state captured by the instrumentation good enough?In other words, can you think of an example where theinstrumentation might discard potentially crashing inputs

Only a heuristic → But, turns out to be very good in practice

14 / 21

Type of instrumentation

Program instrumentation can be done at compile-time or at run-timeCompile-time instrumentation incurs low performance overhead

Checking logic can be optimized before-handSlow down only because of additional instructions and associated I/OBut, hard to encode checking logic for dynamic language features ingeneral

Run-time instrumentations incurs relatively high overheadChecking logic needs to be inserted at program run timeTypically implemented using binary translation =⇒ very slowSince instrumentation occurs at run time, dynamism (e.g., indirectbranches) can be handled relatively easily

Rest of the lecture looks at dynamic analyses that use compile-timeinstrumentation

15 / 21

Dynamic analysis

Why bother if a good fuzzer gives us crashing inputs?A normal program crash happens only when there is a serious problemExample: Segmentation fault, assertion failure, stack canary failuresetc.But, it does not uncover latent faultsExample: Heap corruption, stack corruption but canary intact etc.

Dynamic analysis can help uncover latent faultsCan be used in conjunction with a fuzzer for proactive security audits

16 / 21

AddressSanitizer

AddressSanitizer (Serebryany, Bruening, Potapenko, & Vyukov, 2012)is an instrumentation framework for detecting multiple classes ofmemory corruption faults

Heap/stack buffer overflows, use-after-free bugsIdea is similar to coverage guided fuzzing i.e., insert bounds checks byinstrumenting the programEach word of addressable heap maps to a byte of shadow memoryUses custom memory allocation functions to set shadow byte duringallocationUses shadow lookups during load and store instructions to detectfaults e.g. loading from/storing to unaddressable memory location

17 / 21

ASan Instrumentation

Every 8-byte aligned word on x86 64 has 9 statesFirst n bytes 0 ≤ n ≤ 8 are addressable, rest not =⇒ 1 shadow bytefor 8 bytes of application memory

Instrumentation looks likechar *shadow = MemToShadow(addr);if (*shadow && *shadow <= func(addr))

ReportError(addr)...

18 / 21

Limitations

Fuzz testing and dynamic analysis covered in the lecture areprogram-input centricThis means, if a fuzzed input cannot uncover a program path, bugs inthat program path cannot be foundData on false negatives (bugs missed) by fuzzers is hard to come by

Exception: Recent research on bug insertion (Dolan-Gavitt et al., 2016)Preliminary results show that only 10% of inserted bugs found by aSotA fuzzer

Future: Use symbolic execution to maximize path coverage?Awaiting a public study of efficacy

19 / 21

Conclusions

Dynamic analysis attempts to uncover bugs during program executionWe discussed fuzz testing and program instrumentationDiscussed techniques and tools remain relevant for real-world securityaudits

Anecdotal evidence shows that over 90% of C/C++ bugs found usinga white-box fuzzer

Recent data on bugs missed by fuzzing tells us that there is room forimprovement

20 / 21

References I

Dolan-Gavitt, B., Hulin, P., Kirda, E., Leek, T., Mambretti, A., Robertson,W., . . . Whelan, R. (2016). Lava: Large-scale automatedvulnerability addition.

Miller, B. P., Fredriksen, L., & So, B. (1990). An empirical study of thereliability of unix utilities. Communications of the ACM, 33(12),32–44.

Serebryany, K., Bruening, D., Potapenko, A., & Vyukov, D. (2012).Addresssanitizer: A fast address sanity checker. In Presented as partof the 2012 usenix annual technical conference (usenix atc 12) (pp.309–318).

Zalewski, M. (n.d.). American fuzzy lop.http://lcamtuf.coredump.cx/afl/.

21 / 21