Binary Analysis Tools in Cyber TestingJohn Merrill and Arch Owen

2

PROBLEM STATEMENT

• Lack of source code in cyber engagements drives need for binary analysis tools, however...

• Binary-only analysis tools are limited and less developed than source analysis tools

• Draper has implemented Modular, Open software frameworks for static and dynamic analysis of binary code:– Dynamic analysis – VADER

– Static analysis – SHREDDER.

• These frameworks allow for shared development and use by a broad contractor community.

• Draper has leveraged the inherent modularity of these frameworks to implement new advanced binary analysis capabilities.

3

SURVEY OF EXAMPLE BINARY TOOLS

Tool Static Analysis

Dynamic Analysis

Usability/Adaptability

Auto

mat

ed

Stat

ic

Anal

ysis

Rev

erse

En

gine

erin

g

Fuzz

ing

Synb

olic

/ C

onco

licEx

ecut

ion

Mul

ti-Pl

atfo

rm

Use

rPlu

gins

Auto

mat

ed

Ope

n

Mod

ular

Join

t Use

CodeSonar ✓ ✓ ✓

IDA ✓ ✓ ✓

Ghidra ✓ ✓ ✓ ✓ ✓

Binary Ninja ✓ ✓

AFL ✓ ✓ ✓

Angr ✓ ✓ ✓ ✓ ✓

Mayhem ✓ ✓ ✓

Binary analysis tools exist BUT • Few are open AND modular• None are designed for joint use

4

QUALITATIVE COMPARISON OF BINARY TOOLS

Automated StaticAnalysis

Reverse Engineering Fuzzing Symbolic / ConcolicExecution

Ease of Deployment Memory Corruption Bugs Logic Bugs

We need a means to efficiently leverage all techniques AND facilitate more rapid binary tool development and maturation.

Very easy to employ, but lower performance

than other options

Potentially very effective but VERY manpower intensive

and dependent

A balance between ease-of-deployment

and performance

5

DRAPER’S APPROACH FOR RAPID TOOL DEVELOPMENT

• Rapid, robust tool development and maturation requires a 2-step approach

• Step 1: Develop a modular, open, automated framework– Automation: easy deployment

– Modular and Open: easy development of new features

• Step 2: Rapidly develop and integrate novel techniques

VADERDynamic Analysis

SHREDDERStatic Analysis

AutomatedStatic

Analysis

ReverseEngineering

Fuzzing Symbolic /ConcolicExecution

VADER Dynamic Analysis

7

WHAT IS FUZZING.....

ExecutableProgram

Unit Test #1 Data

Unit Test #2 Data

Unit Test #3 Data

Unit Test #4 Data

ExecutableProgram

Random Number Generator

Traditional Unit Testing Fuzzing Based Testing

Traditional software testing handles known edge cases. Fuzzing, in contrast, attempts to expose unknown edge cases through randomly generated tests.

8

TRADITIONAL FUZZING LIMITATIONS

Source code

Source code

Binarycode

Crash Output

AFL(American Fuzzy Lop)

Crash OutputLibFuzzer

Crash OutputRadamsa

Fuzzers have tended to focus on source code, and the monolithic nature of available options limits innovation and advanced features.

Source only

9

VADER MODULAR OPEN FUZZING FRAMEWORK

Source code

BinaryBinary

Seed Generation Module

Crash TriageTrace Collection Module

Genetic Algorithm Mutation ModuleGenetic Algorithm

Mutation ModuleGenetic Algorithm Mutation Module

Crash Output

Seed Generation Module

Comprehensive Crash Report

Seed Generation Module

Execution Engine Module

VADER open modularity leverages the best features of different fuzzers AND facilitates binary support, more rapid adaptation, and faster tool development.

10

VADER CURRENT FEATURES

• Modular support for many existing open-source fuzzers– AFL

– Radamsa

– Domato

• Automated sample generation and crash triage

• Draper developed capabilities leveraging open modularity:– Custom Black Box Taint Tracking subsystem

◦ Ability to traverse complex code paths

◦ Full support for binary code

– Real time, automated fuzzer switching - faster AND deeper code search

SHREDDER Static Analysis

12

LIMITATIONS TO REVERSE ENGINEERING

Firmware Extractor

Control Flow Graph

Data Flow Analysis

Binary Unpacker

Disassembly / IR

Reverse engineering is a time-consuming task involving numerous steps of information gathering. Many of these steps can be automated; however, the means to do so are limited.

13

SHREDDER – STATIC ANALYSIS FRAMEWORK

1 00 1 SHREDDER

UserInterface

Firmware Extractor

Binary Unpacker

Control Flow Graph

Data Flow Analysis

Disassembly / IR

User Jobs

The SHREDDER framework automates a large collection of common reverse engineering tasks and provides a simple interface for user extensions.

1 00 1

1 00 1

1 00 1

14

SHREDDER CURRENT FEATURES

• Integration of numerous static analysis jobs

• IDA / Binary Ninja plugin support

• Draper developed capabilities leveraging open modularity:– Library Identification capability

◦ Precise signature detection for library versions in binary

◦ Correlation with CVE database

15

PROBLEM STATEMENT

• Lack of source code in cyber engagements drives need for binary analysis tools, however...

• Binary-only analysis tools are limited and less developed than source analysis tools

• Draper has implemented Modular, Open software frameworks for static and dynamic analysis of binary code:– Dynamic analysis – VADER

– Static analysis – SHREDDER.

• These frameworks allow for shared development and use by a broad contractor community.

• Draper has leveraged the inherent modularity of these frameworks to implement new advanced binary analysis capabilities.