Model Checking an Entire Linux Distribution for Security Violations

Work by Benjamin Schwarz, Hao Chen, David Wagner, Geoff Morrison, Jacob West, Jeremy Lin and Wei Tu

Jacob West,Security Research Group, Fortify Software

ACSAC 2005

2

Outline

Introduction MOPS Background Analyzing Red Hat 9

Tool performance Human performance Security properties

Vulnerability Examples TOCTTOU Standard File Descriptors Temporary Files strncpy()

Results

3

Introduction

Over 50% of security vulnerabilities caused by coding errors Automated detection possible

Rapidly expanding field Academic and commercial Feasible at large scale

4

MOPS(MOdelchecking Programs for Security properties)

Static analysis for security C programs Enforce temporal safety rules

5

Analyzing Red Hat 9:Overview

Tool performance Analysis of large code base feasible Compaction improves performance Reasonable resource requirements

Human performance Integration with existing build processes False positives Easy-to-review error traces Grouped error traces

Security properties Temporal safety properties Employable by other tools Iteratively refined for low false positives

6

Analyzing Red Hat 9:Tool Performance

Red Hat 9: 839 packages, 60 million TLOC 732 packages (87%) 107 failures caused by parse errors

73 packages contained C++ code 34 packages used unsupported C99 constructs

Compaction improves performance Only consider relevant operations

Reasonable resource requirements TOCTTOU takes about 10 hours on P4 1.5 GHZ / 1GB

7

Analyzing Red Hat 9:Human Performance

Integration with existing build processes Integrated with rpmbuild, make Interposed on gcc Analyze multiple packages easily

False positives Relatively low, permits human review

Easy-to-review error traces Navigate code quickly to verify error traces

Grouped error traces Understand multiple traces through representative samples

8

Analyzing Red Hat 9:Security Properties

Temporal safety properties Security properties expressed as Finite State Automata (FSA)

Pattern variables e.g. foo(x); bar(x); where x is the same

Iteratively refined to reduce false positives Employable by other tools Properties include

TOCTTOU: Time-of-check, to time-of-use race conditions Standard File Descriptors: Vulnerable uses of stdin, stdout and stderr Temporary Files: Insecure creation of temporary files strncpy(): Dangerous uses of strncpy()

9

Security Properties :TOCTTOU

Time-of-check to time-of-use race conditions occur when a program checks the access permission of an object and, if the check succeeds, makes a privileged system call on the object.

Example:if (access(pathname, R_OK) == 0) fd = open(pathname, O_RDONLY);

10

Security Properties :TOCTTOU

Checks: access(), stat(), etc. Uses: creat(), open(), unlink(), etc.

11

Vulnerability Example:TOCTTOU – binutils :: ar

exists = lstat (to, &s) == 0;/* Use rename only if TO is not a symboliclink and has only one hard link. */if (! exists || (!S_ISLNK (s.st_mode) && s.st_nlink == 1)){

ret = rename (from, to); if (ret == 0) { if (exists) { chmod (to, s.st_mode & 0777);

if (chown (to, s.st_uid, s.st_gid) >= 0) { chmod (to, s.st_mode & 07777); } ...

12

Security Properties: Standard File Descriptors

Since the kernel does require that stdin, stdout and stderr point to terminal devices, an attacker may cause a victim program open one of them to a sensitive file.

Example /* victim.c */fd = open("/etc/passwd", O_RDWR);if (!process_ok(argv[0])) perror(argv[0]);

/* attack.c */int main(void) { close(2); execl("victim", "foo:<pw>:0:1:Super-User-2:...", NULL);}

13

Security Properties: Standard File Descriptors

States correspond to the status of the three standard file descriptors and transitions occur on a "safe" open (/dev/null and /dev/tty).

open(…)

open(…)

14

Vulnerability Example: Standard File Descriptors - gnuchess

void BookBuilder(short depth, ...){ FILE *wfp,*rfp; if (depth == -1 && score == -1) { if ((rfp = fopen(BOOKRUN,"r+b")) != NULL) { printf("Opened existing book!\n"); } else { printf("Created new book!\n"); wfp = fopen(BOOKRUN,"w+b"); fclose(wfp); if ((rfp = fopen(BOOKRUN,"r+b")) == NULL) { printf("Could not create %s file\n", BOOKRUN); return; } ...

15

Security Properties:Temporary Files

Because many of the functions in the C standard library that create temporary files are insecure an adversary that is able to predict the filename can gain control of the file by precreating it.

Examplefd = mkstemp(action_file_name);...unlink(action_file_name);

16

Security Properties:Temporary Files

tmpnam(), tempnam(), mktemp() and tmpfile() are always unsafe

mkstemp() is safe if the generated filename is not used

17

Vulnerability Example:Temporary Files - yacc

static void open_files() { ... fd = mkstemp(action_file_name); if (fd < 0 || (action_file = fdopen(fd, "w")) == NULL){ ... open_error(action_file_name);

} }void open_error(char *filename) { warnx("f - cannot open \"%s\"", filename); done(2);}void done(int k) { ... if (action_file_name[0]) unlink(action_file_name);

18

Security Properties:strncpy()

First strncpy() encourages off-by-one errors if the programmer is not careful to compute the value of n precisely. Secondly, because the function does not automatically null-terminate a string in all cases it is a common mistake for a program to create unterminated strings during its execution.

Examplebuf[sizeof(buf)-1] = '\0';strncpy(buf, ..., sizeof(buf));

19

Security Properties:strncpy()

20

Vulnerability Example:strncpy() - xloadimage

newopt->info.dump.type = argv[++a];...dumpImage(dispimage, dump->info.dump.type,dump->info.dump.file, verbose);

void dumpImage(Image *image, char *type, char *filename, int verbose) { int a; char typename[32]; char *optptr; optptr = index(type, ','); if (optptr) { strncpy(typename, type, optptr - type); typename[optptr - type] = '\0'; ...}

21

Results

1358 strncpy() warnings; 53 audited; 11 real bugs* 200 human hours found 108 real bugs in 50 million lines of code Order of magnitude larger in scale than previous academic work Static analysis will be feasible and integral part of building systems

Property Reported Warnings

% FP Real Bugs

TOCTTOU 790 95% 41

Standard File Descriptors 56 61% 22

Insecure Temporary Files 108 69% 34

Total 954 90% 97

strncpy() 53/1358 79% 11/258*

Projected Total 2312 85% 355

Questions?Want to talk more about software security?

jwest@fortifysoftware.com