The Impact of Programming Language Theory on Computer Security

Drew DeanComputer Science LaboratorySRI International

2

What I’m not Talking About

Cryptographic Protocol Verification See, e.g., Computer Security

Foundations Workshop Type Systems for Non-Interference

See, e.g., POPL

3

Much of security is:

“Program P exactly implements Specification S and no more.”

For this talk, we assume that the specification is correct

4

Security Tripos

No undefined usermode behavior

Proper system call use

Correctnesswrt criticalrequirements

5

Correctness wrt Security

No system that misses security checks can be secure Program Verification Architectural Support

Stack inspection Security Passing Style [WAF]

6

Program Verification

Obvious connections Lambda calculus, Curry-Howard Hoare Logic …

7

Architectural Support

Stack Inspection Access control based on endorsement of

code: answers “Who called me?” Designed to prevent untrusted code from

bypassing access controls, while allowing higher level code to assert that it knows what it’s doing

8

Stack Inspection Example

Applet wants to use the Helvetica font May require JVM to read a file Solution:

Font handling code checks arguments If successful, asserts privilege Attempts to read file

Which notes that font code (privileged) has asserted everything’s OK

9

Stack Inspection: Critique

Exposes call stack Tail call elimination painful Function inlining also painful Goodbye, Church-Rosser, goodbye!

10

Security Passing Style

Wallach, Appel, Felten, TOSEM 9/00 A la CPS, pass security context as an

extra (implicit) argument Restores tail call elimination and

function inlining Doesn’t restore Church-Rosser

11

Observation

SPS is in closer analogy to CPS than its authors say

Shivers: “Threads are paths through continuation space”

Continuations are the right semantic object to attach permissions to

Would a dependent type system work out?

12

Properly Using System Calls

If a program handles its own security, e.g., ftpd, it better use system calls correctly

Many programs don’t Wu-ftpd Sendmail …

13

How Can PLT help?

Joint work with David Wagner and Hao Chen, UC Berkeley

Given a program, morph control flow graph into an automaton that accepts language of system calls

14

IEEE S&P 2001

Take automaton, check runtime trace of system calls for anomaly detection

(Most of) Benefits of specification-based intrusion detection without needing the non-existent spec

15

Current Work

Take abstracted specification, throw it and library of security “best practices” (and known attacks) at (custom) model checker

But this requires understanding system calls

Usually the POSIX spec is reasonable But not for set*uid()

16

Understanding set*uid

Absolutely necessary for writing secure setuid Unix programs

Linux, FreeBSD, Solaris all subtly different Even if all POSIX compliant

Kernel code unreadable Reverse engineer formal model Will appear at USENIX Security 2002

17

No Undefined User-mode Behavior

Buffer overflows are still a problem in 2002

PL people think this is stupid It is

Like it or not, most of the world codes in C or unsafe C++

18

Not Just Buffer Overflows

Any corruption of program state can cause vulnerability Nearly science fiction attack based on a

C program double freeing a pointer

19

Observation

Memory comes in two colors Storage of variables Compiler/runtime support

20

Partition Property

“All variables only refer to memory locations that the compiler has mapped to program variables, not compiler/runtime support (e.g., return addresses, temporaries for evaluating expressions, memory management overhead, etc.)”

21

Partition Properties

Note that this is weaker than non-interference Values obviously depend on program

values Stronger than some forms of memory

& type safety Should be a theorem of modern (safe)

languages

22

Conclusions

This was a brief survey of a wide field

“and no more” is hard to implement

Hopefully, breaking it down helps

No undefined behavior

Proper system call use

Correctnesswrt criticalrequirements