Cosc 4010

Sandboxing

Last lecture

• Last time, we covered chroot, which is a method to "sandbox" a problem.– Not full proof by any means.

• Many simple mistakes can cause it fail• vulnerabilities in the lib's and file that need to be

copied into the sandbox can allow "processes" to possible escape.

Untrusted programs

• We must use applications from outside our computer in order to run many things.– Web browsers for example.

• java applets, viewers and many helper applications

– P2P applications• many of which install other applications without telling you.

– Code from vendors may have bugs and problems.

• So we end up running untrusted/unknown/unreliable programs a lot

What to do?

• In the end, the goal would be to be able to run the code, but not comprise the entire system– IE vulnerabilities could not break into the O/S

– This is normally termed sandboxing.• chroot from the last lecture is one example of

sandboxing.

Quick Overview of O/S

• An application runs and requests resources from the kernel

• The kernel grants resources to the process if it has privileges

• example: It can open a file, it the UID can access the file.

• Of course this fails for security with the administrator/root users, since they normally have access to everything.

Review of O/S (2)

• Many "resources" are not privileged.– What other processes are running?

• Which is how viruses know if there is an anti-virus program running (and then attempt to turn it off)

– All programs have the same level of access to the system call interface or API

• they may be denied by the O/S for the resource, but they can still ask.

– Which could be problem. WHY???

Approaches for Sandboxing

• System Monitoring– Software Fault Isolation

• Modify compiled program to catch memory errors– Wrap/trap system calls

• Check and trap bad calls from the application to the O/S

• Virtual machines– An extra layer of security between the application and

the O/S• Formal methods

– The vendor provides proof that the code is correct– That proof is also verifiable by the O/S

• Difficult

Virtual Machines

• The VM works as an interface between the application and the O/S– The VM is trusted, so it will "always do the

right thing"– Security and Isolation

• Separate the program from the O/S, – no bad API calls to the O/S– I/O is interoperated through the VM– Also separates the program from the networking

Virtual Machine Implementation

• This implementation is used on top of linux or windows

Virtual Machine Implementation (2)

• IBM's implementation for the A/S 400

Isolation

• Data security– Each VM is managed independently

• Different OS, disks (files, registry), MAC address (IP address)• Data sharing is not possible between VM’s except through standard

channels (i.e. network, shared network file systems)

• Faults– Crashes are contained within a VM

• Security claim– No assumptions required for software inside a VM

• Can run multiple OS’s on a single machine and communicate between them via dedicated network interface (Groupware)

Mandatory I/O Interposition

• Two levels1. No direct Guest I/O

• All guest I/O operations are mediated by VM

2. VM uses host I/O• VM uses system calls to execute all I/O requests

Observable by the "host"

• “See without being seen” advantage– Very difficult within a computer, possible on a host– Attacker can cook logs, disable monitors on

compromised machine– Hard to disable if not visible because they’re on

• Observation points:– Networking (through vmnet) Physical memory– Disk I/O (read and write) Any other I/O

Advantages

• A VM can crash and fail, but the host doesn't.– Possibly easy reloading of the comprised system– With VMware, the VM can be "paused" and restarted

later, with shutting down the guest O/S

• The Host can act as a firewall, filter, and router.– The outside world doesn't have to see the vm

computer running at all.

Disadvantages

• VM for intel boxes are emulators– So there are performance hits

• especially for drive I/O

• IBM claims this is not an issue for there boxes.

Software Fault Isolation (SFI)

• Wahbe, Lucco, Anderson, Graham [SOSP’93]– Collusa Software (founded ’94, bought by Microsoft ‘96)– Support multiple applications in same address space

• This is an old hardware idea that was originally used to enforce isolation before hardware support for privilege protection domains and virtual memory

• Useful to achieve OS-level protection (or better) without overhead of OS context switch

• Current interest is in supporting applications that sometimes need to call other applications– Plug-ins for web browsers

SFI: How it works

• Partition memory space into segments

• Add instructions to binary executables– Check every jump and memory access– Target location must be in correct segment

• All locations in segment have same high-order bits

System Call Based Approach

• Rather than checking every memory access, check system calls instead– Idea is that process cannot interact with outside world

except through system call interface with the OS

• Several projects– Janus (Berkeley)– Systrace (Michigan)

• Trap system calls– Check parameters, deny unauthorized calls– Enforce mandatory access control in OS that does not

provide mandatory access control

Janus

• One of the first papers to propose system call interposition to limit the interface a process has to the operating system:– Idea is to invoke Janus every time process makes a system call– Janus is provided with the arguments of the system call– Janus either decides to allow the system call, or deny the

system call

• Two implementation approaches:– Use ptrace facility: this is the same facility used by gdb to track

another process’ execution– Use /proc file system

• Solaris implementation allows tracing process to get arguments and return value

Janus +

• David Wagner who worked on Janus, later extended the idea– Policies were derived from creating models from

programs via static analysis of the source code:

• A push down automata model of the program is created, including which system calls can be made when

• If program deviates form the model, the program has had a fault– Assumes that the programs are not malicious initially,

just may have vulnerabilities that can be exploited

Systrace

• Created by Niels Provos at Michigan– Via user configuration, a policy for what system calls and what

arguments an application does is derived.

• One main difference from Janus is that systrace has support at the kernel level.– Systrace solves some of the problems that affected Janus:

• Kernel interface solves race conditions• Normalizes the file system to resolve symlinks• Kernel implementation is also fail-safe, i.e. if systrace

fails, application is denied access by default– Policies are quite long and complex– More info at: http://www.citi.umich.edu/u/provos/systrace/

Java Virtual Machine (JVM)

• Java VM works on similar ideas– The JVM controls the java code, so that it can

not violate security policies• access to the filesystem• access to the network• etc.

• One issue to note with java applets– Signed applets can break the security policy

• That's reason they are signed.• But it can still be a malicious program.

A matter of trust

• Once again, most of this is a matter of trust.

• Do you trust the application or not?

– IE will ask you when downloading from Microsoft

• Always trust microsoft?

• So should you?

QA&