Attacking Malicious Code:A Report to the Infosec Research Council

Attacking Malicious Code:A Report to the Infosec Research Council

Kim Sung-Moo

Contents

What is malicious code?Defense against malicious codeCurrent defensesPromising new defensesPolicy as Achilles’ HeelConclusions

What is Malicious Code?

Any code added, changed, or removed from a software system.

To intentionally cause harm or subvert the system’s intended function.

Examples of malicious code

Viruses attach to host programs and propagate when an infected

program executes. Worms

Particular to networked computers. Attack to jump from one to another across network.

Trojan Horses Hide malicious intent inside a host program that appears to do

something useful. Attack Scripts

Exploit security weaknesses like buffer overflows. Java attack applets

Embedded in Web pages. Dangerous ActiveX controls

A Growing Problem

Networks are everywhere. Internet makes an attack easy.

System complexity is rising. Applications becoming larger, bugs can’t be avoided. Hard to find malicious code in API. The use of unsafe programming language (C or C++)

Systems are easily extensible. Malicious code can slip in systems as an unwanted

extension. (scripts, applets, etc.)

Defense against Malicious Code

4 approaches that the host can protect itself. Analyze the code and reject it if there is the potential

that executing it will cause harm. Rewrite the code before executing it so that it can do

no harm. Monitor the code while its executing and stop it

before it does harm. Audit the code during executing and take policing

action if it did some harm.

Code Analysis

Scan a file or do dataflow analysis. Help finding security-related bugs. (buffer overflow)

Necessarily limited Because determining if code will misbehave is as

hard as the halting problem.

Code Rewriting

Less pervasive but more important.Inserts extra code to perform dynamic checks

that ensure bad things cannot happen.Ex) array bounds check in Java

Code Monitoring

Traditional approachMonitor the code while its executing and stop it

before it does harm.Ex) JVM interpreter

Interpreter monitors execution of applets and mediates access to system calls

By examining the execution stack to determine who is issuing the system call request.

Code Auditing

If we want to recover damage, the damage must be properly assessed and addressed.

Create an audit trail that captures program behavior.

Security Principles

The Principle of Least Privilege A component should be given the minimum access

necessary to accomplish its intended task.

The Principle of Minimum Trusted Computing Base (TCB). The best way to assure that your system is secure is

to keep your TCB small and simple. TCB is the set of hardware and software components

Current Defenses

OS-Based Reference MonitorsScanning for Known Malicious CodeCode Signing

OS-Based Reference Monitors

Provided by the computer hardware and OS. Examples

Address translation hardware Supervisor-mode and user-mode Timer interrupt System calls for invoking a trusted software

Disadvantage Work with a fixed system-call interface and a fixed vocabulary. Hard to extend interface and vocabulary.

Advantage Effective for protecting OS resources.

Scanning for Known Malicious Code

Blacklisting matches programs against a database of known virus signatures.

Disadvantage Weak to unknown malicious codes Each file must be scanned Too easy to change malicious code a little.

Advantage Cheap to implement Easily find naïve attacks

Code Signing

Authenticate code based on public-key cryptography and digital signatures.

Poor understanding of most people If a key is stolen, anyone can use the key to sign any

code. We can’t know developer’s bad thought.

Lack of a Public Key Infrastructure (PKI)

Promising New Defenses

Software-Based Reference MonitorsType-Safe LanguagesProof-Carrying Code

Software-Based Reference Monitors

Alternative to OS-based reference monitorsIdea

Rewrite binary code Insert extra checks and states on each memory

access and control transfer.

Current research Implementation of Java’s stack inspection security

policy

Type-Safe Languages

Type-safe programming language Java, scheme, ML Ensure that operations are only applied to values of

the appropriate type. Specify new, abstract types for operations. Enable offline enforcement through static type

checking instead of each time a operation is performed.

Current research Eliminate more runtime checks (array bounds checks) Eliminate type-checking machine code

Proof-Carrying Code

Require any untrusted code to come equipped with an explicit, machine-checkable proof Before executing Prove that the code respects a given security policy.

Proof checker is very simple.Can enforce any security policy with proofCurrent research

How to integrate compilers and modern theorem provers to produce PCC.

Policy as Achilles’ Heel

Policy can be of service.But there is the problem of policy.

Bad policy Incorrectly enforced policy

There is the many levels of policy.

Bad Policy

Misunderstanding of context policy makes no sense in the context where it was ap

plied.

Over restriction the policy prevents useful work when it is enforced.

Noncomprehensiveness policy fails to cover some situation or exists at the wro

ng level of abstraction.

Incorrectly Enforced Policy

The enforcement mechanism is too weak to implement the desired policy.

There are bugs in the implementation of the enforcement mechanism.

The enforcement mechanism is misconfigured.

Example

Bad Policy ExampleIncorrect

policy enforcement

Example

Context misunderstood

Script languages(Visual Basic)

Mechanism is too weak

Privacy in smart cards

Overly restrictive

Changing passwords too frequently

Mechanism is buggy

Buffer overflows in kernels

Noncompre-hensive

Executable content in e-mail(love virus)

Mechanism is misconfigured

Sendmail debug mode

The Many Levels of Policy

By collection of lower-level enforcement mechanisms, define metalevel policies. We must consider safety, liveness, confidentiality. Most of technologies can enforce particular aspects of

software behavior.

Question How to express reasonable security policy that can be

directly transformed into solutions.

Answer Understand policy as a layered set of abstractions.

Conclusions

The malicious code problem will continue to grow as the Internet grows.

Our hope is creating sound policy about software behavior enforcing that policy through the use of technologies