cs265: program analysis, testing, and debugging

CS265: Program Analysis, Testing, and Debugging

Lecture 1Koushik Sen

EECS, UC Berkeley

My Background

• Assistant Professor in CS since Fall 2006– Office: Parlab (581 Soda Hall)– Email: ksen@cs

• Ph.D. and M.S. in Computer Science– University of Illinois at Urbana Champaign (2001-2006)

• Spent 1 year in software industry as a software developer• B-Tech

– IIT Kanpur (1995-1999)• Research Interests:

– Software Engineering, – Programming Languages, and – Formal Methods– Verification, Testing, Model Checking, Program Analysis

Please introduce yourselves

Logistics

Course Goals

• To help students start research in the area of program analysis, model checking, testing, and debugging of sequential, concurrent, and distributed systems.

• To help students to apply the techniques learned in this course in their ongoing research in other areas such as operating systems, computer networks, security, and database systems.

• Should meet your Th+Sys prelims requirement

Course Communication

• All class materials will be on the website– http://fa09.pbwiki.com/

• See website for announcements• Class meets on Monday and Wednesday, 1:00

PM – 2:30 PM at 320 Soda Hall• Office hour by appointment

http://fa09.pbwiki.com/

http://fa09.pbwiki.com/

Course Structure

• We will study papers– You will read a paper and write a review of the

paper before each class– There will be some guest lectures

• You need to do 2-3 programming assignments– Test generation– Dynamic analysis of concurrent programs– Points-to analysis

• Project in teams of 1-2.

Course Grading

• Reviews and class participation: 30%• 2-3 homework assignments: 30%• Project: 40%

– A project must involve new research– Some sample projects will be posted online– Choose a project topic by 25th September, 2009– 1-2 page project proposal due by 9th October, 2009– A 5-7 minute mid-semester project presentation– Final project demo or presentation– 6 page project report in ACM SIGPLAN conference format

About the course

• This course is about software reliability• Why is software reliability important?• As society becomes more dependent on software, the

consequences of software failures are non-trivial. – Money lost.– Lives lost.– Market share lost.– Clients lost.– Jobs lost.

Software Bug => Space Disaster• Ariane 5 Space mission• $7, 000, 000, 000• 10 Years in the making• 40 seconds after take off

the rocket exploded

Software Bug => Space Disaster

Attempt to cram a 64-floating point number to a 16-bit integer failed

AT&T long distance service failed for 9 hours

• On January 15, 1990, one of AT&T's #4ESS toll switching systems in New York City experienced an intermittent failure that caused a major service outage.

• Wrong BREAK statement in C-Code• Complete coverage could have revealed this bug during

testing

/* ``C'' Fragment to Illustrate AT&T Defect */

do { switch expression { ... case (value): if (logical) { … break; } else { … }

case (value2): … } …

} while (expression)

Software Bugs: Cause of Deaths

• Several deaths of cancer patients were due to overdoses of radiation resulting from a race condition between concurrent tasks in the Therac-25 software.

Torpedoes, that deviate more than 90 degree, explode to avoid self destruction of the ship.Once upon a time a ship fired a torpedo but the torpedo was jammed in the tube. Then the captain gave the command: Let's turn around and return to the harbour! What happened next is no mystery.

180 Degree Bug

Cost of Failure

• Software failures were estimated to cost the US economy about $60 billion annually [NIST 2002]– Improvements in software testing infrastructure

may save one-third of this cost• Testing accounts for an estimated 50%-80%

of the cost of software development [Beizer 1990]

Methods for Building Reliable Software

Safe Programming Languages and Type

systems

Static Program Analysis

Dynamic Program Analysis

Model Based Software Development and Analysis

Model Checking

and Theorem Proving

Testing

Course Contents

• Automated test generation• Software model checking and various

theoretical results that form the foundation of software model checking

• Concurrent program analysis• Abstract interpretation and points-to analysis

Automated Test Generation

• Korat:– Reading assignment: Korat: Automated Testing

Based on Java Predicates. Chandrasekhar Boyapati, Sarfraz Khurshid, Darko Marinov (ISSTA 2002)

– Must submit review to the course wiki by 9/8 11:59 PM

• Concolic Testing: Homework 1

Automated Test Generation

• Generate test inputs– To reveal bugs: assertion violation, crashes, wrong

output– Improve software reliability

• Often cannot prove program correct– Need to check program for all possible inputs– Input domain is often infinite– Pick inputs to satisfy certain coverage criteria

• Generate all legal inputs of bounded size• Generate all legal inputs for full branch coverage

Course Contents




Software Model Checking

• Attempt to prove programs correct– Abstract domain– Create an abstraction of the program– Show that the abstraction does not contain a “bad

state”• Predicate abstraction and boolean programs • Successfully used for model checking device

drivers

Software Model Checking

• BLAST: predicate abstraction• Decidability results for various boolean program

models– Forms theoretical foundation of various software

model checking algorithms– Pushdown systems– Pushdown systems with multiset– Petri-Nets– Multi-pushdown systems with bounded context switch

Course Contents




Concurrent Program Analysis

• Bugs due to concurrency are notorious– Intermittent and hard to reproduce– Common causes: data race, atomicity violations,

deadlocks, and other synchronization issues– Much more difficult to analyze than sequential

programs– Need to check program

• all schedules• all inputs

– Pushdown system with stacks => undecidable

Concurrent Program Analysis

• Classic dynamic race detection algorithms• Atomicity checking and deadlock analysis• Explicit State Model Checking• Partial Order Reduction• Iterative preemption bounded model checking• Homework 2

Course Contents




Abstract Interpretation and Points-to Analysis

• Abstract interpretation– Note software model checking is an instantiation of

abstract interpretation– forms basis of most static program analyses

• Mayur Naik on Points-to analysis– The most important analysis– Anderson’s points-to analysis– BDDs (Binary Decision Diagrams) for scalable points-to

analysis– And a homework

Course Contents




Summary

• 3 homework assignments will give you hands-on experience of program analysis– In my opinion, this is quite useful to get a deep

understanding of the subject, and– to jump start research in program analysis

Symbolic Execution

• Use symbolic values for input variables• Execute the program symbolically on symbolic

input values• Collect symbolic path constraints• Use constraint solver to generate test inputs

for each execution path

Computation Tree: Execution Paths of a Program

• Can be seen as a binary tree with possibly infinite depth– Computation tree

• Each node represents the execution of a “if then else” statement

• Each edge represents the execution of a sequence of non-conditional statements

• Each path in the tree represents an equivalence class of inputs

0 1

0 0

0

0

1

1

1

1

1

1

Example of Computation Treevoid test_me(int x, int y) { if(2*x==y){

if(x != y+10){ printf(“I am fine here”);} else { printf(“I should not reach here”); ERROR;}

}}

2*x==y

x!=y+10

N Y

N Y

ERROR

Concrete Execution

int x = read();int y = read();int t;

t = x;x = x + y;if (x > y) {

y = 2*t; if (x+1 == y) {

assert(false);}

}

Concrete Execution


t = x;x = x + y;if (x > y) {

y = 2*t; if (x+1 == y) {

assert(false);}

}

x = 4y = 9

t = 4x = 13(13 > 9)y = 8(14 == 8)

END

Symbolic Execution


t = x;x = x + y;if (x > y) {

y = 2 * t; if (x+1 == y) {

assert(false);}

}

(true | x = x0 )

(true | x = x0,y=y0)

(true | x = x0,y=y0,t=x0)

(true | x = x0+y0,y=y0,t=x0)

x0+y0 > y0

(x0 <= 0 | x = x0+y0,y=y0,t=x0) (x0>0 | x = x0+y0,y=2x0,t=x0)

x0+y0 + 1 == 2x0

(x0 > 0 && y0 + 1 == x0

| x = x0+y0,y=2x0,t=x0)

(x0 > 0 && y0 + 1 x0

| x = x0+y0,y=2x0,t=x0)

Test Generation


t = x;x = x + y;if (x > y) {

y = 2 * t; if (x+1 == y) {

assert(false);}

}

(true | x = x0 )

(true | x = x0,y=y0)

(true | x = x0,y=y0,t=x0)

(true | x = x0+y0,y=y0,t=x0)

x0+y0 > y0

(x0 <= 0 | x = x0+y0,y=y0,t=x0) (x0 > 0 | x = x0+y0,y=2x0,t=x0)

x0+y0 + 1 == 2x0

(x0 > 0 && y0 + 1 == x0

| x = x0+y0,y=2x0,t=x0)

(x0 > 0 && y0 + 1 x0

| x = x0+y0,y=2x0,t=x0)

Solve: x0<= 0Solution: x = 0, y=0

Solve: x0 > 0 && y0+1 x0

Solution: x = 1, y=9

Solve: x0 > 0 && y0+1 = x0

Solution: x = 2, y=1

Abstract Interpretation: Test to Proof

Example ( ) {int new = read(); lock = 0;1: do{ lock = 1; old = new; b = read();2: if (b){3: lock=0; new ++; }4:} while(new != old);5: assert (lock==1);}


Example ( ) {int new = read(); lock = 0;1: do{ lock = 1; old = new; b = read();2: if (b){3: lock=0; new ++; }4:} while(new != old);5: assert (lock==1);}Predicates: LOCK, new==old


Example ( ) {int new = read(); lock = 0;1: do{ lock = 1; old = new; b = read();2: if (b){3: lock=0; new ++; }4:} while(new != old);5: assert (lock==1);}

: LOCK1

2

3

4

1

LOCK , new=old

4

5

Reachability Tree

LOCK , new==old

LOCK , new==old

: LOCK , : new = old

: LOCK, : new == old

Predicates: LOCK, new==old

cs265: program analysis, testing, and debugging

Documents

software bug

software engineering

software reliabilitywhy

software industry

software flaw

software reliability

project topic

page project proposal