specifying multithreaded java semantics for program verification abhik roychoudhury national...

Specifying Multithreaded Java semantics for Program

VerificationAbhik Roychoudhury

National University of Singapore(Joint work with Tulika Mitra)

ICSE 2002, Orlando FL 2

Java MultithreadingThreads communicate via shared variablesThreads run on uni- or multi-processorsSemantics given as abstract rules : Java Memory Model (JMM)Supports locking of shared variables via synchronized statementsLocking may be avoided ….

ICSE 2002, Orlando FL 3

Shared variable access without locks

Initially : A = 0, B = 0

B = 1;

A = 1;

while (A != 1) {};

return B||

Expected returned value = 1Locks not used, but using A as flag.

ICSE 2002, Orlando FL 4

What may happen

Returned value = 0

A= 1

return B

B = 1

May happen if …..

Threads are executed on different processors.

B = 1;

A = 1;

while (A != 1) {};

return B||

ICSE 2002, Orlando FL 5

Sequential Consistency

||Op1;Op2;

Op’ ;Op’’ ;

Programmer expects statements within a thread to complete in program order: Sequential Consistency

I. Each thread proceeds in program order

II. Operations across threads are interleaved

Op1 Op’’ Op2 Op’ violates SC

ICSE 2002, Orlando FL 6

Is this a problem ?Programmers expect SCVerification techniques assume SC executionSC not guaranteed by execution platformsNot demanded by Java language spec.Unrealistic for any future spec. to demand

YES !!

ICSE 2002, Orlando FL 7

OrganizationShared variable access without locksCandidate solutionsSpecifying the Java Memory Model (JMM)Using JMM for verification

ICSE 2002, Orlando FL 8

1. Program with cautionAlways synchronize (and acquire lock) before writing a shared objectFor these programs, any execution is SCUnacceptable performance overheads for low-level librariesSoftware libraries from other sources cannot be guaranteed to be properly synchronized

ICSE 2002, Orlando FL 9

2. Change the SemanticsCurrent semantics called Java Memory Model (JMM). Part of Java language spec.Weaker than Sequential Consistency. Specifies which re-orderings are allowed within a thread.Currently being considered for revision Existing/future JMM bound to be weaker than SC – Does not solve the problem

ICSE 2002, Orlando FL 10

3. Use the semantics Develop a formal executable description of the Java Memory ModelUse it for static checking of programsJMM captures all possible behaviors for any implementation – Platform independent reasoningAdds value to existing program verification techniques

ICSE 2002, Orlando FL 11

OrganizationShared variable access without locksCandidate solutionsSpecifying the Java Memory Model (JMM)Using JMM for verification

ICSE 2002, Orlando FL 12

JMM OverviewEach shared variable v has• A master copy• A local copy in each thread

Threads read and write local/master copies by actionsImposes ordering constraints on actionsConstraints are informal and declarativeHard to understand !!

ICSE 2002, Orlando FL 13

JMM Actions

Master copy of v

read(v,t) <

load(v,t)

write(v,t)>

store(v,t)

Actions invoked by Program Execution: use/assign(t,v) Read from/ Write into local copy of v in t lock/unlock(t) Acquire/Release lock on shared variables

Local copy of v

in t

ICSE 2002, Orlando FL 14

Formal SpecificationAsynchronous concurrent composition

• Th1 || Th2 || … || Thn || MM

Local state of each thread modeled as cache• ( Local copy, Stale bit, Dirty bit )• Queues for incomplete reads/writes

Local state of MM • ( Master copies, Lock ownership info )

ICSE 2002, Orlando FL 15

Specifying JMM ActionsEach action is a guarded command G BOnly 8 actions capture all the rules.Example: Use of variable v by thread t stale[t,v] return local_copy[t,v]

Applicability of action stated as guardIn rule-based description, several rules determine the applicability

ICSE 2002, Orlando FL 16

Understanding JMM

assign(t, v)

<

load(t,v)

assign(t,v)

<

store(t,v)

<

load(t,v)A store must intervene between an assign

and a load action for a variable v by thread t

ICSE 2002, Orlando FL 17

Understanding JMMassign(t, v)

<

store(t,v)

<

load(t,v)

assign(t,v)

< store(t,v)

< write(t,v)

< read(t,v)

< load(t,v)

assign(t,v)

<

load(t,v)

read < assign <load is not possible. We specifyassign(t,v) : empty(read_queue[t,v]) -> ….

ICSE 2002, Orlando FL 18

Executable modelJava threads invoke use,assign,lock,unlockThreads block if the next action not enabledTo enable these, store,write,load,read are executed in any order provided guard holdsExample: To enable assign, a load is executed (if there was an earlier read)Proof of equivalence between executable and rule-based JMM.

ICSE 2002, Orlando FL 19

Other JMM FeaturesExecutable model also captures:• Volatile variables : Every access of

these variables accesses master copy• Prescient stores: Writing master copy

ahead of local copy• Waiting and Notification

Modeled by additional guarded commands.

ICSE 2002, Orlando FL 20

OrganizationShared variable access without locksCandidate solutionsSpecifying the Java Memory Model (JMM)Using JMM for verification

ICSE 2002, Orlando FL 21

Verifying Unsynchronized Code

assign(B,1)

assign(A,1)

While (use(A) !=1){}

use(B)||

use/assign invoke corresponding guarded commands

load/store/read/write executed to enable use/assign

Exhaustive state space exploration shows use(B) may return 1 or 0

ICSE 2002, Orlando FL 22

Program verificationComposing executable JMM allows searchThe state space explosion problem Most Java programs are “properly synchronized” and hence SC executionUnsynchronized code appears in low-level fragments which are frequently executedThese programs are small, so …

ICSE 2002, Orlando FL 23

One possibilityUser chooses one program path in each thread (Creative step)Exhaustively check all possible execution traces from chosen program paths (Automated state space exploration: Can verify invariants)Choosing program paths requires understanding source code, not JMM

ICSE 2002, Orlando FL 24

Case Study: Double Checked LockingIdiom for lazy instantiation of a singletonCheck whether garbage data-fields can be returned by this object initialization routineVerification by composing the JMM reveals:• Incompletely instantiated object can be returned• Due to re-ordering of writes within constructor• Detected by prototype invariant checker in 0.15 sec

ICSE 2002, Orlando FL 25

Summary Executable specification of Multithreaded Java semanticsUsing the specification for checking multithreaded programsSimilar approach has been studied before in the context of hardware multiprocessorsHow to correct the bugs found (the harmful re-orderings) ?