testing multithreaded java applications for synchronization problems, ista 2011
Post on 10-May-2015
206 Views
Preview:
TRANSCRIPT
www.istabg.orgwww.vmware.com www.istabg.orgwww.vmware.com
Testing Multithreaded Java Applications for
Synchronization Problems
Vassil Popovski
www.istabg.orgwww.vmware.com
Why multithreading?
A modern CPU
…core 1
cache
core N
cache
shared memory
www.istabg.orgwww.vmware.com
Why multithreading?
…Thread 1 Thread N
Java heap
A modern JavaApplication
synchronization
thread 1 thread N
Java heap
www.istabg.orgwww.vmware.com
• Non-determinism
• New types of defects exist such as deadlocks, livelocks and race conditions
New concept -> new problems
www.istabg.orgwww.vmware.com
Thread 2Thread 1
Thread interleaving
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
www.istabg.orgwww.vmware.com
Thread 2Thread 1
Thread interleaving
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
www.istabg.orgwww.vmware.com
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2 T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
www.istabg.orgwww.vmware.com
Number of different thread interleavings
www.istabg.orgwww.vmware.com
Q: What is the number of all interleavings for 3 threads with 2 blocks each?
QUIZ#1
www.istabg.orgwww.vmware.com
• safety: nothing bad happens
• liveness: something good eventually happens
What to test for?
www.istabg.orgwww.vmware.com
Image source: http://www.doc.ic.ac.uk/~jnm/book/ppt/ch7.ppt
Example 1: Single Lane Bridge
• safety:– no car crash
• liveness:– every car eventually get an opportunity to cross the bridge
www.istabg.orgwww.vmware.com
Example 2: BoundedBuffer
Producer 1
Producer 2
Producer 3
Producer N
Consumer 1
Consumer 2
Consumer M…
…
• safety:– If empty, must not allow Get(); If full, must not allow Put()
• liveness (quiz#2):– Put() on empty and Get() on full will eventually be allowed
www.istabg.orgwww.vmware.com
How to test for synchronization issues?Why
• NASA’s Remote Agent
• Therac-25
www.istabg.orgwww.vmware.com
How to test for synchronization issues?
• Load/Stress testing (blackbox, whitebox)• Specific interleavings testing (whitebox)• All interleavings testing (whitebox)
• Instrumentation (blackbox)
www.istabg.orgwww.vmware.com
• A lot of threads and operations to exercise different interleavings
Load/Stress testing
www.istabg.orgwww.vmware.com
Demo!
www.istabg.orgwww.vmware.com
Deadlock example
Thread 2Thread 1
synchronized ( A ) {
synchronized ( B ) {
synchronized ( B ) {
synchronized ( A ) {
Deadlock !!!
www.istabg.orgwww.vmware.com
• Tools– ExecutorService -
http://download.oracle.com/javase/6/docs/api/java/util/concurrent/ExecutorService.html#invokeAll(java.util.Collection)
– TestNG - http://testng.org/doc/documentation-main.html#annotations
– JUnit - http://www.junit.org/apidocs/junit/extensions/ActiveTestSuite.html
• JUnitPerf - http://testng.org/doc/documentation-main.html#annotations
– Custom Threading– GroboUtils -
http://groboutils.sourceforge.net/testing-junit/using_mtt.html
Load/Stress testing
www.istabg.orgwww.vmware.com
• Deterministic and repeatable tests
Specific interleavings testing
www.istabg.orgwww.vmware.com
• Based on internal clock
– Tick increases when all threads are blocked
– waitForTick(tick) – blocks the thread until tick reaches certain value
– assertTick(tick) – asserts the current tick
MultithreadedTC
www.istabg.orgwww.vmware.com
MultithreadedTC
Thread 2Thread 1
put 42
get 42
put 17(blocks)
BoundedBuffer with size = 1
get 17
www.istabg.orgwww.vmware.com
MultithreadedTC
Thread 2Thread 1
put 42
get 42
BoundedBuffer with size = 1
get 17
Tick 0
put 17(blocks)
Tick 1
waitForTick(1)
assertForTick(1)
www.istabg.orgwww.vmware.com
Demo!
www.istabg.orgwww.vmware.com
• Events based
– IMUnit.fireEvent(“event1”)
– @Schedule(“event1 -> event2”)
– @Schedule(“[event1] -> event2”)
IMUnit
www.istabg.orgwww.vmware.com
Demo!
www.istabg.orgwww.vmware.com
• Two threads only – Main and Secondary
• By default: For each line of Main thread (T1) -> block and execute fully Secondary thread (T2)
ThreadWeaver
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
www.istabg.orgwww.vmware.com
• Two threads only – Main and Secondary
• By default: For each line of Main thread (T1) -> block and execute fully Secondary thread (T2)
• Powerful Breakpoints
ThreadWeaver
www.istabg.orgwww.vmware.com
Demo!
www.istabg.orgwww.vmware.com
• Tools– MultithreadedTC -
http://code.google.com/p/multithreadedtc/• Enhanced version for Junit 4 -
http://code.google.com/p/multithreadedtc-junit4/
– IMUnit - http://mir.cs.illinois.edu/imunit/– ThreadWeaver -
http://code.google.com/p/thread-weaver/– Awaitility - http://code.google.com/p/awaitility/
Specific interleavings testing
www.istabg.orgwww.vmware.com
• Exercise all possible interleavings
All interleavings testing
www.istabg.orgwww.vmware.com
JavaPathFinder
Traditional testing
OK Code
Testing
error Image source: http://javapathfinder.sourceforge.net/events/JPF-workshop-050108/tutorial.ppt
www.istabg.orgwww.vmware.com
JavaPathFinder
Model Checking with JavaPathFinder
Image source: http://javapathfinder.sourceforge.net/events/JPF-workshop-050108/tutorial.ppt
OKCode
properties
Model Checking
error trace
Line 5: …Line 12: ……Line 41:…Line 47:…
www.istabg.orgwww.vmware.com
Program testing can be used to show the presence of bugs, but never to show their absence!
--Edsger Dijkstra
JavaPathFinder
www.istabg.orgwww.vmware.com
Demo!
www.istabg.orgwww.vmware.com
• Tools– JavaPathFinder (JPF) -
http://babelfish.arc.nasa.gov/trac/jpf
• JavaRaceFinder - http://www.cise.ufl.edu/research/JavaRacefinder/Java_RaceFinder/JRF_Home.html
All interleavings testing
www.istabg.orgwww.vmware.com
• Instrument the code to catch problems easier
Instrumentation
www.istabg.orgwww.vmware.com
AspectJ
Original code.class or .jar
AspectJ Compiler
Instrumented .class or .jar
Aspect Definition.aj
www.istabg.orgwww.vmware.com
Demo!
www.istabg.orgwww.vmware.com
• Tools– AspectJ - www.eclipse.org/aspectj/
• RacerAJ - http://www.bodden.de/tools/raceraj/
– CalFuzzer - http://srl.cs.berkeley.edu/~ksen/calfuzzer/– (commercial) Flashlight / Jsure -
http://www.surelogic.com/concurrency-tools.html
Instrumentation
www.istabg.orgwww.vmware.com
• CHESS (native DLLs & managed executables) - http://research.microsoft.com/en-us/projects/chess/
• MoonWalker (.NET)- http://code.google.com/p/moonwalker/
Other tools
www.istabg.orgwww.vmware.com
Recommended books
www.istabg.orgwww.vmware.com
Thank you!
www.istabg.orgwww.vmware.com
Q & A
www.istabg.orgwww.vmware.com
• org.ista2011.multithreaded.common– Several implementations of AccountManager, Counter, BounderBuffer and
MultipleReadersSingleWriter
• org.ista2011.multithreaded.executorservice– Load/stress tests using ExecutorService
• org.ista2011.multithreaded.testng– Load/stress tests using TestNG
• org.ista2011.multithreaded.mtc– MultithreadedTC tests
• org.ista2011.multithreaded.imunit– IMUnit tests
• org.ista2011.multithreaded.threadweaver– ThreadWeaver tests
Source code packages (1)
www.istabg.orgwww.vmware.com
• org.ista2011.multithreaded.javapathfinder– JavaPathFinder examples
• ca.mcgill.sable.racer – RacerAJ source code
• org.ista2011.multithreaded.aspectj– Custom aspect that increase the chance of hitting multithreading problem
during testing
• org.ista2011.multithreaded.blockbox.*– Rest based AccountManager client/test and server (in .server package)
Source code packages (2)
www.istabg.org
top related