dongyun jin , patrick meredith, dennis griffith, grigore rosu

Dongyun Jin, Patrick Meredith, Dennis Griffith, Grigore Rosu

University of Illinois at Urbana-Champaign

Garbage Collection for Monitoring Parametric

Properties

2

Outline

Motivation Property Static Analysis Double Lazy Garbage Collection Evaluation Conclusion

3

Monitoring Scalable technique for ensuring software

reliability Observe run of a program

Analyze execution trace against desired properties

React/Report using handlers (if needed)

Program

Event 1

Event 2

Event 3

Event 4

…

…

Execute & Observe

Analyze

Execution Trace

?

4

Applications of Monitoring Development

Debugging Testing

Deployment Security Reliability Runtime Verification

5

Parametric Properties Properties referring to object instances The following property describes a bad behavior

between Each Collection c and Iterator i :

Generalize typestates Typestates are parametric properties with one

parameter

update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i )

6

Parametric Property Monitoring Systems Tracematches [de Moor et al. 05]

Regular Patterns PQL [Lam et al. 05]

Context-Free Patterns PTQL [Aiken et al. 05]

SQL-like language Eagle then RuleR [Barringer et al. 04]

Rule-Based Rewrite Properties (e.g. Context-Free Patterns, ..)

MOP [Rosu et al. 03] Formalism Independent Approach

many others

7

Parametric Monitoring in MOP Keep one monitor for each parameter instance

A parameter instance binds parameters to objects E.g.,

Each monitor knows nothing of parameters;operates exclusively on only one trace slice

8




Trace: create(c1, i1) create(c1, i2) next(i1) update(c1) next(i2)

9





(c1, i1) Trace Slice: create next update

(c1, i2) Trace Slice: create update next

10




update(c ) next(i )

create(c , i )

update(c )

update(c )

next(i )

update(c ) next(i )

create(c , i )

update(c )

update(c )

next(i )




11




update(c ) next(i )

create(c , i )

update(c )

update(c )

next(i )

update(c ) next(i )

create(c , i )

update(c )

update(c )

next(i )

Challenge: Large number of monitors




12

Previously… Benchmark based on DaCapo 9.12 [Blackburn et

al. 06] On average for 65 cases (13 examples 5

properties) Tracematches [de Moor et al. 05]

One of the most memory efficient monitoring systems

142% Runtime overhead 1

12% Memory overhead 1

MOP [Rosu et al. 03] 33% Runtime overhead 140% Memory overhead 1: Except 6(of 65)

crashes

13

Worst Cases – Runtime Overhead

bloat, UnsafeSyncMap

bloat, UnsafeSyncColl

pmd, UnsafeMapIterator

bloat, UnsafeMapIterator

avrora, UnsafeIterator

bloat, UnsafeIterator

bloat, HasNext

0% 500% 1000% 1500% 2000% 2500%

Tracematches MOP

2119%

19194%

637%

OOM

OOM

1359%

1942%

448%

569%

311%

1203%

571%

746%

716%

From DaCapo 9.12 and DaCapo 2006-10 MR2

~ 4M Monitors

14

Worst Cases – Memory Overhead







bloat, HasNext

10% 100% 1000% 10000%

Tracematches MOP

OOM

OOM


1059%294%

57%2896%

2799%5%

3439%

1082%

39%2510%

41%1030%

15

Goal: Reduce the Runtime/Memory Overhead Static

Program Analysis [Bodden et al. 09, 10], [Dwyer et al. 10], …

Property Analysis Dynamic

By using information from property analysis,garbage collect monitorswhen they become useless during monitoring

16

Outline


17

Property Static Analysis

update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i )

Property analysis tells us when monitors are unnecessary

Our analysis is formalism independent; we only show it for FSM monitors (see paper for CFG monitors)

The Last EventParameter instances that need to be alive

18


update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i )




create

19


update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i )




create

20


Needed Events




create

update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i )

21

Property Static Analysis Property analysis tells us when monitors are

unnecessary Our analysis is formalism independent; we only

show it for FSM monitors (see paper for CFG monitors)

The Last Event Parameter instances that need to be alive

create {c , i }

Needed Events

update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i )

22


create {c , i }


If any of these parameters are garbage collected,this monitor cannot trigger; can be collected



Needed Events

update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i )

23


update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i )




create {c , i }

update

24


update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i )




create {c , i }

update

25


update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i ) Needed Events




create {c , i }

update

26





create {c , i }

update {i }

update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i ) Needed Events

27


update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i )Needed Events




create {c , i }

update{i }

28





create {c , i }

update {i }{c , i }

update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i )Needed Events

29



create {c , i }

update {i }{c , i }

next {c , i }

update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i )



30



create {c , i }

update {i }{c , i }

next {c , i }

update(c )

next(i )

create(c , i )

update(c )

update(c )

next(i )



If any of these are dead,then the current monitorcan be collected

31

Outline


32

Double Lazy Garbage Collection Indexing Trees map parameter instances to monitors

Lazy propagation of info about dead parameter instances Eager propagation yields high runtime overhead (like TM) Idea: propagate when monitors are updated for other

reasons

Lazy removal of useless monitors Removal is expensive (monitor can belong to many trees) Idea: just mark unnecessary monitors, then clean up later

(technical; see paper for details and proof of correctness)

33

Outline


34

Evaluation – Overall On average, based on the DaCapo 9.12

Benchmark 13 Programs and 5 Properties

1: Except 6(of 65) crashes

Tracematches MOPRV

(MOP + Garbage Collection)

Runtime Overhead 142%1 33% 15%

Memory Overhead 12%1 140% 39%

35

Evaluation – Worst Cases Runtime Overhead







bloat, HasNext

0% 500% 1000% 1500% 2000% 2500%

Tracematches MOP RV2119%

19194%

637%

OOM

OOM

1359%

1942%

448%

569%

311%

1203%

571%

746%

716%

116%

251%

118%

178%

188%

212%

130%


~ 4M Monitors

36

Evaluation – Worst Cases Memory Overhead







bloat, HasNext

10% 100% 1000% 10000%

Tracematches MOP RV

OOM

OOM


1059%294%

57%2896%

2799%5%

3439%

1082%

39%2510%

41%1030%

169%

1512%

236%

1045%

420%

886%

159%

37

Conclusions Garbage collection for parametric monitoring

Formalism-independent Directed by property static analysis

Double Lazy Garbage Collection Lazy propagation of info Lazy removal of monitors

Evaluation Lowest runtime overhead parametric monitoring

system Reasonable memory overhead

dongyun jin , patrick meredith, dennis griffith, grigore rosu

Documents

parametric monitoring

mop8 trace

possible trace

mop9 trace

observeanalyzeexecution

i1 trace slice

i2 trace slice

parametric propertiesthis