1 compression techniques to simplify the analysis of large execution traces abdelwahab hamou-lhadj...
TRANSCRIPT
1
Compression Techniques to Simplify the Analysis of Large Execution Traces
Abdelwahab Hamou-Lhadj and Dr. Timothy C. Lethbridge
{ahamou, tcl}@site.uottawa.ca
University of Ottawa - Canada
IWPC 2002 - Paris
2
Introduction
Execution traces are important to understand the behavior and sometimes the structure of a software system
Execution traces tend to be very large and need to be compressed
In this presentation, we present techniques for compressing traces of procedure calls
We also show the results of our techniques when applied to two different software systems
3
Why Traces of Procedure Calls?
Many of today’s legacy systems were developed using the procedural paradigm
The flow of procedure calls can be useful to comprehend the execution of a particular software feature
The level of abstraction of traces of procedure calls tend to be not too low and not too high
Traces of method invocation become crucial when it comes to understand the behavior of object-oriented systems
4
Traditional Compression Techniques
They are two types of compression techniques: lossy and lossless compression
In Information theory, most of the compression algorithms are based on the same principle (David Salomon, 2000)
Compressing data by removing redundancy
These techniques produce good results, however The information, once compressed, is no longer readable by
humans. Such algorithms certainly will not help in program comprehension
5
Trace Compression Steps
Preprocess the trace by removing the contiguous redundancies due to loops and recursion
Represent the trace as a rooted ordered labeled tree
Detect the non-contiguous redundancies and represent them only once
this problem is also known as the common subexpression problem and can be solved in linear time
Analyze the compressed version and estimate the gain
6
Preprocessing Stage
Redundant calls caused by loops and recursion tend to encumber the trace and should be removed
the number of occurrences is stored to reconstruct the original trace
Removing the redundant calls is one form of compression that could make the trace more readable
If the trace is perceived as a tree, removing contiguous redundancies reduce the depth of the tree and the degree of its nodes
7
The Common Subexpression ProblemIntroduced by J.P. Downey, R. Sethi and R.E. Tarjan
“Any tree can be represented in a maximally compact form as a directed acyclic graph where common subtrees are factored and shared, being represented only once” - Flajolet, Sipala and Steyaert
The process of compacting the tree is known as the common subexpression problem also called “subtree factoring”
If we consider trees with a finite number of nodes so that the degrees are bounded by some constant ... “The compacted form of a tree can be computed in expected time O(n) using a top-down recursive procedure in conjecture with hashing...” - Flajolet, Sipala and Steyaert
8
Example
A
D
CB
E
C
CB
D
1 2
3221
3 4
5
A E D C B12345
Input tree: 9 nodes and 8 links The Compressed form: 5 nodes and 6 links
9
The Algorithm Introduced by P. Flajolet, P. Sipala, J.–M. Steyaert and improved by G. Valiente
The algorithm assigns a positive number called certificate to each node
Two nodes have the same certificate if, and only if the trees rooted at them are isomorphic.
The certificate of a node n is obtained by building a sequence [L(n), a1, .... , am] called the signature of the
node, where L(n) is the label of the node, a1,..., am are the certificates of the children of the node.
The certificates and signatures are stored in a global table
10
Example
Node Signature Certificate
A [A, 4, 3] 5
E [E, 3, 2] 4
D [D, 2, 1] 3
C [C, 0, 0] 2
B [B, 0, 0] 1
A
D
CB
E
C
CB
D
1 2
3221
3 4
5
11
The Algorithm Steps (iterative version)
The algorithm performs a bottom-up traversal of the tree using a queue
1. For each node n2. Build a signature for n 3. If the signature already exists in the global table then
4. Return the corresponding certificate Else
5. Create a new certificate6. Update the table
7. Assign the certificate to the node
If the degree of the tree is bounded by a constant and a hash table is used to store the certificates then this algorithm performs in linear time
12
Experiment
We experimented with traces of the following systems: XFIG (a drawing system under UNIX) A real world telecommunication system
We are interested in the following results: The initial size of the trace n The size of the trace after preprocessing it n1
The compression ratio r1 such that r1 = n1 / n
The size of the trace after using the common subexpression algorithm n2.
The compression ratio r2 such that r2 = n2 / n
13
Results of the Experiment (XFIG System)
Trace n n1 r1 (%) n2 r2(%)
1 2198 623 28.34 218 9.92
2 9076 647 7.13 223 2.46
3 5140 889 17.30 236 4.59
4 2710 1136 41.92 248 9.15
5 3077 700 22.75 236 7.67
6 6215 869 13.98 261 4.20
7 3381 839 24.82 253 7.48
8 4336 830 19.14 267 6.16
14
Some Considerations Regarding the Telecommunication System
It is a large legacy system
The traces are generated using an internal mechanism
The traces tend to be incomplete. This is reflected as an inconsistency in the trace with respect to the nesting levels.
Our solution to this problem is to complete the trace by filling up the gaps with virtual procedure calls estimate the error ratio, which is the number of missing calls to the
size of the original trace.
e = g / (g+n)
15
Trace n g e(%) n1 r1 (%) n2 r2 (%)
1 17465 589 3.37 14396 82.43 2452 14.04
2 11095 313 2.82 9715 87.56 3308 29.82
3 10175 381 3.74 8654 85.05 2361 23.20
4 3621 121 3.34 3226 89.09 694 19.16
5 3609 109 3.02 3281 90.91 1150 31.86
Results of the Experiment (Telecom. System)
16
Before the preprocessing step After the preprocessing step
Variation of the degrees of the tree according to depth (3 traces of XFIG)
0
50
100
150
200
250
300
350
400
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Depth
Deg
ree
0
50
100
150
200
250
300
350
400
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Depth
Deg
ree Polyline
Text
Move
17
Variation of the degrees of the tree according to depth (3 traces of the telecom. system)
Before the preprocessing step After the preprocessing step
0
10
20
30
40
50
60
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Depth
Deg
ree
Process1
Process2
Process3
0
10
20
30
40
50
60
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Depth
Degr
ee
18
Discussion
Procedure-call traces could be considerably compressed in a way that preserves the ability for humans to understand them
Possible improvement look for procedures that are not of a great interest to software
engineers remove them before the compression process
The preprocessing stage could be very useful to reduce the trace size increase of the performance of the common subexpression
algorithm
19
Conclusions and future directions The results shown in this presentation can help build better tools based on execution traces
We intend to conduct more experiments with this framework to see how helpful it is to software engineers
Future directions should focus on lossy compression.Types of information eliminated can include:
the number of repetitions, the order of calls, and some lower-level utility procedures
The non-contiguous redundancies can be used to determine other features of the system
20
21
Results of the Experiment (XFIG System)With procedures and files
Trace n n1 r1 (%) n2 r2(%) # Proc. # Files
1 2198 623 28.34 218 9.92 167 30
2 9076 647 7.13 223 2.46 174 31
3 5140 889 17.30 236 4.59 178 30
4 2710 1136 41.92 248 9.15 190 28
5 3077 700 22.75 236 7.67 190 33
6 6215 869 13.98 261 4.20 197 31
7 3381 839 24.82 253 7.48 187 33
8 4336 830 19.14 267 6.16 190 33
22
Results of the Experiment (Telecom. System) with procedures and files
Trace n g e(%) n1 r1 (%) n2 r2 (%) # Proc. # Files
1 17465 589 3.37 14396 82.43 2452 14.04 802 189
2 11095 313 2.82 9715 87.56 3308 29.82 828 184
3 10175 381 3.74 8654 85.05 2361 23.20 876 190
4 3621 121 3.34 3226 89.09 694 19.16 657 160
5 3609 109 3.02 3281 90.91 1150 31.86 668 164