impact analysis yury chebiryak urriy@wjpserver.cs.uni-sb.de university of saarland department of...

Impact Analysis

Yury Chebiryak

urriy@wjpserver.cs.uni-sb.de

University of SaarlandDepartment of Computer Science

2/96

Impact Analysis is often used to assess the effects of a

change after change has been made

is rarely used to predict the effect of change before it is instantiated

3/96

Techniques

ImpactAnalysis

TraceabilityAnalysis

DependencyAnalysis

4/96

Dependency-based IA

DependencyAnalysis

Program Slicing Call Graph

Static Dynamic

Whole Program Path-Based Dynamic Impact Analysis

James Law

Gregg Rothermel

6/96

PathImpact Dynamic Call-based Works on binaries Relatively low cost Relatively low overhead

7/96

Call Graph

M

A D

C E

F

G

B

8/96

Call Graph

M

A D

C E

F

G

B

B is changed

9/96

M

A D

C E

F

G

B

Call Graph

Assumption:change in B has a

potential impact on nodes reachable

from B

10/96

Execution Traces (ET)

M

A D

C E

F

G

B

Run M()

ET: M

11/96

Execution Traces

Run B()

ET: MB

M

A D

C E

F

G

B

12/96

Execution Traces

M

A D

C E

F

G

B

Return

ET: MBr

13/96

Execution Traces

ET: MBrA

M

A D

C E

F

G

B

Run A()

14/96

Execution Traces

Run C()

ET: MBrAC

M

A D

C E

F

G

B

15/96

Execution Traces

Run D()

ET: MBrACD

M

A D

C E

F

G

B

16/96

Execution Traces

Return

ET: MBrACDr

M

A D

C E

F

G

B

17/96

Execution Traces

Run E()

ET: MBrACDrE

M

A D

C E

F

G

B

18/96

Execution Traces

Return

ET: MBrACDrEr

M

A D

C E

F

G

B

19/96

Execution Traces

Return

ET: MBrACDrErr

M

A D

C E

F

G

B

20/96

Execution Traces

Return

ET: MBrACDrErrr

M

A D

C E

F

G

B

21/96

Execution Traces

M

A D

C E

F

G

B

Return

ET: MBrACDrErrrrx

22/96

Execution Traces II Single

Multiple

MBrACDrErrrrx

MBrACDrErrrrxMBGrrrrxMBCFrrrrx

Programs with loops => very long trace

23/96

SEQUITUR algorithm

Data compression algorithm by Larus: Online Created grammar reproduces trace exactly O(N) running time O(N) size in the worst case O(log N) size in the best case e.g. 2GB trace → 100 MB

24/96

SEQUITUR algorithmalgorithm SEQUITUR( S )input Execution Trace Soutput Grammar GGrammar GRule T1. for each token in S2. append token to end of production for T3. if duplicate digram appears4. if other occurrence is a rule g in G5. replace new digram with non-terminal of g.6. else7. form a new rule and replace duplicate8. digrams with the new non-terminal.9. if any rule in G is used only once10. remove the rule by substituting the production.11. return G

25/96

SEQUITUR: example

ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx

26/96

SEQUITUR: example

ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx

T → MBrACDrErrrrx

27/96

SEQUITUR: example

ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx

T → MBrACDrE x1 → rr

1 1

28/96

SEQUITUR: example

ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx

T → MBrACDrE11x1 → rr

29/96

SEQUITUR: example

ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx

T → MBrACDrE11xM1 → rr

30/96

SEQUITUR: example

ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx

T → rACDrE11x 1 → rr

MBMB

31/96

SEQUITUR: example

ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx

T → rACDrE11x 1 → rr2 → MB

22

32/96

SEQUITUR: example

ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx

T → 2rACDrE11x2 1 → rr2 → MB

33/96

SEQUITUR: example

ET: MBrACDrErrrrxMBGrrrrxMBCFrrrrx

T → 2rACDrE113G43CF4rx 1 → rr2 → MB3 → x24 → 1r

grammar => whole path DAG

34/96

Whole path DAG

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

35/96

PathImpact

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r x

UP()

E

36/96

PathImpact

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r x

UP()

E

XBACKWARD() FORWARD()

37/96

PathImpact

PathImpact(E): I {E} I = {}

Returns=0

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

38/96

PathImpact

PathImpact(E): up(T, E) I = {E}

Returns=0

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

39/96

PathImpact

PathImpact(E): forward(1) I = {E}

Returns=0

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

40/96

PathImpact

PathImpact(E): forward(r) I = {E}

Returns=0

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

41/96

PathImpact

PathImpact(E): Returns++ I = {E}

Returns=0

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

42/96

PathImpact

PathImpact(E): forward(r) I = {E}

Returns=1

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

43/96

PathImpact

PathImpact(E): Returns++ I = {E}

Returns=1

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

44/96

PathImpact

PathImpact(E): after forward(1) I = {E}

Returns=4

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

45/96

PathImpact

PathImpact(E): forward(3) -> x I = {E}

Returns=4

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

46/96

PathImpact

PathImpact(E): backward(r) I = {E}

Returns=4

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

Skip=0

47/96

PathImpact

PathImpact(E): Skip++ I = {E}

Returns=4Skip=0

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

48/96

PathImpact

PathImpact(E): backward(D)->Skip-- I = {E}

Returns=4Skip=1

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

49/96

PathImpact

PathImpact(E): backward(C)->I := I U {C},Returns--

I = {E}

Returns=4Skip=0

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

50/96

PathImpact

PathImpact(E): backward(A)->I := I U {A},Returns--

I = {E, C}

Returns=3Skip=0

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

51/96

PathImpact

PathImpact(E): backward(r)->Skip++ I = {E, C, A}

Returns=2Skip=0

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

52/96

PathImpact

PathImpact(E): after backward(2) I = {E, C, A, M}

Returns=1Skip=0

T 2 r A C D r E 1 1 3 G 4 3 C F 4 r x

1 r r 2 M B 3 x 2 4 1 r

M A B C D GF r xE

Chianti: A Tool for ChangeImpact Analysis of Java Programs

Ophelia ChesleyXiaoxia Ren

Barbara G. RyderFenil ShahFrank Tip

54/96

Chianti: high-level concept

Inputs:

1) test suite T

2) original version of code

3) edited version of code

55/96

Chianti: high-level concept

1) set of interdependent atomic changes A

56/96

Chianti: high-level concept

1) set of interdependent atomic changes A

2) call graph for each test in T

57/96

Chianti: high-level concept

1) set of interdependent atomic changes A

2) call graph for each test in T

3) T´ T – potentially affected by cA

58/96

Chianti: high-level concept

1) set of interdependent atomic changes A

2) call graph for each test in T

3) T´ T – potentially affected by cA

4) for tiT´ determine A´A that may have affected ti

59/96

Chianti: high-level concept

1) set of interdependent atomic changes A

2) call graph for each test in T

3) T´ T – potentially affected by cA

4) for tiT´ determine A´A that may have affected ti

determine subset of methods affected by

a set of changes

isolate subset of changes that affect

a given test

60/96

Chianti: architectureSet of Unit Tests

Changedprogram P’

Originalprogram P

Call Graphof Tests

in P’

AffectedTests

AffectingChanges

Call GraphBuilder

CI analyzer

CHIANTI

Call Graphof Tests

in P

Atomic ChangeDecoder

AtomicChanges &

Dependencies

61/96

Chianti: architectureSet of Unit Tests

Changedprogram

P’

Originalprogram

P

Call Graphof Tests

in P’

AffectedTests

AffectingChanges

Call GraphBuilder

CI analyzer

CHIANTI

Call Graphof Tests

in P

Atomic ChangeDecoder

AtomicChanges &

Dependencies

62/96

Chianti: architectureSet of Unit Tests

Changedprogram P’

Originalprogram P

Call Graphof Tests

in P’

AffectedTests

AffectingChanges

Call GraphBuilder

CI analyzer

CHIANTI

Call Graphof Tests

in P

Atomic ChangeDecoder

AtomicChanges &

Dependencies

63/96

Example: original versionclass A{public A() {}public void foo (){}

}class B extends A{public B(){}public void foo(){}

}class C extends A{public C(){}

}

64/96

Example: changed version

class A{void foo (){}int x;

}class B extends A{void foo(){ B.bar(); }static void bar(){ y=17; }static int y;

}class C extends A{void foo(){ x=18; }int z;

}

65/96

Example: testsclass Tests {public static void test1(){

A a = new A();a.foo();

}public static void test2(){

A a = new B();a.foo();

}public static void test3(){

A a = new C();a.foo();

}}

66/96

Review: Atomic Changes

Change Description

AC Add an empty class

DC Delete an empty class

AM Add an empty method

DM Delete an empty method

CM Change body of a method

LC Change virtual method lookup

AF Add a field

DF Delete a field

67/96

Review: Atomic Changes II

Change Description

CFI Change definition of an instance field initializer

CSFI Change definition of a static field initializer

AI Add an empty instance initializer

DI Delete an empty instance initializer

CI Change definition of an instance initializer

ASI Add an empty static initializer

DSI Delete an empty static initializer

CSI Change definition of a static initializer

68/96

Example: Step 1

class A{void foo (){}int x;

}class B extends A{void foo(){ B.bar(); }static void bar(){ y=17; }static int y;

}class C extends A{void foo(){ x=18; }int z;

}

69/96

Example: Step 1

AMB.bar()

class A{void foo (){}int x;

}class B extends A{void foo(){ B.bar(); }static void bar(){ y=17; }static int y;

}class C extends A{void foo(){ x=18; }int z;

}

CMB.foo()

AFB.y

CMB.bar()

70/96

Example: Step 1

class A{void foo (){}int x;

}class B extends A{void foo(){ B.bar(); }static void bar(){ y=17; }static int y;

}class C extends A{void foo(){ x=18; }int z;

}

AFA.x

CMC.foo()

AMC.foo()

LCC, A.foo()

71/96

Example: Step 1

class A{void foo (){}int x;

}class B extends A{void foo(){ B.bar(); }static void bar(){ y=17; }static int y;

}class C extends A{void foo(){ x=18; }int z;

}

AFC.z

72/96

Example: atomic changesAM

B.bar()AFA.x

CMC.foo()

AMC.foo()

LCC, A.foo()

CMB.foo()

AFB.y

CMB.bar()

AFC.z

73/96

Recall: high-level concept

1) set of interdependent atomic changes A

2) call graph for each test in T

3) T´ T – potentially affected by cA

4) for tiT´ determine A´A that may have affected ti

74/96

Example: affected tests

A test is affected if its call graph contains:

1) node that corresponds to CM or DM, or

2) edge that corresponds to LC

75/96

Example: affected tests

76/96

Example: test1

void test1(){ A a = new A(); a.foo();}

<A, A.foo()>Tests.test1()

A.A() A.foo()

77/96

Example: test2

void test2(){ A a = new B(); a.foo();}

Tests.test2()<B, A.foo()>

B.B() B.foo()

A.A()

CMB.foo()

78/96

Example: test3

void test3(){ A a = new C(); a.foo();}

Tests.test3()<C, A.foo()>

C.C() A.foo()

A.A()

LCC, A.foo()

79/96

Recall: high-level concept

1) set of interdependent atomic changes A

2) call graph for each test in T

3) T´ T – potentially affected by cA

4) for tiT´ determine A´A that may have affected ti

80/96

Example: affecting changes

The set of atomic changes that affect a given affected test includes:

1) all atomic changes for nodes with AM and CM,

2) atomic changes in LC category, and

3) their transitively prerequisite atomic changes

81/96

Example: affecting changes

82/96

Example: AffectingChanges(test2)

Tests.test2()<B, A.foo()>

B.B() B.foo()

A.A() B.bar()

CMB.foo()

CMB.bar()

83/96

Example: AffectingChanges(test2)

AMB.bar()

CMB.foo()

AFB.y

CMB.bar()

84/96

Example

AFC.z

This atomic change does not affect any of the tests

=> additional tests are needed.

Remember the atomic change:

85/96

Snapshot

86/96

Chianti: Evaluation

Number of atomic changes between each pair of Daikon versions.

87/96

Chianti: Evaluation

Categorization of the atomic changes.

88/96

Chianti: Evaluation

Percentage of affected tests for each of the Daikon versions.

89/96

Chianti: Evaluation

Average percentage of affecting changes, per affected test.

90/96

Summary

PathImpact dynamic – uses dynamic traces online works on binariesChianti uses dynamic call graphs applicable for OOP (e.g. Java) determines affecting changes for every test

91/96

References[1] Chesley, O., Dolby, J., Ren, X., Ryder, B., Shah, F., and Tip, F. Chianti:

A prototype change impact analysis tool for Java. Tech. Rep. DCS-TR-533, Rutgers University Department of Computer Science, September 2003.

[2] Chesley, O., Ren, X., Ryder, B., Shah, F., and Tip, F. Chianti: a tool for change impact analysis of java programs. In Proceedings of the 19th annual ACM SIGPLAN Conference on Object-oriented programming, systems, languages, and applications, pp. 432448.

[3] Law, J., Rothermel, G. Whole program Path-Based dynamic impact analysis. In Proc. of the 25th International Conference on Software Engineering, May 03-10, 2003, Portland, Oregon.

[4] Ryder, B., Tip, F. Change impact analysis for object-oriented programs. In Proceedings of the 2001 ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering, pp. 4653, June 2001, Snowbird, Utah, United States.

92/96

Thanks for listening.

Questions?