extracting executable transformations from distilled code changes

Extracting Executable Transformations from

Distilled Code ChangesReinout Stevens

[email protected] @ReinoutStevens

Coen De Roover [email protected]

@oniroi

1

mailto:[email protected]

mailto:[email protected]?subject=

Context: Distilled Code Changes

B. Fluri, M. Würsch, M. Pinzger, and H. C. Gall.Change distilling: Tree differencing for fine-grained source code change extraction. Transactions on Software Engineering, 2007.

2

1. update(0,1) 2. move(int y = 0;) 3. insert(public int foo()…) 4. delete(int z = 0;)

Output:

00 public class Example {01 public Integer run(Integer x) {02 return x;03 }0405 public void test() {06 int x = 0;07 int y = 0;08 int z = 0;09 run(x);10 }11 }

00 public class Example {01 public Integer run(Integer x) {02 int y = 0;03 return x;04 }0506 public int foo() {07 return 42;08 }0910 public void test() {11 int x = 1;12 run(x);13 }14 }

Insert

Delete

Move

Update

00 public class Example {01 public Integer run(Integer x) {02 return x;03 }0405 public void test() {06 int x = 0;07 int y = 0;08 int z = 0;09 run(x);10 }11 }

00 public class Example {01 public Integer run(Integer x) {02 int y = 0;03 return x;04 }0506 public int foo() {07 return 42;08 }0910 public void test() {11 int x = 1;12 run(x);13 }14 }

Revision 1 Revision 2

Goal: Extracting Executable Transformations

3

Insert Insert Move Delete Update Insert Move Delete

Delete Insert Insert Move Insert Delete Update Delete

Update Move Insert Insert Update Update Move …

Naive Approach: Logic Queries over Changes

public class Example { int x = 0;}

public class Example { int y = 0;}

1. update(“x”, “y”)

1 (defn rename-field [changes] 2 (run* [?update] 3 (member ?update changes) 4 (change|update ?update) 5 (change|update-original ?update ?val|source) 6 (ast :SimpleName ?val|source) 7 (ast-parent ?val|source ?parent|source) 8 (ast :VariableDeclarationFragment ?parent|source) 9 (update-newval ?update ?val|target) 0 (ast :SimpleName ?val|target) 1 (name-name|different ?val|source ?val|target))

4


changes

look for an update

that modifies a field

its name

1. delete(“int x = 0;”)2. insert(“int aLongerName = 0;”)

1 (defn rename-field [changes] 2 (run* [?insert ?delete] 3 (fresh [?insert|source’ ?delete|original ?i-name ?d-name] 4 (member ?delete changes) 5 (member ?insert changes) 6 (== ?sequence (list ?insert ?delete)) 7 (change|insert ?insert) 8 (change|delete ?delete) 9 (insert-node ?insert ?insert|source’) 0 (ast :VariableDeclarationFragment ?insert|source’) 1 (delete-node ?delete ?delete|original) 2 (ast :VariableDeclarationFragment ?delete|source) 3 (has :name ?insert|source’ ?i-name) 4 (has :name ?delete|source ?d-name) 5 (name-name|different ?i-name ?d-name))) 5


public class Example { int aLongerName = 0;}

changes


look for a delete and insert

that introduce a fieldand remove a field with

a different name

Naive Approach: Logic Queries over Changes

Problem: Change Equivalence

1. update(“x”, “y”) 1. delete(“int x = 0;”)2. insert(“int y = 0;”)

1. insert(“int y = 0;”)2. delete(“int x = 0;”)

Possible Changes Sequences

public class Example { int x = 0; }

Revision 1

public class Example { int y = 0; }

Revision 2

6

One change distiller may produce different change sequences for the same source code transformation in different commits: • different change types • different length • different subjects of changes

7

Problem: Change Equivalence

or

or…

Problem Summary

Multiple change sequences implement the same source code transformation

It is not possible for a user to enumerate all the change sequences that implement a transformation in a query

8

9

initial state

sought-after state

1 (defn field-rename [esg] 2 (run* [?es] 3 (query-changes esg ?es [?orig-ast ?field] 4 (in-current-es [es ast] 5 (== ?orig-ast ast) 6 (ast-field ast ?field))

7 change->+

8 (in-current-es [es ast] 9 (fresh [?renamed ?new-name] 0 (ast-field ast ?renamed) 1 (ast-field|absent ast ?field) 2 (ast-field|absent ?orig-ast ?renamed)))))



Approach: Before&After AST Specification

10

public class Example { int x = 0; int y = 1;}

public class Example {}

Source AST Target AST

1 3

2 4

Regular Dependency

List Dependency

Change Dependency Graph

1. insert(int x, Example, Example, nil, :BodyDeclarations, 0)2. insert(int y, Example, Example, nil, :BodyDeclarations, 1)3. insert(0, nil, int x, nil, :Initializer, nil)4. insert(1, nil, int y, nil, :Initializer, nil)

Distilled Change Sequence



public class Example { int x;}


public class Example { int x = 0; int y;}

public class Example { int x; int y;}

public class Example { int x; int y = 1;}

public class Example { int y;}


1

2

3

2

1

4

2

3 4

1

34

Evolution State Graph

Implementation Overview

11

1 3

2 4

Regular Dependency

List Dependency


Solution: Change Dependency Graph

Insert Dependency List Dependency

Source SourceTarget Target

Move Dependency

Source Targetα

12

Solution: Change Dependency Graph

13











1

2

3

2

1

4

2

3 4

1

34

Solution: Evolution State Graph

• Detect instances of refactorings in open-source projects using a single evolution query per refactoring

• Ensure returned change sequences are minimal and executable

• Compare our approach with the naive approach

K. Prete, N. Rachatasumrit, N. Sudan, and M. Kim, “Template-based reconstruction of complex refactorings,” in Proc. of the 2010 Int. Conf. on Software Maintenance (ICSM10)

E. Murphy-Hill, C. Parnin, and A. P. Black, “How we refactor, and how we know it,” Transactions on Software Engineering, vol. 38, pp. 5–18, 2012.

14

Evaluation: Outline

15

ChangeNodes Distiller

Known Refactorings

Magic Constant

Field Rename

Unused Method










1

2

3

2

1

4

2

3 4

1

34

Evolution State Graph

Change Dependency Graph

1 3

2 4

Regular Dependency

List Dependency

Evolution Query

1 (query-changes esg ?es2 [?absent ?method …]3 (in-current-es [es ast]4 (== ast ?absent)5 (ast-method ast ?method) 6 (child+ ?method ?literal) 7 (literal-value ?literal ?value))8 change->*9 (in-current-es [es ast]0 (ast-ast-field|introduced ?absent ast ?field)1 (field-value|initialized ?field ?value)2 (ast-method-method|corresponding …) 3 (child+ ?cmethod ?field-access)4 (field-name|accessed ?field ?field-access)))

Minimal Executable Code Transformation

Insert Move Insert

Code Rev1Code Rev2

Insert Move …

Evaluation: Outline

16

1 1 public class Jar extends Zip {2 2 private static final String INDEX_NAME="META-INF/INDEX.LIST";

3 + private static final String MANIFEST_NAME="META-INF/MANIFEST.MF";3 4 private Manifest configuredManifest;4 5 private Manifest savedConfiguredManifest;5 6 6 7 protected void zipFile( ) throws IOException {7 - if ("META-INF/MANIFEST.MF".equalsIgnoreCase(vPath)) {

8 + if (MANIFEST_NAME.equalsIgnoreCase(vPath)) {8 9 if (!doubleFilePass || (doubleFilePass && skipWriting)) {9 10 filesetManifest(fromArchive,is);

10 11 }11 12 }12 13 else {13 14 super.zipFile(is,zOut,vPath,lastModified,fromArchive,mode);14 15 }15 16 }16 17 }

Evaluation: Replace Magic Constant

17

1 (query-changes esg ?es 2 [?absent ?method ?literal value ?cmethod ?field ?field-access] 3 (in-current-es [es ast] 4 (== ast ?absent) 5 (ast-method ast ?method) 6 (child+ ?method ?literal) 7 (literal-value ?literal ?value)) 8 change->* 9 (in-current-es [es ast] 0 (ast-ast-field|introduced ?absent ast ?field) 1 (field-value|initialized ?field ?value) 2 (ast-method-method|corresponding ast ?method ?cmethod) 3 (child+ ?cmethod ?field-access) 4 (field-name|accessed ?field ?field-access)))

protected void zipFile(…) { if (“META-INF/MANIFEST.MF” .equalsIgnoreCase(vPath)) { …

private String MANIFEST_NAME = “META-INF/MANIFEST.MF"; … protected void zipFile(…) { if (MANIFEST_NAME .equalsIgnoreCase(vPath)) { …

Evaluation: Replace Magic Constant

18

Legend

insert

delete

move

solution

dependency

update

• Total Changes: 74 • Solution Length Our Approach: 4 • Solution Length Replaying Sequence: 23

Evaluation: Inspecting the Solution

19

Evaluation: Result SummaryN

umbe

r of C

hang

es

0

50

100

150

200

250

300

350

RefactoringConstant Constant Constant Constant Constant Constant Method Method Method Field Field Field Field

61513

3

39

126551048

213

24

57

9

264

1214

118

199

1000

23

82

221

27

63

10

291

511

25

149

245

1244

74

202

Total Changes Solution Length Replaying Sequence Solution Length Our Approach

Conclusion• The change equivalence problem renders specifying

a sought-after code transformation in terms of changes difficult

• Our approach supports specifying code transformations using before&after states in logic queries

• Solutions to our queries are a minimal, executable subsequence of changes that implements the sought-after transformation

20

Discussion

Do ad-hoc solutions to the change equivalence problem invalidate existing MSR studies?

21

extracting executable transformations from distilled code changes

Presentations & Public Speaking