Sung Kim and Michael D. Ernst{hunkim, mernst}@csail.mit.edu

CSAIL • MIT

Sung Kim and Michael D. Ernst{hunkim, mernst}@csail.mit.edu

CSAIL • MIT

Static analysis I can find bugs for

you!

Bug-finding tool

Warnings

Warning examples

Are warnings useful?

Are all warnings Are all warnings important to be important to be

fixed?fixed?

Does tools’ Does tools’ prioritization help prioritization help

totoidentify important identify important

ones?ones?

Buggy lines

Rev 100

……

Mark as a Mark as a buggybuggy lineline if a line is changed if a line is changed in future fixes.in future fixes.

Warning evaluation

Rev 100

50% of warnings are fixed later

50% of warnings are fixed later

0 5 10 15 20

Scarab

Lucene

Columba

% of warnings that are fixed later

Proj

ects

FindBugs

JLint

PMD

NOonly 3~18% are fixed!

Are warnings important?

0 5 10 15

Scarab

Lucene

Columba

% of warnings that are fixed later

Proj

ects

All

Tool's priority 1

NOTools’ priority 1

warnings are not better.

Does tools’ prioritization help?

Warning A

File ……

Warning B

Foo ……

Our solution: using change history

Reprioritization algorithmWarning A

File 1 ……

Score(A) +=non-fix-weight (0.1)

Score(A) +=fix-weight (0.9)

Fixed Fixed bug bug #3576#3576

Score(A) +=Non-fix-weight(0.1)

Score(A)=1.1

Warning B Foo ……

Score(B)=0

fix

Score(A)=0Score(A)=0.1Score(A)=1

Evaluation

Rev n/2Rev 1

Train warnings

Rev n/2-1 Rev n

Mark buggy lines

Rev n/2+1

Comparison• History-based warning prioritization• Tools’ warning prioritization

• Compare precision of top 100 warnings

Columba

Columba

Lucene

Lucene

Scarab

Scarab

Summary

Sung Kim and Michael D. Ernst{hunkim, mernst}@csail.mit.edu

CSAIL • MIT