oss java analysis - what you might be missing
DESCRIPTION
We think FindBugs is a great tool for finding coding style and best practice types of issues, so we conducted a little experiment a few months ago. We analyzed Jenkins core code with both FindBugs and Coverity. These slides provide a high-level summary of our results.TRANSCRIPT
OSS Java AnalysisWhat You Might Be Missing
Copyright 2013, Coverity, Inc.2
Overview
• Open Source Static Analysis Options for Java
• Case Study: Analysis of Jenkins Project
• Detailed Findings
Copyright 2013, Coverity, Inc.
Consistent Use = More Maintainable Code
Numerous Static Analysis Options for Java
3
CheckstyleDefines and enforces coding standards
Finds common programming flaws and duplicated code
Finds bugs
Finds coding errors, ensures proper testing coverage and more
Copyright 2013, Coverity, Inc.4
• Will you use it?• Does it find issues that you care about? Does it fit into
your team’s workflow? Is it worth the effort?
• Is it actionable?• Does it just find issues, or does it help you understand how
and where to fix them?
• Is it accurate?• Will you waste significant time on false positives? Are
results consistent across builds?
• Is it integrated? • Do you have to go out of your way to use it, or does it bring
the problems to you?
Key Considerations
Copyright 2013, Coverity, Inc.5
Case Study: Jenkins
Coverity Development Testing Platform and FindBugs
Copyright 2013, Coverity, Inc.6
Background
• Analyzed Jenkins 1.496 core code using up-to-date Coverity and FindBugs (as of Dec 2012)
• Both solutions focus on finding defects, have IDE and command line interfaces
FindBugs
• Often used automatically in Eclipse
• Extensible tool
• Management capabilities focused on individual developers
Coverity Platform
• Most often run in Eclipse or via automated build
• Extensible tool, open platform
• Sophisticated management capabilities addressing the needs of developers, teams & organizations
Copyright 2013, Coverity, Inc.7
Little OverlapOnly 28 relevant issues found
by both solutions
Copyright 2013, Coverity, Inc.
Defect Summary (Real Bugs Only)
8
Type Coverity FindBugs Both
Resource leaks 86 12 13
Security problems 1 0 0
Concurrency problems 22 10 9High Severity Subtotal 109 22 22
Coding Standards, Best Practices, Other 88 605 6
Total Bugs 197 627 28
Coverity
FindBugs
Impact Distribution
High Med-Low
Each solution found unique issues (neither is sufficient to replace the other)
Copyright 2013, Coverity, Inc.9
Here is a resource leak identified by FindBugs:
• Identifies the problem and general location• But there’s no indication of the affected
object/variable, the relevant code path or events• Developers need to figure it out on their own
• Not a big deal in this simple code• Complex code will be a pain!
Example: FindBugs Resource Leak
A leak might affect something on this line
Copyright 2013, Coverity, Inc.10
Here is a leak identified by Coverity:
The object will continue to hold resources until the garbage collector eventually finalizes it—which may never happen.
Example: Coverity Resource Leak
“listener” allocated. Implements Closeable, so it needs to be closed
“listener” used, but still needs to be closed
Leak: no longer usable, and never
closed
Copyright 2013, Coverity, Inc.11
Example: Coverity Race ConditionThis is an HTTP request
handler
“o” is associated with session
Another thread may modify the session
Session attribute is deleted. The right one?
Erroneous code path identified
You may delete the wrong key! Good luck finding this manually!
Copyright 2013, Coverity, Inc.12
Example: Coverity Null DereferencePassing null to function
Dereferenced two layers deep
You should check for null before or during the call to rebuild()
Copyright 2013, Coverity, Inc.13
Conclusion
• Different analysis tools often find different (important and complementary) issues
• Use multiple tools that find issues important to you
• Avoid tools that take too long to analyze, require too much manual work or have too many false positives
Copyright 2013 Coverity, Inc.