static code analysis and code review - soberit-software business
Post on 03-Feb-2022
7 Views
Preview:
TRANSCRIPT
HELSINKI UNIVERSITY OF TECHNOLOGY
T-76.5613 Software testing and quality assurance
Static code analysis and code review
Mika MäntyläSoberIT
2
Different static methods
Mathematical verificationProve the program correctness with mathematical methods
Code reviewsRead the source code in order to find defects and quality issues
Tool based static analysis“Automated code inspections”
Subjective design and code quality indicators and improvement
3
Introduction – Static versus Systematic testing
Systematic software testingRequires a large set of tests
Each test tend to discover only one or few faults
Cannot be used in early stages
Static methods Does not require execution
Each error can be considered in isolation No problems where you don't know whether anomalies are from new fault or side-effects of existing one
Can consider other quality attributes as maintainability, reusability, security
4
Contents
Static methodsMathematical verification
Code review defects
Tool based analysis
Subjective design and code quality indicators and improvement
5
Mathematical verification
Research in this area has been going on 40 years This can offer error free programs
Assuming that programs specification is correct
Mathematical formal verification requires Program itself must be formally specifiedSemantics of programming language must be formally defined
The formal semantics of a language is given by a mathematical model to represent the possible computations described by the languageSyntax is always formally definedSemantics of most programming languages are not formally defined
— We must use only a subset of language
6
Mathematical verification cont’d
Hoare’s triplets are example of mathematical verification {P} S {Q}
P is a precondition, S is program statements, Q is a post conditionExample { x = 0 } x := x + 2 { x ≥ 1 },
Mathematical verification is too time consuming and expensive for many programs
Can be used in systems or parts of systems that are safety critical require high reliabilityE.g. telecom systems, space-shuttle software, embedded systems
ISDN-protocol bugs enabled free calls -> discovered with mathematical verification
Automated mathematical verification, i.e. model checkingUsed to verify and debug systems specified with temporal logic
Software needs to be abstracted for the language of the model checker
State explosion problem Algorithms: symbolic algorithms, partial order reduction
7
Design-By-Contract (DBC)
DBC is practical application of Hoare’s tripletsUses contract as program specifications
Each routine has “two contracts”, pre and post conditionsCaller must satisfy preconditionsThe routine must satisfy post conditions
Inheritance and polymorphism is also supported through “subcontracting”
Pre condition may be kept or weakenPost condition may be kept or strengthen
Classes can have contracts. E.g. invariants that must hold for all instances
Program itself never verifies the contract conditionsFail hard
Contract code not used in production
8
Design-By-Contract (DBC) – Example Routine
put (x: ELEMENT; key: STRING) is
-- Insert x so that it will be retrievable through key.
require
count <= capacity
not key.empty
do
... Some insertion algorithm ...
ensure
has (x)
item (key) = x
count = old count + 1
end
9
Design-By-Contract (DBC)
Benefits:Errors are immediately detected and located -> Better error handlingProgrammers must specify their intentions -> Better software design and structureSpecified intentions serve as documentation -> Better documentation
DrawbacksLack programming language support
Eiffel is the only widely used language with built in support for DBCTools exists for many languages C++ and Java
— Wikipedia has good list of tools
Assertions can be used in DBC fashion— Discipline required
10
CleanRoom
CleanRoom is development approach utilizing statistic / mathematical techniques.
Bugs are not allowed to enter the softwareCleanRoom has
Incremental developmentPrograms are seen as mathematical functions
Stepwise refinement (blackbox->state-boxes->clear-boxes (procedural design))
Formal specification of the programsStatistic verification of program correctnessIncrement components are not executed or tested in any wayIntegrated system is not tested to find defects but to ensure reliability with operational profile
Operational profile = Probable pattern of the usage of the softwareAll possible usage patterns definedDefined based on box structure specifications of system function
— plus information on system usage probabilities obtained from prospective users, actual usage of prior versions, etc.
11
CleanRoom Cases
Ericsson OS-32 (350Kloc) operating system project70% improvement in development productivity, 100% improvement in testing productivity, Testing error rate of 1.0 errors per KLOC (all errors found in all testing) Project that contributed the most to the company in 1993
NASA Satellite Control (40Kloc)80% improvement in productivity
780 LOC/person month50% improvement in quality4,5 errors per KLOC
Other ClearRoom project domainsFlight control softwareLan Software Real-time embedded tape drive softwareMedical device software (2004)
12
Summary of Mathematical Methods
Mathematical verification is expensive, but can be used when high reliability is required
Design-by-Contract is a feasible optionTechnique based on pre and post conditions
CleanRoom is development approach utilizing statistical techniques.
13
Contents
Static methodsMathematical verification
Code review defects
Tool based analysis
Subjective design and code quality indicators and improvement
14
What is a defect?
No universally adopted definition
Three approaches in code review contextanomalies that will cause a system failure
incorrect code, which may (or may not) result in a system failure
all deviations from quality are defects
In this study the last definition is used
15
Code Review - Defect types
0,00 %
10,00 %
20,00 %
30,00 %
40,00 %
50,00 %
60,00 %
70,00 %
80,00 %
90,00 %
Evolvability "True defects" FP
Industry case QIndustry case SStudents
16
What are Evolvability Defects? (1/3)
Not just simple issues dealing with layout
Three types of evolvability defectsVisual Representation
Documentation
Structure
Visual Representation has roughly 10% share of evolvability defects
This number was consistent in the three data sets (IQ 10%, Stu 11%, IS 12%)
Examples of VR: Indentation, Blank line usage, Grouping...
17
What are Evolvability Defects? (2/3)
Documentation - Communicates the intent of the codeTextual
Code comments
Naming of code elements
Debuggin information
Supported by languageDeclare immutable (keyword: const, final)
Visibility of code elements (keyword: static, private)
Share of documentation fluctuats between studiesIQ 35%, Stu 46%
Textual documentation: IQ 24%, Stu 34%, IS 52%
18
What are Evolvability Defects? (3/3)
StructureRedundant
Dead code, Duplicate code
Semantic dead code, Semantic duplicate code
Re-organize Move Functionality, Long sub-routine
Alternative ApproachChange function, Magic number, Create new functionality, Use Exceptions
Share of structure fluctuats between studiesIQ 55%, Stu 43%, IS 26%
19
What are functional defects (1/2)?
Data & ResourceVariable initialization
Memory management
Data & Resource manipulation
SharesIQ 27.7%, Stu 2%, IC 15.3%, IK 10%-13%,
ChecksCheck return value
Valid value returned
No error code given
Check memory allocation
Check variableParameters are valid
Checking loop variables
Check pointer
SharesIQ 16,9%, Stu 56%, IC 12.6%, IK 8%-13%,
20
What are functional defects (2/2)?
InterfaceDefect made when interacting with other parts of the software or with code library
Incorrect or missign function call
Incorrect or missignparameter
SharesIQ 16,9%, Stu 2%, IC 12.6%, IK 8%-13%,
LogicComparison
Computation
Wrong location
Off by one
Algorithm / Performance
SharesIQ 14.5%, Stu 37.5%, IC 21.6%, IK 4%-5%,
21
Summary of Code review data
What is a defect
1:3 ratio funtional defects: evolvability defects
Evolvability defectVisual representation (layout)
Documentation (Naming & Commenting)
Structure
Functional defectsData & Resource
Checks
Interface
Logic
22
Contents
Static methodsMathematical verification
Code inspections
Tool based static analysis
Subjective design and code quality indicators and improvement
23
Summary of Code review defects
60% are evolvability defectsDocumentation
NamingCommenting
Visual representation (layout)Constant 10% share in studies
StructureRefundantRe-organizeAlternative Approach (re-think)
20% are functional defectsData&ResourceChecksInterfaceLogic
24
Contents
Static methodsMathematical verification
Code review defects
Tool based analysis
Subjective design and code quality indicators and improvement
25
Tool Based Static analysis
“Automated code inspections”Descended from compiler technology
A compiler statically analyses the code and knows a lot about it, e.g. Variable usage; finds syntax faultsStatic analysis tools extend this knowledgeReverse-engineering tools also extend compiler knowledge
But they have different target e.g. Source Code -> UML
Goals:Check violation of code
Documentation Visual Representation
Ensure the system design and structure qualityDetect defects
26
Layout & Visual Representation
Visual representation has great impact to program comprehension
Usage of white space, indentation and other layout issues
Different layout styles: K&R Style, Allman Style, GNU Style
What can static analysis do to enforceLayout / Visual representation
Layout can be completely enforced
Pretty printers can even restore the wanted layout
27
Documenting Code (Naming)
Descriptive naming greatly increase the program comprehensionDifferent naming styles
Sun’s Java Coding ConventionsClass names are nouns Methods are verbs Variable names should reflect the variables intended use
Hungarian naming style (“Microsoft Style”)Identifiers prefixes state their type lprgst -> long pointer to array of zero terminated strings
Figuring good names right away is difficultName should be improved as better one is discoveredRename is most used “refactoring” feature in Eclipse
28
Documenting Code (Commenting)
Studies have shown that too much commenting is harmful
Commenting is often abusedComment are used as deodorant
— Using comments to explain a complex code rather than improving the code
Comments re-stating the method name— Maybe caused by outside pressure to comment all code
Commenting should be used wiselyProvide extra information not obvious from the naming
“Check needed because of bug in Java virtual machine 1.4.1”
“Used by X”
“If parameter is <0 this disables all buttons”
29
Static Analysis – Documenting
What can static analysis do to enforceDocumenting
Naming— Regular expression checks
— E.g int’s are lower case, starting with ‘i’, length< 6 letters
Commenting— Check that classes, variables, and methods have comments and
— Existence of different java doc tags can be checked (@return, @param)
— No tool can say anything about the sanity of the comments
Tool CheckStyle http://checkstyle.sourceforge.net/
Peer pressure and standards are still among the most effective tools
30
Structure
Does the following look acceptable
The method had over 500 lines of code and cyclocmaticcomplexity over 100
Writing for example a unit test or a small description of the methods behavior is impossible
31
Structure
The software structure affects many software quality attributesMaintainability, portability, reliability, and functionality
Reasons for controlling internal software structureThe future development depends on the current structure Agreement on what is acceptable software structure
What is acceptable size for method, how big can class be?
To stop bad programmersAnecdotal evidence suggest that some programmers actually do more harm than good
To prevent side effects of software evolution. Laws of software evolution from 1970’s by Lehman & Belady at IBM with OS/360 state that
— Software which is used in a real-world environment must change or become less and less useful in that environment
— As an evolving program changes, its structure becomes more complex, unless active efforts are made to avoid this phenomenon
32
Case: Construction Quality Measurement at HP
Purpose Create construction quality (maintainability) metric for software developers in the trenches
Measure combined three dimension of maintainability 1) control structure, 2) information structure, 3) typography, naming, commenting
Polynomial metric equationProcedure quality:
Equation was adjusted against developers opinions
Analysis assisted HP in Buy-versus-build decision, Controlling software entropy over several versionsIdentify change prone subcomponentsFinding targets and assessing the efforts of reengineering
))46,2sin(50()ln(2,16
)'(23,0)ln(2,5171
perCMaveLOC
gaveVaveVol
××+×
−×−×−
33
Summary: Static Analysis – Structure
Code metrics are used to control internal software qualityBased on change rate data: Cyclomatic complexity not >14
Code metrics can protect software from the harmful side-effects of software evolutionBy measuring you send a message of it’s importance Problems
You should to define your own limitsHistorical data needed
If you want to use hard evidence to base the measures
Lack qualitative elements“You may replace that entire function with one function call”“Measures don’t really tell whether it is well programmed”
Programs: CCCC, Metrics (Eclipse plugin)To measure duplication: Same and CloneFinder (commercial)
34
Static analysis – Error detection
Static analysis can also detect anomalies in the code unreachable code,
undeclared variables,
parameter type mismatches,
uncalled functions & procedures,
array bound violations
thread synchronization problems,
misuse of variables
35
Example: Errors
void input_reporter(){
int i;
while (i == 0)
{
sleep();
i = doWeHaveInput();
}
weHaveInput(i);
}
Where is the bug?
Is this even a bug?C -> Yes
Java -> No
36
Data flow analysis
Study of program variablesVariable defined where a value is stored into it
Variable used where the stored value is accessed
Variable is undefined before it is defined or when it goes out of scope
X = y + z
IF a>b THEN read(S)
x is defined, y and z are used
a and b are used, S is defined
37
Static analysis – Defect detection
Static analysis can detect anomalies in the code Static analysis can find real and difficult to spot defects
For example Microsoft has but such tools as part of their build process
Whenever warning is found it is treated as a build breakerTheir developers are using such tools in their desktops
Problems of static defect detectorsNoise = “messages people don’t care about” (not just “bogus”messages)
Too much noise — people won’t use the tool— missing all the defects
Need to make sure that all error messages are considered important
38
Summary on tool based static analysis
Static analysis is “Automated code inspections”Descended from compiler technologyGoal:
Check problems of code Documentation (Naming & Documenation)
— Important but difficult to automate
Layout / Visual representation— This is not currently so big problem
Ensure the system design and structure qualityCan good or poor design be measured? Yes and NoPrevent the code erosion caused by software evolution
Detect defectsAutomatic detection can find many hard to spot defects
Problems: NoisePeople can starting ignoring the messages and miss all defects detected
Need to make sure that all error messages are considered important
Tools often can focus one or more of these viewpointsProgramming errors (Lint, Findbugs)Documentation (Checkstyle)Layout (IDE’s auto reformat)Structure (Code metrics tools)
39
Contents
Static methodsMathematical verification
Code inspections
Tool based static analysis
Subjective design and code quality indicators and improvement
41
Pattern History
Design PatternsRepresent reusable designs
Strategy, Command, Singleton, etc
HistoryBased on pattern languages which focused on traditional architecture (1977)Rediscovered by software people 1987 and published by GoF in 1994
Motivation Capsulate years of industry experience on software development
Anti-PatternsRepresent frequently occurring problemsHistory
Have been around longer than Design PatternsFred Brooks and The Mythical Man-Month 1979
— “Adding manpower to a late software project makes it later”Term Anti-Pattern appeared soon after the book by GoF in 1994
MotivationLearning from mistakes
42
Anti-Patterns & Bad Code Smells
Anti-Pattern cover wider range of topics
DevelopmentSoftware Development
ArchitecturalSystems Architecture
ManagerialOrganization and Process
Bad code smellsAre in fact development level anti-patterns
43
Anti-Pattern examples – Golden Hammer
Also know as: Old Yeller, Head-in-the sand
Description: High knowledge on particular solution or vendor product
This knowledge is applied everywhere
Causes:Several successes with the particular approach
Large investment for the product
Isolated Group
44
Anti-Pattern examples – One Size Fits All
Also know as: Silver Bullet Software Process
Description: Software process does not fit in the business environment or lifecycle
Too bureaucratic process for small project
Own process created under the official one
Causes:IT management wants to have same process for all projects to avoid overrunning time and budget
45
Anti-Patterns & Bad Code Smells
Anti-Pattern cover wider range of topics
DevelopmentSoftware Development
ArchitecturalSystems Architecture
ManagerialOrganization and Process
Bad code smellsAre in fact development level anti-patterns
46
When to Refactor
Bad smells in the code tell us when to apply refactorings
Smells are structures that indicate bad software designIdea introduced by Fowler and Beck (2000)
Of course, the list of bad smells can never be complete
They are introduce with the set of refactoring that can remove the smells
Why are they called smells?Fowler & Beck think that when it comes to refactoring decision: “no set of metrics rivals informed human intuition”
47
Bad Code Smells - Taxonomy
BloatersWhen something is too large
Examples: Long Method, Large Class, Long Parameter List, These smells likely grow little bit a time
Hopefully nobody designs e.g. Long Methods
Object-Orientation abusersObject Oriented concept not fully understood and utilized
Examples: Switch statements, Temporary Field,, Alternative Classes with Different Interfaces
Change PreventersThese smells make changing the system unnecessarily difficult
Example: Shotgun surgeryViolate principle one external change should effect to one classonly
Change the database from Oracle to SQLServer
48
Bad Code Smells - Taxonomy Cont’d
DispensablesAll code needs effort to understand and maintain
If code is not used or redundant it needs to be removedExamples: Duplicate & Dead code, Speculative Generality
Couplers Low coupling between objects/classes is one the desirable goals of OO software
Examples: Feature Envy, Message Chains
Too much delegation (= reduction of coupling) can be bad as well
Examples: MiddleMan
49
Refactoring Example – Switch statement
Class Engine{ …
int cylinderCount(){
switch (type){
case FAMILY_CAR_ENGINE:
return getBaseSylinderNumber();
case EXECUTIVE_CAR_ENGINE:
return getBaseSylinderNumber() * 2;
case RACE_CAR_ENGINE:
return 8;
}
} +cylinderCount() : int
Engine
+cylinderCount() : int
FamilyCarEngine
+cylinderCount() : int
ExecutiveCarEngine
+cylinderCount() : int
RaceCarEngine
50
Summary subjective design and code quality indicators
Refactoring is improving the code structure with out changing it’s external behavior
Bad code smells and AntiPatterns are subjective quality indicators
They cannot be exactly measured and human mind provides the final judgment
It is possible that people have quite different opinions on them
51
ReferencesAnon (CheckStyle) http://checkstyle.sourceforge.net/Anon (Eiffel Software Inc.) Building bug-free O-O software: An introduction to Design by ContractTMhttp://archive.eiffel.com/doc/manuals/technology/contract/Bansiya, J., David, C.G. 2002, "A Hierarchical Model for Object-Oriented Design Quality", Software Engineering, IEEE Transactions on, vol. 28, no. 1, pp. 4-17.Barnard, J., Price, A. 1994, "Managing code inspection information", Software, IEEE, vol. 11, no. 2, pp. 59-69. Brown, W. J., Malveau, R. C., McCormick, H. W., & Mowbray, T. J. 1998, AntiPatternsRefactoring Software, Architectures, and Projects in Crisis Wiley, New York..Fowler, M. 2000, Refactoring: Improving the Design of Existing Code Addison-Wesley, Canada. Coleman, D., Ash, D., Lowther, B. & Oman, P.W. 1994, "Using Metrics to Evaluate Software System Maintainability", Computer, vol. 27, no. 8, pp. 44-49.Dunsmore, A., Roper, M. & Wood, M. 2003, "The development and evaluation of three diverse techniques for object-oriented code inspection", Software Engineering, IEEE Transactions on, vol. 29, no. 8, pp. 677-686. Dunsmore, A., Roper, M. & Wood, M. 2003, "Practical code inspection techniques for object-oriented systems: an experimental comparison", Software, IEEE, vol. 20, no. 4, pp. 21-29.Enseling Oliver, iContract: Design by Contract in Java, http://www.javaworld.com/javaworld/jw-02-2001/jw-0216-cooltools.htmlHoare, C.A.R, 1969 ”An Axiomatic Basis for Computer Programming”, Communications of the ACM 12,Laitenberger,Oliver; DeBaud,Jean-Marc, An encompassing life cycle centric survey of software inspection, J.Syst.Software, 2000, 50, 1, 5-31Li, W., Henry, S.M. 1993, "Object-Oriented Metrics that Predict Maintainability", Journal of Systems and Software, vol. 23, no. 2, pp. 111-122.Linger,Richard C. 1993, “Cleanroom software engineering for zero-defect software”, 2-13, IEEE Computer Society PressM. Lowry, M. Boyd, and D. Kulkarn Towards a Theory for Integration of Mathematical Verification and Empirical Testing p. 322, 13th IEEE International Conference on Automated Software Engineering (ASE'98), 1998. Martin, John C. ”Formal methods software engineering for the CARA system”, International-Journal-on-Software-Tools-for-Technology-Transfer. 2004; 5(4): 301-7McArthy J., 1962, “Towards a mathematical science of computation”, In Proceedings of IFIP 62, Munich, 21,8McConnell, S. 1993. Code complete. Redmond, Washington, USA: Microsoft Press. Porter, A.A., Siy, H.P., Toman, C.A. & Votta, L.G. 1997, "An experiment to assess the cost-benefits of code inspections in large scale software development", Software Engineering, IEEE Transactions on, vol. 23, no. 6, pp. 329-346. Pincus Jon 2002 “PREfix, PREfast, and other tools and technologies“ presentation at ICSM 2002 MontrealRoush, W. 2003, "Writing Software Right", MIT Technology Review, vol. 106, no. 3, pp. 26-28 Shull, F., et al. 2002. What we have learned about fighting defects. , 249-258., Software Metrics, 2002.Proceedings.Eighth IEEE Symposium on SEI, ClearnRoom, http://www.sei.cmu.edu/str/descriptions/cleanroom_body.htmlSethi, R. 1996, Programming Languages - Concepts and Constructs, Addison & Wesley. Sommerville, I. 2001, Software Engineering, Addison-Wesley / Pearson, Reading, MA, USA.
top related