![Page 1: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/1.jpg)
Software Engineering Prof. Dr. Bertrand Meyer
Dr. Manuel Oriol
Dr. Bernd Schoeller
Chair of Software Engineering
Lectures 17-18: Estimation Techniques and Software Metrics
![Page 2: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/2.jpg)
Today
• We present software metrics How they are calculated What they represent Why they are used Their limitations
• We show evaluation techniques How to use them What to deduce from them
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 3: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/3.jpg)
SOFTWARE METRICS
![Page 4: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/4.jpg)
Metrics?
• What are the metrics you know? height
weight
length
depth
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 5: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/5.jpg)
Definition of Metrics
• A system or standard of measurement (Meriam-Webster Dictionary)
-> A metric is a value expressed with units associated to a property of an object, a person, a process…
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 6: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/6.jpg)
Software Metrics
• A system or standard of measurement on software
-> A metric is a value expressed with units associated to a software
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 7: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/7.jpg)
Why use metrics?
• Metrics give an indication of how the system is
• They are important because they allow people to compare systems and guess what they can achieve
• Example Alice is 1.8m tall and Bob is 1.6m tall
Alice is taller than Bob
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 8: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/8.jpg)
Size-related metrics
• Lines of Code (Locs)
• Number of classes or header files
• Number of methods per class
• Number of attributes per class
• Size of compiled code
• Memory footprint
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 9: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/9.jpg)
Lines of Code
• The most known (and used!) metrics to assess the general size of a project. Usually called locs. Can also be called SLOC (Source line of code). Sometimes: 1 klocs = 1000 locs, 1 mlocs = 1000000 locs
• How to count it: Remove all lines devoted to comments and blank lines
then count the number of carriage return (‘\n’ or ‘%N’) • Pitfalls: It is not completely accurate, e.g.
Software Engineering, lecture: Estimation Techniques and Software Metrics
print (“Hello World”); Result := 2
print (“Hello World”); Result := 2
vs
![Page 10: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/10.jpg)
Examples
• STRING_32 has 2766 locs
• EiffelBase has 65416 locs (in 251 classes)
• Apache has around 89000 locs
• EiffelStudio has 3millions locs
• Windows XP has 40 millions locs
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 11: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/11.jpg)
Evolution in time: Windows NT
0
10
20
30
40
50
60
1992 1994 1996 1998 2000 2002 2004 2006
Windows NT Size Evolution
Mlocs Windows
NT 3.1
Software Engineering, lecture: Estimation Techniques and Software Metrics
NT 3.5 NT 4.0
Win 2k
Win XP
Win Vista
Source for Data: Wikipedia, SLOC
![Page 12: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/12.jpg)
Evolution in time: vsftpd
Software Engineering, lecture: Estimation Techniques and Software Metrics
Source: Neamtiu, Hicks, Stoyle, Oriol, (PLDI 2006)
![Page 13: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/13.jpg)
Evolution in time: sshd
Software Engineering, lecture: Estimation Techniques and Software Metrics
Source: Neamtiu, Hicks, Stoyle, Oriol, (PLDI 2006)
![Page 14: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/14.jpg)
Number of classes or header files
• Another very used one
• How to count it: Count classes (inner classes are not counted)
• Pitfalls: It is very dependent on the programming style E.g. C programmers have more difficulties with cutting
application in different pieces with separate headers
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 15: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/15.jpg)
Examples
• EiffelBase (Estudio 6.1) has 251 classes
• XCode for MacOSX Leopard has 14040 header files
• Java 6 SDK has 7556 classes
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 16: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/16.jpg)
Number of methods per class
• Used to devise how big classes are
• How to count it: Count routines in classes
• Pitfalls: The number of routines is more adapted to view the
complexity of a class with a client point of view or with a maintenance point of view.
Providers tend to consider that a higher number indicates how complete the class is
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 17: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/17.jpg)
Examples
• EiffelBase (Eiffel Studio 6.1) has 58 routines per class on average (14541 routines in total, 251 classes)
• Gobo (in EiffelStudio 6.1) has 9.6 routine per class on average (19214 routines in total, 1993 classes)
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 18: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/18.jpg)
Number of attributes per class
• Used to devise how complex the data is
• How to count it: Count attributes in classes In C: number of fields per struct
• Pitfalls: People tend to think that a class with a higher number
of attributes contains more complex information: this is plain wrong!
e.g. a class with 10 integer attributes is easier to understand than a class with 4 linked lists
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 19: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/19.jpg)
Examples
• EiffelBase has 2.3 attribute per class (578 attributes, 251 classes)
• Gobo (in EiffelStudio 6.1) has less than one attribute per class on average (1237 attribute in total, 1993 classes)
• Time cluster (in EiffelStudio 6.1) has 2.18 attribute on average per class (72 attributes over 33 classes)
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 20: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/20.jpg)
Evolution in time: additions and deletions of attributes
Software Engineering, lecture: Estimation Techniques and Software Metrics Source: Neamtiu, Hicks, Foster, (MSR'05)
![Page 21: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/21.jpg)
Size of compiled code
• Used to devise how big the code to load in memory is (can be used to devise startup time)
• How to count it: Use GUI Under Unix: ls –al, size
• Pitfalls: Compiled code is not really representative of the
actual size in memory, but it is easier to compute Architectures make these numbers change
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 22: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/22.jpg)
Examples
• Firefox is 48.2 MB big (on MacOS X Leopard, including images and all)
• EiffelStudio 6.1 is 72 MB big (on MacOS X Leopard, including images and all)
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 23: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/23.jpg)
Memory footprint/Performance
• Used to devise how big the code loaded in memory is, or how fast the code runs
• How to count it: Measure at least 11 times the memory footprint (this
can vary) Use GUI tools Under Unix: pmap
• Pitfalls: Taking one measure is clearly NOT acceptable
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 24: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/24.jpg)
Examples
• EiffelStudio 6.1: 157 MB
Software Engineering, lecture: Estimation Techniques and Software Metrics Source: Neamtiu, Hicks, Stoyle, Oriol, (PLDI 2006)
![Page 25: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/25.jpg)
Quality/Complexity metrics
• Cyclomatic Complexity
• Number of states
• Bugs per lines of code
• Coupling metrics
• Inheritance metrics
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 26: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/26.jpg)
Cyclomatic Complexity
• The metric measures the number of possible paths in a program (in a subroutine)
• How to count it: M=E-N+2P (M= Cyclomatic Complexity, E number of edges in
control-flow graph, N number of nodes of the graph, P number of entries)
M = 1+ number of if, from…until, select • Pitfalls:
Difficult to understand for non-specialists What does it really measure?
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 27: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/27.jpg)
Example
Software Engineering, lecture 2: Fundamentals 2
a:=1 b:=a+2
b>3
Result:=b Result:=a b:=a
b:=a
yes no
E=5, N=5, P=1
M=2
![Page 28: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/28.jpg)
Number of distinct states
• The metric measures the number of states in a model of the program
• How to count it: Have a model (sic!) and count the number of nodes
• Pitfalls: Model generally non-trivial to make
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 29: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/29.jpg)
Example
Software Engineering, lecture: Estimation Techniques and Software Metrics
open closed
Distinct states: 2
![Page 30: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/30.jpg)
Bugs per line of code
• The metric measures the number of bugs found per line of code
• How to count it: Count the number of bugs divide it by the number of
lines of code
• Pitfalls: What if the program is not sufficiently tested? How to compare two different values? Is it relevant?
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 31: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/31.jpg)
Examples
Software Engineering, lecture: Estimation Techniques and Software Metrics
• Firefox 1.5: .000077 (locs: 2’172’520, open bugs: 169)
• Apache Http 2: .00065 (locs: 89000, open bugs: 58)
![Page 32: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/32.jpg)
Interdependence
• The metric measures the number of interdependences between two modules
• How to count it: Count all direct references in one class to another
• Pitfalls: How to count dependencies that use inheritance? Are direct calls to be used?
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 33: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/33.jpg)
Inheritance Metrics
• These metrics measures how much classes inherit one from another.
• How to count them: Measure the number of parents or descendants
• Pitfalls: How does that relate to the quality of the code?
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 34: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/34.jpg)
Inheritance Example: EiffelBase
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 35: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/35.jpg)
Examples
Software Engineering, lecture: Estimation Techniques and Software Metrics
• In Eiffel most classes have a couple of parents (max 5 or 6, mostly 2 or 3)
• See new study at ECOOP 2008…
![Page 36: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/36.jpg)
Process metrics
• Failed Builds
• Defects per hour
• Requirements changes
• Programming Time
• Patches after release
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 37: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/37.jpg)
Failed Builds
• The metric measures the number of overnight build that failed
• How to count it: Count the number of times the overnight build failed
• Pitfalls: A higher value is not necessarily bad depending on the
development model Languages more restrictive on typing are more likely
to have higher value
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 38: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/38.jpg)
Defects per hour
• The metric measures the number of defects per hour of developer time
• How to count it: Count the number of bugs divide it by the number of
hours spent on the project
• Pitfalls: Does not take into account the difficulty of programs
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 39: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/39.jpg)
Examples
Software Engineering, lecture: Estimation Techniques and Software Metrics
• Apparently more experienced programmers are responsible for more bugs that rookies.
How to explain that?
![Page 40: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/40.jpg)
Requirements Changes
• The metric measures the number requirements changes
• How to count it: Count the number of requirements that changed
• Pitfalls: Who actually introduced the changes is also relevant!
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 41: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/41.jpg)
Programming Time
• The metric measures the number of hours spent by programmers
• How to count it: Ask programmers to fill in some form every week to
track their time
• Pitfalls: Is there really a correlation from one project to
another
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 42: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/42.jpg)
Patches after release
• The metric measures the number of patches distributed after the release
• How to count it: Count patches
• Pitfalls: Can work both ways:
Higher = higher maintenance Higher = software more bug-prone
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 43: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/43.jpg)
Did we present all Metrics?
Software Engineering, lecture: Estimation Techniques and Software Metrics
CLEARLY NOT!!!!!
![Page 44: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/44.jpg)
How many???
• M. Xenos, D. Stavrinoudis, K. Zikouli and D. Christodoulaki, “OBJECT-ORIENTED METRICS – A SURVEY” in FESMA 2000 studied over 200 software metrics!!!
• The question is: what metrics are adapted to what use?
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 45: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/45.jpg)
What about using several?
• As an example:
Maintainability = 171−5.2ln(V)−0.23V(G)−16.2ln(L) +50sin (sqrt(2.4 C))
Software Engineering, lecture: Estimation Techniques and Software Metrics
Cited by Andreas Zeller at the Laser 2006 Summer School Oman, P. & Hagemeister, J. "Constructing and Testing of Polynomials Predicting Software Maintainability.”
Journal of Systems and Software 24, 3 (März 1994): 251–266.
% of comment lines
locs cyclomatic Halstead Complexity
![Page 46: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/46.jpg)
How to avoid stupid things with metrics?
• Treat them like statistics: talk about correlation rather than implication, and then make an interpretation!
• Famous example from statistics: People who wear a hat have more often cancer
Deduction: wearing hat provokes cancer
Software Engineering, lecture: Estimation Techniques and Software Metrics
Real interpretation: it might be that people who wear a hat are more exposed to cancer anyway
(e.g. they are bald and thus more exposed to skin cancer)
![Page 47: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/47.jpg)
Probably the most interesting recent work on metrics in the last few years… (1/3)
• http://www.st.cs.uni-sb.de/softevo/
• Prof. Zeller and his team got access to 5 major projects at Microsoft and they tried 15 -20 metrics on all projects to predict what modules would be more bug-prone once in production.
• Results: there is no single metric that predicts bugs for all projects!
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 48: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/48.jpg)
Probably the most interesting recent work on metrics in the last few years… (2/3)
• Idea: combine metrics to make the One?
• It does not work either!
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 49: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/49.jpg)
Probably the most interesting recent work on metrics in the last few years… (3/3)
• Main result:
If a set of metrics is good in one project at one point, it is good to predict failures in future iterations!
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 50: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/50.jpg)
The paradox of metrics
• Once a metric expresses a negative property and programmers tend to use it, then it becomes less and less valid to predict errors.
• It behaves like “Goto statements”, Dijkstra
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 51: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/51.jpg)
What metrics to use?
• Locs • Number of classes • Memory footprints • Cyclomatic complexity for testing • Number of states • Programming time
Are all easy to use and do not really imply anything bad for programmers.
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 52: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/52.jpg)
What metrics to use? (cont.)
In the end, use the ones that work for you!
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 53: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/53.jpg)
How to present results
• Use graphs if possible in addition to raw data in tables
• When aggregating data (like in memory footprints), show confidence intervals
• Show the progression of values: make sure that the intuitive understanding is not contradicted by the actual graphs
• Label correctly all axes and make sure to explain all abbreviations
• Select carefully your data and the way you show them Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 54: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/54.jpg)
Example: Present data and Graphs
Software Engineering, lecture: Estimation Techniques and Software Metrics
Source: Neamtiu, Hicks, Stoyle, Oriol, (PLDI 2006)
![Page 55: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/55.jpg)
Example: Confidence Intervals
Software Engineering, lecture: Estimation Techniques and Software Metrics Source: Neamtiu, Hicks, Stoyle, Oriol, (PLDI 2006)
![Page 56: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/56.jpg)
Example: Intuitive Understanding
Software Engineering, lecture: Estimation Techniques and Software Metrics
NEVER, EVER, DO THAT!!!!! (We did it… )
Source: Ciupa, Leitner, Oriol, Meyer (ISSTA 2007)
![Page 57: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/57.jpg)
Example: Label Axes
Software Engineering, lecture: Estimation Techniques and Software Metrics
NEVER, EVER, DO THAT!!!!! (We did it… )
Source: Ciupa, Pretschner, Leitner, Oriol, Meyer (ICMT 2007)
![Page 58: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/58.jpg)
Example: Select and present data
Software Engineering, lecture: Estimation Techniques and Software Metrics Source: Leitner, Ciupa, Eugster, Oriol, Meyer (TOOLS 2007)
![Page 59: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/59.jpg)
In practice: Metrics Tool in Estudio
Software Engineering, lecture: Estimation Techniques and Software Metrics
Input Domain
Run
Metrics
![Page 60: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/60.jpg)
In practice: Metrics Tool in Estudio
Software Engineering, lecture: Estimation Techniques and Software Metrics
Create Custom
Custom definition
![Page 61: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/61.jpg)
ESTIMATION TECHNIQUES
![Page 62: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/62.jpg)
Estimation Techniques
• The basic measure: the man-month
• Discussion:
How would you quantify the time needed to develop a web browser?
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 63: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/63.jpg)
Cone of Uncertainty
Software Engineering, lecture: Estimation Techniques and Software Metrics Source: http://www.construx.com/Page.aspx?hid=1648
![Page 64: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/64.jpg)
Cloud of Uncertainty
Software Engineering, lecture: Estimation Techniques and Software Metrics Source: http://www.construx.com/Page.aspx?hid=1648
![Page 65: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/65.jpg)
Narrowing the cloud
Software Engineering, lecture: Estimation Techniques and Software Metrics Source: http://www.construx.com/Page.aspx?hid=1648
![Page 66: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/66.jpg)
What makes it so unstable?
• Forgetting necessary tasks!!! Missing requirements
Functional and non-functional (e.g. setup, data conversion, glue code, help system, interfaces…)
Missing software development activities E.g. new team members, creation of test data,
coordination with sub contractors…
Missing other activities E.g. vacation, training, weekends, holidays,
hardware problems Software Engineering, lecture: Estimation Techniques and Software Metrics Source:Mc Connel, Software Estimation
![Page 67: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/67.jpg)
First question: estimate the size
• Assumption:If you already coded some program that took around the same amount (up to factor 3) of work to code than the one you are going to make
• What is the size of the project?
How to express it?
Is it already starting to design?
What needs to be written?
How can an expert help? Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 68: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/68.jpg)
Why estimate the size?
• The cost in terms of man-month is as follows: (C and k are constants, usually the size is in kLoc)
Cost=c * (size)^k
Software Engineering, lecture: Estimation Techniques and Software Metrics
Source: Fundamentals of Software Engineering, Ghezzi, Jazayeri, Mandrioli
![Page 69: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/69.jpg)
Basic COCOMO
E=ab(KLOC)bb (effort in man-month)
D=cb(E)db (development time)
P=E/D (people required)
Software project ab bb cb db Organic 2.4 1.05 2.5 0.38 Semi-detached 3.0 1.12 2.5 0.35 Embedded 3.6 1.20 2.5 0.32
Organic: small simple projects, small experienced teams Semi-detached: intermediate, mixed experience Embedded: developed with set of tight hardware etc.
constraints Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 70: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/70.jpg)
Basic COCOMO?
• Good for rough and fast estimations
• Somewhat big variability
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 71: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/71.jpg)
Intermediate COCOMO?
E=ai(KLoC)(bi).EAF
Software project ai bi Organic 3.2 1.05 Semi-detached 3.0 1.12 Embedded 2.8 1.20
• EAF?
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 72: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/72.jpg)
Effort Adjustment Factor
Software Engineering, lecture: Estimation Techniques and Software Metrics The product of all these values…
![Page 73: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/73.jpg)
Another Estimation: Function-Points
Idea: replace locs with function points
Types of FP: External Inputs: screens forms, dialog boxes… External Outputs: screens, reports, graphs External Queries: input/output combination with a
query leads to simple output Internal Logical files: major groups of end-user-
data External Interface Files: files controlled by other
programs
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 74: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/74.jpg)
Calculation
Complexity Low Med High
• External Inputs x3 x4 x6 • External Outputs x4 x5 x7 • External Queries x3 x4 x6 • Internal Logic Files x4 x10 x15 • External Interface Files x5 x7 x10
• Count them, then multiply by the factors, then make the addition…
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 75: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/75.jpg)
From Function Points to Locs
• Depending on the language it generates a different number of lines of code:
• E.g.: C: 165.4, C#: 55, C++: 72.8, Cobol: 116, Smalltalk: 24.2, Java: 52.6, Eiffel: 21
(This is not an advertisement for Eiffel, but…)
Source: http://www.softwareestimator.comIndustryData2.htm#SAIL
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 76: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/76.jpg)
Today
• We presented software metrics How they are calculated What they represent Why they are used Their limitations
• We showed evaluation techniques How to use them What to deduce from them
Software Engineering, lecture: Estimation Techniques and Software Metrics
![Page 77: Lectures 17-18: Estimation Techniques and Software Metricsse.inf.ethz.ch/old/teaching/2008-S/se-0204/slides/15-Estimation... · Lectures 17-18: Estimation Techniques and ... Is there](https://reader031.vdocuments.us/reader031/viewer/2022021900/5b707f707f8b9aab548d74ee/html5/thumbnails/77.jpg)
Conclusions
• None of the techniques that were presented here are silver bullet
• They need to be adapted and tested in your environment with your people and to build up some know how
• That’s why being a project manager is implying keeping track of what people do and how projects went in the past
Software Engineering, lecture: Estimation Techniques and Software Metrics