test estimation in practice
DESCRIPTION
Anyone who has ever attempted to estimate software testing effort realizes just how difficult the task can be. The number of factors that can affect the estimate is virtually unlimited. The key to good estimates is to understand the primary variables, compare them to known standards, and normalize the estimates based on their differences. This is easy to say but difficult to accomplish because estimates are frequently required even when very little is known about the project and what is known is constantly changing. Throw in a healthy dose of politics and a bit of wishful thinking and estimation can become a nightmare. Rob Sabourin provides a foundation for anyone who must estimate software testing work effort. Learn about the test team’s and tester’s roles in estimation and measurement, and how to estimate in the face of uncertainty. Analysts, developers, leads, test managers, testers, and QA personnel can all benefit from this tutorial.TRANSCRIPT
TR Half-day Tutorials
5/6/2014 1:00:00 PM
Test Estimation in Practice
Presented by:
Rob Sabourin
AmiBug.com
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
Rob Sabourin AmiBug.com
Rob Sabourin, P. Eng., has more than thirty years of management experience leading teams of software development professionals. A well-respected member of the software engineering community, Rob has managed, trained, mentored, and coached hundreds of top professionals in the field. He frequently speaks at conferences and writes on software engineering, SQA, testing, management, and internationalization. Rob wrote I am a Bug!, the popular software testing children's book; works as an adjunct professor of software engineering at McGill University; and serves as the principle consultant (and president/janitor) of AmiBug.Com, Inc. Contact Rob at Contact Rob at [email protected].
IntroductionTEST Estimation IN PRACTICE
Course timingCourse timing
Electronic
devices
Electronic
devices
Smoking
Smoking
MealsMeals
Facilities
Facilities
Breaks
Breaks
Administrivia
6© 2012 SQE Training V3.0
Introductions
Name
CompanyNameBusiness
EnvironmentHardwareSoftware
ExperienceTestingManagement
Personal Course Objective(s)
Name
CompanyNameBusiness
EnvironmentHardwareSoftware
ExperienceTestingManagement
Personal Course Objective(s)
7© 2012 SQE Training V3.0
Course Agenda
1. Estimation2. Estimation Techniques
8© 2012 SQE Training V3.0
1estimation
Estimation
Estimate: – A tentative evaluation or rough calculation – A preliminary calculation of the cost of a
project – A judgment based upon one’s impressions;
opinion —The American Heritage Dictionary
It is very difficult to make a vigorous, plausible, and job-risking estimate that is derived by no quantitative method, supported by little data and certified chiefly by hunches of the managers.
— Fred Brooks10© 2012 SQE Training V3.0
The best estimatesThe best estimates
Represent the collective wisdom of practitioners (testers, developers, users, etc.) and have their buy-in
Provide specific, detailed catalogs of the costs, resources, tasks, and people involved
Present, for each activity estimated, the most likely cost, effort, and duration
Estimation: the creation of an approximate target for costs and completion dates Estimation: the creation of an approximate target for costs and completion dates
Test Estimation
11© 2012 SQE Training V3.0
Factors that can influence cost, effort, and duration include:Factors that can influence cost, effort, and duration include:
Required level of quality of the system
Size of the system to be tested
Historical data
Process factors (process maturity, etc.)
Material factors (tools, data, etc.)
People factors (skills, experience, managers, etc.)
Test Estimation (cont.)
12© 2012 SQE Training V3.0
Test Estimation (cont.)
• Delivery of estimates should include justification
• Negotiation and re-work of estimates is normal
• Final estimates represent a balance of organizational and project goals in the areas of quality, schedule, budget, and features
13© 2012 SQE Training V3.0
How Good an Estimator Are You?Low Estimate High Estimate
___________ to ____________
.
.
.
© Steve McConnell, Software Estimation, Microsoft Press, 2006
Surface temperature of the sun
Latitude of ShanghaI
Area of the Asian continent
Birth year of Alexander the Great
Total value of US currency in 2004
Total volume of the Great Lakes
Worldwide box office receipts of Titanic
Total length of Pacific Ocean coastline
Heaviest blue whale ever recorded
# of books published in US since 1776
14© 2012 SQE Training V3.0
How Good an Estimator Are You?• Surface temperature of the sun 10,0000 F / 6,000 C
• Latitude of ShanghaI 31° 10′ N
• Area of the Asian continent 17,212,000 sq mi
• Birth year of Alexander the Great 356 BC
• Total value of US currency in 2004 $719.9 Billion
• Total volume of the Great Lakes 5,500 cu mi
• Worldwide box office receipts of Titanic $1.835 billion
• Total length of Pacific Ocean coastline 84,300 miles
• Heaviest blue whale ever recorded 380,000 pounds
• Number of books published in US since 1776 22 million
15© 2012 SQE Training V3.0
How Good Is Our Industry (at Estimating)?
• Tata: 62% of projects fail to meet schedule
49% have budget overruns• Moiokken and Jorgensen: 30-40%
overruns
16© 2012 SQE Training V3.0
Class Discussion
Why is estimating not done well?
Your top five reasons:
1) Too many variables____________________
2) ____________________________________
5) ____________________________________
6) ____________________________________
5) ____________________________________
17© 2012 SQE Training V3.0
Why Estimates Are Inaccurate ― Part I
• Lack of estimating experience• Lack of historical data on which to base estimates• Lack of systematic estimation process, sound techniques, or
models suited to the project• Failure to include essential activities and products within the
scope of the estimates• Unrealistic expectations or assumptions• Failure to recognize and address the uncertainty inherent in
project estimates
Practical Software Measurement
Addison-Wesley, 2001
18© 2012 SQE Training V3.0
Why Estimates Are Inaccurate ― Part II
• Lack of education and training• Confusing the target with the estimate• Hope-based planning• Inability to communicate and support estimates• Incomplete, changing, and creeping requirements• Quality surprises (test and re-fix)
—adapted from Linda M. Laird
The Limitations of Estimation
19© 2012 SQE Training V3.0
Bohem’s Cone of Uncertainty
20© 2012 SQE Training V3.0
NHC Track Forecast Cone
21© 2012 SQE Training V3.0
“Testing” Track Forecast Cone
Best ca
se
Expected case
Worst case
T i m e
Tas
ks(or why it is important to constantly re-estimate)
22© 2012 SQE Training V3.0
What would have to happen to deliver this in four weeks?What would have to happen to deliver this in four weeks?
What should the estimate have been?What should the estimate have been?
The Fantasy Factor
Weeks
Today
0 1 2 3 4 5 6 7 8 9
1st 3rd2nd
23© 2012 SQE Training V3.0
Time
Time
SizeSize
Quality
Quality
Resources
Resources
Estimation
1, 2, 3, or 4 Variables + Many Modifiers:
If it’s not variable, then it’s fixed.
24© 2012 SQE Training V3.0
Time vs. Resources
=
25© 2012 SQE Training V3.0
2Estimation Techniques
Test Estimation Techniques ― Examples
• Intuition and guess• Work-breakdown-structures• Three-point estimates• Company standards and norms• % of project effort or staffing• Industry averages and predictive models (e.g., FP, TPA )• Team estimation sessions
– Wideband Delphi– Story point sizing– Poker estimation– T-shirt sizing
28© 2012 SQE Training V3.0
What do intuition and guess imply?What do intuition and guess imply?
No formal estimation process
Based on the experience of the team member(s) usually without the benefit of recorded metrics
Intuition and Guess
29© 2012 SQE Training V3.0
When would we use this technique?When would we use this technique?
Experienced staff members
Little or no recorded metrics available
Great accuracy is not required
Same type of project we’ve done in the past
Or at the other extreme, we don’t have a clue (often called a WAG or SWAG)
Intuition and Guess (cont.)
30© 2012 SQE Training V3.0
Project LevelProject Level
Phase LevelPhase Level
Task LevelTask Level
Phase LevelPhase Level
Task LevelTask Level
------
Task LevelTask Level
------
------
------
Task LevelTask Level
------
Phase LevelPhase Level
Task LevelTask Level
Phase LevelPhase Level
Task LevelTask Level
Work Breakdown Structure
31© 2012 SQE Training V3.0
When would we use this technique?When would we use this technique?
When the test team lacks credibility with their estimates
When we are doing an estimate for a project that is dissimilar to projects we have done in the past
When the project is too large to get our “arms around it” (e.g., it is too big to easily see the big picture)
Work Breakdown Structure (cont.)
32© 2012 SQE Training V3.0
When would we not use this technique?When would we not use this technique?
When the project has lots of turbulence!
Work Breakdown Structure (cont.)
33© 2012 SQE Training V3.0
Approach:Approach:
Develop goal
Break goal into deliverables
Break deliverables into activities
Decompose work into small manageable components that are of sufficient detail to allow you to make estimates of time and/or resources
Work Breakdown Structure ― Top Down
34© 2012 SQE Training V3.0
Work Breakdown Structure (cont.)
Testing for Completeness
• Status of a task is measurable• Start and end events are clearly defined• Further breakdown makes no sense• Each activity has a deliverable• Time or resources for an activity can be estimated• Duration of activity is within acceptable limits• Each activity is independent• Each activity forms a unique package of work that could
be outsourced
35© 2012 SQE Training V3.0
Work Breakdown Structure (cont.)
• Level of detail guidelines:– No single activity or group of activities should
exceed 80 hours of effort*– No activity or group of activities should exceed
the reporting period (e.g., if the reporting period is weekly, no activity should exceed one week)
– “If it makes sense” rule: The duration of each activity must pass the common sense rule
36© 2012 SQE Training V3.0
Three–Point Estimates
• Three-point estimates is an analytical technique using three cost or duration estimates:– Optimistic– Most likely – Pessimistic
• This technique is applied to improve the accuracy of the estimates of cost or duration
• Three-point estimation techniques can be employed with other estimating techniques such as Delphi
37© 2012 SQE Training V3.0
% of Project Effort or Staffing
• Based on the premise that there is a predictable correlation between development effort/time and test effort/time
• The preferred/“dictated” method of estimation in some organizations
• Sometimes associated with developer/tester ratios
• One method recognized by the ISTQB
38© 2012 SQE Training V3.0
When Would We Use This Technique?When Would We Use This Technique?An organization is producing more or less the same type of product on an ongoing basis
The organization (development and testing) uses consistent practices and has a stable staff
A crude estimate is needed
It is mandated
It works
% of Project Effort or Staffing
39© 2012 SQE Training V3.0
Issues:Issues:
We are basing our estimates on development estimates which are probably suspect
Some products that are relatively easy to develop may be difficult to test and vice versa
This technique may not take into account test automation, etc.
This method may not take the stated quality goals into account
Test Estimates Based on Development Estimates
40© 2012 SQE Training V3.0
Class DiscussionWhat is the ideal developer/tester ratio?
= ?
41© 2012 SQE Training V3.0
McConnell Developer to Tester Ratio
Environment Relative Importance Ratio(Developer:Tester Ratio)
Business Systems 3:1 to 20:1
Shrink Wrap 1:1 to 5:1
Scientific Projects 5:1 to 20:1
Systems Projects 1:1 to 5:1
Safety Critical Software 5:1 to 1:2
Microsoft Windows 2000 1:2
NASA Space Shuttle Flight Control Software 1:10
42© 2012 SQE Training V3.0
Test Point Analysis
• A method with the possibility to perform a technology-independent measurement of the test size of an information system on the basis of a function point analysis, and to use this measurement as a basis for a productivity measurement, an estimate of the required resources, and project management
Software Testing: A Guide to the TMap ApproachMartin Pol, Ruud Teunissen, Erik van Veenendaal
43© 2012 SQE Training V3.0
Test Point Analysis (cont.)
For more details on TPA,look in this book.
44© 2012 SQE Training V3.0
Principles of Test Point Analysis
• TPA only considers measurable quality characteristics that fall within the range of system or acceptance testing
• TPA is (in principle) analyst independent• TPA depends on the availability of function points• Sufficient knowledge of the test team is a precondition• TPA estimates are based on the assumption that, on average,
one complete re-test will be conducted
45© 2012 SQE Training V3.0
Team Estimates
• Teams often produce better estimates than individuals (See The Wisdom of Crowds)
• Team estimates can facilitate buy-in and commitment from the entire team
• Almost all estimating methods can be done using teams
46© 2012 SQE Training V3.0
The Wisdom of Crowds
Four elements required to form a wise crowd:
• Diversity of opinion• Independence• Decentralization• Aggregation
47© 2012 SQE Training V3.0
Test Estimation Sessions ― Wideband Delphi
48© 2012 SQE Training V3.0
Test Estimation Sessions ― Wideband Delphi
• Developed by Barry Boehm and John Farquhar in the 1970s*
• A variant of the existing Delphi method but with more interaction
• Popularized in Boehm’s book Software Engineering Economics in 1981
• Similar to the Platinum Poker method used in some agile projects
49© 2012 SQE Training V3.0
When would we use this technique?When would we use this technique?
Estimation for a new product or technology
To develop an “order of magnitude” estimate
Requirements are shaky or incomplete
Multiple disciplines are involved
Wideband Delphi
50© 2012 SQE Training V3.0
Test Estimation Sessions — Wideband Delphi
Steps from Barry Boehm’s book:
1. Coordinator presents each expert with a specification and estimation form
2. Experts discuss issues in group meeting3. Experts fill out form anonymously4. Coordinator distributes a summary5. In a group meeting experts discuss variances6. Experts fill out forms again for as many rounds as
necessary
51© 2012 SQE Training V3.0
Wideband Delphi Process Flow
Assemble estimates
and assumptions
Estimation meeting
Individual preparation
Planning and kickoff
meeting
Re-estimate
Acceptable
Done
No Yes
Adapted from Karl Wiegers
52© 2012 SQE Training V3.0
Sample Estimation Form — Wideband Delphi
Task Estimate #1 Estimate #2 Estimate #3 Estimate #4 Final
Change
Total
53© 2012 SQE Training V3.0
Discussion Items:Discussion Items:
What are story points?● Measure of size/complexity, not time● Relative unit of measure
Is there any relation to function points or test points?
Estimating Using Story Points ― Agile
54© 2012 SQE Training V3.0
Estimating Using Story Points ― Agile
55© 2012 SQE Training V3.0
Estimating Using Story Points ― Agile
• Story points are used for long term or release planning and tracking
• Point-estimated stories along with team velocity can be used to provide rough release-level scheduling and project progress
• Since story points are relative size indicators, a two-point story is always twice as big as a one-point story
56© 2012 SQE Training V3.0
Why Use Story Points?
• Cheaper• Allow us to change our minds as new
information becomes available• Don’t take a lot of time• Foster collaboration• Consistent• Provide credibility• Can use Planning Poker Cards with story points
57© 2012 SQE Training V3.0
Planning Poker
58© 2012 SQE Training V3.0
Planning Poker
• Based on the Wideband Delphi method espoused by Barry Boehm and others in the 60s and 70s (and Boehm’s work was possibly based on earlier works dating to the 40s)
• Most commonly used in agile software development
• A study published by the IEEE in April 2007 indicated that Planning Poker achieved less optimistic and more accurate estimates than those obtained through mechanical combinations of individual estimates
59© 2012 SQE Training V3.0
Planning Poker Rules
1. Form a group of no more than ten estimators and a moderator. The product owner is usually present but cannot estimate
2. Each participant gets a deck of cards. The decks frequently use a doubling sequence (½, 1, 2, 4, 8, 16, 32…) or the Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144…) or a “modified” Fibonacci sequence such as (0, 1, 2, 3, 5, 8, 13, 20, 40, 100)
3. The moderator reads one user story at a time to the team
4. The product owner answers questions about the story
60© 2012 SQE Training V3.0
Planning Poker Rules (cont.)
5. Each estimator privately selects a card from his or her deck representing their estimate of the “size” of the user story
6. When everyone is ready, all of the cards are flipped over at the same time
7. If there is a consensus, the results are recorded, and the team moves on to the next story
8. If the estimates vary widely, the owners of the high and low estimates defend their positions to the rest of the team
61© 2012 SQE Training V3.0
Planning Poker Rules (cont.)
9. The group briefly debates the arguments
10. Repeat from step 5 until the estimates converge
11. Repeat for all stories
62© 2012 SQE Training V3.0
Planning Poker Rules ― Tips
1. Those who actually could do the work are the ones who vote
2. Managers don’t vote
3. When there is a tie in the voting between two sizes that are consecutive, just pick the larger size and move on
4. Stop implementation discussions before they go too deep
5. Use an “I need a break” card
63© 2012 SQE Training V3.0
Planning Poker Rules ― Tips (cont.)
6. Use a timer to limit discussions
7. If consensus cannot be reached by the end of the third round of voting, pick the largest size and move on
8. Have the person creating the user stories meet with QA and Development leads prior to playing poker to answer the most obvious questions
9. Remember the baseline
10.Have fun!
64© 2012 SQE Training V3.0
T-Shirt Sizing
An estimation technique, usually used in agile projects, that uses relative sizing based upon a limited number of values (e.g., t-shirt sizes): S, M, L, XL
Steps:– Make S, M, L, XL cards– Stories are read one at a time– Each developer gives each story a t-shirt size– All developers raise their cards simultaneously– Discuss differences– Go back to step 3– Compile the stories into size buckets– Estimate the time to complete a S, M, L,XL
65© 2012 SQE Training V3.0
Hadden’s Size/Complexity Technique
Rita HaddenSmall Medium Large
Simple
Moderate
Complex
Complexity
Size
66© 2012 SQE Training V3.0
Sample Estimate
67© 2012 SQE Training V3.0
Test Objective Size Complexity Effort
1 to 3 1 to 3 Sessions 1 2 3to001 2 1 4 1 1 4 8to002 2 3 16 2 4 8 16to003 2 3 16 3 8 16 32to004 2 1 4to005 3 1 8to006 2 2 8to007 3 2 16Typical Usage Scenarios 2 2 8
total 80 sessions20 days
Siz
e
Complexity
Simple Defect Estimation Models
• Defect Detection Percentage (DDP)• Historical Extrapolation• “Bug Budgets”
68© 2012 SQE Training V3.0
Karl Wiegers’s Estimation Safety Tips
• A goal is not an estimate• The estimate you produce should be unrelated
to what you think the requester wants to hear• The correct answer to any request for an
estimate is “Let me get back to you on that”• Avoid giving single point estimates• Incorporate contingency buffers into estimates
69© 2012 SQE Training V3.0
Rick Craig’s Tips for Better Estimates
• Do it!• Collect metrics• Remember the “fantasy” factor• Don’t “pad” your estimates*• Don’t spend a ton of time• Estimates don’t have to be perfect
– Estimates are just estimates– They will change/constantly as you re-estimate– Remember planning risks and contingencies– Remember Brooke’s Law
• If the date is fixed, estimate something else• Use tools• Use ranges of value instead of discrete numbers
70© 2012 SQE Training V3.0
Robert Sabourin’s Tips for Better Estimates
• Consider estimating the amount of test coverage possible given a fixed amount of testing effort
• Present stakeholders with alternatives (Plan-a,Plan-b,Plan-9)
• Use ranges of values• Highlight business organizational and
technical factors which impact the estimate
• Using used envelopes to present rough estimates
71© 2012 SQE Training V3.0
Thank You
• On behalf of SQE Training, we appreciate your attendance at this course
• If you have additional training or consulting needs, please think of us
73© 2012 SQE Training V3.0
AppendixInstructor Material
Brainstorming Testing Estimates
• Min Max Missing– Identify participants– Hold kick off meeting– Individuals build list– Logging meeting– Estimate depth of testing– Estimate size in sessions– Identify new, missing, or
redundant ideas
75© 2012 SQE Training V3.0
BrainstormingTesting Ideas
• Min Max Missing– PMI Estimation– Given three values
● MIN = BEST CASE● MAX = WORST CASE● EXPECTED = NORMAL CASE● Estimate = (MIN + 4*EXPECTED + MAX)/6
76© 2012 SQE Training V3.0
Effort Estimation
ReadingSteve McConnell, Rapid Development: Taming Wild
Software Schedules. Chapter 8
77© 2012 SQE Training V3.0
Estimating Software Projects• Duration (on time)
– Schedule time– Delivery
• Quality (on quality)– Defect density– Defect arrival
• Cost (on budget)– Money
● Fixed● Variable
– Opportunity● What else could we be doing?
78© 2012 SQE Training V3.0
Estimating Software Projects
• Effort– Work– Manpower
• Staffing– Team definition– Skills, experiences
79© 2012 SQE Training V3.0
Estimation Process
McConnell Basic Steps:
1. Estimate size2. Estimate effort3. Estimate schedule
80© 2012 SQE Training V3.0
Estimation Process
1 - Estimate Size
81© 2012 SQE Training V3.0
Estimation Size
• SLOC– Source lines of code
• FP– Function points– Synthetic measure of software size
82© 2012 SQE Training V3.0
Estimation Size
• Software Tools– Calibration base on relevant experience– Combine several methods– Estimator (free!)
● www.construx.com
83© 2012 SQE Training V3.0
Estimation Size
• Analogy– Similar past projects– Relative to past experience– Must take into account
● Team● Technology● Complexity● Development process maturity
84© 2012 SQE Training V3.0
Estimating Size
• Algorithmic– Function points– Given requirements determine the number and
complexity of● Inputs● Outputs● Inquiries● Logical files● External interfaces
– Calculate and adjust based on “type of project/technology”
85© 2012 SQE Training V3.0
Function Points
Low Complexity Medium Complexity High Complexity
Program Characteristic count multiplier count multiplier count multiplier total
Inputs 6 3 2 4 3 6 44
Outputs 7 4 7 5 0 7 63
Inquiries 0 3 2 4 4 6 32
Logical Internal Files 5 7 2 10 3 15 100
External Interface Files 9 5 0 7 2 10 65
Unadjusted Function Point Total 304
Influence Multiplier 1.15
Adjusted Function Point Total 349.6
86© 2012 SQE Training V3.0
Function Points ― Influence Factors
1 Data communications How many communication facilities are there to aid in the transfer or exchange of information with the application or system?
2 Distributed data processing How are distributed data and processing functions handled?
3 Performance Did the user require response time or throughput?
4 Heavily used configuration How heavily used is the current hardware platform where the application will be executed?
5 Transaction rate How frequently are transactions executed daily, weekly, monthly, etc.?
6 On-line data entry What percentage of the information is entered on-line?
7 End-user efficiency Was the application designed for end-user efficiency?
87© 2012 SQE Training V3.0
Function Points ― Influence Factors
8 On-line update How many ILFs are updated by on-Llne transaction?
9 Complex processing Does the application have extensive logical or mathematical processing?
10 Reusability Was the application developed to meet one or many user’s (users’) needs?
11 Installation ease How difficult is conversion and installation?
12 Operational ease How effective and/or automated are start-up, back up, and recovery procedures?
13 Multiple sites Was the application specifically designed, developed, and supported to be installed at multiple sites for multiple organizations?
14 Facilitate change Was the application specifically designed, developed, and supported to facilitate change?
88© 2012 SQE Training V3.0
Function Points ― Influence Factors Score
Score Influence
0 Not present, or no influence
1 Incidental influence
2 Moderate influence
3 Average influence
4 Significant influence
5 Strong influence throughout
89© 2012 SQE Training V3.0
Function Points Influence Factors Score
INF = 0.65 + SCORE/100
90© 2012 SQE Training V3.0
Function Points ― Input
• Input– Number of screens, forms, dialogues, controls, or
messages through which an end user or another program adds, deletes, or changes data
91© 2012 SQE Training V3.0
Function Points ― Output
• Output– Screens, reports, graphs, or messages generated for
use by end users or other programs
92© 2012 SQE Training V3.0
Function Points ― Inquiries
• Inquiries– Direct access to data in database
93© 2012 SQE Training V3.0
Function Points ― Logical Files
• Logical Files– Major groups of end user data; could be a “file” or
“database table”
94© 2012 SQE Training V3.0
Function Points ― External Interfaces
• External Interface Files– Files controlled by other applications with which the
program interacts
95© 2012 SQE Training V3.0
Function Points ― Advantages
• Advantages include– Technology independent– Strong relationship to effort– Commonly used
• Disadvantages include– Requires specialized training to
compute
96© 2012 SQE Training V3.0
Estimation Process
2 - Estimate Effort
97© 2012 SQE Training V3.0
Effort Estimate
• Jones’s First-Order Estimation Practice
• Effort = (FP count) exponent
98© 2012 SQE Training V3.0
Jones’s First Order Exponents
Organization Capability
Kind of Software Best Average Worst
Systems 0.43 0.45 0.48
Business 0.41 0.43 0.46
Shrink Wrap 0.39 0.42 0.45
99© 2012 SQE Training V3.0
Estimation Process
3 - Estimate Schedule
100© 2012 SQE Training V3.0
Estimating Schedule
• Rule of Thumb (McConnell 8-1)
• Schedule = 3.0 * (effort) 1/3
101© 2012 SQE Training V3.0
Estimating Schedule
• Software Engineering Past Data– Example McConnell table 8-8, 8-9
● LOC vs. Schedule and Effort● Different project types
102© 2012 SQE Training V3.0
Estimation Convergence
Effort Range Schedule Range
Project Phase Optimistic Pessimistic Optimistic Pessimistic
Initial product definition 0.25 4.00 0.60 1.60
Approved product definition 0.50 2.00 0.80 1.25
Requirement specification 0.67 1.50 0.85 1.15
Product design specification 0.80 1.25 0.90 1.10
Detailed design specification 0.90 1.10 0.95 1.05
Product complete 1.00 1.00 1.00 1.00
103© 2012 SQE Training V3.0
Initialproductdefinition
Approvedproductdefinition
Requirementsspecification
Productdesignspecification
Detaileddesignspecification
Productcomplete
1.0×
0.25×
4×
2×
0.5×
1.5×
0.67×
1.25×
0.8×1.0×
0.6×
1.6×
1.25×
0.8×
1.15×
0.85×
1.1×
0.9×
Project Cost (effort and size) Project Schedule
Estimate ― Convergence Graph
104© 2012 SQE Training V3.0
More Relationships
Code Volume Approx. 100 LOC/FP, varies widely [Pressman, p. 94]
Schedule (FP)0.4Calendar months
Staffing (FP)/100Average – follows Raleigh curve
Effort Schedule * Staffing = (FP1.4)/150
105© 2012 SQE Training V3.0
Rules of Thumb
106© 2012 SQE Training V3.0
Rules of Thumb (cont.)
• Experience from several projects indicates that the amount of testing required in a project is proportional to the amount of development required
• Ratio depends on
– History– Organization, culture, structure– Projects– Teams
107© 2012 SQE Training V3.0
Rules of Thumb (cont.)
• Hadden’s Size/Complexity Technique
Small Medium Large
Simple
Moderate
Complex
Complexity
Size
108© 2012 SQE Training V3.0
Developer/Tester Ratios
109© 2012 SQE Training V3.0
Sample Estimate Project A
110© 2012 SQE Training V3.0
Test Objective Size Complexity Effort
1 to 3 1 to 3 Sessions 1 2 3to001 2 1 4 1 1 4 8to002 2 3 16 2 4 8 16to003 2 3 16 3 8 16 32to004 2 1 4to005 3 1 8to006 2 2 8to007 3 2 16Typical Usage Scenarios 2 2 8
total 80 sessions20 days
Siz
e
Complexity
Sample Estimate Project B
111© 2012 SQE Training V3.0
Test Objective Size Complexity Effort1 to 3 1 to 3 Sessions 1 2 3
to001 1 1 1 1 1 4 8to002 1 1 1 2 4 8 16to003 2 1 4 3 8 16 32to004 2 1 4to005 2 1 4to006 2 1 4to007 1 1 1to008 1 1 1to009 3 2 16to010 2 2 8to011 2 2 8to012 2 2 8to013 2 2 8to014 3 2 16to015 3 2 16to016 2 2 8Typical Usage Scenarios 2 2 8
total 116 sessions29 days
Complexity
Siz
e
Sample Estimate Project C
112© 2012 SQE Training V3.0
Test Objective Size Complexity Effort1 to 3 1 to 3 Sessions 1 2 3
to001 3 2 16 1 1 4 8to002 1 1 1 2 4 8 16to003 1 1 1 3 8 16 32to004 1 1 1to005 1 1 1to006 1 1 1to007 1 2 4to008 1 2 4to009 1 1 1to010 1 2 4to011 1 2 4to012 2 2 8to013 2 2 8to014 2 2 8to015 1 1 1to016 1 1 1to017 2 1 2to018 2 2 8to019 3 2 16Typical Usage Scenarios 2 2 8
total 98 sessions24.5 days
Complexity
Siz
e
Sample Estimate Project D
113© 2012 SQE Training V3.0
Test Objective Size Complexity Effort1 to 3 1 to 3 Sessions 1 2 3
t001 1 1 1 1 1 4 8t002 1 1 1 2 4 8 16t003 1 1 1 3 8 16 32t004 2 1 4t005 2 1 4t006 2 2 8t007 1 1 1t008 1 1 1t009 1 1 1t010 2 1 4t011 2 2 8t012 1 1 1t013 1 2 4t014 1 2 4t015 1 2 4t016 1 1 1t017 2 1 4t018 1 1 1t019 1 2 4t020 2 1 4t021 1 1 1t022 1 1 1t023 2 2 4t024 1 1 1t025 2 1 4t026 1 1 1t027 1 1 1t028 2 1 4t029 1 2 4t030 1 2 4t031 2 1 4t032 1 1 1Scenarios 2 2 8
total 99 sessions24.75 days
Complexity
Siz
e