se 450 software processes & product metrics 1 software quality engineering roadmap
Post on 19-Dec-2015
220 views
TRANSCRIPT
SE 450 Software Processes & Product Metrics 3
Maximize Revenue
Improve market share: Improve product quality Increase customer satisfaction Better marketing Improve cycletime
Increase product value: Increase product quality Define value proposition and match product capabilities and
quality to customer needs
SE 450 Software Processes & Product Metrics 4
Improve Market Share Improve product quality
Increase customer satisfaction Identify and address key customer concerns Continuous improvement process Practices for measuring and improving satisfaction Expectation management practices
Better marketing Match product to marketing needs Improved requirements practices Practices for measuring and improving marketing effectiveness
Improve cycletime Cycletime measurement and continuous improvement
Increase product value
Define value proposition and match product capabilities and quality to customer needs
SE 450 Software Processes & Product Metrics 5
Minimize Costs – Improve Productivity
Productivity measurement and continuous improvement
Practices for improving productivity: Tools usage Technology selection Reuse and common architecture development Streamlining and automation Workflow, templates, tools Task elimination
Developer capability improvement through training
SE 450 Software Processes & Product Metrics 6
Improve Project Management – Activity Management
Practices: planning, estimation, progress tracking
Metrics: estimation accuracy, progress metrics
Estimation practices: Maintain database of past projects Estimate based on history of comparable projects COCOMO (constructive cost model) provides questionnaire and
formula for estimation based on organizational and project profile
SE 450 Software Processes & Product Metrics 7
Improve Project Management – Risk Management
Identify and track risks
Metrics examples: Percentage of unidentified high-impact risks among those that
actually occurred Percentage of unmitigated high-impact risks among those that
actually occurred Percentage of effort spent on risk management and mitigation
SE 450 Software Processes & Product Metrics 8
Improve Project Management – People Management
Practices to measure and improve employee satisfaction
Teaming practices and effectiveness:
Metrics: Turnover %
SE 450 Software Processes & Product Metrics 9
Improve Project Management: In-Process Metrics
Slippage, progress
Inspection effectiveness (evolve as project proceeds)
Defect rates
Test progress
Reliability growth curves
SE 450 Software Processes & Product Metrics 10
Product Quality - Features
Requirements quality & effectiveness
Requirements prioritization & release planning
SE 450 Software Processes & Product Metrics 11
Product Quality - Performance
Defining performance objectives
Practices: designing for performance, performance analysis, performance testing
Primarily measurements and not metrics
SE 450 Software Processes & Product Metrics 12
Product Quality - Dependability
Reliability engineering: objective setting, practices and metrics
Availability: objective setting, practices and metrics
Security: Practices for identifying threats (defining objectives) Security engineering practices Security testing: threat injection
Safety (similar to security and reliability)
SE 450 Software Processes & Product Metrics 13
Product Quality - Usability
Specifying usability objectives
Usability engineering practices
Usability measurements: surveys, avg. learning time, keystrokes for frequent operations etc.
Usability metrics: SUMI
SE 450 Software Processes & Product Metrics 14
Product Quality - Evolvability
Evolvability goals
Practices for improving evolvability
SE 450 Software Processes & Product Metrics 15
Development Effectiveness
Requirements
Design
Testing & Inspections
Configuration Management
Quality Engineering
Creative Research Systems
SE 450 Software Processes & Product Metrics 16
Development Effectiveness - Requirements
Requirements effectiveness practices
Elicitation practices, prototyping, requirements analysis practices, traceability
Requirements metrics
Activity metrics, also requirements defect injection rate
SE 450 Software Processes & Product Metrics 17
Development Effectiveness - Design
Practices for design effectiveness
Modularity practices, design patterns for achieving various quality attributes, design diagramming, design analysis techniques
Design metrics
Activity metrics, injection rates, product quality metrics, reuse percentage
SE 450 Software Processes & Product Metrics 18
Development Effectiveness – Testing & Inspections
Practices for testing effectiveness Multistage testing, test strategy & planning, test case identification
through equiv classes, automation of test generation, randomized parameters, automated testing
Reliability engineering Test metrics DRE charts, defect rates and densities, reliability … Inspection practices and metrics
SE 450 Software Processes & Product Metrics 19
Development Effectiveness – Configuration Management
Configuration management practices
Metrics for configuration management effectiveness
Configuration management defect injection rate
SE 450 Software Processes & Product Metrics 20
Development Effectiveness – Quality Engineering Practices
Processes
Process Improvement / Continuous Improvement
Assessments
SE 450 Software Processes & Product Metrics 21
Quality Engineering Process
Initial process definition and tailoring Process compliance metrics (certifications, assessments) Process improvement Improve effectiveness of practices Improve breadth of process & practices (increased “maturity level”, CMMI-
style) Minimize volume of process, maximize effectiveness (agile development) Define set of processes that address the most frequent causes of defects and
achieve more of the business objectives: maximizing cost-benefit ratio for process practices
Requires deep understanding of process/quality relationships, high developer maturity and competence, and extensive customization to situational needs
A stage of evolution beyond simply adding process breadth and doing improvement.
SE 450 Software Processes & Product Metrics 22
Continuous Improvement
Quality tools and causal analysis, defect elimination Identify problem area Use fishbone diagrams to brainstorm and capture possible causes Use results to design data collection scheme and gather data about
actual causes of problem - This is an extremely important step. The fishbone itself is just a set of guesses. It does not tell you which causes need addressing
Use pareto charts to identify most significant causes of the problem Use defect elimination techniques (checklists, templates etc) to try to
prevent the problem in the future Repeat continuously or until objective is achieved Effectiveness measured through trend lines for various process metrics
SE 450 Software Processes & Product Metrics 23
Metrics Concepts Defining Metrics
GQM paradigm Metrics must have significant value: High impact (value provided
to user of metric) compared to effort spent in collecting data. Meaningfulness of metrics
Reliability Validity
Importance of interpretation
Gathering Measurements
SE 450 Software Processes & Product Metrics 24
Goal-Question-Metric (GQM) Paradigm
What are the objectives for the metric? Should identify the purpose of the metric – who wants to do what with it? Includes “viewpoint”: whose viewpoint are we interested in for this? E.g.
developer, project manager, organization manager, quality engineer, external assessor, customer.
What specific questions do we want answered? Identify specific aspects of the problem that we care about Includes operational definitions of the quantities we want to measure
Define measurements and metrics What data will need to be gathered? Who will gather it and how? What metric will be computed from the measurements? Metrics must provide a basis for action Metrics must have an interpretation: given a value, how will this be interpreted? Metrics must have significant value: high impact (value provided to user of metric)
compared to effort spent in collecting data Meaningfulness of metrics Reliability Validity
SE 450 Software Processes & Product Metrics 25
Importance of Interpretation
Understanding the intrinsic limitations of the metric
Understanding the limitations of the data collection process
Understanding situational factors that provide context to the numbers
SE 450 Software Processes & Product Metrics 26
Gathering Measurements Minimizing measurement effort:
Effort spent on providing data is “wasted” effort from project perspective – needs to be minimized!
Often, person who has to put in effort is different from the one who gets immediate value! Higher the effort, less the motivation for the provider to supply good data
Need for tools: Tools improve accuracy of data, especially if collection is automated May sometimes need to provide for manual adjustments to tool-generated data Tools can automate processing of data and generate useful views (graphs) directly
Motivating measurement: Set high bar on value: only ask for measurements that provide considerable value Make value visible to people providing measurements (clear how the gathered data
will be used) Try to provide value to those providing measurements:
e.g. customer concerns actually get addressed e.g. developers get useful feedback that helps them avoid problems and reduce
effort
SE 450 Software Processes & Product Metrics 27
Metrics Perspective
Metrics are a small part of the overall quality engineering process.
Define objectives for metrics.
Metrics interpretation is key.
Actions and improvements based on metrics interpretation is the primary value achieved in the use of metrics.