chapter 19: quality models and measurements types of quality assessment models data requirements...
TRANSCRIPT
Chapter 19: Quality Models andMeasurements
Types of Quality Assessment Models
Data Requirements and Measurement
Comparing Quality Assessment Models
Measurement and Model Selection
Introduction Analytical models provide quantitative
assessment of selected quality characteristics Applied over time, provide accurate prediction of
future quality Purpose of measurement and analysis is to
make corrective actions =>improvement provide timely feedback/assessment identify problematic areas prediction, anticipating/planning for scheduling and
resource allocation
Models for Quality Assessment Direct indicators of quality
defect measurements - defect density for correctness probability of failure-free operation for reliability measured at end of software development
Indirect indicators of quality product internal attributes (e.g. KLOC, McCabe’s) interaction between product and user development process general characteristics of product (e.g. telecom) may be available early enough to make predictions
Models for Quality Assessment
Generalized Models for Quality Assessment Require little or no project-specific data Three categories
Overall model – provides a single estimate of overall product quality
Segmented model – provides different quality estimates for different industrial segments
Dynamic model – provides quality trend or distribution over time or development process
Overall Models Most general subtype of generalized quality
models Provide a rough estimate of product quality, e.g.
defect density = total defects / product size
Lump all products together – abstraction of commonly observed facts about quality generally true over all kinds of application domains, e.g. 80:20 rule which states 80% of defects are concentrated in 20% of
product modules/components linkages between software defect, risk, process maturity to quality
Segmented Models Abstraction of commonly observed facts about
quality over product market segments, e.g. reliability levels (measured by failure rate)
safety-critical SW – medical devices and nuclear reactors commercial SW – telecommunications and business auxiliary SW – games and low-cost PC SW
Dynamic Models Provide information about quality over time
or development phases, e.g. defect distribution profile over dev. phases Putnam model – effort and defect profiles over
time reliability growth during product testing
Can be combined with segmented models to give us segmented dynamic models
Product-Specific Models Provide more precise quality assessments using
product-specific data Three categories
Semi-customized models – extrapolate product history to predict quality for the current project (Table 2)
Observation-based models – estimate quality based on observations from the current project
Measurement-driven predictive models – establish predictive relations between various early measurements and product quality
Semi-Customized Models Use general characteristics and historical
information about product, process, or envt Provide quality extrapolations Examples:
Defect removal models (DRMs) provide defect distribution profile over development phases based on previous releases of the same product
Combine DRM with orthogonal defect classification (ODC) model - profiles defects by individual phases in which they where injected, discovered, and by categories => identify high-defect areas
Observation-based Models
Relate observations of the software system behavior to information about related activities for more precise quality assessments, e.g. SRGMs – estimate parameters based on observation
data
Usually use data from current project
Measurement-driven predictive models Establish predictive relations between quality and other
measurements from historical data Provide early predictions of quality Identify problems early for timely actions Use statistical analysis techniques / learning algorithms Examples:
Relationships between defect fixes and design and code measurements
high-defect modules of legacy products associated with numerous changes and high data complexity
high-defect modules of new products associated with complex design and control structures
Identify High-risk areas in Development Relationship between defect fixes and
various design and code measurements High-defect modules of legacy products associated with numerous
changes and high data complexity High-defect modules of new projects associated with complex design
and control structures
Model Comparison and Interconnections Comparisons based on
usefulness of modeling results, how accurate quality estimates are, and applicability of models to different environments
Model inter-connections examined in two opposite directions
Customization required of generalized quality models to create product-specific models
Generalization of product specific models when enough empirical evidence from different products or projects is accumulated
Comparisons Usefulness can be weighted against cost (such as
collecting data) Generalized models more widely applicable and
less expensive to use (do not require product-specific measurements)
Generalized models more useful in product planning stage and early development phases – when product-specific data unavailable, except when historical data exists in which case semi-customized models are better
More Comparisons Observation-based and Measurement-
based predictive models better manage QA activities and later development and maintenance activities as more measurement data collected
More Comparisons Counterparts in generalized models to product-
specific models and vice versa Generalized models can be customized into product-
specific ones Product-specific models can be generalized
Depends on kind of measurement data collected and analysis results available
Data Requirements and Measurement Different models have different data
requirements (direct and/or indirect)
Generalized models based on industrial averages and general profiles for
all products or product segment. No data from current project needed directly But measurement taken at current project can be
accumulated into empirical base to calibrate models for future applications
Data Requirements and Measurement for Product-Specific Models Measurement-driven models
need direct quality measurements and indirect quality measurements (process, product and people)
need early measurements from historical / current releases
Semi-customized models indirect environmental measurements to characterize
current project extrapolate quality estimates from previous releases use course-grain activity measures
Data Requirements and Measurement for Product-Specific Models Observation-based models
direct quality measurements environmental characteristics assumed
Data Requirements and Measurement (Table 19.5)
Models Supported by Kinds of Data Direct and indirect quality measurements from
industry form empirical basis for generalized models
Direct quality measurements used in all product-specific models product-specific extrapolations in semi-customized
models development activities in observation-based models predicted by early measurements in measurement-
driven models
Models Supported by Kinds of Data Environmental measurements mainly used in semi-
customized models characterize current product to make extrapolations
Product internal measurements used in measurement-driven predictive models early assessment of product quality identify problematic areas
Activity measurements used by various models course-grained used in semi-customized models, e.g.
defect data grouped by phase. fine-grained used in observation-based models
Summarized in Figure 19.3
Selecting Measurements and Models Use a goal-oriented approach (GQM)
Set specific quality goals (e.g. high reliability) Choose specific quality assessment models that can
answer our concerns (e.g. SRGMs) Choose appropriate measurements (e.g. failure and
test execution time measurements)
Examples A - C in text.