acceptance testing and measuring quality - ida.liu.setddc93/timetable/4acceptancetesting-1.pdf ·...
TRANSCRIPT
![Page 1: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/1.jpg)
Acceptance testing and measuring
quality
Lecture 4
Kristian Sandahl
Department of Computer and Information Science
Linköping University, Sweden
Software Engineering
TDDC88/TDDC93
Autumn 2008
![Page 2: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/2.jpg)
2
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Theory Lecture Plan
L1 - Course Introduction and Overview
L2 - Project Management
L3 - Requirements
L4 - Acceptance Testing and Quality Factors
L5 – UML
L6 - Design Patterns
L7 - System Design and Architecture
L8 - Testing Theory
L9 - Testing in Practice
L10 - Inspection
L11 - Software Life Cycles and Configuration Management
L12 - Software Quality Management
L13 - Course Summary, Exam examples, Questions
L4 - Acceptance Testing and Quality Factors
![Page 3: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/3.jpg)
3
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
A Software Life-cycle Model
Which part will we talk about today?
Requirements
System Design(Architecture,
High-level Design)
Module Design(Program Design,Detailed Design)
Implementationof Units (classes, procedures,
functions)
Unit testing
Module Testing(Integration testing of units)
System Testing(Integration testing of modules)
Acceptance Test(Release testing)
Validate Requirements, Verify Specification
Verify System Design
Verify Module Design
Verify Implementation
Project Management, Software Quality Assurance (SQA), Supporting Tools, Education
Maintenance
![Page 4: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/4.jpg)
4
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Agenda - What will you learn today?
Part I – Acceptance testing Part II – Software quality factors
Part III – Critical systems
"Reliability"
0,9
0,91
0,92
0,93
0,94
0,95
0,96
0,97
0,98
0,99
1
0 5 10 15 20
Series1
![Page 5: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/5.jpg)
5
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Part I
Acceptance testing
![Page 6: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/6.jpg)
6
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Requirements - short revisit
Elicitation Analysis Specification Validation
Many sources
IdeasConceptual model
Negotiation
Formal analysis
Written SRS
Detailed use-cases
Detailed models
SimulationNotes
Requirements
System
Result
Decision
![Page 7: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/7.jpg)
7
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Validation of requirements
Before design and coding
� Inspections
� Cross-referencing
� Interviews
� Checklists
� Scenarios
� Proofs
� Model validation
� Simulation
� Prototyping
![Page 8: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/8.jpg)
8
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Acceptance testing
� Purpose
� Benchmarking
� Pilot testing� alpha test
� beta test
� Installation testing
� Parallel testing
![Page 9: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/9.jpg)
9
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Acceptance testing - documentation
� Test plan� Identify test items� Plan resources, tasks, schedule
� Test design specifications� Test techniques to be used� Analysis methods� How to handle groups of tests
� Test case specifications2.1. Test case identifier2.2. Objective eg priority2.3. Inputs2.4. Outcome(s)2.5. Environmental needs2.6. Special procedural requirements – pre- and post processing2.7. Intercase dependencies – order of execution
IEEE-std 829
![Page 10: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/10.jpg)
10
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Acceptance testing - documentation
� Test procedure specifications� Order of each step for all participants
� Test logs� Results
� Changes to plans
� Test anomaly report� Things worth investigating
� Known impact
� Recommendation
� Test summary reports
IEEE-std 829
![Page 11: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/11.jpg)
11
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Part II
Software quality factors"Reliability"
0,9
0,91
0,92
0,93
0,94
0,95
0,96
0,97
0,98
0,99
1
0 5 10 15 20
Series1
![Page 12: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/12.jpg)
12
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Many opinions ⇒⇒⇒⇒
Statistical
techniques
Many opinions ⇒⇒⇒⇒
Statistical
techniques
Perspectives of quality
� Transcendent – something we learn to recognize
� Product-based – measurable variable
� Usage-based – in the eyes of the beholder
� Manufacturing-based – conformance to requirements
� Value-based – market sets the value
![Page 13: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/13.jpg)
13
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Quality factors
� Correctness
� Reliability
� Efficiency
� Usability
� Integrity
� Maintainability
� Flexibility
� Testability
� Security
� Portability
� Reusability
� Interoperability
� Survivability
� Safety
� Manageability
� Supportability
� Replaceability
� Functionality
Price?
![Page 14: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/14.jpg)
14
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Quantification of Quality Factors
Quality factors must be quantified to:
� State precise requirements
� Validate the results
� An important class of non-fuctional requirements
� Original: � The system’s usability must be optimal
� Better:� The users shall consult the help function no more than once
a week on average
![Page 15: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/15.jpg)
15
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Testing the usability requirements
![Page 16: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/16.jpg)
16
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Usability
� Relevance
� Efficiency
� Attitude
� Learnability
� Metrics:� http://www.ida.liu.se/~TDDC93/timetable/Usability metrics.pdf
![Page 17: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/17.jpg)
17
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Usability testing
� Think-aloud test
� Observation test
� Automatic logging
� Heuristic evaluation 50% hit rate
� User review with expert users
� Avoid trade-union representatives and IT-experts
Normal users
![Page 18: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/18.jpg)
18
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Testing process
� Input: artefact, test tasks (realistic, closed)
� Participants: test user, facilitator, log keeper
� Explain purpose
� Background and daily matters
� Task
� Observe
� If necessary: give hints
� Debriefing
� Output: List of problems
![Page 19: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/19.jpg)
19
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Usability problem severity classes
� Missing function
� Task failure
� Annoying
� Medium problem
� Minor problem
![Page 20: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/20.jpg)
20
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
User stories in eXtreme Programming
� The customer is always available
� The customer writes user stories, ex:� The user is presented with information of all phones
connected
� The user changes parameters for a phone
� The user activates a dialup connection
� The user inactivates a dialup connection
� Embrace change
� Make frequent releases – one per week
Lundsten & Holst LiTH-IDA-Ex-03/068-SELundsten & Holst LiTH-IDA-Ex-03/068-SE
![Page 21: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/21.jpg)
21
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Reliability
� The probability that the software executes with no failures during a specified time interval
� Approximation: MTTF/(1+MTTF)
� Example:� http://www.ida.liu.se/~TDDC93/timetable/failure-based.xls
![Page 22: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/22.jpg)
22
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Failure intensity
� Easier to manage: Failure intensity, [failures / hours of execution time]
� Another approximation: λ = (1-R)/t
� Example
![Page 23: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/23.jpg)
23
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Software reliability testing
� Define target failure intensity
� Develop operational profile
� Plan tests
� Execute test
� Apply data to decisions
usage testing
![Page 24: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/24.jpg)
24
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Failure intensity guideline
1 h1$1 cost
10 h10-1$10 cost
100 h10-2$100 cost
6 weeks10-3$1000 cost
114 years10-61-2 deaths, $106 cost
114 000 years10-9Hundreds of deaths, $109 cost
Time btwn failuresFailure intensityImpact
![Page 25: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/25.jpg)
25
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Develop operational profiles
� Find operations and their relative probability
� An operation is a major system logical task of short duration which returns control to the system when complete and whose processing is substantially different from other operations.
� Operational modes (e.g. day, night)
� Representation (tabular, graphical)
� Ignore non-critical with p < 1-Rtarget
� How do you find and asses probabilities of operations?
![Page 26: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/26.jpg)
26
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Prepare for test
� Feature test – check that each operation work properly
� Load test – mimic filed usage
� Tests are executed in runs comprising:� Operation,
� input state,
� direct and indirect variables
� A test case is defined as a run with named direct variables and values
� A test case is independent of the operational mode
� Thus a test case can generate multiple runs
� Problem: how to select test cases?
![Page 27: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/27.jpg)
27
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Select test cases
� Determine the total number of test cases.
� Sanity check: #new operations + 100
� Distribute test cases proportionally to operations
� Assign at least one case per operation
� Accelerate rare, critical operation
� Maximise variable distance in input space
![Page 28: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/28.jpg)
28
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Execute tests
� Run test cases
� When a failure is found, log the time, correct the fault (don’t count twice)
� Decisions made according to reliability growth criteria
� Don’t count multiple failures due to single fault
� Run performance tests simultaneously
![Page 29: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/29.jpg)
29
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Part III
Critical systems
![Page 30: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/30.jpg)
30
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Critical systems
� When failed a critical system can cause injury of people, damage the environment or loose immense sum of money.
� Criticality is often expressed in terms of:� Reliability
� Availability
� Maintainability
� Safety
� Security
� Critical systems makes expensive methods worthwhile and needs experience
What horror stories
(true or false) do you
know of?
What horror stories
(true or false) do you
know of?
![Page 31: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/31.jpg)
31
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Hazard, Incident, Accident
� Hazard – System state or condition that may lead to an incident.
� Incident – Potentially dangerous system behaviour
� Accident – Unplanned sequence of events that lead to human death, severe injury or big loss of resources
![Page 32: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/32.jpg)
32
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Create safety requirements checklists
� Encapsulation of experience
� Avoids errors by omission
� Keep the list short, and there is only small costs
Examples:
� Safe start-up
� Initialise variables
� Manual override
� Off-line behaviour
� Time-out
� Unexpected input
� Possible sensor data
� Output values produced
� Value ranges
� Reasonabless of output
� Timing delays
� Overload behaviour
� Alarm management
� Data ageing
� Exception handling
� Reversible commands
� Fail-safe states
� Error detection and recovery
![Page 33: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/33.jpg)
33
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Involve external reviewers in
the validation process
� External people lacks commitment to the product and the company
� Wisely selected, the union of competence can cover a vast area
� High costs in time and money
� The inspection process can limit the contribution of members
� Nominal groups perform best
� A meeting can be needed to gain personal commitment
![Page 34: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/34.jpg)
34
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Identify and analyse hazards
� A panel of experts can anticipate hazards
� Experiment with a suitable process in non-critical applications
� Spend time on training
� Use brainstorm-like mode of communication
� Costly
� Analyse the system from different viewpoints, for example:
� potential victims
� events in the system environment
� behaviour of sub-systems
� properties of hardware platform
� properties of special software
![Page 35: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/35.jpg)
35
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Define safety requirements
from hazard analysis
� Natural continuation of analysis
� Specify:� avoidance
� tolerance
� detection
� recovery
� minimising damage
� handling sub-sequent hazards
� Often a matter of severity analysis and prioritisation
![Page 36: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/36.jpg)
36
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Cross-check operational and functional requirements against safety
requirements
� Complements the hazard analysis
� Check the do’s against the dont’s
� Create a traceability matrix Safety requirements ☓Functional requirements
� Determine the relationship
� Handle the potential hazards
![Page 37: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/37.jpg)
37
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Specify systems using
formal specification
� Problems seem to be finding the most appropriate formalism and training
� Experiment, and hire competent people!
![Page 38: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/38.jpg)
38
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Collect incident experience
� An incident database helps to:� training
� create and refine checklists
� The key to success is the retrieval functions
� A problem is that elements on very different levels of abstraction contribute to incidents
![Page 39: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/39.jpg)
39
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Learn from incident experience
Use incident data to:� refine checklists
� refine incident description
� refine severity classes
� create general requirements
� reuse requirements
![Page 40: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/40.jpg)
40
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Establish an organisational safety culture
� Clear message from management
� Problem reporting channels
� Clear responsibility
� Egoless style of working
� Incentive and reward system
� Competence development
![Page 41: Acceptance testing and measuring quality - ida.liu.seTDDC93/timetable/4AcceptanceTesting-1.pdf · System Design (Architecture, High-level Design) Module Design (Program Design, Detailed](https://reader035.vdocuments.us/reader035/viewer/2022062911/5c97615f09d3f29f7b8c12d9/html5/thumbnails/41.jpg)
41
Part I
Software quality factors
Part II
Usability
Kristian Sandahl
Part III
Reliability
Part IV
Critical systems
Summary - What have we learned today?
� Software quality is multi-faceted often in the eyes of the beholder.
� Usability can and shall be measured during a prototyping development project.
� Reliability can be assured through statistical usage testing
� Critical systems development needs experience.