trust iv introduction to non functional and performance testing...avoiding unpleasant surprises
DESCRIPTION
The following presentation is an introduction to Non Functional and Performance Testing at Trust IV. Written by sales lead, Ian Mills, the presentation looks to define and describe the types of non functional testing. For more information visit Trust IV online - http://trustiv.co.uk/ or check out our blog - http://blog.trustiv.co.uk/TRANSCRIPT
Non Functional Testing
Avoiding unpleasant surprises
Agenda
• Definition• Types of Non Functional Testing• Performance Testing– When to Performance Test– Performance Testing Process– Trust IV Performance Testing
• Other Non Functional Testing
Non Functional Requirements
• Requirements of the system not related to business function for example:– Scale– Response time– Service hours– Availability– Recoverability– Security– Operability– Servicability
Non Functional Requirements
• Often– Not captured– Not considered– Assumed– Not tested
• But when not met can make lead to:– total business failure of the application– brand damage– the nastiest of nasty surprises
Non Functional Testing
• Involves the use of tools to simulate user activity, with the benefit that once captured the test script is re-usable
• Needs a near production infrastructure available for it
• Requires much more understanding of the infrastructure
• Is not easy• Is often abandoned
Performance Testing
• Tests for Scale and Response Time in a statistical framework
• Is the most common non functional testing carried out
• Is almost never an exact copy of the full production system
• Is not required for every application
When to Performance Test
• As a result of a risk analysis when one or more of the following apply: – First use of new infrastructure– Exacting requirements for response time or
scale– Business criticality – Regulatory change– Public visibility
• As soon as you can
Performance Testing
• A consultancy lead process:– Identify elements that require test– Identify test characteristics for those elements– Design test environment and tests
• Build test environment , simulating elements not to be tested
• Capture test scripts • Run test• Analyse results
Trust IV Performance Testing
• Making Performance Testing more affordable :– Experienced test consultants who ensure that the most
appropriate choices are made to protect test integrity but minimise the effort
– Skilled technicians that can operate a range of tools effectively– Proven test project managers who ensure that they are carried
out to time and budget
• Making it possible to test more widely and regularly
• Getting rid of unpleasant surprises
Other Non Functional Testing
• Uses simulated load to test other aspects of the system:– Failover, DNoS, batch overrun, etc.
• Many similarities to Performance Testing:– Consultancy lead– Extensive use of specialist tools– Complex test environments
• Done less frequently and hence an even greater value in experience
The Changing Balance Between Risk and Cost• The Risk is combination of probability and
impact– Probability is increasing as complexity rises– Impact is increasing as more systems are
exposed to the public • With Trust IV’s experience the Cost of Non
Functional Testing is now lower and can be confidently predicted
• The balance has changed significantly