An empirical process to get an assessment of the quality of the quality

of a systemNever exhaustive

one can never prove the system will workComparison against expected results

requires correct preparation of expected results

Assumes a sequence of events what if this sequence is not always followed ?

Forms• Static : review, inspection,…• Dynamic : execution

Functional • Unit – examples : input validation, calculation, formatting• System – examples : website with interfaces stubbed out, website

running against test customer database• Integration – full functioning system – as close as possible to

production• Conversion – when you upgrade• Acceptance – the customer signs off

Non functional• Scaling• Duration• Security• Performance

Project Initiation 4%

Project Management 10%

Training 12%

Communications 4%

Analysis 9%

Design 10%

Build 25%

Test 17%

Roll Out 8%

Evaluation 1%

For larger – 10 + people projectsFor smaller projects – testing cost increase

testing.avi

Unstated mental modelsIt compiled so it will workI hate MS$&^*T stuff It is written in PHP so it worksShow that the website worksEveryone has broadband…

Imperative that you break this mental block

Repeat/recreate the errorOther browsers, OS-s,…Different sequenceDifferent data – input and DB-sDifferent HW (video, memory, disk,..), SWDifferent connection speedsDifferent connectivity paths…

First assumption has to be: Your application has an error

The “owner’ is way to invested into the system to truly test it• Main focus will be to prove the system

works – not on trying to break the system• To aware of the many assumptions that

went in the design/implementation• Quite often the system is too large to be

tested by an individual; there are too many variations

Create unit tests for complex lower design issues Test driven development• Helps clarify the requirements/design• Helps automate

Document larger tests in a form that works for you • scenarios, use cases in word document, excel,…• Document data, steps and outcomes• If possible automate• Do not forget the exceptions, errors,..

Execute these tests in an environment that is not your development environment; maybe everything you execute them use a different browser, OS VM-s are your friend

If none of your tests fail:find someone else to test

Focus first on the common scenarios Then focus on common exceptions Then focus on rare exceptions Then….

• MONKEY test

Continuously test – if you wait till the end – it will be very, very, very tedious

If you find lots of errors in a certain subsystem, and they keep coming back (ie you can not seem to fix it), toss it and rewrite it

It is probably the most undervalued activity to ensure you site works

There are truly testing geniuses – if you find them try and hire them..• Able to perform repetitive tasks with high focus• Have fingers that make code break• Have an incredible attention for detail• Are very productive in finding high quality errors

Questions…..