test automation nightmares - antti heimola, qentinel

© Qentinel Group 2016

GUI Test Automation Nightmares

Test Automation Clinic Nov-10, 2016Antti Heimola

© Qentinel Group 2016 CONFIDENTIAL 2

Content

#1 Strategy and Operations

#2 Design for Test Automation

#3 Human Psychology

#4 Success – Hype vs. Chasm

Some hints how to avoid Nightmares

Some other things to avoid Nightmares


#1 TA Strategy and Operations

Die Quickly

Die Slowly

Thrive

Survive

Low Effectiveness High

Lo

w

E

ffic

ien

cy

H

igh

Doing the right things….

…right

Stephen Bungay:

“Strategy is about doing

the right things.

Operations is about

doing things right.”

#1 Rarely seen any proper Test Automation strategy, nor Test Automation Operations – “We need

a Test Automation Engineer”


#2 Design for Test Automation (back in 2000)

• Test automation is a fulltime effort, not a sideline

• The test design and the test framework are totally separate entities

• The test framework should be application-independent

• The test framework must be easy to expand, maintain, and perpetuate

• The test strategy/design vocabulary should be framework independent

• The test strategy/design should remove most testers from the complexities of

the test frameworkhttp://safsdev.sourceforge.net/DataDrivenTestAutomationFrameworks.htm

#2 Nothing new since 1990’s, but still TA projects keep repeating the same mistakes – “We need a

TA Engineer to design automated test scripts in the project”


#3 Human Psychology

Miller's law, 1959: “The Magical Number Seven, Plus or Minus Two”

Dunbar's Number, 1990s: “Why We Can't Have More Than 150 Friends”

Lessons Learned in Test Automation projects: “Try to keep number of KeyWords

below 10 and number of ActionWords below 150” – GUI based Test Automation

is NOT like a software project having hundreds of separate functions in dozens

of different files

#3 Keep it simple to avoid your TA system corruption – “Every GUI SW developer is responsible

for designing own automated test scripts the way they want”


#4 Success - Hype vs. Chasm

https://setandbma.wordpress.com/2012/05/28/tech

nology-adoption-shift/

#4 Most TA projects just

fail to cross the chasm

becoming a burden.

“Test automation is

progressing well”


Some hints to avoid Nightmares

1. Strategy and Operations

a) What you want and why - next year, in 3-5 years

b) Think about How & Who – get the best possible people

c) …..

2. Design for Test Automation

a) Separate test automation system development and scripting

b) Support several projects, not one project only

c) Lean Thinking

d) ……


Some hints to avoid Nightmares

3. Human Psychology – 7/150

a) Make it simple to use, so that people want to use your test

automation system – not just you or your team

b) …

4. Success

a) When the hype is over, work harder and harder

b) You must show things that surprise people continuously to cross

the chasm and to get the market share

c) …


Some other things to avoid Nightmares

• Management interest and support is needed

• Competence is very important, because TA is very difficult

• Focus on SW development process, because issues come from there

• Force contractors to release high quality SW only, define DoD

• Set goals, like Continuous Deployment, Continuous Improvement, etc.

• Test Environment and Data management – things to do first

• Remove waste – managers, meetings, useless features, etc.

• Do not try to reach 100%. Spending 20% of your time you reach 80%

which, in most cases, is enough – 5 x productivity!


Keskusteluaihe:

Miksi testiautomaatio yleensä epäonnistuu?

• Voi käyttää ”5 Whys”-tekniikkaa

• Jos ei kokemuksia epäonnistumisista, niin voi myös keskustella

onnistumisista

30 minuutin ryhmäkeskustelu


Contact [email protected] Qentinel Group www.qentinel.com

test automation nightmares - antti heimola, qentinel

Technology