heuristics, mnemonics and other greek words in the exploratory testing of mobile apps
TRANSCRIPT
Software quality assurance days20 Международная конференция по вопросам качества ПОsqadays.com
Минск. 24–26 ноября 2016
Владислав РоманенкоIntellectsoft. Минск, Беларусь
Эвристики, мнемоники и другие греческие слова в исследовательском тестировании мобильных приложений
Тестирую мобильные и веб приложения
ISTQB Certified Test Analyst
QA-инженер в Intellectsoft
2
@laechoppe
Reasons forExperience-based testing:
- lack of time - lack of documentation - issues could hardly be located using more systematic approaches
3
Experience-based techniquesError guessing
Checklist-based testing
Exploratory testing
4
Error guessing
5
Checklist-based testing Checklist-based testing
6
Exploratory testing- tests are designed and executed concurrently- usually guided by a charter- lightweight documentation (if any)
7
+ more interesting+ more apparent bugs first+ good if time is limited+ works if docs are absent+ more checks due to variations of tests+ ….
Exploratory testing
- less experienced testers won't get as much as more experienced
- could be hard to reproduce the issues- not suitable if requirements are mandatory- hard to collect metrics- …
8
Heuristic - a set of research methods to facilitate the discovery of a previously unknown
Mnemonic - a learning technique, allowing to remember the sequence and structure of something.
Richard Of York Gave Battle In Vained
range
ellow
reen
lue
ndigo
iolet
9
Mnemonic
Not Every Cat Eats Sardines (Some Are Really Yummy)
?
10
MnemonicNotEveryCatEatsSardines(SomeAreReallyYummy)
11
Goldilocks Principle - in any dataset there’s always a U-shaped distribution
12
Goldilocks Principletoo short text too long text just right text
13
SFDPOT (San Francisco Depot)
S StructureF FunctionD DataP PlatformO OperationsT Time
14
S StructureF FunctionD DataP PlatformO OperationsT Time
What files does it have? Can I download the app?Can I download an update?Does the app write to an SD card? What if the card is filled, removed or changed?
SFDPOT
15
S StructureF FunctionD DataP PlatformO OperationsT Time
SFDPOT
16
S StructureF FunctionD DataP PlatformO OperationsT Time
Does the app perform the tasks or features it was designed to?Does the app or site prevent, block or not include features not intended on a mobile device?Does the app direct me to turn on necessary related services?
SFDPOT
17
S StructureF FunctionD DataP PlatformO OperationsT Time
SFDPOT
18
S StructureF FunctionD DataP PlatformO OperationsT Time
What kinds of input does it process?What does its output look like?Does it come packaged with preset dataDoes the app integrate with any hardware such as bar code scanners for payment or other purposes?
SFDPOT
19
S StructureF FunctionD DataP PlatformO OperationsT Time
SFDPOT
20
S StructureF FunctionD DataP PlatformO OperationsT Time
What operating systems does it run on?Does the environment have to be configured in any special way? Does it depend on third-party components?Does the app depend on any device settings?
SFDPOT
21
S StructureF FunctionD DataP PlatformO OperationsT Time
SFDPOT
22
S StructureF FunctionD DataP PlatformO OperationsT Time
Where and how the app will be used?Does the app function when I am moving and traveling around (switch wi-fi <-> 3G)?What happens when a mobile interruption occurs such as an incoming call or text message?
SFDPOT
23
S StructureF FunctionD DataP PlatformO OperationsT Time
SFDPOT
24
S StructureF FunctionD DataP PlatformO OperationsT Time
What happens if the time zone is switched?What happens when my location is switched?Does the device auto detect a change in time zones and the day and date?Does the app find time-related data based on my device time?
SFDPOT
25
S StructureF FunctionD DataP PlatformO OperationsT Time
SFDPOT
26
SFDPOTIS StructureF FunctionD DataP PlatformO OperationsT TimeI Interface
Does the app work well on phones and tablets?Does the app work well when used with “one hand?”Is the mobile version consist or complementary to the desktop version?Is the app consistent with UI current constructs in the marketplace?
27
SFDPOTIS StructureF FunctionD DataP PlatformO OperationsT TimeI Interface
28
SFDPOTIES StructureF FunctionD DataP PlatformO OperationsT TimeI InterfaceE Errors
Does the app continue gracefully after an error?Is the message helpful, instructive?If a user provokes multiple errors in succession, does the app continue?Can I get to an error state that forces the app to shut down?What happens when I restart the crashed app – is previous data or state saved?Is there a method to send an error report?
29
SFDPOTIES StructureF FunctionD DataP PlatformO OperationsT TimeI InterfaceE Errors
30
I SLICED UP FUNI InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
31
I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
I SLICED UP FUN
32
I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
I SLICED UP FUN
33
I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
I SLICED UP FUN
34
I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
I SLICED UP FUN
35
I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
I SLICED UP FUN
36
I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
I SLICED UP FUN
37
I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
I SLICED UP FUN
38
I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
I SLICED UP FUN
39
I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
I SLICED UP FUN
40
I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
I SLICED UP FUN
41
I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
I SLICED UP FUN
42
I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
I SLICED UP FUN
43
I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network
Start from
I SLICED UP FUN
44
Heuristics help to:
find out what to start testing fromremember the context of app usageavoid repeating the mistakes…
TO SUM UP:
45
эвристикамнемоникатехникатеориядиапазонсистемныйпараллельнометодсимвол
идеяэргономикателефонбиблиотекаграфикмикрофонпараметрриск…
Some of the Greek words:
46
http://www.rockynook.com/press-release-software-test-engineers-handbook-2nd-edition/
http://www.computing.dcu.ie/~ray/teaching/CA358/dorothy_graham.pdf
http://karennicolejohnson.com/2012/05/applying-the-sfdpot-heuristic-to-mobile-testing/
http://karennicolejohnson.com/2012/12/applying-the-sfdpot-heuristic-to-mobile-testing-adding-i/
http://karennicolejohnson.com/wp-content/uploads/2012/11/KNJohnson-2012-heuristics-mnemonics.pdf
https://www.utest.com/articles/rotation-tour-in-mobile-testing
http://www.kohl.ca/articles/ISLICEDUPFUN.pdf
http://www.satisfice.com/articles/sfdpo.shtml
http://michaeldkelly.com/blog/2005/9/20/touring-heuristic.html
https://goritskov.com/blog/heuristics-in-testing.html
Pictures from: mycutegraphics.com
References
47