my 10 mobile automation questions
TRANSCRIPT
Hello!I’m Ran Ben Aharon
Automation & Tools lead @ EverythingMehttp://ranbena.com@ranbena
Y u so hard
▣ No widely used solution▣ Hard to get advice▣ Not everything fits▣ Requires lots of attention▣ Devices fragmentation
18,796 devicesDistinct devices (Aug ‘14)
150 sizesDistinct screen dimensions
2.5% usersLatest OS adoption rate
via OpenSignal
Different from desktop
▣ Not yet mature▣ No common open source solution▣ Device fragmentation▣ Users interact differently▣ Device resources are limited▣ App store limitations▣ Rapid deployment
No solid open source solution
▣ Maturity▣ Single platform*▣ Single dev language*▣ No one-stop-shop
CI > Testing fw > UI simulation fw > Visibility
▣ Each app has it’s unique needs□ None have ALL features□ Concentrate on UI testing
* Excluding Appium
The big boys
EbayCalabash
Mobile automation at Ebay Slides
MozillaRobocop for AndroidMarionette for Firefox OS
Automated testing for Firefox OS
SoundCloudRobotium for AndroidFrank for iOS
FacebookSelendroid for Android
YelpKIF for iOSEspresso for Android
ZapposKIF for iOS
Emulator▣ Cheap▣ Fast▣ Multiple on 1 comp▣ Sharable▣ Mock battery,
sensors, network▣ No hw fatigue▣ Low maintenance
Real or Emulator?
Real device▣ Reliable results▣ Any model available▣ No need for
resource emulation
Pros
▣ Wide range of devices▣ Scale▣ Zero maintenance▣ Real locations*▣ Real cell networks*▣ Your test fw + CI▣ Built-in stress testing*▣ Manual testing*
Device cloud
To name a few
Appthwack
Appurify
Experitest
Keynote
Perfectomobile
SauceLabs
TestDroid
...
Pros▣ Free▣ Community▣ Your code
Open source
To name a few▣ Appium▣ Calabash▣ Espresso▣ Frank▣ KIF▣ Robotium▣ Selendroid▣ MonkeyTalk▣ ...
Android▣ UiAutomator
API 16+ (Appium)
▣ UiAutomationAPI 18+ (Robotium)
▣ InstrumentationAPI 1+ (Espresso, Selendroid, Calabash)
User Action Simulation & Introspection
iOS▣ UiAutomation
(Appium, Zucchini)
▣ UiSpec (Kif, Frank)
Pros▣ Working solution▣ Robust reports▣ Script recording▣ Device cloud
integration*
Proprietary solutions
To name a few▣ Experitest▣ Jamo▣ Keynote▣ Ranorex▣ SOASTA▣ TenKod▣ ...
▣ Unreliable results▣ Not part of release flow▣ No visibility▣ Cumbersome tools
Distrust and disengagement
▣ Reliable results▣ Integral part of release flow▣ Full visibility▣ Delightful tools
Trust and cooperation
Much much more.UI testing is just one aspect out of many.
This is where it gets even more interesting!
Much more than UI testing
▣ Broken connectivity simulation▣ Offline mode▣ Excessive resource usage▣ AB testing / feature switches▣ Non visual events assertions▣ Jank detection▣ API mocks▣ Stats testing
▣ OTA updates▣ Stress testing▣ Image recognition▣ Crash detection▣ Purchase/advert flow▣ Sensors▣ OS event simulation
More info
▣ GTAC - 2014 vids▣ SmashingMagazine article▣ TestDroid blog