W2 Track Session 4/20/2016 10:00 AM

"Uber’s Fascinating World of Inter-App Communications"

Presented by:

Apple Chow & Bian Jiang Uber

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ info@techwell.com ∙ www.techwell.com

Apple Chow Uber Apple Chow is the engineering manager for Uber’s mobile test infra team, which focuses on building tools and infrastructure for Uber’s fast-growing mobile organization. Previously, Apple was the engineering productivity lead at Google where she led releases and worked on building out the testing infrastructure for many of Google’s mobile and web products including Google Hangouts, Google Maps, YouTube, Google Now, Google CRM, Google Offers, Google Checkout, and OpenSocial.

Bian Jiang Uber Bian Jiang has extensive experience in both mobile application development and mobile application testing. For the mobile test infra team at Uber, he helps design and build testing tools and infrastructure, such as Octopus, a multi-app and multi-device coordinating test runner. Bian was a key member of the team which built and shipped the new Yahoo! Mail iOS client from scratch in just twelve weeks.

4/13/16

1

Evolution of Octopus at Uber Apple Chow, Bian Jiang Mobile Dev & Test (April 20, 2016)

Our Uber Challenge Soon after we started at Uber, we encountered a fresh testing challenge. Can you guess what it is?

4/13/16

2

Our Uber Challenge

What makes testing Uber’s mobile apps significantly

different from, say, testing Google Maps?

4/13/16

3

Our Uber Challenge - The Answer

The Driver and Rider App are tied together!

Not Only Our Challenge

Messaging

Apps!

Games!

Anything involving multiple user communication!

4/13/16

4

❶

❷

❸

➍

❺

Octopus Our Uber Useful Solution

4/13/16

5

Why Octopus?

•  Unified •  Extensible •  Parallelized •  Signaling

Octopus Overview

•  Streamlines the following:

- Prepare test targets

- Run tests (handles signaling)

- Create test results reports

- Pull test artifacts

- Perform clean ups

What does it do?

4/13/16

6

Octopus in Action (Parallelized Test Run)

4/13/16

7

Octopus - Deep Dive Bian

Octopus Design Philosophy

•  Easy to Use

•  Robust

•  Integrate with Existing UI Automation Frameworks (UiAutomator, Espresso)

4/13/16

8

Octopus - Signaling

Signaling - Terminology

•  Test Host

•  Test Target

•  Channel

•  Signal

4/13/16

9

Signaling - How it works?

Test Target (driver)

Test Target (rider)

driver_online

requested_trip

Signaling - (very simple) API

•  readSignal(channel, timeout)

- Blocking till a signal or timeout

•  writeSignal(channel, content)

- Nonblocking call

4/13/16

10

Signaling - API calls

Test Target (driver)

Test Target (rider)

writeSignal(“driver_inbox”, “requested_trip”)

readSignal(“driver_inbox”, 10)

writeSignal(“rider_inbox”, “driver_online”)

channel signal readSignal(“rider_inbox”, 10)

channel

timeout

Signaling - Logical Flow

Test Target

Test Target

signal

signal

4/13/16

11

Signaling - Octopus Workflow

Two-way communication Test Target Test Target

Virtual two-way communication Test Target Test Target

Test Host

Signaling in Octopus: File-based Signaling

Test Target (driver)

Test Target (rider)

Test Host

writeSignal(“driver_to_rider”, “driver_arrived”)

“driver_online”

“driver_online”

“driver_online”

4/13/16

12

Signaling in Octopus: Interacting with Test Host

•  How to send a file/string from test target to test host? writeSignal

•  How to watch a file on test host from test target? readSignal

Octopus - Signaling on iOS

•  UIAHost.performTaskWithPathArgumentsTimeout The equivalent of “exec()”

•  writeSignal: exec(“echo ${signal} > ${signal_file}”) •  readSignal: exec(“cat ${signal_file}”)

4/13/16

13

Octopus - Signaling on Android

•  Unfortunately, there’s no direct “exec()”. Everything should be initiated from test host.

•  We use “adb shell” as relayer.

•  readSignal: “adb pull”

•  writeSignal: “adb push”

Octopus Demo #2 (UberPool)

4/13/16

14

Octopus Evolution (Hermetic Tests)

4/13/16

15

New Octopus Challenge: Making the tests stable

Want stable tests to guard against our builds Solution: •  Network record/replay •  Record mode:

•  Records the network responses •  Replay mode:

•  Plays back the network responses from cache

Octopus Evolution (Image Diffing)

4/13/16

16

•  Perceptual diff •  Post-processors (thresholds)

{ "app_name": "rider", "test_name": "com.ubercab.tests.rider.AppLaunchTest#testSignInScreenshot", "post_processors": [ ["ImageDiffPipeline", {"threshold": 5000}] ] }

New Octopus Challenge: Image diffing

Octopus Evolution (Locale Testing)

4/13/16

17

New Octopus Challenge: Locale Testing What is it & Why? •  Localization can mean many things besides Translation:

‐  Date/time format ‐  Number format ‐  Currency ‐  Text direction ‐  Encoding etc..

•  Why? -> Hotfixes ‐  Chinese users see English UI ‐  Hebrew/Arabic users unable to request a ride ‐  Crashes for users from certain regions ‐  Layout/encoding issues

Locale Testing: Our Octopus Solution

•  Test buddy app: Set the locale/network country ISO on simulators/emulators/devices

•  Ability to loop thru different locales for a test, and upload the results to our test results repository

•  Test related ‐  Test should be locale agnostic

▪  text as UI element identifier ▪  text as verifications

‐  Logic should be locale agnostic ▪  some features are not available in all countries ▪  flows may be different etc...

4/13/16

18

Sign-in Test (run against 35 locales) for android rider app Test artifacts (logs, video recording, screenshots) for each locale is stored in its own folder on Jenkins.

Questions?

4/13/16

19

Presented by: Apple Chow & Bian Jiang

Special thanks to mobile-test-infa@, octopus-eng@, mobile-eng@ and all the amazing people at Uber :) Uber eng blog: https://eng.uber.com/rescued-by-octopus/

Come join us: https://www.uber.com/careers/list/13540