ebay - a look behind the scenes

EBAY A LOOK BEHIND THE SCENES

MICHAEL PALOTAS – HEAD OF QUALITY ENGINEERING EUROPE JANUARY 8 2013

TOPICS TODAY

• Overview about eBay • Our current challenges • How we see the future • How do we build and test software • A little bit of management talk • Q&A

FH HEILBRONN /// EBAY INTERNATIONAL 2

WHO AM I?

MICHAEL PALOTAS HEAD OF QUALITY ENGINEERING EUROPE [email protected] [email protected] +41 79 6690708 http://ch.linkedin.com/in/michaelpalotas http://www.xing.com/profile/Michael_Palotas


THE COMPANY

4 FH HEILBRONN /// EBAY INTERNATIONAL

EBAY INC.

5 FH HEILBRONN /// EBAY INTERNATIONAL

LOCATIONS


EBAY FACTS

• Founded in 1995

• Based in San Jose, California

• 28000 employees worldwide

• >100 million active buyers and sellers worldwide

• Revenue: 11.6B USD

• 68 Billion USD transaction volume in 2011

• 2000 USD transaction volume every second


• >10 Petabytes of data in Hadoop and Teradata clusters

• 2 billion page views every day

• 75 billion database calls every day

• >100 million downloads of eBay Inc’s mobile apps

• Mobile transaction volume will double in 2012 to 10 billion USD

• 1.7 million items listed via mobile apps every week

EBAY FUN FACTS


• Every 30 seconds, a woman’s handbag is purchased on eBay Mobile.

• Every 2 minutes, a tablet is bought through eBay Mobile

• Every week, more than 8,000 cars and 340,000 car parts are sold through eBay’s mobile apps.

• 7,493 tablets sold weekly from April – June; enough to cover the surface of 3 Olympic-sized pools.

EBAY MARKETPLACES ORGANIZATION

• Offices around the globe

• 1000+ developers

• 300+ test engineers

• Main development center in San Jose

• European Headquarter in Zürich

• Waterfall … Agile… and everything in between


OUR CHALLENGES & OPPORTUNITIES

• Scaling • Mobile

• Cross Border Trade

• Ever faster release cycles • Trust


WHAT’S IN THE FUTURE


http://www.wahrsagen-online24.de/

HOW WE WORK


LOCALIZATION


SETUP

14

EU Quality

Engineering (Zürich, London,

Berlin, Paris)

Global Product

San Jose

USA

Global Product

Shanghai

China

Global Product

Chennai

India

Local Product

Zürich, London

Mobile

Portland USA

Adjacencies (i.e. Brands for Friends)

FH HEILBRONN /// EBAY INTERNATIONAL

NOT ONLY THE WEB

15

Windows Desktop

Mobile


TESTING – OWNERSHIP OF QUALITY

16

Manual Test

Automation

Consulting +

Coaching


CORE TESTING

MOBILE

INNOVATION

TEST AUTOMATION – WHY?

• NOT to replace people

• To save time and to reinvest that time into manual testing

• To get early and instant feedback

• Regression testing only

• Safety net


AUTOMATION IS GOOD…BUT

•  Tools help, but tools don’t test •  Manual testing is equally important •  Manual (exploratory) and automated testing complement each other •  Most bugs are found during manual testing / while automating


Automation

Manual

WHAT DO WE DO WITH SELENIUM?

End to End functional test automation


Unit Tests

E2E Test

s

Manual Tests

Integration Tests

Selenium space

HIGH LEVEL INFRASTRUCTURE


vCenter Server

Selenium GRID HUB

Web

Frontend ESX

ESX

API

DB

ESX

ESX

Test Type

TestNG

ESX

ESX

SELENIUM GRID


Sequential Execution

Execution time: 7000 minutes (~116 hours = ~5 days)

Test 1

Test 2

Test …

Test 4500

Execution Time

Test 3

-  4500 end-to-end regression test cases -  2-3 minutes execution time for each test

Parallel Execution

Execution time: 60 minutes by using 150 Virtual Machines / Browsers using the Selenium GRID

Test Test Test

Execution Time

Test

Test Test Test Test

Test Test Test Test

Par

alle

l Exe

cutio

n

Par

alle

l Exe

cutio

n

BUG QUALITY

22

Bug Quality = accepted bugs / submitted bugs

75

80

85

90

95

100

Q1 08 Q2 08 Q3 08 Q4 08 Q1 09 Q2 09 Q3 09 Q4 09 Q1 10 Q2 10 Q3 10 Q4 10

Bug Quality in % (2008-2010)

Bug Quality 2008-2010

Start of automation efforts


MOBILE

EBAY’S MOBILE PRODUCTS


Android Native App iOS Native App Mobile Web

FIRST CHALLENGE: LOCALIZATION (L10N)


SECOND CHALLENGE: DEVICE DIVERSITY


DEVICES VS. EMULATORS


MANUAL MOBILE TESTING

• Feature testing is manual

• Concentrate on top 5 devices

• Layout, User Experience, Localization (i.e. does the text fit on the button)

• In depth EU specific domain knowledge


MOBILE TEST AUTOMATION REQUIREMENTS

• Use load balancer features of Selenium GRID for parallel execution

• Manage multiple applications / versions / languages

• Native app should not need to be modified

• Native app runtime inspection support


THE IMPORTANCE OF PARALLEL TEST EXECUTION

•  Multiple Apps •  12 European Sites •  iOS Devices

•  iPhone vs. iPad •  Version

•  Android Devices •  Manufacturers •  Screen Sizes •  OS Modifications


CHECK IT OUT ON GITHUB

• https://github.com/freynaud/ios-driver

• http://calabash-driver.github.com


A LITTLE MANAGEMENT TALK

THE PERFECT EMPLOYEE


Sackmesser.ch

WHO ARE WE LOOKING FOR?


Sackmesser.ch

INNOVATION BY PRESCRIPTION


OFFSHORING


http://www.guardian.co.uk/business/2011/jan/06/viewpoint-bp

INNOVATE…AND TALK ABOUT IT

Sydney, Australia, October 5th 2011

Selenium GRID has 1.5K downloads / day


THE LONG LEASH


HYGIENE FACTORS


http://betterafter50.com/wp/wp-content/uploads/2011/10/google-adsense-more-money-tips.jpg

THANK YOU!

COMMENTS?

WE ARE HIRING!

http://ebaycareers.com/ Or contact me: [email protected]

ebay - a look behind the scenes

Documents

challenge

www

localization