© 2014 eBay Inc. 1

Monitoring and Managing Performance

© 2014 eBay Inc. 2© 2014 eBay Inc. 2

© 2014 eBay Inc. 3© 2014 eBay Inc. 3

© 2014 eBay Inc. 4© 2014 eBay Inc. 4

© 2014 eBay Inc. 5© 2014 eBay Inc. 5

eBay Classifieds drives connected commerce by creating local marketplaces across the globe, powered

by simplicity, enriched by personal interaction.

eCG’s vision is to be the undisputed global classifieds leader

© 2014 eBay Inc. 6© 2014 eBay Inc. 6

eCG Group Maturity

Market Share

Presence Seeds High Growth Established

Canada

Denmark

Germany

UK

South Africa

SpainUS

Italy

AR

Colombia

Poland

Taiwan

Singapore

Australia

Netherlands

HK

Mexico Revenues

Develop Geo Winners

Deploy Classifieds

Full Potential

Innovate to Sustain Growth

Mobile `

© 2014 eBay Inc. 7© 2014 eBay Inc. 7

Our history of acquisitions has been a major driver of Tech complexity

eCG Growth Story

Partnerships

Source: eCG analysis

Acquired

Growth

Organic

Growth

Tech Stack

s

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

© 2014 eBay Inc. 8© 2014 eBay Inc. 8

eCG NOC

Protecting the experiences our customers love

A team working in 24x7 rotation dedicated to:

© 2014 eBay Inc. 9© 2014 eBay Inc. 9

Monitoring Through the Stack and Beyond

Outside view

Network Infrastructure

Internal Systems

© 2014 eBay Inc. 10© 2014 eBay Inc. 10

Use Case: Validating performance of own Datacenters

Alert notification arrives:

Identify ‘Own Datacenter’ as culprit by inspecting waterfall drill down (First Byte Time)

© 2014 eBay Inc. 11© 2014 eBay Inc. 11

Use Case: Validating performance of own Datacenters (continued)

Find Corresponding Public VIP, policy, pools and pool members in Observium

At the Public VIP level we establish we are not seeing a spike in requests, we’re operating well within capacity limits

© 2014 eBay Inc. 12© 2014 eBay Inc. 12

Use Case: Validating performance of own Datacenters (continued)

We only get connections and requests at pool level, we need to descend to pool member level to getTime-to-first-byte. A spike in time-to-first-byte will typically manifest itself as an increase in connections at pool level

In this example we identified a hiccup in time to first byte for one member of the pool serving this particular VIP, which trickles up in the number of open connections at the pool level. We establish there is no further site impact, and hand over to responsible operations team for root cause analysis.

We might also find an overall increase of time-to-first-byte after a new release. At release time these trends are monitored proactively, so typically we’re on top of such cases before external monitoring triggers an alarm.

© 2014 eBay Inc. 13© 2014 eBay Inc. 13

Use Case: Monitoring CDN performance of image delivery

Some of our territories do not benefit from image delivery through CDN, as our own network is ‘closer’ to the end user than any of the CDN nodes. By measuring through Gomez / APMaaS Last Mile measurements we were able to identify these cases. Based on the results, we have moved to a solution where we’re caching in our own datacenter.

As a follow up, we performed a CDN shoot out between two CDN vendors, we ended up switching vendor to improve user experience across our portfolio worldwide.

© 2014 eBay Inc. 14© 2014 eBay Inc. 14

Use Case: Monitoring CDN performance of image delivery (continued)

i.eba

yimg.

com

(2.1

6.62

.x)

i.eba

yimg.

com

(88.

221.

93.x)

i.eba

yimg.

com

(213

.241

.88.

x)

i.eba

yimg.

com

(77.

67.9

6.x)

i.eba

yimg.

com

(92.

122.

216.

x)

i.eba

yimg.

com

(2.1

6.21

8.x)

i.eba

yimg.

com

(88.

221.

92.x)

i.eba

yimg.

com

(5.1

78.4

0.x)

i.eba

yimg.

com

(90.

84.6

0.x)

i.eba

yimg.

com

(80.

157.

149.

x)

i.eba

yimg.

com

(193

.45.

10.x)

i.eba

yimg.

com

(80.

239.

228.

x)

i.eba

yimg.

com

(217

.212

.225

.x)

i.eba

yimg.

com

(80.

239.

205.

x)

i.eba

yimg.

com

(2.1

6.54

.x)

i.eba

yimg.

com

(184

.84.

182.

x)0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0.00%

2.00%

4.00%

6.00%

8.00%

10.00%

12.00%

14.00%

16.00%

18.00%

ePS performance in Germany measured from Backbone Nodes

Average Response Time (sec) Percentage of hits

© 2014 eBay Inc. 15© 2014 eBay Inc. 15

Use Case: Monitoring CDN performance of image delivery (continued)

i.eba

yimg.

com

(2.1

6.21

8.x)

i.eba

yimg.

com

(88.

221.

93.x)

i.eba

yimg.

com

(88.

221.

92.x)

i.eba

yimg.

com

(77.

67.9

6.x)

i.eba

yimg.

com

(5.1

78.4

0.x)

i.eba

yimg.

com

(217

.212

.225

.x)

i.eba

yimg.

com

(79.

140.

81.x)

i.eba

yimg.

com

(149

.3.1

80.x)

i.eba

yimg.

com

(92.

122.

216.

x)

i.eba

yimg.

com

(92.

122.

124.

x)

i.eba

yimg.

com

(184

.25.

107.

x)

i.eba

yimg.

com

(88.

221.

212.

x)

i.eba

yimg.

com

(46.

33.7

5.x)

i.eba

yimg.

com

(90.

84.5

4.x)

i.eba

yimg.

com

(165

.254

.134

.x)

i.eba

yimg.

com

(96.

17.1

5.x)

i.eba

yimg.

com

(213

.155

.157

.x)

i.eba

yimg.

com

(46.

33.7

6.x)

i.eba

yimg.

com

(46.

33.7

2.x)

i.eba

yimg.

com

(204

.2.2

22.x)

0

2

4

6

8

10

12

14

16

18

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

ePS performance in Germany measured from Last Mile

Average Response Time (sec) Percentage of hits

© 2014 eBay Inc. 16© 2014 eBay Inc. 16

Use Case: Monitoring Performance trends vs. Competitors

Weighed Page Load times for Home Page, Search Results Page and Item detail pages are reported to the business on a monthly basis.

Downside: Current method is labor intensive; Competitor tests need frequent repair as items expire.

© 2014 eBay Inc. 17© 2014 eBay Inc. 17

Future steps

- Move to Last Mile tests for competitor comparisons- Speed up the correlation process:

- Push Observium data into Graphite (feature development sponsored by eBay)

- Use Gomez / APMaaS API to pull performance data into Graphite

- Have custom text in Gomez / APMaaS Notifications to have direct url pointing to relevant Graphite graphs

- Create custom reports to help keep third party advertisers honest

- Create customer reports to aid in monthly site speed reporting

- Measure Mobile API call latency from Gomez / APMaaS

© 2014 eBay Inc. 18© 2014 eBay Inc. 18

Questions?