google cloud lightning talk

Google Cloud Lightning Talk Magnus Jern, CEOLondon 8 October 2013

Content !   About myself and Golden Gekko !   Why GAE? !   Four case studies !   Forecasting GAE cost !   Summary

2

Who am I?

!   Programmer since childhood

!   Bulletin Boards

!   University

! Framfab

! Driftbolaget

! Blokks

!   Vodafone

!   Golden Gekko

! NComVA

Software engineer, entrepreneur, CEO and SVP

4

200 brands and agency clients. 160+ people that love and breathe mobile technology.

5

Part of DMI Group

Key Partnerships

Awards

Global Footprint ..............................................................

•  Bethesda, MD (HQ) •  New York, NY •  Barcelona, Spain •  London, United Kingdom •  Pune, India •  Washington, DC

Growth ..............................................................

•  $250 Million Revenue •  1,600 Employees •  Inc 500/5000 Six years in

a row •  Top 5 Enterprise Mobility

provider in the world

Certifications ..............................................................

•  ISO 9001: 2008 Certified •  ISO 27001: 2005 Certified •  ISO 20000-1: 2011

Certified •  CMMI Level 3 Appraised:

Development and Services

6

Local Offers, Product and store locator incl front-end apps, middleware and backoffice Strategy, development framework and long term solution for Anheuser-Busch InBev, managing all mobile services across 200 brands in 50 countries. Sales enterprise solution powering one of UK’s most recognizable food brands. All mobile services for 6 airports across the UK and Scotland, including London’s Heathrow. Mobile media solution providing intelligent pespectives on news

Used Google App Engine since 2010 Other Examples .............................................................................................................. •  Telenor SoBazar

Social mCommerce platform mixing online and instore experiences

•  NokadiPutting all your loyalty cards on a mobile device.

•  Unilever Smartswap Intelligent recommendations of healthier and better products based on shopping

•  TwoForOnePreviously a paper based coupon book company, that has now gone mobile.

•  OvivoFree mobile MVNO in the UK

•  Yellow PagesPowering the Yellow Pages in 8 countries

•  O2 HealthyEncouraging walking, biking and more through social

Helps avoid common pitfalls incl. file system, time outs and threading

8

Runtime Helps avoid common pitfalls incl. file system, time outs and threading

?

9

Boot-strapping Up and running in hours.

?

Consolidation Several years of dev created fragmented environment and hosting approach.

?

Scale Small team with BIG ambitions.

Reliability High availability solution.

?

Operations Managing monitoring and maintenance

?

14

Spring

Mardao

GAE

Base Services

Component Library Our d

ev st

ack

15

For build and project management.

For continuous integration.

For storing and distributing built SW.

For monitoring of 24/7 operations.

Toolbox

16

1 Pre-trip

2 At the airport

3 On-board

4 At the destination

5 Post-trip

§  Business critical §  Peak traffic with

weather, strikes, etc

Heathrow Airport

!   BAA will update status file for each airport once every 5 minutes using FTP

!   The Ingestion Daemon get all the flights and the related information

!   Updated flights information is send to the Status Service

!   For each flight received from the ingestion Daemon, If the flight already exists it will merge the existing status with the new status, if it is a new status it will added

!   The status service request all arrival and departure flights for a specific airport from FlightStats during the next 14 hours (one single request that can be slow and contain a lot of data)

!   For each of the received statues (from BAA) the service try to find the same flight in the response from FlightStats. If it is found the service will, first update the status with the information from FlightStats and then If the status from FlightStats indicate that the flight have coflights this information will be added to the status.

!   Updated status are stored to be usedfor the next 5 min

ESB Integration - Overview"

FlightInfo Service

Status Service

Controllers

Apps

BAA CSV File

Ingestion Daemon

1 per airport

Persistence Manager

1

2

1 2

3

3

4

5

6

7

5

6 7

4

17

18

Track180

§  The “Flipboard” of getting different perspectives §  Using GAE as CMS and content delivery platform

19

One of UK’s most recognizable brands who’s retail team is responsible for visiting 20.000 stores across the UK.

§  Business critical Enterprise solution

20

Architecture Overview

21

Datastore Diagram

22

§  Scale to handle between 10-100M active users during World Cup §  Peak bursts of 5-30 min during football game

Budweiser – Man of The Match

How to calculate GAE cost?

1.  Type of service 2.  System Design 3.  Supporting Services 4.  Usage

23

Summary 1

!   Super fast bootstrapping and template projects - developer can get up and running in a few hours.

!   Run-time will support and lead developers into good habits, especially dealing with resources (files, requests etc).

!   Small operations team can mange a large number of services - everything in one place.

!   Auto-scaling is nice and important BUT less then the items above.

GAE is a fantastic service for complex, high availability and scalable service for demanding customers

24

Summary 2

!   Hire or grow an internal datastore champion that can police/help projects.

!   Invest in sharing best practices, internal tech talks/workshops to spread the knowledge.

!   Denormalise, duplicate properties between entities to avoid an extra read, always use queries that run a list of results, never request single result on a loop. Rule of thumb is to do max 1 read per request.

!   Memcache is your best friend, use it well.

Good understanding and strategy for using the datastore is fundamental for achieving a good service

25

26

Contributors

Contact me on mj@goldengekko.com if you have more questions.

Mattias Levin, Head of Production Ola Sandstrom, CTO Mardao - database mapper and domain generator that hides many of the complexities of the datastore and provide automatic cachinghttps://github.com/sosandstrom/mardao Open services - A set of base services for rapid development of web services on GAE and Spring. Take a look at the CRUD controller, combine it with Mardao and and it will reduce your boilerplate coding with 80-90%, its awesomehttps://github.com/sosandstrom/open-server Gaelic - a light weight web service framework tailored for GAE, supporting dynamic scaling by offering support fast startup times (in the range of seconds). Will replace Spring in future projects https://github.com/sosandstrom/gaelic Ricotta - A neat service for managing a projects translations, no more Excel files. Let you customer translate token online and use Maven to pull down your localised resource at build time. https://github.com/sosandstrom/ricotta And many more https://github.com/sosandstromhttps://github.com/mattiaslevin

Thank You