going cloud first at the ft
TRANSCRIPT
Going cloud first at the FT
John O’Donovan CTO, Financial Times
@jodbod
Why cloud…
Going to talk about our approach
Finding our key project – The data Warehouse
Going cloud first
Finding balance…
For us getting to this point has been about finding balance…
Some FT Perspective
A disruption problem…
0 50,000
100,000 150,000 200,000 250,000 300,000 350,000 400,000 450,000
Q3 2010
Q4 2010
Q1 2011
Q2 2011
Q3 2011
Q4 2011
Q1 2012
Q2 2012
Q3 2012
Q4 2012
Q1 2013
Q2 2013
Q3 2013
Print circ
A disruption problem…
0 50,000
100,000 150,000 200,000 250,000 300,000 350,000 400,000 450,000
Print circ Digital subs
200+k more digital subs than print
A disruption problem…
0 50,000
100,000 150,000 200,000 250,000 300,000 350,000 400,000 450,000
Print circ Digital subs
200+k more digital subs than print
Total circulation grew 11% year-on-year to 677,000 (Deloitte assured, Q1 2014).
Online subscribers increasing 32% year-on-year to 435,000.
Digital readers represent two-thirds of total audience.
Mobile readership continues to increase, driving majority of subscriber consumption and 50% of total traffic
How people use content is changing…
How we measure content changes…
And platforms keep changing…
And platforms keep changing…
Some Redshift Points
We quickly came to realise that actually, the real power of the subscription relationship …comes from the data
John Ridding, CEO, Financial Times
Understand data from the top…
Mature + successful data driven CRM
programme
Optimisation embedded across digital business
Measure cross-platform effectiveness
Shapes our strategy
Powers on and off-site marketing
Provides insight into customer content preferences
Our outcomes…
Delivered on timeTo budget
Decrease in costs by 80%Pay as you go with no upfront commitment
Flexibility to scaleReal-time data instead of reportsNot a black box Data Warehouse
Analyst quote…
“As an analyst I generate a usage trend for specific content over 4 months. This meant I had to create 4
individual data sets, one for each month. In the current system it took me 25-30 minutes to run the query for generating a data set for a single month.
When I migrated to Redshift I was able to run the query for all four months in about 2.5 minutes! This is a big win both for the business and the
analytics team. 98% reduction in processing time or 40 times faster!”
This is our outcome…
“blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah
98% reduction in processing time or 40 times faster!”
This is our outcome…
“This is Fantastic, guys!”
Securing the cloud…
Data security at network and application layers
Everything is encrypted and transported over https or ssh
Redshift runs mandatory SSL on client connections
Roles and privileges work in a controlled environment
Making it easier to stay in the “system”…
“Today the Data Science team had a problem that would take 300 hours to solve on their local
laptops so we created a Windows box in AWS that they could log on to, install R and crunch
through their problem overnight.”
Automation again…
Use Puppet to deploy from Stash
So it's not possible to merge an invalid job
Or deploy random “data munging” scripts
Confidence that what's in Stash is what’s in Prod
Tidy up! Data Debt is one of the worst kinds of Technical Debt
Automation…
Destroy and create environments easily
Don’t need a Test environment all the time, for eg.
With Puppet we could recreate the Linux environment
FT Platform installs monitoring & Splunk logging
Some General Points
FT Platform…
Our environment is complex
So it’s not a free for all
FT Platform is our interface
FT Platform…
FT Platform…
DevOps
FT Platform
FT Platform…
FT Platform…
FT Platform…all these things…
FT Platform…to all these places…
Remove the FEAR!!!!!
Controlling your environments…
Seamless AD integration
Make it easier to use than not to use
Using Roles & Least Privilege Principle
Simple security scales
Chaos Snail…coming to get you eventually…
All Hail the Chaos Snail
Based on Chaos Monkey but it’s more chilled Slows things down and attacks IO
Written in shell…
- Bash to be precise…
- Seemed like a good idea at the time…
Reboot, reboot, reboot…
No one should be proud of this anymore…
We reboot at least monthly
Breeds confidence, changes are easier
HeartBlead & ShellShock patching was easy
Automation…
Tidy up all the time
Build confidence you can reinstate
It’s cheaper than duplication…
Meet Tagbot…
Tagging environments and the Tagbot
AWS provides lots of services to help monitor
Work out how to control your spend
Don’t need all the sweets in the sweet shop…
Getting away from managing all the complexity…
Who knows how many disks they will need next year?
What's going into the cloud?
Nearly everything…
But managing for Armageddon…
In summary…
Trying to keep it simple
Automation is key
Take the savings
Focus on business value
Please note that this is a PowerPoint 2003 (ppt or pot) file. DO NOT work on or save this file on PowerPoint 2007 or 2010 as this will CORRUPT some slide master settings (even in the compatibility mode!!)
John O’Donovan @jodbod
www.ft.com
AWS Enterprise Summit
@AWS_UKI
#AWSSummit