sv jug - mar 2013 - sl
DESCRIPTION
TRANSCRIPT
Next Generation Software Development:Continuous Cloud Deployment
@SachaLaboureyCEO, CloudBees, Inc.
March 20, 2013 – Silicon Valley JUGPhoto credit: @romainguy
©2013 CloudBees, Inc. All Rights Reserved
2©2012 CloudBees, Inc. All Rights Reserved
• Born in 1975 in Switzerland
• JBoss– Clustering lead – 2001– GM Europe - 2003– CTO - 2005
• Red Hat– JBoss acquisition in June 2006– co-GM of Red Hat’s middleware division– Left Red Hat in April 2009
• CloudBees– Started in April 2010– About 35 bees in 7 countries
Introduction – Sacha Labourey
X
3
• What’s the life of a developer today?• Challenges ahead…• Solution!• Cloud 101• Read my lips: SERVICE• Quick demos
Agenda
©2012 CloudBees, Inc. All Rights Reserved
©2013 CloudBees, Inc. All Rights Reserved
Life of a Developer Today
5©2013 CloudBees, Inc. All rights reserved
• Release v1.0
• Gather requirements for v2.0
• Work on v2.0 • Release v2.0
don’t screw it up: once it is out, it is out.
• Have your customer upgrade
– Support v1.0 for years
• Support and patch dozen of “branches” in parallel
• Gather requirements for v3.0
• Work on v3.0
Typical Software Development
Mon
ths/Y
ears
Packaged Software
6
Developers vs. Sys Admins (aka Love Story)
Photo Credit: Joseph Viles/ The CW Photo Credit: http://www.sysadminday.com/people.html
7
M. Developer
Interests & Hobbies:• Give a try to the last
framework• Put in the production
application the last shiny object seen at a conference
• Participate to the Scala vs. Ceylon debate, static vs. dynamic typing, etc.
• Dream of meeting with James Gosling
M. Sys Admin/ops
Interests and Hobbies:• Not to be beeped at 2am in
the morning• Not to be beeped at 4am in
the morning• Participate to the Debian vs.
Fedora vs. Ubuntu debate• Dream of meeting with Linus
Torvald
<sarcastic mode=“on” />
8
Development? Friction!
SourceCode
Managemnt
JenkinsContinuousIntegration
UnitTest
FunctionalTest
LoadTest
IntegrationTest
Developer
“Cloud you please provision a server for me?”
“Could I get X, Y and Z installed on that server?”
“Could you please patch my server?”
“Could I please get a database dump?”
Could I get a new Git server?
Where could I run my Jenkins integration tests?
IT
9
Deployment? Friction!
Configure OS
Secure OS Setup JVM
Install AS on OS
Configure AS
Secure AS
provision nodes
Install load-balancer
(LB)Configure LB
Secure LBSetup BC for
clusterQA cluster
Configure DNS
Provision node:
Provision cluster:
Provision node
Install DBConfigure
DB
Secure DBSetup BC for
DB
Provision DB:
Provision cluster
Provision DBDeploy schema
Deploy App to each node
Deploy App:
(not a transparent update process)
Analyse sec. bulletins
Analyse QA bulletins
Analyse AS/OS logs
maintain OSmaintain
JVMmaintain AS
maintain DB
Manage environment (recurring)
maintain FWValidate/QA
stack
Measure App
performance
Acquire Hardware
Provision node
Update Cluster
Update LB
Metering/Scaling (recurring)
Update Security
10
Friction, friction, friction
provision nodes
Install load-
balancer (LB)
Configure LB
Secure LBSetup BC
for clusterQA cluster
Configure DNS
Provision cluster:
Provision cluster
Provision DB
Deploy schema
Deploy App to
each node
Deploy App:
(not a transparent update process)
Configure OS
Secure OS Setup JVM
Install AS on OS
Configure AS
Secure AS
provision nodes
Install load-balancer
(LB)Configure LB
Secure LBSetup BC for
clusterQA cluster
Configure DNS
Provision node:
Provision cluster:
Provision node
Install DBConfigure
DB
Secure DBSetup BC for
DB
Provision DB:
Provision cluster
Provision DBDeploy schema
Deploy App to each node
Deploy App:
(not a transparent update process)
Analyse sec. bulletins
Analyse QA bulletins
Analyse AS/OS logs
maintain OSmaintain
JVMmaintain AS
maintain DB
Manage environment (recurring)
maintain FWValidate/QA
stack
Measure App
performance
Acquire Hardware
Provision node
Update Cluster
Update LB
Metering/Scaling (recurring)
Update Security
11
Why do we do software development again?
12
To serve the BUSINESS obviously!
Business Needs
Specification
Implementa-tion
Deployment
13
IT serves the BUSINESS?
Increase Revenue
s
Implement e-
commerce site
JVM? Ruby? PHP? Java? Scala?
Play? Groovy? Lift? Jruby? CFML? Grails? Spring? EE6?JBoss? Tomcat? Glassfish? Jetty? MySQL? MongoDB?
CouchDB? Cassandra? JMS? AMQP? Memcache?
Ehcache? Quartz? Lucene? Solr? Jquery? GWT? iPhone?
Android? JIRA? Redmine? SVN? Git? Maven? Ant/Ivy? Jenkins? Artifactory/Nexus?
etc.
AMD/Intel? HP? DELL? SPARC? UX? AWS? Linux? Windows?
Solaris? RHEL? Ubuntu? Debian? Arch? EXT2/3?
NFS/CIFS? Backup? Chef? Puppet? Continuous
Deployment? Apache HTTPD? Mod_jk? Mod_proxy? Nginx?
Haproxy? Firewall? SSL? Server provisioing? Load-
balancing? Failover? Monitoring? Management?
Alerting? Escalation procedure? etc.
15 seconds
2 weeks
18 months
1414
Didn’t we simply ask him to build an e-commerce site?
15©2013 CloudBees, Inc. All Rights Reserved
• Lots of FRICTION• Less time spent on
DELIVERING VALUE
And that’s about to get worse…
Software Development got Difficult
©2013 CloudBees, Inc. All Rights Reserved
Challenges ahead!
17
• In parallel, demand for new needs are growing fast!– Mobile– Big Data– Social– Local– …
At the same time…
18
Entering the post-PC era
Thanks to Mary Meeker and Matt Murphy, From David Skok presentation @ MassTLC
1919
Source: Mary Meeker , 2011, www.kpcb.com/internettrends2011
2020
Source: Mary Meeker , 2011, www.kpcb.com/internettrends2011
21
Mobile Internet - 10x bigger!
Thanks to Mary Meeker and Matt Murphy, From David Skok presentation @ MassTLC
22©2013 CloudBees, Inc. All Rights Reserved
Small but big!
Source: Mary Meeker , 2012, http://www.kpcb.com/insights/2012-internet-trends
23©2013 CloudBees, Inc. All Rights Reserved
India just made the switch!
Source: Mary Meeker , 2012, http://www.kpcb.com/insights/2012-internet-trends
24©2012 CloudBees, Inc. All Rights Reserved
• Drivers are accelerating the need for NEW applications– Mobile, Mobile, Mobile, Social, Big Data, Etc.
• Time-to-market is more important than ever!– “I need it for yesterday!”– Can’t afford 18m cycles
Plus, you must be in DISCOVER MODE, since
…You don’t really know what’s needed!
Drivers & Acceleration
25©2013 CloudBees, Inc. All Rights Reserved
• You are (t)asked– To do many more new applications– To deliver them much earlier– To deliver incremental changes very quickly– To measure whether what you are doing
works– To scale quickly the few apps that will be
successful– To have them interact with an increasing
number of different devices
… Yet, nothing else should change?
Impact
©2013 CloudBees, Inc. All Rights Reserved
Looking For a Solution?
©2013 CloudBees, Inc. All Rights Reserved
Cloud 101
Read my lips: A-S A S-E-R-V-I-C-E
28
Easy: « Just use a XaaS! »
©2012 CloudBees, Inc. All Rights Reserved
29
Traditional software stackWe have done this for 20 years!
©2012 CloudBees, Inc. All Rights Reserved
Server
Hypervisor
VM
AS
JVM
LB
Application
setup
monitor
Patch
update
validate
You
30©2012 CloudBees, Inc. All Rights Reserved
Servers
OS
VM
AS
JVM
LB
Applications
setup
Monitor
Patch
Update
Validate
Cloud Computing: How to do it? Who does what?
Cloud Provider
?
31
• SalesForce.com, Zendesk, NetSuite, etc.
• Development environment– Rigid – mostly through
CONFIGURATION– When available “development” takes
place within the SaaS itself
• THINK– Standard applications
SaaS
©2012 CloudBees, Inc. All Rights Reserved
32
Must probably be the same…
IaaS – Let’s try not to change too much…
©2012 Cloud Bees, Inc. All Rights Reserved
?
SUN/HP/IBM/XYZ
AWS/Rackspace/etc.
33
• AWS – the most popular example• Server Lego blocks – VM, storage, IP, etc.• Development environment:
– “Give me a server, an OS, a virtualization layer, an application server, a firewall, a database, I’ll deal with it! And patch it. And monitor it. And…”
– Flexible but cOmpLiCaTed
• THINK– Custom-built systems
IaaS
©2012 CloudBees, Inc. All Rights Reserved
34
IaaS – Congratulations!
©2012 Cloud Bees, Inc. All Rights Reserved
“Great” news: you have now become responsible for the data-center!
35©2012 CloudBees, Inc. All Rights Reserved
Directly managing your IaaS resources provides you with resource elasticity (CAPEX-free), but…
The amount of soft-IT typically required to do so is … higher!– All of the traditional IT activities remain
(maintain/patch/monitor OS+JVM+AS+DB++, etc.)
– + cloud-specific items: elasticity/security/automation
• I can read your mind: this is where you will start
• My advice? Do it and then move on…
IaaS – Consequences
36©2012 CloudBees, Inc. All Rights Reserved
• Cloud concepts are applied to Applications– On-demand, pay-as-you-go, elasticity, etc.– No need to handle updates, patches, scalability,
failover, etc.
• Development environment– “Give me my typical development
environment and manage everything else for me – servers, scalability, etc.”
• THINK– Custom applications– Harder to “grasp” initially
PaaS
37©2012 CloudBees, Inc. All Rights Reserved
• We focus a lot on the P/I/S on XaaS• But not enough on the … S!• We are entering a SERVICE-ERA
– IaaS - You are CONSUMING IT resources– SaaS - You are CONSUMING an online Service– PaaS - You are CONSUMING development services
This is not about software anymore!
The Cloud is refocusing IT roles
©2013 CloudBees, Inc. All Rights Reserved
Again: Looking For a Solution?
Read my lips: A-S A S-E-R-V-I-C-E
39©2013 CloudBees, Inc. All rights reserved
• Release v1.0
• Gather requirements for v2.0
• Work on v2.0 • Release v2.0
don’t screw it up: once it is out, it is out.
• Have your customer upgrade
– Support v1.0 for years
• Support and patch dozen of “branches” in parallel
• Gather requirements for v3.0
• Work on v3.0
• Release v0.1• Implement micro-feature,
deploy, measure success, keep or kill
• Implement micro-feature, deploy, measure success, keep or kill
• Patch a bug• Implement micro-feature,
deploy, measure success, keep or kill
• Implement micro-feature, deploy, measure success, keep or kill
• Implement micro-feature, deploy, measure success, keep or kill
To Be Or Not To Be…M
on
ths/Y
ears
Days/W
eeks
Packaged Software As-A-Service
40©2012 CloudBees, Inc. All Rights Reserved
• What’s the big difference?– They’re just a long series of 1 and 0 at the end of
the day, right?
• A Services is not just “hosted software”– i.e. take your current software, host it, and keep
doing what you’ve been doing to date
Software vs. Service
41©2012 CloudBees, Inc. All Rights Reserved
Software != Service
Not the same …
[ requirement gathering |
development | QA |
success measurement | release | support ]
… processes
42
Packaged Software vs Cloud Software Delivery
Version 1
• Shipped May 2011
Version 2
• Shipped May 2012
Version 3
• Shipped May 2013
Development Methodology VariesBonus: “We use CI”
Update CSS1st Feb
Move button1st Feb
New tab2nd Feb
Overhaul UI13th Feb
Bug fix14th Feb
Unachievable without Continuous Integration and Continuous Delivery
©2013 CloudBees, Inc. All rights reserved
42
43
Time
v1.0 Release
v2.0 Release
Time
v1.0.x ContinuousReleasesRi
sk/
Cost
Risk/
Cost
Packaged Software As-A-Service
Lower Risk and Cheap Experimentation
Same basic value prop as agile, but code is being released continuously
44©2012 CloudBees, Inc. All Rights Reserved
• In theory, fast-iteration is great! But…
FAST_ITERATION + FRICTION => HEAT!
You need a no-friction environment
Fast Iteration is good, but…
45©2012 CloudBees, Inc. All Rights Reserved
• Integrated & automated “pipeline”• Always be in a “release-ready”
state
Continuous Cloud Deployment
Code Build Test Stage Deploy
#Fail#Fail #Fail#Fail
46©2012 CloudBees, Inc. All Rights Reserved
Complex Scenarios are possible!
46
SQE Job
Database Tests
Compliance Tests
Stress Tests
Developer Job
Developer Tests
Functional Tests
X-Team Tests
Security Tests
Access Tests
Production Job
Backup Production
Deploy
Monitor
Developer Org
QA Org
Production Org
Promote
Promote
47©2012 CloudBees, Inc. All Rights Reserved
• Your Code ~= Your Service• Always be in a release-ready state
– This requires a NO-friction environment
• Enables high-frequency iterations– Code, test, deploy, measure, keep-or-
kill
Continuous Cloud Deployment
Why am I telling you all of this?
Enter into the world of CloudBees!
©2012 Cloud Bees, Inc. All Rights Reserved
49
Public Edition - IaaS Providers
©2012 CloudBees, Inc. All Rights Reserved
CloudBees Platform as a Service
Continuous
Deployment
Production
Build
Test
Provisioning
Maintenance
Jenkins
SaaS Vendor Applications
Enterprise UserApplications
JVM – Java EEJava, Python, Scala,
…
CloudBees Core Platform Services
RUN@DEV@
Repositories
Data Services
MeteringBilling
MonitoringManagement
Messaging IdentityLogging/Auditing
Back-end services
Runtime services
Development Projects
CloudBees Ecosystem
GIT Maven …SVN
APIs
CloudBees Smart
Plugins
Databases
Other SaaS
On Premise
Private Edition - On-Premise
Code
BuildTest
Deploy
Stage Deploy Scale
Iterate
50
What You Interact With When You Use CloudBees
50©2013 CloudBees, Inc. All rights reserved
CLI/SDK GrandCentral Web Console IDE
51©2012 CloudBees, Inc. All Rights Reserved
Development? No friction!
DeveloperCloudBees
PaaS
52©2012 CloudBees, Inc. All Rights Reserved
Deployment? No Friction!
Configure OS Secure OS Setup JVM
Install AS on OS Configure AS Secure AS
provision nodesInstall load-
balancer (LB)Configure LB
Secure LBSetup BC for
clusterQA cluster
Configure DNS
Provision node:
Provision cluster:
Provision node Install DB Configure DB
Secure DB Setup BC for DB
Provision DB:
Provision cluster
Provision DB Deploy schema
Deploy App to each node
Deploy App:
(not a transparent update process)
Analyse sec. bulletins
Analyse QA bulletins
Analyse AS/OS logs
maintain OS maintain JVM maintain AS
maintain DB
Manage environment (recurring)
maintain FW Validate/QA stack
Measure App performance
Acquire Hardware
Provision node
Update Cluster Update LB
Metering/Scaling (recurring)
Update Security
(transparent, multi-version,HA, scalable)
• No need to care about servers, load-balancers, firewalls, backups, etc.
• The environment is constantly managed and monitored
• Scalability happens in realtime
• Integrated failover/HA
• We do « Ops », you do « Dev »
Deploy app to traditional Java platform
Deploy app to CloudBees
DEPLOY
53©2012 CloudBees, Inc. All Rights Reserved
Value à la carte – NO FRICTION
Code
BuildTest
Deploy
OURYOUR TEAM
YOUR PROJECT
54
• Lose it!– Mobile application– >> millions of users– > 30,000 transactions per minute, at peak time
• And… only 4 employees and …2 developers!– No IT, no servers, no DevOps– Complete focus on SOFTWARE DEVELOPMENT– Anything else is handled by CloudBees
• Unmatched productivity level!– This is possible TODAY– Will you wait for your competitors to shoot first?
Example: Lose it!
55
Continuous Cloud Delivery at CloudBees
Jenkins Core
Jenkins Plugins
Staging
Production
Web Console
SDK
API
Identity
Provisioning
Monitoring
Messaging
Services Platform
Generic App Container
Database
Router
Curated Stack
Git
SVN
Forge
Unit Tests StageCross Service Tests UAT Prod
As needed, multiple times
per day
56
Typical Mobile App with Java back-end
???CustomMobile
Back-end
Fast-IterationDiscovery
StableNO-discovery
First Demo• Complete Java Server Application…
• (Backbone.js front-end, REST back-end, DB)
• … with full lifecycle automation …
• (Code, build, test, deploy)
• … and do it in a single click!
ClickStart
58
cloudbees.com
What’s up with this new
framework!
Get started in a snap! “ClickStart”
58
CloudBeesGit
Repo
Database
Build/Test
db:create
App
PartnerServicePartnerService
CloudBees API
Grand Central Web
Console
app:creategit clone
Jenkins job create
Bind
Bind
Trigger
Deploy
Wow! At yourservice
©2013 CloudBees, Inc. All rights reserved
59
cloudbees.com
Continous Delivery at work
CloudBeesGit
Repo
Database
Build/Test
App
PartnerServicePartnerService
Bind
Bind
TriggerDeployI want to make
some changes!
Local GitRepo
git clonegi
t com
mit git push
Chan
ges
Wow! At yourservice
#fail #fail #fail
Mobile!
CodeBuild App
Code
BuildTest
Deploy
TestBetaDeplo
y
GADeplo
y
• Get started fast!
• Team Work• Automatically
push new code to test
#fail
• Build on several targets: iOS, Android, etc.
• If it fails, stop
• Test is costly!• Easy-first, then
increase!• Basic test• Multi-device• Gesture• Etc.
• If it fails, stop
• Deploy in the field on real users
• Gather input• Retrieve
dumps• Etc.• If it fails, stop
• D-Day! • Push apps to
real-life users!• If it fails, stop
Build Matrix• Testing mobile application requires testing
on lots of different devices, form factors, etc.
• Create one build job per form factor/generation?
• No!
• Jenkins Build Matrix!
Jenkins Build Matrix
http://wiki.cloudbees.com/bin/view/DEV/Matrixed_Jenkins_Android_Builds
So much more!
Getting started• Lots of new things to learn!
• (delegation of trust, access to legacy data, latency, elasticity, iterativity, etc.)
• Identify an application
• Start small & Non-critical
• Focus on « Learning »
• And extend from there
• (and enjoy!)
65©2012 CloudBees, Inc. All Rights Reserved
• We are entering a SERVICE-era• Packaged software is dead
– Too long cycles, no ability to measure, etc.
• How can you remain competitive?
I’m ready to deploy! Are you?
TIME TO WAKE-UP!
Thank you
@SachaLabourey
“It is easier to ask forgiveness than it is to ask permission”
-- Grace Hopper