devops in practice - when does "practice" become "doing"
Post on 17-Jul-2015
429 Views
Preview:
TRANSCRIPT
© 2013 IBM Corporation
Accelerating Product and Service Innovation
DevOps in Practice: When does "Practice" Become "Doing"?
Michael Elder, IBM Senior Technical Staff Member
mdelder@us.ibm.com
http://linkedin.com/in/mdelder
@mdelder
© 2013 IBM Corporation
"Success is not delivering a feature; success is learning how to
solve the customer’s problem.”
- Mark Cook
© 2013 IBM Corporation
Examples of DevOps and Continuous Delivery
4
http://nflx.it/1dAJEBs
http://slidesha.re/1mXJ6Mo
© 2013 IBM Corporation
Accelerate
software delivery
Balance speed, cost,
quality and risk
Reduce time to
customer feedback
People Process Technology
Develop/Test
Deploy
Operate
Steer
IBM DevOps
Enterprise capability for continuous software delivery that enables clients to seize market opportunities and reduce time to customer feedback
© 2013 IBM Corporation
Software delivery challenges
Failures due to inconsistent dev and production environments
Bottlenecks trying to deliver more
frequent releases to meet market
demands
Complex, manual, processes for release lack
repeatability and speed
Poor visibility into dependencies across releases, resources,
and teams
Daily
Build
Monthly
Delivery
Who did
this last
time?
Dave…
Dave’s
not here
man…
Dev
Prod
© 2013 IBM Corporation
High performing teams adopt DevOps
Reference: 2013 State of DevOps Report by PuppetLabs
© 2013 IBM Corporation
About your philosophy
Culture of continual experimentation and learning
• Production like environments • Fully automated deployments • Accelerated delivery cycles
Build – Measure - Learn “Success is not delivering a feature; success is learning how to solve the customer’s problem.”
http://bit.ly/KM4JlQ
© 2013 IBM Corporation
Increase the operational awareness of your software earlier in the development process.
What does DevOps mean to me?
How do you do that? Architecture
Automated Testing
Automated Deployments
Production-Like Environments
Automated Release Promotion Process
Version control of all software, automation, and configuration
Why do this? Because faster feedback loops enable rapid evolution of ideas and
therefore faster iterations of your software
© 2013 IBM Corporation
“Cool factor” .. seen as pushing to the “next big thing” in our industry
Better communication between those who create and those who operate (same people in some cases)
Reduced fear of breaking the build/deployment/environment
Fail small before you “fail all”
Heavy focus on experimentation and learning
Improve the speed of your feedback loop to enable rapid evolution of ideas
What are the characteristics of
teams who practice DevOps?
© 2013 IBM Corporation
Architecture
Automated Testing
Automated Deployments
Production-Like Environments
Automated Release Promotion Process
Version control of all software, automation, and configuration
About our next topics
© 2013 IBM Corporation
About your architecture
Architecture should support DevOps principles such as staged roll out, operational insights, and scriptability
Each resource provides some very practical advice for building systems which are focused on reliability and feedback loops
Experiment!: Website conversion rate
optimization with A/B and multivariate testing
Release It!: Design and Deploy Production-Ready Software
http://netflix.github.io/#repo
http://slidesha.re/1mXJ6Mo
© 2013 IBM Corporation
Design for availability, continuous deployment
Change your perspective: as we move into hosted offerings, scalability and availability are critical to our business
Netflix Runtime Libraries provide capabilities you can adopt in your products today – regardless of whether you’re building a hosted service
The Libraries promote good development practices and architectural patterns
See the talk Andrew Spyker and I gave on this: http://slidesha.re/1mXJ6Mo
© 2013 IBM Corporation
Moving from monolithic applications to micro-services
14
Monolithic app Micro services
Scaling Scaling
© 2013 IBM Corporation
Compartmentalized business capability
Cross-functional teams
Communication via API ONLY!!
Use messaging to remove peer-to-peer dependencies
REST communication
Decentralized data
Design for failure
Pluggable architecture
Enables continuous delivery
Properties of a micro-service architecture
15
© 2013 IBM Corporation
About automated testing
This is THE HARDEST part of the whole experience
NO ONE is great at this, so don’t get discouraged
Some companies’ process for code review include stats on how many tests are consistently broken to decide whether the test failures are noteworthy
Continuously improve your coverage, flexibility, and discipline
Consider all aspects of testing:
Programmatic API (JUnit)
REST API (HTTPUnit, REST Fuse)
Web UI (Selenium)
Behavior driven testing (Cucumber)
Performance testing (Apache JMeter)
Security testing (Rational AppScan)
© 2013 IBM Corporation
Embedding unit tests in your classes directly
Promoted approach by Netflix to reduce friction and introduces limited additional bytes relative to third party libraries
Makes it easy to write commands, the UnitTest becomes the test harness and verification
Always testSuccess() and testFailure() use cases to ensure expected behavior
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
public class OSDeployEnvironmentCommand extends
OSAbstractOrchestrationCommand<Response> {
public static class UnitTest {
@Test
public void testSuccess() throws
InterruptedException,
ExecutionException, JSONException {
…
Future<Response> request =
new OSDeployEnvironmentCommand(…)
.queue();
Response response = request.get();
assertNotNull(response);
assertEquals(HttpStatus.SC_OK,
response.getStatus());
}
}
© 2013 IBM Corporation
Testing Your Deployed Application – REST API
We’re experimenting with REST Fuse which is an Eclipse project
Captures REST calls as annotations, freeing up your test method to just verify behavior
Requires a little “hoop jumping” for testing a series of calls (e.g. POST .. PUT .. GET)
import static
com.eclipsesource.restfuse.Assert.assertCreated;
import ...restfuse.annotation.Authentication;
import ...restfuse.annotation.HttpTest;
import ...restfuse.MediaType;
@HttpTest (method =
com.eclipsesource.restfuse.Method.POST,
contentMethod="getBlueprintRequestBody",
authentications = {
@Authentication( type = BASIC,
user = "{keystoneUser}",
password = "{keystonePassword}") },
path = "/rest/blueprint",
type = MediaType.APPLICATION_JSON
)
public void testCreateBlueprintOK() throws Exception {
assertCreated(response);
}
© 2013 IBM Corporation
Testing Your Deployed Application – Web UI
Selenium is the only “apparent” choice that we’ve come across that’s effective
Couples automated test to some details of your web UI and must be kept in sync
We’ve developed a few best practices around location elements for maximum flexibility, could publish this if there’s interest
public class EditorsPage extends UCDPPage {
…
@FindBy(id = "new-blueprint-button")
private WebElement newButton;
…
public void createFile(String name, String id,
String[] values) {
newButton.click();
util().waitForElementPresentAndVisible(
By.className("new-file-dialog"));
WebElement widget =
util().findElement(
By.className("new-file-dialog"));
Dialog dialog =
Widget.init(this, widget, Dialog.class);
dialog.setValues(values, true, true);
dialog.clickPrimaryButton();
…
}
…
}
© 2013 IBM Corporation 21
About automated deployments
Visibility and automated control of your application deployment process
• Manage application components and versions
• Manage configurations across all environments
• Offer secure ‘self-service’ capabilities
• Increase transparency • Ensure governance and compliancy
© 2013 IBM Corporation
Application model
Reliably capture application content and configuration settings with versioning and traceability
© 2013 IBM Corporation
Application environments
23
Define where components are deployed and capture configuration settings per deployment environment for an application
© 2013 IBM Corporation
Define automated processes for each application component
24
Each component defines a series of steps, backed by plugin tasks or custom scripts, to deploy it
© 2013 IBM Corporation
About production-like environments
Use any and all means at your disposal
Clouds can be helpful to provide “infinite” resources and drive the creation of new environments through automation See UrbanCode Deploy with Patterns (Landscaper):
https://ibm.biz/BdFc2X Or UrbanCode Deploy integration with Pure/SCO:
https://ibm.biz/BdFc2E
Vagrant (http://www.vagrantup.com) and Docker (http://www.docker.com) are good alternatives to clouds or to be used in combination with clouds
© 2013 IBM Corporation
DevOps and Cloud adoption Automating for faster delivery with DevOps and Cloud
Networking
Storage
Servers
Virtualization
O/S
Middleware
Mid Config
Data
Applications
Infrastructure as a Service
UrbanCode Deploy
Networking
Storage
Servers
Virtualization
O/S
Middleware
Mid Config
Data
Applications
Platform as a Service
IBM Bluemix
PureApplication System
SmartCloud Orchestrator
SoftLayer
IBM DevOps Services
New
UrbanCode Deploy with Patterns
26
IBM Patterns
© 2013 IBM Corporation
About using clouds
We’ve used an internal environment based on the IBM Pattern Engine Long history of usage Lots of work to stabilize Still impacted by performance of
transferring images, etc
We’re building software to work with OpenStack Cloud availability often behind “jump
boxes” at this stage of maturity Longer term, this will be our
approach along with Heat Orchestration Templates (HOT) for our applications
SmartCloud Orchestrator
IBM PureApplication System
IBM UrbanCode Deploy
with Patterns
© 2013 IBM Corporation IBM CONFIDENTIAL
OpenStack is a global collaboration of developers & cloud computing technologists working to produce an ubiquitous Infrastructure as a Service (IaaS) open source cloud computing platform for public & private clouds.
OpenStack is our strategic direction for cloud
Platinum Sponsors Gold Sponsors
Compute (core) Provision and manage large networks of virtual machines
Dashboard (core) Enables administrators and users to access & provision cloud-based resources through a self-service portal.
Heat (core) orchestrates multiple composite cloud applications using templates
Ceilometer (shared service) Collect monitoring, metering, and other measurements
Storage (core) Provision and manage block-based and object storage
Network (core) Provision and manage network connectivity
Identity (shared service) Unified authentication across all OpenStack projects and integrates with existing authentication systems.
Identity (shared service) Unified authentication across all OpenStack projects and integrates with existing authentication systems.
© 2013 IBM Corporation
Our internal cloud: “FIT”
The Federated Integrated Testing (FIT) cloud service provides Infrastructure as a Service capabilities from IBM
Very stable, reasonably fast (has improved dramatically over time)
Resources are “leased” so some zones aren’t meant for long running usage
Great for automated deployment and testing of builds CLM has been doing this against FIT
for a long time
© 2013 IBM Corporation
About using Vagrant
Some challenges with networking across platform, particularly if you’re trying to use it to “run a cloud” such as OpenStack *within* the VM
But for most applications, provide a simple way to share a common infrastructure configuration within a team or community
http://www.vagrantup.com
© 2013 IBM Corporation
About using Docker
Create one or more “Dockerfiles” capturing the set of steps to setup your application, middleware, and configuration
Build the “container” for each build
Ship the “container” along your delivery pipeline
IBM is active in this community – recently giving a keynote at Dockercon
http://www.docker.com
© 2013 IBM Corporation
Faster, repeatable, reliable full-stack deployment processes
SCM Build Automation
QA PROD
Execute application
deployment and
manage settings
across environments
Publish build
Pull
changes
SmartCloud Orchestrator
Environments DEV DEV DEV DEV
IBM PureApplication System
IBM UrbanCode Deploy
Cloud Provisioning
Deploy Automation
IBM UrbanCode Deploy
with Patterns
32
Learn more at https://ibm.biz/BdFcmV
© 2013 IBM Corporation
Introducing IBM UrbanCode Deploy with Patterns
IBM UrbanCode Deploy
with Patterns
New
VMware® vCenter Server™
Full stack Application Design & Provisioning
– Design open, full stack application environments from diagram or source
Portability to heterogeneous clouds
– Capture the expertise of a heterogeneous DevOps team, from solution architects to IT specialists
Hybrid clouds: Blend your application with PaaS, SaaS, and Traditional IT
– Enables you to balance speed, cost, and flexibility according to your specific needs
Continuous Delivery of Applications into the Cloud
– Eliminate weeks of waiting for new environments when starting new projects or testing new app changes
© 2013 IBM Corporation 36
Composite Applications
Components
Re-usable Workflows Environment
Management
SIT
PROD
The “What”
The “How”
The “Where”
Deployment
Automation
Intuitive scalable model-driven deployment automation
Images from
image registry Deploy containers from
image component
Deploy to docker hosts or
IBM Container Service
© 2013 IBM Corporation 37
Release & Deploy Automation
Deployment of Applications across Environments Applications are
managed for
deployment
Components
from the app
Specific versions of
the Components’
artifacts are deployed
Compliancy keeps track
of whether the actual
versions matches the
expected
Each Component
knows its relevant
deployment processes
Compare versions across
environments and define
Snapshots of known working
sets of Components
View deployment
requests against
the environment
https://developer.ibm.com/urbancode/
© 2013 IBM Corporation
Configuration Management - Snapshots
Creating a Snapshot
Component Versions
1
1
2
2
3
3
321Web
Mid. Code
DB
Snapshot
3
2
1
Mid. Config 1 2 3 3
Snapshot Deployment
ENVSnapshot
3
2
1
WEB HOST
MID HOST
DB HOST
2
1
1
3
2
X
3 3 1
Contents of environments that pass tests
• contain deployable version and configuration
Deployments are based on deltas
They help with:
Automation, Audit, and Visibility
uDeploy
© 2013 IBM Corporation
Version applications with Snapshots
39
Capture tested component versions and configuration settings as a snapshot that is promoted to other environments for better visibility and control
© 2013 IBM Corporation
IBM Container Service
Automate the build of Docker images
Manage and distribute Docker images in private image registries
Easily host containers in the cloud
Scale and auto-recovery built-in
Logging and Monitoring built-in
40
© 2013 IBM Corporation
Dev/Test with Hybrid Cloud
41
Off-P
rem
O
n-P
rem
Image Registry
IBM Bluemix
IBM UrbanCode Deploy PROD STAGING
© 2013 IBM Corporation
About version control
All of your source code is likely already version controlled
All of your automation scripts, configuration files, tests, etc should also version controlled
UrbanCode Deploy automatically understands versions of components from your build process, but also versions all changes to configuration properties and automation processes
https://hub.jazz.net/create
© 2013 IBM Corporation
IBM Bluemix DevOps Services – Project Mgmt
Source Control Management
• Hosted source control management within the cloud
• Support for Git and Jazz repositories • Manage users access via projects • Projects have developer centric tools
such as IDE, defect tracking, and a delivery pipeline
© 2013 IBM Corporation
IBM Bluemix DevOps Services – Delivery Pipeline
• Integration test, System test, Production
• Automation ensures the same procedure in each environment
• Fully hosted, managed, and integrated in the cloud for the cloud
© 2013 IBM Corporation
Code review should be part of your versioning process
Gerrit is a popular code review integration for git Used for all UrbanCode products
Enables you to create “pre-review”
verifications which automate running tests before a human reviews the change
Provides immediate, automated feedback about the quality of the change based on your test suites
© 2013 IBM Corporation
Every aspect of our product delivery should be driven from the
principles of validated learning; no more anecdotes, only
meaningful results from our fast cycle prototypes.
- Michael D. Elder
© 2013 IBM Corporation
Architecture
Automated Testing
Automated Deployments
Production-Like Environments
Automated Release Promotion Process
Version control of all software, automation, and configuration
About this deck
mdelder@us.ibm.com http://linkedin.com/in/mdelder @mdelder
Download from https://ibm.biz/BdFc2b
© 2013 IBM Corporation 48
© Copyright IBM Corporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
www.ibm.com/software
top related