devops in practice - when does "practice" become "doing"

© 2013 IBM Corporation

Accelerating Product and Service Innovation

DevOps in Practice: When does "Practice" Become "Doing"?

Michael Elder, IBM Senior Technical Staff Member

[email protected]

http://linkedin.com/in/mdelder

@mdelder

mailto:[email protected]





"Success is not delivering a feature; success is learning how to

solve the customer’s problem.”

- Mark Cook


Examples of DevOps and Continuous Delivery

4

http://nflx.it/1dAJEBs

http://slidesha.re/1mXJ6Mo






Accelerate

software delivery

Balance speed, cost,

quality and risk

Reduce time to

customer feedback

People Process Technology

Develop/Test

Deploy

Operate

Steer

IBM DevOps

Enterprise capability for continuous software delivery that enables clients to seize market opportunities and reduce time to customer feedback


Software delivery challenges

Failures due to inconsistent dev and production environments

Bottlenecks trying to deliver more

frequent releases to meet market

demands

Complex, manual, processes for release lack

repeatability and speed

Poor visibility into dependencies across releases, resources,

and teams

Daily

Build

Monthly

Delivery

Who did

this last

time?

Dave…

Dave’s

not here

man…

Dev

Prod


High performing teams adopt DevOps

Reference: 2013 State of DevOps Report by PuppetLabs


About your philosophy

Culture of continual experimentation and learning

• Production like environments • Fully automated deployments • Accelerated delivery cycles

Build – Measure - Learn “Success is not delivering a feature; success is learning how to solve the customer’s problem.”

http://bit.ly/KM4JlQ




Increase the operational awareness of your software earlier in the development process.

What does DevOps mean to me?

How do you do that? Architecture

Automated Testing

Automated Deployments

Production-Like Environments

Automated Release Promotion Process

Version control of all software, automation, and configuration

Why do this? Because faster feedback loops enable rapid evolution of ideas and

therefore faster iterations of your software


“Cool factor” .. seen as pushing to the “next big thing” in our industry

Better communication between those who create and those who operate (same people in some cases)

Reduced fear of breaking the build/deployment/environment

Fail small before you “fail all”

Heavy focus on experimentation and learning

Improve the speed of your feedback loop to enable rapid evolution of ideas

What are the characteristics of

teams who practice DevOps?


Architecture

Automated Testing





About our next topics


About your architecture

Architecture should support DevOps principles such as staged roll out, operational insights, and scriptability

Each resource provides some very practical advice for building systems which are focused on reliability and feedback loops

Experiment!: Website conversion rate

optimization with A/B and multivariate testing

Release It!: Design and Deploy Production-Ready Software

http://netflix.github.io/#repo







Design for availability, continuous deployment

Change your perspective: as we move into hosted offerings, scalability and availability are critical to our business

Netflix Runtime Libraries provide capabilities you can adopt in your products today – regardless of whether you’re building a hosted service

The Libraries promote good development practices and architectural patterns

See the talk Andrew Spyker and I gave on this: http://slidesha.re/1mXJ6Mo




Moving from monolithic applications to micro-services

14

Monolithic app Micro services

Scaling Scaling


Compartmentalized business capability

Cross-functional teams

Communication via API ONLY!!

Use messaging to remove peer-to-peer dependencies

REST communication

Decentralized data

Design for failure

Pluggable architecture

Enables continuous delivery

Properties of a micro-service architecture

15


Leveraging Hystrix’s built-in event stream


About automated testing

This is THE HARDEST part of the whole experience

NO ONE is great at this, so don’t get discouraged

Some companies’ process for code review include stats on how many tests are consistently broken to decide whether the test failures are noteworthy

Continuously improve your coverage, flexibility, and discipline

Consider all aspects of testing:

Programmatic API (JUnit)

REST API (HTTPUnit, REST Fuse)

Web UI (Selenium)

Behavior driven testing (Cucumber)

Performance testing (Apache JMeter)

Security testing (Rational AppScan)


Embedding unit tests in your classes directly

Promoted approach by Netflix to reduce friction and introduces limited additional bytes relative to third party libraries

Makes it easy to write commands, the UnitTest becomes the test harness and verification

Always testSuccess() and testFailure() use cases to ensure expected behavior

import static org.junit.Assert.assertEquals;

import static org.junit.Assert.assertNotNull;

public class OSDeployEnvironmentCommand extends

OSAbstractOrchestrationCommand<Response> {

public static class UnitTest {

@Test

public void testSuccess() throws

InterruptedException,

ExecutionException, JSONException {

…

Future<Response> request =

new OSDeployEnvironmentCommand(…)

.queue();

Response response = request.get();

assertNotNull(response);

assertEquals(HttpStatus.SC_OK,

response.getStatus());

}

}


Testing Your Deployed Application – REST API

We’re experimenting with REST Fuse which is an Eclipse project

Captures REST calls as annotations, freeing up your test method to just verify behavior

Requires a little “hoop jumping” for testing a series of calls (e.g. POST .. PUT .. GET)

import static

com.eclipsesource.restfuse.Assert.assertCreated;

import ...restfuse.annotation.Authentication;

import ...restfuse.annotation.HttpTest;

import ...restfuse.MediaType;

@HttpTest (method =

com.eclipsesource.restfuse.Method.POST,

contentMethod="getBlueprintRequestBody",

authentications = {

@Authentication( type = BASIC,

user = "{keystoneUser}",

password = "{keystonePassword}") },

path = "/rest/blueprint",

type = MediaType.APPLICATION_JSON

)

public void testCreateBlueprintOK() throws Exception {

assertCreated(response);

}


Testing Your Deployed Application – Web UI

Selenium is the only “apparent” choice that we’ve come across that’s effective

Couples automated test to some details of your web UI and must be kept in sync

We’ve developed a few best practices around location elements for maximum flexibility, could publish this if there’s interest

public class EditorsPage extends UCDPPage {

…

@FindBy(id = "new-blueprint-button")

private WebElement newButton;

…

public void createFile(String name, String id,

String[] values) {

newButton.click();

util().waitForElementPresentAndVisible(

By.className("new-file-dialog"));

WebElement widget =

util().findElement(

By.className("new-file-dialog"));

Dialog dialog =

Widget.init(this, widget, Dialog.class);

dialog.setValues(values, true, true);

dialog.clickPrimaryButton();

…

}

…

}

© 2013 IBM Corporation 21

About automated deployments

Visibility and automated control of your application deployment process

• Manage application components and versions

• Manage configurations across all environments

• Offer secure ‘self-service’ capabilities

• Increase transparency • Ensure governance and compliancy


Application model

Reliably capture application content and configuration settings with versioning and traceability


Application environments

23

Define where components are deployed and capture configuration settings per deployment environment for an application


Define automated processes for each application component

24

Each component defines a series of steps, backed by plugin tasks or custom scripts, to deploy it


About production-like environments

Use any and all means at your disposal

Clouds can be helpful to provide “infinite” resources and drive the creation of new environments through automation See UrbanCode Deploy with Patterns (Landscaper):

https://ibm.biz/BdFc2X Or UrbanCode Deploy integration with Pure/SCO:

https://ibm.biz/BdFc2E

Vagrant (http://www.vagrantup.com) and Docker (http://www.docker.com) are good alternatives to clouds or to be used in combination with clouds

https://ibm.biz/BdFc2X

https://ibm.biz/BdFc2X



http://www.vagrantup.com


http://www.docker.com




DevOps and Cloud adoption Automating for faster delivery with DevOps and Cloud

Networking

Storage

Servers

Virtualization

O/S

Middleware

Mid Config

Data

Applications

Infrastructure as a Service

UrbanCode Deploy

Networking

Storage

Servers

Virtualization

O/S

Middleware

Mid Config

Data

Applications

Platform as a Service

IBM Bluemix

PureApplication System

SmartCloud Orchestrator

SoftLayer

IBM DevOps Services

New

UrbanCode Deploy with Patterns

26

IBM Patterns


About using clouds

We’ve used an internal environment based on the IBM Pattern Engine Long history of usage Lots of work to stabilize Still impacted by performance of

transferring images, etc

We’re building software to work with OpenStack Cloud availability often behind “jump

boxes” at this stage of maturity Longer term, this will be our

approach along with Heat Orchestration Templates (HOT) for our applications


IBM PureApplication System

IBM UrbanCode Deploy

with Patterns

© 2013 IBM Corporation IBM CONFIDENTIAL

OpenStack is a global collaboration of developers & cloud computing technologists working to produce an ubiquitous Infrastructure as a Service (IaaS) open source cloud computing platform for public & private clouds.

OpenStack is our strategic direction for cloud

Platinum Sponsors Gold Sponsors

Compute (core) Provision and manage large networks of virtual machines

Dashboard (core) Enables administrators and users to access & provision cloud-based resources through a self-service portal.

Heat (core) orchestrates multiple composite cloud applications using templates

Ceilometer (shared service) Collect monitoring, metering, and other measurements

Storage (core) Provision and manage block-based and object storage

Network (core) Provision and manage network connectivity

Identity (shared service) Unified authentication across all OpenStack projects and integrates with existing authentication systems.

Identity (shared service) Unified authentication across all OpenStack projects and integrates with existing authentication systems.


Our internal cloud: “FIT”

The Federated Integrated Testing (FIT) cloud service provides Infrastructure as a Service capabilities from IBM

Very stable, reasonably fast (has improved dramatically over time)

Resources are “leased” so some zones aren’t meant for long running usage

Great for automated deployment and testing of builds CLM has been doing this against FIT

for a long time


About using Vagrant

Some challenges with networking across platform, particularly if you’re trying to use it to “run a cloud” such as OpenStack *within* the VM

But for most applications, provide a simple way to share a common infrastructure configuration within a team or community





About using Docker

Create one or more “Dockerfiles” capturing the set of steps to setup your application, middleware, and configuration

Build the “container” for each build

Ship the “container” along your delivery pipeline

IBM is active in this community – recently giving a keynote at Dockercon





Faster, repeatable, reliable full-stack deployment processes

SCM Build Automation

QA PROD

Execute application

deployment and

manage settings

across environments

Publish build

Pull

changes


Environments DEV DEV DEV DEV

IBM PureApplication System


Cloud Provisioning

Deploy Automation


with Patterns

32

Learn more at https://ibm.biz/BdFcmV

https://ibm.biz/BdFcmV




Introducing IBM UrbanCode Deploy with Patterns


with Patterns

New

VMware® vCenter Server™

Full stack Application Design & Provisioning

– Design open, full stack application environments from diagram or source

Portability to heterogeneous clouds

– Capture the expertise of a heterogeneous DevOps team, from solution architects to IT specialists

Hybrid clouds: Blend your application with PaaS, SaaS, and Traditional IT

– Enables you to balance speed, cost, and flexibility according to your specific needs

Continuous Delivery of Applications into the Cloud

– Eliminate weeks of waiting for new environments when starting new projects or testing new app changes


IBM Bluemix DevOps Services

Enabling rapid software delivery

34


About your promotion process

UrbanCode Deploy

with Patterns


Composite Applications

Components

Re-usable Workflows Environment

Management

SIT

PROD

The “What”

The “How”

The “Where”

Deployment

Automation

Intuitive scalable model-driven deployment automation

Images from

image registry Deploy containers from

image component

Deploy to docker hosts or

IBM Container Service


Release & Deploy Automation

Deployment of Applications across Environments Applications are

managed for

deployment

Components

from the app

Specific versions of

the Components’

artifacts are deployed

Compliancy keeps track

of whether the actual

versions matches the

expected

Each Component

knows its relevant

deployment processes

Compare versions across

environments and define

Snapshots of known working

sets of Components

View deployment

requests against

the environment

https://developer.ibm.com/urbancode/




Configuration Management - Snapshots

Creating a Snapshot

Component Versions

1

1

2

2

3

3

321Web

Mid. Code

DB

Snapshot

3

2

1

Mid. Config 1 2 3 3

Snapshot Deployment

ENVSnapshot

3

2

1

WEB HOST

MID HOST

DB HOST

2

1

1

3

2

X

3 3 1

Contents of environments that pass tests

• contain deployable version and configuration

Deployments are based on deltas

They help with:

Automation, Audit, and Visibility

uDeploy


Version applications with Snapshots

39

Capture tested component versions and configuration settings as a snapshot that is promoted to other environments for better visibility and control


IBM Container Service

Automate the build of Docker images

Manage and distribute Docker images in private image registries

Easily host containers in the cloud

Scale and auto-recovery built-in

Logging and Monitoring built-in

40


Dev/Test with Hybrid Cloud

41

Off-P

rem

O

n-P

rem

Image Registry

IBM Bluemix

IBM UrbanCode Deploy PROD STAGING


About version control

All of your source code is likely already version controlled

All of your automation scripts, configuration files, tests, etc should also version controlled

UrbanCode Deploy automatically understands versions of components from your build process, but also versions all changes to configuration properties and automation processes

https://hub.jazz.net/create




IBM Bluemix DevOps Services – Project Mgmt

Source Control Management

• Hosted source control management within the cloud

• Support for Git and Jazz repositories • Manage users access via projects • Projects have developer centric tools

such as IDE, defect tracking, and a delivery pipeline


IBM Bluemix DevOps Services – Delivery Pipeline

• Integration test, System test, Production

• Automation ensures the same procedure in each environment

• Fully hosted, managed, and integrated in the cloud for the cloud


Code review should be part of your versioning process

Gerrit is a popular code review integration for git Used for all UrbanCode products

Enables you to create “pre-review”

verifications which automate running tests before a human reviews the change

Provides immediate, automated feedback about the quality of the change based on your test suites


Every aspect of our product delivery should be driven from the

principles of validated learning; no more anecdotes, only

meaningful results from our fast cycle prototypes.

- Michael D. Elder


Architecture

Automated Testing





About this deck

[email protected] http://linkedin.com/in/mdelder @mdelder

Download from https://ibm.biz/BdFc2b





https://ibm.biz/BdFc2b




© Copyright IBM Corporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.

www.ibm.com/software

http://www.ibm.com/software/rational

devops in practice - when does "practice" become "doing"

Technology

ibm corporation success

ibm corporation high

ibm corporation cool

continuous software

devops reference

devops principles

continuous delivery

architecture architecture