Continuous Delivery on Z at IBM—Stewart FrancisCICS Explorer Architectstewartfrancis@uk.ibm.com

CICS TS / August 2018 / © 2018 IBM Corporation

What is continuous delivery?—What are the advantages?

Produce software in short cycles ensuring the codebase is kept in a state where it could be released at any time

Reduces the time taken to deliver new features. For us this typically means moving from a multi-year delivery window to potentially monthly.

Reduces the risk of adopting new features as code drops are smaller, and introduce less instability to the codebase

Reduces the cost of developing new features

How does continuous delivery relate to continuous integration?

Continuous integration describes a build / test cycle that underpins continuous delivery.

Recommends frequent, small introduction of code to a shared codebase.

Code is built and tested as it is introduced to the code base by automated processes.

Reduces the amount of time it takes to find bugs, as regular integration builds trigger automated testing

Reduces the amount of time code remain isolated with one developer. Isolated code can be difficult to integrate if it has adopted a differing approach from other ongoing work.

Continuous integration is an approach to software development that often underpins continuous delivery, as it helps software be ready to release at any time.

Is continuous delivery the same as continuous deployment?

Or…

How often do you have to relase?

Continuous deployment is an extension of continuous delivery where changes are automatically deployed to production upon passing automated testing in a CI pipeline.

Practicing continuous delivery does not mean that you are obliged to release as fast as possible. It just means that you could!

Normally when we’re developing something for a continuous delivery release, we have some kind of intended release date planned. Releases are still allowed to be strategic!

Some software is well suited to continuous delivery. Amazon.compushes many updates to production every minute!

Different products have different release cadences. It all depends how they’re used.

The main benefit you should get from us adopting continuous delivery is that you don’t have to wait multiple years to get your hands on new features in our products.

Continuous delivery approaches on z

5

IBM MQ: long term support and continuous delivery

6

2016 2017 201820152014

V8.0 V9.0 LTS V9.0.1 CD V9.0.2 CD V9.0.3 CD V9.0.4 CDV9.0.5 CD

IBM MQ ApplianceM2000

IBM MQ ApplianceM2001

MQ on IBM Cloud

Long Term Support

Approximately every two years a new LTS version is released, rolling up CD capabilities into a release with 5+3 support.

Required by those looking for fixed function.

Mix and Match

Both are available under the same license.

Both can interoperate, just like any previous version of MQ.

Continuous Delivery

New CD versions of MQ are released approx. every 4 months, incrementally introducing new product capabilities.

Intended for those that can continually integrate.

In 2016 MQ introduced a dual Long Term Support and Continuous Delivery model

Followed pilot of introducing gated new function on top of V8.0

FAQs for LTS and CD releases of IBM MQ

7

https://www-01.ibm.com/support/docview.wss?uid=swg27047919

Addresses issues such as how long particular releases are supported, release frequency, etc.

Continuous delivery for Db2

8

Db2 adopted continuous delivery in v12

Application compatibility extended to support function levels provided by CD releases

Function levels activated by applying CD PTFs, but not available to applications unless they specifically enable that capability using application compatibility mechanism

Run the same version of Db2 for all applications, with varying functionality enabled

Red paper addressing Db2 CD approach -https://www.redbooks.ibm.com/redpapers/pdfs/redp5469.pdf

Currently available function levels -https://www.ibm.com/support/knowledgecenter/en/SSEPEK_12.0.0/wnew/src/tpc/db2z_db2functionlevels.html

Continuous delivery for CICS TS

9

CICS TS has delivered content through continuous delivery streams since CICS TS 5.1

We continue to deliver CD updates to CICS TS releases in the field, though typically we only deliver new features to the latest release

CD features are delivered as APARs

Example recent CICS TS CD deliveries:

CICS TS 5.3:

• OpenID support for Liberty in CICS

CICS TS 5.4

• OpenID support for Liberty in CICS

• Multi-factor authentication support for CMCI and CICS Explorer

Full list of CD features for CICS TS

Continuous delivery for CICS TS-Feature toggles

10

Because we use the same stream for CD and service, PTFs for CD can become pre-requisites of regular service PTFs

We introduced a mechanism called feature togglesto help manage this

Depends on the case, but typically changes to existing behaviour can be toggled off (and are usually off by default).

Toggles are specified in files in HFS, and can be specified for regions, or region groups

For example, multi-factor authentication support was delivered in CICS TS 5.4 CD toggled off.

More information about feature toggles

What about Eclipse tooling for z runtimes?

11

We have a whole suite of tooling based on Eclipse for administering runtimes on z:

• z/OS Explorer

• CICS Explorer

• MQ Explorer

• Db2 Data Studio

• IMS Explorer

• ID/z

• CICS Tools plugins for CICS Explorer

• CICS transaction gateway

• z/OS Connect

developerWorks download site

Group Name / DOC ID / Month XX, 2018 / © 2018 IBM Corporation 12

What about Eclipse tooling for z runtimes?

13

Typically we’ve always followed a CD approach with our Eclipse tooling, though we do tend to reserve larger changes for separately supported releases.

Most Eclipse tooling has 2 actively developed streams, one supporting Eclipse 4.4, and one supporting Eclipse 4.6.

Co-ordinated releases of products in these streams each month, delivering a mixture of new functionality and service updates

If you have internet access, you can get updates from the embedded updater in the products.

If not, you can mirror the update sites internally within your organizations for distributing updates to many users

Continuous integration in CICS TS

14

Every night we run a full build of CICS TS which serves as the baseline for the next day’s builds.

Throughout the day, every 30 minutes, we batch changes into logical groups and run a delta build using the nightly baseline.

Automated test are selected for delta builds based on the areas of the product that are affected by the changes.

Delta builds

Nightly baseline

Continuous integration in CICS TS

Every 8 hours we schedule an increment build. This runs a full build, and generates the SMP/E data required for installation.

We run our core regression suite against increment builds. If we don’t encounter any regressions, we consider these builds candidates for betas.

There are a lot more tests that get run pre-release!

We also produce a nightly developers integrated build, which is used by development teams as a baseline for their development environments.

Increment build

Nightly baseline

Continuous integration in CICS TS – what technology are we using?

16

RTC for source control + custom extensions

Jenkins for orchestration

Custom JUnit extensions for automated testing

Custom ant extensions for build

Continuous integration in CICS Explorer

17

For CICS Explorer, we’re actively working on 2 streams at once: CICS Explorer 5.4 for CD releases, and CICS Explorer 5.5 for bigger functional changes that pre-req CICS TS 5.5.

For both streams, we have a nightly build which runs a suite of automated unit tests and integration tests.

We also run a large suite of functional verification tests regularly, which perform automated UI testing.

Continuous integration in CICS Explorer – what technologies are we using?

18

Jenkins for orchestrationRTC for source control

Artifactory for asset management

Maven for builds

Junit for unit testing Rational Functional

Tester for FV

Continuous integration in CMCI GraphQL server

In CICS TS 5.5 we introduced a new API for system management data built on a technology called GraphQL.

We’re taking a different approach to how we build this component, and how it’s integrated with the rest of CICS.

The API is provided by a Java application running in Liberty, within CICS

We’re using git for SCM, hosting our code internally on an instance of GitHub Enterprise.

Similar to CICS Explorer & others, the build is written using Maven, but scheduled using Travis.

Build is triggered on source code delivery to GHE and results are pushed to Artifactory.

Main CICS TS build integrates this component by retrieving a named build from Artifactory at CICS TS build time.

Continuous integration in CMCI GraphQL server – what technologies are we using?

GitHub Enterprise

Travis

Artifactory Maven for builds