© 2014 IBM Corpora/on
Please Note IBM’s statements regarding its plans, direc/ons, and intent are subject to change or withdrawal without no/ce at IBM’s sole discre/on. Informa/on regarding poten/al future products is intended to outline our general product direc/on and it should not be relied on in making a purchasing decision.
The informa/on men/oned regarding poten/al future products is not a commitment, promise, or legal obliga/on to deliver any material, code or func/onality. Informa/on about poten/al future products may not be incorporated into any contract. The development, release, and /ming of any future features or func/onality described for our products remains at our sole discre/on.
Performance is based on measurements and projec/ons using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considera/ons such as the amount of mul/programming in the user’s job stream, the I/O configura/on, the storage configura/on, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
1
What will this session cover? • WMB and IIB have support for two main user roles
– Integra/on Developer – Administrator
• This session will aim to cover accepted best prac>ce for the most typical administrator tasks – Including some essen/al “top /ps”
• Topics to discuss – Tools of the trade – Common administra/ve tasks
• I’ll focus on WMB V8 and IIB V9 – but will point out some V10 Open Beta improvements too!
Interac>on With Tools
Integra>on Toolkit Command line
Integra>on Node
Third Party Tools
Integra>on API (CMP)
REST API
Integra>on Explorer
Web Admin
REST API WAS Admin
Integra>on Toolkit View for Developers
Integra>on Explorer
• Management op/on designed for administrators
• Plug-‐in to MQ Explorer
• Extra features − Create/Manage Configurable
Services
− Performance Views
− Group integra/on nodes
− Administra/on Log
− Administra/on Queue
− Manage sta/s/cs & tracing
BIP1121I: Creates an execution group. Syntax: mqsicreateexecutiongroup brokerSpec -e egName [-w timeoutSecs] [-v traceFileName] Command options: 'brokerSpec' is one of: (a) 'brokerName' : Name of a locally defined broker (b) '-n brokerFileName' : File containing remote broker connection parameters (*.broker) (c) '-i ipAddress -p port -q qMgr' : hostname, port and queue manager of a remote broker '-e egName' name of the new execution group '-w timeoutSecs' maximum number of seconds to wait for the execution group to be created '-v traceFileName' send verbose internal trace to the specified file.
Command line tools • A wide selec>on of tools for scrip>ng ac>ons • Requires a configured environment
– mqsiprofile or mqsicommandconsole (Windows) mqsiprofile (Linux/UNIX) JCL or ISPF (z/OS)
• Most commands work against local or remote integra>on nodes
Integra>on API (CMP) • Java interface that enables the administra>on tools • Use for custom administra>on requirements • Fully documented and samples available
• MB v8 and later allows you to create and edit message flows too – Build your en/re system programma/cally!
Web Visualisa>on and Analy>cs § A comprehensive tool for web management
– Manage all integra/on resources from zero-‐footprint client – Analyze integra/on performance in real-‐/me – Supported on a variety of browsers: IE10, Firefox, Safari… – Complements MQ Explorer and WAS Admin consoles
§ Managing Integra>on Resources – View top-‐level integra/on node proper/es – Add/remove/change integra/on servers – Start/Stop integra/on data flows – Role based access to control usage – Advanced op/ons include data replay, policy & monitoring – Exploits underlying public REST/JSON API
§ Integra>on Performance Analysis – Opera/onal experience; no developer interven/on required
• New and exis/ng flows can exploit without change
– Many metrics of integra/on flow available in real-‐/me • CPU & I/O /me shown by default in integra/on analyzer • Other metrics include thread, data sizes, errors…
– Flexible display includes data tables and flow profile • Drill down to understand detailed behaviour
– Exploits underlying MQTT web sockets technology • Asynchronous no/fica/on at low CPU cost
8
Significant Tool changes in V10 Open Beta
• Single install package and single installed component – Unit test integra/on node automa/cally started with IIB development tool – Can create addi/onal integra/on nodes as required
• No longer any MQ pre-‐requisite – All exis/ng MQ use-‐cases s/ll work (and now support remote MQ connec/ons) – All administra/on done through the web UI rather than the Integra/on Explorer – Remote administra/on commands need to point at the integra/on node’s administra/on
port rather than the queue manager
• Uses a single command ‘iib’ – e.g. iib studio, iib help, iib list – Equivalent of ‘mqsicommand’ is ‘iib command’ – Exis/ng commands con/nue to work
(no/ng remote administra/on caveat above)
Development Lifecycle and Environments
• Integra>on Developer – Develops message flows, message
models etc. – Unit Tests on local machine – Creates archive (BAR) files
containing required artefacts
• Administrator – Customizes BAR for target
environment (message flow proper/es including queues, database names etc.)
– Deploys BAR to target environment – Management and opera/onal control
Development Test QA Produc/on
Common Administra>ve Tasks
• Planning and configura>on – Bringing a new integra/on node online – Making an integra/on node highly available – Planning for disaster recovery – Securing an integra/on node
• Managing integra>on nodes – Deployment and redeployment – Understanding behaviour – Op/mizing and tuning – Migra/on – Maintenance
Bringing a new integra>on node online
• Typical steps required – Preconfigura/on (e.g. OS, userids) – Installa/on (MQ, IIB, addi/onal sojware) – Crea/ng the integra/on node – Crea/ng integra/on servers – Crea/ng Configurable Services – Deploying BAR files – Addi/onal configura/on (e.g. security, userids)
• And don’t forget:
– Documenta/on – Scrip/ng – Virtualiza/on
Scrip>ng and Automa>on
• Four general approaches for IIB provisioning – Crea/ng and configuring brokers manually – Use of commandline tools from within scripts (e.g. shell scripts, Jenkins, Ant, Maven…) – Hypervisor images for deployment directly on public or private cloud – Use of technologies such as Chef and Puppet for all environments (cloud, non-‐cloud)
• Choose an approach that enables your environment to be reproduced easily
<project> <target name=“deploy”> <cvs command=“checkout” … /> <mqsipackagebar … /> <mqsiapplybaroverride … /> <mqsideploy … /> </target> </project>
Making the integra>on node highly available
• How do I ensure that the integra>on node is con>nually processing messages? – Ac/ve/Ac/ve vs. Ac/ve/Passive – Agree SLAs with the business (% up/me)
• The integra>on node includes restart recovery of integra>on servers – But this is usually not sufficient on its own – Does not cover machine restart
• Two main op>ons – Third-‐party solu/ons (e.g. VCS/HACMP/MSCS) – Mul/-‐instance queue managers and integra/on nodes
• Distributed Cache – Built in cache support in 8.0.0.1 and later. – Enables HA scenarios
Planning for disaster recovery
• What would you do if your primary WMB loca>on goes down?
• Distribute WMB to mul>ple sites if possible – This introduces data replica/on and latency concerns
• Keep DR concerns separate from HA! – HA: Systems at a single site with a single configura/on
DR: Systems at mul/ple sites with replicated configura/ons – An HA failover can be a planned ac/vity – DR is unplanned.
Another approach to disaster recovery…
• Consider an architecture where no one service was taken for granted… – Trea/ng all elements of a system, regardless of business u/lity, as ‘disposable’ – Forces you to design in a way that enforces resilience and scalability – Requires up-‐front effort, but takes many concerns off the table (e.g. backup)
IIB IIB IIB
• IIB enables an architecture that removes single points of failure – Even easier with IIB V10 Open Beta due to remote queue manager support
…
Cloud Integra>on • The idea of disposability is facilitated by cloud…
– Trea/ng compu/ng resources like u/li/es such as gas, electricity – Pay for only what you use
• IIB has a range of IaaS cloud op>ons – Private Cloud
• IBM Integra/on Bus Hypervisor Edi/on • IBM Workload Deployer • Pure Applica/on Systems Papern • Chef scrip/ng
– Public Cloud • Support for public cloud providers, e.g. Sojlayer • Chef scrip/ng
– Future plans around IaaS (e.g. Sojlayer), PaaS (e.g. Bluemix) and SaaS connec/vity
• Consider virtualiza>on for new environments – Makes it easy to provision systems (and restore known state!) – Understand maintenance and performance aspects
SaaS
IaaS
PaaS
Securing WMB and IIB
• Simplified administra>ve security • 3 levels of authorisa>on for administra>ve ac>ons
– Reading (e.g. View integra/on node proper/es -‐ mqsilist) – Wri/ng (e.g. Change integra/on node proper/es -‐ mqsicreateexecu/ongroup)
– Execu/ng (i.e. star/ng and stopping) – See IIB Knowledge Center topic bp43540 for more details
• On two object types: – “Integra/on node” and “Integra/on servers”
• Administra>ve Security is not enabled by default – Enable through mqsichangebroker / mqsicreatebroker
• Access controlled using MQ queues on the integra>on node’s queue manager – Give authority to users or groups
+inq = Read
+put = Write
+set = Execute
Security Queues SYSTEM.BROKER.AUTH SYSTEM.BROKER.AUTH.<egname>
Managing what’s deployed
• It can be difficult to understand what individual deployed resources are used for • Applica>ons and libraries can allow you to understand why each file is there
– Applica/on: Encapsulates a single use case or scenario – Library: Promotes re-‐use of a shared set of files
– Libraries are sta/cally bound to the applica/on (V8, V9, V10 OB) or shared (V10 OB only)
• Concepts are shared between developers and administrators – The developer chooses to create an applica/on or library; the collec/on is then carried all the way through
to the run/me
Backing Up
• Consider using mqsibackupbroker to backup the integra>on node’s configura>on – Ideally, ajer all configura/on changes – Best run when stopped – But can be used whilst ac/ve as long as not undergoing configura/on changes
• Also consider any other required resources, for example: – Database tables – Source artefacts (message flows, BAR files)
• Ensure regular restore tes>ng – Prove your process works
• Paeern to recons>tute message flows from a running integra>on node – Available on MQSeries.net
Understanding integra>on node behaviour • The tools include a lot of informa>on that is useful to the administrator
– Administra/on queue and log, Message flow and resource sta/s/cs – Warnings when features that affect performance are enabled
• Use this informa>on to understand recent configura>on changes
− How the integra/on server is performing, connected endpoints, etc.
Understanding integra>on node behaviour • The Ac>vity Log shows you all recent ac>vity on a message flow or resource manager
– For example, “Show me all recent JMS ac/vity” • Visible in Integra>on Explorer and/or wrieen to a file • Customisable rota>on rules (based on age or size) and content
Op>mizing and tuning
• The tools allow you to modify the configura>on opera>onally • These tweaks more efficient to make than modifying message flows • Encourage developers to create message flows that enable opera>onal tweaks to be made
– User-‐defined proper/es, Configurable Services, User-‐defined configurable services, Policies
Managing Unresponsive Integra>on Flows § Target unresponsive flows through policy to improve overall system reliability
– Addi/onal WLM op/on aimed at unresponsive integra/on flows – An integra/on flow can become unresponsive for mul/ple reasons
• e.g. Wai/ng for external system, infinite loop, deadlock, malformed XML
§ Flexible configura>on, ac>ons and repor>ng op>ons – Specify threshold at which flows are considered unresponsive, e.g. 30 seconds for processing
• Configured via WLM policy, or directly on the flow in the BAR file
– Define ac/on to trigger when flow considered unresponsive • Administra/ve no/fica/on through a new “/meout exceeded” event message
– If flow eventually con/nues through to comple/on, a second event is published • Restart the integra/on server (execu/on group) on which the unresponsive flow is running
– New command op/on to forcibly stop integra/ons manually: mqsistopmsgflow –f
25
Waiting for response
Version-‐to-‐Version Migra>on § Migra>on to IIB V9 from WMB V6.1, V7 and V8
– All development assets (e.g. message flows, ESQL, DFDL, Java, Maps and XSLT) import directly • Right-‐click convert ac/on for pre-‐V8 maps; some manual tasks may be required
– Migrate brokers using a single command, or create new integra/on nodes for phased migra/on • No redeployment necessary when using built-‐in migrate command • All exis/ng BAR files can be deployed to IB V9 integra/on nodes without change
§ Migra>on commands for in-‐place migra>on – Includes migra/on of configura/on data including databases, queues and registry – Forwards and backwards migra/on of exis/ng components, in situ
• mqsimigratecomponents command (includes –t op/on for rollback to V7 and V8)
§ Flexible co-‐existence op>ons remove the need for addi>onal hardware when migra>ng – IB V9 co-‐exists on the same OS with all previous MB versions – MQ V7.5.0.1 required for all IB V9 integra/on nodes
• MQ V7.5.01 supported with V7 and V8 brokers for the purposes of V9 migra/on • For V6.1 migra/on, upgrade MQ and MB simultaneously
26
4 3 2 1 Install IB V9 Stop broker Run migrate command Start broker
4 3 2 1 Install IB V9 Create new broker Deploy exis/ng assets Stop old broker
or
Maintenance
• Schedule regular maintenance windows – IBM recommends that you are on the latest maintenance level – Plan exactly what will be applied and when – A highly available environment ensures that there is no down/me
Summary
• We have discussed a number of different ways that help ensure that administra>on is trouble-‐free
• My Top Tips
– Always ensure your environment is reproducible – Treat DR and HA separate – Ensure regular backups – Encourage developers to create message flows that enables opera/onal tweaks to be made
– Schedule regular maintenance windows