devops with activemq, camel, fabric8, and hawtio
DESCRIPTION
DevOps and Continuous Delivery slides with Fabric8 (http://fabric8.io), HawtIO (http://hawt.io), Camel, ActiveMQ, Docker Jenkins, etc.TRANSCRIPT
DevOps with ActiveMQ,
Camel, Fabric8 and HawtIO
Christian Posta 6/11/14
RED HAT | ADD NAME 2
Agenda
• DevOps… What is that?
• Enterprise Integration
• Automated Delivery
• To the Cloud
RED HAT | ADD NAME 3
Your speaker Christian Posta
http://christianposta.com/blog
@christianposta
• Principal Middleware Specialist
• Based in Phoenix, AZ
• Committer on Apache Camel, ActiveMQ, Apollo, PMC on ActiveMQ
• Author: Essential Camel Components DZone Refcard
Shorten the lifecycle from inception to production so that the business can make money! Developers and Operations must cooperate. Rely on tools/frameworks to automate, automate, and automate.
DevOps
RED HAT | ADD NAME 6
“WTF is DevOps?” • IT is a core competency • Set of principles • There’s more to applications than coding! • Feedback • Repetition • Communication • People!
RED HAT | ADD NAME 7
Shorten the feedback loop • Developers • QA • Operations • Business • Customers!!!
Enterprise Integration
RED HAT | ADD NAME 9
Integration is easy!
RED HAT | ADD NAME 10
• Off the shelf? Home Grown? Acquisition? • Platforms • Protocols / Data Formats • Data Formats • Timing • Organizational mismatch
Why is integration hard?
RED HAT | ADD NAME 11
• Light-weight integration library • Domain Specific Language • Enterprise Integration Patterns • Components • Routing and Mediation (like an ESB?) • Runs in any container (or stand alone)
What is Apache Camel?
RED HAT | ADD NAME 12
• Message Routing • Transformation • Aggregation • Splitting • Resequencer • Routing Slip • Enricher • All 65 from the book!
Enterprise Integration Patterns
RED HAT | ADD NAME 13
Components • ActiveMQ, Websphere, Weblogic (JMS)
• AMQP
• ATOM feeds
• AWS (S3, SQS, SNS, others)
• Bean
• Cache (EHCache)
• CXF (JAX-WS, JAX-RS)
• EJB
• Drools
• File
• FTP/SFTP
• Google App Engine
• GMail
• HTTP
• IRC
• jclouds
• JDBC
• Jetty
• MQTT
• MyBatis
• JPA
• Spring Integration
• Spring Web Services
http://camel.apache.org/components.html
To see list of all components!!
RED HAT | ADD NAME 14
Java DSL
public class OrderProcessorRouteBuilder extends RouteBuilder { @Override public void configure() throws Exception { from(“activemq:orders”) .choice()
.when(header(“customer-rating”).isEqualTo(“gold”)) .to(“ibmmq:topic:specialCustomer”) .otherwise() .to(“ftp://user@host/orders/regularCustomers”) .end() .log(“received new order ${body.orderId}”) .to(“ibatis:storeOrder?statementType=Insert”); }}
RED HAT | ADD NAME 15
Spring XML DSL
<route id=“processOrders”> <from uri=“activemq:orders”/>
<choice> <when> <simple>${header.customer-rating} == ‘gold’</simple> <to uri=“ibmmq:topic:specialCustomer”> </when>
<otherwise> <to uri=“ftp://user@host/orders/regularCustomers” /> </otherwise>
</choice> <log message=“received new order ${body.orderId}”/> <to uri=“ibatis:storeOrder?statementType=Insert”/> </route>
RED HAT | ADD NAME 16
• Batch file transfers • Shared Database • RPC • Messaging
Integration Options
RED HAT | ADD NAME 17
• Asynchronous architectures • Reliable message passing • Loose coupling • Heterogeneous integration • Fault tolerant • Scalable • Real-time data
Why messaging?
RED HAT | ADD NAME 18
• The most widely used open-source messaging broker
• Highly configurable • Friendly license (no license fees!) • Vibrant community (TLP) • Backbone of top enterprises in retail, e-retail,
financial services, shipping, many others!
Apache ActiveMQ
RED HAT | ADD NAME 19
• High performance • High availability • Light-weight • Multi-protocol (AMQP, MQTT, STOMP) • Multi-transport (TCP,SSL,WS,VM,HTTP) • JMS compliant • Supported in production by Red Hat!
ActiveMQ Features
RED HAT | ADD NAME 20
Network of Master/Slave
RED HAT | ADD NAME 21
High-performance, real-time systems
Head Office
STORE
STORE
outlet
shop
24 houra
STORE
shop
Broker Clusters
• Clustering, Network of Brokers
• Client-aware failover
• Master/Slave HA
• Fabric
RED HAT | ADD NAME 22
Ingestion for BigData Architecture
Broker Cluster
Broker Cluster
Web Servers
Web Servers
Web Servers
Web Servers
Camel HTTP Log
Reader
Camel HTTP Log
Reader
Camel HTTP Log
Reader
Camel HTTP Log
Reader
ESB
ESB
Camel HBase/HDFS
Camel Hbase/HDFS
HDFS
RED HAT | ADD NAME 23
Integration Everywhere – Internet of Things
Connecting Things
• mobile devices • meters • industrial controls • smart buildings • asset tracking • traffic control • monitors • sensors • actuators
Broker Clusters
Arrival Airport 1
RED HAT | ADD NAME 24
JBoss Fuse Integrate Everything!
RED HAT | ADD NAME 25
More info on JBoss Fuse…
https://www.redhat.com/products/jbossenterprisemiddleware/fuse/
http://www.jboss.org/products/fuse
Demo
Management
RED HAT | ADD NAME 28
RED HAT | ADD NAME 29
Current issues with deployments
• Installation • SSH, download, unpack, install, permissions, ENV var
• Configuration • Container, individual apps/services, brokers, security
• Upgrading • Install, configure, rollback
• Introspection • Big picture statistics (JVM/OS memory usage, CPU) • JMX (jconsole, visualVM)
RED HAT | ADD NAME 30
Clients aware of Topology
• Brokers • failover:(tcp://host1:port1,tcp://host2:port2)
• Camel endpoints • from(“jetty:http://22.33.44.55:8080/endpoint”) • to(“http4://11.22.33.44:9000/endpoint”)
• Web service endpoints • Access a specific endpoint: http://22.33.44.55:9091/endpoint
RED HAT | ADD NAME 31
Current best practices
• Use templates for configuration w/ template engine • Puppet/Chef and/or Capistrano/Ansible • Store configuration (templates + values) in SCM • Separate configuration from binary deployments • Verifiable build and release process
RED HAT | ADD NAME 32
• Simplifies deployments • Provides management tools for centralized
configuration • Visualize your middleware with HawtIO • Polycontainer • Blurs the line of PaaS
http://fabric8.io
RED HAT | ADD NAME 33
• Provides cluster capabilities, coordination • Service discovery, load balancing, failover • Deploy to cloud (IaaS, PaaS) • Supported as JBoss Fuse (managed)
• fabric8 1.0 is in Fuse 6.1 • 1.1 about to be released
• RC1 released 6/10
http://fabric8.io
RED HAT | ADD NAME 34
fabric8 1.1 release
• Java Container • Tomcat, TomEE, Jetty • Spring Boot • fabric:watch * with various containers • Profile import/export • fabric8 maven plugin enhancements
RED HAT | ADD NAME 35
Architecture
Clustered Registry
Agent
Agent
Zookeeper
Agent
Zookeeper
Agent
Zookeeper
Version 1.0
Profile CXF Profile Camel
Profile Default
Features Configuration
Registry Content
containers that form an “ensemble”
Pulls profile data
Registers / Listens for changes
RED HAT | ADD NAME 36
Core concepts
• Fabric registry
• Holds all configuration data • Runtime registry for looking up distributed services
• Profile
• Describes the container set up • Features, Bundles, ConfigAdmin PIDs, system properties • Hierarchical structure • Container versioning
• Agent
• Runs on each container • Communicates with registry to make sure container provisioned
correctly
RED HAT | ADD NAME 37
How’s fabric8 different than Puppet/Chef?
• Middleware centric • Container agnostic • Deep knowledge about the running processes • Consistent configuration • Visualizations • Versioning built in, upgrades, rollbacks, selective
upgrade, etc • Use puppet/chef to provision your machines, use
fabric8 to provision and manage your middleware/apps
RED HAT | ADD NAME 38
Fuse Management Console
RED HAT | ADD NAME 39
HawtIO http://hawt.io
RED HAT | ADD NAME 40
HawtIO http://hawt.io
RED HAT | ADD NAME 41
FuFuse Management Console
Demo
Continuous Delivery
RED HAT | ADD NAME 44
Continuous Delivery • Builds on continuous integration • Establish a concrete pipeline to production • Build/Test/Release often! • Bottlenecks? • Involves Dev and Ops to be successful • Every build is a “release candidate”
RED HAT | ADD NAME 45
Automate everything! • Developers
• Unit tests • Integration tests • Builds • Deployments in dev
• Operations • VMs • Provisioning software • Deployments in QA/UAT/PROD
RED HAT | ADD NAME 46
Tools for a CD pipeline • Puppet/Chef to provision VMs • Git for SCM • Gerrit/Gitlab for code reviews • Maven • Jenkins + plugins • and of course… Fabric8! • What’s this Docker thingy?
RED HAT | ADD NAME 47
Fabric8 maven plugin • fabric8:deploy • fabric8:zip • fabric8:aggregate-zip • fabric8:branch • fabric8:script (for karaf only) • http://fabric8.io/gitbook/mavenPlugin.html
RED HAT | ADD NAME 48
Sample flow • Check your code in • Gerrit for code reviews • Jenkins for build + CD pipeline • Use fabric8:zip to deploy profiles to Maven repo • Use fabric8:branch to automate deploying multiple
profiles to QA/UAT/PROD • Can use profile-import to manually import zips • Build the binary once!
Demo
Containers
RED HAT | ADD NAME 51
Docker • http://docker.io • Open source • Lightweight VEs • Linux containers (LXC) • Portable packaging • Versioned • Repositories
RED HAT | ADD NAME 52
How different than VMs?
RED HAT | ADD NAME 53
Fabric8 Docker support
• Docker images • Build docker containers for your java apps • Flat classpath • Microservices
Demo
OpenShift PaaS
RED HAT | ADD NAME 56
What is OpenShift?
Red Hat’s free platform as a service for applications in the cloud.
RED HAT | ADD NAME 57
What’s supported?
RED HAT | ADD NAME 58
Flavors
origin
Public Cloud Service
On-premise or Private Cloud Software
Open Source Project
RED HAT | ADD NAME 59
Terminology l Broker – Management host, orchestration of Nodes
l Node – Compute host containing Gears
l Gear – Allocation of fixed memory, compute, and storage resources for running applications
l Cartridge – A technology/framework (PHP, Perl, Java/JEE, Ruby, Python, MySQL, etc.) to build applications
l Application – Instantiation of a Cartridge
l Client Tools – CLI, Eclipse Plugin, Web Console, Java API, REST API
RED HAT | ADD NAME 60
xPaaS/iPaaS
l xPaaS
- Cartridges for Entire JBoss & Fuse Portfolio
- EAP and EWS Enterprise Cartridges Today
- Several Community Cartridges Available Today
l iPaaS
- Integration PaaS
- Fuse/Fabric Cartridge
RED HAT | ADD NAME 61
More Info
http://fabric8.io
http://hawt.io
http://docker.io
http://jboss.org/products/fuse
http://activemq.apache.org
http://camel.apache.org