automation in cloud

• Welcome to Pune Cloud Engineers and Architects First Meetup.

Automation in Cloud

Cloud Quiz• Number of SSL certificates supported by ELB at given point of

time ?• What does Amazon S3 stands for?• What does RRS stands for when talking about S3?• What are the 2 permissions provided by AWS?• What are the Amazon Ec2 API tools?

Some Interesting Facts About Cloud Computing

• Cloud service revenue will top $148.8 billion in 2015 and about $241 billion in 2020, according to Gartner.

• According to AMD (1/12), 70 percent of businesses are either using or investigating cloud computing solutions.82 percent of all companies saved money in their last cloud adoption project.

• 65 percent of the companies chose cloud based solutions more than a year ago.

• Cloud computing gives organizations a green boost. About 64 percent of organizations say that adopting cloud based solutions has helped them reduce waste and lower energy consumption.

• More than 90 percent of all companies saw at least one area of improvement in their IT department since they moved to the cloud.

• 52 percent reported increased data center efficiency and utilization while 47 percent companies said that they witnessed lower operating costs after cloud adoption

• 60% reported cloud computing as their highest IT priority.• 74% are already using some form of cloud computing technology.• 64% are investing in training new and current employees on their cloud

expertise.

Why Automation?

• In addition to efficiency, there are also considerations on consistency, repeatability, and predictability to programmatically carry out tasks.

• It keeps you making lazy.

What we are going to cover?

• Fabric• Boto• Provisioning , Deployment and System Admin Tasks using

Fabric and Boto.

What is Boto?

• Boto is a Python interface to the family of Amazon Web Services. In this article I’ll only be referencing the Boto EC2 API.

• OpenSource project hosted on github. https://github.com/boto/boto Fork it .• Also supports services like Openstack ,Eucalyptus.

What is Fabric?Fabric is a Python library for automating application deployment and other systems administration tasks. Fabric allows you to easily run your Python programs .• Python SSH library• Opensource project hosted on Github • Encapsulates common SSH/SCP options.• Typical usage involves creating a python module containing

one or more functions, then executing them via the fab command-line tool.

• You can execute shell commands over ssh, so you only need to have ssh running on the remote m/c. It Interact with the remote m/c tha you specify as if they were local.

How to install Fabric and Boto?

• Fabric and Boto requires Python 2.5 or later, the setuptools packaging/installation library, the ssh Python library, and SSH and its dependencies.

• For the most part, you won't have to worry about any of this, because Fabric can be installed easily through various package managers

• The easiest, and most prolific way to install Fabric is using pip (or easy_install). On most systems, you can use your systems package manager (apt-get, install, and so on) to install it (the package either will be fabric or python-fabric).

• Once installed, you will have access to the fab script from the command line.

• Pip install fabric boto• Sudo easy install fabric boto • Sudo apt-get install fabric boto

Why Fabric and Boto?

• We can use fabric for deployments and system admin tasks.• Fabric is task runner , can run commands over ssh locally and

remotely.• Boto is Amazon Web Service python API , and can connect ec2

and other services.

• With the use of Fabric and Boto we can create instance , deploy to instance , scale instance and manage instance.

Our First Fab File

• To start just create a blank file called fabfile.py in the directory you’d like to use the fabric commands from.

• You basically write rules that do something and then you (can) specify on which servers the rules will run on.

• Fabric then logs into one or more servers in turn and executes the shell commands defined in “fabfile.py” .

• If you are located in the same dir as "fabfile.py" you can go "fab --list“• Below is a small but complete "fabfile" containing a single task:

#!/usr/bin/pythonfrom fabric.api import rundef host_type():run (`uname –s`)

• Once a task is defined, it may be run on one or more servers, like so# $ fab –H localhost.linuxbox host_type# [localhost] run: uname –s# [localhost] out: Linux# [linuxbox] run: uname –s# [linuxbox] out: Fedora# Done.# Disconnecting from localhost… done# Disconnecting from linuxbox.. done

• You can run fab -h for a full list of command line options

Fabric provides a set of commands in fabric.api that are simple but powerful.

With Fabric, you can use simple Fabric calls like

• Local # execute a local command)• Run # execute a remote command on all specific hosts, user-

level permissions)• Sudo # sudo a command on the remote server)• Put # copy over a local file to a remote destination)• Get # download a file from the remote server)• Prompt # prompt user with text and return the input (like

raw_input))• Reboot # reboot the remote system, disconnect, and wait for wait

seconds)

Execution Strategy

Fabric by defaults runs in single, serial execution method, from Fabric 1.3 parallel mode is also available.

• A list of tasks is created. • For each task , a task-specific host list is generated from

various sources.• The task list is walked through in order , and each task is run

once per host in its host list.• Tasks with no hosts in their host list are considered local-only• We can run task on globally on all hosts or we can define

hosts per tasks or roles.

Roles in Fabric• We know how to run task on single server , but what if we

want to run particular task on set of servers ? We can define Roles in Fabric

from fabric.api import env env.roledefs[‘webservers’] = [‘web1’,’web2’]

Parallel Execution• Fabric by default executes tasks serially. If we need to run

specified tasks parallel we need to tell fabric .• Suppose we need to install nginx and start nginx on set of

servers, by default fabric run serially.1. Install nginx servera2. Install nginx serverb3. Restart nginx servera4. Restart nginx serverb

By using parallel execution 5. Install nginx servera serverb

6. Restart nginx servera serverb

Failure Handling in Fabric

• Once we define task list fabric start executing them one by one as per the execution strategy.

• By default fabric works in fail-fast method , if any thing goes wrong fabric gets halts.

env.warn_only = False (By Default)

If we make it env.warn_only = true Fabric will display warning and continue executing next tasks.

Some Fabric Commands Example

Few fabric commands• fab task1 task2 task3• fab -f firstfab.py command:"hostname –f”• fab set_hosts:tagname,us-east-1 task1 task2 task3