xen summit2010 globocom

case studyIaaS using XCP and XAPI

marco sinhorelivirtualization project leader

Monday, May 17, 2010

• Internet branch of Globo organizations

• Biggest media conglomerate in Latin America and 4th around the world

• Largest vertical web portals in Brazil

• More than 150 million page views per day

• Average of 27 million unique visitors in February 2010

• More than 1500 physical servers in the production datacenter

• More than 60 Gbps internet link

• Considered benchmark for high-volume web distribution

• Responsible for the largest simultaneous video streaming audiences in the country

about globo.com

Monday, May 17, 2010

• Automate existing manual workflow between teams’ processes

• Keep existing automated systems used for infrastructure setup

• Identify new infrastructure processes to be automated

• The IaaS needs to fit the company and not the reverse

making decisions

Monday, May 17, 2010

new infrastructure processes

Switches

Virtual MachinesPhysical Servers

LoadBalancers

Monitoring

NetworkAPIcore

Core applicationXAPI

REST

REST

Deploy

All infrastructure processes are integrated in a single core application responsible for the flux organization. Each process has an API controlling its physical/logical/virtual resources.

Monday, May 17, 2010

• Relationship between server and switch ports

• needed for adding tagged vlan into switch port trunk when a VM is installed or started

• Switch communication by SNMP or SSH

• check/add/remove/list vlans in switch ports

• Load-balancer communication by API or SNMP or SSH

• management of virtual IPs

• Users profiles

• implement policies for human or system users

networkAPI

Monday, May 17, 2010

• The monitoring is a part of the VM life-cycle

• when the VM is started all information about the VM is included in the monitoring system and then enabled

• when the VM is stopped or is tagged as “maintenance”, the monitoring is disabled

• if the VM is removed, the VM monitoring is removed as well

monitoring

Monday, May 17, 2010

• Used for internal application deployment

• Install all dependencies needed in OS

• Setup the virtual machine for applications

• Policies for users and groups

• Dev/QA/Staging/Production environment

deployment

Monday, May 17, 2010

• Pooling

• One place to get status and setup all pool virtual resources

• Easy API

• Fast to program and maintain

• Many client bindings for many flavors

• Paravirtualization

• Large and devoted community

motivation to use XCP

Monday, May 17, 2010

• Django application running on Apache

• Users access front-end and authentication

• Unifies all others IaaS components

• Defines all workflow and system interactions

IaaS front-end component

Monday, May 17, 2010

• Django application running on Apache

• VNC console proxy encapsulated with token to identify client

• Translates REST calls from front-end application to native xapi calls

• All calls to XCP pools are only natively accessible from the integration component

IaaS xapi integration component

Monday, May 17, 2010

• Hosts

• PXE boot

• answerfile for unattended installation

• rewrite /etc/firstboot.d files to not change the static IPs

• VM

• kickstart for unattended installation integrated into system

• static IP configured by system inserting information on Linux kernel cmdline

host and VM installation

Monday, May 17, 2010

system view (traditional)

Monday, May 17, 2010

system view (logical)

Monday, May 17, 2010

benefits of this new approach

Monday, May 17, 2010

• Defines the workflow between teams

• Integrated with ticket system to request an human action

• Keeps the same flux previously used when the actions weren’t automated

• Checklist of each item to release the resources

workflow

Monday, May 17, 2010

user profile

• Each profile has a group of actions in the system

• Some profiles inherit group actions of others

Monday, May 17, 2010

user profile

Monday, May 17, 2010

user profile

Monday, May 17, 2010

user profile

Monday, May 17, 2010

user profile

Monday, May 17, 2010

user profile

Project administrator

Monday, May 17, 2010

user profile

Project administrator

Monday, May 17, 2010

user profile

Project administrator

Monday, May 17, 2010

user profile

Monday, May 17, 2010

user profile

Monday, May 17, 2010

user profile

Monday, May 17, 2010

user profile

Monday, May 17, 2010

user profile

Monday, May 17, 2010

roadmap• Deployment (Integrate with the IaaS core to tag the deployment in projects)

• DNS (Manage the DNS using an API)

• Monitoring (rich details)

• XCP

• Permit many VMs to use the same shared storage block in web farms

• Master recovery

• Virtual Machine HA

Monday, May 17, 2010

thanks

Questions?

Monday, May 17, 2010