tools and virtualization to manage our operations at puppet labs - puppetconf 2014

VIRTUAL OPERATIONSVIRTUAL OPERATIONSTOOLS AND CHALLENGES TO MANAGING A VIRTUAL IN‐TOOLS AND CHALLENGES TO MANAGING A VIRTUAL IN‐

FRASTRUCTURE AT PUPPET LABSFRASTRUCTURE AT PUPPET LABSCreated by Cody Herriges

INTRODUCTIONINTRODUCTIONPuppet LabsOperations EngineerSystems Team lead

GENERAL STRUCTURE AND PHILOSOPHYGENERAL STRUCTURE AND PHILOSOPHYThree sub-teams that make up a core Technical OperationsteamProvide a collection of core services

Everything from core networking to web hosting andapplication managementMy team specifically: virtualization/cloud, storage, andbackup platforms

We're only the beginningTHERE IS NO DEVOPS TEAM!Fairly agnostic

HOW HETEROGENEOUS...?HOW HETEROGENEOUS...?VMware vSphereOpenStack NovaSolaris ZonesLinux ContainersVanilla KVM/libvirtAIX LPARsSolaris LDOMsVMware FusionVMware WorkstationVirtualBox

HOW HETEROGENEOUS...?HOW HETEROGENEOUS...?Google Compute EngineAmazon EC2LinodeRackspace

Going to save everyone a similar list for compute, storage, andnetworking.

HOW HETEROGENEOUS...?HOW HETEROGENEOUS...?

WHY THE LONG LISTWHY THE LONG LISTNecessityBest tool for the job (agnostic)Don't use a large complicated tool to solve a simple problemLack of effective planning or follow through

OUR MAJOR CHALLENGESOUR MAJOR CHALLENGESCross platform insightAutomatable APIsMonitoringDebt

Are you surprised that orchestration isn't here?

MINOR ISSUESMINOR ISSUESIts not all automatedData normalization

OBTAINING INSIGHTOBTAINING INSIGHTELIMINATE DASHBOARD SPRAWLELIMINATE DASHBOARD SPRAWL

THE TOOLS THAT MADE IT POSSIBLETHE TOOLS THAT MADE IT POSSIBLEElasticSearchLogstashKibanaGrafanaGraphiteDiamondStatsFeederCloudTrail

ALL THE PUPPET CODEALL THE PUPPET CODEpuppet-graphitepuppet-statsfeederpuppet-elasticsearchpuppet-logstashgarethr-diamond

RESULTSRESULTSReplaced dashboards, VMware, EMC, Jenkins, AmazonDiscovered, processed, and presented metrics that were onceimpossible

"AN" API IS NOT ENOUGH"AN" API IS NOT ENOUGH

UNIVERSAL ACCESSIBILITYUNIVERSAL ACCESSIBILITYcapable of being reachedeasy to interact with and understandopen to influence/suggestioncapable of being appreciated

INTERNALLY DEVELOPED TOOLSINTERNALLY DEVELOPED TOOLSvmpoolervmwr

WHAT CAN YOU DO IN A FEW HUNDRED LINES OF CODE AND AWHAT CAN YOU DO IN A FEW HUNDRED LINES OF CODE AND ADAY OF YOUR TIME?DAY OF YOUR TIME?

GETTING OUT OF DEBTGETTING OUT OF DEBT

STANDARIZE AND OPTIMIZESTANDARIZE AND OPTIMIZE

ENDENDCreated by Cody Herriges

PuppetConf '14