©2015,  Amazon  Web  Services,  Inc.  or  its  affiliates.   All  rights  reserved

Shiva  NSolutions  Architect,  AWS

@shivamaan

AWS  +  Puppet  =  Dynamic  ScalePuppetCamp,  Melbourne  2015

Application  Rate  of  Change  Is  Increasing

Source  2013  State  of  DevOps  Report  Puppet  Labs,  IT  Revolution  Press

Top  Performers30x  more  frequent  code  deployments

More  than  25%  can  deploy on  demand

~11.6sMean  timebetween  deployments  (weekday)

~1,079Max  number  of  deployments  in  a  single  hour

~10,000Mean  number  of  hosts  simultaneously  receiving  a  deployment  

~30,000Max  number  of  hosts  simultaneously  receiving  a  deployment  

DEPLOYMENTS  AT  AMAZON.COM

75%Reduction  in  outages  triggered  by  software  deployments  since  2006

90%Reduction  in  outage  minutes  triggered  by  software  deployments

Business  Value  of  Frequent  Deployments

~0.001%Software  deployments  cause  anoutage

example.com – Online  auction

AWS Sydney Region

Public subnet App subnetHAProxy Vertx

Glassfish

Data subnet

Puppet  Stack

ELKStackMgmt subnet

Jenkins

Let’s  build  

[  ]  OS  [  ]  Patches[  ]  OS  Configuration[  ]  VM  Templates  (Amazon  Machine  Image)[  ]  VPC[  ]  Networking  (Subnets,  ELBs)[  ]  Network  Security  (NACLs  and  SGs)[  ]  Software[  ]  Application  (Code)[  ]  Services

Start  with  a  Base  Image

Machine  Images  can  come  from  several  sources..- AWS  Provided  (Quick  Start)- AWS  Marketplace- Community  AMIs- Build  your  own  AMIs- Migrate  On  Premise  Image We  need  a  server  to  

build  an  image  to  build  a  server

Let’s  build  

[X]  OS  [  ]  Patches[  ]  OS  Configuration[  ]  VM  Templates  (Amazon  Machine  Image)[  ]  VPC[  ]  Networking  (Subnets,  ELBs)[  ]  Network  Security  (NACLs  and  SGs)[  ]  Software[  ]  Application  (Code)[  ]  Services

Layers  to  Foundational  Image

CI-­‐CD  Pipeline- git push  puppetmanifests- Packer/AMINator- puppet  apply- Test  image- Register  AMI,  meta-­‐data  in  CMDB

Let’s  build  

[X]  OS  [X]  Patches[X]  OS  Configuration[X]  VM  Templates  (Amazon  Machine  Image)[  ]  VPC[  ]  Networking  (Subnets,  ELBs)[  ]  Network  Security  (NACLs  and  SGs)[  ]  Software[  ]  Application  (Code)[  ]  Services

Cloudformation

Let’s  build  

[X]  OS  [X]  Patches[X]  OS  Configuration[X]  VM  Templates  (Amazon  Machine  Image)[X]  VPC[X]  Networking  (Subnets,  ELBs)[X]  Network  Security  (NACLs  and  SGs)[  ]  Software[  ]  Application  (Code)[  ]  Services

Build  Puppet  Stack

- EC2  Metadata  service  (or  Resource  Tags)$  aws s3  cp s3://puppetmanifests /etc/puppet/modules   (or  git clone)$  curl  http://169.254.169.254/latest/user-­‐data…hiera_role=PuppetMaster

- Self  awareness  by  facter/hiera magic- Self  manifestation  with  puppet  apply

Build  Software  Stack

- CloudFormation- RDS- Elasticache- S3- CloudFront- Route  53- ELB- SNS- Autoscaling groups  of  EC2  Instances

- PuppetMaster- Build  ELK  Stack  (PuppetForge)- Build  Glassfish  layer  (Custom  types  and  providers)- Build  Vertx layer  (Custom  types  and  providers)- Build  HAProxy layer  (PuppetForge)

Let’s  build  

[X]  OS  [X]  Patches[X]  OS  Configuration[X]  VM  Templates  (Amazon  Machine  Image)[X]  VPC[X]  Networking  (Subnets,  ELBs)[X]  Network  Security  (NACLs  and  SGs)[X]  Software[  ]  Application  (Code)[  ]  Services

Application  Deployment

define  glassfish::app  ($user   =  'glassfish',$action =  'deploy',$bindaddr =  '127.0.0.1',$mgmtport =  '8080',$deploy_source =  '${glassfish::real_instance_basedir/deployments/${name}.war',)  {require glassfishif ($action ==  'deploy')  {app {  "${name}":ensure   =>  present,mgmtport =>  $mgmtport,user   =>  $user,source   =>  $deploy_source,

}}

}

Let’s  build  

[X]  OS  [X]  Patches[X]  OS  Configuration[X]  VM  Templates  (Amazon  Machine  Image)[X]  VPC[X]  Networking  (Subnets,  ELBs)[X]  Network  Security  (NACLs  and  SGs)[X]  Software[X]  Application  (Code)[X]  Services

Let’s  run

[  ]  Tune[  ]  Release[ ]  Orchestrate  and  Automate

Decision,  Decisions…

- In  place  deployments  (Puppet  run)- MCollective- CodeDeploy

- Immutable  deployments  - Blue-­‐Green  deployment- AMI  baking

AWS  CodeDeploy

Bake  an  AMI Puppet  +

Sweet  spot  

To  AMI  or  not  to  AMI

Start  an  instance

Configure  the  instance

Create  an  AMI  from  your  instance

Start  new  ones  from  the  AMI

Launch  an  instance

Use  metadata  service  to  get  hiera roles

Puppet  run  performs  post  launch  config

The  Butler  orchestrates

*  Golden  AMI  – Glassfish  +  HAProxy +  Vertx +  Applications  with  all  services  turned  off

Minor  Release

Major  Release

+

+

CodeDeploy Autoscaling group ScheduledAutoscalinggroup

Packer AMI* DynamoDB

Autoscalinggroup

Packer AMI* DynamoDB

Let’s  run

[X]  Tune[X]  Release[X]  Orchestrate  and  Automate

example.com – Online  auction

AWS Singapore Region

Public subnet App subnetHAProxy Vertx

Glassfish

Data subnet

Puppet  Stack

ELKStackMgmt subnet

Jenkins

Thank  You