practical chef and capistrano for your rails app

Practical Chef and CapistranoFor Your Rails Application

Dan Ivovich

SLS Internal Dec 201212/17/12

Dan Ivovich

http://smartlogicsolutions.com

Twitter - @danivovich

Dan Ivovich on GitHub

SmartLogic Solutions

What is the goal?

● Build a machine that can run the application quickly and repeatedly

● Make deploying the app, upgrading components, adding components, etc seamless and easy

Who does what?● Chef

○ User / SSH Keys○ Web server○ Database○ Postfix○ Redis / Memcached○ Monit○ NewRelic Server monitoring○ /etc/hosts○ rbenv & Ruby○ Application binary dependencies (i.e.

Sphinx)

Who does what?

● Capistrano○ Virtual Hosts○ Unicorn init.d script○ Unicorn.rb○ Monit process monitors○ Normal Capistrano Stuff

Why both?

● Use each for what it is best at● Chef is for infrastructure● Capistrano is for the app● Could have more than one Capistrano app with

the same Chef config● Chef config changes infrequently, Capistrano

config could change more frequently

How? - Chef

● Standard Recipes● Custom Recipes● Recipes assigned to Roles● Roles assigned to Nodes● Nodes with attributes to tailor the install

How? - Capistrano

● Standard Tasks● Custom Tasks● Templating files

Chef - Getting started● Gemfile

● knife kitchen chef-repo○ Create the folder structure you need

● Solo specific stuff (chef-repo/.chef/knife.rb)

● knife cookbook site install nginx○ Get the nginx cookbook and anything it

needs

Custom cookbook● knife cookbook create your_app_custom

● Edit:chef-repo/cookbooks/your_app_custom/recipes/default.rb

package "logrotate"

rbenv_ruby node['your_app']['ruby_version']

rbenv_gem "bundler" do ruby_version node['your_app']['ruby_version']end

template "/etc/hosts" do source "hosts.erb" mode "0644" owner "root" group "root"end

directory "#{node['your_app']['cap_base']}" do action :create owner 'deploy' group 'deploy' mode '0755'end

directory "#{node['your_app']['deploy_to']}/shared" do action :create owner 'deploy' group 'deploy' mode '0755'end

template "#{node['your_app']['deploy_to']}/shared/database.yml" do source 'database.yml.erb' owner 'deploy' group 'deploy' mode '0644'end

Recipe Templates● chef-repo/cookbooks/your_app_custom/templates/default/database.yml.erb

<%= node['your_app']['environment'] %>: adapter: <%= node['your_app']['adapter'] %> database: <%= node['your_app']['database'] %> username: <%= node['your_app']['database_user'] %><% if node['your_app']['database_password'] %> password: <%= node['your_app']['database_password'] %><% end %> host: <%= node['your_app']['database_host'] %> encoding: utf8 min_messages: warning

Node Attributes● Application namespace

○ chef-repo/cookbooks/your_app_custom/attributes/default.rbdefault['your_app']['cap_base'] = '/home/deploy/apps'default['your_app']['deploy_to'] = '/home/deploy/apps/your_app'default['your_app']['environment'] = 'production'default['your_app']['database'] = 'your_app'default['your_app']['adapter'] = 'postgresql'default['your_app']['database_user'] = 'postgres'default['your_app']['database_password'] = (node['postgresql']['password']['postgres'] rescue nil)default['your_app']['database_host'] = 'localhost'default['your_app']['ruby_version'] = '1.9.2-p320'

Node Attributes

"your_app" : { "environment" : "production", "database" : "your_app", "database_user" : "your_app_db_user", "database_host" : "db1", "hosts" : { "db1" : "nn.nn.nn.nn" }},

● For your Node configuration

Define Roles

name "web_server"description "web server setup"run_list [ "recipe[build-essential]", "recipe[annoyances]", "recipe[openssl]", "recipe[openssh]", "recipe[sudo]", "recipe[postgresql::client]", "recipe[users_solo::admins]", "recipe[sphinx]", "recipe[imagemagick]", "recipe[nginx]", "recipe[rbenv]", "recipe[postfix]", "recipe[monit]", "recipe[your_app_custom]"]default_attributes 'build-essential' => { 'compiletime' => true }

chef-repo/roles/web_server.rb

Node Configuration{ "openssh" : { "permit_root_login" : "no", "password_authentication": "no" }, "authorization" : { "sudo" : { "groups" : [ "admin", "sudo" ], "passwordless" : true } }, "rbenv" : { "group_users" : [ "deploy" ] }, "sphinx" : { "use_mysql" : false, "use_postgres" : true }, "your_app" : { "environment" : "production", "database" : "your_app", "database_user" : "your_app_db_user", "database_host" : "db1", "hosts" : { "db1" : "nn.nn.nn.nn" } }, "run_list": [ "role[web_server]" ]}

Not so bad!

Go!● bundle exec knife bootstrap -x super_user node_name \

--template-file=ubuntu-12.04-lts.erb

● bundle exec knife cook super_user@node_name

● Relax!

Capistrano - Getting Started

● Add capistrano and capistrano-ext● Capify● deploy.rb

Capistrano - deploy.rbrequire 'bundler/capistrano'require 'capistrano/ext/multistage'

load 'config/recipes/base'load 'config/recipes/nginx'load 'config/recipes/unicorn'load 'config/recipes/monit'

set :default_environment, { 'PATH' => "/opt/rbenv/shims:/opt/rbenv/bin:$PATH", 'RBENV_ROOT' => "/opt/rbenv"}set :bundle_flags, "--deployment --quiet --binstubs --shebang ruby-local-exec"set :use_sudo, falseset :application, 'your_app'set :repository, 'git@github.com:you/your_app.git'set :deploy_to, '/home/deploy/apps/your_app'set :deploy_via, :remote_cache

Capistrano - deploy.rbset :branch, 'master'set :scm, :gitset :target_os, :ubuntuset :maintenance_template_path, File.expand_path("../recipes/templates/maintenance.html.erb", __FILE__)

default_run_options[:pty] = truessh_options[:forward_agent] = true

namespace :custom do desc 'Create the .rbenv-version file' task :rbenv_version, :roles => :app do run "cd #{release_path} && rbenv local 1.9.2-p320" endend

before 'bundle:install', 'custom:rbenv_version'

Capistrano - recipes/base.rb

def template(from, to) erb = File.read(File.expand_path("../templates/#{from}", __FILE__)) put ERB.new(erb).result(binding), toend

def set_default(name, *args, &block) set(name, *args, &block) unless exists?(name)end

Capistrano - recipes/monit.rbset_default(:alert_email, "dan@smartlogicsolutions.com")namespace :monit do desc "Setup all Monit configuration" task :setup do unicorn syntax restart end after "deploy:setup", "monit:setup"

task(:unicorn, roles: :app) { monit_config "unicorn" }

%w[start stop restart syntax].each do |command| desc "Run Monit #{command} script" task command do with_user "deploy" do sudo "service monit #{command}" end end endend

Capistrano - recipes/monit.rb

def monit_config(name, destination = nil) destination ||= "/etc/monit/conf.d/#{name}.conf" template "monit/#{name}.erb", "/tmp/monit_#{name}" with_user "deploy" do sudo "mv /tmp/monit_#{name} #{destination}" sudo "chown root #{destination}" sudo "chmod 600 #{destination}" endend

Capistrano - recipes/nginx.rb

namespace :nginx do desc "Setup nginx configuration for this application" task :setup, roles: :web do template "nginx_unicorn.erb", "/tmp/nginx_conf" sudo "mv /tmp/nginx_conf /etc/nginx/sites-enabled/#{application}" sudo "rm -f /etc/nginx/sites-enabled/default" restart end after "deploy:setup", "nginx:setup"

%w[start stop restart].each do |command| desc "#{command} nginx" task command, roles: :web do sudo "service nginx #{command}" end endend

Capistrano - templates/nginx_unicorn.erbupstream unicorn { server unix:/tmp/unicorn.<%= application %>.sock fail_timeout=0;}server { listen 80 default deferred; server_name your_app_domain.com; root <%= current_path %>/public; if (-f $document_root/system/maintenance.html) { return 503; } error_page 503 @maintenance; location @maintenance { rewrite ^(.*)$ /system/maintenance.html last; break; } location ^~ /assets/ { gzip_static on; expires max; add_header Cache-Control public; } try_files $uri/index.html $uri @unicorn; location @unicorn { proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-Proto $scheme; proxy_redirect off; proxy_pass http://unicorn; } error_page 500 502 /500.html; error_page 504 /504.html; client_max_body_size 4G; keepalive_timeout 10; server_tokens off;}

Capistrano - recipes/unicorn.rb

set_default(:unicorn_user) { user }set_default(:unicorn_pid) { "#{current_path}/tmp/pids/unicorn.pid" }set_default(:unicorn_config) { "#{shared_path}/config/unicorn.rb" }set_default(:unicorn_log) { "#{shared_path}/log/unicorn.log" }set_default(:unicorn_workers) { if rails_env == "production" 10 else 3 end}set_default(:unicorn_timeout, 30)

Capistrano - recipes/unicorn.rbnamespace :unicorn do desc "Setup Unicorn initializer and app configuration" task :setup, roles: :app do run "mkdir -p #{shared_path}/config" template "unicorn.rb.erb", unicorn_config template "unicorn_init.erb", "/tmp/unicorn_init" run "chmod +x /tmp/unicorn_init" sudo "mv /tmp/unicorn_init /etc/init.d/unicorn_#{application}" sudo "update-rc.d -f unicorn_#{application} defaults" end after "deploy:setup", "unicorn:setup"

%w[start stop restart].each do |command| desc "#{command} unicorn" task command, roles: :app do sudo "service unicorn_#{application} #{command}" end after "deploy:#{command}", "unicorn:#{command}" endend

Capistrano - templates/unicorn.rb.erb

root = "<%= current_path %>"working_directory rootpid "#{root}/tmp/pids/unicorn.pid"stderr_path "#{root}/log/unicorn.log"stdout_path "#{root}/log/unicorn.log"listen "/tmp/unicorn.<%= application %>.sock"worker_processes <%= unicorn_workers %>timeout <%= unicorn_timeout %>preload_app true

before_exec { |server| ENV['BUNDLE_GEMFILE'] = "#{root}/Gemfile" }

Capistrano - templates/unicorn.rb.erb

before_fork do |server, worker| # Disconnect since the database connection will not carry over if defined? ActiveRecord::Base ActiveRecord::Base.connection.disconnect! end # Quit the old unicorn process old_pid = "#{server.config[:pid]}.oldbin" if File.exists?(old_pid) && server.pid != old_pid begin Process.kill("QUIT", File.read(old_pid).to_i) rescue Errno::ENOENT, Errno::ESRCH # someone else did our job for us end endend

Capistrano - templates/unicorn.rb.erb

after_fork do |server, worker| # Start up the database connection again in the worker if defined?(ActiveRecord::Base) ActiveRecord::Base.establish_connection end child_pid = server.config[:pid].sub(".pid", ".#{worker.nr}.pid") system("echo #{Process.pid} > #{child_pid}")end

Capistrano - t/monit/unicorn.erb

check process <%= application %>_unicorn with pidfile <%= unicorn_pid %> start program = "/etc/init.d/unicorn_<%= application %> start" stop program = "/etc/init.d/unicorn_<%= application %> stop"

<% unicorn_workers.times do |n| %> <% pid = unicorn_pid.sub(".pid", ".#{n}.pid") %> check process <%= application %>_unicorn_worker_<%= n %> with pidfile <%= pid %> start program = "/bin/true" stop program = "/usr/bin/test -s <%= pid %> && /bin/kill -QUIT `cat <%= pid %>`" if mem > 200.0 MB for 1 cycles then restart if cpu > 50% for 3 cycles then restart if 5 restarts within 5 cycles then timeout alert <%= alert_email %> only on { pid } if changed pid 2 times within 60 cycles then alert<% end %>

Whoa!

But really, it is just a bunch of Erb for files you

already have

Did you see the trick?

● after "deploy:setup", "nginx:setup"

So we can...

● cap staging deploy:setup deploy:migrations

From the top!

Ready?!? Here we go!

1. New VM at my_web_app in your .ssh/config2. Create chef-repo/nodes/my_web_app.json3. In chef-repo:

bundle exec knife bootstrap node_name \ --template-file=ubuntu-12.04-lts.erb

4. bundle exec knife cook root@my_web_app5. In app directory:

create/edit config/deploy/staging.rb6. cap staging deploy:setup deploy:migrations7. Hit the bars

Thoughts....● Vagrant and VMs are you friend. Rinse and repeat

● It is ok to tweak your Chef stuff and re-cook, but I always

like to restart with a fresh VM once I think I'm done

● Capistrano tweaks should be easy to apply, especially with

tasks like nginx:setup, unicorn:setup etc.

● Chef issues are harder to debug and more frustrating than

Capistrano issues, another reason to put more app specific

custom stuff in Capistrano and do standard things in Chef

Questions?

http://smartlogicsolutions.com

http://twitter.com/smartlogic

http://github.com/smartlogic http://fb.me/smartlogic