A Supermarket of Your Own: Running a Private Supermarket

Nell Shamrell-Harrington

Who am I?

Nell Shamrell-Harrington• Software Development Engineer at Chef• Twitter: @nellshamrell• Email: nshamrell@chef.io

Why Private Supermarket?

Supermarket – in the beginning…• Started as the public community cookbooks site• Can browse cookbooks from anywhere• People really liked it!

However…• Some companies create very company-specific cookbooks• And cannot share those cookbooks publicly• Still want to share internally

Before Private Supermarket• Store Cookbooks on Github Enterprise

PROBLEM: Hard to know exactly what to search for

• Use Berkshelf with Github Enterprise PROBLEM: Resolving dependencies directly from Github repos is slow

Private Supermarket – in the beginning…• Chef Summit 2014 - Customers requested ability to run their own copy

of Supermarket• Hadn’t thought of this before…but ready to try it• Build a stand alone Supermarket package using Chef Omnibus

Packager https://github.com/chef/omnibus

Benefits of Private Supermarket• Easily searchable cookbook repository that can on a company’s

internal network• Friendly GUI and command line interfaces• Supermarket API simplifies and speeds up Berkshelf runs• Easier to formalize process of release cookbooks

Sounds Great…How Do I Get My Own Private

Supermarket?

Installing Private Supermarket

Requirements• Running Chef Server – need for oc-id (Chef Oauth Provider)

Requirements• Running Chef Server – need for oc-id (Chef Oauth Provider)• Admin User Account on Chef Server

Requirements• Running Chef Server – need for oc-id (Chef Oauth Provider)• Admin User Account on Chef Server• Key for User Account on Chef Server

Requirements• Running Chef Server – need for oc-id (Chef Oauth Provider)• Admin User Account on Chef Server• Key for User Account on Chef Server• Fresh VM with 1 GB Memory (we use an Ubuntu instance on AWS)

What is oc-id?

Chef Server

oc-id• Chef’s Authentication/Authorization

Service• Use credentials for Chef Server

users to access other applications• Uses OAuth2 to talk to any

applications authorized by Chef Server

What is oc-id?

Chef Server

oc-id

Application

What is oc-id?

Chef Server

oc-id

Application

User clicks “Sign In”

What is oc-id?

Chef Server

oc-id

Application

User redirected to Chef Server

What is oc-id?

Chef Server

oc-id

Application

User signs into Chef Server

What is oc-id?

Chef Server

oc-id

Application

User is redirected to Supermarket, now signed in!

Preparing Your Chef Server• SSH into your Chef Server

$ (your workstation) ssh user@your-chef-server

Preparing Your Chef Server• Open up /etc/opscode/chef-server.rb

$ (your chef server) sudo vim /etc/opscode/chef-server.rb

Preparing Your Chef Server• Add this content, then save and exit the file

oc_id[‘applications’] = { ‘supermarket’ => { ‘redirect_uri’ => ‘https://supermarket-hostname/auth/chef_oauth2/callback’ }}

Preparing Your Chef Server• Reconfigure your Chef Server

$ (your chef server) sudo chef-server-ctl reconfigure

Preparing Your Chef Server• Look at /etc/opscode/oc-id-applications/supermarket.json – will need

these values later$ (your chef server) sudo cat /etc/opscode/oc-id-applications/supermarket.json{ “name”: “supermarket”, “uid”: “123……989” “secret”: “1234…897” “redirect uri”: “https://supermarket-hostname/auth/chef_oauth2/callback”}

Preparing Your Supermarket• Will be using Irving Popovetsky’s supermarket-omnibus-cookbook

https://github.com/irvingpop/supermarket-omnibus-cookbook• There IS another community cookbook, but uses an outdated method for installing

Supermarket and is (as of 8/17) no longer supported by Chef• A blog post on converting from the old community cookbook to the new Supermarket cookbook

is coming soon!

• Good practice – using a wrapper cookbook with Irving’s cookbook• Will need to supply certain attributes through wrapper cookbook

Preparing Your Supermarket

Preparing Your Supermarket

Wrapper Cookbook

• Defines node[supemarket_omnibus] attributes

Preparing Your Supermarket

Wrapper Cookbook

OmnibusSupermarket

Cookbook

• Defines node[supemarket_omnibus] attributes

• Installs supermarket omnibus package

• Writes node[supermarket_omnibus] attributes to

/etc/supermarket/supermarket.json

Preparing Your Supermarket

Wrapper Cookbook

OmnibusSupermarket

Cookbook

OmnibusSupermarket

Package

• Has internal Chef cookbook to run

• Configures package using attributes defined in:

/etc/supermarket/supermarket.json• supermarket-ctl reconfigure• Like running chef solo

• Defines node[supemarket_omnibus] attributes

• Installs supermarket omnibus package

• Writes node[supermarket_omnibus] attributes to

/etc/supermarket/supermarket.json

Preparing Your Supermarket• Create wrapper cookbook on your local workstation

$ (your workstation) chef generate cookbook my-supermarket-wrapper $ (your workstation) cd my-supermarket-wrapper

Preparing Your Supermarket• Open your cookbook’s metadata.rb

$ (your workstation) vim metadata.rb

Preparing Your Supermarket• Add dependency on the supermarket-omnibus-cookbook to

metadata.rb. Save and close file.depends ‘supermarket-omnibus-cookbook’

Preparing Your Supermarket• Open your cookbook’s default recipe at recipes/default.rb

$ (your workstation) vim recipes/default.rb

Preparing Your Supermarket• Include the default recipe of the supermarket-omnibus-cookbook

include_recipe ‘supermarket-omnibus-cookbook’

Preparing Your Supermarket• Next, need to define attributes for your Supermarket and how it

connects to Chef Server• Can hard code attributes in your wrapper cookbook’s default recipe • Better practice – place attributes in a databag, then reference it in

your recipe• At minimum – must define

• chef_server_url• chef_oauth2_app_id• chef_oauth2_secret

Preparing Your Supermarket• Add attributes to wrapper cookbook’s default recipe

# calling the data bagapp = data_bag_item(‘apps’, ‘supermarket’)

Preparing Your Supermarket• Add attributes to wrapper cookbook’s default recipe, save and close.

# calling the data bagapp = data_bag_item(‘apps’, ‘supermarket’)

node.set[‘supermarket_omnibus’][‘chef_server_url’] = app[‘chef_server_url’]node.set['supermarket_omnibus']['chef_oauth2_app_id'] = app[‘app_id’]node.set['supermarket_omnibus']['chef_oauth2_secret'] = app[‘secret’]

Preparing Your Supermarket• Download dependent cookbooks using Berkshelf

$ (your workstation) berks install

Preparing Your Supermarket• Then upload all dependent cookbooks to your Chef Server

$ (your workstation) cd ~/.berkshelf/cookbooks$ (your workstation) knife cookbook upload -a

Preparing Your Supermarket• Now upload your wrapper cookbook

$ (your workstation) cd path/to/wrapper/cookbook/..$ (your workstation) knife cookbook upload -a

Preparing Your Supermarket• Bootstrap and register your Supermarket node with your Chef Server

# For an Ubuntu node on AWS$ (your workstation) knife bootstrap ip_address –N supermarket-node –x ubuntu --sudo

Preparing Your Supermarket• Bootstrap and register your Supermarket node with your Chef Server

# For an Ubuntu node on AWS$ (your workstation) knife bootstrap ip_address –N supermarket-node –x ubuntu –-sudo# -N flag defines name of node

Preparing Your Supermarket• Bootstrap and register your Supermarket node with your Chef Server

# For an Ubuntu node on AWS$ (your workstation) knife bootstrap ip_address –N supermarket-node –x ubuntu –sudo# -N flag defines name of node# -x flag defines username to use (default on AWS is ubuntu)

Preparing Your Supermarket• Bootstrap and register your Supermarket node with your Chef Server

# For an Ubuntu node on AWS$ (your workstation) knife bootstrap ip_address –N supermarket-node –x ubuntu –sudo# -N flag defines name of node# -x flag defines username to use (default on AWS is ubuntu)# --sudo runs bootstrap command as sudo on node

Preparing Your Supermarket• Edit your Supermarket node

$ (your workstation) knife node edit supermarket-node

Preparing Your Supermarket• Add wrapper cookbook’s default recipe to Supermarket node’s run list,

save and quit file"run_list": [ “recipe[my_supermarket_wrapper::default]” ]

Preparing Your Supermarket• SSH into your Supermarket node

# For an Ubuntu instance on AWS$ (your workstation) ssh ubuntu@your-supermarket-node-public-dns

Preparing Your Supermarket• Run Chef Client

$ (your-supermarket-node) sudo chef-client

Using Private Supermarket

Using Private Supermarket• Open up /etc/hosts on your local workstation

$ (your workstation) vim /etc/hosts

Using Private Supermarket• Add an entry for your Supermarket instance

# Supermarket ip address Supermarket hostname00.00.000.000 supermarket-hostname

Using Private Supermarket• Visit Supermarket hostname in browser (accept SSL certificate)• Click the “Create Account” link

Using Private Supermarket• Log into your Chef Server

Using Private Supermarket• Authorize the app to use your Chef Server account

Using Private Supermarket• And you’re in!

Using Private Supermarket• Knife Supermarket Plugin

Best current way to interact with a private Supermarket https://github.com/chef/knife-supermarket

Using Private Supermarket• Install knife-supermarket

# If using the Chef DK$ chef gem install knife-supermarket

# If not using the Chef DK$ gem install knife-supermarket

Using Private Supermarket• Open up knife.rb

$ vim .chef/knife.rb

Using Private Supermarket• Define supermarket site in knife.rb, then save and close.

knife[:supermarket_site] = ‘https://your-private-supermarket’

Using Private Supermarket• knife supermarket commands• same as knife cookbook site commands• see https://docs.chef.io/knife_cookbook_site.html

Using Private Supermarket• Using knife supermarket share

$ (your-workstation) knife supermarket share ‘my_cookbook’

Using Private Supermarket• When you first use this command, might see an SSL error

$ (your-workstation) knife supermarket share ‘my_cookbook’ Making tarball my_cookbook.tgzERROR: Error uploading cookbook my_cookbook to the Opscode Cookbook Site: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed. Increase log verbosity (-VV) for more information.

Using Private Supermarket• By default, Chef 12 enforces ssl when sharing cookbooks (Chef 11 did

NOT do this)• Private Supermarkets (by default) use self-signed certificates• When using chef-client 12.4 or above, can fix error through fetching

Supermarket’s ssl certificate• When using < chef-client 12.4, must set knife[:ssl_verification]

= :verify_none

Using Private Supermarket• When using >= Chef 12.4, fetch the ssl certificate of your Private

Supermarket$ (your workstation) knife ssl fetch https://your-private-supermarket

Using Private Supermarket• When using >= Chef 12.4, fetch the ssl certificate of your Private

Supermarket$ (your workstation) knife ssl fetch https://your-private-supermarketWARNING: Certificates from your-private-supermarket will be fetched and placed in your trusted_certdirectory (/home/ubuntu/.chef/trusted_certs).

Knife has no means to verify these are the correct certificates. You shouldverify the authenticity of these certificates after downloading.

Using Private Supermarket• Then check the ssl certificate

$ (your-workstation) knife ssl check https://your-private-supermarket

Using Private Supermarket• Then check the ssl certificate

$ (your-workstation) knife ssl check https://your-private-supermarket[Connecting to host your-private-supermarket:443Successfully verified certificates from `your-private-supermarket'

Using Private Supermarket• Now try sharing the cookbook again

$ (your-workstation) knife supermarket share ‘my_cookbook’

Using Private Supermarket• Success!

$ (your workstation) knife supermarket share ‘my_cookbook’Generating metadata for my_cookbook from /tmp/chef-my_cookbook-build20150722-14976-8e8bdz/my_cookbook/metadata.rbMaking tarball my_cookbook.tgzUpload complete!

Managing Private Supermarket

Managing Private Supermarket• You can manage services and other settings of your instance with

supermarket-ctl

Managing Private Supermarket• You can manage services and other settings of your instance with

supermarket-ctl• Examples:

supermarket-ctl reconfigure - does an internal Chef run based on the cookbooks included

within the omnibus package supermarket-ctl make-admin user - makes a supermarket user an admin user

supermarket-ctl restart - stops services if they are running, then starts them again

Managing Private Supermarket• Restarting services on your Supermarket instance

$ (your-supermarket-node) sudo supermarket-ctl restartok: run: nginx: (pid 16737) 1sok: run: postgresql: (pid 16742) 0sok: run: rails: (pid 16751) 1sok: run: redis: (pid 16755) 0sok: run: sidekiq: (pid 16760) 0s

Managing Private Supermarket• Monitoring URL: https://your_private_supermarket/status

Other Tools You Can Use With Private Supermarket

Stovehttps://github.com/sethvargo/stove

Berkshelf Integration

Berksfile

source 'https://supermarket.yourcompany.com' source 'https://supermarket.chef.io' ...

Dependency Resolution

• Multiple sources can be defined in a Berksfile• Cookbook dependency resolution is performed from the top down. i.e. the source defined first in the Berksfile will be searched for a cookbook version before the second source and so on.

Alternative Datastores / HA

Public Supermarket HA configuration

Why?

• Make better use of Amazon's services RDS, Elasticache and S3• Make your company’s existing infrastructure better• Handle a heavy traffic load spike• Improve uptime

Load Balancing

Load Balancer Health Checkshttp://app1-staging-supermarket.internal/status

{ "status":"ok", ...}

External Database

Postgres / RDS

node.set['supermarket_omnibus']['postgresql']['enable'] = falsenode.set['supermarket_omnibus']['database']['user'] = 'supermarket'node.set['supermarket_omnibus']['database']['name'] = 'supermarket'node.set['supermarket_omnibus']['database']['host'] = 'yourcompany...rds.amazon.com'node.set['supermarket_omnibus']['database']['port'] = '5432'node.set['supermarket_omnibus']['database']['pool'] = '25'node.set['supermarket_omnibus']['database']['password'] = 'topsecretneverguessit'

Cache

Cache Configuration

node.set['supermarket_omnibus']['redis']['enable'] = false node.set['supermarket_omnibus']['redis_url'] = 'redis://supermarket.yourcompany.use1.cache.amazonaws.com:6379'

Storage

Cookbook Storage Methods

• Local Filesystem• S3

S3 Configuration

node.set['supermarket_omnibus']['s3_access_key_id'] = 'XYZXYZXYZXYZXYZXYZ'node.set['supermarket_omnibus']['s3_bucket'] = 'supermarket'node.set['supermarket_omnibus']['s3_secret_access_key'] = 'wXYzwXYzwXYzwXYzwXYz'

Additional Configuration Options

• https://github.com/chef/omnibus-supermarket/blob/master/cookbooks/omnibus-supermarket/attributes/default.rb