Automating That "Other" OSCooking with Chef on Windows

Julian Dunn <jdunn@getchef.com>

Engineering Lead – Field Solutions

Chef Software, Inc.

CloudDevelop Conference

Columbus, OH

October 2014

Doesn't scale

Jeffrey Snover and the Monad Manifesto

2002

Evolution of Automation on Windows

1996: WMI

1999:Services for UNIX

2003: Subsystem for Unix Applications

2006:PowerShell 1.0

2008:PowerShell 2.0&Server Core

2012:PowerShell 3.0

2013:PowerShell 4.0/Desired State Configuration

2014:PowerShell 5.0(Preview)

2002:MonadManifesto

The GUI isn't dead, but it's dying on the server OS. Don't be caught off-

guard when it's finally gone.- Don Jones, Redmond Magazine, July 2011

http://redmondmag.com/articles/2011/07/01/the-gui-is-dead.aspx

Declarative Configuration Management

Model your IT by describing what it should look like -- not how it should

be done.

Imperative versus Declarative

Imperative:

PS C:\> Add-WindowsFeature Web-WebServer

Declarative:

Configuration MySite {    WindowsFeature IIS {          Ensure          = "Present"          Name            = "Web-Server"      }

}

Platform-Neutral Domain-Specific Language

package 'httpd'

template '/etc/httpd/conf/httpd.conf' do

owner 'root'

group 'root'

action :create

source 'httpd.conf.erb'

notifies :reload, 'service[httpd]'

End

service 'httpd' do

action [:start, :enable]

end

windows_feature 'IIS-WebServerRole'

template 'c:\inetpub\wwwroot\index.html' do

owner 'Administrator'

group 'IIS_IUSRS'

action :create

source 'index.html.erb'

notifies :reload, 'service[W3SVC]'

End

service 'W3SVC' do

action [:start, :enable]

end

Chef Mechanics

• Recipes go in cookbooks• Cookbooks are uploaded to a Chef

server• Nodes periodically check in and

get their recipes to run ("run list")• If system state is already desired

state, Chef makes no changes

• "Convergence"

What's the Purpose of Declarative CM?

• Consistent, reproducible configurations• Manage & deploy thousands of machines correctly• Deploy applications correctly• Keep them in compliance with declared policy

Infrastructure Applications Service

+ =

Wait, What About System Center?

Microsoft System Center

• Advisor• App Controller• Configuration Manager (SCCM)• Data Protection Manager• Endpoint Protection• Orchestrator/SMA• Operations Manager• Service Manager• Virtual Machine Manager

System Center Configuration Manager

• Origin: Started as Systems Management Server (1994), renamed SCCM in ~2007

• Purpose: Manage large groups of computers running Windows, Windows Embedded, Mac OS X, and/or Linux/UNIX

• Components:• Remote control

• Patch management

• Software distribution

• OS deployment using MDT

• Hardware/software inventory

• System configuration

SCCM: The Good, Bad and the Ugly

• Good:• Easy-to-use UI

• Lots of functionality

• Great for managing desktops

• Integrates with other System Center products

• Bad:• Prescriptive workflow

• Point-and-click

• Needs Active Directory

• Hard to automate the automation

• No easily versionable artifacts

SCCM and Chef

• SCCM (Compliance Settings)• Configuration settings set via UI

• Configuration item primitives• WMI, registry, scripts, applications

• Shareable artifacts (baselines)

• Restrictive workflow

• Idempotence is up to you

• Agent-based

• Chef:• Configuration settings via plain text files

• Resource primitives• file, template, service, powershell_script, etc.

• Shareable & versionable artifacts (cookbooks)

• Flexible workflow

• Built-in idempotence

• Agent-based

Demo

Provisioning with Chef on Microsoft Azure

1. Upload content (cookbooks, roles, etc.)

2. Request VM

3. Create VM, install Azure and Chef agents

4. Register with Chef server

5. Execute run_list

Provisioning with Chef

$ knife azure server create

--azure-source-image a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-R2-201409.01-en.us-127GB.vhd

--bootstrap-protocol cloud-api

--winrm-user chef

--winrm-password DELETED

--azure-dns-name DELETED

-r "role[base-windows], role[fourthcoffee-classic]"

...........

Waiting for virtual machine to reach status 'provisioning'............vm state 'provisioning' reached after 2.6 minutes.

Waiting for virtual machine to reach status 'ready'..........................vm state 'ready' reached after 6.23 minutes.

.

DNS Name: DELETED.cloudapp.net

VM Name: DELETED

Size: Medium

Azure Source Image: a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-R2-201409.01-en.us-127GB.vhd

Azure Service Location: East US

Public Ip Address: XXXXXXXX

Private Ip Address: YYYYYYYY

WinRM Port: 5985

Environment: _default

Provisioning with ChefWaiting for Resource Extension to reach status 'wagent provisioning'....

Resource extension state 'wagent provisioning' reached after 0.03 minutes.

Waiting for Resource Extension to reach status 'installing'....................

Resource extension state 'installing' reached after 2.17 minutes.

Waiting for Resource Extension to reach status 'provisioning'....................................

Resource extension state 'provisioning' reached after 4.33 minutes.

Waiting for Resource Extension to reach status 'ready'....................

Resource extension state 'ready' reached after 2.16 minutes.

.

DNS Name: DELETED.cloudapp.net

VM Name: DELETED

Size: Medium

Azure Source Image: a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-R2-201409.01-en.us-127GB.vhd

Azure Service Location: East US

Public Ip Address: XXXXXX

Private Ip Address: YYYYYY

WinRM Port: 5985

Environment: _default

Runlist: ["role[base-windows]", "role[fourthcoffee-classic]"]

Welcome to Fourth Coffee Corporation of Seattle

Pay no attention to the man behind the curtain

windows_feature 'IIS-WebServerRole' do

action :install

end

# Pre-requisite features for IIS-ASPNET45 that need to be installed first, in this order.

%w{IIS-ISAPIFilter IIS-ISAPIExtensions NetFx3ServerFeatures NetFx4Extended-ASPNET45 IIS-NetFxExtensibility45}.each do |f|

windows_feature f do

action :install

end

end

windows_feature 'IIS-ASPNET45' do

action :install

end

Pay no attention to the man behind the curtainremote_directory node['fourthcoffee']['install_path'] do

source 'fourthcoffee'

action :create

end

iis_pool 'FourthCoffee' do

runtime_version '4.0'

action :add

end

iis_site 'FourthCoffee' do

protocol :http

port 80

path node['fourthcoffee']['install_path']

application_pool 'FourthCoffee'

action [:add,:start]

end

Challenges to Automation on Windows

• No real package manager• Many COTS vendors don’t understand automation• UAC (User Access Control)• WinRM Quotas• Win32 Redirector• Not all preferences/state stored in registry• Reboots!• Other annoyances (KB2773898, KB2918614, KB2842230)

Desired State ConfigurationThe Future of Automation on Windows

PowerShell DSC: The Future of Automation

"DSC represents a significant break in administration, because it asks … administrators to not actually configure anything themselves. Instead, DSC asks administrators to describe, in fairly simple text files, how they would like a computer to be configured. The computer, in turn, reads that text file, and configures itself accordingly."

- The DSC Book, Don Jones & Steve Murawski

The Relationship between DSC and Chef

• As PerfMon is to Solarwinds, DSC is to Chef• DSC provides automation primitives that Chef recipes can call• It deliberately lacks the ecosystem:

• Content distribution

• Cross-platform support

• Monitoring/logging/analytics

• However, it brings a standard base for automation to Windows• No MSFT product in the future may ship without DSC modules!

Example DSC Code

Configuration FourthCoffee {

    # Install the IIS role     WindowsFeature IIS      {          Ensure          = "Present"          Name            = "Web-Server"      }        # Install the ASP .NET 4.5 role     WindowsFeature AspNet45      {          Ensure          = "Present"          Name            = "Web-Asp-Net45"      }

...

}

DSC Invoked from Chef

Configuration FourthCoffee {

    # Install the IIS role     WindowsFeature IIS      {          Ensure          = "Present"          Name            = "Web-Server"      }        # Install the ASP .NET 4.5 role     WindowsFeature AspNet45      {          Ensure          = "Present"          Name            = "Web-Asp-Net45"      }

...

}

dsc_resource 'webserver' do

resource_name :windowsfeature

property :name, 'Web-Server'

property :ensure, 'Present'

end

dsc_resource 'dotnet45' do

resource_name :windowsfeature

property :name, 'Web-Asp-Net45'

property :ensure, 'Present'

end

Testing Infrastructure Code

DevOps is a Two-Way Street

• It's great when developers care about:• Uptime!

• Scaling!

• Deployment!

• Argh! Put them on call! That'll teach them!

DevOps is a Two-Way Street

• Sysadmins/infracoders have a lot to learn from developers as well!

• Good developers:• Write unit tests

• Write acceptance tests

• Practice test-driven-development

• Build confidence that their program code works correctly

• Avoid breaking their applications

• Good infracoders:• Do all of the above

• Avoid breaking ALL THE THINGS

Testing on the desktop

• Chef Ecosystem Tools:• Test Kitchen

• Acceptance testing (ServerSpec)

• Bring-your-own hypervisor (VirtualBox, VMWare Fusion/Workstation, Hyper-V…) and/or middleware (Vagrant)

• Demo

Example Test Suite

describe windows_feature('IIS-WebServer') do it { should be_installed }end

describe port(80) do it { should be_listening }end

describe file('C:\inetpub\FourthCoffee\Default.cshtml') do it { should be_file }end

Test Kitchen Demo

fourthcoffee ~$ kitchen test default-windows-2012R2 --destroy=never

-----> Starting Kitchen (v1.3.0)

-----> Cleaning up any prior instances of <default-windows-2012R2>

-----> Testing <default-windows-2012R2>

-----> Creating <default-windows-2012R2>...

Bringing machine 'default' up with 'virtualbox' provider...

==> default: Importing base box 'win2012r2-datacenter-chef11.16.2'...

Vagrant instance <default-windows-2012R2> created.

Finished creating <default-windows-2012R2> (2m57.54s).

-----> Converging <default-windows-2012R2>...

-----> Chef Omnibus installation detected (true)

Transferring files to <default-windows-2012R2>

Concurrent threads set to :max_threads => 2

[2014-10-13T19:16:36-07:00] INFO: Starting chef-zero on host localhost, port 8889 with repository at repository at C:/tmp/kitchen

One version per cookbook

[2014-10-13T19:16:40-07:00] INFO: *** Chef 11.16.2 ***

[2014-10-13T19:16:40-07:00] INFO: Chef-client pid: 1656

Test Kitchen Demo

[2014-10-13T19:19:10-07:00] INFO: Chef Run complete in 142.572914 seconds

[2014-10-13T19:19:10-07:00] INFO: Running report handlers

[2014-10-13T19:19:10-07:00] INFO: Report handlers complete

Finished converging <default-windows-2012R2> (22m55.08s).

-----> Setting up <default-windows-2012R2>...

-----> Running postinstall for serverspec plugin

Finished setting up <default-windows-2012R2> (0m45.62s).

-----> Verifying <default-windows-2012R2>...

-----> Running serverspec test suite

Windows feature "IIS-WebServer" should be installed

Port "80" should be listening

File "C:\inetpub\FourthCoffee\Default.cshtml" should be file

Finished in 13.41 seconds (files took 0.48432 seconds to load)

3 examples, 0 failures

Finished verifying <default-windows-2012R2> (0m22.73s).

Finished testing <default-windows-2012R2> (27m11.16s).

-----> Kitchen is finished. (27m12.60s)

Summary

• Don't point-and-click to administer your Windows servers• Learn PowerShell!• Learn declarative configuration management• Test your infrastructure code

Q&A