using puppet for deploying hyper-v openstack compute nodes - puppetconf 2013
DESCRIPTION
"Using Puppet for Deploying Hyper-V OpenStack Compute Nodes" by Peter Pouliot, Sr. SDET OpenStack, Microsoft. Presentation Overview: A discussion about using Puppet to deploy openstack on Hyper-V and my experiences writing puppet manifests for Windows. Speaker Bio: Peter has been evangelizing for OpenStack Hyper-V integration since March 2011, now as a Microsoft employee within Microsoft’s Open Source Technology Center, he is working on OpenStack Hyper-V integration. He is the official subject matter expert on OpenStack for Microsoft as well as the Hyper-V/OpenStack community manager and is currently working to build the continuous integration infrastructure to support the ongoing community development and integration of OpenStack and Microsoft technologies.TRANSCRIPT
Puppet, Hyper-V And OpenStack PETER POULIOT, CISSP
MICROSOFT N.E.R.D., CAMBRIDGE, MA
Bio u Peter Pouliot, CISSP
u Email: [email protected]
u Education: u BA Philosophy - Rhode Island College
u Certificate Windows NT Administration - Boston University CEC
u Oracle DBA 8i – Oracle University
u MS Telecommunication Security - Boston University MET College
u Graduate Certificate Digital Forensics - Boston University MET College
u Certifications: u CISSP, MCSE+I, MCT (NT4/2000), OCP (8i), Network+
u 9 years working on Datacenter/Network Infrastructure and Linux/Windows interoperability for Ximian/Novell/SuSE.
u May 2011 Deployed first Openstack/Hyper-V cloud while working in the Novell/Microsoft Joint Interoperability Lab
u March 2012 Started working for Microsoft to return Hyper-V support to Folsom release
u Currently tasked with building a CI infrastructure to support Hyper-V in OpenStack
Agenda
u Puppet at MS
u Brief introduction to Hyper-V and OpenStack
u Why Puppet?
u A history of the OpenStack Hyper-V puppet modules
u Challenges of Using Puppet on Windows.
u State of Devops for Windows IT Pros
OpenStack/Hyper-V
u Nova: Hyper-V can be used as a compute node in OpenStack
u Python application running on Hyper-V
u Publically Available MSI
u MSI Includes Neutron/(formerly Quantum) agent for Hyper-V
u Neutron
u Currently has VLAN Support
u OpenVSwitch for Hyper-V coming
u Cinder: Windows can be used as a block storage node.
u Cloudinit functionality for windows provided by Cloudbaseinit.
u Public MSI available.
Puppet at MS
u Currently using puppet to build the OpenStack CI Infrastructure for Hyper-V
u Puppet 3.2+ on Windows and Linux.
u Running Puppetmaster/Passenger
u Using Kickstart/Preseed/Unattend.xml/Cloudinit as entry for puppet deployment and registration
u Using both centralized and decentralized puppet.
Puppet on Windows?
u Needed a way to configure Windows Hyper-V nodes consistently and rapidly.
u OpenStack deployments on Hyper-V using public binaries is complicated.
u Lots of Hyper-V tidbits need to be configured.
u Lots of additional software requirements.
u Manual process does not scale.
u Wanted to help CERN deploy Hyper-V into their OpenStack infrastructure.
Brief History of Puppet and Openstack on Hyper-V
u 8 months ago Began automating deployment and configuration of Hyper-V using puppet 2.7+ on windows server 2012.
u Created a basic framework for deploying OpenStack nova using public python binaries, with locally served files on Windows.
u Included Windows configuration, vswitch creation and nova configuration.
u Deployment of additional software. (Git, 7zip, Python, python Modules)
u After initial configuration moved to building supporting linux deployment automation.
u Development pace quickened and skill level increased.
Hyper-V/Puppet History Continued
Began automation of windows PE creation for windows pxe automation, including integration with Linux PXE infrastructure.
Refactor of original Hyper-V/OpenStack module to automatically retrieve all components with help from Octavian Ciuhandu from Cloudbase Solutions.
Inclusion of deployment from MSI using Cloudbase Solutions prebuilt MSI for Nova Compute on Hyper-V
A move to puppet 3.2+ on Windows 2012 and Hyper-V Server 2012
Major refactor including a Virtual Switch type/provider from Luis Fernández Álvarez of CERN including dependency on Josh Cooper’s Powershell Modules
Puppet Projects
Puppet-quartermaster: A pxe provisioning infrastructure for deploying linux (kickstart, preseed, unattend.xml) based installations of linux and windows puppetized nodes.
https://github.com/ppouliot/puppet-quartermaster
Puppet-petools: Generates a x86_64 Windows PE image using the Automated Deployment Kit. Includes retrieval and installation of the adk. Slipstreaming of virtio-drivers for windows and other
https://github.com/ppouliot/puppet-petools
Puppet-openstack_hyper_v: hyper-v compute node configuration and openstack nova deployment.
https://github.com/ppouliot/puppet-openstack_hyper_v
puppet-openstack_hyper_v
u Installs a complete Python environment
u Creates external virtual switches
u Enables live migration
u Enables the iScsi initiator service
u Enables RDP Remote access
u Configures NTP
u Joins node to Domain
u Installs Git, 7zip, and other dependancies
u Installs OpenStack (Package, Public Binaries, Source)
puppet-quartermaster
u Creates a basic PXE infrastructure
u TFTP, HTTP, Samba, NFS
u Squid-Proxy (package caching)
u DNSMasq (Proxy DHCP)
u Puppetmaster
u Retrieves necessary pxe kernels and files, generates menus, and necessary automation (kickstart, preseed, unattend.xml) for deploying operating system
u Includes as OS automation retrieval and installation of puppet and registration with puppetmaster.
Puppet-petools
u Downloads and Installs Microsoft’s Automated Deployment Kit
u Creates a Windows PE x86_64 image
u Mounts the Windows PE Image
u Installs necessary Windows PE Packages
u Retrieves and Installs KVM VirtIO drivers into WinPE Images
u Installs additional Windows Drivers into winpe image.
u Modifies startnet.cmd to map appropriate drives to launch a script on a windows share.
u Finalizes WinPE image and Copies all necessary files to allow for WinPE to PXE onto a windows file share.
Challenges of Puppet on Windows
u Windows isn’t Linux.
u Lack of public modules for Windows.
u No windows package management
u Spaces in path naming
u Reboots
u Previously was no powershell provider for puppet/windows.
u Move from using native .exe commands to the powershell equivalent.
Windows IT Professionals and Devops: A problem
u Problem:
u Devops is a new concept for the Windows IT Pro
u Continuous Integration and Delivery are also new concepts for Windows IT Pro
u Skills need to updated
u Enterprise shops usually have highly segregated skill sets.
u Typical windows workflows are not automated or automation
How do we fix the problem?
u Organize the Windows Puppet Community.
u Share experiences
u Share code
u Create a rich catalog of public Puppet Modules for the Windows IT Pro
u Evangelism: Windows can be automated too
u Education:
u Remember, It’s only computer science!
More information
u http://wiki.openstack.org/Hyper-V
u IRC: Freenode: #openstack-hyper-v
u Meetup: OpenStack Boston
u Hyper-V MSIs for OpenStack:
u http://www.cloudbase.it
u Email: [email protected]