December4–9,2016|Boston,MAwww.usenix.org/lisa16#lisa16
ReleasePipelinesinMicrosoftEcosystems
WarrenFrame,HarvardUniversity
MichaelGreene,Microsoft
whoami• WarrenFrame
• ResearchComputingatHarvardUniversity
• @pscookiemonster• Ramblingcookiemonster• wframe
• MichaelGreene• EnterpriseCloudEngineeringCAT
TeamatMicrosoft• @migreene• mgreenegit• migreene
bit.ly/lisa16pipeline
Stuff• Slides• Demos!• Slidesatbit.ly/lisa16pipeline• Cleanup,
ConfigurationasCode• Everything-as-a-service,APIsgalore• Livingdocumentation• Abstractoutcomplexity.Scripts->Modules->DSC->key:value• PowerShellDSCisaplatformthatallsolutionscanusetodeployand
manageWindowsServer• AzureResourceManagertemplates• Youstillneedtoknowtheunderlyingsystemsyouwillmanage• Releasepipelinescanbringsanityandconsistencytomanagingthis
ReleasePipeline
aka.ms/trpmWhy?
ReleasePipelines
Prodenvironment(etc.)…• Systems/Services• Modules• Scripts• Config files
ExampleWorkflow• Makeachange,pushtosourcecontrol*• Buildsystem doestherest.Forexample:
• Runtestsagainstyourcode• Spinuptestservices/infrastructureformoretests• Buildartifacts(packages,configs,etc.)• Deploythings(artifacts,systems,services,etc.)
*Youmightrunthroughsource-build-testloopslocallyuntilhappy,beforepushing
Tooling“abunchofrandomopensourceprojectsboundtogetherwith
ducttapeandchewinggum”
Tools:SourceGit?Mercurial?SVN?CLI:• Git forWindows• PoshGitGUI:• GitHubDesktop• Atlassian SourceTree• Manyothers
Demo:SourceGitVisualStudioCode
Tools:BuildSystems• Jenkins,GitLab CI,VSTS,etc.• Preferbuild-as-code
• e.g.Jenkinsfile,appveyor.yml,.gitlab-ci.yml
Tools:BuildAutomation• Invoke-Build,psake• Similartorake,make,bake,cake,grunt,gulp,msbuild,etc.
Demo:BuildTFS2017psake (buildautomation)github.com/powershell/demo_ci
Tools:Testing• Pester: Testframework• poshspec: infrastructuretesting• OVF: Operation-Validation-Framework- simplifyorganizing,
execution,andsharingoftests.
Demo:TestPesterposhspec
Tools:Release• OctopusDeployandVSTS
• Manypre-cannedtasks• Flexible• Pretty• Potentially$$
• PSDeploy• Somepre-cannedtasks• Deploymentascode• Poorlywritten• Opensource
• RandomPowerShellcode• Funtoreadandmaintain!
Demo:ReleaseTFS2017- Releasemanagement
Tools:TestHarness• Test-Kitchen• NotjustforChef• Roughly:
• Runtestswithaverifier(Pester)
• againstplatforms(differentvagrantboxes)
• convergedwithaprovisioner (dsc)
• withthelifecyclemanagedbyadriver(vagrant)
• Andtest,configuration,otherfilescopiedtoplatforms viaatransport(WinRM)
Drivers• AmazonEC2• AzureResourceManager• DigitalOcean• Docker• GoogleComputeEngine• Hyper-V• OpenStack• Vagrant• vRealizeAutomation,Orchestrator• vSphereProvisioners• Ansible• CFEngine• ChefSolo,Zero• DSC• Puppet• Salt• ShellVerifiers• Inspec• Pester• Shell(Bats,Serverspec,etc.)
Source: GitHubBuild: AppVeyor
Builddependencies: PSDependBuildautomation: Invoke-BuildBuildhelpers: BuildHelpers
Test: PesterRelease: PSDeploy
ExamplePipeline
Demo:ExamplePipelinehttps://github.com/RamblingCookieMonster/lisa-kitchen-demo
Whatabout…• Secrets
• Insourcecontrol• Builtintobuildsystem?• Secretmanagement– vault,passwordstate,SecretServer,credstash,etc.
• Images• Packer!• Images-as-code• BuildimagesforAmazon,VirtualBox,Azure,Hyper-V(ish),etc.
Wheretostart• SourceControland/orTests overentirepipelineatonce• Existingtools overresume-driven-development• Newservice(s)/valueproposition overre-engineering
everything• Noluckinhouse?PlaywithGitHub+AppVeyor,VSTS,etc.
Nextsteps• Opensourceprojectscoulduseyourhelp!• JITprovisioningoradynamicpoolsofWindowsbuildagents• WindowsDockercontainersfortesting• Focusonephemeraldeploymentsoverincrementalchanges• Planforday100
CommunityProjects
Mapyourrequirements
Plaster Invoke-Build,psake
Pester
poshspec
OVF
PSDeploy
Lability
PowerShellSlack
References,DivingDeeper• TheReleasePipelineModel - MichaelGreene,StevenMurawski
• BuildingaSimpleReleasePipelineinPowerShellUsingpsake,Pester,andPSDeploy - BrandonOlin
• StackOverflow:HowWeDoDeployment- 2016Edition - NickCraver
• DevOpsReadingList - StevenMurawski• ReadingList - ChrisHunt
• ThePesterPipeline- ChrisHunt
• BestPracticeswithPackerandWindows - MattHodgkins
• IntroductiontoKitchen-DSC - GaelColas
• TestingAnsibleRolesAgainstWindowswithTest-Kitchen- MattHodgkins• Twitter,Slack,andothercommunities
• Etc.