git + docker tutorial - nersc.gov
TRANSCRIPT
Tony Wildish!
Git + Docker tutorial
Preamble
• Thispresenta,on,thetutorialmaterial– h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip– h"ps://www.nersc.gov/users/computa>onal-systems/genepool/genepool-training-and-tutorials/
• Pre-requisites– Seeh"ps://bitbucket.org/TWildish/git-docker-tutorial/overview
– PleasetellmeyoudidthatalreadyJ• Today:– 3:00–4:00:gitoverview+hands-onexercises– 4:00–5:00:dockeroverview+hands-onexercises– Familiaritywithwhat’spossible,ratherthanadeep-dive– Workedexamplesofhowtodothings
-2-
This tutorial
• Git– Basicsofrepositories,localandremote– Howtorecoverfrommistakes– Workingwithbranches– Workingwithteams
• Docker– Variouswaystorun&managedockercontainers– Arealbioinforma>csapplica>onexample
• ThankstoMichaelBarton– Howtogetdatainto/outofadockercontainer– Howtobuildasimpledockercontainer– ShiTer–dockeronCori,Edison,and(eventually)Genepool
-3-h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip
Git history
• Gitisa‘VersionControlSystem’,(VCS)• Gitmanagescollec,onsoffiles(text,smallbinaries)– Trackstheirhistory,versions– Tracksmul>pledevelopmentpaths– Letsyourecoverpreviousversions
• GitistheVCS,don’tbotherwithanythingelse– CVS:ConcurrentVersionSystem->completelyobsolete– SVN:SubVersioN->mostlyobsolete(shouldbe!)
• DesignedbyLinusTorvalds(hewhogaveusLinux!)• Q:Whatdoes‘git’standfor?
-4-h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip
Why use git?
• Security– Neverloseyourcodeagain– Codeissafeagainstdiskfailure/earthquakes/meteors
• Convenience– Easilydeployyourcodeinseveralplaces– Easilymanageseveralversions(prod,dev,…)
• Community– Shareyourcodewithothers– Acceptbug-fixes&contribu>onsincontrolledmanner
• DidImen,on…– Neverloseyourcodeagain
-5-h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip
Git components
• Command-lineinterface,the‘git’command• Server‘hos,ng’plaRorms,web-interface,API– Github.com:theoriginalgithos>ngservice– Bitbucket.com:usedbyLBNL/JGI– Gitlab.com:recentplacormwithcon>nuousintegra>on
• Hos,ngplaRormsbringaddedvalue– Issuetracking:bugreports,coupledtogithistory– Wiki:managingdocumenta>on– Teammgmt:differentroles(admin,developer,user)– Accessmgmt:read/write,read-only,private,public– ‘web-hooks’:performcustomac>onsbasedontriggers
-6-h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip
Git concepts
• Repository– Localorremote,aplacewheregitkeepsyourfiles
• Onyourlocaldisk,oronaremoteserver
• Workingarea– Partofyourlocalrepository,youedityourcodethere
• Stagingarea– Partofthelocalrepositorywheregittrackschangestoyourworkingarea
• Branches,tags– Waystomanagesub-groupsoffilesinarepository
-7-h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip
Git workflow
• Changefilesinyourworkingarea• Tellgitaboutthechanges– Thisaddsthefilestothe‘stagingarea’– Atthispoint,s>llpossibletoundo,leavingnotrace
• Committhosechanges– Makethempermanent,addthemtotherepository– Nowthosechangescanberecovered,any>melater
• Pushthechangestoaremoterepository– Copyyourlocalrepositorytoaremoteserver– Nowyouhavearemotebackup
-8-h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip
More git concepts
• Clone– Alocalcopyofaremoterepository– Youcanchangetheclone–youownit– Accesstoremoterepositorycontrolledbyitsowner
• Fork– Aremotecopyofanotherremoterepository– Youownthefork,whichyoucannowcloneandchange
• Anon-concept:‘TheCentralRepository’– Gitiscompletelydecentralized– Canworkwithmul>pleremoterepositories,simultaneously
• Confused?– Let’sgetstuckintotheexercises...
-9-h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip
Git exercises
• Cookbookapproach:– Cancut-&-paste,butbe"ertotypeincommandsyourself
• Today:doexercises1,3,and4ifyouhave,me– 1)BasicCommitandTag– 2)UndoingMistakes– 3)UsingARemoteRepository– 4)UsingBranches– 5)WorkinginTeams
• Feelfreetoworkthroughtherestatyourownpace
-10-h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip
Docker
-11-
Docker overview
• Dockerisa‘containertechnology’– Linux-specific
• can’trunMacOSX,Windowsindockercontainers• ButcanrundockercontainersonMacOSX&Windows
• Similartovirtualmachines,butmorelightweight– Smaller,fastertostart,easiertomaintainandmanage– Lighteronsystemresources=>vastlymorescalable
• Notavirtualmachine– Sharestheunderlyinghostopera>ngsystem– Lessfullyisolatedfromthehost=>securityconcerns– Moreofanapplica>on-wrapperonsteroids
-12-h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip
Docker components
• The‘docker’command-linetool– Abitofakitchen-sink,yourone-stopshopforeverythingdocker
• Thedocker-daemon– Worksbehindthescenestocarryoutac>ons– Managescontainerimages,processes– Buildscontainerswhenrequested– Runsasroot,notauser-spacedaemon
• Docker.com– Allthingsdocker:installa>on,documenta>on,tutorials
• Dockerhub.com– Repositoryofdockercontainers.Manyotherrepositoriesexist
-13-h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip
Docker concepts • Image
– Ashrink-wrappedchunkofs/w+itsexecu>onenvironment• Imagetags
– Iden>fydifferentversionsofanimage– Anamespaceforsepara>ngyourimagesfromotherpeoples
• Imageregistry– Aplaceforsharingimageswithawidercommunity– Dockerhub.com,plussomedomain-specificregistries
• Container– Aprocessinstan>atedfromanimage
• Dockerfile– Arecipeforbuildinganimage:download,compile,configure…– CanshareeithertheDockerfile,ortheimage,orboth
-14-h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip
Docker images: layers
• Imagesusethe‘overlayfilesystem’concept– Imageisbuiltbyaddinglayerstoabase– EachcommandintheDockerfileaddsanewlayer– Eachlayeriscachedindependently– Layerscanbesharedbetweenmul>pleimages– Changeinonelayerinvalidatesallfollowinglayers
• Forcesrebuild(similarto‘make’dependencies…)
• Performanceconsidera,ons– Toomanylayerscanimpedeperformance– Toofewcancauseexcessiverebuilding– Buildingproduc>on-qualityimagestakescare,prac>ce
-15-h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip
Docker exercises
• Again,acookbookapproach• Today:1,3and4aremostinteres,ng– 1)RunningImages– 2)Cleaningup– 3)RunningaBioboxContainer– 4)Crea,ngaDockerImage– 5)RunningonCoriwithShiTer
-16-
h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip
-17-