Download - Contributing to OpenStack
Agenda
• Part I – Introduc-ons – Introduc-on to OpenStack
• Part II – Deep dive of a OpenStack project -‐ Solum
• Part III – Contribu-ng to OpenStack – OpenStack CI system – Development best prac-ces – Hands-‐on exercise
• Ques-ons and Answers
Agenda
• Part I – Introduc,ons – Introduc,on to OpenStack
• Part II – Deep dive of a OpenStack project -‐ Solum
• Part III – Contribu-ng to OpenStack – OpenStack CI system – Development best prac-ces – Hands-‐on exercise
• Ques-ons and Answers
Introduc-on
• Me – Devda8a Kulkarni – First contributor on Solum – Currently Project Team Lead (PTL) of Solum for Mitaka cycle
– PhD, University of Minnesota Minneapolis
• You
What is OpenStack?
A family of “open” projects that together enable different aspects of cloud compu-ng Openness of OpenStack h8ps://wiki.openstack.org/wiki/Open
• Open source • Open design • Open development • Open community
OpenStack family
• Family of projects enabling different aspects of cloud compu-ng – Authen-ca-on and authoriza-on layer
• Keystone – Infrastructure-‐as-‐a-‐service layer
• Nova, Glance – Storage layer
• Swi[, Cinder – Networking layer
• Neutron – Pla\orm-‐as-‐a-‐service layer
• Solum, Heat, Murano, Mistral, Trove
• Project types – OpenStack big tent vs. OpenStack related projects
OpenStack by numbers
• 49 projects in OpenStack big tent • Several hundred organiza-ons involved • Several thousand developers par-cipa-ng
Agenda
• Part I – Introduc-ons – Introduc-on to OpenStack
• Part II – Deep dive of a OpenStack project -‐ Solum
• Part III – Contribu-ng to OpenStack – OpenStack CI system – Development best prac-ces – Hands-‐on exercise
• Ques-ons and Answers
What is Solum?
OpenStack project that provides easy mechanism for applica-on developers to deploy and run their applica-ons on OpenStack star-ng from applica-on’s source code Why should you care? (the value proposi-on)
– For operators, Solum provides ability to make your OpenStack cloud more useful for your clouds’ applica-on developers by simplifying the process of deploying applica-ons to it
– For developers, Solum provides an easy-‐to-‐use pla\orm for building, tes-ng, and deploying applica-ons on OpenStack clouds
Project goals
Applica-on Stack Flexibility Ability to support applica-ons wri8en in different languages and using different applica-on frameworks
Developer Produc-vity Ability to perform CI/CD, integra-on with github Add-‐On Services Extensibility Ability to support different add-‐on services
Applica-on Portability Ability to deploy applica-ons across different OpenStack clouds (use na-ve OpenStack services)
Project details
• Wiki: h8ps://wiki.openstack.org/wiki/Solum
• Mee-ngs: Tuesdays 1700 UTC in #openstack-‐mee-ng-‐3
• IRC: Channel: #solum on freenode
• Code: h8ps://github.com/openstack/solum h8ps://github.com/openstack/python-‐solumclient
Solum abstrac-ons -‐ App
Declara-ve applica-on defini-on
• source code repository • run command • port • languagepack
version: 1 name: cherrypy description: python web app languagepack: python source: repository: https://github.com/rackspace-solum-samples/solum-python-sample-app.git revision: master workflow_config: run_cmd: python app.py ports: - 80
Solum abstrac-ons -‐ Languagepack (LP)
• Docker image which contains applica-on specific build and run-me libraries
• A LP needs to implement Solum’s languagepack contract A well-‐known file available at a known loca-on on the languagepack image /solum/bin/build.sh
• How to build a LP? Dockerfile
• Operator-‐defined or user-‐defined
Solum abstrac-ons -‐ Languagepack Example: Python
FROM ubuntu:precise MAINTAINER Murali Allada <[email protected]> RUN apt-get -yqq update RUN apt-get -yqq install python-pip RUN apt-get -yqq install python-dev COPY build.sh /solum/bin/
https://github.com/rackspace-solum-samples/solum-languagepack-python
#!/bin/bash # Check if pip is installed pip help [[ $? != 0 ]] && echo python-pip is not installed. && exit 1 # Install app dependencies cd /app pip install -r requirements.txt
build.shDockerfile
Solum abstrac-ons -‐ Deployment Unit (DU) (1/2)
• Docker image that is formed from the languagepack image with applica-on’s source code added to it
DU = LP + applica-on source code
• Solum guarantees the DU contract Applica-on source code available at a known loca-on on the DU image /app
• How to build a DU? – Construct Dockerfile with languagepack as the base image, applica-on source code
injected, run command as the entry point – Build the DU image from this Dockerfile
Solum abstrac-ons -‐ Deployment Unit (DU) (2/2)
• Building a DU – Start from the specified languagepack LP – Specify execu-on of languagepack’s ‘build.sh’ – Inject applica-on source code – Use the run command specified in app defini-on as the default Entrypoint to
run the DU
• LP and DU storage Glance, Swi[, Docker registry
18
Solum abstrac-ons -‐ Workflow
• Abstrac-on to represent execu-on of applica-on deployment consis-ng of one or more workflow stages
• Supported workflow stages
– Build DU (and store it for future use) – Run unit tests, build DU – Run unit tests, build DU, deploy DU (if unit tests pass) – Build DU, deploy DU – Deploy a previously built DU (not yet available)
• A workflow can be triggered from github webhooks
Solum abstrac-ons -‐ Add-‐ons
• Services needed by an applica-on E.g.: rela-onal database such as Trove
• DU parameters
Solum supports ability to pass service’s connec-on parameters to applica-on DU
High-‐level architecture
Solum API
Solum Worker
Solum Deployer Queue Queue
Queue
Solum Conductor
Heat
Solum Database
Glance, Swi[, Docker Registry
Builds LP and DU Deploys DU
-‐ Decoupled architecture
-‐ Asynchronous communica-on
Agenda
• Part I – Introduc-ons – Introduc-on to OpenStack
• Part II – Deep dive of a OpenStack project -‐ Solum
• Part III – Contribu,ng to OpenStack – OpenStack CI system – Development best prac,ces – Hands-‐on exercise
• Ques-ons and Answers
Tools/Systems
• Gerrit: Code review system – h8ps://review.openstack.org/
• Launchpad: Work (bugs/blueprints) tracking system – h8ps://launchpad.net/
• Github: Code repository – h8ps://github.com/openstack
• Git: Code contribu-on • Mailing list: openstack-‐dev – h8p://lists.openstack.org/cgi-‐bin/mailman/lis-nfo/openstack-‐dev
• IRC: chat.freenode.net
Setup steps (1/2)
• h8p://docs.openstack.org/infra/manual/developers.html
• Create Launchpad account • Join OpenStack founda-on as founda-on member • Sign individual contributor’s license agreement • Install git-‐review • Install git • Run git review -‐s
Setup steps: Problems and troubleshoo-ng
• Problem: We don't know where your gerrit is. Please manually create a remote
• Solu-on: – Create a username in gerrit if not already created – Manually set gerrit remote. E.g.: git remote add gerrit "ssh://[email protected]:29418/openstack/solum.git”
Setup steps: Problems and troubleshoo-ng
• Problem: Problem running 'git remote update gerrit’
Fetching gerrit Permission denied (publickey). fatal: Could not read from remote repository. • Solu-on: – Generate sshkey and add it to gerrit – ssh-‐keygen
Steps to submit a patch
• Create a Launchpad bug or assign an exis-ng bug to yourself
• Fixing the bug – Itera-ve process
• Understand the issue (be in touch with project developers on project’s IRC channel or on openstack-‐dev mailing list)
• Design solu-on • Write code • Write tests • Commit code • Provide good commit message • Submit patch for review
Steps to merge a patch
• Patch needs to pass configured automated tests • Patch needs to get two +2 votes • Patch needs to be approved for merging by project’s core reviewer
Con-nuous Integra-on of patches
• Systems involved: – Zuul – Gerrit – Jenkins – Github
• Pre-‐merge checking – Patch is merged to master only if all the configured tests pass
OpenStack’s CI setup
Gerrit Zuul Jenkins
Github
2 3
4 5
1 Patch submi8ed for review
Patch picked up for tes-ng Tests ran on the patch
Jenkins gives its vote
Code merged
Devstack
• OpenStack environment consis-ng of all the openstack-‐services – Keystone, Nova, Heat, Solum, etc.
• Can be run within a virtual machine or independently
• Useful for development
Vagrant and VirtualBox
• Vagrant – Configura-on management tool for configuring virtual machines
• VirtualBox – Virtualiza-on system
• Spinning up a development environment using Vagrant + VirtualBox – h8ps://github.com/rackerlabs/vagrant-‐solum-‐dev – git clone <solum-‐vagrant-‐repo> – vagrant up -‐-‐provision
Sexng up Solum development env
• Develop on Host, debug on the VM • Install Vagrant and Virtualbox • Clone solum and python-‐solumclient repositories on the Host
• Set following environment variables: – SOLUM=<path on Host system to solum repo> – SOLUMCLIENT=<path on Host system to python-‐solumclient repo>
• vagrant up -‐-‐provision • vagrant ssh
Python
• PyCharm IDE • pdb – Put breakpoint in the code where you want to break • import pdb; pdb.set_trace()
– pdb commands • s: Execute the current line, stop at first possible occasion • c: Con-nue execu-on, only stop when a breakpoint is encountered
• h8ps://docs.python.org/2/library/pdb.html
Tes-ng
• Run tests locally before submixng patch for review – From within solum directory • tox –epep8 • tox –epy27
• Tox – Command line tool for running tests within virtualenv
Code paths on Devstack
• solum /opt/stack/solum
• python-‐solumclient /opt/stack/python-‐solumclient
• Solum logs – /var/log/solum/deployer – /var/log/solum/worker
• Devstack logs (of various screen sessions) – /opt/stack/logs/
Screen
• Program that allows sharing physical terminal between different processes
• Entering Solum’s screen session in Devstack – screen –x stack
• Screen commands – Move to next window: Ctrl+a n – Move to previous window: Ctrl+a p – See all windows: Ctrl+a “ – Enter scrollback/copy mode: Ctrl+a [ – Leave scrollback/copy mode: Ctrl+a ] – h8p://www.pixelbeat.org/lkdb/screen.html
Debugging
• Debug code/feature – Put breakpoints in the code – Invoke code-‐under-‐test from CLI – Use screen to enter breakpoint and then debug using Pdb
• Debug failing func-onal test – h8ps://review.openstack.org/#/c/254344/
• Debug failing unit test
Git
• Crea-ng dependent commits • Squashing commits – git rebase -‐i HEAD~<number-‐of-‐commits> – pick/squash/edit
• Upda-ng dependent commit and submixng for review
• h8ps://ask.openstack.org/en/ques-on/31633/gerrit-‐best-‐way-‐to-‐make-‐a-‐series-‐of-‐dependent-‐commits/
Hands-‐on exercise
• Submit first patch to OpenStack (45 minutes)
• h8ps://etherpad.openstack.org/p/solum-‐low-‐hanging-‐fruit-‐bugs