making your first openstack contribution (europython)
DESCRIPTION
Hands-on workshop to guide newcomers through the OpenStack contribution process. For EuroPython in Berlin, July 22rd 2014.TRANSCRIPT
Making your first contribution to OpenStack
Julie Pichon, EuroPython, July 22nd 2014
Etherpad: http://tinyurl.com/ep-openstack-etherpad
Slides: http://tinyurl.com/ep-openstack
The Cloud
http://en.wikipedia.org/wiki/OpenStack#Components
Compute (Nova) Object Storage (Swift)
Image Service (Glance) Identity Service (Keystone)
Dashboard (Horizon) Block Storage (Cinder)
Networking (Neutron) Orchestration (Heat)
Telemetry (Ceilometer) Database (Trove)
http://en.wikipedia.org/wiki/OpenStack#Components
Compute (Nova) Object Storage (Swift)
Image Service (Glance) Identity Service (Keystone)
Dashboard (Horizon) Block Storage (Cinder)
Networking (Neutron) Orchestration (Heat)
Telemetry (Ceilometer) Database (Trove)
http://en.wikipedia.org/wiki/OpenStack#Components
Compute (Nova) Object Storage (Swift)
Image Service (Glance) Identity Service (Keystone)
Dashboard (Horizon) Block Storage (Cinder)
Networking (Neutron) Orchestration (Heat)
Telemetry (Ceilometer) Database (Trove)
http://en.wikipedia.org/wiki/OpenStack#Components
Compute (Nova) Object Storage (Swift)
Image Service (Glance) Identity Service (Keystone)
Dashboard (Horizon) Block Storage (Cinder)
Networking (Neutron) Orchestration (Heat)
Telemetry (Ceilometer) Database (Trove)
http://en.wikipedia.org/wiki/OpenStack#Components
Compute (Nova) Object Storage (Swift)
Image Service (Glance) Identity Service (Keystone)
Dashboard (Horizon) Block Storage (Cinder)
Networking (Neutron)(as Quantum)
Orchestration (Heat)
Telemetry (Ceilometer) Database (Trove)
http://en.wikipedia.org/wiki/OpenStack#Components
Compute (Nova) Object Storage (Swift)
Image Service (Glance) Identity Service (Keystone)
Dashboard (Horizon) Block Storage (Cinder)
Networking (Neutron) Orchestration (Heat)
Telemetry (Ceilometer) Database (Trove)
http://en.wikipedia.org/wiki/OpenStack#Components
Compute (Nova) Object Storage (Swift)
Image Service (Glance) Identity Service (Keystone)
Dashboard (Horizon) Block Storage (Cinder)
Networking (Neutron) Orchestration (Heat)
Telemetry (Ceilometer) Database (Trove)
http://en.wikipedia.org/wiki/OpenStack#Components
Compute (Nova) Object Storage (Swift)
Image Service (Glance) Identity Service (Keystone)
Dashboard (Horizon) Block Storage (Cinder)
Networking (Neutron) Orchestration (Heat)
Telemetry (Ceilometer) Database (Trove)
Meta:
● Documentation● Quality Assurance (Tempest)● Infrastructure (not today)
Also:
● Clients
Nova (Compute)
$ nova list
Helpful trick: $ nova --debug list
python-novaclient
REST
APIs
$ curl -i 'http://my_ip:5000/v2.0/tokens' -X POST -H "Accept: application/json" -H "Content-Type: application/json" -d '{"auth": {"tenantName": "demo", "passwordCredentials": {"username": "demo", "password": "123456"}}}'$ curl -i 'http://my_ip:8774/v2/project_id/servers/detail' -X GET -H "Accept: application/json"-H "X-Auth-Project-Id: demo" -H "X-Auth-Token: my_token"
Keystone (Identity)
REST
APIs
Session objectives● Have all the necessary accounts and tools set
up for contributing● Know how to submit a contribution to an
OpenStack component● Understand the OpenStack contribution
process● Have a first contribution completed or
underway, and know what are the next steps
https://wiki.openstack.org/wiki/How_To_Contribute
Also: http://docs.openstack.org/infra/manual/developers.html
Setting up (one time only)
● Launchpad (bug tracking, single sign-on)● The OpenStack Foundation (voting)● Gerrit and the Contributor Licence Agreement
(reviews, submitting patches)● Keypair (so you can upload your patches to
Gerrit)● Basic check: git review -s
Protip: Use the same email for all these accounts
contributor
Launchpad
sign in
Gerrit
sign up
upload keypair OpenStackFoundation
sign up
compare email addresses to confirm membership
Extra reading: https://wiki.openstack.org/wiki/GerritWorkflow
● Launchpad– https://launchpad.net/ -> Register -> Create account
● The OpenStack Foundation– https://www.openstack.org/join/ (Use the same email as LP)
– “Affiliation:” None (unless you're here for your company!)
– “Statement of interest:” e.g. “I want to contribute code”, “I want to improve the documentation”, ...
● Gerrit and the CLA– https://review.openstack.org (Sign in with LP account)
– Sign the Individual CLA (ICLA)
● Keypair– Upload at https://review.openstack.org/#/settings/ssh-keys
Bonus levelSimple check (gerrit account + keypair) with git review:
● Clone a repo from https://github.com/openstack or cd to /opt/stack (DevStack)● sudo pip install git-review● git review -s
DevStack tips and tricks● What is DevStack?
– Best tested on Ubuntu 12.04 and Fedora 20
– Extra reading: http://devstack.org/overview.html
● Always in a VM● Good for testing and actually seeing● Not a requirement for contributing though
– /opt/stack/
– https://github.com/openstack/
● Need help troubleshooting? Ask for help on IRC.– #openstack-101, #openstack-dev (Freenode)
– https://wiki.openstack.org/wiki/IRC
Tip #1: Use the local.conf file
$ cat devstack/local.conf
[[local|localrc]]
OS_PASSWORD=123456
ADMIN_PASSWORD=123456
MYSQL_PASSWORD=123456
RABBIT_PASSWORD=123456
SERVICE_PASSWORD=$ADMIN_PASSWORD
SERVICE_TOKEN=123456
# If you need the latest version
RECLONE=yes
# Can be handy to help debug stack.sh failures
VERBOSE=True
# Neutron (naming based on Quantum)
disable_service nnet
enable_service qsvc qagt qdhcp ql3 qmeta quantum qlbaas
(See http://devstack.org/configuration.html for even more options)
Tip #2: On using screen
● http://www.gnu.org/software/screen/manual/screen.html#Default-Key-Bindings
● screen -r, screen -ls● Detaching the screen session: C-a, C-d● Navigating: C-a “● (C-a ?)
Applied screen: restarting a service
Example: Imagining you made a change in /opt/stack/keystone
screen r
Ca “ # Choose “key” for keystone
Cc # To kill the service
Press uparrow + Enter, to restart the last command
Ca Cd # To leave
Tip #3: openrc files and communicating with OpenStack
$ nova list
$ source ~/devstack/openrc # credentials
$ nova list
$ nova help
$ keystone userlist
$ source ~/devstack/openrc admin
$ openstack help # New common client
A simple openrc example
$ cat ./example_openrc
export OS_USERNAME=username
export OS_PASSWORD=password
export OS_TENANT_NAME=projectName
export OS_AUTH_URL=https://identityHost:portNumber/v2.0
See http://docs.openstack.org/user-guide/content/cli_openrc.html#openrc-create
Getting started
Soft skills
“Assume good faith.”
Getting started
Culture
● Friendly community● Reviewer vs. Committer team
Getting started
Process● +1, -1
Everyone is welcome and encouraged to participate in reviews
● +2, -2Core reviewers team
● Continuous Integration (CI) gets a vote too● Unit tests, integration tests, style
● To get your patch merged:● +1 from CI● +2 from two cores (+A)
the contributor guidelines
Read
Usethe bug tracker
the contributor guidelines
Buildthe software
Findthe bug
Fixit!
Submityour patch
The Open-Source contribution process
Sources
● Code:– /opt/stack (DevStack)
– github.com/openstack
● Docs:– github.com/openstack-manuals– https://wiki.openstack.org/wiki/Documentation/HowTo/FirstTimers
● QA/Integration tests:– github.com/openstack/tempest (DevStack required)
– http://docs.openstack.org/developer/tempest/
How to choose a bug?
● Low hanging fruits– e.g. https://bugs.launchpad.net/horizon/+bugs?field.tag=low-hanging-fruit
● Browsing the bug tracker– e.g. https://bugs.launchpad.net/neutron
● Sometimes a project has additional docs– e.g. https://wiki.openstack.org/wiki/NeutronStarterBugs
● Launchpad tips and tricks:
– Low priority bugs can sometimes be smaller (not always true)
– Advanced search: by tag (low-hanging-fruits)
– Advanced search: Assignee -> “Nobody”
– Advanced search: Status New/Confirmed/Triaged
– Find something that piques your interest or touches an area of knowledge
Launchpad
● run_tests.sh often has a coverage flag– usually ./run_tests.sh coverage
– otherwise: ./run_tests.sh help
● Reports should appear in a new ./reports/ directory
● Unit tests and additional non-Python requirements: have a look at the README file– e.g. for Nova: python-dev libssl-dev python-pip git-core libxml2-
dev libxslt-dev pkg-config libffi-dev libpq-dev libmysqlclient-dev
How to choose a bug?Adding a unit test
● Find and fix a typo!
● Main docs: guides, API references, ...– http://docs.openstack.org/
– Sources: https://github.com/openstack/openstack-manuals
– More info on how to build the docs: https://wiki.openstack.org/wiki/Documentation/HowTo
● Developer docs: within the code repositories– e.g. http://docs.openstack.org/developer/heat/ is generated from
/doc/ at https://github.com/openstack/heat/
– Find a typo, fix Sphinx syntax errors... (http://sphinx-doc.org/)
– Usually can build the docs with e.g. “tox -e docs” (pip install tox first)
How to choose a bug?Documentation patch
● Often out of date or copy-paste errors● Projects: python-<codename>client● Where? Usually:
– python-<codename>client/<codename>client/<version>/
– Pick a resource type and read the file!
● Correct the entire file
How to choose a bug?Client doc strings
commit ea8c9554c947b7fd30adfb1249a9317f258901ab
Author: Julie Pichon <[email protected]>
Date: Mon Jan 27 14:28:28 2014 +0000
Fix typos in the volumes and snapshots docstrings
ChangeId: I0e4f3813b8e263ea59e182db153c5b4c6fd2a177
diff git a/cinderclient/v2/volumes.py b/cinderclient/v2/volumes.py
index 524456d..7f77bb9 100644
a/cinderclient/v2/volumes.py
+++ b/cinderclient/v2/volumes.py
@@ 187,7 +187,7 @@ class VolumeManager(base.ManagerWithFind):
def get(self, volume_id):
"""Get a volume.
:param volume_id: The ID of the volume to delete.
+ :param volume_id: The ID of the volume to get.
:rtype: :class:`Volume`
"""
return self._get("/volumes/%s" % volume_id, "volume")
@@ 225,7 +225,7 @@ class VolumeManager(base.ManagerWithFind):
def update(self, volume, **kwargs):
"""Update the name or description for a volume.
:param volume: The :class:`Volume` to delete.
+ :param volume: The :class:`Volume` to update.
"""
if not kwargs:
return
● https://wiki.openstack.org/wiki/How_To_Contribute ● Help with translations, break things and file bugs (DevStack),
do reviews, ...● Bug triage – This can help you find a task!
– See the excellent blog post, “A path towards contributing (via commits) in OpenStack” http://www.mattfischer.com/blog/?p=602
– Bugs in “NEW” state: confirm if can reproduce (DevStack)● Yes? Set status to CONFIRMED● No? Ask questions, ask for clearer steps to reproduce, optionally mark
“INCOMPLETE”
– http://wiki.openstack.org/BugTriage
How to choose a bug?Other tasks
Now
● This should cover most non-Python dependencies:– sudo apt-get install python-dev libssl-dev libxml2-dev libxslt-dev
pkg-config libffi-dev libpq-dev libmysqlclient-dev– sudo yum install python-devel openssl-devel gcc libxslt-devel
mysql-devel postgresql-devel libffi-devel
● Choose a project● Go into the repository and start ./run_tests.sh
This will create the virtual environment in .venv and download the required dependencies● Short Break
Find a task and work on it
● Browse the Launchpad bug tracker (low hanging fruit or other) – maybe better for sprints
● Increase the test coverage by adding a unit test● Improve the docs● Fix a client's doc strings● Confirm bugs with DevStack● https://wiki.openstack.org/wiki/How_To_Contribute
● (http://tinyurl.com/ep-openstack )
Formatting your patch to community standards
● See “Summary of GIT commit message structure” in wiki
https://wiki.openstack.org/wiki/GitCommitMessages#Summary_of_GIT_commit_message_structure
● Style guidelines: http://docs.openstack.org/developer/hacking/
● Check no pep8/hacking errors:
– ./run_tests.sh -p● Check the tests pass:
– ./run_tests.sh
Switch libvirt get_cpu_info method over to use config APIs
The get_cpu_info method in the libvirt driver currently uses XPath queries to extract information from the capabilities XML document. Switch this over to use the new config class LibvirtConfigCaps. Also provide a test case to validate the data being returned.
DocImpact
ClosesBug: #1003373
Implements: blueprint libvirtxmlcpumodel
ChangeId: I4946a16d27f712ae2adf8441ce78e6c0bb0bb657
Submitting a patch
● Preparing your patch
– Git commit message structure, pep8, unit tests● sudo pip install git-review● In the repository:
– git review -s● To add a ChangeId to your existing commit:
– git commit --amend– (exit editor, no need to change anything)
git-review
● For the cautious:– git log -n1 # check your commit has a ChangeId– git review -n # dry run, doesn't push anything out yet
● git review # sends your patch to Gerrit
● “The Gate”– Unit tests, Integration tests, style checks
$ git review n
Please use the following command to send your commits to review:
git push gerrit HEAD:refs/publish/master/bug/1324982
$ git review
remote: Resolving deltas: 100% (2/2)
remote: Processing changes: new: 1, refs: 1, done
remote:
remote: New Changes:
remote: https://review.openstack.org/96815
remote:
To ssh://[email protected]:29418/openstack/horizon.git
* [new branch] HEAD > refs/publish/master/bug/1324982
Gate jobs
git review bonus commands
● git review can help you do reviews, too– git review -l
– git review -d <review_id>
The Next Steps
● Your patch will get reviewed, be patient● You can help by doing reviews of other patches yourself
– Learn about the project's culture and structure
● Where to ask for help?– IRC: #openstack, #openstack-dev, #openstack-101
● https://wiki.openstack.org/wiki/IRC
– Mailing list: [email protected] (filters!)● https://wiki.openstack.org/wiki/MailingLists
– http://ask.openstack.org/– Don't stay stuck! Ask for help on IRC.
What to do next?
● EuroPython sprints● Re-read https://wiki.openstack.org/wiki/How_To_Contribute● Find a slightly harder bug. After resolving one, try to leave the remaining low
hanging fruits for the next newcomer.● Review, triage, translate... Lots of ways to help!● Respond to review feedback, if you receive any● Join the mailing list (filters, filters), idle on IRC, become part of the
community● Summit! Conference + Design Summit. Participate in the conversation.
– Next one in November in Paris
– Free ticket for “ATC”
– http://www.openstack.org/summit/
What did we learn?
● What accounts are required and why● The OpenStack development process● Glimpse into the OpenStack developer toolbox
– DevStack, Gerrit, Launchpad
● Where/How you can help● The Life of a Contribution● Where to ask for help
● “Next action” - plan it now– Take a couple of minutes to figure out what you want to do next
and write it down
Questions?
Feedback form: http://tinyurl.com/ep-openstack-survey