juc europe 2015: optimizing your ci: lessons learned from a successful jenkins rebuild
TRANSCRIPT
![Page 1: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/1.jpg)
Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild
Europe, June 2015 Jonathann Zenou @zenoujohn
![Page 2: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/2.jpg)
Footer
#jenkinsconf
Greetings fellow humans
• Hi, my name is Jonathann • Living in Tel Aviv, Israel • Originally from Marseille, France • Leading the DevOps effort at Ravello Systems • [email protected]
2
![Page 3: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/3.jpg)
#jenkinsconf
Footer
What is Ravello Systems ?
![Page 4: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/4.jpg)
#jenkinsconf
Footer
Ravello Systems
• 30 developers • 4 teams • 7 major projects • Kanban like
4
![Page 5: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/5.jpg)
#jenkinsconf
Footer
A part of our stack
5
![Page 6: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/6.jpg)
#jenkinsconf
Footer
What are we going to talk about ?
6
![Page 7: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/7.jpg)
#jenkinsconf
Footer
Old problems
7
![Page 8: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/8.jpg)
#jenkinsconf
Footer
Jobs Leftovers
• Causes full disk • Workspace Cleanup Plugin • Can pollute your maven repo or npm cache • Solution needed : a new clean environment for every job
8
![Page 9: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/9.jpg)
Footer
#jenkinsconf
Better resource distribution
9
Backend Team Frontend Team
![Page 10: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/10.jpg)
Footer
#jenkinsconf
Better resource distribution
10
Backend Team Frontend Team
![Page 11: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/11.jpg)
Footer
#jenkinsconf
Better resource distribution
11
Backend Team Frontend Team
![Page 12: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/12.jpg)
Footer
#jenkinsconf
Better resource distribution
12
Backend Team Frontend Team
![Page 13: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/13.jpg)
Footer
#jenkinsconf
Better resource distribution
13
Backend Team Frontend Team
“waiting for next available executor …”
![Page 14: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/14.jpg)
Footer
#jenkinsconf
Better resource distribution
14
Backend Team/Frontend Team
![Page 15: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/15.jpg)
Footer
#jenkinsconf
Better resource distribution
15
Backend Team/Frontend Team
![Page 16: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/16.jpg)
Footer
#jenkinsconf
Better resource distribution
16
Backend Team/Frontend Team
![Page 17: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/17.jpg)
Footer
#jenkinsconf
Frontend Team leader’s reaction
17
![Page 18: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/18.jpg)
#jenkinsconf
Footer
Better resource distribution
• Solution needed : a rapid and scalable way to spawn dynamic environments
18
![Page 19: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/19.jpg)
#jenkinsconf
Footer
Better resource distribution
19
![Page 20: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/20.jpg)
#jenkinsconf
Footer
Using Docker in Jenkins
• Using Jenkins Docker plugin • Dedicated Docker host for each team • Hosts machine capabilities are a function of team size and
requirements • All images on all hosts • Problems solved : jobs leftovers and resource distribution
20
![Page 21: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/21.jpg)
#jenkinsconf
Footer
Security
• No active directory or LDAP • Jim uses Dave’s account • Employee leaves the company
21
![Page 22: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/22.jpg)
#jenkinsconf
Footer
Security
22
![Page 23: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/23.jpg)
#jenkinsconf
Footer
Security
23
![Page 24: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/24.jpg)
#jenkinsconf
Footer
Security
• Solution needed: third party dynamic authentication mechanism
24
![Page 25: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/25.jpg)
#jenkinsconf
Footer
Security
• Using Jenkins Google Login plugin • Coupled with Matrix Authorization Strategy plugin • 2-Factor authentication • Open our Jenkins to the world
25
![Page 26: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/26.jpg)
#jenkinsconf
Footer
Too much code in jobs
26
![Page 27: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/27.jpg)
#jenkinsconf
Footer
Too much code in jobs
• Hard to maintain • Can be easily broken • “Who touched this ??”
27
![Page 28: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/28.jpg)
#jenkinsconf
Footer
Too much code in jobs
• Solution: • Put your scripts in revision control (Github) • Using Jenkins Multiple SCM plugin
28
![Page 29: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/29.jpg)
#jenkinsconf
Footer
Jobs Versioning
29
![Page 30: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/30.jpg)
#jenkinsconf
Footer
Jobs Versioning • Solution:
• Using Jenkins JobConfigHistory plugin • Saves old configurations • Diff between job configurations • Revert job configuration
• Coupled with Jenkins SCM sync configuration plugin • Saves you whole Jenkins configuration in revision
control (Github)
30
![Page 31: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/31.jpg)
#jenkinsconf
Footer
Jobs data sharing
31
Backend Job Frontend Job
Parameters: - Backend version: 1.1 - Frontend version: stable
Parameters: - Frontend version: 1.1 - Backend version: stable
![Page 32: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/32.jpg)
#jenkinsconf
Footer
Jobs data sharing
• Solution needed: an interface/service that will store data that will be accessible from all the jobs
• Using Jenkins Global Variable String Parameter Plugin • Cannot alter them from a job • But you can using Groovy …
32
![Page 33: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/33.jpg)
#jenkinsconf
Footer
Jobs data sharing
33
![Page 34: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/34.jpg)
#jenkinsconf
Footer
Jobs data sharing
• Keep a global parameter for each of product stable version • Let all the other jobs use these values • Change the global parameter value of a product stable
version automatically when this product passes a green CI • Gist link: https://gist.github.com/johnyzed/2af71090419af2b20c5a
34
![Page 35: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/35.jpg)
#jenkinsconf
Footer
CI at Ravello Systems
35
build +
unit tests
environment creation
code deployment test suites stop
environments
Artifactory
log collection
![Page 36: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/36.jpg)
#jenkinsconf
Footer
CI at Ravello Systems
36
environment creation
![Page 37: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/37.jpg)
#jenkinsconf
Footer
Before digging further
37
![Page 38: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/38.jpg)
#jenkinsconf
Footer
The pieces are ready • What Do Developers Want ? • Blackbox • No duplication • C.D.R.S • Single form, with a lot of triggers • Using Multijob Plugin
38
![Page 39: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/39.jpg)
#jenkinsconf
Footer
Jobs sandbox • Improving a job without:
• Breaking it • Stopping other people using it
• Solution needed: a parallel Jenkins environment almost identical to our production Jenkins environment to test the new changes on
39
![Page 40: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/40.jpg)
#jenkinsconf
Footer
Jobs sandbox • Jenkins environment blueprint on Ravello Systems • Using Ansible:
• git clone • minor changes
• Running regular jobs • All that orchestrated from a Jenkins job
40
![Page 41: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/41.jpg)
#jenkinsconf
Footer
Jenkins sandbox flow
41
changes on Jenkins script
branch
create dynamic Jenkins
environment run jobs stop dynamic
environment
![Page 42: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/42.jpg)
#jenkinsconf
Footer
Jobs sandbox
42
![Page 43: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/43.jpg)
#jenkinsconf
Footer
Comparing Jenkins environment
43
Old Jenkins New Jenkins
Number of jobs 137 56 Number of slaves 10 0-50 Build and test duration 70 43
![Page 44: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/44.jpg)
#jenkinsconf
Footer
Comparing Jenkins environment
44
![Page 45: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/45.jpg)
#jenkinsconf
Footer
Take-aways
• Ask your developers how to make their life easier • Don’t be afraid to break things • Culture • Follow @jenkins_release • Follow @devopsreactions
45
![Page 46: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/46.jpg)
#jenkinsconf
Footer
Project Handover
46
![Page 47: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/47.jpg)
#jenkinsconf
Footer
One last thing
• Ravello Repo • http://www.ravellosystems.com/repo/ • http://www.ravellosystems.com/repo/blueprints/60391436
47
![Page 48: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/48.jpg)
#jenkinsconf
Footer 48
Questions ?
![Page 49: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/49.jpg)
#jenkinsconf
Footer
Please Share Your Feedback
• Did you find this session valuable? • Please share your thoughts in the
Jenkins User Conference Mobile App. • Find the session in the app and click
on the feedback area.
49
![Page 50: JUC Europe 2015: Optimizing Your CI: Lessons Learned from a Successful Jenkins Rebuild](https://reader033.vdocuments.us/reader033/viewer/2022042723/587b826b1a28ab9d448b564d/html5/thumbnails/50.jpg)
#jenkinsconf
Thank you
50