going literate in amadeus juc berlin june 25th 2014
DESCRIPTION
Everything that you need to know about the Literate plugin for Jenkins, presented during the JUC Berlin 2014.TRANSCRIPT
![Page 1: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/1.jpg)
Jenkins User Conference Europe #jenkinsconf
Vincent LatombeAmadeus IT Grouphttp://www.amadeus.com
June 25, 2014
Going Literate in Amadeus
#jenkinsconf
![Page 2: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/2.jpg)
Jenkins User Conference Europe #jenkinsconf
About me
Vincent Latombe a.k.a vlatombeDeveloper advocate at Amadeus
Used to build for Java/JEE env., now also C++/Python/RubyHeavy Jenkins user since 2010Maintainer of the Clearcase pluginContributes to core (windows fixes), git and literate
![Page 3: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/3.jpg)
Jenkins User Conference Europe #jenkinsconf
Agenda
What is Literate ?Literate in PracticeLiterate in AmadeusUnder the hood
![Page 4: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/4.jpg)
Jenkins User Conference Europe #jenkinsconf
WHAT IS LITERATE?Part I
WTF IS LITERATE???
![Page 5: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/5.jpg)
Jenkins User Conference Europe #jenkinsconf
Literate Programming
• Introduced by Donald Knuth in 1983
• Explain program logic using natural language with snippets of code and macros
![Page 6: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/6.jpg)
Jenkins User Conference Europe #jenkinsconf
Literate Builds
• Applies Literate paradigm to builds
• Describes the build steps in a marker file or simply in README.md
• Build definition stored in the same SCM as the code it is building
![Page 7: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/7.jpg)
Jenkins User Conference Europe #jenkinsconf
Literate Builds
• Marker file within the SCM
• May contain the build definition
• Fallback to README.md if empty
![Page 8: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/8.jpg)
Jenkins User Conference Europe #jenkinsconf
Markdown Cheatsheet
A section=========This is text inside section.And `this is inline code`.
some code
![Page 9: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/9.jpg)
Jenkins User Conference Europe #jenkinsconf
Hello world exampleHello world literate project============================
Build-----
Let's say hello
echo "Hello world"
![Page 10: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/10.jpg)
Jenkins User Conference Europe #jenkinsconf
Ant example
Environments===========Notice how we specify the environment to build with by providing Jenkins node labels or tool installer names in code snippet sections attached to bullet points in an "environment" section?
* `ant-1.8`, `java-1.7`
Build=====
ant clean dist
![Page 11: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/11.jpg)
Jenkins User Conference Europe #jenkinsconf
Matrix build? No problem
Complex project===============
Environments===========
We have two different environments that the build must be run on:
* `linux`, `gcc-4.2`, `ant-1.8`, `maven-3.0.5`, `java-1.7`* `windows`, `vs-pro-2012`, `ant-1.8`, `maven-3.0.5`, `java-1.7`
![Page 12: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/12.jpg)
Jenkins User Conference Europe #jenkinsconf
Build commands per environment
Build=====
We have two different sets of build instructions, one for building with visual studio and the other for building with GCC
* On `gcc-4.2`, we start by building the native code
./configure make
* On `vs-pro-2012`, we have a batch file to do the native steps
call build-native.bat
![Page 13: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/13.jpg)
Jenkins User Conference Europe #jenkinsconf
Additional tasks
• Defined in the marker file• Enabled per branch using the job
configuration• Lightweight promotion– Self-promotion– Manual promotion (with optional parameters)
![Page 14: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/14.jpg)
Jenkins User Conference Europe #jenkinsconf
Extensions
• Files stored in SCM under .jenkins folder• Basic: <extensionName>.xml, contains
the config snippet• Implement Agent to expose higher-level
configuration files
![Page 15: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/15.jpg)
Jenkins User Conference Europe #jenkinsconf
LITERATE IN PRACTICEPart II
![Page 16: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/16.jpg)
Jenkins User Conference Europe #jenkinsconf
DEMO TIME !
![Page 17: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/17.jpg)
Jenkins User Conference Europe #jenkinsconf
LITERATE IN AMADEUSPart III
![Page 18: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/18.jpg)
Jenkins User Conference Europe #jenkinsconf
Our context
• About 3000 developers• Commercial products– ~1000 Java/JEE– ~2000 C++
• For internal tooling, add– Python– Ruby– Perl
![Page 19: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/19.jpg)
Jenkins User Conference Europe #jenkinsconf
It’s about control!
• Fully open• Locked down• Balanced– Template-based solutions: only defined
attributes can be customized– Literate: commands can be customized, post-
build actions can be whitelisted
![Page 20: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/20.jpg)
Jenkins User Conference Europe #jenkinsconf
Infrastructure
Stash Jenkins
1.Enable CI
2.Creates
3.Index branches
4.Parses marker
Literate Multi-branch project
Literate branch project
Project/Repository
Literate env.
project
Literate env.
project
5.Build status
![Page 21: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/21.jpg)
Jenkins User Conference Europe #jenkinsconf
YAML Parser
• Markdown considered too textual, too much space for syntax errors, even if using README.md was tempting
• We have some YAML lovers internally ;-)• Easy to implement thanks to existing
Literate architecture
![Page 22: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/22.jpg)
Jenkins User Conference Europe #jenkinsconf
Markdown vs YAMLEnvironments============
* `windows`, `java-1.6`* `linux`, `java-1.7`
Build=====
mvn –B clean verify
environments: - [windows, java-1.6] - [linux, java-1.7]build: mvn –B clean verify
![Page 23: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/23.jpg)
Jenkins User Conference Europe #jenkinsconf
UNDER THE HOODPart IV
![Page 24: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/24.jpg)
Jenkins User Conference Europe #jenkinsconf
Architecture
Literate-plugin
Literate-api Branch-api
Scm-api
![Page 25: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/25.jpg)
Jenkins User Conference Europe #jenkinsconf
Literate-api
• Doesn’t depend on Jenkins• Pluggable model builders (Markdown,
YAML, …)
![Page 26: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/26.jpg)
Jenkins User Conference Europe #jenkinsconf
Scm-API
• jenkins.scm.api.SCMSource : provider for hudson.scm.SCM instances
• Able to comprehend multiple heads on a SCM
![Page 27: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/27.jpg)
Jenkins User Conference Europe #jenkinsconf
Branch-api
• Toolkit to handle multi-branch– Dead branch policy– Untrusted branches– Build retention– Throttling
• Foundation for– Multi-branch freestyle project– Multi-branch template project
![Page 28: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/28.jpg)
Jenkins User Conference Europe #jenkinsconf
Literate-plugin
• New extension points– LiterateBranchProperty : decorate branch and
environments with anything (buildwrapper, properties…)
– Agent : Builds Publisher instances from files in the SCM.
![Page 29: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/29.jpg)
Jenkins User Conference Europe #jenkinsconf
Towards a release
• Develop the ecosystem– More plugins integration– Github/Bitbucket/Stash pull request support
• Integration with isolation features– Sandbox commands within a container
![Page 30: Going literate in Amadeus JUC Berlin June 25th 2014](https://reader034.vdocuments.us/reader034/viewer/2022052618/554bc102b4c90594278b515c/html5/thumbnails/30.jpg)
Jenkins User Conference Europe #jenkinsconf
Corporate Community
Thank You To Our Sponsors
Platinum Gold
Silver