managing dependencies at build time
Post on 11-May-2015
2.219 Views
Preview:
TRANSCRIPT
Build Time Dependencies
Build Dependencies
Part of a wider dependency challenge
Develop / Build RuntimeIn
fras
truc
ture
source
3rd Party libs
Internal Libs
Config templates
VM ImagesEnvironmentTemplates
Deployment Manifests
Deployable Builds
EnvironmentManifests
Sub-Projects
Middleware
“Cod
e”
Build Tools
Today’s Focus
Libraries
Sub-Projects
Interface Definitions
Reusing compiled code
4
The plan for today
The challenge
Basics of a dependency solution
Best practices for CI at scale
Managing 3rd party dependencies
5
The plan for today
The challenge
Basics of a dependency solution
Best practices for CI at scale
Managing 3rd party dependencies
6
Why not source level reuse?
Release / Reuse Equivalency
7
Why not source level reuse?
Changing Project A breaks Project B
Release / Reuse Equivalency
8
Why not source level reuse?
Release / Reuse Equivalency
9
Why not source level reuse?
Release / Reuse Equivalency
A bug fix in Project A never gets to Project B
New features have to be done repeatedly
10
Why not source level reuse?
Release / Reuse Equivalency
11
Why not source level reuse?
Use versioned copies of C- Bugs found in C can be tracked and fixed
Release / Reuse Equivalency
http://www.urbancode.com/html/resources/articles/reuse-maturity-model.html
“The granule of reuse is the granule of release. Only components that are released through a tracking
system can be effectively reused.”
- Robert C. Martin’s C++ Report 1997
Dependency Graph
Represent relationship visually
13
Dependency graphs
Complex, rarely clean, and important
14
The Challenge
Successfully build the app
Easily provide new versions to dependents
Rebuild when my dependency changes
Get a BOM
Manage 3rd party libs
15
The plan for today
The challenge
Basics of a dependency solution
Best practices for CI at scale
Managing 3rd party dependencies
16
Elements of a solution
1.A description of our dependencies (rules)
2.A repository of versioned components
3.Automated retrieval at build time
4.A manifest of what was retrieved
5.Intelligent build triggering for CI
17
Elements of a solution#1 - A description of our dependencies (rules)
Project?
Version?
What if something goes wrong?
Subset of files?
18
Elements of a solution
Authoritative Source
Tamper Resistant
Access Control
Release Meta-Data
Retention Policies
#2 - A repository of versioned components
19
Elements of a solution
Authoritative Source
Tamper Resistant
Access Control
Release Meta-Data
Retention Policies
#2 - A repository of versioned components
Examples:• Maven Repos:• (Artifactory, Nexus…)
• CodeStation
• NuGet
• Ivy
20
Repository Failure Patterns
Binary dependencies are versioned with the source code
It’s in the “lib” directory
Repository Failure Patterns
Using a public repository
Don’t mind me, I’m just waiting for the whole internet to
download…
22
Elements of a solution
Pulling the files down
Support various build types- On the developer’s desktop
- In the authoritative build environment
04/12/2023 UrbanCode Inc. Proprietary and Confidential ©2012
#3 - Automated retrieval at build time
23
Elements of a solution
What dependencies at what version
#4 - A manifest of what was retrieved
http://mvnrepository.com/artifact/com.sun.jersey/jersey-bundle/1.16
24
Elements of a solution
…Harder than it looks
#5 - Intelligent build triggering for CI
25
5 Elements of a solution
1.A description of our dependencies (rules)
2.A repository of versioned components
3.Automated retrieval at build time
4.A manifest of what was retrieved
5.Intelligent build triggering for CI
26
The plan for today
The challenge
Basics of a dependency solution
Best practices for CI at scale
Managing 3rd party dependencies
Best Practices for CI at Scale
Early in dev cycle rules should point to “latest”
- Bill of Materials must still know what the latest is
When nearing release, lock versions
In maintenance: sub-projects are branched as necessary using main project numbering
#1 – Dynamic rules early, fixed later
Best Practices for CI at Scale
Push builds from the bottom of the graph
Best Practices for CI at Scale
Use multiple build machines
Independent builds run in parallel
Distribute the work
Best Practices for CI at Scale
Big graphs can take a while to build- Introduce race conditions as new changes
come in
Consistent sets:- Fetch code from same date / time- Fetch from a snapshot / label / baseline
Use a consistent set of code
31
The plan for today
The challenge
Basics of a dependency solution
Best practices for CI at scale
Managing 3rd party dependencies
Managing 3rd party libraries
Do we really need another XML parser?- Be suspicious of new libraries
Don’t reference external repositories
Is that the real version?- Require some seniority to load a new versions
Lifecycle libraries- Flag them as under test, approved, or
deprecated as your repo allows
General concerns
Managing 3rd party libraries
Some open source licenses are risky
Involve legal without wishing you didn’t- Get approval for (versioned) licenses, not
specific libs
- Make approved & rejected licenses well known
Open source and licensing
No, I’m not a lawyer
Managing 3rd party libraries
Compliance with licensing agreements: tricky
Build a reverse BOM- What is this library used by?- Consider restricting permissions to it
Commercial licensing
No, I’m not a lawyer
Key take-aways
Build time dependencies require- Having the files- Knowing the dependency rules- Resolving those dependency rules- Recording a bill of materials
CI exposes subtle challenges with dependencies
3rd Party libraries require special care
Reference Material
Urbancode.com/resources
Stay in touch:Blogs.urbancode.comTwitter.com/UrbanCodeSoft Twitter.com/EricMinickSlideshare.net/Urbancode
Yes, we’re a products company
uBuild - Build automation and CI that scales- Integrated dependency repository- Integrated dependency definitions
uDeploy- Deployment and release management
uRelease- Release & Environment management /
planning
Q&A
Contact Eric Minick
eminick@usa.ibm.com @UrbanCodeSoft @EricMinick
www.urbancode.com
top related