Git LFS at Autodesk
LARS SCHNEIDER
@kit3bus
@kit3bus
Lars Schneider Autodesk Inc.
Git and Git LFS contributor
Technical Lead forGit at Autodesk
@kit3bus
• Best known for AutoCAD2D and 3D computer-aided design
• 33 years in business
• 4000 engineers, hundreds of products, terabytes of code and asset data
Lars Schneider Autodesk Inc.
@kit3bus
Architecture, Engineering & Construction
Image by Dave Tyner, Autodesk Plant 3D - P&ID
@kit3bus
Manufacturing
@kit3bus
Media & Entertainment
@kit3bus
3D Printing
"Future of Making Things"
Image courtesy of Local Motors Inc.
@kit3bus
Git Git LFS
+
@kit3bus
Why are large files a problem?
All history is local. Good for source files.
Problem for large files.
@kit3bus
Elephant-Video.mp4
Why are large files a problem?
@kit3bus
What files are not suited for Git?
Files that do not compress well and
change frequently....
@kit3bus
Mon Tue Wed
What files are not suited for Git?
Files that do not compress well and
change frequently.
@kit3bus
Example: Git repowith large files
codecode
Git Repo
mastercodecode
301MB
codecode
@kit3bus
Git LFS to the rescue!
git lfs track "*.mp4"
@kit3bus 1 MB
Example: Git LFS repo with large files
master
LFS Server
LFS Ptr
code
Git LFS Repo
LFS Ptr
codecode
codecode LFS Ptr
code
@kit3bus 1 MB
Example: Git LFS repo with large files
master
LFS Server
LFS Ptr
code
LFS Ptr
codecode
codecode LFS Ptr
code
Git LFS Repo
Clo
ne
@kit3bus 1 MB
Example: Git LFS repo with large files
master
LFS Server
LFS Ptr
code
LFS Ptr
codecode
codecode LFS Ptr
code
Git LFS Repo
Clo
ne +
Che
ckou
t
@kit3bus
Technical Details
https://autode.sk/git-lfs-talk
@kit3bus
What do we use Git LFS for?
Integration Test Data(3D Models, ...)
Auxiliary Data(Documentation, Images, Videos, ...)
Build Artifacts(not recommended)
@kit3bus
D E V E L O P E R
M I G R ATO R
A D M I N I S T R ATO R
- Git LFS - What have we learned?
@kit3bus
Migrator
@kit3bus
Migration Process
1 Identify an engineer with deep code knowledge
Create a "demo" migration on Git migration server
Iterate on "demo" migration until repo and CI are OK
Ask broader team to "play" with the "demo" migration
Perform migration on Git production server
2
3
4
5
@kit3bus
How to migrate?
+
git-svn / git-p4 / git-tfs ...
git filter branch / git-lfs-migrate
@kit3bus
How to migrate?
+
git-p4
git-svn / git-p4 / git-tfs ...
git filter branch / git-lfs-migrate
@kit3bus
Git LFSmigration gotchas
Discard large file history
1998 2007 2016
code code code +++
@kit3bus
Git LFSmigration gotchas
master LFS Ptr
code
LFS ServerGit LFS Repo
Avoid "orphaned" LFS files
@kit3bus
Git LFSmigration gotchas
rebasedmaster code
LFS ServerGit LFS Repo
?
Avoid "orphaned" LFS files
@kit3bus
( INCLUDES DESIGNER, TESTER, . . . )
Developer
@kit3bus
Mon Tue Wed
Teach why "Large" files are a problem!
Files that do not compress well and
change frequently.
@kit3bus
Files smaller than 500kb are OK.
Rule of Thumb
Teach why "Large" files are a problem!
@kit3bus
git lfs track "*.png"How to track Git LFS files?
@kit3bus
git lfs track "*.lfs.*"
e.g. /images/elephant.lfs.png
How to track Git LFS files?
@kit3bus
git lfs track "/big/*"
e.g. /big/elephant.png
How to track Git LFS files?
@kit3bus
git lfs track "/xxl.png"How to track Git LFS files?
@kit3bus
Less than 1000 files in LFS are OK.
Rule of Thumb
How to track Git LFS files?
@kit3bus
Less than 1000 files in LFS are OK.
Rule of Thumb
How to track Git LFS files?
https://autode.sk/git-filter
"Git checkout for a test repo with 12k LFS files on Windows takes 57 seconds instead of 55 minutes!"
Git 2.11 + GitLFS 1.5
@kit3bus
git lfs track "*.png"
git lfs track "*.[pP][nN][gG]"
Case sensitive:
Case in-sensitive:
Git LFS Tips & Tricks
@kit3bus
No line ending conversions on
LFS files!Git LFS Tips & Tricks
FirstLineSecondLine
FirstLine▯SecondLineWindowsmacOS
@kit3bus
git lfs clone <URL>Git LFS Tips & Tricks
@kit3bus
Setup your Git credential helper
(or use SSH)!
Git LFS Tips & Tricks
@kit3bus
Use the latestGit / Git LFS
version!
Git LFS Tips & Tricks
@kit3bus
Administrator
@kit3bus
How to make sure Git LFS is used properly?
Setup Git LFS on all dev machines!
Enterprise Config for Githttps://autode.sk/git
LFS Ptr
@kit3bus
How to make sure Git LFS is used properly?
"What happens in Git, stays in Git."
@kit3bus
How to make sure Git LFS is used properly?
Rewriting history can cause a lot of
trouble!
@kit3bus
How to make sure Git LFS is used properly?
Configure file size limit on Git server!
@kit3bus
How to make sure Git LFS is used properly?
Configure file size limit with localGit pre-commit
hooks!
@kit3bus
How to make sure Git LFS is used properly?
Initially, perform Git LFS usage reviews.
@kit3bus
Takeaways
• Git LFS works at scale
• Use the latest Git/Git LFS versions
• Automate Git LFS setuphttps://autode.sk/git
• Use fast clone commandgit lfs clone <URL>
• Use smart tracking patternsgit lfs track "*.lfs.*"
• Reject large files in Git