NICOLA PAOLUCCI • DEVELOPER INSTIGATOR • ATLASSIAN • @DURDN

Advanced Git TechniquesSubtrees, grafting and other fun

Get more out of your version control.

Today we’ll cover some powerful Git goodies

Interactive stagingPainless sub-projects Collating historyInteractive commits are an amazing tool at your disposal as you work on complex code changes

You can use git subtree to handle external libraries in a clean and efficient way

Joining together history of your projects using git replace is useful when migrating to Git

git subtreeExtract project

Alternative to git submodule to handle external dependencies.

Inject dependencyIt allows you to inject an external project into a sub-folder

Introduced in 1.7.11

It can be also used to extract a sub-folder as separate project

Clean integration pointsStores in regular commitsNo training

When and why is git subtree a great choice

Does not require your entire team to be trained in the use of the command

The command stores the external dependency in regular Git commits. Squashing the history optionally.

It makes it easy to see when integrations happened and makes it easy to revert them.

Syntax to inject a project

Command to inject project

git subtree add \ --prefix target-folder \ https://bitbucket.org/team/sub.git \ master --squash

Folder where to insert code

Repository URL

v1.1

Under the hood of git subtree

commit ab54c4e0b75c3107e3e773ab9b39268abddca002Author: Nicola Paolucci <npaolucci@atlassian.com>Date: Tue Sep 29 15:27:35 2015 +0200

Squashed ‘src/sub-project‘ content from commit df563ed git-subtree-dir: src/sub-project git-subtree-split: df563ed15fa6…6b2e95d3

Result of git subtree add

commit 8fb507baf7b270c30c822b27e262d0b44819b4c5Merge: 606cd3e ab54c4eAuthor: Nicola Paolucci <npaolucci@atlassian.com>Date: Tue Sep 29 15:27:35 2015 +0200

Merge commit 'ab54c4e0b75c3107e3e773ab9b39268abddca002' as '.vim/bundle/fireplace'

commit ab54c4e0b75c3107e3e773ab9b39268abddca002Author: Nicola Paolucci <npaolucci@atlassian.com>Date: Tue Sep 29 15:27:35 2015 +0200

Squashed '.vim/bundle/fireplace/' content from commit df563ed git-subtree-dir: .vim/bundle/fireplace git-subtree-split: df563ed15fa685ce2508bf16b3ca7e176b2e95d3

To keep the sub-project up to date

git subtree pull \ --prefix target-folder \ https://bitbucket.org/team/sub.git \ master --squash

Command to pull project

Folder where to insert code

Repository URL

v1.5

Under the hood of git subtree

commit ab54c4e0b75c3107e3e773ab9b39268abddca002Author: Nicola Paolucci <npaolucci@atlassian.com>Date: Tue Sep 29 15:27:35 2015 +0200

Squashed ‘src/sub-project‘ content from commit df563ed git-subtree-dir: src/sub-project git-subtree-split: df563ed15fa6…6b2e95d3

Find the symbolic ref matching a hash (sha-1)

sha-1 of last commit pulled

Git plumbing to list all remote refs

Repository URL

git ls-remote https://bitbucket.org/team/sub.git | grep df563eddf563ed15fa685ce2508bf16b3ca7e176b2e95d3 v1.15eaff1232acedeca565er7e1333234dacccebfff v1.5

git ls-remote https://bitbucket.org/team/sub.git | grep <sha-1>

Aliases to make your life easier!

[alias] sba = "!f() { git subtree add --prefix $2 $1 master --squash; }; f" sbu = "!f() { git subtree pull --prefix $2 $1 master --squash; }; f"

Alias section of your .gitconfig

http://bit.do/git-aliases

How to use the aliases

git sba <repo URL> <destination-folder>

git sba https://bitbucket.org/team/sub.git src/sub

When everyone in the team must work on

sub-projects

When you have constant updates to your dependencies

When you have many dependencies

When NOT to use git subtree

For complex project dependencies Use a dependency tool. Really.

Alternatives? Read my rant on project dependencies

http://bit.do/git-dep

How to extract a projectLet’s learn how to use subtree split

Git subtree to extract a project

Command to split out project

git subtree split \ --prefix my/project/ \ --branch extracted

Folder prefix to extract

where we store it

Push to new remote

We can remove the contents of the folder from the repo

Import extracted branchInitialise a new repo and import the extracted branch

Remove from old repo

After we imported the code we can push it to a new repository

git rm -rf my/project

git initgit pull ../path/ extracted

git remote add origin …git push origin -u master

Interactive commitSplitting commits semantically in flight!

Fine grained control on your commits

Knowing this technique frees you from worrying

about what to do first

Atomic commits make your changes readable

Why split changes interactively?

We modify a single file (README) in 3 parts

# Title

Content of my article

## Subtitle second heading

Some more paragraph content

## Conclusions

Here are your conclusions

We modify a single file in 3 different parts

[:~/p/demo] master(+3/-0) ± git diff@@ -1,11 +1,14 @@ # Title Content of my article+Adding a second line to Title. ## Subtitle second heading Some more paragraph content+Adding another line to Subtitle 1 ## Conclusions Here are your conclusions+Adding to the Conclusions.

To split those changes in separate commits:

git commit --interactive

staged unstaged path 1: unchanged +3/-0 README.md

*** Commands *** 1: status 2: update 3: revert 4: add untracked 5: patch 6: diff 7: quit 8: helpWhat now>

Overview of interactive staging

revert

The status of the files, whether they are staged or unstaged.

updateChoose which files to add to the staging area

status

Undo any action done previously in this session.

Overview of interactive staging [2]

add untracked

Check the unstaged or staged changes in the workspace.

patchAdd parts of a file to the staging area. This allows you to split commits.

diff

Add new untracked files to the repository.

I finally hit “p” for patch, select the file and ENTER:

What now> p staged unstaged path 1: unchanged +3/-0 README.mdPatch update>> 1 staged unstaged path* 1: unchanged +3/-0 README.mdPatch update>>

I finally hit “p” for patch, select the file and ENTER:

@@ -1,11 +1,14 @@ # Title Content of my article+Adding a second line to Title. ## Subtitle second heading Some more paragraph content+Adding another line to Subtitle 1 ## Conclusions Here are your conclusions+Adding to the Conclusions.Stage this hunk [y,n,q,a,d,/,s,e,?]?

Stage this hunk?!?!

A hunk is just a piece of text in a larger file. Diff and patch commands tend to understand changes by clustering them in blocks of continuous text.

“

”

If the default hunk size is too big, you can split it:

Stage this hunk [y,n,q,a,d,/,s,e,?]? sSplit into 3 hunks.@@ -1,7 +1,8 @@ # Title Content of my article+Adding a second line to Title. ## Subtitle second heading Some more paragraph contentStage this hunk [y,n,q,a,d,/,j,J,g,e,?]?

Let’s stage this hunk, or add it to the staging area:

Stage this hunk [y,n,q,a,d,/,j,J,g,e,?]? y@@ -4,8 +5,9 @@ ## Subtitle second heading Some more paragraph content+Adding another line to Subtitle 1 ## Conclusions Here are your conclusionsStage this hunk [y,n,q,a,d,/,K,j,J,g,e,?]?

Now so skip all the rest pressing “q” twice:

The result is a neat new commit and the rest left:

14a0be4 [4 minutes ago] (HEAD -> master) Add content to Title [Nick]

git diffdiff --git i/README.md w/README.mdindex fc26295..0ef85c4 100644--- i/README.md+++ w/README.md@@ -6,7 +6,9 @@ Adding a second line to Title. ## Subtitle second heading Some more paragraph content+Adding another line to Subtitle 1 ## Conclusions Here are your conclusions

When you need help just press “?”Stage this hunk [y,n,q,a,d,/,s,e,?]? ?y - stage this hunkn - do not stage this hunkq - quit; do not stage this hunk or any of the remaining onesa - stage this hunk and all later hunks in the filed - do not stage this hunk or any later hunks in the fileg - select a hunk to go to/ - search for a hunk matching the given regexj - leave this hunk undecided, see next undecided hunkJ - leave this hunk undecided, see next hunkk - leave this hunk undecided, see previous undecided hunkK - leave this hunk undecided, see previous hunks - split the current hunk into smaller hunkse - manually edit the current hunk? - print help

Collating History

Cross VCS mergesUnify two reposFast migration from svn

Why collate history?

Migrate immediately from Subversion, attach the earlier history after the migration.

You have two repositories that should actually be only one.

You need to perform Subversion merges in a Git branch

One relevant example: Linux kernel

Today use git replace

The entire history of the Linux kernel is split over three different repos.

Originally in GraftsWhich are local pair of ids connecting a commit id (SHA-1) to the next

Linux kernel is split

Available in the stock git distribution since version 1.6.5

git replace is capable to replace any object with any other object. It tracks these swaps via refs which you can push and pull between repositories.

“

”

git replace in practice

shallow

first

last

legacy

shallow clone with cut history

git replace first last

First commit of restarted repo

git checkout -b shallow origin/shallowgit log --max-parents=0 master

git tag 56eacf first -m”Tag… commit”

shallow

first

Last commit of legacy repo

git checkout -b legacy origin/legacygit rev-parse --verify legacy

git tag 84abb last -m”Tag… commit”

last

legacy

git replace in practice

shallow

first

last

legacy

shallow clone with cut history

git replace first last

Replacements are persisted

cat .git/refs/replace/56eac…7cc84abb39d9aab234dfba2e41f13f693fa5edbfe22

git push origin ‘refs/replace/*’

If you want to make the git replace changes permanent and free the team from pulling those refs, do a final pass usinggit filter-branch

“

”

Go forth and enrich your Git experience

Thank you!

NICOLA PAOLUCCI • DEVELOPER INSTIGATOR • ATLASSIAN • @DURDN

Twitter: @durdn

• GIF backdrops taken from giphy.com • Icons from thenounproject.com credits below

Image credits