will my patch make it? and how fast?
DESCRIPTION
The Linux kernel follows an extremely distributed reviewing and integration process supported by 130 developer mailing lists and a hierarchy of dozens of Git repositories for version control. Since not every patch can make it and of those that do, some patches require a lot more reviewing and integra- tion effort than others, developers, reviewers and integrators need support for estimating which patches are worthwhile to spend effort on and which ones do not stand a chance. This paper cross- links and analyzes eight years of patch reviews from the kernel mailing lists and committed patches from the Git repository to understand which patches are accepted and how long it takes those patches to get to the end user. We found that 33% of the patches makes it into a Linux release, and that most of them need 3 to 6 months for this. Furthermore, that patches developed by more experienced developers are more easily accepted and faster reviewed and integrated. Additionally, reviewing time is impacted by submission time, the number of affected subsystems by the patch and the number of requested reviewers.TRANSCRIPT
Yujuan Jiang, Bram Adams (MCIS, Polytechnique Montréal) Daniel M. German (University of Victoria )
Will My Patch Make It? And How Fast?
11Sunday, 19 May, 13
I do hold out hope that Google does come around and works to fix their codebase to get it merged upstream to stop the huge blockage that they have now caused in a large number of embedded Linux hardware companies […] But I need the help of the Google developers to make it happen, without them, nothing can change.
http://www.kroah.com/log/linux/android-kernel-problems.html2
GregKroah-Hartman
2Sunday, 19 May, 13
Integration Process
33Sunday, 19 May, 13
Integration Process
3
Reviewing Integration Staging
3Sunday, 19 May, 13
Integration Process
3
Reviewing Integration Staging
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
Reviewing Integration Staging
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
Reviewing Integration Staging
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
linux-usb
linux-scsi
lkml
Reviewing Integration Staging
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
linux-usb
linux-scsi
lkml
Reviewing Integration Staging
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
linux-usb
linux-scsi
lkml
Reviewing Integration Staging
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
linux-usb
linux-scsi
lkml
Reviewing Integration Staging
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
linux-usb
linux-scsi
lkml
subsystemmaintainer1
subsystemmaintainer1
Reviewing Integration Staging
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
linux-usb
linux-scsi
lkml
subsystemmaintainer1
subsystemmaintainer1
Reviewing Integration Staging
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
linux-usb
linux-scsi
lkml
subsystemmaintainer1
subsystemmaintainer1
Reviewing Integration Staging
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
linux-usb
linux-scsi
lkml
subsystemmaintainer1
subsystemmaintainer1
Reviewing Integration Staging
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
linux-usb
linux-scsi
lkml
subsystemmaintainer1
subsystemmaintainer1
Reviewing Integration Staging
maintainer
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
linux-usb
linux-scsi
lkml
subsystemmaintainer1
subsystemmaintainer1
Reviewing Integration Staging
maintainer
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
linux-usb
linux-scsi
lkml
subsystemmaintainer1
subsystemmaintainer1
Reviewing Integration Staging
maintainer Linus Torvalds
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
linux-usb
linux-scsi
lkml
subsystemmaintainer1
subsystemmaintainer1
Reviewing Integration Staging
maintainer Linus Torvalds
3Sunday, 19 May, 13
Integration Process
3
contributor 1
contributor 2
contributor 3
linux-usb
linux-scsi
lkml
linux 3.5
subsystemmaintainer1
subsystemmaintainer1
Reviewing Integration Staging
maintainer Linus Torvalds
3Sunday, 19 May, 13
Research Questions
4
RQ1: How manypatches are
merged?
RQ2: What kind of
patch is merged more likely?
RQ 3: What kind of
patch is accepted faster?
4Sunday, 19 May, 13
Setup Of Case Study
5
Reviewing Integration Staging
Linus Torvalds
linux-usb
linux-scsi
lkml
linux 3.5
contributor 1
contributor 2
contributor 3
subsystemmaintainer1
subsystemmaintainer1
maintainer
5Sunday, 19 May, 13
Setup Of Case Study
5
Reviewing Integration Staging
Linus Torvalds
linux-usb
linux-scsi
lkml
linux 3.5
contributor 1
contributor 2
contributor 3
subsystemmaintainer1
subsystemmaintainer1
maintainer
5Sunday, 19 May, 13
Setup Of Case Study
5
Reviewing Integration Staging
Linus Torvalds
linux-usb
linux-scsi
lkml
5Sunday, 19 May, 13
Setup Of Case Study
5
Reviewing Integration Staging
Linus Torvalds
linux-usb
linux-scsi
lkml
5Sunday, 19 May, 13
Setup Of Case Study
5
Reviewing Integration Staging
Linus Torvalds
5Sunday, 19 May, 13
Setup Of Case Study
5
Reviewing Integration Staging
Linus Torvalds
email1
email3
email2 email patch2
email patch1
email patch3
...
5Sunday, 19 May, 13
Setup Of Case Study
5
Reviewing Integration Staging
email1
email3
email2 email patch2
email patch1
email patch3
...
5Sunday, 19 May, 13
Setup Of Case Study
5
Reviewing Integration Staging
email1
email3
email2 email patch2
email patch1
email patch3
... commit3
commit2
commit1commit patch1
commit patch2
commit patch3
...
5Sunday, 19 May, 13
Setup Of Case Study
5
Reviewing Integration Staging
email1
email3
email2 email patch2
email patch1
email patch3
... commit3
commit2
commit1commit patch1
commit patch2
commit patch3
...
checksum1
checksum3
checksum2
...
5Sunday, 19 May, 13
Setup Of Case Study
5
Reviewing Integration Staging
email1
email3
email2 email patch2
email patch1
email patch3
... commit3
commit2
commit1commit patch1
commit patch2
commit patch3
...
checksum1
checksum3
checksum2
...
5Sunday, 19 May, 13
Experience
ReviewExperience
Patch
Experience
Commit
6
5 Dimensions of 29 Patch Metrics
6Sunday, 19 May, 13
size: LOC > 50
Number of reviewers > 3 ?
not accepted Number of review messages > 3 ?
Is this first patch in thread?
not acceptedaccepted
Decision Tree
Building Decision Trees
77Sunday, 19 May, 13
8
RQ1: How manypatches are
merged?
RQ2: What kind of
patch is merged more likely?
RQ 3: What kind of
patch is accepted faster?
8Sunday, 19 May, 13
9
RQ1: How manypatches are
merged?
RQ2: What kind of
patch is merged more likely?
RQ 3: What kind of
patch is accepted faster?
9Sunday, 19 May, 13
RQ1:33% of patches make it!
102005 2006 2007 2008 2009 2010 2011 2012
accepted/rejected patches
year
perc
enta
ge o
f pat
ches
0
20000
40000
60000
80000
100000
120000
28.6328.7
27.03
32.83 32.79 33.8733.55 30.74
71.37
71.3
72.97
67.1767.21 66.13
66.45
69.26
% accepted by linus% rejected by linus
# o
f p
atch
es
72.97%
67.17%
71.3%
71.73%
69.26%
66.45%
66.13%67.21%
28.63%
28.7%
32.79%32.83%
27.03%
30.74%33.55%33.87%
ACCEPT
REJECT
10Sunday, 19 May, 13
RQ1:Requiring 1~6months!
11
2005 2006 2007 2008 2009 2010 2011 2012
year
perc
enta
ge o
f acc
epte
d pa
tche
s of
eac
h ye
ar
020
4060
80
instantlywithin_hourwithin_day
within_weekwithin_monthwithin_quarter
within_half_yearwithin_yeartook_ages
Text
% acceptedpatches
11Sunday, 19 May, 13
RQ1: reviewing time speeds up & integration slows down
12reviewing time integration time
12Sunday, 19 May, 13
13
RQ1: How manypatches are
merged?
RQ2: What kind of
patch is merged more likely?
RQ 3: What kind of
patch is accepted faster?
13Sunday, 19 May, 13
RQ2: What kind of patch is merged more likely?
precision:73% recall:68.47%
1414Sunday, 19 May, 13
RQ2: What kind of patch is merged more likely?
precision:73% recall:68.47%
1414Sunday, 19 May, 13
RQ2: What kind of patch is merged more likely?
precision:73% recall:68.47%
1414Sunday, 19 May, 13
RQ2: What kind of patch is merged more likely?
precision:73% recall:68.47%
1414Sunday, 19 May, 13
RQ2: What kind of patch is merged more likely?
precision:73% recall:68.47%
1414Sunday, 19 May, 13
15
RQ1: How manypatches are
merged?
RQ2: What kind of
patch is merged more likely?
RQ 3: What kind of
patch is accepted faster?
15Sunday, 19 May, 13
RQ3: What kind of patch is accepted faster?
1616Sunday, 19 May, 13
RQ3: What kind of patch is accepted faster?
1616Sunday, 19 May, 13
RQ3: What kind of patch is accepted faster?
1616Sunday, 19 May, 13
RQ3: What kind of patch is accepted faster?
1616Sunday, 19 May, 13
RQ3: What kind of patch is accepted faster?
1616Sunday, 19 May, 13
RQ3: What kind of patch is accepted faster?
1616Sunday, 19 May, 13
RQ3: What kind of patch is accepted faster?
16
Acceptance is determined by
integration phase
16Sunday, 19 May, 13
1717Sunday, 19 May, 13
1717Sunday, 19 May, 13
2005 2006 2007 2008 2009 2010 2011 2012
accepted/rejected patches
year
perc
enta
ge o
f pat
ches
0
20000
40000
60000
80000
100000
120000
28.6328.7
27.03
32.83 32.79 33.8733.55 30.74
71.37
71.3
72.97
67.1767.21 66.13
66.45
69.26
% accepted by linus% rejected by linus
72.97%
67.17%
71.3%
71.73%
66.45%
66.13%67.21%
28.63%28.7%
32.79%32.83%
27.03%
30.74%33.55%
33.87%
1717Sunday, 19 May, 13
2005 2006 2007 2008 2009 2010 2011 2012
accepted/rejected patches
year
perc
enta
ge o
f pat
ches
0
20000
40000
60000
80000
100000
120000
28.6328.7
27.03
32.83 32.79 33.8733.55 30.74
71.37
71.3
72.97
67.1767.21 66.13
66.45
69.26
% accepted by linus% rejected by linus
72.97%
67.17%
71.3%
71.73%
66.45%
66.13%67.21%
28.63%28.7%
32.79%32.83%
27.03%
30.74%33.55%
33.87%
1717Sunday, 19 May, 13
2005 2006 2007 2008 2009 2010 2011 2012
accepted/rejected patches
year
perc
enta
ge o
f pat
ches
0
20000
40000
60000
80000
100000
120000
28.6328.7
27.03
32.83 32.79 33.8733.55 30.74
71.37
71.3
72.97
67.1767.21 66.13
66.45
69.26
% accepted by linus% rejected by linus
72.97%
67.17%
71.3%
71.73%
66.45%
66.13%67.21%
28.63%28.7%
32.79%32.83%
27.03%
30.74%33.55%
33.87%
1717Sunday, 19 May, 13