beyond counting: new perspectives on the active ipv4
TRANSCRIPT
![Page 1: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/1.jpg)
Beyond Counting:New Perspectives on the Active IPv4 Address Space
Philipp Richter Georgios Smaragdakis David Plonka Arthur BergerTU Berlin MIT Akamai Akamai/MIT
@IETF 96 Berlin (maprg) July 2016
work under submission comments highly appreciated!
preprint: http://arxiv.org/abs/1606.00360
![Page 2: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/2.jpg)
Philipp Richter | INET / TU Berlin 1
IPv4 Address Space Exhaustion
IPv4 Standard
1981
RIR Framework
InitiationFirst RIR (RIPE)
founded
1992
APNICexhausted
2011
RIPEexhausted
2012
ARINexhausted
2015
Early Registration Needs-Based Provision Depletion & Exhaustion
2005
Last RIR(AFRINIC)founded
LACNICexhausted
2014 /8
equ
ivale
nts
● ●●
●●
●●
●
●
●
●
●
●
●●
●●
0
50
100
150
200
250
Nov 1997 Jan 2002 Jan 2006 Jan 2010 Jan 2014
routable address space limit (220.7 /8 equivalents)
total address space limit (256 /8 equivalents)
● allocated address blocksrouted address blocks
Figures: P. Richter, M. Allman, R. Bush, V. Paxson: A Primer on IPv4 Scarcity, ACM CCR 45(2), 2015.
• IPv4 has been around for ~35 years • Theoretically routable IP addresses: 3.7B, ~2.8B routed • IANA exhausted its address pool in 2011 • Today: Less than 2% of the IPv4 address space “free”
http://arxiv.org/abs/1606.00360
![Page 3: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/3.jpg)
Operators' Community Efforts
Efforts in the IETF community: • IPv6 transition mechanisms • IPv4 multiplexing/sharing mechanisms (e.g., EnIP, A+P) • Efforts to conserve IPv4 address space
IANA/Regional Internet Registries: • Establishment of address transfer policies • Incentives for increasing address space utilization
e.g., draft-fleischhauer-ipv4-addr-saving-05, RFC6346, draft-chimiak-enhanced-ipv4-03
Philipp Richter | INET / TU Berlin 2http://arxiv.org/abs/1606.00360
![Page 4: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/4.jpg)
Academic Community Efforts
• Measurements to understand “where we are” right now • Internet-wide: Number of actively used IPv4 addresses:
“1.2B IP addresses in use in 2014”, statistical estimation
“5.3M /24 address blocks in use in 2013”, passive+active measurement
• Challenge: No single vantage point captures all activity
Philipp Richter | INET / TU Berlin 3http://arxiv.org/abs/1606.00360
Zander et al., IMC ‘14
Dainotti et al., JSAC ‘16
![Page 5: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/5.jpg)
Academic Community Efforts
• Measurements to understand “where we are” right now • Internet-wide: Number of actively used IPv4 addresses:
“1.2B IP addresses in use in 2014”, statistical estimation
“5.3M /24 address blocks in use in 2013”, passive+active measurement
• Challenge: No single vantage point captures all activity
Philipp Richter | INET / TU Berlin 3http://arxiv.org/abs/1606.00360
Zander et al., IMC ‘14
Dainotti et al., JSAC ‘16
What can we say from our CDN’s perspective?Can we do more than counting active IP addresses?
![Page 6: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/6.jpg)
The CDN as an Observatory
CDN front-end servers
HTTP(S) requests
• 200,000+ servers • 3 trillion requests per day • CDN logs: number of requests per IP per day
Totals for the entirety of 2015: • 1.2B active IPv4 addresses (42% of routed) • 6.5M active /24 address blocks (59% of routed)
Philipp Richter | INET / TU Berlin 4http://arxiv.org/abs/1606.00360
Visibility: CDN logs vs. ICMP scan (ZMap project, 8 snapshots)
% IPv4 addresses visibly active (N=950M, Oct. 2015)
CD
N o
nly
CD
N &
ICM
P
ICM
P on
ly
0 20 40 60 80 100
![Page 7: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/7.jpg)
Peak IPv4?
●●●●●●●
●●●●●●●
●●●●●
●●●●●●●
●●●●●
●●●●●●●
●●●●●●●●●●●●●
●●●●
●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●
date [ticks: January of each year]
uniq
ue IP
v4 a
ddre
sses
200M
400M
600M
800M
1B
2008 2009 2010 2011 2012 2013 2014 2015 2016
● unique active IPv4 addresses per monthlinear regression until 2014−01
●
IANA exhaustion●
RIPE exhaustion
●
ARIN exhaustion
●
APNIC exhaustion●
LACNIC exhaustion
Active IPv4 address counts have stagnated since 2014
Philipp Richter | INET / TU Berlin 5http://arxiv.org/abs/1606.00360
![Page 8: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/8.jpg)
Daily IPv4 Activity and Churn
●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●
●●●●●●●
●●●●●●●
●●●●●●●
●●
●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●
●●●●
●
●●
days from 2015−08−17 to 2015−12−06
uniq
ue IP
v4 a
ddre
sses
020
0M40
0M60
0M
0 14 28 42 56 70 84 98 112
● active IPv4 addressesup eventsdown events
Philipp Richter | INET / TU Berlin 6http://arxiv.org/abs/1606.00360
![Page 9: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/9.jpg)
Churn on all Timescales
day-to-day: ~7% come, ~7% go
week-to-week: ~5% come, ~5% go
month-to-month: ~5% come, ~5% go
The number of active IPv4 addresses stays constantthe set of active addresses varies on all timescales
Philipp Richter | INET / TU Berlin 7http://arxiv.org/abs/1606.00360
![Page 10: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/10.jpg)
Long-term Effect of Address Churn
time lag from 2015−01−01
chan
ge in
act
ive IP
v4 a
ddre
sses
1 week 26 weeks 52 weeks−200
M−1
00M
010
0M20
0M
appear
−25%
−12.
5%0
12.5
%25
%
disappear
Over the course of one year, 25% of the active IP address pool changed
Philipp Richter | INET / TU Berlin 8http://arxiv.org/abs/1606.00360
![Page 11: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/11.jpg)
Address Activity Matrix
130.149.0.6 130.149.0.5 130.149.0.4 130.149.0.3 130.149.0.2 130.149.0.1
…
…addr
ess
spac
e
days
for each day on which an IP address was active (requested content), we draw a red dot.
Philipp Richter | INET / TU Berlin 9http://arxiv.org/abs/1606.00360
![Page 12: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/12.jpg)
Patterns: “In situ” Address Activity
time [months]
IP a
ddre
ss a
ctiv
ity w
ithin
/24
0 1 2 3 4
.0.1
27.2
55
time [months]IP
add
ress
act
ivity
with
in /2
40 1 2 3 4
.0.1
27.2
55
time [months]
IP a
ddre
ss a
ctiv
ity w
ithin
/24
0 1 2 3 4
.0.1
27.2
55
time [months]
IP a
ddre
ss a
ctiv
ity w
ithin
/24
0 1 2 3 4
.0.1
27.2
55static block DE University DHCP pool US University
residential users US ISP residential users DE ISP
“in situ” activity:address assignment practice
+user behavior
(no visible modification of address assignment practice)
Philipp Richter | INET / TU Berlin 10http://arxiv.org/abs/1606.00360
![Page 13: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/13.jpg)
Patterns: Operational Change
time [months]
IP a
ddre
ss a
ctiv
ity w
ithin
/24
0 1 2 3 4
.0.1
27.2
55
time [months]
IP a
ddre
ss a
ctiv
ity w
ithin
/24
0 1 2 3 4
.0.1
27.2
55
DE University DE University
Philipp Richter | INET / TU Berlin 11http://arxiv.org/abs/1606.00360
![Page 14: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/14.jpg)
Activity Matrix at Scale
Philipp Richter | INET / TU Berlin 12http://arxiv.org/abs/1606.00360
20k adjacent IP addresses (in active /24s), University Network
![Page 15: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/15.jpg)
Metric 1: Filling Degree per /24
Number of active IP addresses per /24 [1…256]
time [months]
IP a
ddre
ss a
ctiv
ity w
ithin
/24
0 1 2 3 4
.0.1
27.2
55
rather low (degree = 29)
time [months]
IP a
ddre
ss a
ctiv
ity w
ithin
/24
0 1 2 3 4
.0.1
27.2
55
Philipp Richter | INET / TU Berlin 13http://arxiv.org/abs/1606.00360
high (degree = 254)
![Page 16: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/16.jpg)
Metric 1: Filling Degree per /24
Philipp Richter | INET / TU Berlin 14http://arxiv.org/abs/1606.00360
active IP addresses within /24
CD
F: a
ctive
/24
bloc
ks
1 64 128 192 256
0.0
0.2
0.4
0.6
0.8
1.0
![Page 17: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/17.jpg)
Metric 1: Filling Degree per /24
Philipp Richter | INET / TU Berlin 14http://arxiv.org/abs/1606.00360
active IP addresses within /24
CD
F: a
ctive
/24
bloc
ks
1 64 128 192 256
0.0
0.2
0.4
0.6
0.8
1.0
only less than 50% of all active /24 blocks have
filling degree > 250
![Page 18: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/18.jpg)
Addressing: Static vs. Dynamic
Philipp Richter | INET / TU Berlin 14http://arxiv.org/abs/1606.00360
active IP addresses within /24
CD
F: a
ctive
/24
bloc
ks
1 64 128 192 256
0.0
0.2
0.4
0.6
0.8
1.0
static all dynamic
• We tagged likely static/dynamic blocks using PTR records • We identified 262K static blocks and 456K dynamic blocks
![Page 19: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/19.jpg)
Addressing: Static vs. Dynamic
Philipp Richter | INET / TU Berlin 14http://arxiv.org/abs/1606.00360
active IP addresses within /24
CD
F: a
ctive
/24
bloc
ks
1 64 128 192 256
0.0
0.2
0.4
0.6
0.8
1.0
static all dynamic
• We tagged likely static/dynamic blocks using PTR records • We identified 262K static blocks and 456K dynamic blocks
more than 70%of “static”-tagged blocks have filling degree < 64
![Page 20: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/20.jpg)
Addressing: Static vs. Dynamic
Philipp Richter | INET / TU Berlin 14http://arxiv.org/abs/1606.00360
active IP addresses within /24
CD
F: a
ctive
/24
bloc
ks
1 64 128 192 256
0.0
0.2
0.4
0.6
0.8
1.0
static all dynamic
• We tagged likely static/dynamic blocks using PTR records • We identified 262K static blocks and 456K dynamic blocks
more than 70%of “static”-tagged blocks have filling degree < 64
more than 80% of “dynamic”-tagged blocks have filling degree > 250
![Page 21: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/21.jpg)
Metric 2: Spatio-temporal Utilization
low utilization (18%)time [months]
IP a
ddre
ss a
ctiv
ity w
ithin
/24
0 1 2 3 4
.0.1
27.2
55
time [months]
IP a
ddre
ss a
ctiv
ity w
ithin
/24
0 1 2 3 4
.0.1
27.2
55
Philipp Richter | INET / TU Berlin 15http://arxiv.org/abs/1606.00360
sum(<active IP, day>)sum(all possible <active IP, day>)
= redred + grey
rather high (75%)
Dynamic addressing: Configuration/Pool sizes matter
![Page 22: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/22.jpg)
Utilization: Blocks w/ > 250 active IPs
% of max possible spatio−temporal utilization
activ
e /2
4 bl
ocks
0 20 40 60 80 100
040
K80
K12
0K
Philipp Richter | INET / TU Berlin 16http://arxiv.org/abs/1606.00360
![Page 23: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/23.jpg)
Utilization: Blocks w/ > 250 active IPs
% of max possible spatio−temporal utilization
activ
e /2
4 bl
ocks
0 20 40 60 80 100
040
K80
K12
0K
Philipp Richter | INET / TU Berlin 16http://arxiv.org/abs/1606.00360
majority of - likely dynamic - blocks show high utilization
![Page 24: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/24.jpg)
Utilization: Blocks w/ > 250 active IPs
% of max possible spatio−temporal utilization
activ
e /2
4 bl
ocks
0 20 40 60 80 100
040
K80
K12
0K
Philipp Richter | INET / TU Berlin 16http://arxiv.org/abs/1606.00360
a third of - likely dynamic - blocks show low utilization
![Page 25: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/25.jpg)
spatio-temporal utilizationtraffi
c contribution
rela
tive
host
cou
nt
Summary
Philipp Richter | INET / TU Berlin 17http://arxiv.org/abs/1606.00360
• Comprehensive study of IPv4 address activity • Metrics “beyond” binary notion of IPv4 activity • Can inform: Network operations, address [re]assigment • Can inform: Network security and host reputation
Figure: active /24 address blocks • Spatio-temporal utilization • Traffic contribution • Relative host count
![Page 26: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/26.jpg)
Backup: IPv4 Traffic Consolidation
Philipp Richter | INET / TU Berlin http://arxiv.org/abs/1606.00360
months [2015]
% tr
affic
sha
re o
f top
10%
IPs
4950
5152
53
01 06 12
weeklymoving average (4 weeks)
![Page 27: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/27.jpg)
Backup: Churn Visibility in BGP
Philipp Richter | INET / TU Berlin http://arxiv.org/abs/1606.00360
1 day 7 days 28 days
aggregation window size
% e
vent
s co
rrela
ted
with
BG
P ch
ange
0.0
0.5
1.0
1.5
2.0
2.5
up eventsdown eventsactive (no change)
![Page 28: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/28.jpg)
Backup: Classification ICMP-only IPs
Philipp Richter | INET / TU Berlin http://arxiv.org/abs/1606.00360
ASes(N=2k)
BGP prefixes(N=55k)
/24s(N=495k)
IPs(N=77m)
0.0
0.2
0.4
0.6
0.8
1.0
server server/router router unknown
server identification: ZMap scans HTTP(S), POP3(S), IMAP(S)router identification: Ark, TTL exceeded received
ASes(N=51k)
BGP prefixes(N=460k)
/24s(N=6m)
IPs(N=950m)
0.0
0.2
0.4
0.6
0.8
1.0
CDN only CDN & ICMP ICMP only
Visibility CDN/ICMP ICMP-only hosts
![Page 29: Beyond Counting: New Perspectives on the Active IPv4](https://reader034.vdocuments.us/reader034/viewer/2022052301/6288e46ae95ee057837c04ba/html5/thumbnails/29.jpg)
Backup: IPv6 /64 Growth
Philipp Richter | INET / TU Berlin http://arxiv.org/abs/1606.00360
Mar-20
14
Apr-20
14
May-20
14
Jun-20
14
Jul-20
14
Aug-20
14
Sep-20
14
Oct-20
14
Nov-20
14
Dec-20
14
Jan-20
15
Feb-20
15
Mar-20
15
Apr-20
15
May-20
15
Jun-20
15
Jul-20
15
Aug-20
15
Sep-20
15
Oct-20
15
Nov-20
15
Dec-20
15
Jan-20
16
Feb-20
16
Mar-20
16
200 M
400 M
Active WWW client IPv6 /64 count