roll, roll, roll your root - sigcomm-sponsored events · introduction •dnssec brings integrityto...

53
Roll, Roll, Roll Your Root A Comprehensive Analysis of the First Ever DNSSEC Root KSK Rollover Internet Measurement Conference 2019 – Amsterdam, 2019-10-21 Moritz Müller 3,4 , Matthew Thomas 6 , Duane Wessels 6 , Wes Hardaker 5 , Taejoong Chung 2 , Willem Toorop 1 , Roland van Rijswijk-Deij 1,4 1 NLnet Labs, 2 Rochester Institute of Technology, 3 SIDN Labs, 4 University of Twente, 5 USC/Information Sciences Institute, 6 Verisign

Upload: others

Post on 27-Jun-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Roll, Roll, Roll Your RootA Comprehensive Analysis of the First Ever DNSSEC Root KSK RolloverInternet Measurement Conference 2019 – Amsterdam, 2019-10-21

Moritz Müller3,4, Matthew Thomas6, Duane Wessels6, Wes Hardaker5, Taejoong Chung2, Willem Toorop1, Roland van Rijswijk-Deij1,4

1NLnet Labs, 2Rochester Institute of Technology, 3SIDN Labs, 4University of Twente, 5USC/Information Sciences Institute, 6Verisign

Page 2: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Introduction

• DNSSEC brings integrity to the DNS• Validators need the public key of the

Root and configure it as trust-anchor• In 2018, the trust-anchor was replaced

(or “rolled”) for the first time

• The old key: KSK-2010• The new key: KSK-2017

Recursive Resolver

Root

What are the name servers of

.com?

2

Page 3: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Introduction

• DNSSEC brings integrity to the DNS• Validators need the public key of the

Root and configure it as trust-anchor• In 2018, the trust-anchor was replaced

(or “rolled”) for the first time

• The old key: KSK-2010• The new key: KSK-2017

Recursive Resolver

Root

a.gtld-servers.net.b.gtld-servers.net.

3

Page 4: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Introduction

• DNSSEC brings integrity to the DNS• Validators need the public key of the

Root and configure it as trust-anchor• In 2018, the trust-anchor was replaced

(or “rolled”) for the first time

• The old key: KSK-2010• The new key: KSK-2017

Recursive Resolver

Root

a.gtld-servers.net.b.gtld-servers.net.

4

Page 5: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Introduction

• DNSSEC brings integrity to the DNS• Validators need the public key of the

Root and configure it as trust-anchor• In 2018, the trust-anchor was replaced

(or “rolled”) for the first time

• The old key: KSK-2010• The new key: KSK-2017

Recursive Resolver

Root

a.gtld-servers.net.b.gtld-servers.net.

5

Page 6: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Introduction

• DNSSEC brings integrity to the DNS• Validators need the public key of the

Root and configure it as trust-anchor• In 2018, the trust-anchor was replaced

(or “rolled”) for the first time

• The old key: KSK-2010• The new key: KSK-2017

Recursive Resolver

Root

a.gtld-servers.net.b.gtld-servers.net.

6

Page 7: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Why is rolling hard?

• No key à No validation à No DNS responses • Every validator needs to have KSK-2017, but:

• Validators use hard-coded keys• Containers challenge key update• People tend to forget about DNS

Photo by Icons8 team on Unsplash

7

Page 8: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Timeline

IV V VII STOP

ICANN resumes rollover process18 Sep 2018

KSK-2017 published in Root Zone11 Jul 2017

Revocation of KSK-201011 Jan 2019

The Rollover11 Oct 2018

KSK-2010 removed from Root Zone22 Mar 2019

ICANN halts rollover process27 Sep 2017

II III

8

Page 9: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

IV V VII STOP

ICANN resumes rollover process18 Sep 2018

KSK-2017 published in Root Zone11 Jul 2017

ICANN halts rollover process27 Sep 2017

Before the Rollover

II III

Page 10: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Resolver Telemetry: RFC 8145

• The goal: estimating how many validators had KSK-2017• The solution: resolvers signal to the root

which keys they trust

• Data from ICANN from A, B, and J root• Signals from up to 100,000 validators daily

IV V VII STOP

10

Page 11: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Uptake of KSK-2017

Resolver Telemetry: RFC 8145

KS

K−

20

17

ad

de

d t

o z

on

e

RFC 5011add

hold−down

0.00

0.25

0.50

0.75

1.00

May Jun Jul Aug Sep Oct

Fra

ctio

n o

f si

gn

alle

rs

KSK−2010

KSK−2017

IV V VII STOP

2017

11

Page 12: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Uptake of KSK-2017

Resolver Telemetry: RFC 8145

KS

K−

20

17

ad

de

d t

o z

on

e

RFC 5011add

hold−down

0.00

0.25

0.50

0.75

1.00

May Jun Jul Aug Sep Oct

Fra

ctio

n o

f si

gn

alle

rs

KSK−2010

KSK−2017

IV V VII STOP

8% of resolvers don’t have KSK-2017

2017

12

Page 13: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

IV V VII STOP

Photo by John Matychuk on Unsplash13

Page 14: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Zooming in on resolvers that only have KSK-2010

• Lots of RFC 8145 sources sent only one signal• Many sent only a few queries

IV V VII STOP

Query Count_ta-4a5c 15,447. 9,182VPN domain 3,156VPN alternate domain 415_sip._udp.otherdomain 86

Domains, queried by resolvers

14

Page 15: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Zooming in on resolvers that only have KSK-2010

Resolver Telemetry: RFC 8145

VP

N r

ele

ase

1

VP

N r

ele

ase

2

VP

N r

ele

ase

3

Act

ual r

ollo

ver

0.0

0.1

0.2

0.3

0.4

0.5

Feb '18 Apr '18 Jun '18 Aug '18 Oct '18 Dec '18

Fra

ctio

n o

f R

FC

8145 s

ignalle

rs

IPv4

IPv6

IV V VII STOP

15

Page 16: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Zooming in on resolvers that only have KSK-2010

Resolver Telemetry: RFC 8145

VP

N r

ele

ase

1

VP

N r

ele

ase

2

VP

N r

ele

ase

3

Act

ual r

ollo

ver

0.0

0.1

0.2

0.3

0.4

0.5

Feb '18 Apr '18 Jun '18 Aug '18 Oct '18 Dec '18

Fra

ctio

n o

f R

FC

8145 s

ignalle

rs

IPv4

IPv6

IV V VII STOP

16

Page 17: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Takeaways from before the Rollover

• Most validators correctly picked up KSK-2017• But one single application can influence the trust-anchor signal• Validation in applications might become more common

à Influence on telemetry

IV V VII STOP

17

Page 18: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

IV V VII STOP

The Rollover11 Oct 2018

During the Rollover

II III

Page 19: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

The User’s Perspective: RIPE Atlas

• The goal: measuring how users perceive the rollover • The approach: Measuring with all RIPE Atlas probes

once per houra) If they have cached KSK-2017b) If they validate correctly

• We observed 35,719 resolver addresses in 3,141 ASesand correlated failing resolvers with DNSKEY queries with DITL data

IV V VII STOP

19

Page 20: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Activating KSK-20171

2

3

0%

25%

50%

75%

100%

Oct 11−16:00h

Oct 12−00:00h

Oct 12−08:00h

Oct 12−16:00h

Oct 13−00:00h

Oct 13−08:00h

Oct 13−16:00h

Oct 14−00:00h

Oct 14−08:00h

Oct 14−16:00h

% V

Ps

with

Key

Cach

ed

KSK−2010KSK−2017

IV V VII STOP

2017

20

Page 21: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Activating KSK-20171

2

3

0%

25%

50%

75%

100%

Oct 11−16:00h

Oct 12−00:00h

Oct 12−08:00h

Oct 12−16:00h

Oct 13−00:00h

Oct 13−08:00h

Oct 13−16:00h

Oct 14−00:00h

Oct 14−08:00h

Oct 14−16:00h

% V

Ps

with

Key

Cach

ed

KSK−2010KSK−2017

IV V VII STOP

Large resolvers start validating with KSK-2017

2017

21

Page 22: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Reaction to Validation Failures

218 138 3

35,719 unique resolver sources in RIPE Atlas

970 747

sending excessDNSKEY queries519

359

fixed within 1h fixed after 1h never fixed

secure before, insecure after rollover

secure before, bogus after rollover

sending 1.5x more DNSKEY queries after rollover

970secure before, bogus after rollover

34,002always secure or always insecure

IV V VII STOP

22

Page 23: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Reaction to Validation Failures

218 138 3

35,719 unique resolver sources in RIPE Atlas

970 747

sending excessDNSKEY queries519

359

fixed within 1h fixed after 1h never fixed

secure before, insecure after rollover

secure before, bogus after rollover

sending 1.5x more DNSKEY queries after rollover

970secure before, bogus after rollover

34,002always secure or always insecure

IV V VII STOP

23

Page 24: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Reaction to Validation Failures

218 138 3

35,719 unique resolver sources in RIPE Atlas

970 747

sending excessDNSKEY queries519

359

fixed within 1h fixed after 1h never fixed

secure before, insecure after rollover

secure before, bogus after rollover

sending 1.5x more DNSKEY queries after rollover

970secure before, bogus after rollover

34,002always secure or always insecure

IV V VII STOP

24

Page 25: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Reaction to Validation Failures

218 138 3

35,719 unique resolver sources in RIPE Atlas

970 747

sending excessDNSKEY queries519

359

fixed within 1h fixed after 1h never fixed

secure before, insecure after rollover

secure before, bogus after rollover

sending 1.5x more DNSKEY queries after rollover

970secure before, bogus after rollover

34,002always secure or always insecure

IV V VII STOP

25

Page 26: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

https://www.irishtimes.com/business/technology/broadband-restored-to-eir-customers-after-outage-1.3663004

IV V VII STOP

26

Page 27: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

EIR Outage - Was it DNS(SEC)?

Ro

llove

r

Revo

catio

n

Re

mova

l

0

25000

50000

75000

100000

125000

Aug '18Sep '18

Oct '18

Nov '18Dec '18

Jan '19Feb '19

Mar '19

Apr '19

Qu

eri

es

pe

r d

ay

IV V VII STOP

Massive increase after the rollover

27

Page 28: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

EIR Outage - Was it DNS(SEC)?

Ro

llove

r

Revo

catio

n

Re

mova

l

0

25000

50000

75000

100000

125000

Aug '18Sep '18

Oct '18

Nov '18Dec '18

Jan '19Feb '19

Mar '19

Apr '19

Qu

eri

es

pe

r d

ay

IV V VII STOP

Mysterious bump after removal of KSK-2010

28

Page 29: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Takeaways from during the Rollover

• Few resolvers had serious problems• The ones that had problems recovered fast• Less than 0.01% of the resolvers we monitored experienced problems

IV V VII STOP

29

Page 30: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

IV V VII STOP

Revocation of KSK-201011 Jan 2019

KSK-2010 removed from Root Zone22 Mar 2019

After the Rollover

II III

Page 31: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Increase in DNSKEY queries

Ro

llove

r

Revo

catio

n

Re

mova

l

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●1 ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●2

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●3

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●40 M

250 M

500 M

750 M

1 000 M

1 250 M

Aug '18Sep '18

Oct '18

Nov '18Dec '18

Jan '19Feb '19

Mar '19

Apr '19

Qu

eri

es

pe

r d

ay

IV V VII STOP

31

Page 32: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Increase in DNSKEY queries

Ro

llove

r

Revo

catio

n

Re

mova

l

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●1 ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●2

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●3

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●40 M

250 M

500 M

750 M

1 000 M

1 250 M

Aug '18Sep '18

Oct '18

Nov '18Dec '18

Jan '19Feb '19

Mar '19

Apr '19

Qu

eri

es

pe

r d

ay

IV V VII STOP

Partially expected increase

32

Page 33: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Increase in DNSKEY queries

Ro

llove

r

Revo

catio

n

Re

mova

l

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●1 ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●2

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●3

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●40 M

250 M

500 M

750 M

1 000 M

1 250 M

Aug '18Sep '18

Oct '18

Nov '18Dec '18

Jan '19Feb '19

Mar '19

Apr '19

Qu

eri

es

pe

r d

ay

IV V VII STOP

Partially expected increase

Very unexpected increase

33

Page 34: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Increase in DNSKEY queries

Ro

llove

r

Revo

catio

n

Re

mova

l

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●1 ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●2

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●3

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●40 M

250 M

500 M

750 M

1 000 M

1 250 M

Aug '18Sep '18

Oct '18

Nov '18Dec '18

Jan '19Feb '19

Mar '19

Apr '19

Qu

eri

es

pe

r d

ay

IV V VII STOP

Partially expected increase

Very unexpected increase

7% of total query load 34

Page 35: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Increase in DNSKEY queries

Ro

llove

r

Revo

catio

n

Re

mova

l

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●1 ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●2

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●3

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●40 M

250 M

500 M

750 M

1 000 M

1 250 M

Aug '18Sep '18

Oct '18

Nov '18Dec '18

Jan '19Feb '19

Mar '19

Apr '19

Qu

eri

es

pe

r d

ay

IV V VII STOP

Partially expected increase

Very unexpected increase

Return to load after rollover

357% of total query load

Page 36: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Who’s behind the query floods?

• DNS CHAOS queries to sources reveal mostly older versions of BIND• Outreach• A large French cloud hosting provider

confirmed a source running BIND 9.8.2 on CentOS• Large midwestern university confirmed DNS

lab exercise and provided BIND config

IV V VII STOP

Photo by Kelly Sikkema on Unsplash

36

Page 37: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Reproducing Key Floods with BIND

0

400

800

1200

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

#Q

ue

rie

s

Experiment

• Conditions for reproducing DNSKEY floods with BIND:• DNSSEC managed keys contains KSK-2010, but not KSK-2017• The dnssec-enable flag was set to false• The dnssec-validation flag was unset, leaving it in its default state of “yes.”

IV V VII STOP

37

Page 38: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Reproducing Key Floods with BIND

0

400

800

1200

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

#Q

ue

rie

s

Experiment

• Conditions for reproducing DNSKEY floods with BIND:• DNSSEC managed keys contains KSK-2010, but not KSK-2017• The dnssec-enable flag was set to false• The dnssec-validation flag was unset, leaving it in its default state of “yes.”

IV V VII STOP

Bursts occur only occasionally

38

Page 39: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Resolver Telemetry: The return of KSK-2010

Ro

llove

r

Revo

catio

n

Re

mova

l

0.00

0.25

0.50

0.75

1.00

Oct 2018 Jan 2019 Apr 2019 Jul 2019

Fra

ctio

n o

f si

gn

alle

rs KSK−2010

KSK−2017

IV V VII STOP

39

Page 40: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Takeaways from after the Rollover

• No one expected the massive flood of DNSKEY queries• Trust anchor management comes in different shapes and colors• Shipping trust anchors with software has long-lasting effects

IV V VII STOP

40

Page 41: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Discussion

VI

Page 42: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Do we need to improve telemetry?• RFC 8145 and RFC 8509 are useful but should be

improved• Allowing to identify the true source of a signal• Provide an estimate for how many users a signal

representsPhoto by Chunlea Ju on Unsplash

42

Page 43: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Do we need to improve telemetry?• RFC 8145 and RFC 8509 are useful but should be

improved• Allowing to identify the true source of a signal• Provide an estimate for how many users a signal

represents

Do we need to change trust anchor management?E.g. shipping TAs centrally in OSes?

Photo by Chunlea Ju on Unsplash

43

Page 44: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Conclusions and broader Lessons

• The rollover was a success• Independent analysis and measurements on the internet are valuable• Telemetry must be kept in mind at an early stage of protocol development• Trust anchors should be managed centrally

44

Page 45: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Conclusions and broader Lessons

• The rollover was a success• Independent analysis and measurements on the internet are valuable• Telemetry must be kept in mind at an early stage of protocol development• Trust anchors should be managed centrally

Questions, suggestions, comments?Data available at

https://github.com/SIDN/RollRollRollYourRootContact

Moritz Müller | [email protected] | sidnlabs.nl

45

Page 46: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Bonus Slides

Page 47: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Increase in DNSKEY queries after revocation

ZSKrollover

RFC 5011hold−down

for revocation

KSK−2010revoked

0.000

0.025

0.050

0.075

0.100

Jan '19 Feb '19 Mar '19

Fra

ction o

f tr

affic

AA*BCDEFHIJJ*KLM

IV V VII STOP

Most root servers see the increase

47

Page 48: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Increase in DNSKEY queries after revocation

ZSKrollover

RFC 5011hold−down

for revocation

KSK−2010revoked

0.000

0.025

0.050

0.075

0.100

Jan '19 Feb '19 Mar '19

Fra

ction o

f tr

affic

AA*BCDEFHIJJ*KLM

IV V VII STOP

Most root servers see the increase

But not all of them

48

Page 49: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Resolver Telemetry: RFC 8509 “Root Sentinel”

Rollo

ver

Revo

catio

n

Rem

ova

l

0

500

1000

1500

2000

2500

Aug '18Sep '18

Oct '18Nov '18

Dec '18Jan '19

Feb '19Mar '1

9Apr '1

9May '19

Jun '19Jul '1

9Aug '19

Num

ber

of re

solv

ers

KSK−2010

KSK−2017

IV V VII STOP

49

Page 50: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Resolver Telemetry: RFC 8509 “Root Sentinel”

Rollo

ver

Revo

catio

n

Rem

ova

l

0

500

1000

1500

2000

2500

Aug '18Sep '18

Oct '18Nov '18

Dec '18Jan '19

Feb '19Mar '1

9Apr '1

9May '19

Jun '19Jul '1

9Aug '19

Num

ber

of re

solv

ers

KSK−2010

KSK−2017

IV V VII STOP

The return of KSK-2010

50

Page 51: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Failure Modes

0204060

0100020003000

0

500

1000

Oct 11 − 00:00

Oct 11 − 08:00

Oct 11 − 16:00

Oct 12 − 00:00

Oct 12 − 08:00

Oct 12 − 16:00

Oct 13 − 00:00

Oct 13 − 08:00

Oct 13 − 16:00

Oct 14 − 00:00

Oct 14 − 08:00

Oct 14 − 16:00

IV V VII STOP

Failing and then Insecure

Failing and Recover

Failure and Bogus

51

Page 52: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Validation Failure Modes

0204060

0100020003000

0

500

1000

Oct 11 − 00:00

Oct 11 − 08:00

Oct 11 − 16:00

Oct 12 − 00:00

Oct 12 − 08:00

Oct 12 − 16:00

Oct 13 − 00:00

Oct 13 − 08:00

Oct 13 − 16:00

Oct 14 − 00:00

Oct 14 − 08:00

Oct 14 − 16:00

IV V VII STOP

Failing and then Insecure

Failing and then Recover

Failure and Bogus

52

Page 53: Roll, Roll, Roll Your Root - SIGCOMM-Sponsored Events · Introduction •DNSSEC brings integrityto the DNS •Validators need the public key of the Root and configure it as trust-anchor

Validation Failure Modes

0204060

0100020003000

0

500

1000

Oct 11 − 00:00

Oct 11 − 08:00

Oct 11 − 16:00

Oct 12 − 00:00

Oct 12 − 08:00

Oct 12 − 16:00

Oct 13 − 00:00

Oct 13 − 08:00

Oct 13 − 16:00

Oct 14 − 00:00

Oct 14 − 08:00

Oct 14 − 16:00

IV V VII STOP

Failing and then Insecure

Failing and then Recover

Failing and then Bogus

53