1

Jul 12, 2017

Dinesh G Dutt | Cumulus Networks

Life After Configuration

Network DevOps 202

2Cumulus Networks

Disclaimers

• Examples shown, software tools used, demos displayed will be due to my own experience and familiarity

• Modern data center focus▪ Data comes from talking with network operators

• Not focused on public cloud deployments

• Not focused on security/compliance

3Cumulus Network

Because you want to build Scalable, Reliable, Predictable, Easy to Operate Data Center Networks

Why Should I Care ?

4Cumulus Network

The Story So Far

5Cumulus Networks

Applications Evolved...

Mainframe CloudClient-Server

Simple applications on complex infrastructure

Complex applications on generic infrastructure

Complex applications on complex infrastructure

6Cumulus Networks

And So Too Did Networks...

L2L3

SPINE

LEAFL3

7Cumulus Networks

And So Too Did Networks...

L2L3

SPINE

LEAFL3

8Cumulus Networks

And The Way You Built These Networks...

9Cumulus Networks

But Still A Ways To Go In Changing How We Operate...

10Cumulus Networks

Network Automation is Hard

S1

S4

SPINE

LEAF L1 L2 L16

S1 S2 S4S310.1.1.1

10.1.1.0

10.1.4.33

10.1.4.32

• Scale: Many things to configure

• Every interface is assigned an IP address

• Each end of the link SHOULD belong to the same subnet

• Information is duplicated

• Matching data across nodes is hard to do without some programming

11Cumulus Networks

So We Introduced BGP Unnumbered

12Cumulus Networks

Savings in IP Address Utilization

Spine Leaf Total

Unnumbered 4 16 20

Traditional BGP 4+ 4*16= 68 16+ 16*4= 80 148

Spine Leaf Total

Unnumbered 16 96 112

Traditional BGP 16 + 16*96 = 1552 96 + 96*16 = 1632 3184

Case 1

Case 2

13Cumulus Networks

Savings in Ansible Playbook Variables with BGP Unnumbered

Spine Leaf Total

BGP Unnumbered

1 + 1 (loopback subnet + spine ASN) 1 (Leaf ASN base, same loopback subnet) 3

Traditional BGP

4+(4*16)+1 = 69(Router IDs + Total switches*TOR IPv4 +ASN)

16+(16*4) +16 = 96(Router IDs + Total switches*uplink IPv4 +ASN)

165

Spine Leaf Total

BGP Unnumbered

1 + 1 (loopback subnet + spine ASN) 1 (Leaf ASN base, same loopback subnet) 3

Traditional BGP

16+(16*96)+1 = 1552(Router IDs + Total switches*TOR IPv4 +ASN)

96+(96*16) +96 = 1728(Router IDs + Total switches*uplink IPv4 +ASN)

3280

Case 1

Case 2

14Cumulus Networks

So What’s Left ?

Everything after the initial configuration….

15Cumulus Networks

Avoiding the Pitfalls of Automation...

Validating the automation playbooks

Testing changes before deploying

16Cumulus Networks

Making Changes Post Initial Deployment With Confidence

17Cumulus Networks

Troubleshooting Networks

18Cumulus Networks

And What About Just Plain Finding Information...

Where is a route originating from ?

What does the mac address look like across my fabric ?

What version is my router running ?

19Cumulus Networks

This Three-Part Webinar Addresses These Questions

Part 1

The Story So Far

Identifying What’s Left

Codifying Validation

Easing Finding Information

Part 2

Troubleshooting

Part 3

Deep Dive Into the Tools - Part 2

20

Validate Configuration

21Cumulus Networks

Why Validate ?

• With DevOps, if automation is code, validation is testing/QA.

• Validating after changes are applied avoids having to troubleshoot a problem later

• Requires the operator to know the desired state to check against

22

Why Validation is Hard

• Ansible is not a programming language▪ Designed more as a configuration

automation tool

• Validating state across the network can be hard, especially if needing to correlate multiple pieces

23

Validating BGP Config

- name: Get bgp summary command: vtysh -c 'sh ip bgp summary json' register: cmd_out become: true

- name: Get the peer count set_fact: peer_count: "{{ ((cmd_out.stdout|from_json).totalPeers) }}"

- name: Get the peer list set_fact: bgp_peers: "{{ (cmd_out.stdout|from_json).peers }}“

- name: Validate peer count matches the expected number of leaves assert: { that: '(peer_count|int) == num_leaves' } when: "{{ 'spine' in group_names }}"

- name: Validate peer count matches the expected number of spines assert: { that: '(peer_count|int) == num_spines' } when: "{{ 'leaf' in group_names }}"

- name: Verify all BGP sessions are in established state assert: { that: 'bgp_peers[item]["state"] == "Established"' } with_items: "{{ bgp_peers }}"

24Cumulus Networks

But...

This is very elementary and doing more complicated validation requires programming

25Cumulus Networks

Validating Configuration, Take Two

• NetQ is a fresh-off-the-presses product from Cumulus Networks

• Provides constructs to simplify validation:▪ Built for automation suites such as Ansible/Puppet/Chef etc.

▪ Also works well for manual workflows

26Cumulus Networks

- name: Get bgp summary

command: vtysh -c 'sh ip bgp summary json' register: cmd_out become: true

- name: Get the peer count set_fact: peer_count: "{{ ((cmd_out.stdout|from_json).totalPeers) }}"

- name: Get the peer list set_fact: bgp_peers: "{{ (cmd_out.stdout|from_json).peers }}“

- name: Validate peer count matches the expected number of leaves assert: { that: '(peer_count|int) == num_leaves' } when: "{{ 'spine' in group_names }}"

- name: Validate peer count matches the expected number of spines assert: { that: '(peer_count|int) == num_spines' } when: "{{ 'leaf' in group_names }}"

- name: Verify all BGP sessions are in established state assert: { that: 'bgp_peers[item]["state"] == "Established"' } with_items: "{{ bgp_peers }}"

Comparing Validation with NetQ

And NetQ validates:● More than what the playbook

does● Works across more complex

topologies and configuration● Can live in the past as

comfortably as the present

27Cumulus Networks

Validating CLAG

---- hosts: 'leaf*' vars_files: - properties.yml gather_facts: false tasks:

- name: Get clagctl output command: clagctl -j register: cmd_out

- name: Get the status set_fact: clag_status: "{{ (cmd_out.stdout|from_json).status }}"

- name: Get the Individual Bond status set_fact: clag_ifs: "{{ (cmd_out.stdout|from_json).clagIntfs }}"

- name: Verify CLAG Peer is up and alive assert: { that: 'clag_status["peerAlive"] == true' }

- name: Verify all bonds are dual attached assert: { that: 'clag_ifs[item]["status"] == "dual"' } with_items: "{{ clag_ifs }}"

And NetQ validates so much more than what the playbook does:

● Duplicate sysmac use

● Proper backup IP configuration

● ...

28

And What About When Validation Fails ?

29Cumulus Networks

NetQ Validation

Simplifying automating validation just like BGP and OSPF unnumbered simplified automating configuration

30

Simplifying Searching For Information

31Cumulus Networks

Sample Topology

32Cumulus Networks

NetQ Show

Run command anywhere, including not on any switch

Provide easy access to network information to non-networking folks

Safe: Netq is read-only and doesn’t touch any switches to provide information

33

Thank you!Visit us at cumulusnetworks.com or follow us @cumulusnetworks or

slack.cumulusnetworks.com

© 2017 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark

Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.