automate all the things! - internet2€¦ · automate all the things! david wilde cto, aarnet...

33
AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet [email protected]

Upload: others

Post on 26-May-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

AUTOMATE ALL THE THINGS!

David Wilde

CTO, [email protected]

Page 2: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

AUTOMATE ALL THE THINGS!

David Wilde

CTO, [email protected]

Page 3: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

WHY AUTOMATE?

Page 4: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

4

WHY AUTOMATE?NUMBER OF SERVICES IN 2014

© AARNet Pty Ltd |

Page 5: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

5

WHY AUTOMATE?NUMBER OF SERVICES IN 2016

© AARNet Pty Ltd |

Page 6: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

6

WHY AUTOMATE?NUMBER OF SERVICES IN 2017

© AARNet Pty Ltd |

Page 7: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

7

OPERATIONAL STAFF UTILISATION

© AARNet Pty Ltd |

Deploying new customers

10%

Planning + deploying upgrades to

infrastructure

10%

Attending meetings with clients and

internally

5%

Troubleshooting

faults

75%

Page 8: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

SO, WHY?

Page 9: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

9

WHY: REALLY

© AARNet Pty Ltd |

Because we’re LAZY

Page 10: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

10© AARNet Pty Ltd |

THE INTENDED (MEASURABLE) BENEFITS

Easier operation & improved security

through increased

consistency,reduced human

error and continual auditing

Improved customer

experience

Easier delivery of services

and reduced staff effort through

self-service

through telemetry and

analytics

through integration of

network, systems, applications &

cloud into a single platform

Better visibility for operations and

customers

Page 11: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

WHAT’S OUR APPROACH?

Page 12: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

12

BUILDING BLOCKS

© AARNet Pty Ltd |

Billing CRMNetwork Elements

Service inventory

Customers Staff

Page 13: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

13© AARNet Pty Ltd |

VISION

Automation and data analytics software toolkit

Improved customer experience through operational excellence

API-driven network + systems platform

Sector and institutional platforms

AARNet Network &

systems3rd Party Cloud

Providers

Service provisioning

Incident management

Operational assurance

22© AARNet Pty Ltd |

Billing CRMNetwork Elements

Service inventory

Customers Staff

Page 14: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

HOW ARE WE APPROACHING THIS?

Page 15: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

Incident management tools Operational assurance tools Service provisioning tools

Customer list: linecard failure

Customer list: fibre cut

Customer list: POP power outage

Identify recurring faults

PE/NTU configs Monitor/alarming

DNS updates Routing

Redundant service Upgrades

Service deletion End-to-end svc

Status page

Reconcile actual vs provisioned vs billing

Standardise configs

Net status pre/post planned change

Traffic graphing for capacity planning

etc… etc… IRR updates Etc…

API-driven network & systems building blocks

Information stores

Service inventory DB

Secure document storage

Element DB Billing system

Virtual + lab TND instance to simulate

platform

CI/CD toolchain

VirtualisationContainerisation

Orchestration

NetworkServersStorage

CRM

Page 16: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

API-driven network & systems building blocks

Automation building blocks

Incident management tools Operational assistance tools Service provisioning tools

Customer list: linecard failure

Customer list: fibre cut

Customer list: POP power outage

Identify recurring faults

PE/NTU configs Monitor/alarming

DNS updates Routing

Redundant service Upgrades

Service deletion End-to-end svc

Status page

Reconcile actual vs provisioned vs billing

Standardise configs

Net status pre/post planned change

Traffic graphing for capacity planning

Virtual + lab TND instance to simulate

platform

CI/CD toolchain

VirtualisationContainerisation

Orchestration

etc… etc…

NetworkServersStorage

IRR updates Etc…

Service inventory DB

Secure document storage

Element DB Billing system CRM

Page 17: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

17

Example: Network Audit Tool

Page 18: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

18

Example: IRR update tool

Page 19: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

19

Example: NOC Tool

Page 20: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

20

Example: Data analytics

Page 21: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

HOW ARE WE DOING THIS?

Page 22: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

22

HOW

© AARNet Pty Ltd |

Warning - unfortunate truth follows:

Automation is hard.(Although not always technically hard.)

Page 23: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

NETWORK AUTOMATION…

Page 24: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

24

1. BUILD A VIRTUAL MODEL

© AARNet Pty Ltd |

Grey Box Monitoring & Analysis

http://www.eve-ng.net/

Page 25: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

25

2. BUILD A PHYSICAL LAB

© AARNet Pty Ltd |

http://www.flickriver.com/photos/anachrocomputer/3080420597/

Grey Box Monitoring & Analysis

Page 26: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

26

3. START SMALL AND BUILD UNIT TESTS

© AARNet Pty Ltd |

Grey Box Monitoring & Analysis

https://aws.amazon.com/blogs/mobile/automated-device-testing-with-aws-device-farm-and-jenkins/

Page 27: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

27

4. THE HOLY GRAIL: CI-CD

© AARNet Pty Ltd |

Page 28: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

28

4. THE HOLY GRAIL: CI-CD

© AARNet Pty Ltd |

Page 29: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

29

MAKE IT VISIBLE – AGILE

© AARNet Pty

Ltd |

Page 30: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

30

MAKE IT VISIBLE – AGILE

Page 31: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

31

NEXT LEVEL 31337

© AARNet Pty Ltd |

Grey Box Monitoring & Analysis

Page 32: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

32

USEFUL LINKS

© AARNet Pty Ltd |

Grey Box Monitoring & Analysis

Virtual testbed

• EVE-NG (http://www.eve-ng.net)

• VRNetLab (https://github.com/plajjan/vrnetlab)

• VIRL (http://virl.cisco.com)

• Wistar (https://github.com/Juniper/wistar)

• GNS3 (http://www.gns3.com)

CI/CD

• Jenkins (https://jenkins.io/)

• Travis (https://travis-ci.org/)

• TeamCity (http://www.jetbrains.com/teamcity/)

Automation

• Ansible (https://www.ansible.com/)

• SaltStack (https://saltstack.com/)

• Puppet (https://puppet.com/)

• Chef (https://www.chef.io/)

Juniper

• PyEZ (https://github.com/Juniper/py-junos-eznc)

• VMX (https://www.juniper.net/us/en/products-services/routing/mx-series/vmx/)

Page 33: AUTOMATE ALL THE THINGS! - Internet2€¦ · AUTOMATE ALL THE THINGS! David Wilde CTO, AARNet david.wilde@aarnet.edu.au

THANK YOU

David Wilde

[email protected]