ams-ix provision system

AMS-IX provision system

Maksym Tulyuk, maksym.tulyuk@ams-ix.netMore-IP event, Amsterdam, 22nd May 2014

Agenda

1. Definition of the provision system

2. Evolution of the provision system

3. Achievements and plans

Definition of provision system

Maksym Tulyuk, maksym.tulyuk@ams-ix.netMore-IP event, Amsterdam, 22nd May 2014

Provision system as Black Box

Customerrequests a link

AMS-IX provides access to a

VLAN

AMS-IXprovisio

nsystem

• link is a physical port(s) or rate-limit on a partner’s port• in 95% cases VLAN is Internet Peering VLAN

Inside The Black Box

reservation

assignment

testing

enabling

customer’s request

signed documents

link is up

router is ready

Deeply inside The Black Box

Complexity inside each step!

Evolution of provision system

Maksym Tulyuk, maksym.tulyuk@ams-ix.netMore-IP event, Amsterdam, 22nd May 2014

Provision 4 years ago• customer: sends email to MR team

• MR: copy&paste information into my.ams-ix and RT, and sends email to NOC

• NOC: copy&paste to browser to find free port

• NOC: copy&paste to XML (internal database)

• NOC: copy&paste to console to configure port

• NOC: copy&paste to email to send reserved port to MR team

• MR: copy&paste into ...

“Issues” with old systems

• each engineer did it in his/her own way

• a NOC engineer made a lot of errors

• it took a lot of time and energy

• it required a lot of the engineers

Improvements

• Internal procedures

• Scripts

• Email templates in my.ams-ix.net

Procedures

• Procedure = best practice

• Write down existing processes

• Constantly check and improve them

Scripts

• “One step" scripts: change configuration, find free resources (port, IP address), etc

• “Do-it-all” scripts: replacement of procedures

• Require careful thinking

• Have a lot of checks

Email templates

• Replacement of copy&paste operations

• Integration with my.ams-ix and our database

• Flexibility via simple programming language

Historical perspective

• Write down best practices as procedures

• Replace the procedures with scripts

• Generate emails via templates

Achievements and plans

Maksym Tulyuk, maksym.tulyuk@ams-ix.netMore-IP event, Amsterdam, 22nd May 2014

Fixing the “issues”

• his/her own way

• a lot of errors

• a lot of time and energy

• a lot of the engineers

• standardisation

• no errors

• minimum and predictive time

• no engineers involved

Scalability

• Four new sites in Amsterdam: Equinix 3, EvoSwitch, Telecity 5, Terremark

• Four new exchanges: CW, EA, HK, NY

Some numbers

Achievements

• Free up engineers

• Speed up provision

• Scale up for new sites and exchanges

Plans

• customer can add, remove and change mac addresses

• customer can configure BGP sessions with route-servers and our router

• partners can modify virtual links

Even more ideas

• automatic testing and enabling

• automatic addition of mac addresses, BGP sessions, IPv6 addresses, etc

• automatic operations (show logs, port status, etc)

Summary

• Simple concept of provision: request from customer -> link from AMS-IX

• From procedures to scripts

• Automatic emails and processes

• Scalability and growth

Big thanks!

• m6_provision: Martin Pels

• my.ams-ix: Martin Lyden, Petros and Peter provision scripts: Gloria

• XML: Steven

• NOC and MR teams: feedback

Questions? Feedback?

Maksym Tulyukmaksym.tulyuk@ams-ix.net