Network Monitoring and Management- case study of BKNIX @33rd WUNCA

Kittinan SriprasertNetwork EngineerBKNIX Co., Ltd.nan@bknix.co.th14/07/2016

Agendau Introduction to BKNIXu Monitoring and Managementu Conclusionu Extra : RIPE Atlas project

Introduction to

What is the Internet?u “The Internet is the global system of interconnected computer networks that use the

Internet protocol suite (TCP/IP) to link billions of devices worldwide. It is a network of networks that consists of millions of private, public, academic, business, and government networks of local to global scope,…”— Internet. In Wikipedia. Retrieved June 23, 2016, from https://en.wikipedia.org/wiki/Internet

u any network connect to each other

u originally for education purposes

u Now, bring everything to Life (Live)

How to connect 2 network?

u Link : 1 or 2 (with backup)

u Port : 2 or 4 (with backup)

u Simply

u Working fine!!!

How to connect 3 network?

u Link : 3 or 6 (with backup)

u Port : 6 or 12 (with backup)

u Still Simply

u Working

How to connect 4 network?

u Link : 6 or 12 (with backup)

u Port : 12 or 24 (with backup)

u More Complex

u More Cost

Compare between 2, 3 and 4 networks

u Link : 1 or 2 (with backup)

u Port : 2 or 4 (with backup)

u Link : 3 or 6 (with backup)

u Port : 6 or 12 (with backup)

u Link : 6 or 12 (with backup)

u Port : 12 or 24 (with backup)

u Link : or

u Port : n(n-1) or 2n(n-1) (with backup)

n! 2(n-2)!

2n! (with backup) 2(n-2)!

Layer 3 vs Layer 2 connections (Full-mesh)

u Higher amount of Link

u Higher amount of Port

u Difficult to manage

u Link : 6 or 12 (with backup)

u Port : 12 or 24 (with backup)

u Link : 4 or 8 (with backup)

u Port : 8 or 16 (with backup)

u Lower amount of Link

u Lower amount of Port

u Easy to manage

Port and Link growth rate

0

22.5

45

67.5

90

2 3 4 5 6 7 8 9 10

Port

Link

Layer 3 vs Layer 2 Full-mesh Interconnection Characteristic

u Not Scalable

u Non cost effective

u Higher Latency (More hop)*

Port and Link growth rate

0

5

10

15

20

2 3 4 5 6 7 8 9 10

u Scalable

u Cost Effective

u Lower Latency (Less hop)*

Port

Link

“BKNIX” ?u Bangkok Neutral Internet Exchange Pointu Layer 2 Internet Exchange (IX)u Non-profitu Independenceu Neutral (Carriers, …)u Design by network engineer group with

20-years experiencedu Located at a world-class datacenter

facility that has N+1 supported infrastructure

u Supported by THNICF and ISOCu Licensed by NBTC

VDO Presentation

u https://www.youtube.com/watch?v=WJYvOXrv-OU

13

To become “BKNIX” u 2013, Internet Hall of Fame : PIONEER

u Prof. Kanchana Kanchanasut

Acceptance Speech : https://www.youtube.com/watch?v=uXXZ8K36bRo

Feb Mar Apr Jun JulJan

First Member

Officially Lauch

First WorkshopBGP/IXP

First Commercial

ISP

First ExoticJ-root, .com, .net

Setup Connection

5 Local ISPin a row

be a member of APIX

Aug Sep

APIX#12@Jakarta, Indonesia

Oct

Members Meeting #1

DecNov

First Mobile Operator

LatestMember

(Suspend)

2016 Feb Mar AprJan May

PEERING FORUM 20169-10 May 2016      

@ VIE Hotel Bangkok

Members Meeting #2

June July

Second Location

APIX#13@Auckland,

New Zealand

DTAC AS133543 For Route Server

- Open - High Availability - Redundancy

AS63529Route Server

For Public services - NTP - RPKI - DNS - .th (Thailand TLD)

OverviewUninet

AS4621

Triple T AS45758

TT&T AS55465

INETAS4618

TCCTAS45667

BKNIXAS63528

KirzAS24187

BBConnectAS45796

EGAAS9835

VerisignAS26415

KSCAS7693

CDNSAS42090

Total11

Government1

Non-Commercial1

Commercial ISP7

Root Server, CDN, Cache 2

Port Type

9

2

1 GE 10 GE

Member

Peak 9.03 Gb/s (24 Mar 2016)

Latency in Layer 2 IX

u Traceroute from any probe in Thailand to ku.ac.th

u using RIPE Atlas Probe u https://atlas.ripe.net/

measurements/4426555/#!probes

NONTRINET-AS-AP

AIT-CS-ASNTHNIC-ASN-AP

INET-TH-AS

JASTEL-NETWORK-TH-IDC-AP

TTT-AS-AP

PSU-TH-AS-AP

JASTEL-NETWORK-TH-AP

UNINET-TH

KIRZ-AS-TH

Monitoring and Management

ServicesPeering

Performance

Traffic

Port

bytes, packets, errors, discards

Transceiver(Tx, Rx quality)

Traffic Diff.

Routing (BGP)

Route Server

Direct Peer

Web-Portal

IXP-Manager

NTP

RPKI

c.thains.co.th .th

Speedtest

speedtest.bknix.co.th

LG

Latency

Cacti

u Tx/Rx Transceiver

u Prefixes

u Traffic (for Internal)

u Traffic (for member)

u Weathermap

u Route server

Nagiosu Availability check

u eg. Ping, Load, Users, Disk, Process, Uptime

u Custom check

u check_snmp - Interface status, uptime

u check BGP - Route server peer v4&v6

u Notification

u by email

u Alternative

u Icinga (https://www.icinga.org)

IXP-Manager

u Web Portal for member

u Member Detail, Billing Detail

u Users

u Ports

u IP Addresses

u Routing Information

u Statistics

u individual

u peer-to-peer

u overall25

MRTG

u integrated with IXP-Manager u run along with Cacti u only IX port

Smokingping

u Latency u Loss u Jitter u integrated with IXP-Manager

Rancid (with WebSVN)

u Accounting (Track changes)

u Configuration Management

u Backup

Nfsen

u Investigation u Trend u Routing u DOS

IPFIXCOL

uGithub : https://github.com/CESNET/ipfixcol u IPFIX (Netflow V10) Collector

u Internet Protocol Flow Information Export (IPFIX) u Profiler - create p-2-p stats

Syslog-ng

u run on every devices (Network, Server) u Log rotate everyday, with 90 day history u export to

u Email u ELK (Elasticsearch - Logstash - Kibana)

31

Librenms (observium)u Management

u wealth check

u Inventory

u housekeeping

u Auto-discovery

NETDOT (NETwork Documentation Tool)

u Documentation

u Device

u Cable

u Inventory

u IP Address

u IPv4

u IPv6

u VLAN

u Contact

u Export Configuration

TACACS+

u Terminal Access Controller Access Control System

u AAA model

u only use for network devices

Retrieve from : http://srijit.com/monitoring-securing-your-cisco-devices-with-syslog-tacacs-server/

Looking Glass

u CLI

u quagga

u Web-based

u Bird-lg - Github : https://github.com/zorun/bird-lg

RT (Request-Tracker)

u Email-base Ticket u Problem tracking

u Link up/down u BGP up/down u Routing

BKNIX NOC (In-house)

u In-house developing

u Web-based Application (mobile support)

u PHP, Javascript and MySQL

u Monitor

u Peering Status

u Route Server

u Report

u Downtime

u Member statistic

Conclusion

About BKNIX Monitoring & Management

u Open-source basis u monitor what we use u Our selected tool won’t fit in some organization u Pick the best for you!!!

**Remark**

This is not an installation guide or tutorials, only sharing purpose

please find a presentation from NSRC for a full walk-through

https://nsrc.org/workshops/2015/apricot2015/wiki/Track4Agenda

RIPE Atlas project

Where performance is measured, performance improves.

Where performance is measured and reported, the rate of improvement accelerates.

— Thomas S. Monson —16th President of The Church of Jesus Christ of Latter-day Saints

“

”

What is RIPE Atlas?

A measurement network of 13000+ devices

For the community, by the community

42

https://atlas.ripe.net/results/maps/network-coverage/?filter=Thailand+(th)

RIPE Atlas in Thailand

Connected: 18

Disconnected: 8

Abandoned: 11

What is measured?

44

u “Built-in” measurements From all the probes, towards root name servers and RIPE Atlas infrastructure — Pings, traceroutes, SSLcert and DNS queries

Results available to everyone! – https://atlas.ripe.net/results/

u Customised (user defined) measurements (UDM) using the targets of their choice and desired frequency. – You need “credits” to do this

Built-in measurement

45

UDMs

More informationu https://bknix.co.thu https://www.facebook.com/bknixthu https://twitter.com/bknixthu noc@bknix.co.th

Question ?