1-855-mikrotik network · pdf filenetwork architecture – validated designs utilizing...
TRANSCRIPT
Network Architecture – Validated designs utilizing MikroTik in the Data Center P R E S E N T E D B Y: K E V I N M Y E R S , N E T W O R K A R C H I T E C T / M A N AG I N G PA R T NE R I P A R C H I T E C H S M A N AG E D S E R V I C E S
www.iparchitechs.com 1-855-MIKROTIK
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com
Background
• Kevin Myers
• 16 + years in IT/Network Engineering
• Designed and implemented networks in Service Provider, Enterprise, Ecommerce and Government environments
• Areas of Design Focus:
• MikroTik integration with multi-vendor networks
• Design of BGP/MPLS/OSPF Service Provider Triple-Play networks
• Design of large enterprise Data Center networks
• Certifications
• MTCINE #1409INE006
• Certified – CCNP, CCNA, MCP, MTCRE, MTCTCE, MTCNA
1-855-MIKROTIK
IP ArchiTechs Managed Services
• Exhibitor at 2013 and 2014 MUM – Please stop by our exhibitor booth and register to win an RC Helicopter !
• The first Carrier-Grade 24/7/365 MikroTik TAC (Technical Assistance Center)
• Three tiers of engineering support
• Monthly and per incident pricing available
• 1-855-MIKROTIK or support.iparchitechs.com
• AirMPLS - Private Nationwide 4G LTE MPLS backbone
• Partnership with Verizon Wireless - available anywhere in the Verizon service area
• Not Internet facing – privately routed over our MPLS infrastructure
• Multiple Deployment options to carry public and private traffic including L2 adjacency
• Proactive Monitoring / Ticketing / Change Control / IPAM (IP Address Management)
• Carrier-Grade Network Engineering / Design in large (100,000+ nodes) environments
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Introduction – The MikroTik enabled Data Center
• Role within the Data Center
• Layer 3 Core – Designs using CCRs with 10 Gbps interfaces
• Top of rack / End of Row – L3 options for core connectivity
• External / Internal Firewall – Internet reachability / protect critical internal networks with multiple layers (PCI)
• VPN Aggregation – Multiple Vendors / Remote Mgmt Access
• MPLS P/PE router - Segregation of traffic within Data Center
• Role between Data Centers
• MPLS L2 VPN – VLAN extension between Data Centers for VM mobility
• MPLS L3 VPN – Segregate traffic as it routes between data centers
• VLAN Rewrites – Used to deal with VLAN overlap between two or more sites
• Multiple Gateways for the same subnet at more than one site
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Conventional Data Center
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Multi-Million dollar DCs - Where does MikroTik fit in ?
• MikroTik routers can be used in different areas of the Data Center and compete with mainstream vendors like Cisco, Juniper and HP within a specific set of design parameters.
• The goal of this presentation is to display the design elements required to build a Layer 3 infrastructure capable of up to 320 Gbps forwarding with off the shelf 10 Gig switches
• Why 320 Gbps? • Relies on ECMP (server side) – assuming 16 BGP paths
• 16 Paths is a conservative value for ECMP – some go as high as 128 paths
• 16 CCRs with 20 Gbps LACP channels = 320 Gbps
• Design validation was tested with 2 CCRs which yielded 40 Gbps between servers
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Multi-Million dollar DCs - Where does MikroTik fit in ?
•Why?
• The business case for MikroTik in the Data Center •CAPEX (Capital Expenditure) Savings
•Lower hardware replacement cost when a node fails
•Cisco Nexus Deployment for 320 Gbps
•$2,000,000 to $5,000,000 CAPEX
•MikroTik Deployment for 320 Gbps
•$50,000 to $100,000 CAPEX
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
The MikroTik enabled Data Center
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 1 – Desigining for High Availability – 99.999% uptime
• Getting to five 9’s isn’t easy – can only have 5 minutes of unplanned downtime per year – maintenance windows aren’t included
• HA design elements
• Stackable switches – enables multi-chassis LAG for CCRs and servers to provide survivability in the event of a failure of one of the switches
• LACP – channeling/bonding at Layer 2 allows devices to aggregate speeds as well as prevent routing topology changes when a link fails
• Load Balancers – Provide the ability to use multiple CCR chassis as a single firewall without breaking state. The LB has the ability to return traffic to the same source and track that relationship dynamically.
• BFD – Bi-Directonal Forwarding Detection allows a network t o converge much more quickly than standard timers
• Multiple Internet BGP Peers – When used along with BFD if the upstream carrier supports it, multiple tier 1 peerings provide a level of redundancy to ensure Internet traffic is uninterrupted
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 2 – Achieving 320 Gbps throughput
• Two mechanisms for achieving high throughput
• Method 1 - ECMP
• Equal Cost Multipath (ECMP) on the CCR – RouterOS is capable of up to 128 gateways .
• Example below shows 16 gateways for one route
• Routes can be installed by either OSPF or Static. BGP can also be run on top of OSPF and utilize ECMP as well.
• ECMP Route with 16 Gateways
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 2 – Achieving 320 Gbps throughput
• ECMP Continued
• Using multiple gateway allows traffic egressing the router to balance along multiple paths but what about ingress?
• Server side ECMP is the key to scaling throughput when using independent routers.
• Support in multiple operating systems – Microsoft and Linux both support ECMP in static routes along with OSPF and BGP
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Achieving 320 Gbps throughput
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 2 – Achieving 320 Gbps throughput
• Utilizing BGP and OSPF at the server for dynamic ECMP
• Role of OSPF
• Converges quickly using adjusted standard timers (1 second hello 3 second dead)
• Converges even faster with Bidirectional Forwarding Detection (BFP)
• Provides Loopback reachability for BGP
• Is needed to implement ECMP dynamically through MikroTik routers until ECMP is added to BGP.
• Role of BGP
• Advertise data center subnets for servers, databases, web apps, etc, to the 16 router CCR core
• Can be utilized for traffic management
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Achieving 320 Gbps throughput
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 2 – Achieving 320 Gbps throughput
• Method 2 – Offset VRRP Gateways
• Each CCR is the VRRP master for one or more VLANS
• Requires setting priority for each VLAN/CCR
• Can be used in conjunction with ECMP when servers cannot be setup for ECMP
• CCR 1 – Master for VLAN 100 Backup for VLAN 200
• CCR2 – Backup for VLAN 100 Master for VLAN 200
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 2 – Achieving 320 Gbps throughput
• LACP Channels for Routers and Servers
• LACP is an open standard for aggregating Layer 2 links – 802.3ad
• Referred to as Channeling, Bonding, Teaming, Link Aggregation
• Can be trunked with multiple VLANs and multiple Layer 3 gateways
• Can be used with VRRP
• 20 Gigabit LACP channel on CCR-1036-8G-2S+
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 2 – Achieving 320 Gbps throughput
• LACP Channels for Routers and Servers
• Microsoft LACP example:
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 2 – Achieving 320 Gbps throughput
• LACP Channels for Routers and Servers
• Linux LACP example:
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 2 – Achieving 320 Gbps throughput • Final Result – 40 Gbps throughput
• Only 2 CCRs in the Core – with 16 CCRs, the throughput will be roughly 320 Gbps
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 3 – Multiple Data Centers
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 3 – Multiple Data Centers
•Using MPLS in the Data Center
• CCRs can be used as MPLS edge routers to connect Data Centers.
• Used to segregate traffic within and between Data Centers
• L2VPN (VPLS ) – Provides Layer 2 Connectivity and isolation
• L3VPN – Provides Layer 3 connectivity and isolation
• VRF (Routing Marks) – Used to separate customer routing tables so that more than one customer can use the same subnet without overlap
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 3 – Multiple Data Centers
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
• MPLS Customer Isolation at Layer 3
Part 3 – Multiple Data Centers
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
EoIP provides Layer 2 Connectivity and will allow MPLS to function across an encrypted internet link. Either EoIP or VPLS can be used for L2 connectivity.
Part 3 – Multiple Data Centers
•VLAN Rewrites • Problem:
• Data Center 1 uses Vlan 100 for web Servers on 10.1.1.0/24
• Data Center 2 uses Vlan 100 for storage replication on 192.168.222.0/24
• When extending the VLAN between Data Centers, one side must be rewritten
• CCRs can do this via bridging
• MikroTik routers with switch chips can use /switch to perform vlan rewrites
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 3 – Multiple Data Centers
•VLAN Rewrites – change VLAN 100 traffic to VLAN 3100
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 3 – Multiple Data Centers
•VLAN Rewrites – change VLAN 100 traffic to VLAN 3100
•Create VLAN 100 and 3100 interface VLANs
Create Bridge and add VLAN interface ports
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 3 – Multiple Data Centers
•Dual VRRP Gateways • Problem – when extending VLANs between Data Centers, If there is
not a local gateway for hosts in that subnet, traffic must go all the way to the other Data Center via Layer 2 to hit the default gateway
• Solution: Dual VRRP gateways
• Data Center 1 – VRRP GW – 100.64.100.1/24
• Data Center 2 – VRRP GW – 100.64.100.2/24
• These are duplicate IPs – How can this work?
• Because VRRP uses MAC addresses derived from the VRRP Group Number – Hosts will always find the gateway in their own data center before going to the other Data Center
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 3 – Multiple Data Centers
•Dual VRRP Gateways
• Add input filter for VRRP on both edge routers – IP Protocol 112 to prevent either gateway from becoming master for the other (bridges must be set to use IP Firewall)
Add VRRP Gateway for 100.64.100.1 in both Data Centers
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
Part 3 – Multiple Data Centers
•Dual VRRP Gateways
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK
24/7/365 MikroTik TAC | Nationwide Private 4G LTE MPLS | Proactive Network Monitoring | Design / Engineering / Operations
www.iparchitechs.com
1-855-MIKROTIK
2014 Pittsburgh MUM RC Heli Giveaway !! • 4 To Give Away!!! 17” RC Helicopters
Questions?
• The content of this presentation will be available at mum.iparchitechs.com
• Please come see us at the IP ArchiTechs booth in the Exhibitor Hall
• Email: [email protected]
• Office: (303) 590-9943
• Web: www.iparchitechs.com
•Thank you for your time and enjoy the MUM!!
24/7/365 MikroTik TAC Nationwide Private 4G LTE MPLS Proactive Network Monitoring Design / Engineering / Operations
www.iparchitechs.com 1-855-MIKROTIK