What’s status:

IPv6 Implementation in ITB

Affan Basalamah

100NGN Workshop

# whoami

• Affan Basalamah

• IT Infra Manager

• Unit Sumber Daya

Informasi ITB

• affan@itb.ac.id

• @affanzbasalamah

Outline

The past

10 years

The

present

The

future

THE PAST 10 YEARS

Background

• ITB already implement IPv6 since 2001

– From SOI-ASIA program (www.soi.asia)

– Allocated subnet: 2001:d30:3::/48

• Registering IPv6 allocation to APNIC in 2007

– Allocated subnet: 2403:8000::/32

IPV6 NETWORK ADDRESS &

ROUTING

IPv6 External Connection

• IPv6 address-family to all eBGP peers IPv6 Tunnel

HE.net

BGP Native

IPv6 Address Allocation

Distribution

• 2001:d30:3::/48 for NOC

• 2043:8000::/32 for ITB

– 1 buah /35 for ITB campus

– 7 buah /35 for future allocation

IPv6 Routing Implementation (1)

• FreeBSD/Linux-based PC Router (pre-2010)

– Quagga Routing Suite

– RIPng OSPFv3

– BGP

• Dedicated Core Router/Switches (2010)

– Cisco Catalyst 6500 on Sup720-3B/Sup32

– Juniper SRX650

IPv6 Routing Implementation (2)

• PC Router with IPv4-only Layer 3 Switch

IPv4-only

Layer 3 Switch

IPv6

PC Router-1

IPv6 Gateway

PC Router

IPv6

Network

IPv6 Routing Implementation (3)

• PC Router with VLAN interface

• OSPFv3 IPv4-only

Layer 3 Switch

IPv6

PC Router-1

IPv4-only

Layer 3 Switch

IPv4-only

Layer 3 Switch

IPv6

PC Router-2 IPv6

PC Router-3

Tunnel & VLAN

IPv6 Routing Implementation (4)

• 802.1Q Trunk VLAN to distribute IPv6

subnet with Router Advertisement (RA)

Layer 2 Switch

Layer 2 Switch

Layer 2 Switch

IPv4-only

Layer 3 Switch PC

PC

PC 802.1Q Trunk

IPv4-only

Layer 3 Switch

IPv6

PC Router

Campus Network

IPV6 APPLICATION

Aplikasi IPv6 di ITB

• Operating System

• DNS

• WWW & FTP Server

• Mail Exchange Server

• Web Cache Proxy

• Unicast & Multicast Stream

Operating System for Server

• FreeBSD 9.x, 8.x, 7.x

• CentOS Linux 6.x dan 5.x

• OpenSolaris 2009.x

• Windows Server 2003

Domain Name System (DNS)

• BIND 9.8.x

• Forward zone

– AAAA record for MX & selected Server

• Reverse zone

– PTR record for 2403:8000::/32 delegated from

APNIC

Web Server

• Apache Web Server 2.2.x

– Serve IPv4 and IPv6 at the same time

• IPv6 PHP script to detect v6 client

• Website IPv6 ITB

– http://www.itb.ac.id

– http://ipv6.itb.ac.id

– Masih banyak lagi

ITB Official Website

Mail Exchange (MX) Server

• Postfix 2.10

• mx.itb.ac.id

• http://www.postfix.org/IPV6_README.html

Web Cache Proxy Server

• Squid 2.7 (IPv4 only) and 3.1 (IPv6 support)

• Web Cache Parenting over IPv6

– to WIDE Project Japan

• Some IPv6 content observed

– Google IPv6

– Youtube IPv6

• Serving IPv6 client in ITB

• User Authentication with LDAP

Access.log Squid IPv6

Multicast Stream

• VLC

– IPv6 Unicast

– IPv6 Multicast

• Dokodemo SOI-ASIA

(http://dokodemo.soi.asia)

Dokodemo

IPv6 Day Activities

• Work together with SOI-ASIA

(http://ipv6day.soi.asia)

• IPv6-only video-on-demand streaming

– Adobe Flash Media Streaming Server on Linux

– 2 video of Indonesia cultural show

• IPv6-only website, embedding video

content

– http://ipv6day.itb.ac.id Apache on FreeBSD

Evaluating

IPv6 Server Load Balancer

• Provide IPv6 SLB for v6 client to v4 server

• IPv6 SLB that can translate:

– v6 client – v6 server

– v6 client – v6/v4 server

– v6 client – v4 server

Why IPv6 Load Balancer?

• To solve questions:

– Which is comes first, network or application?

– What is IPv6 killer apps?

• How it’s going to solve:

– IPv4 killer apps can directly migrated to IPv6

– No apps rewrite or migration

• At least in the theory

– Evaluation in the real world will tell you

Experience with IPv6 SLB (1)

• Basic services works just fine

• Translate IPv4 web server to IPv6 client

• Translate IPv4 cache server to IPv6 client

– real server(s) TCP4/8080 translated to virtual

IP on TCP6/8080

– virtual server client TCP6 server IPv6 client

Experience with IPv6 SLB (2)

• HTTP Layer 7 switching is mandatory

– or else cookie-based apps is not working

– Show stopper for webmaster to put webserver

behind SLB

• Managing SLB is quite hard for ordinary

network admin

– Lots of L7 feature to learn

Screenshot

Video

Stream

from

streaming

server

Simple

script to

identify

IPv6 client

Website Statistics (1)

Website Statistics (2)

User statistics

• Viewer observed from ITB campus

– Most of ITB campus network is IPv6 dual-stack

• Viewer also observed from Indonesia ISP

• Also observed from WIDE Project Japan

• No reverse address for IPv6

– It’s hard to see which ISP has IPv6 address

– Had to manually doing WHOIS on address

IPv6 tunnel broker for

Indonesia Universities • Deployed on ITB router (Juniper SRX650)

– Ask INHERENT community to join

• Cleanup IPv6 prefix-list in TEIN3 ID-POP to

advertise new IPv6 prefix form

ITB/INHERENT

IPv6 BGP peering in ITB router

2001:470:17:72::1 6939 176059 23178 0 18 1w0d9h Establ

inet6.0: 5917/6690/6690/0

2001:7fa:f::1 7717 2907 2496 0 265 19:07:49 Establ

inet6.0: 118/123/123/0

2403:8000:10::2 18007 1 3 0 15950 1 Establ

inet6.0: 672/672/672/0

inet6.2: 79/79/79/0

2403:8000:666:dead::2 46047 2031 101953 0 149 16:53:49 Establ

inet6.0: 1/1/1/0

2403:8000:666:dead::6 55687 35 2699 0 123 20:57:09 Active

2403:8000:666:dead::a 45304 1 30432 0 9 9w4d1h Active

2403:8000:666:dead::18 46052 0 0 0 0 9w5d18h Idle

2403:8000:666:dead::22 55674 0 0 0 0 9w5d18h Active

• Red: IPv6 Tunnel BGP peering (AS6939 above is HE.net)

• Blue: IPv6 BGP peering to Indonesia OpenIXP

• Green: IPv6 BGP peering to INHERENT router to TEIN3 network

Statistics

• At least 5 tunnel registered, 3 of them observed

alive, only 1 currently active

• Unable to run IPv6 network monitoring, because

we haven’t setup the the infrastructure

• NetFlow v9 collector

• NFSen as NetFlow viewer

Hurricane Electric Tunnel

everywhere • From simple show route protocol bgp, I see

most Indonesia ISP has HE.net tunnel

• AS6939 everywhere

– Makes BGP path adjustment difficult

• Path to AS6939 is preferred compared to TEIN3

• e.g., ITB needs to advertise /33 instead of /32 to TEIN3

– ITB has some IPv6 BGP peering

• Internet commercial IPv6 via HE.net

THE PRESENT

Status per 2013 (1)

• IPv6 stack is maturing:

– Router OS: Linux/BSD, Cisco, Juniper, Brocade,

HP, Huawei, Mikrotik, Force10, dsb

– Switch OS: Cisco Catalyst/Nexus, Juniper,

Brocade, HP, Huawei, Force10, dsb

Status per 2013 (2)

– Firewall: Cisco ASA, Juniper, Mikrotik, Palo Alto

– Load Balancer: F5 LTM, Brocade ADX, Apache

Traffic Server, Nginx, Varnish, Apache

mod_proxy module

– OS: Windows 7/8, Server 2008R2/2012, Mac

OS X, Linux/BSD

– Hypervisor: vSphere 5.x, RHEV, Hyper-V

Status per 2013 (3)

• OpenIXP provide IPv6 BGP

• Other ISP? Indosat? Telkom? Anyone?

• Temporary (permanently) solutions:

www.tunnelbroker.net

– bisa tunnel + peering BGP juga

However...

There are few things that gets in the way...

IPv6 without DNS =~ headache

• IPv6 address below is very hard to remember:

– 2403:8000:2e3b:6738:a573:c1bd:4b6c:31b7

• Especially when you create IN PTR record

• In order to use IPv6 network sniffer

• In order to see access_log apache/squid

• In order to see awstat/webalizer

• We should automate IN PTR creation in DNS

Happy Eyeball (1)

• Broken experience on IPv6 dual stack

means user won’t use IPv6

– https://ripe64.ripe.net/presentations/78-2012-

04-16-ripe64.pdf dari Geoff Huston

Happy Eyeball (2)

• Need patch for all browser

• Most sysadmin choose to disable IPv6 for

end-user to mitigate complaints

• Or directry migrate to IPv6 only network

with NAT64/DNS64

– Small number of apps with literal IPv4

addressing won’t run

Slide happy eyeball

IPv6 Addressing scheme

• Or use existing IPv4 addressing scheme – Easy to remember

• “Human-readable” IPv6 address: – face:b00c (www.facebook.com)

– dead:beef

• IPAM is mandatory – BlueCat Networks http://www.bluecatnetworks.com/ipam/

– GestioIP www.gestioip.net

– phpIPAM www.phpipam.net

• IPv6 Subnetting BCOP: http://www.ipbcop.org/ratified-bcops/bcop-ipv6-subnetting/

Application guys don’t care

• They only care about their apps, without knowing any networking property – Managing responsive web, CSS and support

for IE6 is taking their time

• Solution: IPv6 load balancer – Dual stack SLB, IPv4-only web server

– Enable Layer 7 features, or else problems with sticky apps

– Test your apps!

Security Issues

• Developing practices for IPv6

snort/IDS/IPS

• Port scanning is impossible

– You can’t run nmap -sP subnet/64

• Fragmentation attack

• RH0, source route

• Security compliance additional checklist

Bandwidth accounting

• How to inspect/police IPv6 bittorrent?

• Squid cache proxy

– Stable version don’t support IPv6 (2.7)

– IPv6 support in 3.2 is not as stable as 2.7

• Yes, you can put Squid behind IPv6 SLB

– But how about squid access log?

• This is problem in regular enterprise without

separate accounting/billing infra (telco)

User/client Provisioning

• DHCPv6 is not really like DHCPv4

• Two choices, which one to choose?

– IPv6 RA (ICMPv6) or DHCPv6?

• No DNS server record from IPv6 RA

– (you don't say?)

• Security issue in ICMPv6

– SEND = Secure ND

It feels like marathon

• Implementing IPv6 requires clear

milestone, resources and determination

• There are no deadline

• But sometimes you are out of resources

– Our team members come and go

– Higher priority jobs gets in the way

THE FUTURE

What’s next for IPv6?

• Part of the ITB nextgen network blueprint

• IPv6 in hardware for all network devices

• Simpler transition mechanism

– NAT64/DNS64

– IPv6 SLB

• Simpler operation

– IPv6 full telemetry

– IPv6 address management

Roadmaps

IT

Cluster

BigData

Cluster

HPC

Cluster

Compute Cluster

Mgmt Inter

connect

Core

Routing

Network Cluster

Live

Storage

Repo

Storage

Archive

Storage

Storage Cluster

I/O Inter

connect Disk Storage Memory Processor

Infrastructure

as a Service

Software as

a Service

Platform as

a Service

Email File

Sharing HPC

Web

Hosting Identity

Provider

Online

Learning IS BigData

Telepres

ence

Self service

Portal OS/Hypervisor

Cloud

Orchestration

Network Blueprint

Networking for NGN Enterprise

• Basic IP routing

– IPv4/v6 unicast/multicast

– Policy-based routing/forwarding

• Advanced: MPLS on enterprise

– L3VPN, L2VPN, VPLS w/ TE/FRR

• Next generation network

– Ethernet fabric

– SDN: Software Defined Network

(programmable network) OpenFlow

MPLS on Enterprise

• Enterprise ingin punya network yg flexible

seperti Telco

• Feature sets:

– L3VPN

– L2VPN

– VPLS

• High Availability

– MPLS TE

– FRR

MPLS Use Case for Campus

• L3VPN (IPv4 and IPv6, unicast & multicast)

– IP surveillance, RFID gate/reader, BMS

– Resell ISP bandwidth

• L2VPN

– Direct L2 connectivity from ISP

• VPLS

– Datacenter connectivity for cloud computing

– Single subnet wireless LAN deployment

• Router

– Unicast/multicast in Global Routing Table

– Unicast/multicast in VRF

• Firewall & NAT gateway

– IPv6 traffic inspection

– NAT64

• Server Load Balancer

– IPv6 SLB

IPv6 on all network devices (1)

IPv6 on all network devices (2)

• Network management infrastructure

– Devices telemetry: SNMP, Syslog

– Network telemetry: Netflow v9 / sFlow

– Authentication: RADIUS/Tacacs+

• Security management infrastructure

– Traffic inspection (IPS/IDS)

– Security Information & Event Management

(SIEM)

Simpler transition mechanism

• NAT64/DNS64 for IPv6-only network

– Good-enough IPv6-only experience

• IPv6 SLB for IPv4-only server

– Providing IPv6 content in an instant

• In the end, dual stack is not for everybody

– Only in network infrastructure

– Not good for endpoint

Simpler Operation

• IPAM (IP Address Management) is

mandatory

• In the future, tracking network resources

to IP address will not scale

– Track by User ID

– Track by application

– Track by content

SDN AND OPENFLOW

Glimpse to the future:

Software Defined Networking

(SDN) In the SDN architecture, the control and data planes are

decoupled, network intelligence and state are logically

centralized, and the underlying network infrastructure is

abstracted from the applications.

Open Networking Foundation white paper

• OpenFlow is one of the SDN tool

– It’s the most popular ones

OpenFlow (1)

• Traditionally, control plane & forwarding

plane is integrated in same system

– Control plane: management, routing protocol

(OSPF, BGP) -> RIB, routing table

– Forwarding plane: packet forwarding -> FIB,

forwarding table

• SDN will decouple control plane function

to single controller

OpenFlow (2)

• Controller wil centrally manage routing for

the network

• Forwarding plane will forward the packet

based on decision from controller

– Forward, drop, send to controller, etc.

• Beberapa router menawarkan fitur

OpenFlow Hybrid Port

– One port/VLAN can simultaneously managed by

OpenFlow or by traditional routing protocol

Control/Data Plane Separation

•Control / Management plane in a dedicated controller

•Networking devices perform forwarding and maintenance functions

•IP / SSL connectivity between controller and OpenFlow switch

•OpenFlow = Forwarding table (TCAM) download protocol

Controller & Agents

Protocol Details

What’s so exciting about SDN?

• Sysadmin can centrally managed the

network without configuring each devices

• Sysadmin can program the network via

manual decision or automated, e.g. cloud

computing: OpenStack, VMware

• Flexibility above the traditional solution

• At least that’s the promise

Early SDN/OpenFlow Use Cases

• “Policy-based routing” or “packet filter”

• Replace traditional Layer 2 MAC learning

and propagation mechanisms

• Source: – http://blog.ioshints.info/2011/11/openflow-

enterprise-use-cases.html

– http://datacenteroverlords.com/2011/11/07/openflow

-overlords/

And the challenges are...

• Building the network from scratch

– Event-driven network programming

– Fluency with TCP/IP layer

– Start learning now

• Things can fail massively

Troubleshooting gets complex

• IGP/EGP routing -> RIB table

• MPLS -> MPLS label table, VPN table

• Also troubleshooting L2 is hard (VPLS, QinQ)

• And there’s another one: SDN controller

• You need to wrap around your head to

manage all of these abstraction

When should we adopt SDN?

• Start small, build virtual SDN labs

– OpenFlow controller

– Open vSwitch

• Evaluate SDN offering from vendors

• Collect SDN practices

CONCLUSION

Learned Lessons

• Put IPv6 as a requirement for next

generation network RFP

• Continuous milestone is essential to keep

IPv6 development under track

• Experience IPv6 operation early to

recognize pitfall and find solution

Reference

• Analysing Dual Stack Behaviour and IPv6 Quality – Geoff Huston &

George Michaelson - https://ripe64.ripe.net/presentations/78-2012-

04-16-ripe64.pdf

• IPv6 Security – Scott Hogg & Eric Vyncke, Cisco Press -

http://www.amazon.com/IPv6-Security-Scott-Hogg/dp/1587055945

• NAT64 and DNS64 in 30 minutes – Ivan Pepelnjak ipSpace

http://blog.ioshints.info/2010/05/nat64-and-dns64-in-30-

minutes.html

• IPv6 Address Management – 6Help Australia

http://ipv6now.com.au/addresses.php

• OpenFlow and SDN: hype, useful tools or panacea? – Ivan Pepelnjak

- https://ripe65.ripe.net/presentations/19-

OpenFlow_and_SDN_(RIPE).pdf

Thanks!

It’s time to QA!