1

Experiences in Deploying Experiences in Deploying Machines Registration and Machines Registration and Integrated Linux Firewall Integrated Linux Firewall

with Traffic Shaper for Large with Traffic Shaper for Large Campus NetworkCampus Network

-Kasom Koth -Kasom Koth aarsarsa11 , Surasak Sanguanpong , Surasak Sanguanpong22 , , Pirawat Pirawat WatanpongseWatanpongse22 ,,

Surachai ChitpinityonSurachai Chitpinityon3 3 , Chalermpol Chatampan, Chalermpol Chatampan33 {{ Kasom.K, Surasak.S, Kasom.K, Surasak.S, Pirawat.W, Pirawat.W, Surachai.ChSurachai.Ch, cpccpc, cpccpc}@ku.ac.th}@ku.ac.th

11 Engineering Computer Center, Faculty of Engineering Engineering Computer Center, Faculty of Engineering22 Department of Computer Engineering, Faculty of Engineering Department of Computer Engineering, Faculty of Engineering

33 Office of Computer Services Office of Computer Services

Kasetsart UniversityKasetsart University

APAN, Xi’an, Network Security, 29APAN, Xi’an, Network Security, 29thth August 2007 August 2007

This work is partially supported by Commission of Higher Education (CHE), UniNET, Thailand

2

Kasetsart UniversityKasetsart University

Established in Established in 1943 1943 A.D.A.D.

7 campuses with 7 campuses with ~43,000~43,000 students students, , ~9600 academic and ~9600 academic and supported staffssupported staffs

3

NontriNet Quick FactsNontriNet Quick Facts

University Network - NontriNetUniversity Network - NontriNet 41,992 MAC addresses (As of 41,992 MAC addresses (As of

2007/08/28)2007/08/28) 8,852 Clients (Personal, Wired)8,852 Clients (Personal, Wired) 3,269 Clients (Service, Wired)3,269 Clients (Service, Wired) 29,342 Clients (Wireless)29,342 Clients (Wireless) 495 Servers495 Servers 34 misc. devices34 misc. devices

Avg. In/out TrafficAvg. In/out Traffic 550/490 Mbps550/490 Mbps

1 Gbps

ThaiSARN UniNet

1 Gbps (backup)

1 Gbps

34 Mbps34 Mbps

34 Mbps

Bangkhen

SriRachaKampaengsaen

SakonNakhon

2 Mbps

Supan Buri

630 Mbps

Internet

45 Mbps

JGN TIEN2

155 Mbps

10 GigE

10 GigE

4

Obstacles & Obstacles & OpportunitiesOpportunities

Large number of hostsLarge number of hosts Hard to keep trackHard to keep track

Non-productive bandwidth usageNon-productive bandwidth usage P2P file sharingP2P file sharing

QoS issuesQoS issues Security issuesSecurity issues

5

Special RequirementsSpecial Requirements

Fully-integrated information Fully-integrated information databasedatabase

Low costLow cost CustomizableCustomizable ExtensibleExtensible ScalableScalable

6

Our Designed FeaturesOur Designed Features

Web-based Machines RegistrationWeb-based Machines Registration Linux Firewall & Traffic Shaper Linux Firewall & Traffic Shaper

extension extension

7

SMARTSMART(Simple Machine Address (Simple Machine Address

Registration Tool)Registration Tool) Mandatory Web-based Machines Mandatory Web-based Machines

RegistrationRegistration Registration Enforcement Agent: Registration Enforcement Agent:

The OverlordThe Overlord Centralized Database: Command Centralized Database: Command

CenterCenter Distributed Data Entry: the Interface Distributed Data Entry: the Interface

8

SMART: Architecture SMART: Architecture DiagramDiagram

Command-CenterOverlord Observer

Detected Incident

Sniffed Packets

PoliciesDetection Rules

Statistics

Sniffed Packets

Injected Packets (TCP hijacking)

Target Subnetwork

9

Command CenterCommand CenterCommand-Center

Overlord

Observer

Policies

Statistics

Detection Rules

Detected Incident

Administrators

Users

Web Interface

Communicator

Database Manager

MACPolicy

Users

Overlords,Observers

Logs

NetworkAnomaly

DetectionRules

Statistics Documents

10

OverlordOverlord (TCP Hijack) (TCP Hijack)

Command Center

Overlord

Policies

Statistics

Table of MACs’Policy + Statistics

Target Subnetwork

Packet Sniffer

Packet Injector

Policy Checker

Communicator

Sniffed Packets

Injected Packets (TCP hijacking)

11

ObserverObserver

Command Center

Observer

Detection Rules

Detected Incident

Table of DetectionRules

Target Subnetwork

Packet Sniffer

Pattern Matcher

Communicator

Sniffed Packets

12

Linux Firewall & Traffic Linux Firewall & Traffic Shaper ExtensionShaper Extension

Intelligent Master ControllerIntelligent Master Controller User-friendly configuration interfaceUser-friendly configuration interface Automatic egress SYN-flood/P2P Automatic egress SYN-flood/P2P

blockingblocking Per-host traffic shapingPer-host traffic shaping

13

MechanismMechanism

Use Linux server as a bridgeUse Linux server as a bridge Traffic classification through Traffic classification through

iptablesiptables Traffic control through Traffic control through tctc Use Use IPP2PIPP2P and our in-house daemon and our in-house daemon

to identify P2P trafficto identify P2P traffic Use our in-house daemon to detect Use our in-house daemon to detect

some problematic network patternsome problematic network pattern

14

HardwareHardware

Dell Power Edge 2900Dell Power Edge 2900 Xeon 5160 Dual core(3.0GHz)Xeon 5160 Dual core(3.0GHz) 1 GB of RAM1 GB of RAM 160 GB SATA hard disk160 GB SATA hard disk 2 x SUN 10 Gigabit Ethernet Controller 2 x SUN 10 Gigabit Ethernet Controller

PCI Express Card (SR module)PCI Express Card (SR module)

15

SoftwareSoftware

Linux 2.6.18-8.1.8.el5 (CentOS’s Linux 2.6.18-8.1.8.el5 (CentOS’s stocked kernel) on CentOS 5 (64 bit)stocked kernel) on CentOS 5 (64 bit)

bridge-utilsbridge-utils ebtablesebtables iptablesiptables IPP2PIPP2P Our in-house developed daemon for Our in-house developed daemon for

automatically adjust the automatically adjust the shaping/blocking policy.shaping/blocking policy.

16

Traffic Shaper/Firewall(Bridge)

GatewayRouter

(OSPF/BGP)

Core Router(OSPF)

Bypass/failover path for IPv4, main connection for IPv6 and multicast IPv4.

NECTEC

UniNet

Simplified Network Simplified Network DiagramDiagram

Gigabit Ethernet Link

10 GigE

Gigabit Ethernet Link

Gig

ab

it E

thern

et

Lin

ks

10 GigE

10 GigE

17

How we shape the trafficHow we shape the traffic

Use iptables’ ‘MARK’ target to mark Use iptables’ ‘MARK’ target to mark the class of traffic for every packetsthe class of traffic for every packets

Hierarchical Token Bucket (HTB) as Hierarchical Token Bucket (HTB) as packet shaperpacket shaper

Stochastic Fairness Queuing (SFQ) Stochastic Fairness Queuing (SFQ) as queuing algorithmas queuing algorithm

18

Traffic ClassificationTraffic Classification

Port-based Port-based Content based (L7)Content based (L7)

using IPP2P through iptablesusing IPP2P through iptables Automatically adjust iptables’ rules Automatically adjust iptables’ rules

using our daemonusing our daemon

19

Sample Reports - Sample Reports - BandwidthBandwidth

Turn off shaping during Friday morning to Monday morning

Incoming Traffic Outgoing Traffic

Stop Shaping Restart Shaping

20

Sample Reports - PacketSample Reports - Packet

Turn off shaping during Friday morning to Monday morning

Incoming Traffic Outgoing Traffic

Stop Shaping Restart Shaping

21

Sample Reports - SYN Flood Sample Reports - SYN Flood BlockingBlocking

A host infected with an Internet worm send a large amount of SYN packets at 9:19.

Bandwidth

Packet

Real Outgoing Traffic

Attempt Outgoing Traffic

22

Sample Reports - Shaping Sample Reports - Shaping by Classesby Classes

Traffic shaping was turned off during 21:21 to 21:53.

23

Sample Reports - Shaping Sample Reports - Shaping by Classesby Classes

P2P Traffic allow in the night.

No P2P allow

P2P allow in the night

24

Misc. reportsMisc. reportsLast seen IP matrix

Detected hosts

Number of last seen hosts

25

ConclusionsConclusions

Complete control of unregistered Complete control of unregistered machinesmachines Prevent unauthorized/unregistered net usagePrevent unauthorized/unregistered net usage

Automatic co-operate between Automatic co-operate between registration and firewall/traffic shapingregistration and firewall/traffic shaping

Complete control of P2P traffics under Complete control of P2P traffics under desired policy (class, usage period, desired policy (class, usage period, bandwidth, etc.)bandwidth, etc.)

Prevent our machines from becoming a Prevent our machines from becoming a source of SYN-flood attacksource of SYN-flood attack

26

Conclusions (cont.)Conclusions (cont.)

Free up NOC officer’s timeFree up NOC officer’s time Real-world, low-cost, high-efficiency Real-world, low-cost, high-efficiency

implementation (currently online)implementation (currently online)

27

ReferencesReferences The Official BitTorrent Home Page The Official BitTorrent Home Page http://whttp://w

ww.bittorrent.org/ww.bittorrent.org/ Kazaa Kazaa http://www.kazaa.com/http://www.kazaa.com/ Netfilter/iptables project homepage Netfilter/iptables project homepage http://http://

www.netfilter.org/www.netfilter.org/ Official IPP2P homepage Official IPP2P homepage http://www.ipp2phttp://www.ipp2p

.org/.org/ HTB home http://luxik.cdi.cz/~devik/qos/ht HTB home http://luxik.cdi.cz/~devik/qos/ht

b/b/ SFQ queuing discipline SFQ queuing discipline

http://www.opalsoft.net/qos/DS-25.htmhttp://www.opalsoft.net/qos/DS-25.htm

28

Questions?

29

Thank you