traffic control - funtoo linux

7/27/2019 Traffic Control - Funtoo Linux

1/6

From Funtoo Linux

1 Introduction2 Incoming and Outgoing Traffic3 Recommended Resources4 Recommended Approaches5 State of the Code6 Inspect Your Rules7 Matching8 Sample Traffic Control Code

8.1 tc code walkthrough8.2 iptables code walkthrough8.3 Further optimizations

8.3.1 SSH8.3.2 ACKs

9 Other Links of Interest

Linux's traffic control functionality offers a lot of capabilities related to influencing the rate of flow, as well aslatency, of primarily outgoing but also in some cases incoming network traffic. It is designed to be a"construction kit" rather than a turn-key system, where complex network traffic policing and shaping decisionscan be made using a variety of algorithms. The Linux traffic control code is also often used by academia forresearch purposes, where is it can be a useful mechanism to simulate and explore the impact of a variety ofdifferent network behaviors. See netem (http://www.linuxfoundation.org/collaborate/workgroups/networking/netem) for an example of a simulation framework that can be used for this purpose.

Of course, Linux traffic control can also be extremely useful in an IT context, and this document is intended tofocus on the practical, useful applications of Linux traffic control, where these capabilities can be applied tosolve problems that are often experienced on modern networks.

One common use of Linux traffic control is to configure a Linux system as a Linux router or bridge, so that theLinux system sits between two networks, or between the "inside" of the network and the real router, so that itcan shape traffic going to local machines as well as out to the Internet. This provides a way to prioritize, shapeand police both incoming (from the Internet) and outgoing (from local machines) network traffic, because it iseasiest to create traffic control rules for traffic flowing outof an interface, since we can control when the systemsends data, but controlling when we receive data requires an additional intermediate queue to be created to

c Control - Funtoo Linux http://www.funtoo.org/wiki/Traffi

6/29/2011


2/6

buffer incoming data. When a Linux system is configured as a firewall or router with a physical interface foreach part of the network, we can avoid using intermediate queues.

A simple way to set up a layer 2 bridge using Linux involves creating a bridge device with brctl , adding twoEthernet ports to this bridge (again using brctl ), and then apply prioritization, shaping and policing rules to both

interfaces. The rules will apply to outgoing traffic on each interface. One physical interface will be connected toan upstream router on the same network, while the other network port will be connected to a layer 2 accessswitch to which local machines are connected. This allows powerful egress shaping policies to be created onboth interfaces, to control the flows in and out of the network.

Resources you should take a look at, in order:

HTB documentation (http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm) by Martin Devera. Best way tocreate different priority classes and bandwidth allocations.Differentiated Services On Linux HOWTO (http://www.opalsoft.net/qos/DS.htm) by Leonardo Balliache.Good general docs.

A Practical Guide to Linux Traffic Control (http://blog.edseek.com/~jasonb/articles/traffic_shaping/index.html) by Jason Boxman. Good general docs.IFB - replacement for Linux IMQ (http://www.linuxfoundation.org/collaborate/workgroups/networking/ifb) , with examples. This is the official best way to do inboundtraffic control, when youdon't have dedicated in/out interfaces.Use of iptables hashlimit (http://seclists.org/fulldisclosure/2006/Feb/702) - Great functionality in iptables.There's a hashlimit example below as well.

Related Interesting Links:

Second Life Bandwidth Testing Protocol (http://wiki.secondlife.com/wiki/BLT) - example of Netem

UDP Buffer Sizing (http://www.29west.com/docs/THPM/udp-buffer-sizing.html) , part of Topics in HighPerformance Messaging (http://www.29west.com/docs/THPM/index.html)

Daniel Robbins has had very good results with the HTB queuing discipline (http://luxik.cdi.cz/~devik/qos/htb/) -it has very good features, and also has very good documentation (http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm) , which is just as important, and is designed to deliver useful results in a production environment.And it works. If you use traffic control under Funtoo Linux, please use the HTB queuing discipline as the rootqueuing discipline because you will get good results in very little time. Avoid using any other queuing discipline

under Funtoo Linux as the rootqueuing discipline on any interface. If you are creating a tree of classes andqdiscs, HTB should be at the top, and you should avoid hanging classes under any other qdisc unless you haveplenty of time to experiment and verify that your QoS rules are working as expected. Please see State of theCode for more info on what Daniel Robbins considers to be the current state of the traffic controlimplementation in Linux.

If you are using enterprise kernels, especially any RHEL5-based kernels, you must be aware that the trafficcontrol code in these kernels is about 5 years old and contains many significant bugs. In general, it is possible to


6/29/2011


3/6

avoid these bugs by using HTB as your root queueing discipline and testing things carefully to ensure that youare getting the proper behavior. The pr i o queueing discipline is known to not work reliably in RHEL5 kernels.

See Broken Traffic Control for more information on known bugs with older kernels.

If you are using a more modern kernel, Linux traffic control should be fairly robust. The examples below shouldwork with RHEL5 as well as newer kernels.

If you are implementing Linux traffic control, you should be running these commands frequently to monitor thebehavior of your queuing discipline. Replace $wani f with the actual network interface name.

t c - s qdi sc ls dev $wani f

t c - s c l ass ls dev $wani f

Here are some examples you can use as the basis for your own filters/classifiers:

prot ocol ar p u32 mat ch u32 0 0 - match ARP packets1.pr ot ocol i p u32 mat ch i p pr otocol 0x11 0xf f - match UDP packets2.pr ot ocol i p u32 mat ch i p pr otocol 17 0xf f - (also) match UDP packets3.pr ot ocol i p u32 mat ch i p pr otocol 0x6 0xf f - match TCP packets4.pr ot ocol i p u32 mat ch i p pr otocol 1 0xf f - match ICMP (ping) packets5.pr otocol i p u32 match i p dst 4. 3. 2. 1/ 32 - match all IP traffic headed for IP 4.3.2.16.pr otocol i p u32 match i p sr c 4. 3. 2. 1/ 32 match i p spor t 80 0xf f f f - match all IP traffic from4.3.2.1 port 80

7.

pr otocol i p u32 match i p spor t 53 0xf f f f - match originating DNS (both TCP and UDP)8.pr ot ocol i p u32 mat ch i p dport 53 0xf f f f - match response DNS (both TCP and UDP)9.pr ot ocol i p u32 mat ch i p pr ot ocol 6 0xf f mat ch u8 0x10 0xf f at next hdr +13 - matchpackets with ACK bit set

10.

prot ocol i p u32 mat ch i p prot ocol 6 0xf f mat ch u8 0x10 0xf f at next hdr +13 mat ch u16

0x0000 0xf f c0 at 2 - packets less than 64 bytes in size with ACK bit set11.

pr ot ocol i p u32 mat ch i p t os 0x10 0xf f - match IP packets with "type of service" set to"Minimize delay"/"Interactive"

12.

pr ot ocol i p u32 mat ch i p t os 0x08 0xf f - match IP packets with "type of service" set to"Maximize throughput"/"Bulk" (see "QDISC PARAMETERS" in t c - pr i o man page)

13.

pr ot ocol i p u32 mat ch t cp dport 53 0xf f f f match i p pr otocol 0x6 0xf f - match TCP

packets heading for dest. port 53 (my not work)

14.

modemi f=et h4i pt abl es - t mangl e - A POSTROUTI NG - o $modemi f - p t cp - m t os - - t o s Mi ni mi ze- Del ay - j CLi pt abl es - t mangl e - A POSTROUTI NG - o $modemi f - p t cp - - dpor t 53 - j CLASSI FY - - set - cl asi pt abl es - t mangl e - A POSTROUTI NG - o $modemi f - p t cp - - dpor t 80 - j CLASSI FY - - set - cl asi pt abl es - t mangl e - A POSTROUTI NG - o $modemi f - p t cp - - dpor t 443 - j CLASSI FY - - s et - cl a


6/29/2011


4/6

t c qdi sc add dev $modemi f r oot handl e 1: ht b def aul t 12t c cl ass add dev $modemi f par ent 1: cl assi d 1: 1 ht b r at e 1500kbi t cei l 1500kbi t bur stt c cl ass add dev $modemi f par ent 1: 1 cl assi d 1: 10 ht b r at e 700kbi t cei l 1500kbi t pr i ot c cl ass add dev $modemi f par ent 1: 1 cl assi d 1: 12 ht b r at e 800kbi t cei l 800kbi t pr i o 2t c f i l t er add dev $modemi f pr ot ocol i p parent 1: 0 pr i o 1 u32 mat ch i p pr ot ocol 0x11 0xt c qdi sc add dev $modemi f parent 1: 10 handl e 20: sf q pert ur b 10t c qdi sc add dev $modemi f parent 1: 12 handl e 30: sf q pert ur b 10

The code above is a working traffic control script that is even compatible with RHEL5 kernels, for a 1500kbpsoutbound link (T1, Cable or similar.) In this example, et h4 is part of a bridge. The code above should workregardless of whetheret h4 is in a bridge or not -- just make sure that modemi f is set to the interface on which

traffic is flowing outand you wish to apply traffic control.

tc code walkthrough

This script uses the t c command to create two priority classes - 1:10 and 1:12. By default, all traffic goes into

the low-priority class, 1:12. 1:10 has priority over 1:12 (pr i o 1 vs. pr i o 2,) so if there is any traffic in 1:10

ready to be sent, it will be sent ahead of 1:12. 1:10 has a rate of 700kbit but can use up to the full outbound

bandwidth of 1500kbit by borrowing from 1:12.

UDP traffic (traffic that matches i p pr ot ocol 0x11 0xf f) will be put in the high priority class 1:10. This can

be good for things like FPS games, to ensure that latency is low and not drowned out by lower-priority traffic.

If we stopped here, however, we would get a bit worse results than if we didn't use t c at all. We have basically

created two outgoing sub-channels of different priorities. The higher priority class can drown out the lower-priority class, and this is intentional so it isn't the issue -- in this case we wantthat functionality. The problem isthat the high priority and low priority classes can both be dominated by high-bandwidth flows, causing othertraffic flows of the same priority to be drowned out. To fix this, two sf q queuing disciplines are added to thehigh and low priority classes and will ensure that individual traffic flows are identified and each given a fair shotat sending data out of their respective classes. This should prevent starvation within the classes themselves.

iptables code walkthrough

First note that we are adding netfilter rules to the POSTROUTI NGchain, in the mangl e table. This table allows us

to modify the packets right before they are queued to be sent out of an interface, which is exactly what wewant. At this point, these packets could have been locally-generated or forwarded -- as long as they are on theirway to going out ofmodemi f (eth4 in this case), the mangl ePOSTROUTI NGchain will see them and we canclassify them and perform other useful tweaks.

The iptables code puts all traffic with the "minimize-delay" flag (interactive ssh traffic, for example) in the highpriority traffic class. In addition, all HTTP, HTTPS and DNS TCP traffic will be classified as high-priority.Remember that all UDP traffic is being classified as high priority via the t c rule described above, so this will

take care of DNS UDP traffic automatically.

Further optimizations

SSH


6/29/2011


5/6

i pt abl es - t mangl e - N t osf i xi pt abl es - t mangl e - A t osf i x - p t cp - m l engt h - - l engt h 0: 512 - j RETURN#allow screen redraws under interactive SSH sessions to be fast:

i pt abl es - t mangl e - A t osf i x - m hashl i mi t - - hashl i mi t 20/sec - - hashl i mi t - burst 20 \

- - hashl i mi t - mode src i p, srcport , dst i p, dstport - - hashl i mi t - name mi nl at - j RETURNi pt abl es - t mangl e - A t osf i x - j TOS - - s e t - t o s Maxi mi ze- Thr oughputi pt abl es - t mangl e - A t osf i x - j RETURN

i pt abl es - t mangl e - A POSTROUTI NG - p t cp - m t os - - t o s Mi ni mi ze- Del ay - j t osf i x

To use this code, place it near the top of the file, just below the modemi f =" et h4" line, but before the main

i pt abl es andt c rules. These rules will apply to all packets about to get queued to any interface, but this is not

necessarily a bad thing, since the TCP flags being set are not just specific to our traffic control functionality. Tomake these rules specific to modemi f, add "-o $modemif" after "-A POSTROUTING" on the last line, above.

As-is, the rules above will set the TCP flags on all packets flowing out of all interfaces, but the the traffic controlrules will only take effect formodemi f, because they are only configured for that interface.

SSH is a tricky protocol. By default, all the outgoing SSH traffic is classified as "minimize-delay" traffic, which

will cause it to all flow into our high-priority class, even if it is a bulkscp transfer running in the background.This code will grab all "minimize-delay" traffic such as SSH and telnet and route it through some special rules.Any individual keystrokes (small packets) will be left as "minimize-delay" packets. For anything else, we willrun the hashl i mi t iptables module, which will identify individual outbound flows and allow small bursts oftraffic (even big packets) to remain "minimize-delay" packets. These settings have been specifically tuned sothat most GNU scr een screen changes (^A^N) when logging into your server(s) remotely will be fast. Any traffic

over these burst limits will be reclassified as "maximize-throughput" and thus will drop to our lower-priorityclass 1:12. Combined with the traffic control rules, this will allow you to have very responsive SSH sessions intoyour servers, even if they are doing some kind of bulk outbound copy, like rsync over SSH.

Code in our main i pt abl es rules will ensure that any "minimize-delay" traffic is tagged to be in the

high-priority 1:10 class.

What this does is keep interactive SSH and telnet keystrokes in the high-priority class, allow GNU screen fullredraws and reasonable full-screen editor scrolling to remain in the high-priority class, while forcing bulktransfers into the lower-priority class.

ACKs

i pt abl es - t mangl e - N acki pt abl es - t mangl e - A ack - m t os ! - - t o s Normal - Servi ce - j RETURN

i pt abl es - t mangl e - A ack - p t cp - m l engt h - - l engt h 0: 128 - j TOS - - s e t - t o s Mi ni mi ze- Dei pt abl es - t mangl e - A ack - p t cp - m l engt h - - l engt h 128: - j TOS - - s e t - t o s Maxi mi ze- Thri pt abl es - t mangl e - A ack - j RETURNi pt abl es - t mangl e - A POSTROUTI NG - p t cp - m t cp - - t c p- f l ags SYN, RST, ACK ACK - j ack

To use this code, place it near the top of the file, just below the modemif="eth4" line, butbefore the main

iptables andtc rules.

ACK optimization is another useful thing to do. If we prioritize small ACKs heading out to the modem, it will


6/29/2011


6/6

allow TCP traffic to flow more smoothly without unnecessary delay. The lines above accomplish this.

This code basically sets the "minimize-delay" flag on small ACKs. Code in our main i pt abl es rules will then

tag these packets so they enter high-priority traffic class 1:10.

http://pupa.da.ru/tc/http://manpages.ubuntu.com/manpages/maverick/en/man8/ufw.8.htmlhttps://help.ubuntu.com/community/UFW

Retrieved from "http://www.funtoo.org/wiki/Traffic_Control"Categories: Investigations | Articles | Featured | Networking

This page was last modified on 29 June 2011, at 04:44.This page has been accessed 1,609 times.

Privacy policyAbout Funtoo LinuxDisclaimers


traffic control - funtoo linux

Documents