cs 498 lecture 9 traffic control for qos
TRANSCRIPT
![Page 1: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/1.jpg)
CS 498 Lecture 9Traffic Control for QoS
Jennifer Hou
Department of Computer Science
University of Illinois at Urbana-Champaign
Reading: Chapters 18, The Linux Networking Architecture: Design and Implementation of Network Protocols in the Linux Kernel
![Page 2: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/2.jpg)
Traffic ControlTwo major functions Policing
Usually implemented at the router. Data connections are monitored and packets that are
transmitted violating a specified strategy are discarded.
Traffic shaping Usually implemented at end hosts. Data connections are regulated to conform to certain
rate. Surplus packets are either marked and then sent or delayed at the sender side until the rate constraint no longer holds true.
![Page 3: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/3.jpg)
Processing of Network Data
Input de-multiplexing Forwarding Output queuing
Upper layers (TCP, UDP, …)
Traffic control
Ingress policing
![Page 4: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/4.jpg)
Traffic Control in Linux Kernel
net/core/dev.cdriver.c
net/ipv4/ip_input.c
net/sched/sch_ingress.cTraffic control in
Incoming direction
Forwarding
Local delivery Locally created data
net/core/dev.c
dev_queue_xmit
net/sched/sch_*.cnet/sched/cls_*.cTraffic control inouting direction
dev-> hard_start_xmit
![Page 5: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/5.jpg)
Traffic Control in Linux Kernel......
....
dev.cdev.c
dev.cdev.c
driver.cdriver.c
net_interrupt
netif_rx
net_rx_action
Schedulerdo_softirq
br_input.cbr_input.chandle_bridge
CONFIG_BRIDGE
dev_alloc_skb()
eth_type_trans()
CPU1 CPU2
softnet_data[cpun].input_pkt_queue
arp_rcv ip_rcv p8022_rcv
dev.cdev.c
dev_queue_xmit
dev->qdisc->enqueue
dev.cdev.c
driver.cdriver.cdev->hard_start_xmit
qdisc_run
qdisc_restart
dev->qdisc->dequeue
Scheduler
eth0 eth1
ip_queue_xmitarp_send
...
net_tx_action
ETH_P_802_2
![Page 6: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/6.jpg)
Components of Traffic ControlQueuing disciplines Packets sent are passed to a queueing discipline and
sorted within the queue in compliance with specific rules.
Packets can be removed no earlier than when the queueing discipline has marked them as ready for transmission.
Classes (within a queuing disciplines) Within a queue discipline, packets can be allocated to
different classes.
Filters: are used to allocate packets to classes with a queueing discipline.
![Page 7: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/7.jpg)
Queuing DisciplineEach network device has a queuing discipline
It controls how packets are enqueued on the device are treatedPossible operations: keep, drop, mark
A simple one may just consist of a single queue
Queuing discipline
![Page 8: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/8.jpg)
Complex Queuing Discipline
Queuing discipline May use filters to distinguish
among different classes of packets
Process each class in a specific way
Two filters can point to one class
Classes do not store packets They use another queuing
discipline to do that
Queueing discipline
Filter Filter Filter
Class 2Class 1
QueueingDiscipline
QueueingDiscipline
Enqueuedequeue
![Page 9: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/9.jpg)
Complex Queuing Discipline
![Page 10: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/10.jpg)
Policing
When packets of a connection are enqueued, the connection can be policed:Letting the packets goDropping the packetsLetting the packets go but mark them
![Page 11: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/11.jpg)
Data Structures
Include/net/pkt_sched.h
Include/net/sch_generic.h
![Page 12: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/12.jpg)
Traffic Control in Linux Kernel
Traffic control kernel code resides mainly in net/schedTraffic control in the incoming direction is
handled by net/sched/sch_ingress.c.Various scheduling disciplines in the
outgoing direction are given in net/sched/sch_*.cnet/sched/cls_*.c
![Page 13: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/13.jpg)
Traffic Control in Linux Kernel
Interface used inside the kernel can be found in
/usr/src/linux-(version)/include/net/pkt_cls.h /usr/src/linux-(version)/include/net/pket_sched.h
Interfaces between kernel traffic control and user space programs are decared in
/usr/include/linux/pkt_cls.h /usr/include/linux/pkt_sched.h.
![Page 14: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/14.jpg)
Inserting Traffic Controldev.c, net/sched/*dev.c, net/sched/*
softirq.c, netdevice.hsoftirq.c, netdevice.h
dev->qdisc->enqueue
driver.cdriver.cdev->hard_start_xmit
qdisc_run
qdisc_restart dev->qdisc->dequeue
Scheduler
net_tx_action
dev_queue_xmit
timer_handler netif_scheduleTimer
cpu_raise_softirq
do_softirq
NET_TX_SOFTIRQQueueing discipline
Filter Filter Filter
Class 2Class 1
QueueingDiscipline
QueueingDiscipline
Enqueuedequeue
![Page 15: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/15.jpg)
Queueing Discipline -- Qdiscstruct Qdisc{ int (*enqueue)(struct sk_buff *skb, struct Qdisc *dev); struct sk_buff * (*dequeue)(struct Qdisc *dev); unsigned flags;32 #define TCQ_F_BUILTIN 1#define TCQ_F_THROTTLED 2#define TCQ_F_INGRESS 4 int padded; struct Qdisc_ops *ops; u32 handle; u32 parent; atomic_t refcnt; struct sk_buff_head q; struct net_device *dev; struct list_head list;
struct gnet_stats_basic bstats; struct gnet_stats_queue qstats; struct gnet_stats_rate_est rate_est; spinlock_t *stats_lock; struct rcu_head q_rcu; int (*reshape_fail)(struct sk_buff *skb, struct Qdisc *q); struct Qdisc *__parent;
};
The network device to which the Qdisc isallocated
The Qdisc_ops data structure
The socket buffer queue governed bythis qdisc
When an outer queue passes a packet to an inner queuethe packet may have to be discarded. If the outer queueing discipline implements the callback functionreshape_fail then it can be invoked by the inner queueingdiscipline.
![Page 16: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/16.jpg)
Queuing Disciplines –Qdisc_opsstruct Qdisc_ops {
struct Qdisc_ops *next; struct Qdisc_class_ops *cl_ops;char id[IFNAMSIZ];int priv_size;int (*enqueue)(struct sk_buff *, struct Qdisc *);struct sk_buff * (*dequeue)(struct Qdisc *);int (*requeue)(struct sk_buff *, struct Qdisc *);unsigned int (*drop)(struct Qdisc *);int (*init)(struct Qdisc *, struct rtattr *arg);void (*reset)(struct Qdisc *);void (*destroy)(struct Qdisc *);int (*change)(struct Qdisc *, struct rtattr *arg);int (*dump)(struct Qdisc *, struct sk_buff *); };
The packet should bearranged at the positionin the queueing disciplinewhere it has been before
A queueing discipline can be added via register_qdisc() in init_module()
![Page 17: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/17.jpg)
Qdisc_opsenqueue() Enqueues a packet Return values are
NET_XMIT_SUCCESS, if the packet is accepted NET_XMIT_DROP, if the packet is discarded NET_XMIT_CN, if the packet is discarded because of
buffer overflow NET_XMIT_POLICED, if the packet is discarded
because of violation of a policing rule. NET_XMIT_BYPASS, if the packet is accepted, but will
not leave the queue via the regular dequeue() function.
![Page 18: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/18.jpg)
Qdisc_opsdequeue() Returns a pointer to a packet (skb) eligible for sending A return value of null means that there are no packets ready
to be sent. (The total number of packets in the queue is given in struct Qdisc* qq.len.)
requeue() Puts a packet back into the original position in the queue
where it had been before. The number of packets running through the queue should
not be increased. drop() Drops one packet from the queue
![Page 19: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/19.jpg)
Qdisc_opsinit() Initializes the queuing discipline
reset() Resets the queuing discipline to its initial state (empty queue,
reset counter, delete times)
destroy() Removes a queuing discipline and frees all the resources
reserved during the runtime of the queueing discipline.
change() Changes the parameters of a queuing discipline
dump() Returns output configuration parameters and statistics of a
queueing discipline.
![Page 20: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/20.jpg)
Qdisc_class_opsstruct Qdisc_class_ops { /* Child qdisc manipulation */ int (*graft)(struct Qdisc *, unsigned long cl, struct Qdisc *, struct Qdisc **); struct Qdisc * (*leaf)(struct Qdisc *, unsigned long cl); /* Class manipulation routines */ unsigned long (*get)(struct Qdisc *, u32 classid); void (*put)(struct Qdisc *, unsigned long); int (*change)(struct Qdisc *, u32, u32, struct rtattr **, unsigned long *); int (*delete)(struct Qdisc *, unsigned long); void (*walk)(struct Qdisc *, struct qdisc_walker * arg); /* Filter manipulation */ struct tcf_proto ** (*tcf_chain)(struct Qdisc *, unsigned long); unsigned long (*bind_tcf)(struct Qdisc *, unsigned long, u32 classid); void (*unbind_tcf)(struct Qdisc *, unsigned long); /* rtnetlink specific */ int (*dump)(struct Qdisc *, unsigned long, struct sk_buff *skb, struct tcmsg*); int (*dump_stats)(struct Qdisc *, unsigned long, struct gnet_dump *); };
![Page 21: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/21.jpg)
Qdisc_class_opsgraft(): binds a queueing discipline to a classleaf(): returns a pointer to the queueing discipline currently bound to the classget(): maps the classid to the internal identification and increments the reference counter by one. Each class is associated with two ids
classid (of type u32) is used by the user and the configuration tools used in the user space.
Internal identification (of type unsigned long) is used within the kernel
put(): decrements the usage counter.
![Page 22: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/22.jpg)
Qdisc_class_opschange(): changes the class parametersdelete(): checks if the class is not referenced; and if not, deletes the class.walk(): walks through the linked list of the all the classes of a queueing discipline and invokes the associated callback functions to obtain configuration/statistics data.tcf_chain(): returns a pointer to the linked list for the filter bound to the class.bind_tcf(): binds a filter to a class.dump_class(): gives configuration and statistics data of a class.
![Page 23: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/23.jpg)
tcf_protostruct tcf_proto{ /* Fast access part */ struct tcf_proto *next; void *root; int (*classify)(struct sk_buff*, struct tcf_proto*, struct tcf_result *); u32 protocol; /* All the rest */ u32 prio; u32 classid; struct Qdisc *q; void *data; struct tcf_proto_ops *ops;};
![Page 24: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/24.jpg)
tcf_proto_opsstruct tcf_proto_ops{ struct tcf_proto_ops *next; char kind[IFNAMSIZ]; int (*classify)(struct sk_buff*, struct tcf_proto*, struct tcf_result *); int (*init)(struct tcf_proto*); void (*destroy)(struct tcf_proto*); unsigned long (*get)(struct tcf_proto*, u32 handle); void (*put)(struct tcf_proto*, unsigned long); int (*change)(struct tcf_proto*, unsigned long, u32 handle, struct rtattr **, unsigned long *); int (*delete)(struct tcf_proto*, unsigned long); void (*walk)(struct tcf_proto*, struct tcf_walker *arg); /* rtnetlink specific */ int (*dump)(struct tcf_proto*, unsigned long, struct sk_buff *skb, struct tcmsg*); struct module *owner;};
![Page 25: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/25.jpg)
tcf_proto_ops
classify(): classifies a packet (checks if the filtering rule applies to the packet) Possible return values are
TC_POLICE_OK: the packet is accepted by the filter. TC_POLICE_RECLASSIFY: the packet violates agreed
parameters and should be allocated to a different class. TCP_POLICE_SHOT: the packet was dropped because
of violation of agreed parameters TCP_POLICE_UNSPEC: The rule does not match the
packet, and the packet should be passed to the next filter. tcf_result contains the classid and the internal
identification of the class.
![Page 26: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/26.jpg)
Queueing Discipline Example
net/sched/sch_red.c
![Page 27: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/27.jpg)
RED
![Page 28: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/28.jpg)
Dropping Probability pa
ap
pb
minth maxth
maxp
1
countpp ba * Linux implementation
![Page 29: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/29.jpg)
RED implementation Istruct red_sched_data { /* Parameters */
u32 limit; /* HARD maximal queue length */ u32 qth_min; /* Min average length threshold: A scaled */ u32 qth_max; /* Max average length threshold: A scaled
*/ char Wlog; /* log(W) */ char Plog; /* random number bits */ …unsigned long qave; /* Average queue length: A scaled */ int qcount; /* Packets since last random number
generation */ u32 qR; /* Cached random number */ psched_time_t qidlestart; /* Start of idle period */ struct tc_red_xstats st; };
![Page 30: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/30.jpg)
RED implementation II: Compute average queue length
We want: avg = avg * (1- w) +w * backlog
Code in Linux:q->qave += sch->stats.backlog - (q-
>qave >> q->Wlog);
Why:avg = q->qave * ww = 2^(-wlog)
![Page 31: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/31.jpg)
RED implementation III
Ideally avg should be calculated every constant clock interval
In Linux it is updated every packet outgoing
Care need to be taken for idle period
![Page 32: CS 498 Lecture 9 Traffic Control for QoS](https://reader035.vdocuments.us/reader035/viewer/2022070315/554f3e1bb4c905cd048b528a/html5/thumbnails/32.jpg)
RED implementation IV:Decide dropping probabilityWe want: enqueue if
Linux code: if (((q->qave - q->qth_min)>>q-
>Wlog)*q->qcount < q->qR) goto enqeue;
max_P = (qth_max – qth_min)/2^Plog
q->qR = rnd * 2^Plog
rndcountPavg
thth
th
*max_*minmax
min