Research ArticleOptimal Control for Bufferbloat Queue ManagementUsing Indirect Method with Parametric Optimization

Amr Radwan1 Hoang-Linh To1 and Won-Joo Hwang2

1Department of Information and Communication System Inje University Gimhae Gyeongnam 50834 Republic of Korea2Department of Information and Communications Engineering HSV-TRC Inje University GimhaeGyeongnam 50834 Republic of Korea

Correspondence should be addressed to Won-Joo Hwang ichwanginjeackr

Received 20 May 2016 Accepted 31 July 2016

Academic Editor Junhu Ruan

Copyright copy 2016 Amr Radwan et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Because memory buffers become larger and cheaper they have been put into network devices to reduce the number of losspackets and improve network performance However the consequences of large buffers are long queues at network bottlenecksand throughput saturation which has been recently noticed in research community as bufferbloat phenomenon To address suchissues in this article we design a forward-backward optimal control queue algorithmbased on an indirect approachwith parametricoptimization The cost function which we want to minimize represents a trade-off between queue length and packet loss rateperformance Through the integration of an indirect approach with parametric optimization our proposal has advantages ofscalability and accuracy compared to direct approaches while still maintaining good throughput and shorter queue length thanseveral existing queue management algorithms All numerical analysis simulation in ns-2 and experiment results are provided tosolidify the efficiency of our proposal In detailed comparisons to other conventional algorithms the proposed procedure can runmuch faster than direct collocation methods while maintaining a desired short queue (asymp40 packets in simulation and 80 (ms) inexperiment test)

1 Introduction

Nowadays modern computer networks are incredibly com-plex and we rely on them to transport huge quantities ofdata across the globe in seconds Although this works wellthere are some foreseen issues that need to be tackled Asbandwidth-heavy applications such as peer-to-peer networksand websites relying on user-generated content have becomemore prevalent especially the relatively slow residentialbroadband links have been used at full capacity and interrup-tions in connectivity have become more common [1] Recentresearch has shown that the culprit is the buffers built intonetwork equipment [2] Accordingly bufferbloat term hasbeen used to describe related issues whenever these buffersmisbehave to produce unnecessary latency [3]

In order to efficiently manage queues which are gen-erated due to the bufferbloat phenomenon active queuemanagement (AQM) algorithms have been recommendedto use in network equipment [4] Most of the existing

approaches to AQM design exploit feedback control theorywith the linearized TCP core model that was proposed byHollot et al [5] The well-known AQMs that monitor theaverage queue size and drops (or marks) packets based onstatistical probabilities are Random Early Detection (RED)[6] or Random Early Marking (REM) [7] If the buffer isempty all incoming packets are accepted When the bufferis full the probability has reached 1 and all incoming packetsare dropped When queue is growing the probability growsaccording to a piecewise linear function and RED (or REM)drops (or marks) packets using the updated probabilityThe main drawback of RED or REM is that it is sensitiveto network parameter changes and requires careful tuningof its parameters in order to provide optimal performancein any scenarios Recently in [8] a Proportional IntegralEnhanced (PIE) controller as a lightweight AQM is proposedwithout the need of per-packet extra processing Such PI-type controllers are known to provide queue control with zerooffset (the mean queue length converges to the target value)

Hindawi Publishing CorporationScientific ProgrammingVolume 2016 Article ID 4180817 10 pageshttpdxdoiorg10115520164180817

2 Scientific Programming

but are consequently less stable and slower reacting Therehave been several other control-theoretic solutions based onfeedback fluid-flow model proposed in [9ndash12] to improvestable and robust control mechanisms but none of themtackle the issue of system optimality in terms of minimizingboth queue length and packet dropping rate performance andsearching for an optimal control trajectory

To address such issues a promising design direction is toreformulate the network queuing problem as an optimal con-trol queue (OCQ) problem [13] where themain state variableis queue length and the control variable is the actual input rateto form the queue [14] or packet dropping rate Then one ofthe approaches to solve OCQ is that we can priorly discretizethe governing ordinary differential equations (ODEs) and theintegral terms in the cost function or constraint functionsand thereby replace the infinite dimensional optimal controlproblemwith a large nonlinear optimization problem (NOP)This is known as the direct method in the literature for solvingOCQ This approach is typically easier to use especially forOCQ with a state equality or inequality constraints Themain difference among direct approaches is how to handlethe constraints corresponding to the system dynamics Thethree most common direct approaches are direct singleshooting direct multiple shooting and direct collocationDirect methods have been used with references in [15ndash18]

Alternatively one can first form the optimality condi-tions using the calculus of variations and Pontryagin mini-mum principle and then solve the resulting boundary valueproblem This is known as the indirect method for solvingOCQ The references present just a small sample of the workthat discusses or applies indirect methods for the solution ofoptimal control problems [15 16 18 19] In rare cases thesolution can be obtained in closed form from the optimalityconditions but in general approximation methods are usedto solve the problem numerically The optimality conditionsof these problems generally take the form of differentialalgebraic equations (DAEs) with boundary conditions (BCs)The approximate solution to the OCQ can be obtained byusing a boundary value problem (BVP) solver Perhaps themost popular methods aremultiple shooting and collocationMore recently a combination of direct and indirect methodswas proposed leading to hybrid methods [16]

In this paper we firstly derive methods to solve OCQusing both direct and indirect approaches We show how toapply the direct collocation approach to solve OCQ problemwhich can be solved in popular optimization solvers suchas JModelica [20] and GAlib [21] Indirect method withforward-backward for optimal control queue algorithm (FB-OCQ) is designed as an alternative to tackle it as well Ourkey difference from existing works is that we provide a novelmethod to update the control step 120575119906 by solving a parametricoptimization subproblem This method is scalable whichmeans that we can expand for larger problems with morevariables as well The numerical results for both direct andindirect approaches are discussed in Section 51 to emphasizeour choice of integrating an indirect method with parametricoptimization for active queue management design which isdemonstrated more efficiently (ie faster reaction and morestable) than directmethods Finally we evaluate the proposed

actual rateinput rate

drop rate

service rate

w(t) u(t)

d(t) = w(t) minus u(t)

x(t) queue length

120583

Figure 1 OCQ model

algorithm using network simulator ns-2 and compare it toother AQMs including RED REM and PI The droppingfeature of the proposed algorithm makes the average queuelength shortened and stabilized compared with others whilethroughput is not reduced so much

Our Contributions(i) We present both indirect and direct methods for an

optimal control problem which is applied in queuemanagement field (Section 3) The step update 120575119906 ofcontrol variable is calculated by solving a subproblemof parametric optimization with the advantage ofscalability Numerical results show that all of thembring nearly similar results but indirect method (FB-OCQ) is much faster than other solvers and gives thebest cost function value (Section 51)

(ii) We evaluate FB-OCQ in simulation and show that adesired small queue length value at 40 packets canbe obtained Nevertheless we cannot avoid the trade-off between queue length and throughput FB-OCQrsquosthroughput is slightly smaller than REDrsquos one (0075versus 0082Mbps) however it can be an acceptablevalue when compared to REMrsquos and PIrsquos (Section 52)

(iii) We implement FB-OCQ in Linux kernel (Ubuntu1604) and test it in the worst case using RealtimeResponse Under Load (RRUL) test suite The exper-iment result shows that our algorithm brings lowlatency ping value compared with the other existingalgorithms in Linux kernel (DropTail and RED)

2 Problem Formulation

We consider an optimal control model of queuemanagementproblem named (OCQ) in [14] (Figure 1) The main ideais to minimize the cost function which implies a trade-offbetween queue length anddropping rate subject to a dynamicconstraint of queue length along time 119905

0to 119905119891as follows

minimize119909

119869 = int

119905119891

119905=0

119871 (119909 (119905) 119906 (119905) 119905)

subject to = 119891 (119909 (119905) 119906 (119905) 119905)

0 le 119906 (119905) le 119908 (119905)

(OCQ)

where 119871(119909 119906 119905) = 119909(119905) + 119877(119908 minus 119906(119905)) is cost function119891(119909(119905) 119906(119905) 119905) = 119906(119905) minus 120583119909(119886 + 119909) 119877 is weight on droppingrate 119905

119891is final time 120583 is service rate (bandwidth capacity)

and 119886 is parameter for different types of queuing model forexample when 119886 = 1 we obtain an MM1 queue

Scientific Programming 3

3 Numerical Methods for OCQ

31 Direct Method Collocation Method In this section wepresent how to apply the direct collocation [18] for OCQ Inthis method we discretize the time interval 119905

0= 1199051

lt 1199052

lt

1199053

lt sdot sdot sdot lt 119905119896

= 119905119891into 119873 elements The state and the

total number of packets and control variables at each nodeare 119909119895= 119909(119905119895) and 119908

119895= 119908(119905

119895) and 119906

119895= 119906(119905119895) such that the

state control and packets variables at the nodes are definedas nonlinear programming (NLP) variables

119884 = [119909 (1199051) 119909 (119905

119896) 119906 (119905

1) 119906 (119905

119896) 119908 (119905

1) 119908 (119905

119896)] (1)

The controls are chosen as piecewise linear interpolatingfunctions between 119906(119905

119895) and 119906(119905

119895+1) for 119905

119895le 119905 le 119905

119895+1as

follows

119906app (119905) = 119906 (119905119895) +

119905 minus 119905119895

ℎ119895

[119906 (119905119895+1

) minus 119906 (119905119895)]

ℎ119895= 119905119895+1

minus 119905119895

(2)

The value of the control variables at the center 119905119895+12

is givenby

119906 (119905119895+12

) =

119906 (119905119895) + 119906 (119905

119895+1)

2

119895 = 1 119896 minus 1(3)

The piecewise linear interpolation is used to preparefor the possibility of discontinuous solutions in controlSimilarly we can derive the approximate of the total numberof the packets 119908app(119905) and 119908(119905

119895+12) as above The state

variable 119909(119905) is approximated by a continuously differentiableand piecewise Hermite-Simpson cubic polynomial between119909(119905119895) and 119909(119905

119895+1) on the interval 119905

119895le 119905 le 119905

119895+1of length ℎ

119895

119909app (119905) =

3

sum

119903=0

119888

119895

119903

119905 minus 119905119895

ℎ119895

119888

119895

0= 119909 (119905

119895)

119888

119895

1= ℎ119895119891119895

119888

119895

2= minus3119909 (119905

119895) minus 2ℎ

119895119891119895+ 3119909 (119905

119895+1) minus (ℎ

119895) 119891119895+1

119888

119895

3= 2119909 (119905

119895) + ℎ119895119891119895minus 2119909 (119905

119895+1) + (ℎ

119895) 119891119895+1

(4)

where

119891119895= 119891 (119909 (119905

119895) 119906 (119905

119895) 119908 (119905

119895) 119905119895)

119905119895le 119905 le 119905

119895+1 119895 = 1 119896 minus 1

(5)

The value of the state variables at the center point 119905119895+12

of thecubic approximation is

119909 (119905119895+12

) =

119909 (119905119895) + 119909 (119905

119895+1)

2

+

119905119895+1

+ 119905119895

8

(119891119895+ 119891119895+1

)

119895 = 1 119896 minus 1

(6)

and the derivative is

(119905119895+12

) =

3 (119909119895+ 119909119895+1

)

2 (119905119895+1

+ 119905119895)

minus

1

4

(119891119895+ 119891119895+1

)

119895 = 1 119896 minus 1

(7)

In addition the chosen interpolating polynomial for the stateand control variablesmust satisfy themidpoint conditions forthe differential equations as follows

119891 (119909app (119905119895+12) 119906app (119905119895+12) 119908app (119905119895+12) 119905119895+12

)

minus app (119905119895+12) = 0

(8)

Equations (OCQ) can now be defined as a discretizedproblem as follows

min 119891 (119884) (9)

subject to 119891 (119909app (119905) 119906app (119905) 119908app (119905) 119905) minus app

= 0

119909app (1199051) minus 1199091= 0

0 le 119906app (119905) le 119908app (119905)

(10)

where 119909app 119906app and119908app are the approximations of the statethe control and the total number of packets constituting119884 in(9) This above discretization problem (9)-(10) can be solvedusing the following

(i) JModelica which is a package for simulation andoptimization ofModelicamodels (formore details see[20])

(ii) GAlib which is C++ library of genetic algorithm (formore details see [21])

32 Indirect Method Forward-Backward Sweeping We solveOCQ by using indirect method approach

(i) forming optimality conditions(ii) solving BVP by First-Order Sweeping algorithm

Let 119909(119905) be the adjoint variable At time 119905 let 119906lowast(119905) denotethe optimum controls and let 119909

lowast(119905) and 119909

lowast(119905) denote the

state and adjoint evaluated at the optimumUsing Pontryaginminimum principle we get the following equations

Hamiltonian Function

119867(119909 (119905) 119906 (119905) 119909 (119905)) = 119909 (119905) + 119877 (119908 (119905) minus 119906 (119905))

+ 119909 (119905) (119906 (119905) minus

120583119909 (119905)

119886 + 119909 (119905)

)

(11)

Adjoint Equations

119909 (119905) = minus1 + 119906 (119905)

119886120583

(119886 + 119909 (119905))

2 (12)

4 Scientific Programming

Transversality Condition

119909 (119905119891) = 0 (13)

Hamiltonian Minimization Condition Derivative of theHamiltonian is evaluated to zero at interior points hence

119867119906 (

119909 (119905) 119906 (119905) 119909 (119905) 120582 (119905) 119905)

= minus119877 + 119909 (119905) minus 1205821(119905) + 120582

2(119905) = 0

(14)

If 119906

lowast(119905) is optimal in (OCQ) then it must satisfy the

minimum condition119867(119909

lowast 119906

lowast 119909

lowast 119905) = min

Visin119880119867(119909 V 119909 119905) forall119905 isin 119873 (15)

Furthermore we assume the following

(H1) the Hamiltonian is strictly convex with respect tocontrol variable 119906

(H2) (119905) = argmin119906isin119880

119867(119909 119906 119909) 119905 isin [1199050 119905119891] is continu-

ous on [1199050 119905119891]

(H3) 119906 isin 119862

119903[1199050 119905119891]

Note that 119906(119905) is determined in a unique way for each119905 since 119880 is convex and 119867 is convex Now we consider theproblem of minimizing the Hamiltonian with respect to 119901

min119906isin119880

119867(119909 119906 119909) for each 119905 isin [1199050 119905119891] (16)

Usually in the literature 119906 is found explicitly as a function119906 = 119906(119909 119909 119905) and after substituting it into the system theproblem reduces to the boundary value problem

Theorem 1 Assume that conditions (H1)ndash(H3) hold andproblem has an optimal solution (119906

lowast 119909

lowast) Then for a given 120598

there exists a finite discretization

1199050= 1205910lt 1205911lt sdot sdot sdot lt 120591

119894lt sdot sdot sdot lt 120591

119873= 119905119891 (17)

and an approximate solution (119905) 119905 isin [1199050 119905119891] such that

1003817100381710038171003817119906

lowast(119905119894) minus (119905

119894)

1003817100381710038171003817lt 120598 119894 = 1 2 119873 (18)

Proof Let 119906

lowast be an optimal solution of problem Then(119901

lowast 119909

lowast) satisfies the conditions

lowast

119894=

120597119867 (119909

lowast 119906

lowast 119909

lowast)

120597119906119894

119909

lowast

119894= minus

120597119867 ((119909

lowast 119906

lowast 119909

lowast))

120597119909119894

119894 = 1 2 119873

(19)

where

119867(119909 119906 119909) =

119899

sum

119894=1

119909119894(119905) 119891119894(119909 119906) minus 119891

0(119909 119906)

119905 isin [1199050 119905119891]

(20)

and (119906

lowast 119909

lowast) satisfies the minimum principle

119867(119909

lowast 119906

lowast 119909

lowast) = min119906isin119880

119867(119909

lowast 119906 119909

lowast) 119905 isin [119905

0 119905119891] (21)

(a) First-Order Sweeping Method We linearize the Hamilto-nian around a reference solution119906(119905) and obtain the followingequation for variation 120575119906(119905)

0 = 119867119906

⊤+ 119867119906119906

120575119906 (119905) (22)

With the strong Legendre-Clebsch condition one canapproximate the above equation as

120575119906 (119905) = arg min120575119906isin119880minus119906

119867aug (119909 119906 + 120575119906 119909 119905) (23)

where

119867aug (119909 119906 + 120575119906 119909 119905) = 119867 (119909 119906 + 120575119906 119909 119905) + 120573 120575119906

2 (24)

is the augmented Hamiltonian function and 120573120575119906

2 is apenalty term This is really parametric optimization with120573 = 0 initially if119880 is a convex set and119867 is a convex functionwith respect to 119906 then 120575119906 is a descent direction If 119906(119905)+120575119906(119905)

does not yield a reduction then we set 120573 = 1 and then doubleit repeatedly until the objective is really reduced

Remark 2 An alternative way to ensure descent is to applyBacktracking Line-Search Procedure

(b) Parametric Optimization to OCQ Now we considerproblem (23) as one parametric minimization problemSince 119867aug(119909

lowast 119909

lowast 119906

lowast 120575119906 119905) is twice differentiable in 120575119906 and

assumptions (H1)ndash(H3) hold we can apply Theorem 1 to theproblem Then as a result it generates a discretization 119905

0=

1205910lt 1205911lt sdot sdot sdot lt 120591

119894lt sdot sdot sdot lt 120591

119873= 119905119891 and corresponding points

119894= (119905119894) such that1003817100381710038171003817120575119906

lowast(119905119894) minus 120575 (119905

119894)

1003817100381710038171003817lt 120598 119894 = 1 2 119873 (25)

which proves the assertion

Remark 3 Parametric optimization also can be applied infinding nominal optimal control given in [22] It is easy to seethat at each iteration 119896 the Hamiltonian function is a scalarfunction of 119906 isin 119880 sub 119877

119903 and 119905 isin 119879 = [1199050 119905119891] that is

119866119896(120575119906 119905) = 119867aug (119909

119896(119905) 119909

119896(119905) 120575119906 119905) (26)

The latter states that 120575

119896(119905) must be a minimizer of the

following problem

min120575119906isin119880

119866119896(120575119906 119905) 119905 isin 119879 (27)

which is a problem of parametric optimization as formulatedin various papers from [23] where the independent variable 119905is now considered as unknown parameter 119905 isin 119879 = [119905

0 119905119891]We

can also consider a case when the set of admissible controls istime-varying that is 119880 = 119880(119905) 119905 isin 119879 = [119905

0 119905119891] In this case

a general theory of parametric optimization is also applicablefor finding the nominal optimal controls

Scientific Programming 5

Let (119909lowast 120575119906lowast) be an optimal process in problem Introducethe function 119891 119880 times 119877 rarr 119877

119891 (120575119906 119905) = minus119867 (119909

lowast 120575119906 119909

lowast 119905) 119905 isin [119905

0 119905119891] (28)

where a parametric optimization problem is defined as

min120575119906isin119880

119891 (120575119906 119905) 119905 isin [1199050 119905119891]

119880 = 120575119906 isin 119877

119903 119892119894(120575119906) le 0 119894 isin 119869

119869 = 1 2 119904

(29)

The KKT conditions for the problem state that

119863120575119906119891 (120575119906 119905) + sum

119895isin119869

120583119894119863120575119906119892119894(120575119906 119905) = 0

119892119894(120575119906 119905) 120583

119894le 0 120583

119894ge 0 119894 isin 119869

120583119894119892119894(120575119906 119905) = 0

119894 isin 119869 119905 isin [1199050 119905119891]

(30)

where

119863120575119906119891 (120575119906 119905) = (

120597119891 (120575119906 119905)

1205971205751199061

120597119891 (120575119906 119905)

120597120575119906119903

) (31)

Consider the auxiliary parametric optimization problem

min 119891 (120575119906 119905) 119905 isin [1199050 119905119891]

subject to 119892119894(120575119906) = 0 119894 isin

119869 sub 119869

(32)

Let V0 = (120575119906

0(119905) 120583

0(119905)) satisfy the KKT conditions for

problem with 119869 = 119869

0 This system can be written in the

following compact notation

119865 (120575119906 119905) = 0 119905 isin [1199050 119905119891] (33)

where V = (120575119906

0(119905) 120583

0(119905)) In order to apply Newtonrsquos method

to system we have to solve a linear system with119863120575119906119865(V(119905) 119905)

as matrix The same matrix is used to compute V(119905)

119863120575119906119865 (120575119906 (119905) 119905) V (119905) = 119863

120575119906119891 (120575119906 (119905) 119905) (34)

Therefore using the Newton method as corrector wehave

119863120575119906119865 (120575119906

119896119894minus1

119894 119905119894) (120575119906

119896119894

119894minus 120575119906

119896119894minus1

119894) = minus119865 (120575119906

119896119894minus1

119894 119905119894)

119863120575119906119865 (120575119906

119896119894minus1

119894 119905119894) (

120575119906

119896119894minus1

119894) = minusΔ

119905119865 (120575119906

119896119894minus1

119894 119905119894)

(35)

4 Proposed Algorithm FB-OCQ

The first-order indirect approach motivates us to design FB-OCQ algorithm Shortly this algorithm uses gradient and

(1) Initialization119906(119905) 120575119906(119905) = 0 119906

+(119905) = 119906(119905) forall119905 isin [119905

0 119905119891]

Jminus= infin 120574 = 0 120573 120588 isin (0 1)

Iter = 1 MaxIter TOL(2) while Iter leMaxIter do(3) procedure Forward Sweep(4) 119909

+(1199050) = 1199090J+(1199050) = 0

(5) Integrate Forward 119905 1199050rarr 119905119891

(6) if Iter gt 1 then(7) 120575119906(119905) = argmin

120575119906119867aug(119909(119905) 119906(119905) + 120575119906 119909(119905) 119905)

(8) 119906+(119905) = 119906(119905) + 120575119906(119905)

(9) end if(10)

+(119905) = 119891(119909

+(119905) 119906+(119905) 119905)

(11) J+(119905) = 119897(119909

+(119905) 119906+(119905) 119905)

(12) end procedure(13) J

+= J+(119905119891)

(14) if J+lt Jminusthen

(15) Jminus= J+ 120573 = 120573120588

(16) else(17) 120573 = 120573120588 go to Forward Sweep(18) end if(19) procedure Adjoint Sweep(20) 119909(119905

119891) = 120601

⊤

119909(119909+(119905119891)) 120574(119905

0) = 0

(21) Integrate Backward 119905 119905119891rarr 1199050

(22) 119909(119905) = 119909+(119905) 119906(119905) = 119906

+(119905)

(23) 119909(119905)

⊤= minus(120597119867(119909(119905) 119906(119905) 119909(119905) 119905)120597119909)

(24) 119906(119905) = 120597119867(119909(119905) 119906(119905) 119909(119905) 119905)120597119906(25) (119905) = 119906(119905)

2(26) 120574 = 120574(119905

119891)

(27) end procedure(28) if 120574 lt TOL then(29) stop(30) end if(31) Iter larr Iter + 1(32) end while

Algorithm 1 FB-OCQ

does forward-backward searching for the optimal controlsolution Let us choose an initial control trajectory

(1199060 119906

119873) (1205751199060 120575119906

119873)

(119906+0

= 1199060 119906

+119873= 119880119873)

119869minus= infin 120574 Iter = 1 Δ

0 TOL MaxIter 120588 isin [0 1]

FromAlgorithm 1 (FB-OCQ) it is obvious that the algorithmterminates if the norm of the gradient of the Hamiltonianwith respect to 119906 120574 = 119867

1199062 during the run time of the

program is smaller than the tolerance and the parameter 120573

must change at the inner iterations if we do not have a descentdirection so we must divide 120573 by parameter 120588 isin (0 1) untilwe get a reduction in the cost function Figure 2 explainsour algorithm steps forward sweep (control variable and costfunction value) and backward sweep (adjoint variable) indetail

6 Scientific Programming

t0 t1 tf

Forward Backwardsweepsweep u = u + 120575u

J+ = 1 lt Jminus = infinrarr Jminus = J+ = 1

rarr Jminus = J+ = 05

Decrease 120573

Increase 120573J+ = 2 gt Jminus = 1

J+ = 05 lt Jminus = 1

x(t) u(t)

Figure 2 FB-OCQ algorithm explanation

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 3 Indirect method FB-OCQ

5 Performance Evaluation

51 Numerical Results In this section we provide someiteration results for both direct and indirect approaches asfollows

Indirect Method FB-OCQ The ODE was solved on equidis-tant discretization with 1000 discretization points The opti-mal control and optimal state are depicted in Figure 3 Werealize by looking at the final control that there exist somepoints where the set of active constraints changed due to thesingularity that is the optimal control is of the bang-bangtype with the possibility of a singular arc To the meaningof such singular arc for queue management it presents thesudden changes of input rate from 60 (packetssec) to 35

(packetssec) which accordingly result in the changes ofbuffer load (or queue length) In the 120573 history in Figure 3the parameter120573 changed at the inner iterations becausewe donot have a descent direction so wemust divide 120573 by 120588 isin (0 1)

until we get a reduction in the cost functionJ = 681280316The processing time of central processing unit (CPU) forthis algorithm is 004 s and the norm of the gradient of

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 4 Direct method nonlinear optimization with JModelicasolver

the Hamiltonian function 1198671199062with respect to the control

119906 goes to approximately 10119890 minus 7

Direct CollocationMethod JModelica and GAlib Solvers With119873 = 1000 and the number of interpolation points being 3we tested the discretized problem (9)-(10) by JModelica andGAlib solvers (see [20 21]) as follows

(i) JModelica Solver The cost function is reduced to J =

68678362 and the CPU processing time is 052 sThe optimalcontrol and optimal state trajectories are obtained duringJModelica running and illustrated in Figure 4 An advantageof this approach is that we do not need to derive the adjointequations

(ii) GAlib Solver For this problem we use the number ofgenerations 119899gen = 2500 and the population size is 200 Theoptimal control and optimal state are depicted in Figure 5obtained during the run of the GAlib The cost function isreduced toJ = 706744 during the run time of the programThe CPU processing time is 1290 s Although the genetic

Scientific Programming 7

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 5 Direct method genetic algorithm with GAlib solver

Table 1 Numerical result summary

Criteriamethods DirectGAlib

DirectJModelica

IndirectFB-OCQ

Cost function value J = 7067 J = 6867 J = 6812

CPU process time 129 (sec) 052 (sec) 004 (sec)

algorithm has an advantage that it needs no derivatives orHessianrsquos information the control functions produced by itare useless and the convergence is very slow in comparison toAlgorithm 1 (FB-OCQ) and the one using JModelica solver

Table 1 summarizes and compares the three methods thatwe develop numerically We conclude that Algorithm 1 ismuch faster than the other solvers and gives the best costfunction By using genetic library GAlib the cost functiondoes not reach the local solution as it claims in finding theglobal solution The limitation of the indirect methods (FB-OCQ) is that one should derive the adjoint equations whichare not easy to derive in some applications

52 Simulation Results In this section the performance ofthe obtained algorithm (FB-OCQ) is evaluated by comparingit with some popular AQMs including RED REM and PIThe credibility of results is confirmed using ns-2 simulator[24] We investigate a network topology with 40 sourcesan intermediate router and one destination (Figure 6)All sources simultaneously send packets to the destinationthrough the router Hence a large queue is built up at therouter or bottleneck pointMaximum buffer size of the routeris 100 packets All of the compared RED REM and PIalgorithms are configured to obtain the desired queue lengthat 40 packets Simulation lasts for 60 seconds and is repeatedusing a built-in random generator in ns-2 to obtain morecredible results

DestinationRouter

Source 1

Source 2

Source 40

Figure 6 Simulation topology

FB-OCQRED

000

001

002

003

004

005

Dro

ppin

g pr

obab

ility

10 20 30 40 50 600Time (sec)

Figure 7 Dropping probability

Figure 7 shows dropping probability of FB-OCQ andRED algorithms With the proposed square-root drop func-tion FB-OCQ drops queuing packets more aggressivelythan RED although dropping frequency is nearly the sameMaximum dropping ratio is 0045 for FB-OCQ and 0039 forRED In fact when the algorithm drops more packets thequeue stability will be increased but we will have to sacrificethe system throughput performanceWe can see this trade-offin next results

Figure 8 presents average queue length values measuredat the bottleneck link from router to destination for differentalgorithms Due to aggressive dropping FB-OCQ maintainsthe shortest queue at 40 packets which is also the desiredvalue Only REM can obtain the same value but in alonger time asymp50 seconds PI in fact can achieve the sameperformance only if its parameters are well configured andbe dynamically changed to different network scenarios InFigure 8 we can see that PI performs not so well even weset the desired queue length variable at 40 packets and exploitdefault parameters in ns-2 for PI controller

Finally we investigate throughput performance of pro-posed FB-OCQ algorithm It is confirmed from Figure 9that there exists a trade-off in the relationship between

8 Scientific Programming

FB-OCQRED

REMPI

0

10

20

30

40

50

60

70

Aver

age q

ueue

leng

th (p

acke

ts)

10 20 30 40 50 600Time (sec)

Figure 8 Average queue length versus time

FB-OCQ RED REM PI000

001

002

003

004

005

006

007

008

009

Thro

ughp

ut (M

bps)

Figure 9 Throughput comparison

throughput and queue lengthdropping probability PI andREM achieve nearly the same throughput at 006 (Mbps)while FB-OCQandREDobtain the better throughput perfor-mance at 008 (Mbps) value In Figure 8 although REM canachieve the desired queue length 40 packets in this scenarioREM still maintains a large queue during simulation timefrom 0 (sec) up to 25 (sec) That reason leads to throughputperformance of REM being lower than RED and FB-OCQOur proposed FB-OCQ drops aggressively the packets sothat its throughput value (0075904Mbps) is slightly less thanRED (0082146Mbps)

53 Experiment Results We examine the effectiveness ofour state-of-the-art optimal control queue method in theLinux kernel (eg Ubuntu 1604) We conduct experimentfrom a desktop computer through the Internet gateway toan outside server (Figure 10) To manage working queuingdisciplines (qdiscs) we use the scheduler qdisc in the Linuxkernel The chosen server is a dedicated bufferbloat server

Bufferbloat server

DropTail

REDFB-OCQ

Experiment

Figure 10 Experiment test setup from the test client to the testserver

which is able to stand very high congestion due to manydata flows at the same time We exploit the Flent FLExibleNetwork Tester [25] and Realtime Response Under Load(RRUL) test [26] to evaluate our proposal RRUL test puts anetwork under worst case conditions reliably saturates thenetwork link and thus recreates bufferbloat phenomenonfor queue algorithm testing Simulation time duration is 60

secondsWe compare latency under load test with queue being

handled by different AQM schemes (DropTail RED and FB-OCQ) in turnDropTail queuingmethod is by far the simplestapproach to network router queue management The routeraccepts and forwards all the packets that arrive as long as itsbuffer space is available for the next incoming packets If apacket arrives and the queue is currently full the incomingpacket will be dropped The sender then detects the packetlost event and shrinks its sendingwindowWhile it is themostwidely used due to simplicity and relatively high efficiencyDropTail has some weakness such as the bad fairness sharingamongTCP connections and throughput and bottleneck linkefficiency suffer severe degradation if congestion is gettingworse

RED [5 6] was presented with the objective of min-imizing packet loss and queuing delay Moreover it cancompensate the weakness of DropTail by avoiding globalsynchronization of TCP sources so that it improves fair-ness To achieve these goals RED utilizes two thresholdsminth and maxth and an exponentially weighted movingaverage (EWMA) formula to estimate average queue length[27] When the average queue length exceeds a predefinedthreshold the link is implied to be in congested state anddrop action is taken A temporary increase in the queuelength notifies the transient congestion while an increasein the average queue length reflects long-lived congestionBased on such information RED router sends randomizedfeedback signals to the senders tomake decision of decreasingtheir congestion windows RED has good fairness amongconnections because of the feedback randomizedmechanism[28]

Figure 11 presents latency ping results under RRUL testsuite Ping is a networking utility and operates by sendingInternet Control Message Protocol (ICMP) echo requestpackets to the target server and waits for an ICMP echoreply The program measures the round-trip time fromtransmission to reception reporting errors and packet lossOur proposed algorithm FB-OCQ achieves the lowest packetlatency compared with the other two algorithms inside Linux

Scientific Programming 9

DropTailREDFB-OCQ

0 10 20

Simulation time (s)

30 40 50 60

101

102

103

Pack

et la

tenc

y (m

s)

Figure 11 ICMP ping result using Realtime Response Under Loadtest

kernel Specifically the packet latency when using FB-OCQis about 80 (ms) while about 120 (ms) if using DropTail(pfifo fast in Ubuntu) and 100ms if using RED algorithm

6 Conclusions

We proposed a queue management algorithm namedforward-backward optimal control queue (FB-OCQ) tosolve the OCQ problem Derived from indirect approachesin dynamic optimization this algorithm demonstrates fasterreaction while still achieving the same performance innumerical analysis compared to direct methods Employingunder network simulation ns-2 we see that the proposedalgorithm drops packets more aggressively than thetraditional RED algorithm in a higher frequency andmagnitude As a result average queue length can be reducedmuch more while an acceptable value of throughput stillcan be maintained In future works we try to investigatethe memory efficiency of FB-OCQ under wireless sensornetworks

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work was supported by the National Research Founda-tion of Korea (NRF) grant funded by the Korean government(MEST) (no NRF-2016R1A2B1013733)

References

[1] T Hoiland-Jorgensen ldquoBattling bufferbloat an experimentalcomparison of four apapproach to queuemanagement in linuxrdquoTech Rep Roskilde University Roskilde Denmark 2012

[2] J Gettys ldquoBufferbloat dark buffers in the Internetrdquo IEEEInternet Computing vol 15 no 3 pp 95ndash96 2011

[3] J Gettys ldquoBufferbloat views of the elephantrdquo in Proceedingsof the International Summit for Community Wireless NetworksOctober 2012

[4] B Braden D Clark J Crowcroft et al ldquoRecommendations onqueue management and congestion avoidance in the internetrdquoRFC 2309 IETF San Francisco Calif USA 1998

[5] C Hollot V Misra D Towsley and W-B Gong ldquoA controltheoretic analysis of redrdquo in Proceedings of the 20th Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo01) vol 3 pp 1510ndash1519 IEEE Anchorage AlaskaUSA 2001

[6] S Floyd and V Jacobson ldquoRandom early detection gateways forcongestion avoidancerdquo IEEEACM Transactions on Networkingvol 1 no 4 pp 397ndash413 1993

[7] S Athuraliya S Low and D Lapsley ldquoRandom early markingrdquoin Quality of Future Internet Services J Crowcroft J Robertsand M I Smirnov Eds vol 1922 of Lecture Notes in ComputerScience pp 43ndash54 Springer Berlin Germany 2000

[8] R Pan PNatarajan C Piglione et al ldquoPIE a lightweight controlscheme to address the bufferbloat problemrdquo in Proceedings ofthe IEEE 14th International Conference on High PerformanceSwitching and Routing (HPSR rsquo13) pp 148ndash155 Taipei TaiwanJuly 2013

[9] F Ren C Lin and BWei ldquoDesign a robust controller for activequeue management in large delay networksrdquo in Proceedings ofthe 9th International Symposium on Computers and Communi-cations (ISCC rsquo04) vol 2 pp 748ndash754 June-July 2004

[10] R Zhu H Teng and J Fu ldquoA predictive PID controller forAQM router supporting TCP with ECNrdquo in Proceedings ofthe 5th International Symposium on Multi-Dimensional MobileCommunications The 2004 Joint Conference of the 10th Asia-Pacific Conference on Communications vol 1 pp 356ndash360August-September 2004

[11] J Sun M Zukerman and M Palaniswami ldquoStabilizing REDusing a fuzzy controllerrdquo in Proceedings of the IEEE Interna-tional Conference on Communications (ICC rsquo07) pp 266ndash271Glasgow Scotland June 2007

[12] H-L To T M Thi and W-J Hwang ldquoCascade probabilitycontrol to mitigate bufferbloat under multiple real-world TCPstacksrdquoMathematical Problems in Engineering vol 2015 ArticleID 628583 13 pages 2015

[13] M Hassan and H Sirisena ldquoOptimal control of queues in com-puter networksrdquo in Proceedings of the International Conferenceon Communications (ICC rsquo01) vol 2 pp 637ndash641 June 2001

[14] V Guffens and G Bastin ldquoOptimal adaptive feedback controlof a network bufferrdquo in Proceedings of the American ControlConference (ACC rsquo05) vol 3 pp 1835ndash1840 Portland Ore USAJune 2005

[15] J T Betts Practical Methods for Optimal Control Using Nonlin-ear Programming Society for Industrial andAppliedMathemat-ics Philadelphia Pa USA 2001

[16] R Bulirsch E Nerz H Pesch and O von Stryk ldquoCombiningdirect and indirect methods in optimal control Range maxi-mization of a hang gliderrdquo in Optimal Control R Bulirsch AMiele J Stoer and K Well Eds vol 111 of ISNM InternationalSeries of Numerical Mathematics pp 273ndash288 Birkhuser BaselSwitzerland 1993

[17] A Radwan Utilization of parametric programming and evo-lutionary computing in optimal control [PhD dissertation]Humboldt-Universitat zu Berlin 2012

10 Scientific Programming

[18] O von Stryk and R Bulirsch ldquoDirect and indirect methods fortrajectory optimizationrdquo Annals of Operations Research vol 37no 1 pp 357ndash373 1992

[19] A Bryson and Y Ho Applied Optimal Control OptimizationEstimation and Control Hemisphere CRC Press WashingtonDC USA 1975

[20] M Ab Jmodelicaorg user guide 160 2011 httpwwwjmode-licaorgpage236

[21] M Wall GAlib A C++ Library of Genetic Algorithm Compo-nents Version 24 Documentation Revision B MassachusettsInstitute of Technology 1996

[22] A Radwan O Vasilieva R Enkhbat A Griewank and J Gud-dat ldquoParametric approach to optimal controlrdquo OptimizationLetters vol 6 no 7 pp 1303ndash1316 2012

[23] J Guddat FGuerraVazquez andHT JongenParametricOpti-mization Singularities Pathfollowing and Jumps John Wiley ampSons 1990

[24] T Issariyakul and EHossain Introduction to Network SimulatorNS2 Springer Berlin Germany 1st edition 2008

[25] T Hiland-Jrgensen Flent The flexible network tester 2012ndash2015 httpsflentorg

[26] DTht ldquoRealtime response under load (rrul) testrdquo 2013 httpsgithubcomdtahtdeBloatblobmasterspecrruledoc

[27] J Koo S Ahn and J Chung ldquoA comparative study of queuedelay and loss characteristics of AQM schemes in QoS-enablednetworksrdquo Computing and Informatics vol 23 no 4 pp 317ndash335 2004

[28] T B Reddy and A Ahammed ldquoPerformance comparison ofactive queue management techniquesrdquo Journal of ComputerScience vol 4 no 12 pp 1020ndash1023 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

2 Scientific Programming

but are consequently less stable and slower reacting Therehave been several other control-theoretic solutions based onfeedback fluid-flow model proposed in [9ndash12] to improvestable and robust control mechanisms but none of themtackle the issue of system optimality in terms of minimizingboth queue length and packet dropping rate performance andsearching for an optimal control trajectory

To address such issues a promising design direction is toreformulate the network queuing problem as an optimal con-trol queue (OCQ) problem [13] where themain state variableis queue length and the control variable is the actual input rateto form the queue [14] or packet dropping rate Then one ofthe approaches to solve OCQ is that we can priorly discretizethe governing ordinary differential equations (ODEs) and theintegral terms in the cost function or constraint functionsand thereby replace the infinite dimensional optimal controlproblemwith a large nonlinear optimization problem (NOP)This is known as the direct method in the literature for solvingOCQ This approach is typically easier to use especially forOCQ with a state equality or inequality constraints Themain difference among direct approaches is how to handlethe constraints corresponding to the system dynamics Thethree most common direct approaches are direct singleshooting direct multiple shooting and direct collocationDirect methods have been used with references in [15ndash18]

Alternatively one can first form the optimality condi-tions using the calculus of variations and Pontryagin mini-mum principle and then solve the resulting boundary valueproblem This is known as the indirect method for solvingOCQ The references present just a small sample of the workthat discusses or applies indirect methods for the solution ofoptimal control problems [15 16 18 19] In rare cases thesolution can be obtained in closed form from the optimalityconditions but in general approximation methods are usedto solve the problem numerically The optimality conditionsof these problems generally take the form of differentialalgebraic equations (DAEs) with boundary conditions (BCs)The approximate solution to the OCQ can be obtained byusing a boundary value problem (BVP) solver Perhaps themost popular methods aremultiple shooting and collocationMore recently a combination of direct and indirect methodswas proposed leading to hybrid methods [16]

In this paper we firstly derive methods to solve OCQusing both direct and indirect approaches We show how toapply the direct collocation approach to solve OCQ problemwhich can be solved in popular optimization solvers suchas JModelica [20] and GAlib [21] Indirect method withforward-backward for optimal control queue algorithm (FB-OCQ) is designed as an alternative to tackle it as well Ourkey difference from existing works is that we provide a novelmethod to update the control step 120575119906 by solving a parametricoptimization subproblem This method is scalable whichmeans that we can expand for larger problems with morevariables as well The numerical results for both direct andindirect approaches are discussed in Section 51 to emphasizeour choice of integrating an indirect method with parametricoptimization for active queue management design which isdemonstrated more efficiently (ie faster reaction and morestable) than directmethods Finally we evaluate the proposed

actual rateinput rate

drop rate

service rate

w(t) u(t)

d(t) = w(t) minus u(t)

x(t) queue length

120583

Figure 1 OCQ model

algorithm using network simulator ns-2 and compare it toother AQMs including RED REM and PI The droppingfeature of the proposed algorithm makes the average queuelength shortened and stabilized compared with others whilethroughput is not reduced so much

Our Contributions(i) We present both indirect and direct methods for an

optimal control problem which is applied in queuemanagement field (Section 3) The step update 120575119906 ofcontrol variable is calculated by solving a subproblemof parametric optimization with the advantage ofscalability Numerical results show that all of thembring nearly similar results but indirect method (FB-OCQ) is much faster than other solvers and gives thebest cost function value (Section 51)

(ii) We evaluate FB-OCQ in simulation and show that adesired small queue length value at 40 packets canbe obtained Nevertheless we cannot avoid the trade-off between queue length and throughput FB-OCQrsquosthroughput is slightly smaller than REDrsquos one (0075versus 0082Mbps) however it can be an acceptablevalue when compared to REMrsquos and PIrsquos (Section 52)

(iii) We implement FB-OCQ in Linux kernel (Ubuntu1604) and test it in the worst case using RealtimeResponse Under Load (RRUL) test suite The exper-iment result shows that our algorithm brings lowlatency ping value compared with the other existingalgorithms in Linux kernel (DropTail and RED)

2 Problem Formulation

We consider an optimal control model of queuemanagementproblem named (OCQ) in [14] (Figure 1) The main ideais to minimize the cost function which implies a trade-offbetween queue length anddropping rate subject to a dynamicconstraint of queue length along time 119905

0to 119905119891as follows

minimize119909

119869 = int

119905119891

119905=0

119871 (119909 (119905) 119906 (119905) 119905)

subject to = 119891 (119909 (119905) 119906 (119905) 119905)

0 le 119906 (119905) le 119908 (119905)

(OCQ)

where 119871(119909 119906 119905) = 119909(119905) + 119877(119908 minus 119906(119905)) is cost function119891(119909(119905) 119906(119905) 119905) = 119906(119905) minus 120583119909(119886 + 119909) 119877 is weight on droppingrate 119905

119891is final time 120583 is service rate (bandwidth capacity)

and 119886 is parameter for different types of queuing model forexample when 119886 = 1 we obtain an MM1 queue

Scientific Programming 3

3 Numerical Methods for OCQ

31 Direct Method Collocation Method In this section wepresent how to apply the direct collocation [18] for OCQ Inthis method we discretize the time interval 119905

0= 1199051

lt 1199052

lt

1199053

lt sdot sdot sdot lt 119905119896

= 119905119891into 119873 elements The state and the

total number of packets and control variables at each nodeare 119909119895= 119909(119905119895) and 119908

119895= 119908(119905

119895) and 119906

119895= 119906(119905119895) such that the

state control and packets variables at the nodes are definedas nonlinear programming (NLP) variables

119884 = [119909 (1199051) 119909 (119905

119896) 119906 (119905

1) 119906 (119905

119896) 119908 (119905

1) 119908 (119905

119896)] (1)

The controls are chosen as piecewise linear interpolatingfunctions between 119906(119905

119895) and 119906(119905

119895+1) for 119905

119895le 119905 le 119905

119895+1as

follows

119906app (119905) = 119906 (119905119895) +

119905 minus 119905119895

ℎ119895

[119906 (119905119895+1

) minus 119906 (119905119895)]

ℎ119895= 119905119895+1

minus 119905119895

(2)

The value of the control variables at the center 119905119895+12

is givenby

119906 (119905119895+12

) =

119906 (119905119895) + 119906 (119905

119895+1)

2

119895 = 1 119896 minus 1(3)

The piecewise linear interpolation is used to preparefor the possibility of discontinuous solutions in controlSimilarly we can derive the approximate of the total numberof the packets 119908app(119905) and 119908(119905

119895+12) as above The state

variable 119909(119905) is approximated by a continuously differentiableand piecewise Hermite-Simpson cubic polynomial between119909(119905119895) and 119909(119905

119895+1) on the interval 119905

119895le 119905 le 119905

119895+1of length ℎ

119895

119909app (119905) =

3

sum

119903=0

119888

119895

119903

119905 minus 119905119895

ℎ119895

119888

119895

0= 119909 (119905

119895)

119888

119895

1= ℎ119895119891119895

119888

119895

2= minus3119909 (119905

119895) minus 2ℎ

119895119891119895+ 3119909 (119905

119895+1) minus (ℎ

119895) 119891119895+1

119888

119895

3= 2119909 (119905

119895) + ℎ119895119891119895minus 2119909 (119905

119895+1) + (ℎ

119895) 119891119895+1

(4)

where

119891119895= 119891 (119909 (119905

119895) 119906 (119905

119895) 119908 (119905

119895) 119905119895)

119905119895le 119905 le 119905

119895+1 119895 = 1 119896 minus 1

(5)

The value of the state variables at the center point 119905119895+12

of thecubic approximation is

119909 (119905119895+12

) =

119909 (119905119895) + 119909 (119905

119895+1)

2

+

119905119895+1

+ 119905119895

8

(119891119895+ 119891119895+1

)

119895 = 1 119896 minus 1

(6)

and the derivative is

(119905119895+12

) =

3 (119909119895+ 119909119895+1

)

2 (119905119895+1

+ 119905119895)

minus

1

4

(119891119895+ 119891119895+1

)

119895 = 1 119896 minus 1

(7)

In addition the chosen interpolating polynomial for the stateand control variablesmust satisfy themidpoint conditions forthe differential equations as follows

119891 (119909app (119905119895+12) 119906app (119905119895+12) 119908app (119905119895+12) 119905119895+12

)

minus app (119905119895+12) = 0

(8)

Equations (OCQ) can now be defined as a discretizedproblem as follows

min 119891 (119884) (9)

subject to 119891 (119909app (119905) 119906app (119905) 119908app (119905) 119905) minus app

= 0

119909app (1199051) minus 1199091= 0

0 le 119906app (119905) le 119908app (119905)

(10)

where 119909app 119906app and119908app are the approximations of the statethe control and the total number of packets constituting119884 in(9) This above discretization problem (9)-(10) can be solvedusing the following

(i) JModelica which is a package for simulation andoptimization ofModelicamodels (formore details see[20])

(ii) GAlib which is C++ library of genetic algorithm (formore details see [21])

32 Indirect Method Forward-Backward Sweeping We solveOCQ by using indirect method approach

(i) forming optimality conditions(ii) solving BVP by First-Order Sweeping algorithm

Let 119909(119905) be the adjoint variable At time 119905 let 119906lowast(119905) denotethe optimum controls and let 119909

lowast(119905) and 119909

lowast(119905) denote the

state and adjoint evaluated at the optimumUsing Pontryaginminimum principle we get the following equations

Hamiltonian Function

119867(119909 (119905) 119906 (119905) 119909 (119905)) = 119909 (119905) + 119877 (119908 (119905) minus 119906 (119905))

+ 119909 (119905) (119906 (119905) minus

120583119909 (119905)

119886 + 119909 (119905)

)

(11)

Adjoint Equations

119909 (119905) = minus1 + 119906 (119905)

119886120583

(119886 + 119909 (119905))

2 (12)

4 Scientific Programming

Transversality Condition

119909 (119905119891) = 0 (13)

Hamiltonian Minimization Condition Derivative of theHamiltonian is evaluated to zero at interior points hence

119867119906 (

119909 (119905) 119906 (119905) 119909 (119905) 120582 (119905) 119905)

= minus119877 + 119909 (119905) minus 1205821(119905) + 120582

2(119905) = 0

(14)

If 119906

lowast(119905) is optimal in (OCQ) then it must satisfy the

minimum condition119867(119909

lowast 119906

lowast 119909

lowast 119905) = min

Visin119880119867(119909 V 119909 119905) forall119905 isin 119873 (15)

Furthermore we assume the following

(H1) the Hamiltonian is strictly convex with respect tocontrol variable 119906

(H2) (119905) = argmin119906isin119880

119867(119909 119906 119909) 119905 isin [1199050 119905119891] is continu-

ous on [1199050 119905119891]

(H3) 119906 isin 119862

119903[1199050 119905119891]

Note that 119906(119905) is determined in a unique way for each119905 since 119880 is convex and 119867 is convex Now we consider theproblem of minimizing the Hamiltonian with respect to 119901

min119906isin119880

119867(119909 119906 119909) for each 119905 isin [1199050 119905119891] (16)

Usually in the literature 119906 is found explicitly as a function119906 = 119906(119909 119909 119905) and after substituting it into the system theproblem reduces to the boundary value problem

Theorem 1 Assume that conditions (H1)ndash(H3) hold andproblem has an optimal solution (119906

lowast 119909

lowast) Then for a given 120598

there exists a finite discretization

1199050= 1205910lt 1205911lt sdot sdot sdot lt 120591

119894lt sdot sdot sdot lt 120591

119873= 119905119891 (17)

and an approximate solution (119905) 119905 isin [1199050 119905119891] such that

1003817100381710038171003817119906

lowast(119905119894) minus (119905

119894)

1003817100381710038171003817lt 120598 119894 = 1 2 119873 (18)

Proof Let 119906

lowast be an optimal solution of problem Then(119901

lowast 119909

lowast) satisfies the conditions

lowast

119894=

120597119867 (119909

lowast 119906

lowast 119909

lowast)

120597119906119894

119909

lowast

119894= minus

120597119867 ((119909

lowast 119906

lowast 119909

lowast))

120597119909119894

119894 = 1 2 119873

(19)

where

119867(119909 119906 119909) =

119899

sum

119894=1

119909119894(119905) 119891119894(119909 119906) minus 119891

0(119909 119906)

119905 isin [1199050 119905119891]

(20)

and (119906

lowast 119909

lowast) satisfies the minimum principle

119867(119909

lowast 119906

lowast 119909

lowast) = min119906isin119880

119867(119909

lowast 119906 119909

lowast) 119905 isin [119905

0 119905119891] (21)

(a) First-Order Sweeping Method We linearize the Hamilto-nian around a reference solution119906(119905) and obtain the followingequation for variation 120575119906(119905)

0 = 119867119906

⊤+ 119867119906119906

120575119906 (119905) (22)

With the strong Legendre-Clebsch condition one canapproximate the above equation as

120575119906 (119905) = arg min120575119906isin119880minus119906

119867aug (119909 119906 + 120575119906 119909 119905) (23)

where

119867aug (119909 119906 + 120575119906 119909 119905) = 119867 (119909 119906 + 120575119906 119909 119905) + 120573 120575119906

2 (24)

is the augmented Hamiltonian function and 120573120575119906

2 is apenalty term This is really parametric optimization with120573 = 0 initially if119880 is a convex set and119867 is a convex functionwith respect to 119906 then 120575119906 is a descent direction If 119906(119905)+120575119906(119905)

does not yield a reduction then we set 120573 = 1 and then doubleit repeatedly until the objective is really reduced

Remark 2 An alternative way to ensure descent is to applyBacktracking Line-Search Procedure

(b) Parametric Optimization to OCQ Now we considerproblem (23) as one parametric minimization problemSince 119867aug(119909

lowast 119909

lowast 119906

lowast 120575119906 119905) is twice differentiable in 120575119906 and

assumptions (H1)ndash(H3) hold we can apply Theorem 1 to theproblem Then as a result it generates a discretization 119905

0=

1205910lt 1205911lt sdot sdot sdot lt 120591

119894lt sdot sdot sdot lt 120591

119873= 119905119891 and corresponding points

119894= (119905119894) such that1003817100381710038171003817120575119906

lowast(119905119894) minus 120575 (119905

119894)

1003817100381710038171003817lt 120598 119894 = 1 2 119873 (25)

which proves the assertion

Remark 3 Parametric optimization also can be applied infinding nominal optimal control given in [22] It is easy to seethat at each iteration 119896 the Hamiltonian function is a scalarfunction of 119906 isin 119880 sub 119877

119903 and 119905 isin 119879 = [1199050 119905119891] that is

119866119896(120575119906 119905) = 119867aug (119909

119896(119905) 119909

119896(119905) 120575119906 119905) (26)

The latter states that 120575

119896(119905) must be a minimizer of the

following problem

min120575119906isin119880

119866119896(120575119906 119905) 119905 isin 119879 (27)

which is a problem of parametric optimization as formulatedin various papers from [23] where the independent variable 119905is now considered as unknown parameter 119905 isin 119879 = [119905

0 119905119891]We

can also consider a case when the set of admissible controls istime-varying that is 119880 = 119880(119905) 119905 isin 119879 = [119905

0 119905119891] In this case

a general theory of parametric optimization is also applicablefor finding the nominal optimal controls

Scientific Programming 5

Let (119909lowast 120575119906lowast) be an optimal process in problem Introducethe function 119891 119880 times 119877 rarr 119877

119891 (120575119906 119905) = minus119867 (119909

lowast 120575119906 119909

lowast 119905) 119905 isin [119905

0 119905119891] (28)

where a parametric optimization problem is defined as

min120575119906isin119880

119891 (120575119906 119905) 119905 isin [1199050 119905119891]

119880 = 120575119906 isin 119877

119903 119892119894(120575119906) le 0 119894 isin 119869

119869 = 1 2 119904

(29)

The KKT conditions for the problem state that

119863120575119906119891 (120575119906 119905) + sum

119895isin119869

120583119894119863120575119906119892119894(120575119906 119905) = 0

119892119894(120575119906 119905) 120583

119894le 0 120583

119894ge 0 119894 isin 119869

120583119894119892119894(120575119906 119905) = 0

119894 isin 119869 119905 isin [1199050 119905119891]

(30)

where

119863120575119906119891 (120575119906 119905) = (

120597119891 (120575119906 119905)

1205971205751199061

120597119891 (120575119906 119905)

120597120575119906119903

) (31)

Consider the auxiliary parametric optimization problem

min 119891 (120575119906 119905) 119905 isin [1199050 119905119891]

subject to 119892119894(120575119906) = 0 119894 isin

119869 sub 119869

(32)

Let V0 = (120575119906

0(119905) 120583

0(119905)) satisfy the KKT conditions for

problem with 119869 = 119869

0 This system can be written in the

following compact notation

119865 (120575119906 119905) = 0 119905 isin [1199050 119905119891] (33)

where V = (120575119906

0(119905) 120583

0(119905)) In order to apply Newtonrsquos method

to system we have to solve a linear system with119863120575119906119865(V(119905) 119905)

as matrix The same matrix is used to compute V(119905)

119863120575119906119865 (120575119906 (119905) 119905) V (119905) = 119863

120575119906119891 (120575119906 (119905) 119905) (34)

Therefore using the Newton method as corrector wehave

119863120575119906119865 (120575119906

119896119894minus1

119894 119905119894) (120575119906

119896119894

119894minus 120575119906

119896119894minus1

119894) = minus119865 (120575119906

119896119894minus1

119894 119905119894)

119863120575119906119865 (120575119906

119896119894minus1

119894 119905119894) (

120575119906

119896119894minus1

119894) = minusΔ

119905119865 (120575119906

119896119894minus1

119894 119905119894)

(35)

4 Proposed Algorithm FB-OCQ

The first-order indirect approach motivates us to design FB-OCQ algorithm Shortly this algorithm uses gradient and

(1) Initialization119906(119905) 120575119906(119905) = 0 119906

+(119905) = 119906(119905) forall119905 isin [119905

0 119905119891]

Jminus= infin 120574 = 0 120573 120588 isin (0 1)

Iter = 1 MaxIter TOL(2) while Iter leMaxIter do(3) procedure Forward Sweep(4) 119909

+(1199050) = 1199090J+(1199050) = 0

(5) Integrate Forward 119905 1199050rarr 119905119891

(6) if Iter gt 1 then(7) 120575119906(119905) = argmin

120575119906119867aug(119909(119905) 119906(119905) + 120575119906 119909(119905) 119905)

(8) 119906+(119905) = 119906(119905) + 120575119906(119905)

(9) end if(10)

+(119905) = 119891(119909

+(119905) 119906+(119905) 119905)

(11) J+(119905) = 119897(119909

+(119905) 119906+(119905) 119905)

(12) end procedure(13) J

+= J+(119905119891)

(14) if J+lt Jminusthen

(15) Jminus= J+ 120573 = 120573120588

(16) else(17) 120573 = 120573120588 go to Forward Sweep(18) end if(19) procedure Adjoint Sweep(20) 119909(119905

119891) = 120601

⊤

119909(119909+(119905119891)) 120574(119905

0) = 0

(21) Integrate Backward 119905 119905119891rarr 1199050

(22) 119909(119905) = 119909+(119905) 119906(119905) = 119906

+(119905)

(23) 119909(119905)

⊤= minus(120597119867(119909(119905) 119906(119905) 119909(119905) 119905)120597119909)

(24) 119906(119905) = 120597119867(119909(119905) 119906(119905) 119909(119905) 119905)120597119906(25) (119905) = 119906(119905)

2(26) 120574 = 120574(119905

119891)

(27) end procedure(28) if 120574 lt TOL then(29) stop(30) end if(31) Iter larr Iter + 1(32) end while

Algorithm 1 FB-OCQ

does forward-backward searching for the optimal controlsolution Let us choose an initial control trajectory

(1199060 119906

119873) (1205751199060 120575119906

119873)

(119906+0

= 1199060 119906

+119873= 119880119873)

119869minus= infin 120574 Iter = 1 Δ

0 TOL MaxIter 120588 isin [0 1]

FromAlgorithm 1 (FB-OCQ) it is obvious that the algorithmterminates if the norm of the gradient of the Hamiltonianwith respect to 119906 120574 = 119867

1199062 during the run time of the

program is smaller than the tolerance and the parameter 120573

must change at the inner iterations if we do not have a descentdirection so we must divide 120573 by parameter 120588 isin (0 1) untilwe get a reduction in the cost function Figure 2 explainsour algorithm steps forward sweep (control variable and costfunction value) and backward sweep (adjoint variable) indetail

6 Scientific Programming

t0 t1 tf

Forward Backwardsweepsweep u = u + 120575u

J+ = 1 lt Jminus = infinrarr Jminus = J+ = 1

rarr Jminus = J+ = 05

Decrease 120573

Increase 120573J+ = 2 gt Jminus = 1

J+ = 05 lt Jminus = 1

x(t) u(t)

Figure 2 FB-OCQ algorithm explanation

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 3 Indirect method FB-OCQ

5 Performance Evaluation

51 Numerical Results In this section we provide someiteration results for both direct and indirect approaches asfollows

Indirect Method FB-OCQ The ODE was solved on equidis-tant discretization with 1000 discretization points The opti-mal control and optimal state are depicted in Figure 3 Werealize by looking at the final control that there exist somepoints where the set of active constraints changed due to thesingularity that is the optimal control is of the bang-bangtype with the possibility of a singular arc To the meaningof such singular arc for queue management it presents thesudden changes of input rate from 60 (packetssec) to 35

(packetssec) which accordingly result in the changes ofbuffer load (or queue length) In the 120573 history in Figure 3the parameter120573 changed at the inner iterations becausewe donot have a descent direction so wemust divide 120573 by 120588 isin (0 1)

until we get a reduction in the cost functionJ = 681280316The processing time of central processing unit (CPU) forthis algorithm is 004 s and the norm of the gradient of

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 4 Direct method nonlinear optimization with JModelicasolver

the Hamiltonian function 1198671199062with respect to the control

119906 goes to approximately 10119890 minus 7

Direct CollocationMethod JModelica and GAlib Solvers With119873 = 1000 and the number of interpolation points being 3we tested the discretized problem (9)-(10) by JModelica andGAlib solvers (see [20 21]) as follows

(i) JModelica Solver The cost function is reduced to J =

68678362 and the CPU processing time is 052 sThe optimalcontrol and optimal state trajectories are obtained duringJModelica running and illustrated in Figure 4 An advantageof this approach is that we do not need to derive the adjointequations

(ii) GAlib Solver For this problem we use the number ofgenerations 119899gen = 2500 and the population size is 200 Theoptimal control and optimal state are depicted in Figure 5obtained during the run of the GAlib The cost function isreduced toJ = 706744 during the run time of the programThe CPU processing time is 1290 s Although the genetic

Scientific Programming 7

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 5 Direct method genetic algorithm with GAlib solver

Table 1 Numerical result summary

Criteriamethods DirectGAlib

DirectJModelica

IndirectFB-OCQ

Cost function value J = 7067 J = 6867 J = 6812

CPU process time 129 (sec) 052 (sec) 004 (sec)

algorithm has an advantage that it needs no derivatives orHessianrsquos information the control functions produced by itare useless and the convergence is very slow in comparison toAlgorithm 1 (FB-OCQ) and the one using JModelica solver

Table 1 summarizes and compares the three methods thatwe develop numerically We conclude that Algorithm 1 ismuch faster than the other solvers and gives the best costfunction By using genetic library GAlib the cost functiondoes not reach the local solution as it claims in finding theglobal solution The limitation of the indirect methods (FB-OCQ) is that one should derive the adjoint equations whichare not easy to derive in some applications

52 Simulation Results In this section the performance ofthe obtained algorithm (FB-OCQ) is evaluated by comparingit with some popular AQMs including RED REM and PIThe credibility of results is confirmed using ns-2 simulator[24] We investigate a network topology with 40 sourcesan intermediate router and one destination (Figure 6)All sources simultaneously send packets to the destinationthrough the router Hence a large queue is built up at therouter or bottleneck pointMaximum buffer size of the routeris 100 packets All of the compared RED REM and PIalgorithms are configured to obtain the desired queue lengthat 40 packets Simulation lasts for 60 seconds and is repeatedusing a built-in random generator in ns-2 to obtain morecredible results

DestinationRouter

Source 1

Source 2

Source 40

Figure 6 Simulation topology

FB-OCQRED

000

001

002

003

004

005

Dro

ppin

g pr

obab

ility

10 20 30 40 50 600Time (sec)

Figure 7 Dropping probability

Figure 7 shows dropping probability of FB-OCQ andRED algorithms With the proposed square-root drop func-tion FB-OCQ drops queuing packets more aggressivelythan RED although dropping frequency is nearly the sameMaximum dropping ratio is 0045 for FB-OCQ and 0039 forRED In fact when the algorithm drops more packets thequeue stability will be increased but we will have to sacrificethe system throughput performanceWe can see this trade-offin next results

Figure 8 presents average queue length values measuredat the bottleneck link from router to destination for differentalgorithms Due to aggressive dropping FB-OCQ maintainsthe shortest queue at 40 packets which is also the desiredvalue Only REM can obtain the same value but in alonger time asymp50 seconds PI in fact can achieve the sameperformance only if its parameters are well configured andbe dynamically changed to different network scenarios InFigure 8 we can see that PI performs not so well even weset the desired queue length variable at 40 packets and exploitdefault parameters in ns-2 for PI controller

Finally we investigate throughput performance of pro-posed FB-OCQ algorithm It is confirmed from Figure 9that there exists a trade-off in the relationship between

8 Scientific Programming

FB-OCQRED

REMPI

0

10

20

30

40

50

60

70

Aver

age q

ueue

leng

th (p

acke

ts)

10 20 30 40 50 600Time (sec)

Figure 8 Average queue length versus time

FB-OCQ RED REM PI000

001

002

003

004

005

006

007

008

009

Thro

ughp

ut (M

bps)

Figure 9 Throughput comparison

throughput and queue lengthdropping probability PI andREM achieve nearly the same throughput at 006 (Mbps)while FB-OCQandREDobtain the better throughput perfor-mance at 008 (Mbps) value In Figure 8 although REM canachieve the desired queue length 40 packets in this scenarioREM still maintains a large queue during simulation timefrom 0 (sec) up to 25 (sec) That reason leads to throughputperformance of REM being lower than RED and FB-OCQOur proposed FB-OCQ drops aggressively the packets sothat its throughput value (0075904Mbps) is slightly less thanRED (0082146Mbps)

53 Experiment Results We examine the effectiveness ofour state-of-the-art optimal control queue method in theLinux kernel (eg Ubuntu 1604) We conduct experimentfrom a desktop computer through the Internet gateway toan outside server (Figure 10) To manage working queuingdisciplines (qdiscs) we use the scheduler qdisc in the Linuxkernel The chosen server is a dedicated bufferbloat server

Bufferbloat server

DropTail

REDFB-OCQ

Experiment

Figure 10 Experiment test setup from the test client to the testserver

which is able to stand very high congestion due to manydata flows at the same time We exploit the Flent FLExibleNetwork Tester [25] and Realtime Response Under Load(RRUL) test [26] to evaluate our proposal RRUL test puts anetwork under worst case conditions reliably saturates thenetwork link and thus recreates bufferbloat phenomenonfor queue algorithm testing Simulation time duration is 60

secondsWe compare latency under load test with queue being

handled by different AQM schemes (DropTail RED and FB-OCQ) in turnDropTail queuingmethod is by far the simplestapproach to network router queue management The routeraccepts and forwards all the packets that arrive as long as itsbuffer space is available for the next incoming packets If apacket arrives and the queue is currently full the incomingpacket will be dropped The sender then detects the packetlost event and shrinks its sendingwindowWhile it is themostwidely used due to simplicity and relatively high efficiencyDropTail has some weakness such as the bad fairness sharingamongTCP connections and throughput and bottleneck linkefficiency suffer severe degradation if congestion is gettingworse

RED [5 6] was presented with the objective of min-imizing packet loss and queuing delay Moreover it cancompensate the weakness of DropTail by avoiding globalsynchronization of TCP sources so that it improves fair-ness To achieve these goals RED utilizes two thresholdsminth and maxth and an exponentially weighted movingaverage (EWMA) formula to estimate average queue length[27] When the average queue length exceeds a predefinedthreshold the link is implied to be in congested state anddrop action is taken A temporary increase in the queuelength notifies the transient congestion while an increasein the average queue length reflects long-lived congestionBased on such information RED router sends randomizedfeedback signals to the senders tomake decision of decreasingtheir congestion windows RED has good fairness amongconnections because of the feedback randomizedmechanism[28]

Figure 11 presents latency ping results under RRUL testsuite Ping is a networking utility and operates by sendingInternet Control Message Protocol (ICMP) echo requestpackets to the target server and waits for an ICMP echoreply The program measures the round-trip time fromtransmission to reception reporting errors and packet lossOur proposed algorithm FB-OCQ achieves the lowest packetlatency compared with the other two algorithms inside Linux

Scientific Programming 9

DropTailREDFB-OCQ

0 10 20

Simulation time (s)

30 40 50 60

101

102

103

Pack

et la

tenc

y (m

s)

Figure 11 ICMP ping result using Realtime Response Under Loadtest

kernel Specifically the packet latency when using FB-OCQis about 80 (ms) while about 120 (ms) if using DropTail(pfifo fast in Ubuntu) and 100ms if using RED algorithm

6 Conclusions

We proposed a queue management algorithm namedforward-backward optimal control queue (FB-OCQ) tosolve the OCQ problem Derived from indirect approachesin dynamic optimization this algorithm demonstrates fasterreaction while still achieving the same performance innumerical analysis compared to direct methods Employingunder network simulation ns-2 we see that the proposedalgorithm drops packets more aggressively than thetraditional RED algorithm in a higher frequency andmagnitude As a result average queue length can be reducedmuch more while an acceptable value of throughput stillcan be maintained In future works we try to investigatethe memory efficiency of FB-OCQ under wireless sensornetworks

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work was supported by the National Research Founda-tion of Korea (NRF) grant funded by the Korean government(MEST) (no NRF-2016R1A2B1013733)

References

[1] T Hoiland-Jorgensen ldquoBattling bufferbloat an experimentalcomparison of four apapproach to queuemanagement in linuxrdquoTech Rep Roskilde University Roskilde Denmark 2012

[2] J Gettys ldquoBufferbloat dark buffers in the Internetrdquo IEEEInternet Computing vol 15 no 3 pp 95ndash96 2011

[3] J Gettys ldquoBufferbloat views of the elephantrdquo in Proceedingsof the International Summit for Community Wireless NetworksOctober 2012

[4] B Braden D Clark J Crowcroft et al ldquoRecommendations onqueue management and congestion avoidance in the internetrdquoRFC 2309 IETF San Francisco Calif USA 1998

[5] C Hollot V Misra D Towsley and W-B Gong ldquoA controltheoretic analysis of redrdquo in Proceedings of the 20th Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo01) vol 3 pp 1510ndash1519 IEEE Anchorage AlaskaUSA 2001

[6] S Floyd and V Jacobson ldquoRandom early detection gateways forcongestion avoidancerdquo IEEEACM Transactions on Networkingvol 1 no 4 pp 397ndash413 1993

[7] S Athuraliya S Low and D Lapsley ldquoRandom early markingrdquoin Quality of Future Internet Services J Crowcroft J Robertsand M I Smirnov Eds vol 1922 of Lecture Notes in ComputerScience pp 43ndash54 Springer Berlin Germany 2000

[8] R Pan PNatarajan C Piglione et al ldquoPIE a lightweight controlscheme to address the bufferbloat problemrdquo in Proceedings ofthe IEEE 14th International Conference on High PerformanceSwitching and Routing (HPSR rsquo13) pp 148ndash155 Taipei TaiwanJuly 2013

[9] F Ren C Lin and BWei ldquoDesign a robust controller for activequeue management in large delay networksrdquo in Proceedings ofthe 9th International Symposium on Computers and Communi-cations (ISCC rsquo04) vol 2 pp 748ndash754 June-July 2004

[10] R Zhu H Teng and J Fu ldquoA predictive PID controller forAQM router supporting TCP with ECNrdquo in Proceedings ofthe 5th International Symposium on Multi-Dimensional MobileCommunications The 2004 Joint Conference of the 10th Asia-Pacific Conference on Communications vol 1 pp 356ndash360August-September 2004

[11] J Sun M Zukerman and M Palaniswami ldquoStabilizing REDusing a fuzzy controllerrdquo in Proceedings of the IEEE Interna-tional Conference on Communications (ICC rsquo07) pp 266ndash271Glasgow Scotland June 2007

[12] H-L To T M Thi and W-J Hwang ldquoCascade probabilitycontrol to mitigate bufferbloat under multiple real-world TCPstacksrdquoMathematical Problems in Engineering vol 2015 ArticleID 628583 13 pages 2015

[13] M Hassan and H Sirisena ldquoOptimal control of queues in com-puter networksrdquo in Proceedings of the International Conferenceon Communications (ICC rsquo01) vol 2 pp 637ndash641 June 2001

[14] V Guffens and G Bastin ldquoOptimal adaptive feedback controlof a network bufferrdquo in Proceedings of the American ControlConference (ACC rsquo05) vol 3 pp 1835ndash1840 Portland Ore USAJune 2005

[15] J T Betts Practical Methods for Optimal Control Using Nonlin-ear Programming Society for Industrial andAppliedMathemat-ics Philadelphia Pa USA 2001

[16] R Bulirsch E Nerz H Pesch and O von Stryk ldquoCombiningdirect and indirect methods in optimal control Range maxi-mization of a hang gliderrdquo in Optimal Control R Bulirsch AMiele J Stoer and K Well Eds vol 111 of ISNM InternationalSeries of Numerical Mathematics pp 273ndash288 Birkhuser BaselSwitzerland 1993

[17] A Radwan Utilization of parametric programming and evo-lutionary computing in optimal control [PhD dissertation]Humboldt-Universitat zu Berlin 2012

10 Scientific Programming

[18] O von Stryk and R Bulirsch ldquoDirect and indirect methods fortrajectory optimizationrdquo Annals of Operations Research vol 37no 1 pp 357ndash373 1992

[19] A Bryson and Y Ho Applied Optimal Control OptimizationEstimation and Control Hemisphere CRC Press WashingtonDC USA 1975

[20] M Ab Jmodelicaorg user guide 160 2011 httpwwwjmode-licaorgpage236

[21] M Wall GAlib A C++ Library of Genetic Algorithm Compo-nents Version 24 Documentation Revision B MassachusettsInstitute of Technology 1996

[22] A Radwan O Vasilieva R Enkhbat A Griewank and J Gud-dat ldquoParametric approach to optimal controlrdquo OptimizationLetters vol 6 no 7 pp 1303ndash1316 2012

[23] J Guddat FGuerraVazquez andHT JongenParametricOpti-mization Singularities Pathfollowing and Jumps John Wiley ampSons 1990

[24] T Issariyakul and EHossain Introduction to Network SimulatorNS2 Springer Berlin Germany 1st edition 2008

[25] T Hiland-Jrgensen Flent The flexible network tester 2012ndash2015 httpsflentorg

[26] DTht ldquoRealtime response under load (rrul) testrdquo 2013 httpsgithubcomdtahtdeBloatblobmasterspecrruledoc

[27] J Koo S Ahn and J Chung ldquoA comparative study of queuedelay and loss characteristics of AQM schemes in QoS-enablednetworksrdquo Computing and Informatics vol 23 no 4 pp 317ndash335 2004

[28] T B Reddy and A Ahammed ldquoPerformance comparison ofactive queue management techniquesrdquo Journal of ComputerScience vol 4 no 12 pp 1020ndash1023 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Scientific Programming 3

3 Numerical Methods for OCQ

31 Direct Method Collocation Method In this section wepresent how to apply the direct collocation [18] for OCQ Inthis method we discretize the time interval 119905

0= 1199051

lt 1199052

lt

1199053

lt sdot sdot sdot lt 119905119896

= 119905119891into 119873 elements The state and the

total number of packets and control variables at each nodeare 119909119895= 119909(119905119895) and 119908

119895= 119908(119905

119895) and 119906

119895= 119906(119905119895) such that the

state control and packets variables at the nodes are definedas nonlinear programming (NLP) variables

119884 = [119909 (1199051) 119909 (119905

119896) 119906 (119905

1) 119906 (119905

119896) 119908 (119905

1) 119908 (119905

119896)] (1)

The controls are chosen as piecewise linear interpolatingfunctions between 119906(119905

119895) and 119906(119905

119895+1) for 119905

119895le 119905 le 119905

119895+1as

follows

119906app (119905) = 119906 (119905119895) +

119905 minus 119905119895

ℎ119895

[119906 (119905119895+1

) minus 119906 (119905119895)]

ℎ119895= 119905119895+1

minus 119905119895

(2)

The value of the control variables at the center 119905119895+12

is givenby

119906 (119905119895+12

) =

119906 (119905119895) + 119906 (119905

119895+1)

2

119895 = 1 119896 minus 1(3)

The piecewise linear interpolation is used to preparefor the possibility of discontinuous solutions in controlSimilarly we can derive the approximate of the total numberof the packets 119908app(119905) and 119908(119905

119895+12) as above The state

variable 119909(119905) is approximated by a continuously differentiableand piecewise Hermite-Simpson cubic polynomial between119909(119905119895) and 119909(119905

119895+1) on the interval 119905

119895le 119905 le 119905

119895+1of length ℎ

119895

119909app (119905) =

3

sum

119903=0

119888

119895

119903

119905 minus 119905119895

ℎ119895

119888

119895

0= 119909 (119905

119895)

119888

119895

1= ℎ119895119891119895

119888

119895

2= minus3119909 (119905

119895) minus 2ℎ

119895119891119895+ 3119909 (119905

119895+1) minus (ℎ

119895) 119891119895+1

119888

119895

3= 2119909 (119905

119895) + ℎ119895119891119895minus 2119909 (119905

119895+1) + (ℎ

119895) 119891119895+1

(4)

where

119891119895= 119891 (119909 (119905

119895) 119906 (119905

119895) 119908 (119905

119895) 119905119895)

119905119895le 119905 le 119905

119895+1 119895 = 1 119896 minus 1

(5)

The value of the state variables at the center point 119905119895+12

of thecubic approximation is

119909 (119905119895+12

) =

119909 (119905119895) + 119909 (119905

119895+1)

2

+

119905119895+1

+ 119905119895

8

(119891119895+ 119891119895+1

)

119895 = 1 119896 minus 1

(6)

and the derivative is

(119905119895+12

) =

3 (119909119895+ 119909119895+1

)

2 (119905119895+1

+ 119905119895)

minus

1

4

(119891119895+ 119891119895+1

)

119895 = 1 119896 minus 1

(7)

In addition the chosen interpolating polynomial for the stateand control variablesmust satisfy themidpoint conditions forthe differential equations as follows

119891 (119909app (119905119895+12) 119906app (119905119895+12) 119908app (119905119895+12) 119905119895+12

)

minus app (119905119895+12) = 0

(8)

Equations (OCQ) can now be defined as a discretizedproblem as follows

min 119891 (119884) (9)

subject to 119891 (119909app (119905) 119906app (119905) 119908app (119905) 119905) minus app

= 0

119909app (1199051) minus 1199091= 0

0 le 119906app (119905) le 119908app (119905)

(10)

where 119909app 119906app and119908app are the approximations of the statethe control and the total number of packets constituting119884 in(9) This above discretization problem (9)-(10) can be solvedusing the following

(i) JModelica which is a package for simulation andoptimization ofModelicamodels (formore details see[20])

(ii) GAlib which is C++ library of genetic algorithm (formore details see [21])

32 Indirect Method Forward-Backward Sweeping We solveOCQ by using indirect method approach

(i) forming optimality conditions(ii) solving BVP by First-Order Sweeping algorithm

Let 119909(119905) be the adjoint variable At time 119905 let 119906lowast(119905) denotethe optimum controls and let 119909

lowast(119905) and 119909

lowast(119905) denote the

state and adjoint evaluated at the optimumUsing Pontryaginminimum principle we get the following equations

Hamiltonian Function

119867(119909 (119905) 119906 (119905) 119909 (119905)) = 119909 (119905) + 119877 (119908 (119905) minus 119906 (119905))

+ 119909 (119905) (119906 (119905) minus

120583119909 (119905)

119886 + 119909 (119905)

)

(11)

Adjoint Equations

119909 (119905) = minus1 + 119906 (119905)

119886120583

(119886 + 119909 (119905))

2 (12)

4 Scientific Programming

Transversality Condition

119909 (119905119891) = 0 (13)

Hamiltonian Minimization Condition Derivative of theHamiltonian is evaluated to zero at interior points hence

119867119906 (

119909 (119905) 119906 (119905) 119909 (119905) 120582 (119905) 119905)

= minus119877 + 119909 (119905) minus 1205821(119905) + 120582

2(119905) = 0

(14)

If 119906

lowast(119905) is optimal in (OCQ) then it must satisfy the

minimum condition119867(119909

lowast 119906

lowast 119909

lowast 119905) = min

Visin119880119867(119909 V 119909 119905) forall119905 isin 119873 (15)

Furthermore we assume the following

(H1) the Hamiltonian is strictly convex with respect tocontrol variable 119906

(H2) (119905) = argmin119906isin119880

119867(119909 119906 119909) 119905 isin [1199050 119905119891] is continu-

ous on [1199050 119905119891]

(H3) 119906 isin 119862

119903[1199050 119905119891]

Note that 119906(119905) is determined in a unique way for each119905 since 119880 is convex and 119867 is convex Now we consider theproblem of minimizing the Hamiltonian with respect to 119901

min119906isin119880

119867(119909 119906 119909) for each 119905 isin [1199050 119905119891] (16)

Usually in the literature 119906 is found explicitly as a function119906 = 119906(119909 119909 119905) and after substituting it into the system theproblem reduces to the boundary value problem

Theorem 1 Assume that conditions (H1)ndash(H3) hold andproblem has an optimal solution (119906

lowast 119909

lowast) Then for a given 120598

there exists a finite discretization

1199050= 1205910lt 1205911lt sdot sdot sdot lt 120591

119894lt sdot sdot sdot lt 120591

119873= 119905119891 (17)

and an approximate solution (119905) 119905 isin [1199050 119905119891] such that

1003817100381710038171003817119906

lowast(119905119894) minus (119905

119894)

1003817100381710038171003817lt 120598 119894 = 1 2 119873 (18)

Proof Let 119906

lowast be an optimal solution of problem Then(119901

lowast 119909

lowast) satisfies the conditions

lowast

119894=

120597119867 (119909

lowast 119906

lowast 119909

lowast)

120597119906119894

119909

lowast

119894= minus

120597119867 ((119909

lowast 119906

lowast 119909

lowast))

120597119909119894

119894 = 1 2 119873

(19)

where

119867(119909 119906 119909) =

119899

sum

119894=1

119909119894(119905) 119891119894(119909 119906) minus 119891

0(119909 119906)

119905 isin [1199050 119905119891]

(20)

and (119906

lowast 119909

lowast) satisfies the minimum principle

119867(119909

lowast 119906

lowast 119909

lowast) = min119906isin119880

119867(119909

lowast 119906 119909

lowast) 119905 isin [119905

0 119905119891] (21)

(a) First-Order Sweeping Method We linearize the Hamilto-nian around a reference solution119906(119905) and obtain the followingequation for variation 120575119906(119905)

0 = 119867119906

⊤+ 119867119906119906

120575119906 (119905) (22)

With the strong Legendre-Clebsch condition one canapproximate the above equation as

120575119906 (119905) = arg min120575119906isin119880minus119906

119867aug (119909 119906 + 120575119906 119909 119905) (23)

where

119867aug (119909 119906 + 120575119906 119909 119905) = 119867 (119909 119906 + 120575119906 119909 119905) + 120573 120575119906

2 (24)

is the augmented Hamiltonian function and 120573120575119906

2 is apenalty term This is really parametric optimization with120573 = 0 initially if119880 is a convex set and119867 is a convex functionwith respect to 119906 then 120575119906 is a descent direction If 119906(119905)+120575119906(119905)

does not yield a reduction then we set 120573 = 1 and then doubleit repeatedly until the objective is really reduced

Remark 2 An alternative way to ensure descent is to applyBacktracking Line-Search Procedure

(b) Parametric Optimization to OCQ Now we considerproblem (23) as one parametric minimization problemSince 119867aug(119909

lowast 119909

lowast 119906

lowast 120575119906 119905) is twice differentiable in 120575119906 and

assumptions (H1)ndash(H3) hold we can apply Theorem 1 to theproblem Then as a result it generates a discretization 119905

0=

1205910lt 1205911lt sdot sdot sdot lt 120591

119894lt sdot sdot sdot lt 120591

119873= 119905119891 and corresponding points

119894= (119905119894) such that1003817100381710038171003817120575119906

lowast(119905119894) minus 120575 (119905

119894)

1003817100381710038171003817lt 120598 119894 = 1 2 119873 (25)

which proves the assertion

Remark 3 Parametric optimization also can be applied infinding nominal optimal control given in [22] It is easy to seethat at each iteration 119896 the Hamiltonian function is a scalarfunction of 119906 isin 119880 sub 119877

119903 and 119905 isin 119879 = [1199050 119905119891] that is

119866119896(120575119906 119905) = 119867aug (119909

119896(119905) 119909

119896(119905) 120575119906 119905) (26)

The latter states that 120575

119896(119905) must be a minimizer of the

following problem

min120575119906isin119880

119866119896(120575119906 119905) 119905 isin 119879 (27)

which is a problem of parametric optimization as formulatedin various papers from [23] where the independent variable 119905is now considered as unknown parameter 119905 isin 119879 = [119905

0 119905119891]We

can also consider a case when the set of admissible controls istime-varying that is 119880 = 119880(119905) 119905 isin 119879 = [119905

0 119905119891] In this case

a general theory of parametric optimization is also applicablefor finding the nominal optimal controls

Scientific Programming 5

Let (119909lowast 120575119906lowast) be an optimal process in problem Introducethe function 119891 119880 times 119877 rarr 119877

119891 (120575119906 119905) = minus119867 (119909

lowast 120575119906 119909

lowast 119905) 119905 isin [119905

0 119905119891] (28)

where a parametric optimization problem is defined as

min120575119906isin119880

119891 (120575119906 119905) 119905 isin [1199050 119905119891]

119880 = 120575119906 isin 119877

119903 119892119894(120575119906) le 0 119894 isin 119869

119869 = 1 2 119904

(29)

The KKT conditions for the problem state that

119863120575119906119891 (120575119906 119905) + sum

119895isin119869

120583119894119863120575119906119892119894(120575119906 119905) = 0

119892119894(120575119906 119905) 120583

119894le 0 120583

119894ge 0 119894 isin 119869

120583119894119892119894(120575119906 119905) = 0

119894 isin 119869 119905 isin [1199050 119905119891]

(30)

where

119863120575119906119891 (120575119906 119905) = (

120597119891 (120575119906 119905)

1205971205751199061

120597119891 (120575119906 119905)

120597120575119906119903

) (31)

Consider the auxiliary parametric optimization problem

min 119891 (120575119906 119905) 119905 isin [1199050 119905119891]

subject to 119892119894(120575119906) = 0 119894 isin

119869 sub 119869

(32)

Let V0 = (120575119906

0(119905) 120583

0(119905)) satisfy the KKT conditions for

problem with 119869 = 119869

0 This system can be written in the

following compact notation

119865 (120575119906 119905) = 0 119905 isin [1199050 119905119891] (33)

where V = (120575119906

0(119905) 120583

0(119905)) In order to apply Newtonrsquos method

to system we have to solve a linear system with119863120575119906119865(V(119905) 119905)

as matrix The same matrix is used to compute V(119905)

119863120575119906119865 (120575119906 (119905) 119905) V (119905) = 119863

120575119906119891 (120575119906 (119905) 119905) (34)

Therefore using the Newton method as corrector wehave

119863120575119906119865 (120575119906

119896119894minus1

119894 119905119894) (120575119906

119896119894

119894minus 120575119906

119896119894minus1

119894) = minus119865 (120575119906

119896119894minus1

119894 119905119894)

119863120575119906119865 (120575119906

119896119894minus1

119894 119905119894) (

120575119906

119896119894minus1

119894) = minusΔ

119905119865 (120575119906

119896119894minus1

119894 119905119894)

(35)

4 Proposed Algorithm FB-OCQ

The first-order indirect approach motivates us to design FB-OCQ algorithm Shortly this algorithm uses gradient and

(1) Initialization119906(119905) 120575119906(119905) = 0 119906

+(119905) = 119906(119905) forall119905 isin [119905

0 119905119891]

Jminus= infin 120574 = 0 120573 120588 isin (0 1)

Iter = 1 MaxIter TOL(2) while Iter leMaxIter do(3) procedure Forward Sweep(4) 119909

+(1199050) = 1199090J+(1199050) = 0

(5) Integrate Forward 119905 1199050rarr 119905119891

(6) if Iter gt 1 then(7) 120575119906(119905) = argmin

120575119906119867aug(119909(119905) 119906(119905) + 120575119906 119909(119905) 119905)

(8) 119906+(119905) = 119906(119905) + 120575119906(119905)

(9) end if(10)

+(119905) = 119891(119909

+(119905) 119906+(119905) 119905)

(11) J+(119905) = 119897(119909

+(119905) 119906+(119905) 119905)

(12) end procedure(13) J

+= J+(119905119891)

(14) if J+lt Jminusthen

(15) Jminus= J+ 120573 = 120573120588

(16) else(17) 120573 = 120573120588 go to Forward Sweep(18) end if(19) procedure Adjoint Sweep(20) 119909(119905

119891) = 120601

⊤

119909(119909+(119905119891)) 120574(119905

0) = 0

(21) Integrate Backward 119905 119905119891rarr 1199050

(22) 119909(119905) = 119909+(119905) 119906(119905) = 119906

+(119905)

(23) 119909(119905)

⊤= minus(120597119867(119909(119905) 119906(119905) 119909(119905) 119905)120597119909)

(24) 119906(119905) = 120597119867(119909(119905) 119906(119905) 119909(119905) 119905)120597119906(25) (119905) = 119906(119905)

2(26) 120574 = 120574(119905

119891)

(27) end procedure(28) if 120574 lt TOL then(29) stop(30) end if(31) Iter larr Iter + 1(32) end while

Algorithm 1 FB-OCQ

does forward-backward searching for the optimal controlsolution Let us choose an initial control trajectory

(1199060 119906

119873) (1205751199060 120575119906

119873)

(119906+0

= 1199060 119906

+119873= 119880119873)

119869minus= infin 120574 Iter = 1 Δ

0 TOL MaxIter 120588 isin [0 1]

FromAlgorithm 1 (FB-OCQ) it is obvious that the algorithmterminates if the norm of the gradient of the Hamiltonianwith respect to 119906 120574 = 119867

1199062 during the run time of the

program is smaller than the tolerance and the parameter 120573

must change at the inner iterations if we do not have a descentdirection so we must divide 120573 by parameter 120588 isin (0 1) untilwe get a reduction in the cost function Figure 2 explainsour algorithm steps forward sweep (control variable and costfunction value) and backward sweep (adjoint variable) indetail

6 Scientific Programming

t0 t1 tf

Forward Backwardsweepsweep u = u + 120575u

J+ = 1 lt Jminus = infinrarr Jminus = J+ = 1

rarr Jminus = J+ = 05

Decrease 120573

Increase 120573J+ = 2 gt Jminus = 1

J+ = 05 lt Jminus = 1

x(t) u(t)

Figure 2 FB-OCQ algorithm explanation

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 3 Indirect method FB-OCQ

5 Performance Evaluation

51 Numerical Results In this section we provide someiteration results for both direct and indirect approaches asfollows

Indirect Method FB-OCQ The ODE was solved on equidis-tant discretization with 1000 discretization points The opti-mal control and optimal state are depicted in Figure 3 Werealize by looking at the final control that there exist somepoints where the set of active constraints changed due to thesingularity that is the optimal control is of the bang-bangtype with the possibility of a singular arc To the meaningof such singular arc for queue management it presents thesudden changes of input rate from 60 (packetssec) to 35

(packetssec) which accordingly result in the changes ofbuffer load (or queue length) In the 120573 history in Figure 3the parameter120573 changed at the inner iterations becausewe donot have a descent direction so wemust divide 120573 by 120588 isin (0 1)

until we get a reduction in the cost functionJ = 681280316The processing time of central processing unit (CPU) forthis algorithm is 004 s and the norm of the gradient of

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 4 Direct method nonlinear optimization with JModelicasolver

the Hamiltonian function 1198671199062with respect to the control

119906 goes to approximately 10119890 minus 7

Direct CollocationMethod JModelica and GAlib Solvers With119873 = 1000 and the number of interpolation points being 3we tested the discretized problem (9)-(10) by JModelica andGAlib solvers (see [20 21]) as follows

(i) JModelica Solver The cost function is reduced to J =

68678362 and the CPU processing time is 052 sThe optimalcontrol and optimal state trajectories are obtained duringJModelica running and illustrated in Figure 4 An advantageof this approach is that we do not need to derive the adjointequations

(ii) GAlib Solver For this problem we use the number ofgenerations 119899gen = 2500 and the population size is 200 Theoptimal control and optimal state are depicted in Figure 5obtained during the run of the GAlib The cost function isreduced toJ = 706744 during the run time of the programThe CPU processing time is 1290 s Although the genetic

Scientific Programming 7

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 5 Direct method genetic algorithm with GAlib solver

Table 1 Numerical result summary

Criteriamethods DirectGAlib

DirectJModelica

IndirectFB-OCQ

Cost function value J = 7067 J = 6867 J = 6812

CPU process time 129 (sec) 052 (sec) 004 (sec)

algorithm has an advantage that it needs no derivatives orHessianrsquos information the control functions produced by itare useless and the convergence is very slow in comparison toAlgorithm 1 (FB-OCQ) and the one using JModelica solver

Table 1 summarizes and compares the three methods thatwe develop numerically We conclude that Algorithm 1 ismuch faster than the other solvers and gives the best costfunction By using genetic library GAlib the cost functiondoes not reach the local solution as it claims in finding theglobal solution The limitation of the indirect methods (FB-OCQ) is that one should derive the adjoint equations whichare not easy to derive in some applications

52 Simulation Results In this section the performance ofthe obtained algorithm (FB-OCQ) is evaluated by comparingit with some popular AQMs including RED REM and PIThe credibility of results is confirmed using ns-2 simulator[24] We investigate a network topology with 40 sourcesan intermediate router and one destination (Figure 6)All sources simultaneously send packets to the destinationthrough the router Hence a large queue is built up at therouter or bottleneck pointMaximum buffer size of the routeris 100 packets All of the compared RED REM and PIalgorithms are configured to obtain the desired queue lengthat 40 packets Simulation lasts for 60 seconds and is repeatedusing a built-in random generator in ns-2 to obtain morecredible results

DestinationRouter

Source 1

Source 2

Source 40

Figure 6 Simulation topology

FB-OCQRED

000

001

002

003

004

005

Dro

ppin

g pr

obab

ility

10 20 30 40 50 600Time (sec)

Figure 7 Dropping probability

Figure 7 shows dropping probability of FB-OCQ andRED algorithms With the proposed square-root drop func-tion FB-OCQ drops queuing packets more aggressivelythan RED although dropping frequency is nearly the sameMaximum dropping ratio is 0045 for FB-OCQ and 0039 forRED In fact when the algorithm drops more packets thequeue stability will be increased but we will have to sacrificethe system throughput performanceWe can see this trade-offin next results

Figure 8 presents average queue length values measuredat the bottleneck link from router to destination for differentalgorithms Due to aggressive dropping FB-OCQ maintainsthe shortest queue at 40 packets which is also the desiredvalue Only REM can obtain the same value but in alonger time asymp50 seconds PI in fact can achieve the sameperformance only if its parameters are well configured andbe dynamically changed to different network scenarios InFigure 8 we can see that PI performs not so well even weset the desired queue length variable at 40 packets and exploitdefault parameters in ns-2 for PI controller

Finally we investigate throughput performance of pro-posed FB-OCQ algorithm It is confirmed from Figure 9that there exists a trade-off in the relationship between

8 Scientific Programming

FB-OCQRED

REMPI

0

10

20

30

40

50

60

70

Aver

age q

ueue

leng

th (p

acke

ts)

10 20 30 40 50 600Time (sec)

Figure 8 Average queue length versus time

FB-OCQ RED REM PI000

001

002

003

004

005

006

007

008

009

Thro

ughp

ut (M

bps)

Figure 9 Throughput comparison

throughput and queue lengthdropping probability PI andREM achieve nearly the same throughput at 006 (Mbps)while FB-OCQandREDobtain the better throughput perfor-mance at 008 (Mbps) value In Figure 8 although REM canachieve the desired queue length 40 packets in this scenarioREM still maintains a large queue during simulation timefrom 0 (sec) up to 25 (sec) That reason leads to throughputperformance of REM being lower than RED and FB-OCQOur proposed FB-OCQ drops aggressively the packets sothat its throughput value (0075904Mbps) is slightly less thanRED (0082146Mbps)

53 Experiment Results We examine the effectiveness ofour state-of-the-art optimal control queue method in theLinux kernel (eg Ubuntu 1604) We conduct experimentfrom a desktop computer through the Internet gateway toan outside server (Figure 10) To manage working queuingdisciplines (qdiscs) we use the scheduler qdisc in the Linuxkernel The chosen server is a dedicated bufferbloat server

Bufferbloat server

DropTail

REDFB-OCQ

Experiment

Figure 10 Experiment test setup from the test client to the testserver

which is able to stand very high congestion due to manydata flows at the same time We exploit the Flent FLExibleNetwork Tester [25] and Realtime Response Under Load(RRUL) test [26] to evaluate our proposal RRUL test puts anetwork under worst case conditions reliably saturates thenetwork link and thus recreates bufferbloat phenomenonfor queue algorithm testing Simulation time duration is 60

secondsWe compare latency under load test with queue being

handled by different AQM schemes (DropTail RED and FB-OCQ) in turnDropTail queuingmethod is by far the simplestapproach to network router queue management The routeraccepts and forwards all the packets that arrive as long as itsbuffer space is available for the next incoming packets If apacket arrives and the queue is currently full the incomingpacket will be dropped The sender then detects the packetlost event and shrinks its sendingwindowWhile it is themostwidely used due to simplicity and relatively high efficiencyDropTail has some weakness such as the bad fairness sharingamongTCP connections and throughput and bottleneck linkefficiency suffer severe degradation if congestion is gettingworse

RED [5 6] was presented with the objective of min-imizing packet loss and queuing delay Moreover it cancompensate the weakness of DropTail by avoiding globalsynchronization of TCP sources so that it improves fair-ness To achieve these goals RED utilizes two thresholdsminth and maxth and an exponentially weighted movingaverage (EWMA) formula to estimate average queue length[27] When the average queue length exceeds a predefinedthreshold the link is implied to be in congested state anddrop action is taken A temporary increase in the queuelength notifies the transient congestion while an increasein the average queue length reflects long-lived congestionBased on such information RED router sends randomizedfeedback signals to the senders tomake decision of decreasingtheir congestion windows RED has good fairness amongconnections because of the feedback randomizedmechanism[28]

Figure 11 presents latency ping results under RRUL testsuite Ping is a networking utility and operates by sendingInternet Control Message Protocol (ICMP) echo requestpackets to the target server and waits for an ICMP echoreply The program measures the round-trip time fromtransmission to reception reporting errors and packet lossOur proposed algorithm FB-OCQ achieves the lowest packetlatency compared with the other two algorithms inside Linux

Scientific Programming 9

DropTailREDFB-OCQ

0 10 20

Simulation time (s)

30 40 50 60

101

102

103

Pack

et la

tenc

y (m

s)

Figure 11 ICMP ping result using Realtime Response Under Loadtest

kernel Specifically the packet latency when using FB-OCQis about 80 (ms) while about 120 (ms) if using DropTail(pfifo fast in Ubuntu) and 100ms if using RED algorithm

6 Conclusions

We proposed a queue management algorithm namedforward-backward optimal control queue (FB-OCQ) tosolve the OCQ problem Derived from indirect approachesin dynamic optimization this algorithm demonstrates fasterreaction while still achieving the same performance innumerical analysis compared to direct methods Employingunder network simulation ns-2 we see that the proposedalgorithm drops packets more aggressively than thetraditional RED algorithm in a higher frequency andmagnitude As a result average queue length can be reducedmuch more while an acceptable value of throughput stillcan be maintained In future works we try to investigatethe memory efficiency of FB-OCQ under wireless sensornetworks

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work was supported by the National Research Founda-tion of Korea (NRF) grant funded by the Korean government(MEST) (no NRF-2016R1A2B1013733)

References

[1] T Hoiland-Jorgensen ldquoBattling bufferbloat an experimentalcomparison of four apapproach to queuemanagement in linuxrdquoTech Rep Roskilde University Roskilde Denmark 2012

[2] J Gettys ldquoBufferbloat dark buffers in the Internetrdquo IEEEInternet Computing vol 15 no 3 pp 95ndash96 2011

[3] J Gettys ldquoBufferbloat views of the elephantrdquo in Proceedingsof the International Summit for Community Wireless NetworksOctober 2012

[4] B Braden D Clark J Crowcroft et al ldquoRecommendations onqueue management and congestion avoidance in the internetrdquoRFC 2309 IETF San Francisco Calif USA 1998

[5] C Hollot V Misra D Towsley and W-B Gong ldquoA controltheoretic analysis of redrdquo in Proceedings of the 20th Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo01) vol 3 pp 1510ndash1519 IEEE Anchorage AlaskaUSA 2001

[6] S Floyd and V Jacobson ldquoRandom early detection gateways forcongestion avoidancerdquo IEEEACM Transactions on Networkingvol 1 no 4 pp 397ndash413 1993

[7] S Athuraliya S Low and D Lapsley ldquoRandom early markingrdquoin Quality of Future Internet Services J Crowcroft J Robertsand M I Smirnov Eds vol 1922 of Lecture Notes in ComputerScience pp 43ndash54 Springer Berlin Germany 2000

[8] R Pan PNatarajan C Piglione et al ldquoPIE a lightweight controlscheme to address the bufferbloat problemrdquo in Proceedings ofthe IEEE 14th International Conference on High PerformanceSwitching and Routing (HPSR rsquo13) pp 148ndash155 Taipei TaiwanJuly 2013

[9] F Ren C Lin and BWei ldquoDesign a robust controller for activequeue management in large delay networksrdquo in Proceedings ofthe 9th International Symposium on Computers and Communi-cations (ISCC rsquo04) vol 2 pp 748ndash754 June-July 2004

[10] R Zhu H Teng and J Fu ldquoA predictive PID controller forAQM router supporting TCP with ECNrdquo in Proceedings ofthe 5th International Symposium on Multi-Dimensional MobileCommunications The 2004 Joint Conference of the 10th Asia-Pacific Conference on Communications vol 1 pp 356ndash360August-September 2004

[11] J Sun M Zukerman and M Palaniswami ldquoStabilizing REDusing a fuzzy controllerrdquo in Proceedings of the IEEE Interna-tional Conference on Communications (ICC rsquo07) pp 266ndash271Glasgow Scotland June 2007

[12] H-L To T M Thi and W-J Hwang ldquoCascade probabilitycontrol to mitigate bufferbloat under multiple real-world TCPstacksrdquoMathematical Problems in Engineering vol 2015 ArticleID 628583 13 pages 2015

[13] M Hassan and H Sirisena ldquoOptimal control of queues in com-puter networksrdquo in Proceedings of the International Conferenceon Communications (ICC rsquo01) vol 2 pp 637ndash641 June 2001

[14] V Guffens and G Bastin ldquoOptimal adaptive feedback controlof a network bufferrdquo in Proceedings of the American ControlConference (ACC rsquo05) vol 3 pp 1835ndash1840 Portland Ore USAJune 2005

[15] J T Betts Practical Methods for Optimal Control Using Nonlin-ear Programming Society for Industrial andAppliedMathemat-ics Philadelphia Pa USA 2001

[16] R Bulirsch E Nerz H Pesch and O von Stryk ldquoCombiningdirect and indirect methods in optimal control Range maxi-mization of a hang gliderrdquo in Optimal Control R Bulirsch AMiele J Stoer and K Well Eds vol 111 of ISNM InternationalSeries of Numerical Mathematics pp 273ndash288 Birkhuser BaselSwitzerland 1993

[17] A Radwan Utilization of parametric programming and evo-lutionary computing in optimal control [PhD dissertation]Humboldt-Universitat zu Berlin 2012

10 Scientific Programming

[18] O von Stryk and R Bulirsch ldquoDirect and indirect methods fortrajectory optimizationrdquo Annals of Operations Research vol 37no 1 pp 357ndash373 1992

[19] A Bryson and Y Ho Applied Optimal Control OptimizationEstimation and Control Hemisphere CRC Press WashingtonDC USA 1975

[20] M Ab Jmodelicaorg user guide 160 2011 httpwwwjmode-licaorgpage236

[21] M Wall GAlib A C++ Library of Genetic Algorithm Compo-nents Version 24 Documentation Revision B MassachusettsInstitute of Technology 1996

[22] A Radwan O Vasilieva R Enkhbat A Griewank and J Gud-dat ldquoParametric approach to optimal controlrdquo OptimizationLetters vol 6 no 7 pp 1303ndash1316 2012

[23] J Guddat FGuerraVazquez andHT JongenParametricOpti-mization Singularities Pathfollowing and Jumps John Wiley ampSons 1990

[24] T Issariyakul and EHossain Introduction to Network SimulatorNS2 Springer Berlin Germany 1st edition 2008

[25] T Hiland-Jrgensen Flent The flexible network tester 2012ndash2015 httpsflentorg

[26] DTht ldquoRealtime response under load (rrul) testrdquo 2013 httpsgithubcomdtahtdeBloatblobmasterspecrruledoc

[27] J Koo S Ahn and J Chung ldquoA comparative study of queuedelay and loss characteristics of AQM schemes in QoS-enablednetworksrdquo Computing and Informatics vol 23 no 4 pp 317ndash335 2004

[28] T B Reddy and A Ahammed ldquoPerformance comparison ofactive queue management techniquesrdquo Journal of ComputerScience vol 4 no 12 pp 1020ndash1023 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

4 Scientific Programming

Transversality Condition

119909 (119905119891) = 0 (13)

Hamiltonian Minimization Condition Derivative of theHamiltonian is evaluated to zero at interior points hence

119867119906 (

119909 (119905) 119906 (119905) 119909 (119905) 120582 (119905) 119905)

= minus119877 + 119909 (119905) minus 1205821(119905) + 120582

2(119905) = 0

(14)

If 119906

lowast(119905) is optimal in (OCQ) then it must satisfy the

minimum condition119867(119909

lowast 119906

lowast 119909

lowast 119905) = min

Visin119880119867(119909 V 119909 119905) forall119905 isin 119873 (15)

Furthermore we assume the following

(H1) the Hamiltonian is strictly convex with respect tocontrol variable 119906

(H2) (119905) = argmin119906isin119880

119867(119909 119906 119909) 119905 isin [1199050 119905119891] is continu-

ous on [1199050 119905119891]

(H3) 119906 isin 119862

119903[1199050 119905119891]

Note that 119906(119905) is determined in a unique way for each119905 since 119880 is convex and 119867 is convex Now we consider theproblem of minimizing the Hamiltonian with respect to 119901

min119906isin119880

119867(119909 119906 119909) for each 119905 isin [1199050 119905119891] (16)

Usually in the literature 119906 is found explicitly as a function119906 = 119906(119909 119909 119905) and after substituting it into the system theproblem reduces to the boundary value problem

Theorem 1 Assume that conditions (H1)ndash(H3) hold andproblem has an optimal solution (119906

lowast 119909

lowast) Then for a given 120598

there exists a finite discretization

1199050= 1205910lt 1205911lt sdot sdot sdot lt 120591

119894lt sdot sdot sdot lt 120591

119873= 119905119891 (17)

and an approximate solution (119905) 119905 isin [1199050 119905119891] such that

1003817100381710038171003817119906

lowast(119905119894) minus (119905

119894)

1003817100381710038171003817lt 120598 119894 = 1 2 119873 (18)

Proof Let 119906

lowast be an optimal solution of problem Then(119901

lowast 119909

lowast) satisfies the conditions

lowast

119894=

120597119867 (119909

lowast 119906

lowast 119909

lowast)

120597119906119894

119909

lowast

119894= minus

120597119867 ((119909

lowast 119906

lowast 119909

lowast))

120597119909119894

119894 = 1 2 119873

(19)

where

119867(119909 119906 119909) =

119899

sum

119894=1

119909119894(119905) 119891119894(119909 119906) minus 119891

0(119909 119906)

119905 isin [1199050 119905119891]

(20)

and (119906

lowast 119909

lowast) satisfies the minimum principle

119867(119909

lowast 119906

lowast 119909

lowast) = min119906isin119880

119867(119909

lowast 119906 119909

lowast) 119905 isin [119905

0 119905119891] (21)

(a) First-Order Sweeping Method We linearize the Hamilto-nian around a reference solution119906(119905) and obtain the followingequation for variation 120575119906(119905)

0 = 119867119906

⊤+ 119867119906119906

120575119906 (119905) (22)

With the strong Legendre-Clebsch condition one canapproximate the above equation as

120575119906 (119905) = arg min120575119906isin119880minus119906

119867aug (119909 119906 + 120575119906 119909 119905) (23)

where

119867aug (119909 119906 + 120575119906 119909 119905) = 119867 (119909 119906 + 120575119906 119909 119905) + 120573 120575119906

2 (24)

is the augmented Hamiltonian function and 120573120575119906

2 is apenalty term This is really parametric optimization with120573 = 0 initially if119880 is a convex set and119867 is a convex functionwith respect to 119906 then 120575119906 is a descent direction If 119906(119905)+120575119906(119905)

does not yield a reduction then we set 120573 = 1 and then doubleit repeatedly until the objective is really reduced

Remark 2 An alternative way to ensure descent is to applyBacktracking Line-Search Procedure

(b) Parametric Optimization to OCQ Now we considerproblem (23) as one parametric minimization problemSince 119867aug(119909

lowast 119909

lowast 119906

lowast 120575119906 119905) is twice differentiable in 120575119906 and

assumptions (H1)ndash(H3) hold we can apply Theorem 1 to theproblem Then as a result it generates a discretization 119905

0=

1205910lt 1205911lt sdot sdot sdot lt 120591

119894lt sdot sdot sdot lt 120591

119873= 119905119891 and corresponding points

119894= (119905119894) such that1003817100381710038171003817120575119906

lowast(119905119894) minus 120575 (119905

119894)

1003817100381710038171003817lt 120598 119894 = 1 2 119873 (25)

which proves the assertion

Remark 3 Parametric optimization also can be applied infinding nominal optimal control given in [22] It is easy to seethat at each iteration 119896 the Hamiltonian function is a scalarfunction of 119906 isin 119880 sub 119877

119903 and 119905 isin 119879 = [1199050 119905119891] that is

119866119896(120575119906 119905) = 119867aug (119909

119896(119905) 119909

119896(119905) 120575119906 119905) (26)

The latter states that 120575

119896(119905) must be a minimizer of the

following problem

min120575119906isin119880

119866119896(120575119906 119905) 119905 isin 119879 (27)

which is a problem of parametric optimization as formulatedin various papers from [23] where the independent variable 119905is now considered as unknown parameter 119905 isin 119879 = [119905

0 119905119891]We

can also consider a case when the set of admissible controls istime-varying that is 119880 = 119880(119905) 119905 isin 119879 = [119905

0 119905119891] In this case

a general theory of parametric optimization is also applicablefor finding the nominal optimal controls

Scientific Programming 5

Let (119909lowast 120575119906lowast) be an optimal process in problem Introducethe function 119891 119880 times 119877 rarr 119877

119891 (120575119906 119905) = minus119867 (119909

lowast 120575119906 119909

lowast 119905) 119905 isin [119905

0 119905119891] (28)

where a parametric optimization problem is defined as

min120575119906isin119880

119891 (120575119906 119905) 119905 isin [1199050 119905119891]

119880 = 120575119906 isin 119877

119903 119892119894(120575119906) le 0 119894 isin 119869

119869 = 1 2 119904

(29)

The KKT conditions for the problem state that

119863120575119906119891 (120575119906 119905) + sum

119895isin119869

120583119894119863120575119906119892119894(120575119906 119905) = 0

119892119894(120575119906 119905) 120583

119894le 0 120583

119894ge 0 119894 isin 119869

120583119894119892119894(120575119906 119905) = 0

119894 isin 119869 119905 isin [1199050 119905119891]

(30)

where

119863120575119906119891 (120575119906 119905) = (

120597119891 (120575119906 119905)

1205971205751199061

120597119891 (120575119906 119905)

120597120575119906119903

) (31)

Consider the auxiliary parametric optimization problem

min 119891 (120575119906 119905) 119905 isin [1199050 119905119891]

subject to 119892119894(120575119906) = 0 119894 isin

119869 sub 119869

(32)

Let V0 = (120575119906

0(119905) 120583

0(119905)) satisfy the KKT conditions for

problem with 119869 = 119869

0 This system can be written in the

following compact notation

119865 (120575119906 119905) = 0 119905 isin [1199050 119905119891] (33)

where V = (120575119906

0(119905) 120583

0(119905)) In order to apply Newtonrsquos method

to system we have to solve a linear system with119863120575119906119865(V(119905) 119905)

as matrix The same matrix is used to compute V(119905)

119863120575119906119865 (120575119906 (119905) 119905) V (119905) = 119863

120575119906119891 (120575119906 (119905) 119905) (34)

Therefore using the Newton method as corrector wehave

119863120575119906119865 (120575119906

119896119894minus1

119894 119905119894) (120575119906

119896119894

119894minus 120575119906

119896119894minus1

119894) = minus119865 (120575119906

119896119894minus1

119894 119905119894)

119863120575119906119865 (120575119906

119896119894minus1

119894 119905119894) (

120575119906

119896119894minus1

119894) = minusΔ

119905119865 (120575119906

119896119894minus1

119894 119905119894)

(35)

4 Proposed Algorithm FB-OCQ

The first-order indirect approach motivates us to design FB-OCQ algorithm Shortly this algorithm uses gradient and

(1) Initialization119906(119905) 120575119906(119905) = 0 119906

+(119905) = 119906(119905) forall119905 isin [119905

0 119905119891]

Jminus= infin 120574 = 0 120573 120588 isin (0 1)

Iter = 1 MaxIter TOL(2) while Iter leMaxIter do(3) procedure Forward Sweep(4) 119909

+(1199050) = 1199090J+(1199050) = 0

(5) Integrate Forward 119905 1199050rarr 119905119891

(6) if Iter gt 1 then(7) 120575119906(119905) = argmin

120575119906119867aug(119909(119905) 119906(119905) + 120575119906 119909(119905) 119905)

(8) 119906+(119905) = 119906(119905) + 120575119906(119905)

(9) end if(10)

+(119905) = 119891(119909

+(119905) 119906+(119905) 119905)

(11) J+(119905) = 119897(119909

+(119905) 119906+(119905) 119905)

(12) end procedure(13) J

+= J+(119905119891)

(14) if J+lt Jminusthen

(15) Jminus= J+ 120573 = 120573120588

(16) else(17) 120573 = 120573120588 go to Forward Sweep(18) end if(19) procedure Adjoint Sweep(20) 119909(119905

119891) = 120601

⊤

119909(119909+(119905119891)) 120574(119905

0) = 0

(21) Integrate Backward 119905 119905119891rarr 1199050

(22) 119909(119905) = 119909+(119905) 119906(119905) = 119906

+(119905)

(23) 119909(119905)

⊤= minus(120597119867(119909(119905) 119906(119905) 119909(119905) 119905)120597119909)

(24) 119906(119905) = 120597119867(119909(119905) 119906(119905) 119909(119905) 119905)120597119906(25) (119905) = 119906(119905)

2(26) 120574 = 120574(119905

119891)

(27) end procedure(28) if 120574 lt TOL then(29) stop(30) end if(31) Iter larr Iter + 1(32) end while

Algorithm 1 FB-OCQ

does forward-backward searching for the optimal controlsolution Let us choose an initial control trajectory

(1199060 119906

119873) (1205751199060 120575119906

119873)

(119906+0

= 1199060 119906

+119873= 119880119873)

119869minus= infin 120574 Iter = 1 Δ

0 TOL MaxIter 120588 isin [0 1]

FromAlgorithm 1 (FB-OCQ) it is obvious that the algorithmterminates if the norm of the gradient of the Hamiltonianwith respect to 119906 120574 = 119867

1199062 during the run time of the

program is smaller than the tolerance and the parameter 120573

must change at the inner iterations if we do not have a descentdirection so we must divide 120573 by parameter 120588 isin (0 1) untilwe get a reduction in the cost function Figure 2 explainsour algorithm steps forward sweep (control variable and costfunction value) and backward sweep (adjoint variable) indetail

6 Scientific Programming

t0 t1 tf

Forward Backwardsweepsweep u = u + 120575u

J+ = 1 lt Jminus = infinrarr Jminus = J+ = 1

rarr Jminus = J+ = 05

Decrease 120573

Increase 120573J+ = 2 gt Jminus = 1

J+ = 05 lt Jminus = 1

x(t) u(t)

Figure 2 FB-OCQ algorithm explanation

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 3 Indirect method FB-OCQ

5 Performance Evaluation

51 Numerical Results In this section we provide someiteration results for both direct and indirect approaches asfollows

Indirect Method FB-OCQ The ODE was solved on equidis-tant discretization with 1000 discretization points The opti-mal control and optimal state are depicted in Figure 3 Werealize by looking at the final control that there exist somepoints where the set of active constraints changed due to thesingularity that is the optimal control is of the bang-bangtype with the possibility of a singular arc To the meaningof such singular arc for queue management it presents thesudden changes of input rate from 60 (packetssec) to 35

(packetssec) which accordingly result in the changes ofbuffer load (or queue length) In the 120573 history in Figure 3the parameter120573 changed at the inner iterations becausewe donot have a descent direction so wemust divide 120573 by 120588 isin (0 1)

until we get a reduction in the cost functionJ = 681280316The processing time of central processing unit (CPU) forthis algorithm is 004 s and the norm of the gradient of

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 4 Direct method nonlinear optimization with JModelicasolver

the Hamiltonian function 1198671199062with respect to the control

119906 goes to approximately 10119890 minus 7

Direct CollocationMethod JModelica and GAlib Solvers With119873 = 1000 and the number of interpolation points being 3we tested the discretized problem (9)-(10) by JModelica andGAlib solvers (see [20 21]) as follows

(i) JModelica Solver The cost function is reduced to J =

68678362 and the CPU processing time is 052 sThe optimalcontrol and optimal state trajectories are obtained duringJModelica running and illustrated in Figure 4 An advantageof this approach is that we do not need to derive the adjointequations

(ii) GAlib Solver For this problem we use the number ofgenerations 119899gen = 2500 and the population size is 200 Theoptimal control and optimal state are depicted in Figure 5obtained during the run of the GAlib The cost function isreduced toJ = 706744 during the run time of the programThe CPU processing time is 1290 s Although the genetic

Scientific Programming 7

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 5 Direct method genetic algorithm with GAlib solver

Table 1 Numerical result summary

Criteriamethods DirectGAlib

DirectJModelica

IndirectFB-OCQ

Cost function value J = 7067 J = 6867 J = 6812

CPU process time 129 (sec) 052 (sec) 004 (sec)

algorithm has an advantage that it needs no derivatives orHessianrsquos information the control functions produced by itare useless and the convergence is very slow in comparison toAlgorithm 1 (FB-OCQ) and the one using JModelica solver

Table 1 summarizes and compares the three methods thatwe develop numerically We conclude that Algorithm 1 ismuch faster than the other solvers and gives the best costfunction By using genetic library GAlib the cost functiondoes not reach the local solution as it claims in finding theglobal solution The limitation of the indirect methods (FB-OCQ) is that one should derive the adjoint equations whichare not easy to derive in some applications

52 Simulation Results In this section the performance ofthe obtained algorithm (FB-OCQ) is evaluated by comparingit with some popular AQMs including RED REM and PIThe credibility of results is confirmed using ns-2 simulator[24] We investigate a network topology with 40 sourcesan intermediate router and one destination (Figure 6)All sources simultaneously send packets to the destinationthrough the router Hence a large queue is built up at therouter or bottleneck pointMaximum buffer size of the routeris 100 packets All of the compared RED REM and PIalgorithms are configured to obtain the desired queue lengthat 40 packets Simulation lasts for 60 seconds and is repeatedusing a built-in random generator in ns-2 to obtain morecredible results

DestinationRouter

Source 1

Source 2

Source 40

Figure 6 Simulation topology

FB-OCQRED

000

001

002

003

004

005

Dro

ppin

g pr

obab

ility

10 20 30 40 50 600Time (sec)

Figure 7 Dropping probability

Figure 7 shows dropping probability of FB-OCQ andRED algorithms With the proposed square-root drop func-tion FB-OCQ drops queuing packets more aggressivelythan RED although dropping frequency is nearly the sameMaximum dropping ratio is 0045 for FB-OCQ and 0039 forRED In fact when the algorithm drops more packets thequeue stability will be increased but we will have to sacrificethe system throughput performanceWe can see this trade-offin next results

Figure 8 presents average queue length values measuredat the bottleneck link from router to destination for differentalgorithms Due to aggressive dropping FB-OCQ maintainsthe shortest queue at 40 packets which is also the desiredvalue Only REM can obtain the same value but in alonger time asymp50 seconds PI in fact can achieve the sameperformance only if its parameters are well configured andbe dynamically changed to different network scenarios InFigure 8 we can see that PI performs not so well even weset the desired queue length variable at 40 packets and exploitdefault parameters in ns-2 for PI controller

Finally we investigate throughput performance of pro-posed FB-OCQ algorithm It is confirmed from Figure 9that there exists a trade-off in the relationship between

8 Scientific Programming

FB-OCQRED

REMPI

0

10

20

30

40

50

60

70

Aver

age q

ueue

leng

th (p

acke

ts)

10 20 30 40 50 600Time (sec)

Figure 8 Average queue length versus time

FB-OCQ RED REM PI000

001

002

003

004

005

006

007

008

009

Thro

ughp

ut (M

bps)

Figure 9 Throughput comparison

throughput and queue lengthdropping probability PI andREM achieve nearly the same throughput at 006 (Mbps)while FB-OCQandREDobtain the better throughput perfor-mance at 008 (Mbps) value In Figure 8 although REM canachieve the desired queue length 40 packets in this scenarioREM still maintains a large queue during simulation timefrom 0 (sec) up to 25 (sec) That reason leads to throughputperformance of REM being lower than RED and FB-OCQOur proposed FB-OCQ drops aggressively the packets sothat its throughput value (0075904Mbps) is slightly less thanRED (0082146Mbps)

53 Experiment Results We examine the effectiveness ofour state-of-the-art optimal control queue method in theLinux kernel (eg Ubuntu 1604) We conduct experimentfrom a desktop computer through the Internet gateway toan outside server (Figure 10) To manage working queuingdisciplines (qdiscs) we use the scheduler qdisc in the Linuxkernel The chosen server is a dedicated bufferbloat server

Bufferbloat server

DropTail

REDFB-OCQ

Experiment

Figure 10 Experiment test setup from the test client to the testserver

which is able to stand very high congestion due to manydata flows at the same time We exploit the Flent FLExibleNetwork Tester [25] and Realtime Response Under Load(RRUL) test [26] to evaluate our proposal RRUL test puts anetwork under worst case conditions reliably saturates thenetwork link and thus recreates bufferbloat phenomenonfor queue algorithm testing Simulation time duration is 60

secondsWe compare latency under load test with queue being

handled by different AQM schemes (DropTail RED and FB-OCQ) in turnDropTail queuingmethod is by far the simplestapproach to network router queue management The routeraccepts and forwards all the packets that arrive as long as itsbuffer space is available for the next incoming packets If apacket arrives and the queue is currently full the incomingpacket will be dropped The sender then detects the packetlost event and shrinks its sendingwindowWhile it is themostwidely used due to simplicity and relatively high efficiencyDropTail has some weakness such as the bad fairness sharingamongTCP connections and throughput and bottleneck linkefficiency suffer severe degradation if congestion is gettingworse

RED [5 6] was presented with the objective of min-imizing packet loss and queuing delay Moreover it cancompensate the weakness of DropTail by avoiding globalsynchronization of TCP sources so that it improves fair-ness To achieve these goals RED utilizes two thresholdsminth and maxth and an exponentially weighted movingaverage (EWMA) formula to estimate average queue length[27] When the average queue length exceeds a predefinedthreshold the link is implied to be in congested state anddrop action is taken A temporary increase in the queuelength notifies the transient congestion while an increasein the average queue length reflects long-lived congestionBased on such information RED router sends randomizedfeedback signals to the senders tomake decision of decreasingtheir congestion windows RED has good fairness amongconnections because of the feedback randomizedmechanism[28]

Figure 11 presents latency ping results under RRUL testsuite Ping is a networking utility and operates by sendingInternet Control Message Protocol (ICMP) echo requestpackets to the target server and waits for an ICMP echoreply The program measures the round-trip time fromtransmission to reception reporting errors and packet lossOur proposed algorithm FB-OCQ achieves the lowest packetlatency compared with the other two algorithms inside Linux

Scientific Programming 9

DropTailREDFB-OCQ

0 10 20

Simulation time (s)

30 40 50 60

101

102

103

Pack

et la

tenc

y (m

s)

Figure 11 ICMP ping result using Realtime Response Under Loadtest

kernel Specifically the packet latency when using FB-OCQis about 80 (ms) while about 120 (ms) if using DropTail(pfifo fast in Ubuntu) and 100ms if using RED algorithm

6 Conclusions

We proposed a queue management algorithm namedforward-backward optimal control queue (FB-OCQ) tosolve the OCQ problem Derived from indirect approachesin dynamic optimization this algorithm demonstrates fasterreaction while still achieving the same performance innumerical analysis compared to direct methods Employingunder network simulation ns-2 we see that the proposedalgorithm drops packets more aggressively than thetraditional RED algorithm in a higher frequency andmagnitude As a result average queue length can be reducedmuch more while an acceptable value of throughput stillcan be maintained In future works we try to investigatethe memory efficiency of FB-OCQ under wireless sensornetworks

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work was supported by the National Research Founda-tion of Korea (NRF) grant funded by the Korean government(MEST) (no NRF-2016R1A2B1013733)

References

[1] T Hoiland-Jorgensen ldquoBattling bufferbloat an experimentalcomparison of four apapproach to queuemanagement in linuxrdquoTech Rep Roskilde University Roskilde Denmark 2012

[2] J Gettys ldquoBufferbloat dark buffers in the Internetrdquo IEEEInternet Computing vol 15 no 3 pp 95ndash96 2011

[3] J Gettys ldquoBufferbloat views of the elephantrdquo in Proceedingsof the International Summit for Community Wireless NetworksOctober 2012

[4] B Braden D Clark J Crowcroft et al ldquoRecommendations onqueue management and congestion avoidance in the internetrdquoRFC 2309 IETF San Francisco Calif USA 1998

[5] C Hollot V Misra D Towsley and W-B Gong ldquoA controltheoretic analysis of redrdquo in Proceedings of the 20th Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo01) vol 3 pp 1510ndash1519 IEEE Anchorage AlaskaUSA 2001

[6] S Floyd and V Jacobson ldquoRandom early detection gateways forcongestion avoidancerdquo IEEEACM Transactions on Networkingvol 1 no 4 pp 397ndash413 1993

[7] S Athuraliya S Low and D Lapsley ldquoRandom early markingrdquoin Quality of Future Internet Services J Crowcroft J Robertsand M I Smirnov Eds vol 1922 of Lecture Notes in ComputerScience pp 43ndash54 Springer Berlin Germany 2000

[8] R Pan PNatarajan C Piglione et al ldquoPIE a lightweight controlscheme to address the bufferbloat problemrdquo in Proceedings ofthe IEEE 14th International Conference on High PerformanceSwitching and Routing (HPSR rsquo13) pp 148ndash155 Taipei TaiwanJuly 2013

[9] F Ren C Lin and BWei ldquoDesign a robust controller for activequeue management in large delay networksrdquo in Proceedings ofthe 9th International Symposium on Computers and Communi-cations (ISCC rsquo04) vol 2 pp 748ndash754 June-July 2004

[10] R Zhu H Teng and J Fu ldquoA predictive PID controller forAQM router supporting TCP with ECNrdquo in Proceedings ofthe 5th International Symposium on Multi-Dimensional MobileCommunications The 2004 Joint Conference of the 10th Asia-Pacific Conference on Communications vol 1 pp 356ndash360August-September 2004

[11] J Sun M Zukerman and M Palaniswami ldquoStabilizing REDusing a fuzzy controllerrdquo in Proceedings of the IEEE Interna-tional Conference on Communications (ICC rsquo07) pp 266ndash271Glasgow Scotland June 2007

[12] H-L To T M Thi and W-J Hwang ldquoCascade probabilitycontrol to mitigate bufferbloat under multiple real-world TCPstacksrdquoMathematical Problems in Engineering vol 2015 ArticleID 628583 13 pages 2015

[13] M Hassan and H Sirisena ldquoOptimal control of queues in com-puter networksrdquo in Proceedings of the International Conferenceon Communications (ICC rsquo01) vol 2 pp 637ndash641 June 2001

[14] V Guffens and G Bastin ldquoOptimal adaptive feedback controlof a network bufferrdquo in Proceedings of the American ControlConference (ACC rsquo05) vol 3 pp 1835ndash1840 Portland Ore USAJune 2005

[15] J T Betts Practical Methods for Optimal Control Using Nonlin-ear Programming Society for Industrial andAppliedMathemat-ics Philadelphia Pa USA 2001

[16] R Bulirsch E Nerz H Pesch and O von Stryk ldquoCombiningdirect and indirect methods in optimal control Range maxi-mization of a hang gliderrdquo in Optimal Control R Bulirsch AMiele J Stoer and K Well Eds vol 111 of ISNM InternationalSeries of Numerical Mathematics pp 273ndash288 Birkhuser BaselSwitzerland 1993

[17] A Radwan Utilization of parametric programming and evo-lutionary computing in optimal control [PhD dissertation]Humboldt-Universitat zu Berlin 2012

10 Scientific Programming

[18] O von Stryk and R Bulirsch ldquoDirect and indirect methods fortrajectory optimizationrdquo Annals of Operations Research vol 37no 1 pp 357ndash373 1992

[19] A Bryson and Y Ho Applied Optimal Control OptimizationEstimation and Control Hemisphere CRC Press WashingtonDC USA 1975

[20] M Ab Jmodelicaorg user guide 160 2011 httpwwwjmode-licaorgpage236

[21] M Wall GAlib A C++ Library of Genetic Algorithm Compo-nents Version 24 Documentation Revision B MassachusettsInstitute of Technology 1996

[22] A Radwan O Vasilieva R Enkhbat A Griewank and J Gud-dat ldquoParametric approach to optimal controlrdquo OptimizationLetters vol 6 no 7 pp 1303ndash1316 2012

[23] J Guddat FGuerraVazquez andHT JongenParametricOpti-mization Singularities Pathfollowing and Jumps John Wiley ampSons 1990

[24] T Issariyakul and EHossain Introduction to Network SimulatorNS2 Springer Berlin Germany 1st edition 2008

[25] T Hiland-Jrgensen Flent The flexible network tester 2012ndash2015 httpsflentorg

[26] DTht ldquoRealtime response under load (rrul) testrdquo 2013 httpsgithubcomdtahtdeBloatblobmasterspecrruledoc

[27] J Koo S Ahn and J Chung ldquoA comparative study of queuedelay and loss characteristics of AQM schemes in QoS-enablednetworksrdquo Computing and Informatics vol 23 no 4 pp 317ndash335 2004

[28] T B Reddy and A Ahammed ldquoPerformance comparison ofactive queue management techniquesrdquo Journal of ComputerScience vol 4 no 12 pp 1020ndash1023 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Scientific Programming 5

Let (119909lowast 120575119906lowast) be an optimal process in problem Introducethe function 119891 119880 times 119877 rarr 119877

119891 (120575119906 119905) = minus119867 (119909

lowast 120575119906 119909

lowast 119905) 119905 isin [119905

0 119905119891] (28)

where a parametric optimization problem is defined as

min120575119906isin119880

119891 (120575119906 119905) 119905 isin [1199050 119905119891]

119880 = 120575119906 isin 119877

119903 119892119894(120575119906) le 0 119894 isin 119869

119869 = 1 2 119904

(29)

The KKT conditions for the problem state that

119863120575119906119891 (120575119906 119905) + sum

119895isin119869

120583119894119863120575119906119892119894(120575119906 119905) = 0

119892119894(120575119906 119905) 120583

119894le 0 120583

119894ge 0 119894 isin 119869

120583119894119892119894(120575119906 119905) = 0

119894 isin 119869 119905 isin [1199050 119905119891]

(30)

where

119863120575119906119891 (120575119906 119905) = (

120597119891 (120575119906 119905)

1205971205751199061

120597119891 (120575119906 119905)

120597120575119906119903

) (31)

Consider the auxiliary parametric optimization problem

min 119891 (120575119906 119905) 119905 isin [1199050 119905119891]

subject to 119892119894(120575119906) = 0 119894 isin

119869 sub 119869

(32)

Let V0 = (120575119906

0(119905) 120583

0(119905)) satisfy the KKT conditions for

problem with 119869 = 119869

0 This system can be written in the

following compact notation

119865 (120575119906 119905) = 0 119905 isin [1199050 119905119891] (33)

where V = (120575119906

0(119905) 120583

0(119905)) In order to apply Newtonrsquos method

to system we have to solve a linear system with119863120575119906119865(V(119905) 119905)

as matrix The same matrix is used to compute V(119905)

119863120575119906119865 (120575119906 (119905) 119905) V (119905) = 119863

120575119906119891 (120575119906 (119905) 119905) (34)

Therefore using the Newton method as corrector wehave

119863120575119906119865 (120575119906

119896119894minus1

119894 119905119894) (120575119906

119896119894

119894minus 120575119906

119896119894minus1

119894) = minus119865 (120575119906

119896119894minus1

119894 119905119894)

119863120575119906119865 (120575119906

119896119894minus1

119894 119905119894) (

120575119906

119896119894minus1

119894) = minusΔ

119905119865 (120575119906

119896119894minus1

119894 119905119894)

(35)

4 Proposed Algorithm FB-OCQ

The first-order indirect approach motivates us to design FB-OCQ algorithm Shortly this algorithm uses gradient and

(1) Initialization119906(119905) 120575119906(119905) = 0 119906

+(119905) = 119906(119905) forall119905 isin [119905

0 119905119891]

Jminus= infin 120574 = 0 120573 120588 isin (0 1)

Iter = 1 MaxIter TOL(2) while Iter leMaxIter do(3) procedure Forward Sweep(4) 119909

+(1199050) = 1199090J+(1199050) = 0

(5) Integrate Forward 119905 1199050rarr 119905119891

(6) if Iter gt 1 then(7) 120575119906(119905) = argmin

120575119906119867aug(119909(119905) 119906(119905) + 120575119906 119909(119905) 119905)

(8) 119906+(119905) = 119906(119905) + 120575119906(119905)

(9) end if(10)

+(119905) = 119891(119909

+(119905) 119906+(119905) 119905)

(11) J+(119905) = 119897(119909

+(119905) 119906+(119905) 119905)

(12) end procedure(13) J

+= J+(119905119891)

(14) if J+lt Jminusthen

(15) Jminus= J+ 120573 = 120573120588

(16) else(17) 120573 = 120573120588 go to Forward Sweep(18) end if(19) procedure Adjoint Sweep(20) 119909(119905

119891) = 120601

⊤

119909(119909+(119905119891)) 120574(119905

0) = 0

(21) Integrate Backward 119905 119905119891rarr 1199050

(22) 119909(119905) = 119909+(119905) 119906(119905) = 119906

+(119905)

(23) 119909(119905)

⊤= minus(120597119867(119909(119905) 119906(119905) 119909(119905) 119905)120597119909)

(24) 119906(119905) = 120597119867(119909(119905) 119906(119905) 119909(119905) 119905)120597119906(25) (119905) = 119906(119905)

2(26) 120574 = 120574(119905

119891)

(27) end procedure(28) if 120574 lt TOL then(29) stop(30) end if(31) Iter larr Iter + 1(32) end while

Algorithm 1 FB-OCQ

does forward-backward searching for the optimal controlsolution Let us choose an initial control trajectory

(1199060 119906

119873) (1205751199060 120575119906

119873)

(119906+0

= 1199060 119906

+119873= 119880119873)

119869minus= infin 120574 Iter = 1 Δ

0 TOL MaxIter 120588 isin [0 1]

FromAlgorithm 1 (FB-OCQ) it is obvious that the algorithmterminates if the norm of the gradient of the Hamiltonianwith respect to 119906 120574 = 119867

1199062 during the run time of the

program is smaller than the tolerance and the parameter 120573

must change at the inner iterations if we do not have a descentdirection so we must divide 120573 by parameter 120588 isin (0 1) untilwe get a reduction in the cost function Figure 2 explainsour algorithm steps forward sweep (control variable and costfunction value) and backward sweep (adjoint variable) indetail

6 Scientific Programming

t0 t1 tf

Forward Backwardsweepsweep u = u + 120575u

J+ = 1 lt Jminus = infinrarr Jminus = J+ = 1

rarr Jminus = J+ = 05

Decrease 120573

Increase 120573J+ = 2 gt Jminus = 1

J+ = 05 lt Jminus = 1

x(t) u(t)

Figure 2 FB-OCQ algorithm explanation

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 3 Indirect method FB-OCQ

5 Performance Evaluation

51 Numerical Results In this section we provide someiteration results for both direct and indirect approaches asfollows

Indirect Method FB-OCQ The ODE was solved on equidis-tant discretization with 1000 discretization points The opti-mal control and optimal state are depicted in Figure 3 Werealize by looking at the final control that there exist somepoints where the set of active constraints changed due to thesingularity that is the optimal control is of the bang-bangtype with the possibility of a singular arc To the meaningof such singular arc for queue management it presents thesudden changes of input rate from 60 (packetssec) to 35

(packetssec) which accordingly result in the changes ofbuffer load (or queue length) In the 120573 history in Figure 3the parameter120573 changed at the inner iterations becausewe donot have a descent direction so wemust divide 120573 by 120588 isin (0 1)

until we get a reduction in the cost functionJ = 681280316The processing time of central processing unit (CPU) forthis algorithm is 004 s and the norm of the gradient of

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 4 Direct method nonlinear optimization with JModelicasolver

the Hamiltonian function 1198671199062with respect to the control

119906 goes to approximately 10119890 minus 7

Direct CollocationMethod JModelica and GAlib Solvers With119873 = 1000 and the number of interpolation points being 3we tested the discretized problem (9)-(10) by JModelica andGAlib solvers (see [20 21]) as follows

(i) JModelica Solver The cost function is reduced to J =

68678362 and the CPU processing time is 052 sThe optimalcontrol and optimal state trajectories are obtained duringJModelica running and illustrated in Figure 4 An advantageof this approach is that we do not need to derive the adjointequations

(ii) GAlib Solver For this problem we use the number ofgenerations 119899gen = 2500 and the population size is 200 Theoptimal control and optimal state are depicted in Figure 5obtained during the run of the GAlib The cost function isreduced toJ = 706744 during the run time of the programThe CPU processing time is 1290 s Although the genetic

Scientific Programming 7

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 5 Direct method genetic algorithm with GAlib solver

Table 1 Numerical result summary

Criteriamethods DirectGAlib

DirectJModelica

IndirectFB-OCQ

Cost function value J = 7067 J = 6867 J = 6812

CPU process time 129 (sec) 052 (sec) 004 (sec)

algorithm has an advantage that it needs no derivatives orHessianrsquos information the control functions produced by itare useless and the convergence is very slow in comparison toAlgorithm 1 (FB-OCQ) and the one using JModelica solver

Table 1 summarizes and compares the three methods thatwe develop numerically We conclude that Algorithm 1 ismuch faster than the other solvers and gives the best costfunction By using genetic library GAlib the cost functiondoes not reach the local solution as it claims in finding theglobal solution The limitation of the indirect methods (FB-OCQ) is that one should derive the adjoint equations whichare not easy to derive in some applications

52 Simulation Results In this section the performance ofthe obtained algorithm (FB-OCQ) is evaluated by comparingit with some popular AQMs including RED REM and PIThe credibility of results is confirmed using ns-2 simulator[24] We investigate a network topology with 40 sourcesan intermediate router and one destination (Figure 6)All sources simultaneously send packets to the destinationthrough the router Hence a large queue is built up at therouter or bottleneck pointMaximum buffer size of the routeris 100 packets All of the compared RED REM and PIalgorithms are configured to obtain the desired queue lengthat 40 packets Simulation lasts for 60 seconds and is repeatedusing a built-in random generator in ns-2 to obtain morecredible results

DestinationRouter

Source 1

Source 2

Source 40

Figure 6 Simulation topology

FB-OCQRED

000

001

002

003

004

005

Dro

ppin

g pr

obab

ility

10 20 30 40 50 600Time (sec)

Figure 7 Dropping probability

Figure 7 shows dropping probability of FB-OCQ andRED algorithms With the proposed square-root drop func-tion FB-OCQ drops queuing packets more aggressivelythan RED although dropping frequency is nearly the sameMaximum dropping ratio is 0045 for FB-OCQ and 0039 forRED In fact when the algorithm drops more packets thequeue stability will be increased but we will have to sacrificethe system throughput performanceWe can see this trade-offin next results

Figure 8 presents average queue length values measuredat the bottleneck link from router to destination for differentalgorithms Due to aggressive dropping FB-OCQ maintainsthe shortest queue at 40 packets which is also the desiredvalue Only REM can obtain the same value but in alonger time asymp50 seconds PI in fact can achieve the sameperformance only if its parameters are well configured andbe dynamically changed to different network scenarios InFigure 8 we can see that PI performs not so well even weset the desired queue length variable at 40 packets and exploitdefault parameters in ns-2 for PI controller

Finally we investigate throughput performance of pro-posed FB-OCQ algorithm It is confirmed from Figure 9that there exists a trade-off in the relationship between

8 Scientific Programming

FB-OCQRED

REMPI

0

10

20

30

40

50

60

70

Aver

age q

ueue

leng

th (p

acke

ts)

10 20 30 40 50 600Time (sec)

Figure 8 Average queue length versus time

FB-OCQ RED REM PI000

001

002

003

004

005

006

007

008

009

Thro

ughp

ut (M

bps)

Figure 9 Throughput comparison

throughput and queue lengthdropping probability PI andREM achieve nearly the same throughput at 006 (Mbps)while FB-OCQandREDobtain the better throughput perfor-mance at 008 (Mbps) value In Figure 8 although REM canachieve the desired queue length 40 packets in this scenarioREM still maintains a large queue during simulation timefrom 0 (sec) up to 25 (sec) That reason leads to throughputperformance of REM being lower than RED and FB-OCQOur proposed FB-OCQ drops aggressively the packets sothat its throughput value (0075904Mbps) is slightly less thanRED (0082146Mbps)

53 Experiment Results We examine the effectiveness ofour state-of-the-art optimal control queue method in theLinux kernel (eg Ubuntu 1604) We conduct experimentfrom a desktop computer through the Internet gateway toan outside server (Figure 10) To manage working queuingdisciplines (qdiscs) we use the scheduler qdisc in the Linuxkernel The chosen server is a dedicated bufferbloat server

Bufferbloat server

DropTail

REDFB-OCQ

Experiment

Figure 10 Experiment test setup from the test client to the testserver

which is able to stand very high congestion due to manydata flows at the same time We exploit the Flent FLExibleNetwork Tester [25] and Realtime Response Under Load(RRUL) test [26] to evaluate our proposal RRUL test puts anetwork under worst case conditions reliably saturates thenetwork link and thus recreates bufferbloat phenomenonfor queue algorithm testing Simulation time duration is 60

secondsWe compare latency under load test with queue being

handled by different AQM schemes (DropTail RED and FB-OCQ) in turnDropTail queuingmethod is by far the simplestapproach to network router queue management The routeraccepts and forwards all the packets that arrive as long as itsbuffer space is available for the next incoming packets If apacket arrives and the queue is currently full the incomingpacket will be dropped The sender then detects the packetlost event and shrinks its sendingwindowWhile it is themostwidely used due to simplicity and relatively high efficiencyDropTail has some weakness such as the bad fairness sharingamongTCP connections and throughput and bottleneck linkefficiency suffer severe degradation if congestion is gettingworse

RED [5 6] was presented with the objective of min-imizing packet loss and queuing delay Moreover it cancompensate the weakness of DropTail by avoiding globalsynchronization of TCP sources so that it improves fair-ness To achieve these goals RED utilizes two thresholdsminth and maxth and an exponentially weighted movingaverage (EWMA) formula to estimate average queue length[27] When the average queue length exceeds a predefinedthreshold the link is implied to be in congested state anddrop action is taken A temporary increase in the queuelength notifies the transient congestion while an increasein the average queue length reflects long-lived congestionBased on such information RED router sends randomizedfeedback signals to the senders tomake decision of decreasingtheir congestion windows RED has good fairness amongconnections because of the feedback randomizedmechanism[28]

Figure 11 presents latency ping results under RRUL testsuite Ping is a networking utility and operates by sendingInternet Control Message Protocol (ICMP) echo requestpackets to the target server and waits for an ICMP echoreply The program measures the round-trip time fromtransmission to reception reporting errors and packet lossOur proposed algorithm FB-OCQ achieves the lowest packetlatency compared with the other two algorithms inside Linux

Scientific Programming 9

DropTailREDFB-OCQ

0 10 20

Simulation time (s)

30 40 50 60

101

102

103

Pack

et la

tenc

y (m

s)

Figure 11 ICMP ping result using Realtime Response Under Loadtest

kernel Specifically the packet latency when using FB-OCQis about 80 (ms) while about 120 (ms) if using DropTail(pfifo fast in Ubuntu) and 100ms if using RED algorithm

6 Conclusions

We proposed a queue management algorithm namedforward-backward optimal control queue (FB-OCQ) tosolve the OCQ problem Derived from indirect approachesin dynamic optimization this algorithm demonstrates fasterreaction while still achieving the same performance innumerical analysis compared to direct methods Employingunder network simulation ns-2 we see that the proposedalgorithm drops packets more aggressively than thetraditional RED algorithm in a higher frequency andmagnitude As a result average queue length can be reducedmuch more while an acceptable value of throughput stillcan be maintained In future works we try to investigatethe memory efficiency of FB-OCQ under wireless sensornetworks

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work was supported by the National Research Founda-tion of Korea (NRF) grant funded by the Korean government(MEST) (no NRF-2016R1A2B1013733)

References

[1] T Hoiland-Jorgensen ldquoBattling bufferbloat an experimentalcomparison of four apapproach to queuemanagement in linuxrdquoTech Rep Roskilde University Roskilde Denmark 2012

[2] J Gettys ldquoBufferbloat dark buffers in the Internetrdquo IEEEInternet Computing vol 15 no 3 pp 95ndash96 2011

[3] J Gettys ldquoBufferbloat views of the elephantrdquo in Proceedingsof the International Summit for Community Wireless NetworksOctober 2012

[4] B Braden D Clark J Crowcroft et al ldquoRecommendations onqueue management and congestion avoidance in the internetrdquoRFC 2309 IETF San Francisco Calif USA 1998

[5] C Hollot V Misra D Towsley and W-B Gong ldquoA controltheoretic analysis of redrdquo in Proceedings of the 20th Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo01) vol 3 pp 1510ndash1519 IEEE Anchorage AlaskaUSA 2001

[6] S Floyd and V Jacobson ldquoRandom early detection gateways forcongestion avoidancerdquo IEEEACM Transactions on Networkingvol 1 no 4 pp 397ndash413 1993

[7] S Athuraliya S Low and D Lapsley ldquoRandom early markingrdquoin Quality of Future Internet Services J Crowcroft J Robertsand M I Smirnov Eds vol 1922 of Lecture Notes in ComputerScience pp 43ndash54 Springer Berlin Germany 2000

[8] R Pan PNatarajan C Piglione et al ldquoPIE a lightweight controlscheme to address the bufferbloat problemrdquo in Proceedings ofthe IEEE 14th International Conference on High PerformanceSwitching and Routing (HPSR rsquo13) pp 148ndash155 Taipei TaiwanJuly 2013

[9] F Ren C Lin and BWei ldquoDesign a robust controller for activequeue management in large delay networksrdquo in Proceedings ofthe 9th International Symposium on Computers and Communi-cations (ISCC rsquo04) vol 2 pp 748ndash754 June-July 2004

[10] R Zhu H Teng and J Fu ldquoA predictive PID controller forAQM router supporting TCP with ECNrdquo in Proceedings ofthe 5th International Symposium on Multi-Dimensional MobileCommunications The 2004 Joint Conference of the 10th Asia-Pacific Conference on Communications vol 1 pp 356ndash360August-September 2004

[11] J Sun M Zukerman and M Palaniswami ldquoStabilizing REDusing a fuzzy controllerrdquo in Proceedings of the IEEE Interna-tional Conference on Communications (ICC rsquo07) pp 266ndash271Glasgow Scotland June 2007

[12] H-L To T M Thi and W-J Hwang ldquoCascade probabilitycontrol to mitigate bufferbloat under multiple real-world TCPstacksrdquoMathematical Problems in Engineering vol 2015 ArticleID 628583 13 pages 2015

[13] M Hassan and H Sirisena ldquoOptimal control of queues in com-puter networksrdquo in Proceedings of the International Conferenceon Communications (ICC rsquo01) vol 2 pp 637ndash641 June 2001

[14] V Guffens and G Bastin ldquoOptimal adaptive feedback controlof a network bufferrdquo in Proceedings of the American ControlConference (ACC rsquo05) vol 3 pp 1835ndash1840 Portland Ore USAJune 2005

[15] J T Betts Practical Methods for Optimal Control Using Nonlin-ear Programming Society for Industrial andAppliedMathemat-ics Philadelphia Pa USA 2001

[16] R Bulirsch E Nerz H Pesch and O von Stryk ldquoCombiningdirect and indirect methods in optimal control Range maxi-mization of a hang gliderrdquo in Optimal Control R Bulirsch AMiele J Stoer and K Well Eds vol 111 of ISNM InternationalSeries of Numerical Mathematics pp 273ndash288 Birkhuser BaselSwitzerland 1993

[17] A Radwan Utilization of parametric programming and evo-lutionary computing in optimal control [PhD dissertation]Humboldt-Universitat zu Berlin 2012

10 Scientific Programming

[18] O von Stryk and R Bulirsch ldquoDirect and indirect methods fortrajectory optimizationrdquo Annals of Operations Research vol 37no 1 pp 357ndash373 1992

[19] A Bryson and Y Ho Applied Optimal Control OptimizationEstimation and Control Hemisphere CRC Press WashingtonDC USA 1975

[20] M Ab Jmodelicaorg user guide 160 2011 httpwwwjmode-licaorgpage236

[21] M Wall GAlib A C++ Library of Genetic Algorithm Compo-nents Version 24 Documentation Revision B MassachusettsInstitute of Technology 1996

[22] A Radwan O Vasilieva R Enkhbat A Griewank and J Gud-dat ldquoParametric approach to optimal controlrdquo OptimizationLetters vol 6 no 7 pp 1303ndash1316 2012

[23] J Guddat FGuerraVazquez andHT JongenParametricOpti-mization Singularities Pathfollowing and Jumps John Wiley ampSons 1990

[24] T Issariyakul and EHossain Introduction to Network SimulatorNS2 Springer Berlin Germany 1st edition 2008

[25] T Hiland-Jrgensen Flent The flexible network tester 2012ndash2015 httpsflentorg

[26] DTht ldquoRealtime response under load (rrul) testrdquo 2013 httpsgithubcomdtahtdeBloatblobmasterspecrruledoc

[27] J Koo S Ahn and J Chung ldquoA comparative study of queuedelay and loss characteristics of AQM schemes in QoS-enablednetworksrdquo Computing and Informatics vol 23 no 4 pp 317ndash335 2004

[28] T B Reddy and A Ahammed ldquoPerformance comparison ofactive queue management techniquesrdquo Journal of ComputerScience vol 4 no 12 pp 1020ndash1023 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

6 Scientific Programming

t0 t1 tf

Forward Backwardsweepsweep u = u + 120575u

J+ = 1 lt Jminus = infinrarr Jminus = J+ = 1

rarr Jminus = J+ = 05

Decrease 120573

Increase 120573J+ = 2 gt Jminus = 1

J+ = 05 lt Jminus = 1

x(t) u(t)

Figure 2 FB-OCQ algorithm explanation

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 3 Indirect method FB-OCQ

5 Performance Evaluation

51 Numerical Results In this section we provide someiteration results for both direct and indirect approaches asfollows

Indirect Method FB-OCQ The ODE was solved on equidis-tant discretization with 1000 discretization points The opti-mal control and optimal state are depicted in Figure 3 Werealize by looking at the final control that there exist somepoints where the set of active constraints changed due to thesingularity that is the optimal control is of the bang-bangtype with the possibility of a singular arc To the meaningof such singular arc for queue management it presents thesudden changes of input rate from 60 (packetssec) to 35

(packetssec) which accordingly result in the changes ofbuffer load (or queue length) In the 120573 history in Figure 3the parameter120573 changed at the inner iterations becausewe donot have a descent direction so wemust divide 120573 by 120588 isin (0 1)

until we get a reduction in the cost functionJ = 681280316The processing time of central processing unit (CPU) forthis algorithm is 004 s and the norm of the gradient of

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 4 Direct method nonlinear optimization with JModelicasolver

the Hamiltonian function 1198671199062with respect to the control

119906 goes to approximately 10119890 minus 7

Direct CollocationMethod JModelica and GAlib Solvers With119873 = 1000 and the number of interpolation points being 3we tested the discretized problem (9)-(10) by JModelica andGAlib solvers (see [20 21]) as follows

(i) JModelica Solver The cost function is reduced to J =

68678362 and the CPU processing time is 052 sThe optimalcontrol and optimal state trajectories are obtained duringJModelica running and illustrated in Figure 4 An advantageof this approach is that we do not need to derive the adjointequations

(ii) GAlib Solver For this problem we use the number ofgenerations 119899gen = 2500 and the population size is 200 Theoptimal control and optimal state are depicted in Figure 5obtained during the run of the GAlib The cost function isreduced toJ = 706744 during the run time of the programThe CPU processing time is 1290 s Although the genetic

Scientific Programming 7

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 5 Direct method genetic algorithm with GAlib solver

Table 1 Numerical result summary

Criteriamethods DirectGAlib

DirectJModelica

IndirectFB-OCQ

Cost function value J = 7067 J = 6867 J = 6812

CPU process time 129 (sec) 052 (sec) 004 (sec)

algorithm has an advantage that it needs no derivatives orHessianrsquos information the control functions produced by itare useless and the convergence is very slow in comparison toAlgorithm 1 (FB-OCQ) and the one using JModelica solver

Table 1 summarizes and compares the three methods thatwe develop numerically We conclude that Algorithm 1 ismuch faster than the other solvers and gives the best costfunction By using genetic library GAlib the cost functiondoes not reach the local solution as it claims in finding theglobal solution The limitation of the indirect methods (FB-OCQ) is that one should derive the adjoint equations whichare not easy to derive in some applications

52 Simulation Results In this section the performance ofthe obtained algorithm (FB-OCQ) is evaluated by comparingit with some popular AQMs including RED REM and PIThe credibility of results is confirmed using ns-2 simulator[24] We investigate a network topology with 40 sourcesan intermediate router and one destination (Figure 6)All sources simultaneously send packets to the destinationthrough the router Hence a large queue is built up at therouter or bottleneck pointMaximum buffer size of the routeris 100 packets All of the compared RED REM and PIalgorithms are configured to obtain the desired queue lengthat 40 packets Simulation lasts for 60 seconds and is repeatedusing a built-in random generator in ns-2 to obtain morecredible results

DestinationRouter

Source 1

Source 2

Source 40

Figure 6 Simulation topology

FB-OCQRED

000

001

002

003

004

005

Dro

ppin

g pr

obab

ility

10 20 30 40 50 600Time (sec)

Figure 7 Dropping probability

Figure 7 shows dropping probability of FB-OCQ andRED algorithms With the proposed square-root drop func-tion FB-OCQ drops queuing packets more aggressivelythan RED although dropping frequency is nearly the sameMaximum dropping ratio is 0045 for FB-OCQ and 0039 forRED In fact when the algorithm drops more packets thequeue stability will be increased but we will have to sacrificethe system throughput performanceWe can see this trade-offin next results

Figure 8 presents average queue length values measuredat the bottleneck link from router to destination for differentalgorithms Due to aggressive dropping FB-OCQ maintainsthe shortest queue at 40 packets which is also the desiredvalue Only REM can obtain the same value but in alonger time asymp50 seconds PI in fact can achieve the sameperformance only if its parameters are well configured andbe dynamically changed to different network scenarios InFigure 8 we can see that PI performs not so well even weset the desired queue length variable at 40 packets and exploitdefault parameters in ns-2 for PI controller

Finally we investigate throughput performance of pro-posed FB-OCQ algorithm It is confirmed from Figure 9that there exists a trade-off in the relationship between

8 Scientific Programming

FB-OCQRED

REMPI

0

10

20

30

40

50

60

70

Aver

age q

ueue

leng

th (p

acke

ts)

10 20 30 40 50 600Time (sec)

Figure 8 Average queue length versus time

FB-OCQ RED REM PI000

001

002

003

004

005

006

007

008

009

Thro

ughp

ut (M

bps)

Figure 9 Throughput comparison

throughput and queue lengthdropping probability PI andREM achieve nearly the same throughput at 006 (Mbps)while FB-OCQandREDobtain the better throughput perfor-mance at 008 (Mbps) value In Figure 8 although REM canachieve the desired queue length 40 packets in this scenarioREM still maintains a large queue during simulation timefrom 0 (sec) up to 25 (sec) That reason leads to throughputperformance of REM being lower than RED and FB-OCQOur proposed FB-OCQ drops aggressively the packets sothat its throughput value (0075904Mbps) is slightly less thanRED (0082146Mbps)

53 Experiment Results We examine the effectiveness ofour state-of-the-art optimal control queue method in theLinux kernel (eg Ubuntu 1604) We conduct experimentfrom a desktop computer through the Internet gateway toan outside server (Figure 10) To manage working queuingdisciplines (qdiscs) we use the scheduler qdisc in the Linuxkernel The chosen server is a dedicated bufferbloat server

Bufferbloat server

DropTail

REDFB-OCQ

Experiment

Figure 10 Experiment test setup from the test client to the testserver

which is able to stand very high congestion due to manydata flows at the same time We exploit the Flent FLExibleNetwork Tester [25] and Realtime Response Under Load(RRUL) test [26] to evaluate our proposal RRUL test puts anetwork under worst case conditions reliably saturates thenetwork link and thus recreates bufferbloat phenomenonfor queue algorithm testing Simulation time duration is 60

secondsWe compare latency under load test with queue being

handled by different AQM schemes (DropTail RED and FB-OCQ) in turnDropTail queuingmethod is by far the simplestapproach to network router queue management The routeraccepts and forwards all the packets that arrive as long as itsbuffer space is available for the next incoming packets If apacket arrives and the queue is currently full the incomingpacket will be dropped The sender then detects the packetlost event and shrinks its sendingwindowWhile it is themostwidely used due to simplicity and relatively high efficiencyDropTail has some weakness such as the bad fairness sharingamongTCP connections and throughput and bottleneck linkefficiency suffer severe degradation if congestion is gettingworse

RED [5 6] was presented with the objective of min-imizing packet loss and queuing delay Moreover it cancompensate the weakness of DropTail by avoiding globalsynchronization of TCP sources so that it improves fair-ness To achieve these goals RED utilizes two thresholdsminth and maxth and an exponentially weighted movingaverage (EWMA) formula to estimate average queue length[27] When the average queue length exceeds a predefinedthreshold the link is implied to be in congested state anddrop action is taken A temporary increase in the queuelength notifies the transient congestion while an increasein the average queue length reflects long-lived congestionBased on such information RED router sends randomizedfeedback signals to the senders tomake decision of decreasingtheir congestion windows RED has good fairness amongconnections because of the feedback randomizedmechanism[28]

Figure 11 presents latency ping results under RRUL testsuite Ping is a networking utility and operates by sendingInternet Control Message Protocol (ICMP) echo requestpackets to the target server and waits for an ICMP echoreply The program measures the round-trip time fromtransmission to reception reporting errors and packet lossOur proposed algorithm FB-OCQ achieves the lowest packetlatency compared with the other two algorithms inside Linux

Scientific Programming 9

DropTailREDFB-OCQ

0 10 20

Simulation time (s)

30 40 50 60

101

102

103

Pack

et la

tenc

y (m

s)

Figure 11 ICMP ping result using Realtime Response Under Loadtest

kernel Specifically the packet latency when using FB-OCQis about 80 (ms) while about 120 (ms) if using DropTail(pfifo fast in Ubuntu) and 100ms if using RED algorithm

6 Conclusions

We proposed a queue management algorithm namedforward-backward optimal control queue (FB-OCQ) tosolve the OCQ problem Derived from indirect approachesin dynamic optimization this algorithm demonstrates fasterreaction while still achieving the same performance innumerical analysis compared to direct methods Employingunder network simulation ns-2 we see that the proposedalgorithm drops packets more aggressively than thetraditional RED algorithm in a higher frequency andmagnitude As a result average queue length can be reducedmuch more while an acceptable value of throughput stillcan be maintained In future works we try to investigatethe memory efficiency of FB-OCQ under wireless sensornetworks

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work was supported by the National Research Founda-tion of Korea (NRF) grant funded by the Korean government(MEST) (no NRF-2016R1A2B1013733)

References

[1] T Hoiland-Jorgensen ldquoBattling bufferbloat an experimentalcomparison of four apapproach to queuemanagement in linuxrdquoTech Rep Roskilde University Roskilde Denmark 2012

[2] J Gettys ldquoBufferbloat dark buffers in the Internetrdquo IEEEInternet Computing vol 15 no 3 pp 95ndash96 2011

[3] J Gettys ldquoBufferbloat views of the elephantrdquo in Proceedingsof the International Summit for Community Wireless NetworksOctober 2012

[4] B Braden D Clark J Crowcroft et al ldquoRecommendations onqueue management and congestion avoidance in the internetrdquoRFC 2309 IETF San Francisco Calif USA 1998

[5] C Hollot V Misra D Towsley and W-B Gong ldquoA controltheoretic analysis of redrdquo in Proceedings of the 20th Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo01) vol 3 pp 1510ndash1519 IEEE Anchorage AlaskaUSA 2001

[6] S Floyd and V Jacobson ldquoRandom early detection gateways forcongestion avoidancerdquo IEEEACM Transactions on Networkingvol 1 no 4 pp 397ndash413 1993

[7] S Athuraliya S Low and D Lapsley ldquoRandom early markingrdquoin Quality of Future Internet Services J Crowcroft J Robertsand M I Smirnov Eds vol 1922 of Lecture Notes in ComputerScience pp 43ndash54 Springer Berlin Germany 2000

[8] R Pan PNatarajan C Piglione et al ldquoPIE a lightweight controlscheme to address the bufferbloat problemrdquo in Proceedings ofthe IEEE 14th International Conference on High PerformanceSwitching and Routing (HPSR rsquo13) pp 148ndash155 Taipei TaiwanJuly 2013

[9] F Ren C Lin and BWei ldquoDesign a robust controller for activequeue management in large delay networksrdquo in Proceedings ofthe 9th International Symposium on Computers and Communi-cations (ISCC rsquo04) vol 2 pp 748ndash754 June-July 2004

[10] R Zhu H Teng and J Fu ldquoA predictive PID controller forAQM router supporting TCP with ECNrdquo in Proceedings ofthe 5th International Symposium on Multi-Dimensional MobileCommunications The 2004 Joint Conference of the 10th Asia-Pacific Conference on Communications vol 1 pp 356ndash360August-September 2004

[11] J Sun M Zukerman and M Palaniswami ldquoStabilizing REDusing a fuzzy controllerrdquo in Proceedings of the IEEE Interna-tional Conference on Communications (ICC rsquo07) pp 266ndash271Glasgow Scotland June 2007

[12] H-L To T M Thi and W-J Hwang ldquoCascade probabilitycontrol to mitigate bufferbloat under multiple real-world TCPstacksrdquoMathematical Problems in Engineering vol 2015 ArticleID 628583 13 pages 2015

[13] M Hassan and H Sirisena ldquoOptimal control of queues in com-puter networksrdquo in Proceedings of the International Conferenceon Communications (ICC rsquo01) vol 2 pp 637ndash641 June 2001

[14] V Guffens and G Bastin ldquoOptimal adaptive feedback controlof a network bufferrdquo in Proceedings of the American ControlConference (ACC rsquo05) vol 3 pp 1835ndash1840 Portland Ore USAJune 2005

[15] J T Betts Practical Methods for Optimal Control Using Nonlin-ear Programming Society for Industrial andAppliedMathemat-ics Philadelphia Pa USA 2001

[16] R Bulirsch E Nerz H Pesch and O von Stryk ldquoCombiningdirect and indirect methods in optimal control Range maxi-mization of a hang gliderrdquo in Optimal Control R Bulirsch AMiele J Stoer and K Well Eds vol 111 of ISNM InternationalSeries of Numerical Mathematics pp 273ndash288 Birkhuser BaselSwitzerland 1993

[17] A Radwan Utilization of parametric programming and evo-lutionary computing in optimal control [PhD dissertation]Humboldt-Universitat zu Berlin 2012

10 Scientific Programming

[18] O von Stryk and R Bulirsch ldquoDirect and indirect methods fortrajectory optimizationrdquo Annals of Operations Research vol 37no 1 pp 357ndash373 1992

[19] A Bryson and Y Ho Applied Optimal Control OptimizationEstimation and Control Hemisphere CRC Press WashingtonDC USA 1975

[20] M Ab Jmodelicaorg user guide 160 2011 httpwwwjmode-licaorgpage236

[21] M Wall GAlib A C++ Library of Genetic Algorithm Compo-nents Version 24 Documentation Revision B MassachusettsInstitute of Technology 1996

[22] A Radwan O Vasilieva R Enkhbat A Griewank and J Gud-dat ldquoParametric approach to optimal controlrdquo OptimizationLetters vol 6 no 7 pp 1303ndash1316 2012

[23] J Guddat FGuerraVazquez andHT JongenParametricOpti-mization Singularities Pathfollowing and Jumps John Wiley ampSons 1990

[24] T Issariyakul and EHossain Introduction to Network SimulatorNS2 Springer Berlin Germany 1st edition 2008

[25] T Hiland-Jrgensen Flent The flexible network tester 2012ndash2015 httpsflentorg

[26] DTht ldquoRealtime response under load (rrul) testrdquo 2013 httpsgithubcomdtahtdeBloatblobmasterspecrruledoc

[27] J Koo S Ahn and J Chung ldquoA comparative study of queuedelay and loss characteristics of AQM schemes in QoS-enablednetworksrdquo Computing and Informatics vol 23 no 4 pp 317ndash335 2004

[28] T B Reddy and A Ahammed ldquoPerformance comparison ofactive queue management techniquesrdquo Journal of ComputerScience vol 4 no 12 pp 1020ndash1023 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Scientific Programming 7

0102030405060

05 10 15 2000Time (sec)

Actual input rate u(t)

0123456

05 10 15 2000Time (sec)

Average buffer load x(t)

Figure 5 Direct method genetic algorithm with GAlib solver

Table 1 Numerical result summary

Criteriamethods DirectGAlib

DirectJModelica

IndirectFB-OCQ

Cost function value J = 7067 J = 6867 J = 6812

CPU process time 129 (sec) 052 (sec) 004 (sec)

algorithm has an advantage that it needs no derivatives orHessianrsquos information the control functions produced by itare useless and the convergence is very slow in comparison toAlgorithm 1 (FB-OCQ) and the one using JModelica solver

Table 1 summarizes and compares the three methods thatwe develop numerically We conclude that Algorithm 1 ismuch faster than the other solvers and gives the best costfunction By using genetic library GAlib the cost functiondoes not reach the local solution as it claims in finding theglobal solution The limitation of the indirect methods (FB-OCQ) is that one should derive the adjoint equations whichare not easy to derive in some applications

52 Simulation Results In this section the performance ofthe obtained algorithm (FB-OCQ) is evaluated by comparingit with some popular AQMs including RED REM and PIThe credibility of results is confirmed using ns-2 simulator[24] We investigate a network topology with 40 sourcesan intermediate router and one destination (Figure 6)All sources simultaneously send packets to the destinationthrough the router Hence a large queue is built up at therouter or bottleneck pointMaximum buffer size of the routeris 100 packets All of the compared RED REM and PIalgorithms are configured to obtain the desired queue lengthat 40 packets Simulation lasts for 60 seconds and is repeatedusing a built-in random generator in ns-2 to obtain morecredible results

DestinationRouter

Source 1

Source 2

Source 40

Figure 6 Simulation topology

FB-OCQRED

000

001

002

003

004

005

Dro

ppin

g pr

obab

ility

10 20 30 40 50 600Time (sec)

Figure 7 Dropping probability

Figure 7 shows dropping probability of FB-OCQ andRED algorithms With the proposed square-root drop func-tion FB-OCQ drops queuing packets more aggressivelythan RED although dropping frequency is nearly the sameMaximum dropping ratio is 0045 for FB-OCQ and 0039 forRED In fact when the algorithm drops more packets thequeue stability will be increased but we will have to sacrificethe system throughput performanceWe can see this trade-offin next results

Figure 8 presents average queue length values measuredat the bottleneck link from router to destination for differentalgorithms Due to aggressive dropping FB-OCQ maintainsthe shortest queue at 40 packets which is also the desiredvalue Only REM can obtain the same value but in alonger time asymp50 seconds PI in fact can achieve the sameperformance only if its parameters are well configured andbe dynamically changed to different network scenarios InFigure 8 we can see that PI performs not so well even weset the desired queue length variable at 40 packets and exploitdefault parameters in ns-2 for PI controller

Finally we investigate throughput performance of pro-posed FB-OCQ algorithm It is confirmed from Figure 9that there exists a trade-off in the relationship between

8 Scientific Programming

FB-OCQRED

REMPI

0

10

20

30

40

50

60

70

Aver

age q

ueue

leng

th (p

acke

ts)

10 20 30 40 50 600Time (sec)

Figure 8 Average queue length versus time

FB-OCQ RED REM PI000

001

002

003

004

005

006

007

008

009

Thro

ughp

ut (M

bps)

Figure 9 Throughput comparison

throughput and queue lengthdropping probability PI andREM achieve nearly the same throughput at 006 (Mbps)while FB-OCQandREDobtain the better throughput perfor-mance at 008 (Mbps) value In Figure 8 although REM canachieve the desired queue length 40 packets in this scenarioREM still maintains a large queue during simulation timefrom 0 (sec) up to 25 (sec) That reason leads to throughputperformance of REM being lower than RED and FB-OCQOur proposed FB-OCQ drops aggressively the packets sothat its throughput value (0075904Mbps) is slightly less thanRED (0082146Mbps)

53 Experiment Results We examine the effectiveness ofour state-of-the-art optimal control queue method in theLinux kernel (eg Ubuntu 1604) We conduct experimentfrom a desktop computer through the Internet gateway toan outside server (Figure 10) To manage working queuingdisciplines (qdiscs) we use the scheduler qdisc in the Linuxkernel The chosen server is a dedicated bufferbloat server

Bufferbloat server

DropTail

REDFB-OCQ

Experiment

Figure 10 Experiment test setup from the test client to the testserver

which is able to stand very high congestion due to manydata flows at the same time We exploit the Flent FLExibleNetwork Tester [25] and Realtime Response Under Load(RRUL) test [26] to evaluate our proposal RRUL test puts anetwork under worst case conditions reliably saturates thenetwork link and thus recreates bufferbloat phenomenonfor queue algorithm testing Simulation time duration is 60

secondsWe compare latency under load test with queue being

handled by different AQM schemes (DropTail RED and FB-OCQ) in turnDropTail queuingmethod is by far the simplestapproach to network router queue management The routeraccepts and forwards all the packets that arrive as long as itsbuffer space is available for the next incoming packets If apacket arrives and the queue is currently full the incomingpacket will be dropped The sender then detects the packetlost event and shrinks its sendingwindowWhile it is themostwidely used due to simplicity and relatively high efficiencyDropTail has some weakness such as the bad fairness sharingamongTCP connections and throughput and bottleneck linkefficiency suffer severe degradation if congestion is gettingworse

RED [5 6] was presented with the objective of min-imizing packet loss and queuing delay Moreover it cancompensate the weakness of DropTail by avoiding globalsynchronization of TCP sources so that it improves fair-ness To achieve these goals RED utilizes two thresholdsminth and maxth and an exponentially weighted movingaverage (EWMA) formula to estimate average queue length[27] When the average queue length exceeds a predefinedthreshold the link is implied to be in congested state anddrop action is taken A temporary increase in the queuelength notifies the transient congestion while an increasein the average queue length reflects long-lived congestionBased on such information RED router sends randomizedfeedback signals to the senders tomake decision of decreasingtheir congestion windows RED has good fairness amongconnections because of the feedback randomizedmechanism[28]

Figure 11 presents latency ping results under RRUL testsuite Ping is a networking utility and operates by sendingInternet Control Message Protocol (ICMP) echo requestpackets to the target server and waits for an ICMP echoreply The program measures the round-trip time fromtransmission to reception reporting errors and packet lossOur proposed algorithm FB-OCQ achieves the lowest packetlatency compared with the other two algorithms inside Linux

Scientific Programming 9

DropTailREDFB-OCQ

0 10 20

Simulation time (s)

30 40 50 60

101

102

103

Pack

et la

tenc

y (m

s)

Figure 11 ICMP ping result using Realtime Response Under Loadtest

kernel Specifically the packet latency when using FB-OCQis about 80 (ms) while about 120 (ms) if using DropTail(pfifo fast in Ubuntu) and 100ms if using RED algorithm

6 Conclusions

We proposed a queue management algorithm namedforward-backward optimal control queue (FB-OCQ) tosolve the OCQ problem Derived from indirect approachesin dynamic optimization this algorithm demonstrates fasterreaction while still achieving the same performance innumerical analysis compared to direct methods Employingunder network simulation ns-2 we see that the proposedalgorithm drops packets more aggressively than thetraditional RED algorithm in a higher frequency andmagnitude As a result average queue length can be reducedmuch more while an acceptable value of throughput stillcan be maintained In future works we try to investigatethe memory efficiency of FB-OCQ under wireless sensornetworks

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work was supported by the National Research Founda-tion of Korea (NRF) grant funded by the Korean government(MEST) (no NRF-2016R1A2B1013733)

References

[1] T Hoiland-Jorgensen ldquoBattling bufferbloat an experimentalcomparison of four apapproach to queuemanagement in linuxrdquoTech Rep Roskilde University Roskilde Denmark 2012

[2] J Gettys ldquoBufferbloat dark buffers in the Internetrdquo IEEEInternet Computing vol 15 no 3 pp 95ndash96 2011

[3] J Gettys ldquoBufferbloat views of the elephantrdquo in Proceedingsof the International Summit for Community Wireless NetworksOctober 2012

[4] B Braden D Clark J Crowcroft et al ldquoRecommendations onqueue management and congestion avoidance in the internetrdquoRFC 2309 IETF San Francisco Calif USA 1998

[5] C Hollot V Misra D Towsley and W-B Gong ldquoA controltheoretic analysis of redrdquo in Proceedings of the 20th Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo01) vol 3 pp 1510ndash1519 IEEE Anchorage AlaskaUSA 2001

[6] S Floyd and V Jacobson ldquoRandom early detection gateways forcongestion avoidancerdquo IEEEACM Transactions on Networkingvol 1 no 4 pp 397ndash413 1993

[7] S Athuraliya S Low and D Lapsley ldquoRandom early markingrdquoin Quality of Future Internet Services J Crowcroft J Robertsand M I Smirnov Eds vol 1922 of Lecture Notes in ComputerScience pp 43ndash54 Springer Berlin Germany 2000

[8] R Pan PNatarajan C Piglione et al ldquoPIE a lightweight controlscheme to address the bufferbloat problemrdquo in Proceedings ofthe IEEE 14th International Conference on High PerformanceSwitching and Routing (HPSR rsquo13) pp 148ndash155 Taipei TaiwanJuly 2013

[9] F Ren C Lin and BWei ldquoDesign a robust controller for activequeue management in large delay networksrdquo in Proceedings ofthe 9th International Symposium on Computers and Communi-cations (ISCC rsquo04) vol 2 pp 748ndash754 June-July 2004

[10] R Zhu H Teng and J Fu ldquoA predictive PID controller forAQM router supporting TCP with ECNrdquo in Proceedings ofthe 5th International Symposium on Multi-Dimensional MobileCommunications The 2004 Joint Conference of the 10th Asia-Pacific Conference on Communications vol 1 pp 356ndash360August-September 2004

[11] J Sun M Zukerman and M Palaniswami ldquoStabilizing REDusing a fuzzy controllerrdquo in Proceedings of the IEEE Interna-tional Conference on Communications (ICC rsquo07) pp 266ndash271Glasgow Scotland June 2007

[12] H-L To T M Thi and W-J Hwang ldquoCascade probabilitycontrol to mitigate bufferbloat under multiple real-world TCPstacksrdquoMathematical Problems in Engineering vol 2015 ArticleID 628583 13 pages 2015

[13] M Hassan and H Sirisena ldquoOptimal control of queues in com-puter networksrdquo in Proceedings of the International Conferenceon Communications (ICC rsquo01) vol 2 pp 637ndash641 June 2001

[14] V Guffens and G Bastin ldquoOptimal adaptive feedback controlof a network bufferrdquo in Proceedings of the American ControlConference (ACC rsquo05) vol 3 pp 1835ndash1840 Portland Ore USAJune 2005

[15] J T Betts Practical Methods for Optimal Control Using Nonlin-ear Programming Society for Industrial andAppliedMathemat-ics Philadelphia Pa USA 2001

[16] R Bulirsch E Nerz H Pesch and O von Stryk ldquoCombiningdirect and indirect methods in optimal control Range maxi-mization of a hang gliderrdquo in Optimal Control R Bulirsch AMiele J Stoer and K Well Eds vol 111 of ISNM InternationalSeries of Numerical Mathematics pp 273ndash288 Birkhuser BaselSwitzerland 1993

[17] A Radwan Utilization of parametric programming and evo-lutionary computing in optimal control [PhD dissertation]Humboldt-Universitat zu Berlin 2012

10 Scientific Programming

[18] O von Stryk and R Bulirsch ldquoDirect and indirect methods fortrajectory optimizationrdquo Annals of Operations Research vol 37no 1 pp 357ndash373 1992

[19] A Bryson and Y Ho Applied Optimal Control OptimizationEstimation and Control Hemisphere CRC Press WashingtonDC USA 1975

[20] M Ab Jmodelicaorg user guide 160 2011 httpwwwjmode-licaorgpage236

[21] M Wall GAlib A C++ Library of Genetic Algorithm Compo-nents Version 24 Documentation Revision B MassachusettsInstitute of Technology 1996

[22] A Radwan O Vasilieva R Enkhbat A Griewank and J Gud-dat ldquoParametric approach to optimal controlrdquo OptimizationLetters vol 6 no 7 pp 1303ndash1316 2012

[23] J Guddat FGuerraVazquez andHT JongenParametricOpti-mization Singularities Pathfollowing and Jumps John Wiley ampSons 1990

[24] T Issariyakul and EHossain Introduction to Network SimulatorNS2 Springer Berlin Germany 1st edition 2008

[25] T Hiland-Jrgensen Flent The flexible network tester 2012ndash2015 httpsflentorg

[26] DTht ldquoRealtime response under load (rrul) testrdquo 2013 httpsgithubcomdtahtdeBloatblobmasterspecrruledoc

[27] J Koo S Ahn and J Chung ldquoA comparative study of queuedelay and loss characteristics of AQM schemes in QoS-enablednetworksrdquo Computing and Informatics vol 23 no 4 pp 317ndash335 2004

[28] T B Reddy and A Ahammed ldquoPerformance comparison ofactive queue management techniquesrdquo Journal of ComputerScience vol 4 no 12 pp 1020ndash1023 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

8 Scientific Programming

FB-OCQRED

REMPI

0

10

20

30

40

50

60

70

Aver

age q

ueue

leng

th (p

acke

ts)

10 20 30 40 50 600Time (sec)

Figure 8 Average queue length versus time

FB-OCQ RED REM PI000

001

002

003

004

005

006

007

008

009

Thro

ughp

ut (M

bps)

Figure 9 Throughput comparison

throughput and queue lengthdropping probability PI andREM achieve nearly the same throughput at 006 (Mbps)while FB-OCQandREDobtain the better throughput perfor-mance at 008 (Mbps) value In Figure 8 although REM canachieve the desired queue length 40 packets in this scenarioREM still maintains a large queue during simulation timefrom 0 (sec) up to 25 (sec) That reason leads to throughputperformance of REM being lower than RED and FB-OCQOur proposed FB-OCQ drops aggressively the packets sothat its throughput value (0075904Mbps) is slightly less thanRED (0082146Mbps)

53 Experiment Results We examine the effectiveness ofour state-of-the-art optimal control queue method in theLinux kernel (eg Ubuntu 1604) We conduct experimentfrom a desktop computer through the Internet gateway toan outside server (Figure 10) To manage working queuingdisciplines (qdiscs) we use the scheduler qdisc in the Linuxkernel The chosen server is a dedicated bufferbloat server

Bufferbloat server

DropTail

REDFB-OCQ

Experiment

Figure 10 Experiment test setup from the test client to the testserver

which is able to stand very high congestion due to manydata flows at the same time We exploit the Flent FLExibleNetwork Tester [25] and Realtime Response Under Load(RRUL) test [26] to evaluate our proposal RRUL test puts anetwork under worst case conditions reliably saturates thenetwork link and thus recreates bufferbloat phenomenonfor queue algorithm testing Simulation time duration is 60

secondsWe compare latency under load test with queue being

handled by different AQM schemes (DropTail RED and FB-OCQ) in turnDropTail queuingmethod is by far the simplestapproach to network router queue management The routeraccepts and forwards all the packets that arrive as long as itsbuffer space is available for the next incoming packets If apacket arrives and the queue is currently full the incomingpacket will be dropped The sender then detects the packetlost event and shrinks its sendingwindowWhile it is themostwidely used due to simplicity and relatively high efficiencyDropTail has some weakness such as the bad fairness sharingamongTCP connections and throughput and bottleneck linkefficiency suffer severe degradation if congestion is gettingworse

RED [5 6] was presented with the objective of min-imizing packet loss and queuing delay Moreover it cancompensate the weakness of DropTail by avoiding globalsynchronization of TCP sources so that it improves fair-ness To achieve these goals RED utilizes two thresholdsminth and maxth and an exponentially weighted movingaverage (EWMA) formula to estimate average queue length[27] When the average queue length exceeds a predefinedthreshold the link is implied to be in congested state anddrop action is taken A temporary increase in the queuelength notifies the transient congestion while an increasein the average queue length reflects long-lived congestionBased on such information RED router sends randomizedfeedback signals to the senders tomake decision of decreasingtheir congestion windows RED has good fairness amongconnections because of the feedback randomizedmechanism[28]

Figure 11 presents latency ping results under RRUL testsuite Ping is a networking utility and operates by sendingInternet Control Message Protocol (ICMP) echo requestpackets to the target server and waits for an ICMP echoreply The program measures the round-trip time fromtransmission to reception reporting errors and packet lossOur proposed algorithm FB-OCQ achieves the lowest packetlatency compared with the other two algorithms inside Linux

Scientific Programming 9

DropTailREDFB-OCQ

0 10 20

Simulation time (s)

30 40 50 60

101

102

103

Pack

et la

tenc

y (m

s)

Figure 11 ICMP ping result using Realtime Response Under Loadtest

kernel Specifically the packet latency when using FB-OCQis about 80 (ms) while about 120 (ms) if using DropTail(pfifo fast in Ubuntu) and 100ms if using RED algorithm

6 Conclusions

We proposed a queue management algorithm namedforward-backward optimal control queue (FB-OCQ) tosolve the OCQ problem Derived from indirect approachesin dynamic optimization this algorithm demonstrates fasterreaction while still achieving the same performance innumerical analysis compared to direct methods Employingunder network simulation ns-2 we see that the proposedalgorithm drops packets more aggressively than thetraditional RED algorithm in a higher frequency andmagnitude As a result average queue length can be reducedmuch more while an acceptable value of throughput stillcan be maintained In future works we try to investigatethe memory efficiency of FB-OCQ under wireless sensornetworks

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work was supported by the National Research Founda-tion of Korea (NRF) grant funded by the Korean government(MEST) (no NRF-2016R1A2B1013733)

References

[1] T Hoiland-Jorgensen ldquoBattling bufferbloat an experimentalcomparison of four apapproach to queuemanagement in linuxrdquoTech Rep Roskilde University Roskilde Denmark 2012

[2] J Gettys ldquoBufferbloat dark buffers in the Internetrdquo IEEEInternet Computing vol 15 no 3 pp 95ndash96 2011

[3] J Gettys ldquoBufferbloat views of the elephantrdquo in Proceedingsof the International Summit for Community Wireless NetworksOctober 2012

[4] B Braden D Clark J Crowcroft et al ldquoRecommendations onqueue management and congestion avoidance in the internetrdquoRFC 2309 IETF San Francisco Calif USA 1998

[5] C Hollot V Misra D Towsley and W-B Gong ldquoA controltheoretic analysis of redrdquo in Proceedings of the 20th Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo01) vol 3 pp 1510ndash1519 IEEE Anchorage AlaskaUSA 2001

[6] S Floyd and V Jacobson ldquoRandom early detection gateways forcongestion avoidancerdquo IEEEACM Transactions on Networkingvol 1 no 4 pp 397ndash413 1993

[7] S Athuraliya S Low and D Lapsley ldquoRandom early markingrdquoin Quality of Future Internet Services J Crowcroft J Robertsand M I Smirnov Eds vol 1922 of Lecture Notes in ComputerScience pp 43ndash54 Springer Berlin Germany 2000

[8] R Pan PNatarajan C Piglione et al ldquoPIE a lightweight controlscheme to address the bufferbloat problemrdquo in Proceedings ofthe IEEE 14th International Conference on High PerformanceSwitching and Routing (HPSR rsquo13) pp 148ndash155 Taipei TaiwanJuly 2013

[9] F Ren C Lin and BWei ldquoDesign a robust controller for activequeue management in large delay networksrdquo in Proceedings ofthe 9th International Symposium on Computers and Communi-cations (ISCC rsquo04) vol 2 pp 748ndash754 June-July 2004

[10] R Zhu H Teng and J Fu ldquoA predictive PID controller forAQM router supporting TCP with ECNrdquo in Proceedings ofthe 5th International Symposium on Multi-Dimensional MobileCommunications The 2004 Joint Conference of the 10th Asia-Pacific Conference on Communications vol 1 pp 356ndash360August-September 2004

[11] J Sun M Zukerman and M Palaniswami ldquoStabilizing REDusing a fuzzy controllerrdquo in Proceedings of the IEEE Interna-tional Conference on Communications (ICC rsquo07) pp 266ndash271Glasgow Scotland June 2007

[12] H-L To T M Thi and W-J Hwang ldquoCascade probabilitycontrol to mitigate bufferbloat under multiple real-world TCPstacksrdquoMathematical Problems in Engineering vol 2015 ArticleID 628583 13 pages 2015

[13] M Hassan and H Sirisena ldquoOptimal control of queues in com-puter networksrdquo in Proceedings of the International Conferenceon Communications (ICC rsquo01) vol 2 pp 637ndash641 June 2001

[14] V Guffens and G Bastin ldquoOptimal adaptive feedback controlof a network bufferrdquo in Proceedings of the American ControlConference (ACC rsquo05) vol 3 pp 1835ndash1840 Portland Ore USAJune 2005

[15] J T Betts Practical Methods for Optimal Control Using Nonlin-ear Programming Society for Industrial andAppliedMathemat-ics Philadelphia Pa USA 2001

[16] R Bulirsch E Nerz H Pesch and O von Stryk ldquoCombiningdirect and indirect methods in optimal control Range maxi-mization of a hang gliderrdquo in Optimal Control R Bulirsch AMiele J Stoer and K Well Eds vol 111 of ISNM InternationalSeries of Numerical Mathematics pp 273ndash288 Birkhuser BaselSwitzerland 1993

[17] A Radwan Utilization of parametric programming and evo-lutionary computing in optimal control [PhD dissertation]Humboldt-Universitat zu Berlin 2012

10 Scientific Programming

[18] O von Stryk and R Bulirsch ldquoDirect and indirect methods fortrajectory optimizationrdquo Annals of Operations Research vol 37no 1 pp 357ndash373 1992

[19] A Bryson and Y Ho Applied Optimal Control OptimizationEstimation and Control Hemisphere CRC Press WashingtonDC USA 1975

[20] M Ab Jmodelicaorg user guide 160 2011 httpwwwjmode-licaorgpage236

[21] M Wall GAlib A C++ Library of Genetic Algorithm Compo-nents Version 24 Documentation Revision B MassachusettsInstitute of Technology 1996

[22] A Radwan O Vasilieva R Enkhbat A Griewank and J Gud-dat ldquoParametric approach to optimal controlrdquo OptimizationLetters vol 6 no 7 pp 1303ndash1316 2012

[23] J Guddat FGuerraVazquez andHT JongenParametricOpti-mization Singularities Pathfollowing and Jumps John Wiley ampSons 1990

[24] T Issariyakul and EHossain Introduction to Network SimulatorNS2 Springer Berlin Germany 1st edition 2008

[25] T Hiland-Jrgensen Flent The flexible network tester 2012ndash2015 httpsflentorg

[26] DTht ldquoRealtime response under load (rrul) testrdquo 2013 httpsgithubcomdtahtdeBloatblobmasterspecrruledoc

[27] J Koo S Ahn and J Chung ldquoA comparative study of queuedelay and loss characteristics of AQM schemes in QoS-enablednetworksrdquo Computing and Informatics vol 23 no 4 pp 317ndash335 2004

[28] T B Reddy and A Ahammed ldquoPerformance comparison ofactive queue management techniquesrdquo Journal of ComputerScience vol 4 no 12 pp 1020ndash1023 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Scientific Programming 9

DropTailREDFB-OCQ

0 10 20

Simulation time (s)

30 40 50 60

101

102

103

Pack

et la

tenc

y (m

s)

Figure 11 ICMP ping result using Realtime Response Under Loadtest

kernel Specifically the packet latency when using FB-OCQis about 80 (ms) while about 120 (ms) if using DropTail(pfifo fast in Ubuntu) and 100ms if using RED algorithm

6 Conclusions

We proposed a queue management algorithm namedforward-backward optimal control queue (FB-OCQ) tosolve the OCQ problem Derived from indirect approachesin dynamic optimization this algorithm demonstrates fasterreaction while still achieving the same performance innumerical analysis compared to direct methods Employingunder network simulation ns-2 we see that the proposedalgorithm drops packets more aggressively than thetraditional RED algorithm in a higher frequency andmagnitude As a result average queue length can be reducedmuch more while an acceptable value of throughput stillcan be maintained In future works we try to investigatethe memory efficiency of FB-OCQ under wireless sensornetworks

Competing Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgments

This work was supported by the National Research Founda-tion of Korea (NRF) grant funded by the Korean government(MEST) (no NRF-2016R1A2B1013733)

References

[1] T Hoiland-Jorgensen ldquoBattling bufferbloat an experimentalcomparison of four apapproach to queuemanagement in linuxrdquoTech Rep Roskilde University Roskilde Denmark 2012

[2] J Gettys ldquoBufferbloat dark buffers in the Internetrdquo IEEEInternet Computing vol 15 no 3 pp 95ndash96 2011

[3] J Gettys ldquoBufferbloat views of the elephantrdquo in Proceedingsof the International Summit for Community Wireless NetworksOctober 2012

[4] B Braden D Clark J Crowcroft et al ldquoRecommendations onqueue management and congestion avoidance in the internetrdquoRFC 2309 IETF San Francisco Calif USA 1998

[5] C Hollot V Misra D Towsley and W-B Gong ldquoA controltheoretic analysis of redrdquo in Proceedings of the 20th Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo01) vol 3 pp 1510ndash1519 IEEE Anchorage AlaskaUSA 2001

[6] S Floyd and V Jacobson ldquoRandom early detection gateways forcongestion avoidancerdquo IEEEACM Transactions on Networkingvol 1 no 4 pp 397ndash413 1993

[7] S Athuraliya S Low and D Lapsley ldquoRandom early markingrdquoin Quality of Future Internet Services J Crowcroft J Robertsand M I Smirnov Eds vol 1922 of Lecture Notes in ComputerScience pp 43ndash54 Springer Berlin Germany 2000

[8] R Pan PNatarajan C Piglione et al ldquoPIE a lightweight controlscheme to address the bufferbloat problemrdquo in Proceedings ofthe IEEE 14th International Conference on High PerformanceSwitching and Routing (HPSR rsquo13) pp 148ndash155 Taipei TaiwanJuly 2013

[9] F Ren C Lin and BWei ldquoDesign a robust controller for activequeue management in large delay networksrdquo in Proceedings ofthe 9th International Symposium on Computers and Communi-cations (ISCC rsquo04) vol 2 pp 748ndash754 June-July 2004

[10] R Zhu H Teng and J Fu ldquoA predictive PID controller forAQM router supporting TCP with ECNrdquo in Proceedings ofthe 5th International Symposium on Multi-Dimensional MobileCommunications The 2004 Joint Conference of the 10th Asia-Pacific Conference on Communications vol 1 pp 356ndash360August-September 2004

[11] J Sun M Zukerman and M Palaniswami ldquoStabilizing REDusing a fuzzy controllerrdquo in Proceedings of the IEEE Interna-tional Conference on Communications (ICC rsquo07) pp 266ndash271Glasgow Scotland June 2007

[12] H-L To T M Thi and W-J Hwang ldquoCascade probabilitycontrol to mitigate bufferbloat under multiple real-world TCPstacksrdquoMathematical Problems in Engineering vol 2015 ArticleID 628583 13 pages 2015

[13] M Hassan and H Sirisena ldquoOptimal control of queues in com-puter networksrdquo in Proceedings of the International Conferenceon Communications (ICC rsquo01) vol 2 pp 637ndash641 June 2001

[14] V Guffens and G Bastin ldquoOptimal adaptive feedback controlof a network bufferrdquo in Proceedings of the American ControlConference (ACC rsquo05) vol 3 pp 1835ndash1840 Portland Ore USAJune 2005

[15] J T Betts Practical Methods for Optimal Control Using Nonlin-ear Programming Society for Industrial andAppliedMathemat-ics Philadelphia Pa USA 2001

[16] R Bulirsch E Nerz H Pesch and O von Stryk ldquoCombiningdirect and indirect methods in optimal control Range maxi-mization of a hang gliderrdquo in Optimal Control R Bulirsch AMiele J Stoer and K Well Eds vol 111 of ISNM InternationalSeries of Numerical Mathematics pp 273ndash288 Birkhuser BaselSwitzerland 1993

[17] A Radwan Utilization of parametric programming and evo-lutionary computing in optimal control [PhD dissertation]Humboldt-Universitat zu Berlin 2012

10 Scientific Programming

[18] O von Stryk and R Bulirsch ldquoDirect and indirect methods fortrajectory optimizationrdquo Annals of Operations Research vol 37no 1 pp 357ndash373 1992

[19] A Bryson and Y Ho Applied Optimal Control OptimizationEstimation and Control Hemisphere CRC Press WashingtonDC USA 1975

[20] M Ab Jmodelicaorg user guide 160 2011 httpwwwjmode-licaorgpage236

[21] M Wall GAlib A C++ Library of Genetic Algorithm Compo-nents Version 24 Documentation Revision B MassachusettsInstitute of Technology 1996

[22] A Radwan O Vasilieva R Enkhbat A Griewank and J Gud-dat ldquoParametric approach to optimal controlrdquo OptimizationLetters vol 6 no 7 pp 1303ndash1316 2012

[23] J Guddat FGuerraVazquez andHT JongenParametricOpti-mization Singularities Pathfollowing and Jumps John Wiley ampSons 1990

[24] T Issariyakul and EHossain Introduction to Network SimulatorNS2 Springer Berlin Germany 1st edition 2008

[25] T Hiland-Jrgensen Flent The flexible network tester 2012ndash2015 httpsflentorg

[26] DTht ldquoRealtime response under load (rrul) testrdquo 2013 httpsgithubcomdtahtdeBloatblobmasterspecrruledoc

[27] J Koo S Ahn and J Chung ldquoA comparative study of queuedelay and loss characteristics of AQM schemes in QoS-enablednetworksrdquo Computing and Informatics vol 23 no 4 pp 317ndash335 2004

[28] T B Reddy and A Ahammed ldquoPerformance comparison ofactive queue management techniquesrdquo Journal of ComputerScience vol 4 no 12 pp 1020ndash1023 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

10 Scientific Programming

[18] O von Stryk and R Bulirsch ldquoDirect and indirect methods fortrajectory optimizationrdquo Annals of Operations Research vol 37no 1 pp 357ndash373 1992

[19] A Bryson and Y Ho Applied Optimal Control OptimizationEstimation and Control Hemisphere CRC Press WashingtonDC USA 1975

[20] M Ab Jmodelicaorg user guide 160 2011 httpwwwjmode-licaorgpage236

[21] M Wall GAlib A C++ Library of Genetic Algorithm Compo-nents Version 24 Documentation Revision B MassachusettsInstitute of Technology 1996

[22] A Radwan O Vasilieva R Enkhbat A Griewank and J Gud-dat ldquoParametric approach to optimal controlrdquo OptimizationLetters vol 6 no 7 pp 1303ndash1316 2012

[23] J Guddat FGuerraVazquez andHT JongenParametricOpti-mization Singularities Pathfollowing and Jumps John Wiley ampSons 1990

[24] T Issariyakul and EHossain Introduction to Network SimulatorNS2 Springer Berlin Germany 1st edition 2008

[25] T Hiland-Jrgensen Flent The flexible network tester 2012ndash2015 httpsflentorg

[26] DTht ldquoRealtime response under load (rrul) testrdquo 2013 httpsgithubcomdtahtdeBloatblobmasterspecrruledoc

[27] J Koo S Ahn and J Chung ldquoA comparative study of queuedelay and loss characteristics of AQM schemes in QoS-enablednetworksrdquo Computing and Informatics vol 23 no 4 pp 317ndash335 2004

[28] T B Reddy and A Ahammed ldquoPerformance comparison ofactive queue management techniquesrdquo Journal of ComputerScience vol 4 no 12 pp 1020ndash1023 2008

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

ArtificialNeural Systems

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014