algorithms of informatics volume 3 -...
TRANSCRIPT
ALGORITHMS
OF INFORMATICS
Volume 3
AnTonComBudapest 2011
This electronic book was prepared in the framework of project Eastern HungarianInformatics Books Repository no TAacuteMOP-412-081A-2009-0046
This electronic book appeared with the support of European Union and with theco-financing of European Social Fund
Editor Antal Ivaacutenyi
Authors of Volume 3 Mira-Cristiana Anisiu (Chapter 26) Beacutela Vizvaacuteri (Chapter 24)Antal Ivaacutenyi (Chapter 25) Zoltaacuten Kaacutesa (Chapter 25) Ferenc Szidarovszky (Chapter 10)
Laacuteszloacute Szirmay-Kalos (Chapter 28)
Validators of Volume 3 Gyoumlrgy Kovaacutecs (Chapter 24) Zoltaacuten Kaacutesa (Chapter 25) AntalIvaacutenyi (Chapter 26) Anna Ivaacutenyi (Bibliography)
ccopy 2011 AnTonCom Infokommunikaacutecioacutes KftHomepage httpwwwantoncomhu
Contents
Introduction 1206
26The Branch and Bound Method 1208261 An example the Knapsack Problem 1208
2611 The Knapsack Problem 12092612 A numerical example 12112613 Properties in the calculation of the numerical example 12142614 How to accelerate the method 1216
262 The general frame of the BampB method 12172621 Relaxation 12172622 The general frame of the BampB method 1224
263 Mixed integer programming with bounded variables 12292631 The geometric analysis of a numerical example 12302632 The linear programming background of the method 12322633 Fast bounds on lower and upper branches 12402634 Branching strategies 12442635 The selection of the branching variable 12472636 The numerical example is revisited 1248
264 On the enumeration tree 1252265 The use of information obtained from other sources 1254
2651 Application of heuristic methods 12542652 Preprocessing 1255
266 Branch and Cut 1255267 Branch and Price 1260
27 Comparison Based Ranking 1262271 Introduction to supertournaments 1262272 Introduction to (a b 2)-tournaments 1264273 Existence of a (1 1 2)-tournament with prescribed score sequence 1266274 Existence of an (a a 2)-tournament with prescribed score sequence 1267275 Existence of a tournament with arbitrary degree sequence 1268
2751 Definition of a naive reconstructing algorithm 12682752 Computation of e 12682753 Definition of a construction algorithm 12692754 Computation of f and g 1270
1204 Contents
2755 Definition of a testing algorithm 12712756 Definition of an algorithm computing f and g 12722757 Computing of f and g in linear time 1274
276 Tournament with f and g 12742761 Definition of the minimax reconstruction algorithm 12752762 Definition of the score slicing algorithm 12752763 Analysis of the minimax reconstruction algorithm 1279
277 Imbalances in (0 b 2)-tournaments 12792771 Imbalances in (0 b)-graphs 1280
278 Supertournaments 12852781 Hypertournamentss 1287
279 Summary 1292
28Complexity of Words 1293281 Simple complexity measures 1293
2811 Finite words 12932812 Infinite words 12952813 Word graphs 12972814 Complexity of words 1302
282 Generalized complexity measures 13132821 Rainbow words 13132822 General words 1322
283 Palindrome complexity 13222831 Palindromes in finite words 13232832 Palindromes in infinite words 1326
29 Conflict Situations 1331291 The basics of multi-objective programming 1331
2911 Applications of utility functions 13352912 Weighting method 13372913 Distance-dependent methods 13382914 Direction-dependent methods 1341
292 Method of equilibrium 1344293 Methods of cooperative games 1348294 Collective decision-making 1352295 Applications of Pareto games 1359296 Axiomatic methods 1361
30 General Purpose Computing on Graphics Processing Units 1368301 The graphics pipeline model 1370
3011 GPU as the implementation of incremental image synthesis 1372302 GPGPU with the graphics pipeline model 1375
3021 Output 13753022 Input 13763023 Functions and parameters 1377
303 GPU as a vector processor 13783031 Implementing the SAXPY BLAS function 13803032 Image filtering 1381
Contents 1205
304 Beyond vector processing 13823041 SIMD or MIMD 13823042 Reduction 13843043 Implementing scatter 13853044 Parallelism versus reuse 1387
305 GPGPU programming model CUDA and OpenCL 1389306 Matrix-vector multiplication 1389
3061 Making matrix-vector multiplication more parallel 1391307 Case study computational fluid dynamics 1394
3071 Eulerian solver for fluid dynamics 13963072 Lagrangian solver for differential equations 1401
31 Perfect Arrays 1408311 Basic concepts 1408312 Necessary condition and earlier results 1410313 One-dimensional arrays 1411
3131 Pseudocode of the algorithm Quick-Martin 14113132 Pseudocode of the algorithm Optimal-Martin 14113133 Pseudocode of the algorithm Shift 14123134 Pseudocode of the algorithm Even 1412
314 One dimensional words with fixed length 1413315 Two-dimensional infinite arrays 1413
3151 Pseudocode of the algorithm Mesh 14133152 Pseudocode of the algorithm Cellular 1413
316 Three-dimensional infinite cubes 14143161 Pseudocode of the algorithm Colour 14143162 Pseudocode of the algorithm Growing 1415
317 Examples of constructing growing arrays using colouring 14163171 Construction of growing sequences 14163172 Construction of growing squares 14173173 Construction of growing cubes 1418
318 Proof of the main result 1419319 Multi-dimensional infinite arrays 1420
32 Score Sets and Kings 1422321 Score sets in 1-tournaments 1423
3211 Determining the score set 14233212 Tournaments with prescribed score set 1425
322 Score sets in oriented graphs 14323221 Oriented graphs with prescribed scoresets 1434
323 Unicity of score sets 14393231 1-unique score sets 14403232 2-unique score sets 1441
324 Kings and serfs in tournaments 1443325 Weak kings in oriented graphs 1451
Bibliography 1462
Subject Index 1472
1206 Contents
Name Index 1477
Introduction
The first volume of the book Informatikai algoritmusok appeared in 2004 in Hun-garian [] and the second volume of the book appeared in 2005 [] Two volumescontained 31 chapters 23 chapters of the present book and further chapters on clus-tering frequent elements in data bases geoinformatics inner-point methods numbertheory Petri-nets queuing theory scheduling
The Hungarian version of the first volume contains those chapters which werefinished until May of 2004 and the second volume contains the chapters finisheduntil April of 2005
The printed English version contains the chapters submitted until April of 2007Volume 1 [] contains the chapters belonging to the fundamentals of informaticswhile the second volume [] contains the chapters having closer connection withsome applications
The given book is the extended and corrected electronic version of the printedbook written is English
The chapters of the first volume are divided into three parts The chapters of Part1 are connected with automata Automata and Formal Languages (written by ZoltaacutenKaacutesa Sapientia Hungarian University of Transylvania) Compilers (Zoltaacuten CsoumlrnyeiEoumltvoumls Loraacutend University) Compression and Decompression (Ulrich Tamm Chem-nitz University of Technology Commitment) Reliable Computations (Peacuteter GaacutecsBoston University)
The chapters of Part 2 have algebraic character here are the chapters Algebra(written by Gaacutebor Ivanyos and Lajos Roacutenyai Budapest University of Technologyand Economics) Computer Algebra (Antal Jaacuterai Attila Kovaacutecs Eoumltvoumls Loraacutend Uni-versity) further Cryptology and Complexity Theory (Joumlrg Rothe Heinrich HeineUniversity)
The chapters of the third part have numeric character Competitive Analysis(Csanaacuted Imreh University of Szeged) Game Theory and Risk Analysis (Ferenc Szi-darovszky The University of Arizona) and Scientific Computations (Aureacutel GalaacutentaiAndraacutes Jeney University of Miskolc)
The second volume is also divided into three parts The chapters of Part 4are connected with computer networks Distributed Algorithms (Burkhard EnglertCalifornia State University Dariusz Kowalski University of Liverpool GrzegorzMalewicz University of Alabama Alexander Allister Shvartsman University of Con-
1208 Introduction
necticut) Parallel Algorithms (Claudia Fohry University of Kassel and Antal IvaacutenyiEoumltvoumls Loraacutend University) Network Simulation (Tibor Gyires Illinois State Uni-versity) and Systolic Systems (Eberhard Zehendner Friedrich Schiller University)
The chapters of Part 5 are Relational Databases and Query in RelationalDatabases (Jaacutenos Demetrovics Eoumltvoumls Loraacutend University Attila Sali Alfreacuted ReacutenyiInstitute of Mathematics) Semistructured Data Bases (Attila Kiss Eoumltvoumls LoraacutendUniversity) and Memory Management (Aacutedaacutem Balog Antal Ivaacutenyi Eoumltvoumls LoraacutendUniversity)
The chapters of the third part of the second volume have close connections withbiology Bioinformatics (Istvaacuten Mikloacutes Reacutenyi Institute of Mathematics) Human-Computer Interactions (Ingo Althoumlfer Stefan Schwarz Friedrich Schiller University)and Computer Graphics (Laacuteszloacute Szirmay-Kalos Budapest University of Technologyand Economics)
The chapters are validated by Gaacutebor IvanyosqnevindexIvanyos Gaacutebor LajosRoacutenyai Andraacutes Recski and Tamaacutes Szaacutentai (Budapest University of Technology andEconomics) Saacutendor Fridli Jaacutenos Gonda and Beacutela Vizvaacuteri (Eoumltvoumls Loraacutend Univer-sity) Paacutel Doumlmoumlsi and Attila Pethő (University of Debrecen) Zoltaacuten Fuumlloumlpqnevin-dexFuumlloumlp Zoltaacuten (University of Szeged) Anna GaacutelqnevindexGaacutel Anna (Universityof Texas) Jaacutenos Mayer (University of Zuumlrich)
The book contains verbal description pseudocode and analysis of over 200 algo-rithms and over 350 figures and 120 examples illustrating how the algorithms workEach section ends with exercises and each chapter ends with problems In the bookyou can find over 330 exercises and 70 problems
We have supplied an extensive bibliography in the section Chapter Notes ofeach chapter The web site of book contains the maintained living version of thebibliography in which the names of authors journals and publishers are usuallylinks to the corresponding web site
The LATEX style file was written by Viktor Beleacutenyesi Zoltaacuten Csoumlrnyei and Laacutes-zloacute Domoszlai The figures was drawn or corrected by Korneacutel Locher Anna Ivaacutenyitransformed the bibliography into hypertext
The publication of the printed book was supported by Department of Mathe-matics of Hungarian Academy of Science and the electronic version received supportfrom
We plan to publish the corrected and extended version of this book in printed andelectronic form too This book has a web site httpelekinfeltehuEnglishBooksYou can use this website to obtain a list of known errors report errors or make sug-gestions (using the data of the colofon page you can contact with any of the creatorsof the book) The website contains the maintaned PDF version of the bibliographyin which the names of the authors journals and publishers are usually active linksto the corresponding web sites (the living elements are underlined in the printedbibliography) We welcome ideas for new exercises and problems
Budapest Szeptember 2010
Antal Ivaacutenyi (tonycompalginfeltehu)
26 The Branch and Bound Method
It has serious practical consequences if it is known that a combinatorial problem isNP-complete Then one can conclude according to the present state of science thatno simple combinatorial algorithm can be applied and only an enumerative-typemethod can solve the problem in question Enumerative methods are investigatingmany cases only in a non-explicit ie implicit way It means that huge majorityof the cases are dropped based on consequences obtained from the analysis of theparticular numerical problem The three most important enumerative methods are(i) implicit enumeration (ii) dynamic programming and (iii) branch and boundmethod This chapter is devoted to the latter one Implicit enumeration and dynamicprogramming can be applied within the family of optimization problems mainly if allvariables have discrete nature Branch and bound method can easily handle problemshaving both discrete and continuous variables Further on the techniques of implicitenumeration can be incorporated easily in the branch and bound frame Branch andbound method can be applied even in some cases of nonlinear programmingThe Branch and Bound (abbreviated further on as BampB) method is just a frame of alarge family of methods Its substeps can be carried out in different ways dependingon the particular problem the available software tools and the skill of the designerof the algorithm
Boldface letters denote vectors and matrices calligraphic letters are used forsets Components of vectors are denoted by the same but non-boldface letter Cap-ital letters are used for matrices and the same but lower case letters denote theirelements The columns of a matrix are denoted by the same boldface but lower caseletters
Some formulae with their numbers are repeated several times in this chapter Thereason is that always a complete description of optimization problems is providedThus the fact that the number of a formula is repeated means that the formula isidentical to the previous one
261 An example the Knapsack Problem
In this section the branch and bound method is shown on a numerical exampleThe problem is a sample of the binary knapsack problem which is one of the easiest
1210 26 The Branch and Bound Method
problems of integer programming but it is still NP-complete The calculations arecarried out in a brute force way to illustrate all features of BampB More intelligentcalculations ie using implicit enumeration techniques will be discussed only at theend of the section
2611 The Knapsack Problem
There are many different knapsack problems The first and classical one is the binaryknapsack problem It has the following story A tourist is planning a tour in themountains He has a lot of objects which may be useful during the tour For exampleice pick and can opener can be among the objects We suppose that the followingconditions are satisfied
bull Each object has a positive value and a positive weight (Eg a balloon filled withhelium has a negative weight See Exercises 261-1 and 261-2) The value is thedegree of contribution of the object to the success of the tour
bull The objects are independent from each other (Eg can and can opener are notindependent as any of them without the other one has limited value)
bull The knapsack of the tourist is strong and large enough to contain all possibleobjects
bull The strength of the tourist makes possible to bring only a limited total weight
bull But within this weight limit the tourist want to achieve the maximal total value
The following notations are used to the mathematical formulation of the prob-lem
n the number of objectsj the index of the objectswj the weight of object jvj the value of object jb the maximal weight what the tourist can bring
For each object j a so-called binary or zero-one decision variable say xj isintroduced
xj =
1 if object j is present on the tour0 if object j isnrsquot present on the tour
Notice that
wjxj =
wj if object j is present on the tour0 if object j isnrsquot present on the tour
is the weight of the object in the knapsackSimilarly vjxj is the value of the object on the tour The total weight in the
knapsack is
nsum
j=1
wjxj
261 An example the Knapsack Problem 1211
which may not exceed the weight limit Hence the mathematical form of the problemis
maxnsum
j=1
vjxj (261)
nsum
j=1
wjxj le b (262)
xj = 0 or 1 j = 1 n (263)
The difficulty of the problem is caused by the integrality requirement If con-straint (263) is substituted by the relaxed constraint ie by
0 le xj le 1 j = 1 n (264)
then the Problem (261) (262) and (264) is a linear programming problem (264)means that not only a complete object can be in the knapsack but any part of itMoreover it is not necessary to apply the simplex method or any other LP algorithmto solve it as its optimal solution is described by
Theorem 261 Suppose that the numbers vj wj (j = 1 n) are all positive andmoreover the index order satisfies the inequality
v1
w1ge v2
w2middot middot middot ge vn
wn (265)
Then there is an index p (1 le p le n) and an optimal solution xlowast such that
xlowast1 = xlowast
2 = middot middot middot = xlowastpminus1 = 1 xlowast
p+1 = xlowastp+2 = middot middot middot = xlowast
p+1 = 0
Notice that there is only at most one non-integer component in xlowast This propertywill be used at the numerical calculations
From the point of view of BampB the relation of the Problems (261) (262) and(263) and (261) (262) and (264) is very important Any feasible solution of thefirst one is also feasible in the second one But the opposite statement is not trueIn other words the set of feasible solutions of the first problem is a proper subset ofthe feasible solutions of the second one This fact has two important consequences
bull The optimal value of the Problem (261) (262) and (264) is an upper boundof the optimal value of the Problem (261) (262) and (263)
bull If the optimal solution of the Problem (261) (262) and (264) is feasible in theProblem (261) (262) and (263) then it is the optimal solution of the latterproblem as well
These properties are used in the course of the branch and bound method intensively
1212 26 The Branch and Bound Method
2612 A numerical example
The basic technique of the BampB method is that it divides the set of feasible solutionsinto smaller sets and tries to fathom them The division is called branching as newbranches are created in the enumeration tree A subset is fathomed if it can bedetermined exactly if it contains an optimal solution
To show the logic of BampB the problem
max 23x1 + 19x2 + 28x3 + 14x4 + 44x5
8x1 + 7x2 + 11x3 + 6x4 + 19x5 le 25x1 x2 x3 x4 x5 = 0 or 1
(266)
will be solved The course of the solution is summarized on Figure 2612Notice that condition (265) is satisfied as
238
= 2875 gt197asymp 2714 gt
2811asymp 2545 gt
146asymp 2333 gt
4419asymp 2316
The set of the feasible solutions of (266) is denoted by F ie
F = x | 8x1 + 7x2 + 11x3 + 6x4 + 19x5 le 25 x1 x2 x3 x4 x5 = 0 or 1
The continuous relaxation of (266) is
max 23x1 + 19x2 + 28x3 + 14x4 + 44x5
8x1 + 7x2 + 11x3 + 6x4 + 19x5 le 250 le x1 x2 x3 x4 x5 le 1
(267)
The set of the feasible solutions of (267) is denoted by R ie
R = x | 8x1 + 7x2 + 11x3 + 6x4 + 19x5 le 25 0 le x1 x2 x3 x4 x5 le 1
Thus the difference between (266) and (267) is that the value of the variables mustbe either 0 or 1 in (266) and on the other hand they can take any value from theclosed interval [0 1] in the case of (267)
Because Problem (266) is difficult (267) is solved instead The optimal solutionaccording to Theorem 261 is
xlowast1 = xlowast
2 = 1 xlowast3 =
1011 xlowast
4 = xlowast5 = 0
As the value of xlowast3 is non-integer the optimal value 6754 is just an upper bound
of the optimal value of (266) and further analysis is needed The value 6754 canbe rounded down to 67 because of the integrality of the coefficients in the objectivefunction
The key idea is that the sets of feasible solutions of both problems are dividedinto two parts according the two possible values of x3 The variable x3 is chosen asits value is non-integer The importance of the choice is discussed below
LetF0 = F F1 = F0 cap x | x3 = 0 F2 = F0 cap x | x3 = 1
261 An example the Knapsack Problem 1213
6 6332
R5
7 minusinfinR6
x1 = 1
5 67127
R4
x1 = 0
x2 = 1
4 65
R3
x2 = 0
x1 = x3 = x4 = 1x2 = x5 = 0
3 6728
R2
x3 = 1
2 6526
R1
x3 = 0
1 6745
R0
Figure 261 The first seven steps of the solution
andR0 = R R1 = R0 cap x | x3 = 0 R2 = R0 cap x | x3 = 1
ObviouslyF1 sube R1 and F2 sube R2
Hence the problem
max 23x1 + 19x2 + 28x3 + 14x4 + 44x5
x isin R1 (268)
1214 26 The Branch and Bound Method
is a relaxation of the problem
max 23x1 + 19x2 + 28x3 + 14x4 + 44x5
x isin F1 (269)
Problem (268) can be solved by Theorem 261 too but it must be taken intoconsideration that the value of x3 is 0 Thus its optimal solution is
xlowast1 = xlowast
2 = 1 xlowast3 = 0 xlowast
4 = 1 xlowast5 =
419
The optimal value is 6526 which gives the upper bound 65 for the optimal value ofProblem (269) The other subsets of the feasible solutions are immediately investi-gated The optimal solution of the problem
max 23x1 + 19x2 + 28x3 + 14x4 + 44x5
x isin R2 (2610)
isxlowast
1 = 1 xlowast2 =
67 xlowast
3 = 1 xlowast4 = xlowast
5 = 0
giving the value 6728 Hence 67 is an upper bound of the problem
max 23x1 + 19x2 + 28x3 + 14x4 + 44x5
x isin F2 (2611)
As the upper bound of (2611) is higher than the upper bound of (269) ie thisbranch is more promising first it is fathomed further on It is cut again into twobranches according to the two values of x2 as it is the non-integer variable in theoptimal solution of (2610) Let
F3 = F2 cap x | x2 = 0 F4 = F2 cap x | x2 = 1 R3 = R2 cap x | x2 = 0 R4 = R2 cap x | x2 = 1
The sets F3 and R3 are containing the feasible solution of the original problems suchthat x3 is fixed to 1 and x2 is fixed to 0 In the sets F4 and R4 both variables arefixed to 1 The optimal solution of the first relaxed problem ie
max 23x1 + 19x2 + 28x3 + 14x4 + 44x5
x isin R3
isxlowast
1 = 1 xlowast2 = 0 xlowast
3 = 1 xlowast4 = 1 xlowast
5 = 0
As it is integer it is also the optimal solution of the problem
max 23x1 + 19x2 + 28x3 + 14x4 + 44x5
x isin F3
261 An example the Knapsack Problem 1215
The optimal objective function value is 65 The branch of the sets F3 and R3 iscompletely fathomed ie it is not possible to find a better solution in it
The other new branch is when both x2 and x3 are fixed to 1 If the objectivefunction is optimized on R4 then the optimal solution is
xlowast1 =
78 xlowast
2 = xlowast3 = 1 xlowast
4 = xlowast5 = 0
Applying the same technique again two branches are defined by the sets
F5 = F4 cap x | x1 = 0 F6 = F4 cap x | x1 = 1
R5 = R4 cap x | x2 = 0 R6 = R4 cap x | x2 = 1 The optimal solution of the branch of R5 is
xlowast1 = 0 xlowast
2 = xlowast3 = xlowast
4 = 1 xlowast5 =
119
The optimal value is 6332 It is strictly less than the objective function value of thefeasible solution found in the branch of R3 Therefore it cannot contain an optimalsolution Thus its further exploration can be omitted although the best feasiblesolution of the branch is still not known The branch of R6 is infeasible as objects1 2 and 3 are overusing the knapsack Traditionally this fact is denoted by usingminusinfin as optimal objective function value
At this moment there is only one branch which is still unfathomed It is thebranch of R1 The upper bound here is 65 which is equal to the objective functionvalue of the found feasible solution One can immediately conclude that this feasiblesolution is optimal If there is no need for alternative optimal solutions then theexploration of this last branch can be abandoned and the method is finished Ifalternative optimal solutions are required then the exploration must be continuedThe non-integer variable in the optimal solution of the branch is x5 The subbranchesreferred later as the 7th and 8th branches defined by the equations x5 = 0 andx5 = 1 give the upper bounds 56 and 61 respectively Thus they do not containany optimal solution and the method is finished
2613 Properties in the calculation of the numerical example
The calculation is revisited to emphasize the general underlying logic of the methodThe same properties are used in the next section when the general frame of BampB isdiscussed
Problem (266) is a difficult one Therefore the very similar but much easierProblem (267) has been solved instead of (266) A priori it was not possible toexclude the case that the optimal solution of (267) is the optimal solution of (266)as well Finally it turned out that the optimal solution of (267) does not satisfyall constraints of (266) thus it is not optimal there But the calculation was notuseless because an upper bound of the optimal value of (266) has been obtainedThese properties are reflected in the definition of relaxation in the next section
As the relaxation did not solved Problem (266) therefore it was divided into
1216 26 The Branch and Bound Method
Subproblems (269) and (2611) Both subproblems have their own optimal solutionand the better one is the optimal solution of (266) They are still too difficult to besolved directly therefore relaxations were generated to both of them These problemsare (268) and (2610) The nature of (268) and (2610) from mathematical point ofview is the same as of (267)
Notice that the union of the sets of the feasible solutions of (268) and (2610)is a proper subset of the relaxation (267) ie
R1 cupR2 sub R0
Moreover the two subsets have no common element ie
R1 capR2 = empty
It is true for all other cases as well The reason is that the branching ie thedetermination of the Subproblems (269) and (2611) was made in a way that theoptimal solution of the relaxation ie the optimal solution of (267) was cut off
The branching policy also has consequences on the upper bounds Let ν(S) bethe optimal value of the problem where the objective function is unchanged andthe set of feasible solutions is S Using this notation the optimal objective functionvalues of the original and the relaxed problems are in the relation
ν(F) le ν(R)
If a subset Rk is divided into Rp and Rq then
ν(Rk) ge maxν(Rp) ν(Rq) (2612)
Notice that in the current Problem (2612) is always satisfied with strict inequality
ν(R0) gt maxν(R1) ν(R2) ν(R1) gt maxν(R7) ν(R8) ν(R2) gt maxν(R3) ν(R4) ν(R4) gt maxν(R5) ν(R6)
(The values ν(R7) and ν(R8) were mentioned only) If the upper bounds of a certainquantity are compared then one can conclude that the smaller the better as it iscloser to the value to be estimated An equation similar to (2612) is true for thenon-relaxed problems ie if Fk = Fp cup Fq then
ν(Fk) = maxν(Fp) ν(Fq) (2613)
but because of the difficulty of the solution of the problems practically it is notpossible to use (2613) for getting further information
A subproblem is fathomed and no further investigation of it is needed if either
bull its integer (non-relaxed) optimal solution is obtained like in the case of F3 or
bull it is proven to be infeasible as in the case of F6 or
261 An example the Knapsack Problem 1217
bull its upper bound is not greater than the value of the best known feasible solution(cases of F1 and F5)
If the first or third of these conditions are satisfied then all feasible solutions of thesubproblem are enumerated in an implicit way
The subproblems which are generated in the same iteration are represented bytwo branches on the enumeration tree They are siblings and have the same parentFigure 241 visualize the course of the calculations using the parentndashchild relation
The enumeration tree is modified by constructive steps when new branches areformed and also by reduction steps when some branches can be deleted as one ofthe three above-mentioned criteria are satisfied The method stops when no subsetremained which has to be still fathomed
2614 How to accelerate the method
As it was mentioned in the introduction of the chapter BampB and implicit enumer-ation can co-operate easily Implicit enumeration uses so-called tests and obtainsconsequences on the values of the variables For example if x3 is fixed to 1 then theknapsack inequality immediately implies that x5 must be 0 otherwise the capacityof the tourist is overused It is true for the whole branch 2
On the other hand if the objective function value must be at least 65 which isthe value of the found feasible solution then it possible to conclude in branch 1 thatthe fifth object must be in the knapsack ie x5 must be 1 as the total value of theremaining objects 1 2 and 4 is only 56
Why such consequences accelerate the algorithm In the example there are 5binary variables thus the number of possible cases is 32 = 25 Both branches 1 and2 have 16 cases If it is possible to determine the value of a variable then the numberof cases is halved In the above example it means that only 8 cases remain to beinvestigated in both branches This example is a small one But in the case of largerproblems the acceleration process is much more significant Eg if in a branch thereare 21 free ie non-fixed variables but it is possible to determine the value of one ofthem then the investigation of 1 048 576 cases is saved The application of the testsneeds some extra calculation of course Thus a good trade-off must be found
The use of information provided by other tools is further discussed in Section265
Exercises261-1 What is the suggestion of the optimal solution of a Knapsack Problem inconnection of an object having (a) negative weight and positive value (b) positiveweight and negative value261-2 Show that an object of a knapsack problem having negative weight andnegative value can be substituted by an object having positive weight and positivevalue such that the two knapsack problems are equivalent (Hint Use complementaryvariable)261-3 Solve Problem (266) with a branching strategy such that an integer valuedvariable is used for branching provided that such a variable exists
1218 26 The Branch and Bound Method
262 The general frame of the BampB method
The aim of this section is to give a general description of the BampB method Particularrealizations of the general frame are discussed in later sections
BampB is based on the notion of relaxation It has not been defined yet As thereare several types of relaxations the first subsection is devoted to this notion Thegeneral frame is discussed in the second subsection
2621 Relaxation
Relaxation is discussed in two steps There are several techniques to define relaxationto a particular problem There is no rule for choosing among them It depends onthe design of the algorithm which type serves the algorithm well The different typesare discussed in the first part titled ldquoRelaxations of a particular problemrdquo In thecourse of the solution of Problem (266) subproblems were generated which werestill knapsack problems They had their own relaxations which were not totallyindependent from the relaxations of each other and the main problem The expectedcommon properties and structure is analyzed in the second step under the titleldquoRelaxation of a problem classrdquo
Relaxations of a particular problem The description of Problem (266)consists of three parts (1) the objective function (2) the algebraic constraints and(3) the requirement that the variables must be binary This structure is typical foroptimization problems In a general formulation an optimization problem can begiven as
max f(x) (2614)
g(x) le b (2615)
x isin X (2616)
Relaxing the non-algebraic constraints The underlying logic of generatingrelaxation (267) is that constraint (2616) has been substituted by a looser one Inthe particular case it was allowed that the variables can take any value between 0and 1 In general (2616) is replaced by a requirement that the variables must belongto a set say Y which is larger than X ie the relation X sube Y must hold Moreformally the relaxation of Problem (2614)-(2616) is the problem
max f(x) (2614)
g(x) le b (2615)
x isin Y (2617)
This type of relaxation can be applied if a large amount of difficulty can be eliminatedby changing the nature of the variables
262 The general frame of the BampB method 1219
Relaxing the algebraic constraints There is a similar technique such that(2616) the inequalities (2615) are relaxed instead of the constraints A natural wayof this type of relaxation is the following Assume that there are m inequalities in(2615) Let λi ge 0 (i = 1 m) be fixed numbers Then any x isin X satisfying(2615) also satisfies the inequality
msum
i=1
λigi(x) lemsum
i=1
λibi (2618)
Then the relaxation is the optimization of the (2614) objective function under theconditions (2618) and (2616) The name of the inequality (2618) is surrogateconstraint
The problem
max 23x1 + 19x2 + 28x3 + 14x4 + 44x5
5x1 + 4x2 + 6x3 + 3x4 + 5x5 le 142x1 minus 2x2 minus 3x3 + 5x4 + 6x5 le 41x1 + 5x2 + 8x3 minus 2x4 + 8x5 le 7
x1 x2 x3 x4 x5 = 0 or 1
(2619)
is a general zero-one optimization problem If λ1 = λ2 = λ3 = 1 then the relaxationobtained in this way is Problem (266) Both problems belong to NP-complete classesHowever the knapsack problem is significantly easier from practical point of viewthan the general problem thus the relaxation may have sense Notice that in thisparticular problem the optimal solution of the knapsack problem ie (10110)satisfies the constraints of (2619) thus it is also the optimal solution of the latterproblem
Surrogate constraint is not the only option in relaxing the algebraic constraintsA region defined by nonlinear boundary surfaces can be approximated by tangentplanes For example if the feasible region is the unit circuit which is described bythe inequality
x21 + x2
2 le 1
can be approximated by the square
minus1 le x1 x2 le 1
If the optimal solution on the enlarged region is eg the point (11) which is not inthe original feasible region then a cut must be found which cuts it from the relaxedregion but it does not cut any part of the original feasible region It is done eg bythe inequality
x1 + x2 leradic
2
A new relaxed problem is defined by the introduction of the cut The method issimilar to one of the method relaxing of the objective function discussed below
1220 26 The Branch and Bound Method
Relaxing the objective function In other cases the difficulty of the problem iscaused by the objective function If it is possible to use an easier objective functionsay h(x) but to obtain an upper bound the condition
forallx isin X h(x) ge f(x) (2620)
must hold Then the relaxation is
max h(x) (2621)
g(x) le b (2615)
x isin X (2616)
This type of relaxation is typical if BampB is applied in (continuous) nonlinearoptimization An important subclass of the nonlinear optimization problems is theso-called convex programming problem It is again a relatively easy subclass There-fore it is reasonable to generate a relaxation of this type if it is possible A Problem(2614)-(2616) is a convex programming problem if X is a convex set the functionsgi(x) (i = 1 m) are convex and the objective function f(x) is concave Thusthe relaxation can be a convex programming problem if only the last condition isviolated Then it is enough to find a concave function h(x) such that (2620) issatisfied
For example the single variable function f(x) = 2x2 minus x4 is not concave in theinterval [ minus
radic3
3 radic
33 ]1 Thus if it is the objective function in an optimization problem
it might be necessary that it is substituted by a concave function h(x) such thatforallx isin [ minus
radic3
3 radic
33 ] f(x) le h(x) It is easy to see that h(x) = 8
9 minus x2 satisfies therequirements
Let xlowast be the optimal solution of the relaxed problem (2621) (2615) and(2616) It solves the original problem if the optimal solution has the same objectivefunction value in the original and relaxed problems ie f(xlowast) = h(xlowast)
Another reason why this type of relaxation is applied that in certain cases theobjective function is not known in a closed form however it can be determined inany given point It might happen even in the case if the objective function is concaveAssume that the value of f(x) is known in the points y1 yk If f(x) concavethen it is smooth ie its gradient exists The gradient determines a tangent planewhich is above the function The equation of the tangent plane in point yp is2
nabla(f(yp))(xminus yp) = 0
Hence in all points of the domain of the function f(x) we have that
h(x) = min f(yp) +nabla(f(yp))(xminus yp) | p = 1 k ge f(x)
Obviously the function h(x) is an approximation of function f(x)
1A continuous function is concave if its second derivative is negative fprimeprime
(x) = 4 minus 12x2 which is
positive in the open interval(
minusradic
33
radic
33
)
2The gradient is considered being a row vector
262 The general frame of the BampB method 1221
The idea if the method is illustrated on the following numerical example Assumethat an ldquounknownrdquo concave function is to be maximized on the [05] closed intervalThe method can start from any point of the interval which is in the feasible regionLet 0 be the starting point According to the assumptions although the closed formulaof the function is not known it is possible to determine the values of function andits derivative Now the values f(0) = minus4 and f
prime
(0) = 4 are obtained The generalformula of the tangent line in the point (x0 f(x0)) is
y = fprime
(x0)(xminus x0) + f(x0)
Hence the equation of the first tangent line is y = 4xminus4 giving the first optimizationproblem as
max hh le 4xminus 4x isin [0 5]
As 4x minus 4 is a monotone increasing function the optimal solution is x = 5 Thenthe values f(5) = minus9 and f
prime
(5) = minus6 are provided by the method calculating thefunction The equation of the second tangent line is y = minus6x+ 21 Thus the secondoptimization problem is
max hh le 4xminus 4 h le minus6x+ 21
x isin [0 5]
As the second tangent line is a monotone decreasing function the optimal solutionis in the intersection point of the two tangent lines giving x = 25 Then the valuesf(25) = minus025 and f
prime
(25) = minus1 are calculated and the equation of the tangent lineis y = minusx+ 225 The next optimization problem is
max hh le 4xminus 4 h le minus6x+ 21 h le minusx+ 225
x isin [0 5]
The optimal solution is x = 125 It is the intersection point of the first and thirdtangent lines Now both new intersection points are in the interval [05] In generalsome intersection points can be infeasible The method goes in the same way furtheron The approximated ldquounknowrdquo function is f(x) = minus(xminus 2)2
The Lagrange Relaxation Another relaxation called Lagrange relaxationIn that method both the objective function and the constraints are modified Theunderlying idea is the following The variables must satisfy two different types ofconstraints ie they must satisfy both (2615) and (2616) The reason that theconstraints are written in two parts is that the nature of the two sets of constraints isdifferent The difficulty of the problem caused by the requirement of both constraintsIt is significantly easier to satisfy only one type of constraints So what about toeliminate one of them
1222 26 The Branch and Bound Method
Assume again that the number of inequalities in (2615) is m Let λi ge 0 (i =1 m) be fixed numbers The Lagrange relaxation of the problem (2614)- (2616)is
max f(x) +msum
i=1
λi(bi minus gi(x)) (2622)
x isin X (2616)
Notice that the objective function (2622) penalizes the violation of the constraintseg trying to use too much resources and rewards the saving of resources The firstset of constraints disappeared from the problem In most of the cases the Lagrangerelaxation is a much easier one than the original problem In what follows Problem(2614)- (2616) is also denoted by (P ) and the Lagrange relaxation is referred as(L(λ)) The notation reflects the fact that the Lagrange relaxation problem dependson the choice of λirsquos The numbers λirsquos are called Lagrange multipliers
It is not obvious that (L(λ)) is really a relaxation of (P ) This relation is estab-lished by
Theorem 262 Assume that both (P ) and (L(λ)) have optimal solutions Thenfor any nonnegative λi (i = 1 m) the inequality
ν(L(λ)) ge ν(P )
holds
Proof The statement is that the optimal value of (L(λ)) is an upper bound of theoptimal value of (P ) Let xlowast be the optimal solution of (P ) It is obviously feasiblein both problems Hence for all i the inequalities λi ge 0 bi ge gi(xlowast) hold Thusλi(bi minus gi(xlowast)) ge 0 which implies that
f(xlowast) le f(xlowast) +msum
i=1
λi(bi minus gi(xlowast))
Here the right-hand side is the objective function value of a feasible solution of(L(λ)) ie
ν(P ) = f(xlowast) le f(xlowast) +msum
i=1
λi(bi minus gi(xlowast)) le ν(L(λ))
There is another connection between (P ) and (L(λ)) which is also importantfrom the point of view of the notion of relaxation
Theorem 263 Let xL be the optimal solution of the Lagrange relaxation If
g(xL) le b (2623)
262 The general frame of the BampB method 1223
and
msum
i=1
λi(bi minus gi(xL)) = 0 (2624)
then xL is an optimal solution of (P )
Proof (2623) means that xL is a feasible solution of (P ) For any feasible solutionx of (P ) it follows from the optimality of xL that
f(x) le f(x) +msum
i=1
λi(bi minus gi(x)) le f(xL) +msum
i=1
λi(bi minus gi(xL)) = f(xL)
ie xL is at least as good as x
The importance of the conditions (2623) and (2624) is that they give an opti-mality criterion ie if a point generated by the Lagrange multipliers satisfies themthen it is optimal in the original problem The meaning of (2623) is that the optimalsolution of the Lagrange problem is feasible in the original one and the meaning of(2624) is that the objective function values of xL are equal in the two problems justas in the case of the previous relaxation It also indicates that the optimal solutionsof the two problems are coincident in certain cases
There is a practical necessary condition for being a useful relaxation which isthat the relaxed problem is easier to solve than the original problem The Lagrangerelaxation has this property It can be shown on Problem (2619) Let λ1 = 1λ2 = λ3 = 3 Then the objective function (2622) is the following
(23x1 + 19x2 + 28x3 + 14x4 + 44x5) + (14minus 5x1 minus x2 minus 6x3 minus 3x4 minus 5x5)
+3(4minus 2x1 minus x2 + 3x3 minus 5x4 minus 6x5) + 3(7minus x1 minus 5x2 minus 8x3 + 2x4 minus 8x5)
= 47 + (23minus 5minus 6minus 3)x1 + (19minus 1minus 3minus 15)x2 + (28minus 6 + 9minus 24)x3
+(14minus 3minus 15 + 5)x4 + (44minus 5minus 18minus 24)x5
= 47 + 9x1 + 0x2 + 7x3 + x4 minus 3x5
The only constraint is that all variables are binary It implies that if a coefficient ispositive in the objective function then the variable must be 1 in the optimal solutionof the Lagrange problem and if the coefficient is negative then the variable mustbe 0 As the coefficient of x2 is zero there are two optimal solutions (10110)and (11110) The first one satisfies the optimality condition thus it is an optimalsolution The second one is infeasible
What is common in all relaxation They have three common properties
1 All feasible solutions are also feasible in the relaxed problem
2 The optimal value of the relaxed problem is an upper bound of the optimalvalue of the original problem
1224 26 The Branch and Bound Method
3 There are cases when the optimal solution of the relaxed problem is also opti-mal in the original one
The last property cannot be claimed for all particular case as then the relaxed prob-lem is only an equivalent form of the original one and needs very likely approximatelythe same computational effort ie it does not help too much Hence the first twoproperties are claimed in the definition of the relaxation of a particular problem
Definition 264 Let f h be two functions mapping from the n-dimensional Eu-clidean space into the real numbers Further on let U V be two subsets of the n-dimensional Euclidean space The problem
maxh(x) | x isin V (2625)
is a relaxation of the problem
maxf(x) | x isin U (2626)
if(i) U sub V and(ii) it is known a priori ie without solving the problems that ν(2625) ge ν(2626)
Relaxation of a problem class No exact definition of the notion of problemclass will be given There are many problem classes in optimization A few examplesare the knapsack problem the more general zero-one optimization the travelingsalesperson problem linear programming convex programming etc In what followsproblem class means only an infinite set of problems
One key step in the solution of (266) was that the problem was divided intosubproblems and even the subproblems were divided into further subproblems andso on
The division must be carried out in a way such that the subproblems belongto the same problem class By fixing the value of a variable the knapsack problemjust becomes another knapsack problem of lesser dimension The same is true foralmost all optimization problems ie a restriction on the value of a single variable(introducing either a lower bound or upper bound or an exact value) creates a newproblem in the same class But restricting a single variable is not the only possibleway to divide a problem into subproblems Sometimes special constraints on a setof variables may have sense For example it is easy to see from the first constraintof (2619) that at most two out of the variables x1 x3 and x5 can be 1 Thus it ispossible to divide it into two subproblems by introducing the new constraint whichis either x1 +x3 +x5 = 2 or x1 +x3 +x5 le 1 The resulted problems are still in theclass of binary optimization The same does not work in the case of the knapsackproblem as it must have only one constraint ie if a second inequality is added tothe problem then the new problem is out of the class of the knapsack problems
The division of the problem into subproblems means that the set of feasiblesolutions is divided into subsets not excluding the case that one or more of thesubsets turn out to be empty set R5 and R6 gave such an example
Another important feature is summarized in formula (2612) It says that the
262 The general frame of the BampB method 1225
upper bound of the optimal value obtained from the undivided problem is at mostas accurate as the upper bound obtained from the divided problems
Finally the further investigation of the subset F1 could be abandoned as R1
was not giving a higher upper bound as the objective function value of the optimalsolution on R3 which lies at the same time in F3 too ie the subproblem definedon the set F3 was solved
The definition of the relaxation of a problem class reflects the fact that relax-ation and defining subproblems (branching) are not completely independent In thedefinition it is assumed that the branching method is a priori given
Definition 265 Let P and Q be two problem classes Class Q is a relaxation ofclass P if there is a map R with the following properties
1 R maps the problems of P into the problems of Q
2 If a problem (P) isin P is mapped into (Q) isin Q then (Q) is a relaxation of (P)in the sense of Definition 264
3 If (P) is divided into (P1) (Pk) and these problems are mapped into(Q1) (Qk) then the inequality
ν(Q) ge maxν(Q1) ν(Qk) (2627)
holds
4 There are infinite many pairs (P) (Q) such that an optimal solution of (Q) isalso optimal in (P)
2622 The general frame of the BampB method
As the Reader has already certainly observed BampB divides the problem into subprob-lems and tries to fathom each subproblem by the help of a relaxation A subproblemis fathomed in one of the following cases
1 The optimal solution of the relaxed subproblem satisfies the constraints ofthe unrelaxed subproblem and its relaxed and non-relaxed objective functionvalues are equal
2 The infeasibility of the relaxed subproblem implies that the unrelaxed sub-problem is infeasible as well
3 The upper bound provided by the relaxed subproblem is less (in the caseif alternative optimal solution are sought) or less or equal (if no alternativeoptimal solution is requested) than the objective function value of the bestknown feasible solution
The algorithm can stop if all subsets (branches) are fathomed If nonlinear program-ming problems are solved by BampB then the finiteness of the algorithm cannot bealways guaranteed
In a typical iteration the algorithm executes the following steps
1226 26 The Branch and Bound Method
bull It selects a leaf of the branching tree ie a subproblem not divided yet intofurther subproblems
bull The subproblem is divided into further subproblems (branches) and their relax-ations are defined
bull Each new relaxed subproblem is solved and checked if it belongs to one of theabove-mentioned cases If so then it is fathomed and no further investigation isneeded If not then it must be stored for further branching
bull If a new feasible solution is found which is better than the so far best one theneven stored branches having an upper bound less than the value of the new bestfeasible solution can be deleted without further investigation
In what follows it is supposed that the relaxation satisfies definition 265The original problem to be solved is
max f(x) (2614)
g(x) le b (2615)
x isin X (2616)
Thus the set of the feasible solutions is
F = F0 = x | g(x) le b x isin X (2628)
The relaxed problem satisfying the requirements of definition 265 is
max h(x)
k(x) le b
x isin Y
where X sube Y and for all points of the domain of the objective functions f(x) le h(x)and for all points of the domain of the constraint functions k(x) le h(x) Thus theset of the feasible solutions of the relaxation is
R = R0 = x | k(x) le b x isin Y
Let Fk be a previously defined subset Suppose that it is divided into the subsetsFt+1 Ft+p ie
Fk =p⋃
l=1
Ft+l
Let Rk and Rt+1 Rt+p be the feasible sets of the relaxed subproblems To satisfythe requirement (2627) of definition 265 it is assumed that
Rk supep⋃
l=1
Rt+l
262 The general frame of the BampB method 1227
The subproblems are identified by their sets of feasible solutions The unfath-omed subproblems are stored in a list The algorithm selects a subproblem from thelist for further branching In the formal description of the general frame of BampB thefollowing notations are used
z the objective function value of the best feasible solution found so farL the list of the unfathomed subsets of feasible solutionst the number of branches generated so farF0 the set of all feasible solutionsr the index of the subset selected for branchingp(r) the number of branches generated from Fr
xi the optimal solution of the relaxed subproblem defined on Ri
zi the upper bound of the objective function on subset Fi
L+ Fi the operation of adding the subset Fi to the list LL minus Fi the operation of deleting the subset Fi from the list L
Note that yi = maxh(x) | x isin RiThe frame of the algorithms can be found below It simply describes the basic
ideas of the method and does not contain any tool of acceleration
Branch-and-Bound
1 z larr minusinfin2 L larr F0 3 t larr 04 while L 6= empty5 do determination of r6 L larr LminusFr
7 determination of p(r)8 determination of branching Fr sub R1 cup cupRp(r)
9 for ilarr 1 to p(r) do10 Ft+i larr Fr capRi
11 calculation of (xt+i zt+i)12 if zt+i gt z13 then if xt+i isin F14 then z larr zt+i
15 else L larr L+ Ft+i
16 t larr t+ p(r)17 for ilarr 1 to t do18 if zi le z19 then L larr LminusFi
20 return x
1228 26 The Branch and Bound Method
The operations in rows 5 7 8 and 11 depend on the particular problem class andon the skills of the designer of the algorithm The relaxed subproblem is solved inrow 14 A detailed example is discussed in the next section The handling of the listneeds also careful consideration Section 264 is devoted to this topic
The loop in rows 17 and 18 can be executed in an implicit way If the selectedsubproblem in row 5 has a low upper bound ie zr le z then the subproblem isfathomed and a new subproblem is selected
However the most important issue is the number of required operations includ-ing the finiteness of the algorithm The method is not necessarily finite Especiallynonlinear programming has infinite versions of it Infinite loop may occur even in thecase if the number of the feasible solutions is finite The problem can be caused byan incautious branching procedure A branch can belong to an empty set Assumethat that the branching procedure generates subsets from Fr such that one of thesubsets Ft+1 Ft+p(r) is equal to Fr and the other ones are empty sets Thus thereis an index i such that
Ft+i = Fr Ft+1 = = Ft+iminus1 = Ft+i+1 = = Ft+p(r) = empty (2629)
If the same situation is repeated at the branching of Ft+i then an infinite loop ispossible
Assume that a zero-one optimization problem of n variables is solved by BampBand the branching is made always according to the two values of a free variableGenerally it is not known that how large is the number of the feasible solutionsThere are at most 2n feasible solutions as it is the number of the zero-one vectorsAfter the first branching there are at most 2nminus1 feasible solutions in the two firstlevel leaves each This number is halved with each branching ie in a branch onlevel k there are at most 2nminusk feasible solutions It implies that on level n there isat most 2nminusn = 20 = 1 feasible solution As a matter of fact on that level there isexactly 1 zero-one vector and it is possible to decide whether or not it is feasibleHence after generating all branches on level n the problem can be solved This ideais generalized in the following finiteness theorem While formulating the statementthe previous notations are used
Theorem 266 Assume that(i) The set F is finite(ii) There is a finite set U such that the following conditions are satisfied If a subsetF is generated in the course of the branch and bound method then there is a subsetU of U such that F sube U Furthermore if the branching procedure creates the coverR1 cup cupRp supe F then U has a partitioning such that
U = U1 cup middot middot middot cup Up Ui cap Uj = empty(i 6= j)
F cap Rj sube Uj(j = 1 p)
and moreover
1 le| Uj |lt| U | (j = 1 p) (2630)
(iii) If a set U belonging to set F has only a single element then the relaxed subprob-lem solves the unrelaxed subproblem as well
262 The general frame of the BampB method 1229
Then the Branch-and-Bound procedure stops after finite many steps If z =minusinfin then there is no feasible solution Otherwise z is equal to the optimal objectivefunction value
Remark Notice that the binary problems mentioned above with Uj rsquos of type
Uj = x isin 0 1n | xk = δkj k isin Ij
where Ij sub 1 2 n is the set of fixed variables and δkj isin 0 1 is a fixed valuesatisfy the conditions of the theorem
Proof Assume that the procedure Branch-and-Bound executes infinite manysteps As the set F is finite it follows that there is at least one subset of F say Fr
such that it defines infinite many branches implying that the situation described in(2629) occurs infinite many times Hence there is an infinite sequence of indicessay r0 = r lt r1 lt middot middot middot such that Frj+1
is created at the branching of Frjand
Frj+1= Frj
On the other hand the parallel sequence of the U sets must satisfy theinequalities
| Ur0|gt| Ur1
|gt middot middot middot ge 1
It is impossible because the Us are finite setsThe finiteness of F implies that optimal solution exists if and only if F is
nonempty ie the problem cannot be unbounded and if feasible solution exist thenthe supremum of the objective function is its maximum The initial value of z isminusinfin It can be changed only in Row 18 of the algorithm and if it is changed thenit equals to the objective function value of a feasible solution Thus if there is nofeasible solution then it remains minusinfin Hence if the second half of the statement isnot true then at the end of the algorithm z equal the objective function value of anon-optimal feasible solution or it remains minusinfin
Let r be the maximal index such that Fr still contains the optimal solutionThen
zr ge optimal value gt z
Hence it is not possible that the branch containing the optimal solution has beendeleted from the list in the loop of Rows 22 and 23 as zr gt z It is also sure thatthe subproblem
maxf(x) | x isin Frhas not been solved otherwise the equation zr = z should hold Then only one optionremained that Fr was selected for branching once in the course of the algorithm Theoptimal solution must be contained in one of its subsets say Ft+i which contradictsthe assumption that Fr has the highest index among the branches containing theoptimal solution
If an optimization problem contains only bounded integer variables then the setsUs are the sets the integer vectors in certain boxes In the case of some schedulingproblems where the optimal order of tasks is to be determined even the relaxationshave combinatorial nature because they consist of permutations Then U = R is also
1230 26 The Branch and Bound Method
possible In both of the cases Condition (iii) of the theorem is fulfilled in a naturalway
Exercises262-1 Decide if the Knapsack Problem can be a relaxation of the Linear BinaryOptimization Problem in the sense of Definition 265 Explain your solution regard-less that your answer is YES or NO
263 Mixed integer programming with boundedvariables
Many decisions have both continuous and discrete nature For example in the pro-duction of electric power the discrete decision is to switch on or not an equipmentThe equipment can produce electric energy in a relatively wide range Thus if thefirst decision is to switch on then a second decision must be made on the level ofthe produced energy It is a continuous decision The proper mathematical model ofsuch problems must contain both discrete and continuous variables
This section is devoted to the mixed integer linear programming problem withbounded integer variables It is assumed that there are n variables and a subset ofthem say I sube 1 n must be integer The model has m linear constraints inequation form and each integer variable has an explicit integer upper bound It is alsosupposed that all variables must be nonnegative More formally the mathematicalproblem is as follows
max cT x (2631)
Ax = b (2632)
forall j isin I xj le gj (2633)
xj ge 0 j = 1 n (2634)
forall j isin I xj is integer (2635)
where c and x are n-dimensional vectors A is an mtimesn matrix b is an m-dimensionalvector and finally all gj (j isin I) is a positive integer
In the mathematical analysis of the problem below the the explicit upper boundconstraints (2633) will not be used The Reader may think that they are formallyincluded into the other algebraic constraints (2632)
There are technical reasons that the algebraic constraints in (2632) are claimedin the form of equations Linear programming relaxation is used in the methodThe linear programming problem is solved by the simplex method which needs thisform But generally speaking equations and inequalities can be transformed into
263 Mixed integer programming with bounded variables 1231
one another in an equivalent way Even in the numerical example discussed belowinequality form is used
First a numerical example is analyzed The course of the method is discussedfrom geometric point of view Thus some technical details remain concealed Nextsimplex method and related topics are discussed All technical details can be de-scribed only in the possession of them Finally some strategic points of the algorithmare analyzed
2631 The geometric analysis of a numerical example
The problem to be solved is
max x0 = 2x1 + x2
3x1 minus 5x2 le 03x1 + 5x2 le 15
x1 x2 ge 0x1 x2 is integer
(2636)
To obtain a relaxation the integrality constraints are omitted from the problemThus a linear programming problem of two variables is obtained
The branching is made according to a non-integer variable Both x1 and x2 havefractional values To keep the number of branches as low as possible only two newbranches are created in a step
The numbering of the branches is as follows The original set of feasible solutionsis No 1 When the two new branches are generated then the branch belonging tothe smaller values of the branching variable has the smaller number The numbersare positive integers started at 1 and not skipping any integer Branches having nofeasible solution are numbered too
The optimal solution of the relaxation is x1 = 25 x2 = 15 and the optimalvalue is 13
2 as it can be seen from figure 262 The optimal solution is the intersectionpoint the lines determined by the equations
3x1 minus 5x2 = 0
and3x1 + 5x2 = 15
If the branching is based on variable x1 then they are defined by the inequalities
x1 le 2 and x1 ge 3
Notice that the maximal value of x1 is 25 In the next subsection the problem isrevisited Then this fact will be observed from the simplex tableaux Variable x2
would create the branches
x2 le 1 and x2 ge 2
1232 26 The Branch and Bound Method
1 2 3 4 5
Feasible region
O
B
A
1
2
3
4
2x1 + x2 = 132
Figure 262 The geometry of linear programming relaxation of Problem (2636) including thefeasible region (triangle OAB) the optimal solution (x1 = 25 x2 = 15) and the optimal level ofthe objective function represented by the line 2x1 + x2 = 13
2
None of them is empty Thus it is more advantageous the branch according to x1Geometrically it means that the set of the feasible solutions in the relaxed problemis cut by the line x1 = 2 Thus the new set becomes the quadrangle OACD onFigure 263 The optimal solution on that set is x1 = 2 x2 = 18 It is point C onthe figure
Now branching is possible according only to variable x2 Branches 4 and 5 aregenerated by the cuts x2 le 1 and x2 ge 2 respectively The feasible regions ofthe relaxed problems are OHG of Branch 4 and AEF of Branch 5 The methodcontinues with the investigation of Branch 5 The reason will be given in the nextsubsection when the quickly calculable upper bounds are discussed On the otherhand it is obvious that the set AEF is more promising than OHG if the Readertakes into account the position of the contour ie the level line of the objectivefunction on Figure 263 The algebraic details discussed in the next subsection serveto realize the decisions in higher dimensions what is possible to see in 2-dimension
Branches 6 and 7 are defined by the inequalities x1 le 1 and x1 ge 2 respectivelyThe latter one is empty again The feasible region of Branch 6 is AIJF The optimalsolution in this quadrangle is the Point I Notice that there are only three integerpoints in AIJF which are (03) (02) and (12) Thus the optimal integer solution of
263 Mixed integer programming with bounded variables 1233
1 2 3O
B
A
C
D
E
F
GH
I
J
1
2
3
Branch 2 - Branch 3 EMPTY
Branch 4
6Branch 5
Branch 6
Figure 263 The geometry of the course of the solution The co-ordinates of the points areO=(00) A=(03) B=(2515) C=(218) D=(212) E=( 5
32) F=(02) G=( 5
31) H=(01)
I=(124) and J=(12) The feasible regions of the relaxation are as follows Branch 1 OAB Branch2 OACD Branch 3 empty set Branch 4 OHG Branch 5 AEF Branch 6 AIJF Branch 7empty set (not on the figure) Point J is the optimal solution
this branch is (12) There is a technique which can help to leap over the continuousoptimum In this case it reaches directly point J ie the optimal integer solution ofthe branch as it will be seen in the next section too Right now assume that theinteger optimal solution with objective function value 4 is uncovered
At this stage of the algorithm the only unfathomed branch is Branch 4 withfeasible region OHG Obviously the optimal solution is point G=(5
3 1) Its objectivefunction value is 13
3 Thus it cannot contain a better feasible solution than the known(12) Hence the algorithm is finished
2632 The linear programming background of the method
The first ever general method solving linear programming problems were discoveredby George Dantzig and called simplex method There are plenty of versions of thesimplex method The main tool of the algorithm is the so-called dual simplex methodAlthough simplex method is discussed in a previous volume the basic knowledge issummarized here
Any kind of simplex method is a so-called pivoting algorithm An importantproperty of the pivoting algorithms is that they generate equivalent forms of theequation system and ndash in the case of linear programming ndash the objective functionPractically it means that the algorithm works with equations As many variables asmany linearly independent equations exist are expressed with other variables and
1234 26 The Branch and Bound Method
x1 = 1x2 = 2
solution
6 4
43
5 5 13
715
x1 le 1
x2 ge 2
7 minusinfin
x1 = 2
4 4 13
45
x2 le 1
2 5 45
710
x1 le 2
1 6 12
3 minusinfin
x1 ge 3
Figure 264 The course of the solution of Problem (2636) The upper numbers in the circuits areexplained in subsection 2632 They are the corrections of the previous bounds obtained from thefirst pivoting step of the simplex method The lower numbers are the (continuous) upper boundsobtained in the branch
further consequences are drawn from the current equivalent form of the equationsIf there are inequalities in the problem then they are reformulated by introducing
nonnegative slack variables Eg in case of LP-relaxation of Problem (2636) the
263 Mixed integer programming with bounded variables 1235
equivalent form of the problem is
max x0 = 2x1 + x2 + 0x3 + 0x4
3x1 minus 5x2 + x3 + 0x4 = 03x1 + 5x2 + 0x3 + x4 = 15x1 x2 x3 x4 ge 0
(2637)
Notice that all variables appear in all equations including the objective functionbut it is allowed that some coefficients are zeros The current version (2637) can beconsidered as a form where the variables x3 and x4 are expressed by x1 and x2 andthe expression is substituted into the objective function If x1 = x2 = 0 then x3 = 0and x4 = 15 thus the solution is feasible Notice that the value of the objectivefunction is 0 and if it is possible to increase the value of any of x1 and x2 andstill getting a feasible solution then a better feasible solution is obtained It is truebecause the method uses equivalent forms of the objective function The methodobtains better feasible solution by pivoting Let x1 and x2 be the two expressedvariables Skipping some pivot steps the equivalent form of (2637) is
max x0 = 0x1 + 0x2 minus 730x3 minus 13
30x4 + 132
x1 + 0x2 + 16x3 + 1
6x4 = 52
0x1 + x2 minus 110x3 + 1
10x4 = 32
x1 x2 x3 x4 ge 0
(2638)
That form has two important properties First if x3 = x4 = 0 then x1 = 52 and
x2 = 32 thus the solution is feasible similarly to the previous case Generally this
property is called primal feasibility Secondly the coefficients of the non-expressedvariables are negative in the objective function It is called dual feasibility It impliesthat if any of the non-expressed variables is positive in a feasible solution then that isworse than the current one It is true again because the current form of the objectivefunction is equivalent to the original one Thus the current value of the objectivefunction which is 13
2 is optimalIn what follows the sign of maximization and the nonnegativity of the variables
will be omitted from the problem but they are kept in mindIn the general case it may be assumed without loss of generality that all equations
are independent Simplex method uses the form of the problem
max x0 = cT x (2639)
Ax = b (2640)
x ge 0 (2641)
where A is an m times n matrix c and x are n-dimensional vectors and b is an m-dimensional vector According to the assumption that all equations are independentA has m linearly independent columns They form a basis of the m-dimensionallinear space They also form an m times m invertible submatrix It is denoted by BThe inverse of B is Bminus1 Matrix A is partitioned into the basic and non-basic parts
1236 26 The Branch and Bound Method
A = (BN) and the vectors c and x are partitioned accordingly Hence
Ax = BxB + NxN = b
The expression of the basic variables is identical with the multiplication of the equa-tion by Bminus1 from left
Bminus1Ax = Bminus1BxB + Bminus1NxN = IxB + Bminus1NxN = Bminus1b
where I is the unit matrix Sometimes the equation is used in the form
xB = Bminus1bminusBminus1NxN (2642)
The objective function can be transformed into the equivalent form
cT x = cTBxB + cT
N xN
cTB(Bminus1bminusBminus1NxN ) + cT
N xN = cTBBminus1b + (cT
N minus cTBBminus1N)xN
Notice that the coefficients of the basic variables are zero If the non-basic variablesare zero valued then the value of the basic variables is given by the equation
xB = Bminus1b
Hence the objective function value of the basic solution is
cT x = cTBxB + cT
N xN = cTBBminus1b + cT
N 0 = cTBBminus1b (2643)
Definition 267 A vector x is a solution of Problem (2639)-(2641) if it satisfiesthe equation (2640) It is a feasible solution or equivalently a primal feasiblesolution if it satisfies both (2640) and (2641) A solution x is a basic solutionif the columns of matrix A belonging to the non-zero components of x are linearlyindependent A basic solution is a basic feasible or equivalently a basic primalfeasible solution if it is feasible Finally a basic solution is basic dual feasiblesolution if
cTN minus cT
BBminus1N le 0T (2644)
The primal feasibility of a basic feasible solution is equivalent to
Bminus1b ge 0
Let a1 an be the column vectors of matrix A Further on let IB and IN be theset of indices of the basic and non-basic variables respectively Then componentwisereformulation of (2644) is
forall j isin IN cj minus cTBBminus1aj le 0
The meaning of the dual feasibility is this The current value of the objective functiongiven in (2643) is an upper bound of the optimal value as all coefficients in theequivalent form of the objective function is nonpositive Thus if any feasible ienonnegative solution is substituted in it then value can be at most the constantterm ie the current value
263 Mixed integer programming with bounded variables 1237
Definition 268 A basic solution is OPTIMAL if it is both primal and dual fea-sible
It is known from the theory of linear programming that among the optimalsolutions there is always at least one basic solution To prove this statement isbeyond the scope of the chapter
In Problem (2637)
A =(
3 minus5 1 03 5 0 1
)
b =(
015
)
c =
2100
If the basic variables are x1 and x2 then
B =(
3 minus53 5
)
Bminus1 =130
(
5 5minus3 3
)
N =(
1 00 1
)
cB =(
21
)
Hence
cTBBminus1 = (2 1)
130
(
5 5minus3 3
)
=(
730
1330
)
Bminus1b =130
(
5 5minus3 3
)(
015
)
=(
75304530
)
=(
5232
)
Bminus1N = Bminus1
The last equation is true as N is the unit matrix Finally
cTN minus cT
BBminus1N = (0 0)minus(
730
1330
)(
1 00 1
)
=(
minus 730 minus13
30
)
One can conclude that this basic solution is both primal and dual feasibleThere are two types of simplex methods Primal simplex method starts from
a primal feasible basic solution Executing pivoting steps it goes through primalfeasible basic solutions and finally even the dual feasibility achieved The objectivefunction values are monotone increasing in the primal simplex method
The dual simplex method starts from a dual feasible basic solution it goesthrough dual feasible basic solutions until even primal feasibility is achieved in thelast iteration The objective function values are monotone decreasing in the dualsimplex method We discuss it in details as it is the main algorithmic tool of themethod
Each simplex method uses its own simplex tableau Each tableau contains thetransformed equivalent forms of the equations and the objective function In thecase of the dual simplex tableau the elements of it are derived from the form of theequations
xB = Bminus1bminusBminus1NxN = Bminus1b + Bminus1N(minusxN )
where the second equation indicates that the minus sign is associated to non-basicvariables The dual simplex tableau contains the expression of all variables by the
1238 26 The Branch and Bound Method
negative non-basic variables including the objective function variable x0 and thenon-basic variables For the latter ones the trivial
xj = minus(minusxj)
equation is included For example the dual simplex tableau for (2637) is providedthat the basic variables are x1 and x2 (see (2638))
variable constant minusx3 minusx4
x0 132 730 1330x1 52 16 16x2 32 minus110 110x3 0 minus1 0x4 0 0 minus1
Generally speaking the potentially non-zero coefficients of the objective function arein the first row the constant terms are in the first column and all other coefficientsare in the inside of the tableau The order of the rows is never changed On theother hand a variable which left the basis immediately has a column instead of thatvariable which entered the basis
The elements of the dual simplex tableau are denoted by djk where k = 0 refersto the constant term of the equation of variable xj and otherwise k isin IN and djk isthe coefficient of the non-basic variable minusxk in the expression of the variable xj Asx0 is the objective function variable d0k is the coefficient of minusxk in the equivalentform (2642) of the objective function The dual simplex tableau can be seen onFigure 265
Notice that dual feasibility means that there are nonnegative elements in thefirst row of the tableau with the potential exception of its first element ie with thepotential exception of the objective function value
Without giving the proof of its correctness the pivoting procedure is this Theaim of the pivoting is to eliminate the primal infeasibility ie the negative valuesof the variables with the potential exception of the objective function value iethe elimination of the negative terms from the first column Instead of that basicvariable xp a non-basic one will be expressed from the equation such that the negativeconstant term becomes zero and the value of the new basic variable ie the valueof xk becomes positive It is easy to see that this requirement can be satisfied onlyif the new expressed variable say xk has a negative coefficient in the equation iedpk lt 0 After the change of the basis the row of xp must become a negative unitvector as xp became a non-basic variable thus its expression is
xp = minus(minusxp) (2645)
The transformation of the tableau consists of the transformations of the columns suchthat the form (2645) of the row of xp is generated The position of the (-1) in therow is the crossing of the row of xp and the column belonging to xk before pivoting
263 Mixed integer programming with bounded variables 1239
d00
objective function value
d0k+
objective function coefficient
dj06
constant term in the equation of xj
djk6
the coefficient of minusxk in the equation of xj
Figure 265 The elements of the dual simplex tableau
This column becomes the column of xp There is another requirement claiming thatthe dual feasibility must hold on Let dj be the column of the non-basic variable xj
including d0 as the column of the constant terms Then the formulae of the columntransformation are the followings where j is either zero or the index of a non-basicvariable different from k
dnewj = dold
j minusdold
pj
doldpk
doldk (2646)
and
dnewp = minus 1
doldpk
doldk
To maintain dual feasibility means that after the change of the basis the relationdnew
0j ge 0 must hold for all non-basic indices ie for all j isin InewN It follows from
(2646) that k must be chosen such that
k = argmax
dold0j
doldpj
| doldpj lt 0
(2647)
In the course of the branch method in the optimization of the relaxed subproblemsdual simplex method can save a lot of computation On the other hand what is usedin the description of the method is only the effect of one pivoting on the value of
1240 26 The Branch and Bound Method
the objective function According to (2646) the new value is
dnew00 = dold
00 minusdold
p0
doldpk
dold0k
Notice that doldp0 d
oldpk lt 0 and dold
0k ge 0 Hence the objective function value decreasesby the nonnegative value
doldp0
doldpk
dold0k (2648)
The formula (2648) will be used if a new constraint is introduced at branchingand it cuts the previous optimal solution As the new constraint has nothing to dowith the objective function it will not destroy dual feasibility but of course theoptimal solution of the relaxed problem of the branch becomes primal infeasible
For example the inequality x1 le 2 is added to the relaxation (2637) defining anew branch then it is used in the equation form
x1 + x5 = 2 (2649)
where x5 is nonnegative continuous variable According to the simplex tableau
x1 =52
+16
(minusx3) +16
(minusx4)
Hence
x5 = minus12minus 1
6(minusx3)minus 1
6(minusx4) (2650)
(2649) is added to the problem in the form (2650) Then the dual simplex tableauis
variable constant minusx3 minusx4
x0 132 730 1330x1 52 16 16x2 32 minus110 110x3 0 minus1 0x4 0 0 minus1x5 minus12 minus16 minus16
Only x5 has a negative value thus the first pivoting must be done in its row Rule(2647) selects x3 for entering the basis Then after the first pivot step the value ofthe objective function decreases by
minus 12
minus 16
times 730
=710
If the optimal solution of the relaxed problem is not reached after the first pivoting
263 Mixed integer programming with bounded variables 1241
variable constant minusx3 minusx4
x0 132 730 1330x1 52 16 16x2 32 minus110 110x3 0 minus1 0x4 0 0 minus1x6 minus12 16 16
then further decrease is possible But decrease of 07 is sure compared to the previousupper bound
Another important property of the cuts is that if it has no negative coefficientin the form how it is added to the simplex tableau then there is no negative pivotelement ie the relaxed problem is infeasible ie the branch is empty For examplethe cut x1 ge 3 leading to an empty branch is added to the problem in the form
x1 minus x6 = 3
where x6 is also a nonnegative variable Substituting the value of x1 again theequation is transformed to
x6 = minus12
+16
(minusx3) +16
(minusx4)
Hence the simplex tableau is obtained There is a negative value at the crossing pointof the first column and the row of x6 But it is not possible to choose a pivot elementin that row as there is no negative element of the row It means that feasibility cannot be achieved ie that branch is infeasible and thus it is completely fathomed
2633 Fast bounds on lower and upper branches
The branching is always based on a variable which should be integer but in thecurrent optimal solution of the linear programming relaxation it has fractional valueIf it has fractional value then its value is non-zero thus it is basic variable Assumethat its index is p Remember that I IB and IN are the index sets of the integerbasic and non-basic variables respectively Hence p isin I capIB According to the lastsimplex tableau xp is expressed by the non-basic variables as follows
xp = dp0 +sum
jisinIN
dpj(minusxj) (2651)
As dp0 has fractional value
1 gt fp = dp0 minus bdp0c gt 0
The branch created by the inequality
xp le bdp0c (2652)
1242 26 The Branch and Bound Method
is called lower branch and the inequality
xp ge bdp0c+ 1
creates the upper branchLet J+ and Jminus be the set of indices of the nonbasic variables according to the
signs of the coefficients in (2651) ie
J +(Jminus) = j | j isin IN dpj gt 0 (dpj lt 0)
First the lower branch is analyzed It follows from (2651) that the inequality(2652) is equivalent to
xp minus bdp0c = fp +sum
jisinIN
dpj(minusxj) le 0
Thus
s = minusfp +sum
jisinIN
(minusdpj)(minusxj) (2653)
is a nonnegative variable and row (2653) can be added to the dual simplex tableauIt will contain the only negative element in the first column that is the optimizationin the lower branch starts by pivoting in this row (2653) can be reformulatedaccording to the signs of the coefficients as
s = minusfp +sum
jisinJ +
(minusdpj)(minusxj) +sum
jisinJ minus
(minusdpj)(minusxj) (2654)
The pivot element must be negative thus it is one of minusdpj rsquos with j isin J+ Hencethe first decrease (2648) of the objective function is
Plp = min
d0j
dpjfp | j isin J +
(2655)
In the upper branch the inequality (2652) is equivalent to
xp minus bdp0c = fp +sum
jisinIN
dpj(minusxj) ge 1
Again the nonnegative slack variable s should be introduced Then the row whichcan be added to the simplex tableau is
s = (fp minus 1) +sum
jisinJ +
dpj(minusxj) +sum
jisinJ minus
dpj(minusxj) (2656)
Thus the pivot element must belong to one of the indices j isin Jminus giving the value
Pup = min
d0j
minusdpj(1minus fp) | j isin Jminus
(2657)
263 Mixed integer programming with bounded variables 1243
Let z be the upper bound on the original branch obtained by linear programmingThen the quantities Plp and Pup define the upper bounds of the objective functionszminusPlp and zminusPup on the lower and upper subbranches respectively They are notsubstituting complete optimization in the subbranches On the other hand they areeasily computable and can give some orientation to the selection of the next branchfor further investigation (see below)
The quantities Plp and Pup can be improved ie increased The claim that thevariable s defined in (2654) is nonnegative implies that
minus fp gesum
jisinJ +
dpj(minusxj) (2658)
In a similar way the nonnegativity of variable s in (2656) implies that
fp minus 1 gesum
jisinJ minus
(minusdpj)(minusxj) (2659)
If (2659) is multiplied by the positive number
fp
1minus fp
then it gets the form
minus fp gesum
jisinJ minus
fp
1minus fp(minusdpj)(minusxj) (2660)
The inequalities (2658) and (2660) can be unified in the form
minus fp gesum
jisinJ +
dpj(minusxj) +sum
jisinJ minus
fp
1minus fp(minusdpj)(minusxj) (2661)
Notice that (2661) not the sum of the two inequalities The same negative numberstands on the left-hand side of both inequalities and is greater or equal than theright-hand side Then both right-hand sides must have negative value Hence theleft-hand side is greater than their sum
The same technique is applied to the variable xprime
p instead of xp with
xprime
p = xp +sum
jisinIcapIN
microjxj
where microj rsquos are integer values to be determined later xprime
p can be expressed by thenon-basic variables as
xprime
p = dp0 +sum
jisinIcapIN
(dpj minus microj)(minusxj) +sum
jisinIN Idpj(minusxj)
Obviously xprime
p is an integer variable as well and its current value if the non-basic
1244 26 The Branch and Bound Method
variables are fixed to zero is equal to the current value of dp0 Thus it is possible todefine the new branches according to the values of x
prime
p Then the inequality of type(2661) which is defined by x
prime
p has the form
minusfp gesum
j isin I cap IN
dpj minus microj ge 0
(dpj minus microj)(minusxj) +sum
j isin I cap IN
dpj minus microj lt 0
fp
1minus fp(microj minus dpj)(minusxj)
+sum
j isin IN Idpj gt 0
dpj(minusxj) +sum
j isin IN Idpj lt 0
fp
1minus fp(minusdpj)(minusxj)
The appropriate quantities Pprime
lp and Pprime
up are as follows
Pprime
lp = mina b
where
a = min
d0j
dpj minus microjfp | j isin I cap IN dpj minus microj gt 0
and
b = min
d0j
dpjfp | j isin IN I dpj gt 0
furtherP
prime
up = minc d where
c = min
d0j(1minus fp)2
(microj minus dpj)fp| j isin I cap IN dpj minus microj lt 0
and
d = min
minusd0j(1minus fp)2
fpdpj| j isin IN I dpj lt 0
The values of the integers must be selected in a way that the absolute values of thecoefficients are as small as possible because the inequality cuts the greatest possiblepart of the polyhedral set of the continuous relaxation in this way (See Exercise263-1) To do so the absolute value of dpjminusmicroj must be small Depending on its signit can be either fj or fjminus1 where fj is the fractional part of dpj ie fj = dpjminusbdpjc
Assume that fj gt 0 If dpj + microj = fj then the term
d0jfp
fj(2662)
is present among the terms of the minimum of the lower branch If dpj gt 0 then itobviously is at least as great as the term
d0jfp
dpj
263 Mixed integer programming with bounded variables 1245
which appears in Plp ie in the right-hand side of (2655) If dpj lt 0 then there isa term
d0j(fp minus 1)dpj
(2663)
is in the right-hand side of (2657) doj is a common multiplier in the terms (2662)and (2663) therefore it can be disregarded when the terms are compared Underthe assumption that fj le fp it will be shown that
fp
fjge fp minus 1
dpj
As dpj is supposed to be negative the statement is equivalent to
dpjfp le (fp minus 1)fj
Hence the inequality
(bdpjc+ fj) fp le fpfj minus fj
must hold It can be reduced to
bdpjc fp le minusfj
It is true as bdpjc le minus1 and
minus1 le minusfj
fplt 0
If dpj + microj = fj minus 1 then according to (2657) and (2661) the term
d0j(1minus fj)2
fp(1minus fj)
is present among the terms of the minimum of the upper branch In a similar wayit can be shown that if fj gt fp then it is always at least as great as the term
d0j(fj minus 1)dpj
which is present in the original formula (2657)Thus the rule of the choice of the integers microj rsquos is
microj =
bdpjc if fj le fp ddpje if fj gt fp
(2664)
2634 Branching strategies
The BampB frame doesnrsquot have any restriction in the selection of the unfathomednode for the next branching in row 7 of Branch-and-Bound First two extremestrategies are discussed with pros and cons The same considerations have to betaken in almost all applications of BampB The third method is a compromise betweenthe two extreme ones Finally methods more specific to the integer programming arediscussed
1246 26 The Branch and Bound Method
The LIFO Rule LIFO means ldquoLast-In-First-Outrdquo ie one of the branchesgenerated in the last iteration is selected A general advantage of the rule is that thesize of the enumeration tree and the size of the list L remains as small as possible Inthe case of the integer programming problem the creation of the branches is based onthe integer values of the variables Thus the number of the branches is at most gj +1if the branching variable is xj In the LIFO strategy the number of leaves is strictlyless then the number of the created branches on each level with the exemption ofthe deepest level Hence at any moment the enumeration tree may not have morethan
nsum
j=1
gj + 1
leavesThe drawback of the strategy is that the flexibility of the enumeration is lost
The flexibility is one of the the main advantage of BampB in solving pure integerproblems
If the algorithm skips from one branch to another branch far away from thefirst one then it is necessary to reconstruct the second branch including not only thebranching restrictions on the variables but any other information which is necessaryto the bounding procedure In the particular algorithm the procedure determiningthe bound is linear programming more precisely a simplex method If a new re-striction as a linear constraint is added to the problem which cuts off the previousoptimal solution then the simplex tableau looses the primal feasibility but the dualfeasibility still holds Thus a dual simplex method can immediately start withoutcarrying out a first phase (The purpose of the first phase which itself is a completeoptimization is to find a primal or dual feasible basic solution depending for primalor dual simplex method respectively) If the BampB method skips to another branchthen to get the new bound by the simplex method will require the execution of thefirst phase too
A further consideration concerns to the construction of feasible solutions Gener-ally speaking if good feasible solutions are known in the early phase of the algorithmthen the whole procedure can be accelerated In the current algorithm branching hasa constructive nature It means that the value of the branching variable becomesmore restricted therefore it either becomes integer in the further optimal solutionsin the subbranches of the branch or it will be restricted further on Thus it can beexpected that sooner or later a complete integer solution is constructed which mightbe feasible or infeasible On the other hand if the algorithm skips frequently in thephase when no feasible solution is known then it is very likely that any constructionwill be finished only later ie the acceleration will not take place because of thelack of feasible solution
If a LIFO type step is to be done and the branching variable is xp then the lowerbranch should be chosen in step 7 of the algorithm if
zr minus Plp ge zr minus Pup ie Plp le Pup
263 Mixed integer programming with bounded variables 1247
The maximal bound The other extreme strategy is that the branch having themaximal bound is selected in each iteration The idea is simple and clear it is themost promising branch therefore it worth to explore it
Unfortunately the idea is not completely true The bounds of the higher levelbranches are not accurate enough This phenomenon has been discussed during theanalysis of the numerical example in the subsection 2613 in relation (2612) Thusa somewhat smaller upper bound in a lower branch can indicate a more promisingbranch
The maximal bound strategy can lead to a very wide enumeration tree whichmay cause memory problems Moreover the construction of feasible solutions will beslow and therefore the relatively few solutions will be enumerated implicitly ie thenumber of steps will be high ie the method will be slow
Fast bounds and estimates If the optimal solution of the relaxed problemis non-integer then it can have several fractional components All of them must bechanged to be integer to obtain the optimal integer programming solution of thebranch The change of the value of each currently fractional variable as a certaincost The cost of the individual changes are estimated and summed up The costmeans the loss in the value of the objective function An adjusted value of the boundof the branch is obtained if the sum of the estimated individual costs is subtractedfrom the current bound It is important to emphasize that the adjusted value is notan upper or lower bound of the optimal value of integer programming solution ofthe branch but it is only a realistic estimation
There are two ways to obtain the estimation The first one uses the crude valuesof the fractionality Let fj and f0
j be the fractional part of variable xj in the currentbranch and in the relaxed problem of the original problem respectively Further onlet zr z0 and z be the optimal value of the relaxed problem in the current branchin the original problem and the value of the best feasible integer solution found sofar Generally speaking the measure of the fractionality of a real number α is thathow far is α to the closest integer ie
minαminus bαc dαe minus α
Hence the estimate is
zr minus (z0 minus z)sum
jisinI minfj 1minus fjsum
jisinI minf0j 1minus f0
j (2665)
(2665) takes into account the average inaccuracy of the boundsThe fast bounds defined in (2655) and (2657) can be used also for the same
purpose They concern to the correction of the fractionality of a single variable inthe current branch Hence the estimate
zr minussum
jisinIminPlj Puj
is a natural choice
1248 26 The Branch and Bound Method
A Rule based on depth bound and estimates The constraints definingthe branches are integer valued lower and upper bounds on the branching variablesThus one can expect that these new constraints force the majority of the branchingvariables to be integer It means that the integrality of the optimal solution of therelaxed problem improves with the depth of the branch Thus it is possible to connectthe last two rules on the following way The current bound is abandoned and thealgorithm selects the best bound is the improvement based on estimates is above acertain threshold
2635 The selection of the branching variable
In selecting the branching variable again both the fractional part of the non-integervariables and the fast bounds have critical role A further factor can be the infor-mation obtained from the user
Selection based on the fractional part The most significant change can beexpected from that variable which is farthest from integers as the cuts defining thetwo new branches cut the most As the measure of fractionality is minfj 1 minus fjthe rule suggest to choose the branching variable xp as
p = argmaxminfj 1minus fj | j isin I
Selection based on fast bounds Upper bounds are
zr minus Plp and zr minus Pup
in the lower and upper branches of branch r if the branching variable is xpHere are five possible selection criteria
maxp
maxzr minus Plp zr minus Pup (2666)
maxp
minzr minus Plp zr minus Pup (2667)
minp
maxzr minus Plp zr minus Pup (2668)
minp
minzr minus Plp zr minus Pup (2669)
maxp| Plp minus Pup | (2670)
Which one can be offered for a BampB algorithmNotice that
maxzr minus Plp zr minus Pupis a correct upper bound of branch r as it has been mentioned earlier Thus (2666)selects according to the most inaccurate upper bound It is obviously not good(2668) makes just the opposite it selects the variable giving the most accuratebound On the other hand
minzr minus Plp zr minus Pup (2671)
263 Mixed integer programming with bounded variables 1249
is the upper bound in the worse one of the two subbranches The interest of thealgorithm is that it will be fathomed without explicit investigation ie the boundof this subbranch will be less than the objective function value of an integer feasiblesolution Thus it is good if (2671) is as small as possible Hence (2669) is a goodstrategy and (2667) is not Finally (2670) tries to separate the good and low qualityfeasible solutions The conclusion is that (2669) and (2670) are the two best onesand (2668) is still applicable but (2666) and (2667) must be avoided
Priority rule Assume that the numerical problem (2631)-(2635) is the modelof an industrial problem Then the final user is the manager andor expert whomust apply the decisions coded into the optimal solution The expert may knowthat which factors (decisions) are the most critical ones from the point of view ofthe managerial problem and the industrial system The variables belonging to thesefactors may have a special importance Therefore it has sense if the user may definea priority order of variables Then the first non-integer variable of the order can beselected as branching variable
2636 The numerical example is revisited
The solution of the problem
max x0 = 2x1 + x2
3x1 minus 5x2 le 03x1 + 5x2 le 15
x1 x2 ge 0x1 x2 is integer
(2636)
has been analyzed from geometric point of view in subsection 2631 Now the above-mentioned methods will be applied and the same course of solution will be obtained
After introducing the slack variables x3 and x4 the (primal) simplex methodgives the equivalent form (2638) of the equations and the objective function
max x0 = 0x1 + 0x2 minus 730x3 minus 13
30x4 + 132
x1 + 0x2 + 16x3 + 1
6x4 = 52
0x1 + x2 minus 110x3 + 1
10x4 = 32
x1 x2 x3 x4 ge 0
(2638)
Hence it is clear that the solution x1 = 52 and x2 = 3
2 (2638) gives the followingoptimal dual simplex tableaux
minusx3 minusx4
x0 132 730 1330x1 52 16 16x2 32 minus110 110x3 0 minus1 0x4 0 0 minus1
1250 26 The Branch and Bound Method
The first two branches were defined by the inequalities x1 le 2 and x1 ge 3 Thesecond one is an empty branch The algebraic evidence of this fact is that there isno negative element in the row of x1 thus it is not possible to find a pivot elementfor the dual simplex method after introducing the cut Now it will be shown in adetailed way Let s be the appropriate slack variable ie the cut introduced in theform
x1 minus s = 3 s ge 0
The new variable s must be expressed by the non-basic variables ie by x3 and x4
3 = x1 minus s =52minus 1
6x3 minus
16x4 minus s
Hences = minus1
2+
16
(minusx3) +16
(minusx4)
When this row is added to the dual simplex tableaux it is the only row having anegative constant term but there is no negative coefficient of any non-basic variableproving that the problem is infeasible Notice that the sign of a coefficient is animmediate consequence of the sign of the coefficient in the row of x1 ie it is notnecessary to carry out the calculation of the row of s and it is possible to concludeimmediately that the branch is empty
The fractional part f1 equals 12 Hence the fast bound (2655) of the lower branch
defined by x1 le 2 is
12
min 7
3016
133016
=710
It means that the fast upper bound in the branch is 132-710=58 The bound canbe rounded down to 5 as the objective function is integer valued
Let x5 be the slack variable of the cut x1 le 2 ie x1 + x5 = 2 Hence
x5 =12minus(
minus16
)
(minusx3) minus(
minus16
)
(minusx4)
If it is added to the simplex tableaux then the pivot element is d53 After the firstpivot step the tableaux becomes optimal It is
minusx5 minusx4
x0 295 75 15x1 2 1 0x2 95 minus35 15x3 3 minus6 1x4 0 0 minus1x5 0 minus1 0
(2672)
Notice that the optimal value is 58 ie exactly the same what was provided bythe fast bound The reason is that the fast bound gives the value of the objectivefunction after the first pivot step In the current case the first pivot step immediately
263 Mixed integer programming with bounded variables 1251
produced the optimal solution of the relaxed problemx2 is the only variable having non-integer value in simplex tableaux Thus the
branching must be done according to x2 The two new cuts defining the branchesare x2 le 1 and x2 ge 2 There are both positive and negative coefficients in the rowof x2 thus both the lower and upper branches exist Moreover
Pl2 =45times 15
15=
45 Pu2 =
15times 75
35=
715
Thus the continuous upper bound is higher on the upper branch therefore it isselected first for further branching
The constraintx2 minus x6 = 2 x6 ge 0
are added to the problem By using the current simplex tableaux the equation
x6 = minus15minus 3
5(minusx5) +
15
(minusx4)
is obtained It becomes the last row of the simplex tableaux In the first pivotingstep x6 enters the basis and x5 leaves it The first tableaux is immediately optimaland it is
minusx6 minusx4
x0 163 73 23x1 53 53 13x2 2 minus1 0x3 5 minus10 minus1x4 0 0 minus1x5 13 minus53 minus13x6 0 minus1 0
Here both x1 and x5 are integer variables having non-integer values Thus branchingis possible according to both of them Notice that the upper branch is empty in thecase of x1 while the lower branch of x5 is empty as well x1 is selected for branchingas it is the variable of the original problem Now
Pl1 =23
min
7353
2313
=1415
On the other hand the bound can be improved in accordance with (2664) as d16 gt 1ie the coefficient of minusx6 may be 23 instead of 53 It means that the inequality
x1 + x6 le 1
is claimed instead of
x1 le 1
It is transferred to the form
x1 + x6 + x7 = 1
1252 26 The Branch and Bound Method
Hence
x7 = minus23minus 2
3(minusx6)minus 1
3(minusx4)
The improved fast bound is obtained from
Pprime
l1 =23
min
72 2
=43
It means that the objective function can not be greater than 4 After the first pivotingthe simplex tableau becomes
minusx6 minusx7
x0 4 1 2x1 1 1 1x2 2 minus1 0x3 7 minus8 minus3x4 2 2 minus3x5 1 minus1 minus1x6 0 minus1 0x7 0 0 minus1
giving the feasible solution x1 = 1 and x2 = 2 with objective function value 4There is only one unfathomed branch which is to be generated from tableaux
(2672) by the constraint x2 le 1 Let x8 be the slack variable Then the equation
1 = x2 + x8 =95minus 3
5(minusx5) +
15
(minusx4) + x8
gives the cut
x8 = minus45
+35
(minusx5)minus 15
(minusx4)
to be added to the tableaux After two pivoting steps the optimal solution is
minusx3 minusx6
x0 133 23 133x1 53 13 53x2 1 0 1x3 5 minus1 0x4 5 minus1 minus10x5 13 minus13 minus53x6 0 0 minus1
Although the optimal solution is not integer the branch is fathomed as the upperbound is under 5 ie the branch can not contain a feasible solution better than thecurrent best known integer solution Thus the method is finished
Exercises263-1 Show that the rule of the choice of the integers microj (2664) is not necessarilyoptimal from the point of view of the object function (Hint Assume that variablexj enters into the basis in the first pivoting Compare the changes in the objectivefunction value if its coefficient is minusfj and fj minus 1 respectively)
264 On the enumeration tree 1253
264 On the enumeration tree
One critical point of BampB is the storing of the enumeration tree When a branch isfathomed then even some of its ancestors can become completely fathomed providedthat the current branch was the last unfathomed subbranch of the ancestors Theancestors are stored also otherwise it is not possible to restore the successor AsBampB uses the enumeration tree on a flexible way it can be necessary to store a largeamount of information on branches It can causes memory problems On the otherhand it would be too expensive from the point of view of calculations to check theancestors every time if a branch becomes fathomed This section gives some ideashow to make a trade-off
The first thing is to decide is that which data are describing a branch Thereare two options The first one is that all necessary informations are stored for eachbranch It includes all the branching defining constraints In that case the sameconstraint is stored many times because a branch on a higher level may have manysubbranches As matter of fact the number of branches is very high in the case oflarge scale problems thus the memory required by this solution is very high
The other option is that only those informations are stored which are necessaryto the complete reconstruction of the branch These ones are
bull the parent branch ie the branch from which it was generated directly
bull the bound of the objective function on the branch
bull the index of the branching variable
bull the branch defining constraint of the branching variable
For technical reasons three other attributes are used as well
bull a Boolean variable showing if the branch has been decomposed into subbranches
bull another Boolean variable showing if any unfathomed subbranch of the branchexists
bull and a pointer to the next element in the list of branches
Thus a branch can be described by a record as follows
acute
record Branchbegin
Parent BranchBound integerVariable integerValue integerDecomposition BooleanDescendant Booleansuc Branch
end
The value of the Parent attribute is none if and only if the branch is the initialbranch ie the complete problem It is the root of the BampB tree The reconstruction
1254 26 The Branch and Bound Method
of the constraints defining the particular branch is the simplest if it is supposed thatthe branches are defined by the fixing of a free variable Assume that Node is avariable of type Branch At the beginning its value is the branch to be reconstructedThen the algorithm of the reconstruction is as follows
Branch-Reconstruction
1 while Node 6= none2 do x[NodeVariable] larr NodeValue3 4 Node larr NodeParent5 return Node
The value of a previously fixed variable is set to the appropriate value in row2 Further operations are possible (row 4) Node becomes its own parent branch inrow 5 If it is none then the root is passed and all fixings are done
Sometimes it is necessary to execute some operations on all elements of thelist L The suc attribute of the branches point to the next element of the listThe last element has no next element therefore the value of suc is none in thiscase The procedure of changing all elements is somewhat similar to the BranchReconstruction procedure The head of the list L is Tree ie the first element ofthe list is Treesuc
BampB-List
1 Node larr Treesuc2 while Node 6=none3 4 Node larr Nodesuc5 return Node
The loop runs until there is no next element The necessary operations areexecuted in row 4 The variable Node becomes the next element of the list in row5 To insert a new branch into the list is easy Assume that it is NewNode of typeBranch and it is to be inserted after Node which is in the list Then the necessarytwo commands are
NewNodesuc larr NodesucNodesuc larr NewNode
If the branches are not stored as objects but they are described in long arraysthen the use of attribute suc is superflous and instead of the procedure BampB Lista for loop can be applied
The greatest technical problem of BampB from computer science point of viewis memory management Because branches are created in enormous large quantitythe fathomed branches must be deleted from the list time to time and the memoryoccupied by them must be freed It is a kind of garbage collection It can be done inthree main steps In the first one value false is assigned to the attribute Descendant
265 The use of information obtained from other sources 1255
of all elements of the list In the second main step an attribute Descendant is changedto true if and only if the branch has unfathomed descendant(s) In the third stepthe unnecessary branches are deleted It is assumed that there is a procedure Outwhich gets the branch to be deleted as a parameter and deletes it and frees the partof the memory
Garbage-Collection
1 Node larr Treesuc2 while Node 6= none3 NodeDescendant larr False4 Node larr Nodesuc5 Node larr Treesuc6 while Node 6= none7 do if not NodeDecomposition and NodeBound gt z8 then Pont larr NodeParent9 while Pont 6= none do
10 PontDescendant larr True11 Pont larr PontParent12 Node larr Nodesuc13 Node larr Treesuc14 while Node 6= none do15 Pont larr Nodesuc16 if (not NodeDescendant and NodeDecomposition) or NodeBound le z17 then Out(Node)18 Node larr Pont19 return
265 The use of information obtained from othersources
The method can be sped up by using information provided by further algorithmictools
2651 Application of heuristic methods
The aim of the application of heuristics methods is to obtain feasible solutions Fromtheoretical point of view to decide if any feasible solution exists is NP-complete aswell On the other hand heuristics can produce feasible solutions in the case of themajority of the numerical problems The methods to be applied depend on the natureof the problem in question ie pure binary bounded integer mixed integer problemsmay require different methods For example for pure integer problems local searchand Lagrange multipliers can work well Lagrange multipliers also provide upperbound (in the case of maximization) of the optimal value
1256 26 The Branch and Bound Method
If a feasible solution is known then it is immediately possible to disregardbranches based on their bounds See row 15 of algorithm Branch and BoundThere the branches having not good enough bounds are automatically eliminatedIn the case of pure binary problem an explicit objective function constraint can givea lot of consequences as well
2652 Preprocessing
Preprocessing means to obtain information on variables and constraints based onalgebraic constraints and integrality
For example if the two constraints of problem (2636) are summed up then theinequality
6x1 le 15
is obtained implying that x1 le 2Let
gi(x) le bi (2673)
be one of the constraints of problem (2614)-(2616) Many tests can be based on thefollowing two easy observations
1 If the maximal value of the left-hand side of (2673) of x isin X is not greaterthan the right-hand side of (2673) then the constraint is redundant
2 If the minimal value of the left-hand side of (2673) if x isin X is greater thanthe right-hand side of (2673) then it is not possible to satisfy the constraintie the problem (2614)-(2616) has no feasible solution
If under some further restriction the second observation is true then the restrictionin question can be excluded A typical example is that certain variables are supposedto have maximalminimal possible value In this way it is possible to fix a variableor decrease its range
Lagrange relaxation can be used to fix some variables too Assume that theoptimal value of Problem (2622) and (2616) is ν(L(λ | xj = δ)) under the furthercondition that xj must take the value δ If z is the objective function value of aknown feasible solution and z gt ν(L(λ | xj = δ)) then xj can not take value δFurther methods are assuming that the LP relaxation of the problem is solved andbased on optimal dual prices try to fix the values of variables
266 Branch and Cut
Branch and Cut (BampC) in the simplest case is a BampB method such that the acertain kind of information is collected and used during the whole course of thealgorithm The theoretical background is based on the notion of integer hull
266 Branch and Cut 1257
Definition 269 Let
P = x | Ax le b
be a polyhedral set where A is an mtimes n matrix x and b are n and m dimensionalvectors All elements of A and b are rationals The convex hull of the integer pointsof P is called the integer hull of P ie it is the set
conv(P cap Zn)
The integer hull of the polyhedral set of problem (2636) is the convex hull ofthe points (00) (03) (12) and (11) as it can be seen on Figure 242 Thus thedescription of the integer hull as a polyhedral set is the inequality system
x1 ge 0 x1 + x2 le 3 x1 le 1 x1 minus x2 le 0
Under the conditions of the definition the integer hull is a polyhedral set tooIt is a non-trivial statement and in the case of irrational coefficients it can be nottrue If the integer hull is known ie a set of linear inequalities defining exactly theinteger hull polyhedral set is known then the integer programming problem can bereduced to a linear programming problem Thus problem (2636) is equivalent to theproblem
max x0 = 2x1 + x2
x1 ge 0x1 + x2 le 3x1 le 1x1 minus x2 le 0
(2674)
As the linear programming problem easier to solve than the integer programmingproblem one may think that it worth to carry out this reduction It is not completelytrue First of all the number of the linear constraint can be extremely high Thusgenerating all constraints of the integer hull can be more difficult than the solutionof the original problem Further on the constraints determining the shape of theinteger hull on the side opposite to the optimal solution are not contributing to thefinding of the optimal solution For example the optimal solution of (2674) will notchange if the first constraint is deleted and it is allowed both x1 and x2 may takenegative values
On the other hand the first general integer programming method is the cuttingplane method of Gomory Its main tool is the cut which is based on the observationthat possible to determine linear inequalities such that they cut the non-integeroptimal solution of the current LP relaxation but they do not cut any integer feasiblesolution A systematic generation of cuts leads to a finite algorithm which finds anoptimal solution and proves its optimality if optimal solution exist otherwise itproves the non-existence of the optimal solution From geometrical point of view theresult of the introducing of the cuts is that the shape of the polyhedral set of the lastLP relaxation is very similar to the integer hull in the neighborhood of the optimalsolution
1258 26 The Branch and Bound Method
There is the generalization of Gomoryrsquos cut called Chvaacutetal (or Chvaacutetal-Gomory)cut If the two inequalities of (2636) are summed such that both have weight 1
6 thenthe constraint
x1 le 25
is obtained As x1 must be integer the inequality
x1 le 2 (2675)
follows immediately It is not an algebraic consequence of the original constraintsTo obtain it the information of the integrality of the variables had to be used Butthe method can be continued If (2675) has weight 2
5 and the second constraint of(2636) has weight 1
5 then
x1 + x2 le 38
is obtained implying
x1 + x2 le 3
If the last inequality has weight 58 and the first inequality of (2636) has weight 1
8then the result is
x1 le158
implying
x1 le 1
Finally the integer hull is obtained In general the idea is as follows Assume that apolyhedral set is defined by the linear inequality system
Ax le b (2676)
Let y ge 0macr
be a vector such that AT y is an integer vector and yT b is a nonintegervalue Then
yT Ax le byT bc
is a valid cut ie all integer points of the polyhedral set satisfy it As a matter offact it can be proven that a systematic application of the method creates a completedescription of the integer hull after finite many steps
The example shows that Gomory and Chvaacutetal cuts can help to solve a problemOn the other hand they can be incorporated in a BampB frame easily But in the verygeneral case it is hopeless to generate all effective cuts of this type
The situation is significantly different in the case of many combinatorial prob-lems There are many theoretical results known on the type of the facet definingconstraints of special polyhedral sets Here only one example is discussed It is the
266 Branch and Cut 1259
Traveling Salesperson Problem (TSP) A salesman must visit some cities and atthe end of the tour he must return to his home city The problem is to find a tourwith minimal possible length TSP has many applications including cases when thecities are products or other objects and the ldquodistancerdquo among them doesnrsquot satisfythe properties of the geometric distances ie symmetry and triangle inequality maybe violated
The first exact mathematical formulation of the problem is the so-called Dantzig-Fulkerson-Johnson (DFJ) model DFJ is still the basis of the numerical solutionsAssume that the number of cities is n Let dij the distance of the route from city ito city j (1 le i j le n i 6= j) DFJ uses the variables xij such that
xij =
1 if the salesman travel from city i to city j0 otherwise
The objective function is the minimization on the total travel length
minnsum
i=1
sum
i6=j
dijxij (2677)
The set of the constraints consists of three parts The meaning of the first part isthat the salesman must travel from each city to another city exactly once
nsum
j=1j 6=i
xij = 1 i = 1 n (2678)
The second part is very similar It claims that the salesman must arrive to each cityfrom somewhere else again exactly once
nsum
i=1i 6=j
xij = 1 j = 1 n (2679)
Constraints (2678) and (2679) are the constraints of an assignment problem Takinginto account that the variables must be binary Problem (2677)-(2679) is really anassignment problem They donrsquot exclude solutions consisting of several smaller toursFor example if n = 6 and x12 = x23 = x31 = 1 and x45 = x56 = x64 = 1 then allother variables must be zero The solution consists of two smaller tours The firstone visits only cities 1 2 and 3 the second one goes through the cities 4 5 and 6The small tours are called subtours in the language of the theory of TSP
Thus further constraints are needed which excludes the subtours They are calledsubtour elimination constraints There are two kinds of logic how the subtours canbe excluded The first one claims that in any subset of the cities which has at leasttwo elements but not the complete set of the cities the number of travels must beless than the number of elements of the set The logic can be formalized as follows
forallS sub 1 2 n 1 le |S| le nminus 1 sum
iisinS
sum
jisinSj 6=i
xij le |S| (2680)
1260 26 The Branch and Bound Method
The other logic claims that the salesman must leave all such sets Let S =1 2 n S Then the subtour elimination constraints are the inequalities
forallS sub 1 2 n 1 le |S| le nminus 1 sum
iisinS
sum
jisinS
xij ge 1 (2681)
The numbers of the two types of constraints are equal and exponential Althoughthe constraints (2678)ndash(2680) or (2678) (2679) and (2681) are satisfied by onlybinary vectors being characteristic vectors of complete tours but the polyhedral setof the LP relaxation is strictly larger than the integer hull
On the other hand it is clear that it is not possible to claim all of the subtourelimination constraints in the real practice What can be done It is possible toclaim only the violated once The difficulty is that the optimal solution of the LPrelaxation is a fractional vector in most of the cases and that subtour eliminationconstraint must be found which is violated by the fractional solution provided thatsuch constraint exists as the subtour elimination constraints are necessary to thedescription of the integer hull but further constraints are needed too Thus it ispossible that there is no violated subtour elimination constraint but the optimalsolution of the LP relaxation is still fractional
To find a violated subtour elimination constraint is equivalent to the findingof the absolute minimal cut in the graph which has only the edges having positiveweights in the optimal solution of the relaxed problem If the value of the absoluteminimal cut is less than 1 in the directed case or less than 2 in the non-directed casethen such a violated constraint exists The reason can be explained based on thesecond logic of the constraints If the condition is satisfied then the current solutiondoesnrsquot leaves at least one of the two sets of the cut in enough number There aremany effective methods to find the absolute minimal cut
A general frame of the numerical solution of the TSP is the following In a BampBframe the calculation of the lower bound is repeated until a new violated subtourelimination constraint is obtained that is the new inequality is added to the relaxedproblem and the LP optimization is carried out again If all subtour eliminationconstraints are satisfied and the optimal solution of the relaxed problem is still non-integer then branching is made according to a fractional valued variable
The frame is rather general The violated constraint cuts the previous optimalsolution and reoptimization is needed Gomory cuts do the same for the generalinteger programming problem In the case of other combinatorial problems specialcuts may work if the description of the integer hull is known
Thus the general idea of BampC is that a cut is generated until it can be foundand the improvement in the lower bound is great enough Otherwise branching ismade by a non-integer variable If the cut generation is made only at the root of theenumeration tree then the name of the method is Cut and Branch (CampB) If a cutis generated in a branch then it is locally valid in that branch and in its successorsThe cuts generated at the root are valid globally ie in all branches In some casesee in binary optimization it is possible to modify it such that it is valid in theoriginal problem too
For practical reasons the type of the generated cut can be restricted It is thecase in TSP as the subtour elimination constraints can be found relatively easily
267 Branch and Price 1261
267 Branch and Price
The Branch and Price method is the dual of BampC in a certain sense If a problemhas very large number of variables then it is often possible not to work explicitelywith all of them but generate only those which may enter the basis of the LP re-laxation This is column generation and is based on the current values of the dualvariables called shadow prices Similarly to BampC the type of the generated columnsis restricted If it is not possible to find a new column then branching is made
Problems
26-1 Continuous Knapsack ProblemProve Theorem 261 (Hint Let x be a feasible solution such that there are twoindices say j and k such that 1 le j lt k le n and xj lt 1 and xk gt 0 Show thatthe solution can be improved)26-2 TSPrsquos relaxationDecide if the Assignment Problem can be a relaxation of the Traveling SalespersonProblem in the sense of definition 265 Explain your solution regardless that youranswer is YES or NO26-3 Infeasibility testBased on the the second observation of Subsection 2652 develop a test for theinfeasibility of a linear constraint of binary variables26-4 Mandatory fixingBased on the previous problem develop a test for the mandatory fixing of binaryvariables satisfying a linear constraint
Chapter Notes
The principle of BampB first appears in [138] It solves problems with bounded integervariables The fast bounds were introduced in [16] and [229] A good summary of thebounds is [68] To the best knowledge of the author of this chapter the improvementof the fast bounds appeared first in [236]
BampB can be used as an approximation scheme too In that case a branch canbe deleted even in the case if its bound is not greater than the objective functionvalue of the current best solution plus an allowed error [107] showed that there areclasses such that the approximate method requires more computation than to solvethe problem optimally BampB is very suitable for parallel processing This issue isdiscussed in [35]
Based on the theoretical results of [147] a very effective version of BampC methodwas developed for pure binary optimization problem by [217] and independently [10]Especially Egon Balas and his co-authors could achieve a significant progress Theirmethod of lifting cuts means that a locally generated cut can be made globally validby solving a larger LP problem and modify the cut according to its optimal solution
1262 26 The Branch and Bound Method
The first integer programming method to solve an IP problem with general ienon-bounded integer variables is Ralph Gomoryrsquos cutting plane method [80] In acertain sense it is still the only general method Strong cuts of integer programmingproblems are discussed in [11] The surrogate constraint (2618) has been introducedby [79] The strength of the inequality depends on the choice of the multipliers λiA rule of thumb is that the optimal dual variables of the continuous problem give astrong inequality provided that the original problem is linear
The DFJ model of TSP appeared in [50] It was not only an excellent theoreticalresult but is also an enormous computational effort as the capacities and speed ofthat time computers were far above the recent ones One important cut of the TSPpolyhedral set is the so-called comb inequality The number of edges of a completetour is restricted in a special subgraph The subgraph consists of a subset of citiescalled hand and odd number of further subsets of cities intersecting the hand Theyare called teeth and their number must be at least three Numerical problems of TSPare exhaustively discussed in [209]
A good summary of Branch and Price is [14]
27 Comparison Based Ranking
In the practice often appears the problem how to rank different objects Researchersof these problems often mention different applications eg in biology Landau [139]in chemistry Hakimi [87] in networks Kim Toroczkai Mikloacutes Erdős and Szeacutekely[127] in business Bozoacuteki Fuumlloumlp Keacuteri Poesz and Roacutenyai [27 28 134] in sportsIvaacutenyi Pirzada and Zhou [116 200]
An often used method is the comparison of twomdashand simetimes morendash objectsin all possible manner and distribution some amount of points among the comparedobjects
In this chapter we introduce a general model for such ranking and study someconnected problems
271 Introduction to supertournaments
Let n m be positive integers a = (a1 a2 am) b = (b1 b2 bm) and k =(k1 k2 km) vectors of nonnegative integers with ai le bi (i = 1 2 m) and0 lt k1 lt k2 lt middot middot middot lt km
Then an (abkm n)-supertournament is an x times n sized matrix M whosecolumns correspond to the players of the tournament (they represent those objectswhich we wish to rank) and the lines correspond to the comparisons of the objectsThe permitted elements of M belong to the set 0 1 2 bmax cup lowast wheremij = lowast means that the player Pj is not a participants of the match correspondingto the i-th line mij = k means that Pj received k points in the match correspondingto the i-th line and brmmax = max1leilen bi
Using the terminology of the sports a supertournament can combine the math-ches of different sports For example in Hungary there are popular chess-bridgechess-tennis and tennis-bridge tournaments
A sport is characterized by the set of the permitted results For example in tennisthe set of permitted results is Stennis = 0 1 for chess is the set Schess = 0 2 1 1 for football is the set Sfootball = 0 3 1 1 and in the Hungarian card gamelast trick is Slast trick = (0 1 1) (0 0 2) There are different possible rules for anindividual bridge tournament eg Sbridge = (0 2 2 2) (1 1 1 3)
1264 27 Comparison Based Ranking
matchplayer P1 P2 P3 P4
P1-P2 1 1 P1-P3 0 2 P1-P4 0 2P2-P3 0 2 P2-P4 0 2P3-P4 lowast 1 1
P1-P2-P3 1 1 0 P1-P2-P4 1 0 2P1-P3-P4 1 1 0P2-P3-P4 0 0 2
P1-P2-P3-P4 3 1 1 1
Total score 7 3 8 10
Figure 271 Point matrix of a chess+last trick-bridge tournament with n = 4 players
The number of participants of a match of a given sport Si is denoted by kithe minimal number of the distributed points in a match is denoted by ai and themaximal number of points is denoted by bi
If a supertournament consists of only the matches of one sport then we use a band k instead of vectors a b and k and omit the parameter m When the numberof the players is not important then the parameter n is also omitted
According to this notations chess is a (222)-sport while football is a (232)-sport If the points can be distributed according to arbitrary integer partition thenthe given sport is called complete otherwise it is called incomplete
Since a set containing n elements has(
nk
)
k-element subsets an (a b k)-tournament consists of
(
nk
)
matches If all matches are played then the tournamentis finished otherwise it is partial
In this chapter we deal only with complete and finished tournamentsFigure 271 contains the results of a full and finished chess+last trick+bridge
supertournament In this example n = 4 a = b = (2 2 6)k = (2 3 4) and x =(
42
)
+(
43
)
+(
44
)
= 11In this example the score vector of the given supertournament is (7 3 8 10) and
its score sequence is (3 7 8 10)In this chapter we investigate the problems connected with the existence and con-
struction of different types of supertournaments having prescribed score sequencesAt first we give an introduction to (a b 2)-tournaments (Section 272) then sum-
marize the simplest results on (112)-tournaments (Section 273) then for (a a 2)-tournaments (Section 274) and for (a b 2)-tournaments (Section 275)
In Section 277 we deal with imbalance sequences and in Section 278 withsupertournaments
Exercises271-1 Design
272 Introduction to (a b 2)-tournaments 1265
271-2 How many
272 Introduction to (a b 2)-tournaments
In this section we consider the traditional round-robin tournament in which eachpair of players plays a match and the winner gets one point the loser gets zero pointA natural way to represent such a tournament using a directed graph in which everypair of vertices is connected with a directed arc
Let D = (d1 d2 dn) be a sequence of the number of the points of the players(the sequence of the out-degrees of the vertices Then s is called the score vectorof the given tournament T If we form a nondecreasing sequence q = (q1 q2 qn)from the elements of s then we get the score sequence of T
Clearly 0 le si le nminus1 Further no two scores can be zero and no two scores canbe n minus 1 Tournament score sequences have also been called score structures [139]and score lists [25]
We use u rarr v to denote an arc from player (vertex) u to v and also to denotethe fact that u wins against v
Now we give the characterization of score sequences of tournaments which is dueto Landau [139] This result has attracted quite a bit of attention as nearly a dozenof different proofs appear in the literature Early proofs tested the readers patiencewith special choices of subscripts but eventually such gymnastics were replaced bymore elegant arguments Many of the existing proofs are discussed in a survey byReid [205] and the proof we give here is due to Thomassen [227] Further two newproofs can be found in [83]
Let a b (b ge a) and n (n ge 2) be nonnegative integers and let T (a b n) bethe set of such generalized tournaments in which every pair of distinct players isconnected at most with b and at least with a arcs In [114] we gave a necessaryand sufficient condition to decide whether a given sequence of nonnegative integersD = (d1 d2 dn) can be realized as the outdegree sequence of a T isin T (a b n)Extending the results of [114] we show that for any sequence of nonnegative integersD there exist f and g such that some element T isin T (g f n) has D as its outdegreesequence and for any (a b n)-tournament T prime with the same outdegree sequence Dhold a le g and b ge f We propose a Θ(n) algorithm to determine f and g and anO(dnn
2) algorithm to construct a corresponding tournament T Let a b (b ge a) and n (n ge 2) be nonnegative integers and let T (a b n) be
the set of such generalised tournaments in which every pair of distinct players isconnected at most with b and at least with a arcs The elements of T (a b n) arecalled (a b n)-tournaments The vector D = (d1 d2 dn) of the outdegreesof T isin T (a b n) is called the score vector of T If the elements of D are innondecreasing order then D is called the score sequence of T
An arbitrary vector D = (d1 d2 dn) of nonnegative integers is called graph-ical vector iff there exists a loopless multigraph whose degree vector is D and Dis called digraphical vector (or score vector) iff there exists a loopless directedmultigraph whose outdegree vector is D
A nondecreasingly ordered graphical vector is called graphical sequence and a
1266 27 Comparison Based Ranking
nondecreasinly ordered digraphical vector is called digraphical sequence (or scoresequence)
The number of arcs of T going from player Pi to player Pj is denoted by mij (1 lei j le n) and the matrix M = [1 n 1 n] is called point matrix or tournamentmatrix of T
In the last sixty years many efforts were devoted to the study of both types ofvectors resp sequences Eg in the papers [21 59 74 82 87 89 90 95 124 175215 218 222 231 239] the graphical sequences while in the papers [1 8 12 2133 66 77 78 83 85 89 96 128 129 139 149 158 159 160 164 165 170 176205 208 211 234 240 242] the score sequences were discussed
Even in the last two years many authors investigated the conditions when D isgraphical (eg [15 24 32 43 70 71 75 72 100 101 121 127 130 131 152 184187 210 230 235 237 244]) or digraphical (eg [17 92 114 125 132 146 163201 197 196 198 199 212 213 227 246])
In this chapter we deal only with directed graphs and usually follow the termi-nology used by K B Reid [208 206] If in the given context a b and n are fixedor non important then we speak simply on tournaments instead of generalized or(a b n)-tournaments
We consider the loopless directed multigraphs as generalized tournaments inwhich the number of arcs from vertexplayer Pi to vertexplayer Pj is denoted bymij where mij means the number of points won by player Pi in the match withplayer Pj
The first question how one can characterise the set of the score sequences of the(a b n)-tournaments Or with another words for which sequences D of nonnega-tive integers does exist an (a b n)-tournament whose outdegree sequence is D Theanswer is given in Section
If T is an (a b n)-tournament with point matrix M = [1 n 1 n] then letE(T ) F (T ) and G(T ) be defined as follows E(T ) = max1leijlen mij F (T ) =max1leiltjlen(mij +mji) and g(T ) = min1leiltjlen(mij +mji) Let ∆(D) denote theset of all tournaments having D as outdegree sequence and let e(D) f(D) and g(D)be defined as follows e(D) = min E(T ) | T isin ∆(D) f(D) = min F (T ) | T isin∆(D) and g(D) = max G(T ) | T isin ∆(D) In the sequel we use the shortnotations E F G e f g and ∆
Hulett Will and Woeginger [101 241] Kapoor Polimeni and Wall [123] andTripathi et al [232 230] investigated the construction problem of a minimal sizegraph having a prescribed degree set [202 243] In a similar way we follow a mini-max approach formulating the following questions given a sequenceD of nonnegativeintegers
bull How to compute e and how to construct a tournament T isin ∆ characterised bye In Section 2752 a formula to compute e and an algorithm to construct acorresponding tournament are presented
bull How to compute f and g In Section 2754 an algorithm to compute f and g isdescribed
bull How to construct a tournament T isin ∆ characterised by f and g In Section276 an algorithm to construct a corresponding tournament is presented and
273 Existence of a (1 1 2)-tournament with prescribed score sequence 1267
analysed
We describe the proposed algorithms in words by examples and by the pseu-docode used in [47]
In the following sections we characterize the score sequences of (1 1 2)-tournaments in Section then the score sequences of (a a 2)-tournaments in Sec-tion In Section we show that for arbitrary score sequence d we can choosesuitable a and b such that there exists an (a b 2)-tournament whose score sequenceis d
273 Existence of a (1 1 2)-tournament withprescribed score sequence
Now we give the characterization of score sequences of tournaments which is dueto Landau [139] This result has attracted quite a bit of attention as nearly a dozendifferent proofs appear in the literature Early proofs tested the readers patiencewith special choices of subscripts but eventually such gymnastics were replaced bymore elegant arguments Many of the existing proofs are discussed in a survey by[205] The proof we give here is due to Thomassen [227] Further two new proofscan be found in in the paper due to Griggs and Reid [83]
Theorem 271 (Landau 1953) A sequence of nonnegative integers D = (d1 d minus2 dn) is the score vector of a (1 1 n)-tournament if and only if for each subsetI isin 1 2 n
sum
iisinI
si ge( |I|
2
)
(271)
with equality when |I| = n
This is a nice necessary and sufficient condition but its direct application canrequire the test of exponential number of subsets
If instead of the score vector we consider the nondecreasingly ordered scoresequence D = (d minus 1 d2 dn) then due to the monotonity d1 le d2 le theinequalities ( called Landau inequalities are equivalent to
ksum
i=1
dminus i ge(
k
2
)
(272)
for 1 le k le n with equality for k = nProof Necessity If a nondecreasing sequence of nonnegative integers D is thescore sequence of an (1 1 n)-tournament T then the sum of the first k scores inthe sequence counts exactly once each arc in the subtournamnent W induced byv1 v2 vk plus each arc from W to T -W Therefore the sum is at least(
k(kminus1)2
)
the number of arcs in W Also since the sum of the scores of the ver-
tices counts each arc of the tournament exactly once the sum of the scores is the
1268 27 Comparison Based Ranking
total number of arcs that is(
n(nminus1)2
)
Sufficiency Thomassen 1981) Let n be the smallest integer for which there isa nondecreasing sequence S of non-negative integers satisfying Landaursquos conditions() but for which there is no (1 1 n)-tournament with score sequence S Amongall such S pick one for which s1 is as small as possible
First consider the case where for some k lt n
ksum
i=1
si =(
k
2
)
By the minimality of n the sequence S1 = [s1 sminus2 sk] is the score sequenceof some tornament T1 Further
msum
i=1
(sk+i minus k) =m+ksum
i=1
si minusmk ge(
m+ k
2
)
minus(
k
2
)
minusmk =(m
2
)
for each m 1 le m le n minus k with the equality when m = n minus k Therefore by theminimality of n the sequence S2 = [sk+1minusk sk+2minusk snminusk] is the score sequenceof some tournament T2 By forming the disjoint union of T1 and T2 and adding allarcs from T2 to T1 we obtain a tournament with score sequence S
Now consider the case where each inequality in () is strict when k lt n (inparticular s1 gt 0) Then the sequence S3 = [s1 minus 1 s2 snminus1 sn + 1] satisfies( and by the minimality of s1 S3 is the score sequence of some tournament T3Let u and v be the vertices with scores sn +1 and s1minus1 respectively Since the scoreof u is larger than that of v T3 has a path P from u to v of length le 2 By reversingthe arcs of P we obtain a tournament with score sequence S a contradiction
Landaursquos theorem is the tournament analog of the Erdős-Gallai theorem forgraphical sequences [59] A tournament analog of the Havel-Hakimi theorem [95 88]for graphical sequences is the following result the proof of which can be found inthe paper of Reid and Beineke [207]
Theorem 272 Reid and beineke A non-decreasing sequence [s1 sminus2 sn]of nonnegative integers n ge 2 is the score sequence of an (1 1 n)-tournament ifand only if the new sequence
[s1 s2 sm sm+1 minus 1 snminus1 minus 1
where m = sn when arranged in nondecreasing order is the score sequence of some(1 1 nminus 1)minustournament
274 Existence of an (a a 2)-tournament withprescribed score sequence
For the (a a 2)-tournament Moon [159] proved the following extension of Landaursquostheorem
275 Existence of a tournament with arbitrary degree sequence 1269
Theorem 273 A nondecreasing sequeuence of nonnegative integers
Later kemnitz and Dulff [125] reproved this theorem
275 Existence of a tournament with arbitrary degreesequence
Since the numbers of points mij are not limited it is easy to construct a (0 dn n)-tournament for any D
Lemma 274 If n ge 2 then for any vector of nonnegative integers D = (d1d2 dn) there exists a loopless directed multigraph T with outdegree vector Dso that E le dn
Proof Let mn1 = dn and mii+1 = di for i = 1 2 n minus 1 and let the remainingmij values be equal to zero
Using weighted graphs it would be easy to extend the definition of the (a b n)-tournaments to allow arbitrary real values of a b and D The following algorithmNaive-Construct works without changes also for input consisting of real numbers
We remark that Ore in 1956 [170] gave the necessary and sufficient conditionsof the existence of a tournament with prescribed indegree and outdegree vectorsFurther Ford and Fulkerson [66 Theorem111] published in 1962 necessary and suf-ficient conditions of the existence of a tournament having prescribed lower and upperbounds for the indegree and outdegree of the vertices They results also can serveas basis of the existence of a tournament having arbitrary outdegree sequence
2751 Definition of a naive reconstructing algorithm
Sorting of the elements of D is not necessaryInput n the number of players (n ge 2)
D = (d1 d2 dn) arbitrary sequence of nonnegative integer numbersOutput M = [1 n 1 n] the point matrix of the reconstructed tournamentWorking variables i j cycle variables
Naive-Construct(nD)01 for i = 1 to n02 for j = 1 to n03 mij = 004 mn1 = dn
05 for i = 1 to nminus 106 mii+1 = di
07 return M
The running time of this algorithm is Θ(n2) in worst case (in best case too) Sincethe point matrix M has n2 elements this algorithm is asymptotically optimal
1270 27 Comparison Based Ranking
2752 Computation of e
This is also an easy question From here we suppose that D is a nondecreasingsequence of nonnegative integers that is 0 le d1 le d2 le le dn Let h = ddn(nminus1)e
Since ∆(D) is a finite set for any finite score vector D e(D) = minE(T )|T isin∆(D) exists
Lemma 275 If n ge 2 then for any sequence D = (d1 d2 dn) there exists a(0 b n)-tournament T such that
E le h and b le 2h (273)
and h is the smallest upper bound for e and 2h is the smallest possible upper boundfor b
Proof If all players gather their points in a uniform as possible manner that is
max1lejlen
mij minus min1lejlen i 6=j
mij le 1 for i = 1 2 n (274)
then we get E le h that is the bound is valid Since player Pn has to gather dn
points the pigeonhole principle [18 19 53] implies E ge h that is the bound isnot improvable E le h implies max1leiltjlen mij + mji le 2h The score sequenceD = (d1 d2 dn) = (2n(n minus 1) 2n(n minus 1) 2n(n minus 1)) shows that the upperbound b le 2h is not improvable
Corollary 276 If n ge 2 then for any sequence D = (d1 d2 dn) holds e(D) =ddn(nminus 1)e
Proof According to Lemma 275 h = ddn(nminus 1)e is the smallest upper bound fore
2753 Definition of a construction algorithm
The following algorithm constructs a (0 2h n)-tournament T having E le h for anyD
Input n the number of players (n ge 2)D = (d1 d2 dn) arbitrary sequence of nonnegative integer numbers
Output M = [1 n 1 n] the point matrix of the tournamentWorking variables i j l cycle variables
k the number of the rdquolarger parts in the uniform distribution of the points
Pigeonhole-Construct(nD)01 for i = 1 to n02 mii = 003 k = di minus (nminus 1)bdi(nminus 1)c
275 Existence of a tournament with arbitrary degree sequence 1271
04 for j = 1 to k05 l = i+ j (mod n)06 mil = ddn(nminus 1)e07 for j = k + 1 to nminus 108 l = i+ j (mod n)09 mil = bdn(nminus 1)c10 return M
The running time of Pigeonhole-Construct is Θ(n2) in worst case (in bestcase too) Since the point matrixM has n2 elements this algorithm is asymptoticallyoptimal
2754 Computation of f and g
Let Si (i = 1 2 n) be the sum of the first i elements of D Bi (i = 1 2 n)be the binomial coefficient n(nminus1)2 Then the players together can have Sn pointsonly if fBn ge Sn Since the score of player Pn is dn the pigeonhole principle impliesf ge ddn(nminus 1)e
These observations result the following lower bound for f
f ge max(lceil
Sn
Bn
rceil
lceil
dn
nminus 1
rceil)
(275)
If every player gathers his points in a uniform as possible manner then
f le 2lceil
dn
nminus 1
rceil
(276)
These observations imply a useful characterisation of f
Lemma 277 If n ge 2 then for arbitrary sequence D = (d1 d2 dn) there existsa (g f n)-tournament having D as its outdegree sequence and the following boundsfor f and g
max(lceil
S
Bn
rceil
lceil
dn
nminus 1
rceil)
le f le 2lceil
dn
nminus 1
rceil
(277)
0 le g le f (278)
Proof (277) follows from (275) and (276) (278) follows from the definition of f
It is worth to remark that if dn(n minus 1) is integer and the scores are identicalthen the lower and upper bounds in (277) coincide and so Lemma 277 gives theexact value of F
In connection with this lemma we consider three examples If di = dn = 2c(nminus1) (c gt 0 i = 1 2 n minus 1) then dn(n minus 1) = 2c and SnBn = c that isSnBn is twice larger than dn(nminus 1) In the other extremal case when di = 0 (i =1 2 n minus 1) and dn = cn(n minus 1) gt 0 then dn(n minus 1) = cn SnBn = 2c so
1272 27 Comparison Based Ranking
dn(nminus 1) is n2 times larger than SnBnIf D = (0 0 0 40 40 40) then Lemma 277 gives the bounds 8 le f le 16
Elementary calculations show that Figure 272 contains the solution with minimalf where f = 10
PlayerPlayer P1 P2 P3 P4 P5 P5 ScoreP1 mdash 0 0 0 0 0 0P2 0 mdash 0 0 0 0 0P3 0 0 mdash 0 0 0 0P4 10 10 10 mdash 5 5 40P5 10 10 10 5 mdash 5 40P6 10 10 10 5 5 mdash 40
Figure 272 Point matrix of a (0 10 6)-tournament with f = 10 for D = (0 0 0 40 40 40)
In [114] we proved the following assertion
Theorem 278 For n ge 2 a nondecreasing sequence D = (d1 d2 dn) of non-negative integers is the score sequence of some (a b n)-tournament if and only if
aBk leksum
i=1
di le bBn minus Lk minus (nminus k)dk (1 le k le n) (279)
where
L0 = 0 and Lk = max
(
Lkminus1 bBk minusksum
i=1
di
)
(1 le k le n) (2710)
The theorem proved by Moon [159] and later by Kemnitz and Dolff [125] for(a a n)-tournaments is the special case a = b of Theorem 278 Theorem 314 of[119] is the special case a = b = 2 The theorem of Landau [139] is the special casea = b = 1 of Theorem 278
2755 Definition of a testing algorithm
The following algorithm Interval-Test decides whether a given D is a score se-quence of an (a b n)-tournament or not This algorithm is based on Theorem 278and returns W = True if D is a score sequence and returns W = False otherwise
Input a minimal number of points divided after each matchb maximal number of points divided after each match
Output W logical variable (W = True shows that D is an (a b n)-tournamentLocal working variable i cycle variable
L = (L0 L1 Ln) the sequence of the values of the loss functionGlobal working variables n the number of players (n ge 2)
D = (d1 d2 dn) a nondecreasing sequence of nonnegative integersB = (B0 B1 Bn) the sequence of the binomial coefficients
275 Existence of a tournament with arbitrary degree sequence 1273
S = (S0 S1 Sn) the sequence of the sums of the i smallest scores
Interval-Test(a b)01 for i = 1 to n02 Li = max(Liminus1 bBn minus Si minus (nminus i)di)03 if Si lt aBi
04 W = False05 return W06 if Si gt bBn minus Li minus (nminus i)di
07 W larr False08 return W09 return W
In worst case Interval-Test runs in Θ(n) time even in the general case 0 lta lt b (n the best case the running time of Interval-Test is Θ(n)) It is worthto mention that the often referenced HavelndashHakimi algorithm [87 95] even in thespecial case a = b = 1 decides in Θ(n2) time whether a sequence D is digraphical ornot
2756 Definition of an algorithm computing f and g
The following algorithm is based on the bounds of f and g given by Lemma 277and the logarithmic search algorithm described by D E Knuth [132 page 410]
Input No special input (global working variables serve as input)Output b f (the minimal F )
a g (the maximal G)Local working variables i cycle variable
l lower bound of the interval of the possible values of F u upper bound of the interval of the possible values of F
Global working variables n the number of players (n ge 2)D = (d1 d2 dn) a nondecreasing sequence of nonnegative integersB = (B0 B1 Bn) the sequence of the binomial coefficientsS = (S0 S1 Sn) the sequence of the sums of the i smallest scoresW logical variable (its value is True when the investigated D is a score sequence)
1274 27 Comparison Based Ranking
MinF-MaxG
01 B0 = S0 = L0 = 0 Initialization02 for i = 1 to n03 Bi = Biminus1 + iminus 104 Si = Siminus1 + di
05 l = max(dSnBne ddn(nminus 1)e)06 u = 2 ddn(nminus 1)e07 W = True Computation of f08 Interval-Test(0 l)09 if W == True10 b = l11 go to 2112 b = d(l + u)2e13 Interval-Test(0 f)14 if W == True15 go to 1716 l = b17 if u == l + 118 b = u19 go to 3720 go to 1421 l = 0 Computation of g22 u = f23 Interval-Test(b b)24 if W == True25 alarr f26 go to 3727 a = d(l + u)2e28 Interval-Test(0 a)29 if W == True30 llarr a31 go to 3332 u = a33 if u == l + 134 a = l35 go to 3736 go to 2739 return a b
MinF-MaxG determines f and g
Lemma 279 Algorithm MinG-MaxG computes the values f and g for arbitrarysequence D = (d1 d2 dn) in O(n log(dn(n)) time
Proof According to Lemma 277 F is an element of the interval [ddn(n minus1)e d2dn(nminus1)e] and g is an element of the interval [0 f ] Using Theorem B of [132page 412] we get that O(log(dnn)) calls of Interval-Test is sufficient so the O(n)
276 Tournament with f and g 1275
run time of Interval-Test implies the required running time of MinF-MaxG
2757 Computing of f and g in linear time
Analysing Theorem 278 and the work of algorithm MinF-MaxG one can observethat the maximal value of G and the minimal value of F can be computed indepen-dently by Linear-MinF-MaxG
Input No special input (global working variables serve as input)Output b f (the minimal F )
a g (the maximal G)Local working variables i cycle variableGlobal working variables n the number of players (n ge 2)
D = (d1 d2 dn) a nondecreasing sequence of nonnegative integersB = (B0 B1 Bn) the sequence of the binomial coefficientsS = (S0 S1 Sn) the sequence of the sums of the i smallest scores
Linear-MinF-MaxG01 B0 = S0 = L0 = 0 Initialization02 for i = 1 to n03 Bi = Biminus1 + iminus 104 Si = Siminus1 + di
05 a = 0)06 b = min 2 ddn(nminus 1)e07 for i = 1 to n Computation of g08 ai =
lceil
(2Si(n2 minus n)rceil
) lt a09 if ai gt a10 a = ai
11 for i = 1 to n Computation of f12 Li = max(Liminus1 bBn minus Si minus (nminus i)di
13 bi = (Si + (nminus i)di + Li)Bi
14 if bi lt b15 b = bi
16 return a b
Lemma 2710 Algorithm Linear-MinG-MaxG computes the values f and g forarbitrary sequence D = (d1 d2 dn) in Θ(n) time
Proof Lines 01ndash03 07 and 18 require only constant time lines 04ndash06 09ndash12 and13ndash17 require Θ(n) time so the total running time is Θ(n)
276 Tournament with f and g
The following reconstruction algorithm Score-Slicing2 is based on balancing be-tween additional points (they are similar to ldquoexcessrdquo introduced by Brauer et al
1276 27 Comparison Based Ranking
[29]) and missing points introduced in [114] The greediness of the algorithm HavelndashHakimi [87 95] also characterises this algorithm
This algorithm is an extended version of the algorithm Score-Slicing proposedin [114]
2761 Definition of the minimax reconstruction algorithm
The work of the slicing program is managed by the following program Mini-MaxInput n the number of players (n ge 2)
D = (d1 d2 dn) a nondecreasing sequence of integers satisfying (279)OutputM = [1 n 1 n] the point matrix of the reconstructed tournamentLocal working variables i j cycle variablesGlobal working variables p = (p0 p1 pn) provisional score sequence
P = (P0 P1 Pn) the partial sums of the provisional scoresM[1 n 1 n] matrix of the provisional points
Mini-Max(nD)01 MinF-MaxG(nD) Initialization02 p0 = 003 P0 = 004 for i = 1 to n05 for j = 1 to iminus 106 M[i j] = b07 for j = i to n08 M[i j] = 009 pi = di
10 if n ge 3 Score slicing for n ge 3 players11 for k = n downto 312 Score-Slicing2(k)13 if n == 2 Score slicing for 2 players14 m12 = p1
15 m21 = p2
16 return M
2762 Definition of the score slicing algorithm
The key part of the reconstruction is the following algorithm Score-Slicing2 [114]During the reconstruction process we have to take into account the following
boundsa le mij +mji le b (1 le i lt j le n) (2711)
modified scores have to satisfy (279) (2712)
mij le pi (1 le i j le n i 6= j) (2713)
the monotonicity p1 le p2 le le pk has to be saved (1 le k le n) (2714)
mii = 0 (1 le i le n) (2715)
276 Tournament with f and g 1277
Input k the number of the actually investigated players (k gt 2)pk = (p0 p1 p2 pk) (k = 3 4 middot middot middot n) prefix of the provisional score sequencepM[1 n 1 n] matrix of provisional points
Output Local working variables A = (A1 A2 An) the number of the addi-tional pointsM missing points the difference of the number of actual points and the number ofmaximal possible points of Pkd difference of the maximal decreasable score and the following largest scorey number of sliced points per playerf frequency of the number of maximal values among the scores p1 p2 pkminus1i j cycle variablesm maximal amount of sliceable pointsP = (P0 P1 Pn) the sums of the provisional scoresx the maximal index i with i lt k and mik lt b
Global working variables n the number of players (n ge 2)B = (B0 B1 B2 Bn) the sequence of the binomial coefficientsa minimal number of points divided after each matchb maximal number of points divided after each match
Score-Slicing2(k)01 for i = 1 to k minus 1 Initialization02 Pi = Piminus1 + pi
03 Ai = Pi minus aBi
04 M = (k minus 1)bminus pk
05 while M gt 0 and Akminus1 gt 0 There are missing and additional points06 x = k minus 107 while rxk = b08 x = xminus 109 f = 110 while pxminusf+1 = pxminusf
11 f = f + 112 d = pxminusf+1 minus pxminusf
13 m = min(b d dAxbe dMbe)14 for i = f downto 115 y = min(bminus rx+1minusikmMAx+1minusi px+1minusi)16 rx+1minusik = rx+1minusik + y17 px+1minusi = px+1minusi minus y18 rkx+1minusi = bminus rx+1minusik
19 M = M minus y20 for j = i downto 121 Ax+1minusi = Ax+1minusi minus y22 while M gt 0 No missing points23 i = k minus 124 y = max(mki +mik minus amkiM)
1278 27 Comparison Based Ranking
25 rki = rki minus y26 M = M minus y27 i = iminus 128 return πkM
Letrsquos consider an example Figure 273 shows the point table of a (2 10 6)-tournament T
PlayerPlayer P1 P2 P3 P4 P5 P6 ScoreP1 mdash 1 5 1 1 1 09P2 1 mdash 4 2 0 2 09P3 3 3 mdash 5 4 4 19P4 8 2 5 mdash 2 3 20P5 9 9 5 7 mdash 2 32P6 8 7 5 6 8 mdash 34
Figure 273 The point table of a (2 10 6)-tournament T
The score sequence of T is D = (9919203234) In [114] the algorithm Score-Slicing2 resulted the point table represented in Figure 274
PlayerPlayer P1 P2 P3 P4 P5 P6 ScoreP1 mdash 1 1 6 1 0 9P2 1 mdash 1 6 1 0 9P3 1 1 mdash 6 8 3 19P4 3 3 3 mdash 8 3 20P5 9 9 2 2 mdash 10 32P6 10 10 7 7 0 mdash 34
Figure 274 The point table of T reconstructed by Score-Slicing2
The algorithm Mini-Max starts with the computation of f MinF-MaxG calledin line 01 begins with initialization including provisional setting of the elements ofM so that mij = b if i gt j and mij = 0 otherwise Then MinF-MaxG sets thelower bound l = max(9 7) = 9 of f in line 07 and tests it in line 10 Interval-TestThe test shows that l = 9 is large enough so Mini-Max sets b = 9 in line 12 andjumps to line 23 and begins to compute g Interval-Test called in line 25 showsthat a = 9 is too large therefore MinF-MaxG continues with the test of a = 5 inline 30 The result is positive therefore comes the test of a = 7 then the test ofa = 8 Now u = l + 1 in line 35 so a = 8 is fixed and the control returns to line 02of Mini-Max
Lines 02ndash09 contain initialization and Mini-Max begins the reconstruction ofa (8 9 6)-tournament in line 10 The basic idea is that Mini-Max successively de-termines the won and lost points of P6 P5 P4 and P3 by repeated calls of Score-Slicing2 in line 12 and finally it computes directly the result of the match between
276 Tournament with f and g 1279
P2 and P1At first Mini-Max computes the results of P6 calling calling Score-Slicing2
with parameter k = 6 The number of additional points of the first five players isA5 = 89 minus 8 middot 10 = 9 according to line 03 the number of missing points of P6 isM = 5 middot 9 minus 34 = 11 according to line 04 Then Score-Slicing2 determines thenumber of maximal numbers among the provisional scores p1 p2 p5 (f = 1according to lines 09ndash14) and computes the difference between p5 and p4 (d = 12according to line 12) In line 13 we get that m = 9 points are sliceable and P5
gets these points in the match with P6 in line 16 so the number of missing pointsof P6 decreases to M = 11 minus 9 = 2 (line 19) and the number of additional pointdecreases to A = 9 minus 9 = 0 Therefore the computation continues in lines 22ndash27and m64 and m63 will be decreased by 1 resulting m64 = 8 and m63 = 8 as theseventh line and seventh column of Figure 275 show The returned score sequenceis p = (9 9 19 20 23)
PlayerPlayer P1 P2 P3 P4 P5 P6 ScoreP1 mdash 4 4 0 0 0 9P2 4 mdash 4 1 0 0 9P3 4 4 mdash 7 4 0 19P4 7 7 1 mdash 5 0 20P5 8 8 4 3 mdash 9 32P6 9 9 8 8 0 mdash 34
Figure 275 The point table of T reconstructed by Mini-Max
Second time Mini-Max calls Score-Slicing2 with parameter k = 5 and getA4 = 9 and M = 13 At first A4 gets 1 point then A3 and A4 get both 4 pointsreducing M to 4 and A4 to 0 The computation continues in line 22 and resultsthe further decrease of m54 m53 m52 and m51 by 1 resulting m54 = 3 m53 = 4m52 = 8 and m51 = 8 as the sixth row of Figure 275 shows
Third time Mini-Max calls Score-Slicing2 with parameter k = 4 and getA3 = 11 and M = 11 At first P3 gets 6 points then P3 further 1 point and P2 andP1 also both get 1 point resulting m34 = 7 m43 = 2 m42 = 8 m24 = 1 m14 = 1and m14 = 8 further A3 = 0 and M = 2 The computation continues in lines 22ndash27and results a decrease of m43 by 1 point resulting m43 = 1 m42=8 and m41 = 8 asthe fifth row and fifth column of Figure 275 show The returned score sequence isp = (9 9 15)
Fourth time Mini-Max calls Score-Slicing2 with parameter k = 3 and getsA2 = 10 and M = 9 At first P2 gets 6 points then The returned point vector isp = (4 4)
Finally Mini-Max sets m12 = 4 and m21 = 4 in lines 14ndash15 and returns thepoint matrix represented in Figure 275
The comparison of Figures 274 and 275 shows a large difference between thesimple reconstruction of Score-Slicing2 and the minimax reconstruction of Mini-Max while in the first case the maximal value of mij +mji is 10 and the minimal
1280 27 Comparison Based Ranking
value is 2 in the second case the maximum equals to 9 and the minimum equals to8 that is the result is more balanced (the given D does not allow to build a perfectlybalanced (k k n)-tournament)
2763 Analysis of the minimax reconstruction algorithm
The main result of this paper is the following assertion
Theorem 2711 If n ge 2 is a positive integer and D = (d1 d2 dn) is a non-decreasing sequence of nonnegative integers then there exist positive integers f andg and a (g f n)-tournament T with point matrix M such that
f = min(mij +mji) le b (2716)
g = maxmij +mji ge a (2717)
for any (a b n)-tournament and algorithm Linear-MinF-MaxG computes f andg in Θ(n) time and algorithm Mini-Max generates a suitable T in O(dnn
2) time
Proof The correctness of the algorithms Score-Slicing2 MinF -MaxG impliesthe correctness of Mini-Max
Lines 1ndash46 of Mini-Max require O(log(dnn)) uses of MinG-MaxF and onesearch needs O(n) steps for the testing so the computation of f and g can beexecuted in O(n log(dnn)) times
The reconstruction part (lines 47ndash55) uses algorithm Score-Slicing2 whichruns in O(bn3) time [114] Mini-Max calls Score-Slicing2 n minus 2 times with f le2ddnne so n3dnn = dnn
2 finishes the proof
The property of the tournament reconstruction problem that the extremal val-ues of f and g can be determined independently and so there exists a tournamentT having both extremal features is called linking property One of the earliest oc-curences appeared in a paper Mendelsohn and Dulmage [153] It was formulated byFord and Fulekerson [66 page 49] in a theorem on the existence of integral matri-ces for which the row-sums and the column-sums lie between specified bounds Theconcept was investigated in detail in the book written by Mirsky [155] A Frankused this property in the analysis of different different problems of combinatorialoptimization [69 73]
277 Imbalances in (0 b 2)-tournaments
A directed graph (shortly digraph) without loops and without multi-arcs is calleda simple digraph [84] The imbalance of a vertex vi in a digraph as bvi
(or simplybi) = d+
viminus dminus
vi where d+
viand dminus
viare respectively the outdegree and indegree of
vi The imbalance sequence of a simple digraph is formed by listing the verteximbalances in non-increasing order A sequence of integers F = [f1 f2 fn]with f1 ge f2 ge ge fn is feasible if the sum of its elements is zero and satisfies
277 Imbalances in (0 b 2)-tournaments 1281
ksum
i=1
fi le k(nminus k) for 1 le k lt n
The following result [84] provides a necessary and sufficient condition for asequence of integers to be the imbalance sequence of a simple digraph
Theorem 2712 A sequence is realizable as an imbalance sequence if and only ifit is feasible
The above result is equivalent to saying that a sequence of integers B =[b1 b2 bn] with b1 ge b2 ge ge bn is an imbalance sequence of a simple di-graph if and only if
ksum
i=1
bi le k(nminus k)
for 1 le k lt n with equality when k = nOn arranging the imbalance sequence in non-decreasing order we have the fol-
lowing observation
Corollary 2713 A sequence of integers B = [b1 b2 bn] with b1 le b2 le lebn is an imbalance sequence of a simple digraph if and only if
ksum
i=1
bi ge k(k minus n)
for 1 le k lt n with equality when k = n
Various results for imbalances in simple digraphs and oriented graphs can befound in [114 116 192 193]
2771 Imbalances in (0 b)-graphs
A multigraph is a graph from which multi-edges are not removed and which hasno loops [84] If r ge 1 then an r-digraph (shortly r-graph) is an orientation of amultigraph that is without loops and contains at most r edges between the elementsof any pair of distinct vertices Clearly 1-digraph is an oriented graph Let D bean f -digraph with vertex set V = v1 v2 vn and let d+
v and dminusv respectively
denote the outdegree and indegree of vertex v Define bvi(or simply bi) = d+
viminus dminus
ui
as imbalance of vi Clearly minusr(nminus 1) le bvile r(nminus 1) The imbalance sequence of
D is formed by listing the vertex imbalances in non-decreasing orderWe remark that r-digraphs are special cases of (a b)-digraphs containing at least
a and at most b edges between the elements of any pair of vertices Degree sequencesof (a b)-digraphs are studied in [162 192]
Let u and v be distinct vertices in D If there are f arcs directed from u to v andg arcs directed from v to u we denote this by u(f minus g)v where 0 le f g f + g le r
A double in D is an induced directed subgraph with two vertices u and v having
1282 27 Comparison Based Ranking
the form u(f1 minus f2)v where 1 le f1 f2 le r and 1 le f1 + f2 le r and f1 is thenumber of arcs directed from u to v and f2 is the number of arcs directed from vto u A triple in D is an induced subgraph with tree vertices u v and w havingthe form u(f1 minus f2)v(g1 minus g2)w(h1 minus h2)u where 1 le f1 f2 g1 g2 h1 h2 le rand 1 le f1 + f2 g1 + g2 h1 + h2 le r and the meaning of f1 f2 g1 g2 h1 h2
is similar to the meaning in the definition of doubles An oriented triple in D is aninduced subdigraph with three vertices An oriented triple is said to be transitiveif it is of the form u(1 minus 0)v(1 minus 0)w(0 minus 1)u or u(1 minus 0)v(0 minus 1)w(0 minus 0)u oru(1minus0)v(0minus0)w(0minus1)u or u(1minus0)v(0minus0)w(0minus0)u or u(0minus0)v(0minus0)w(0minus0)uotherwise it is intransitive An r-graph is said to be transitive if all its oriented triplesare transitive In particular a triple C in an r-graph is transitive if every orientedtriple of C is transitive
The following observation can be easily established and is analogues to Theorem22 of Avery [8]
Lemma 2714 If D1 and D2 are two r-graphs with same imbalance sequence thenD1 can be transformed to D2 by successively transforming (i) appropriate orientedtriples in one of the following ways either (a) by changing the intransitive orientedtriple u(1minus 0)v(1minus 0)w(1minus 0)u to a transitive oriented triple u(0minus 0)v(0minus 0)w(0minus0)u which has the same imbalance sequence or vice versa or (b) by changing theintransitive oriented triple u(1 minus 0)v(1 minus 0)w(0 minus 0)u to a transitive oriented tripleu(0 minus 0)v(0 minus 0)w(0 minus 1)u which has the same imbalance sequence or vice versaor (ii) by changing a double u(1 minus 1)v to a double u(0 minus 0)v which has the sameimbalance sequence or vice versa
The above observations lead to the following result
Theorem 2715 Among all r-graphs with given imbalance sequence those with thefewest arcs are transitive
Proof Let B be an imbalance sequence and let D be a realization of B that isnot transitive Then D contains an intransitive oriented triple If it is of the formu(1 minus 0)v(1 minus 0)w(1 minus 0)u it can be transformed by operation i(a) of Lemma 3to a transitive oriented triple u(0 minus 0)v(0 minus 0)w(0 minus 0)u with the same imbalancesequence and three arcs fewer If D contains an intransitive oriented triple of theform u(1minus 0)v(1minus 0)w(0minus 0)u it can be transformed by operation i(b) of Lemma3 to a transitive oriented triple u(0minus 0)v(0minus 0)w(0minus 1)u same imbalance sequencebut one arc fewer In case D contains both types of intransitive oriented triples theycan be transformed to transitive ones with certainly lesser arcs If in D there is adouble u(1minus 1)v by operation (ii) of Lemme 4 it can be transformed to u(0minus 0)vwith same imbalance sequence but two arcs fewer
The next result gives necessary and sufficient conditions for a sequence of integersto be the imbalance sequence of some r-graph
Theorem 2716 A sequence B = [b1 b2 bn] of integers in non-decreasing order
277 Imbalances in (0 b 2)-tournaments 1283
is an imbalance sequence of an r-graph if and only if
ksum
i=1
bi ge rk(k minus n) (2718)
with equality when k = n
Proof Necessity A multi subdigraph induced by k vertices has a sum of imbal-ances rk(k minus n)
Sufficiency Assume that B = [b1 b2 bn] be the sequence of integers in non-decreasing order satisfying conditions (1) but is not the imbalance sequence of anyr-graph Let this sequence be chosen in such a way that n is the smallest possibleand b1 is the least with that choice of n We consider the following two cases
Case (i) Suppose equality in (1) holds for some k le n so that
ksum
i=1
bi = rk(k minus n)
for 1 le k lt nBy minimality of n B1 = [b1 b2 bk] is the imbalance sequence of some
r-graph D1 with vertex set say V1 Let B2 = [bk+1 bk+2 bn]Consider
fsum
i=1
bk+i =k+fsum
i=1
bi minusksum
i=1
bi
ge r(k + f)[(k + f)minus n]minus rk(k minus n)
= r(k2 + kf minus kn+ fk + f2 minus fnminus k2 + kn)
ge r(f2 minus fn)
= rf(f minus n)
for 1 le f le nminus k with equality when f = nminus k Therefore by the minimality forn the sequence B2 forms the imbalance sequence of some r-graph D2 with vertexset say V2 Construct a new r-graph D with vertex set as follows
Let V = V1 cup V2 with V1 cap V2 = φ and the arc set containing those arcs whichare in D1 and D2 Then we obtain the r-graph D with the imbalance sequence Bwhich is a contradictionCase (ii) Suppose that the strict inequality holds in (1) for some k lt n so that
ksum
i=1
bi gt rk(k minus n)
for 1 le k lt n Let B1 = [b1minus1 b2 bnminus1 bn +1] so that B1 satisfy the conditions(1) Thus by the minimality of b1 the sequences B1 is the imbalances sequence of
1284 27 Comparison Based Ranking
some r-graph D1 with vertex set say V1) Let bv1= b1 minus 1 and bvn
= an + 1 Sincebvn
gt bv1+ 1 there exists a vertex vp isin V1 such that vn(0 minus 0)vp(1 minus 0)v1 or
vn(1minus0)vp(0minus0)v1 or vn(1minus0)vp(1minus0)v1 or vn(0minus0)vp(0minus0)v1 and if these arechanged to vn(0minus 1)vp(0minus 0)v1 or vn(0minus 0)vp(0minus 1)v1 or vn(0minus 0)vp(0minus 0)v1 orvn(0minus 1)vp(0minus 1)v1 respectively the result is an r-graph with imbalances sequenceB which is again a contradiction This proves the result
Arranging the imbalance sequence in non-increasing order we have the followingobservation
Corollary 2717 A sequence B = [b1 b2 bn] of integers with b1 ge b2 ge gebn is an imbalance sequence of an r-graph if and only if
ksum
i=1
bi le rk(nminus k)
for 1 le k le n with equality when k = n
The converse of an r-graphD is an r-graphDprime obtained by reversing orientationsof all arcs of D If B = [b1 b2 bn] with b1 le b2 le le bn is the imbalancesequence of an r-graphD then Bprime = [minusbnminusbnminus1 minusb1] is the imbalance sequenceof Dprime
The next result gives lower and upper bounds for the imbalance bi of a vertexvi in an r-graph D
Theorem 2718 If B = [b1 b2 bn] is an imbalance sequence of an r-graph Dthen for each i
r(iminus n) le bi le r(iminus 1)
Proof Assume to the contrary that bi lt r(iminus n) so that for k lt i
bk le bi lt r(iminus n)
That isb1 lt r(iminus n) b2 lt r(iminus n) bi lt r(iminus n)
Adding these inequalities we get
isum
k=1
bk lt ri(iminus n)
which contradicts Theorem 3Therefore r(iminus n) le biThe second inequality is dual to the first In the converse r-graph with imbalance
sequence B = [bprime1 b
prime2 b
primen] we have by the first inequality
bprimenminusi+1 ge r[(nminus i+ 1)minus n]
= r(minusi+ 1)
277 Imbalances in (0 b 2)-tournaments 1285
Since bi = minusbprimenminusi+1 therefore
bi le minusr(minusi+ 1) = r(iminus 1)
Hence bi le r(iminus 1)
Now we obtain the following inequalities for imbalances in r-graphs
Theorem 2719 If B = [b1 b2 bn] is an imbalance sequence of an r-graph withb1 ge b2 ge ge bn then
ksum
i=1
b2i le
ksum
i=1
(2rnminus 2rk minus bi)2
for 1 le k le n with equality when k = n
Proof By Theorem 3 we have for 1 le k le n with equality when k = n
rk(nminus k) geksum
i=1
bi
implying
ksum
i=1
b2i + 2(2rnminus 2rk)rk(nminus k) ge
ksum
i=1
b2i + 2(2rnminus 2rk)
ksum
i=1
bi
from where
ksum
i=1
b2i + k(2rnminus 2rk)2 minus 2(2rnminus 2rk)
ksum
i=1
bi geksum
i=1
b2i
and so we get the required
b21 + b2
2 + + b2k + (2rnminus 2rk)2 + (2rnminus 2rk)2 + + (2rnminus 2rk)2
minus 2(2rnminus 2rk)b1 minus 2(2rnminus 2rk)b2 minus minus 2(2rnminus 2rk)bk
geksum
i=1
b2i
orksum
i=1
(2rnminus 2rk minus bi)2 geksum
i=1
b2i
The set of distinct imbalances of vertices in an r-graph is called its imbalanceset The following result gives the existence of an r-graph with a given imbal-ance set Let (p1 p2 pm q1 q2 qn) denote the greatest common divisor ofp1 p2 pn q1 q2 qn
1286 27 Comparison Based Ranking
Theorem 2720 If P = p1 p2 pm and Q = minusq1minusq2 minusqn wherep1 p2 pm q1 q2 qn are positive integers such that p1 lt p2 lt lt pm andq1 lt q2 lt lt qn and (p1 p2 pm q1 q2 qn) = t 1 le t le r then thereexists an r-graph with imbalance set P cupQ
Proof Since (p1 p2 pm q1 q2 qn) = t 1 le t le r there exist positive inte-gers f1 f2 fm and g1 g2 gn with f1 lt f2 lt lt fm and g1 lt g2 lt lt gn
such thatpi = tfi
for 1 le i le m andqi = tgi
for 1 le j le nWe construct an r-graph D with vertex set V as followsLet
V = X11 cupX1
2 cup cupX1mcupX2
1 cupX31 cup cupXn
1 cupY 11 cupY 1
2 cup cupY 1mcupY 2
1 cupY 31 cup cupY n
1
with Xji capX l
k = φ Y ji cap Y l
k = φ Xji cap Y l
k = φ and|X1
i | = g1 for all 1 le i le m|Xi
1| = gi for all 2 le i le n|Y 1
i | = fi for all 1 le i le m|Y i
1 | = f1 for all 2 le i le nLet there be t arcs directed from every vertex of X1
i to each vertex of Y 1i for all
1 le i le m and let there be t arcs directed from every vertex of Xi1 to each vertex of
Y i1 for all 2 le i le n so that we obtain the r-graph D with imbalances of vertices as
underFor 1 le i le m for all x1
i isin X1i
bx1i
= t|Y 1i | minus 0 = tfi = pi
for 2 le i le n for all xi1 isin Xi
1
bxi1
= t|Y i1 | minus 0 = tf1 = p1
for 1 le i le m for all y1i isin Y 1
i
by1i
= 0minus t|X1i | = minustgi = minusqi
and for 2 le i le n for all yi1 isin Y i
1
byi1
= 0minus t|Xi1| = minustgi = minusqi
Therefore imbalance set of D is P cupQ
278 Supertournaments 1287
278 Supertournaments
Let n m be positive integers a = (a1 a2 middot middot middot an) b = (b1 b2 middot middot middot bm) and k =(k1 k2 middot middot middot km) vectors of nonnegative integers with ai le biq(i = 1 2 n) and0 lt k1 lt k2 lt lt km
Then an abk n-supertournament is an xtimes y sized matrix Info 22 PirzadaZhouIvanyiHypergraphs are generalizations of graphs [21 22] While edges of a graph are
pairs of vertices of the graph edges of a hypergraph are subsets of the vertex setconsisting of at least two vertices An edge consisting of k vertices is called a k-edgeA k-hypergraph is a hypergraph all of whose edges are k-edges A k-hypertournamentis a complete k-hypergraph with each k-edge endowed with an orientation that isa linear arrangement of the vertices contained in the hyperedge Instead of scoresof vertices in a tournament Zhou et al [246] considered scores and losing scores ofvertices in a k-hypertournament and derived a result analogous to Landaursquos theorem[139] The score s(vi) or si of a vertex vi is the number of arcs containing vi andin which vi is not the last element and the losing score r(vi) or ri of a vertex vi
is the number of arcs containing vi and in which vi is the last element The scoresequence (losing score sequence) is formed by listing the scores (losing scores) innon-decreasing order
The following characterizations of score sequences and losing score sequences ink-hypertournaments can be found in G Zhou et al [247]
Theorem 2721 Given two non-negative integers n and k with n ge k gt 1 a non-decreasing sequence R = [r1 r2 rn] of non-negative integers is a losing scoresequence of some k-hypertournament if and only if for each j
jsum
i=1
ri ge(
j
k
)
with equality when j = n
Theorem 2722 Given non-negative integers n and k with n ge k gt 1 a non-decreasing sequence S = [s1 s2 sn] of non-negative integers is a score sequenceof some k-hypertournament if and only if for each j
jsum
i=1
si ge j(
nminus 1k minus 1
)
+(
nminus jk
)
minus(n
k
)
with equality when j = n
Some more results on k-hypertournaments can be found in [30 133 185 187246] The analogous results of Theorem 2721 and Theorem 2722 for [h k]-bipartitehypertournaments can be found in [184] and for [α β γ]-tripartite hypertournamentscan be found in [194]
Throughout this paper i takes values from 1 to k and ji takes values from 1 toni unless otherwise stated
1288 27 Comparison Based Ranking
A k-partite hypergraph is a generalization of k-partite graph Given non-negativeintegers ni and αi (i = 1 2 k) with ni ge αi ge 1 for each i an [α1 α2 αk]-k-partite hypertournament (or briefly k-partite hypertournament) M of order
sumk1 ni
consists of k vertex sets Ui with |Ui| = ni for each i (1 le i le k) together with an arcset E a set of
sumk1 αi tuples of vertices with exactly αi vertices from Ui called arcs
such that anysumk
1 αi subset cupk1U
primei of cupk
1Ui E contains exactly one of the(
sumk1 αi
)
sumk1 αi-tuples whose αi entries belong to U prime
i Let e = (u11 u12 u1α1
u21 u22 u2α2 uk1 uk2 ukαk
) with uijiisin
Ui for each i (1 le i le k 1 le ji le αi) be an arc in M and let h lt t we let e(u1h u1t)denote to be the new arc obtained from e by interchanging u1h and u1t in e An arccontaining αi vertices from Ui for each i (1 le i le k) is called an (α1 α2 αk)-arc
For a given vertex uijiisin Ui for each i 1 le i le k and 1 le ji le αi the score
d+M (uiji
) (or simply d+(uiji)) is the number of
sumk1 αi-arcs containing uiji
and inwhich uiji
is not the last element The losing score dminusM (uiji
) (or simply dminus(uiji)) is
the number ofsumk
1 αi-arcs containing uijiand in which uiji
is the last element Byarranging the losing scores of each vertex set Ui separately in non-decreasing orderwe get k lists called losing score lists of M and these are denoted by Ri = [riji
]ni
ji=1
for each i (1 le i le k) Similarly by arranging the score lists of each vertex set Ui
separately in non-decreasing order we get k lists called score lists of M which aredenoted as Si = [siji
]ni
ji=1 for each i (1 le i le k)
2781 Hypertournamentss
The following two theorems are the main results
Theorem 2723 Given k non-negative integers ni and k non-negative integers αi
with 1 le αi le ni for each i (1 le i le k) the k non-decreasing lists Ri = [riji]ni
ji=1 ofnon-negative integers are the losing score lists of a k-partite hypertournament if andonly if for each pi (1 le i le k) with pi le ni
ksum
i=1
pisum
ji=1
rijige
kprod
i=1
(
pi
αi
)
(2719)
with equality when pi = ni for each i (1 le i le k)
Theorem 2724 Given k non-negative integers ni and k nonnegative integers αi
with 1 le αi le ni for each i (1 le i le k) the k non-decreasing lists Si = [siji]ni
ji=1 ofnon-negative integers are the score lists of a k-partite hypertournament if and onlyif for each pi (1 le i le k) with pi le ni
ksum
i=1
pisum
ji=1
sijige(
ksum
i=1
αipi
ni
)(
kprod
i=1
(
ni
αi
)
)
+kprod
i=1
(
ni minus pi
αi
)
minuskprod
i=1
(
ni
αi
)
(2720)
with equality when pi = ni for each i (1 le i le k)
278 Supertournaments 1289
We note that in a k-partite hypertournament M there are exactlyprodk
i=1
(
ni
αi
)
arcs and in each arc only one vertex is at the last entry Therefore
ksum
i=1
nisum
ji=1
dminusM (uiji
) =kprod
i=1
(
ni
αi
)
In order to prove the above two theorems we need the following Lemmas
Lemma 2725 If M is a k-partite hypertournament of ordersumk
1 ni with score listsSi = [siji
]ni
ji=1 for each i (1 le i le k) then
ksum
i=1
nisum
ji=1
siji=
[(
ksum
1=1
αi
)
minus 1
]
kprod
i=1
(
ni
αi
)
Proof We have ni ge αi for each i (1 le i le k) If rijiis the losing score of uiji
isin Uithen
ksum
i=1
nisum
ji=1
riji=
kprod
i=1
(
ni
αi
)
The number of [αi]k1 arcs containing uijiisin Ui for each i (1 le i le k) and
1 le ji le ni is
αi
ni
kprod
t=1
(
nt
αt
)
Thus
ksum
i=1
nisum
ji=1
siji=
ksum
i=1
nisum
ji=1
(
αi
ni
) kprod
1
(
nt
αt
)
minus(
ni
αi
)
=
(
ksum
i=1
αi
)
kprod
1
(
nt
αt
)
minuskprod
1
(
ni
αi
)
=
[(
ksum
1=1
αi
)
minus 1
]
kprod
1
(
ni
αi
)
Lemma 2726 If Ri = [riji]ni
ji=1 (1 le i le k) are k losing score lists of a k-
partite hypertournament M then there exists some h with r1h ltα1
n1
prodk1
(
np
αp
)
so that
Rprime1 = [r11 r12 r1h + 1 r1n1
] Rprimes = [rs1 rs2 rstminus 1 rsns
] (2 le s le k)and Ri = [riji
]ni
ji=1 (2 le i le k) i 6= s are losing score lists of some k-partitehypertournament t is the largest integer such that rs(tminus1) lt rst = = rsns
1290 27 Comparison Based Ranking
Proof Let Ri = [riji]ni
ji=1 (1 le i le k) be losing score lists of a k-partite hypertour-
nament M with vertex sets Ui = ui1 ui2 uiji so that dminus(uiji
) = rijifor each
i (1 le i le k 1 le ji le ni)Let h be the smallest integer such that
r11 = r12 = = r1h lt r1(h+1) le le r1n1
and t be the largest integer such that
rs1 le rs2 le le rs(tminus1) lt rst = = rsns
Now letRprime
1 = [r11 r12 r1h + 1 r1n1]
Rprimes = [rs1 rs2 rst minus 1 rsns
(2 le s le k) and Ri = [riji]ni
ji=1 (2 le i le k) i 6= sClearly Rprime
1 and Rprimes are both in non-decreasing order
Since r1h lt α1
n1
prodk1
(
np
αp
)
there is at least one [αi]k1-arc e containing both u1h
and ust with ust as the last element in e let eprime = (u1h ust) Clearly Rprime1 Rprime
s andRi = [riji
]ni
ji=1 for each i (2 le i le k) i 6= s are the k losing score lists of M prime =(M minus e) cup eprime
The next observation follows from Lemma 2726 and the proof can be easilyestablished
Lemma 2727 Let Ri = [riji]ni
ji=1 (1 le i le k) be k non-decreasing sequences
of non-negative integers satisfying (1) If r1n1lt α1
n1
prodk1
(
nt
αt
)
then there exists s
and t (2 le s le k) 1 le t le ns such that Rprime1 = [r11 r12 r1h + 1 r1n1
]Rprime
s = [rs1 rs2 rst minus 1 rsns] and Ri = [riji
]ni
ji=1 (2 le i le k) i 6= s satisfy(1)
Proof of Theorem 2723 Necessity Let Ri (1 le i le k) be the k losingscore lists of a k-partite hypertournament M(Ui 1 le i le k) For any pi with αi le pi
le ni let U primei = uiji
pi
ji=1(1 le i le k) be the sets of vertices such that dminus(uiji) = riji
for each 1 le ji le pi 1 le i le k Let M prime be the k-partite hypertournament formedby U prime
i for each i (1 le i le k)Then
ksum
i=1
pisum
ji=1
rijige
ksum
i=1
pisum
ji=1
dminusM prime(uiji
)
=kprod
1
(
pt
αt
)
278 Supertournaments 1291
Sufficiency We induct on n1 keeping n2 nk fixed For n1 = α1 the result isobviously true So let n1 gt α1 and similarly n2 gt α2 nk gt αk Now
r1n1=
ksum
i=1
nisum
ji=1
rijiminus
n1minus1sum
j1=1
r1j1+
ksum
i=2
nisum
ji=1
riji
lekprod
1
(
nt
αt
)
minus(
n1 minus 1α1
) kprod
2
(
nt
αt
)
=[(
n1
α1
)
minus(
n1 minus 1α1
)] kprod
2
(
nt
αt
)
=(
n1 minus 1α1 minus 1
) kprod
2
(
nt
αt
)
We consider the following two cases
Case 1 r1n1=(
n1 minus 1α1 minus 1
)
prodk2
(
nt
αt
)
Then
n1minus1sum
j1=1
r1j1+
ksum
i=2
nisum
ji=1
riji=
ksum
i=1
nisum
ji=1
rijiminus r1n1
=kprod
1
(
nt
αt
)
minus(
n1 minus 1α1 minus 1
) kprod
2
(
nt
αt
)
=[(
n1
α1
)
minus(
n1 minus 1α1 minus 1
)] kprod
2
(
nt
αt
)
=(
n1 minus 1α1
) kprod
2
(
nt
αt
)
By induction hypothesis [r11 r12 r1(n1minus1)] R2 Rk are losing score
lists of a k-partite hypertournament M prime(U prime1 U2 Uk) of order
(
sumki=1 ni
)
minus 1
Construct a k-partite hypertournament M of ordersumk
i=1 ni as follows In M prime letU prime
1 = u11 u12 u1(n1minus1) Ui = uijini
ji=1 for each i (2 le i le k) Adding a
new vertex u1n1to U prime
1 for each(
sumki=1 αi
)
-tuple containing u1n1 arrange u1n1
on the last entry Denote E1 to be the set of all these(
n1 minus 1α1 minus 1
)
prodk2
(
nt
αt
)
(
sumki=1 αi
)
-tuples Let E(M) = E(M prime) cup E1 Clearly Ri for each i (1 le i le k) arethe k losing score lists of M
Case 2 r1n1lt
(
n1 minus 1α1 minus 1
)
prodk2
(
nt
αt
)
Applying Lemma 2727 repeatedly on R1 and keeping each Ri (2 le i le k)fixed until we get a new non-decreasing list Rprime
1 = [rprime11 r
prime12 r
prime1n1
] in which now
1292 27 Comparison Based Ranking
prime1n1
=(
n1 minus 1α1 minus 1
)
prodk2
(
nt
αt
)
By Case 1 Rprime1 Ri (2 le i le k) are the losing
score lists of a k-partite hypertournament Now apply Lemma 2726 on Rprime1 Ri
(2 le i le k) repeatedly until we obtain the initial non-decreasing lists Ri for each i(1 le i le k) Then by Lemma 2726 Ri for each i (1 le i le k) are the losing scorelists of a k-partite hypertournament
Proof of Theorem 2724 Let Si = [siji]ni
ji=1(1 le i le k) be the k scorelists of a k-partite hypertournament M(Ui 1 le i le k) where Ui = uiji
ni
ji=1 withd+
M (uiji) = siji
for each i (1 le i le k) Clearly
d+(uiji) + dminus(uiji
) = αi
ni
prodk1
(
nt
αt
)
(1 le i le k 1 le ji le ni)
Let ri(ni+1minusji) = dminus(uiji) (1 le i le k 1 le ji le ni)
Then Ri = [riji]ni
ji=1(i = 1 2 k) are the k losing score lists of M Converselyif Ri for each i (1 le i le k) are the losing score lists of M then Si for each i(1 le i le k) are the score lists of M Thus it is enough to show that conditions
(1) and (2) are equivalent provided siji+ ri(ni+1minusji) =
(
αi
ni
)
prodk1
(
nt
αt
)
for each i
(1 le i le k and 1 le ji le ni)First assume (2) holds Then
ksum
i=1
pisum
ji=1
riji=
ksum
i=1
pisum
ji=1
(
αi
ni
)
(
kprod
1
(
nt
αt
)
)
minusksum
i=1
pisum
ji=1
si(ni+1minusji)
=ksum
i=1
pisum
ji=1
(
αi
ni
)
(
kprod
1
(
nt
αt
)
)
minus
ksum
i=1
nisum
ji=1
rijiminus
ksum
i=1
niminuspisum
ji=1
siji
ge
ksum
i=1
pisum
ji=1
(
αi
ni
)
(
kprod
1
(
nt
αt
)
)
minus[((
ksum
1
αi
)
minus 1
)
kprod
1
(
ni
αi
)
]
+ksum
i=1
(ni minus pi)(
αi
ni
) kprod
1
(
nt
αt
)
+kprod
1
(
ni minus (ni minus pi)αi
)
minuskprod
1
(
ni
αi
)
=kprod
1
(
ni
αi
)
with equality when pi = ni for each i (1 le i le k) Thus (1) holdsNow when (1) holds using a similar argument as above we can show that (2)
holds This completes the proof
279 Summary 1293
279 Summary
A nondecreasing sequence of nonnegative integers D = (d1 d2 dn) is a scoresequence of a (1 1 1)-tournament iff the sum of the elements of D equals to Bn andthe sum of the first i (i = 1 2 nminus 1) elements of D is at least Bi [139]
D is a score sequence of a (k k n)-tournament iff the sum of the elements of Dequals to kBn and the sum of the first i elements of D is at least kBi [125 158]
D is a score sequence of an (a b n)-tournament iff (279) holds [114]In all 3 cases the decision whether D is digraphical requires only linear timeIn this paper the results of [114] are extended proving that for any D there exists
an optimal minimax realization T that is a tournament having D as its outdegreesequence and maximal G and minimal F in the set of all realization of D
In a continuation [116] of this chapter we construct balanced as possible tourna-ments in a similar way if not only the outdegree sequence but the indegree sequenceis also given
Exercises279-1 How many
Problems
27-1 Football score sequencesLet
Chapter Notes
[8] [12] [15] [17] [24] [29] [33] [32] [84][87] [95][114] [116] [115] [117] [118][132] [139] [158] [159] [162][185] [184]Imbalances [162] [185]Acknowledgement The author thanks Andraacutes Frank (Eoumltvoumls Loraacutend Univer-
sity) for valuable advices concerning the application of flow theory and Peacuteter LErdős (Alfreacuted Reacutenyi Institute of Mathematics of HAS) for the consultation
The research of the third author was supported by the European Union and theEuropean Social Fund under the grant agreement no TAacuteMOP 421B-091KMR-2010-0003
28 Complexity of Words
The complexity of words is a continuously growing field of the combinatorics ofwords Hundreds of papers are devoted to different kind of complexities We try topresent in this chapter far from beeing exhaustive the basic notions and results forfinite and infinite words
First of all we summarize the simple (classical) complexity measures givingformulas and algorithms for several cases After this generalized complexities aretreated with different type of algorithms We finish this chapter by presenting thepalindrome complexity
Finally references from a rich bibliography are given
281 Simple complexity measures
In this section simple (classical) complexities as measures of the diversity of thesubwords in finite and infinite words are discussed First we present some usefulnotions related to the finite and infinite words with examples Word graphs whichplay an important role in understanding and obtaining the complexity are presentedin detail with pertinent examples After this the subword complexity (as number ofsubwords) with related notions is expansively presented
2811 Finite words
Let A be a finite nonempty set called alphabet Its elements are called letters orsymbols A string a1a2 an formed by (not necessary different) elements of Ais a word The length of the word u = a1a2 an is n and is denoted by |u| Theword without any element is the empty word denoted by ε (sometimes λ) The setof all finite words over A is denoted by Alowast We will use the following notations too
A+ = Alowast ε An =
u isin Alowast ∣∣ |u| = n
=
a1a2 an | ai isin A
that is A+ is the set of all finite and nonempty words over A whilst An is the set ofall words of length n over A Obviously A0 = ε The sets Alowast and A+ are infinitedenumerable sets
281 Simple complexity measures 1295
We define in Alowast the binary operation called concatenation (shortly catena-tion) If u = a1a2 an and v = b1b2 bm then
w = uv = a1a2 anb1b2 bm |w| = |u|+ |v|
This binary operation is associative but not commutative Its neutral element isε because εu = uε = u The set Alowast with this neutral element is a monoid Weintroduce recursively the power of a word
bull u0 = εbull un = unminus1u if n ge 1
A word is primitive if it is no power of any word so u is primitive if
u = vn v 6= ε rArr n = 1
For example u = abcab is a primitive word whilst v = abcabc = (abc)2 is notThe word u = a1a2 an is periodic if there is a value p 1 le p lt n such that
ai = ai+p for all i = 1 2 nminus p
and p is the period of u The least such p is the least period of uThe word u = abcabca is periodic with the least period p = 3Let us denote by (a b) the greatest common divisor of the naturals a and b The
following result is obvious
Theorem 281 If u is periodic and p and q are periods then (p q) is a period too
The reversal (or mirror image) of the word u = a1a2 an is uR =ananminus1 a1 Obviously
(
uR)R
= u If u = uR then u is a palindromeThe word u is a subword (or factor) of v if there exist the words p and q such
that v = puq If pq 6= ε then u is a proper subword of v If p = ε then u is a prefixof v and if q = ε then u is a suffix of v The set of all subwords of length n of u isdenoted by Fn(u) F (u) is the set of nonempty subwords of u so
F (u) =|u|⋃
n=1
Fn(u)
For example if u = abaab then
F1(u) = a b F2(u) = ab ba aa F3(u) = aba baa aabF4(u) = abaa baab F5(u) = abaab
The words u = a1a2 am and v = b1b2 bn are equal if
bull m = n andbull ai = bi for i = 1 2 n
Theorem 282 (FinendashWilf) If u and v are words of length n respective m and ifthere are the natural numbers p and q such that up and vq have a common prefix oflength n+mminus (nm) then u and v are powers of the same word
1296 28 Complexity of Words
The value n + m minus (nm) in the theorem is tight This can be illustrated by thefollowing example Here the words u and v have a common prefix of length n+mminus(nm)minus 1 but u and v are not powers of the same word
u = abaab m = |u| = 5 u2 = abaababaab v = aba n = |v| = 3 v3 = abaabaaba
By the theorem a common prefix of length 7 would ensure that u and v are powersof the same word We can see that u2 and v3 have a common prefix of length 6(abaaba) but u and v are not powers of the same word so the length of the commonprefix given by the theorem is tight
2812 Infinite words
Beside the finite words we consider infinite (more precisely infinite at right) wordstoo
u = u1u2 un where u1 u2 isin A The set of infinite words over the alphabet A is denoted by Aω If we will studytogether finite and infinite words the following notation will be useful
Ainfin = Alowast cupAω
The notions as subwords prefixes suffixes can be defined similarly for infinite wordstoo
The word v isin A+ is a subword of u isin Aω if there are the words p isin Alowast q isin Aωsuch that u = pvq If p 6= ε then p is a prefix of u whilst q is a suffix of u HereFn(u) also represents the set of all subwords of length n of u
Examples of infinite words over a binary alphabet
1) The power word is defined as
p = 010011000111 0n1n = 0102120313 0n1n
It can be seen that
F1(p) = 0 1 F2(p) = 01 10 00 11F3(p) = 010 100 001 011 110 000 111 2) The Champernowne word is obtained by writing in binary representation
the natural numbers 0 1 2 3
c = 0 1 10 11 100 101 110 111 1000 1001 1010 1011 1100 1101 1110 1111 10000
It can be seen that
F1(p) = 0 1 F2(p) = 00 01 10 11F3(p) = 000 001 010 011 100 101 110 111 3) The finite Fibonacci words can be defined recursively as
f0 = 0 f1 = 01fn = fnminus1fnminus2 if n ge 2
281 Simple complexity measures 1297
From this definition we obtainf0 = 0f1 = 01f2 = 010f3 = 01001f4 = 01001010f5 = 0100101001001f6 = 010010100100101001010
The infinite Fibonacci word can be defined as the limit of the sequence of finiteFibonacci words
f = limnrarrinfin
fn
The subwords of this word are
F1(f) = 0 1 F2(f) = 01 10 00 F3(f) = 010 100 001 101F4(f) = 0100 1001 0010 0101 1010 The name of Fibonacci words stems from the Fibonacci numbers because the
length of finite Fibonacci words is related to the Fibonacci numbers |fn| = Fn+2ie the length of the nth finite Fibonacci word fn is equal to the (n+2)th Fibonaccinumber
The infinite Fibonacci word has a lot of interesting properties For example fromthe definition we can see that it cannot contain the subword 11
The number of 1rsquos in a word u will be denoted by h(u) An infinite word u isbalanced if for arbitrary subwords x and y of the same length we have |h(x) minush(y)| le 1 ie
x y isin Fn(u) rArr |h(x)minus h(y)| le 1
Theorem 283 The infinite Fibonacci word f is balanced
Theorem 284 Fn(f) has n+ 1 elements
If word u is concatenated by itself infinitely then the result is denoted by uωThe infinite word u is periodic if there is a finite word v such that u = vω This
is a generalization of the finite case periodicity The infinite word u is ultimatelyperiodic if there are the words v and w such that u = vwω
The infinite Fibonacci word can be generated by a (homo)morphism too Let usdefine this morphism
χ Alowast rarr Alowast χ(uv) = χ(u)χ(v) forallu v isin Alowast
Based on this definition the function χ can be defined on letters only A morphismcan be extended for infinite words too
χ Aω rarr Aω χ(uv) = χ(u)χ(v) forallu isin Alowast v isin Aω
The finite Fibonacci word fn can be generated by the following morphism
σ(0) = 01 σ(1) = 0
In this case we have the following theorem
1298 28 Complexity of Words
Figure 281 The De Bruijn graph B(2 3)
Theorem 285 fn+1 = σ(fn)
Proof The proof is by induction Obviously f1 = σ(f0) Let us presume that fk =σ(fkminus1) for all k le n Because
fn+1 = fnfnminus1
by the induction hypothesis
fn+1 = σ(fnminus1)σ(fnminus2) = σ(fnminus1fnminus2) = σ(fn)
From this we obtain
Theorem 286 fn = σn(0)
The infinite Fibonacci word f is the fixed point of the morphism σ
f = σ(f)
2813 Word graphs
Let V sube Am be a set of words of length m over A and E sube AV cap V A We definea digraph whose vertices are from V and whose arcs from E There is an arc fromthe vertex a1a2 am to the vertex b1b2 bm if
a2 = b1 a3 = b2 am = bmminus1 and a1a2 ambm isin E
that is the last mminus 1 letters in the first word are identical to the first mminus 1 lettersin the second word This arc is labelled by a1a2 ambm (or a1b1 bm)
De Bruijn graphs If V = Am and E = Am+1 where A = a1 a2 an thenthe graph is called De Bruijn graph denoted by B(nm)
Figures 281 and 282 illustrate De Bruijn graphs B(2 3) and B(3 2)
281 Simple complexity measures 1299
Figure 282 The De Bruijn graph B(3 2)
To a walk1 x1x2 xm x2x3 xmxm+1 z1z2 zm in the De Bruijngraph we attach the label x1x2 zmminus1zm which is obtained by maximum overlapof the vertices of the walk In Figure 281 in the graph B(2 3) the label attachedto the walk 001 011 111 110 (which is a path) is 001110 The word attached to aHamiltonian path (which contains all vertices of the graph) in the graph B(nm) isan (nm)-type De Bruijn word For example words 0001110100 and 0001011100are (2 3)-type De Bruijn word An (nm)-type De Bruijn word contains all wordsof length m
A connected digraph2 is Eulerian3 if the in-degree of each vertex is equal to itsout-degree4
Theorem 287 The De Bruijn graph B(nm) is Eulerian
Proof a) The graph is connected because between all pair of vertices x1x2 xm andz1z2 zm there is an oriented path For vertex x1x2 xm there are n leaving arcswhich enter vertices whose first mminus 1 letters are x2x3 xm and the last letters inthis words are all different Therefore there is the path x1x2 xm x2x3 xmz1 xmz1 zmminus1 z1z2 zm
b) There are incoming arcs to vertex x1x2 xm from vertices yx1 xmminus1
1In a graph a walk is a sequence of neighbouring edges (or arcs with the same orientation) If theedges or arcs of the walk are all different the walk is called trail and when all vertices are differentthe walk is a path2A digraph (oriented graph) is connected if between every pair of vertices there is an oriented pathat least in a direction3A digraph is Eulerian if it contains a closed oriented trail with all arcs of the graph4In-degree (out-degree) of a vertex is the number of arcs which enter (leave) this vertex
1300 28 Complexity of Words
where y isin A (A is the alphabet of the graph ie V = Am) The arcs leavingvertex x1x2 xm enter vertices x2x3 xmy where y isin A Therefore the graph isEulerian because the in-degree and out-degree of each vertex are equal
From this the following theorem is simply obtained
Theorem 288 An oriented Eulerian trail of the graph B(nm) (which containsall arcs of graph) is a Hamiltonian path in the graph B(nm + 1) preserving theorder
For example in B(2 2) the sequence 000 001 010 101 011 111 110 100 ofarcs is an Eulerian trail At the same time these words are vertices of a Hamiltonianpath in B(2 3)
Algorithm to generate De Bruijn words Generating De Bruijn words isa common task with respectable number of algorithms We present here the well-known Martin algorithm Let A = a1 a2 an be an alphabet Our goal is togenerate an (nm)-type De Bruijn word over the alphabet A
We begin the algorithm with the word am1 and add at its right end the letter ak
with the greatest possible subscript such that the suffix of length m of the obtainedword does not duplicate a previously occurring subword of length m Repeat thisuntil such a prolongation is impossible
When we cannot continue a De Bruijn word is obtained with the length nm +m minus 1 In the following detailed algorithm A is the n-letters alphabet and B =(b1 b2 ) represents the result an (nm)-type De Bruijn word
Martin(Anm)
1 for ilarr 1 to m2 do bi larr a1
3 ilarr m4 repeat5 done larr true6 k larr n7 while k gt 18 do if biminusm+2biminusm+3 biak 6sub b1b2 bi Not a subword9 then ilarr i+ 1
10 bi larr ak
11 done larr false12 exit while13 else k larr k minus 114 until done15 return B B = (b
1b2 bnm+m+1)
Because this algorithm generates all letters of a De Bruijn word of length (nm +mminus1) and n and m are independent its time complexity is Ω(nm) The more precisecharacterization of the running time depends on the implementation of line 8 The
281 Simple complexity measures 1301
repeat statement is executed nmminus1 times The while statement is executed at mostn times for each step of the repeat The test biminusm+2biminusm+3 biak 6sub b1b2 bi canbe made in the worst case in mnm steps So the total number of steps is not greaterthan mn2m+1 resulting a worst case bound Θ(nm+1) If we use Knuth-Morris-Prattstring mathching algorithm then the worst case running time is Θ(n2m)
In chapter a more efficient implementation of the idea of Martin is presentedBased on this algorithm the following theorem can be stated
Theorem 289 An (nm)-type De Bruijn word is the shortest possible among allwords containing all words of length m over an alphabet with n letters
To generate all (nm)-type De Bruijn words the following recursive algorithm isgiven Here A is also an alphabet with n letters and B represents an (nm)-type DeBruijn word The algorithm is called for each position i with m+1 le i le nm +mminus1
All-De-Bruijn(B im)
1 for j larr 1 to n2 do bi larr aj
3 if biminusm+1biminusm+2 bi 6sub b1b2 biminus1 Not a subword4 then All-De-Bruijn(b i+ 1m)5 else if length(B) = nm +mminus 16 then print B A De Bruijn word7 exit for
The call of the procedure
for i = 1 to mdo bi larr a1
All-De-Bruijn (Bm+ 1m)
This algorithm naturally is exponentialIn following related to the De Bruijn graphs the so-called De Bruijn trees will
play an important roleA De Bruijn tree T (nw) with the root w isin Am is a n-ary tree defined
recursively as follows
i The word w of length m over the alphabet A = a1 a2 an is the root ofT (nw)ii If x1x2 xm is a leaf in the tree T (nw) then each word v of the formx2x3 xma1 x2x3 xma2 x2x3 xman will be a descendent of x1x2 xmif in the path from root to x1x2 xm the vertex v does not appearsiii The rule ii is applied as many as it can
In Figure 283 the De Bruijn tree T (2 010) is given
Rauzy graphs If the word u is infinite and V = Fn(u) E = Fn+1(u) thenthe corresponding word graph is called Rauzy graph (or subword graph) Figure
1302 28 Complexity of Words
Figure 283 The De Bruijn tree T (2 010)
Figure 284 Rauzy graphs for the infinite Fibonacci word
284 illustrates the Rauzy graphs of the infinite Fibonacci word for n = 3 and n = 4As we have seen the infinite Fibonacci word is
f = 0100101001001010010100100101001001
and F1(f) = 0 1 F2(f) = 01 10 00F3(f) = 010 100 001 101 F4(f) = 0100 1001 0010 0101 1010F5(f) = 01001 10010 00101 01010 10100 00100
In the case of the power word p = 01001100011100001111 0n1n whereF1(p) = 0 1 F2(p) = 01 10 00 11F3(p) = 010 100 000 001 011 111 110F4(p) = 0100 1001 0011 0110 1100 1000 0000 0001 0111 1110 1111the corresponding Rauzy graphs are given in Figure 285
As we can see in Fig 284 and 285 there are subwords of length n which can be
281 Simple complexity measures 1303
Figure 285 Rauzy graphs for the power word
continued only in a single way (by adding a letter) and there are subwords whichcan be continued in two different ways (by adding two different letters) These lattersubwords are called special subwords A subword v isin Fn(u) is a right specialsubword if there are at least two different letters a isin A such that va isin Fn+1(u)Similarly v isin Fn(u) is left special subword if there are at least two differentletters a isin A such that av isin Fn+1(u) A subword is bispecial if at the same timeis right and left special For example the special subwords in Figures 284 and 285)are
left special subwords 010 0100 (Figure 284)110 000 111 1110 0001 1111 0011 (Figure 285)
right special subwords 010 0010 ( Figure 284)011 000 111 0111 1111 0011 (Figure 285)
bispecial subwords 010 (Figure 284)000 111 1111 0011 (Figure 285)
2814 Complexity of words
The complexity of words measures the diversity of the subwords of a word In thisregard the word aaaaa has smaller complexity then the word abcab
We define the following complexities for a word
1) The subword complexity or simply the complexity of a word assigns toeach n isin N the number of different subwords of length n For a word u the number
1304 28 Complexity of Words
of different subwords of length n is denoted by fu(n)
fu(n) = Fn(u) u isin Ainfin
If the word is finite then fu(n) = 0 if n gt |u|2) The maximal complexity is considered only for finite words
C(u) = maxfu(n) | n ge 1 u isin Alowast
If u is an infinite word then Cminusu (n) is the lower maximal complexity respectively
C+u (n) the upper maximal complexity
Cminusu (n) = min
iC(uiui+1 ui+nminus1) C+
u (n) = maxiC(uiui+1 ui+nminus1)
3) The global maximal complexity is defined on the set An
G(n) = maxC(u) | u isin An
4) The total complexity for a finite word is the number of all differentnonempty subwords5
K(u) =|u|sum
i=1
fu(i) u isin Alowast
For an infinite word Kminusu (n) is the lower total complexity and K+
u (n) is the uppertotal complexity
Kminusu (n) = min
iK(uiui+1 ui+nminus1) K+
u (n) = maxiK(uiui+1 ui+nminus1)
5) A decomposition u = u1u2 uk is called a factorization of u If each ui
(with the possible exception of uk) is the shortest prefix of uiui+1 uk which doesnot occur before in u then this factorization is called the Lempel-Ziv factorizationThe number of subwords ui in such a factorization is the Lempel-Ziv factoriza-tion complexity of u For example for the word u = ababaaabb the Lempel-Zivfactorization is u = ababaaabb So the Lempel-Ziv factorization complexity of uis lz(u) = 4
6) If in a factorization u = u1u2 uk each ui is the longest possible palin-drome then the factorization is called a palindromic factorization and the num-ber of subwords ui in this is the palindromic factorization complexity Foru = aababbabbabb = aababbabbabb so the palindromic factorization complexity ofu is pal(u) = 3
7) The window complexity Pw is defined for infinite words only For u =u0u1u2 un the window complexity is
Pw(u n) =
uknukn+1 u(k+1)nminus1
∣
∣ k ge 0
5Sometimes the empty subword is considered too In this case the value of total complexity isincreased by 1
281 Simple complexity measures 1305
Subword complexity As we have seen
fu(n) = Fn(u) forallu isin Ainfin n isin N
fu(n) = 0 if n gt |u|For example in the case of u = abacab
fu(1) = 3 fu(2) = 4 fu(3) = 4 fu(4) = 3 fu(5) = 2 fu(6) = 1
In Theorem 284 was stated that for the infinite Fibonacci word
ff (n) = n+ 1
In the case of the power word p = 010011 0k1k the complexity is
fp(n) =n(n+ 1)
2+ 1
This can be proved if we determine the difference fp(n+ 1)minus fp(n) which is equalto the number of words of length n which can be continued in two different ways toobtain words of length n+ 1 In two different ways can be extended only the wordsof the form 0k1nminusk (it can be followed by 1 always and by 0 when k le nminus k) and1k0nminusk (it can be followed by 0 always and by 1 when k lt n minus k) Consideringseparately the cases when n is odd and even we can see that
fp(n+ 1)minus fp(n) = n+ 1
and from this we get
fp(n) = n+ fp(nminus 1) = n+ (nminus 1) + fp(nminus 2) =
= n+ (nminus 1) + + 2 + fp(1) =n(n+ 1)
2+ 1
In the case of the Champernowne word
c = u0u1 un = 0 1 10 11 100 101 110 111 1000
= 0110111001011101111000
the complexity is fc(n) = 2n
Theorem 2810 If for the infinite word u isin Aω there exists an n isin N such thatfu(n) le n then u is ultimately periodic
Proof fu(1) ge 2 otherwise the word is trivial (contains just equal letters) Thereforethere is a k le n such that fu(k) = fu(k + 1) But
fu(k + 1)minus fu(k) =sum
visinFk(u)
(
a isin A | va isin Fk+1(u)
minus 1)
It follows that each subword v isin Fk(u) has only one extension to obtain va isin
1306 28 Complexity of Words
Fk+1(u) So if v = uiui+1 ui+kminus1 = ujuj+1 uj+kminus1 then ui+k = uj+k Be-cause Fk(u) is a finite set and u is infinite there are i and j (i lt j) for whichuiui+1 ui+kminus1 = ujuj+1 uj+kminus1 but in this case ui+k = uj+k is true tooThen from ui+1ui+2 ui+k = uj+1uj+2 uj+k we obtain the following equalityresults ui+k+1 = uj+k+1 therefore ui+l = uj+l is true for all l ge 0 values Thereforeu is ultimately periodic
A word u isin Aω is Sturmian if fu(n) = n+ 1 for all n ge 1Sturmian words are the least complexity infinite and non periodic words The
infinite Fibonacci word is Sturmian Because fu(1) = 2 the Sturmian words aretwo-letters words
From the Theorem 2810 it follows that each infinite and not ultimately periodicword has complexity at least n+ 1 ie
u isin Aω u not ultimately periodic rArr fu(n) ge n+ 1
The equality holds for Sturmian wordsInfinite words can be characterized using the lower and upper total complexity
too
Theorem 2811 If an infinite word u is not ultimately periodic and n ge 1 then
C+u (n) ge
[n
2
]
+ 1 K+u (n) ge
[
n2
4+ n
]
For the Sturmian words equality holds
Let us denote by x the fractional part of x and by bxc its integer partObviously x = bxc + x The composition of a function R by itself n times willbe denoted by Rn So Rn = R R R (n times) Sturmian words can becharacterized in the following way too
Theorem 2812 A word u = u1u2 is Sturmian if and only if there exists anirrational number α and a real number z such that for R(x) = x+ α
un =
0 if Rn(z) isin (0 1minus α) 1 if Rn(z) isin [1minus α 1)
or
un =
1 if Rn(z) isin (0 1minus α) 0 if Rn(z) isin [1minus α 1)
In the case of the infinite Fibonacci number these numbers are α = z = (radic
5+1)2Sturmian words can be generated by the orbit of a billiard ball inside a square
too A billiard ball is launched under an irrational angle from a boundary point ofthe square If we consider an endless move of the ball with reflection on boundariesand without friction an infinite trajectory will result We put an 0 in the word ifthe ball reaches a horizontal boundary and 1 when it reaches a vertical one In sucha way we generate an infinite word This can be generalized using an (s + 1)-letter
281 Simple complexity measures 1307
u fu(1) fu(2) fu(3) fu(4) fu(5) fu(6) fu(7) fu(8)00100011 2 4 5 5 4 3 2 100100100 2 3 3 3 3 3 2 100100101 2 3 4 4 4 3 2 100100110 2 4 5 5 4 3 2 100100111 2 4 5 5 4 3 2 100101000 2 3 5 5 4 3 2 100101001 2 3 4 5 4 3 2 100101011 2 4 4 4 4 3 2 101010101 2 2 2 2 2 2 2 111111111 1 1 1 1 1 1 1 1
Figure 286 Complexity of several binary words
alphabet and an (s+ 1)-dimensional hypercube In this case the complexity is
fu(n s+ 1) =min(ns)sum
i=0
ns(nminus i)i(sminus i)
If s = 1 then fu(n 2) = fu(n) = n+ 1 and if s = 2 then fu(n 3) = n2 + n+ 1
Maximal complexity For a finite word u
C(u) = maxfu(n) | n ge 1
is the maximal complexity In Figure 286 the values of the complexity function forseveral words are given for all possible length From this we can see for examplethat C(00100011) = 5 C(00100100) = 3 etc
For the complexity of finite words the following interesting result is true
Theorem 2813 If w is a finite word fw(n) is its complexity then there are thenatural numbers m1 and m2 with 1 le m1 le m2 le |w| such that
bull fw(n+ 1) gt fw(n) for 1 le n lt m1bull fw(n+ 1) = fw(n) for m1 le n lt m2bull fw(n+ 1) = fw(n)minus 1 for m2 le n le |w|
From the Figure 286 for example ifw = 00100011 then m1 = 3 m2 = 4w = 00101001 then m1 = 4 m2 = 4w = 00101011 then m1 = 2 m2 = 5
Global maximal complexity The global maximal complexity is
G(n) = maxC(u) | u isin An
1308 28 Complexity of Words
fu(i)u i = 1 i = 2 i = 3
000 1 1 1001 2 2 1010 2 2 1011 2 2 1100 2 2 1101 2 2 1110 2 2 1111 1 1 1
Figure 287 Complexity of all 3-length binary words
that is the greatest (maximal) complexity in the set of all words of length n on agiven alphabet The following problems arisebull what is length of the subwords for which the global maximal complexity is
equal to the maximal complexitybull how many such words exist
Example 281 For the alphabet A = 0 1 the Figure 287 and 288 contain the complexityof all 3-length and 4-length words
In this case of the 3-length words (Figure 287) the global maximal complexity is 2and this value is obtained for 1-length and 2-length subwords There are 6 such words
For 4-length words (Figure 288) the global maximal complexity is 3 and this value isobtained for 2-length words The number of such words is 8
To solve the above two problems the following notations will be used
R(n) = i isin 1 2 n | existu isin An fu(i) = G(n)
M(n) = u isin An C(u) = G(n)
In the table of Figure 289 values of G(n) R(n) M(n) are given for length upto 20 over on a binary alphabet
We shall use the following result to prove some theorems on maximal complexity
Lemma 2814 For each k isin Nlowast the shortest word containing all the qk words oflength k over an alphabet with q letters has qk + k minus 1 letters (hence in this wordeach of the qk words of length k appears only once)
Theorem 2815 If A = q and qk + k le n le qk+1 + k then G(n) = nminus k
Proof Let us consider at first the case n = qk+1 + k k ge 1From Lemma 2814 we obtain the existence of a word w of length qk+1 +k which
contains all the qk+1 words of length k+1 hence fw(k+1) = qk+1 It is obvious thatfw(l) = ql lt fw(k+ 1) for l isin 1 2 k and fw(k+ 1 + j) = qk+1minus j lt fw(k+ 1)for j isin 1 2 qk+1minus 1 Any other word of length qk+1 + k will have the maximal
281 Simple complexity measures 1309
fu(i)u i = 1 i = 2 i = 3 i = 4
0000 1 1 1 10001 2 2 2 10010 2 3 2 10011 2 3 2 10100 2 3 2 10101 2 2 2 10110 2 3 2 10111 2 2 2 11000 2 2 2 11001 2 3 2 11010 2 2 2 11011 2 3 2 11100 2 3 2 11101 2 3 2 11110 2 2 2 11111 1 1 1 1
Figure 288 Complexity of all 4-length binary words
complexity less than or equal to C(w) = fw(k + 1) hence we have G(n) = qk+1 =nminus k
For k ge 1 we consider now the values of n of the form n = qk+1 + k minus r withr isin 1 2 qk+1 minus qk hence qk + k le n lt qk+1 + k If from the word w oflength qk+1 + k considered above we delete the last r letters we obtain a wordwn of length n = qk+1 + k minus r with r isin 1 2 qk+1 minus qk This word will havefwn
(k + 1) = qk+1 minus r and this value will be its maximal complexity Indeed it isobvious that fwn
(k+1+j) = fwn(k+1)minusj lt fwn
(k+1) for j isin 1 2 nminuskminus1for l isin 1 2 k it follows that fwn
(l) le ql le qk le qk+1 minus r = fwn(k + 1) hence
C(wn) = fwn(k + 1) = qk+1 minus r Because it is not possible for a word of length
n = qk+1 + k minus r with r isin 1 2 qk+1 minus qk to have the maximal complexitygreater than qk+1 minus r it follows that G(n) = qk+1 minus r = nminus k
Theorem 2816 If A = q and qk + k lt n lt qk+1 + k + 1 then R(n) = k + 1if n = qk + k then R(n) = k k + 1
Proof In the first part of the proof of Theorem 2815 we proved for n = qk+1 + kk ge 1 the existence of a word w of length n for which G(n) = fw(k + 1) = n minus kThis means that k + 1 isin R(n) For the word w as well as for any other word wprime oflength n we have fwprime(l) lt fw(k+1) l 6= k+1 because of the special construction ofw which contains all the words of length k + 1 in the most compact way It followsthat R(n) = k + 1
As in the second part of the proof of Theorem 2815 we consider n = qk+1 +kminusr
1310 28 Complexity of Words
n G(n) R(n) M(n)
1 1 1 22 2 1 23 2 1 2 64 3 2 85 4 2 46 4 2 3 367 5 3 428 6 3 489 7 3 40
10 8 3 1611 8 3 4 55812 9 4 71813 10 4 85414 11 4 92015 12 4 95616 13 4 96017 14 4 91218 15 4 70419 16 4 25620 16 4 5 79006
Figure 289 Values of G(n) R(n) and M(n)
with r isin 1 2 qk+1minusqk and the word wn for which G(n) = fwn(k+1) = qk+1minusr
We have again k + 1 isin R(n) For l gt k + 1 it is obvious that the complexityfunction of wn or of any other word of length n is strictly less than fwn
(k + 1)We examine now the possibility of finding a word w with fw(k + 1) = n minus k forwhich fw(l) = n minus k for l le k We have fw(l) le ql le qk le qk+1 minus r hence theequality fw(l) = nminus k = qk+1minus r holds only for l = k and r = qk+1minus qk that is forw = qk + k
We show that for n = qk + k we have indeed R(n) = k k+ 1 If we start withthe word of length qk + kminus 1 generated by the Martinrsquos algorithm (or with anotherDe Bruijn word) and add to this any letter from A we obtain obviously a word v oflength n = qk + k which contains all the qk words of length k and qk = nminus k wordsof length k + 1 hence fv(k) = fv(k + 1) = G(n)
Having in mind the Martin algorithm (or other more efficient algorithms)words w with maximal complexity C(w) = G(n) can be easily constructed for eachn and for both situations in Theorem 2816
Theorem 2817 If A = q and qk + k le n le qk+1 + k then M(n) is equal to thenumber of different paths of length nminus k minus 1 in the de Bruijn graph B(q k + 1)
Proof From Theorems 2815 and 2816 it follows that the number M(n) of the
281 Simple complexity measures 1311
words of length n with global maximal complexity is given by the number of wordsw isin An with fw(k + 1) = nminus k It means that these words contain nminus k subwordsof length k + 1 all of them distinct To enumerate all of them we start successivelywith each word of k + 1 letters (hence with each vertex in B(q k + 1)) and we addat each step in turn one of the symbols from A which does not duplicate a word oflength k + 1 which has already appeared Of course not all of the trials will finishin a word of length n but those which do this are precisely paths in B(q k + 1)starting with each vertex in turn and having the length n minus k minus 1 Hence to eachword of length n with fw(k + 1) = n minus k we can associate a path and only one oflength nminus kminus 1 starting from the vertex given by the first k+ 1 letters of the initialword conversely any path of length n minus k minus 1 will provide a word w of length nwhich contains nminus k distinct subwords of length k + 1
M(n) can be expressed also as the number of vertices at level n minus k minus 1 in the
set
T (q w)∣
∣w isin Ak+1
of De Bruijn trees
Theorem 2818 If n = 2k + k minus 1 then M(n) = 22kminus1
Proof In the De Bruijn graph B(2 k) there are 22kminus1minusk different Hamiltonian cyclesWith each vertex of a Hamiltonian cycle a De Bruijn word begins (containing all
k-length subwords) which has maximal complexity so M(n) = 2k middot22kminus1minusk = 22kminus1
which proves the theorem
A generalization for an alphabet with q ge 2 letters
Theorem 2819 If n = qk + k minus 1 then M(n) = (q)qkminus1
Total complexity The total complexity is the number of different nonemptysubwords of a given word
K(u) =|u|sum
i=1
fu(i)
The total complexity of a trivial word of length n (of the form an n ge 1) isequal to n The total complexity of a rainbow word (with pairwise different letters)
of length n is equal ton(n+ 1)
2
The problem of existence of words with a given total complexity are studied inthe following theorems
Theorem 2820 If C is a natural number different from 1 2 and 4 then thereexists a nontrivial word of total complexity equal to C
Proof To prove this theorem we give the total complexity of the following k-length
1312 28 Complexity of Words
words
K(akminus1b) = 2k minus 1 for k ge 1
K(abkminus3aa) = 4k minus 8 for k ge 4
K(abcdkminus3) = 4k minus 6 for k ge 3
These can be proved immediately from the definition of the total complexity1 If C is odd then we can write C = 2k minus 1 for a given k It follows that
k = (C + 1)2 and the word akminus1b has total complexity C2 If C is even then C = 2`
21 If ` = 2h then 4k minus 8 = C gives 4k minus 8 = 4h and from this k = h+ 2results The word abkminus3aa has total complexity C
22 If ` = 2h + 1 then 4k minus 6 = C gives 4k minus 6 = 4h + 2 and from thisk = h+ 2 results The word abcdkminus3 has total complexity C
In the proof we have used more than two letters in a word only in the case ofthe numbers of the form 4h+ 2 (case 22 above) The new question is if there existalways nontrivial words formed only of two letters with a given total complexityThe answer is yes anew We must prove this only for the numbers of the form 4h+2If C = 4h+ 2 and C ge 34 we use the followings
K(abkminus7abbabb) = 8k minus 46 for k ge 10
K(abkminus7ababba) = 8k minus 42 for k ge 10
If h = 2s then 8kminus 46 = 4h+ 2 gives k = s+ 6 and the word abkminus7abbabb has totalcomplexity 4h+ 2
If h = 2s+ 1 then 8k minus 42 = 4h+ 2 gives k = s+ 6 and the word abkminus7ababbahas total complexity 4h + 2 For C lt 34 only 14 26 and 30 are feasible The wordab4a has total complexity 14 ab6a has 26 and ab5aba 30 Easily it can be provedusing a tree that for 6 10 18 and 22 such words does not exist Then the followingtheorem is true
Theorem 2821 If C is a natural number different from 1 2 4 6 10 18 and 22then there exists a nontrivial word formed only of two letters with the given totalcomplexity C
The existence of a word with a given length and total complexity is not alwaysassured as we will prove in what follows
In relation with the second problem a new one arises How many words of lengthn and complexity C there exist For small n this problem can be studied exhaustivelyLet A be of n letters and let us consider all words of length n over A By a computerprogram we have got Figure 2810 which contains the frequency of words with givenlength and total complexity
Let |A| = n and let φn(C) denote the frequency of the words of length n over Ahaving a complexity C Then we have the following easy to prove results
281 Simple complexity measures 1313
n = 2C 2 3φn(C) 2 2n = 3
C 3 4 5 6φn(C) 3 0 18 6n = 4
C 4 5 6 7 8 9 10φn(C) 4 0 0 36 48 144 24n = 5
C 5 6 7 8 9 10 11 12 13 14 15φn(C) 5 0 0 0 60 0 200 400 1140 1200 120n = 6
C 6 7 8 9 10 11 12 13φn(C) 6 0 0 0 0 90 0 0
C 14 15 16 17 18 19 20 21φn(C) 300 990 270 5400 8280 19800 10800 720
Figure 2810 Frequency of words with given total complexity
φn(C) = 0 if C lt n or C gtn(n+ 1)
2
φn(n) = nφn(2nminus 1) = 3n(nminus 1)
φn
(
n(n+ 1)2
minus 1)
=n(nminus 1)n
2
φn
(
n(n+ 1)2
)
= n
As regards the distribution of the frequency 0 the following are true
If C = n+ 1 n+ 2 2nminus 2 then φn(C) = 0 If C = 2n 2n+ 1 3nminus 5 then φn(C) = 0
The question is if there exists a value from which up to n(n+1)2 no more 0
frequency exist The answer is positive Let us denote by bn the least number betweenn and n(n+ 1)2 for which
φn(C) 6= 0 for all C with bn le C len(n+ 1)
2
The number bn exists for any n (in the worst case it may be equal to n(n+1)2)
Theorem 2822 If ` ge 2 0 le i le ` n =`(`+ 1)
2+ 2 + i then
bn =`(`2 minus 1)
2+ 3`+ 2 + i(`+ 1)
1314 28 Complexity of Words
Figure 2811 Graph for (2 4)-subwords when n = 6
282 Generalized complexity measures
As we have seen in the previous section a contiguous part of a word (obtained byerasing a prefix orand a suffix) is a subword or factor If we eliminate arbitraryletters from a word what is obtained is a scattered subword sometimes calledsubsequence Special scattered subwords in which the consecutive letters are atdistance at least d1 and at most d2 in the original word are called (d1 d2)-subwordsMore formally we give the following definition
Let n d1 le d2 s be positive integers and let u = x1x2 xn isin An be a wordover the alphabet A The word v = xi1
xi2 xis
wherei1 ge 1d1 le ij+1 minus ij le d2 for j = 1 2 sminus 1is le n
is a (d1 d2)-subword of length s of uFor example the (2 4)-subwords of aabcade are a ab ac aba aa acd abd aaeabae ace abe ad b ba bd bae be c cd ce ae d e
The number of different (d1 d2)-subwords of a word u is called (d1 d2)-complexity and is denoted by Cu(d1 d2)For example if u = aabcade then Cu(2 4) = 23
2821 Rainbow words
Words with pairwise different letters are called rainbow words The (d1 d2)-complexity of a rainbow word of length n does not depends on what letters itcontains and is denoted by C(n d1 d2)
To compute the (d1 d2)-complexity of a rainbow word of length n let us considerthe word a1a2 an (if i 6= j then ai 6= aj) and the corresponding digraph G =(VE) with
V =
a1 a2 an
E =
(ai aj) | d1 le j minus i le d2 i = 1 2 n j = 1 2 n
For n = 6 d1 = 2 d2 = 4 see Figure 2811The adjacency matrix A =
(
aij
)
i=1n
j=1n
of the graph is defined by
aij =
1 if d1 le j minus i le d20 otherwise
for i = 1 2 n j = 1 2 n
282 Generalized complexity measures 1315
Because the graph has no directed cycles the entry in row i and column j in Ak
(where Ak = Akminus1A with A1 = A) will represent the number of k-length directedpaths from ai to aj If A0 is the identity matrix (with entries equal to 1 only on thefirst diagonal and 0 otherwise) let us define the matrix R = (rij)
R = A0 +A+A2 + middot middot middot+Ak where Ak+1 = O (the null matrix)
The (d1 d2)-complexity of a rainbow word is then
C(n d1 d2) =nsum
i=1
nsum
j=1
rij
The matrix R can be better computed using a variant of the well-known Warshallalgorithm
Warshall(An)
1 W larr A2 for k larr 1 to n3 do for ilarr 1 to n4 do for j larr 1 to n5 do wij larr wij + wikwkj
6 return W
From W we obtain easily R = A0 +W The time complexity of this algorithmsis Θ(n3)
For example let us consider the graph in Figure 2811 The corresponding adja-cency matrix is
A =
0 0 1 1 1 00 0 0 1 1 10 0 0 0 1 10 0 0 0 0 10 0 0 0 0 00 0 0 0 0 0
After applying the Warshall algorithm
W =
0 0 1 1 2 20 0 0 1 1 20 0 0 0 1 10 0 0 0 0 10 0 0 0 0 00 0 0 0 0 0
R =
1 0 1 1 2 20 1 0 1 1 20 0 1 0 1 10 0 0 1 0 10 0 0 0 1 00 0 0 0 0 1
and then C(6 2 4) = 19 the sum of entries in RThe Warshall algorithm combined with the Latin square method can be used
to obtain all nontrivial (with length at least 2) (d1 d2)-subwords of a given rainbowword a1a2 an of length n Let us consider a matrix A with the elements Aij which
1316 28 Complexity of Words
n = 6 n = 7
d1d2 1 2 3 4 5
1 21 46 58 62 632 - 12 17 19 203 - - 9 11 124 - - - 8 95 - - - - 7
d1d2 1 2 3 4 5 6
1 28 79 110 122 126 1272 - 16 25 30 32 333 - - 12 15 17 184 - - - 10 12 135 - - - - 9 106 - - - - - 8
Figure 2812 (d1 d2)-complexity for rainbow words of length 6 and 7
are set of words Initially this matrix is defined as
Aij =
aiaj if d1 le j minus i le d2empty otherwise
for i = 1 2 n j = 1 2 n
If A and B are sets of words AB will be formed by the set of concatenation of eachword from A with each word from B
AB =
ab∣
∣ a isin A b isin B
If s = s1s2 sp is a word let us denote by primes the word obtained from s by erasingthe first character primes = s2s3 sp Let us denote by primeAij the set Aij in which weerase from each element the first character In this case primeA is a matrix with entriesprimeAij
Starting with the matrix A defined as before the algorithm to obtain all non-trivial (d1 d2)-subwords is the following
Warshall-Latin(A n)
1 W larr A2 for k larr 1 to n3 do for ilarr 1 to n4 do for j larr 1 to n5 do if Wik 6= empty and Wkj 6= empty6 then Wij larrWij cupWik
primeWkj
7 return W
The set of nontrivial (d1 d2)-subwords is⋃
ijisin12nWij The time complexity
is also Θ(n3)For n = 7 d1 = 2 d2 = 4 the initial matrix is
282 Generalized complexity measures 1317
A =
empty empty ac ad ae empty emptyempty empty empty bd be bf emptyempty empty empty empty ce cf cgempty empty empty empty empty df dgempty empty empty empty empty empty egempty empty empty empty empty empty emptyempty empty empty empty empty empty emptyempty empty empty empty empty empty empty
and
W =
empty empty ac ad ace ae adf acf aeg aceg adg acgempty empty empty bd be bdf bf beg bdgempty empty empty empty ce cf ceg cgempty empty empty empty empty df dgempty empty empty empty empty empty egempty empty empty empty empty empty emptyempty empty empty empty empty empty empty
Counting the one-letter subwords too we obtain C(7 2 4) = 30
The case d1 = 1d1 = 1d1 = 1 In this case instead of d2 we will use d For a rainbow wordaid we will denote the number of (1 d)-subwords which finish at the position i Fori = 1 2 n
aid = 1 + aiminus1d + aiminus2d + + aiminusdd (281)
For simplicity let us denote C(n 1 d) by N(n d) The (1 d)-complexity of arainbow word can be obtained by the formula
N(n d) =nsum
i=1
aid
Because of (281) we can write in the case of d ge 2
aid +1
dminus 1=(
aiminus1d +1
dminus 1
)
+ middot middot middot+(
aiminusdd +1
dminus 1
)
Denoting
bid = aid +1
dminus 1 and cid = (dminus 1)bid
we getcid = ciminus1d + ciminus2d + + ciminusdd
and the sequence cid is one of Fibonacci-type For any d we have a1d = 1 andfrom this c1d = d results Therefore the numbers cid are defined by the followingrecurrence equations
cnd = cnminus1d + cnminus2d + + cnminusdd for n gt 0cnd = 1 for n le 0
1318 28 Complexity of Words
These numbers can be generated by the following generating function
Fd(z) =sum
nge0
cndzn =
1 + (dminus 2)z minus z2 minus middot middot middot minus zd
1minus 2z + zd+1
=1 + (dminus 3)z minus (dminus 1)z2 + zd+1
(1minus z)(1minus 2z + zd+1)
The (1 d)-complexity N(n d) can be expressed with these numbers cnd by thefollowing formula
N(n d) =1
dminus 1
(
nsum
i=1
cid minus n)
for d gt 1
and
N(n 1) =n(n+ 1)
2
orN(n d) = N(nminus 1 d) +
1dminus 1
(cnd minus 1) for d gt 1 n gt 1
If d = 2 then
F2(z) =1minus z2
1minus 2z + z3=
1 + z
1minus z minus z2=F (z)z
+ F (z)
where F (z) is the generating function of the Fibonacci numbers Fn (with F0 =0 F1 = 1) Then from this formula we have
cn2 = Fn+1 + Fn = Fn+2
and
N(n 2) =nsum
i=1
Fi+2 minus n = Fn+4 minus nminus 3
Figure 2813 contains the values of N(n d) for k le 10 and d le 10
N(n d) = 2n minus 1 for any d ge nminus 1
The following theorem gives the value of N(n d) in the case n ge 2dminus 2
Theorem 2823 For n ge 2dminus 2 we have
N(n nminus d) = 2n minus (dminus 2) middot 2dminus1 minus 2
The main step in the proof is based on the formula
N(n nminus dminus 1) = N(n nminus d)minus d middot 2dminus1
The value of N(n d) can be also obtained by computing the number of sequences
282 Generalized complexity measures 1319
n d 1 2 3 4 5 6 7 8 9 101 1 1 1 1 1 1 1 1 1 12 3 3 3 3 3 3 3 3 3 33 6 7 7 7 7 7 7 7 7 74 10 14 15 15 15 15 15 15 15 155 15 26 30 31 31 31 31 31 31 316 21 46 58 62 63 63 63 63 63 637 28 79 110 122 126 127 127 127 127 1278 36 133 206 238 250 254 255 255 255 2559 45 221 383 464 494 506 510 511 511 511
10 55 364 709 894 974 1006 1018 1022 1023 1023
Figure 2813 The (1 d)-complexity of words of length n
of length k of 0rsquos and 1rsquos with no more than dminus1 adjacent zeros In such a sequenceone 1 represents the presence one 0 does the absence of a letter of the word in agiven (1 d)-subword Let bnd denote the number of n-length sequences of zeros andones in which the first and last position is 1 and the number of adjacent zeros is atmost dminus 1 Then it can be proved easily that
bnd = bnminus1d + bnminus2d + + bnminusdd for k gt 1 b1d = 1 bnd = 0 for all n le 0
because any such sequence of length nminus i (i = 1 2 d) can be continued in orderto obtain a similar sequence of length n in only one way (by adding a sequence ofthe form 0iminus11 on the right) For bnd the following formula also can be derived
bnd = 2bnminus1d minus bnminus1minusdd
If we add one 1 or 0 at an internal position (eg at the (n minus 2)th position) of eachbnminus1d sequences then we obtain 2bnminus1d sequences of length n but from thesebnminus1minusdd sequences will have d adjacent zeros
The generating function corresponding to bnd is
Bd(z) =sum
nge0
bndzn =
z
1minus z middot middot middot minus zd=
z(1minus z)1minus 2z + zd+1
By adding zeros on the left andor on the right to these sequences we can obtainthe number N(k d) as the number of all these sequences Thus
N(k d) = bkd + 2bkminus1d + 3bkminus2d + middot middot middot+ kb1d
(i zeros can be added in i + 1 ways to these sequences 0 on the left and i on theright 1 on the left and iminus 1 on the right and so on)
From the above formula the generating function corresponding to the complexi-ties N(k d) can be obtained as a product of the two generating functions Bd(z) and
1320 28 Complexity of Words
A(z) =sum
nge0 nzn = 1(1minus z)2 thus
Nd(z) =sum
nge0
N(n d)zn =z
(1minus z)(1minus 2z + zd+1)
The case d2 = nminus 1d2 = nminus 1d2 = nminus 1 In the sequel instead of d1 we will use d In this case thedistance between two letters picked up to be neighbours in a subword is at least d
Let us denote by bnd(i) the number of (d n minus 1)-subwords which begin at theposition i in a rainbow word of length n Using our previous example (abcdef ) wecan see that b62(1) = 8 b62(2) = 5 b62(3) = 3 b62(4) = 2 b62(5) = 1 andb62(6) = 1
The following formula immediately results
bnd(i) = 1 + bnd(i+d) + bnd(i+d+1) +middot middot middot+ bnd(n) (282)
for n gt d and 1 le i le nminus d
bnd(1) = 1 for n le d
For simplicity C(n d n) will be denoted by K(n d)The (d nminus 1)-complexity of rainbow words can be computed by the formula
K(n d) =nsum
i=1
bnd(i) (283)
This can be expressed also as
K(n d) =nsum
k=1
bkd(1) (284)
because of the formula
K(n+ 1 d) = K(n d) + bn+1d(1)
In the case d = 1 the complexity K(n 1) can be computed easily K(n 1) =2n minus 1
From (282) we get the following algorithm for the computation of bnd(i) Thenumbers bnd(k) (k = 1 2 ) for a given n and d are obtained in the array b =(b1 b2 ) Initially all these elements are equal to minus1 The call for the given n andd and the desired i is
Input (n d i)for k larr 1 to n
do bk larr minus1B(n d i) Array b is a global oneOutput b1 b2 bn
282 Generalized complexity measures 1321
nd 1 2 3 4 5 6 7 8 9 10 11
1 1 1 1 1 1 1 1 1 1 1 12 3 2 2 2 2 2 2 2 2 2 23 7 4 3 3 3 3 3 3 3 3 34 15 7 5 4 4 4 4 4 4 4 45 31 12 8 6 5 5 5 5 5 5 56 63 20 12 9 7 6 6 6 6 6 67 127 33 18 13 10 8 7 7 7 7 78 255 54 27 18 14 11 9 8 8 8 89 511 88 40 25 19 15 12 10 9 9 9
10 1023 143 59 35 25 20 16 13 11 10 1011 2047 232 87 49 33 26 21 17 14 12 1112 4095 376 128 68 44 33 27 22 18 15 13
Figure 2814 Values of K(n d)
The recursive algorithm is the following
B(n d i)
1 plarr 12 for k larr i+ d to n3 do if bk = minus14 then B(n d k)5 plarr p+ bk
6 bi larr p7 return
This algorithm is a linear oneIf the call is B(8 2 1) the elements will be obtained in the following order
b7 = 1 b8 = 1 b5 = 3 b6 = 2 b3 = 8 b4 = 5 and b1 = 21
Lemma 2824 bn2(1) = Fn where Fn is the nth Fibonacci number
Proof Let us consider a rainbow word a1a2 an and let us count all its (2 nminus 1)-subwords which begin with a2 If we change a2 for a1 in each (2 n minus 1)-subwordwhich begin with a2 we obtain (2 nminus1)-subwords too If we add a1 in front of each(2 nminus 1)-subword which begin with a3 we obtain (2 nminus 1)-subwords too Thus
bn2(1) = bnminus12(1) + bnminus22(1)
So bn2(1) is a Fibonacci number and because b12(1) = 1 we obtain that bn2(1) =Fn
Theorem 2825 K(n 2) = Fn+2 minus 1 where Fn is the nth Fibonacci number
1322 28 Complexity of Words
Proof From equation (284) and Lemma 2824
K(n 2) = b12(1) + b22(1) + b32(1) + b42(1) + middot middot middot+ bn2(1)
= F1 + F2 + middot middot middot+ Fn
= Fn+2 minus 1
If we use the notation Mnd = bnd(1) because of the formula
bnd(1) = bnminus1d(1) + bnminusdd(1)
a generalized middle sequence will be obtained
Mnd = Mnminus1d +Mnminusdd for n ge d ge 2 (285)
M0d = 0 M1n = 1 Mdminus1d = 1
Let us call this sequence d-middle sequence Because of the equality Mn2 =Fn the d-middle sequence can be considered as a generalization of the Fibonaccisequence
Then next linear algorithm computes Mnd by using an array M0M1 Mdminus1
to store the necessary previous elements
Middle(n d)
1 M0 larr 02 for ilarr 1 to dminus 13 do Mi larr 14 for ilarr d to n5 do Mi mod d larrM(iminus1) mod d +M(iminusd) mod d
6 print Mi mod d
7 return
Using the generating function Md(z) =sum
nge0
Mndzn the following closed formula
can be obtainedMd(z) =
z
1minus z minus zd (286)
This can be used to compute the sum snd =nsum
n=1
Mid which is the coefficient of
zn+d in the expansion of the function
zd
1minus z minus zdmiddot 1
1minus z =zd
1minus z minus zd+
z
1minus z minus zdminus z
1minus z
So snd = Mn+(dminus1)d +Mnd minus 1 = Mn+dd minus 1 Therefore
nsum
i=1
Mid = Mn+dd minus 1 (287)
283 Palindrome complexity 1323
Theorem 2826 K(n d) = Mn+ddminus1 where n gt d and Mnd is the nth elementsof d-middle sequence
Proof The proof is similar to that in Theorem 2825 taking into account the equation(287)
Theorem 2827 K(n d) =sum
kge0
(
nminus (dminus 1)kk + 1
)
for n ge 2 d ge 1
Proof Let us consider the generating function G(z) =1
1minus z = 1 + z + z2 + middot middot middot Then taking into account the equation (286) we obtain Md(z) = zG(z + zd) =z+ z(z+ zd)+ z(z+ zd)2 + middot middot middot+ z(z+ zd)i + middot middot middot The general term in this expansionis equal to
zi+1isum
k=1
(
i
k
)
z(dminus1)k
and the coefficient of zn+1 is equal to
sum
kge0
(
nminus (dminus 1)kk
)
The coefficient of zn+d is
Mn+dd =sum
kge0
(
n+ dminus 1minus (dminus 1)kk
)
(288)
By Theorem 2826 K(n d) = Mn+dd minus 1 and an easy computation yields
K(n d) =sum
kge0
(
nminus (dminus 1)kk + 1
)
2822 General words
The algorithm Warshall-Latin can be used for nonrainbow words too with theremark that repeating subwords must be eliminated For the word aabbbaaa andd1 = 2 d2 = 4 the result is ab abb aba abba abaa aa aaa bb ba bba baa and witha and b we have Caabbbaaa(2 4) = 13
283 Palindrome complexity
The palindrome complexity function palw of a finite or infinite word w attachesto each n isin N the number of palindrome subwords of length n in w denoted by
1324 28 Complexity of Words
palw(n)The total palindrome complexity of a finite word w isin Alowast is equal to the
number of all nonempty palindrome subwords of w ie
P (w) =|w|sum
n=1
palw(n)
This is similar to the total complexity of words
2831 Palindromes in finite words
Theorem 2828 The total palindrome complexity P (w) of any finite word w sat-isfies P (w) le |w|
Proof We proceed by induction on the length n of the word w For n = 1 we haveP (w) = 1
We consider n ge 2 and suppose that the assertion holds for all words of lengthnminus 1 Let w = a1a2 an be a word of length n and u = a1a2 anminus1 its prefix oflength nminus 1 By the induction hypothesis it is true that P (u) le nminus 1
If an 6= aj for each j isin 1 2 nminus 1 the only palindrome in w which is not inu is an hence P (w) = P (u) + 1 le n
If there is an index j 1 le j le n minus 1 such that an = aj then P (w) gt P (u) ifand only if w has suffixes which are palindromes Let us suppose that there are atleast two such suffixes aiai+1 an and ai+kai+k+1 an 1 le k le nminus i which arepalindromes It follows that
ai = an = ai+k
ai+1 = anminus1 = ai+k+1
middot middot middotanminusk = ai+k = an
hence ai+k an = ai anminusk The last palindrome appears in u (because of k ge 1)and has been already counted in P (u) It follows that P (w) le P (u) + 1 le n
This result shows that the total number of palindromes in a word cannot belarger than the length of that word We examine now if there are words which arelsquopoorrsquo in palindromes In the next lemma we construct finite words wn of arbitrarylength n ge 9 which contain precisely 8 palindromes
Let us denote by wp
q the fractional power of the word w of length q which isthe prefix of length p of wp
Lemma 2829 If wn = (001011)n6 n ge 9 then P (wn) = 8
Proof In wn there are the following palindromes 0 1 00 11 010 101 0110 1001Because 010 and 101 are situated in wn between 0 on the left and 1 on the rightthese cannot be continued to obtain any palindromes The same is true for 1001 and0110 which are situated between 1 on the left and 0 on the right excepting the
283 Palindrome complexity 1325
cases when 1001 is a suffix So there are no other palindromes in wn
Remark 2830 If u is a circular permutation of 001011 and n ge 9 then P (un6 ) = 8
too Because we can interchange 0 with 1 for any n there will be at least 12 wordsof length n with total complexity equal to 8
We shall give now beside the upper delimitation from Theorem 2828 lowerbounds for the number of palindromes contained in finite binary words (In thetrivial case of a 1-letter alphabet it is obvious that for any word w P (w) = |w| )Theorem 2831 If w is a finite word of length n on a 2-letter alphabet then
P (w) = n for 1 le n le 7 7 le P (w) le 8 for n = 8 8 le P (w) le n for n ge 9
Proof Up to 8 the computation can be made by a computer program For n ge 9Lemma 2829 gives words vn for which P (vn) = 8 The maximum value is obtainedfor words of the form an a isin A n isin N
Remark 2832 For all the short binary words (up to |w| = 7) the palindromecomplexity takes the maximum possible value given in Theorem 2828 from the wordswith |w| = 8 only four (out of 28) have P (w) = 7 namely 00110100 00101100 andtheir complemented words
In the following lemmas we construct binary words which have a given totalpalindrome complexity greater than or equal to 8
Lemma 2833 If uk` = 0k10110`1 for k ge 2 and 1 le ` le k minus 1 then P (uk`) =k + 6
Proof In the prefix of length k of uk` there are always k palindromes (1 1k)The other palindromes different from these are 1 11 010 101 0110 and 10`1 (for` ge 2) respectively 101101 (for ` = 1) In each case P (uk`) = k + 6
Lemma 2834 If vk` = (0k1011)k+`+5
k+4 for k ge 2 and k le ` le n minus k minus 5 thenP (vk`) = k + 6
Proof Since ` ge k the prefix of ukj is at least 0k10110k1 which includes thepalindromes 0 0k 1 11 010 101 0110 and 10k1 hence P (vk`) ge k + 6 Thepalindromes 010 and 101 are situated between 0 and 1 while 0110 and 10k1 arebetween 1 and 0 (excepting the cases when they are suffixes) no matter how largeis ` It follows that vk` contains no other palindromes hence P (vk`) = k + 6
Remark 2835 If k = 2 then the word v2` is equal to w`+7 with wn defined inLemma 2829
1326 28 Complexity of Words
We can determine now precisely the image of the restriction of the palindromecomplexity function to An n ge 1
Theorem 2836 Let A be a binary alphabet Then
P (An) =
n
for 1 le n le 7
7 8
for n = 8
8 n
for n ge 9
Proof Having in mind the result in Theorem 2831 we have to prove only that foreach n and i so that 8 le i le n there exists always a binary word wni of length nfor which the total palindrome complexity is P (wni) = i Let n and i be given sothat 8 le i le n We denote k = iminus 6 ge 2 and ` = nminus k minus 5
If ` le kminus 1 we take wni = uk` (from Lemma 2833) if ` ge k wni = vk` (fromLemma 2834) It follows that |wni| = n and P (wni) = k + 6 = i
Example 282 Let us consider n = 25 and i = 15 Then k = 15minus6 = 9 ` = 26minus15 = 11
Because ` gt k minus 1 we use v911 = (091011)2513 = 09101109101 whose total palindrome
complexity is 15
We give similar results for the case of alphabets with q ge 3 letters
Theorem 2837 If w is a finite word of length n over a q-letter (q ge 3) alphabetthen
P (w) = n for n isin 1 2 3 le P (w) le n for n ge 3
Proof For n isin 1 2 it can be checked directly Let us consider now n ge 3 and aword of length at least 3 If this is a trivial word (containing only one letter n times)its total palindrome complexity is n ge 3 If in the word there appear exactly twoletters a1 and a2 it will have as palindromes those two letters and at least one ofa2
1 a22 a1a2a1 or a2a1a2 hence again P (w) ge 3 If the word contains a third letter
then obviously P (w) ge 3 So the total complexity cannot be less then 3
Theorem 2838 Let A be a q-letter (q ge 3) alphabet Then for
P (An) =
n
for 1 le n le 2
3 n
for n ge 3
Proof It remains to prove that for each n and i so that 3 le i le n there exists alwaysa word wni of length n for which the total palindrome complexity is P (wni) = i
Such a word is wni = aiminus31 (a1a2a3)
nminusi+3
3 which has iminus 2 palindromes in its prefixof length iminus 2 and other two palindromes a2 and a3 in what follows
283 Palindrome complexity 1327
2832 Palindromes in infinite words
Sturmian words The number of palindromes in the infinite Sturmian words isgiven by the following theorem
Theorem 2839 If u is an infinite Sturmian word then
palu(n) =
1 if n is even 2 if n is odd
Power word Let us recall the power word as beingp = 01001100011100001111 0n1n
Theorem 2840 The palindrome complexity of the power word p is
palp(n) = 2lfloorn
3
rfloor
+ 1 + ε
where
ε =
0 if n divisible by 3 1 otherwise
Proof There exist the following cases
Case n = 3k Palindrome subwords are
0i13kminus2i0i for i = 0 1 k 1i03kminus2i1i for i = 0 1 k minus 1 so palp(3k) = 2k + 1
Case n = 3k + 1 Palindrome subwords are
0i13k+1minus2i0i for i = 0 1 k 1i03k+1minus2i1i for i = 0 1 k so palp(3k + 1) = 2k + 2
Case n = 3k + 2 Palindrome subwords are
0i13k+2minus2i0i for i = 0 1 k 1i03k+2minus2i1i for i = 0 1 k so palp(3k + 2) = 2k + 2
The palindrome subwords of the power word have the following propertiesbull Every palindrome subword which contains both 0rsquos and 1rsquos occurs only once in thepower wordbull If we use the notations Uiji = 0i1j0i and Viji = 1i0j1i then there are the uniquedecompositions
p = U111U121U232U242U353U363 Uk2kminus1kUk2kk
p = 0V121V232V141V353V262V474V383 Vk+12k+1k+1Vk2k+2k
Champernowne word The Champernowne word is defined as the concatenationof consecutive binary written natural numbers
c = 0 1 10 11 100 101 110 111 1000 1001 1010 1011 1100
1328 28 Complexity of Words
Theorem 2841 The palindrome complexity of the Champernowne word is
palc(n) = 2b n2
c+ε
where
ε =
0 if n is even 1 if n is odd
Proof Any palindrome w of length n can be continued as 0w0 and 1w1 to obtainpalindromes of length n + 2 This theorem results from the following palc(1) = 2palc(2) = 2 and for n ge 1 we have
palc(2n+ 1) = 2palc(2nminus 1) palc(2n+ 2) = 2palc(2n)
The following algorithm generates all palindromes up to a given length of aSturmian word beginning with the letter a and generated by the morphism σ
The idea is the following If p is the least value for which σp(a) and σp(b) areboth of odd length (such a p always exists) we consider conjugates6 of these wordswhich are palindromes (such conjugates always exists) and we define the followingmorphism
π(a) = conj(
σp(a))
π(b) = conj
(
σp(b))
where conj(u) produces a conjugate of u which is a palindrome
The sequences(
πn(a))
nge0and
(
πn(b))
nge0generate all odd length palindromes
and the sequence(
πn(aa))
nge0all even length palindromes
If α is a word then primeαprime represents the word which is obtained from α by erasingits first and last letter More generally mprimeαprimem is obtained from α by erasing its firstm and last m letters
6If w = uv then vu is a conjugate of w
283 Palindrome complexity 1329
Sturmian-Palindromes(n)
1 if n is even2 then nlarr nminus 13 let p be the least value for which σp(a) and σp(b) are both of odd length4 let define the morphism π(a) = conj
(
σp(a))
and π(b) = conj(
σp(b))
5 αlarr a6 while |α| lt n7 do αlarr π(α)8 mlarr (|α| minus n)29 αlarr mprimeαprimem
10 β larr b11 while |β| lt n12 do β larr π(β)13 mlarr (|β| minus n)214 β larr mprimeβprimem
15 repeat print α β Printing odd length palindromes16 αlarrprimeαprime
17 β larrprimeβprime
18 until α = ε and β = ε19 γ larr aa20 while |γ| lt n+ 121 do γ larr π(γ)22 mlarr (|γ| minus nminus 1)223 γ larr mprimeγprimem
24 repeat print γ Printing even length palindromes25 γ larrprimeγprime
26 until γ = ε
Because any substitution requires no more than cn steps where c is a constantthe algorithm is a linear one
In the case of the Fibonacci word the morphism σ is defined byσ(a) = ab σ(b) = a
and becauseσ(a) = ab σ2(a) = aba σ3(a) = abaab |σ3(a)| = |abaab| = 5σ(b) = a σ(b) = ab σ3(b) = aba |σ3(b)| = |aba| = 3
both being odd numbers p will be equal to 3The word abaab is not a palindrome and for the morphism π we will use the
adequate conjugate ababa which is a palindromeIn this case the morphism π is defined byπ(a) = ababaπ(b) = aba
For example if n = 14 the following are obtainedπ2(a) = ababa aba ababa aba ababa and then α = aabaababaabaaπ2(b) = ababa aba ababa and β = ababaabaababaπ3(aa) = ababaabaababaabaababaababaabaababaabaababa and
1330 28 Complexity of Words
γ = baababaababaabThe odd palindromes obtained are
aabaababaabaa ababaabaababaabaababaaba babaabaababbaababaab abaabaabaaababaa baabaabababa aabaabab abaa b
The even palindromes obtained are
baababaababaabaababaababaaababaababababaabababaababaabaa
Problems
28-1 Generating function 1Let bnd denote the number of sequences of length n of zeros and ones in which thefirst and last position is 1 and the number of adjacent zeros is at most dminus 1 Provethat the generating function corresponding to bnd is
Bd(z) =sum
nge0
bndzn =
z(1minus z)1minus 2z + zd+1
Hint See Subsection 2821)28-2 Generating function 2Prove that the generating function of N(n d) the number of all (1 d)-subwords ofa rainbow word of length n is
Nd(z) =sum
nge0
N(n d)zn =z
(1minus z)(1minus 2z + zd+1)
(Hint (See Subsection 2821)28-3 Window complexityCompute the window complexity of the infinite Fibonacci word28-4 Circuits in De Bruijn graphsProve that in the De Bruijn graph B(qm) there exist circuits (directed cycles) ofany length from 1 to qm
Chapter Notes
The basic notions and results on combinatorics of words are given in Lothairersquos [142143 144] and Foggrsquos books [65] Neither Lothaire nor Fogg is a single author theyare pseudonyms of groups of authors A chapter on combinatorics of words writtenby Choffrut and Karhumaumlki [42] appeared in a handbook on formal languages
The different complexities are defined as follows total complexity in Ivaacutenyi [111]maximal and total maximal complexity in Anisiu Blaacutezsik Kaacutesa [5] (1 d)-complexity
Notes for Chapter 28 1331
in Ivaacutenyi [111] (called d-complexity) and used also in Kaacutesa [135]) (d n minus 1)-complexity (called super-d-complexity) in Kaacutesa [137] scattered complexity in Kaacutesa[136] factorization complexity in Ilie [109] and window complexity in CassaigneKaboreacute Tapsoba [37]
The power word lowerupper maximaltotal complexities are defined in Fer-enczi Kaacutesa [62] In this paper a characterization of Sturmian words by upper maxi-mal and upper total complexities (Theorem 2811) is also given The maximal com-plexity of finite words is studied in Anisiu Blaacutezsik Kaacutesa [5] The total complexityof finite words is described in Kaacutesa [135] where the results of the Theorem 2822 isconjectured too and proved later by Leveacute and Seacuteeacutebold [140]
Different generalized complexity measures of words are defined and studied byIvaacutenyi [111] and Kaacutesa [135 137 136]
The results on palindrome complexity are described in M-C Anisiu V AnisiuKaacutesa [4] for finite words and in Droubay Pirillo [54] for infinite words The algorithmfor palindrome generation in Sturmian words is from this paper too
Applications of complexities in social sciences are given in Elzinga [57 56] andin biology in Troyanskaya et al [233]
It is worth to consult other papers too such as [7 36 51 61 216] (on complexityproblems) and [2 9 23 26 31 49 108] (on palindromes)
29 Conflict Situations
In all areas of everyday life there are situations when conflicting aspects have tobe taken into account simultaneously A problem becomes even more difficult whenseveral decision makersrsquo or interest groupsrsquo mutual agreement is needed to find asolution
Conflict situations are divided in three categories in terms of mathematics
1 One decision maker has to make a decision by taking several conflicting aspectsinto account
2 Several decision makers have to find a common solution when every decisionmaker takes only one criterion into account
3 Several decision makers look for a common solution but every decision makertakes several criteria into account
In the first case the problem is a multi-objective optimization problem where theobjective functions make up the various aspects The second case is a typical problemof the game theory when the decision makers are the players and the criteria meanthe payoff functions The third case appears as Pareto games in the literature whenthe different players only strive to find Pareto optimal solutions instead of optimalones
In this chapter we will discuss the basics of this very important and complextopic
291 The basics of multi-objective programming
Suppose that one decision maker wants to find the best decision alternative on thebasis of several usually conflicting criteria The criteria usually represent decisionobjectives At first these are usually defined verbally such as clean air cheap main-tenance etc Before the mathematical model is given firstly these objectives haveto be described by quantifiable indices It often occurs that one criterion is describedby more than one indices such as the quality of the air since the simultaneous pres-ence of many types of pollution have an effect on it In mathematics it is usually
291 The basics of multi-objective programming 1333
assumed that the bigger value of the certain indices (they will be called objectivefunctions) means favorable value hence we want to maximize all of the objectivefunctions simultaneously If we want to minimize one of the objective functions wecan safely multiply its value by (minus1) and maximize the resulting new objectivefunction If in the case of one of the objective functions the goal is to attain somekind of optimal value we can maximize the deviation from it by multiplying it by(minus1)
If X denotes the set of possible decision alternatives and fi X rarr R denotes theith objective function (i = 1 2 I) the problem can be described mathematicallyas follows
fi(x)rarr max (i = 1 2 I) (291)
supposing that x isin XIn the case of a single objective function we look for an optimal solution
Optimal solutions satisfy the following requirements
(i) An optimal solution is always better than any non-optimal solution
(ii) There is no such possible solution that provides better objective functions thanan optimal solution
(iii) If more than one optimal solution exist simultaneously they are equivalent inthe meaning that they have the same objective functions
These properties come from the simple fact that the consequential space
H = u|u = f(x) for some x isin X (292)
is a subset of the real number line which is totally ordered In the case of multipleobjective functions the
H = u = (u1 uI)|ui = fi(x) i = 1 2 I for some x isin X (293)
consequential space is a subset of the I-dimensional Euclidean space which is onlypartially ordered Another complication results from the fact that a decision alterna-tive that could maximize all of the objective functions simultaneously doesnrsquot usuallyexist
Letrsquos denotef
i = maxfi(x)|x isin X (294)
the maximum of the ith objective function then the
f = (f1 f
I )
point is called ideal point If f isin H then there exits an x decision for whichfi(x) = f
i i = 1 2 I In such special cases x satisfies the previously defined(i)-(iii) conditions However if f isin H the situation is much more complicated Inthat case we look for Pareto optimal solutions instead of optimal ones
1334 29 Conflict Situations
Definition 291 An alternative x isin X is said to be Pareto optimal if there isno x isin X such that fi(x) ge fi(x) for all i = 1 2 I with at least one strictinequality
It is not necessary that a multi-purpose optimization problem has Pareto optimalsolution as the case of the
H = (f1 f2)|f1 + f2 lt 1
set shows it Since H is open set (f1 + ε1 f2 + ε2) isin H for arbitrary (f1 f2) isin Hand for a small enough positive ε1 and ε2
Theorem 292 If X bounded closed in a finite dimensional Euclidean space andall of the objective functions are continuous there is Pareto optimal solution
The following two examples present a discrete and a continuous problem
Example 291 Assume that during the planning of a sewage plant one out of two optionsmust be chosen The expenditure of the first option is two billion Ft and its daily capacityis 1500 m3 The second option is more expensive three billion Ft with 2000 m3 dailycapacity In this case X = 1 2 f1 = minusexpenditure f2 = capacity The following tablesummarizes the data
Options f1 f2
12
minus2minus3
15002000
Figure 291 Planning of a sewage plant
Both options are Pareto optimal since minus2 gt minus3 and 2000 gt 1500The H consequentialspace consists of two points (minus2 1500) and (minus3 2000)
Example 292 The optimal combination of three technology variants is used in a sewagestation The first variant removes 321 mgm3 from one kind of pollution and 132 mgm3
quantity from the another kind of pollution Let x1 x2 and 1minusx1minusx2 denote the percentagecomposition of the three technology variants
The restrictive conditions
x1 x2 ge 0
x1 + x2 le 1
the quantity of the removed pollution
3x1 + 2x2 + (1 minus x1 minus x2) = 2x1 + x2 + 1
x1 + 3x2 + 2(1 minus x1 minus x2) = minusx1 + x2 + 2
Since the third term is constant we get the following two objective-function optimumproblem
291 The basics of multi-objective programming 1335
Figure 292 The image of set X
2x1 + x2 minusx1 + x2 minusrarr max
provided that
x1 x2 ge 0
x1 + x2 le 1
A H consequential space can be determined as follows From the
f1 = 2x1 + x2
f2 = minusx1 + x2
equations
x1 =f1 minus f2
3and x2 =
f1 minus 2f2
3
and from the restrictive conditions the following conditions arises for the f1 and f2 objectivefunctions
x1 ge 0 lArrrArr f1 minus f2 ge 0
x2 ge 0 lArrrArr f1 + 2f2 ge 0
x1 + x2 le 1 lArrrArr 2f1 + f2 le 3
Figures and display the X and H setsOn the basis of the image of the H set it is clear that the points of the straight section
joining (1 1) to (2 minus1) are Pareto optimal Point (2 minus1) isnrsquot better than any possiblepoint of H because in the first objective function it results the worst possible planes Thepoints of the section are not equivalent to each other either going down from the point(1 1) towards point (2 1) the first objective function is increasing but the second one iscontinually decreasing Thus the (ii) and (iii) properties of the optimal solution doesnrsquot
1336 29 Conflict Situations
Figure 293 The image of set H
remain valid in the case of multi-objection
As we saw in the previous example the different Pareto optimal solutions resultin different objective function values so it is primary importance to decide which oneshould be selected in a particular case This question is answered by the methodologyof the multi-objective programming Most methodsrsquo basis is to substitute some real-valued
rdquo ldquovalue-functionrdquo for the objective functions that is the preference generated
by the objective functions is replaced by a single real-valued function In this chapterthe most frequently used methods of multi-objective programming are discussed
2911 Applications of utility functions
A natural method is the following We assign one utility function to every objectivefunction Let ui(fi(x)) denote the utility function of the ith objective function Theconstruction of the ui function can be done by the usual manner of the theoryof utility functions for example the decision maker can subjectively define the ui
values for the given fi values then a continuous function can be interpolated on theresulting point In the case of additive independent utility function additive whereasin the case of independent of usefulness utility function additive or multiplicativeaggregate utility function can be obtained That is the form of the aggregate utilityfunction is either
u(f) =Isum
i=1
kiui(fi) (295)
291 The basics of multi-objective programming 1337
or
ku(f) + 1 =Iprod
i=1
kkiui(fi) + 1 (296)
In such cases the multi-objective optimization problem can be rewrite to oneobjective-function form
u(f) minusrarr max (297)
provided that x isin X and thus u(f) means the value-function
Example 293 Consider again the decision making problem of the previous example Therange of the first objective function is [0 2] while the range of the second one is [minus1 1]Assuming linear utility functions
u1(f1) =1
2(f1) and u2(f2) =
1
2(f2) + 1
In addition suppose that the decision maker gave the
u(0 minus1) = 0 u(2 1) = 1 and the u(0 1) =1
4
values Assuming linear utility functions
u(f1 f2) = k1u1(f1) + k2u2(f2)
and in accordance with the given values
0 = k10 + k20
1 = k11 + k211
4= k10 + k21
By the third equation k2 = 14 and by the second one we obtain k1 = 3
4 so that
u(f1 f2) =3
4u1(f1) +
1
4u2(f2) =
3
4
1
2(2x1 + x2) +
1
4
1
2(minusx1 + x2 + 1) =
5
8x1 +
4
8x2 +
1
8
Thus we solve the following one objective-function problem
5
8x1 +
4
8x2 minusrarr max
provided that
x1 x2 ge 0
x1 + x2 le 1
Apparently the optimal solution is x1 = 1 x2 = 0 that is the first technology must bechosen
Assume that the number of objective functions is n and the decision maker givesN vectors (f (l)
1 f(l)n ) and the related u(l) aggregated utility function values
1338 29 Conflict Situations
Then the k1 kn coefficients can be given by the solution of the
k1u1(f (l)1 ) + middot middot middot+ knun(f (l)
n ) = u(l) (l = 1 2 N)
equation system We always suppose that N ge n so that we have at least as manyequations as the number of unknown quantities If the equation system is contradic-tory we determine the best fitting solution by the method of least squares Supposethat
U =
u11 middot middot middot u1n
u21 middot middot middot u2n
uN1 middot middot middot uNn
eacutes u =
u(1)
u(2)
u(N)
The formal algorithm is as follows
Utility-Function-Method(u)
1 for ilarr 1 to N2 do for j larr 1 to n
3 do uij larr uj(f (i)j )
4 klarr (UT U)minus1UT u the vector of solutions5 return k
2912 Weighting method
Using this method the value-function is chosen as the linear combination of theoriginal object functions that is we solve the
Isum
i=1
αifi(x) minusrarr max (x isin X) (298)
problem If we measure the certain objective functions in different dimensions theaggregate utility function canrsquot be interpreted since we add up terms in differentunits In this case we generally normalize the objective functions Let mi and Mi
the minimum and maximum of the fi objective function on the set X Then thenormalized ith objective function is given by the
fi(x) =fi(x)minusmi
Mi minusmi
formula and in the (298) problem fi is replaced by fi
Isum
i=1
αifi(x) minusrarr max (x isin X) (299)
It can be shown that if all of the αi weights are positive the optimal solutions of(299) are Pareto optimal with regard to the original problem
291 The basics of multi-objective programming 1339
Example 294 Consider again the case of Example 292 From Figure 293 we can seethat m1 = 0 M1 = 2 m2 = minus1 and M2 = 1 Thus the normalized objective functions are
f1(x1 x2) =2x1 + x2 minus 0
2 minus 0= x1 +
1
2x2
and
f2(x1 x2) =minusx1 + x2 + 1
1 + 1= minus
1
2x1 +
1
2x2 +
1
2
Assume that the objective functions are equally important so we choose equivalent weightsα1 = α2 = 1
2 in this way the aggregate objective function is
1
2(x1 +
1
2x2) +
1
2(minus
1
2x1 +
1
2x2 +
1
2) =
1
4x1 +
1
2x2 +
1
4
It is easy to see that the optimal solution on set X
x1 = 0 x2 = 1
that is only the second technology variant can be chosen
Suppose that α = (α1 α2 αI) The formal algorithm is as follows
Weighting-Method(α)
1 for ilarr 1 to I2 do mi larr (fi(x) minusrarr min)3 Mi larr (fi(x) minusrarr max)4 k larr (
sumIi=1 αifi minusrarr max)
5 return k
2913 Distance-dependent methods
If we normalize the objective functions the certain normalized objective functionsmost favorable value is 1 and the most unfavourable is 0 So that 1 = (1 1 1) isthe ideal point and 0 = (0 0 0) is the worst yield vector
In the case of distance-dependent methods we either want to get nearest to thevector 1 or get farthest from the point 0 so that we solve either the
(f(x)1) minusrarr min (x isin X) (2910)
or the(f(x)0) minusrarr max (x isin X) (2911)
problem where denotes some distance function in RI
In practical applications the following distance functions are used most fre-quently
1(ab) =Isum
i=1
αi|ai minus bi| (2912)
1340 29 Conflict Situations
Figure 294 Minimizing distance
2(ab) =
(
Isum
i=1
αi|ai minus bi|2)
12
(2913)
infin(ab) = maxiαi|ai minus bi| (2914)
g(ab) =Iprod
i=1
|ai minus bi|αi (2915)
The 1 1 infin distance functions the commonly known Minkowski distance for p =1 2infin The g geometric distance doesnrsquot satisfy the usual requirements of distancefunctions however it is frequently used in practice As we will see it later Nashrsquosclassical conflict resolution algorithm uses the geometric distance as well It is easyto prove that the methods using the 1 distance are equivalent of the weightingmethod Notice firstly that
1(f(x)1) =Isum
i=1
αi|fi(x)minus 1| =Isum
i=1
αi|1minus fi(x)| =Isum
i=1
αi minusIsum
i=1
αifi(x) (2916)
where the first term is constant while the second term is the objective function ofthe weighting method Similarly
1(f(x)0) =Isum
i=1
αi|fi(x)minus 0| =Isum
i=1
αi(fi(x)minus 0) =Isum
i=1
αifi(x) (2917)
which is the objective function of the weighting methodThe method is illustrated in Figures 294 and 295
291 The basics of multi-objective programming 1341
Figure 295 Maximizing distance
Figure 296 The image of the normalized set H
Example 295 Consider again the problem of the previous example The normalized con-sequences are shown by Figure 296 The two coordinates are
f1 =f1
2and f2 =
f2 + 1
2
Choosing the α1 = α2 = 12
and the 2 distances the nearest point of H to the idealpoint is
f1 =3
5 f2 =
4
5
1342 29 Conflict Situations
Hence
f1 = 2f1 = 2x1 + x2 =6
5and f2 = 2f1 minus 1 = minusx1 + x2 =
3
5
that is the optimal decision is
x1 =1
5 x2 =
4
5 1 minus x1 minus x2 = 0
Therefore only the first two technology must be chosen in 20 and 80 proportionLetrsquos choose again equivalent weights (α1 = α2 = 1
2) and the 2 distance but look for
the farthest point of H from the ideal worst point We can see from Figure 295 that thesolution is
f1 =f1
2 f2 = 1
so
f1 = 2f1 = 1 f2 = 2f2 minus 1 = 1
Thus the optimal decision is x1 = 0 and x2 = 1
The formal algorithm is as follows
Distance-Dependent-Method( f)
1 for ilarr 1 to I2 do mi larr (fi(x) minusrarr min)3 Mi larr (fi(x) minusrarr max)4 fi(x)larr (fi(x)minusmi)(Mi minusmi)5 k larr ((f(x)1) minusrarr min) or k larr ((f(x)0) minusrarr max)6 return k
2914 Direction-dependent methods
Assume that we have a flowast point in set H on which wersquod like to improve flowast denotesthe present position on which the decision maker wants to improve or at designlevel we can choose the worst point for the starting one Furthermore we assumethat the decision maker gives an improvement direction vector which is denoted byv After that the task is to reach the farthest possible point in set H starting fromflowast along the v direction vector Thus mathematically we solve the
t minusrarr max (flowast + tv isin H) (2918)
optimum task and the related decision is given by the solution of the
f(x) = flowast + tv (2919)
equation under the optimal t value The method is illustrated in Figure 297
Example 296 Letrsquos consider again the problem of Example 292 and assume that flowast =(0 minus1) which contains the worst possible objective function values in its components Ifwe want to improve the objective functions equally we have to choose v = (1 1) The
291 The basics of multi-objective programming 1343
Figure 297 Direction-dependent methods
Figure 298 The graphical solution of Example 296
graphical solution is illustrated in Figure 298 that
f1 =4
3and f2 =
1
3
so the appropriate values of the decision variables are the following
x1 =1
3eacutes x2 =
2
3
A very rarely used variant of the method is when we diminishes the object func-tion values systematically starting from an unreachable ideal point until a possiblesolution is given If flowast denotes this ideal point the (2918) optimum task is modifiedas follows
t minusrarr min (flowast minus tv isin H) (2920)
1344 29 Conflict Situations
Figure 299 The graphical solution of Example 297
and the appropriate decision is given by the solution of the
f = flowast minus tv (2921)
equation
Example 297 To return to the previous example consider again that f lowast = (2 1) andv = (1 1) that is we want to diminish the object functions equally Figure 299 shows thegraphical solution of the problem in which we can see that the given solution is the sameas the solution of the previous example
Applying the method is to solve the (2918) or the (2920) optimum tasks andthe optimal decision is given by the solution of the (2919) or the (2921) equations
Exercises291-1 Determine the consequence space H for the following exercise
x1 + x2 minusrarr max x1 minus x2 minusrarr max
provided that
x1 x2 ge 0
3x1 + x2 le 3
x1 + 3x2 le 3
291-2 Consider the utility functions of the decision maker u1(f1) = f1 eacutes u2(f2) =12f2 Furthermore assume that the decision maker gave the u(0 0) = 0 u(1 0) =u(0 1) = 1
2 values Determine the form of the aggregate utility function291-3 Solve Exercise 291-1 using the weighting-method without normalizing theobjective functions Choose the α1 = α2 = 1
2 weights
292 Method of equilibrium 1345
291-4 Repeat the previous exercise but do normalize the objective functions291-5 Solve Exercise 291-1 with normalized objective functions α1 = α2 = 1
2weights and minimizing the
(i) 1 distance
(ii) 2 distance
(iii) infindistance
291-6 Repeat the previous exercise but maximize the distance from the 0 vectorinstead of minimizing it291-7 Solve Exercise 291-1 using the direction-dependent method choosing flowast =(0minus1) and v = (1 1)291-8 Repeat the previous exercise but this time choose flowast = ( 3
2 1) and v = (1 1)
292 Method of equilibrium
In this chapter we assume that I decision makers interested in the selection of amutual decision alternative Let fi X 7rarr R denote the objective function of the ithdecision maker which is also called payoff function in the game theory literatureDepending on the decision makers relationship to each other we can speak aboutcooperative and non-cooperative games In the first case the decision makers careabout only their own benefits while in the second case they strive for an agreementwhen every one of them are better off than in the non-cooperative case In thischapter we will discuss the non-cooperative case while the cooperative case will betopic of the next chapter
Letrsquos denote Hi(x) for i = 1 2 I and x isin X the set of the decision alterna-tives into which the ith decision maker can move over without the othersrsquo supportEvidently Hi(x) sube X
Definition 293 An xlowast isin X alternative is equilibrium if for all i and x isin Hi(xlowast)
fi(x) le fi(xlowast) (2922)
This definition can also be formulated that xlowast is stable in the sense that noneof the decision makers can change the decision alternative from xlowast alone to changeany objective function value for the better In the case of non-cooperative gamesthe equilibrium are the solutions of the game
For any x isin X and i decision maker the set
Li(x) = z|z isin Hi(x) and for all y isin Hi(x) fi(z) ge fi(y) (2923)
is called the set of the best answers of the ith decision maker to alternative x It isclear that the elements of Li(x) are those alternatives which the ith decision makercan move over from x and which ensure the best objective functions out of all the
1346 29 Conflict Situations
i = 21 2
i = 112
(1 2)(2 4)
(2 1)(0 5)
Figure 2910 Game with no equilibrium
possible alternatives According to inequality (2922) it is also clear that xlowast is anequilibrium if and only if for all i = 1 2 I xlowast isin Li(xlowast) that is xlowast is mutualfixed point of the Li point-to-set maps Thus the existence of equilibrium can betraced to the existence of mutual fixed point of point-to-set maps so the problemcan be solved by the usual methods
It is a very common case when the collective decision is made up by the personaldecisions of the certain decision makers Let Xi denote the set of the ith decisionmakerrsquos alternatives let xi isin Xi be the concrete alternatives and let fi(x1 xI)be the objective function of the ith decision maker That is the collective decision isx = (x1 xI) isin X1 timesX2 times middot middot middot timesXI = X In this case
Hi(x1 xI) = (x1 ximinus1 zi xi+1 xI)|zi isin Xi
and the (2922) definition of equilibrium is modified as follows
fi(xlowast1 x
lowastiminus1 xi x
lowasti+1 x
lowastI) le fi(xlowast
i xlowastI) (2924)
In the game theory literature the equilibrium is also called Nash-equilibriumThe existence of an equilibrium is not guaranteed in general To illustrate this
letrsquos consider the I = 2 case when both decision makers can choose between toalternatives X1 = 1 2 and X2 = 1 2 The objective function values are shown inFigure 2910 where the the first number in the parentheses shows the first the secondnumber shows the second decision makerrsquos objective function value If equilibriumexists it might not be unique what can be proved by the case of constant objectivefunctions when every decision alternative is an equilibrium
If the X1 XI sets are finite the equilibrium can be found easily by themethod of reckoning when we check for all of the x = (x1 xI) decision vectorswhether the component xi can be changed for the better of the fi objective functionIf the answer is yes x is not equilibrium If none of the components can be changedin such manner x is equilibrium For the formal algorithm letrsquos assume that X1 =1 2 ni
292 Method of equilibrium 1347
Equilibrium-Search
1 for i1 larr 1 to n1
2 do for i2 larr 1 to n2
3
4 do for iI larr 1 to nI
5 do key larr 06 for k larr 1 to n7 do for j larr 1 to nk
8 do if fk(i1 ikminus1 j ik+1 iI) gt f(i1 iI)9 then key larr 1 and go to 10
10 if key = 011 then (i1 iI) is equilibrium
The existence of equilibrium is guaranteed by the following theorem
Theorem 294 Assume that for all i = 1 2 I
(i) Xi is convex bounded and closed in a final dimensional Euclidean space
(ii) fi is continuous on the set X
(iii) for any fixed x1 ximinus1 xi+1 xI fi is concave in xi
Then there is at least one equilibrium
Determination of the equilibrium is usually based on the observation that for alli xlowast
i is the solution of the
fi(xlowast1 x
lowastiminus1 xi x
lowasti+1 x
lowastI) minusrarr max (xi isin Xi) (2925)
optimum task Writing the necessary conditions of the optimal solution (for exam-ple the Kuhn-Tucker conditions) we can get an equality-inequality system whichsolutions include the equilibrium To illustrate this method letrsquos assume that
Xi = xi|gi(xi) ge 0
where xi is a finite dimensional vector and gi is a vector-valued function In this way(2925) can be rewritten as follows
fi(xlowast1 x
lowastiminus1 xi x
lowasti+1 x
lowastI) minusrarr max (gi(xi) ge 0) (2926)
In this case the Kuhn-Tucker necessary conditions are
ui ge 0gi(xi) ge 0
nablaifi(x) + uTi nablaigi(xi) = 0T
uTi gi(xi) = 0
(2927)
where nablai denotes the gradient at xi and ui is a vector which has the same length
1348 29 Conflict Situations
as gi If we formulate the (2927) conditions for i = 1 2 I we get an equality-inequality system which can be solved by computer methods It is easy to see that(2927) can also be rewritten to an nonlinear optimization task
sumIi=1 u
Ti gi(xi) minusrarr min
ui ge 0gi(xi) ge 0
nablaifi(x) + uTi nablaigi(xi) = 0T
(2928)
If the optimal objective function value is positive the (2927) system doesnrsquot havea solution and if the optimal objective function value is zero any optimal solutionis also a solution of the (2927) system so the equilibrium are among the optimalsolutions We know about the sufficiency of the Kuhn-Tucker conditions that if fi isconcave in xi with all i the Kuhn-Tucker conditions are also sufficient thus everysolution of (2927) gives an equilibrium
The formal algorithm is as follows
KuhnndashTucker-Equilibrium
1 for ilarr 1 to I2 do gi larr nablaifi
3 Ji larr nablaigi(xi)4 (x1 xI)larr the solution of the (2928) optimum task5 if
sumIi=1 u
Ti gi(xi) gt 0
6 then return there is no equilibrium7 else return (x1 xI)
Example 298 Assume that I production plant produce some water purification devicesold into households Let xi denote the quantity produced by the ith production plant letci(xi) be the cost function of it and let p(
sumI
j=1xj) be the sale price which depends on
the total quantity to be put on the market Furthermore be Li is the capacity of the ithproduction plant Thus the possible Xi decision set is the [0 Li] closed interval which canbe defined by the
xi ge 0Li minus xi ge 0
(2929)
conditions so
gi(xi) =(
xi
Li minus xi
)
The objective function of the ith production plant is the profit of that
fi(x1 xn) = xip(x1 + middot middot middot + xn) minus ci(xi) (2930)
Since gi(xi) is two-dimensional ui is a two-element vector as well and the (2928)optimum task has the following form
sumI
i=1(u
(1)i xi + u
(2)i (Li minus xi)) minusrarr min
u(1)i u
(2)i ge 0xi ge 0
Li minus xi ge 0
p(sumI
j=1xj) + xip
prime(sumI
j=1xj) minus cprime
i(xi) + (u(1)i u
(2)i )(
1
minus1
)
= 0
(2931)
293 Methods of cooperative games 1349
Letrsquos introduce the αi = u(1)i minusu
(2)i new variables and for the sake of notational convenience
be βi = u(2)i then taking account of the last condition we get the following problem
sumI
i=1(minusxi(p(
sumI
j=1xj) + xip
prime(sumI
j=1xj) minus cprime
i(xi)) + βiLi) minusrarr min
βi ge 0xi ge 0xi le Li
(2932)
Letrsquos notice that in case of optimum βi = 0 so the last term of the objective function canbe neglected
Consider the special case of I = 3 ci(xi) = ix3i + xi Li = 1 p(s) = 2 minus 2s minus s3 The
(2932) problem is now simplified as followssum3
i=1xi(2 minus 2s minus s2 minus 2xi minus 2xis minus 3ix2
i minus 1) minusrarr maxxi ge 0xi le 1
x1 + x2 + x3 = s
(2933)
Using a simple computer program we can get the optimal solution
xlowast1 = 01077 xlowast
2 = 00986 xlowast3 = 00919
which is the equilibrium as well
Exercises292-1 Let I = 2 X1 = X2 = [0 1] f1(x1 x2) = x1 + x2 minus x2
1 f2(x1 x2) =x1 + x2 minus x2
2 Formulate the (2927) conditions and solve them as well292-2 Formulate and solve the optimum problem (2928) for the previous exercise
292-3 Let again I = 2 X1 = X2 = [minus1 1] f1(x1 x2) = minus(x1 + x2)2 + x1 + 2x2f2(x1 x2) = minus(x1 + x2)2 + 2x1 + x2 Repeat Exercise 292-1292-4 Repeat Exercise 292-2 for the problem given in the previous exercise
293 Methods of cooperative games
Similarly to the previous chapter let Xi denote again the decision set of the ithdecision maker and let xi isin Xi be the concrete decision alternatives Furthermorelet fi(x1 xI) denote the objective function of the ith decision maker Let S besome subset of the decision makers which is usually called coalition in the gametheory For arbitrary S sube 1 2 I letrsquos introduce the
v(S) = maxxiisinXi
minxjisinXj
sum
kisinS
fk(x1 xI) (i isin S j isin S) (2934)
function which is also called the characteristic function defined on all of the subsetsof the set 1 2 I if we add the v(empty) = 0 and
v(1 2 I) = maxxiisinXi
Isum
k=1
fk(x1 xI) (1 le i le I)
1350 29 Conflict Situations
special cases to definition (2934)Consider again that all of the Xi sets are finite for Xi = 1 2 ni i =
1 2 I Be S a coalition The value of v(S) is given by the following algorithmwhere |S| denotes the number of elements of S k1 k2 k|S| denotes the elementsand l1 l2 lIminus|S| the elements which are not in S
Characteristic-Function(S)
1 v(S)larr minusM where M a very big positive number2 for i1 larr 1 to nk1
3
4 do for i|S| larr 1 to nk|S|
5 do for j1 larr 1 to nl1
6
7 do for jIminus|S| larr 1 to nlIminus|S|
8 do Z larrM where M a very big positive number9 V larrsum|S|
t=1 fit(i1 i|S| j1 jIminus|S|)
10 if V lt Z11 then Z larr V12 if Z gt v(S)13 then v(S)larr Z14 return v(S)
Example 299 Letrsquos return to the problem discussed in the previous example and assumethat I = 3 Li = 3 p(
sumI
i=1xi) = 10 minus
sumI
i=1xi eacutes ci(xi) = xi + 1 for i = 1 2 3 Since the
cost functions are identical the objective functions are identical as well
fi(x1 x2 x3) = xi(10 minus x1 minus x2 minus x3) minus (xi + 1)
In the following we determine the characteristic function At first be S = i then
v(S) = maxxi
minxj
xi(10 minus x1 minus x2 minus x3) minus (xi + 1) (j 6= i)
Since the function strictly decreases in the xj(i 6= j) variables the minimal value of it isgiven at xj = 3 so
v(S) = maxi
xi(4 minus xi) minus (xi + 1) = max0lexile3
(minusx2i + 3xi minus 1) =
5
4
what is easy to see by plain differentiation Similarly for S = i j
v(S) = maxij
mink 6=ij
(xi + xj)(10 minus x1 minus x2 minus x3) minus (xi + 1) minus (xj + 1)
Similarly to the previous case the minimal value is given at xk = 3 so
v(S) = max0lexixj le3
(xi + xj)(7 minus xi minus xj) minus (xi + xj + 2) = max0lexle6
x(7 minus x) minus (x + 2) =
= max0lexle6
minusx2 + 6x minus 2 = 7
293 Methods of cooperative games 1351
where we introduced the new x = xi + xj variable In the case of S = 1 2 3
v(S) = max0lex1x2x3le3
(x1 + x2 + x3)(10 minus x1 minus x2 minus x3) minus (x1 + 1) minus (x2 + 1) minus (x3 + 1) =
= max0lexle9
x(10 minus x) minus (x + 3) = max0lexle9
minusx2 + 9x minus 3) = 1725
where this time we introduced the x = x1 + x2 + x3 variable
Definition (2934) can be interpreted in a way that the v(S) characteristic func-tion value gives the guaranteed aggregate objective function value of the S coalitionregardless of the behavior of the others The central issue of the theory and practiceof the cooperative games is how should the certain decision makers share in the maxi-mal aggregate profit v(1 2 I) attainable together An (φ1 φ2 φI) divisionis usually called imputation if
φi ge v(i) (2935)
for i = 1 2 I andIsum
i=1
φi = v(1 2 I) (2936)
The inequality system (2935)ndash(2936) is usually satisfied by infinite number ofimputations so we have to specify additional conditions in order to select one specialelement of the imputation set We can run into a similar situation while discussingthe multi-objective programming when we looks for a special Pareto optimal solutionusing the concrete methods
Example 2910 In the previous case a (φ1 φ2 φ3) vector is imputation if
φ1 φ2 φ3 ge 125
φ1 + φ2 φ1 + φ3 φ2 + φ3 ge 7
φ1 + φ2 + φ3 = 172
The most popular solving approach is the Shapley value which can be definedas follows
φi =sum
Ssube12I
(sminus 1)(I minus s)I
(v(S)minus v(S minus i)) (2937)
where s denotes the number of elements of the S coalitionLetrsquos assume again that the decision makers are fully cooperating that is they
formulate the coalition 12 I and the certain decision makers join to the coali-tion in random order The difference v(S)minus v(Sminusi) indicates the contribution tothe S coalition by the ith decision maker while expression (2937) indicates the av-erage contribution of the same decision maker It can be shown that (φ1 φ2 φI)is an imputation
The Shapley value can be computed by following algorithm
1352 29 Conflict Situations
Shapley-Value
1 for forallS sube 1 I2 do v(S)larr Characteristic-Function(S)3 for ilarr 1 to I4 do use (2937) for calculating φi
Example 2911 In the previous example we calculated the value of the characteristicfunction Because of the symmetry φ1 = φ2 = φ3 must be true for the case of Shapleyvalue Since φ1 + φ2 + φ3 = v(1 2 3) = 1725 φ1 = φ2 = φ3 = 575 We get the samevalue by formula (2937) too Letrsquos consider the φ1 value If i isin S v(S) = v(S minusi) so theappropriate terms of the sum are zero-valued The non-zero terms are given for coalitionsS = 1 S = 1 2 S = 1 3 and S = 1 2 3 so
φ1 =02
3(5
4minus 0) +
11
3(7 minus
5
4) +
11
3(7 minus
5
4) +
20
3(69
4minus 7) =
1
6(10
4+
23
4+
23
3+
82
4) =
138
24= 575
An alternative solution approach requires the stability of the solution It is saidthat the vector φ = (φ1 φI) majorizes the vector ψ = (ψ1 ψI) in coalitionS if
sum
iisinS
φi gtsum
iisinS
ψi
that is the S coalition has an in interest to switch from payoff vector φ to payoffvector ψ or ψ is instabil for coalition S The NeumannndashMorgenstern solutionis a V set of imputations for which
(i) There is no φψ isin V that φ majorizes ψ in some coalition (inner stability)
(ii) If ψ isin V there is φ isin V that φ majorizes ψ-t in at least one coalition (outerstability)
The main difficulty of this conception is that there is no general existence the-orem for the existence of a non-empty V set and there is no general method forconstructing the set V
Exercises293-1 Let I = 3 X1 = X2 = X3 = [0 1] fi(x1 x2 x3) = x1 + x2 + x3 minus x2
i (i =1 2 3) Determine the v(S) characteristic function293-2 Formulate the (2935) (2936) condition system for the game of the previousexercise293-3 Determine the ψi Shapley values for the game of Exercise 293-1
294 Collective decision-making 1353
294 Collective decision-making
In the previous chapter we assumed that the objective functions are given by nu-merical values These numerical values also mean preferences since the ith decisionmaker prefers alternative x to z if fi(x) gt fi(z) In this chapter we will discusssuch methods which donrsquot require the knowledge of the objective functions but thepreferences of the certain decision makers
Let I denote again the number of decision makers and X the set of decisionalternatives If the ith decision maker prefers alternative x to y this is denoted byx i y if prefers alternative x to y or thinks to be equal it is denoted by x i yAssume that
(i) For all x y isin X x i y or y i x (or both)
(ii) For x i y and y i z x i z
Condition (i) requires that the i partial order be a total order while condition (ii)requires to be transitive
Definition 295 A group decision-making function combines arbitrary individual(12 I) partial orders into one partial order which is also called the collec-tive preference structure of the group
We illustrate the definition of group decision-making function by some simpleexample
Example 2912 Be x y isin X arbitrary and for all i
αi =
1 ha x i y0 ha x simi yminus1 ha x ≺i y
Let βi β2 βI given positive constant and
α =
Isum
i=1
βiαi
The group decision-making function means
x y lArrrArr α gt 0
x sim y lArrrArr α = 0
x ≺ y lArrrArr α lt 0
The majority rule is a special case of it when β1 = β2 = middot middot middot = βI = 1
Example 2913 An i0 decision maker is called dictator if his or her opinion prevails ingroup decision-making
x y lArrrArr x i0 y
x sim y lArrrArr x simi0 y
x ≺ y lArrrArr x ≺i0 y
1354 29 Conflict Situations
This kind of group decision-making is also called dictatorship
Example 2914 In the case of Borda measure we assume that α is a finite set andthe preferences of the decision makers is expressed by a ci(x) measure for all x isin X Forexample ci(x) = 1 if x is the best ci(x) = 2 if x is the second best alternative for the ithdecision maker and so on ci(x) = I if x is the worst alternative Then
x y lArrrArr
Isum
i=1
ci(x) gt
Isum
i=1
ci(y)
x sim y lArrrArr
Isum
i=1
ci(x) =
Isum
i=1
ci(y)
x ≺ y lArrrArr
Isum
i=1
ci(x) lt
Isum
i=1
ci(y)
A group decision-making function is called Pareto or Pareto function if forall x y isin X and x i y (i = 1 2 I) x y necessarily That is if all the decisionmakers prefer x to y it must be the same way in the collective preference of thegroup A group decision-making function is said to satisfy the condition of pairwiseindependence if any two (1 I) and (prime
1 primeI) preference structure satisfy
the followings Let x y isin X such that for arbitrary i x i y if and only if x primei y
and y i x if and only if y primei x Then x y if and only if x prime y and y x if and
only if y prime x in the collective preference of the group
Example 2915 It is easy to see that the Borda measure is Pareto but it doesnrsquot satisfythe condition of pairwise independence The first statement is evident while the secondone can be illustrated by a simple example Be I = 2 α = x y z Letrsquos assume that
x 1 z 1 y
y 2 x 2 z
and
x prime1 y prime
1 z
y prime2 z prime
2 x
Then c(x) = 1 + 2 = 3 c(y) = 3 + 1 = 4 thus y x However cprime(x) = 1 + 3 = 4 cprime(y) =2 + 1 = 3 so x y As we can see the certain decision makers preference order between xand y is the same in both case but the collective preference of the group is different
Let RI denote the set of the I-element full and transitive partial orders onan at least three-element X set and be the collective preference of the groupwhich is Pareto and satisfies the condition of pairwise independence Then isnecessarily dictatorial This result originated with Arrow shows that there is nosuch group decision-making function which could satisfy these two basic and naturalrequirements
294 Collective decision-making 1355
Decision makers Alternatives Weights1 2 N
12I
a11
a21
aI1
a12
a22
aI2
a1N
a2N
aIN
α1
α2
αI
Figure 2911 Group decision-making table
Example 2916 The method of paired comparison is as follows Be x y isin X arbitraryand letrsquos denote P (x y) the number of decision makers to which x i y After that thecollective preference of the group is the following
x y lArrrArr P (x y) gt P (y x)
x sim y lArrrArr P (x y) = P (y x)
x ≺ y lArrrArr P (x y) lt P (y x)
that is x y if and only if more than one decision makers prefer the x alternative to yLetrsquos assume again that X consists of three elements X = x y z and the individualpreferences for I = 3
x 1 y 1 z
z 2 x 2 y
y 3 z 3 x
Thus in the collective preference x y because P (x y) = 2 and P (y x) = 1 Similarlyy z because P (y z) = 2 and P (z y) = 1 and z x since P (z x) = 2 and P (x z) = 1Therefore x y z x which is inconsistent with the requirements of transitivity
The methods discussed so far didnrsquot take account of the important circumstancethat the decision makers arenrsquot necessarily in the same position that is they canhave different importance This importance can be characterized by weights In thisgeneralized case we have to modify the group decision-making methods as requiredLetrsquos assume that X is finite set denote N the number of alternatives We denotethe preferences of the decision makers by the numbers ranging from 1 to N where1 is assigned to the most favorable while N is assigned to most unfavorable alter-native Itrsquos imaginable that the two alternatives are equally important then we usefractions For example if we canrsquot distinguish between the priority of the 2nd and3rd alternatives then we assign 25 to each of them Usually the average value of theindistinguishable alternatives is assigned to each of them In this way the problemof the group decision can be given by a table which rows correspond to the decisionmakers and columns correspond to the decision alternatives Every row of the tableis a permutation of the 1 2 N numbers at most some element of it is replacedby some average value if they are equally-preferred Figure 2911 shows the giventable in which the last column contains the weights of the decision makers
In this general case the majority rule can be defined as follows For all of the j
1356 29 Conflict Situations
alternatives determine first the aggregate weight of the decision makers to which thealternative j is the best possibility then select that alternative for the best collectiveone for which this sum is the biggest If our goal is not only to select the best butto rank all of the alternatives then we have to choose descending order in this sumto rank the alternatives where the biggest sum selects the best and the smallestsum selects the worst alternative Mathematically be
f(aij) =
1 ha aij = 10 otherwise
(2938)
and
Aj =Isum
i=1
f(aij)αi (2939)
for j = 1 2 I The j0th alternative is considered the best by the group if
Aj0= max
jAj (2940)
The formal algorithm is as follows
Majority-Rule(A)
1 A1 larr 0 A2 larr 0 AN larr 0maxlarr 02 for ilarr 1 to N3 do for j larr 1 to I4 do if aji = 15 then Ai larr Ai + αj
6 if Ai gt max7 then maxlarr Ai
8 indlarr i9 return ind
Applying the Borda measure let
Bj =Isum
i=1
aijαi (2941)
and alternative j0 is the result of the group decision if
Bj0= min
jBj (2942)
The Borda measure can be described by the following algorithm
294 Collective decision-making 1357
Borda-Measure-Method(Aα)
1 B1 larr 0 B2 larr 0 BN larr 0maxlarr 02 for j larr 1 to N3 do for ilarr 1 to I4 do Bj larr Bj + aijαi
5 if Bj gt max6 then maxlarr Bj
7 indlarr j8 return ind
Applying the method of paired comparison let with any j jprime isin X
P (j jprime) =sum
i|aijltaijprime αi (2943)
which gives the weight of the decision makers who prefer the alternative j to jprime Inthe collective decision
j jprime lArrrArr P (j jprime) gt P (jprime j)
In many cases the collective partial order given this way doesnrsquot result in a clearlybest alternative In such cases further analysis (for example using some othermethod) need on the
Slowast = j|j isin X and theres is no such jprime isin X for which jprime jnon-dominated alternative set
By this algorithm we construct a matrix consists of the 0 1 elements whereajl = 1 if and only if the j alternative is better in all then alternative l In the caseof draw ajl = 1
2
Paired-Comparison(A)
1 for j larr 1 to N minus 12 do for llarr j to N3 do z larr 04 for ilarr 1 to I5 do if aij gt ail
6 then z larr z + 17 if z gt N
28 then ajl larr 19 if z = N
210 then ajl larr 1
2
11 if z lt N2
12 then ajl larr 013 alj larr ajl
14 return A
Example 2917 Four proposal were received by the Environmental Authority for the
1358 29 Conflict Situations
Committee Alternatives WeightsMembers 1 2 3 4
123456
121231
313314
242142
434423
030202010101
Figure 2912 The database of Example 2917
cleaning of a chemically contaminated site A committee consists of 6 people has to choosethe best proposal and thereafter the authority can conclude the contract for realizing theproposal Figure 2912 shows the relative weight of the committee members and the personalpreferences
Majority ruleUsing the majority rule
A1 = 03 + 02 + 01 = 06
A2 = 02 + 01 = 03
A3 = 01
A4 = 0
so the first alternative is the bestUsing the Borda measure
B1 = 03 + 04 + 02 + 02 + 03 + 01 = 15
B2 = 09 + 02 + 06 + 03 + 01 + 04 = 25
B3 = 06 + 08 + 04 + 01 + 04 + 02 = 25
B4 = 12 + 06 + 08 + 04 + 02 + 03 = 35
In this case the first alternative is the best as well but this method shows equally good thesecond and third alternatives Notice that in the case of the previous method the secondalternative was better than the third one
In the case of the method of paired comparison
294 Collective decision-making 1359
Figure 2913 The preference graph of Example 2917
P (1 2) = 03 + 02 + 01 + 01 = 07
P (2 1) = 02 + 01 = 03
P (1 3) = 03 + 02 + 02 + 01 + 01 = 09
P (3 1) = 01
P (1 4) = 03 + 02 + 02 + 01 + 01 = 09
P (4 1) = 01
P (2 3) = 02 + 01 + 01 = 04
P (3 2) = 03 + 02 + 01 = 06
P (2 4) = 03 + 02 + 02 + 01 + 01 = 09
P (4 2) = 01
P (3 4) = 03 + 02 + 01 + 01 = 07
P (4 3) = 02 + 01 = 03
Thus 1 2 1 3 1 4 3 2 2 4 and 3 4 These references are showed by Figure2913 The first alternative is better than any others so this is the obvious choice
In the above example all three methods gave the same result However in severalpractical cases one can get different results and the decision makers have to chooseon the basis of other criteria
Exercises294-1 Letrsquos consider the following group decision-making table
Apply the majority rule294-2 Apply the Borda measure to the previous exercise294-3 Apply the method of paired comparison to Exercise 294-1294-4 Letrsquos consider now the following group decision-making table
Repeat Exercise 294-1 for this exercise294-5 Apply the Borda measure to the previous exercise294-6 Apply the method of paired comparison to Exercise 294-4
1360 29 Conflict Situations
Decision makers Alternatives Weights1 2 3 4 5
1234
1154
3443
2512
5231
4325
3221
Figure 2914 Group decision-making table
Decision makers Alternatives Weights1 2 3
1234
1321
2213
3132
1111
Figure 2915 Group decision-making table
295 Applications of Pareto games
Let I denote again the number of decision makers but suppose now that the deci-sion makers have more than one objective functions separately There are severalpossibility to handle such problems
(A) In the application of multi-objective programming let αi denote the weight ofthe ith decision maker and let βi1 βi2 βic(i) be the weights of this decisionmakerrsquos objective functions Here c(i) denote the number of the ith decisionmakerrsquos objective functions Thus we can get an optimization problem with thesumI
i=1 c(i) objective function where all of the decision makersrsquo all the objectivefunctions mean the objective function of the problem and the weights of thecertain objective functions are the αiβij sequences We can use any of themethods from Chapter 291 to solve this problem
(B) We can get another family of methods in the following way Determine anutility function for every decision maker (as described in Chapter 2911)which compresses the decision makerrsquos preferences into one function In theapplication of this method every decision maker has only one (new) objectivefunction so any methods and solution concepts can be used from the previouschapters
(C) A third method can be given if we determine only the partial order of thecertain decision makers defined on an alternative set by some method insteadof the construction of utility functions After that we can use any method ofChapter 294 directly
295 Applications of Pareto games 1361
Decision maker Objective function Alternatives Objective function Decision1 2 3 4 weight weigh
11 90 75 80 85 05
042 09 08 07 08 05
21 85 80 70 90 06
032 08 09 08 085 04
31 80 90 75 70 07
032 085 08 09 08 03
Figure 2916 The database of Example 2918
Example 2918 Modify the previous chapter as follows Letrsquos suppose again that we choosefrom four alternatives but assume now that the committee consists of three people andevery member of it has two objective functions The first objective function is the technicalstandards of the proposed solution on a subjective scale while the second one are the oddsof the exact implementation The latter one is judged subjectively by the decision makersindividually by the preceding works of the supplier The data is shown in Figure 2916where we assume that the first objective function is judged on a subjective scale from 0to 100 so the normalized objective function values are given dividing by 100 Using theweighting method we get the following aggregate utility function values for the separatedecision makers
1 Decision maker
First alternative 09(05) + 09(05) = 09Second alternative 075(05) + 08(05) = 0775Third alternative 08(05) + 07(05) = 075Fourth alternative 085(05) + 08(05) = 0825
2 Decision maker
First alternative 085(06) + 08(04) = 083Second alternative 08(06) + 09(04) = 084Third alternative 07(06) + 08(04) = 074Fourth alternative 09(06) + 085(04) = 088
3 Decision maker
First alternative 08(07) + 085(03) = 0815Second alternative 09(07) + 08(03) = 087Third alternative 075(07) + 09(03) = 0795Fourth alternative 07(07) + 08(03) = 073
The preferences thus are the following
1 1 4 1 2 1 3 4 2 2 2 1 2 3 and 2 3 1 3 3 3 4
1362 29 Conflict Situations
For example in the application of Borda measure
B1 = 1(04) + 3(03) + 2(03) = 19
B2 = 3(04) + 2(03) + 1(03) = 21
B3 = 4(04) + 4(03) + 3(03) = 37
B4 = 2(04) + 1(03) + 4(03) = 23
are given so the group-order of the four alternatives
1 2 4 3
Exercises295-1 Letrsquos consider the following table
Decision maker Objective function Alternatives Objective function Decision maker1 2 3 weight weight
11 06 08 07 06
052 09 07 06 04
21 05 03 04 05
0252 06 08 07 05
31 04 05 06 04
0252 07 06 06 043 05 08 06 02
Figure 2917
Letrsquos consider that the objective functions are already normalized Use method(A) to solve the exercise295-2 Use method (B) for the previous exercise where the certain decision makersrsquoutility functions are given by the weighting method and the group decision makingis given by the Borda measure295-3 Solve Exercise 295-2 using the method of paired comparison instead ofBorda measure
296 Axiomatic methods
For the sake of simplicity letrsquos consider that I = 2 that is wersquod like to solve theconflict between two decision makers Assume that the consequential space H isconvex bounded and closed in R
2 and there is given a flowast = (f1lowast f2lowast) point whichgives the objective function values of the decision makers in cases where they areunable to agree We assume that there is such f isin H that f gt flowast The conflict ischaracterized by the (H flowast) pair The solution obviously has to depend on both H
296 Axiomatic methods 1363
and flowast so it is some function of them φ(H flowast)For the case of the different solution concepts we demand that the solution
function satisfies some requirements which treated as axioms These axioms requirethe correctness of the solution the certain axioms characterize this correctness indifferent ways
In the case of the classical Nash solution we assume the following
(i) φ(H flowast) isin H (possibility)
(ii) φ(H flowast) ge flowast (rationality)
(iii) φ(H flowast) is Pareto solution in H (Pareto optimality)
(iv) If H1 sube H and φ(H flowast) isin H1 necessarily φ(H1 flowast) = φ(H flowast) (independenceof irrelevant alternatives)
(v) Be T R2 7rarr R2 such linear transformation that T (f1 f2) = (α1f1 +β1 α2f2 +
β2) is positive for α1 and α2 Then φ(T (H) T (flowast)) = T (φ(H flowast)) (invariantto affine transformations)
(vi) If H and flowast are symmetrical that is f1lowast = f2lowast and (f1 f2) isin H lArrrArr (f2 f1) isinH then the components of φ(H flowast) be equals (symmetry)
Condition (i) demands the possibility of the solution Condition (ii) requiresthat none of the rational decision makers agree on a solution which is worse thanthe one could be achieved without consensus On the basis of condition (iii) thereis no better solution than the friendly solution According to requirement (iv) ifafter the consensus some alternatives lost their possibility but the solution is stillpossible the solution remains the same for the reduced consequential space If thedimension of any of the objective functions changes the solution canrsquot change Thisis required by (v) and the last condition means that if two decision makers are in theabsolutely same situation defining the conflict we have to treat them in the sameway in the case of solution The following essential result originates from Nash
Theorem 296 The (i)-(vi) conditions are satisfied by exactly one solution func-tion and φ(H flowast) can be given by as the
(f1 minus f1lowast)(f2 minus f2lowast) minusrarr max ((f1 f2) isin H)f1 ge f1lowastf2 ge f2lowast
(2944)
optimum problem unique solution
Example 2919 Letrsquos consider again the consequential space showed in Figure 293 be-fore and suppose that (f1lowast f2lowast) = (0 minus1) that is it comprises the worst values in itscomponents Then Exercise (2944) is the following
f1(f2 + 1) minusrarr max
f2 le f1
f2 le 3 minus 2f1
f2 ge minus1
2f1
1364 29 Conflict Situations
Itrsquos easy to see that the optimal solution is f1 = f2 = 1
Notice that problem (2944) is a distance dependent method where we maximizethe geometric distance from the (f1lowast f2lowast) point The algorithm is the solution of the(2944) optimum problem
Condition (vi) requires that the two decision makers must be treated equallyHowever in many practical cases this is not an actual requirement if one of them isin stronger position than the other
Theorem 297 Requirements (i)-(v) are satisfied by infinite number of functionsbut every solution function comprises such 0 le α le 1 that the solution is given byas the
(f1 minus f1lowast)α(f2 minus f2lowast)1minusα minusrarr max ((f1 f2) isin H)f1 ge f1lowastf2 ge f2lowast
(2945)
optimum problem unique solution
Notice that in the case of α = 12 problem (2945) reduces to problem (2944)
The algorithm is the solution of the (2945) optimum problemMany author criticized Nashrsquos original axioms and beside the modification of the
axiom system more and more new solution concepts and methods were introducedWithout expose the actual axioms we will show the methods judged to be of theutmost importance by the literature
In the case of the KalaindashSmorodinsky solution we determine firstly the idealpoint which coordinates are
flowasti = maxfi|(f1 f2) isin H (f1 f2) ge flowast
then we will accept the last mutual point of the half-line joining flowast to the ideal pointand H as solution Figure 2918 shows the method Notice that this is an directiondependent method where the half-line shows the direction of growing and flowast is thechosen start point
The algorithm is the solution of the following optimum problem
t minusrarr max
provided thatflowast + t(flowast minus flowast) isin H
Example 2920 In the case of the previous example flowast = (0 minus1) and f lowast = (2 1) Wecan see in Figure 2919 that the last point of the half-line joining flowast to f lowast in H is theintersection point of the half-line and the section joining (1 1) to (2 minus1)
The equation of the half-line is
f2 = f1 minus 1
while the equation of the joining section is
f2 = minus2f1 + 3
296 Axiomatic methods 1365
Figure 2918 KalaindashSmorodinsky solution
Figure 2919 Solution of Example 2920
so the intersect point f1 = 43 f2 = 1
3
In the case of the equal-loss method we assume that starting from the idealpoint the two decision makers reduce the objective function values equally until theyfind a possible solution This concept is equivalent to the solution of the
t minusrarr min ((flowast1 minus t flowast
2 minus t) isin H) (2946)
optimum problem Let tlowast denote the minimal t value then the (flowast1 minustlowast flowast
2 minustlowast) pointis the solution of the conflict The algorithm is the solution of the (2946) optimumproblem
Example 2921 In the case of the previous example f lowast = (2 1) so starting from this pointgoing by the 45 line the first possible solution is the f1 = 4
3 f2 = 1
3point again
1366 29 Conflict Situations
Figure 2920 The method of monotonous area
In the case of the method of monotonous area the (f1 f2) solution is given byas follows The linear section joining (f1lowast f2lowast) to (f1 f2) divides the set H into twoparts if (f1 f2) is a Pareto optimal solution In the application of this concept werequire the two areas being equal Figure 2920 shows the concept The two areasare given by as follows
int f1
f1lowast
(g(t)minus f2lowast)dtminus 12
(f1 minus f1lowast)(g(f1)minus f2lowast)
and12
(f1 minus f1lowast)(g(f1)minus f2lowast) +int flowast
1
f1
(g(t)minus flowast2 )dt
where we suppose that f2 = g(f1) defines the graph of the Pareto optimal solutionThus we get a simple equation to determine the unknown value of f1
The algorithm is the solution of the following nonlinear univariate equation
int f1
f1lowast
(g(t)minus f2lowast)dtminusint f1lowast
f1
(g(t)minus flowast2 )dtminus (f1 minus f1lowast)(g(f1)minus f2lowast) = 0
Any commonly known (bisection secant Newtonrsquos method) method can be used tosolve the problem
Exercises296-1 Consider that H = (f1 f2)|f1 f2 ge 0 f1 + 2f2 le 4 Be f1lowast = f2lowast = 0 Usethe (2944) optimum problem296-2 Assume that the two decision makers are not equally important in theprevious exercise α = 1
3 1minus α = 23 Solve the (2945) optimum problem
296-3 Use the KalaindashSmorodinsky solution for Exercise 296-1296-4 Use the equal-loss method for Exercise 296-1296-5 Use the method of monotonous area for Exercise 296-1
Notes for Chapter 29 1367
Problems
29-1 Első feladat ciacutemeProve that the solution of problem (299) is Pareto optimal for any positiveα1 α2 αI values29-2 Masodik feladat ciacutemeProve that the distance dependent methods always give Pareto optimal solution for1 Is it also true for infin29-3 Harmadik feladat ciacutemeFind a simple example for which the direction dependent methods give non Paretooptimal solution29-4 Negyedik feladat ciacutemeSuppose in addition to the conditions of 294 that all of the fi functions are strictlyconcave in xi Give an example for which there are more than one equilibrium29-5 Oumltoumldik feladat ciacutemeProve that the Shapley values result imputation and satisfy the (2935)ndash(2936) con-ditions29-6 Hatodik feladat ciacutemeSolve such a group decision making table where the method of paired comparisondoesnrsquot satisfy the requirement of transitivity That is there are such i j k alterna-tives for which i j j k but k i29-7 Hetedik feladat ciacutemeConstruct such an example where the application of Borda measure equally qualifiesall of the alternatives29-8 Nyolcadik feladat ciacutemeProve that using the KalaindashSmorodinsky solution for non convex H the solution isnot necessarily Pareto optimal29-9 Kilencedik feladat ciacutemeShow that for non convex H neither the equal-loss method nor the method ofmonotonous area can guarantee Pareto optimal solution
Chapter Notes
Readers interested in multi-objective programming can find addition details andmethods related to the topic in the [220] book There are more details about themethod of equilibrium and the solution concepts of the cooperative games in the[67] monograph The [225] monograph comprises additional methods and formulasfrom the methodology of group decision making Additional details to Theorem296 originates from Hash can be found in [166] One can read more details aboutthe weakening of the conditions of this theorem in [94] Details about the KalaindashSmorodinsky solution the equal-loss method and the method of monotonous areacan found respectively in [122] [45] and [3] Note finally that the [228] summarypaper discuss the axiomatic introduction and properties of these and other newermethods
1368 29 Conflict Situations
The results discussed in this chapter can be found in the book of Molnaacuter Saacutendorand Szidarovszky Ferenc [157] in details
30 General Purpose Computing onGraphics Processing Units
GPGPU stands for General-Purpose computation on Graphics Processing Unitsalso known as GPU Computing Graphics Processing Units (GPU ) are highly par-allel multithreaded manycore processors capable of very high computation and datathroughput Once specially designed for computer graphics and programmable onlythrough graphics APIs todayrsquos GPUs can be considered as general-purpose parallelprocessors with the support for accessible programming interfaces and industry-standard languages such as C
Developers who port their applications to GPUs often achieve speedups of or-ders of magnitude vs optimized CPU implementations This difference comes fromthe high floating point performance and peak memory bandwidth of GPUs This isbecause the GPU is specialized for compute-intensive highly parallel computationmdashexactly what graphics rendering is aboutmdashand therefore designed such that moretransistors are devoted to data processing rather than data caching and flow con-trol From the developerrsquos point of view this means that hardware latencies are nothidden they must be managed explicitly and writing an efficient GPU program isnot possible without the knowledge of the architecture
Another reason of discussing GPGPU computing as a specific field of computerscience is that although a GPU can be regarded as a parallel system its architectureis not a clean implementation of parallel computing models (see Chapter 15 of thisbook titled Parallel Computations) Instead it has the features of many differentmodels like pipelines vector or array processors Single-Instruction Multiple-Data (SIMD) machines stream-processors multi-processors connected via sharedmemory hard-wired algorithms etc So when we develop solutions for this specialarchitecture the ideas applicable for many different architectures should be com-bined in creative ways
GPUs are available as graphics cards which must be mounted into computer sys-tems and a runtime software package must be available to drive the computationsA graphics card has programmable processing units various types of memory andcache and fixed-function units for special graphics tasks The hardware operationmust be controlled by a program running on the host computerrsquos CPU through Ap-plication Programming Interfaces (API ) This includes uploading programsto GPU units and feeding them with data Programs might be written and compiled
1370 30 General Purpose Computing on Graphics Processing Units
Graphics API programming model CUDA programming model
Figure 301 GPU programming models for shader APIs and for CUDA We depict here a ShaderModel 4 compatible GPU The programmable stages of the shader API model are red the fixed-function stages are green
from various programming languages some originally designed for graphics (likeCg [168] or HLSL [154]) and some born by the extension of generic programminglanguages (like CUDA C) The programming environment also defines a program-ming model or virtual parallel architecture that reflects how programmable andfixed-function units are interconnected Interestingly different programming modelspresent significantly different virtual parallel architectures (Figure 301) GraphicsAPIs provide us with the view that the GPU is a pipeline or a stream-processorsince this is natural for most of the graphics applications CUDA [169] or OpenCL[126] on the other hand gives the illusion that the GPU is a collection of multi-processors where every multiprocessor is a wide SIMD processor composed of scalarunits capable of executing the same operation on different data The number ofmultiprocessors in a single GPU can range nowadays up to a few hundreds and asingle multiprocessor typically contains 8 or 16 scalar units sharing the instructiondecoder
The total number of scalar processors is the product of the number of multi-processors and the number of SIMD scalar processors per multiprocessor which canbe well over a thousand This huge number of processors can execute the same pro-gram on different data A single execution of the program is called the thread Amultiprocessor executes a thread block All processors have some fast local mem-ory which is only accessible to threads executed on the same processor ie to athread block There is also global device memory to which data can be uploadedor downloaded from by the host program This memory can be accessed from mul-
301 The graphics pipeline model 1371
tiprocessors through different caching and synchronization strategies Compared tothe CPU this means less transistors for caching less cache performance in generalbut more control for the programmer to make use of the memory architecture in anefficient way
The above architecture favours the parallel execution of short coherent compu-tations on compact pieces of data Thus the main challenge of porting algorithms tothe GPU is that of parallelization and decomposition to independent computationalsteps GPU programs which perform such a step when executed by the processingunits are often called kernels or shaders the former alludes to the parallel dataprocessing aspect and the latter is a legacy of the fundamental graphics task thesimulation of light reflection at object surfaces better known as shading
GPU programming languages and control APIs have grown pretty similar to eachother in both capabilities and syntax but they can still be divided into graphicsand GPGPU solutions The two approaches can be associated with two differentprogrammer attitudes While GPGPU frameworks try to add some constructs toprogramming languages to prepare regular code for parallel execution graphics APIsextend previously very limited parallel shader programs into flexible computationaltools This second mindset may seem obsolete or only relevant in specific graphics-related scenarios but in essence it is not about graphics at all it is about theimplicit knowledge of how parallel algorithms work inherent to the incrementalimage synthesis pipeline Therefore we first discuss this pipeline and how the GPUdevice is seen by a graphics programmer This will not only make the purpose andoperation of device components clear but also provides a valid and tried approachto general purpose GPU programming and what GPU programs should ideallylook like Then we introduce the GPGPU approach which abandons most of thegraphics terminology and neglects task-specific hardware elements in favour of ahigher abstraction level
301 The graphics pipeline model
The graphics pipeline model provides an abstraction over the GPU hardware wherewe view it as a device which performs incremental image synthesis [221] (seeChapter 22 of this book titled Computer Graphics of this book) Incremental imagesynthesis aims to render a virtual world defined by a numerical model by transform-ing it into linear primitives (points lines triangles) and rasterizing these primitivesto pixels of a discrete image The process is composed of several algorithmic stepswhich are grouped in pipeline stages Some of these stages are realized by dedicatedhardware components while others are implemented through programs run by GPUsWithout going into details let us recap the image synthesis process (Figure 302)
bull The virtual world is a collection of model instances The models are approxi-mated using triangle meshes This is called
bull In order to perform shading the objects have to be transformed into the coordi-nate system where the camera and lights are specified This is either the worldspace or the camera space
1372 30 General Purpose Computing on Graphics Processing Units
Figure 302 Incremental image synthesis process
bull Triangle vertices are projected on-screen according to the camera settings Wherea vertex should appear on the screen is found by applying the camera transfor-mation the perspective transformation and finally the viewport trans-formation In camera space the camera is in the origin and looks at the minuszdirection Rays originating at the camera focus called the eye position andpassing through points on the window that represent the pixels of our displayform a perspective bundle The role of perspective transformation is to convertthis perspective bundle into a parallel bundle of rays thus to replace perspec-tive projection by a parallel projection After perspective transformation thevertices are in normalized device space where the visible volume is an axisaligned cube defined by inequalities minus1 le x le 1 minus1 le y le 1 minus1 le z le 1Parts of the geometric primitives that are outside of this volume are removedby clipping Normalized device space is further transformed to screen spacewhere the target image resolution and position are taken into account Points ofnormalized device space coordinates x = minus1 y = minus1 are mapped to the lowerleft corner of the viewport rectangle on the screen Points of x = 1 y = 1 areprojected to the upper right corner Meanwhile the z range of minus1 le z le 1 isconverted to [0 1]
bull In screen space every projected triangle is rasterized to a set of pixels When aninternal pixel is filled its properties including the z coordinate also called thedepth value and shading data are computed via incremental linear interpo-lation from the vertex data For every pixel a shading color is computed fromthe interpolated data The shading color of a pixel inside the projection of thetriangle might be the color interpolated from the vertex colors Alternativelywe can map images called textures onto the meshes Texture images are 2Darrays of color records An element of the texture image is called the texelHow the texture should be mapped onto triangle surfaces is specified by texturecoordinates assigned to every vertex
301 The graphics pipeline model 1373
bull Pixel colors are finally written to the frame buffer that is displayed on thecomputer screen Besides the frame buffer we maintain a depth buffer (alsocalled z-buffer or depth stencil texture) containing screen space depth whichis the z coordinate of the point whose color value is in the frame buffer Whenevera triangle is rasterized to a pixel the color and the depth are overwritten only ifthe new depth value is less than the depth stored in the depth buffer meaningthe new triangle fragment is closer to the viewer As a result we get a renderingof triangles correctly occluding each other in 3D This process is commonly calledthe depth buffer algorithm The depth buffer algorithm is also an exampleof a more general operation which computes the pixel data as some functionof the new data and the data already stored at the same location This generaloperation is called merging
3011 GPU as the implementation of incremental image synthe-sis
The GPU architecture as presented by the graphics API is the direct implementationof the image synthesis pipeline (left part of Figure 301) This pipeline is configuredby the CPU via graphics API calls and its operation is initiated by the draw call Asequence of draw calls during which the configuration of the pipeline does not change(but the inputs do) is called a pass A single draw call operates on a sequence ofvertices the attributes of which are stored in a vertex buffer
Vertices are expected to be specified in modeling space with homogeneous coor-dinates A point of Cartesian coordinates (x y z) can be defined by the quadrupleof homogeneous coordinates [xw yw zww] using an arbitrary non-zero scalarw (for more details see Chapter 21 Computer Graphics of this book) This represen-tation owns its name to the fact that if the elements of the quadruple are multipliedby the same scalar then the represented point will not change From homogeneousquadruple [XYZw] the Cartesian coordinates of the same point can be obtainedby homogeneous division that is as (Xw YwZw) Homogeneous coordinateshave several advantages over Cartesian coordinates When homogeneous coordinatesare used even parallel lines have an intersection (an ideal point) thus the singu-larity of the Euclidean geometry caused by parallel lines is eliminated Homogeneouslinear transformations include perspective projection as well which has an im-portant role in rendering but cannot be expressed as a linear function of Cartesiancoordinates Most importantly the widest class of transformations that preservelines and planes are those which modify homogeneous coordinates linearly
Having set the vertex buffer vertices defined by their coordinates and attributeslike texture coordinates or color begin their journey down the graphics pipelinevisiting processing stages implemented by programmable shader processors or fixed-function hardware elements We consider these stages one-by-one
Tessellation If the vertices do not directly define the final triangle mesh but theyare control points of a parametric surface or define just a coarse version of the meshthe first step is the development of the final mesh which is called tessellation Asthe programmability of this stage is limited and its GPGPU potential is small we
1374 30 General Purpose Computing on Graphics Processing Units
do not discuss this stage further but assume that the vertex buffer contains the finemesh that needs no tessellation
Vertex processing Objects must be transformed to normalized device spacefor clipping which is typically executed by a homogeneous linear transformationAdditionally GPUs may also take the responsibility of illumination computationat the vertices of the triangle mesh These operations are executed in the vertexshader From a more general point of view the vertex shader gets a single vertex ata time modifies its attributes including position color and texture coordinates andoutputs the modified vertex Vertices are processed independently and in parallel
The geometry shader The geometry shader stage receives vertex recordsalong with primitive information It may just pass them on as in the fixed-function pipeline or spawn new vertices Optionally these may be writ-ten to an output buffer which can be used as an input vertex buffer ina consecutive pass A typical application of the geometry shader is proce-dural modeling when a complex model is built from a single point or atriangle [148]
While vertex shaders have evolved from small specialized units to general streamprocessors they have kept the one record of output for every record of input schemeThe geometry shader on the other hand works on vertex shader output records(processed vertices) and outputs a varying (but limited) number of similar records
Clipping The hardware keeps only those parts of the primitives that are insidean axis aligned cube of corners (minus1minus1minus1) and (1 1 1) in normalized device spaceIn homogeneous coordinates a point should meet the following requirements to beinside
minusw le x le w minusw le y le w minusw le z le w This formulation complies to the OpenGL [167] convention It is valid eg in theCg language when compiling for an OpenGL vertex shader profile The last pair ofinequalities can also be defined as 0 le z le w as Direct3D assumes This is thecase for Cg Direct3D profiles and in the HLSL standard The difference is hiddenby compilers which map vertex shader output to what is expected by the clippinghardware
Clipping is executed by a fixed-function hardware of the GPU so its operationcan neither be programmed nor modified However if we wish our primitives tocontinue their path in further stages of the pipeline the conditions of the clippingmust be satisfied In GPGPU the clipping hardware is considered as a streamfilter If it turns out that a data element processed by vertex and geometry shaderprograms needs to be discarded vertices should be set to move the primitive out ofthe clipping volume Then the clipping hardware will delete this element from thepipeline
After clipping the pipeline executes that is it converts homogeneous coordinatesto Cartesian ones by dividing the first three homogeneous coordinates by the fourth(w) The points are then transformed to where the first two Cartesian coordinatesselect the pixel in which this point is visible
301 The graphics pipeline model 1375
Rasterization with linear interpolation The heart of the pipeline is the non-programmable rasterization stage This is capable of converting linear primitives(triangles line segments points) into discrete fragments corresponding to digitalimage pixels More simply put it draws triangles if the screen coordinates of thevertices are given Pipeline stages before the rasterizer have to compute these vertexcoordinates stages after it have to process the fragments to find pixel colors
Even though the base functionality of all stages can be motivated by rasteriza-tion GPGPU applications do not necessarily make use of drawing triangles Stillthe rasterizer can be seen to work as a stream expander launching an array of frag-ment computations for all primitive computations only the triangles have to be setup cleverly
Rasterization works in screen space where the x y coordinates of the verticesare equal to those integer pixel coordinates where the vertices are projected Thevertices may have additional properties such as a z coordinate in screen space tex-ture coordinates and color values When a triangle is rasterized all those pixels areidentified which fall into the interior of the projection of the triangle The prop-erties of the individual pixels are obtained from the vertex properties using linearinterpolation
Fragment shading The fragment properties interpolated from vertex propertiesare used to find the fragment color and possibly a modified depth value The classicaloperation for this includes fetching the texture memory addressed by the interpolatedtexture coordinates and modulating the result with the interpolated color
Generally fragment shader programs get the interpolated properties of the frag-ment and output the color and optionally modify the depth of the fragment Like thevertex shader the fragment shader is also one-record-in one-record-out type pro-cessor The fragment shader is associated with the target pixel so it cannot writeits output anywhere else
Merging When final fragment colors are computed they may not directly bewritten to the image memory but the output merger stage is responsible for thecomposition First the depth test against the depth buffer is performed Note thatif the fragment shader does not modify the z value depth testing might be movedbefore the execution of the fragment shader This early z-culling might improveperformance by not processing irrelevant fragments
Finally the output merger blends the new fragment color with the existing pixelcolor and outputs the result This feature could implement blending needed fortransparent surface rendering (Figure 303)
In GPGPU blending is mainly useful if we need to find the sum minimum ormaximum of results from consecutive computations without a need of reconfiguringthe pipeline between them
1376 30 General Purpose Computing on Graphics Processing Units
Figure 303 Blending unit that computes the new pixel color of the frame buffer as a function ofits old color (destination) and the new fragment color (source)
302 GPGPU with the graphics pipeline model
In general purpose programming we are used to concepts like input data tempo-rary data output data and functions that convert input data to temporary andfinally to output data according to their parameters If we wish to use the GPUas presented by a graphics API our programming concepts should be mapped ontothe concepts of incremental image synthesis including geometric primitives ver-texprimitivefragment processing rasterization texturing merging and final im-age There are many different possibilities to establish this correspondence and theircomparative advantages also depend on the actual algorithm Here we consider a fewgeneral approaches that have proven to be successful in high performance computingapplications First we discuss how our general purpose programming concepts canbe related to GPU features
3021 Output
GPUs render images ie two-dimensional arrays of pixels The render target canbe the frame buffer that is displayed or an output texture (in the latter case the pixelis often referred to as a texel) In GPGPU the output is usually a texture since texelscan be stored in floating point format unlike the final frame buffer values that areunsigned bytes Furthermore textures can be used later on as inputs of subsequentcomputation passes ie the two-dimensional output texture can be interpreted asone or two-dimensional input texture in the next rendering pass or as a single layerof a three-dimensional texture In older GPUs a pixel was capable of storing at mostfive floating point values since a color is typically identified by red green blue andopacity values and hidden surface elimination needed a single distance value whichis the z coordinate of the point in screen coordinates Later with the emergenceof multiple render targets a pixel could be associated with several eg fourtextures which means that the maximum size of an output record could grow to 17
302 GPGPU with the graphics pipeline model 1377
floats In current most advanced Shader Model 50 GPUs even this limitation hasbeen lifted so a single pixel can store a list of varying number of values
Which pixel is targeted by the rendering process is determined by the geometricelements Each primitive is transformed to screen space and its projection is raster-ized which means that those pixels are targeted that are inside the projection Ifmore than one element is sent down the pipeline their projections may overlap sothe pixel value is calculated multiple times The merging unit combines these par-tial results it may keep only one eg the fragment having minimal screen space zcoordinate if depth testing is enabled or it may add up partial results using blending
An important property of the render target is that it can be read directly by noneof the shader processors and only the fragment shader processor can indirectly writeinto it via the possible merging operation Different fragment shaders are assignedto different parts of the render target so no synchronization problem may occur
3022 Input
In image synthesis the inputs are the geometry stream and the textures used tocolor the geometry As a triangle mesh geometry has usually no direct meaning ina GPGPU application we use the geometry stream only as a control mechanism todistribute the computational load among the shader processors The real GPGPUinput will be the data stored in textures The texture is a one- two- or three-dimensional array of color data elements which can store one two three or fourscalars In the most general case the color has red green blue and opacity channelsThese color values can be stored in different formats including for example unsignedbytes or 32 bit floats From the point of view of GPGPU 32 bit floats are the mostappropriate
A one-dimensional float texture is similar to the linear CPU memory wherethe usual data structures like arrays lists trees etc can be encoded However theequivalence of the CPU memory and the GPU texture fails in two important aspectsIn one the texture is poorer in the other it is better than the linear CPU memory
An apparent limitation is that a texture is parallel read-only for all pro-grammable shaders with the exception of the render target that cannot be readby the shaders and is accessible only for the merger unit Read-modify-write cycleswhich are common in the CPU memory are not available in shader programs GPUdesigners had a good reason not to allow read-modify-write cycles and to classifytextures as parallel read-only and exclusive write-only In this way the writes do nothave to be cached and during reads caches get never invalidated
On the other hand the texture memory has much more addressing modes than alinear memory and more importantly they are also equipped with built-in texturefilters With the filters a texture is not only an array of elements but also a finiteelement representation of a one- two- or three-dimensional spatial function (referto Section 307 to learn more of the relation between finite element representationsand textures)
For one-dimensional textures we can use linear filtering which means that ifthe texture coordinate u points to a location in between two texels of coordinates Uand U + 1 then the hardware automatically computes a linear interpolation of the
1378 30 General Purpose Computing on Graphics Processing Units
two texel values Let these texels be T (U) and T (U + 1) The filtered value returnedfor u is then
T (u) = (1minus ulowast)T (U) + ulowastT (U + 1) where ulowast = uminus U Two-dimensional textures are filtered with bi-linear filtering taking the four
texels closest to the interpolated texture coordinate pair (u v) Let these be T (U V )T (U + 1 V ) T (U + 1 V + 1) and T (U V + 1) The filtered value returned for (u v)is then
T (U V )ulowastvlowast + T (U + 1 V )(1minus ulowast)vlowast + T (U + 1 V + 1)(1minus ulowast)(1minus vlowast)
+T (U V + 1)ulowast(1minus vlowast)
where ulowast = uminus U and vlowast = v minus V For three-dimensional textures tri-linear filtering is implemented
3023 Functions and parameters
As the primitives flow through the pipeline shader processors and fixed-functionelements process them determining the final values in each pixel The programs ofshader processors are not changed in a single rendering pass so we can say that eachpixel is computed by the very same program The difference of pixel colors is dueto data dependencies So in conclusion a GPU can be regarded as a hardware thatcomputes an array of records
In the GPU primitives are processed by a series of processors that are eitherprogrammable or execute fixed algorithms while output pixels are produced It meansthat GPUs can also be seen as stream processors Vertices defining primitives entera single virtual stream and are first processed by the vertex shader With streamprocessing terminology the vertex shader is a mapping since it applies a functionto the vertex data and always outputs one modified vertex for each input vertex Sothe data frequency is the same at the output as it was at the input The geometryshader may change the topology and inputting a single primitive it may outputdifferent primitives having different number of vertices The data frequency maydecrease when the stream operation is called reduction or may increase when itis called expansion The clipping unit may keep or remove primitives or may evenchange them if they are partially inside of the clipping volume If we ignore partiallykept primitives the clipping can be considered as a By setting the coordinates ofthe vertices in the vertex shader to be outside of the clipping volume we can filterthis primitive out of the further processing steps Rasterization converts a primitiveto possibly many fragments so it is an expansion The fragment shader is also amapping similarly to the vertex shader Finally merging may act as a selectionfor example based on the z coordinate or even as an accumulation if blending isturned on
Shader processors get their stream data via dedicated registers which are filledby the shader of the preceding step These are called varying input On the otherhand parameters can also be passed from the CPU These parameters are calleduniform input since they are identical for all elements of the stream and cannotbe changed in a pass
303 GPU as a vector processor 1379
full
screen
quad
rasterizer
fragment
shader
data
index
input
data
texture
output
merger
output
data
texture
fragments
Figure 304 GPU as a vector processor
303 GPU as a vector processor
If the computation of the elements is done independently and without sharing tem-porary results the parallel machines are called vector processors or array proces-sors As in the GPU hardware the fragment shader is associated with the elementsof the output data we use the fragment shader to evaluate output elements Ofcourse the evaluation in a given processor must also be aware which element is be-ing computed which is the fundamental source of data dependency (it would notmake sense to compute the very same data many times on a parallel machine) Inthe fragment shader the index of the data element is in fact the pair of the pixelcoordinates This is available in screen space as a pair of two integers specifying therow and the column where the pixel is located
In the simplest but practically the most important case we wish to have a resultin all pixels in a single rendering pass So we have to select a geometric primitive thatis mapped to all pixels in screen space and a single pixel is mapped only once Such ageometric primitive is the virtual display itself thus we should render a rectangle or aquadrilateral that represents the window of our virtual camera In screen space thisis the viewport rectangle in clipping space this is a square on the x y plane andhaving corners in homogeneous coordinates (minus1minus1 0 1) (1minus1 0 1) (1 1 0 1)(minus1 1 0 1) This rectangle is also called the full screen quad and is processed bythe hardware as two triangles (Figure 304)
Suppose that we want to compute an output array y of dimension N from aninput array x of possibly different dimension M and a global parameter p withfunction F
yi = F (ix p) i = 1 N
To set up the GPU for this computation we assign output array y to the outputtexture that is the current render target Texture size is chosen according to the out-put size and the viewport is set to cover the entire render target A two-dimensionalarray of H horizontal resolution and V vertical resolution is capable of storing HtimesVelements If H times V ge N then it is up to us how horizontal and vertical resolutionsare found However GPUs may impose restrictions eg they cannot be larger than212 or if we wish to use them as input textures in the next pass or compute binaryreductions the resolutions are preferred to be powers of two If power of two dimen-sions are advantageous but the dimension of the array is different we can extend
1380 30 General Purpose Computing on Graphics Processing Units
the array by additional void elementsAccording to vector processing principles different output values are computed
independently without sharing temporary results As in the GPU hardware the frag-ment shader is associated with the elements of the output data and can run inde-pendently of other elements we use the fragment shader to evaluate function F Tofind its parameters we need to know i ie which element is currently computed andshould have an access to input array x The simplest way is to store the input arrayas an input texture (or multiple input textures if that is more convenient) since thefragment shader can access textures
The only responsibility of the CPU is to set the uniform parameters specify theviewport and send a full screen quad down the pipeline Uniform parameters selectthe input texture and define global parameter p Assuming the OpenGL API thecorresponding CPU program in C would look like the following
StartVectorOperation( ) Set uniform parameters p and arrayX identifying the input texture
glViewport(0 0 H V) Set horizontal and vertical resolutions H and VglBegin(GL_QUADS) The next four vertices define a quad
glVertex4f(-1-1 0 1) Vertices assuming normalized device spaceglVertex4f(-1 1 0 1)glVertex4f( 1 1 0 1)glVertex4f( 1-1 0 1)
glEnd( )
Note that this program defines the rectangle directly in normalized device spaceusing homogeneous coordinates passed as input parameters of the glVertex4f func-tions So in the pipeline we should make sure that the vertices are not transformed
For the shader program the varying inputs are available in dedicated registersand outputs must also be written to dedicated registers All of these registers are oftype float4 that is they can hold 4 float values The role of the register is explainedby its name For example the current value of the vertex position can be fetchedfrom the POSITION register Similar registers can store the texture coordinatesor the color associated with this vertex
The vertex shader gets the position of the vertex and is responsible for trans-forming it to the normalized device space As we directly defined the vertices innormalized device space the vertex shader simply copies the content of its inputPOSITION register to its output POSITION register (the input and output classi-fication is given by the in and out keywords in front of the variable names assignedto registers)
void VertexShader( in float4 inputPos POSITIONout float4 outputPos POSITION )
outputPos = inputPos
The geometry shader should keep the rectangle as it is without changing thevertex coordinates As this is the default operation for the geometry shader we donot specify any program for it The rectangle leaving the geometry shader goes tothe clipping stage which keeps it since we defined our rectangle to be inside theclipping region Then Cartesian coordinates are obtained from the homogeneousones by dividing the first three coordinates by the fourth one As we set all fourth
303 GPU as a vector processor 1381
homogeneous coordinates to 1 the first three coordinates are not altered After ho-mogeneous division the fixed-function stage transforms the vertices of the rectangleto the vertices of a screen space rectangle having the x y coordinates equal to thecorners of the viewport and the z = 0 coordinate to 05 Finally this rectangle israsterized in screen space so all pixels of the viewport are identified as a target andthe fragment shader is invoked for each of them
The fragment shader is our real computing unit It gets the input array andglobal parameter p as uniform parameters and can also find out which pixel is beingcomputed by reading the WPOS register
float FragmentShaderF(in float2 index WPOS target pixel coordinatesuniform samplerRECT arrayX input arrayuniform float p global parameter p) COLOR output is interpreted as a pixel color
float yi = F(index arrayX p) F is the function to be evaluatedreturn yi
In this program two input parameters were declared as uniform inputs by theuniform keyword a float parameter p and the texture identification arrayX Thetype of the texture is samplerRECT that determines the addressing modes how atexel can be selected In this addressing mode texel centers are on a two-dimensionalinteger grid Note that here we used a different syntax to express what the output ofthe shader is Instead of declaring a register as out the output is given as a returnvalue and the function itself and is assigned to the output COLOR register
3031 Implementing the SAXPY BLAS function
To show concrete examples we first implement the level 1 functionality of the Ba-sic Linear Algebra Subprograms (BLAS) library (httpwwwnetliborgblas)that evaluates vector functions of the following general form
y = px + y
where x and y are vectors and p is a scalar parameter This operation is calledSAXPY in the BLAS library Now our fragment shader inputs two textures vectorx and the original version of vector y One fragment shader processor computes asingle element of the output vector
float FragmentShaderSAXPY(in float2 index WPOS target pixel coordinatesuniform samplerRECT arrayX input array xuniform samplerRECT arrayY original version of yuniform float p global parameter p) COLOR output is interpreted as a pixel color
float yoldi = texRECT(arrayY index) yoldi = arrayY[index]float xi = texRECT(arrayX index) xi = arrayX[index]float yi = p xi + yoldireturn yi
Note that instead of indexing an array of CPU style programming here we fetchthe element from a texture that represents the array by the texRECT Cg function
1382 30 General Purpose Computing on Graphics Processing Units
The first parameter of the texRECT function is the identification number of a two-dimensional texture which is passed from the CPU as a uniform parameter and thesecond is the texture address pointing to the texel to be selected
Here we can observe how we can handle the limitation that a shader can onlyread textures but is not allowed to write into it In the operation vector y is an inputand simultaneously also the output of the operation To resolve this we assign twotextures to vector y One is the original vector in texture arrayY and the other oneis the render target While we read the original value the new version is producedwithout reading back from the render target which would not be possible
3032 Image filtering
Another important example is the discrete convolution of two textures an image anda filter kernel which is the basic operation in many image processing algorithms
L(XY ) asympMsum
i=minusM
Msum
j=minusM
L(X minus i Y minus j)w(i j) (301)
where L(XY ) is the filtered value at pixel XY L(XY ) is the original image andw(x y) is the filter kernel which spans over (2M + 1)times (2M + 1) pixels
Now the fragment shader is responsible for the evaluation of a single output pixelaccording to the input image given as texture Image and the filter kernel stored ina smaller texture Weight The half size of the filter kernel M is passed as a uniformvariable
float3 FragmentShaderConvolution(in float2 index WPOS target pixel coordinatesuniform samplerRECT Image input imageuniform samplerRECT Weight filter kerneluniform float M size of the filter kernel) COLOR a pixel of the filtered image
float3 filtered = float3(0 0 0)
for(int i = -M i lt= M i++)for(int j = -M j lt= M j++)
float2 kernelIndex = float2(i j)float2 sourceIndex = index + kernelIndexfiltered += texRECT(Image sourceIndex) texRECT(Weight kernelIndex)
return filtered
Note that this example was a linear ie convolution filter but non-linear filters(eg median filtering) could be implemented similarly In this program we appliedarithmetic operators ( += =) for float2 and float3 type variables storing twoand three floats respectively The Cg compiler and the GPU will execute theseinstructions independently on the float elements
Note also that we did not care what happens at the edges of the image thetexture is always fetched with the sum of the target address and the shift of thefilter kernel A CPU implementation ignoring image boundaries would obviously bewrong since we would over-index the source array However the texture fetching
304 Beyond vector processing 1383
hardware implementing for example the texRECT function automatically solvesthis problem When the texture is initialized we can specify what should happen ifthe texture coordinate is out of its domain Depending on the selected option we getthe closest texel back or a default value or the address is interpreted in a periodicway
Exercises303-1 Following the vector processing concept write a pixel shader which when afull screen quad is rendered quantizes the colors of an input texture to a few levelsin all three channels achieving a cell shading effect303-2 Following the gathering data processing scheme write a pixel shader whichwhen a full screen quad is rendered performs median filtering on an input grayscaleimage achieving dot noise reduction The shader should fetch nine texel values froma neighborhood of 3times 3 outputting the fifth largest303-3 Implement an anisotropic edge preserving low-pass image filter with thegathering data processing scheme In order to preserve edges compute the Euclideandistance of the original pixel color and the color of a neighboring pixel and includethe neighbor in the averaging operation only when the distance is below a threshold
303-4 Write a parallel Mandelbrot set rendering program by assuming that pixelx y corresponds to complex number c = x + iy and deciding whether or not thezn = z2
nminus1 + c iteration diverges when started from z0 = c The divergence maybe checked by iterating n = 106 times and examining that |zn| is large enoughDivergent points are depicted with white non-divergent points with black
304 Beyond vector processing
Imagining the GPU as a vector processor is a simple but efficient application of theGPU hardware in general parallel processing If the algorithm is suitable for vectorprocessing then this approach is straightforward However some algorithms are notgood candidates for vector processing but can still be efficiently executed by theGPU In this section we review the basic approaches that can extend the vectorprocessing framework to make the GPU applicable for a wider range of algorithms
3041 SIMD or MIMD
Vector processors are usually SIMD machines which means that they execute notonly the same program for different vector elements but always the very same ma-chine instruction at a time It means that vector operations cannot contain datadependent conditionals or loop lengths depending on the actual data There is onlyone control sequence in a SIMD parallel program
Of course writing programs without if conditionals and using only constants asloop cycle numbers are severe limitations which significantly affects the programstructure and the ease of development Early GPU shaders were also SIMD typeprocessors and placed the burden of eliminating all conditionals from the program
1384 30 General Purpose Computing on Graphics Processing Units
on the shoulder of the programmer Current GPUs and their compilers solve thisproblem automatically thus on the programming level we can use conditionals andvariable length loops as if the shaders were MIMD computers On execution leveladditional control logic makes it possible that execution paths of scalar units divergein this case it is still a single instruction which is executed at a time possibly withsome scalar units being idle Operations of different control paths are serialized sothat all of them are completed The overhead of serialization makes performancestrongly dependent on the coherence of execution paths but many transistors ofcontrol logic can be spared for more processing units
The trick of executing all branches of conditionals with possibly disabled writesis called predication Suppose that our program has an if statement like
if (condition(i)) F( )
else G( )
Depending on the data on some processors the condition(i) may be true whileit is false on other processors thus our vector machine would like to execute functionF of the first branch in some processors while it should evaluate function G of thesecond branch in other processors As in SIMD there can be only one control paththe parallel system should execute both paths and disable writes when the processoris not in a valid path This method converts the original program to the followingconditional free algorithm
enableWrite = condition(i)F( )enableWrite = enableWriteG( )
This version does not have conditional instructions so it can be executed by theSIMD machine However the computation time will be the the sum of computationtimes of the two functions
This performance bottleneck can be attacked by decomposing the computationinto multiple passes and by the exploitation of the feature The early z-cull comparesthe z value of the fragment with the content of the depth buffer and if it is smallerthan the stored value the fragment shader is not called for this fragment but thefragment processor is assigned to another data element The early z-cull is enabledautomatically if we execute fragment programs that do not modify the fragmentrsquos zcoordinate (this is the case in all examples discussed so far)
To exploit this feature we decompose the computation into three passes In thefirst pass only the condition is evaluated and the depth buffer is initialized with thevalues Recall that if the z value is not modified our full screen quad is on the xyplane in normalized device space so it will be on the z = 05 plane in screen spaceThus to allow a discrimination according to the condition we can set values in therange (05 1) if the condition is true and in (0 05) if it is false
The fragment shader of the first pass computes just the condition values andstores them in the depth buffer
float FragmentShaderCondition(in float2 index WPOS target pixel coordinatesuniform samplerRECT Input input vector
304 Beyond vector processing 1385
) DEPTH the output goes to the depth buffer
bool condition = ComputeCondition( texRECT(Input index) )return (condition) 08 02 08 is greater than 05 02 is smaller than 05
Then we execute two passes for the evaluation of functions F and G respectivelyIn the first pass the fragment shader computes F and the depth comparison is setto pass those fragments where their z = 05 coordinate is less than the depth valuestored in the depth buffer In this pass only those fragments are evaluated wherethe depth buffer has 08 value ie where the previous condition was true Then inthe second pass the fragment shader is set to compute G while the depth bufferis turned to keep those fragments where the fragmentrsquos depth is greater than thestored value
In Subsection 3071 we exploit early z-culling to implement a variable lengthloop in fragment processors
3042 Reduction
The vector processing principle assumes that the output is an array where elementsare obtained independently The array should be large enough to keep every shaderprocessor busy Clearly if the array has just one or a few elements then only one ora few shader processors may work at a time so we loose the advantages of parallelprocessing
In many algorithms the final result is not a large array but is a single valuecomputed from the array Thus the algorithm should reduce the dimension of theoutput Doing the in a single step by producing a single texel would not benefitfrom the parallel architecture Thus reduction should also be executed in parallelin multiple steps This is possible if the operation needed to compute the result fromthe array is associative which is the case for the most common operations like sumaverage maximum or minimum
Σ
Σ
Σ
Σ
Σ
Σ
Σ
inp
ut
dat
a
ou
tpu
t d
ata
thread 1
thread 2
thread 3
thread 4
Figure 305 An example for parallel reduction that sums the elements of the input vector
Suppose that the array is encoded by a two-dimensional texture At a singlephase we downsample the texture by halving its linear resolution ie replacing fourneighboring texels by a single texel The fragment shaders will compute the operationon four texels If the original array has 2ntimes2n resolution then n reduction steps are
1386 30 General Purpose Computing on Graphics Processing Units
needed to obtain a single 1times 1 output value In the following example we computethe sum of the elements of the input array (Figure 305) The CPU program rendersa full screen quad in each iteration having divided the render target resolution bytwo
Reduction( ) Set uniform parameter arrayX to identify the input texture
for(N = 2 N gt= 1 N = 2) log_2 N iterationsglViewport(0 0 N N) Set render target dimensions to hold NxN elementsglBegin(GL_QUADS) Render a full screen quad
glVertex4f(-1-1 0 1)glVertex4f(-1 1 0 1)glVertex4f( 1 1 0 1)glVertex4f( 1-1 0 1)
glEnd( )
Copy render target to input texture arrayX
The fragment shader computes a single reduced texel from four texels as a sum-mation in each iteration step
float FragmentShaderSum( ) (in float2 index WPOS target pixel coordinatesuniform samplerRECT arrayX input array x) COLOR output is interpreted as a pixel color
float sum = texRECT(arrayX 2 index)sum += texRECT(arrayX 2 index + float2(1 0))sum += texRECT(arrayX 2 index + float2(1 1))sum += texRECT(arrayX 2 index + float2(0 1))return sum
Note that if we exploited the bi-linear filtering feature of the texture memorythen we could save three texture fetch operations and obtain the average in a singlestep
3043 Implementing scatter
In vector processing a processor is assigned to each output value ie every processorshould be aware which output element it is computing and it is not allowed to derouteits result to somewhere else Such a static assignment is appropriate for gatheringtype computations The general structure of gathering is that we may rely on adynamically selected set of input elements but the variable where the output isstored is known a-priory
index = ComputeIndex( ) index of the input datay = F(x[index])
Opposed to gathering algorithms may have scattering characteristics ie agiven input value may end up in a variable that is selected dynamically A simplescatter operation is
index = ComputeIndex( ) index of the output datay[index] = F(x)
Vector processing frameworks and our fragment shader implementation are un-able to implement scatter since the fragment shader can only write to the pixel it
304 Beyond vector processing 1387
rasterizerfragment
shaderoutput
data
indexinput
data
texture
output
data
texture
vertex
shader
0 1 2 3 4 5 6 vertex buffer
input data index
Figure 306 Implementation of scatter
has been assigned toIf we wish to solve a problem having scattering type algorithm on the GPU
we have two options First we can restructure the algorithm to be of gatheringtype Converting scattering type parallel algorithms to gathering type ones requiresa change of our viewpoint how we look at the problem and its solution For examplewhen integral equations or transport problems are considered this corresponds tothe solution of the adjoint problem [] Secondly we can move the index calculationup to the pipeline and use the rasterizer to establish the dynamic correspondencebetween the index and the render target (Figure 306)
Let us consider a famous scattering type algorithm histogram generationSuppose we scan an input array x of dimension M evaluate function F for theelements and calculate output array y of dimension N that stores the number offunction values that are in bins equally subdividing range (Fmin Fmax)
A scalar implementation of histogram generation would be
Histogram( x ) for(int i = 0 i lt M i++)
index = (int)((F(x[i]) - Fmin)(Fmax - Fmin) N) binindex = max(index 0)index = min(index N-1)y[index] = y[index] + 1
We can see that the above function writes to the output array at random loca-tions meaning it cannot be implemented in a fragment shader which is only allowedto write the render target at its dedicated index The problem of scattering will besolved by computing the index in the vertex shader but delegating the responsibilityof incrementing to the rest of the pipeline The indices are mapped to output pix-els by the rasterization hardware The problem of read-modify-write cycles might besolved by starting a new pass after each increment operation and copying the currentrender target as an input texture of the next rendering pass However this solutionwould have very poor performance and would not utilize the parallel hardware atall A much better solution uses the arithmetic capabilities of the merging unitThe fragment shader generates just the increment (ie value 1) where the histogramneeds to be updated and gives this value to the merging unit The merging unit inturn adds the increment to the content of the render target
1388 30 General Purpose Computing on Graphics Processing Units
The CPU program generates a point primitive for each input data elementAdditionally it sets the render target to match the output array and also enablesthe merging unit to execute add operations
ScanInputVector( ) Set uniform parameters Fmin Fmax N
glDisable(GL_DEPTH_TEST) Turn depth buffering offglBlendFunc(GL_ONE GL_ONE) Blending operation dest = source 1 + dest 1glEnable(GL_BLEND) Enable blending
glViewport(0 0 N 1) Set render target dimensions to hold N elementsglBegin(GL_POINTS) Assign a point primitive to each input elementsfor(int i = 0 i lt M i++)
glVertex1f( x[i] ) an input element as a point primitiveglEnd( )
The vertex positions in this level are not important since it turns out later wherethis point will be mapped So we use the first coordinate of the vertex to pass thecurrent input element x[i]
The vertex shader gets the position of the vertex currently storing the inputelement and finds the location of this point in normalized device space First func-tion F is evaluated and the bin index is obtained then we convert this index to the[minus1 1] range since in normalized device space these will correspond to the extremesof the viewport
void VertexShaderHistogram(in float inputPos POSITIONout float4 outputPos POSITIONuniform float Fminuniform float Fmaxuniform float N )
float xi = inputPosint index = (int)((F(xi) - Fmin)(Fmax - Fmin) N) binindex = max(index 0)index = min(index N-1)float nindex = 20 index N - 10 normalized device spaceoutputPos = float4(nindex 0 0 1) set output coordinates
The above example is not optimized Note that the index calculation and thenormalization could be merged together and we do not even need the size of theoutput array N to execute this operation
The fragment shader will be invoked for the pixel on which the point primitiveis mapped It simply outputs an increment value of 1
float FragmentShaderIncr( ) COLOR output is interpreted as a pixel color
return 1 increment that is added to the render target by merging
3044 Parallelism versus reuse
Parallel processors running independently offer a linear speed up over equivalentscalar processor implementations However scalar processors may benefit from rec-ognizing similar parts in the computation of different output values so they can
304 Beyond vector processing 1389
Figure 307 Caustics rendering is a practical use of histogram generation The illumination in-tensity of the target will be proportional to the number of photons it receives (images courtesy ofDaacutevid Balambeacuter)
increase their performance utilizing reuse As parallel processors may not reusedata generated by other processors their comparative advantages become less at-tractive
GPUs are parallel systems of significant streaming capabilities so if data thatcan be reused are generated early we can get the advantages of both independentparallel processing and the reuse features of scalar computing
Our main stream expander is the rasterization Thus anything happens beforerasterization can be considered as a global computation for all those pixels that arefilled with the rasterized version of the primitive Alternatively the result of a passcan be considered as an input texture in the next pass so results obtained in theprevious pass can be reused by all threads in the next pass
Exercises304-1 Implement a parallel regula falsi equation solver for (2 minus a minus b)x3 + ax2 +bx minus 1 = 0 that searches for roots in [0 1] for many different a and b parametersThe a and b parameters are stored in a texture and the pixel shader is responsiblefor iteratively solving the equation for a particular parameter pair Terminate theiteration when the error is below a given threshold Take advantage of the early z-culling hardware to prevent further refinement of the terminated iterations Analyzethe performance gain304-2 Based on the reduction scheme write a program which applies simple lineartone mapping to a high dynamic range image stored in a floating-point texture Thescaling factor should be chosen to map the maximum texel value to the value of oneFind this maximum using iterative reduction of the texture304-3 Based on the concept of scatter implement a caustics renderer program(Figure 307) The scene includes a point light source a glass sphere and a diffusesquare that is visualized on the screen Photons with random directions are generatedby the CPU and passed to the GPU as point primitives The vertex shader tracesthe photon through possible reflections or refractions and decides where the photonwill eventually hit the diffuse square The point primitive is directed to that pixeland the photon powers are added by additive alpha blending304-4 Based on the concept of scatter given an array of GSM transmitter towercoordinates compute cell phone signal strength on a 2D grid Assume signal strength
1390 30 General Purpose Computing on Graphics Processing Units
diminishes linearly with the distance to the nearest transmitter Use the rasterizerto render circular features onto a 2D render target and set up blending to pick themaximum
305 GPGPU programming model CUDA andOpenCL
The Compute Unified Device Architecture (CUDA) and the interfaces pro-vide the programmer with a programming model that is significantly different fromthe graphics pipeline model (right of Figure 301) It presents the GPU as a col-lection of multiprocessors where each multiprocessor contains several SIMD scalarprocessors Scalar processors have their own registers and can communicate insidea multiprocessor via a fast shared memory Scalar processors can read cachedtextures having built-in filtering and can read or write the slow global memory Ifwe wish even read-modify-write operations can also be used Parts of the globalmemory can be declared as a texture but from that point it becomes read-only
Unlike in the graphics API model the write to the global memory is not ex-clusive and atomic add operations are available to support semaphores and dataconsistency The fixed-function elements like clipping rasterization and merging arenot visible in this programming model
Comparing the GPGPU programming model to the graphics API model wenotice that it is cleaner and simpler In the GPGPU programming model parallelprocessors are on the same level and can access the global memory in an unrestrictedway while in the graphics API model processors and fixed-function hardware formstreams and write is possible only at the end of the stream When we programthrough the GPGPU model we face less restrictions than in the graphics pipelinemodel However care should be practiced since the graphics pipeline model for-bids exactly those features that are not recommended to use in high performanceapplications
The art of programming the GPGPU model is an efficient decomposition of theoriginal algorithm to parallel threads that can run with minimum amount of datacommunication and synchronization but always keep most of the processors busyIn the following sections we analyze a fundamental operation the matrix-vectormultiplication and discuss how these requirements can be met
306 Matrix-vector multiplication
Computational problems are based on mathematical models and their numericalsolution The numerical solution methods practically always rely on some kind oflinearization resulting in algorithms that require us to solve linear systems of equa-tions and perform matrix-vector multiplication as a core of the iterative solutionThus matrix-vector multiplication is a basic operation that can be if implementedefficiently on the parallel architecture the most general building block in any nu-
306 Matrix-vector multiplication 1391
merical algorithm We define the basic problem to be the computation of the resultvector y from input matrix A vectors x and b as
y = Ax + b
We call this the MV problem Let N timesM be the dimensions of matrix A As everyinput vector element may contribute to each of the output vector elements a scalarCPU implementation would contain a double loop one loop scans the input elementswhile the other the output elements If we parallelize the algorithm by assigningoutput elements to parallel threads then we obtain a gathering type algorithm wherea thread gathers the contributions of all input elements and aggregates them to thethreadrsquos single output value On the other hand if we assigned parallel threads toinput elements then a thread would compute the contribution of this input elementto all output elements which would be a scatter operation In case of gatheringthreads share only input data but their output is exclusive so no synchronizationis needed In case of scattering multiple threads may add their contribution to thesame output element so atomic adds are needed which may result in performancedegradation
An implementation of the matrix-vector multiplication on a scalar processorlooks like the followingvoid ScalarMV(int N int M float y const float A const float x const float b)
for(int i=0 iltN i++) float yi = b[i]for(int j=0 jltM j++) yi += A[i M + j] x[j]y[i] = yi
The first step of porting this algorithm to a parallel machine is to determinewhat a single thread would do from this program From the options of gatheringand scattering we should prefer gathering since that automatically eliminates theproblems of non-exclusive write operations In a gathering type solution a threadcomputes a single element of vector y and thus we need to start N threads A GPUcan launch a practically unlimited number of threads that are grouped in threadblocks Threads of a block are assigned to the same multiprocessor So the nextdesign decision is how the N threads are distributed in blocks A multiprocessortypically executes 32 threads in parallel so the number of threads in a block shouldbe some multiple of 32 When the threads are halted because of a slow memoryaccess a hardware scheduler tries to continue the processing of other threads so itis wise to assign more than 32 threads to a multiprocessor to always have threadsthat are ready to run However increasing the number of threads in a single blockmay also mean that at the end we have just a few blocks ie our program will runjust on a few multiprocessors Considering these we assign 256 threads to a singleblock and hope that N256 exceeds the number of multiprocessors and thus we fullyutilize the parallel hardware
There is a slight problem if N is not a multiple of 256 We should assign thelast elements of the vector to some processors as well so the thread block numbershould be the ceiling of N256 As a result of this we shall have threads that are notassociated with vector elements It is not a problem if the extra threads can detect
1392 30 General Purpose Computing on Graphics Processing Units
it and cause no harm eg they do not over-index the output arraySimilarly to the discussed vector processing model a thread should be aware
which output element it is computing The CUDA library provides implicit inputparameters that encode this information blockIdx is the index of the thread blockblockDim is the number of threads in a block and threadIdx is the index of thethread inside the block
The program of the CUDA kernel computing a single element of the outputvector is now a part of a conventional CPU program
__global__ void cudaSimpleMV(int N int M float y float A float x float b)
Determine element to process from thread and block indicesint i = blockIdxx blockDimx + threadIdxxif(i lt N) if the index is out of the range of the output array skip
float yi = b[i]for(int j=0 jltM j++) yi += A[i M + j] x[j]y[i] = yi
The global keyword tells the compiler that this function will run not on theCPU but on the GPU and it may be invoked from the CPU as well The parametersare passed according to the normal C syntax The only special feature is the use ofthe implicit parameters to compute the identification number of this thread whichis the index of the output array
The kernels are started from a CPU program that sets the parameters and alsodefines the number of thread blocks and the number of threads inside a block
__host__ void run_cudaSimpleMV()
int threadsPerBlock = 256 number of threads per blockint blockNum = (N + threadsPerBlock - 1)threadsPerBlock number of blockscudaSimpleMVltltltblockNum threadsPerBlockgtgtgt(N M y A x b)
The compiler will realize that this function runs on the CPU by reading thehost keyword The parallel threads are started like a normal C function call withthe exception of the ltblockNum threadsPerBlockgt tag which defines how manythreads should be started and how they are distributed among the multiprocessors
3061 Making matrix-vector multiplication more parallel
So far we assigned matrix rows to parallel threads and computed scalar productAix serially inside threads If the number of matrix rows is less than the numberof parallel scalar processors this amount of parallelization is not enough to supplyall processing units with work to do and the execution of individual threads will belengthy Reformulating the scalar product computation is a well known but tougherparallelization problem as the additions cannot be executed independently and werequire a single scalar to be written for every row of the matrix However parts ofthe summation can be executed independently and then the results added This is aclassic example of It is required that the threads whose results are to be added bothfinish execution and write their results to where they are accessible for the threadthat needs to add them Thus we use thread synchronization and available onlyfor the threads of the same block
306 Matrix-vector multiplication 1393
Let us assume firstmdashunrealisticallymdashthat we can have M threads processing arow and the shared memory can hold M floating point values Let Q be the vector oflength M residing in shared memory Then every thread can compute one elementQj as Aijxj Finally elements of Q must be reduced by summation Let us furtherassume that M = 2k The reduction can be carried out in k steps terminatinghalf of the threads while each surviving thread adds the value in Q computed bya terminated one to its own The final remaining thread outputs the value to theglobal memory
define M THE_NUMBER_OF_MATRIX_COLUMNS__global__ void cudaReduceMV(int N float y float A float x float b)
int i = blockIdxxint j = threadIdxx
__shared__ float Q[M] in the shader memory inside a multiprocessor
Q[j] = A[i M + j] x[j] a parallel part of matrix-vector multiplication
for(int stride = M 2 stride gt 0 stride gtgt= 1) reduction
__syncthreads() wait until all other threads of the block arrive this pointif(j + stride lt M)
Q[j] += Q[j + stride]
if(j == 0) reduced to a single elementy[i] = Q[0] + b[i]
__host__ void run_cudaReduceMV()
cudaReduceMVltltlt N M gtgtgt(N y A x b)
For practical matrix dimensions (M gt 104) neither the number of possiblethreads of a single multiprocessor nor the size of the shared memory is enough toprocess all elements in parallel In our next example we use a single block of threadswith limited size to process a large matrix First we break the output vector intosegments of size T Elements within such a segment are evaluated in parallel then thethreads proceed to the next segment Second for every scalar product computationwe break the vectors Ai and x into segments of length Z We maintain a sharedvector Qt of length Z for every row being processed in parallel We can compute theelementwise product of the Ai and x segments in parallel and add it to Qt As Trows are being processed by Z threads each the block will consist of T timesZ threadsFrom one threadrsquos perspective this means it has to loop over y with a stride of Tand for every such element in y loop over Ai and x with a stride of Z Also forevery element in y the contents of Qt must be summed by reduction as before Thecomplete kernel which works with large matrices would then be
__global__ void cudaLargeMV(int N int M float y float A float x float b)
__shared__ float Q[T Z] stored in the shared memory inside a multiprocessor
int t = threadIdxx Zint z = threadIdxx Z
for(int i = t i lt N i += T)
1394 30 General Purpose Computing on Graphics Processing Units
Q[t Z + z] = 0for(int j = z j lt M j += Z)
Q[t Z + z] += A[i M + j] x[j]
for(int stride = Z 2 stride gt 0 stride gtgt= 1)
__syncthreads()if(z + stride lt Z)
Q[t Z + z] += Q[t Z + z + stride]
if(z == 0)y[i] = Q[t Z + 0] + b[i]
__host__ void run_cudaLargeMV()
cudaReduceMVltltlt 1 TZ gtgtgt(N M y A x b)
This can easily be extended to make use of multiple thread blocks by restrictingthe outer loop to only a fraction of the matrix rows based on the blockIdx parameter
The above algorithm uses shared memory straightforwardly and allows us toalign memory access of threads through a proper choice of block sizes Howeverevery element of vector x must be read once for the computation of every row Wecan improve on this if we read values of x into the shared memory and have threadsin one block operate on multiple rows of the matrix This however means we can useless shared memory per line to parallelize summation The analysis of this trade-offis beyond the scope of this chapter but a block size of 64times 8 has been proposed in[76] With such a strategy it is also beneficial to access matrix A as a texture asdata access will exhibit 2D locality supported by texture caching hardware
Even though matrix-vector multiplication is a general mathematical formulationfor a wide range of computational problems the arising matrices are often large butsparse In case of sparse matrices the previously introduced matrix-vector multipli-cation algorithms will not be efficient as they explicitly compute multiplication withzero elements Sparse matrix representations and MV algorithms are discussed in[20]
Exercises306-1 Implement matrix-vector multiplication for large matrices in CUDA Com-pare results to a CPU implementation306-2 Implement an inverse iteration type Julia set renderer The Julia set is theattractor of the zn = z2
nminus1 +c iteration where zn and c are complex numbers Inverseiteration starts from a fixed point of the iteration formula and iterates the inversemapping zn = plusmnradiczn minus c by randomly selecting either
radiczn minus c or minusradiczn minus c from
the two possibilities Threads must use pseudo-random generators that are initializedwith different seeds Note that CUDA has no built-in random number generator soimplement one in the program
307 Case study computational fluid dynamics 1395
B1
B2
B3
1
1
1
functionapproximation
B1
B2
B3
1
1
1
B41
B1
B2
B3
1
1
1
B1
B2
B3
1
1
1
Piece-wise constant Piece-wise linear Harmonic Haar wavelet
Figure 308 Finite element representations of functions The texture filtering of the GPU directlysupports finite element representations using regularly placed samples in one- two- and three-dimensions and interpolating with piece-wise constant and piece-wise linear basis functions
307 Case study computational fluid dynamics
Problems emerging in physics or engineering are usually described mathematicallyas a set of partial differential or integral equations As physical systems expand inspace and time derivatives or integrals should be evaluated both in temporal andspatial domains
When we have to represent a value over space and time we should use functionshaving the spatial position and the time as their variables The representation ofgeneral functions would require infinite amount of data so in numerical methodswe only approximate them with finite number of values Intuitively these valuescan be imagined as the function values at discrete points and time instances Thetheory behind this is the finite element method If we need to represent functionf(~r) with finite data we approximate the function in the following finite series form(Figure 308)
f(~r) asymp f(~r) =Nsum
i=1
fiBi(~r)
where B1(~r) BN (~r) are pre-defined basis functions and f1 fN are thecoefficients that describe f
A particularly simple finite element representation is the piece-wise linear schemethat finds possibly regularly placed sample points ~r1 ~rN in the domain evalu-ates the function at these points to obtain the coefficients fi = f(~ri) and linearlyinterpolates between ~ri and ~ri+1
When the system is dynamic solution f will be time dependent so a new finiteelement representation is needed for every time instance We have basically two
1396 30 General Purpose Computing on Graphics Processing Units
options for this We can set sample points ~r1 ~rN in a static way and allow onlycoefficients fi to change in time This approach is called Eulerian On the otherhand we can also allow the sample points to move with the evaluation of the systemmaking also sample points ~ri time dependent This is the Lagrangian approachwhere sample locations are also called particles
Intuitive examples of Eulerian and Lagrangian discretization schemes are howtemperature and other attributes are measured in meteorology In ground stationsthese data are measured at fixed locations However meteorological balloons canalso provide the same data but from varying positions that follow the flow of theair
In this section we discuss a case study for GPU-based scientific computationThe selected problem is computational fluid dynamics Many phenomena thatcan be seen in nature like smoke cloud formation fire and explosion show fluid-likebehavior Understandably there is a need for good and fast fluid solvers both inengineering and in computer animation
The mathematical model of the fluid motion is given by the Navier-Stokes equa-tion First we introduce this partial differential equation then discuss how GPU-based Eulerian and Langrangian solvers can be developed for it
A fluid with constant density and temperature can be described by its velocity~v = (vx vy vz) and pressure p fields The velocity and the pressure vary both inspace and time
~v = ~v(~r t) p = p(~r t)
Let us focus on a fluid element of unit volume that is at point ~r at time t At anearlier time instance t minus dt this fluid element was in ~r minus ~vdt and according to thefundamental law of dynamics its velocity changed according to an acceleration thatis equal to total force ~F divided by mass ρ of this unit volume fluid element
~v(~r t) = ~v(~r minus ~vdt tminus dt) +~F
ρdt
Mass ρ of a unit volume fluid element is called the fluid density Moving thevelocity terms to the left side and dividing the equation by dt we can express thesubstantial derivative of the velocity
~v(~r t)minus ~v(~r minus ~vdt tminus dt)dt
=~F
ρ
The total force can stem from different sources It may be due to the pressuredifferences
~Fpressure = minus~nablap = minus(
partp
partxpartp
partypartp
partz
)
where ~nablap is the gradient of the pressure field The minus sign indicates that thepressure accelerates the fluid element towards the low pressure regions Here we usedthe nabla operator which has the following form in a Cartesian coordinate system
~nabla =(
part
partxpart
partypart
partz
)
307 Case study computational fluid dynamics 1397
Due to friction the fluid motion is damped This damping depends on the vis-cosity ν of the fluid Highly viscous fluids like syrup stick together while low-viscosity fluids flow freely The total damping force is expressed as a diffusion termsince the viscosity force is proportional to the Laplacian of the velocity field
~Fviscosity = ν ~nabla2~v = ν
(
part2~v
partx2+part2~v
party2+part2~v
partz2
)
Finally an external force field ~Fexternal may also act on our fluid element causingacceleration In the gravity field of the Earth assuming that the vertical directionis axis z this external acceleration is (0 0minusg) where g = 98 [ms2]
Adding the forces together we can obtain the Navier-Stokes equation for thevelocity of our fluid element
ρ~v(~r t)minus ~v(~r minus ~vdt tminus dt)
dt= minus~nablap+ ν ~nabla2
~v + ~Fexternal
In fact this equation is the adaptation of the fundamental law of dynamics forfluids If there is no external force field the momentum of the dynamic system mustbe preserved This is why this equation is also called momentum conservationequation
Closed physics systems preserve not only the momentum but also the mass sothis aspect should also be built into our fluid model Simply put the mass conser-vation means that what flows into a volume must also flow out so the divergence ofthe mass flow is zero If the fluid is incompressible then the fluid density is constantthus the mass flow is proportional to the velocity field For incompressible fluids themass conservation means that the velocity field is divergence free
~nabla middot ~v =partvx
partx+partvy
party+partvz
partz= 0 (302)
3071 Eulerian solver for fluid dynamics
The Eulerian approach tracks the evolution of the velocity and pressure fields onfixed uniform grid points The grid allows a simple approximation of spatial deriva-tives by finite differences If the grid points are in distances ∆x ∆y and ∆z alongthe three coordinate axes and the values of scalar field p and vector field ~v at gridpoint (i j k) are pijk and ~vijk respectively then the gradient the divergence andthe Laplacian operators can be approximated as
~nablap asymp(
pi+1jk minus piminus1jk
2∆xpij+1k minus pijminus1k
2∆ypijk+1 minus pijkminus1
2∆x
)
(303)
~nabla middot ~v asymp vi+1jkx minus viminus1jk
x
2∆x+vij+1k
y minus vijminus1ky
2∆y+vijk+1
z minus vijkminus1z
2∆z (304)
~nabla2p asymp pi+1jk minus 2pijk + piminus1jk
(∆x)2+pij+1k minus 2pijk + pijminus1k
(∆x)2
1398 30 General Purpose Computing on Graphics Processing Units
+pijk+1 minus 2pijk + pijkminus1
(∆x)2 (305)
The Navier-Stokes equation and the requirement that the velocity is divergencefree define four scalar equations (the conservation of momentum is a vector equation)with four scalar unknowns (vx vy vz p) The numerical solver computes the currentfields advancing the time in discrete steps of length ∆t
~v(~r t) = ~v(~r minus ~v∆t tminus∆t) +ν∆tρ
~nabla2~v +
∆tρ~Fexternal minus
∆tρ~nablap
The velocity field is updated in several steps each considering a single term on theright side of this equation Let us consider these steps one-by-one
Advection To initialize the new velocity field at point ~r we fetch the previousfield at position ~r minus ~v∆t since the fluid element arriving at point ~r was there in theprevious time step [219] This step computes advection ie the phenomenon thatthe fluid carries its own velocity field
~w1(~r) = ~v(~r minus ~v∆t tminus∆t)
Diffusion To damp the velocity field we could update it proportionally to adiffusion term
~w2 = ~w1 +ν∆tρ
~nabla2~w1
However this type of forward Euler integrator is numerically unstable The rea-son of instability is that forward methods predict the future based on the presentvalues and as time passes each simulation step adds some error which may accu-mulate and exceed any limit
Unlike forward integrators a backward method can guarantee stability A back-ward looking approach is stable since while predicting the future it simultaneouslycorrects the past Thus the total error converges to a finite value and remainsbounded Here a backward method means that the Laplacian is obtained from thefuture yet unknown velocity field and not from the current velocity field
~w2 = ~w1 +ν∆tρ
~nabla2~w2 (306)
At this step of the computation the advected field ~w1 is available at the grid pointsthe unknowns are the diffused velocity ~wijk
2 for each of the grid points Using (305)to compute the Laplacian of the x y z coordinates of unknown vector field ~w2 atgrid point (i j k) we observe that it will be a linear function of the ~w2 velocitiesin the (i j k) grid point and its neighbors Thus (306) is a sparse linear system ofequations
w2 = w1 + A middotw2 (307)
where vector w1 is the vector of the known velocities obtained by advection w2 isthe vector of unknown velocities of the grid points and matrix-vector multiplicationA middotw2 represents the discrete form of (ν∆tρ)nabla2 ~w2(~r)
307 Case study computational fluid dynamics 1399
Such systems are primary candidates for Jacobi iteration (see Chapter 12 ofthis book titled Scientific Computation) Initially we fill vector w2 with zero andevaluate the right side of (307) iteratively moving the result of the previous stepto vector w2 of the right side Thus we traced back the problem to a sequence ofsparse vector-matrix multiplications Note that matrix A needs not be stored Whenvelocity field ~w2 is needed at a grid point the neighbors are looked up and the simpleformula of (305) gives us the result
Updating a value in a grid point according to its previous value and the values ofits neighbors are called image filtering Thus a single step of the Jacobi iterationis equivalent to an image filtering operation which is discussed in Section 3032
External force field The external force accelerates the velocity field at eachgrid point
~w3 = ~w2 +∆tρ~Fexternal
Projection So far we calculated an updated velocity field ~w3 without consideringthe unknown pressure field In the projection step we compute the unknown pressurefield p and update the velocity field with it
~v(t) = ~w3 minus∆tρ~nablap
The pressure field is obtained from the requirement that the final velocity fieldmust be divergence free Let us apply the divergence operator to both sides of thisequation After this the left side becomes zero since we aim at a divergence freevector field for which ~nabla middot ~v = 0
0 = ~nabla middot(
~w3 minus∆tρ~nablap)
= ~nabla middot ~w3 minus∆tρ~nabla2p
Assuming a regular grid where vector field ~w3 is available searching the unknownpressure at grid positions and evaluating the divergence and the Laplacian withfinite differences of equations (304) and (305) respectively we again end up witha sparse linear system for the discrete pressure values and consequently for thedifference between the final velocity field ~v and ~w3 This system is also solved withJacobi iteration Similarly to the diffusion step the Jacobi iteration of the projectionis also a simple image filtering operation
Eulerian simulation on the GPU The discretized velocity and pressure fieldscan be conveniently stored in three-dimensional textures where discrete variablesare defined at the centers of elemental cubes called voxels of a grid [93] At eachtime step the content of these data sets should be refreshed (Figure 309)
The representation of the fields in textures has an important advantage whenthe advection is computed The advected field at voxel center ~ri is obtained bycopying the field value at position ~ri minus ~vi∆t Note that the computed position isnot necessarily a voxel center but it can be between the grid points Accordingto the finite element concept this value can be generated from the finite element
1400 30 General Purpose Computing on Graphics Processing Units
Figure 309 A time step of the Eulerian solver updates textures encoding the velocity field
Advection Jacobi iteration
Figure 3010 Computation of the simulation steps by updating three-dimensional textures Ad-vection utilizes the texture filtering hardware The linear equations of the viscosity damping andprojection are solved by Jacobi iteration where a texel (ie voxel) is updated with the weighted sumof its neighbors making a single Jacobi iteration step equivalent to an image filtering operation
representation of the data If we assume piece-wise linear basis functions then thetexture filtering hardware automatically solves this problem for us at no additionalcomputation cost
The disadvantage of storing vector and scalar fields in three-dimensional texturesis that the GPU can only read these textures no matter whether we take the graphicsAPI or the GPGPU approach The updated field must be written to the render targetin case of the graphics API approach and to the global memory if we use a GPGPUinterface Then for the next simulation step the last render target or global memoryshould be declared as an input texture
In order to avoid write collisions we follow a gathering approach and assignthreads to each of the grid points storing output values If GPUs fetch global datavia textures then the new value written by a thread becomes visible when the pass orthe thread run is over and the output is declared as an input texture for the next runThus the computation of the time step should be decomposed to elemental updatesteps when the new output value of another grid point is needed It means that wehave and advection pass a sequence of Jacobi iteration passes of the diffusion stepan external force calculation pass and another sequence of Jacobi iteration passesof the projection step With a GPGPU framework a thread may directly read the
307 Case study computational fluid dynamics 1401
Figure 3011 Flattened 3D velocity (left) and display variable (right) textures of a simulation
data produced by another thread but then synchronization is needed to make surethat the read value is already valid so not the old but the new value is fetchedIn such cases synchronization points have the same role and passes or decomposedkernels
In case of graphics APIs there is one additional limitation The render target canonly be two-dimensional thus either we flatten the layers of the three-dimensionalvoxel array into a large two-dimensional texture or update just a single layer ata time Flattened three-dimensional textures are shown by Figure 3011 Once thetextures are set up one simulation step of the volume can be done by the renderingof a quad covering the flattened grid
The graphics API approach has not only drawbacks but also an advantage overthe GPGPU method when the linear systems are solved with Jacobi iteration Thegraphics API method runs the fragment shader for each grid point to update thesolution in the texel associated with the grid point However if the neighbor ele-ments of a particular grid point are negligible we need less iteration steps than ina grid point where the neighbor elements are significant In a quasi-SIMD machinelike the GPU iterating less in some of the processors is usually a bad idea Howeverthe exploitation of the early z-culling hardware helps to sidestep this problem andboosts the performance [224] The z coordinate in the depth value is set propor-tionally to the maximum element in the neighborhood and to the iteration countThis way as the iteration proceeds the GPU processes less and less number of frag-ments and can concentrate on important regions According to our measurementsthis optimization reduces the total simulation time by about 40
When we wish to visualize the flow we can also assume that the flow carries ascalar display variable with itself The display variable is analogous with some paintor confetti poured into the flow The display variable is stored in a float voxel array
Using the advection formula for display variable D its field can also be updated
1402 30 General Purpose Computing on Graphics Processing Units
Figure 3012 Snapshots from an animation rendered with Eulerian fluid dynamics
in parallel with the simulation of time step ∆t
D(~r t) = D(~r minus ~v∆t tminus∆t)
At a time the color and opacity of a point can be obtained from the display variableusing a user controlled transfer function
We can use a 3D texture slicing rendering method to display the resulting displayvariable field which means that we place semi-transparent polygons perpendicularto the view plane and blend them together in back to front order (Figure 3012) Thecolor and the opacity of the 3D texture is the function of the 3D display variablefield
3072 Lagrangian solver for differential equations
In the Lagrangian approach the space is discretized by identifying ie followingjust finite number of fluid elements Let us denote the position and the velocity ofthe ith discrete fluid element by ~ri and ~vi respectively We assume that all particlesrepresent fluid elements of the same mass m but as the density varies in space andwill be the attribute of the particle every particle is associated with a differentvolume ∆Vi = mρi of the fluid The momentum conservation equation has thefollowing form in this case
d~ri
dt= ~vi
md~vi
dt=(
minus~nablap(~ri) + ν ~nabla2~v(~ri) + ~Fexternal(~ri)
)
∆Vi (308)
If particles do not get lost during the simulation the mass is automatically conservedHowever temporarily this mass may concentrate in smaller parts of the volumeso the simulated fluid is not incompressible In Lagrangian simulation we usuallyassume compressible gas
From the knowledge of the system at discrete points attributes are obtained atan arbitrary point via interpolation Suppose we know an attribute A at the particlelocations ie we have A1 AN Attribute A is interpolated at location ~r by a
307 Case study computational fluid dynamics 1403
weighted sum of contributions from the particles
A(~r) =Nsum
i=1
Ai∆ViW (|~r minus ~ri|)
where ∆Vi is the volume represented by the particle in point ~ri and W (d) is asmoothing kernel also called radial basis function that depends on distanced between the particle location and the point of interest From a different point ofview the smoothing kernel expresses how quickly the impact of a particle diminishesfarther away The smoothing kernel is normalized if smoothing preserves the totalamount of the attribute value which is the case if the kernel has unit integral overthe whole volumetric domain An example for the possible kernels is the spiky kernelof maximum radius h
W (d) =15πh6
(hminus d)3 if 0 le d le h and zero otherwise
For normalized kernels the particle density at point ~rj is approximated as
ρj = ρ(~rj) =Nsum
i=1
mW (|~rj minus ~ri|)
As each particle has the same mass m the volume represented by particle j is
∆Vj =m
ρj=
1sumN
i=1 W (|~rj minus ~ri|)
According to the ideal gas law the pressure is inversely proportional to the volumeon constant temperature thus at particle j the pressure is
pj =k
∆Vj
where constant k depends on the temperatureThe pressure at an arbitrary point ~r is
p(~r) =Nsum
i=1
pi∆ViW (|~r minus ~ri|)
The acceleration due to pressure differences requires the computation of the gradientof the pressure field As spatial variable ~r shows up only in the smoothing kernelthe gradient can be computed by using the gradient of the smoothing kernel
~nablap(~r) =Nsum
i=1
pi∆Vi~nablaW (|~r minus ~ri|)
Thus our first guess for the pressure force at particle j is
~Fpressurej = minus~nablap(~rj) = minusNsum
i=1
pi∆Vi~nablaW (|~rj minus ~ri|)
1404 30 General Purpose Computing on Graphics Processing Units
However there is a problem here Our approximation scheme could not guaranteeto satisfy the physical rules including symmetry of forces and consequently the con-servation of momentum We should make sure that the force on particle i due toparticle j is always equal to the force on particle j due to particle i The symmetricrelation can be ensured by modifying the pressure force in the following way
~Fpressurej = minusNsum
i=1
pi + pj
2∆Vi
~nablaW (|~rj minus ~ri|)
The viscosity term contains the Laplacian of the vector field which can becomputed by using the Laplacian of the smoothing kernel
~Fviscosityj = ν ~nabla2~v = ν
Nsum
i=1
~vi∆Vi~nabla2W (|~rj minus ~ri|)
Similarly to the pressure force a symmetrized version is used instead that makesthe forces symmetric
~Fviscosityj = ν
Nsum
i=1
(~vi minus ~vj)∆Vi~nabla2W (|~rj minus ~ri|)
External forces can be directly applied to particles Particle-object collisions aresolved by reflecting the velocity component that is perpendicular to the surface
Having computed all forces and approximating the time derivatives of (308) byfinite differences we may obtain the positions and velocities of each of the particlesin the following way
~ri(t+ ∆t) = ~ri(t) + ~vi(t)∆t
~vi(t+ ∆t) = ~vi(t) + (~Fpressurei + ~Fviscosityi + ~Fexternali)∆Vi∆tm
Note that this is also a forward Euler integration scheme which has stabilityproblems Instead of this we should use a stable version for example the Verletintegration [55]
The Lagrangian approach tracks a finite number of particles where the forcesacting on them depend on the locations and actual properties of other particlesThus to update a system of N particles O(N2) interactions should be examinedSuch tasks are generally referred to as the N-body problem
Lagrangian solver on the GPU In a GPGPU framework the particle at-tributes can be stored in the global memory as a one-dimensional array or can befetched via one-dimensional textures In graphics API frameworks particle attributescan only be represented by textures The advantage of reading the data via texturesis only the better caching since now we cannot utilize the texture filtering hardwareA gathering type method would assign a thread to each of the controlled particlesand a thread would compute the effect of other particles on its own particle As thesmoothing kernel has finite support only those particles can interact with the con-sidered one which are not farther than the maximum radius of the smoothing filter
307 Case study computational fluid dynamics 1405
Figure 3013 Data structures stored in arrays or textures One-dimensional float3 arrays store theparticlesrsquo position and velocity A one-dimensional float2 texture stores the computed density andpressure Finally a two-dimensional texture identifies nearby particles for each particle
Figure 3014 A time step of the Lagrangian solver The considered particle is the red one and itsneighbors are yellow
It is worth identifying these particles only once storing them in a two-dimensionaltexture of in the global memory and using this information in all subsequent kernels
A GPGPU approach would need three one-dimensional arrays representing theparticle position velocity density and pressure and a two-dimensional array for theneighboring particles (Figure 3013) In a graphics API approach these are one-or two-dimensional textures We can run a kernel or a fragment shader for each ofthe particles In a GPGPU solution it poses no problem for the kernel to output acomplete column of the neighborhood array but in the fragment shaders of olderGPUs the maximum size of a single fragment is limited To solve this we may limitthe number of considered neighbor particles to the number that can be outputtedwith the available multiple render target option
The processing of a single particle should be decomposed to passes or kernelruns when we would like to use the already updated properties of other particles(Figure 3014) The first pass is the identification of the neighbors for each particlesie those other particles that are closer than the support of the smoothing kernelThe output of this step is a two-dimensional array where columns are selected bythe index of the considered particle and the elements in this column store the indexand the distance of those particles that are close by
The second pass calculates the density and the pressure from the number andthe distance of the nearby particles Having finished this pass the pressure of everyparticle will be available for all threads The third pass computes the forces from thepressure and the velocity of nearby particles Finally each particle gets its updatedvelocity and is moved to its new position
Having obtained the particle positions the system can be visualized by different
1406 30 General Purpose Computing on Graphics Processing Units
Figure 3015 Animations obtained with a Lagrangian solver rendering particles with spheres(upper image) and generating the isosurface (lower image) [99]
methods For example we can render a point or a small sphere for each particle(upper image of Figure 3015) Alternatively we can splat particles onto the screenresulting in a rendering style similar to that of the Eulerian solver (Figure 3012)Finally we can also find the surface of the fluid and compute reflections and re-fractions here using the laws of geometric optics (lower image of Figure 3015) Thesurface of fluid is the isosurface of the density field which is the solution of thefollowing implicit equation
ρ(~r) = ρiso
This equation can be solved for points visible in the virtual camera by ray march-ing We trace a ray from the eye position through the pixel and make small stepson it At every sample position ~rs we check whether the interpolated density ρ(~rs)has exceeded the specified isovalue ρiso The first step when this happens is the in-tersection of the ray and the isosurface The rays are continued from here into thereflection and refraction directions The computation of these directions also requiresthe normal vector of the isosurface which can be calculated as the gradient of thedensity field
Exercises307-1 Implement a game-of-life in CUDA On a two-dimensional grid of cells everycell is either populated of unpopulated In every step all cell states are re-evaluatedFor populated cells
bull Each cell with one or no neighbors dies as if by loneliness
bull Each cell with four or more neighbors dies as if by overpopulation
bull Each cell with two or three neighbors survives
Notes for Chapter 30 1407
For unpopulated cells
bull Each cell with three neighbors becomes populated
Store cell states in arrays accessible as textures Always compute the next iterationstate into a different output array Start with a random grid and display resultsusing the graphics API307-2 Implement a wave equation solver The wave equation is a partial differentialequation
part2z
partt2= c2
(
part2z
partx2+part2z
party2
)
where z(x y t) is the wave height above point x y in time t and c is the speed ofthe wave
Chapter Notes
The fixed transformation and multi-texturing hardware of GPUs became pro-grammable vertex and fragment shaders about a decade ago The high floating pointprocessing performance of GPUs has quickly created the need to use them not onlyfor incremental rendering but for other algorithms as well The first GPGPU al-gorithms were also graphics related eg ray tracing or the simulation of naturalphenomena An excellent review about the early years of GPGPU computing can befound in [171] Computer graphics researchers have been very enthusiastic to workwith the new hardware since its general purpose features allowed them to implementalgorithms that are conceptually different from the incremental rendering includ-ing the physically plausible light transport called global illumination [221] physicssimulation of rigid body motion with accurate collision detection fluid dynamicsetc which made realistic simulation and rendering possible in real-time systemsand games The GPU Gems book series [63 145 183] and the ShaderX (currentlyGPU Pro [58]) series provide a huge collection of such methods
Since the emergence of GPGPU platforms like CUDA and OpenCL GPU solu-tions have showed up in all fields of high performance computing Online warehousesof papers and programs are the gpgpuorg homepage and the NVIDIA homepage[168 169] which demonstrate the wide acceptance of this approach in many fieldsWithout aiming at completeness successful GPU applications have targeted highperformance computing tasks including simulation of all kinds of physics phenom-ena differential equations tomographic reconstruction computer vision databasesearches and compression linear algebra signal processing molecular dynamics anddocking financial informatics virus detection finite element methods Monte Carlomethods simulation of computing machines (CNN neural networks quantum com-puters) pattern matching DNA sequence alignment cryptography digital hologra-phy quantum chemistry etc
To get a scalable system that is not limited by the memory of a single GPUcard we can build GPU clusters A single PC can be equipped with four GPUs andthe number of interconnected PCs is unlimited [245] However in such systems the
1408 30 General Purpose Computing on Graphics Processing Units
communication will be the bottleneck since current communication channels cannotcompete with the computing power of GPUs
31 Perfect Arrays
An (n a b)-perfect double cube is a btimes btimes b sized n-ary periodic array containingall possible a times a times a sized n-ary array exactly once as subarray A growing cubeis an array whose cj times cj times cj sized prefix is an (nj a cj)-perfect double cube forj = 1 2 where cj = n
v3j v = a3 and n1 lt n2 lt middot middot middot We construct the smallest
possible perfect double cube (a 256times256times256 sized 8-ary array) and growing cubesfor any a
311 Basic concepts
Cyclic sequences in which every possible sequence of a fixed length occurs exactlyonce have been studied for more than a hundred years [64] The same problem whichcan be applied to position localization was extended to arrays [60]
Let Z be the set of integers For u v isin Z we denote the set j isin Z | u le j le vby [uv] and the set j isin Z | j ge u by [uinfin] Let d isin [1infin] and k n isin [2infin]bi ci ji isin [1infin] (i isin [1d]) and ai ki isin [2infin] (i isin [1d]) Let a = 〈a1 a2 ad〉b = 〈b1 b2 bd〉 c = 〈c1 c2 cd〉 j = 〈j1 j2 jd〉 and k = 〈k1 k2 kd〉be vectors of length d n = 〈n1 n2 〉 an infinite vector with 2 le n1 lt n2 lt middot middot middot
A d-dimensional n-ary array A is a mapping A [1infin]d rarr [0 nminus 1]If there exist a vector b and an array M such that
forallj isin [1infin]d A[j] = M [(j1 mod b1) + 1 (j2 mod b2) + 1 (jd mod bd) + 1]
then A is a bperiodic array and M is a period of AThe a-sized subarrays of A are the a-periodic n-ary arraysAlthough our arrays are infinite we say that a b-periodic array is b-sizedIndexset Aindex of a b-periodic array A is the Cartesian product
Aindex = timesdi=1[1bi]
A d dimensional b-periodic n-ary array A is called (n dab)-perfect if allpossible n-ary arrays of size a appear in A exactly once as a subarray
Here n is the alphabet size d gives the number of dimensions of the ldquowindowrdquoand the perfect array M the vector a characterizes the size of the window and thevector b is the size of the perfect array M
An (n dab)-perfect array A is called c-cellular if ci divides bi for i isin [1d]
1410 31 Perfect Arrays
A cellular array consists of b1c1 times b2c2 times middot middot middot times bdcd disjoint subarrays of size ccalled cells In each cell the element with smallest indices is called the head of thecell The contents of the cell is called pattern
The product of the elements of a vector a is called the volume of the vectorand is denoted by |a| The number of elements of the perfect array M is called thevolume of M and is denoted by |M |
If b1 = b2 = middot middot middot = bd then the (n dab)-perfect array A is called symmetricIf A is symmetric and a1 = a2 = middot middot middot = ad then A is called doubly symmetric IfA is doubly symmetric and
1 d = 1 then A is called a double sequence
2 d = 2 then A is called a double square
3 d = 3 then A is called a double cube
According to this definition all perfect sequences are doubly symmetric In thecase of symmetric arrays we use the notion (n da b) and in the case of doublysymmetric arrays we use (n d a b) instead of (n dab)
The first known result originates from Flye-Sainte [64] who proved the existenceof (2 1 a 2a)-perfect sequences for all possible values of a in 1894
One dimensional perfect arrays are often called de Bruijn [34] or Good [81]sequences Two dimensional perfect arrays are called also perfect maps [173] or deBruijn tori [102 103 106]
De Bruijn sequences of even length ndash introduced in [120] ndash are useful in con-struction of perfect arrays when the size of the alphabet is an even number and thewindow size is 2times 2 Their definition is as follows
If n is an even integer then an (n 1 2 n2)-perfect sequence M = (m1m2 mn2) is called even if mi = x mi+1 = y x 6= ymj = y and mj+1 = ximply j minus i is even
Ivaacutenyi and Toacuteth [120] and later Hurlbert and Isaak [103] provided a constructiveproof of the existence of even sequences
Lexicographic indexing of an array M = [mj1j2jd] = [mj] (1 le
ji le bi) for i isin [1d] means that the index I(mj) is defined as
I(mj) = j1 minus 1 +dsum
i=2
(
(ji minus 1)iminus1prod
m=1
bm
)
The concept of perfectness can be extended to infinite arrays in various ways Ingrowing arrays [103] the window size is fixed the alphabet size is increasing andthe prefixes grow in all d directions
Let a and d be positive integers with a ge 2 and n = 〈n1 n2 〉 be a strictlyincreasing sequence of positive integers An array M = [mi1i2id
] is called (n d a)-growing if the following conditions hold
1 M = [mi1i2id] (1 le ij ltinfin) for j isin [1d]
2 mi1i2idisin [0nminus 1]
312 Necessary condition and earlier results 1411
3 the prefix Mk = [mi1i2id] (1 le ij le n
addk for j isin [1d]) of M is
(nk d a naddk )-perfect array for k isin [0infin]
For the growing arrays we use the terms growing sequence growing square andgrowing cube
For a n isin [2infin] the new alphabet size N(n a) is
N(n a) =
n if any prime divisor of a divides n nq otherwise
(311)
where q is the product of the prime divisors of a not dividing nNote that alphabet size n and new alphabet size N have the property that
n | N furthermore n = N holds in the most interesting case d = 3 and n = a1 =a2 = a3 = 2
The aim of this chapter is to prove the existence of a double cube As a side-effectwe show that there exist (n d a)-growing arrays for any n d and a
312 Necessary condition and earlier results
Since in the period M of a perfect array A each element is the head of a patternthe volume of M equals the number of the possible patterns Since each pattern ndashamong others the pattern containing only zeros ndash can appear only once any size ofM is greater then the corresponding size of the window So we have the followingnecessary condition [46 103] If M is an (n dab)-perfect array then
|b| = n|a| (312)
andbi gt ai for i isin [1d] (313)
Different construction algorithms and other results concerning one and two di-mensional perfect arrays can be found in the fourth volume of The Art of ComputerProgramming written by D E Knuth [] Eg a (21532)-perfect array [ page22] a 36-length even sequence whose 4-length and 16-length prefixes are also evensequences [ page 62] a (2224)-perfect array [ page 38] and a (42216)-perfectarray [ page 63]
It is known [34 ] that in the one-dimensional case the necessary condition (312)is sufficient too There are many construction algorithms like the ones of Cock [46]Fan Fan Ma and Siu [60] Martin [150] or any algorithm for constructing of directedEuler cycles []
Chung Diaconis and Graham [44] posed the problem to give a necessary andsufficient condition of the existence of (n 2ab)-perfect arrays
The conditions (2) and (3) are sufficient for the existence of (22ab)-perfectarrays [60] and (n2ab)-perfect arrays [172] Later Paterson in [173 174] suppliedfurther sufficient conditions
Hurlbert and Isaak [103] gave a construction for one and two dimensional growingarrays
1412 31 Perfect Arrays
313 One-dimensional arrays
In the construction of one-dimensional perfect arrays we use the following algorithmsAlgorithm Martin generates one-dimensional perfect arrays Its inputs are the
alphabet size n and the window size a Its output is an n-ary perfect sequence oflength na The output begins with a zeros and always continues with the maximalpermitted element of the alphabet
3131 Pseudocode of the algorithm Quick-Martin
A natural implementation of Martinrsquos algorithm can be found in the chapter Com-plexity of words of this book The following effective implementation of Martin isdue to M Horvaacuteth and A Ivaacutenyi
Quick-Martin(n a)
1 for i = 0 to naminus1 minus 12 C[i] = nminus 13 for i = 1 to a4 w[i] = 05 for i = a+ 1 to na
6 k = w[iminus a+ 1]7 for j = 1 to aminus 18 k = kn+ w[iminus a+ j]9 w[i] = C[k]10 C[k] = C[k]minus 111 return w
This algorithm runs in Θ(ana) time The following implementation of Martinalgorithm requires even smaller time
3132 Pseudocode of the algorithm Optimal-Martin
Optimal-Martin(n a)
1 for i = 0 to naminus1 minus 12 C[i] = nminus 13 for i = 1 to a4 w[i] = 05 for i = a+ 1 to na
6 k = w[iminus a+ 1]7 for j = 1 to aminus 18 k = kn+ w[iminus a+ j]9 w[i] = C[k]10 C[k] = C[k]minus 111 return w
The running time of any algorithm which constructs a on perfect array isΩ(na) since the sequance contains na elements The running time of Optimal-
313 One-dimensional arrays 1413
Martin is Θ(na)
3133 Pseudocode of the algorithm Shift
Algorithm Shift proposed by Cook in 1988 is a widely usable algorithm to constructperfect arrays We use it to transform cellular (N d ab)-perfect arrays into (N d+1 a c)-perfect arrays
Shift(N d a Pd Pd+1)
1 Martin(Nad
aminus 1w)2 for j = 0 to Nadminusadminus1 minus 13 transform wi to an ad digit N -ary number4 produce the (j + 1)-st layer of the output Pd+1 by multiple shifting
the jth layer of Pd by the transformed number (the first a digitsgive the shift size for the first direction then the next a2 minus a digitsin the second direction etc)
5 return Pd+1
3134 Pseudocode of the algorithm Even
If N is even then this algorithm generates the N2-length prefix of an even growingsequence [103]
Even(Nw)
1 if N == 22 w[1] = 03 w[2] = 04 w[3] = 15 w[4] = 16 return w7 for i = 1 to N2minus 18 for j = 0 to 2iminus 19 w[4i2 + 2j + 1] = j10 for j = 0 to iminus 111 w[4i2 + 2 + 4j] = 2i12 for j = 0 to iminus 113 w[4i2 + 4 + 4j] = 2i+ 114 for j = 0 to 4iminus 115 w[4i2 + 4i+ 1 + j] = w[4i2 + 4iminus j]16 w[4i2 + 8i+ 1] = 2i+ 117 w[4i2 + 8i+ 2] = 2i18 w[4i2 + 8i+ 3] = 2i19 w[4i2 + 8i+ 4] = 2i+ 120 return w
Algorithm Even [103] produces even de Bruijn sequences
1414 31 Perfect Arrays
314 One dimensional words with fixed length
315 Two-dimensional infinite arrays
Chung Diaconis and Graham posed the problem to give a necessary and sufficientcondition of the existence of (n 2ab)-perfect arrays
As Fan Fan and Siu proved in 1985 the conditions (2) and (3) are sufficientfor the existence of (22ab)-perfect arrays Paterson proved the same in 1994 for(n 2 a b)-perfect arrays leter Paterson supplied further sufficient conditions
Hurlbert and Isaak in 1993 gave a construction for one and two dimensionalgrowing arrays
3151 Pseudocode of the algorithm Mesh
The following implementation of Mesh is was proposed by Ivaacutenyi and Toacuteth in 1988
Mesh(Nw S)
1 for i = 1 to N2
2 for j = 1 to N2
3 if i+ j is even4 S[i j] = w[i]5 else S[i j] = w[j]6 return S
3152 Pseudocode of the algorithm Cellular
This is an extension and combination of the known algorithms Shift MartinEven and Mesh
Cellular results cellular perfect arrays Its input data are n d and a itsoutput is an (N dab)-perfect array where b1 = Na1 and bi = Na1a2aiminusa1a2aiminus1
for i = 2 3 d Cellular consists of five parts
1 Calculation (line 1 in the pseudocode) determining the new alphabet size Nusing formula (311)
2 Walking (lines 2ndash3) if d = 1 then construction of a perfect symmetric sequenceS1 using algorithm Martin (walking in a de Bruijn graph)
3 Meshing (lines 4ndash6) if d = 2 N is even and a = 2 then first construct anN -ary even perfect sequence e = 〈e1 e2 eN2〉 using Even then constructan N2 timesN2 sized N -ary square S1 using meshing function ()
4 Shifting (lines 7ndash12) if d gt 1 and (N is odd or a gt 2) then use Martin oncethen use Shift dminus 1 times receiving a perfect array P
5 Combination (lines 13ndash16) if d gt 2 N is even and a = 2 then construct aneven sequence with Even construct a perfect square by Mesh and finally useof Shift dminus 2 times results a perfect array P
316 Three-dimensional infinite cubes 1415
Cellular(n d aNA)
1 N = N(n a)2 if d = 13 Martin(N d aA)4 return A5 if d == 2 and a == 2 and N is even6 Mesh(N aA)7 return A8 if N is odd or a 6= 29 Martin(N a P1)10 for i = 1 to dminus 111 Shift(N i Pi Pi+1)12 A = P1
13 return A14 Mesh(N a P1)15 for i = 2 to dminus 116 Shift(N i Pi Pi+1)17 Alarr Pd
18 return Pd
316 Three-dimensional infinite cubes
3161 Pseudocode of the algorithm Colour
Colour transforms cellular perfect arrays into larger cellular perfect arrays Itsinput data are
bull d ge 1 ndash the number of dimensions
bull N ge 2 ndash the size of the alphabet
bull a ndash the window size
bull b ndash the size of the cellular perfect array A
bull A ndash a cellular (N dab)-perfect array
bull k ge 2 ndash the multiplication coefficient of the alphabet
bull 〈k1 k2 kd〉 ndash the extension vector having the property k|a| = k1timesk2timesmiddot middot middottimeskd
The output of Colour is
bull a (kN)-ary cellular perfect array P of size b = 〈k1a1 k2a2 kdad〉Colour consists of three steps
1 Blocking (line 1) arranging k|a| copies (blocks) of a cellular perfect array Ainto a rectangular array R of size k = k1timesk2timesmiddot middot middottimeskd and indexing the blockslexicographically (by 0 1 k|a| minus 1)
1416 31 Perfect Arrays
2 Indexing (line 2) the construction of a lexicographic indexing scheme I con-taining the elements 0 1 k|a|minus1 and having the same structure as the arrayR then construction of a colouring matrix C transforming the elements of Iinto k-ary numbers consisting of |a| digits
3 Colouring (lines 3-4) colouring R into a symmetric perfect array P using thecolouring array C that is adding the N -fold of the j-th element of C to each cellof the j-th block in R (considering the elements of the cell as lexicographicallyordered digits of a number)
The output P consists of blocks blocks consist of cells and cells consists of el-ements If e = P [j] is an element of P then the lexicographic index of the blockcontaining e is called the blockindex of e the lexicographic index of the cell con-taining e is called the cellindex and the lexicographic index of e in the cell is calledelementindex Eg the element S2[7 6] = 2 in Table 3 has blockindex 5 cellindex2 and elementindex 1
Input parameters are N d a k k a cellular (N d ab)-perfect array A theoutput is a (kN da c)-perfect array P where c = 〈a1k1 a2k2 adkd〉
Colour(N da kk A P )
1 arrange the copies of P into an array R of sizek1 times k2 times middot middot middot times kd blocks
2 construct a lexicographic indexing scheme I containing the elementsof [0kad minus 1] and having the same structure as R
3 construct an array C transforming the elements of I into k-arynumbers of v digits and multiplying them by N
4 produce the output S adding the j-th (j isin [0kad minus 1]) element of Cto each cell of the j-th block in R for each block of R
5 return S
3162 Pseudocode of the algorithm Growing
Finally algorithm Growing generates a prefix Sr of a growing array G Its inputdata are r the number of required doubly perfect prefixes of the growing array Gthen n d and a It consists of the following steps
1 Initialization construction of a cellular perfect array P using Cellular
2 Resizing if the result of the initialization is not doubly symmetric then con-struction of a symmetric perfect array S1 using Colour otherwise we take Pas S1
3 Iteration construction of the further r minus 1 prefixes of the growing array Grepeatedly using Colour
Input parameters of Growing are n d a and r the output is a doubly sym-metric perfect array Sr which is the rth prefix of an (n d a)-growing array
Growing(n d a r Sr)
317 Examples of constructing growing arrays using colouring 1417
1 Cellular(n d aN P )2 calculation of N using formula (311)3 if P is symmetric4 S1 = P5 if P is not symmetric6 n1 = Ndgcd(dad)
7 k = n1N
8 k1 = (n1)ad3Na
9 for i = 2 to d
10 ki = (n1)addNaiminusaiminus1
11 Colour(n1 d a kk P S1)12 k = Ndgcd(d ad)13 for i = 1 to d
14 ki = (n2)addNaiminusaiminus1
15 for i = 2 to r
16 ni = Ndigcd(dad)
17 Colour(ni da kk Siminus1 Si)18 return Sr
317 Examples of constructing growing arrays usingcolouring
In this section particular constructions are presented
3171 Construction of growing sequences
As the first example let n = 2 a = 2 and r = 3 Cellular calculates N = 2 andMartin produces the cellular (2124)-perfect sequence P = 00|11
Since P is symmetric S1 = P Now Growing chooses multiplication coefficientk = n2n1 = 2 extension vector k = 〈4〉 and uses Colour to construct a 4-aryperfect sequence
Colour arranges k1 = 4 copies into a 4 blocks sized arrray receiving
R = 00|11 || 00|11 || 00|11 || 00|11 (314)
Colouring receives the indexing scheme I = 0 1 2 3 and the colour-ing matrix C transforming the elements of I into a digit length k-ary numbersC = 00 || 01 || 10 || 11
Finally we colour the matrix R using C ndash that is multiply the elements of C byn1 and adding the j-th (j = 0 1 2 3) block of C1 = n1C to both cells of the j-thcopy in R
S2 = 00|11 || 02|13 || 20|31 || 22|33 (315)
Since r = 3 we use Colour again with k = n3n2 = 2 and get the (81264)-perfect sequence S3 repeating S2 4 times using the same indexing array I and
1418 31 Perfect Arrays
311 Table a) A (2244)-square b) Indexing scheme I of size 4 times 4
columnrow 1 2 3 4 columnrow 1 2 3 4
1 0 0 0 1 1 0 1 2 32 0 0 1 0 2 4 5 6 73 1 0 1 1 3 8 9 10 114 0 1 1 1 4 12 13 14 15
312 Table Binary colouring matrix C of size 8 times 8
columnrow 1 2 3 4 5 6 7 81 0 0 0 0 0 0 0 02 0 0 0 1 1 0 1 13 0 1 0 1 0 1 0 14 0 0 0 1 1 0 1 15 1 0 1 0 1 0 1 06 0 0 0 1 1 0 1 17 1 1 1 1 1 1 1 18 0 0 0 1 1 0 1 1
colouring array C prime = 2CAnother example is a = 2 n = 3 and r = 2 To guarantee the cellular prop-
erty now we need a new alphabet size N = 6 Martin produces a (61236)-perfectsequence S1 then Colour results a (1212144)-perfect sequence S2
3172 Construction of growing squares
Let n = a = 2 and r = 3 Then N(2 2) = 2 We construct the even sequenceW4 = e1e2e3e4 = 0 0 1 1 using Even and the symmetric perfect array A in Table311a using the meshing function () Since A is symmetric it can be used as S1Now the greatest common divisor of a and ad is 2 therefore indeed n1 = N22 = 2
Growing chooses k = n1N = 2 and Colour returns the array R repeatingthe array A k2 times k2 = 4times 4 times
Colour uses the indexing scheme I containing k4 indices in the same 4 times 4arrangement as it was used in R Table 311b shows I
Transformation of the elements of I into 4-digit k-ary form results the colouringmatrix C represented in Table 312
Colouring of array R using the colouring array 2C results the (42216)-squareS2 represented in Table 313
In the next iteration Colour constructs an 8-ary square repeating S2 4 times 4times using the same indexing scheme I and colouring by 4C The result is S3 a(8 2 2 64)-perfect square
317 Examples of constructing growing arrays using colouring 1419
313 Table A (42216)-square generated by colouring
columnrow 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 12 0 0 1 0 0 2 1 2 2 0 3 0 2 2 3 23 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 14 0 1 1 1 0 3 1 3 2 1 3 1 2 3 3 3
5 0 2 0 3 0 2 0 3 0 2 0 3 0 2 0 36 0 0 1 0 0 2 1 2 2 0 3 0 2 2 3 27 1 2 1 3 1 2 1 3 1 2 1 3 1 2 1 38 0 1 1 1 0 3 1 3 2 1 3 1 2 3 3 3
9 2 0 2 1 2 0 2 1 2 0 2 1 2 0 2 110 0 0 1 0 0 2 1 2 2 0 3 0 2 2 3 211 3 0 3 1 3 0 3 1 3 0 3 1 3 0 3 112 0 1 1 1 0 3 1 3 2 1 3 1 2 3 3 3
13 2 2 2 3 2 2 2 3 2 2 2 3 2 2 2 314 0 0 1 0 0 2 1 2 2 0 3 0 2 2 3 215 3 2 3 3 3 2 3 3 3 2 3 3 3 2 3 316 0 1 1 1 0 3 1 3 2 1 3 1 2 3 3 3
314 Table 8 layers of a (23216)-perfect array
Layer 0 Layer 1 Layer 2 Layer 3 Layer 4 Layer 5 Layer 6 Layer 70 0 0 1 0 0 0 1 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 10 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 11 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 10 1 1 1 0 1 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 1 1
3173 Construction of growing cubes
If d = 3 then the necessary condition (2) is b3 = (n)a3
for double cubes implying nis a cube number or a is a multiple of 3 Therefore either n ge 8 and then b ge 256or a ge 3 and so b ge 512 that is the smallest possible perfect double cube is the (83 2 256)-cube
As an example let n = 2 a = 2 and r = 2 Cellular computes N = 2 Meshconstructs the (2 2 2 4)-perfect square in Table 311a then Shift uses Martinwith N = 16 and a = 1 to get the shift sizes for the layers of the (2 3 2b)-perfectoutput P of Cellular where b = 〈4 4 16〉 Shift uses P as zeroth layer and thejth (j isin [1 15]) layer is generated by cyclic shifting of the previous layer downwardsby wi (div 4) and right by wi (mod 4) where w = 〈0 15 14 13 12 11 10 9 8 7 6 5 43 2 1〉 8 layers of P are shown in Table 314
Let A3 be a 4 times 4 times 16 sized perfect rectangular matrix whose 0 layer is thematrix represented in Table 311 and the (2 3 ab)-perfect array P in Table 314where a = (2 2 2) and b = (4 4 8)
Growing uses Colour to retrieve a doubly symmetric cube n1 = 8 thus
1420 31 Perfect Arrays
b = 256 k = n1N = 4 and k = 〈2564 2564 25664〉 that is we construct thematrix R repeating P 64times 64times 16 times
I has the size 64 times 64 times 16 and I[i1 i2 i3] = 642(i1 minus 1) + 64(i2 minus 1) + i3 minus 1Colour gets the colouring matrix C by transforming the elements of I into 8-digit4-ary numbers ndash and arrange the elements into 2times2times2 sized cubes in lexicographicorder ndash that is in order (000) (001) (010) (011) (100) (101) (110) (111)Finally colouring results a double cube S1
S1 contains 224 elements therefore it is presented only in electronic form (on thehomepage of the corresponding author)
If we repeat the colouring again with k = 2 then we get a 64-ary 65536times64536times64536 sized double cube S2
318 Proof of the main result
The main result of this paper can be formulated as follows
Theorem 311 If n ge 2 d ge 1 a ge 2 nj = Ndjgcd(dad) with N = N(n a) givenby (1) for j isin [0infin] then there exists an (n d a)-growing array
The proof is based on the following lemmas
Lemma 312 (Cellular lemma) If n ge 2 d ge 1 and a ge 2 then algorithm Cel-lular produces a cellular (N d ab)-perfect array A where N is determined by
formula (311) b1 = Na and bi = Naiminusaiminus1
(i isin [2d])
Proof It is known that algorithms Even+Mesh and Martin+Shift result perfectoutputs
Since Mesh is used only for even alphabet size and for 2times 2 sized window thesizes of the constructed array are even numbers and so the output array is cellular
In the case of Shift we exploit that all prime divisors of a divide the newalphabet size N and bi = N (aminus1)(aiminus1) and (aminus 1)(aiminus1) ge 1
Lemma 313 (Indexing lemma) If n ge 2 d ge 2 k ge 2 C is a d dimensionala-cellular array with |b| = k|a| cells and each cell of C contains the correspondingcellindex as an |a| digit k-ary number then any two elements of C having the sameelementindex and different cellindex are heads of different patterns
Proof Let P1 and P2 be two such patterns and let us suppose they are identical Letthe head of P1 in the cell have cellindex g and head of P2 in the cell have cellindexh (both cells are in array C) Let g minus h = u
We show that u = 0 (mod k|b|) For example in Table 2 let the head of P1 be(2 2) and the head of P2 be (2 6) Then these heads are in cells with cellindex 0 and2 so here u = 2
In both cells let us consider the position containing the values having local value1 of some number (in our example they are the elements (32) and (36) of C) Sincethese elements are identical then k|u Then let us consider the positions with local
319 Multi-dimensional infinite arrays 1421
values k (in our example they are (31) and (35)) Since these elements are alsoidentical so k2|u We continue this way up to the elements having local value k|b|
and get k|b||u implying u = 0This contradicts to the conditon that the patterns are in different cells
Lemma 314 (Colouring lemma) If k ge 2 ki isin [2infin] (i isin [1d]) A is a cellu-lar (n dab)-perfect array then algorithm Colour(N da kk A S) produces acellular (kN da c)-perfect array P where c = 〈k1a1 k2a2 kdad〉
Proof The input array A is N -ary therefore R is also N -ary The colouring arrayC contains the elements of [0N(k minus 1)] so elements of P are in [0kN minus 1]
The number of dimensions of S equals to the number of dimensions of P thatis d
Since A is cellular and ci is a multiple of bi (i isin [1d]) P is cellularAll that has to be shown is that the patterns in P are differentLetrsquos consider two elements of P as heads of two windows and their contents ndash
patterns p and q If these heads have different cellindex then the considered patternsare different due to the periodicity of R Eg in Table 313 P [11 9] has cellindex8 the pattern headed by P [9 11] has cellindex 2 therefore they are different (seeparity of the elements)
If two heads have identical cellindex but different blockindex then the indexinglemma can be applied
Proof of the main theorem Lemma 18 implies that the first call of Colour inline 10 of Growing results a doubly symmetric perfect output S1 In every iterationstep (in lines 14ndash16 of Growing) the nzeroth block of Si is the same as Siminus1 sincethe zeroth cell of the colouring array is filled up with zeros
Thus S1 is transformed into a doubly symmetric perfect output Sr having therequired prefixes S1 S2 Srminus1
319 Multi-dimensional infinite arrays
Chapter Notes
For Section 313For Section 314For Section 315For Section 316[6] [44] [46][47] [48][34] [52] [64] [81] [98][102] [103] [104] [105][106] [110]
1422 31 Perfect Arrays
[111] [112] [113] [120] [132][] [150] [156] [172][173] [174] [177] [178][] [238]For Section 319
32 Score Sets and Kings
The idea of comparison-based ranking has been discussed earlier in the chapterComparison based ranking where score sequence was introduced as a way of rankingvertices in a tournament Oriented graphs are generalizations of tournaments In factjust like one can think of a tournament as expressing the results of a round-robincompetition without ties (with vertices representing players and arrows pointing tothe defeated players) one can think of an oriented graph as a round-robin competi-tion with ties allowed (ties are represented by not drawing the corresponding arcs)Figure 321 shows the results of a round-robin competition involving 4 players a b c
Figure 321 A round-robin competition involving 4 players
and d with (a) ties not allowed and (b) ties allowed In the first instance there isalways a winner and a loser whenever two players square off while in the latter caseplayer a ties with player d and player b ties with player c
In 2009 Antal Ivaacutenyi studied directed graphs in which every pair of differentvertices is connected with at least a and at most b arcs He named them (a b n)-tournaments or simply (a b)-tournament
If a = b = k then the (a b)-tournaments are called k-tournaments In this chap-ter we deal first of all with 1-tournaments and (0 1)-tournaments (0 1)-tournamentsare in some sense equivalent with (2 2)-tournaments We use the simple notations1-tournament T 1
n 2-tournament T 2n k-tournament T k
n It is worth mention-ing that T 1
n is a classical tournament while oriented graphs are (0 1)-tournamentsIf we allow loops then every directed graph is some (a b n)-tournament (see theChapter (Comparison Based Ranking) of this book)
We discuss two concepts related with (a b)-tournaments namely score sets andkings A score set is just the set of different scores (out-degrees) of vertices while a
1424 32 Score Sets and Kings
king is a dominant vertex We shall study both concepts for 1-tournaments first andthen extend these to the more general setting of oriented graphs
Although we present algorithms for finding score sets and kings in 1-tournamentsand (0 1)-tournaments much of the focus is on constructing tournaments with spe-cial properties such as having a prescribed score set or a fixed number of kings Sinceplayers in a tournament are represented by vertices we shall use the words playerand vertex interchangeably throughout this chapter without affecting the meaning
We adopt the standard notation T (VA) to denote a tournament with vertex setV and arc set A We denote the number of vertices by n and the out-degree matrixbyM and the in-degree matrix by N Furthermore we use the term n-tournamentand the notation T k
n to represent a tournament with n vertices and exactly k arcsbetween the elements of any pair of different vertices In a similar way Rk
n and Nn
denote a regular resp a null graph When there is no ambiguity we omit one oreven both indices shall refer to the corresponding tournaments as T R and N
In Section 321 the score sets of 1-tournaments are discussed while Section 322deals with the sore sets of oriented graphs In Section 323 the conditions of theunique reconstruction of the score sets are considered at first for k-tournamentsthen in more details for 1-tournaments and 2-tournaments In Section 324 andSection 325 results connected with different kings of tournaments are presented
Some long and accessible proofs are omitted In these cases the Reader can findthe coordinates of the proof in Chapter notes and Bibliography
321 Score sets in 1-tournaments
In a round-robin competition with no ties allowed what are the sets of nonnegativeintegers that can arise as scores of players Note that here we are not interested inthe scores of individual players (the score sequence) rather we are looking for thesets of nonnegative integers with each integer being the score of at least one playerin the tournament This question motivates the study of score sets of tournaments
The set of different scores of vertices of a tournament is called the score setof the tournament In other words the score set is actually the score sequence of atournament with repetitions removed For example the tournament given in Figure322 has score sequence [0 2 2 2] whereas the score set of this tournament is 0 2Figure 323 shows the out-degree matrix of the tournament represented on Figure322
3211 Determining the score set
Determining the score set of a tournament T (VA) is quite easy The followingalgorithm Set1 takes the data of a tournament T (VA) as input and returns thescore set S of T
The procedures of this chapter are written according to the third edition of thetextbook Introduction to Algorithms published by T H Cormen C E LeisersonR L Rivest and C Stein in 2009
321 Score sets in 1-tournaments 1425
Figure 322 A tournament with score set 0 2
vertexvertex a b c d Scorea mdash 0 0 0 0b 1 mdash 1 0 2c 1 0 mdash 1 2d 1 1 0 mdash 2
Figure 323 Out-degree matrix of the tournament represented in Figure 322
Set1(n VA)
1 S = empty2 for all vertex u isin V3 s = 04 for all vertex v isin V5 if (u v) isin A is (u v) an arc of T6 s = s+ 17 if s isin S is the found score new8 S = S cup s9 return S
Since the scores of the vertices depend on n(n minus 1) out-degrees any algorithmdetermining the score set requires Ω(n2) time Due to the embedded loops in lines02ndash08 the running time of Set1 is Ω(n2) even in the best case The precise order ofthe running time depends among others on the implementation of the if instructionin line 07 Eg if line 07 is implemented by the comparison of the actual score withthe elements of S then the running time is Θ(n3) for a score sequence containingdifferent elements and is Θ(n2) for a regular tournament
Out-degree matrix Mntimesn = [mij ]ntimesn is a useful tool in the implementation ofgraph algorithms The input of the following algorithm Quick-Set1 is n and Mand the output is the score sequence s as a nonincreasingly ordered sequence and thescore set S as an increasingly ordered sequence Quick-Set1 calls the well-knownsorting procedure Insertion-Sort
1426 32 Score Sets and Kings
Quick-Set1(nM)
1 S = empty2 for i = 1 to n3 si = 04 for j = 1 to n5 si = si +mij score sequence is computed6 S1 = s1
7 Insertion-Sort(s) if s isin S sorting of the score vector8 for i = 2 to n9 if si 6= siminus1
10 Sk = si
11 k = k + 112 return s S
Since the embedded loops in lines 02ndash05 need Θ(n2) time and the remainingpart of the code requires less the running time of Quick-Set1 is Θ(n2) in all cases
3212 Tournaments with prescribed score set
Constructing a tournament with a prescribed score set is more difficult than deter-mining the score set Quite surprisingly if sufficiently many players participate in atournament then any finite set of nonnegative integers can arise as a score set Thiswas conjectured by K B Reid in 1978 and turned out to be a relatively challengingproblem
Reid proved the result when | S | = 1 2 or 3 or if S contains consecutive termsof an arithmetic or geometric progression That is Reid showed that any set of onetwo or three nonnegative integers is a score set of some tournament and additionallyany set of the form s s+d s+2d s+pd for s gt 0 d gt 1 or s sd sd2 sdpfor s ge 0 d gt 0 is a score set of some tournament Hager settled the cases |S| = 4and |S| = 5 in 1986 and finally in 1987 T Yao gave an existence proof of the generalReidrsquos conjecture based on arithmetic analysis
Theorem 321 (Yao 1988) Every finite nonempty set S of nonnegative integers isthe score set of some tournament
Let us try to formulate Reidrsquos conjecture purely as a statement about num-bers Let S = s1 sp be an increasing sequence of nonnegative integers Theconjecture means that there exist positive integers x1 xp such that
S = (sx1
1 sx2
2 sxpp )
is the score sequence of some 1-tournament withsump
i=1 xi = n vertices By Landaursquostheorem a = (a1 an) with a1 le middot middot middot le an is the score sequence of some 1-
tournament Tn if and only ifsumk
i=1 ai ge(
k2
)
for k = 1 nminus1 andsumn
i=1 ai =(
n2
)
Thus it can be readily seen that Reidrsquos conjecture is equivalent to the followingstatement
For every nonempty set of nonnegative integers S = s1 sp where s1 lt
321 Score sets in 1-tournaments 1427
Figure 324 Construction of tournament T with odd number of distinct scores
middot middot middot lt sp there exist positive integers x1 xp such that
ksum
i=1
sixi ge(
sumki=1 xi
2
)
for k = 1 pminus 1 (321)
psum
i=1
sixi =(sump
i=1 xi
2
)
(322)
It is this equivalent formulation of Reidrsquos conjecture that led to Yaorsquos proof Theproof is not combinatorial in nature but uses first of all some results of numbertheory Commenting on Yaorsquos proof Qiao Li wrote in 2006 in the Annals of NewYork Academy of Sciences
Yaorsquos proof is the first proof of the conjecture but I do not think itis the last one I hope a shorter and simpler new proof will be coming inthe near future
However the prophecized constructive proof has not been discovered yet Thisis in sharp contrast with Landaursquos theorem on score sequences for which severalproofs have emerged over the years Recently S Pirzada and T A Naikoo gavea constructive combinatorial proof of a new special case of Reidrsquos theorem Theirproof gives an algorithm for constructing a tournament with the prescribed scoreset provided the score increments are increasing
Theorem 322 (Pirzada and Naikoo 2008) If a1 a2 ap are nonnegative in-tegers with a1 lt a2 lt middot middot middot lt ap then there exists a 1-tournament T with scoreset
S =
s1 = a1 s2 =2sum
i=1
ai sp =psum
i=1
ai
(323)
Since any set of nonnegative integers can be written in the form of 323 theabove theorem is applicable to all sets of nonnegative integers S = s1 s2 sp
1428 32 Score Sets and Kings
Figure 325 Construction of tournament T with even number of distinct scores
with increasing increments (ie s1 lt s2 minus s1 lt s3 minus s2 lt middot middot middot lt sp minus spminus1) Theimportance of Pirzada-Naikoo proof of Theorem 322 is augmented by the fact thatYaorsquos original proof is not constructive and is not accessible to a broad audience1
The following recursive algorithm is based on Pirzada and Naikoorsquos proof ofTheorem 322 The algorithm takes the set of increments Ip = a1 lt a2 lt middot middot middot lt apof the score set S as input and returns a tournament T whose score set is S LetXt = a1 lt a2 lt middot middot middot lt at for 1 le t le p Let Rn denote the regular tournament onn vertices and let T (1)oplusT (2) denote the vertex and arc disjoint union of tournamentsT (1) and T (2)
Score-Reconstruction1(p Ip)
1 if p is odd2 print Odd(p Ip)3 else print Even(p Ip)
This algorithm calls one of the two following recursive procedures ODD andEven according to the parity of p The input of both algorithm is some prefix Xt ofthe sequence of the increments a1 a2 at and the output is a tournament havingthe score set corresponding to the given increments
1Yaorsquos proof originally appeared in Chinese in the journal Kexue Tongbao Later in 1989 the proofwas published in English in the Chinese Science Bulletin Unfortunately neither are accessiblethrough the world wide web although the English version is available to subscribers of the ChineseScience Bulletin In Hungary this journal is accessible in the Library of Technical and EconomicalUniversity of Budapest
321 Score sets in 1-tournaments 1429
Odd(tXt)
1 if t == 12 return R2a1+1
3 else T(3)t = R(2(atminusatminus1+atminus2minusatminus3+middotmiddotmiddot+a3minusa2+a1)+1)
4 T(2)t = R2(atminus1minusatminus2+atminus3minusatminus2+middotmiddotmiddot+a4minusa3+a2minusa1minus1)+1
5 t = tminus 26 T
(1)t = Odd(tXt)
7 Tt = T(3)t oplus T (2)
t oplus T (1)t
8 Tt = T+ arcs such that9 T
(2)t dominates T (1)
t
10 T(3)t dominates T (1)
t
11 T(3)t dominates T (2)
t
12 return Tt
We can remark that the tournament constructed by the first execution of line 03of Odd contains the vertices whose score is ap while the tournament constructedin line 04 contains the vertices whose score is apminus1 in the tournament appearing asoutput The vertices having smaller scores appear during the later execution of lines03 and 04 with exception of the vertices having score a1 since those vertices will beadded to the output in line 02
Even(tXt)
1 T(2)t = R2(atminusatminus1+atminus2minusatminus3+middotmiddotmiddot+a4minusa3+a2minusa1minus1)+1
2 t = tminus 13 T
(1)t = Odd(tXt)
4 Tt = T(2)t oplus T (1)
t
5 Tt = T+ arcs such that T (2)t dominates T (1)
t
6 return Tt
Since the algorithm is complicated letrsquos consider an example
Example 321 Let p = 5 and I5 = 0 1 2 3 4 Since p is odd Score-Reconstruction1calls Odd in line 02 with parameters 5 and I5
The first step of Odd is the construction of T(3)5 = T2(4minus3+2minus1+0)+1 = T5 in line 03
Denoting the vertices of this regular 5-tournament by v1 v2 v3 v4 v5 and using theresult of Exercise 321-1 we get the out-degree matrix shown in Figure 326
The second step of Odd is the construction of T(2)5 = T2(3minus2+1minus0minus1)+1 = T3 Let v6 v7
and v8 be the vertices of this tournamentThe third step of Odd is the recursive call with parameters p = 3 and X3 = 2 1 0
The fourth action of Odd is the construction of T(3)3 = T2(2minus1+0)+1 = T3 Let v9 v10
and v11 be the vertices of this tournament The fifth step is the construction of T(2)3 =
T2(2minus1+0minus1)+1 = T1 Let v12 be the only vertex of this graph The sixth action is the callof Odd with parameters t = 1 and X1 = 0 Now the number of increments equals to 1
therefore the algorithm constructs T(1)1 = T1 in line 02
The seventh step is the construction of T in line 07 then the eighth step is adding new
1430 32 Score Sets and Kings
vertexvertex v1 v2 v3 v4 v5 Scorev1 mdash 1 1 0 0 2v2 0 mdash 1 1 0 2v3 0 0 mdash 1 1 2v4 1 0 0 mdash 1 2v5 1 1 0 mdash 0 2
Figure 326 Out-degree matrix of the tournament T(3)5
vertexvertex v9 v10 v11 v12 v13 Scorev9 mdash 1 0 1 1 3v10 0 mdash 1 1 1 3v11 1 0 mdash 1 1 3v12 0 0 0 mdash 1 1v13 0 0 0 0 mdash 0
Figure 327 Out-degree matrix of the tournament T(3)5
vv v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 Scorev1 mdash 0 0 0 0 0 0 0 0 0 0 0 0 0v2 1 mdash 0 0 0 0 0 0 0 0 0 0 0 1v3 1 1 mdash 1 0 0 0 0 0 0 0 0 0 3v4 1 1 0 mdash 1 0 0 0 0 0 0 0 0 3v5 1 1 1 0 mdash 0 0 0 0 0 0 0 0 3v6 1 1 1 1 1 mdash 1 0 0 0 0 0 0 6v7 1 1 1 1 1 0 mdash 1 0 0 0 0 0 6v8 1 1 1 1 1 1 0 mdash 0 0 0 0 0 6v9 1 1 1 1 1 1 1 1 mdash 1 0 1 1 10v10 1 1 1 1 1 1 1 1 0 mdash 1 1 0 10v11 1 1 1 1 1 1 1 1 0 0 mdash 1 1 10v12 1 1 1 1 1 1 1 1 1 0 0 mdash 1 10v13 1 1 1 1 1 1 1 1 1 1 0 0 mdash 10
Figure 328 Out-degree matrix of the tournament T5
arcs (according to lines 08ndash11) to the actual T constructed in line 07 and consisting from3 regular tournaments having altogether 5 vertices (v13 v12 v11 v10 v9) The result isshown in Figure 327
Ninth step of Odd is joining the tournaments T5 and T3 to T and the final step isadding of the domination arcs The out-degree matrix of the output T5 of Odd is shownin Figure 328
321 Score sets in 1-tournaments 1431
Correctness of the algorithm Let I = a1 a2 ap be a set of p nonnegativeintegers with a1 lt a2 lt middot middot middot lt ap Score-Reconstruction1 performs two typesof recursions first if p is odd and the second if p is even Assume p to be odd Forp = 1 the set I contains one nonnegative integer a1 and the algorithm returns theregular tournament T2a1+1 as output Note that each vertex of T2a1+1 has score(
2a1+1minus12
)
= a1 so that score set of T2a1+1 is S = s1 = a1 This shows that thealgorithm is correct for p = 1
If p = 3 then the set of increments I consists of three nonnegative integersa1 a2 a3 with a1 lt a2 lt a3 Now a3 gt a2 therefore a3 minus a2 gt 0 so thata3minusa2 +a1 gt 0 as a1 ge 0 Let T (3) be a regular tournament having 2(a3minusa2 +a1)+1
vertices Then each vertex of T (3) has score(
2(a3minusa2+a1)+1minus12
)
= a3 minus a2 + a1
Again since a2 gt a1 therefore a2 minus a1 gt 0 so that a2 minus a1 minus 1 ge 0 Let T (2) bea regular tournament having 2(a2 minus a1 minus 1) + 1 vertices Then each vertex of T (2)
has score(
2(a2minusa1minus1)+1minus12
)
= a2 minus a1 minus 1 Also since a1 ge 0 let T (1) be a regular
tournament having 2a1+1 vertices Then each vertex of T1 has score(
2a1+1minus12
)
= a1If p = 3 Score-Reconstruction1 outputs a tournament T whose vertex set
is the disjoint union of vertex sets of T (1) T (2) and T (3) and whose arc set containsall the arcs of T (1) T (2) and T (3) such that every vertex of T (2) dominates eachvertex of T (1) and every vertex of T (3) dominates each vertex of T (1) and T (2)Thus T has 2a1 +1+2(a2minusa1minus1)+1+2(a3minusa2 +a1)+1 = 2(a1 +a3)+1 verticeswith score set
S = a1 a2 minus a1 minus 1 + 2a1 + 1 a3 minus a2 + a1 + 2(a2 minus a1 minus 1) + 1 + 2a1 + 1
=
a12sum
i=1
ai3sum
i=1
ai
This shows that the algorithm is correct for p = 3 too When the set I of incre-ments consists of an odd number of nonnegative integers the algorithm recursivelybuilds the required tournament by using the procedure Odd To see this assumethat the algorithm works for all odd numbers upto p That is if a1 a2 ap are pnonnegative integers with a1 lt a2 lt middot middot middot lt ap then the algorithm outputs a tourna-ment having 2(a1+a3+ +ap)+1 vertices with score set a1
sum2i=1 ai
sumpi=1 ai
Let us call this tournament T (1)We now show how the algorithm constructs a tournament with p+2 vertices with
score set a1sum2
i=1 ai sump+2
i=1 ai where a1 a2 ap+2 are p + 2 nonnegativeintegers with a1 lt a2 lt middot middot middot lt ap+2
Since a2 gt a1 a4 gt a3 apminus1 gt apminus2 ap+1 gt ap therefore a2 minus a1 gt 0a4 minus a3 gt 0 apminus1 minus apminus2 gt 0 ap+1 minus ap gt 0 so that ap+1 minus ap + apminus1 minus apminus2 + +a4minusa3+a2minusa1 gt 0 that is ap+1minusap+apminus1minusapminus2+ +a4minusa3+a2minusa1minus1 ge 0
The procedure Odd constructs T (2) as a regular tournament having 2(ap+1 minusap + apminus1 minus apminus2 + middot middot middot+ a4 minus a3 + a2 minus a1 minus 1) + 1 vertices Each vertex of T (2) hasscore
2(ap+1 minus ap + apminus1 minus apminus2 + + a4 minus a3 + a2 minus a1 minus 1) + 1minus 12
= ap+1 minus ap + apminus1 minus apminus2 + middot middot middot+ a4 minus a3 + a2 minus a1 minus 1
1432 32 Score Sets and Kings
Again a3 gt a2 ap gt apminus1 ap+2 gt ap+1 therefore a3 minus a2 gt 0 ap minusapminus1 gt 0 ap+2 minus ap+1 gt 0 so that ap+2 minus ap+1 + ap minus apminus1 + middot middot middot+ a3 minus a2 + a1 gt 0as a1 ge 0
The procedure Odd constructs T (3) as a regular tournament having 2(ap+2 minusap+1 + ap minus apminus1 + middot middot middot+ a3 minus a2 + a1) + 1 vertices Each vertex of T (3) has score
2(ap+2 minus ap+1 + ap minus apminus1 + middot middot middot+ a3 minus a2 + a1) + 1minus 12
= ap+2 minus ap+1 + ap minus apminus1 + middot middot middot+ a3 minus a2 + a1
Now Score-Reconstruction1 sets T = T (1)oplusT (2)oplusT (3) and adds additionalarcs in such a way that every vertex of T (2) dominates each vertex of T (1) and everyvertex of T (3) dominates each vertex of T (1) and T (2) Therefore T is a tournamenthaving
2(a1 + a3 + middot middot middot+ ap) + 1 + 2(ap+1ap + ap1ap2 + middot middot middot+ a4a3 + a2a1) + 1
+2(ap+2ap+1 + apapminus1 + middot middot middot+ a3a2 + a1) + 1
= 2(a1 + a3 + middot middot middot+ ap+2) + 1
vertices with score set
S =
a1
2sum
i=1
ai
psum
i=1
ai
p+1sum
i=1
ai
p+2sum
i=1
ai
Hence by induction the algorithm is correct for all odd pTo prove the correctness for even case note that if p is odd then p+1 is even Let
a1 a2 ap+1 be p+ 1 nonnegative integers with a1 lt a2 lt middot middot middot lt ap+1 Thereforea1 lt a2 lt middot middot middot lt ap where p is odd The procedure Even uses the procedure Oddto generate a tournament T (1) having 2(a1 + a3 + middot middot middot + ap) + 1 vertices with scoreset S = a1
sum2i=1 ai
sumpi=1 ai
Also since a2 gt a1 a4 gt a3 apminus1 gt apminus2 ap+1 gt ap the procedure Evengenerates a regular tournament T (2) having 2(ap+1 minus ap + apminus1 minus apminus2 + middot middot middot+ a4 minusa3 +a2minusa1minus1)+1 vertices such that the score for each vertex is ap+1minusap +apminus1minusapminus2 + middot middot middot+ a4 minus a3 + a2 minus a1 minus 1
Finally the algorithm generates the tournament T (1)oplusT (2) and adds additionalarcs so that every vertex of T (2) dominates each vertex of T (1) The resulting tour-nament T consists of
2(a1 + a3 + middot middot middot+ apminus2 + ap) + 1
+2(ap+1 minus ap + apminus1 minus apminus2 + middot middot middot+ a4 minus a3 + a2 minus a1 minus 1) + 1
= 2(a2 + a4 + middot middot middot+ ap+1)
vertices and has score set
322 Score sets in oriented graphs 1433
S = a1
2sum
i=1
ai
psum
i=1
ai
ap+1 minus ap + apminus1 minus apminus2 + middot middot middot+ a4 minus a3 + a2 minus a1 minus 1
+2(a1 + a3 + middot middot middot+ apminus2 + ap) + 1
= a1
2sum
i=1
ai
p+1sum
i=1
ai
This shows that the algorithm is correct for even p as well
Computational complexity The running time of Score-Reconstruction1depends on the size of the score set |S| as well as the largest increment ap = spminusspminus1The details are left as a problem for the Reader (see Exercise 321-1)
Exercises321-1 The out-degree matrixM of a tournament is defined as a 0minus 1 matrix with(i j) entry equal to 1 if player vi defeats player vj and 0 otherwise (see (3213))A tournament is completely determined by its out-degree matrix Write an O(n2)algorithm to generate the out-degree matrix of a regular tournament on n verticeswhere n is any odd positive integer Hint Circularly place
(
nminus12
)
ones in each row
321-2 Use Exercise 321-1 and the discussion in this section to determine the worst-case running time of Score-Reconstruction1321-3 Obtain the out-degree matrix of a tournament with score set 1 3 6How many vertices does this tournament have Draw this tournament and giveits outdegree-matrix321-4 Use the tournament obtained in Exercise 321-3 to generate the out-degreematrix of a 1-tournament with score set 1 3 6 10 Write the score sequence ofyour tournament
322 Score sets in oriented graphs
Oriented graphs are generalizations of tournaments Formally an oriented graphD(VA) with vertex set V and arc set A is a digraph with no symmetric pairs ofdirected arcs and without loops In other words oriented graph is a directed graph inwhich every pair of different vertices is connected with at most one arc or orientedgraphs are (0 1)-tournaments
Figure 329 shows an oriented graph with score sequence [1 3 3 5] and thecoressponding score set 1 3 5
Thus tournaments are complete oriented graphs in the sense that any pair ofvertices in a tournament is joined exactly by one arc Several concepts defined fortournaments can be extended in a meaningful way to oriented graphs For examplescore of a player (vertex) in a tournament is defined as its out-degree as a player
1434 32 Score Sets and Kings
Figure 329 An oriented graph with score sequence [1 3 3 5] and score set 1 3 5
either wins (and earns one point) or looses (earning no points) a two-way clash In1991 Peter Avery introduced the score structure for oriented graphs based on theintuition that in a round-robin competition with ties allowed a player may earn twoone or no points in case the player wins looses or makes a tie respectively
More precisely the score of a vertex vi in a k-tournament D with n vertices isdefined as
a(vi) = ai = nminus 1 + d+viminus dminus
vi
where d+vi
and dminusvi
are the out-degree and in-degree respectively of vi The scoresequence of an oriented graph is formed by listing the vertex scores in non-decreasingorder If we denote the number of non-arcs in D containing the vertex vi as dlowast
vi then
ai = 2d+vi
+ dlowastvi
With this score structure an oriented graph can be interpreted as the result of around-robin competition in which ties (draws) are allowed that is the players playeach other once with an arc from player u to v if and only if u defeats v A playerreceives two points for each win and one point for each tie
It is worth to remark that this is a sophisticated score structure comparing withthe simple and natural structure of 2-tournaments
Avery gave a complete characterization of score sequences of oriented graphssimilar to Landaursquos theorem
Theorem 323 (Avery 1991) A nondecreasing sequence A = [a1 an] of non-negative integers is the score sequence of an oriented graph if and only if
ksum
i=1
ai ge k(k minus 1) (324)
for 1 le k le n with equality when k = n
Proof This theorem is a special case of the theorem proved by Moon in 1963 orthe theorem proved by Kemnitz and Dulff in 1997 (see the theorem and its proof inChapter 27 that is chapter Comparison Based Ranking)
322 Score sets in oriented graphs 1435
Just as in the case of 1-tournaments the score set of an oriented graph isdefined as the set of scores of its vertices It is worth noting that a (0 1)-tournamenthas different score sets under Averyrsquos and Landaursquos score structures In fact the scoreof a vertex v under Averyrsquos score structure is twice the score of v under Landaursquosscore structure This is obviously due to Averyrsquos assumption that a win contributes2 points to the score
The score set of an oriented graph can be determined by adapting Quick-Set2as follows
Quick-Set2(nM)
1 S = empty2 for i = 1 to n3 si = 04 for j = 1 to n5 si = si + 2mij
6 if mij==0 and mji == 07 si = si + 1 score sequence is computed8 S1 = s1
9 k = 210 for i = 2 to n11 if si 6= siminus1 is the found score new12 Sk = si
13 k = k + 114 return s S
The running time of Quick-Set2 is Θ(n2) since the nested loop in lines 02ndash07requires Θ(n2) the remaining lines require Θ(n) time
3221 Oriented graphs with prescribed scoresets
In Section we discussed score sets of tournaments and noted that every non-emptyset of nonnegative integers is the score set of some tournament In this section westudy the corresponding question for oriented graphs ie which sets of nonnegativeintegers can arise as score sets of oriented graphs Pirzada and Naikoo investigatedthis question and gave two sufficient conditions for a set of nonnegative integers tobe the score set of some oriented graph
Theorem 324 (Pirzada Naikoo 2008) Let a d n nonnegative integers and S =a ad ad2 adn with d gt 2 or d = 2 and n gt 1 Then there exists an orientedgraph with score set A except for a = 1 d = 2 n gt 0 and for a = 1 d = 3 n gt 0
Theorem 325 (Pirzada Naikoo 2008) If n is a positive integer and a1 a2 an
are nonnegative integers with a1 lt a2 lt middot middot middot lt an then there exists an oriented graphwith an + 1 vertices and with score set S = aprime
1 aprime2 a
primen where
aprimei =
aiminus1 + ai + 1 for i gt 1 ai for i = 1
(325)
1436 32 Score Sets and Kings
Thus any set of positive integers whose elements form a geometric progression isthe score set of some oriented graph with few exceptions and any set of nonnegativeintegers whose elements are of the form (325) is also a score set It follows that everysingleton set of nonnegative integers is the score set of some oriented graph On theother hand for any positive integer n the sets 1 2 22 2n and 1 3 32 3ncannot be the score sets of an oriented graph Therefore unlike in the case of tour-naments not all sets of nonnegative integers are score sets of oriented graphs So farno complete characterization of score sets of oriented graphs is known
The proof of Theorem 324 depends on the following auxiliary assertion
Lemma 326 Naikoo Pirzada 2008) The number of vertices in an oriented graphwith at least two distinct scores does not exceed its largest score
Proof This assertion is the special case k = 2 of Lemma due to Ivaacutenyi andPhong
Here we omit formal proofs of Theorems 324 and 325 since they can be foundon the internet and since we will implicitly prove these theorems when we check thecorrectness of Geometric-Construction and Adding-Construction respec-tively
We first present a recursive algorithm that takes positive integers a d andn satisfying the condition of Theorem as input and generates a 2-tournamentD(VA) with score set a ad ad2 adn Let Np denote the null digraph on pvertices ie the digraph with n vertices and no arcs
Geometric-Construction(a d n)
1 if a = 0 or n = 02 D = Na+1
3 return D4 else5 D(1) = Geometric-Construction(a d nminus 1)6 U = vertex set of D(1)
7 D = D(1) oplusNadnminus2|U |+1
8 Add arcs to D such that9 Nadnminus2|U |+1 dominates D(1)
10 return D
322 Score sets in oriented graphs 1437
Geometric-Construction(a d n)
1 if n = 02 D = Na+1
3 return D4 if n = 14 if n ge 25 n = nminus 16 D(1) = Geometric(adn)6 U = vertex set D(1)7 D = D(1)oplusN
Example 322 Let a = 2 d = 2 and n = 2 Then the prescribed score set is 2 4 8 Thefirst step is the call of Geometric with parameters (2 2 2)
Algorithm description If n = 0 then the algorithm returns the null digraphNa+1 Note that Na+1 is well-defined as a + 1 gt 0 Each vertex of Na+1 has scorea+1minus1+0minus0 = a Therefore the score set of Na+1 is S = a Thus the algorithmis correct for n = 0
Now we prove the correctness of Geometric by induction That is we showthat if the algorithm is valid for n = 0 1 p for some integer p ge 1 then it is alsovalid for n = p + 1 Let a and d be positive integers with a gt 0 and d gt 1 suchthat for a = 1 d 6= 2 3 By the induction hypothesis the algorithm can construct anoriented graph D(1) with score set a ad adp and a ad adp are the distinctscores of the vertices of D(1) Let U be the vertex set of D(1)
There are three possibilities
bull a = 1 and d gt 3
bull a gt 1 and d = 2 or
bull a gt 1 and d gt 2
Obviously for d gt 1 in all the above cases we have adp+1 ge 2adp Also the score setof D(1) namely a ad adp has at least two distinct scores for p ge 1 Thereforeby Lemma 326 we have |U | le adp Hence adp+1 ge 2|U | so that adp+1minus2|U |+1 gt 0
Let Nadp+1minus2|U |+1 be the null digraph with vertex set X The algorithm nowgenerates the vertex and arc disjoint union D = D(1)oplusNadp+1minus2|U |+1 and adds an arcdirected from each vertex in Nadp+1minus2|U vert+1 to every vertex of D(1) The outputD(VA) of Geometric-Seq-Construction therefore has |V | = |U | + adp+1 minus2|U |+1 = adp+1minus|U |+1 vertices Moreover a+ |X|minus|X| = a ad+ |X|minus|X| = adad2 +|X|minus|X| = ad2 adp +|X|minus|X| = adp are the distinct scores of the verticesin U while ax = |U |minus1 + |V |minus0 = adp+1minus|V |+ 1minus1 + |V | = adp+1 for all verticesx isin X
Therefore the score set of D is S = a ad ad2 adp adp+1 which shows thatthe algorithm works for n = p + 1 Hence the algorithm is valid for all a d and nsatisfying the hypothesis of Theorem
1438 32 Score Sets and Kings
The recursive procedure Geometric runs n times and during its ith run theprocedure adds O(adn+1minusi) arcs to the oriented graph D The overall complexity ofthe algorithm is therefore O(nadn)
As noted in Theorem there exists no 1-tournament when either a = 1 d =2 n gt 0 or a = 1 d = 3 n gt 0 It is quite interesting to investigate theseexceptional cases as it provides more insight into the problem
Let us assume that S = 1 2 22 2n is a score set of some oriented graphD for n gt 0 Then there exist positive integers say x1 x2 x3 xn+1 such that
S1 = [1x1 2x2 (22)x3 (2n)xn+1
is the score sequence of D Therefore by relations (324) of score sequences of 1-tournaments we have
x1 + 2x2 + 22x3 + middot middot middot+ 2nxn+1 =
(
n+1sum
i=1
xi
)(
n+1sum
i=1
xi minus 1
)
which implies that x1 is even However x1 is a positive integer therefore x1 ge 2Let the scores be a1 = 1 a2 = 1 and a3 ge 1 By inequalities (324) a1 + a2 + a3 ge3(3minus 1) = 6 or in other words a3 ge 4 This implies that x2 = 0 a contradiction
The proof of the other exceptional case (S = 1 3 32 3n) is left as anexercise (Exercise 322-1)
The next algorithm takes the set I = a1 lt a2 lt middot middot middot lt an consisting of nnonnegative integers as input and recursively constructs a 2-tournament D(VA)the score set S = aprime
1 aprime2 a
primen where aprime
i are of the form 325
Adding-Construction(n In)
1 if n = 02 D = Na1+1
3 return D4 n = nminus 15 D(1) = Adding-Construction(n In)6 D = D1 oplusNan+1minusan
7 Add arcs to D such that8 Nn dominates D(1)9 return D
322 Score sets in oriented graphs 1439
Adding-Construction(n In)
1 if n = 02 D = Na1+1
3 else4 D(1) = Adding-Construction(nminus 1 In minus an)5 D = D(1) oplusNanminusanminus1
7 Add arcs to D such that8 Nanminusanminus1
dominates D(1)
9 return D
Algorithm description If n = 1 the algorithm returns the null digraph Na1+1Each vertex of Na1+1 has the score a1 + 1minus 1 + 0minus 0 = a1 = aprime
1 Therefore the scoreset of Na1+1 is S = aprime
1 as requiredWe prove the correctness of General-Construction in general by induction
on n Assume that the algorithm is valid for n = 1 2 p for some integer p ge 2We show that the algorithm is also valid for n = p + 1 Let a1 a2 ap+1 benonnegative integers with a1 lt a2 lt middot middot middot lt ap+1 Since a1 lt a2 lt middot middot middot lt ap by theinduction hypothesis the algorithm returns an oriented graph D(1) on ap +1 verticeswith score set aprime
1 aprime2 a
primep where aprime
i is given by equations (325) That is scoreset of D(1) is a1 a1 + a2 + 1 a2 + a3 + 1 apminus1 + ap + 1 So a1 a1 + a2 + 1a2 +a3 +1 apminus1 +ap +1 are the distinct scores of the vertices of D Let X be thevertex set of D(1) so that |X| = ap +1 Since ap+1 gt ap ap+1minusap gt 0 the algorithmconstructs a new oriented graph D = D(1)oplusNp+1 with vertex set V = X cupY whereY is the vertex set of Np+1 and |Y | = ap+1 minus ap Arcs are added to D such thatthere is an arc directed from each vertex in Y to every vertex in X Thus D has|V | = |X|+ |Y | = ap +1+ap+1minusap = ap+1 +1 vertices The distinct score of verticesin X are a1 + |Y |minus|Y | = a1 = aprime
1 a1 +a2 +1+ |Y |minus|Y | = a1 +a2 +1 = aprime2 a2 +a3 +
1 + |Y | minus |Y | = a2 + a3 + 1 = aprime3 apminus1 + ap+ 1 + |Y | minus |Y | = apminus1 + ap + 1 = aprime
pwhile ay = |X| minus 1 + |V | minus 0 = ap+1 + 1minus 1 + ap + 1 = ap + ap+1 + 1 = aprime
p+1 for ally isin Y
Therefore the score set of D is S = aprime1 a
prime2 a
primep a
primep+1 which proves the va-
lidity of algorithm for n = p+ 1 Hence by induction General-Construction isvalid for all n
The analysis of computational complexity of General-Construction is leftas an exercise (Exercise 322-2)
Exercises322-1 Prove that there exists no oriented graph with score set 1 3 32 3n forany n gt 0322-2 Adding-Construction is a recursive algorithm Analyse its running timeand compare its performance with the performance of Geometric-Construction
322-3 Implement Adding-Construction in a suitable programming language
1440 32 Score Sets and Kings
and use it to construct an oriented graph with score set 2 4 8 Write the scoresequence of your oriented graph322-4 Implement Adding-Construction in a suitable programming languageand use it to construct an oriented graph with score set 1 4 6 9 Write the scoresequence of your oriented graph322-5 Give a proof of Lemma 326322-6 For any nonnegative integer n what is the score set of the regular tourna-ment T2n+1 when considered as an oriented graph322-7 Determine the score set of the oriented graph D = T3 oplus T5 where T5 domi-nates T3 ie there is an arc directed from every vertex of T5 to every vertex of T3
322-8 Write an O(n) algorithm to determine the score set of directed cycles (iecycles with directed edges) How can we make this algorithm work for directed wheels(note that a wheel is a cycle with an additional vertex joined to all the vertices onthe cycle)
323 Unicity of score sets
k-tournaments (multitournaments) are directed graphs in which each pair of ver-tices is connected with exactly k arcs
Reid formulated the following conjecture in [202]
Conjecture 327 Any set of nonnegative integers is the score set of some 1-tournament T
Using Landaursquos theorem this conjecture can be formulated in the following arith-metic form too
Conjecture 328 If 0 le r1 lt r2 lt middot middot middot lt rm then there exist such positive integersx1 x2 xm that
jsum
i=1
xiri ge(sumj
i=1 xi)(sumj
i=1 xi minus 1)2
j isin [1 m]
andmsum
i=1
xiri =(summ
i=1 xi)(summ
i=1 xi minus 1)2
In this case we say that the sequence s = 〈s1 sn〉 = 〈rx1
1 rxmm 〉 realizes
the sequence r = 〈r1 rm〉 or s is a solution for rReid gave a constructive proof of his conjecture for sets containing one two or
three elements [202]Later Hager published a constructive proof for sets with four and five elements
[86] and Yao [243] published the outline of a nonconstructive proof of the generalcase
A score set is called k-unique if there exists exactly 1 score sequence of k-tournaments generating the given set In the talk we investigate the following ques-tions
323 Unicity of score sets 1441
1 characterization of the unique score sets of 1-tournaments
2 extension of the Reidrsquos conjecture to 2-tournaments
3231 1-unique score sets
At first we formulate a useful necessary condition
Lemma 329 (Ivaacutenyi and Phong 2004) If k ge 1 then for any (n k)-tournamentholds that the sequence s is a solution for r then in the case m = 1 we have
n = 2r1 + 1 (326)
and in the case m ge 2 we have
2r1
k+ 1 lt n lt
2rm
k+ 1 (327)
andn ge rm + 1 (328)
Proof If
This lemma implies the exact answer for the case m = 1
Corollary 3210 (Ivaacutenyi and Phong 2004) If r = 〈r1〉 then exactly the sequences = 〈r2r1+1
1 〉 is a solution for r
Proof Lemma implies that only this solution is acceptable One can check thatit satisfies the required inequality and equality
Now we present a useful method of the investigation of the uniqueness Letr = 〈a a+ d〉 Then according to the Reid-equality we get
2ax+ 2(a+ d)y = n(nminus 1)
implying
y =n(nminus 2aminus 1)
2d (329)
But here only the values n = 2a+ 1 + i (i isin [1 2dminus 1]) are permitted where
i ge d+ 1minus a (3210)
By substitution a = (q minus 1)d from (329) we get
y =(2qdminus 2d+ 2r + 1 + i)i
2d (3211)
Here y must be an integer so transform this formula into
y = i(q minus d) +i(2r + 1 + i)
2d (3212)
1442 32 Score Sets and Kings
Theorem 3211 If 0 le a lt b then there exist positive integers x and y satisfying
ax ge x(xminus 1)2
and
ax+ by =(x+ y)(x+ y minus 1)
2
In the following cases there is only one solution
bull a = 0
bull d = 1
bull d = 2
In the following case there are at least two solutions
bull d is odd and 3 le d le a
Proof a) Existence of a solution Let d = bminusa and i = 2dminus2rminus1 Then n = 2(bminusr)y = q(2dminus 2r minus 1) x = q(2r + 1) satisfy all requirements
b) Uniqueness If a = 0 then d = b q = 1 and y is integer only if i = 2bminus 1 Sowe get the unique 〈01 b2bminus1〉 solution
If d = 1 then only i = 1 is permitted implying the unique solution 〈ab bb〉If d = 2 or d is odd then we also can analyse formula (3212)
This theorem left open the case when the difference d is odd and the investigatedset is sparse and also the case when the difference is an even number greater then 2
3232 2-unique score sets
Now we present a new form of Reid-problem for 2-tournamentsFor a fixed sequence q[m] = 〈q1 qm〉 with q1 lt middot middot middot lt qm of positive integers
we shall denote by G(q[m]) the set G of sequences g = 〈g1 gm〉 such that
ksum
i=1
qigi ge(
ksum
i=1
gi
)2
k isin [1 mminus 1]
andmsum
i=1
qigi =
(
msum
i=1
gi
)2
Here we also say that g is a solution for qWe wish to give necessary and sufficient conditions for q[m] to have a solution
that is a nonempty G(q[m]))
Theorem 3212 For the sequence q[1] = 〈q1〉 we have G(q[1]) = 〈q1〉Proof If q[1] = 〈q1〉 then it is obvious that the solution of q1g1 = g2
1 is given in theform g1 = q1 Hence we have G(q[1]) = 〈q1〉
323 Unicity of score sets 1443
Theorem 3213 Let q[2] = 〈q1 q2〉 be a sequence of positive integers with d =q2 minus q1 gt 0 Then G(q[2]) 6= empty if and only if either d 6 |(q1 q2) or d|(q1 q2) and thereis a prime p such that p2|d
Proof According to the definition of G(q[m]) we need only find positive integersg1 g2 such that q1 ge g1 and q1g1 + q2g2 = (g1 + g2)2
Let q r be integers for which q2 = qd+ r where 0 le r lt d If d 6 |(q1 q2) thenr 6= 0 and let g1 = rq and g2 = q2 minus r(q + 1) Hence we have
g1 = rq = rq2 minus rq2 minus q1
= r + rq1 minus rq2 minus q1
lt r + (R1 minus r) = R1
g2 = R2 minus r(q + 1) =
q2 minus (q2 minus r)r
q2 minus q1minus r
gt q2 minus (q2 minus r)minus r = 0
andq1g1 + q2g2 = q1rq + q2
2 minus q2r(q + 1)
= q22 + r(q1q minus q2q + q2)minus 2q2r =
= (q2 minus r)2 = (g1 + g2)2
Now assume that d|(q1 q2) and there is a prime p such that p2|d In this case r = 0and we choose g1 g2 as follows
g1 =q2
pminus d
p2and g2 = g1(pminus 1)
It is obvious thatg1 gt 0 g2 gt 0 g1 le R1
andq1g1 + q2g2 = g1(q1 + (pminus 1)q2)
= g1(pq2 minus d) =
= g1p2(q2
pminus d
p2) = (g1p)2 = (g1 + g2)2
Finally assume that d = 1 or d|(q1 q2) and d is the product of distinct primesIf there are positive integers g1 g2 such that q1 ge g1 and q1g1 + R2g2 = (g1 + g2)2then we have d|g1 + g2 and
1d
(g1 + g2)2 minus q1
d(g1 + g2) = g2 gt 0
1d
(g1 + g2)2 minus R2
d(g1 + g2) = minusg1 lt 0
1444 32 Score Sets and Kings
consequentlyq2
d=q1
d+ 1 gt
g1 + g2
dgtq1
d
This is impossible
Theorem 3214 Ivaacutenyi Phong 2004 Let q[2] =lt q1 q2 gt be the sequence ofpositive integers with conditions q1 lt R2 (q1 q2) = 1 2q1 gt q2 and d = q2minusR1 hass distinct prime factors Then
|G(q[2])| = 2s minus 1
Proof Since d = q2 minus q1 lt q1 and (q1 q2) = 1 the congruence x2 equiv q2x (mod d)has 2s minus 1 solutions in positive integers less than d For each solution x we set
g1 = x(q2minusx)d and g2 = (d minus x) q2minusx
d One can check that g1 g2 satisfy conditionsq1 ge g1 and q1g1 + q2g2 = (g1 + g2)2
Exercises323-1 How many 323-2 Design an algorithm
324 Kings and serfs in tournaments
Sociologists are often interested in determining the most dominant actors in a socialnetwork Moreover dominance in animal societies is an important theme in ecologyand population biology Social networks are generally modelled as digraphs withvertices representing actors and arcs representing dominance relations among ac-tors The concept of ldquokingrdquo is very closely related to dominance in digraphs Kingsand serfs were initially introduced to study dominance in round-robin competitionsThese concepts were latter extended to more general families of digraphs such asmultipartite tournaments quasi-transitive digraphs semicomplete multipartite di-graphs and oriented graphs In this section our focus will be on algorithmic aspects ofkings and serfs in tournaments and their applications in majority preference voting
A king in a tournament dominates every other vertex either directly or throughanother vertex To make the idea more formal we define a path of length k froma vertex u to a vertex v in a tournament (or any digraph) as a sequence of arcse1 e2 ek where u is the initial vertex of e1 v is the terminal vertex of ek and theterminal vertex of ei is the same as the initial vertex of ei+1 for all 1 le i le k minus 1If there is a path of length 1 or 2 from a vertex u to a vertex v then v is said to bereachable from u within two steps Analogously if there is a path of length 1 2 or r from u to v then v is said to be reachable from u within r steps Let T be ann-tournament A vertex u in T is called an r-king where 1 le r le n minus 1 if everyother vertex v in the tournament is reachable within r steps from u A vertex u iscalled an r-serf if u is reachable within r if u is reachable within r steps from every
324 Kings and serfs in tournaments 1445
Figure 3210 A tournament with three kings u v y and three serfs u v x Note that z isneither a king nor a serf and uv are both kings and serfs
other vertex v in T In particular a 2-king is simply called a king and a 2-serf iscalled a serf
S B Maurer introduced the dual terms of king and serf in a delightful expo-sition of a tournament model for dominance in flocks of chicken In his influentialseries of papers on dominance in animal societies H G Landau proved that everytournament has a king (although he did not use the word king) In fact he showedthe following
Theorem 3215 (Landau 1953) Every vertex of maximum score in a tournamentis a king
The proof is quite intuitive Suppose to the contrary that u is a vertex with maximumscore in a tournament T and u is not a king Then there exists another vertex v in Tsuch that v is not reachable from u within 2 steps But this means that u and all out-neighbours of u are reachable from v in 1 step and so s(v) gt s(u) a contradictionAnother classical result by J W Moon states that
Theorem 3216 (Moon 1968) A tournament without transmitters (vertices within-degree 0) contains at least three kings
It is natural to ask if the bound on the number of kings given in Theorem 3216is tight The answer is yes as demonstrated by the following example
Example 323 Let T be a tournament with vertex set v1 v2 v5 Let us denote by(u v) an arc directed from u to v Suppose that the arc set of T consists of the arcs(v3 v5) (v4 v3) all arcs of the form (vjminus1 vj) with 1 lt j le 5 and all arcs of the form(vj+2 vj) (vj+3 vj) (vn vj) with j = 1 2 4 Then it can be easily verified (Exercise324-2) that T has no transmitters and v2 v3 and v4 are the only kings in T
K B Reid proved the existence of a tournament with an arbitrary number ofvertices and an arbitrary number of kings with few exceptions
Theorem 3217 (Reid 1982) For all integers n ge k ge 1 there exists a tournamenton n vertices with exactly k kings except when k = 2 or when n = k = 4 (in whichcase no such n-tournament exists)
1446 32 Score Sets and Kings
Hence no tournament has exactly two kings The above theorems can be statedjust as well in terms of serfs To see this note that the converse T prime of a tournamentT obtained by reversing the arcs of T is also a tournament and that the kings andserfs of T and T prime are interchanged
The king set of a tournament consists of all kings in the tournament We candefine the serf set analogously The problem of determining the king set of a tour-nament is very important both for theoretical and practical considerations In votingtheory literature political scientists often refer to the uncovered set in majority pref-erence voting This uncovered set is actually the king set for the tournament whosevertices consist of the candidates to be elected and arcs represent the outcomes ofthe two-way race between candidates Here we present a simple polynomial timealgorithm for determining the king set of a tournament Given an n-tournament Tlet us define an ntimes n matrix D+
T as
(D+T )ij =
1 if (vi vj) is an arc of T 0 otherwise
(3213)
We call D+T the out-degree matrix of T When there is no danger of ambiguity
we will drop the subscript T and simply denote the out-degree matrix by D+ King-Set takes a tournament T (VA) as input calculates the out-degree matrix D+ of Tand uses it to generate the king set K of T Let O be the ntimes n zero matrix and letI be the ntimes n identity matrix
King-Set(VA)
1 D+ =2 K = empty3 for i = 1 to n4 for j = 1 to n5 if (vi vj) isin A6 (D+)ij = 17 M = I +D+ + (D+)2
8 K = vi isin V |forallvj isin V (M)ij 6= 09 Nn dominates D(1)9 return K
Algorithm description The algorithm works on the same principle as thealgorithm for finding the number of paths from one vertex to another in a digraph(Exercise 324-1 asks you to derive this algorithm) The (i j) entry of the matrix(D+)2 is equal to the number of paths of length two from vertex vi to vertex vj
(check this) Therefore the (i j) entry of matrix D+ + (D+)2 counts the numberof paths of length one or two from vi to vj and if vertex vi is a king all entries inthe ith row of I +D+ + (D+)2 must be non-zero
The computational complexity of Algorithm King-Set depends on the way(D+
T )2 is computed If naive matrix multiplication is used the algorithm runs inΘ(n3) time However using the fast matrix multiplication by Coppersmith andWinograd the running time can be reduced to O(n238) The Reader should note
324 Kings and serfs in tournaments 1447
that by using the duality of kings and serfs King-Set can be adapted for findingthe serf set of a tournament
King sets in majority preference voting Kings frequently arise in politicalscience literature A majority preference voting procedure asks each voter torank candidates in order of preference The results can be modeled by a tournamentwhere vertices represent the candidates and arcs point toward the loser of each twoway race where candidate u defeats candidate v if some majority of voters preferu to v Political scientists are often interested in determining uncovered vertices inthe resulting tournament A vertex u is said to cover another vertex v if u defeats vand also defeats every vertex that v defeats
The covering relation is clearly transitive and has maximal elements called un-covered vertices An uncovered vertex u has the strategically important propertythat u defeats any other vertex v in no more than two steps ie either
1 u defeats v or
2 there is some third alternative w such that u defeats w and w defeats v
Thus an uncovered vertex is actually a king In fact the uncovered set consisting ofall uncovered vertices is precisely the set of all kings (see Exercise 324-8)
The idea behind finding kings in a tournament can be easily extended to findingr-kings for any positive integer r
rKing-Set(VA r)
1 D+ = 02 K = empty3 for i = 1 to n4 for j = 1 to n5 if (vi vj) isin A6 (D+)ij = 17 M = I +D+ + + (D+)r
8 K = vi isin V |forallvj isin V (M)ij 6= 09 return K
The above algorithm runs in O(rn3) if the matrix multiplications are performednaively and in O(rn238) time if fast matrix multiplication is incorporated
As we have seen kings dominate in tournaments However there exists a strongernotion of dominance in tournaments in the form of strong kings Let us write urarr vto denote that u defeats v in a tournament T or in other words (u v) is an arc of T IfU1 and U2 are disjoint subsets of vertices of T then we write U1 rarr U2 to denote thatall vertices in U1 defeat all vertices in U2 We define BT (u v) = w isin V minus u v urarr w and w rarr v where V denotes the vertex set of T Let bT (u v) = |BT (u v)|When no ambiguity arises we drop the subscript T from the notation
A vertex u in a tournament T is said to be a strong king if urarr v or b(u v) gtb(v u) for every other vertex v of T