arxiv:1803.10803v2 [math.oc] 28 jan 2019march 28, 2018; revised on jan 28, 2019 abstract in this...
TRANSCRIPT
On the Equivalence of Inexact Proximal ALM and ADMM for aClass of Convex Composite Programming
Liang Chen∗ Xudong Li† Defeng Sun‡ and Kim-Chuan Toh§
March 28, 2018; Revised on Jan 28, 2019
Abstract
In this paper, we show that for a class of linearly constrained convex composite optimization problems,an (inexact) symmetric Gauss-Seidel based majorized multi-block proximal alternating direction methodof multipliers (ADMM) is equivalent to an inexact proximal augmented Lagrangian method (ALM). Thisequivalence not only provides new perspectives for understanding some ADMM-type algorithms but alsosupplies meaningful guidelines on implementing them to achieve better computational efficiency. Even forthe two-block case, a by-product of this equivalence is the convergence of the whole sequence generated bythe classic ADMM with a step-length that exceeds the conventional upper bound of (1+
√5)/2, if one part
of the objective is linear. This is exactly the problem setting in which the very first convergence analysisof ADMM was conducted by Gabay and Mercier in 1976, but, even under notably stronger assumptions,only the convergence of the primal sequence was known. A collection of illustrative examples are providedto demonstrate the breadth of applications for which our results can be used. Numerical experiments onsolving a large number of linear and convex quadratic semidefinite programming problems are conductedto illustrate how the theoretical results established here can lead to improvements on the correspondingpractical implementations.
Keywords. Alternating direction method of multipliers; Augmented Lagrangian method; Symmetric Gauss-Seidel decomposition; Proximal term
AMS Subject Classification. 90C25; 65K05; 90C06; 49M27; 90C20
1 Introduction
Let X, Y and Z be three finite-dimensional real Hilbert spaces each endowed with an inner product denotedby 〈·, ·〉 and its induced norm denoted by ‖ · ‖, where Y := Y1 × · · · ×Ys is the Cartesian product of s finite-dimensional real Hilbert spaces Yi, i = 1, . . . , s, each endowed with the inner product, as well as the inducednorm, inherited from Y. For any given y ∈ Y, we can write y = (y1; . . . ; ys) with yi ∈ Yi,∀ i = 1, . . . , s. Here,
∗College of Mathematics and Econometrics, Hunan University, Changsha, 410082, China ([email protected]), and Departmentof Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong ([email protected]).The research of this author was supported by the National Natural Science Foundation of China (11801158, 11871205) and theFundamental Research Funds for the Central Universities in China.†School of Data Science, Fudan University, Shanghai 200433, China, and Shanghai Center for Mathematical Sciences, Fudan
University, Shanghai 200433, China ([email protected]). The research of this author was supported by the FundamentalResearch Funds for the Central Universities in China.‡Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong (defeng.
[email protected]). The research of this author was supported in part by a start-up research grant from the Hong KongPolytechnic University.§Department of Mathematics, and Institute of Operations Research and Analytics, National University of Singapore, 10
Lower Kent Ridge Road, Singapore 119076 ([email protected]). The research of this author was supported in part by theMinistry of Education, Singapore, Academic Research Fund (R-146-000-257-112).
1
arX
iv:1
803.
1080
3v2
[m
ath.
OC
] 2
8 Ja
n 20
19
and throughout this paper, we use the notation (y1; . . . ; ys) to mean that the vectors y1, . . . , ys are writtensymbolically in a column format.
In this paper, we shall focus on the following multi-block convex composite optimization problem
miny∈Y,z∈Z
p(y1) + f(y)− 〈b, z〉 | F∗y + G∗z = c , (1.1)
where p : Y1 → (−∞,+∞] is a (possibly nonsmooth) closed proper convex function, f : Y → (−∞,+∞) isa continuously differentiable convex function whose gradient is Lipschitz continuous, b ∈ Z and c ∈ X arethe given data, and F∗ and G∗ are the adjoints of the given linear mappings F : X → Y and G : X → Z,respectively. Despite the simple appearance of problem (1.1), we shall see in the next section that thismodel actually encompasses various important classes of convex optimization problems in both classical coreconvex programming as well as recently emerged models from a broad range of real-world applications. Aquintessential example of problem (1.1) is the dual of the following convex composite quadratic programming
minx
ψ(x) +
1
2〈x,Qx〉 − 〈c, x〉 | Gx = b
, (1.2)
where ψ : X→ (−∞,+∞] is a closed proper convex function, Q : X→ X is a self-adjoint positive semidefinitelinear operator, G : X→ Z is a linear mapping, and c ∈ X, b ∈ Range(G) (i.e., b is in the range space of thelinear operator G) are the given data. The dual of problem (1.2) in the minimization form can be written asfollows:
miny1,y2,z
ψ∗(y1) +
1
2〈y2,Qy2〉 − 〈b, z〉
∣∣ y1 +Qy2 − G∗z = c
, (1.3)
where ψ∗ is the Fenchel conjugate of ψ, y1 ∈ X, y2 ∈ X and z ∈ Z, so that problem (1.3) constitutes aninstance of problem (1.1).
To solve problem (1.1), one of the most preferred approaches is the augmented Lagrangian method(ALM) initiated by Hestenes [23] and Powell [42], and elegantly studied for general (without taking intoaccount of the multi-block structure) convex optimization problems in the seminal work of Rockafellar [45].Given a penalty parameter σ > 0, the augmented Lagrangian function corresponding to problem (1.1) isdefined by
Lσ(y, z;x) := p(y1) + f(y)− 〈b, z〉+ 〈x,F∗y + G∗z − c〉+σ
2‖F∗y + G∗z − c‖2,
∀ (x, y, z) ∈ X×Y ×Z.
Starting from a given initial multiplier x0 ∈ X, the ALM performs the following steps at the k-th iteration:
(1) compute (yk+1, zk+1) to (approximately) minimize the function Lσ(y, z;xk), and
(2) update the multipliers xk+1 := xk + τσ(F∗yk+1 + G∗zk+1 − c), where τ ∈ (0, 2) is the step-length.
While one would really want to solve miny,z Lσ(y, z;xk) as it is without modifying the augmented Lagrangianfunction, it can be expensive to minimize Lσ(y, z;xk) with respect to both y and z simultaneously, dueto the coupled quadratic term in y and z. Thus, in practice, unless the ALM is converging rapidly, onewould generally want to replace the augmented Lagrangian subproblem with an easier-to-solve surrogate bymodifying the augmented Lagrangian function to decouple the minimization with respect to y and z. Such amodification is especially desirable during the initial phase of the ALM when its local superlinear convergencehas yet to kick in. The most obvious approach to decouple the subproblem for obtaining (yk+1, zk+1) is toadd to Lσ(y, z;xk) the proximal term σ
2 ‖(y; z)− (yk; zk)‖2Λ, where Λ = λ2I − (F ;G)(F ;G)∗ with λ being thelargest singular value of (F ;G) and I being the identity operator in Y ×Z. However, such a modification tothe augmented Lagrangian function is generally too drastic and has the undesirable effect of significantlyslowing down the convergence of the ALM [6, Section 7]. This naturally leads us to the important questionon what is an appropriate proximal term to add to Lσ(y, z;xk) such that the ALM subproblem is easier to
2
solve while at the same time it is less drastic than the obvious choice we have just mentioned in the previoussentences.
We shall show in this paper that by adding an appropriately designed proximal term to Lσ(y, z;xk), wecan reduce the computation of the modified ALM subproblem to sequentially updating y and z via computing
yk+1 ≈ miny
Lσ(y, zk;xk)
and zk+1 ≈ min
z
Lσ(yk+1, z;xk)
.
The reader would have observed that the resulting proximal ALM updating scheme is the same as the classictwo-block ADMM (pioneered by Glowinski and Marroco [21] and Gabay and Mercier [18]) that is appliedto problem (1.1). However, there is a crucial difference in that our convergence result holds true for thestep-length τ in the range (0, 2), whereas the classic two-block ADMM only allows the step-length to be inthe interval
(0, (1 +
√5)/2
)if the convergence of the full sequence generated by the algorithm is required. It
is important to note that even with the sequential minimization of y and z in the modified ALM subproblem,the minimization subproblem with respect to y can still be very difficult to solve due to the coupling of theblocks y1, . . . , ys in (1.1). One of the main contributions we made in this paper is to show that by majorizingthe function f(y) at yk with a quadratic function and by adding an extra proximal term that is derived basedon the block symmetric Gauss-Seidel (sGS) decomposition theorem [32] for the quadratic term associatedwith y, we are able to update the sub-blocks in y individually in a symmetric Gauss-Seidel fashion. A crucialimplication of this result is that the (inexact) block sGS decomposition based multi-block majorized ADMMis equivalent to an inexact majorized proximal ALM. Consequently, we are able to prove the convergence ofthe whole sequence generated by the former even when the step-length is in the range (0, 2).
In this paper, we shall not delve into the vast literature on both ALM and ADMM, as well as theirvariants, and their relationships to the proximal point method and operator splitting methods. Theyare simply too abundant for us to list even a few of them here. Thus, we shall only refer to those thatare most relevant for our work in this paper. Here we should mention that many attempts have beenmade in recent years on designing convergent multi-block ADMM-type algorithms that can outperformthe directly extended multi-block (proximal ADMM) numerically. While the latter is not guaranteed toconverge even under the strong assumption that f ≡ 0, paradoxically its practical numerical performanceis often better than many convergent variants that have been developed in the past; see for example [48].Against this backdrop, we should mention that the ADMM-type algorithms that have been progressivelydesigned in [48, 30, 6] not only come with convergence guarantee but they have also been demonstratedto have superior numerical performance than the directly extended ADMM, at least for a large number ofconvex conic programming problems. More recently, those algorithms have found applications in variousareas [1, 2, 9, 17, 27, 31, 53, 55, 56]. Among those algorithms, the most general and versatile one is therecently developed inexact majorized multi-block proximal ADMM in Chen et al. [6], which we shall brieflydescribe in the next paragraph.
Under the assumption that the gradient of f is Lipschitz continuous, we know that one can specify afixed self-adjoint positive semidefinite linear operator Σf : Y → Y and define at each y′ ∈ Y the followingconvex quadratic function
f(y, y′) := f(y′) + 〈∇f(y′), y − y′〉+1
2‖y − y′‖2
Σf, ∀ y ∈ Y, (1.4)
such thatf(y) ≤ f(y, y′), ∀y, y′ ∈ Y and f(y′) = f(y′, y′), ∀y′ ∈ Y.
Thus, we say that at each y′ ∈ Y, the function f(·, y′) constitutes a majorization of the function f . Let σ > 0be the penalty parameter. Based on the notion of majorization described above, the majorized augmentedLagrangian function of problem (1.1) is defined by
Lσ(y, z; (x, y′)
):= p(y1) + f(y, y′)− 〈b, z〉+ 〈F∗y + G∗z − c, x〉
+σ
2‖F∗y + G∗z − c‖2, ∀ (y, z, x, y′) ∈ Y ×Z×X×Y.
(1.5)
3
Let (x0, y0, z0) ∈ X×Y ×Z be a given initial point with y01 ∈ dom p, and Di : Yi → Yi, i = 1, . . . , s be the
given self-adjoint linear operators, for the purpose of facilitating the computations of the subproblems. Forconvenience, we denote for any y = (y1; . . . ; ys) ∈ Y1 × · · · ×Ys,
y<i := (y1; . . . ; yi−1) and y>i := (yi+1; . . . ; ys), ∀ i = 1, . . . , s.
Then, the k-th step of the (inexact) block sGS decomposition based majorized multi-block proximal ADMMin [6], when applied to problem (1.1), takes the following form
yk+ 1
2i ≈ arg min
yi∈Yi
Lσ((yk<i; yi; y
k+ 12
>i
), zk; (xk, yk)
)+ 1
2‖yi − yki ‖2Di
, i = s, . . . , 2 ;
yk+1i ≈ arg min
yi∈Yi
Lσ((yk+1<i ; yi; y
k+ 12
>i
), zk; (xk, yk)
)+ 1
2‖yi − yki ‖2Di
, i = 1, . . . , s ;
zk+1 ≈ arg minz∈Z
Lσ(yk+1, z; (xk, yk)
);
xk+1 = xk + τσ(F∗yk+1 + G∗zk+1 − c
),
(1.6)
where τ ∈(0, (1 +
√5)/2
)was allowed in [6]. As one can observe from (1.5) and (1.6), the quadratic
majorization technique in Li et al. [29] was used to replace the original augmented Lagrangian functionby the majorized augmented Lagrangian function. This in turn enables us to employ the inexact blocksGS decomposition technique in Li et al. [32] to sequentially update the sub-blocks of y individually. Moreimportantly, the algorithm is highly flexible in that all the subproblems are allowed to be solved approximatelyto overcome possible numerical obstacles such as, for example, when iterative solvers must be employed tosolve large-scale linear systems to overcome extreme memory requirement and prohibitive computing cost. Ithas already been demonstrated in [6] that the inexact block sGS decomposition based multi-block ADMMis far superior to the directly extended ADMM in solving high-dimensional linear and convex quadraticsemidefinite programming with the step-length in (1.6) being restricted to be less than (1 +
√5)/2.
Our focus in this paper is to investigate whether the framework in (1.6) can be proven to be convergentfor problem (1.1) when the step-length τ is in the range (0, 2). In particular, we will show that the inexactblock sGS decomposition based multi-block ADMM (1.6) is equivalent to an inexact majorized proximalALM in the sense that computations of yk+1, zk+1 and xk+1 in (1.6) can equivalently be written as
(yk+1, zk+1
)≈ arg min
(y, z)∈Y×Z
Lσ(y, z; (xk, yk)
)+ 1
2‖(y; z)−(yk; zk
)‖2T
;
xk+1 = xk + τσ(F∗yk+1 + G∗zk+1 − c
),
where T : Y ×Z→ Y ×Z is a self-adjoint (not necessarily positive definite) linear operator whose precisedefinition will be given later, and ‖(y; z)‖2T := 〈(y; z), T (y; z)〉, ∀ (y, z) ∈ Y × Z. This connection not onlyprovides new theoretical perspectives for analyzing multi-block ADMM-type algorithms, but also has thepotential of allowing them to achieve even better computational efficiency since a larger step-length beyond(1 +
√5)/2 can now be taken in (1.6), without adding any extra conditions or any additional verification
steps such as those extensively used in [48, 30, 6, 5].The main contributions of this paper are as follows.
• We derive the equivalence of an (inexact) block sGS decomposition based multi-block majorized proximalADMM to an inexact majorized proximal ALM, and establish the global and local convergence propertiesof the latter with the step-length τ ∈ (0, 2). As a result, the global and local convergence properties ofthe former even with τ ∈ (0, 2) are also established.
• Even for the most conventional two-block case, we are able for the first time to rigorously characterizethe connection between ADMM and proximal ALM. Note that given the form of the updating rulesof the classic ADMM and ALM, although it is natural to view ADMM as an approximate versionof the ALM, this is not completely true as can be seen from our analysis in this paper. Indeed, to
4
alleviate the difficulty of solving the subproblems in the ALM, the classic ADMM uses a single cycleof the Gauss-Seidel block minimization to replace the full minimization of the augmented Lagrangianfunction in the ALM. This viewpoint in fact motivated the study of the classic ADMM in the veryfirst paper [21]. However, as was mentioned in [13, 14], there were no known results in quantifying thisinterpretation.
• As a by-product of the second contribution, this paper gives an affirmative answer to the open questionon whether the dual sequence generated by the classic ADMM with τ ∈ (0, 2) is convergent if one ofthe two functions in the objective is linear1. This is the problem setting of the very first proof forthe ADMM in Gabay and Mercier [18, Theorem 3.1] in which the dual sequence is only guaranteedto be bounded, even under very strong assumptions. The later proof of Glowinski [20, Chapter 5,Theorem 5.1] established stronger results than [18] but it requires τ ∈
(0, (1 +
√5)/2
). Thereafter,
only the latter interval, and especially the unit step-length, has been considered. In fact, in a rigorousproof presented recently in [5] for the classic two-block ADMM with τ ∈
(0,(1 +√
5)/2), it was shown
that the convergence of the dual sequence can be guaranteed under pretty weak conditions but theconvergence of the primal sequence requires more. Hence, it is of much theoretical interest to clarifywhether the dual sequence is convergent if the objective contains a linear part while τ ≥
(1 +√
5)/2.
• We provide a fairly general criterion for choosing the possibly indefinite2 linear operators Di, i = 1, . . . , s,in the proximal terms, which unifies those used in Chen et al. [6] and those used in Zhang et al. [57] toguarantee the viability of the block sGS decomposition techniques and the convergence of the wholesequence generated by the algorithm in (1.6). Recall that the proximal terms in [6] should be positivesemidefinite while in [57] the functions being majorized should be separable with respect to each blockof variables. Here, we do not require f to be separable and indefinite proximal terms are allowed.
• We use a unified criterion, which is weaker than those used in [6], for choosing the proximal terms inthe algorithmic framework (1.6) and analyzing its convergence. Note that in [6], compared with thecondition [6, (3.2)] imposed on choosing the proximal terms, a stronger condition ([6, (5.26) of Theorem5.1]) was used to guarantee the convergence of the algorithm. Here, we are able to get rid of such a gapwhile using a weaker condition.
• We conduct extensive numerical experiments on solving the linear and convex quadratic semidefiniteprogramming (SDP) problems to demonstrate how the theoretical results obtained here can be exploitedto improve the numerical efficiency of the implementation on ADMM. Based on the numerical results,together with the theoretical analysis in this paper, we are able to give a plausible explanation as towhy ADMM often performs well when the dual step-length is chosen to be the golden ratio of 1.618.Meanwhile, a guiding principle on choosing the step-length during the practical implementation of thealgorithmic framework in (1.6) is derived.
Here we emphasize again that for solving large-scale instances of the multi-block problem (1.1), asuccessful multi-block ADMM-type algorithm must not only possess convergence guarantee but should alsonumerically perform at least as fast as the directly extended ADMM. Based on our work in this paper, wecan conclude that the inexact block sGS decomposition based majorized proximal ADMM studied in [6]indeed does possess those desirable properties. Moreover, this algorithm is a versatile framework and one canapply it to problem (1.1) in different routines other than (1.6). The reason that we are more interested in theiteration scheme (1.6) is not only for the theoretical improvement one can achieve, but also for the practicalmerit it features for solving large scale problems, especially when the dominating computational cost is inperforming the evaluations associated with the linear mappings G and G∗. A particular case in point is thefollowing problem:
minx∈X
ψ(x) +
1
2〈x,Qx〉 − 〈c, x〉 | GEx = bE , GIx ≥ bI
, (1.7)
1This question was first resolved in [48] when the initial multiplier x0 satisfies Gx0 − b = 0 and all the subproblems are solvedexactly.
2One may refer to [29] for the details that motivating the use of indefinite proximal terms in the 2-block majorized proximalADMM, especially [29, Section 6] on their computational merits, as well as [57] for the similar results in multi-block cases.
5
where Q, ψ, and c have the same meaning as in (1.3), GE : X→ ZE and GI : X→ ZI are the given linearmappings, and b = (bE ; bI) ∈ Z := ZE ×ZI is a given vector. By introducing a slack variable x′ ∈ ZI , theabove problem can be equivalently reformulated as
minx∈X,x′∈ZI
ψ(x) +
1
2〈x,Qx〉 − 〈c, x〉 |
(GE 0GI IZI
)(xx′
)= b, x′ ≤ 0
,
where IZI is the identity operator in ZI . The corresponding dual problem in the minimization form is thengiven by
miny1,y′2,z
p(y1) +
1
2〈y2,Qy2〉 − 〈b, z〉 |
(y11
y12
)+
(Q0
)y2 −
(G∗E G∗I0 IZ2
)z =
(c0
),
where y1 := (y11; y12) ∈ X × ZI , p(y1) := ψ∗1(y11) + δ+(y12) with δ+ being the indicator function of thenonnegative orthant in ZI , y2 ∈ X and z ∈ Z. It is clear that when problem (1.7) has a large number ofinequality constraints, the dimension of Z can be much larger than that of X. For such a scenario, theiteration scheme (1.6) is more preferable since the more difficult subproblem involving z is solved only oncein each iteration.
Organization
This paper is organized as follows. In Section 2, we present a few important classes of problems that can behandled by (1.1) to illustrate the wide applicability of this model. In Section 3, we design an inexact majorizedproximal ALM framework and establish its global and local convergence properties. In Section 4, we showthe key result that the sequence generated by the inexact block sGS decomposition based majorized proximalADMM (1.6), together with a simple error tolerance criterion, is equivalent to the sequence generated bythe inexact ALM framework introduced in Section 3. Accordingly, the convergence of the two-block ADMMwith the step-length in the interval of (0, 2) is also established for problem (1.1) with s = 1. In Section 5, weconduct extensive numerical experiments on the 2-block dual linear SDP problems and the multi-block dualconvex quadratic SDP problems to illustrate the numerical efficiency of the proposed algorithm, as well asthe impact of the step-length on its numerical performance. A few important practical observations from thenumerical results are also presented. Finally, we conclude this paper in the last section.
Notation
• Let H and H′ be two finite-dimensional real Hilbert spaces each endowed with an inner product 〈·, ·〉and its induced norm ‖ · ‖. We also use ‖ · ‖ to denote the norm induced on the product space H×H′by the inner product 〈(ν1, ν
′1), (ν2, ν
′2)〉 := 〈ν1, ν2〉+ 〈ν′1, ν′2〉,∀ν1, ν2 ∈ H,∀ν′1, ν′2 ∈ H′.
• For any linear map O : H→ H′, we use O∗ to denote its adjoint, O−1 to denote its inverse (if invertible),O† to denote its Moore–Penrose pseudoinverse, Range(O) to denote its range space, and ‖O‖ to denoteits spectral norm.
• If H′ = H and O is self-adjoint and positive semidefinite, there must be a unique self-adjoint positivesemidefinite operator, denoted by O1/2, such that O1/2O1/2 = O. In this case, for any ν, ν′ ∈ H wedefine 〈ν, ν′〉O := 〈Oν, ν′〉 and ‖ν‖O :=
√〈ν,Oν〉 = ‖O1/2ν‖. If O is also invertible, O1/2 is invertible
and we use the notation that O−1/2 := (O1/2)−1 .
• Let O1, . . . ,Ok be k self-adjoint linear operators, we used Diag(O1, . . . ,Ok) to denote the block-diagonallinear operator whose block-diagonal elements are in the order of O1, . . . ,Ok.
• For any convex set H ⊆ H, we denote the relative interior of H by ri(H). When the self-adjoint linearoperator O : H→ H positive definite, we define, for any ν ∈ H,
distO(ν,H) := infν′∈H
‖ν − ν′‖O and ΠOH(ν) = arg minν′∈H
‖ν − ν′‖O.
6
If O is the identity operator we just omit it from the notation so that dist(·, H) and ΠH(·) are thestandard distance function and the metric projection operator, respectively.
• Let θ : H → (−∞,+∞] be an arbitrary closed proper convex function. We use dom θ to denote itseffective domain, ∂θ to denote its subdifferential mapping, and θ∗ to denote its conjugate function.Moreover, for a given self-adjoint and positive definite linear operator O : H → H, we use ProxOθ todenote the Moreau-Yosida proximal mapping of θ, which is defined by
ProxOθ (ν) := arg minν′∈H
θ(ν′) + 1
2‖ν − ν′‖2O, ∀ν ∈ H.
Note that the mapping ProxOθ is globally Lipschitz continuous. If O is the identity operator, we willdrop O from ProxOθ (·).
2 Illustrative Examples
In this section, we present a few important classes of concrete problems, including those in the classic coreconvex programming as well as those which are popularly used in various real-world applications. As willbe shown, these problems and/or their dual problems have the form given by (1.1), so that the algorithmdesigned in this paper can be utilized to solve them.
2.1 Convex Composite Quadratic Programming
It is well known that many problems are subsumed under the convex composite quadratic programming model(1.2) or the more concrete form (1.7). For example, it includes the important classes of convex quadraticprogramming (QP), the convex quadratic semidefinite programming (QSDP), and the convex quadraticprogramming and weighted centering [41] (QPWC). As an illustration, consider a convex QSDP problem inthe following form
minX∈Sn
1
2〈X,QX〉 − 〈C,X〉
∣∣∣ AEX = bE , AIX ≥ bI , X ∈ Sn+, (2.1)
where Sn is the space of n×n real symmetric matrices and Sn+ is the closed convex cone of positive semidefinitematrices in Sn, Q : Sn → Sn is a positive semidefinite linear operator, C ∈ Sn is a given matrix, and AE
and AI are the linear maps from Sn to the two finite-dimensional Euclidean spaces RmE and RmI thatcontaining bE and bI , respectively. To solve this problem, one may consult the recently developed softwareQSDPNAL in Li et al. [31] and the references therein. The algorithm implemented in QSDPNAL is atwo-phase augmented Lagrangian method in which the first phase is an inexact sGS decomposition basedmulti-block proximal ADMM whose convergence was established in [6, Theorem 5.1]. The solution generatedin the first phase is used as the initial point to warm-start the second phase algorithm, which is an ALMwith the inner subproblem in each iteration being solved via an inexact semismooth Newton algorithm. InSection 5, we will use the QSDP problem (2.1) to test the algorithm studied in this paper.
Besides the core optimization problems just mentioned above, there are many problems from real-wordapplications that can be cast in the form of (1.2) and the following are only a few such examples.
Penalized and Constrained Regression Models
In various statistical applications, the penalized and constrained (PAC) regression [25] often arises in high-dimensional generalized linear models with linear equality and inequality constraints. A concrete example ofthe PAC regression is the following constrained lasso problem
minx∈Rn
1
2‖Φx− η‖2 + λ‖x‖1 | AEx = bE , AIx ≥ bI
, (2.2)
7
where Φ ∈ Rm×n, AE ∈ RmE×n, AI ∈ RmI×n, η ∈ Rm, bE ∈ RmE and bI ∈ RmI are the given data, andλ > 0 is a given regularization parameter. The statistical properties of problem (2.2) have been studied in[25]. For more details on the applications of the model (2.2), one may refer to [25, 19] and the referencestherein. In Gaines et al. [19], the authors considered solving (2.2) by first reformulating it as a conventionalQP via letting x = x+ − x− and adding the extra constraints x+ ≥ 0, x− ≥ 0, and then applying the primalADMM to solve the conventional QP, in which all the subproblems should be solved exactly (or to veryhigh accuracy) by iterative methods. Such a combination may perform well for low dimensional problemswith moderate sample sizes. But for the more challenging and interesting high-dimensional cases where n isextremely large and m n, the approach in [19] is likely to face severe numerical difficulties because of thepresence of a huge number of constraints. Fortunately, the algorithm we designed in this paper can preciselyhandle those difficult cases because the large linear systems associated with the huge number of constraintsare not required to solve to very high accuracy by an iterative solver.
Noisy Matrix Completion and Rank-Correction Step
In Miao et al. [36], the authors introduced a rank-correction step for matrix completion with fixed basiscoefficients to overcome the shortcomings of the nuclear norm penalization model for such problems. LetX ∈ Vn1×n2 (where Vn1×n2 may represent the space of n1 × n2 real or complex matrices or the space of
n × n real symmetric or Hermitian matrices) be the unknown true low-rank matrix and Xm is an initialestimator of X from the nuclear norm penalized least squares model. The rank-correction step is to solve thefollowing convex optimization problem
minX
12m‖y −Po(X)‖2 + ρm
(‖X‖∗ − 〈F (Xm), X〉
)s.t. PA(X) = PA(X), ‖PB(X)‖∞ ≤ b,
(2.3)
where y = Po(X) + ε ∈ Rm is the observed data for the matrix X, Po is the linear map corresponding to theobserved entries, ε ∈ Rm is the unknown error, ρm > 0 is a given penalty parameter, and F : Vn1×n2 → Vn1×n2
is a spectral operator [10] whose precise definition can be found in [36, Section 5]. Here the constraintsPA(X) = PA(X) and ‖PB(X)‖∞ ≤ b represent the fixed elements and bounded elements of X, respectively.If F and the equality constraints are vacuous, problem (2.3) is exactly the noise matrix completion modelconsidered in [37], and a similar matrix completion model can be found in [26]. One may view (2.3) as aninstance of problem (1.2), and whose corresponding linear operator Q admits a very simple form.
2.2 Two-Block Problems
Next we present a few important classes of two-block problems whose objective functions contain a linearpart.
Semidefinite Programming
One of the most prominent examples of problem (1.1) with 2 blocks of variables (i.e., s = 1) is the dual linearsemidefinite programming (SDP) problem given by
minY, z
δSn+(Y )− 〈b, z〉 | Y + A∗z = C
, (2.4)
where A : Sn → Rm is a given linear map, and b ∈ Rm and C ∈ Sn are given data. The notation δSn+ denotes
the indicator function of Sn+. For problem (2.4), various ADMM algorithms have been employed to solve theproblem. As far as we are aware of, the classic two-block ADMM with unit step-length was first employed inPovh et al. [43] under the name of boundary point method for solving the SDP problem (2.4). It was laterextended in Malick et al. [34] with a convergence proof. The ADMM approach was later used in the softwareSDPNAL developed by Zhao et al. [58] to warm-start a semismooth Newton method based ALM for solvingproblem (2.4).
8
In section 5, we will conduct extensive numerical experiments on solving a few classes of linear SDPproblems via the two-block ADMM algorithm but with the dual step-length being chosen in the interval(0, 2), as it is guaranteed by this paper.
Equality Constrained Problems
Consider the equality constrained problem
minx∈X
θ(x) | Gx = b
, (2.5)
where G : X → Rm is a linear map, b ∈ Rm is a given vector, and θ : X → (−∞,+∞] is a simple closedproper convex function such that its proximal mapping can be computed efficiently. The dual problem of(2.5) can be written in the minimization form as
miny,zθ∗(y)− 〈b, z〉 | y − G∗z = 0 . (2.6)
A concrete example of problem (2.5), with X := Rn and θ(x) := ‖x‖1, is the basis pursuit (BP) problem [7],which has been wildly used in sparse signal recovery and image restoration. Another example of (2.5) is thenuclear norm based matrix completion problem for which X := Rn1×n2 and θ(x) = ‖x‖∗. Moreover, the socalled tensor completion problem [33] also falls into this category.
We note that for the application problems just mentioned above, the dimension of X is generally muchlarger than m, i.e., the dimension of the linear constraints. Therefore from the computational viewpoint, it isgenerally more economical to apply the two-block ADMM to the dual problem (2.6) instead of the primalproblem (2.5) (by introducing an extra variable x′ and adding the condition x− x′ = 0) because the formerwill solve smaller m×m linear systems in each iteration whereas the latter will correspondingly need to solvemuch larger linear systems.
Composite Problems
A composite problem can take the following form
minz∈Z
f (c− G∗z) , (2.7)
where f : Z→ (−∞,+∞] is a (possibly nonsmooth) closed proper convex function whose proximal mappingcan be computed efficiently, G : Z→ X is a given linear operator and c ∈ X is given data. By introducing aslack variable, problem (2.7) can be recast as
miny,z
f(y) | y + G∗z = c
.
Problem (2.7) contains many real-world applications such as the well-known least absolute deviation (LAD)problem (also known as least absolute error (LAE), least absolute value (LAV), least absolute residual (LAR),sum of absolute deviations, or the `1-norm condition). The model (2.7) also includes the Huber fittingproblem [24]. We shall not continue with more examples as there are too many applications to be listed hereto serve as a literature review.
Consensus Optimization
Consider the following problem
minz∈Z
n∑i=1
fi (G∗i z)
, (2.8)
where each fi is a closed proper convex function and each Gi : Yi → Z is a linear operator. The model(2.8) includes the global variable consensus optimization and general variable optimization, as well as their
9
regularized versions (see [4, Section 7]), which have been well applied in many areas such as machine learning,signal processing and wireless communication [4, 3, 49, 46, 59]. In the consensus optimization setting, it isusually preferable to solve subproblems each involving a subset of the component functions f1, . . . , fn insteadof all of them. Therefore, one can equivalently recast problem (2.8) as
miny,z
n∑i=1
fi(yi) | yi − G∗i z = 0, 1 ≤ i ≤ n
. (2.9)
Obviously, when applying the two-block ADMM to solve (2.9), the subproblem with respect to y is separatedinto n independent problems that can be solved in parallel. In [4], the variable z in (2.9) is called thecentral collector. Besides, the network based decentralized and distributed computation of the consensusoptimization, such as the distributed lasso in [35], also falls in the problems setting in this paper.
3 An Inexact Majorized ALM with Indefinite Proximal Terms
In this section, we present an inexact majorized indefinite-proximal ALM. This algorithm, as well as its globaland local convergence properties, not only constitutes a generalization of the original (proximal) ALM, butalso paves the way for us to establish its equivalence relationship with the inexact block sGS decompositionbased indefinite-proximal multi-block ADMM in the next section.
Let X and W be two finite-dimensional real Hilbert spaces each endowed with an inner product 〈·, ·〉and its induced norm ‖ · ‖. We consider the following fairly general linearly constrained convex optimizationproblem
minw∈W
ϕ(w) + h(w) | A∗w = c
, (3.1)
where ϕ : W → (−∞,+∞] is a closed proper convex function, h : W → (−∞,+∞) is a continuouslydifferentiable convex function whose gradient is Lipschitz continuous, A : X→W is a linear mapping andc ∈ X is the given data. The Karush-Kuhn-Tucker (KKT) system of problem (3.1) is given by
0 ∈ ∂ϕ(w) +∇h(w) +Ax, A∗w − c = 0. (3.2)
For any (w, x) ∈W ×X that solve the KKT system (3.2), w is a solution to problem (3.1) while x is a dualsolution of (3.1).
The fact that the gradient of h is Lipschitz continuous implies that there exists a self-adjoint positivesemidefinite linear operator Σh : W→W, such that for any w′ ∈W, h(w) ≤ h(w,w′), where
h(w,w′) := h(w′) + 〈∇h(w′), w − w′〉+1
2‖w − w′‖2
Σh, ∀w ∈W. (3.3)
We call the function h(·, w′) : W→ (−∞,+∞) a majorization of h at w′. The following result, whose proofcan be found in [57, Lemma 3.2], will be used later.
Lemma 3.1. Suppose that (3.3) holds for any given w′ ∈W. Then, it holds that⟨∇h(w)−∇h(w′), w′′ − w′
⟩≥ −1
4‖w − w′′‖2
Σh, ∀w,w′, w′′ ∈W.
Let σ > 0 be a given penalty parameter. The majorized augmented Lagrangian function associated withproblem (3.1) is defined by
Lσ(w; (x,w′)) := ϕ(w) + h(w,w′) + 〈A∗w − c, x〉+ σ2 ‖A
∗w − c‖2,
∀(w, x,w′) ∈W ×X×W.(3.4)
In the following, we propose an inexact majorized indefinite-proximal ALM in Algorithm iPALM for solvingproblem (3.1). This algorithm is an extension of the proximal method of multipliers developed by Rockafellar
10
[45], with new ingredients added based on the recent progress on using proximal terms which are not necessarilypositive definite [16, 29, 57] and the implementable inexact minimization criteria studied in [6]. For theconvenience of later convergence analysis, we make the following blanket assumption.
Assumption 3.1. The solution set to the KKT system (3.2) is nonempty and S : W → W is a givenself-adjoint (not necessarily positive semidefinite) linear operator such that
S −1
2Σh and
1
2Σh + σAA∗ + S 0. (3.5)
We are now ready to present Algorithm iPALM that will be studied in this section.
Algorithm iPALM An inexact majorized indefinite-proximal ALM
Let εk be a summable sequence of nonnegative numbers. Choose an initial point (x0, w0) ∈ X×W. Fork = 0, 1, . . ., perform the following steps in each iteration.
Step 1. Compute
wk+1 ≈ wk+1 := arg minw∈W
Lσ(w; (xk, wk)
)+
1
2‖w − wk‖2S
(3.6)
such that there exists a vector dk ∈W satisfying ‖dk‖ ≤ εk and
dk ∈ ∂wLσ(wk+1; (xk, wk)
)+ S(wk+1 − wk). (3.7)
Step 2. Compute xk+1 := xk + τσ(A∗wk+1 − c) with τ ∈ (0, 2) being the step-length.
We shall next proceed to analyze the global convergence, the rate of local convergence and the iterationcomplexity of Algorithm iPALM. For notational convenience, we collect the total quadratic information inthe objective function of (3.6) as the following linear operator
M := Σh + S + σAA∗. (3.8)
The following result presents two important inequalities for the subsequent analysis. The first one characterizesthe distance (with M being involved in the metric) from the computed solution to the true solution of thesubproblem in (3.6), while the second one presents a non-monotone descent property about the sequencegenerated by Algorithm iPALM.
Proposition 3.1. Suppose that Assumption 3.1 holds. Then,
(a) the sequence (xk, wk) generated by Algorithm iPALM and the auxiliary sequence wk defined in (3.6)are well-defined, and it holds that
‖wk+1 − wk+1‖2M ≤ 〈dk, wk+1 − wk+1〉; (3.9)
(b) for any given (x∗, w∗) ∈ X×W that solves the KKT system (3.2) and k ≥ 1, it holds that(1
2τσ‖xk+1
e ‖2 +1
2‖wk+1
e ‖2Σh+S
)−(
1
2τσ‖xke‖2 +
1
2‖wke‖2Σh+S
)≤ −
((2− τ)σ
2‖A∗wk+1
e ‖2 +1
2‖wk+1 − wk‖21
2 Σh+S − 〈dk, wk+1
e 〉),
(3.10)
where xe := x− x∗, ∀x ∈ X and we := w − w∗, ∀w ∈W.
11
Proof. (a) From (3.5) and (3.8) we know that M 0. Hence, each of the subproblems in Algorithm iPALMis strongly convex so that each wk+1 is uniquely determined by (xk, wk). Note that, for the given εk ≥ 0, onecan always find a certain wk+1 such that ‖dk‖ ≤ εk with dk being given in (3.7), see [12, Lemma 4.5]. Hence,Algorithm iPALM is well-defined. According to (3.3) and (3.4), the objective function in (3.6) is given by
ϕ(w) + 〈∇h(wk) +Axk, w〉+σ
2‖A∗w − c‖2 +
1
2‖w − wk‖2
Σh+S ,
so that (3.7) implies that
dk ∈ ∂ϕ(wk+1) +∇h(wk) +Axk + σA(A∗wk+1 − c) + (Σh + S)(wk+1 − wk). (3.11)
Therefore, from the definitions of the Moreau-Yosida proximal mapping and M in (3.8), one has that
wk+1 = ProxMϕ
(M−1
[dk − (∇h(wk) +Axk − σAc− (Σh + S)wk)
]).
Consequently, by the Lipschitz continuity of ProxMϕ [28, Proposition 2.3] and the fact that dk can be set as
zero if wk+1 = wk+1, one can readily get (3.9).
(b) Let (x∗, w∗) ∈ X ×W be an arbitrary solution to the KKT system (3.2). Obviously, one has that−∇h(w∗)−Ax∗ ∈ ∂ϕ(w∗) and A∗w∗ = c. This, together with (3.11) and the maximal monotonicity of ∂ϕ,implies that
〈dk −∇h(wk) +∇h(w∗)−Axke − σA(A∗wk+1 − c)− (Σh + S)(wk+1 − wk), wk+1e 〉 ≥ 0.
Therefore, by using the fact that
A∗wk+1 − c = A∗wk+1e =
1
τσ(xk+1 − xk), (3.12)
one can obtain from the above inequality and Lemma 3.1 that⟨dk, wk+1
e
⟩− 1
τσ
⟨xke , x
k+1 − xk⟩− σ‖A∗wk+1
e ‖2 −⟨
(Σh + S)(wk+1 − wk), wk+1e
⟩≥⟨∇h(wk)−∇h(w∗), wk+1
e
⟩≥ − 1
4‖wk+1 − wk‖2
Σh= − 1
2‖wk+1 − wk‖21
2 Σh.
(3.13)
Note that 〈xke , xk+1 − xk〉 = 12‖x
ke‖2 − 1
2‖xke‖2 − 1
2‖xk+1 − xk‖2 and⟨
(Σh + S)(wk+1 − wk), wk+1e
⟩=
1
2‖wk+1 − wk‖2
Σh+S +1
2‖wk+1
e ‖2Σh+S −
1
2‖wke‖2Σh+S .
Then, (3.10) follows form (3.13) and this completes the proof of the proposition.
3.1 Global Convergence
For the convenience of our analysis, we define the following two linear operators
Ξ := τσ
(1
2Σh + S +
(2− τ)σ
6AA∗
)and Θ := τσ
(Σh + S +
(2− τ)σ
3AA∗
), (3.14)
which will be used in defining metrics in W. Note that τ ∈ (0, 2). If (3.5) in Assumption 3.1 holds, one hasthat
1τσΞ = 2−τ
6
(12 Σh + S + σAA∗
)+(1− 2−τ
6
) (12 Σh + S
) 2−τ
6
(12 Σh + S + σAA∗
) 0,
1τσΘ = 1
τσΞ + 12 Σh + (2−τ)σ
6 AA∗ 1τσΞ 0.
(3.15)
12
Moreover, we define the block-diagonal linear operator Ω : U→ U by
Ω(x;w) :=(x; Θ
12w), ∀(x,w) ∈ X×W, (3.16)
where Θ is given by (3.14). Now we establish the convergence theorem of Algorithm iPALM. The correspondingproof mainly follows from the proof of [6, Theorem 5.1] for the convergence of an inexact majorized semi-proximal ADMM and the following result on quasi-Fejer monotone sequence will be used.
Lemma 3.2. Let akk≥0 be a sequence of nonnegative real numbers sequence satisfying ak+1 ≤ ak + εk forall k ≥ 0, where εkk≥0 is a nonnegative and summable sequence of real numbers. Then the ak convergesto a unique limit point.
Theorem 3.1. Suppose that Assumption 3.1 holds and the sequence (xk, wk) is generated by AlgorithmiPALM. Then,
(a) for any solution (x∗, w∗) ∈ X×W of the KKT system (3.2) and k ≥ 1, we have that∥∥(xk+1e ;wk+1
e )∥∥2
Ω−∥∥(xke ;wke )
∥∥2
Ω
≤ −(
(2− τ)
3τ‖xk+1 − xk‖2 + ‖wk+1 − wk‖2Ξ − 2τσ〈dk, wk+1
e 〉),
(3.17)
where xe := x− x∗, ∀x ∈ X and we := w − w∗, ∀w ∈W;
(b) the sequence (xk, wk) is bounded;
(c) any accumulation point of the sequence (xk, wk) solves the KKT system (3.2);
(d) the whole sequence (xk, wk) converges to a solution to the KKT system (3.2).
Proof. (a) By using (3.14), together with the definitions of Ξ and Θ in (3.10), and the fact that A∗wk+1e =
1τσ (xk+1 − xk), one can get∥∥(xk+1
e ;wk+1e )
∥∥2
Ω−∥∥(xke ;wke )
∥∥2
Ω=(‖xk+1
e ‖2 + ‖wk+1e ‖2Θ
)−(‖xke‖2 + ‖wke‖2Θ
)≤ −τσ
(2(2−τ)σ
3 ‖A∗wk+1e ‖2 + (2−τ)σ
3 ‖A∗wke‖2 + ‖wk+1 − wk‖212 Σh+S
− 2〈dk, wk+1e 〉
)≤ −
((2−τ)
3τ ‖xk+1 − xk‖2 + τσ (2−τ)
3
(σ‖A∗wk+1
e ‖2 + σ‖A∗wke‖2)
+τσ‖wk+1 − wk‖212 Σh+S
− 2τσ〈dk, wk+1e 〉
)≤ −
((2−τ)
3τ ‖xk+1 − xk‖2 + στ‖wk − wk+1‖2(2−τ)σ
6 AA∗+ 12 Σh+S
− 2τσ〈dk, wk+1e 〉
),
which, together with the definition of the linear operator Ξ in (3.14), implies (3.17).
(b) Define xk+1 := xk + τσ(A∗wk+1− c), ∀k ≥ 0. From (3.6), (3.7) and (3.17) one can get that for any k ≥ 0,
∥∥(xk+1e ;wk+1
e )∥∥2
Ω≤∥∥(xke ;wke )
∥∥2
Ω−(
(2− τ)
3τ‖xk+1 − xk‖2 + ‖wk+1 − wk‖2Ξ
).
Meanwhile, one can get that ‖(xk+1e ;wk+1
e )‖Ω ≤ ‖(xke ;wke )‖Ω, ∀ k ≥ 0. Therefore, it holds that∥∥(xk+1e ;wk+1
e )∥∥
Ω≤∥∥(xk+1
e ;wk+1e )
∥∥Ω
+∥∥(xk+1 − xk+1;wk+1 − wk+1
)∥∥Ω
≤∥∥(xke ;wke )
∥∥Ω
+∥∥(τσA∗(wk+1 − wk+1); Θ1/2(wk+1 − wk+1)
)∥∥=∥∥(xke ;wke )
∥∥Ω
+∥∥wk+1 − wk+1
∥∥τ2σ2AA∗+Θ
, ∀k ≥ 0.
(3.18)
13
From (3.9) we know that ‖wk+1 − wk+1‖2M ≤ 〈M−1/2dk,M1/2(wk+1 − wk+1)〉, so that
‖M1/2(wk+1 − wk+1)‖ ≤ ‖M−1/2dk‖ ≤ ‖M−1/2‖‖dk‖.
Therefore, it holds that
‖wk+1 − wk+1‖ ≤ ‖M−1/2‖‖M1/2(wk+1 − wk+1)‖
≤ ‖M−1/2‖‖M−1/2‖‖dk‖ ≤ ‖M−1‖εk, ∀k ≥ 0.(3.19)
Therefore, by combining (3.18) and (3.19) together we can get∥∥(xk+1e ;wk+1
e )∥∥
Ω≤∥∥(xke ;wke )
∥∥Ω
+√‖τ2σ2AA∗ + Θ‖ ‖M−1‖εk, ∀k ≥ 0.
Hence, the sequence‖(xk+1
e ;wk+1e )‖Ω
is quasi-Fejer monotone, which converges to a unique limit point by
Lemma 3.2. Since Ω defined in (3.16) is positive definite, we further know that the sequence (xk, wk) isbounded.
(c) From (3.17) we know that for any k ≥ 0,
k∑i=0
∥∥(xie;wie)∥∥2
Ω−∥∥(xi+1
e ;wi+1e )
∥∥2
Ω+ 2τσεi‖wi+1
e ‖
≥k∑i=0
(2− τ)
3τ‖xi+1 − xi‖2 + ‖wi+1 − wi‖2Ξ
.
(3.20)
Since (xk, wk) is bounded and εk is summable, it holds that
∞∑i=0
∥∥(xie;wie)∥∥2
Ω−∥∥(xi+1
e ;wi+1e )
∥∥2
Ω+ 2τσεi‖wi+1
e ‖<∞,
which, together with (3.20) and the fact that Ξ 0, implies
limk→∞
(xk+1 − xk) = 0 and limk→∞
(wk+1 − wk) = 0. (3.21)
Suppose that the subsequence (xkj , wkj ) of (xk, wk) converges to some limit point (x∞, w∞). By takinglimits on both sides of (3.11) and (3.12) along with kj and using (3.21) and [44, Theorem 24.6], one can get
0 ∈ ∂ϕ(w∞) +∇h(w∞) +Ax∞ and A∗w∞ − c = 0,
which implies that (x∞, w∞) is a solution to the KKT system (3.2).
(d) Note that (3.17) holds for any (x∗, w∗) satisfying the KKT system (3.2). Therefore, we can choosex∗ = x∞ and w∗ = w∞ in (3.17):
‖(xk+1 − x∞;wk+1 − w∞)‖2Ω ≤ ‖(xk − x∞;wk − w∞)‖2Ω + 2τσ‖wk+1 − w∞‖εk.
Note that wk is bounded. Then, the above inequality, together with Lemma 3.2, implies that the quasi-Fejermonotone sequence ‖(xk − x∞;wk − w∞)‖2Ω converges. Since (x∞, w∞) is a limit point of (xk, wk), onehas that
limk→0‖(xk − x∞;wk − w∞)‖2Ω = 0,
which, together with the fact that Ω 0, implies that the whole sequence (xk, wk) converges to (x∞, w∞).This completes the proof of the theorem.
14
3.2 Local Convergence Rate
In this section, we present the local convergence rate analysis of Algorithm iPALM. For this purpose, wedenote U := X×W and consider the KKT residual mapping of problem (3.1) defined by
R(u) = R(x,w) :=
(c−A∗w
w − Proxϕ(w −∇h(w)−Ax)
), ∀u = (x,w) ∈ X×W. (3.22)
Note that R(u) = 0 if and only if u = (x,w) is a solution to the KKT system (3.2), whose solution set cantherefore be characterized by K := u |R(u) = 0. Moreover, the residual mapping R has the followingproperty.
Lemma 3.3. Suppose that Assumption 3.1 holds and the sequence uk := (xk, wk) is generated by AlgorithmiPALM. Then, for any k ≥ 0,
‖R(uk+1)‖2 ≤ 1
τ2σ2‖xk − xk+1‖2 +
2‖Σh + S‖τσ
‖wk+1 − wk‖2Θ
+2‖(1− τ−1)A(xk+1 − xk) + dk‖2.(3.23)
Proof. Note that (3.11) holds. Then, one can see that
wk+1 = Proxϕ
(wk+1 + dk −∇h(wk)−A
(xk +
xk+1 − xk
τ
)−(Σh + S
)(wk+1 − wk)
).
By taking the above equality and c−A∗wk+1 = 1τσ (xk − xk+1) into the definition of R(uk+1) in (3.22) and
using the Lipschitz continuity of Proxϕ, one can get
‖R(uk+1)‖2 ≤ 1
τ2σ2‖xk − xk+1‖2 + 2‖(1− τ−1)A(xk+1 − xk) + dk‖2
+2‖∇h(wk+1)−∇h(wk)−(Σh + S
)(wk+1 − wk)‖2.
(3.24)
By using Clarke’s mean value theorem [8, Proposition 2.6.5] we know that for any k ≥ 0 there exists a linear
operator Σk : W→W such that ∇h(wk+1)−∇h(wk) = Σk(wk+1 − wk) with 0 Σk Σh, so that
‖∇h(wk+1)−∇h(wk)−(Σh + S
)(wk+1 − wk)‖2
= ‖(Σh + S − Σk
)(wk+1 − wk)‖2
≤ ‖Σh + S − Σk‖〈wk+1 − wk,(Σh + S − Σk
)(wk+1 − wk)〉
≤ ‖Σh + S‖〈wk+1 − wk,(Σh + S
)(wk+1 − wk)〉
≤ ‖Σh+S‖τσ
⟨wk+1 − wk, τσ
(Σh + S + (2−τ)σ
3 AA∗)
(wk+1 − wk)⟩, ∀k ≥ 0,
(3.25)
where the last inequality comes form the fact that 0 < τ < 2. Then, by using the definition of Θ in (3.14),one can readily see from (3.24) and (3.25) that (3.23) holds. This completes the proof.
To analyze the linear convergence rate of Algorithm iPALM, we shall introduce the following error boundcondition.
Definition 3.1. The KKT residual mapping R defined in (3.22) is said to be metric subregular3 [11, 3.8 [3H]](with the modulus κ > 0) at u ∈ K for 0 ∈ U if there exists a constant r > 0 such that
dist(u,K
)≤ κ‖R(u)‖ , ∀u ∈ u ∈ U | ‖u− u‖ ≤ r. (3.26)
3 This is equivalent to say that R−1 is calm at 0 ∈ U for u ∈ K with the same modulus κ > 0, see [11, Theorem 3H.3].
15
Suppose that Assumption 3.1 holds. We know from (3.14) and (3.15) that Ξ 0. Hence, one can letζ > 0 be the smallest real number such that ζΞ Θ. For notational convenience, we define the followingpositive constants:
ρ := max
6σ2(τ − 1)2‖A∗A‖+ 3
τσ2(2− τ),
2ζ‖Σh + S‖τσ
max
‖Θ‖, 1
, (3.27)
β := max√
ζ,√
3τ/(2− τ), (3.28)
µ :=
√τσ‖Σh + S +
2
3(1 + τ)σAA∗‖ ‖M−1‖. (3.29)
To ensure the local linear rate convergence of Algorithm iPALM, we need extra conditions to control theerror variable dk in each iteration. Hence, we make the following assumption.
Assumption 3.2. There exists an integer k0 > 0 and a sequence of nonnegative real numbers ηk such that
supk≥k0ηk < 1/µ and ‖dk‖ ≤ ηk‖uk − uk+1‖, ∀ k ≥ k0. (3.30)
Now we are ready to present the local convergence rate of Algorithm iPALM.
Theorem 3.2. Suppose that Assumptions 3.1 and 3.2 hold. Let uk = (xk, wk) be the sequence generated byAlgorithm iPALM that converges to u∗ := (x∗, w∗) ∈ K. Suppose that the KKT residual mapping R definedin (3.22) is metric subregular at u∗ for 0 ∈ U with the modulus κ > 0, in the sense that there exists a constantr > 0 such that (3.26) holds with u = u∗. Then, there exists a threshold k > 0 such that for for all k ≥ k,
distΩ
(uk+1,K
)≤ ϑk distΩ
(uk,K
)with ϑk :=
1
1− µηk
(√κ2ρ
1 + κ2ρ+ µηk(1 + β)
). (3.31)
Moreover, if it holds that
supk≥kηk <
1
µ(2 + β)
(1−
√κ2ρ
1 + κ2ρ
), (3.32)
then one has supk≥kϑk < 1, and the convergence rate of distΩ(uk,K) is Q-linear when k ≥ k.
Proof. Denote ue := u− u∗ for all u ∈ U and define xk+1 := xk + τσ(A∗wk+1− c) and uk+1 := (xk+1, wk+1),∀k ≥ 0. Since uk converges to u∗ and dk converges to 0 as k →∞, one has from (3.9) that uk alsoconverges to u∗ as k →∞. Therefore, there exists a threshold k > 0 such that
‖uk+1e ‖ ≤ r and ‖uk+1
e ‖ ≤ r , ∀k ≥ k. (3.33)
According to Lemma 3.3, one can let uk+1 = uk+1 and dk = 0 in (3.23) and use the fact that ζΞ Θ toobtain that
‖R(uk+1)‖2
≤(
2(τ − 1)2‖A∗A‖τ2
+1
(τσ)2
)‖xk+1 − xk‖2 +
2‖Σh + S‖τσ
‖wk+1 − wk‖2Θ
≤ max
6σ2(τ − 1)2‖A∗A‖+ 3
τσ2(2− τ),
2ζ‖Σh + S‖τσ
((2− τ)
3τ‖xk+1 − xk‖2 + ‖wk+1 − wk‖2Ξ
).
(3.34)
Moreover, according to the definition of Ω in (3.16), one has that
dist2Ω
(u,K
)≤ max‖Θ‖, 1dist2
(u,K
), ∀u ∈ U.
16
Then, by using the above inequality, together with (3.26), (3.33) and (3.34), we can obtain with the constantρ > 0 being defined in (3.27) that
dist2Ω(uk+1,K) ≤ κ2 max‖Θ‖, 1‖R(uk+1)‖2
≤ κ2ρ
((2− τ)
3τ‖xk+1 − xk‖2 + ‖wk+1 − wk‖2Ξ
), ∀ k ≥ k.
(3.35)
It is easy to see from (3.17) that, for any k ≥ 0,
dist2Ω(uk+1,K) ≤ dist2
Ω(uk,K)−(
(2− τ)
3τ‖xk+1 − xk‖2 + ‖wk+1 − wk‖2Ξ
). (3.36)
Therefore, by combining (3.35) and (3.36) together we can get
dist2Ω(uk+1,K) ≤ κ2ρ
1 + κ2ρdist2
Ω(uk,K), ∀ k ≥ k. (3.37)
From (3.36) and the fact that ζΞ Θ we know that
dist2Ω(uk,K) ≥ min
(2− τ)
3τ,
1
ζ
‖uk − uk+1‖2Ω, ∀ k ≥ 0.
Therefore, it holds that‖uk − uk+1‖Ω ≤ β distΩ(uk,K), ∀ k ≥ 0, (3.38)
where the constant β > 0 is given in (3.28). By using the triangle inequality, we have that
‖uk −ΠΩK(uk+1)‖Ω ≤ distΩ(uk,K) + ‖ΠΩ
K(uk)−ΠΩK(uk+1)‖Ω, ∀k ≥ 0. (3.39)
Moreover, from [28, Proposition 2.3] we know that
‖ΠΩK(uk)−ΠΩ
K(uk+1)‖2Ω ≤ 〈ΠΩK(uk)−ΠΩ
K(uk+1),Ω(uk − uk+1)〉, ∀ k ≥ 0.
Thus, one has‖ΠΩ
K(uk)−ΠΩK(uk+1)‖Ω ≤ ‖uk − uk+1‖Ω, ∀k ≥ 0,
which together with (3.38) and (3.39), implies that
‖uk −ΠΩK(uk+1)‖Ω ≤ (1 + β)distΩ(uk,K), ∀ k ≥ 0. (3.40)
From the definitions of Θ in (3.14) and Ω in (3.16) we know that
‖uk+1 − uk+1‖2Ω = (τσ)2‖A∗(wk+1 − wk+1)‖2 + ‖wk+1 − wk+1‖2Θ
=⟨wk+1 − wk+1, τσ(Σh + S + 2
3 (1 + τ)σAA∗)(wk+1 − wk+1)⟩.
Based on the above equality, one can see from (3.19) and (3.29) that
‖uk+1 − uk+1‖Ω ≤ µ‖dk‖. (3.41)
Since Assumption 3.2 holds, by using (3.30), (3.41) and the triangle inequality one can get that
‖uk+1 −ΠΩK(uk+1)‖Ω ≤ ‖uk+1 − uk+1‖Ω + distΩ
(uk+1,K
)≤ µ‖dk‖+ distΩ
(uk+1,K
)≤ µηk‖uk − uk+1‖+ distΩ(uk+1,K)
≤ µηk‖uk+1 −ΠΩK(uk+1)‖Ω + µηk‖uk −ΠΩ
K(uk+1)‖Ω + distΩ(uk+1,K), ∀k ≥ k.
17
Then, by using the fact that ‖uk+1 − ΠΩK(uk+1)‖Ω ≥ distΩ(uk+1,K) and (3.40), we can obtain that when
k ≥ 0,(1− µηk)distΩ(uk+1,K) ≤ µηk(1 + β)distΩ(uk,K) + distΩ(uk+1,K),
which, together with (3.37), implies (3.31). Finally, it is easy to see that supk≥kϑk < 1 from (3.30) and(3.32). This completes the proof.
Remark 3.1. Note that if ηk → 0 as k →∞, condition (3.32) holds eventually for k sufficiently large.
3.3 Non-Ergodic Iteration Complexity
With the inequalities established in the previous subsections, one can easily get the following non-ergodiciteration complexity results for Algorithm iPALM.
Theorem 3.3. Suppose that Assumption 3.1 holds. Let uk = (xk, wk) be the sequence generated byAlgorithm iPALM that converges to u∗ := (x∗, w∗) ∈ K. Then, the KKT residual mapping R defined in(3.22) satisfies
min0≤j≤k
‖R(uj)‖2 ≤ %/k and limk→∞
(k · min
0≤j≤k‖R(uj)‖2
)= 0, (3.42)
where the constant % is defined by
% := max
12σ2(τ − 1)2‖A∗A‖+ 3
τσ2(2− τ),
2ζ‖Σh + S‖τσ
e (3.43)
with e := ‖u0e‖2Ω + 2τσ‖Θ−1/2‖(
∑∞j=0 εj)
(‖u0
e‖Ω + µ∑∞j=0 εj
)+ 4
∑∞j=1 ε
2j .
Proof. From (3.17) in Theorem 3.1(a) we know that‖uj+1e ‖Ω ≤ ‖uje‖Ω, ∀ j ≥ 0. Moreover, (3.41) still holds
with µ being given in (3.29), so that
‖uj+1 − uj+1‖Ω ≤ µ‖dj‖, ∀ j ≥ 0.
Therefore,
‖wj+1e ‖Θ ≤ ‖uj+1
e ‖Ω ≤ ‖uje‖Ω + µ‖dk‖ ≤ ‖u0e‖Ω + µ
∞∑j=0
εj , ∀ j ≥ 0.
Consequently, for any k ≥ 0
k∑j=0
〈dj , wj+1e 〉 ≤ ‖Θ−1/2‖
k∑j=0
‖dj‖
‖wj+1e ‖Θ
≤ ‖Θ−1/2‖
∞∑j=0
εj
‖u0e‖Ω + µ
∞∑j=0
εj
.
(3.44)
Also, from (3.17) of Theorem 3.1(a) we know that for any k ≥ 0,
‖u0e‖2Ω ≥ ‖u0
e‖2Ω − ‖uk+1e ‖2Ω =
k∑j=0
(‖uje‖2Ω − ‖uj+1
e ‖2Ω)
≥k∑j=0
(‖wj+1 − wj‖2Ξ +
2− τ3τ‖xj+1 − xj‖2
)− 2τσ
k∑j=0
〈dj , wj+1e 〉.
(3.45)
Moreover, from (3.23) we know that
‖R(uk+1)‖2 ≤ 4σ2(τ − 1)2‖A∗A‖+ 1
τ2σ2‖xk+1 − xk‖2 +
2ζ‖Σh + S‖τσ
‖wk+1 − wk‖2Ξ + 4‖dk‖2.
Therefore, we can get from (3.44) and (3.45) that∑∞j=0 ‖R(uj+1)‖2 ≤ %. From here, we can easily get
required results in (3.42).
18
4 The Equivalence Property
In this section, we establish the equivalence of an inexact block sGS decomposition based multi-blockindefinite-proximal ADMM for solving problem (1.1) to the inexact indefinite-proximal ALM presented inthe previous section. The iteration scheme of the former has already been briefly sketched in (1.6) in theintroduction. Here we shall formally present it as Algorithm sGS-iPADMM.
Algorithm sGS-iPADMM An inexact block sGS decomposition based indefinite-Proximal ADMM
Let εk be a summable sequence of nonnegative real numbers, τ ∈ (0, 2) be the (dual) step-length, and(x0, y0, z0) ∈ X×dom p×Y2×· · ·×Ys×Z be the given initial point. Choose the self-adjoint linear operatorsDi : Yi → Yi, i = 1, . . . , s. For k = 0, 1, . . . , perform the following steps in each iteration.
Step 1. For i = s, . . . , 2, compute
yk+ 1
2i ≈ arg min
yi∈Yi
Lσ((yk<i; yi; y
k+ 12
>i
), zk; (xk, yk)
)+
1
2‖yi − yki ‖2Di
,
such that there exists δki satisfying ‖δki ‖ ≤ εk and
δki ∈ ∂yiLσ((yk<i; y
k+ 12
i ; yk+ 1
2>i
), zk; (xk, yk)
)+Di
(yk+ 1
2i − yki
).
Step 2. For i = 1, . . . , s, compute
yk+1i ≈ arg min
yi∈Yi
Lσ((yk+1<i ; yi; y
k+ 12
>i
), zk; (xk, yk)
)+
1
2‖yi − yki ‖2Di
,
such that there exists δki satisfying ‖δki ‖ ≤ εk and
δki ∈ ∂yiLσ((yk+1<i ; yk+1
i ; yk+ 1
2>i
), zk; (xk, yk)
)+Di
(yk+1i − yki
).
Step 3. Compute zk+1 ≈ arg minz∈Z
Lσ(yk+1, z; (xk, yk)
), such that ‖γk‖ ≤ εk with
γk := ∇zLσ(yk+1, zk+1; (xk, yk)
)= Gxk − b+ σG(F∗yk+1 + G∗zk+1 − c). (4.1)
Step 4. Compute xk+1 := xk + τσ(F∗yk+1 + G∗zk+1 − c
).
Recall that the KKT system of problem (1.1) is defined by
0 ∈(∂p(y1)
0
)+∇f(y) + Fx, Gx− b = 0, F∗y + G∗z = c. (4.2)
We make the following assumption on problem (1.1) throughout this section.
Assumption 4.1. The solution set to the KKT system (4.2) is nonempty.
Note that if Slater’s constraint qualification (SCQ) holds for problem (1.1), i.e.,(y, z) | y1 ∈ ri (dom p), F∗y + G∗z = c
6= ∅,
then we know from [44, Corollaries 28.2.2 & 28.3.1] that a vector (y, z) ∈ Y × Z is a solution to problem(1.1) if and only if there exists a Lagrangian multiplier x ∈ X such that (x, y, z) is a solution to the KKT
19
system (4.2). Therefore, Assumption 4.1 holds if the SCQ holds and (1.1) has at least one optimal solution.Moreover, for any (x, y, z) ∈ X×Y ×Z satisfying (4.2), we know from [44, Corollary 30.5.1] that (y, z) is anoptimal solution to problem (1.1) and x is an optimal solution to its dual problem.
Recall that the majorized augmented Lagrangian function of problem (1.1) was given in (1.5). Note thatone can always write Fx = (F1x; . . . ;Fsx), ∀x ∈ X with each Fi : X → Yi being a given linear mapping.
For later discussions, we symbolically decompose the self-adjoint linear operator Σf in the following from
Σf =
Σf11 Σf12 · · · Σf1s
Σf21 Σf22 · · · Σf2s
......
. . ....
Σfs1 Σfs2 · · · Σfss
with Σfij : Yj → Yi, ∀1 ≤ i, j ≤ s. (4.3)
Based on the above decomposition, we make the following assumption on choosing the proximal terms inAlgorithm sGS-iPADMM .
Assumption 4.2. The self-adjoint linear operators Di : Yi → Yi, i = 1, . . . , s in Algorithm sGS-iPADMMare chosen such that
1
2Σfii + σFiF∗i +Di 0 and D := Diag(D1, . . . ,Ds) −
1
2Σf . (4.4)
We are now ready to prove the equivalence of Algorithm 1 and Algorithm sGS-iPADMM for solvingproblem (1.1). We begin by applying the inexact block sGS decomposition technique in [32, Theorem 1] toexpress the procedure for computing yk+1 in Steps 1 and 2 of Algorithm sGS-iPADMM in a more compactfashion. For this purpose we define the following linear operator
N := Σf + σFF∗ +D. (4.5)
Note that the self-adjoint linear operator N is positive semidefinite, if Assumption 4.2 holds. Moreover, ascan be seen from (1.5), for any given (x, y′, z) ∈ X×Y ×Z, the linear operator N contains all the quadraticinformation of
Lσ (y, z; (x, y′)) +1
2‖y − y′‖2D
with respect to y. Based on (4.3), the linear operator N can be decomposed as N = Nd +Nu +N ∗u with Ndand Nu being the block-diagonal and the strict block-upper triangular parts of N , respectively, i.e.,
Nd := Diag(N11, . . . ,Nss) with Nii := Σfii + σFiF∗i +Di, i = 1, . . . , s
and
Nu :=
0 N12 · · · N1s
0 0. . .
...
......
. . . N(s−1)s
0 0 · · · 0
with Nij = Σfij + σFiF∗j , ∀ 1 ≤ i < j ≤ s. (4.6)
For convenience, we denote in Algorithm sGS-iPADMM for each k ≥ 0, δk1 := δk1 , δk := (δk1 , δ2k . . . , δ
ks ) and
δk := (δk1 , . . . , δks ). Suppose that Assumption 4.2 holds. We can define the sequence δksGS ∈ Y by
δksGS := δk +NuN−1d (δk − δk). (4.7)
20
Moreover, we can define the linear operator
NsGS := NuN−1d N
∗u . (4.8)
Based on the above definitions, we have the following result, which is a direct consequence of [32, Theorem 1].
Lemma 4.1. Suppose that Assumption 4.2 holds. The iterate yk+1 in Step 2 of Algorithm sGS-iPADMM isthe unique solution to the perturbed proximal minimization problem given by
yk+1 = arg miny∈Y
Lσ(y, zk;
(xk, yk
))+
1
2‖y − yk‖2D+NsGS
− 〈δksGS, y〉. (4.9)
Moreover, it holds that N +NsGS = (Nd +Nu)N−1d (Nd +N ∗u ) 0.
Remark 4.1. From (4.9) one can get the interpretation of the linear operator NsGS defined in (4.8). Thatis, by adding the proximal term 1
2‖y − yk‖2D to the majorized augmented Lagrangian function and conduct
one cycle of the block sGS-type block coordinate minimization via Steps 1 and 2 in Algorithm sGS-iPADMM,the resulted yk+1 is then an inexact solution to the following problem
miny∈Y
Lσ(y, zk;
(xk, yk
))+
1
2‖y − yk‖2D +
1
2‖y − yk‖2NsGS
,
where the proximal term 12‖y − y
k‖2NsGSis generated due to the sGS-type iteration with the linear operator
NsGS being defined by (4.8) and (4.5), while δksGS defined in (4.7) represents the error accumulated from δk
and δk after one cycle of the sGS-type update.
The following elementary result4 will be frequently used later.
Lemma 4.2. The self-adjoint linear operator GG∗ is nonsingular (positive definite) on the subspace Range(G)of Z.
Now, we start to establish the equivalence between Algorithm sGS-iPADMM and Algorithm iPALM.The first step is to show that the procedure of obtaining (yk+1, zk+1) in Algorithm sGS-iPADMM can beviewed as the procedure of getting wk+1 in Algorithm iPALM. For this purpose, we define the block diagonallinear operator T : Y ×Z→ Y ×Z by
T(y; z)
:=
((D +NsGS + σFG∗[GG∗]†GF∗)y
0
), ∀
(y, z)∈ Y ×Z. (4.10)
Moreover, we define the sequence
∆k
in Y by
∆k := δksGS −FG∗[GG∗]†(γk−1 − γk − G(xk−1 − xk)
), k ≥ 0 (4.11)
with the convention that x−1 := x0 − τσ(F∗y0 + G∗z0 − c),
γ−1 := −b+ Gx−1 + σG(F∗y0 + G∗z0 − c).(4.12)
Based on the above definitions and Lemma 4.1, we have the following result.
Proposition 4.1. Suppose that Assumption 4.2 holds. Then,
(a) Algorithm sGS-iPADMM is well-defined;
4This lemma can be directly verified via the singular value decomposition of the linear operator G and some basic calculationsfrom linear functional analysis.
21
(b) the sequence (xk, yk, zk) generated by Algorithm sGS-iPADMM satisfies(∆k; γk
)∈ ∂(y,z)Lσ
((yk+1, zk+1
);(xk, yk
))+ T
(yk+1 − yk; zk+1 − zk
), ∀ k ≥ 0. (4.13)
Proof. (a) Since Assumption 4.2 holds, it is easy to see from Lemma 4.1 that Steps 1 and 2 in algorithmsGS-iPADMM are well-defined for any k ≥ 0. Moreover, from (4.1) we know that Step 3 of AlgorithmsGS-iPADMM is well-defined if, for any k ≥ 0, the following linear system with respect to z
Gxk − b+ σG(F∗yk+1 + G∗z − c) = 0 (4.14)
has a solution. Since b ∈ Range(G), we know that (b − Gxk)/σ − G(F∗yk+1 − c) ∈ Range(G). Therefore,Lemma 4.2 implies that the linear system
GG∗z = (b− Gxk)/σ − G(F∗yk+1 − c)
or equivalently the linear system (4.14), has a solution. Consequently, Algorithm sGS-iPADMM is well-defined.
(b) From (4.1) and (4.12) we know that for any k ≥ 0,
γk−1 = −b+ Gxk−1 + σG(F∗yk + G∗zk − c) (4.15)
so that γk−1 ∈ Range(G) and GG∗zk = (γk−1 + b− Gxk−1)/σ − GF∗yk + Gc. Hence,
GG∗(zk − zk+1) =1
σ(γk−1 − γk − Gxk−1 + Gxk)− GF∗(yk − yk+1), ∀ k ≥ 0.
Therefore, one can get5 that for any k ≥ 0,
σFG∗(zk − zk+1)
= FG∗[GG∗]†(γk−1 − γk − G(xk−1 − xk)) + σFG∗[GG∗]†GF∗(yk+1 − yk).(4.16)
From (4.9) in Lemma 4.1 we know that, for any k ≥ 0,
δksGS ∈ ∂yLσ(yk+1, zk; (xk, yk)
)+(D +NsGS
)(yk+1 − yk)
= ∂yLσ(yk+1, zk+1;
(xk, yk
))+(D +NsGS
)(yk+1 − yk) + σFG∗
(zk − zk+1
).
(4.17)
Then, by substituting (4.16) into (4.17) and using the definition of ∆k in (4.11), one has that
∆k ∈ ∂yLσ(yk+1, zk+1; (xk, yk)
)+(D +NsGS + σFG∗[GG∗]†GF∗
)(yk+1 − yk),
which, together with (4.1), implies that (4.13) holds. This completes the proof.
The following important result will be used later.
Proposition 4.2. Suppose that Assumptions 4.1 and 4.2 hold. Let (xk, yk, zk) be the sequence generatedby Algorithm sGS-iPADMM. Define ξ0 := ‖b− Gx0‖ and
ξk := |1− τ |kξ0 + τ∑ki=1 |1− τ |k−iεi−1, ∀ k ≥ 1.
Then, it holds that for all k ≥ 0, ‖b− Gxk‖ ≤ ξk and
∞∑k=0
‖b− Gxk‖ ≤∞∑k=0
ξk < +∞.
5This can be routinely derived by using the singular value decomposition of G and the definition of the Moore-Penrosepseudoinverse.
22
Proof. We know from Step 4 of Algorithm sGS-iPADMM and (4.12) that
xk = xk−1 + τσ(F∗yk + G∗zk − c
), ∀ k ≥ 0.
Hence, one has thatb− Gxk = b− Gxk−1 − τσG
(F∗yk + G∗zk − c
), ∀ k ≥ 0.
Moreover, from (4.12) and (4.15) we know that
τ(γk−1 + b− Gxk−1) = τσG(F∗yk + G∗zk − c
), ∀k ≥ 0.
Thus, by combining the above two equalities together, one can get
b− Gxk = b− Gxk−1 − τ(γk−1 + b− Gxk−1
)= (1− τ)
(b− Gxk−1
)− τγk−1, ∀ k ≥ 0.
Consequently, it holds that
‖b− Gxk‖ ≤ |1− τ | ‖b− Gxk−1‖+ τ‖γk−1‖, ∀ k ≥ 0,
and hence
‖b− Gxk‖ ≤ |1− τ |k ‖b− Gx0‖+ τ
k∑i=1
|1− τ |k−i ‖γi−1‖ ≤ ξk, ∀k ≥ 0. (4.18)
Note that τ ∈ (0, 2). It is easy to see that
∞∑k=0
‖b− Gxk‖ ≤∞∑k=0
ξk ≤
( ∞∑k=0
|1− τ |k)ξ0 + τ
∞∑k=1
k∑i=1
|1− τ |k−iεi−1
≤
( ∞∑k=0
|1− τ |k)ξ0 + τ
( ∞∑k=0
|1− τ |k)( ∞∑
i=0
εi
)< +∞,
which completes the proof.
Now, we start to show that the sequence (xk, yk, zk) generated by Algorithm sGS-iPADMM can beviewed as a sequence generated by Algorithm iPALM from the same initial point. For this purpose, we definethe space V := Y × Range(G), and we define the linear operators B : X→ V and P : V→ V by
Bx :=(Fx;Gx
), ∀x ∈ X and P(y, z) :=
(Σfy ; 0
), ∀ (y, z) ∈ V. (4.19)
Moreover, we define the closed proper convex function φ : V→ (−∞,+∞] by
φ(v) = φ(y, z) := p(y1) + f(y)− 〈b, z〉, ∀ v = (y, z) ∈ V
and defineLσ (v; (x, v′)) := Lσ (y, z; (x, y′)) , ∀ v = (y, z) ∈ V, v′ = (y′, z′) ∈ V. (4.20)
Based on the above definitions, problem (1.1) can be viewed as an instance of problem (3.1). In this case, thefollowing result is for the purpose of viewing Algorithm sGS-iPADMM as an instance of Algorithm iPALM.
Theorem 4.1. Suppose that Assumptions 4.1 and 4.2 hold. Let (xk, yk, zk) be the sequence generated byAlgorithm sGS-iPADMM. Define
vk :=(yk; ΠRange(G)(z
k)), ∀ k ≥ 0. (4.21)
Then, for any k ≥ 0, it holds that
23
(a) the linear operators T , B and P defined in (4.10) and (4.19) satisfy
T − 12P and
⟨v,(
12P + σBB∗ + T
)v⟩> 0, ∀ v ∈ V \ 0; (4.22)
(b) there exists a sequence of nonnegative real numbers εk, such that
‖(∆k; γk)‖ ≤ εk and
∞∑k=0
εk < +∞;
(c) it holds that
vk+1 ≈ arg minv∈V
Lσ(v; (xk, vk)
)+
1
2‖v − vk‖2T
in the sense that(
∆k; γk)∈ ∂vLσ
(vk+1;
(xk, vk
))+ T
(vk+1 − vk
)and
∥∥(∆k; γk)∥∥ ≤ εk.
Proof. (a) According to (4.4) in Assumption 4.2 we know that D − 12 Σf . Moreover, from (4.8) we know
that NsGS 0. Thus, one can readily see from (4.10) and (4.19) that T − 12P . On the other hand, one can
symbolically do the decomposition that
1
2P + σBB∗ + T =
12 Σf + σFF∗ +D +NsGS + σFG∗[GG∗]†GF∗ σFG∗
σGF∗ σGG∗
.
From Lemma 4.2, we know that GG∗ is nonsingular on the Range(G). Therefore, by using the definition of Vand the Schur complement condition for ensuring the positive definiteness of a linear operator, we only needto show that 1
2 Σf + σFF∗ +D +NsGS 0 on Y. Suppose on the contrary that it is not positive definite.Then, there exists a nonzero vector y ∈ Y such that⟨
y,(
12 Σf + σFF∗ +D +NsGS
)y⟩
=⟨y,(
12 Σf +D + σFF∗
)y⟩
+ 〈y,NsGSy〉 = 0.
From (4.4) of Assumption 4.2 and (4.8) we know that 12 Σf +D + σFF∗ 0 and NsGS 0, so that⟨
y,(
12 Σf +D + σFF∗
)y⟩
= 0 = 〈y,NsGSy〉 .
Then, by using (4.8) we can get that N ∗uy = 0. This, together with (4.6), implies that
0 =⟨y,(
12 Σf +D + σFF∗
)y⟩
= 12
⟨y,(
Σf + σFF∗)y⟩
+⟨y,(
12σFF
∗ +D)y⟩
= 12
⟨y,(
Σf + σFF∗)dy⟩
+⟨y,(
12σFF
∗ +D)y⟩
=⟨y,(
12 (Σf )d +D
)y⟩
+ σ2 〈y, (FF
∗)dy〉+ σ2 〈y,FF
∗y〉 ,
(4.23)
where(Σf + σFF∗)d := Diag
(Σf11 + σF1F∗1 , . . . , Σfss + σFsF∗s
),
(FF∗)d := Diag(F1F∗1 , . . . ,FsF∗s
).
Since D − 12 Σf implies 1
2 (Σf )d +D 0, we obtain from (4.23) that⟨y,(
12 (Σf )d +D
)y⟩
= σ2 〈y, (FF
∗)d y〉 = σ2 〈y,FF
∗y〉 = 0,
24
which contradicts the requirement in Assumption 4.2 that 12 Σfii + σFiF∗i + Di 0 for all i = 1, . . . , s.
Therefore, it holds that 12 Σf + σFF∗ +D +NsGS 0, and this completes the proof of (a).
(b) From the definition of ∆k in (4.11) one can see that for all k ≥ 0,
‖∆k‖ ≤ ‖δksGS‖+ ‖FG∗[GG∗]†‖‖γk−1 − γk − G(xk−1 − xk)‖.
Then, by using the fact that max‖δki ‖, ‖δki ‖, ‖γk‖ ≤ εk, we can get from Proposition 4.2 and the definitionof δksGS in (4.7) that for all k ≥ 1,
‖(∆k; γk)‖ ≤ ‖γk‖+ ‖∆k‖
≤ εk := (s+ 1)εk + 2s‖NuN−1d ‖εk + ‖FG∗[GG∗]†‖
(εk−1 + ξk−1 + εk + ξk
).
Moreover, we define ε0 := ‖(∆0; γ0)‖. Then, according to Proposition 4.2 and the fact that the sequenceεk is summable, we know that
∑∞k=0 εk < +∞.
(c) According to (4.10), (4.13) and (4.20), we only need to show that
∂(y,z)Lσ((yk+1, zk+1
);(xk, yk
))= ∂(y,z)Lσ
((yk+1,ΠRange(G)(z
k+1))
;(xk, yk
)), ∀ k ≥ 0.
From (1.4) and (1.5) we can get that
∂yLσ(y, z; (x, y′)) =
(∂y1p(y1)
0
)+∇f(y′) + Σf (y − y′) + Fx+ σF(F∗y + G∗z − c)
and∇zLσ(y, z; (x, y′)) = −b+ Gx+ σG(F∗y + G∗z − c).
Therefore, by using the fact that G∗zk+1 = G∗ΠRange(G)(zk+1), ∀ k ≥ 0, we know that part (c) of the theorem
holds. This completes the proof.
Remark 4.2. One can see that in Algorithm sGS-iPADMM, the sequence (xk, yk, zk) was generated, whilethe sequence ΠRange(G)(z
k) has never been explicitly calculated. Note that once zk is computed, only the
vector G∗zk is needed during the next iteration, instead of zk itself. Since G∗zk = G∗ΠRange(G)(zk),∀ k ≥ 0,
one may view the sequence ΠRange(G)(zk) ∈ Range(G) as a shadow sequence of zk. It has never been
explicitly computed, but still plays an important role on establishing the convergence of the algorithm. In fact,similar observations have been made and extensively used in [30, 31].
By combining the results of Theorem 3.1 and Theorem 4.1, one can readily get the following convergencetheorem of Algorithm sGS-iPADMM.
Theorem 4.2. Suppose that Assumptions 4.1 and 4.2 hold. Let (xk, yk, zk) be the sequence generated byAlgorithm sGS-iPADMM. Then,
(a) the sequence(yk,ΠRange(G)(z
k))
converges to a solution to problem (1.1) and the sequence xkconverges to a solution to the dual of (1.1);
(b) any accumulation point of the sequence (yk, zk) is a solution to problem (1.1);
(c) the sequence p(yk1 ) + f(yk)− 〈b, zk〉 of the objective values converges to the optimal value of problem(1.1), and
limk→∞
(F∗yk + G∗zk − c) = 0;
(d) it holds with K being the solution set to the KKT system (4.2) that
limk→∞
dist((xk, yk, zk),K
)= 0;
25
(e) if the linear operator G is surjective, the whole sequence (xk, yk, zk) converges to a solution to theKKT system (4.2) of problem (1.1).
Proof. (a) Note that the sequencevk =
(yk; ΠRange(G)(z
k))
defined in (4.21) lies in Y × Range(G). By
using Theorem 4.1(c), one can treat the sequence (xk, vk) generated by Algorithm sGS-iPADMM as theone generated by Algorithm iPALM with the given initial point (x0, v0). In addition, (4.22) in Theorem 4.1guarantees that condition (3.5) in Assumption 3.1 holds. Thus, by Theorem 3.1, the sequence (xk, vk)converges to a solution to the KKT system (4.2), i.e., the sequences
(yk,ΠRange(G)(z
k))
and xk convergeto a solution to problem (1.1) and its dual, respectively.
(b) From (a), we see that limk→∞(xk, yk,ΠRange(G)(zk)) = (x∗, y∗, z∗) which is a solution to the KKT system
(4.2). Since G∗zk = G∗ΠRange(G)(zk),∀ k ≥ 1, any accumulation point, say z∞ of zk satisfies G∗z∞ = G∗z∗.
Then, it is easy to verify that (x∗, y∗, z∞) also satisfy the KKT system (4.2). Therefore, (y∗, z∞) is a solutionto problem (1.1).
(c) From (a) and the fact that the objective function of problem (1.1) is continuous on its domain, we knowthat p(yk1 ) +f(yk)−〈b,ΠRange(G)(z
k)〉 converges to the optimal value of problem (1.1). Since b ∈ Range(G),
it holds that for any k ≥ 1, 〈b, zk〉 = 〈b, ΠRange(G)(zk)〉. Thus,
p(yk1 ) + f(yk)− 〈b,ΠRange(G)(zk)〉 = p(yk1 ) + f(yk)− 〈b, zk〉, ∀k ≥ 1.
Therefore, the sequence p(yk1 ) + f(yk)− 〈b, zk〉 converges to the optimal value of problem (1.1). Meanwhile,since G∗zk = G∗ΠRange(G)(z
k), we further have that
limk→∞
(F∗yk + G∗zk − c
)= limk→∞
(F∗yk + G∗ΠRange(G)(z
k)− c)
= 0.
(d) From (a), we have that (x∗, y∗, z∗), the limit point of (xk, yk,ΠRange(G)(zk)), is a solution to the KKT
system (4.2), i.e., (x∗, y∗, z∗) ∈ K. Since G∗(zk −ΠRange(G)(z
k))
= 0 for any k ≥ 1, it is not difficult to seethat (
x∗, y∗, z∗ +(zk −ΠRange(G)(z
k)))∈ K, ∀k ≥ 1.
Therefore, it holds for all k ≥ 1
dist((xk, yk, zk),K
)≤ ‖xk − x∗‖+ ‖yk − y∗‖+ ‖ΠRange(G)(z
k)− z∗‖
and limk→∞ dist((xk, yk, zk),K
)= 0.
(e) In this case, it holds that Range(G) = Z and zk = ΠRange(G)(zk), ∀ k ≥ 0. The result follows from (a),
which completes the proof of the theorem.
We make the following remark on Theorem 4.2.
Remark 4.3. Without any additional assumptions on G, one can observe that the solution set of problem(1.1) is unbounded and the sequence zk generated by Algorithm sGS-iPADMM may also be unbounded.Fortunately, we are still able to show in Theorem 4.2(a) and (c) that the sequence
(xk, yk,ΠRange(G)(z
k))
converges to a solution to the KKT system (4.2), and both the objective and the feasibility converge to theoptimal value and zero, respectively. Meanwhile, we would like to emphasize that the surjectivity assumptionon G in Theorem 4.2(e) is not restrictive at all. Indeed, this assumption simply means that there are noredundant equations in the linear constraints Gx = b in the primal problem (1.2). If necessary, well establishednumerical linear algebra techniques can be used to remove redundant equations from Gx = b.
4.1 The Two-Block Case
Consider the two-block case that Y = Y1 and f is vacuous, i.e., the following problem
miny,zp(y)− 〈b, z〉 | F∗y + G∗z = c . (4.24)
26
Table 1: Comparison between [18] and this paper. In the table ‘SOL’ denotes the solution set to problem(4.24), ‘X’ denotes the set of multipliers (the solution set to the dual problem) to problem (4.24), and ‘K’denotes the solution set to the KKT system or problem (4.24), i.e., K = X× SOL. The symbol → meansthat the sequence on its left-hand-side is convergent, and converges to a point in its right-hand-side.
Item \ Ref [18] This paper
Updating rules z ⇒ y ⇒ x & τ ∈ (0, 2) y ⇒ z ⇒ x & τ ∈ (0, 2)
Assumptions-yp strongly convex p strongly convex p strongly convex
and F the identity operator or F surjective or F surjective
Assumptions-z G surjective - G surjective
Sequences(yk, zk) → SOL dist
((xk, yk, zk),K
)→ 0 (yk, zk) → SOL
xk bounded xk → X xk → X
Assume that the KKT system of problem (4.24) admits a nonempty solution set K. For such a two-blockproblem, Algorithm sGS-iPADMM without the proximal terms and the inexact computations reduces tothe classic ADMM. Then, by Theorem 4.2, the sequence
(xk, yk,ΠRange (G)(z
k))
generated by the classicADMM or its inexact variants with τ ∈ (0, 2) (in the order that the y-subproblem is solved before thez-subproblem) converges to a point in K if either F is surjective or p is strongly convex. Moreover, if G is alsosurjective, we have that the sequence
(xk, yk, zk
)converges to a point in K. Note that the assumptions
we made for problem (4.24) are apparently weaker than those in [18], where F is assumed to be the identityoperator, G is surjective, and p is assumed to be strongly convex. Moreover, in [18, Theorem 3.1], only theconvergence of the primal sequence (yk, zk) and the boundedness of the dual sequence xk were obtained.
The detailed comparison between the results in this paper and those in [18] is presented in Table 1. Ascan be observed from this table, the convergence result on the dual sequence xk is easier to be derivedthan that of the primal sequence (yk, zk), and this result is consistent with the results in [5] for the classicADMM and the ALM in [45]. Hence, the results derived in this paper properly resolves the questions wehave mentioned in the introduction.
At last, we should mention that, in Sun et al. [48, Theorem 3.3 (iv)], a similar result to ours has beenderived with the requirements that the initial multiplier x0 satisfies Gx0 − b = 0 and all the subproblems aresolved exactly. Here, we are able to relax these requirements to the most general case and extend our resultsto the more interesting and challenging multi-block problems.
4.2 Linear Rate of Convergence and Iteration Complexity
Theorem 3.2 has provided a tool, which can be used together with Theorem 4.1 to analyze the linearconvergence rate of the sequence generated by Algorithm sGS-iPADMM, i.e., one only need to verify whether(3.30) is valid for this sequence, provided that the metric subregular property (3.26) holds. However, such averification is not as straightforward as it conceptually seems.
Here, we establish a linear convergence result for the case that the linear system in step 3 of AlgorithmsGS-iPADMM is solve exactly, but leave the general cases as a topic for further study. For this purpose, we
27
view problem (1.1) as an instance of (3.1) withϕ(w) := p(y1),
h(w) := f(y)− 〈b, z〉,
A∗w := F∗y + G∗z,
∀w = (y, z) ∈W := Y ×Z. (4.25)
Then, the corresponding KKT residual mapping of problem (1.1) can be given by (3.22). Moreover, the
self-adjoint linear operator Ω defined in (3.16) is given by Ω(x; (y; z)) = (x; Θ12 (y; z)), where Θ = τσ(P + T +
(2−τ)σ3 AA∗) with T and P being defined in (4.10) and (4.19), respectively. In fact, we further have that
Ω(x; (y; z)
)= Ω
(x;(y; ΠRange (G)(z)
)), ∀(x, y, z) ∈ X×Y ×Z. (4.26)
Theorem 4.3. Suppose that Assumptions 4.1 and 4.2 hold. Let uk = (xk, wk) with wk := (yk; zk) bethe sequence generated by Algorithm sGS-iPADMM such that vk :=
(xk, yk,ΠRange (G)(z
k)) converges to
v∗ ∈ K. It holds thatdistΩ(uk,K) = distΩ(vk,K), ∀k ≥ 0. (4.27)
Suppose that b − Gx0 = 0 and γk = 0 for all k ≥ 0. Suppose that the KKT residual mapping R definedin (3.22) (with the notation in (4.25)) is metric subregular at v∗ for 0 ∈ U with the modulus κ > 0, inthe sense that there exists a constant r > 0 such that (3.26) holds with u = v∗. Let ηk be a givensequence of nonnegative numbers that converges to 0 in the limit. Suppose that in addition to satisfyingmax‖δki ‖, ‖δki ‖ | i = 1, . . . , s ≤ εk, there exists an integer k0 > 0 such that for any k ≥ k0, it holds that
max1≤i≤s
‖δki ‖, ‖δki ‖
≤ ηk‖vk − vk+1‖. (4.28)
Then, for all k sufficiently large, it holds that distΩ(uk+1,K) ≤ ϑk distΩ(uk,K) with supk≥k0ϑk < 1, i.e., the
convergence rate of distΩ(uk,K) is Q-linear when k is sufficiently large.
Proof. By (4.26), we have that for all k ≥ 0,
dist2Ω(uk,K) = inf
u∈K
1
2〈uk − u, Ω(uk − u)〉 = inf
u∈K
1
2〈uk − u, Ω(vk)− Ω(u)〉
= infu∈K
1
2〈vk − u, Ω(vk − u)〉 = dist2
Ω(vk,K),
i.e., (4.27) holds. Since b− Gx0 = 0 and γk = 0 for all k ≥ 0, according to (4.18) one has that
‖b− Gxk‖ ≤ |1− τ |k ‖b− Gx0‖+ τ
k∑i=1
|1− τ |k−i ‖γi−1‖ = 0, ∀k ≥ 0.
Therefore, by (4.7) and (4.11) one knows that
∆k := δksGS −FG∗(GG∗)−1G(xk − xk−1) = δk +NuN−1d (δk − δk).
Thus, we can get that for all k ≥ 0,
‖dk‖ = ‖∆k‖ ≤ (1 + 2‖NuN−1d ‖) max‖δ‖, ‖δ‖
≤√s(1 + 2‖NuN−1
d ‖)ηk‖vk − vk+1‖,
where dk := (∆k; γk) ∈ W. Define ηk =√s(1 + 2‖NuN−1
d ‖)ηk. Then, it holds that ηk → 0 and ‖dk‖ ≤ηk‖vk − vk+1‖. Therefore, by Theorem 3.2, we know that for all k sufficiently large
distΩ(vk,K) ≤ ϑk distΩ(vk,K)
28
with supk≥k0ϑk < 1, which, together with (4.27), implies
distΩ(uk,K) ≤ ϑk distΩ(uk,K) for all k sufficiently large.
This completes the proof.
Remark 4.4. Note that, different from the condition (3.30) in Assumption 3.2, the condition (4.28) here isgenerally not directly verifiable during the numerical implementation. However, Theorem 4.3 does provide us avery important theoretical guideline on implementing Algorithm sGS-iPADMM, i.e., in the k-th iteration, it islikely to be beneficial to solve the subproblems to an accuracy higher than the dual feasibility ‖F∗yk+G∗zk−c‖.In fact, this phenomenon has already been observed during our numerical experiments. We should alsomention that even for the 2-block case, the study on the linear convergence of inexact ADMMs with shorter
step-length τ ∈(0, 1+
√5
2
)is still not as mature as the study for their exact counterparts, especially when
compared with the recently developed results, e.g., in [22, 57]. Suitable criteria that generalize the condition(3.30) for terminating the subproblems are still lacking. We note that the results presented in Theorem 4.3are still far from complete, and more effort should be spent on this part in the future.
Finally, different from the above discussions on the convergence rate, we can establish the following non-ergodic iteration complexity for the sequence generated by Algorithm sGS-iPADMM by a direct applicationof Theorem 4.1.
Theorem 4.4. Suppose that Assumptions 4.2 and 4.1 hold. Let uk = (xk, wk) with wk := (yk; zk) bethe sequence generated by Algorithm sGS-iPADMM such that vk :=
(xk, yk,ΠRange (G)(z
k)) converges to
v∗ ∈ K. It holds that the KKT residual (3.22), with B and P given by (4.19), satisfies
min0≤j≤k
‖R(uj)‖2 ≤ %/k, and limk→∞
k · min0≤j≤k
‖R(uj)‖2 = 0,
where the constant % is defined as in (3.43) but with
e := ‖u0e‖2Ω + 2τσ‖Θ−1/2‖
(∑∞j=0 εj
)(‖u0
e‖Ω + µ∑∞j=0 εj
)+ 4
(∑∞j=1 ε
2j
)and u0
e = u0 − v∗.
Proof. From (4.26), we know that
‖u0 − v∗‖2Ω = 〈Ω(u0 − v∗), u0 − v∗〉 = 〈Ω(v0)− Ω(v∗), u0 − v∗〉 = ‖v0 − v∗‖2Ω. (4.29)
According to (3.22), (4.2) and (4.25), one has that
R(u) =
c−F∗y + G∗zy − ProxP (y −∇f(y)−Fx)
Gx− b
, ∀u = (x, y, z) ∈ X×Y ×Z.
Since for all k ≥ 0, G∗zk = GΠRange(G∗)(zk), one has that R(uk) = R(vk). Therefore, by using (4.22) in
Theorem 4.1(a), Theorem 3.3 and (4.29), one has the results of this theorem holds. This completes theproof.
5 Numerical Experiments
In this section, we conduct numerical experiments on solving dual linear SDP and dual convex quadraticSDP problems via Algorithm sGS-iPADMM with the dual step-length τ taking values beyond the standardrestriction of (1 +
√5)/2. For linear SDP problems, the algorithm reduces to the two-block ADMM, and the
29
aim is two-fold. On the one hand, as the ADMM is among the most important first-order algorithms forsolving SDP problems, it is of importance to know to what extent can the numerical efficiency be improved ifthe observation on the dual step-length made in this paper is incorporated. On the other hand, as the upperbound of the step-length has been enlarged, it is also important to see whether a step-length that is veryclose to the upper bound will lead to better or worse numerical performance.
A standard linear SDP problem has the following form:
minX〈C,X〉 | AX = b, X ∈ Sn+ (5.1)
and its corresponding dual is given as in (2.4). To avoid repetition, we refer the reader to (2.4) for thenotation used. The (majorized) augmented Lagrangian function associated with problem (2.4) is given by
Lσ(S, z;X) = δSn+(S)− 〈b, z〉+ 〈X,S + A∗z− C〉+ σ2 ‖S + A∗z− C‖2,
∀(S, z, X) ∈ Sn ×Rm × Sn,
where σ > 0 is the given penalty parameter. When applied to solving problem problem (2.4), at the k-thstep of the two-block ADMM the following steps are performed:
Sk+1 = ΠSn+(C −A∗zk −Xk/σ),
zk+1 = (AA∗)−1(A(C − Sk+1)− (AXk − b)/σ
),
Xk+1 = Xk + τσ(Sk+1 + A∗zk+1 − C),
where the step-length τ is allowed to be in the range (0, 2) based on Theorem 4.1 and the discussions inSection 4.1. We emphasize again that this is in contrast to the usual interval of (0, (1 +
√5)/2) allowed by
the convergence analysis of Glowinski in [20, Theorem 5.1].On the other hand, as was briefly introduced in Section 2.1, the convex QSDP problem was formally
given in (2.1), whose dual problem, in minimization form, is a multi-block problem given by
minS,W,zE ,zI
δSn+(S) + δRmI+(zI) + 1
2 〈W,QW 〉 − 〈bE , zE〉 − 〈bI , zI〉
s.t. S −QW + A∗EzE + A∗IzI + C = 0.(5.2)
Note that problem (2.1) was subsumed as an instance of the convex quadratic composition optimizationproblem (1.7). Therefore, to fit the framework of Algorithm sGS-iPADMM, we write the dual of problem(2.1) in the minimization form as
minS,W,s,zE ,zI
δSn+(S) + δRmI+(s) + 1
2 〈W,QW 〉 − 〈bE , zE〉 − 〈bI , zI〉
s.t.
S −QW + A∗EzE + A∗IzI + C = 0,
D(s− zI) = 0,
(5.3)
where D ∈ RmI×mI is a given positive definite diagonal matrix which is incorporated here for for the purposeof scaling the variables to ensure the numerical stability.
The convex QSDP problem (2.1) is solved via its dual (5.3), whose (majorized) augmented Lagrangianfunction is defined by
Lσ(S,W, zE , zI , s;X,x) :=(δSn+(S) + δRmI+
(s))
+ 12 〈W, QW 〉 − 〈bE , zE〉 − 〈bI , zI〉
+〈X,S −QW + A∗EzE + A∗IzI + C〉+ 〈D(s− zI),x〉
+σ2 ‖S −QW + A∗EzE + A∗IzI + C‖2 + σ
2 ‖D(s− zI)‖2,
∀(S,W, zE , zI , s;X,x) ∈ Sn × Sn ×RmE ×RmI ×RmI × Sn ×RmI .
30
where σ > 0 is the given penalty parameter and and we have used X ∈ Sn and x ∈ RmI to denote theLagrange multipliers which are introduced for the two groups of equality constraints in (5.3). During the k-thiteration of Algorithm sGS-iPADMM with given (Sk,W k, zkE , z
kI , s
k) and (Xk,xk), we update the variablesin the order of(
zk+1/2E ⇒W k+1/2︸ ︷︷ ︸backward GS
⇒ (Sk+1, sk+1)⇒W k+1 ⇒ zk+1E︸ ︷︷ ︸
forward GS
)⇒ zk+1
I ⇒(Xk+1,xk+1
)︸ ︷︷ ︸
τ∈(0,2)
.
Note that the term 〈bI , zI〉 is treated as the linear term in the framework of (1.1). We made this choicebecause for the test instances that we will consider later, the linear system that must be solved to update zIis much larger than that for updating zE , and in this way, the larger linear system will be solved only once ineach iteration.
The numerical results in the subsequent two subsections are obtained by using Matlab R2017b on aHP Elitedesk (64-bit Windows 10 system) with one Intel Core i7-4770S Processor (4 Cores, 3.1− 3.9 GHz)and 16 GB RAM (with the virtual memory turned off).
5.1 Numerical Results on Linear SDP Problems
Based on the first-order optimality condition for problem (5.1), we terminate all the tested algorithms if
ηSDP := maxηD, ηP , ηS ≤ 10−6,
where ηD =
‖A∗z + S − C‖1 + ‖C‖
, ηP =‖AX − b‖
1 + ‖b‖,
ηS = max
‖X −ΠSn+(X)‖
1 + ‖X‖,
|〈X,S〉|1 + ‖X‖+ ‖S‖
with the maximum number of iterations set at 100, 000. In addition, we also measure the duality gap:
ηgap :=〈C,X〉 − 〈b, z〉
1 + |〈C,X〉|+ |〈b, z〉|.
During our preliminary tests, we found that using a step-length smaller than 1 is not as good as using the unitstep-length. Therefore, we shall only consider the cases that τ ≥ 1. Note that the known theoretical upper
bound of the step-length τ in the classic ADMM for solving general convex programming is 1+√
52 ≈ 1.618034.
Although it has been observed empirically that the ADMM with the step-length τ = 1.618 works quite well,this phenomenon still requires further understanding since the value 1.618 is quite close to the theoreticalupper bound and such an aggressive choice may result in unstable numerical performance. Fortunately, theabove concern is partially alleviated by the theoretical results obtained in this paper. Indeed, for a large classconvex optimization problems, one can use τ = 1.618 confidently since it has a “safe” distance to the renewedtheoretical upper bound of 2. For this class of problems, it is thus very interesting to see what would happenif the step-length τ is very close to 2. Therefore, we tested five choices of the step-length, i.e., τ = 1, 1.618,1.90, 1.99 and 1.999. For convenience, we use ADMM(τ) to denote the algorithm with the specific step-lengthτ .
We tested 6 categories of linear SDP problems, including the random sparse SDP problems tested in[34], the semidefinite relaxation of frequency assignment problems (FAP) [15], the relaxation of maximumstable set problems [50, 52, 47], the SDP relaxation of binary integer quadratic (BIQ) problems from [54],the SDP relaxation of rank-1 tensor approximations (R1TA) [38, 39], and the SDP relaxations of clusteringproblems [40]. One may refer to [58, 56] for detailed descriptions and the data sources of these problems.All these algorithms are tested by running the Matlab package SDPNAL+ (version 1.0, available athttp://www.math.nus.edu.sg/~mattohkc/SDPNALplus.html). The records of the computational results
31
0 0.2 0.4 0.6 0.8 1(100y)% of the problems
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Itera
tion
num
ber
ratio
(co
mpa
red
to A
DM
M(1
))
Efficiency: ratio of iteration numbers
ADMM(1.618)ADMM(1.90)ADMM(1.99)ADMM(1.999)
0 0.2 0.4 0.6 0.8 1(100y)% of the problems
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Itera
tion
num
ber
ratio
(co
mpa
red
to A
DM
M(1
.618
))
Efficiency: ratio of iteration numbers
ADMM(1)ADMM(1.90)ADMM(1.99)ADMM(1.999)
Figure 1: Comparison of the computational efficiency of the classic two-block ADMM with different step-lengths
are provided in Table 3. Here, we should mention that even though all the problems we tested have beensuccessfully solved by at least one of the tested algorithms, there are a few categories of SDP problems thatare beyond the capability of the ADMM, see, e.g., [58].
Figure 1 presents the computational performance of the ADMM with all the five choices of step-lengths.The left panel shows the comparison between ADMM(1) and all the other algorithms, while the right panelshows the comparison between ADMM(1.618) and all the others. As can be seen from Figure 1, ADMM(1.618)has an impressive improvement over ADMM(1) and ADMM(1.9) works even better than ADMM(1.618) formore than 80% of the tested instances. Furthermore, ADMM(1.99) can perform marginally better thanADMM(1.9) for about 60% of the tested instances but for about 10% of them, its performance is apparentlyworse. However, ADMM(1.999) has a significantly worse performance than ADMM(1.99) even though itsstep-length is just slightly larger than 1.99. This can be partially explained by the fact that the step-lengthof 1.999 is too close to the theoretical upper bound of 2.
From both the theoretical analysis and numerical experiments in this paper, one can see that in generalit is a good idea to use a step-length that is larger than 1, e.g., τ = 1.618, when solving linear SDP problems.Meanwhile, we can even set the step-length to be larger than 1.618, say τ = 1.9, to get even better numericalperformance.
32
5.2 Numerical Results on Convex QSDP Problems
The KKT system of problem (5.2) is given byS −QW + A∗EzE + A∗IzI + C = 0, AEX − bE = 0,
QX −QW = 0, X ∈ Sn+, S ∈ Sn+, 〈X,S〉 = 0,
AIX − bI ≥ 0, zI ≥ 0, 〈AIX − b, zI〉 = 0.
(5.4)
Based on the optimality conditions given in (5.4), we measure the accuracy of an approximate (computed)solution (X,Z,W, S, yE , yI) for the convex QSDP (2.1) and its dual (5.2) via
ηqsdp = max ηD, ηP , ηW , ηS , ηI , (5.5)
where
ηD =‖S −QW + A∗EzE + A∗IzI + C‖
1 + ‖C‖, ηP =
‖AEX − bE‖1 + ‖bE‖
,
ηW =‖QX −QW‖
1 + ‖Q‖, ηS = max
‖X −ΠSn+(X)‖
1 + ‖X‖,
|〈X,S〉|1 + ‖X‖+ ‖S‖
,
ηI = max
‖min(0, zI)‖
1 + ‖zI‖,‖min(0, AIX − bI)‖
1 + ‖bI‖,|〈AIX − bI , zI〉|
1 + ‖AIx− bI‖+ ‖zI‖
.
Additionally, we measure the objective values and the duality gap:
ηgap :=Objprimal −Objdual
1 + |Objprimal|+ |Objdual|,
where
Objprimal :=1
2〈X,QX〉 − 〈C,X〉, and Objdual := −1
2〈W, QW 〉+ 〈bE , zE〉+ 〈bI , zI〉.
In our numerical experiments, similar to [6], we used QSDP test instances based on the SDP problems arisingfrom the relaxation of the binary integer quadratic (BIQ) programming with a large number of inequalityconstraints that was introduced by Sun et al. [48] for getting tighter bounds. The problems that we actuallysolve have the following form:
min1
2〈X, QX〉+
1
2〈Q, X〉+ 〈c, x〉
s.t.
diag(X)− x = 0, X =
(X xxT 1
)∈ Sn+,
xi −Xij ≥ 0, xj −Xij ≥ 0, Xij − xi − xj ≥ −1, ∀ 1 ≤ i < j ≤ n− 1.
The data for Q and c are taken from the Biq Mac Library maintained by Wiegele [54].We solve the QSDP (2.1) via its dual (5.3) with the matrix D = (
√‖AI‖/2)IRmI . We use the directly
extended multi-block ADMM with step-length τ = 1 as the benchmark (which we named as ‘DirectlyExtended’), and compare it with Algorithm sGS-iPADMM, which was implemented in 6 different ways, i.e.,2 groups of algorithms with each using 3 types of different step-lengths, i.e., τ = 1, 1.618 and 1.9, whichare chosen according to the numerical results in Section 5.1. For convenience, we use the name ‘sGS-PADMM’to mean that Algorithm sGS-iPADMM is implemented such that all the subproblems are solved exactlyvia direct solvers or adding appropriate proximal terms, and use ‘sGS-iPADMM’ to mean that AlgorithmsGS-iPADMM is implemented such that the subproblems are allowed to be solved inexactly via iterativesolvers. The details of all the seven tested algorithms are presented in Table 2.
33
Table 2: 7 types of algorithms tested for the convex QSDP problems. In the table, ‘Dir’ denotes thecorresponding subproblems are solved via direct solvers, ‘Proj.’ means that the corresponding subproblemsare the calculation of projections, ‘Prox.’ means that the corresponding subproblems are solved via addingappropriate proximal terms to ensure closed-form solutions, ‘Inex.’ means that the subproblems are solvedapproximately via an iterative scheme, ‘(rep.)’ means that the corresponding subproblems in the forwardsGS sweeps are also solved, ‘(check)’ means that the corresponding subproblems in the forward sGS sweepsare not directly solved but the most recently updated variables are checked to see if they are admissibleapproximate solution to the subproblems.
Algorithm \ Variable τ zE W S s zI
Directly Extended 1 Dir. Prox. Proj. Proj. Prox.
sGS-PADMM
1 Dir.(rep.) Prox.(rep.) Proj. Proj. Prox.
1.618 Dir.(rep.) Prox.(rep.) Proj. Proj. Prox.
1.9 Dir.(rep.) Prox.(rep.) Proj. Proj. Prox.
sGS-iPADMM
1 Dir.(check) Inex.(check) Proj. Proj. Inex.
1.618 Dir.(check) Inex.(check) Proj. Proj. Inex.
1.9 Dir.(check) Inex.(check) Proj. Proj. Inex.
For all the algorithms applied to problem (5.3), the subproblems corresponding to the block variable (S, s)can be solved analytically by computing the projections onto Sn+ ×R
mI+ . For the subproblems corresponding
to zE , linear systems of equations must be solved with the same coefficient matrix AEA∗E . As the linearsystems are not too large, we solve them via the Cholesky factorization (computed only once) of AEA∗E . Forthe subproblems corresponding to zI and W , we need to solve very large scale linear systems of equations,so that they are either solved via a preconditioned conjugate gradient method with preconditioners thatare described in [6, Section 7.1] (for sGS-iPADMM), or solved directly by adding an appropriate proximalterm to the subproblems to get closed-form solutions (for sGS-PADMM). Moreover, in the implementationof the sGS-PADMM, all the subproblems in the forward Gauss-Seidel sweep are directly solved, while in theimplementation of sGS-iPADMM we used the strategy described in [6, Remark 4.1(b)] to decide whether thecomputation of the subproblems in the forward GS sweep can be skipped (see [6, Section 7.2] for more detailson using this technique). We used a similar strategy as described in [27, Section 4.4] to adaptively adjustthe penalty parameter σ and used the same technique as in [6] to control the error tolerance for solving thesubproblems, i.e., εkk≥0 is chosen such that αεk ≤ 1/k1.2, where α > 0 is a positive constant based on theproblem data.
We have tested 147 instances of convex QSDP problems with n ranging from 51 to 501. The linearoperator Q was chosen as the symmetrized Kronecker operator Q(X) = 1
2 (AXB + BXA) with A andB being two randomly generated symmetric positive semidefinite matrices such that rank(A) = 10 andrank(B) ≈ n/5, respectively, as was used in [51, 6]. The maximum iteration number is set at 500, 000. Thedetail computational results are provided in Table 4.
34
1 1.5 2 2.5 3 3.5 4 4.5 5at most x times of the best
0
0.2
0.4
0.6
0.8
1(1
00y)
% o
f the
pro
blem
s
Performance profile: time
Directly ExtendedsGS-PADMM-1sGS-PADMM-1.618sGS-PADMM-1.9sGS-iPADMM-1sGS-iPADMM-1.618sGS-iPADMM-1.9
Figure 2: Comparison of the computational efficiency of the 7 algorithms
Figure 2 shows the numerical performance of the 7 tested algorithms described in Table 2 on solvingthe convex QSDP problems to the accuracy of 10−6 in terms of ηqsdp in (5.5). One can readily see from thefigure that sGS-iPADMM overwhelmingly outperforms sGS-PADMM, no matter which step-length τ was used.This evidently shows the considerable advantage of catering for approximate solutions in the subproblems ofAlgorithm sGS-iPADMM. Moreover, for both sGS-PADMM and sGS-iPADMM, the step-length τ = 1.618 is ableto bring a noticeable improvement on the numerical efficiency, compared with using the unit step-length.Meanwhile, the choice of τ = 1.9 can perform even better in general. Even this is more apparent forsGS-PADMM, in which all the subproblems are solved exactly. We can see that sGS-iPADMM with τ = 1.9performs the best among all the tested algorithms for almost 65% of all the tested problems. Hence, thenumerical results clearly demonstrate the merit of using a larger step-length and the flexibility of inexactlysolving the subproblems.
6 Conclusions
In this paper, we have shown that, for a class of convex composite programming problems, the sequencegenerated by an inexact sGS decomposition based multi-block majorized (proximal) ADMM is equivalent to
35
the sequence generated by an inexact proximal ALM starting with the same initial point. The convergence ofthe inexact majorized proximal ALM was first established, and the convergence of the multi-block ADMM-typealgorithm follows readily because of the newly discovered equivalence. As a consequence of this equivalence,we are able to provide a very general answer to the open question on whether the whole sequence generated bythe classic ADMM with τ ∈ (0, 2) for a conventional two-block problem with one part of its objective functionbeing linear, is convergent. Numerical experiments on solving a large number of linear and convex quadraticSDP problems are conducted. The numerical results show that one can achieve even better numericalperformance of the ADMM if the step-length is chosen to be larger than the conventional upper bound of(1 +
√5)/2, and one can get a considerable improvement by allowing inexact subproblems together with
the large step-lengths on the multi-block ADMM for convex quadratic SDP problems. We hope that ourtheoretical analysis and numerical results can inspire more insightful studies on the ADMM-type algorithms.
Acknowledgments
We would like to thank the two anonymous referees for their careful reading of this paper, and their insightfulcomments and suggestions which have helped to improve the quality of this paper.
References
[1] Bai, M., Zhang, X., Ni, G. and Cui, C.: An adaptive correction approach for tensor completion. SIAM J. ImagingSci., 9, 1298–1323 (2016)
[2] Bai, S. and Qi, H.-D.: Tackling the flip ambiguity in wireless sensor network localization and beyond. Digit.Signal Process. 55, 85–97 (2016)
[3] Bertsekas, D.P. and Tsitsiklis, J.N.: Parallel and Distributed Computation: Numerical Methods. Athena Scientific,Belmont, Massachusetts (1997)
[4] Boyd, S., Parikh, N., Chu, E., Peleato, B. and Eckstein. J.: Distributed optimization and statistical learning viathe alternating direction method of multipliers, Found. Trends Mach. Learn. 3, 1–122 (2011)
[5] Chen, L., Sun, D.F. and Toh, K.-C.: A note on the convergence of ADMM for linearly constrained convexoptimization problems. Comput. Optim. Appl. 66, 327-343 (2017)
[6] Chen, L., Sun, D.F. and Toh, K.-C.: An effcient inexact symmetric Gauss-Seidel based majorized ADMM forhigh-dimensional convex composite conic programming. Math. Program. 161(1-2), 237–270 (2017)
[7] Chen, S.S., Donoho, D.L. and Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Rev. 43, 129–159(2001)
[8] Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)
[9] Ding, C. and Qi, H.-D.: Convex optimization learning of faithful Euclidean distance representations in nonlineardimensionality reduction. Math. Program. 164(1-2), 341–381 (2017)
[10] Ding, C., Sun, D.F., Sun, J. and Toh K.-C.: Spectral operators of matrices. Math. Program. 168, 509–531 (2018)
[11] Dontchev, A.L. and Rockafellar, R.T.: Implicit Functions and Solution Mappings, Second Edition. Springer, NewYork (2014)
[12] Du, M.Y.: A Two-Phase Augmented Lagrangian Method for Convex Composite Quadratic Programming. PhDthesis, Department of Mathematics, National University of Singapore (2015)
[13] Eckstein, J.: Augmented Lagrangian and alternating direction methods for convex optimization: A tutorial andsome illustrative computational results. RUTCOR Research Reports (2012)
[14] Eckstein, J. and Yao, W.: Understanding the convergence of the alternating direction method of multipliers:Theoretical and computational perspectives. Pac. J. Optim. 11, 619–644 (2015)
[15] Eisenblatter, A., Grotschel, M. and Koster, A.: Frequency planning and ramification of coloring. Discuss. Math.Graph Theory 22, 51–88 (2002)
[16] Fazel, M., Pong, T.K., Sun D.F. and Tseng, P.: Hankel matrix rank minimization with applications to systemidentification and realization. SIAM J. Matrix Anal. 34(3), 946–977 (2013)
36
[17] Ferreira, J., Khoo, Y. and Singer, A.: Semidefinite programming approach for the quadratic assignment problemwith a sparse graph. Comput. Optim. Appl. 69(3), 677–712 (2018)
[18] Gabay, D. and Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite elementapproximation. Comput. Math. Appl. 2(1), 17–40 (1976)
[19] Gaines, B.R., Kim, J. and Zhou, H.: Algorithms for fitting the constrained lasso, J. Comput. Graph. Stat. 27(4),861–871 (2018)
[20] Glowinski, R.: Lectures on Numerical Methods for Non-Linear Variational Problems. Published for the TataInstitute of Fundamental Research, Bombay. Springer-Verlag (1980)
[21] Glowinski, R. and Marroco, A.: Sur l’approximation, par elements finis d’ordre un, et la resolution, par penalisation-dualite d’une classe de problemes de Dirichlet non lineaires. Revue francaise d’atomatique, Informatique RechercheOperationelle. Analyse Numerique 9(2), 41–76 (1975)
[22] Han, D.R., Sun, D.R. and Zhang, L.W.: Linear rate convergence of the alternating direction method of multipliersfor convex composite programming. Math. Oper. Res. 43(2), 622–637 (2018)
[23] Hestenes, M.: Multiplier and gradient methods. J. Optim. Theory Appl. 4(5), 303–320 (1969)
[24] Huber, P.J.: Robust estimation of a location parameter. Ann. Math. Stat. 35, 73–101 (1964)
[25] James, G.M., Paulson, C. and Rusmevichientong, P.: Penalized and constrained regression. UnpublishedManuscript, the latest version is available at http://www-bcf.usc.edu/~gareth/research/Research.html
(2013)
[26] Klopp, O.: Noisy low-rank matrix completion with general sampling distribution. Bernoulli 20(1), 282–303 (2014)
[27] Lam, X.Y., Marron, J.S., Sun, D.F. and Toh, K.-C.: Fast algorithms for large scale generalized distance weighteddiscrimination. J. Comput. Graph. Stat. 27(2), 368–379 (2018)
[28] Lemarechal, C. and Sagastizabal, C.: Practical aspects of the Moreau-Yosida regularization: theoreticalpreliminaries. SIAM J. Optim. 7(2), 367–385 (1997)
[29] Li, M., Sun, D.F. and Toh, K.-C.: A majorized ADMM with indefinite proximal terms for linearly constrainedconvex composite optimization. SIAM J. Optim. 26(2), 922–950 (2016)
[30] Li, X.D., Sun, D.F. and Toh, K.-C.: A Schur complement based semi-proximal ADMM for convex quadraticconic programming and extensions. Math. Program. 155, 333–373 (2016)
[31] Li, X.D., Sun, D.F. and Toh, K.-C.: QSDPNAL: A two-phase augmented Lagrangian method for convex quadraticsemidefinite programming. Math. Program. Comput. 10(4), 703–743 (2018)
[32] Li, X.D., Sun, D.F. and Toh, K.-C.: A block symmetric Gauss-Seidel decomposition theorem for convex compositequadratic programming and its applications. Math. Program. DOI:10.1007/s10107-018-1247-7 (2018)
[33] Liu, J., Musialski, P., Wonka P. and Ye, J.: Tensor completion for estimating missing values in visual data. IEEETrans. Pattern Anal. Mach. Intell. 35, 208–220 (2013)
[34] Malick, J., Povh, J., Rendl, F. and Wiegele, A.: Regularization methods for semidefinite programming. SIAM J.Optim. 20, 336–356 (2009)
[35] Mateos G., Bazerque, J.-A. and Giannakis G.B.: Distributed sparse linear regression. IEEE Trans. Signal Proces.58, 5262–5276 (2010)
[36] Miao, W.M., Pan, S.H. and Sun, D.F.: A rank-corrected procedure for matrix completion with fixed basiscoefficients. Math. Program. 159, 289–338 (2016)
[37] Negahban, S. and Wainwright, M.J.: Restricted strong convexity and weighted matrix completion: optimalbounds with noise. J. Mach. Learn. Res. 13, 1665–1697 (2012)
[38] Nie, J. and Wang, L.: Regularization methods for SDP relaxations in large-scale polynomial optimization. SIAMJ. Optim. 22, 408–428 (2012)
[39] Nie, J. and Wang, L.: Semidefinite relaxations for best rank-1 tensor approximations. SIAM J. Matrix Anal.Appl. 35, 1155–1179 (2014)
[40] Peng, J. and Wei, Y.: Approximating k-means-type clustering via semidefinite programming. SIAM J. Optim.18, 186–205 (2007)
[41] Potra F.A.: Weighted complementarity problems–a new paradigm for computing equilibria. SIAM J. Optim. 22,1634–1654 (2012)
37
[42] Powell, M.: A method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization,pp. 283–298. Academic, New York (1969)
[43] Povh, J., Rendl, F. and Wiegele, A.: A boundary point method to solve semidefinite programs. Computing 78,277–286 (2006)
[44] Rockafellar, R.T.: Convex Analysis. Princeton University Press (1970)
[45] Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex program-ming. Math. Oper. Res. 1, 97–116 (1976)
[46] Schizas, I.D., Ribeiro A. and Giannakis G.B.: Consensus in ad hoc WSNs with noisy links - Part I: distributedestimation of deterministic signals. IEEE Trans. Signal Process. 56, 350–364 (2008)
[47] Sloane, N.: Challenge Problems: Independent Sets in Graphs, http://www.research.att.com/~njas/doc/
graphs.html
[48] Sun, D.F., Toh, K.-C. and Yang, L.Q.: A convergent 3-block semi-proximal alternating direction method ofmultipliers for conic programming with 4-type constraints. SIAM J. Optim. 25(2), 882–915 (2015)
[49] Teo, C.H., Vishwanathan, S.V.N., Smola A. and V.Le, Q.: Bundle methods for regularized risk minimization. J.Mach. Learn. Res. 11, 313-365 (2010)
[50] Toh, K.-C.: Solving large scale semidefinite programs via an iterative solver on the augmented systems. SIAM J.Optim. 14, 670-698 (2004)
[51] Toh, K.-C.: An inexact primal-dual path-following algorithm for convex quadratic SDP. Math. Program. 112(1),221–254 (2008)
[52] Trick, M., Chvatal, V., Cook, W., Johnson, D., McGeoch, C. and Tarjan, R.: The Second DIMACS ImplementationChallenge: NP Hard Problems: Maximum Clique, Graph Coloring, and Satisfiability. Rutgers University,http://dimacs.rutgers.edu/Challenges/ (1992)
[53] Wang, B. and Zou, H.: Another look at distance-weighted discrimination. J. R. Stat. Soc. B 80, 177–198 (2018)
[54] Wiegele, A.: Biq Mac library–a collection of Max-Cut and quadratic 0− 1 programming instances of mediumsize. Technical report (2007) http://biqmac.uni-klu.ac.at/biqmaclib.pdf
[55] Yan, Z., Gao, S.Y. and Teo C.P.: On the design of sparse but efficient structures in operations. Manage. Sci. 64,2973–3468 (2018)
[56] Yang, L.Q., Sun, D.F. and Toh, K.-C.: SDPNAL+: a majorized semismooth Newton-CG augmented Lagrangianmethod for semidefinite programming with nonnegative constraints. Math. Program. Comput. 7, 331–366 (2015)
[57] Zhang, N., Wu, J. and Zhang, L.W.: A linearly convergent majorized ADMM with indefinite proximal terms forconvex composite programming and its applications. arXiv: 1706.01698v2 (2018)
[58] Zhao, X.Y., Sun, D.F. and Toh K.-C.: A Newton-CG augmented Lagrangian method for semidefinite programming.SIAM J. Optim. 20, 1737–1765 (2010)
[59] Zhu, H., Cano, A. and Giannakis, G.B.: Distributed consensus-based demodulation: algorithms and error analysis.IEEE Trans. Wirel. Commun. 9, 2044–2054 (2010)
38
Tab
le3:
The
per
form
ance
ofA
DM
Mw
ithτ
=1,
1.61
8,1.
90,1.9
9,1.
999
onso
lvin
gSD
Ppro
ble
ms
(acc
ura
cy=
10−
6).
The
max
imum
iter
atio
nnum
ber
isse
tas
100,
000.
Inth
eta
ble
,th
eti
me
ofco
mp
uta
tion
isin
the
form
at
of
”h
ou
rs:m
inu
tes:
seco
nd
s”if
itis
larg
erth
an
on
em
inu
te,
or
else
itis
reco
rded
by
seco
nd
.
itera
tion
ηSD
Pηgap
CP
Uti
me
pro
ble
mm|n
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
theta
41949|2
00
408|3
44|3
14|6
01|3
401
9.8
-7|9
.4-7|9
.9-7|8
.9-7|8
.6-7
9.7
-8|-
5.4
-7|1
.7-6|-
7.3
-10|-
5.5
-92.6|1
.7|1
.6|2
.8|1
5.1
theta
42
5986|2
00
205|1
69|1
73|1
66|1
67
9.8
-7|9
.5-7|9
.5-7|9
.9-7|9
.6-7
-4.0
-8|-
4.5
-8|4
.3-8|3
.9-8|3
.5-8
1.1|0
.9|0
.9|0
.9|0
.9
theta
64375|3
00
400|3
53|3
33|6
72|3
735
9.0
-7|9
.0-7|9
.4-7|9
.9-7|9
.9-7
-1.7
-6|2
.0-6|2
.1-6|-
2.0
-10|1
.5-9
4|3
.6|3
.3|6
.7|3
5.9
theta
62
13390|3
00
186|1
85|1
73|1
63|2
01
9.5
-7|9
.2-7|9
.6-7|9
.6-7|9
.7-7
5.4
-8|1
.4-7|1
.4-7|5
.8-9|-
6.4
-10
2|2|2
.1|1
.8|2
.1
theta
87905|4
00
453|3
80|3
57|6
44|3
401
9.4
-7|8
.1-7|9
.7-7|9
.9-7|8
.8-7
-6.5
-7|1
.7-6|2
.0-6|-
2.1
-10|1
.7-9
9.4|7
.9|7
.4|1
3.3|1
:10
theta
82
23872|4
00
191|1
71|1
72|1
81|1
82
9.1
-7|9
.7-7|9
.2-7|9
.5-7|9
.8-7
-1.5
-9|1
.8-7|6
.8-8|7
.1-8|7
.2-8
4.3|3
.8|3
.8|4|4
theta
83
39862|4
00
173|1
61|1
50|1
47|1
47
9.9
-7|9
.5-7|9
.8-7|9
.7-7|9
.4-7
-6.2
-8|1
.1-7|1
.1-7|1
.1-7|1
.1-7
3.9|3
.7|3
.4|3
.4|3
.3
theta
10
12470|5
00
464|3
86|3
89|6
86|2
801
8.5
-7|8
.9-7|8
.2-7|9
.9-7|9
.4-7
-1.4
-6|2
.0-6|1
.7-6|-
1.2
-10|2
.9-9
14.6|1
2.2|1
2.3|2
1.5|1
:27
theta
102
37467|5
00
197|1
85|1
71|1
72|2
51
9.5
-7|9
.8-7|9
.3-7|9
.9-7|8
.5-7
6.9
-8|1
.1-7|4
.6-8|5
.8-8|-
2.2
-10
6.5|6
.1|5
.7|5
.7|8
.2
theta
103
62516|5
00
154|1
44|1
41|1
45|1
44
9.8
-7|9
.0-7|9
.3-7|9
.3-7|9
.7-7
1.3
-7|6
.9-8|4
.6-8|3
.5-8|3
.6-8
5.2|4
.9|4
.8|4
.9|4
.9
theta
104
87245|5
00
159|1
48|1
55|1
52|1
51
9.8
-7|9
.5-7|9
.7-7|9
.2-7|9
.7-7
1.8
-7|1
.1-7|4
.9-8|4
.3-8|4
.4-8
5.5|5
.2|5
.4|5
.2|5
.2
theta
12
17979|6
00
458|3
92|3
82|6
01|3
101
9.5
-7|8
.2-7|9
.6-7|9
.3-7|8
.4-7
1.6
-6|1
.8-6|-
1.8
-6|4
.6-1
0|-
4.9
-10
24.3|2
0.7|2
0.2|3
1.5|2
:41
theta
123
90020|6
00
150|1
44|1
36|1
42|1
42
9.5
-7|9
.6-7|9
.9-7|9
.4-7|9
.2-7
6.3
-8|6
.6-8|7
.5-8|5
.8-8|5
.6-8
8.8|8
.4|8|8
.4|8
.2
theta
162
127600|8
00
181|1
48|1
38|1
38|1
51
9.4
-7|9
.6-7|9
.2-7|9
.9-7|9
.4-7
6.3
-8|3
.8-8|-
1.3
-8|2
.8-8|1
.1-8
20.6|1
6.8|1
5.9|1
5.5|1
7.2
MA
NN
-a27
703|3
78
1516|1
388|1
372|1
272|2
112
9.1
-7|9
.1-7|7
.5-7|8
.2-7|9
.9-7
1.3
-5|-
1.4
-5|1
.1-5|-
1.3
-5|8
.0-9
22.3|2
0.3|1
9.8|1
8.5|3
4.2
johnso
n8-4
-4561|7
0165|1
61|1
53|1
62|1
62
4.7
-7|8
.8-7|9
.6-7|9
.3-7|6
.2-7
2.3
-6|-
2.4
-6|-
5.5
-6|-
3.4
-6|-
3.4
-60.3|0
.1|0
.1|0
.2|0
.2
johnso
n16-2
-41681|1
20
102|1
03|1
04|1
07|1
07
2.1
-7|5
.0-7|1
.8-7|1
.1-7|3
.5-7
-2.7
-6|-
5.0
-6|2
.4-6|-
1.4
-6|-
1.2
-60.2|0
.2|0
.1|0
.1|0
.2
san200-0
.7-1
5971|2
00
3778|3
898|3
397|3
446|5
301
9.7
-7|9
.7-7|9
.9-7|9
.9-7|9
.7-7
-1.3
-5|-
7.6
-6|-
1.3
-5|-
1.1
-5|-
1.5
-710.5|1
1.6|8
.9|8
.7|1
3.2
sanr2
00-0
.76033|2
00
211|1
78|1
80|1
69|1
69
9.9
-7|9
.7-7|9
.4-7|9
.6-7|9
.6-7
-9.8
-8|-
1.8
-7|-
1.6
-7|-
1.6
-7|-
1.5
-71|0
.8|0
.9|0
.8|0
.8
c-f
at2
00-1
18367|2
00
239|3
70|3
68|5
68|2
701
9.9
-7|7
.7-7|8
.9-7|9
.9-7|9
.8-7
3.8
-6|-
6.4
-6|-
7.3
-6|2
.2-7|-
1.1
-61.1|1
.7|1
.6|2
.5|1
1.3
ham
min
g-6
-41313|6
473|6
8|6
4|6
4|6
46.3
-7|9
.0-7|7
.0-7|7
.0-7|7
.0-7
1.7
-6|-
2.5
-6|3
.7-6|3
.3-6|3
.3-6
0.1|0
.1|0
.1|0
.1|0
.1
ham
min
g-8
-411777|2
56
149|1
44|1
43|1
46|1
46
9.2
-7|5
.5-7|6
.8-7|2
.2-7|8
.1-7
-4.0
-6|-
5.8
-6|3
.1-6|-
2.9
-6|-
2.0
-60.9|0
.8|0
.8|0
.8|0
.8
ham
min
g-9
-82305|5
12
2608|2
666|2
533|2
758|2
656
8.3
-7|9
.0-7|9
.8-7|8
.7-7|9
.3-7
9.9
-6|1
.1-5|-
1.2
-5|1
.0-5|1
.2-5
1:1
7|1
:19|1
:16|1
:25|1
:31
ham
min
g-1
0-2
23041|1
024
700|6
44|6
30|6
31|6
30
8.3
-7|4
.5-7|7
.8-7|7
.2-7|9
.6-7
-7.6
-6|1
.2-5|1
.9-5|-
1.9
-5|-
1.9
-52:1
6|2
:01|2
:02|1
:55|1
:55
ham
min
g-7
-5-6
1793|1
28
532|5
24|5
37|5
39|5
37
6.0
-7|9
.6-7|6
.7-7|9
.7-7|9
.3-7
3.5
-6|-
5.4
-6|-
3.6
-6|3
.7-6|4
.8-6
0.8|0
.8|0
.9|0
.8|0
.9
ham
min
g-8
-3-4
16129|2
56
193|1
73|1
89|1
86|1
86
6.6
-7|8
.1-7|3
.9-7|6
.0-7|5
.1-7
-2.5
-6|6
.1-6|3
.6-6|5
.0-6|4
.2-6
1.2|1
.1|1
.2|1
.2|1
.2
ham
min
g-9
-5-6
53761|5
12
1029|1
068|1
054|1
036|1
065
7.4
-7|7
.0-7|8
.3-7|7
.5-7|8
.2-7
8.8
-6|-
7.9
-6|9
.6-6|8
.6-6|1
.0-5
33.7|3
4.5|3
4.1|3
3.6|3
4.7
bro
ck200-1
5067|2
00
241|1
96|1
86|2
02|2
51
9.4
-7|9
.4-7|9
.1-7|9
.9-7|9
.7-7
-2.2
-7|1
.1-7|1
.2-7|1
.3-7|-
1.4
-91.2|0
.9|0
.9|1|1
.2
bro
ck200-4
6812|2
00
202|1
73|1
67|1
60|1
60
9.8
-7|9
.6-7|9
.5-7|9
.8-7|9
.9-7
-4.7
-8|8
.9-8|9
.9-8|7
.5-8|6
.9-8
1|0
.8|0
.8|0
.8|0
.8
bro
ck400-1
20078|4
00
250|2
19|1
89|2
25|4
51
9.9
-7|9
.9-7|9
.9-7|9
.9-7|8
.7-7
5.0
-8|6
.9-8|-
3.8
-7|1
.2-8|9
.4-1
05.6|4
.8|4
.3|5|1
0.1
keller4
5101|1
71
259|2
11|2
03|1
88|1
88
9.9
-7|9
.8-7|9
.4-7|9
.5-7|9
.7-7
-4.7
-7|-
3.0
-7|-
3.2
-7|-
2.3
-7|-
2.3
-70.7|0
.6|0
.6|0
.5|0
.5
p-h
at3
00-1
33918|3
00
834|5
91|6
23|6
04|6
41
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-8.5
-7|-
2.6
-7|1
.1-7|7
.1-8|-
4.0
-98.8|6
.3|6
.6|6
.3|6
.8
G43
9991|1
000
1527|1
320|1
309|1
310|3
501
9.3
-7|9
.4-7|9
.1-7|8
.8-7|9
.4-7
2.8
-6|3
.2-6|-
3.1
-6|2
.9-6|5
.2-9
4:1
3|3
:34|3
:33|3
:37|9
:35
G44
9991|1
000
1527|1
319|1
309|1
310|3
513
8.8
-7|9
.9-7|8
.8-7|8
.3-7|9
.9-7
3.0
-6|3
.4-6|-
3.2
-6|2
.9-6|6
.4-9
4:1
0|3
:34|3
:33|3
:36|9
:36
G45
9991|1
000
1562|1
318|1
331|1
309|4
001
9.2
-7|9
.1-7|9
.4-7|8
.8-7|9
.4-7
-2.4
-6|3
.6-6|2
.8-6|3
.5-6|2
.5-9
4:1
8|3
:37|3
:39|3
:35|1
0:5
5
G46
9991|1
000
1593|1
342|1
353|1
307|3
601
9.9
-7|8
.9-7|8
.7-7|9
.2-7|8
.0-7
2.5
-6|-
3.4
-6|-
2.7
-6|3
.5-6|4
.8-9
4:2
4|3
:41|3
:43|3
:36|9
:51
G47
9991|1
000
1568|1
323|1
312|1
313|3
502
9.0
-7|9
.2-7|8
.8-7|8
.2-7|9
.9-7
-2.1
-6|2
.7-6|-
2.9
-6|2
.7-6|-
5.8
-94:2
0|3
:38|3
:36|3
:36|9
:38
G51
5910|1
000
7395|6
135|5
035|4
977|7
827
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.9
-7|-
1.4
-7|-
5.4
-8|-
2.2
-7|-
1.1
-623:3
7|2
0:2
1|1
5:3
6|1
5:2
0|2
3:3
1
G52
5917|1
000
26218|2
0278|1
6964|1
5980|1
5793
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-9.4
-6|-
9.7
-6|-
8.9
-6|-
8.9
-6|-
9.1
-61:1
3:4
8|5
7:0
9|4
7:4
5|4
5:1
8|4
4:2
4
G53
5915|1
000
100000|1
6476|1
7439|1
6424|1
7300
1.5
-6|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-7.0
-6|-
8.4
-6|-
7.8
-6|-
7.9
-6|-
7.8
-64:5
7:5
6|4
4:2
9|4
4:5
4|4
2:2
0|4
4:3
4
G54
5917|1
000
4715|4
592|4
152|3
806|4
532
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.3
-5|7
.2-6|-
9.8
-6|-
3.8
-6|7
.8-6
13:1
0|1
2:4
9|1
1:4
0|1
0:3
4|1
2:2
6
39
Table
3(c
onti
nued)
itera
tion
ηSD
Pηgap
CP
Uti
me
pro
ble
mm|n
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
1dc.6
4544|6
44046|2
805|2
559|2
492|3
679
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-9.4
-8|3
.3-6|2
.6-7|6
.6-7|-
3.3
-84.2|2
.4|2
.1|2
.1|2
.9
1et.
64
265|6
4247|2
11|2
07|2
15|3
51
9.8
-7|9
.8-7|9
.6-7|9
.2-7|9
.5-7
-2.3
-6|2
.1-6|-
2.1
-6|1
.2-6|-
3.9
-80.2|0
.2|0
.2|0
.2|0
.3
1tc
.64
193|6
4365|3
42|3
20|7
45|5
395
8.7
-7|7
.6-7|8
.7-7|9
.9-7|9
.9-7
2.4
-6|-
2.1
-6|-
2.3
-6|-
1.6
-8|-
9.9
-80.4|0
.3|0
.3|0
.6|4
.1
1dc.1
28
1472|1
28
100000|1
00000|1
0960|1
00000|1
00000
2.1
-6|2
.8-6|9
.9-7|2
.3-6|4
.7-6
-6.3
-6|-
6.1
-7|-
5.2
-6|-
2.5
-6|-
1.9
-63:1
9|3
:32|2
0.9|3
:26|3
:37
1et.
128
673|1
28
388|3
83|3
72|7
01|3
201
8.3
-7|7
.8-7|8
.9-7|8
.8-7|9
.3-7
-3.7
-6|-
2.5
-6|2
.7-6|1
.3-8|1
.5-7
0.8|0
.7|0
.7|1
.3|5
.4
1tc
.128
513|1
28
1079|9
51|8
83|1
005|4
056
9.8
-7|8
.3-7|8
.7-7|9
.9-7|9
.9-7
-4.4
-6|4
.0-6|3
.9-6|2
.0-6|-
6.9
-81.7|1
.5|1
.4|1
.5|5
.5
1zc.1
28
1121|1
28
245|1
97|1
84|7
51|5
101
8.8
-7|9
.7-7|6
.1-7|9
.9-7|8
.0-7
9.1
-7|1
.1-6|1
.8-6|-
3.3
-8|-
2.2
-70.4|0
.4|0
.4|1
.3|7
.8
1dc.2
56
3840|2
56
8759|6
759|6
542|6
536|6
301
9.2
-7|9
.6-7|9
.4-7|8
.8-7|9
.3-7
-1.7
-5|-
1.5
-5|-
1.6
-5|-
1.5
-5|5
.3-6
52|4
0.9|3
8.7|3
8.7|3
5.7
1et.
256
1665|2
56
1623|1
333|1
242|1
184|4
601
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.6-7
-4.9
-7|-
3.5
-7|-
5.6
-7|-
5.4
-7|-
1.4
-69.8|8
.2|7
.7|7
.3|2
8
1tc
.256
1313|2
56
5673|3
259|3
271|3
911|9
201
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.5-7
-3.4
-6|-
3.5
-7|-
3.4
-7|-
3.1
-6|-
2.3
-644.4|2
4.8|2
5.8|3
0.6|1
:07
1zc.2
56
2817|2
56
295|3
02|2
76|7
09|4
301
9.2
-7|5
.0-7|9
.9-7|9
.9-7|8
.6-7
4.0
-6|2
.3-6|-
2.2
-6|2
.2-8|1
.9-7
1.7|1
.8|1
.6|4|2
3.5
1dc.5
12
9728|5
12
5618|4
875|3
740|4
643|6
065
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.5
-6|-
3.0
-6|-
6.1
-6|-
3.0
-6|-
3.3
-64:2
9|3
:53|2
:58|3
:43|4
:50
1et.
512
4033|5
12
2372|2
037|1
826|1
808|4
601
9.5
-7|9
.9-7|9
.4-7|9
.9-7|8
.5-7
2.4
-8|-
2.6
-7|-
3.2
-7|-
3.5
-7|2
.6-7
1:2
6|1
:15|1
:07|1
:06|3
:00
1tc
.512
3265|5
12
5340|7
138|4
948|4
858|6
972
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-4.2
-6|-
5.5
-6|-
3.9
-6|-
4.7
-6|-
3.7
-63:0
8|4
:15|3
:10|2
:51|4
:07
2dc.5
12
54896|5
12
6361|4
692|4
796|4
697|4
703
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.0
-5|-
1.4
-5|-
2.0
-5|-
2.0
-5|-
1.9
-54:1
5|3
:06|3
:13|2
:50|2
:51
1zc.5
12
6913|5
12
512|4
62|4
71|4
53|1
067
8.2
-7|8
.8-7|9
.2-7|9
.9-7|9
.9-7
3.0
-6|3
.1-6|-
2.6
-6|3
.8-6|5
.4-8
16.7|1
5.8|1
5.2|1
4.4|3
4.1
1dc.1
024
24064|1
024
5332|3
406|3
428|3
291|4
731
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-5.0
-6|-
6.9
-6|-
6.2
-6|-
6.5
-6|-
5.7
-616:4
8|1
1:3
5|1
1:2
7|1
1:1
8|1
6:2
4
1et.
1024
9601|1
024
2854|2
502|2
194|2
474|5
005
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-4.2
-6|-
4.2
-6|-
5.3
-6|-
3.1
-6|-
6.0
-68:5
6|7
:05|6
:16|7
:39|1
5:2
9
1tc
.1024
7937|1
024
6057|1
0345|1
0261|9
771|6
149
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-4.3
-6|-
4.3
-6|-
4.4
-6|-
4.4
-6|-
4.0
-620:4
6|3
6:1
6|3
6:0
9|3
2:4
0|2
1:0
6
1zc.1
024
16641|1
024
816|7
51|7
67|1
001|3
301
9.3
-7|9
.1-7|9
.4-7|8
.4-7|9
.9-7
5.9
-6|4
.2-6|4
.6-6|-
7.5
-8|-
1.5
-62:2
6|2
:15|2
:17|2
:58|9
:47
2dc.1
024
169163|1
024
10074|7
629|7
491|7
893|7
625
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.5
-5|-
2.5
-5|-
2.6
-5|-
2.6
-5|-
2.5
-533:1
5|2
4:5
1|2
4:4
0|2
6:0
4|2
4:4
9
1dc.2
048
58368|2
048
6436|6
115|4
962|5
898|4
801
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.7-7
5.8
-7|-
5.4
-6|-
1.1
-5|-
5.5
-6|-
8.1
-62:3
5:4
0|
2:3
6:5
0|
2:0
2:2
0|
2:5
1:3
9|
2:0
9:4
2
1et.
2048
22529|2
048
4697|7
488|4
501|1
0578|5
301
9.9
-7|9
.9-7|9
.9-7|9
.9-7|8
.7-7
-4.7
-6|-
5.3
-6|-
9.9
-6|-
5.3
-6|-
4.1
-62:1
0:3
4|
3:3
4:0
6|
1:5
5:3
2|
4:3
2:2
1|
2:2
7:1
0
1tc
.2048
18945|2
048
6623|5
975|7
106|1
2207|5
769
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-6.9
-6|-
6.5
-6|-
8.7
-6|-
6.0
-6|-
7.0
-63:0
6:5
7|
2:4
2:5
4|
2:5
7:3
9|
5:1
9:1
3|
2:2
2:2
2
1zc.2
048
39425|2
048
1530|1
395|1
391|1
408|4
201
7.5
-7|9
.9-7|9
.7-7|9
.9-7|8
.4-7
4.3
-6|5
.4-6|4
.9-6|5
.3-6|2
.2-8
34:0
8|3
5:0
7|3
2:3
7|3
3:0
2|
1:3
9:3
1
2dc.2
048
504452|2
048
6561|5
134|4
800|4
745|5
472
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.5
-5|-
2.4
-5|-
2.5
-5|-
2.5
-5|-
2.3
-52:5
7:3
7|
2:0
9:2
1|
2:0
9:4
3|
1:4
7:4
9|
2:0
3:1
9
fap01
1378|5
21839|1
449|1
540|1
419|1
417
9.6
-7|9
.5-7|9
.9-7|9
.9-7|9
.7-7
1.5
-6|-
1.4
-5|-
1.8
-5|-
1.8
-5|-
1.7
-52.4|1
.3|1
.4|1
.3|1
.3
fap02
1866|6
15171|4
199|3
770|5
420|4
257
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7
-5.0
-6|5
.2-7|4
.6-6|-
1.0
-5|5
.0-6
4|3
.2|2
.9|4
.3|3
.3
fap03
2145|6
52600|2
443|2
080|2
351|2
345
9.9
-7|9
.9-7|9
.8-7|9
.9-7|9
.9-7
-4.1
-6|-
1.3
-5|-
1.5
-5|-
1.3
-5|-
1.3
-52.3|2
.2|1
.8|2
.1|2
.2
fap04
3321|8
12174|1
751|1
601|1
641|1
626
9.9
-7|8
.9-7|8
.1-7|9
.8-7|9
.9-7
-5.6
-6|5
.3-6|1
.1-6|1
.1-5|4
.8-6
3.1|2
.7|2
.2|2
.4|2
.2
fap05
3570|8
42457|1
928|1
753|1
739|1
736
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-5.1
-6|9
.9-6|9
.9-6|-
5.0
-6|3
.9-6
3.5|2
.9|2
.5|2
.4|2
.5
fap06
4371|9
31668|1
317|1
223|1
174|1
172
9.9
-7|9
.4-7|9
.9-7|9
.9-7|9
.9-7
-9.5
-7|-
1.5
-7|-
2.3
-6|6
.1-6|6
.2-6
2.9|2
.2|1
.9|1
.9|1
.9
fap07
4851|9
81873|1
558|1
418|1
392|1
389
9.9
-7|9
.1-7|8
.4-7|9
.3-7|9
.4-7
1.5
-6|1
.2-6|2
.5-6|-
7.4
-7|-
7.9
-72.9|2
.4|2
.3|2
.3|2
.4
fap08
7260|1
20
1617|1
293|1
198|1
201|1
201
9.9
-7|9
.9-7|9
.9-7|9
.6-7|8
.4-7
-1.7
-6|-
1.7
-6|-
1.8
-6|-
6.8
-6|1
.7-6
3.6|2
.6|2
.4|2
.5|2
.5
fap09
15225|1
74
951|7
58|7
01|7
18|7
16
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.1
-6|-
1.1
-6|-
8.3
-7|-
1.1
-6|-
1.1
-63.4|2
.8|2
.9|2
.8|3
fap10
14479|1
83
5156|4
038|3
838|3
568|3
470
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-6.6
-5|-
9.0
-5|-
7.5
-5|-
5.8
-5|-
5.8
-526.2|2
0.5|1
9.3|1
8.2|1
7.7
fap11
24292|2
52
5428|4
358|4
062|3
978|3
965
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.1
-4|-1.1
-4|-1.2
-4|-1.2
-4|-1.2
-450.8|4
1.9|4
2|3
5.7|3
5.5
fap12
26462|3
69
7414|6
192|5
716|5
501|5
808
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.0
-4|-1.3
-4|-1.3
-4|-1.2
-4|-1.2
-42:0
6|1
:44|1
:36|1
:33|1
:38
fap25
322924|2
118
11830|9
292|8
504|8
409|1
1289
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-8.0
-5|-
8.2
-5|-
9.1
-5|-
8.1
-5|-
8.0
-55:4
7:2
1|
3:3
2:4
9|
3:0
5:4
2|
3:2
6:0
9|
4:5
0:1
0
fap36
1154467|4
110
8085|6
495|5
929|5
980|9
001
9.9
-7|1
.0-6|1
.0-6|9
.9-7|9
.7-7
-2.8
-5|-
2.8
-5|-
2.8
-5|-
2.8
-5|-
2.4
-521:3
6:2
6|
17:0
7:3
9|
16:0
5:0
0|
15:5
4:3
1|
25:0
5:5
8
Rn3m
20p3
20000|3
00
644|5
27|4
86|4
77|4
76
9.9
-7|9
.9-7|9
.9-7|9
.7-7|9
.7-7
-8.6
-6|-
8.3
-6|-
8.2
-6|-
8.0
-6|-
8.0
-619.4|1
5.7|1
3.4|1
5.1|1
6.2
Rn3m
25p3
25000|3
00
561|4
73|4
42|4
30|4
29
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.1
-5|-
2.8
-5|-
2.8
-5|-
2.8
-5|-
2.7
-545.5|3
8.5|2
9.4|3
2.6|3
2.4
Rn3m
10p4
10000|3
00
1076|8
24|8
16|9
04|5
602
9.9
-7|9
.9-7|9
.9-7|9
.9-7|7
.0-7
-2.6
-5|-
2.6
-5|-
2.6
-5|-
2.6
-5|-
1.7
-645.9|3
9.2|3
0.1|4
3.5|4
:02
40
Table
3(c
onti
nued)
itera
tion
ηSD
Pηgap
CP
Uti
me
pro
ble
mm|n
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
Rn4m
30p3
30000|4
00
781|6
05|5
74|5
61|6
41
9.9
-7|9
.9-7|9
.8-7|9
.9-7|6
.5-7
-2.0
-5|-
1.9
-5|-
1.9
-5|-
1.9
-5|-
1.0
-535.4|2
5.9|2
0.9|2
3.3|2
8.4
Rn4m
40p3
40000|4
00
645|5
02|5
14|5
03|5
02
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-8.5
-6|-
7.8
-6|-
7.9
-6|-
7.9
-6|-
7.9
-61:3
3|1
:08|1
:02|1
:16|1
:14
Rn4m
15p4
15000|4
00
1193|9
93|9
26|9
98|6
002
9.9
-7|9
.9-7|9
.9-7|9
.8-7|7
.7-7
-2.2
-5|-
2.2
-5|-
2.2
-5|-
2.2
-5|-
9.8
-61:3
4|1
:19|5
9.2|1
:18|7
:32
Rn5m
30p3
30000|5
00
1027|8
32|7
47|8
26|5
416
9.9
-7|9
.9-7|9
.9-7|9
.9-7|8
.4-7
-1.2
-3|-1.2
-3|-1.2
-3|-1.2
-3|9
.7-5
42.2|3
4.3|2
3.4|3
2.2|3
:44
Rn5m
40p3
40000|5
00
918|6
89|6
51|6
58|4
103
9.9
-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
-1.1
-4|-1.0
-4|-1.0
-4|-1.0
-4|-
9.7
-650.2|4
1.5|2
7.3|3
5.9|3
:41
Rn5m
50p3
50000|5
00
776|6
02|5
72|5
59|5
58
9.9
-7|9
.9-7|9
.7-7|9
.9-7|9
.8-7
-2.1
-5|-
2.0
-5|-
2.0
-5|-
2.0
-5|-
2.0
-51:2
7|1
:11|5
3.2|1
:06|1
:06
Rn5m
20p4
20000|5
00
1257|1
021|1
003|1
226|6
114
9.9
-7|9
.8-7|9
.9-7|9
.9-7|9
.1-7
-1.7
-5|-
1.6
-5|-
1.7
-5|-
1.7
-5|-
1.2
-52:3
6|1
:51|1
:29|2
:08|1
1:0
1
Rn6m
40p3
40000|6
00
1051|8
27|8
17|9
27|5
701
9.9
-7|9
.8-7|9
.9-7|9
.9-7|9
.7-7
-3.5
-5|-
3.4
-5|-
3.5
-5|-
3.5
-5|-
1.3
-51:0
1|4
7.8|4
1.4|5
4.5|5
:40
Rn6m
50p3
50000|6
00
1018|7
87|7
15|7
47|4
802
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-5|-
1.4
-5|-
1.4
-5|-
1.4
-5|1
.1-6
1:1
0|5
2.2|4
0.4|5
2|5
:36
Rn6m
60p3
60000|6
00
839|6
89|6
39|6
33|3
637
9.9
-7|9
.8-7|9
.8-7|9
.8-7|2
.1-7
-2.3
-5|-
2.4
-5|-
2.4
-5|-
2.4
-5|4
.6-6
1:2
5|1
:12|5
3.3|1
:04|5
:48
Rn6m
20p4
20000|6
00
1414|1
303|1
131|1
343|6
478
9.9
-7|4
.8-7|9
.9-7|9
.9-7|9
.9-7
-2.3
-5|-
9.9
-6|-
2.3
-5|-
2.3
-5|-
2.0
-52:0
2|2
:00|1
:21|1
:54|8
:53
Rn7m
50p3
50000|7
00
1126|9
30|8
94|9
45|5
709
9.9
-7|9
.9-7|6
.0-7|9
.9-7|9
.3-7
-3.5
-5|-
3.5
-5|-
1.0
-5|-
3.5
-5|-
1.2
-51:3
7|1
:20|1
:11|1
:21|8
:12
Rn7m
70p3
70000|7
00
1010|7
50|7
34|7
40|4
648
9.9
-7|9
.8-7|9
.9-7|9
.9-7|2
.9-7
-6.1
-5|-
5.9
-5|-
6.0
-5|-
6.0
-5|1
.1-6
1:4
2|1
:16|1
:04|1
:15|7
:49
Rn8m
70p3
70000|8
00
1131|9
35|9
01|9
36|5
493
9.9
-7|9
.9-7|9
.7-7|9
.9-7|4
.9-7
-2.0
-5|-
2.0
-5|-
5.7
-6|-
2.0
-5|-
5.7
-62:1
2|1
:40|1
:41|1
:40|1
3:0
6
Rn8m
100p3
100000|8
00
949|7
44|7
05|6
91|4
003
9.9
-7|9
.8-7|9
.9-7|9
.9-7|1
.9-7
-4.7
-5|-
4.6
-5|-
4.6
-5|-
4.6
-5|6
.4-6
3:0
7|2
:20|1
:50|2
:09|1
1:4
4
be100.1
101|1
01
2694|2
192|1
989|1
810|5
458
9.9
-7|9
.9-7|9
.5-7|9
.9-7|9
.9-7
-9.3
-7|-
9.3
-7|-
1.3
-6|-
3.4
-6|5
.0-7
4|2
.6|3|2
.4|6
.4
be100.2
101|1
01
2211|1
697|1
738|1
707|4
701
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.2-7
-9.9
-7|-
1.8
-7|-
6.6
-7|-
7.1
-7|-
4.0
-72.8|2
.1|2
.2|2
.1|5
.8
be100.3
101|1
01
2615|2
048|2
033|2
037|4
751
9.4
-7|9
.7-7|9
.1-7|9
.6-7|9
.6-7
1.3
-6|1
.5-6|-
1.5
-6|-
3.6
-6|-
2.9
-73.5|2
.8|2
.7|2
.7|5
.5
be100.4
101|1
01
2223|1
854|1
854|1
787|5
491
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.9
-7|-
9.7
-7|-
5.3
-7|-
4.6
-7|-
5.3
-72.8|2
.4|2
.4|2
.2|6
.7
be100.5
101|1
01
1872|1
421|1
520|1
429|5
383
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-8.2
-7|-
5.2
-7|-
7.5
-7|-
7.1
-7|-
1.4
-72.4|1
.8|2|1
.9|6
.5
be100.6
101|1
01
2329|1
881|1
949|1
873|4
302
9.2
-7|9
.9-7|9
.9-7|9
.7-7|7
.2-7
-1.5
-7|-
4.5
-7|-
4.9
-7|-
9.4
-7|-
2.1
-72.9|2
.2|2
.5|2
.3|5
.3
be100.7
101|1
01
2201|1
947|1
637|1
739|5
501
9.9
-7|9
.9-7|9
.4-7|9
.9-7|9
.9-7
-8.4
-7|-
3.7
-7|-
4.1
-7|-
2.1
-7|-
5.4
-72.5|2
.2|2
.4|2|6
.1
be100.8
101|1
01
2254|1
929|1
797|1
601|4
165
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.8
-7|-
3.8
-7|-
1.7
-7|-
4.9
-7|-
4.7
-72.6|2
.3|2
.3|1
.9|4
.7
be100.9
101|1
01
1755|1
319|1
258|1
192|5
121
9.9
-7|8
.9-7|9
.9-7|9
.9-7|9
.9-7
-6.8
-7|-
2.7
-7|-
9.3
-7|-
6.2
-7|4
.2-8
2|1
.7|1
.7|1
.4|5
.7
be100.1
0101|1
01
1755|1
402|1
321|1
259|5
401
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.8
-7|-
4.4
-7|-
4.8
-7|-
3.3
-7|-
2.7
-72.1|1
.8|1
.7|1
.5|6
be120.3
.1121|1
21
2446|2
122|1
830|1
845|5
201
9.9
-7|9
.9-7|9
.3-7|9
.9-7|9
.3-7
2.4
-7|-
6.7
-7|-
2.5
-7|-
10.0
-7|-
1.7
-73.5|2
.9|3
.1|2
.6|7
.2
be120.3
.2121|1
21
2486|1
729|1
601|1
581|5
201
9.9
-7|9
.3-7|9
.2-7|8
.2-7|8
.0-7
-1.4
-6|-
1.1
-6|-
2.2
-6|-
2.3
-6|-
3.1
-83.5|2
.6|2
.6|2
.4|7
.1
be120.3
.3121|1
21
2465|2
144|1
938|1
839|6
319
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-8.3
-7|-
7.1
-7|-
4.3
-7|-
5.0
-7|-
2.0
-83.5|3
.1|3
.1|2
.8|8
.9
be120.3
.4121|1
21
2921|2
350|2
050|2
032|5
188
9.8
-7|9
.9-7|9
.9-7|9
.9-7|7
.2-7
-5.8
-7|-
4.8
-7|-
2.3
-7|-
6.0
-7|9
.7-8
4.3|3
.3|3
.1|2
.8|7
.3
be120.3
.5121|1
21
2042|1
839|1
699|1
619|5
111
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-6|3
.1-7|2
.6-7|5
.5-7|4
.2-8
2.9|2
.6|2
.6|2
.3|7
be120.3
.6121|1
21
2743|2
135|1
849|1
940|5
957
9.9
-7|9
.9-7|9
.5-7|9
.9-7|9
.9-7
-8.4
-7|-
5.2
-7|-
9.3
-7|-
4.6
-7|-
2.3
-73.6|3|2
.9|2
.6|7
.8
be120.3
.7121|1
21
3229|2
410|2
278|2
248|6
002
9.9
-7|9
.9-7|9
.9-7|9
.9-7|5
.2-7
-9.0
-7|-
7.6
-7|-
4.4
-7|-
5.1
-7|7
.9-8
4.5|3
.4|3
.4|3
.1|8
.2
be120.3
.8121|1
21
2934|2
418|2
166|2
035|5
201
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.3-7
-3.2
-7|-
5.5
-7|-
2.9
-7|-
4.7
-8|-
4.6
-74.3|3
.5|3
.3|3|7
.5
be120.3
.9121|1
21
2133|1
623|1
593|1
564|5
002
9.9
-7|9
.9-7|9
.9-7|9
.9-7|7
.7-7
-9.0
-7|-
9.5
-7|-
8.7
-7|-
9.2
-7|8
.0-8
3|2
.3|2
.4|2
.2|7
be120.3
.10
121|1
21
2355|1
826|1
558|1
632|6
513
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.1
-6|-
1.1
-6|-
1.5
-6|-
9.4
-7|5
.4-7
3.3|2
.6|2
.4|2
.4|8
.9
be120.8
.1121|1
21
2555|1
865|1
974|1
729|4
966
9.9
-7|9
.9-7|9
.9-7|9
.9-7|6
.1-7
-1.4
-8|4
.8-7|4
.5-8|6
.5-7|-
7.8
-83.5|2
.5|3
.2|2
.3|6
.5
be120.8
.2121|1
21
2127|1
768|1
557|1
743|4
719
8.7
-7|9
.9-7|8
.8-7|9
.9-7|9
.9-7
-1.2
-6|-
4.7
-7|-
1.6
-6|-
9.2
-7|4
.3-7
3.2|2
.5|2
.6|2
.5|6
.4
be120.8
.3121|1
21
2267|1
722|1
723|1
658|5
202
9.9
-7|9
.9-7|9
.9-7|9
.9-7|7
.8-7
-5.3
-7|-
1.7
-7|-
9.9
-8|-
5.3
-7|1
.4-7
3.3|2
.5|2
.7|2
.4|7
.5
be120.8
.4121|1
21
2438|1
770|1
810|1
705|5
537
9.9
-7|9
.9-7|9
.9-7|9
.9-7|5
.5-7
-6.2
-7|-
3.3
-6|-
8.3
-7|2
.5-7|-
2.0
-73.5|2
.5|2
.8|2
.4|7
.5
be120.8
.5121|1
21
2657|1
879|1
835|1
859|5
306
9.9
-7|9
.2-7|9
.8-7|8
.6-7|9
.5-7
-2.3
-7|1
.1-6|4
.0-6|-
2.0
-6|-
4.1
-73.5|2
.7|2
.8|2
.7|7
.1
be120.8
.6121|1
21
2023|1
653|1
547|1
525|5
598
9.9
-7|9
.9-7|9
.9-7|9
.9-7|8
.9-7
-1.7
-6|-
1.2
-6|-
1.3
-6|-
1.3
-6|4
.4-8
2.8|2
.3|2
.3|2
.1|7
.7
be120.8
.7121|1
21
2342|1
809|1
894|1
792|5
833
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.3-7
-2.8
-7|-
3.4
-7|-
1.5
-8|-
8.0
-8|-
3.5
-73.3|2
.6|2
.8|2
.5|8
.2
be120.8
.8121|1
21
2260|1
793|1
685|1
500|4
935
9.9
-7|9
.9-7|9
.9-7|9
.9-7|6
.5-7
4.2
-8|8
.2-8|2
.1-7|3
.0-7|5
.4-9
3.4|2
.6|2
.7|2
.2|7
.1
41
Table
3(c
onti
nued)
itera
tion
ηSD
Pηgap
CP
Uti
me
pro
ble
mm|n
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
be120.8
.9121|1
21
1988|1
763|1
631|1
505|6
299
9.6
-7|9
.9-7|9
.9-7|9
.9-7|9
.6-7
-1.0
-6|-
1.0
-6|-
8.7
-7|-
1.5
-6|5
.8-8
3|2
.7|2
.9|2
.2|9
be120.8
.10
121|1
21
2227|1
900|1
624|1
636|5
701
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.1-7
-8.1
-7|-
8.5
-7|-
1.0
-6|-
7.6
-7|-
4.9
-73.1|2
.7|2
.6|2
.3|7
.8
be150.3
.1151|1
51
3270|2
533|2
282|2
210|5
095
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-7.2
-7|-
7.8
-7|-
8.5
-7|-
1.1
-6|2
.7-7
6.7|5
.1|5
.1|4
.5|1
0.2
be150.3
.2151|1
51
3355|2
724|2
472|2
346|4
801
9.9
-7|9
.9-7|9
.9-7|9
.9-7|8
.6-7
-7.1
-7|-
8.4
-7|-
8.2
-7|-
5.1
-7|-
2.6
-76.8|5
.5|5
.2|4
.8|9
.5
be150.3
.3151|1
51
3189|2
306|2
360|1
982|5
601
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.6-7
-1.0
-6|-
7.8
-7|-
8.3
-7|-
1.7
-6|5
.1-7
6.7|4
.8|5
.3|4
.2|1
1.5
be150.3
.4151|1
51
3794|3
040|2
672|2
558|5
307
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.6-7
-5.7
-7|-
4.9
-7|-
1.8
-7|-
5.6
-7|3
.8-7
7.6|6
.1|5
.7|5
.1|1
0.8
be150.3
.5151|1
51
2797|2
352|2
069|2
117|5
320
9.9
-7|9
.9-7|9
.9-7|9
.9-7|5
.7-7
-2.3
-7|-
5.1
-7|-
7.0
-7|-
4.1
-7|-
1.1
-85.7|4
.7|4
.5|4
.3|1
0.5
be150.3
.6151|1
51
2758|2
365|2
068|2
120|5
532
9.9
-7|9
.9-7|9
.9-7|9
.9-7|7
.1-7
-8.0
-7|-
5.9
-7|-
8.4
-7|-
5.5
-7|-
3.9
-75.6|4
.8|4
.5|4
.4|1
1
be150.3
.7151|1
51
2937|2
398|2
074|2
174|4
901
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.4
-7|-
3.5
-7|-
5.6
-7|-
4.2
-7|-
3.7
-75.6|4
.7|4
.4|4
.3|9
.2
be150.3
.8151|1
51
3001|2
215|1
979|1
871|5
901
9.9
-7|9
.9-7|9
.7-7|9
.4-7|8
.0-7
-1.2
-7|-
3.1
-7|-
6.1
-7|-
1.5
-6|-
3.1
-76|4
.5|4
.5|4|1
2
be150.3
.9151|1
51
2370|1
680|1
592|1
452|5
202
9.9
-7|9
.9-7|9
.9-7|9
.9-7|6
.4-7
-1.1
-6|-
7.7
-7|-
1.3
-6|-
1.2
-6|6
.1-8
4.9|3
.5|3
.5|3|1
0.4
be150.3
.10
151|1
51
3227|2
618|2
182|2
258|4
701
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-5.8
-7|-
4.1
-7|-
4.4
-7|-
2.9
-7|2
.9-7
6.8|5
.4|4
.6|4
.7|9
.4
be150.8
.1151|1
51
2717|2
203|1
962|1
834|6
302
9.9
-7|9
.9-7|9
.9-7|9
.9-7|6
.5-7
-1.1
-6|-
1.0
-6|-
10.0
-7|-
8.2
-7|-
2.2
-75.6|4
.5|4
.2|3
.8|1
2.5
be150.8
.2151|1
51
2898|2
326|2
164|2
049|5
717
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-7.3
-7|-
4.4
-7|-
7.1
-7|-
7.2
-7|-
5.4
-76.4|4
.9|4
.8|4
.3|1
1.7
be150.8
.3151|1
51
3367|2
815|2
644|2
528|5
401
9.8
-7|9
.9-7|9
.9-7|9
.9-7|9
.7-7
-2.8
-7|-
6.9
-8|-
1.3
-7|-
3.7
-8|3
.8-7
7.2|5
.7|5
.7|5
.1|1
0.9
be150.8
.4151|1
51
2724|2
114|2
057|1
975|6
025
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-6.5
-7|-
1.1
-6|-
7.7
-7|-
7.8
-7|-
1.1
-75.5|4
.3|4
.4|4
.1|1
2
be150.8
.5151|1
51
3125|2
505|2
150|2
043|4
796
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-5.6
-7|-
4.2
-7|6
.0-7|2
.9-7|1
.8-7
6.3|5
.1|4
.6|4
.1|9
.5
be150.8
.6151|1
51
2329|2
148|1
819|1
926|5
301
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.1-7
-3.2
-6|-
9.0
-7|-
1.9
-6|-
7.8
-7|1
.3-7
4.6|4
.2|3
.7|3
.8|1
0
be150.8
.7151|1
51
3687|2
439|2
315|2
358|6
244
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.6-7
-3.6
-7|-
4.6
-7|-
1.1
-6|-
2.7
-7|-
2.1
-77.3|4
.8|5|4
.6|1
2.5
be150.8
.8151|1
51
3310|2
566|2
415|2
134|4
601
9.9
-7|9
.9-7|9
.9-7|9
.3-7|9
.4-7
-6.3
-7|-
5.9
-7|-
7.6
-7|-
6.7
-7|2
.2-7
6.9|5
.2|5
.3|4
.7|9
.1
be150.8
.9151|1
51
2843|1
975|1
930|1
920|4
595
9.8
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.1
-6|-
1.4
-6|-
1.1
-6|-
1.3
-6|-
4.9
-76.1|4
.1|4
.5|4|9
.2
be150.8
.10
151|1
51
3358|2
517|2
499|2
279|4
901
9.8
-7|9
.9-7|9
.9-7|9
.9-7|9
.0-7
-6.8
-7|-
7.1
-7|-
6.9
-7|-
8.4
-7|1
.4-7
7.1|5
.2|5
.4|4
.7|1
0
be200.3
.1201|2
01
3446|3
055|2
334|2
389|6
347
9.8
-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7
-6.3
-7|-
8.7
-7|-
4.8
-7|-
1.1
-6|-
4.4
-711.7|1
0.2|8
.2|7
.8|2
1
be200.3
.2201|2
01
4035|2
920|2
850|2
555|6
201
9.8
-7|9
.9-7|9
.8-7|9
.9-7|8
.3-7
-5.8
-7|-
6.5
-7|-
1.2
-6|-
9.4
-7|2
.9-7
13.1|9
.3|9
.5|8
.1|1
8.8
be200.3
.3201|2
01
4491|3
515|3
332|3
219|5
901
9.5
-7|9
.7-7|9
.9-7|9
.9-7|9
.4-7
9.5
-8|-
4.8
-7|-
8.0
-7|-
3.4
-7|6
.6-7
15.1|1
1.3|1
0.9|1
0.1|1
8.2
be200.3
.4201|2
01
4254|3
111|3
137|2
894|6
201
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.0-7
-5.5
-7|-
9.3
-7|-
8.4
-7|-
1.1
-6|2
.8-7
13.2|1
0|1
0.4|9
.3|1
9
be200.3
.5201|2
01
3918|3
122|2
608|2
369|6
001
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-8.3
-7|-
1.3
-6|-
1.3
-6|-
8.8
-7|1
.4-7
13|9
.8|8
.4|7
.5|1
8.5
be200.3
.6201|2
01
3493|2
393|2
744|2
642|5
601
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.7-7
3.1
-7|-
2.8
-6|-
8.0
-7|-
1.1
-6|-
2.8
-711.2|7
.8|9
.5|8
.6|1
7.7
be200.3
.7201|2
01
4573|3
489|3
174|3
069|5
851
9.9
-7|9
.9-7|9
.9-7|9
.6-7|9
.9-7
3.8
-9|7
.4-8|-
3.6
-7|-
3.3
-7|6
.0-7
14.6|1
1.3|1
0.4|1
0|1
8.5
be200.3
.8201|2
01
4148|3
211|2
992|2
906|6
301
9.7
-7|9
.9-7|9
.9-7|9
.9-7|8
.5-7
-5.3
-7|-
7.3
-7|-
1.2
-6|-
1.1
-6|2
.9-7
13.5|1
0.2|9
.5|9
.4|1
9.3
be200.3
.9201|2
01
3890|2
910|2
695|2
842|6
019
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.6
-7|-
1.1
-6|-
6.9
-7|-
9.9
-7|4
.3-7
12.3|9
.1|8
.7|9
.3|1
8.9
be200.3
.10
201|2
01
3156|2
753|2
364|2
435|5
677
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.5-7
-5.6
-7|-
4.4
-7|-
2.5
-7|-
3.6
-7|-
3.6
-79.8|8
.8|8|7
.8|1
8
be200.8
.1201|2
01
4599|3
750|3
417|3
430|5
501
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.0-7
-1.3
-6|-
9.6
-7|-
9.9
-7|-
1.0
-6|2
.1-8
14.7|1
1.9|1
1.6|1
0.9|1
7.6
be200.8
.2201|2
01
4094|3
063|3
023|2
806|5
872
9.7
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.9
-7|-
7.7
-7|-
5.5
-7|-
4.2
-7|-
4.0
-713.4|9
.6|9
.9|8
.8|1
8.1
be200.8
.3201|2
01
4257|3
025|3
035|2
895|5
701
9.9
-7|9
.9-7|9
.9-7|9
.9-7|8
.8-7
-3.9
-9|-
6.1
-7|-
7.9
-7|-
9.8
-7|9
.0-8
13.7|9
.8|1
7.2|9
.5|1
7.9
be200.8
.4201|2
01
4358|3
079|2
860|2
660|6
901
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.3-7
-8.9
-7|3
.5-7|-
5.7
-7|-
2.1
-7|-
1.6
-713.8|9
.7|1
0|8
.4|2
1.2
be200.8
.5201|2
01
3776|2
878|2
631|2
242|5
501
9.9
-7|9
.9-7|9
.9-7|9
.4-7|8
.6-7
-5.8
-7|-
7.6
-7|-
7.4
-7|-
4.5
-7|-
3.0
-712.5|9
.1|8
.5|7
.5|1
6.8
be200.8
.6201|2
01
4717|3
307|2
999|2
964|6
427
9.9
-7|9
.9-7|9
.9-7|9
.2-7|9
.9-7
-3.5
-6|-
2.8
-6|2
.4-6|1
.2-6|5
.6-7
14.2|1
0.2|9
.7|9
.8|1
9.1
be200.8
.7201|2
01
4392|3
574|2
968|3
172|5
701
9.9
-7|9
.9-7|9
.9-7|9
.9-7|7
.5-7
-3.5
-7|-
1.3
-6|-
6.2
-7|-
6.4
-7|-
4.7
-714.4|1
1.7|9
.7|1
0.3|1
8.3
be200.8
.8201|2
01
4110|3
295|2
951|2
824|6
101
9.8
-7|9
.9-7|9
.9-7|9
.9-7|8
.8-7
-2.7
-7|-
2.6
-7|-
4.5
-7|-
7.5
-7|3
.5-7
13.3|1
0.1|9
.2|8
.8|1
8.2
be200.8
.9201|2
01
4121|2
974|2
714|2
540|5
901
9.9
-7|9
.9-7|9
.9-7|9
.9-7|8
.7-7
-1.2
-6|-
6.8
-7|-
1.2
-6|-
3.8
-6|1
.1-7
13|9
.4|8
.8|8
.1|1
8.3
be200.8
.10
201|2
01
3586|2
869|2
739|2
743|4
482
9.9
-7|9
.9-7|9
.9-7|9
.8-7|9
.9-7
-5.1
-7|-
5.4
-7|-
8.9
-7|-
8.9
-7|5
.0-7
11.6|9
.3|9
.5|9
.3|1
4.5
be250.1
251|2
51
6404|4
571|4
299|4
293|7
001
9.9
-7|9
.9-7|9
.8-7|9
.7-7|9
.9-7
-3.5
-6|-
1.2
-6|-
5.4
-7|-
5.3
-7|-
8.7
-830.4|2
1.1|2
1.5|2
0.8|3
2.1
42
Table
3(c
onti
nued)
itera
tion
ηSD
Pηgap
CP
Uti
me
pro
ble
mm|n
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
be250.2
251|2
51
5804|4
388|3
916|3
795|7
301
9.9
-7|9
.8-7|9
.9-7|9
.9-7|9
.8-7
3.6
-6|-
8.7
-7|-
6.8
-7|-
1.1
-6|6
.8-7
27.1|2
1.8|1
8.7|1
7.5|3
3.1
be250.3
251|2
51
6255|4
411|3
231|3
150|6
976
9.9
-7|9
.9-7|9
.5-7|9
.9-7|9
.9-7
-3.2
-6|-
4.9
-7|-
1.3
-6|-
3.5
-6|4
.0-7
29.6|1
9.9|1
6.3|1
5|3
2.5
be250.4
251|2
51
6945|5
107|4
207|4
164|6
451
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.1
-6|-
1.4
-6|-
9.4
-7|-
1.1
-6|-
2.2
-734.4|2
4.3|2
1.1|1
9.8|3
1.8
be250.5
251|2
51
4735|3
541|3
275|3
254|6
695
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-7.6
-8|-
1.4
-7|-
5.3
-7|-
4.5
-7|5
.5-7
22|1
6.7|1
6|1
5.4|3
0.4
be250.6
251|2
51
5335|3
888|3
521|3
562|6
437
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7
6.6
-7|-
4.7
-7|-
9.3
-7|-
6.4
-7|-
3.1
-725.8|1
8.6|1
7.5|1
8|3
0.9
be250.7
251|2
51
5803|4
330|3
975|3
434|6
503
9.9
-7|9
.9-7|9
.8-7|9
.9-7|9
.9-7
-1.7
-6|-
1.1
-6|-
7.8
-7|-
1.1
-6|8
.0-7
26.7|2
0.3|1
9.6|1
6|2
9.6
be250.8
251|2
51
5543|4
225|3
790|3
653|6
364
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7
2.1
-6|3
.1-7|-
7.9
-7|-
1.1
-6|3
.0-7
26.7|2
0.3|1
9.2|1
7.7|3
1.3
be250.9
251|2
51
4764|3
732|3
602|3
487|6
701
9.9
-7|9
.9-7|9
.9-7|9
.7-7|9
.5-7
-1.8
-6|-
9.7
-7|-
4.2
-7|-
6.2
-7|2
.2-7
22.1|1
7.4|1
7.3|1
7.3|3
1
be250.1
0251|2
51
5548|3
889|3
461|3
480|6
213
9.8
-7|9
.9-7|9
.9-7|9
.8-7|9
.9-7
-2.1
-6|-
2.5
-7|1
.4-7|-
4.9
-7|-
3.6
-726.6|1
8.2|1
6.8|1
6.7|2
8.5
bqp50-1
51|5
1855|7
50|6
47|6
80|5
201
9.9
-7|9
.9-7|9
.8-7|9
.9-7|9
.5-7
-5.0
-7|-
2.2
-7|1
.7-7|-
4.0
-7|9
.2-8
0.5|0
.4|0
.4|0
.4|2
.7
bqp50-2
51|5
12463|2
424|2
074|1
859|4
502
9.9
-7|9
.9-7|9
.9-7|9
.9-7|6
.9-7
-4.2
-7|-
8.4
-7|-
6.5
-7|-
1.2
-6|4
.1-8
1.3|1
.2|1
.2|1
.1|2
.3
bqp50-3
51|5
13650|2
518|2
312|2
171|5
001
9.5
-7|9
.4-7|9
.9-7|9
.8-7|9
.2-7
-1.7
-6|7
.0-7|-
3.5
-6|3
.3-6|5
.2-7
1.8|1
.3|1
.3|1
.1|2
.3
bqp50-4
51|5
11828|1
436|1
650|1
384|4
501
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.7-7
2.2
-7|9
.3-8|-
9.1
-7|3
.5-7|7
.0-8
1|0
.8|0
.9|0
.7|2
.3
bqp50-5
51|5
11702|1
217|1
129|1
117|4
623
9.8
-7|9
.6-7|9
.8-7|8
.0-7|9
.9-7
-2.7
-6|-
2.9
-6|-
1.8
-6|-
1.5
-6|-
1.8
-70.8|0
.6|0
.7|0
.6|2
.1
bqp50-6
51|5
12538|1
960|1
749|1
758|4
583
9.7
-7|9
.0-7|9
.7-7|9
.3-7|8
.4-7
-1.4
-6|2
.2-6|-
3.8
-6|2
.0-6|5
.2-8
1.3|1
.3|1|1
.4|2
.9
bqp50-7
51|5
11921|1
478|1
479|1
369|5
401
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7
-1.7
-7|-
3.1
-7|-
3.2
-7|-
9.2
-7|8
.2-8
1|0
.8|0
.8|0
.7|2
.7
bqp50-8
51|5
11380|1
261|1
111|1
139|6
701
9.2
-7|9
.1-7|8
.9-7|8
.9-7|9
.1-7
-9.0
-7|-
1.6
-6|2
.1-6|-
7.0
-7|5
.2-7
0.8|0
.7|0
.7|0
.7|3
.3
bqp50-9
51|5
11848|1
534|1
337|1
377|6
101
8.7
-7|9
.0-7|9
.8-7|8
.1-7|9
.5-7
1.2
-6|1
.0-6|2
.9-6|-
1.9
-6|-
1.8
-71|0
.8|0
.8|0
.8|3
.2
bqp50-1
051|5
11961|1
571|1
440|1
435|1
572
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-5.8
-7|-
5.7
-7|-
7.6
-7|-
3.2
-7|-
8.1
-81.1|1
.1|0
.8|0
.9|0
.9
bqp100-1
101|1
01
2003|1
705|1
517|1
645|6
401
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.5-7
-3.3
-7|5
.6-8|-
3.0
-7|1
.2-8|-
5.3
-72.5|2
.1|1
.9|2|7
.8
bqp100-2
101|1
01
2748|2
256|2
124|2
031|6
218
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.8
-7|-
9.5
-7|-
8.2
-7|-
9.1
-7|-
8.9
-83.5|2
.9|2
.6|2
.3|7
bqp100-3
101|1
01
3441|2
664|2
505|2
627|5
801
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.5-7
-9.9
-7|-
4.6
-7|-
1.3
-7|-
1.5
-7|3
.7-7
3.9|3
.1|2
.9|3|6
.4
bqp100-4
101|1
01
2287|1
683|1
850|1
614|5
602
9.9
-7|9
.9-7|9
.9-7|9
.9-7|5
.1-7
-3.4
-7|-
2.0
-7|-
6.9
-7|-
8.8
-7|4
.6-8
2.5|1
.9|2
.3|1
.9|6
.1
bqp100-5
101|1
01
2280|1
755|1
838|1
786|5
402
9.9
-7|9
.9-7|9
.9-7|9
.9-7|6
.5-7
-8.8
-7|-
2.0
-6|-
1.4
-6|-
1.3
-6|7
.3-8
2.6|2|2
.3|2|5
.9
bqp100-6
101|1
01
2198|1
819|1
658|1
606|6
487
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-7.0
-7|-
7.2
-7|-
6.4
-7|1
.4-7|-
5.4
-72.4|2
.1|2|1
.8|7
.1
bqp100-7
101|1
01
2469|1
888|1
895|1
830|5
440
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.6-7
3.0
-7|-
2.2
-6|3
.3-7|-
9.2
-8|-
3.4
-72.8|2
.2|2
.4|2
.1|6
bqp100-8
101|1
01
2611|1
923|1
687|1
716|5
753
9.9
-7|9
.6-7|9
.0-7|9
.9-7|9
.9-7
-2.9
-7|-
1.9
-7|-
1.2
-6|2
.2-6|4
.2-8
2.9|2
.2|2
.3|2|6
.2
bqp100-9
101|1
01
3622|2
840|2
564|2
779|5
605
9.9
-7|9
.9-7|9
.4-7|9
.9-7|9
.9-7
-4.2
-9|-
3.2
-8|-
7.9
-7|-
8.4
-8|-
3.4
-74|3
.1|3
.3|3
.1|6
.2
bqp100-1
0101|1
01
3164|2
366|2
447|2
392|5
301
9.9
-7|9
.7-7|9
.9-7|9
.9-7|9
.1-7
-1.1
-6|-
8.7
-7|-
1.6
-6|-
1.3
-6|-
2.5
-73.6|2
.8|3|2
.7|5
.8
bqp250-1
251|2
51
6482|4
851|4
394|4
427|6
962
9.9
-7|9
.9-7|9
.9-7|9
.7-7|9
.8-7
2.7
-6|-
4.5
-7|-
9.7
-7|-
5.4
-7|2
.4-7
30|2
2.5|2
1.2|2
1.8|3
3.1
bqp250-2
251|2
51
6081|4
463|4
135|4
036|6
953
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.2
-6|-
3.6
-7|-
3.2
-7|-
8.0
-7|3
.6-7
28.3|2
0.6|2
0|1
8.9|3
3
bqp250-3
251|2
51
6863|4
596|4
345|4
233|6
201
9.8
-7|9
.9-7|9
.7-7|9
.8-7|9
.9-7
-2.3
-6|-
2.5
-8|-
5.8
-7|-
5.8
-7|-
3.1
-732.9|2
1.4|2
2.1|2
0.8|2
9.5
bqp250-4
251|2
51
4934|3
732|2
857|2
771|6
601
9.9
-7|9
.9-7|9
.5-7|9
.6-7|9
.6-7
-1.1
-6|-
7.1
-7|7
.9-7|1
.4-7|5
.4-7
23.6|1
7.9|1
5.1|1
4.3|3
1.3
bqp250-5
251|2
51
7959|5
424|4
757|4
589|7
019
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7
5.8
-6|-
5.1
-7|-
1.8
-6|-
1.6
-6|-
3.0
-736.6|2
4.9|2
2.2|2
1.2|3
1.8
bqp250-6
251|2
51
4656|3
531|3
325|3
169|6
085
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.6
-7|-
1.4
-6|-
2.6
-7|-
8.7
-7|9
.2-8
22.6|1
7.2|1
6.5|1
5.4|2
8.9
bqp250-7
251|2
51
6554|4
700|4
349|4
320|7
014
9.9
-7|9
.5-7|9
.9-7|9
.9-7|9
.9-7
-8.3
-7|-
4.7
-7|-
1.7
-6|-
2.1
-6|5
.9-7
31.9|2
2.4|2
0.7|2
0.2|3
2.4
bqp250-8
251|2
51
3942|3
241|3
047|2
944|6
201
9.8
-7|9
.9-7|9
.9-7|9
.9-7|9
.1-7
-5.2
-7|-
9.4
-7|-
7.9
-7|-
8.8
-7|-
4.3
-719.4|1
5.8|1
6.2|1
4.4|3
0.2
bqp250-9
251|2
51
7165|5
012|4
383|4
350|7
601
9.9
-7|9
.9-7|9
.9-7|9
.8-7|9
.5-7
-6.3
-7|-
5.7
-7|-
6.3
-7|7
.8-8|4
.1-7
33.6|2
4|2
0.7|2
1.5|3
5
bqp250-1
0251|2
51
5014|3
660|2
619|3
128|7
001
9.9
-7|9
.6-7|9
.9-7|9
.9-7|9
.9-7
-1.1
-6|-
3.6
-7|-
1.6
-6|-
9.8
-7|-
3.7
-723.4|1
8.6|1
3.9|1
5.1|3
2.3
bqp500-1
501|5
01
10132|7
274|6
866|6
662|7
301
9.9
-7|9
.8-7|9
.8-7|9
.9-7|9
.9-7
2.9
-6|3
.6-7|-
1.8
-7|-
4.1
-7|5
.9-7
4:2
0|3
:16|4
:01|2
:52|3
:07
bqp500-2
501|5
01
15240|8
644|7
766|7
958|7
801
9.9
-7|9
.9-7|9
.6-7|9
.9-7|8
.5-7
-2.9
-6|3
.8-7|9
.1-7|4
.8-7|-
1.1
-66:2
5|3
:40|3
:43|3
:23|3
:19
bqp500-3
501|5
01
12690|9
531|8
091|8
112|7
983
9.9
-7|9
.9-7|9
.9-7|9
.8-7|9
.9-7
-3.2
-6|-
9.5
-7|-
1.1
-6|-
3.3
-7|1
.5-7
5:3
1|4
:07|3
:36|3
:39|3
:33
bqp500-4
501|5
01
14051|8
963|8
024|7
716|7
323
9.9
-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
3.6
-6|-
5.3
-8|3
.4-7|9
.5-8|4
.0-7
6:1
2|4
:00|3
:39|3
:23|3
:12
43
Table
3(c
onti
nued)
itera
tion
ηSD
Pηgap
CP
Uti
me
pro
ble
mm|n
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
bqp500-5
501|5
01
12010|8
797|7
432|7
264|7
601
9.9
-7|9
.9-7|9
.9-7|9
.8-7|8
.1-7
7.0
-6|-
3.1
-7|1
.3-7|-
3.4
-7|-
7.6
-75:1
3|3
:57|3
:19|3
:17|3
:19
bqp500-6
501|5
01
10177|7
333|7
396|6
973|7
353
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.2
-6|-
7.1
-7|-
1.2
-6|-
4.5
-7|-
3.2
-74:2
8|3
:20|3
:24|3
:06|3
:20
bqp500-7
501|5
01
10795|7
128|6
915|6
915|6
961
9.9
-7|9
.6-7|9
.8-7|9
.9-7|9
.9-7
3.7
-7|1
.6-6|-
1.0
-6|-
8.1
-7|-
8.9
-74:4
8|3
:15|3
:12|3
:05|3
:06
bqp500-8
501|5
01
9848|7
942|7
420|7
241|7
195
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.4
-6|5
.3-7|-
1.7
-7|-
2.8
-7|-
7.8
-74:2
6|3
:34|3
:35|3
:16|3
:14
bqp500-9
501|5
01
10779|7
312|6
950|7
121|7
357
9.9
-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
5.5
-7|-
5.1
-7|-
4.6
-7|-
3.9
-7|4
.0-7
4:5
2|3
:26|3
:05|3
:13|3
:19
bqp500-1
0501|5
01
17191|9
623|8
354|8
227|7
801
9.7
-7|9
.9-7|9
.9-7|9
.9-7|9
.7-7
-1.6
-6|-
8.5
-7|-
4.9
-7|8
.3-7|1
.7-6
7:4
6|4
:22|3
:36|3
:45|3
:29
gka1a
51|5
13485|2
877|2
632|2
594|3
679
8.7
-7|8
.7-7|9
.4-7|7
.7-7|9
.9-7
-1.5
-6|2
.4-6|2
.7-6|2
.4-6|2
.0-7
1.8|1
.5|1
.5|1
.4|1
.8
gka2a
61|6
13690|2
789|2
572|2
514|4
201
9.8
-7|9
.9-7|9
.7-7|9
.5-7|9
.6-7
2.8
-6|8
.6-7|1
.2-6|7
.1-7|5
.6-7
2.2|1
.6|1
.7|1
.6|2
.4
gka3a
71|7
11569|1
323|1
101|1
083|4
901
9.0
-7|8
.0-7|9
.9-7|7
.2-7|9
.9-7
1.2
-6|1
.4-6|2
.9-6|1
.6-6|-
1.3
-71.3|1
.1|0
.9|0
.8|3
.4
gka4a
81|8
12573|2
215|1
942|1
866|5
501
9.6
-7|9
.1-7|9
.6-7|8
.6-7|9
.5-7
9.6
-7|-
1.6
-6|2
.5-6|1
.7-6|4
.6-8
2.2|2|1
.9|1
.6|4
.3
gka5a
51|5
11857|1
334|1
272|1
241|5
974
9.9
-7|9
.9-7|9
.5-7|8
.7-7|9
.9-7
-1.2
-6|-
2.3
-6|1
.8-7|4
.4-7|5
.4-8
1|0
.7|0
.8|0
.7|3
.1
gka6a
31|3
1842|7
20|6
90|8
93|6
901
8.7
-7|8
.7-7|8
.3-7|9
.9-7|9
.4-7
7.0
-7|-
1.1
-6|2
.9-7|-
2.3
-7|5
.1-8
0.3|0
.3|0
.4|0
.3|2
.4
gka7a
31|3
1887|7
11|6
65|8
87|5
901
9.9
-7|8
.9-7|8
.7-7|9
.9-7|9
.4-7
-1.7
-6|2
.9-7|5
.8-7|4
.1-7|5
.4-7
0.3|0
.3|0
.3|0
.3|2
.1
gka8a
101|1
01
4405|3
342|3
178|3
009|5
653
9.9
-7|9
.8-7|9
.7-7|9
.9-7|9
.4-7
1.1
-6|-
3.7
-6|2
.4-6|-
3.9
-6|4
.7-7
5.1|4|3
.9|3
.6|6
.5
gka1b
21|2
1312|2
65|2
65|2
87|4
801
9.7
-7|9
.9-7|9
.8-7|9
.9-7|9
.5-7
-1.1
-6|-
7.2
-7|-
7.5
-7|2
.7-7|-
2.4
-60.1|0
.1|0
.1|0
.1|1
.5
gka2b
31|3
1427|3
35|3
24|7
93|6
801
9.7
-7|9
.9-7|9
.9-7|9
.9-7|9
.6-7
-1.9
-6|-
1.7
-6|-
1.4
-6|5
.8-7|2
.2-7
0.2|0
.1|0
.2|0
.3|2
.5
gka3b
41|4
1318|2
39|2
44|6
01|5
101
9.7
-7|8
.7-7|9
.7-7|8
.3-7|9
.7-7
-2.1
-6|-
2.2
-6|-
1.5
-6|-
4.1
-7|-
1.1
-60.2|0
.1|0
.1|0
.3|2
.1
gka4b
51|5
1459|3
97|3
71|7
98|5
997
9.7
-7|9
.9-7|9
.8-7|9
.9-7|9
.9-7
-2.3
-6|-
1.9
-6|-
2.1
-6|-
5.3
-7|4
.6-7
0.3|0
.2|0
.2|0
.4|3
.1
gka5b
61|6
1434|3
64|3
33|8
79|7
101
9.9
-7|9
.7-7|9
.7-7|9
.9-7|9
.5-7
-1.2
-6|-
1.5
-6|-
8.5
-7|5
.3-7|1
.2-7
0.3|0
.2|0
.3|0
.6|4
.3
gka6b
71|7
1475|4
01|3
93|8
01|7
501
9.6
-7|9
.4-7|9
.9-7|8
.0-7|8
.9-7
-3.7
-6|-
2.5
-6|-
2.8
-6|-
6.4
-7|-
3.8
-70.4|0
.3|0
.4|0
.6|5
.3
gka7b
81|8
1526|4
42|4
11|8
61|7
487
9.9
-7|9
.9-7|9
.8-7|9
.9-7|9
.9-7
-1.2
-6|-
1.2
-6|-
1.0
-6|2
.7-7|-
6.9
-70.5|0
.4|0
.5|0
.8|6
.3
gka8b
91|9
1584|4
80|4
69|8
64|7
366
9.9
-7|9
.7-7|9
.8-7|9
.9-7|9
.9-7
-2.2
-6|-
2.2
-6|-
2.6
-6|-
3.6
-7|-
8.4
-70.6|0
.5|0
.6|0
.9|7
.4
gka9b
101|1
01
594|4
57|4
40|8
95|7
601
9.9
-7|9
.8-7|9
.7-7|9
.9-7|9
.7-7
-3.2
-6|-
4.4
-6|-
2.7
-6|4
.3-7|1
.8-7
0.7|0
.5|0
.6|1|8
gka10b
126|1
26
736|6
03|6
06|8
66|7
440
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.8
-6|-
3.8
-6|-
3.8
-6|1
.0-6|-
9.8
-71.2|1|1
.1|1
.4|1
0.9
gka1c
41|4
12085|1
590|1
524|1
461|5
151
9.3
-7|9
.0-7|9
.8-7|8
.9-7|9
.9-7
1.3
-6|-
1.5
-6|-
3.1
-6|-
1.2
-6|-
4.6
-70.9|0
.7|0
.7|0
.6|2
gka2c
51|5
12113|1
751|1
608|1
499|5
001
9.2
-7|9
.9-7|9
.3-7|9
.0-7|9
.9-7
-1.3
-6|4
.1-7|6
.9-7|-
9.8
-7|-
5.6
-71.2|0
.9|0
.9|0
.8|2
.4
gka3c
61|6
11687|1
335|1
180|1
206|4
701
8.8
-7|9
.7-7|9
.8-7|9
.4-7|9
.8-7
-1.4
-6|-
3.3
-6|3
.0-6|-
2.2
-6|-
5.5
-71.2|0
.9|0
.9|0
.8|3
gka4c
71|7
12402|1
761|1
683|1
633|4
413
9.9
-7|9
.0-7|9
.8-7|9
.7-7|9
.9-7
1.7
-6|1
.1-6|2
.9-6|3
.0-6|4
.5-7
1.8|1
.3|1
.3|1
.2|3
.2
gka5c
81|8
12201|1
819|1
598|1
523|4
908
9.9
-7|9
.9-7|9
.9-7|9
.9-7|8
.6-7
-6.8
-7|-
1.8
-7|-
1.1
-6|-
3.8
-7|2
.7-7
1.9|1
.5|1
.6|1
.3|4
.3
gka6c
91|9
12834|2
648|2
236|2
267|6
008
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-4.6
-7|-
3.8
-7|-
1.0
-7|-
2.6
-7|-
5.2
-72.9|2
.6|2
.4|2
.4|6
.3
gka7c
101|1
01
3073|2
369|2
109|2
165|4
651
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-5.8
-7|-
2.4
-6|3
.2-6|-
3.6
-6|2
.2-7
3.6|2
.8|3|2
.7|5
.5
gka1d
101|1
01
3330|2
845|2
622|2
422|4
801
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.7-7
-2.4
-7|-
4.8
-7|-
6.2
-7|-
2.6
-7|-
3.2
-83.9|3
.4|3
.7|2
.9|5
.5
gka2d
101|1
01
1878|1
358|1
409|1
298|5
201
9.9
-7|9
.0-7|9
.9-7|9
.9-7|9
.7-7
-2.2
-7|-
4.8
-7|-
7.2
-7|9
.3-8|-
1.0
-72.2|1
.7|2
.1|1
.5|5
.9
gka3d
101|1
01
1785|1
382|1
259|1
165|5
522
9.3
-7|8
.9-7|9
.0-7|9
.0-7|9
.9-7
-7.1
-7|1
.2-7|2
.7-7|-
2.6
-6|4
.5-8
2.2|1
.7|1
.9|1
.4|6
gka4d
101|1
01
1745|1
462|1
420|1
293|5
102
9.3
-7|9
.9-7|9
.9-7|9
.9-7|6
.1-7
-1.0
-6|-
7.6
-7|-
4.8
-7|-
1.5
-6|4
.2-8
2.1|1
.6|1
.8|1
.5|5
.5
gka5d
101|1
01
2025|1
669|1
537|1
463|5
801
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.4-7
-3.5
-7|-
2.8
-7|-
3.3
-7|-
1.4
-7|-
3.6
-82.4|2|2
.1|1
.7|6
.6
gka6d
101|1
01
2556|2
131|1
935|1
961|6
401
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.7-7
-4.0
-9|-
3.7
-7|-
8.6
-7|-
3.5
-7|-
5.3
-72.9|2
.4|2
.7|2
.3|7
.4
gka7d
101|1
01
2133|1
826|1
694|1
636|5
005
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
6.3
-7|-
4.1
-7|-
3.3
-7|-
6.1
-7|4
.3-8
2.6|2
.2|2
.5|2|6
gka8d
101|1
01
2566|2
111|2
103|2
041|6
001
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7
-9.6
-7|-
6.3
-7|-
8.4
-7|-
9.7
-7|5
.0-7
3|2
.4|2
.8|2
.5|7
.1
gka9d
101|1
01
2197|1
771|1
491|1
475|4
801
9.9
-7|9
.9-7|9
.7-7|9
.9-7|9
.0-7
-4.2
-7|-
2.4
-7|3
.6-7|-
1.4
-7|-
4.1
-72.6|2
.1|2
.2|1
.7|5
.5
gka10d
101|1
01
2451|1
807|1
764|1
785|6
039
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-5.4
-7|9
.2-7|-
8.3
-7|-
4.6
-7|-
5.4
-72.9|2
.1|2
.4|2
.1|6
.9
gka1e
201|2
01
4510|3
288|3
422|3
195|6
441
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.7-7
1.6
-6|-
1.0
-6|-
1.1
-6|-
4.5
-7|-
3.7
-715.3|1
1.4|1
3.5|1
1|2
2.5
gka2e
201|2
01
4104|3
197|2
949|2
787|6
001
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.0-7
-1.5
-7|-
1.1
-6|-
8.1
-7|-
6.5
-7|-
2.0
-713.9|1
1|1
1.7|9
.5|2
0
44
Table
3(c
onti
nued)
itera
tion
ηSD
Pηgap
CP
Uti
me
pro
ble
mm|n
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
gka3e
201|2
01
3413|2
710|2
638|2
661|5
801
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.6-7
-5.6
-7|-
5.8
-7|-
5.9
-7|-
6.1
-7|4
.2-7
11.5|9
.2|9
.7|9
.1|1
9.4
gka4e
201|2
01
4326|3
273|3
044|2
950|6
122
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.7-7
-3.5
-6|-
4.0
-7|1
.2-7|8
.3-8|3
.5-7
15.1|1
1.5|1
1|1
0.3|2
1.7
gka5e
201|2
01
3556|2
292|2
712|2
624|6
201
9.9
-7|9
.4-7|9
.9-7|9
.9-7|9
.2-7
5.1
-7|7
.8-7|-
6.2
-7|-
6.7
-7|5
.8-7
11.8|8
.3|8
.9|8
.8|1
9.9
gka1f
501|5
01
11283|8
087|6
681|6
537|7
101
9.9
-7|9
.9-7|9
.8-7|9
.8-7|8
.8-7
1.3
-6|-
7.5
-7|-
9.4
-8|-
8.8
-7|8
.3-7
5:0
5|3
:39|3
:17|3
:03|3
:12
gka2f
501|5
01
11541|8
035|7
353|6
920|7
739
9.9
-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
1.2
-6|3
.1-7|-
5.6
-7|-
1.4
-6|2
.6-7
5:1
9|3
:51|3
:39|3
:11|3
:39
gka3f
501|5
01
11020|7
870|6
705|6
713|6
901
9.9
-7|9
.9-7|9
.8-7|9
.8-7|9
.5-7
4.4
-6|-
4.9
-7|2
.8-7|-
8.5
-8|-
8.4
-74:5
9|3
:33|3
:18|3
:08|3
:07
gka4f
501|5
01
10970|7
607|7
036|7
157|7
301
9.9
-7|9
.9-7|9
.7-7|9
.9-7|8
.3-7
1.5
-6|-
1.4
-7|3
.8-7|-
8.2
-7|-
6.7
-75:0
9|3
:34|3
:37|3
:17|3
:20
gka5f
501|5
01
10951|7
130|6
926|6
689|7
058
9.9
-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
1.5
-7|3
.7-7|5
.5-7|6
.9-7|-
4.9
-75:0
9|3
:27|3
:28|3
:07|3
:16
soyb
ean-s
mall-2
1176|4
72536|2
646|2
654|1
478|2
945
9.9
-7|8
.1-7|7
.5-7|9
.5-7|9
.9-7
1.1
-7|6
.4-8|6
.0-8|1
.1-7|-
1.3
-93.4|2
.4|2
.3|1
.2|1
.9
soyb
ean-s
mall-3
1176|4
7221|1
67|1
52|4
07|3
177
9.4
-7|6
.1-7|8
.4-7|9
.9-7|9
.9-7
-2.6
-8|5
.1-7|-
6.1
-7|1
.5-9|1
.6-9
0.2|0
.2|0
.2|0
.3|1
.9
soyb
ean-s
mall-4
1176|4
71271|1
084|1
089|1
088|1
558
9.9
-7|8
.4-7|8
.1-7|8
.1-7|9
.9-7
-7.4
-8|1
.9-8|2
.4-8|2
.4-8|-
1.6
-10
1.2|1
.1|1
.1|1
.1|1
.2
soyb
ean-s
mall-5
1176|4
71732|1
057|9
08|8
70|4
369
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.9
-8|-
3.9
-8|-
3.9
-8|-
3.8
-8|-
2.9
-10
1.4|0
.9|0
.7|0
.8|3
.5
soyb
ean-s
mall-6
1176|4
74366|2
634|2
252|2
153|5
171
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-8.1
-8|-
8.2
-8|-
8.3
-8|-
8.3
-8|3
.6-1
24|2
.3|1
.8|1
.7|4
soyb
ean-s
mall-7
1176|4
72343|1
391|1
191|1
139|4
353
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.0
-8|-
3.0
-8|-
3.0
-8|-
3.0
-8|-
1.5
-10
1.8|1
.1|1|0
.9|3
.3
soyb
ean-s
mall-8
1176|4
76274|3
717|3
174|3
034|4
379
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-7.4
-8|-
7.6
-8|-
7.7
-8|-
7.7
-8|-
1.5
-84.9|2
.8|2
.5|2
.9|3
.8
soyb
ean-s
mall-9
1176|4
74255|2
498|2
136|2
042|4
489
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.0
-7|-
2.0
-7|-
2.0
-7|-
2.0
-7|-
2.6
-10
3.6|2
.1|1
.8|1
.8|3
.5
soyb
ean-s
mall-1
01176|4
73188|2
180|1
865|1
783|5
198
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.0
-7|-
3.0
-7|-
3.0
-7|-
3.0
-7|-
5.9
-11
2.5|1
.7|1
.5|1
.4|3
.9
soyb
ean-s
mall-1
11176|4
74199|2
879|2
460|2
351|4
724
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.3
-7|-
2.3
-7|-
2.3
-7|-
2.3
-7|-
1.5
-93.5|2
.3|1
.9|1
.8|3
.8
soyb
ean-l
arg
e-2
47586|3
07
3819|3
655|3
658|3
660|5
065
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-9.0
-8|-
8.9
-8|-
9.5
-8|-
9.8
-8|-
5.4
-853.3|5
2|4
9.7|4
5.6|5
5.5
soyb
ean-l
arg
e-3
47586|3
07
3268|3
256|3
258|1
852|3
651
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.9
-8|-
4.0
-8|-
4.0
-8|-
2.7
-8|-
3.5
-847.2|4
6.4|4
5.1|2
1.6|4
1.2
soyb
ean-l
arg
e-4
47586|3
07
6819|7
454|7
457|7
458|7
479
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-7.2
-8|-
7.1
-8|-
7.1
-8|-
7.0
-8|-
7.0
-81:4
5|1
:58|1
:55|1
:52|1
:43
soyb
ean-l
arg
e-5
47586|3
07
2611|2
788|2
818|2
261|3
793
9.8
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-4.2
-8|-
4.4
-8|-
4.4
-8|-
6.4
-8|-
2.7
-836|3
9.2|3
9.6|2
8.7|4
6.2
soyb
ean-l
arg
e-6
47586|3
07
998|1
092|8
64|8
17|5
964
9.9
-7|6
.9-7|9
.9-7|9
.9-7|9
.9-7
-7.0
-8|-
2.6
-8|-
5.1
-8|-
5.2
-8|-
6.1
-10
12.9|1
6|1
1.5|1
0.2|1
:10
soyb
ean-l
arg
e-7
47586|3
07
2423|2
606|2
628|2
633|5
066
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.7
-8|-
5.6
-8|-
5.5
-8|-
5.5
-8|-
1.8
-837|4
1.3|4
0.6|3
7.5|1
:04
soyb
ean-l
arg
e-8
47586|3
07
3466|3
551|3
550|3
550|5
482
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.9
-8|-
2.8
-8|-
2.8
-8|-
2.7
-8|-
7.3
-954.8|5
6.6|5
5.9|5
3.9|1
:09
soyb
ean-l
arg
e-9
47586|3
07
4426|4
483|4
467|4
464|6
270
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.1
-8|-
3.2
-8|-
3.2
-8|-
3.1
-8|-
1.3
-81:0
8|1
:11|1
:10|1
:07|1
:19
soyb
ean-l
arg
e-1
047586|3
07
749|7
27|7
38|7
42|5
812
9.4
-7|9
.2-7|9
.9-7|9
.3-7|9
.9-7
1.0
-7|-
6.1
-9|5
.6-9|1
.8-8|2
.3-9
9.9|1
0.1|1
0.2|9
.7|1
:11
soyb
ean-l
arg
e-1
147586|3
07
4506|2
745|2
351|2
252|5
771
9.5
-7|9
.9-7|9
.9-7|9
.4-7|9
.9-7
3.8
-7|9
.9-8|-
3.1
-7|-
3.8
-7|-
3.3
-958.3|3
6.5|3
1.1|2
9.5|1
:13
spam
base
-sm
all-2
45451|3
00
986|9
70|9
46|9
27|4
194
9.1
-7|9
.7-7|9
.9-7|9
.9-7|9
.9-7
-3.3
-8|-
1.9
-7|-
4.4
-7|-
4.4
-7|7
.2-1
110.5|1
0.2|1
0.1|1
0|4
3.4
spam
base
-sm
all-3
45451|3
00
1034|1
420|1
111|1
174|5
885
9.9
-7|9
.7-7|9
.7-7|9
.9-7|9
.9-7
-1.4
-7|-
5.0
-8|-
7.3
-8|-
7.9
-8|-
4.6
-911.9|1
8.5|1
3|1
3.1|1
:02
spam
base
-sm
all-4
45451|3
00
2786|2
730|2
707|2
700|6
178
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.2
-8|-
3.4
-8|-
3.4
-8|-
3.4
-8|-
1.5
-834.5|3
6.3|3
5.4|3
3.9|1
:10
spam
base
-sm
all-5
45451|3
00
3522|2
107|1
804|1
726|6
539
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.3
-7|-
1.4
-7|-
1.1
-7|-
1.8
-7|-
1.6
-10
47.5|2
7|2
1.4|2
1.7|1
:17
spam
base
-sm
all-6
45451|3
00
4535|2
679|2
292|2
192|6
402
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-9.0
-8|-
9.9
-8|-
1.0
-7|-
1.0
-7|-
2.1
-91:0
6|3
8.1|3
0.3|2
8.5|1
:19
spam
base
-sm
all-7
45451|3
00
7587|4
998|4
265|4
075|6
402
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.5
-7|-
2.5
-7|-
2.5
-7|-
2.4
-7|-
7.5
-91:3
4|1
:03|5
4.5|4
9.2|1
:17
spam
base
-sm
all-8
45451|3
00
3974|2
434|2
083|1
994|5
921
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.9
-7|-
1.5
-7|-
9.4
-8|-
2.6
-7|3
.3-1
149.2|3
0.1|2
5.8|2
4.6|1
:11
spam
base
-sm
all-9
45451|3
00
6817|4
209|3
594|3
435|6
503
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.3
-7|-
2.4
-7|-
2.4
-7|-
2.4
-7|-
1.7
-91:2
5|5
2.5|4
5.2|4
2.8|1
:20
spam
base
-sm
all-1
045451|3
00
6227|3
998|3
414|3
263|5
818
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.5
-7|-
1.5
-7|-
1.5
-7|-
1.5
-7|-
2.7
-91:2
2|5
2|4
4.2|4
2.4|1
:15
spam
base
-sm
all-1
145451|3
00
7947|5
234|4
469|4
270|6
171
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.7
-7|-
1.7
-7|-
1.7
-7|-
1.7
-7|-
1.7
-81:4
4|1
:08|5
9|5
6.3|1
:20
spam
base
-mediu
m-2
406351|9
00
1110|7
59|7
26|7
16|5
363
9.9
-7|9
.8-7|9
.5-7|9
.9-7|9
.9-7
-2.2
-6|-
4.4
-6|-
1.9
-6|-
1.3
-6|4
.0-1
03:2
3|2
:21|2
:15|2
:13|1
6:0
5
spam
base
-mediu
m-3
406351|9
00
1812|2
695|2
391|2
487|5
127
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-5.4
-7|-
4.3
-7|-
3.1
-8|-
5.4
-7|-
9.7
-85:3
0|8
:50|7
:25|7
:35|1
5:2
9
spam
base
-mediu
m-4
406351|9
00
4370|4
153|4
139|4
110|6
406
9.8
-7|9
.9-7|9
.8-7|9
.4-7|9
.9-7
-2.3
-6|2
.3-6|-
2.2
-6|8
.3-7|-
2.7
-913:5
3|1
2:5
7|1
2:5
9|1
2:4
9|1
9:2
7
spam
base
-mediu
m-5
406351|9
00
2691|2
948|2
013|2
028|7
137
9.9
-7|9
.9-7|9
.8-7|9
.9-7|9
.9-7
3.4
-8|-
5.9
-8|-
2.3
-8|-
1.3
-7|-
8.9
-98:2
3|1
0:0
4|6
:22|6
:21|2
2:0
2
spam
base
-mediu
m-6
406351|9
00
4759|2
640|2
231|2
148|6
879
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.1
-7|-
2.0
-7|-
2.4
-7|-
2.5
-7|-
5.5
-914:5
4|8
:18|7
:02|6
:47|2
1:3
5
45
Table
3(c
onti
nued)
itera
tion
ηSD
Pηgap
CP
Uti
me
pro
ble
mm|n
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
spam
base
-mediu
m-7
406351|9
00
5744|3
884|3
234|2
735|6
550
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.5
-7|-
2.4
-7|-
2.5
-7|-
3.2
-7|-
1.6
-918:0
9|1
2:1
9|1
0:1
9|8
:37|2
0:3
3
spam
base
-mediu
m-8
406351|9
00
7153|4
197|3
474|3
325|6
663
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.0
-7|-
1.9
-7|-
2.0
-7|-
2.2
-7|-
5.6
-10
22:2
4|1
3:1
3|1
0:5
7|1
0:0
4|2
0:0
1
spam
base
-mediu
m-9
406351|9
00
9345|5
145|4
181|4
008|6
607
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.4
-7|-
2.6
-7|-
3.3
-7|-
3.2
-7|-
2.5
-928:1
2|1
5:3
6|1
2:4
1|1
2:1
1|1
9:5
9
spam
base
-mediu
m-1
0406351|9
00
12192|6
592|5
466|5
296|6
511
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.3
-7|-
1.1
-7|-
1.0
-7|-
1.0
-7|-
2.4
-836:5
5|1
9:5
8|1
6:3
4|1
6:0
1|1
9:4
0
spam
base
-mediu
m-1
1406351|9
00
6715|4
640|4
070|3
941|6
431
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
8.8
-8|-
1.4
-7|-
3.8
-7|-
2.8
-8|1
.1-9
20:1
2|1
3:5
8|1
2:1
4|1
1:5
4|1
9:1
4
spam
base
-larg
e-2
1127251|1
500
1108|7
95|7
19|6
82|5
395
9.9
-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
-8.0
-7|1
.3-7|-
1.3
-6|-
1.9
-6|5
.2-1
111:2
8|8
:16|7
:32|7
:12|5
2:3
8
spam
base
-larg
e-3
1127251|1
500
2217|1
709|1
530|1
504|7
218
9.9
-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
3.8
-6|-
3.7
-6|3
.1-6|2
.5-6|2
.3-1
022:2
1|1
7:1
0|1
5:2
7|1
5:1
2|
1:1
0:5
0
spam
base
-larg
e-4
1127251|1
500
4649|4
376|4
382|4
365|7
442
9.5
-7|9
.8-7|9
.9-7|9
.6-7|9
.9-7
-1.6
-6|2
.8-6|3
.2-6|2
.9-6|-
5.3
-846:0
0|4
3:3
1|4
6:3
8|4
7:4
6|
1:1
9:5
6
spam
base
-larg
e-5
1127251|1
500
13500|1
4373|1
4598|1
4555|1
4583
9.9
-7|9
.5-7|9
.8-7|9
.9-7|9
.9-7
2.5
-6|-
2.4
-6|2
.5-6|-
2.4
-6|2
.4-6
2:2
2:1
1|
2:2
5:3
1|
2:2
7:3
7|
2:2
4:5
7|
2:2
5:1
2
spam
base
-larg
e-6
1127251|1
500
5485|3
448|2
725|2
571|7
026
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-7|-
9.6
-8|-
5.9
-7|7
.7-7|-
6.8
-854:4
9|3
4:4
6|2
7:3
2|2
6:0
2|
1:0
9:5
4
spam
base
-larg
e-7
1127251|1
500
6559|4
260|3
965|3
848|6
896
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.3
-7|-
1.6
-7|-
1.4
-7|-
1.3
-7|1
.5-9
1:0
8:0
3|4
5:0
3|4
1:5
7|4
0:4
2|
1:1
3:2
3
spam
base
-larg
e-8
1127251|1
500
8388|5
745|5
395|5
261|6
608
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.3
-7|-
1.3
-7|-
1.2
-7|-
1.2
-7|-
1.8
-81:3
4:5
1|
1:0
2:0
5|5
8:2
9|5
7:4
5|
1:1
2:5
8
spam
base
-larg
e-9
1127251|1
500
13890|8
547|7
973|7
780|7
803
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.1
-7|-
2.1
-7|-
2.1
-7|-
2.1
-7|-
2.1
-72:2
8:5
7|
1:2
9:1
3|
1:2
1:5
4|
1:2
0:0
5|
1:2
0:1
0
spam
base
-larg
e-1
01127251|1
500
22504|1
3523|1
2685|1
2412|1
2384
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.0
-7|-
1.9
-7|-
2.0
-7|-
1.9
-7|-
1.9
-73:5
3:3
0|
2:3
8:2
9|
2:2
0:3
6|
2:3
3:3
1|
2:1
9:2
8
spam
base
-larg
e-1
11127251|1
500
10722|6
408|6
144|6
073|6
410
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
2.1
-7|6
.6-7|8
.9-8|-
5.8
-7|-
2.5
-71:5
3:1
8|
1:0
7:2
2|
1:0
4:4
5|
1:0
3:4
0|
1:0
7:1
3
abalo
ne-s
mall-2
20301|2
00
867|9
01|9
04|8
69|3
282
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.1
-7|-
3.5
-8|-
3.1
-8|-
9.9
-8|7
.6-1
24.6|5
.3|5
.3|5|1
6.6
abalo
ne-s
mall-3
20301|2
00
909|5
77|5
02|5
49|4
771
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.9
-8|-
3.6
-8|-
3.5
-8|-
1.5
-8|-
2.8
-11
4.9|3
.1|2
.7|3|2
4.7
abalo
ne-s
mall-4
20301|2
00
3063|1
798|1
540|1
474|5
180
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-8.9
-8|-
8.9
-8|-
8.9
-8|-
8.9
-8|-
4.0
-11
16.9|9
.9|8
.5|8
.1|2
7.7
abalo
ne-s
mall-5
20301|2
00
4498|2
975|2
543|2
431|5
791
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-5.1
-8|-
5.1
-8|-
5.1
-8|-
5.1
-8|-
2.7
-10
24.9|1
6.4|1
4.2|1
3.6|3
2.7
abalo
ne-s
mall-6
20301|2
00
5194|3
126|2
672|2
554|5
352
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.3
-7|-
1.3
-7|-
1.3
-7|-
1.3
-7|-
2.1
-10
29|1
7.5|1
4.9|1
4.4|2
9.5
abalo
ne-s
mall-7
20301|2
00
15401|8
937|7
587|7
235|7
217
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.0
-7|-
1.0
-7|-
1.0
-7|-
1.0
-7|-
1.0
-71:2
9|5
1.6|4
4|4
2|4
2
abalo
ne-s
mall-8
20301|2
00
21464|1
3345|1
1212|1
0689|1
0639
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.5
-7|-
2.5
-7|-
2.5
-7|-
2.5
-7|-
2.5
-72:0
6|1
:19|1
:06|1
:03|1
:03
abalo
ne-s
mall-9
20301|2
00
25207|1
7213|1
4449|1
3743|1
3675
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.2
-7|-
3.2
-7|-
3.2
-7|-
3.2
-7|-
3.2
-72:3
1|1
:42|1
:26|1
:21|1
:20
abalo
ne-s
mall-1
020301|2
00
15043|8
703|7
369|7
032|7
004
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-7|-
1.4
-7|-
1.4
-7|-
1.5
-7|-
1.4
-71:3
0|5
2.1|4
3.9|4
2.1|4
2
abalo
ne-s
mall-1
120301|2
00
21997|1
3896|1
1717|1
1154|1
1100
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.4
-7|-
2.4
-7|-
2.4
-7|-
2.4
-7|-
2.4
-72:1
6|1
:26|1
:12|1
:10|1
:09
abalo
ne-m
ediu
m-2
80601|4
00
1262|1
081|1
081|1
046|1
848
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
7.3
-10|-
3.9
-8|-
4.9
-8|-
1.4
-7|-
2.7
-934.5|3
0|2
9.9|2
7.6|4
4
abalo
ne-m
ediu
m-3
80601|4
00
1854|1
729|1
724|1
722|4
171
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-8|-
1.5
-8|-
1.5
-8|-
1.5
-8|-
2.0
-954.1|5
3.2|5
3.2|5
3|1
:42
abalo
ne-m
ediu
m-4
80601|4
00
2029|1
227|1
054|1
009|3
939
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.1
-8|9
.5-9|9
.2-9|9
.1-9|9
.7-1
151.1|3
1.1|2
6.7|2
5.6|1
:38
abalo
ne-m
ediu
m-5
80601|4
00
5680|3
185|2
720|2
600|5
770
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.1
-7|-
2.1
-7|-
2.1
-7|-
2.1
-7|-
2.7
-10
2:2
4|1
:21|1
:10|1
:07|2
:27
abalo
ne-m
ediu
m-6
80601|4
00
8387|5
719|4
879|4
661|5
842
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.7
-7|-
1.7
-7|-
1.7
-7|-
1.7
-7|-
5.6
-83:3
5|2
:28|2
:06|2
:01|2
:32
abalo
ne-m
ediu
m-7
80601|4
00
14290|8
539|7
278|6
950|6
945
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-7|-
1.4
-7|-
1.4
-7|-
1.4
-7|-
1.4
-76:1
5|3
:45|3
:10|3
:02|3
:02
abalo
ne-m
ediu
m-8
80601|4
00
13224|7
557|6
409|6
113|6
156
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.1
-7|-
1.1
-7|-
1.2
-7|-
1.2
-7|-
9.2
-85:5
5|3
:22|2
:52|2
:43|2
:45
abalo
ne-m
ediu
m-9
80601|4
00
24862|1
5368|1
2895|1
2286|1
2229
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.9
-7|-
1.9
-7|-
1.9
-7|-
1.9
-7|-
1.9
-711:2
4|7
:02|5
:55|5
:37|5
:37
abalo
ne-m
ediu
m-1
080601|4
00
44753|3
0224|2
5295|2
4031|2
3912
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.4
-7|-
2.4
-7|-
2.4
-7|-
2.4
-7|-
2.4
-720:1
8|1
3:5
8|1
1:2
4|1
0:5
6|1
0:4
9
abalo
ne-m
ediu
m-1
180601|4
00
38177|2
2500|2
1934|2
0841|2
0737
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.8
-7|-
1.8
-7|-
1.8
-7|-
1.8
-7|-
1.8
-717:3
3|1
0:2
0|1
0:0
4|9
:35|9
:37
abalo
ne-l
arg
e-2
501501|1
000
1045|1
122|1
145|1
155|1
991
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
3.8
-7|-
1.2
-7|-
1.2
-7|-
1.3
-7|-
5.9
-94:4
9|5
:24|5
:23|5
:36|9
:06
abalo
ne-l
arg
e-3
501501|1
000
1369|1
397|1
451|1
467|3
499
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-7|-
9.9
-8|-
9.6
-8|-
9.5
-8|-
8.8
-97:2
8|7
:42|7
:57|7
:41|1
4:0
8
abalo
ne-l
arg
e-4
501501|1
000
2814|1
676|1
437|1
375|4
178
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-9.7
-8|-
1.0
-7|-
1.0
-7|-
1.0
-7|-
4.1
-12
11:2
4|6
:56|5
:54|5
:39|1
6:5
2
abalo
ne-l
arg
e-5
501501|1
000
9065|5
080|4
334|4
140|5
868
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.1
-7|-
2.1
-7|-
2.1
-7|-
2.1
-7|-
3.1
-836:5
6|2
0:2
7|1
6:5
5|1
6:1
0|2
5:2
8
abalo
ne-l
arg
e-6
501501|1
000
10882|7
393|6
305|6
022|6
252
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.3
-7|-
1.3
-7|-
1.3
-7|-
1.3
-7|-
1.1
-743:4
8|2
8:4
6|2
4:3
1|2
3:2
7|2
4:2
0
abalo
ne-l
arg
e-7
501501|1
000
11625|7
032|6
006|5
740|5
845
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.6
-7|-
1.6
-7|-
1.6
-7|-
1.6
-7|-
1.5
-745:0
9|2
7:5
1|2
4:2
6|2
3:2
3|2
3:5
8
abalo
ne-l
arg
e-8
501501|1
000
23607|1
3577|1
1481|1
0945|1
0894
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-4.3
-7|-
4.3
-7|-
4.3
-7|-
4.3
-7|-
4.3
-71:3
7:3
3|5
2:4
7|4
4:4
0|4
2:2
5|4
2:1
6
abalo
ne-l
arg
e-9
501501|1
000
38356|2
3874|2
0166|1
9227|1
9138
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.9
-7|-
2.9
-7|-
2.9
-7|-
2.9
-7|-
2.9
-72:2
9:3
6|
1:3
3:0
9|
1:1
8:4
6|
1:1
5:1
1|
1:2
5:4
8
46
Table
3(c
onti
nued)
itera
tion
ηSD
Pηgap
CP
Uti
me
pro
ble
mm|n
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
abalo
ne-l
arg
e-1
0501501|1
000
32936|2
2015|1
8382|1
7449|1
7361
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.0
-7|-
3.0
-7|-
3.0
-7|-
3.0
-7|-
3.0
-72:2
8:4
5|
1:3
4:4
2|
1:1
8:1
0|
1:1
7:3
9|
1:1
6:0
2
abalo
ne-l
arg
e-1
1501501|1
000
35119|2
0956|1
9982|1
8968|1
8872
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.4
-7|-
2.4
-7|-
2.4
-7|-
2.4
-7|-
2.4
-72:2
7:0
4|
1:2
4:0
4|
1:2
0:0
8|
1:1
6:1
2|
1:1
5:4
6
segm
ent-
small-2
80601|4
00
9029|8
616|8
421|8
375|6
336
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-6.1
-8|-
6.4
-8|-
6.4
-8|-
6.5
-8|-
1.2
-74:2
3|4
:10|4
:02|3
:35|2
:22
segm
ent-
small-3
80601|4
00
10139|1
0195|1
0196|1
0196|1
0201
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-10.0
-8|-
9.8
-8|-
9.8
-8|-
9.8
-8|-
9.8
-85:0
1|5
:04|4
:59|4
:46|4
:26
segm
ent-
small-4
80601|4
00
4899|5
085|5
093|5
095|5
163
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-7.1
-8|-
7.4
-8|-
7.4
-8|-
7.4
-8|-
7.3
-82:2
3|2
:31|2
:29|2
:15|2
:14
segm
ent-
small-5
80601|4
00
19208|1
9763|1
9760|1
9756|1
9755
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-7|-
1.4
-7|-
1.4
-7|-
1.4
-7|-
1.4
-711:0
8|1
1:2
7|1
1:2
2|1
1:0
2|1
0:3
4
segm
ent-
small-6
80601|4
00
11858|1
2361|1
2394|1
2407|1
2408
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-8.9
-8|-
9.2
-8|-
9.4
-8|-
9.4
-8|-
9.4
-86:3
8|7
:00|6
:58|6
:47|6
:29
segm
ent-
small-7
80601|4
00
4378|4
544|4
551|4
554|5
137
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.5
-8|-
3.2
-8|-
3.3
-8|-
3.3
-8|-
2.5
-82:2
0|2
:27|2
:25|2
:26|2
:14
segm
ent-
small-8
80601|4
00
2869|3
041|3
060|3
062|5
585
9.9
-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
-1.5
-7|-
2.2
-8|-
2.1
-8|-
2.1
-8|-
1.2
-81:2
2|1
:44|1
:46|1
:44|2
:44
segm
ent-
small-9
80601|4
00
1899|2
116|2
118|2
122|5
633
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.9
-7|-
3.5
-8|-
2.4
-8|-
2.4
-8|-
2.9
-953.7|1
:09|1
:09|2
:22|4
:47
segm
ent-
small-1
080601|4
00
3593|3
640|3
638|3
634|5
637
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.8
-8|-
2.3
-8|-
2.4
-8|-
2.3
-8|-
1.5
-82:4
9|2
:25|2
:23|2
:15|3
:03
segm
ent-
small-1
180601|4
00
3253|2
998|3
010|3
028|5
776
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-7|-
1.6
-8|-
1.6
-8|-
1.5
-8|-
9.0
-91:4
4|1
:50|1
:54|1
:47|3
:03
segm
ent-
mediu
m-2
246051|7
00
2629|1
899|1
738|1
701|5
913
9.8
-7|9
.7-7|9
.6-7|9
.5-7|9
.9-7
1.2
-6|-
1.2
-6|-
1.3
-6|-
1.2
-6|1
.3-9
5:0
8|3
:39|3
:27|3
:27|1
1:0
9
segm
ent-
mediu
m-3
246051|7
00
1016|8
85|8
80|8
86|5
399
9.4
-7|9
.7-7|9
.8-7|9
.9-7|9
.9-7
9.5
-7|4
.8-7|-
4.1
-7|-
2.9
-9|-
1.2
-91:5
4|1
:45|1
:47|1
:50|1
0:2
2
segm
ent-
mediu
m-4
246051|7
00
9440|9
369|9
373|9
375|7
804
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.2
-7|-
1.2
-7|-
1.2
-7|-
1.2
-7|-
1.5
-722:0
0|2
1:4
9|2
2:3
9|2
0:4
2|1
7:2
3
segm
ent-
mediu
m-5
246051|7
00
9655|9
823|9
832|9
838|7
695
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.2
-7|-
1.3
-7|-
1.3
-7|-
1.3
-7|-
1.4
-724:1
1|2
2:1
0|2
3:5
8|2
1:3
5|1
4:3
9
segm
ent-
mediu
m-6
246051|7
00
17891|1
7906|1
7914|1
7922|1
7923
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.7
-7|-
1.7
-7|-
1.7
-7|-
1.7
-7|-
1.7
-743:1
5|4
3:2
7|4
3:0
9|4
3:4
1|4
1:3
4
segm
ent-
mediu
m-7
246051|7
00
20706|1
9013|1
9029|1
9033|1
9035
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.8
-7|-
1.9
-7|-
1.9
-7|-
1.9
-7|-
1.9
-752:4
6|4
9:4
2|4
8:0
6|4
2:5
1|4
2:2
5
segm
ent-
mediu
m-8
246051|7
00
11948|1
0777|1
0795|1
0798|8
532
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.3
-7|-
1.3
-7|-
1.3
-7|-
1.3
-7|-
1.3
-727:0
9|2
4:3
8|2
4:3
0|2
3:4
2|1
6:1
5
segm
ent-
mediu
m-9
246051|7
00
5771|5
020|5
041|5
046|4
980
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-8.2
-8|-
9.3
-8|-
9.2
-8|-
8.8
-8|-
6.9
-812:0
4|1
0:3
1|1
0:3
8|1
0:0
8|9
:13
segm
ent-
mediu
m-1
0246051|7
00
3984|3
291|3
273|3
918|5
226
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-6.2
-8|-
1.3
-7|5
.7-9|-
3.4
-8|-
2.6
-87:5
3|6
:34|6
:32|7
:49|9
:36
segm
ent-
mediu
m-1
1246051|7
00
3140|2
466|2
284|2
244|5
577
9.7
-7|9
.6-7|9
.5-7|9
.9-7|9
.9-7
-4.0
-7|1
.6-7|3
.1-7|2
.3-7|-
5.0
-95:4
5|4
:32|4
:11|4
:08|1
0:0
9
segm
ent-
larg
e-2
501501|1
000
3159|2
178|1
986|1
949|5
143
9.8
-7|9
.8-7|9
.6-7|9
.7-7|9
.9-7
-1.6
-6|-
1.6
-6|1
.3-6|1
.3-6|-
4.9
-10
14:5
9|9
:12|7
:57|7
:47|1
9:4
6
segm
ent-
larg
e-3
501501|1
000
679|6
32|6
54|7
08|6
188
9.9
-7|9
.5-7|9
.1-7|9
.9-7|9
.9-7
9.5
-8|-
5.7
-7|-
5.5
-7|-
3.6
-7|-
1.5
-10
2:4
6|2
:36|2
:41|2
:53|2
3:5
9
segm
ent-
larg
e-4
501501|1
000
9271|1
1243|1
1262|1
1269|8
902
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.2
-7|-
1.3
-7|-
1.3
-7|-
1.3
-7|-
1.5
-744:0
3|5
4:2
1|5
3:5
3|5
3:0
9|4
1:0
9
segm
ent-
larg
e-5
501501|1
000
14070|1
4493|1
4494|1
4494|1
4494
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-7|-
1.4
-7|-
1.4
-7|-
1.4
-7|-
1.4
-71:1
5:5
6|
1:2
0:0
9|
1:2
0:4
2|
1:1
8:3
8|
1:1
4:3
7
segm
ent-
larg
e-6
501501|1
000
16373|1
6940|1
6968|1
6967|1
6966
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.6
-7|-
1.7
-7|-
1.7
-7|-
1.6
-7|-
1.6
-71:2
7:4
2|
1:2
6:5
7|
1:2
5:4
1|
1:2
0:3
9|
1:1
3:3
7
segm
ent-
larg
e-7
501501|1
000
23923|2
3568|2
3574|2
3569|2
3568
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.0
-7|-
2.0
-7|-
2.0
-7|-
2.0
-7|-
2.0
-72:0
1:0
6|
2:0
5:0
2|
2:1
6:5
5|
2:0
7:5
1|
2:0
0:4
1
segm
ent-
larg
e-8
501501|1
000
15689|1
4976|1
4989|1
4992|1
4994
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.5
-7|-
1.5
-7|-
1.5
-7|-
1.5
-7|-
1.5
-71:2
4:2
3|
1:2
1:0
0|
1:2
0:2
7|
1:1
4:1
0|
1:0
7:2
2
segm
ent-
larg
e-9
501501|1
000
8608|8
303|8
357|8
363|8
351
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-9.1
-8|-
9.8
-8|-
9.8
-8|-
9.8
-8|-
9.8
-842:5
3|4
1:3
1|4
1:3
3|4
0:2
7|3
6:4
4
segm
ent-
larg
e-1
0501501|1
000
3604|3
301|3
289|2
271|5
890
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-6.4
-8|-
2.3
-8|-
2.2
-8|-
3.2
-9|-
2.0
-816:1
8|1
5:2
1|1
5:2
1|9
:33|2
3:5
4
segm
ent-
larg
e-1
1501501|1
000
2977|2
286|2
115|2
142|5
836
9.7
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.4
-7|-
3.2
-7|-
5.6
-7|1
.2-7|-
1.2
-812:3
7|9
:29|8
:48|8
:52|2
3:4
5
housi
ng-2
128778|5
06
5125|4
096|3
781|3
662|4
636
9.4
-7|9
.6-7|9
.7-7|9
.9-7|9
.9-7
-1.9
-7|1
.9-7|-
1.8
-7|-
1.9
-7|2
.4-8
3:2
4|2
:44|2
:33|2
:27|2
:57
housi
ng-3
128778|5
06
2237|1
463|1
288|1
207|4
541
9.3
-7|9
.2-7|8
.4-7|9
.8-7|9
.9-7
-1.4
-8|-
3.1
-8|8
.4-9|2
.0-7|-
2.5
-11
1:3
1|1
:00|5
4.4|4
9.3|2
:54
housi
ng-4
128778|5
06
3580|2
118|1
810|1
732|6
077
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-5.9
-8|-
5.9
-8|-
6.0
-8|-
6.2
-8|2
.5-1
02:2
0|1
:23|1
:10|1
:08|3
:55
housi
ng-5
128778|5
06
2579|1
769|1
588|1
425|5
878
9.9
-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-7|1
.7-7|2
.7-7|1
.1-7|1
.4-1
11:4
2|1
:11|1
:04|5
6.3|3
:50
housi
ng-6
128778|5
06
4044|2
462|2
091|1
997|5
430
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-7|-
1.4
-7|-
1.4
-7|-
1.4
-7|-
1.2
-10
2:5
3|1
:54|1
:39|1
:38|4
:06
housi
ng-7
128778|5
06
9120|5
332|4
525|4
317|5
176
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.0
-7|-
2.0
-7|-
2.0
-7|-
2.0
-7|-
6.7
-87:2
1|4
:08|3
:22|3
:01|3
:37
housi
ng-8
128778|5
06
9367|6
157|5
205|4
962|5
290
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.8
-7|-
1.8
-7|-
1.8
-7|-
1.8
-7|-
1.3
-76:4
1|4
:23|3
:43|3
:33|3
:47
housi
ng-9
128778|5
06
11344|7
514|6
312|6
007|6
052
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-9.1
-8|-
9.2
-8|-
9.5
-8|-
9.4
-8|-
8.4
-88:1
3|5
:25|4
:35|4
:20|4
:23
housi
ng-1
0128778|5
06
19551|1
1517|1
1021|1
0457|1
0403
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.7
-7|-
2.7
-7|-
2.7
-7|-
2.7
-7|-
2.7
-714:0
2|8
:16|8
:09|7
:56|8
:12
housi
ng-1
1128778|5
06
12674|7
443|6
320|6
042|6
031
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.8
-7|-
1.8
-7|-
1.8
-7|-
1.8
-7|-
1.7
-710:3
4|5
:52|5
:26|4
:46|4
:33
nonsy
m(5
,4)
3374|1
25
2241|2
140|2
093|2
114|2
086
9.6
-7|9
.1-7|9
.7-7|9
.1-7|9
.9-7
1.3
-5|-
1.2
-5|-
1.2
-5|-
1.1
-5|1
.2-5
4.3|3
.4|3
.3|3
.3|3
.3
47
Table
3(c
onti
nued)
itera
tion
ηSD
Pηgap
CP
Uti
me
pro
ble
mm|n
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
τ=
1|1.6
18|1.9
0|1.9
9|1.9
99
nonsy
m(6
,4)
9260|2
16
3684|3
801|3
746|3
769|3
765
9.9
-7|9
.8-7|9
.9-7|9
.4-7|9
.4-7
1.8
-5|-
7.1
-6|1
.7-5|1
.6-5|1
.6-5
14.4|1
4.9|1
4.8|1
5.3|1
5
nonsy
m(7
,4)
21951|3
43
5059|5
180|5
404|5
324|5
324
9.8
-7|9
.9-7|9
.4-7|9
.9-7|9
.8-7
-8.1
-6|-
9.0
-6|2
.3-5|-
2.3
-5|-
8.0
-647.4|4
7.8|4
9.8|4
9.6|4
9.3
nonsy
m(8
,4)
46655|5
12
8210|7
768|7
734|7
709|7
705
9.8
-7|9
.8-7|9
.9-7|9
.9-7|9
.8-7
-1.0
-5|-
2.9
-5|2
.9-5|-
2.9
-5|-
9.8
-64:2
8|4
:17|4
:11|4
:48|4
:44
nonsy
m(9
,4)
91124|7
29
10876|9
980|1
0751|1
0728|1
0704
9.9
-7|9
.9-7|9
.7-7|9
.8-7|9
.9-7
-2.7
-5|9
.1-6|-
8.0
-6|-
2.6
-5|-
2.6
-513:5
6|1
3:3
4|1
4:4
0|1
4:2
0|1
4:3
4
nonsy
m(1
0,4
)166374|1
000
30089|3
3786|3
3829|3
3820|3
3760
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.1
-5|-
3.5
-5|-
3.5
-5|3
.4-5|-
1.1
-51:4
5:4
4|
1:5
2:4
8|
1:5
3:3
9|
1:5
1:5
1|
1:5
2:3
9
nonsy
m(1
1,4
)287495|1
331
17778|1
7487|1
7408|1
7733|1
7693
9.9
-7|9
.9-7|9
.9-7|9
.7-7|9
.8-7
-4.5
-5|-
1.5
-5|-
1.5
-5|4
.5-5|4
.5-5
1:4
7:4
4|
1:4
0:0
5|
1:4
3:5
7|
1:4
5:2
3|
2:1
0:4
2
nonsy
m(3
,5)
1295|8
12517|2
461|2
447|2
450|2
449
8.7
-7|9
.2-7|9
.7-7|8
.9-7|9
.6-7
-7.9
-6|-
7.3
-6|-
7.4
-6|7
.8-6|7
.7-6
1:3
0|1
:51|1
:15|1
:20|1
:44
nonsy
m(4
,5)
9999|2
56
12494|1
2149|1
2124|1
2082|1
2087
9.7
-7|9
.7-7|9
.9-7|9
.9-7|9
.8-7
-5.9
-6|1
.6-5|-
1.7
-5|1
.7-5|5
.5-6
5:5
6|5
:55|6
:59|7
:10|6
:15
nonsy
m(5
,5)
50624|6
25
8373|7
586|7
512|7
148|6
761
9.9
-7|9
.9-7|9
.8-7|9
.9-7|9
.9-7
1.1
-5|3
.1-5|3
.0-5|3
.0-5|-
1.1
-512:4
8|1
1:4
2|1
2:0
6|1
1:4
0|1
1:5
1
nonsy
m(6
,5)
194480|1
296
16298|1
5117|1
4606|1
4656|1
4735
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.5
-5|-
1.5
-5|-
4.5
-5|-
4.4
-5|1
.7-5
1:5
5:3
3|
1:3
8:5
8|
1:4
7:4
8|
1:3
3:3
6|
1:3
1:4
3
sym
-rd(3
,20)
10625|2
31
2117|1
748|1
633|1
646|1
645
8.1
-7|9
.4-7|9
.5-7|9
.8-7|9
.5-7
4.8
-7|1
.3-5|1
.0-5|4
.5-6|4
.0-6
16:5
7|1
3:3
7|1
2:4
2|1
2:5
5|1
2:4
9
sym
-rd(5
,5)
461|5
6262|2
51|2
53|2
50|2
49
9.9
-7|9
.7-7|3
.5-7|4
.2-7|4
.7-7
-7.6
-6|2
.6-5|2
.7-6|7
.0-6|4
.1-6
1.1|0
.9|0
.3|0
.7|0
.9
sym
-rd(5
,10)
8007|2
86
697|7
61|7
12|7
73|7
73
7.2
-7|8
.8-7|8
.4-7|7
.1-7|9
.0-7
2.5
-5|6
.4-6|3
.7-5|3
.7-5|4
.5-5
7:4
4|9
:01|6
:41|8
:56|8
:06
sym
-rd(6
,5)
209|3
5157|1
71|1
61|1
60|1
60
8.9
-7|4
.2-7|6
.2-7|7
.7-7|9
.3-7
1.1
-5|7
.9-6|2
.2-6|1
.6-5|1
.8-5
0.6|0
.3|0
.1|0
.2|0
.2
sym
-rd(6
,10)
5004|2
20
635|5
88|5
80|5
69|5
68
9.6
-7|8
.6-7|7
.4-7|6
.8-7|6
.9-7
4.5
-5|4
.5-5|4
.4-5|3
.3-5|3
.4-5
2:4
3|2
:18|2
:01|2
:14|2
:14
48
Tab
le4:
Th
ep
erfo
rman
ceof
7m
ult
i-b
lock
AD
MM
-typ
ealg
ori
thm
son
solv
ing
convex
qu
ad
rati
cS
DP
pro
ble
ms
(acc
ura
cy=
10−
6).
Th
em
axim
um
iter
ati
on
num
ber
isse
tas
500,0
00.
Inth
eta
ble
,‘D
-E’
mea
ns
the
dir
ectl
yex
tended
mult
i-blo
ckA
DM
Mw
ith
step
-len
gth
τ=
1,
P1.0
(P1.6
,P
1.9
)m
eans
the
blo
cksG
Sdec
om
posi
tion
base
dm
ult
i-blo
ckpro
xim
al
AD
MM
wit
hst
ep-l
ength
τ=
1(τ
=1.6
18,τ
=1.9
,et
c.),
andiP
1.0
(iP
1.6
,iP
1.9
,et
c.)
mea
ns
the
blo
cksG
Sdec
om
posi
tion
base
din
exact
mult
i-blo
ckA
DM
Mw
ith
step
-len
gth
τ=
1(τ
=1.6
18,τ
=1.9
,et
c.).
The
tim
eof
com
puta
tion
isin
the
form
at
of
”hou
rs:m
inu
tes:
seco
nd
s”if
itis
larg
erth
anon
em
inu
te,
or
else
itis
reco
rded
by
seco
nd
.
iteratio
nηSD
Pηgap
CP
Utim
e
proble
mm|nE|nI
D-E|P
1.0|P
1.6|P
1.9|iP
1.0|iP
1.6|iP
1.9
D-E|P
1.0|P
1.6|P
1.9|iP
1.0|iP
1.6|iP
1.9
D-E|P
1.0|P
1.6|P
1.9|iP
1.0|iP
1.6|iP
1.9
D-E|P
1.0|P
1.6|P
1.9|iP
1.0|iP
1.6|iP
1.9
be100.1
101;1
01;1
4850
26599|2
5983|2
0453|1
9877|6
585|5
190|4
912
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.7
-7|1
.8-7|8
.6-8|4
.8-8|-1.2
-6|-8.7
-7|-1.2
-61:1
1|1
:19|1
:01|1
:00|3
3.5|2
2.8|2
2
be100.2
101;1
01;1
4850
14616|1
4399|1
1747|1
1562|5
292|4
442|4
287
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
4.4
-7|3
.5-7|3
.4-7|3
.6-8|-9.7
-7|-4.8
-7|-7.0
-743.7|4
4.1|3
5.4|3
4.9|2
2|1
9.1|1
8.4
be100.3
101;1
01;1
4850
17930|1
7453|1
3747|1
1606|4
230|3
520|3
414
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.6
-7|-1.5
-7|-1.5
-7|-1.4
-7|-4.4
-7|-3.2
-7|-2.4
-752.3|5
0.5|4
0.6|3
4.1|1
8.5|1
5.5|1
4.8
be100.4
101;1
01;1
4850
28987|3
0096|2
3233|2
1379|6
856|5
214|4
916
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.8
-7|1
.8-7|1
.6-7|1
.6-7|-3.6
-7|-8.8
-7|-5.0
-71:2
4|1
:30|1
:08|1
:02|3
1.1|2
2.6|2
2.2
be100.5
101;1
01;1
4850
13962|1
5264|1
0192|1
0004|4
593|3
686|3
612
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
8.8
-7|1
.0-6|1
.0-6|9
.3-7|-4.2
-7|7
.1-8|-3.9
-741.7|4
6.3|3
0.9|3
0.8|1
9.2|1
6.2|1
5.9
be100.6
101;1
01;1
4850
18215|1
8685|1
4235|1
3690|5
209|4
049|4
287
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
6.5
-7|6
.5-7|6
.4-7|5
.7-7|-1.1
-6|-1.2
-6|-6.9
-753.1|5
4.3|4
2.1|4
0.2|2
2.4|1
8.1|1
9.2
be100.7
101;1
01;1
4850
20092|2
0280|1
5472|1
4825|5
281|4
113|3
952
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
2.8
-7|3
.1-7|2
.1-7|2
.4-7|-9.2
-7|-5.1
-7|-6.1
-759.6|5
9.5|4
5.9|4
3.4|2
3.1|1
8.3|1
7.7
be100.8
101;1
01;1
4850
21370|2
0910|1
6458|1
5425|7
231|5
203|4
617
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
3.8
-7|3
.9-7|4
.1-7|2
.9-7|-2.5
-7|-6.3
-7|-6.1
-71:0
3|1
:00|4
7.2|4
4.8|3
2.1|2
3.5|2
1
be100.9
101;1
01;1
4850
18780|2
0839|1
5095|1
3046|5
920|4
366|4
782
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.7
-7|1
.8-7|1
.8-7|1
.7-7|-3.1
-7|-3.8
-7|-3.1
-756.4|1
:04|4
5.9|4
0.2|2
5.6|1
9.7|2
1.6
be100.1
0101;1
01;1
4850
13409|1
2708|1
0008|9
959|4
480|3
685|3
170
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
3.1
-7|2
.5-7|3
.4-7|3
.7-7|-2.5
-7|-2.1
-7|5
.2-7
39.9|3
8.6|3
0.9|3
0.7|2
0.7|1
9.4|1
6.4
be120.3
.1121;1
21;2
1420
36536|3
3139|2
7803|2
5503|1
0934|7
218|7
601
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7
4.4
-7|5
.2-7|4
.3-7|4
.1-7|-6.7
-7|-8.5
-7|-9.9
-72:2
2|2
:16|1
:53|1
:44|1
:06|4
4.5|4
7.5
be120.3
.2121;1
21;2
1420
22024|2
0823|1
7065|1
4387|6
362|5
239|5
933
9.8
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
6.5
-7|6
.2-7|2
.7-7|6
.9-7|-1.6
-7|-1.8
-7|-1.7
-71:2
8|1
:28|1
:12|5
9.8|3
5.5|2
9.5|3
4
be120.3
.3121;1
21;2
1420
30816|3
0566|2
2729|2
1785|6
505|5
025|4
417
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.1
-8|8
.2-9|-1.1
-9|3
.1-9|-8.1
-7|-8.4
-7|-7.9
-72:1
0|2
:14|1
:39|1
:35|3
7.4|2
9|2
5.9
be120.3
.4121;1
21;2
1420
32611|2
6705|2
1582|2
3609|8
907|7
260|7
300
9.9
-7|9
.7-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
3.0
-8|2
.0-7|2
.4-7|5
.8-8|-6.9
-7|-5.4
-7|-8.7
-82:0
9|1
:53|1
:30|1
:38|5
4.6|4
5.3|4
5.3
be120.3
.5121;1
21;2
1420
21167|2
1348|1
7614|1
6896|8
537|5
728|7
127
9.4
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
9.1
-8|6
.8-8|1
.9-8|-2.0
-9|-1.9
-7|-4.8
-7|-1.7
-71:2
5|1
:28|1
:13|1
:11|5
1.1|3
3.5|4
2.5
be120.3
.6121;1
21;2
1420
33348|3
3295|2
6053|2
3919|8
513|6
072|5
620
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.0
-7|-9.8
-8|-9.8
-8|-9.1
-8|-1.2
-7|-5.1
-7|-1.2
-62:1
2|2
:19|1
:49|1
:41|5
4.1|3
9|3
5.9
be120.3
.7121;1
21;2
1420
42467|4
2294|3
2274|2
9722|8
952|7
096|7
464
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
9.2
-8|9
.9-8|9
.6-8|8
.9-8|-6.7
-7|-5.3
-9|-5.0
-72:5
1|2
:57|2
:13|2
:04|4
9.8|4
3|4
4.7
be120.3
.8121;1
21;2
1420
18853|1
8306|1
5051|1
6448|7
598|5
444|5
999
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
8.5
-7|1
.0-6|5
.4-7|3
.7-8|-5.2
-7|-9.3
-7|-3.3
-71:1
6|1
:18|1
:05|1
:09|4
3.5|3
0.6|3
3.9
be120.3
.9121;1
21;2
1420
17200|1
7015|1
3458|1
2528|5
690|4
125|3
967
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.1
-6|1
.2-6|8
.8-7|1
.2-6|-3.8
-7|-3.6
-7|-3.8
-71:0
9|1
:09|5
5.2|5
1|3
1.9|2
4.5|2
3.6
be120.3
.10
121;1
21;2
1420
20854|2
0129|1
6858|1
5738|1
1939|7
416|8
737
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.4
-6|1
.3-6|1
.5-6|1
.4-6|-5.2
-7|-6.7
-7|-5.8
-71:2
4|1
:24|1
:11|1
:06|1
:08|4
2.4|4
9.8
be120.8
.1121;1
21;2
1420
14493|1
5499|1
1470|1
0034|4
996|4
104|3
919
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
9.2
-7|9
.8-7|9
.7-7|9
.9-7|-2.6
-7|-5.6
-8|-9.4
-858|1
:03|4
6.8|4
1.5|2
9.1|2
4.2|2
3.7
be120.8
.2121;1
21;2
1420
24822|2
4477|2
0548|1
8052|9
010|6
647|6
553
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
3.9
-7|3
.9-7|3
.2-7|2
.7-7|-4.0
-7|-3.6
-7|-3.8
-71:4
2|1
:44|1
:29|1
:17|5
3.7|4
0.5|3
9.4
be120.8
.3121;1
21;2
1420
16108|1
7178|1
3140|1
1817|5
080|4
289|4
401
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
5.0
-7|7
.2-7|3
.2-7|5
.5-7|-1.2
-6|-3.7
-7|-5.5
-71:0
5|1
:12|5
6.1|5
0.4|3
2.8|2
9.5|3
0.1
be120.8
.4121;1
21;2
1420
23850|2
3821|1
8243|1
6852|8
511|7
273|5
807
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.0
-6|1
.0-6|9
.9-7|9
.9-7|-6.7
-7|-4.4
-7|-1.0
-61:3
8|1
:41|1
:19|1
:11|4
8|4
2.8|3
3.6
be120.8
.5121;1
21;2
1420
17486|1
8030|1
4479|1
2872|5
917|4
206|4
473
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
4.7
-7|4
.6-7|3
.7-7|3
.5-7|-3.6
-7|-4.5
-7|-2.3
-71:1
2|1
:17|1
:01|5
4.2|3
2.7|2
3.7|2
5.7
be120.8
.6121;1
21;2
1420
16195|1
5975|1
2294|1
1529|5
417|4
386|4
017
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.2
-6|1
.2-6|8
.9-7|1
.2-6|-8.5
-7|-3.6
-7|-1.8
-71:0
7|1
:07|5
1.9|4
8.1|3
1.3|2
6|2
3.7
be120.8
.7121;1
21;2
1420
22803|2
4455|1
9984|1
7313|6
351|5
041|5
300
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
7.0
-8|7
.5-8|5
.2-8|4
.3-8|-4.2
-7|-2.9
-7|-5.9
-71:2
9|1
:41|1
:23|1
:12|3
5.2|2
7.7|2
9.3
be120.8
.8121;1
21;2
1420
18304|1
9368|1
5242|1
2962|5
493|4
389|4
208
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
8.3
-8|5
.8-8|8
.2-8|1
.2-7|-4.5
-7|-3.2
-7|-4.1
-71:1
4|1
:19|1
:05|5
5.9|3
2.2|2
6.6|2
5.4
be120.8
.9121;1
21;2
1420
17717|1
7698|1
4975|1
3722|4
543|3
928|4
201
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|8
.7-7
2.2
-8|8
.0-8|-6.7
-8|-4.2
-8|7
.9-7|5
.0-7|3
.9-7
1:1
5|1
:17|1
:05|5
9.3|2
5.7|2
4|2
5.3
be120.8
.10
121;1
21;2
1420
14772|1
6218|1
2679|1
0876|6
119|4
223|4
365
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.8
-6|1
.9-6|9
.1-7|1
.7-6|-2.3
-7|-2.7
-7|2
.7-7
1:0
1|1
:10|5
4.3|4
6.5|3
4.8|2
5.1|2
6
be150.3
.1151;1
51;3
3525
28554|2
9936|2
3256|2
2393|8
513|6
313|6
043
9.6
-7|9
.9-7|9
.1-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
4.0
-7|4
.7-7|3
.9-7|3
.9-7|-5.6
-7|-1.2
-6|-8.9
-72:5
4|3
:14|2
:34|2
:28|1
:22|1
:05|1
:00
be150.3
.2151;1
51;3
3525
20017|2
0358|1
5903|1
4062|6
680|5
961|5
577
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.9
-6|1
.9-6|1
.8-6|1
.6-6|-4.1
-7|-4.4
-7|-1.6
-71:5
8|2
:07|1
:40|1
:29|1
:02|5
5.4|5
1.7
be150.3
.3151;1
51;3
3525
17823|1
7904|1
3439|1
2084|4
531|3
913|4
286
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.6
-6|1
.6-6|1
.6-6|1
.5-6|-1.0
-6|9
.5-8|-1.2
-71:4
1|1
:49|1
:23|1
:13|3
9.6|3
4.5|3
7.8
be150.3
.4151;1
51;3
3525
42544|4
4008|3
3882|3
1582|1
2465|9
109|1
0139
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7|9
.9-7
1.0
-7|1
.0-7|1
.2-7|1
.1-7|-6.5
-7|-9.3
-7|-3.8
-73:5
1|4
:18|3
:20|3
:06|2
:24|1
:43|1
:55
be150.3
.5151;1
51;3
3525
29602|2
5107|2
1383|1
9189|7
754|5
014|5
531
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
8.1
-8|5
.8-7|3
.1-7|3
.9-7|-2.0
-7|-2.8
-7|-3.1
-72:5
4|2
:38|2
:17|2
:04|1
:18|5
3.5|5
8.6
be150.3
.6151;1
51;3
3525
23547|2
3575|1
9302|1
7081|5
190|4
414|4
375
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7
1.3
-7|1
.8-7|7
.3-8|7
.2-8|4
.3-9|8
.1-8|2
.6-7
2:1
7|2
:24|1
:58|1
:44|4
3.4|3
9|4
0.6
be150.3
.7151;1
51;3
3525
34355|3
8137|2
5627|2
2630|1
3035|1
0054|9
920
9.5
-7|9
.7-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
4.9
-7|4
.3-7|4
.9-7|5
.3-7|-1.3
-6|-1.0
-6|-1.3
-63:2
9|4
:07|2
:42|2
:24|2
:06|1
:37|1
:36
be150.3
.8151;1
51;3
3525
28913|2
9932|2
2314|2
1019|7
357|6
038|5
815
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
2.6
-7|2
.5-7|2
.7-7|3
.4-7|-2.4
-7|5
.2-8|3
.7-8
2:5
1|3
:06|2
:22|2
:14|1
:08|5
6.9|5
4.2
be150.3
.9151;1
51;3
3525
16832|1
7394|1
3964|1
2810|5
428|4
803|4
210
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
8.3
-7|8
.1-7|6
.3-7|6
.7-7|-3.8
-7|-3.1
-7|-3.0
-81:3
2|1
:41|1
:22|1
:13|4
5.9|4
3|3
8.6
be150.3
.10
151;1
51;3
3525
30567|3
3360|2
0147|2
2104|1
0144|7
516|7
208
9.9
-7|9
.0-7|8
.9-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
3.9
-7|3
.1-7|8
.2-7|4
.8-7|-5.2
-7|-6.3
-7|-5.4
-73:0
1|3
:32|2
:08|2
:21|1
:37|1
:13|1
:08
be150.8
.1151;1
51;3
3525
22101|2
1781|1
7326|1
6353|7
068|5
587|5
528
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
6.6
-7|7
.3-7|4
.1-7|5
.2-7|-2.6
-8|-7.5
-8|-1.1
-72:0
6|2
:13|1
:46|1
:41|1
:06|5
3|5
3.8
be150.8
.2151;1
51;3
3525
18626|1
8037|1
4057|1
3687|6
735|5
412|4
746
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
5.3
-7|5
.8-7|4
.6-7|3
.1-7|6
.5-8|2
.9-8|-2.7
-71:4
6|1
:50|1
:28|1
:25|1
:02|5
0.6|4
7.8
be150.8
.3151;1
51;3
3525
36848|3
7270|2
8677|2
7414|9
596|7
847|7
371
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
2.1
-7|2
.1-7|1
.8-7|1
.7-7|-7.3
-7|-9.8
-7|-5.7
-73:4
7|4
:05|3
:10|3
:01|1
:34|1
:17|1
:12
be150.8
.4151;1
51;3
3525
34316|3
7312|2
5461|2
5550|1
0437|7
564|6
639
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
2.9
-7|2
.9-7|3
.0-7|2
.9-7|-2.1
-7|-2.8
-7|-2.1
-73:2
7|4
:01|2
:46|2
:44|1
:41|1
:13|1
:08
be150.8
.5151;1
51;3
3525
28012|2
8619|2
2723|2
1236|5
490|4
792|4
605
9.9
-7|9
.9-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
3.9
-7|4
.3-7|3
.2-7|3
.5-7|-3.0
-7|-5.0
-7|1
.0-6
2:4
9|3
:02|2
:27|2
:15|4
9.8|4
3.4|4
3.5
49
Table
4(contin
ued)
iteratio
nηSD
Pηgap
CP
Utim
e
proble
mm|nE|nI
D-E|P
1.0|P
1.6|P
1.9|iP
1.0|iP
1.6|iP
1.9
D-E|P
1.0|P
1.6|P
1.9|iP
1.0|iP
1.6|iP
1.9
D-E|P
1.0|P
1.6|P
1.9|iP
1.0|iP
1.6|iP
1.9
D-E|P
1.0|P
1.6|P
1.9|iP
1.0|iP
1.6|iP
1.9
be150.8
.6151;1
51;3
3525
24264|2
5728|1
9256|1
7622|6
491|4
839|4
795
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
6.4
-7|6
.2-7|5
.7-7|5
.6-7|-1.4
-6|-1.4
-6|-5.8
-72:2
2|2
:40|2
:02|1
:50|1
:08|5
1.8|5
2.3
be150.8
.7151;1
51;3
3525
31040|3
1624|2
4243|2
2476|7
344|5
761|5
421
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.4
-7|3
.6-7|3
.4-7|3
.2-7|-3.6
-8|-1.4
-7|-3.0
-73:1
2|3
:28|2
:40|2
:27|1
:11|5
5.1|5
2.6
be150.8
.8151;1
51;3
3525
27941|2
7247|2
0450|1
8710|9
339|5
521|6
209
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.4-7
1.2
-7|1
.7-7|9
.8-8|5
.1-8|-3.6
-7|-1.1
-6|-6.9
-72:5
1|2
:56|2
:13|2
:04|1
:33|5
3.7|1
:04
be150.8
.9151;1
51;3
3525
27997|2
8052|1
9827|1
8572|8
516|7
629|6
622
9.9
-7|9
.9-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.3
-7|1
.5-7|4
.1-7|3
.7-7|-3.5
-7|-2.8
-7|-3.2
-72:4
7|2
:59|2
:05|1
:59|1
:21|1
:13|1
:05
be150.8
.10
151;1
51;3
3525
23508|2
4124|1
9173|1
8945|6
724|5
294|5
205
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.3-7|9
.9-7|9
.9-7
3.3
-7|3
.9-7|2
.0-7|1
.7-7|-9.5
-7|-4.3
-7|-5.4
-72:2
2|2
:34|2
:04|2
:01|1
:05|5
3.1|5
2.5
be200.3
.1201;2
01;5
9700
29477|2
8767|2
2306|2
0318|6
960|5
912|6
226
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.6
-7|1
.9-7|1
.7-7|1
.1-7|-3.9
-7|-7.2
-7|8
.9-7
4:2
3|4
:41|3
:42|3
:24|1
:50|1
:35|1
:47
be200.3
.2201;2
01;5
9700
25578|2
5807|1
7126|1
5520|6
696|6
063|5
571
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
9.2
-7|9
.2-7|1
.4-6|1
.5-6|-6.6
-7|-7.5
-8|1
.0-8
3:4
7|4
:14|2
:44|2
:26|1
:43|1
:36|1
:29
be200.3
.3201;2
01;5
9700
30087|3
0635|2
4874|2
3386|9
016|7
579|6
902
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7
7.7
-7|7
.8-7|4
.6-7|4
.1-7|-1.1
-6|-9.3
-7|-1.0
-64:3
5|5
:06|4
:13|3
:59|2
:22|2
:02|1
:51
be200.3
.4201;2
01;5
9700
46338|4
6575|3
7079|3
4330|9
908|6
601|6
075
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
3.2
-9|3
.1-8|-2.9
-9|-4.6
-9|-9.9
-7|-5.6
-7|-7.4
-77:2
9|8
:04|6
:26|5
:56|2
:58|2
:02|1
:52
be200.3
.5201;2
01;5
9700
32249|3
4880|2
7644|2
3057|7
131|5
928|5
954
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.7-7
3.5
-7|2
.9-7|2
.3-7|2
.7-7|-5.4
-7|-3.6
-7|-2.3
-75:0
1|5
:57|4
:43|3
:51|1
:49|1
:36|1
:37
be200.3
.6201;2
01;5
9700
29111|2
8738|2
3565|2
1866|6
999|4
948|5
292
9.9
-7|9
.9-7|9
.9-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
3.7
-7|5
.6-7|2
.1-7|1
.9-7|-1.0
-6|-5.7
-7|8
.7-8
4:2
6|4
:41|3
:54|3
:40|1
:44|1
:17|1
:24
be200.3
.7201;2
01;5
9700
35410|3
5805|2
8035|2
5610|1
4332|1
1053|9
554
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
5.8
-7|5
.5-7|4
.5-7|4
.9-7|-1.4
-7|-1.9
-7|-1.8
-75:2
7|5
:57|4
:49|4
:24|3
:52|3
:16|2
:38
be200.3
.8201;2
01;5
9700
43650|4
5249|3
7113|3
4005|1
0102|7
651|7
160
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.1
-7|1
.1-7|7
.1-8|6
.6-8|-9.8
-7|-7.2
-7|-8.7
-76:5
5|7
:54|6
:22|5
:49|2
:44|2
:02|1
:57
be200.3
.9201;2
01;5
9700
42442|4
3126|3
3676|3
0112|8
642|7
773|6
776
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
7.3
-8|9
.3-8|5
.2-8|5
.4-8|-2.3
-7|-6.4
-7|-2.5
-76:3
4|7
:17|5
:45|5
:06|2
:34|2
:21|2
:07
be200.3
.10
201;2
01;5
9700
21971|2
2213|2
1379|1
8907|6
456|4
963|5
803
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
5.6
-7|6
.2-7|5
.9-8|9
.2-8|-1.1
-6|-3.5
-8|-1.7
-83:1
6|3
:35|3
:31|3
:09|1
:41|1
:27|1
:45
be200.8
.1201;2
01;5
9700
41652|4
2441|3
1324|3
0128|1
3327|9
510|8
566
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
2.1
-7|2
.7-7|1
.5-7|1
.4-7|-3.2
-7|-5.4
-7|-3.9
-76:4
2|7
:27|5
:30|5
:17|3
:42|2
:37|2
:22
be200.8
.2201;2
01;5
9700
26502|2
6103|2
0051|1
8601|7
538|6
387|5
412
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
3.2
-7|4
.8-7|3
.8-7|4
.4-7|-2.3
-8|-6.3
-8|3
.1-7
3:5
8|4
:12|3
:17|3
:02|1
:56|1
:40|1
:29
be200.8
.3201;2
01;5
9700
37333|3
7889|2
8210|2
6797|1
1003|8
411|7
938
9.9
-7|9
.9-7|9
.8-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
2.3
-7|2
.4-7|2
.6-7|2
.2-7|-4.2
-7|-6.7
-7|-4.6
-75:4
5|6
:21|4
:43|4
:28|3
:03|2
:27|2
:21
be200.8
.4201;2
01;5
9700
32888|3
2062|2
5654|2
4081|7
527|7
312|6
056
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7|9
.9-7|9
.1-7
2.0
-7|2
.7-7|1
.7-7|1
.8-7|-5.0
-7|-4.0
-7|-7.0
-75:0
4|5
:19|4
:19|4
:04|2
:01|1
:59|1
:43
be200.8
.5201;2
01;5
9700
38925|3
8943|2
8609|2
6971|9
550|7
412|5
227
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
5.6
-8|7
.9-8|1
.0-7|9
.3-8|-4.7
-7|-4.8
-7|-6.1
-76:1
0|6
:39|4
:57|4
:27|2
:31|1
:57|1
:25
be200.8
.6201;2
01;5
9700
31948|3
2339|2
3412|2
2468|1
4473|9
133|9
961
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.6
-6|1
.7-6|1
.5-6|1
.5-6|-3.0
-7|-7.1
-7|-3.0
-74:4
4|5
:15|3
:49|3
:37|3
:56|2
:28|2
:44
be200.8
.7201;2
01;5
9700
37078|3
6472|2
9123|2
5885|8
613|7
075|6
355
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
4.8
-8|9
.3-8|4
.4-8|4
.2-8|-5.9
-7|5
.5-9|-3.7
-75:4
4|6
:08|5
:01|4
:27|2
:20|1
:55|1
:47
be200.8
.8201;2
01;5
9700
35815|3
7470|2
6962|2
4955|1
4921|1
2354|1
1543
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
3.0
-7|3
.1-7|3
.3-7|2
.0-7|-4.6
-7|-3.1
-7|-4.0
-75:2
4|6
:07|4
:22|4
:00|4
:27|3
:50|3
:36
be200.8
.9201;2
01;5
9700
24808|2
5601|1
9909|1
8903|8
739|6
843|5
944
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.2
-6|1
.3-6|7
.6-7|6
.8-7|-2.5
-7|9
.5-8|-2.3
-73:3
5|4
:00|3
:11|3
:03|2
:16|1
:50|1
:39
be200.8
.10
201;2
01;5
9700
25044|2
6022|1
9027|1
7467|7
755|5
432|5
219
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.7
-6|1
.7-6|1
.5-6|1
.5-6|-4.0
-7|-5.8
-7|6
.5-8
3:3
3|3
:57|2
:57|2
:40|1
:57|1
:24|1
:22
be250.1
251;2
51;9
3375
60594|5
9445|4
7902|4
4538|1
7487|1
3380|1
2119
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
2.3
-7|2
.3-7|2
.1-7|2
.2-7|-4.9
-7|-5.4
-7|-5.6
-713:5
6|1
5:1
4|1
2:2
2|1
1:2
9|7
:38|5
:53|5
:21
be250.2
251;2
51;9
3375
52532|5
6803|4
1194|3
8447|1
2725|9
050|9
634
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
6.1
-8|5
.0-8|5
.1-8|4
.2-8|-5.2
-7|-3.9
-7|-4.6
-712:1
4|1
4:3
0|1
0:3
9|9
:53|5
:28|3
:58|4
:17
be250.3
251;2
51;9
3375
48310|4
9846|4
0110|3
6528|2
2337|1
8339|1
6113
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.2
-6|1
.6-6|1
.1-6|1
.3-6|-4.6
-7|-3.3
-7|-5.5
-710:5
5|1
2:3
4|1
0:0
5|9
:10|9
:45|8
:07|7
:09
be250.4
251;2
51;9
3375
55860|5
7006|4
3507|4
1110|1
8580|1
5409|1
4453
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
3.5
-7|3
.8-7|3
.2-7|3
.4-7|-3.4
-7|-3.5
-7|-3.4
-713:0
5|1
4:4
2|1
1:1
7|1
0:4
2|8
:18|7
:00|6
:40
be250.5
251;2
51;9
3375
32841|3
3106|2
8314|2
6201|1
3476|8
600|1
0101
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
7.8
-7|1
.0-6|4
.2-7|3
.3-7|-2.7
-7|-2.8
-7|-2.5
-77:0
8|7
:56|7
:00|6
:29|5
:31|3
:48|4
:35
be250.6
251;2
51;9
3375
58449|5
9628|4
2132|4
1389|1
7968|1
3180|1
2357
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
9.3
-8|8
.9-8|1
.0-7|8
.2-8|-2.1
-7|-2.0
-7|-2.4
-713:2
9|1
5:1
8|1
0:4
8|1
0:3
7|7
:56|5
:57|5
:41
be250.7
251;2
51;9
3375
38639|3
7397|3
0589|2
7266|1
5412|1
2418|1
0112
9.9
-7|9
.9-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.2
-6|1
.3-6|1
.1-6|1
.1-6|-3.6
-7|-4.9
-7|-5.0
-78:4
1|9
:23|7
:45|6
:52|6
:40|5
:27|4
:30
be250.8
251;2
51;9
3375
58942|5
9801|4
0847|4
3010|1
8511|1
5377|1
4505
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
3.4
-7|3
.4-7|3
.1-7|3
.3-7|-3.9
-7|-3.4
-7|-3.5
-714:2
6|1
6:0
4|1
1:0
6|1
1:3
4|8
:10|6
:55|6
:31
be250.9
251;2
51;9
3375
32283|3
1800|2
5065|2
3395|8
254|6
464|7
088
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
7.3
-7|1
.1-6|5
.9-7|5
.2-7|5
.1-8|-3.2
-8|-7.8
-87:0
8|7
:41|6
:08|5
:43|3
:21|2
:42|3
:00
be250.1
0251;2
51;9
3375
42664|4
5466|3
2794|3
2014|1
5930|1
1483|1
1013
9.9
-7|9
.9-7|9
.6-7|9
.7-7|9
.9-7|9
.9-7|9
.9-7
9.1
-7|9
.0-7|1
.0-6|9
.6-7|-3.4
-7|-4.2
-7|-5.2
-79:3
6|1
1:2
2|8
:10|8
:05|6
:52|5
:01|4
:48
bqp50-1
51;5
1;3
675
14720|1
4287|1
1974|9
908|2
498|2
540|3
099
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-4.0
-7|-3.4
-7|-3.9
-7|-3.9
-7|-8.5
-7|-7.1
-7|-9.2
-718.7|1
9.8|1
7.3|1
3.7|4
.1|4
.2|5
.3
bqp50-2
51;5
1;3
675
18994|1
9368|1
4001|1
3567|9
276|5
145|4
195
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.5-7
-3.3
-7|-3.0
-7|-2.8
-7|-3.0
-7|-3.7
-8|-8.8
-8|-3.5
-723|2
6.2|1
9.5|1
8.8|1
5.4|8
.5|7
.2
bqp50-3
51;5
1;3
675
177243|6
8211|7
4341|5
6165|1
2257|9
752|1
1149
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.1
-7|-2.3
-7|5
.8-8|3
.8-7|-3.7
-7|-3.7
-7|-3.5
-73:4
2|1
:34|1
:43|1
:18|1
9.6|1
6.5|1
9.2
bqp50-4
51;5
1;3
675
157203|1
62674|1
30657|1
20852|2
8763|2
1304|2
0760
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7|9
.9-7|9
.9-7
-2.3
-7|-2.7
-7|-2.9
-7|-2.5
-7|-3.1
-7|-5.1
-7|-4.5
-73:2
9|3
:59|3
:11|3
:04|5
2.4|5
1.5|5
0
bqp50-5
51;5
1;3
675
109308|1
04266|7
9469|7
8615|1
9132|1
2946|1
3897
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7
-2.2
-7|-1.6
-7|-1.7
-7|-2.5
-7|-2.9
-7|-2.9
-7|-2.8
-72:5
6|2
:38|1
:56|1
:53|3
3.3|2
3.2|2
4.9
bqp50-6
51;5
1;3
675
184039|2
04810|2
32612|1
11458|3
4091|3
9132|4
0007
9.9
-7|9
.7-7|9
.9-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
9.9
-8|6
.2-8|-6.0
-8|2
.0-7|4
.4-7|-5.4
-8|-2.0
-83:5
8|4
:58|5
:43|2
:43|5
5|1
:10|1
:13
bqp50-7
51;5
1;3
675
55655|5
6399|4
4561|3
9690|9
918|7
648|7
503
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.5
-7|-2.7
-7|-2.6
-7|-2.7
-7|-4.2
-7|-3.5
-7|-4.3
-71:1
2|1
:20|1
:03|5
6|1
8.1|1
3.8|1
3.8
bqp50-8
51;5
1;3
675
61129|7
7405|6
0239|5
9698|1
1146|8
679|1
2535
9.9
-7|9
.9-7|9
.9-7|9
.9-7|8
.8-7|9
.4-7|9
.9-7
2.4
-8|1
.5-8|-3.1
-8|-6.6
-8|-1.4
-7|-2.0
-7|-2.1
-71:2
1|1
:53|1
:29|1
:27|1
9.6|1
5.4|2
3.2
bqp50-9
51;5
1;3
675
77506|7
5610|5
8591|5
7791|1
3151|1
0742|1
2249
9.9
-7|9
.9-7|9
.8-7|9
.9-7|9
.8-7|9
.9-7|9
.9-7
-1.9
-7|-1.4
-7|-1.3
-7|-2.2
-7|-2.8
-7|-3.6
-7|-1.1
-71:4
3|1
:50|1
:25|1
:24|2
1|1
8.7|2
1.7
bqp50-1
051;5
1;3
675
47014|4
6988|3
7824|3
4880|1
0039|7
604|7
565
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.9
-7|-2.0
-7|-1.9
-7|-2.0
-7|-5.2
-7|-5.2
-7|-5.4
-71:0
1|1
:07|5
3.8|4
9.3|1
7|1
2.3|1
2.2
bqp100-1
101;1
01;1
4850
16118|1
8635|1
2822|1
1305|5
048|4
871|4
001
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.1
-6|6
.9-7|9
.0-7|9
.2-7|-5.4
-7|-3.1
-7|-6.4
-749.7|5
8.8|3
9.9|3
4.7|2
2.7|2
2.6|1
8.5
bqp100-2
101;1
01;1
4850
60256|5
8439|5
3590|4
7623|1
3614|1
1747|1
1130
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-8.9
-8|-1.1
-7|-7.3
-8|-7.3
-8|-1.3
-7|-1.2
-7|-1.3
-73:0
1|2
:58|2
:47|2
:28|1
:04|5
8.5|5
8.6
bqp100-3
101;1
01;1
4850
233230|2
31991|1
82150|1
69031|4
9066|3
7990|3
4805
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.8-7|8
.6-7
-9.4
-8|-8.8
-8|-8.1
-8|-6.7
-8|-4.1
-7|-4.8
-7|-4.4
-712:3
1|1
2:4
2|9
:51|9
:05|3
:59|3
:32|3
:16
bqp100-4
101;1
01;1
4850
40048|4
4442|3
2258|2
9492|9
772|7
448|7
715
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
4.3
-8|-1.4
-7|-2.5
-8|-1.1
-8|-1.1
-6|-1.2
-6|-1.2
-62:0
2|2
:18|1
:40|1
:32|4
5|3
4|3
5.6
bqp100-5
101;1
01;1
4850
53259|5
2618|4
4316|3
9413|7
204|5
785|6
226
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.7
-8|2
.1-8|-2.3
-8|-3.1
-8|-8.2
-7|-1.2
-6|-1.1
-62:4
5|2
:48|2
:24|2
:09|3
3.2|2
6.7|2
9.1
bqp100-6
101;1
01;1
4850
20692|2
1092|1
5795|1
5390|6
514|5
540|4
436
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-8|5
.0-8|2
.7-8|-2.3
-8|-1.6
-7|-3.6
-7|-9.0
-71:0
4|1
:06|4
9.3|4
8.2|2
9.5|2
5.6|1
9.9
bqp100-7
101;1
01;1
4850
49979|4
8122|3
9677|3
7153|9
132|6
657|7
358
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.3
-7|-2.4
-7|-2.4
-7|-2.3
-7|-1.3
-6|-9.0
-7|-1.4
-62:3
5|2
:32|2
:05|2
:00|4
1.9|3
2.9|3
4.9
50
Table
4(contin
ued)
iteratio
nηSD
Pηgap
CP
Utim
e
proble
mm|nE|nI
D-E|P
1.0|P
1.6|P
1.9|iP
1.0|iP
1.6|iP
1.9
D-E|P
1.0|P
1.6|P
1.9|iP
1.0|iP
1.6|iP
1.9
D-E|P
1.0|P
1.6|P
1.9|iP
1.0|iP
1.6|iP
1.9
D-E|P
1.0|P
1.6|P
1.9|iP
1.0|iP
1.6|iP
1.9
bqp100-8
101;1
01;1
4850
118113|1
17553|9
2922|8
5497|2
8883|2
3297|2
1405
9.9
-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.4
-8|-1.9
-9|-2.7
-8|-2.5
-8|-5.1
-7|-4.8
-7|-5.1
-76:0
8|6
:12|4
:50|4
:22|2
:15|1
:54|1
:54
bqp100-9
101;1
01;1
4850
80369|7
8808|6
4974|5
9661|1
5877|1
5069|1
3732
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-4.0
-8|-3.6
-8|-7.5
-8|-5.2
-8|-5.7
-7|-5.0
-7|-5.3
-74:0
7|4
:06|3
:20|3
:05|1
:11|1
:10|1
:06
bqp100-1
0101;1
01;1
4850
125337|1
25310|1
00168|9
1825|2
5485|1
9581|1
8711
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.9
-7|-1.9
-7|-1.8
-7|-1.8
-7|-5.8
-7|-5.8
-7|-6.8
-76:3
8|6
:44|5
:21|4
:50|1
:59|1
:47|1
:45
bqp250-1
251;2
51;9
3375
75186|6
2326|5
7991|5
4154|1
5516|1
0520|1
0715
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
8.1
-8|3
.9-7|7
.4-8|7
.4-8|-1.2
-6|-1.9
-6|-1.0
-618:2
1|1
6:3
3|1
5:3
6|1
4:3
9|7
:02|4
:56|5
:07
bqp250-2
251;2
51;9
3375
53456|4
5022|4
2776|3
8826|1
3881|1
1713|1
0640
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.7
-7|3
.7-7|1
.2-7|1
.2-7|-4.5
-7|-4.3
-7|-3.6
-712:2
4|1
1:2
0|1
0:5
1|9
:56|5
:57|5
:14|4
:57
bqp250-3
251;2
51;9
3375
51906|5
6050|3
9679|3
9434|1
7002|1
3824|1
2779
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
4.8
-7|4
.2-7|5
.2-7|4
.2-7|-4.0
-7|-3.8
-7|-3.9
-712:0
2|1
4:1
9|1
0:0
9|1
0:0
5|7
:22|6
:12|5
:43
bqp250-4
251;2
51;9
3375
40753|3
8314|3
3944|2
6351|1
0715|8
181|7
554
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.4-7|9
.9-7
2.3
-7|5
.2-7|1
.9-7|4
.7-7|-1.2
-6|-1.2
-6|-1.1
-69:0
4|9
:20|8
:30|6
:29|4
:42|3
:47|3
:40
bqp250-5
251;2
51;9
3375
53411|5
4251|4
4392|3
9628|1
6818|1
2554|1
2191
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.3
-7|1
.6-7|7
.7-8|9
.4-8|-2.8
-7|-2.9
-7|-2.6
-713:0
0|1
4:3
3|1
2:0
8|1
0:3
2|7
:44|6
:02|5
:59
bqp250-6
251;2
51;9
3375
40439|3
9815|3
0536|2
8314|8
210|7
204|7
103
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.8
-7|2
.2-7|1
.4-7|1
.5-7|-4.4
-7|-1.6
-7|-1.7
-79:1
7|1
0:0
3|7
:52|7
:13|3
:27|3
:08|3
:12
bqp250-7
251;2
51;9
3375
47536|4
9671|3
7014|3
6922|1
2214|9
303|7
175
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
3.7
-7|3
.7-7|3
.3-7|2
.9-7|-1.0
-6|-8.0
-7|-7.1
-711:1
6|1
2:4
1|9
:32|9
:33|5
:22|4
:12|3
:14
bqp250-8
251;2
51;9
3375
33602|3
4460|2
8002|2
5838|8
024|6
574|6
268
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
4.3
-7|4
.5-7|2
.9-7|3
.6-7|-2.5
-7|-1.3
-7|9
.0-7
7:1
7|8
:15|6
:48|6
:14|3
:17|2
:50|3
:01
bqp250-9
251;2
51;9
3375
54808|5
5226|4
1413|3
7864|1
6012|1
1409|1
0816
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.1
-7|1
.2-7|1
.3-7|1
.4-7|-6.0
-7|-6.6
-7|-5.5
-712:4
7|1
4:1
7|1
0:4
0|9
:43|6
:47|4
:59|4
:46
bqp250-1
0251;2
51;9
3375
33608|2
9942|2
8932|2
6666|9
832|7
716|7
302
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.3-7
6.2
-7|1
.7-6|3
.3-7|3
.7-7|-4.1
-7|-5.6
-7|-1.5
-77:1
9|7
:08|7
:10|6
:36|4
:00|3
:13|3
:10
bqp500-1
501;5
01;3
74250
500000|5
00000|5
00000|4
7251|2
0402|1
6524|1
3689
3.1
-4|2
.2-2|1
.4-2|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.9
-6|-5.3
-4|-4.1
-4|2
.8-7|-2.4
-8|-2.7
-8|-1.6
-710:3
7:2
0|13:0
2:0
3|12:3
8:3
0|1:1
6:4
3|4
8:5
7|4
1:4
7|3
6:1
6
bqp500-2
501;5
01;3
74250
500000|5
00000|5
00000|5
3886|2
6250|1
9700|1
8089
5.2
-3|1
.8-3|9
.3-3|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.2
-4|1
.5-4|3
.4-4|5
.5-7|-1.5
-7|-1.7
-7|-1.5
-710:4
5:5
2|12:5
1:4
2|12:5
2:0
6|1:3
0:1
4|1:0
3:1
6|4
9:3
3|4
7:2
9
bqp500-3
501;5
01;3
74250
500000|5
00000|5
00000|5
2209|2
4254|1
5924|1
1716
6.6
-4|2
.6-3|1
.0-2|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.8
-5|1
.1-4|3
.5-4|3
.1-7|-2.8
-7|-3.0
-7|-3.2
-710:5
4:1
4|12:4
9:2
8|13:0
3:3
9|1:3
0:3
1|1:0
2:0
8|4
1:1
3|3
1:2
9
bqp500-4
501;5
01;3
74250
500000|5
00000|6
5512|5
9933|1
8255|1
4448|1
2398
3.3
-3|2
.5-3|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.7
-4|-2.6
-4|2
.3-7|2
.1-7|-1.5
-6|-1.3
-6|-1.0
-610:5
0:4
1|13:0
8:5
2|1:5
2:4
0|1:4
4:0
9|4
3:2
2|3
6:0
5|3
1:0
6
bqp500-5
501;5
01;3
74250
500000|5
00000|5
00000|5
5816|2
0575|1
4599|1
5020
1.4
-3|2
.3-3|3
.0-3|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-10.0
-5|8
.0-5|-2.7
-5|1
.0-7|-2.4
-7|2
.5-8|-2.8
-710:5
5:1
4|13:2
5:0
2|13:0
8:5
6|1:3
2:5
2|4
9:4
0|3
6:5
6|3
8:2
5
bqp500-6
501;5
01;3
74250
500000|5
00000|5
00000|4
1561|2
2987|1
6783|1
4644
5.2
-4|7
.0-4|6
.5-3|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-6.8
-6|1
.5-5|2
.4-4|1
.3-6|-2.7
-7|-2.7
-7|-3.9
-711:2
0:0
3|13:0
3:0
8|13:0
8:1
8|1:0
5:4
1|5
5:4
3|4
3:1
9|3
8:1
4
bqp500-7
501;5
01;3
74250
500000|5
00000|5
00000|5
00000|1
4723|1
5152|9
156
1.1
-3|2
.0-3|1
.3-2|8
.8-3|9
.9-7|9
.9-7|9
.9-7
-6.0
-5|6
.9-5|-3.8
-4|-2.6
-4|-6.9
-7|-7.7
-7|-4.3
-710:5
9:4
2|13:1
4:2
6|13:1
6:2
2|13:1
5:2
8|3
6:3
3|3
8:0
3|2
4:1
9
bqp500-8
501;5
01;3
74250
500000|5
00000|5
00000|6
4442|2
4814|1
7420|1
1342
1.1
-3|2
.5-3|1
.3-2|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-4.4
-5|-8.4
-5|-3.4
-4|4
.1-7|-4.2
-7|-4.7
-7|-5.1
-711:2
9:1
2|13:3
3:2
8|13:5
5:4
2|1:4
9:5
6|1:0
3:0
1|4
5:5
8|3
0:0
6
bqp500-9
501;5
01;3
74250
500000|5
00000|5
00000|4
3377|1
9576|1
5101|1
1199
3.1
-4|5
.8-4|1
.0-2|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-5.0
-7|7
.6-5|2
.7-4|5
.0-7|-2.7
-7|-7.7
-8|-9.1
-811:2
0:2
5|13:4
9:3
9|14:0
3:4
4|1:1
5:4
3|5
1:0
9|4
2:0
1|3
2:2
7
bqp500-1
0501;5
01;3
74250
500000|5
00000|6
5291|5
9373|2
4430|1
9293|1
5617
2.6
-3|9
.9-4|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.0
-4|-1.4
-5|2
.2-7|1
.7-7|-3.3
-7|-3.5
-7|-4.2
-711:3
1:5
6|14:1
6:1
5|1:5
5:3
5|1:4
3:5
1|1:0
0:4
7|5
0:1
3|4
2:4
3
gka1a
51;5
1;3
675
34150|3
4737|2
7580|2
6284|4
475|4
398|5
040
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.9
-7|-1.8
-7|-2.1
-7|-2.1
-7|6
.4-8|1
.9-7|-4.2
-747.4|5
0.1|4
0|3
8.1|7
.9|8
.2|9
.3
gka2a
61;6
1;5
310
124262|7
0057|4
3222|4
9568|9
404|7
627|7
871
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
4.5
-7|3
.8-7|4
.1-7|-7.9
-8|-1.5
-7|-1.4
-7|-1.1
-73:0
4|1
:55|1
:11|1
:24|2
3.2|1
9.5|2
0.4
gka3a
71;7
1;7
245
31998|3
1107|2
7090|2
5067|6
936|5
774|5
147
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.7-7|9
.9-7
-1.5
-7|-1.2
-7|-1.9
-7|-1.4
-7|7
.0-8|-4.6
-7|-6.4
-755.4|1
:03|5
4.2|5
0.6|2
0.4|1
6.5|1
5.1
gka8a
101;1
01;1
4850
301490|5
00000|3
61738|3
13698|3
8872|3
1784|3
6337
9.9
-7|1
.8-6|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
9.9
-8|-1.4
-7|-3.3
-8|-4.7
-8|-2.9
-7|-2.6
-7|-2.8
-716:0
5|3
0:1
9|2
2:4
5|1
8:5
3|3
:30|3
:18|3
:40
gka9b
101;1
01;1
4850
7484|7
734|5
039|4
725|1
245|9
47|8
33
9.9
-7|9
.9-7|9
.4-7|9
.8-7|9
.9-7|9
.9-7|9
.9-7
-2.1
-7|-2.0
-7|-2.0
-8|-1.2
-8|-4.7
-7|-4.3
-7|-2.7
-723.6|2
5|1
5.8|1
5.3|8
.1|6|5
.4
gka10b
126;1
26;2
3250
13125|1
3033|1
0232|9
281|1
673|1
340|1
113
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.6
-7|-1.6
-7|-1.6
-7|-1.6
-7|-3.2
-7|-3.2
-7|-4.8
-752.8|5
7.2|4
5.3|4
1.8|1
3.2|1
0.8|8
.9
gka7c
101;1
01;1
4850
270591|2
70353|2
13183|1
99923|6
2652|5
0853|4
5771
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-9.4
-9|-2.6
-9|-1.7
-8|-5.1
-8|-4.5
-7|-4.6
-7|-4.5
-714:5
9|1
5:5
9|1
2:5
2|1
1:4
9|5
:42|4
:42|4
:39
gka1d
101;1
01;1
4850
153347|1
45059|1
17904|1
00814|2
6213|2
1022|1
9709
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.4
-7|-1.1
-7|-1.4
-7|-1.1
-7|-7.0
-7|-7.6
-7|-7.4
-78:2
5|8
:32|6
:43|5
:39|2
:09|1
:47|1
:46
gka2d
101;1
01;1
4850
12761|1
4458|1
1305|9
309|4
388|3
881|3
879
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
8.2
-7|9
.0-7|8
.8-7|1
.1-6|-2.2
-7|-4.5
-8|-1.8
-740.2|4
8.3|3
8.6|3
1.6|2
1.2|2
0|2
1.1
gka3d
101;1
01;1
4850
18783|2
0375|1
6135|1
4463|7
068|4
943|5
486
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
5.0
-7|3
.8-7|4
.2-7|3
.5-7|-3.4
-7|-2.8
-7|-2.9
-759.5|1
:07|5
3.5|4
7.8|3
5.7|2
7.7|3
0.5
gka4d
101;1
01;1
4850
17779|1
8959|1
5273|1
3901|5
723|4
644|4
610
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
2.7
-7|4
.4-7|4
.7-7|4
.8-7|-3.1
-7|-2.6
-7|-3.1
-759|1
:05|5
3|4
7.9|2
8|2
4.1|2
4.4
gka5d
101;1
01;1
4850
24810|2
4278|1
9334|1
7603|5
272|3
978|3
778
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
2.9
-8|4
.6-8|4
.6-8|3
.7-8|-2.4
-7|-1.9
-7|-4.6
-71:2
0|1
:23|1
:06|1
:01|2
6|2
0.5|1
9.1
gka6d
101;1
01;1
4850
31575|3
2317|2
4772|2
2905|8
156|5
738|5
792
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.6
-7|-1.7
-7|-1.5
-7|-1.6
-7|-5.7
-7|-8.9
-7|-1.0
-61:4
1|1
:49|1
:23|1
:17|4
1.1|2
8.4|2
9.1
gka7d
101;1
01;1
4850
14412|1
4697|1
1980|1
1477|5
310|4
247|4
341
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
7.0
-7|9
.8-7|8
.0-7|6
.6-7|-1.6
-6|-9.7
-7|-7.6
-745.8|5
0.1|4
1|3
9.3|2
6.9|2
1.7|2
2.5
gka8d
101;1
01;1
4850
18589|1
7501|1
3844|1
2181|6
633|4
798|4
523
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
4.7
-7|6
.7-7|7
.5-7|8
.4-7|-2.6
-7|-8.0
-7|-1.0
-659.4|5
8.8|4
7.4|4
1.6|3
3.6|2
4|2
2.8
gka9d
101;1
01;1
4850
17204|1
7719|1
2980|1
2321|5
517|4
538|4
410
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
2.4
-7|3
.1-7|2
.8-7|2
.6-7|-7.6
-7|-7.0
-7|-3.6
-755.5|1
:00|4
3.4|4
1.7|2
6.7|2
2.6|2
2.3
gka10d
101;1
01;1
4850
23462|2
1138|1
7548|1
5506|9
770|7
935|7
770
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-2.7
-8|-2.8
-7|-3.0
-7|-2.7
-7|-2.5
-7|-2.5
-7|-2.8
-71:1
5|1
:12|5
8.5|5
1.8|4
9.4|4
0.3|3
9.7
gka1e
201;2
01;5
9700
84099|8
4010|6
8161|6
1687|2
1129|1
7712|1
6953
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
8.9
-8|9
.6-8|1
.7-8|4
.2-8|-5.3
-7|-4.5
-7|-4.4
-713:3
1|1
5:0
2|1
2:2
2|1
1:0
4|6
:19|5
:45|5
:33
gka2e
201;2
01;5
9700
43286|4
3298|3
4608|3
1139|1
0368|8
001|7
451
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.5
-8|-1.3
-8|-4.7
-8|-4.9
-8|-2.1
-7|-2.2
-7|-2.6
-76:3
5|7
:22|5
:54|5
:23|2
:56|2
:16|2
:03
gka3e
201;2
01;5
9700
28104|2
7944|2
0565|1
9928|8
913|7
382|6
417
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.2
-6|1
.2-6|1
.2-6|1
.1-6|-2.9
-7|-2.8
-7|-3.3
-74:0
5|4
:35|3
:22|3
:15|2
:30|2
:08|1
:58
gka4e
201;2
01;5
9700
45967|4
6652|3
5948|3
4469|9
708|7
994|7
284
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
1.1
-7|1
.0-7|1
.2-7|1
.1-7|-1.1
-6|-9.3
-7|-8.9
-77:2
4|8
:20|6
:28|6
:10|2
:49|2
:18|2
:07
gka5e
201;2
01;5
9700
41178|3
9322|3
5176|3
2979|9
756|6
259|6
181
9.9
-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7|9
.9-7
2.3
-7|2
.9-7|1
.2-7|1
.1-7|-3.1
-7|-6.4
-7|-3.6
-76:2
6|6
:55|6
:13|5
:50|2
:52|1
:50|1
:52
gka1f
501;5
01;3
74250
500000|5
00000|5
00000|3
6996|2
0529|1
3666|1
2717
1.0
-3|1
.1-3|8
.6-3|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-7.1
-5|1
.3-5|4
.0-4|1
.4-6|-2.4
-7|-2.4
-7|-7.0
-711:2
7:0
3|13:4
1:1
7|14:0
2:2
4|1:0
2:3
3|5
2:0
2|3
6:2
7|3
4:3
0
gka2f
501;5
01;3
74250
500000|5
00000|5
00000|5
1262|1
8588|1
1922|1
1993
3.8
-3|2
.0-3|5
.1-3|9
.9-7|9
.9-7|9
.9-7|9
.6-7
-3.9
-5|2
.4-4|2
.7-4|1
.7-7|-3.8
-7|2
.5-8|5
.8-8
11:5
8:4
2|14:0
3:1
0|14:0
2:0
0|1:3
4:2
4|4
9:3
5|3
2:4
1|3
3:0
0
gka3f
501;5
01;3
74250
500000|5
00000|5
00000|5
1268|1
6420|1
4608|1
2985
2.0
-3|2
.1-3|5
.2-3|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.1
-5|-2.0
-5|8
.3-5|2
.3-7|-1.1
-8|-1.9
-7|-2.5
-712:0
9:5
8|14:4
1:3
9|14:1
4:5
3|1:3
2:5
3|4
3:0
7|3
9:2
7|3
6:4
1
gka4f
501;5
01;3
74250
500000|5
00000|5
00000|5
6339|1
8917|1
4679|1
1215
6.4
-3|1
.7-3|6
.7-3|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-3.5
-4|-1.7
-4|9
.1-5|9
.0-8|-8.4
-7|-9.3
-7|-1.0
-611:4
6:3
8|14:4
1:0
2|14:0
4:5
1|1:4
3:4
6|4
8:1
1|3
8:4
7|3
0:5
9
gka5f
501;5
01;3
74250
500000|5
00000|5
00000|4
7188|2
3314|1
8453|1
6909
1.1
-3|2
.0-3|2
.3-2|9
.9-7|9
.9-7|9
.9-7|9
.9-7
-1.7
-5|5
.0-6|6
.2-4|7
.2-7|-1.6
-7|-1.6
-7|-1.7
-711:4
1:1
4|14:0
7:2
8|14:1
9:3
9|1:2
1:2
1|1:0
0:3
1|4
9:4
8|5
1:0
1
51