karmarkar algorithm
DESCRIPTION
Karmarkar Algorithm. Anup Das, Aissan Dalvandi, Narmada Sambaturu, Seung Min, and Le Tuan Anh. Contents. Overview Projective transformation Orthogonal projection Complexity analysis Transformation to Karmarkar’s canonical form. LP Solutions. Simplex Dantzig 1947 Ellipsoid - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/1.jpg)
Karmarkar AlgorithmAnup Das, Aissan Dalvandi, Narmada Sambaturu, Seung Min, and Le Tuan Anh
1
![Page 2: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/2.jpg)
Contents• Overview
• Projective transformation
• Orthogonal projection
• Complexity analysis
• Transformation to Karmarkar’s canonical form
2
![Page 3: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/3.jpg)
LP Solutions• Simplex• Dantzig 1947
• Ellipsoid• Kachian 1979
• Interior Point• Affine Method 1967• Log Barrier Method 1977• Projective Method 1984
• Narendra Karmarkar form AT&T Bell Labs3
![Page 4: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/4.jpg)
Linear Programming
5
MSubject to
MSubject to
Karmarkar’s Canonical FormGeneral LP
• Minimum value of is 0
![Page 5: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/5.jpg)
Karmarkar’s Algorithm• Step 1: Take an initial point • Step 2: While • 2.1 Transformation: such that is center of
• This gives us the LP problem in transformed space
• 2.2 Orthogonal projection and movement: Project to feasible region and move in the steepest descent direction. • This gives us the point
• 2.3 Inverse transform: • This gives us
• 2.4 Use as the start point of the next iteration • Set
6
![Page 6: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/6.jpg)
Step 2.1• Transformation: such that is center of - Why?
7
𝑥 ′(𝑘)
𝑟
𝑥(𝑘)
![Page 7: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/7.jpg)
Step 2.2• Projection and movement: Project to feasible region and
move in the steepest descent direction - Why?
8
𝑐 ′
−𝑐 ′
Projection of to feasible region
![Page 8: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/8.jpg)
Big Picture
9
𝑥(𝑘)𝑥 ′(𝑘)
𝑥 ′(𝑘+1)
𝑥(𝑘+1 )
𝑥(𝑘+2 )
![Page 9: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/9.jpg)
Karmarkar’s Algorithm• Step 1: Take an initial point • Step 2: While • 2.1 Transformation: such that is center of
• This gives us the LP problem in transformed space
• 2.2 Orthogonal projection and movement: Project to feasible region and move in the steepest descent direction. • This gives us the point
• 2.3 Inverse transform: • This gives us
• 2.4 Use as the start point of the next iteration • Set
11
![Page 10: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/10.jpg)
Projective Transformation• Transformation: such that is center of
12
Transform
𝑥(𝑘) 𝑥 ′(𝑘)=𝑒𝑛
𝑥(𝑘)=[𝑥1(𝑘)
𝑥2(𝑘)
⋮𝑥𝑛
(𝑘) ] 𝑥 ′(𝑘)=[1𝑛1𝑛⋮1𝑛
]=𝑒𝑛
𝑒= [111…1 ]𝑇
![Page 11: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/11.jpg)
Projective transformation• We define the transform such that
• We transform every other point with respect to point
Mathematically:• Defining D = diag • projective transformation:•
• inverse transformation•
13
![Page 12: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/12.jpg)
Problem transformation:• projective transformation:•
• inverse transformation•
The new objective function is not a linear function.
Minimize
s.t.
MinimizeS.t.
Transform
14
![Page 13: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/13.jpg)
Problem transformation:
MinimizeS.t.
Convert
15
MinimizeS.t.
![Page 14: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/14.jpg)
Karmarkar’s Algorithm• Step 1: Take an initial point • Step 2: While • 2.1 Transformation: such that is center of
• This gives us the LP problem in transformed space
• 2.2 Orthogonal projection and movement: Project to feasible region and move in the steepest descent direction. • This gives us the point
• 2.3 Inverse transform: • This gives us
• 2.4 Use as the start point of the next iteration • Set
16
![Page 15: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/15.jpg)
Orthogonal Projection
• Projecting onto • projection on • normalization
17
MinimizeS.t.
![Page 16: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/16.jpg)
Orthogonal Projection• Given a plane and a point outside the plane, which direction
should we move from the point to guarantee the fastest descent towards the plane?• Answer: perpendicular direction
• Consider a plane and a vector
18
𝑐 ′ 𝑐𝑃
𝑠 𝐴’
ℵ (𝐴′)
![Page 17: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/17.jpg)
Orthogonal Projection• Let and be the basis of the plane ()
• The plane is spanned by the column space of ]
• , so
• is perpendicular to and
• 19
𝑐 ′ 𝑐𝑃
𝑠 𝐴’
ℵ (𝐴′)
![Page 18: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/18.jpg)
Orthogonal Projection
• = projection matrix with respect to ’
• We need to consider the vector • Remember
• is the projection matrix with respect to 20
𝑐 ′ 𝑐𝑃
𝑠 𝐴’
ℵ (𝐴′)
![Page 19: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/19.jpg)
Orthogonal Projection• What is the direction of?
• Steepest descent =
• We got the direction of the movement or projected ’ onto
• Projection on
21
𝑐 ′ 𝑐𝑃
𝑠 𝐴’
![Page 20: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/20.jpg)
Calculate step size
• r = radius of the incircle
22
MinimizeS.t.
![Page 21: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/21.jpg)
Movement and inverse transformation
23
Transform
𝑥(𝑘) 𝑥 ′(𝑘)=𝑒𝑛
𝑥 ′(𝑘+ 1)
𝑥(𝑘+1 ) Inverse Transform
![Page 22: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/22.jpg)
Big Picture
24
𝑥(𝑘)𝑥 ′(𝑘)
𝑥 ′(𝑘+1)
𝑥(𝑘+1 )
𝑥(𝑘+2 )
![Page 23: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/23.jpg)
Matlab Demo
25
![Page 24: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/24.jpg)
Contents• Overview
• Projective transformation
• Orthogonal projection
• Complexity analysis
• Transformation to Karmarkar’s canonical form
26
![Page 25: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/25.jpg)
Running Time• Total complexity of iterative algorithm =
(# of iterations) x (operations in each iteration)
• We will prove that the # of iterations = O(nL)
• Operations in each iteration = O(n2.5)
• Therefore running time of Karmarkar’s algorithm = O(n3.5L)
![Page 26: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/26.jpg)
# of iterations• Let us calculate the change in objective function value at the
end of each iteration
• Objective function changes upon transformation• Therefore use Karmarkar’s potential function
• The potential function is invariant on transformations (upto some additive constants)
![Page 27: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/27.jpg)
# of iterations• Improvement in potential function is
• Rearranging,
![Page 28: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/28.jpg)
# of iterations• Since potential function is invariant on transformations, we
can use
whereandare points in the transformed space.
• At each step in the transformed space,
![Page 29: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/29.jpg)
# of iterations• Simplifying, we get•
Where
• For small
• So suppose both and are small -
![Page 30: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/30.jpg)
# of iterations• It can be shown that • The worst case for
would be if
• In this case
![Page 31: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/31.jpg)
# of iterations• Thus we have proved that
• Thus
where k is the number of iterations to converge
• Plugging in the definition of potential function,
![Page 32: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/32.jpg)
# of iterations• Rearranging, we get
![Page 33: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/33.jpg)
# of iterations• Therefore there exists some constant such that after
iterations,
where , being the number of bits
is sufficiently small to cause the algorithm to converge.
• Thus the number of iterations is in
![Page 34: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/34.jpg)
Rank-one modification• The computational effort is dominated by:
• Substituting A’ in, we have
• The only quantity changes from step to step is D • Intuition:• Let and be diagonal matrices• If and are close the calculation given should be cheap!
![Page 35: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/35.jpg)
Method• Define a diagonal matrix , a “working approximation” to in
step k such that• for • We update D’ by the following strategy• (We will explain this scaling factor in later slides)• For to do
if Then let and update
![Page 36: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/36.jpg)
Rank-one modification (cont)• We have
• Given , computation of can be done in or steps.• If and differ only in entry then
• => rank one modification.• If and differ in entries then complexity is .
![Page 37: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/37.jpg)
Performance Analysis• In each step: • Substituting , and , • Let call • Then Let • Then
![Page 38: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/38.jpg)
Performance analysis - 2• First we scale by a factor
• Then for any entry I such that
• We reset • Define discrepancy between and as• Then just after an updating operation for index
![Page 39: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/39.jpg)
Performance Analysis - 3• Just before an updating operation for index i
Between two successive updates, say at steps k1 and k2
Let call Then
Assume that no updating operation was performed for index I-
![Page 40: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/40.jpg)
Performance Analysis - 4
• Let be the number of updating operations corresponding to index in steps and be the total number of updating operations in steps.
• We have
Because of then
![Page 41: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/41.jpg)
Performance Analysis - 5• Hence
• So
• Finally
![Page 42: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/42.jpg)
Contents• Overview
• Transformation to Karmarkar’s canonical form
• Projective transformation
• Orthogonal projection
• Complexity analysis
44
![Page 43: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/43.jpg)
Transform to canonical form
45
MSubject to
• x
MSubject to
• x
Karmarkar’s Canonical FormGeneral LP
![Page 44: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/44.jpg)
Step 1:Convert LP to a feasibility problem• Combine primal and dual problems
• LP becomes a feasibility problem46
MSubject to
• x
MSubject to
•
Primal Dual
Combined
![Page 45: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/45.jpg)
Step 2: Convert inequality to equality• Introduce slack and surplus variable
47
![Page 46: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/46.jpg)
Step 3: Convert feasibility problem to LP• Introduce an artificial variable to create an interior starting
point.• Let be strictly interior points in the positive orthant.
48
• Minimize • Subject to• • + •
![Page 47: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/47.jpg)
Step 3: Convert feasibility problem to LP
• Change of notation
49
MSubject to
• x
• Minimize • Subject to• • + •
![Page 48: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/48.jpg)
Step4: Introduce constraint • Let ,…, ] be a feasible point in the original LP
• Define a transformation• for
• Define the inverse transformation• for
50
![Page 49: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/49.jpg)
Step4: Introduce constraint • Let denote the column of • If • Then
• We define the columns of a matrix ’ by these equations• for • =
• Then
51
![Page 50: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/50.jpg)
Step 5: Get the minimum value of Canonical form • Substituting
• We get
• Define by
• Then implies
52
![Page 51: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/51.jpg)
Step 5: Get the minimum value of Canonical form • The transformed problem is
53
M
Subject to
![Page 52: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/52.jpg)
References• Narendra Karmarkar (1984). "
A New Polynomial Time Algorithm for Linear Programming”• Strang, Gilbert (1 June 1987). "Karmarkar’s algorithm and its
place in applied mathematics". The Mathematical Intelligencer (New York: Springer) 9 (2): 4–10. doi:10.1007/BF03025891. ISSN 0343-6993. MR '''883185'‘
• Robert J. Vanderbei; Meketon, Marc, Freedman, Barry (1986). "A Modification of Karmarkar's Linear Programming Algorithm". Algorithmica 1: 395–407. doi:10.1007/BF01840454. ^ Kolata, Gina (1989-03-12). "IDEAS & TRENDS; Mathematicians Are Troubled by Claims on Their Recipes". The New York Times. 54
![Page 53: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/53.jpg)
References• Gill, Philip E.; Murray, Walter, Saunders, Michael A., Tomlin, J.
A. and Wright, Margaret H. (1986). "On projected Newton barrier methods for linear programming and an equivalence to Karmarkar’s projective method". Mathematical Programming 36 (2): 183–209. doi:10.1007/BF02592025.
• oi:10.1007/BF02592025. ^ Anthony V. Fiacco; Garth P. McCormick (1968). Nonlinear Programming: Sequential Unconstrained Minimization Techniques. New York: Wiley. ISBN 978-0-471-25810-0. Reprinted by SIAM in 1990 as ISBN 978-0-89871-254-4.
• Andrew Chin (2009). "On Abstraction and Equivalence in Software Patent Doctrine: A Response to Bessen, Meurer and Klemens". Journal Of Intellectual Property Law 16: 214–223.
55
![Page 54: Karmarkar Algorithm](https://reader036.vdocuments.us/reader036/viewer/2022062305/5681692c550346895de06f49/html5/thumbnails/54.jpg)
Q&A
56