ms&e 211 lecture 5: basic feasible solutions; shortest paths ashish goel
TRANSCRIPT
The Knapsack problem
• A thief wants to rob a warehouse, but only has a small knapsack to carry her loot back. What should she carry?– The warehouse has many goods, all divisible, so she can take
50% of a good (for example) and sell it for 50% of the total value. Examples: gold dust, diamonds, sugar, oil, cash
– The knapsack has a limited capacity for the amount of weight, but not for the amount of volume
– The thief wants to maximize her profit
• The thief asks you, the optimization expert, to help her figure out what goods to carry
Modeling: Step 1a
Name the quantities that are given to you as part of the problem. i.e. the parameters
• W: The capacity of the knapsack
• N: The number of goods
• vi: The value of the i-th good
• wi: The weight of the i-th good
Modeling: Step 1b
Name the quantities that you are free to choose, i.e., the decision variables
• xi: The fraction of the i-th good carried away by the thief
Define your goal; i.e. what is your objective in solving the problem
• First in English: Maximize the total value of the goods the thief can carry away
• Then in Math, to obtain your objective Function
Maximize Σ vi xi
Modeling: Step 2a
Ni=1
Then, define the restrictions placed on you by the problem, i.e. the constraints
• You can not carry more of a good than you have available
• You can not carry more total weight than the knapsack capacity
• You can not carry negative amounts of any good
Modeling: Step 2b
For all i, 1 ≤ i ≤ N: xi ≥ 0
For all i, 1 ≤ I ≤ N: xi ≤ 1
Σ wi xi ≤ WNi=1
Then, define the restrictions placed on you by the problem, i.e. the constraints
• You can not carry more of a good than you have available
• You can not carry more total weight than the knapsack capacity
• You can not carry negative amounts of any good
Modeling: Step 2b
For all i, 1 ≤ i ≤ N: xi ≥ 0
For all i, 1 ≤ I ≤ N: xi ≤ 1
Σ wi xi ≤ WNi=1
Then, define the restrictions placed on you by the problem, i.e. the constraints
• You can not carry more of a good than you have available
• You can not carry more total weight than the knapsack capacity
• You can not carry negative amounts of any good
Modeling: Step 2b
For all i, 1 ≤ i ≤ N: xi ≥ 0
For all i, 1 ≤ i ≤ N: xi ≤ 1
Σ wi xi ≤ WNi=1
Then, define the restrictions placed on you by the problem, i.e. the constraints
• You can not carry more of a good than you have available
• You can not carry more total weight than the knapsack capacity
• You can not carry negative amounts of any good
Modeling: Step 2b
For all i, 1 ≤ i ≤ N: xi ≥ 0
For all i, 1 ≤ i ≤ N: xi ≤ 1
Σ wi xi ≤ WNi=1
Then, define the restrictions placed on you by the problem, i.e. the constraints
• You can not carry more of a good than you have available
• You can not carry more total weight than the knapsack capacity
• You can not carry negative amounts of any good
Modeling: Step 2b
For all i, 1 ≤ i ≤ N: xi ≥ 0
For all i, 1 ≤ i ≤ N: xi ≤ 1
Σ wi xi ≤ WNi=1
Then, define the restrictions placed on you by the problem, i.e. the constraints
• You can not carry more of a good than you have available
• You can not carry more total weight than the knapsack capacity
• You can not carry negative amounts of any good
Modeling: Step 2b
For all i, 1 ≤ i ≤ N: xi ≥ 0
For all i, 1 ≤ i ≤ N: xi ≤ 1
Σ wi xi ≤ WNi=1
Putting it all together
The end result of modeling: A mathematical statement of the knapsack problem.
subject to:Maximize Σ vi xi
Ni=1
For all i, 1 ≤ i ≤ N: xi ≤ 1
Σ wi xi ≤ WNi=1
For all i, 1 ≤ i ≤ N: xi ≥ 0
Putting it all together
The end result of modeling: A mathematical statement of the knapsack problem.
subject to:Maximize Σ vi xi
Ni=1
Σ wi xi ≤ WNi=1
( i): xi ≤ 1
( i): xi ≥ 0
An Instance
• An instance of a problem is where we actually give values to all the parameters. Example:– Three goods: gold, diamond dust, and silver.– Weights: (2, 3, 4) and Values: (5, 20, 3), respectively.– Knapsack capacity: 4
Maximize 5x1 + 20x2 + 3x3
Subject to:2x1 + 3x2 + 4x3 ≤ 4
x1 ≤ 1
x2 ≤ 1
x3 ≤ 1x1, x2,
x3 ≥ 0
The Knapsack Instance
– Optimal solution: x1 = 0.5, x2 = 1.0, x3 = 0
• Only one fractional value. Will this always be true?• Three decision variables, hence need three linearly independent
“binding” or “tight” constraints.• N decision variables: need N linearly independent binding
constrains => At most one fractional variable in a bfs
Maximize 5x1 + 20x2 + 3x3
Subject to:2x1 + 3x2 + 4x3 ≤ 4x1 ≥ 0
x1 ≤ 1x2 ≥ 0
x2 ≤ 1x3 ≥ 0
x3 ≤ 1
The Knapsack Instance
– Optimal solution: x1 = 0.5, x2 = 1.0, x3 = 0
• Only one fractional value. Will this always be true?• Three decision variables, hence need three linearly independent
“binding” or “tight” constraints.• N decision variables: need N linearly independent binding
constrains => At most one fractional variable in a bfs
Maximize 5x1 + 20x2 + 3x3
Subject to:2x1 + 3x2 + 4x3 ≤ 4x1 ≥ 0
x1 ≤ 1x2 ≥ 0
x2 ≤ 1x3 ≥ 0
x3 ≤ 1
BFS and LP Solvers
• The Simplex method returns a BFS when possible• Most (all?) modern LP solvers return a BFS when
possible by default• Hence, if you solve the knapsack problem using
Excel, or pretty much any modern LP solver, you will get at most one fractional decision variable in the optimum solution
Example: Matching
• Marry every man to exactly one woman, and vice-versa, while maximizing compatibility
Ci,j : Compatibility of man i with woman j
xi,j: Decision variable indicating whether man i and woman j are married (perhaps fractionally)
COMPATIBILTY Helen Gloria Iris
Dave 1 0 0.5
Eddy 0.75 2 1
Frank 0.5 2.5 1.5
Example: Matching
• One optimum solution: x11 = x22 = x33 = 1, everything else 0
• Another: x11 = x23 = x32 = 1, everything else 0
• A third: x11 = 1, = x22 = x33 = x23 = x32 = 0.5, everything else 0
COMPATIBILTY Helen Gloria Iris
Dave 1 0 0.5
Eddy 0.75 2 1
Frank 0.5 2.5 1.5
Example: Matching
• One optimum solution: x11 = x22 = x33 = 1, everything else 0
• Another: x11 = x23 = x32 = 1, everything else 0
• A third: x11 = 1, = x22 = x33 = x23 = x32 = 0.5, everything else 0
COMPATIBILTY Helen Gloria Iris
Dave 1 0 0.5
Eddy 0.75 2 1
Frank 0.5 2.5 1.5
Example: Matching
• One optimum solution: x11 = x22 = x33 = 1, everything else 0
• Another: x11 = x23 = x32 = 1, everything else 0
• A third: x11 = 1, = x22 = x33 = x23 = x32 = 0.5, everything else 0
COMPATIBILTY Helen Gloria Iris
Dave 1 0 0.5
Eddy 0.75 2 1
Frank 0.5 2.5 1.5
Example: Matching
• One optimum solution: x11 = x22 = x33 = 1, everything else 0
• Another: x11 = x23 = x32 = 1, everything else 0
• A third: x11 = 1, = x22 = x33 = x23 = x32 = 0.5, everything else 0
COMPATIBILTY Helen Gloria Iris
Dave 1 0 0.5
Eddy 0.75 2 1
Frank 0.5 2.5 1.5
Exercise
Which of the three optimum solutions
x11 = x22 = x33 = 1, everything else 0
x11 = x23 = x32 = 1, everything else 0
x11 = 1, x22 = x33 = x23 = x32 = 0.5, everything else 0
can not be a basic feasible solution?
Example: Matchings
• In this example, every basic feasible solution is integral
• In general, for the problem of finding a perfect matching, every basic feasible solution is integral, and the polytope is bounded
=> Excel will always return an integer solution as long as you solve the problem as an LP
Shortest Path Problem
• Find the shortest path from s to t– Using linear programs
• Many direct applications– Web applications to compute routes
• Indirect applications– Currency arbitrage– Most reliable path
• Beautiful structure: BFSs, duality• Generalizes to Min-cost Flows
Notation
• Given a directed graph G– V = Set of nodes in the graph, eg. {s, p, q, r, t}– N = Number of nodes, eg. N = 5
6
2
4
st
p
qr
1
2
3
4
Notation
• Given a directed graph G– E = Set of edges, eg. {(s,p), (p,s), (s,q), (q,s),(p,t), (q,r), (r,t)}– M = Number of edges, eg. M = 7
6
2
4
st
p
qr
1
2
3
4
Notation
• Given a directed graph G– c(u,v) = Cost of edge from u to v, eg. c(s,p) = 4, whereas
c(s,t) is undefined
6
2
4
st
p
qr
1
2
3
4
Notation
• Given a directed graph G– Assume that s is the “start” and t the “terminating point”
of the path we want to find
6
2
4
st
p
qr
1
2
3
4
6
2
4
st
p
qr
1
2
3
4
Notation
• Given a directed graph G– Assume that “s” is the start and t the “terminating point”
of the path we want to find
G, V, E, N, M, c, s, tare all just a convention; we will use other symbolswhere necessary
Linear Program?
• Decision Variables?– A variable for every path?
s
tNumber of s-t paths in this graph = 70
Number of s-t paths in 10 X 10 grid = 184,756
In 20 X 20 grid = 137,846,528,820
Linear Program?
• Decision Variables?– A variable for every edge
x(u,v): The amount of edge (u,v) used in the path, perhaps fractionallyx(u,v) = 1 : The edge is used on the pathx(u,v) = 0 : The edge is not usedx(u,v) = ½ : The edge is used half the time
x(u,v): Leave u, enter v
Constraints
x(u,v) ≥ 0
x(u,v) ≤ 1
If you enter a vertex other than s or t, you must leave it
You must leave s once
You must enter t once
Constraints
x(u,v) ≥ 0
x(u,v) ≤ 1
If you enter a vertex other than s or t, you must leave it exactly as much as you enter it
You must leave s once more than you enter it
You must enter t once more than you leave it
A Useful Mnemonic
• We will employ the following useful shorthand notation for the number of times we leave a node u in solution x:
OUTx(u) = Sum of x(u,v) for all edges (u,v)
• We will use the following useful shorthand notation for the number of times we enter a node u in solution x:
INx(u) = Sum of x(v,u) for all edges (v,u)
Properties of the Solution
Which of the following is true?
1. There can not be a feasible solution which has a positive value for any variable x(v,s).
2. There can not be a feasible solution which has a value of zero for all variables x(v,s).
3. There can not be a feasible solution which has a value of zero for all variables x(v,t).
4. There can not be a feasible solution which has a positive value for any x(t,v).
Another Useful Mnemonic
d(u) = “Demand” of node u, i.e., how many more times must we enter node u compared to how many times we leave it
d(t) = +1
d(s) = -1
d(u) = 0 for all other nodes u
Even Simpler
• We don’t need the x ≤ 1 constraint
• M decision variables, one for each edge• N constraints (other than x ≥ 0), one for each node
– Called “conservation constraints” since they conserve movement into and out of a node, except for the source and the terminus
Negative Cycles
• We saw that the shortest path problem can be infeasible.
• But can it have an unbounded optimum solution?• Sadly, yes, when there is a negative cost cycle.
Negative Cost Edges?
• Negative cost edges are ok, as long as there is no negative cost cycle
-6
2
4
st
p
qr
1
2
3
4