learning to solve combinatorial optimization problems on
TRANSCRIPT
![Page 1: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/1.jpg)
Learning to Solve Combinatorial Optimization Problems on
Real-World Graphs in Linear Time
IddoDrori
BrandonKates
WilliamSickinger
AnantKharkar
BrendaDietrich
DavidWilliamson
MadeleineUdell
1 2 3
1,2,3 233 2 22QiangMa
3 SuwenGe
3 EdenDolev
3
![Page 2: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/2.jpg)
Example Problems Over Graphs
• Polynomial– Minimum Spanning Tree (MST)– Single-Source Shortest Paths (SSP)
• NP-hard– Traveling Salesman Problem (TSP)– Vehicle Routing Problem (VRP)
![Page 3: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/3.jpg)
• NP-hard problem• Approximation algorithms and optimality gaps• Linear time approximation with optimality gap close to 1
TSP Time Complexity
![Page 4: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/4.jpg)
• Graph attention is quadratic• Attention approximation is linear and independent of data• Practical GPU memory bottleneck
Running Time Complexity
![Page 5: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/5.jpg)
Running Time Complexity
• Verification problems for NP-hard problems have polynomial time complexity.
• Polynomial vs. NP-hard problems:– Fast type-1 process: polynomial problems on graphs can be
solved using GNN’s without reinforcement learning or search– Slow type-2 process: NP-hard problems require RL or search
• GNN’s can be used directly for verification
![Page 6: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/6.jpg)
Minimum Spanning Tree (MST)
• Given connected and undirected graph 𝐺 = (V, E, W)
• Find tree T = (VT, ET) with VT = V, ET ⊂ E minimizing sum of edge weights WT ⊂ W.
• Greedy algorithms with time complexity O(|E|log|V|): Boruvka, Prim, Kruskal
![Page 7: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/7.jpg)
Single-Source Shortest Paths (SSP)
• Given connected and directed graph G = (V, E, W) and source vertex.
• Find shortest paths from source to all other vertices.• For SSP with nonnegative weights: Dijkstra’s algorithm
complexity O(|V|log|V| + |E|) using a heap. • For general SSP: Bellman-Ford runs in O(|V||E|). • Floyd–Warshall algorithm solves SSP between all pairs of
nodes with cubic time complexity O(|V|^3)
![Page 8: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/8.jpg)
Traveling Salesman Problem (TSP)
• Graph G = (V, E, W)• V represents list of cities• W represents distances between each pair of cities.• Find shortest tour visiting each city once and returns to start.• NP-hard problem.
![Page 9: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/9.jpg)
Examples
TSP250 TSP500 TSP750 TSP1000
![Page 10: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/10.jpg)
Vehicle Routing Problem (VRP)
• Given M vehicles and graph G = (V, E) with |V| cities• Find optimal routes for vehicles.• Each vehicle m ∈ {1, .., M} starts from same depot node, visits
subset V(m) of cities, and returns to depot node. • Routes of different vehicles do not intersect except at depot;
together, the vehicles visit all cities. • Optimal routes minimize longest tour length of any single route.• TSP is special case of VRP for one vehicle.
![Page 11: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/11.jpg)
Learning Graph Algorithms as Single Player Games
• Represent problem space as search tree.• Leaves of search tree represent all (possibly exponentially
many) possible solutions to problem.• Search traverses tree, choosing path (guided by MCTS+NN)
![Page 12: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/12.jpg)
Action
State
Reward
Learning Graph Algorithms as Single Player Games
![Page 13: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/13.jpg)
Reinforcement Learning
![Page 14: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/14.jpg)
Learning Graph Algorithms as Single Player Games
• Initial state: represented by root node, may be empty set, a random state, or other initial state.
• Each path from root to a leaf consists of moving between nodes (states) along edges (taking actions) reaching a leaf node (reward).
• Actions: adding or removing a node or edge. • Reward (or cost): value of solution, for example sum of
weights or length of tour.
![Page 15: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/15.jpg)
Action
State
Reward
State
![Page 16: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/16.jpg)
Actions
• Add or remove node or edge
![Page 17: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/17.jpg)
Line Graph
• Problem: problems involve both actions on nodes and edges.
• Solution: use edge-to vertex dual (line graph)Perform actions on nodes.
![Page 18: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/18.jpg)
Graph
• G = (V, E)
![Page 19: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/19.jpg)
• Each edge in primal graph corresponds to node in line graph
Line Graph: Edge-to-Vertex Dual
![Page 20: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/20.jpg)
• Edges in primal graph
Graph
![Page 21: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/21.jpg)
• Correspond to nodes in line graph
Line Graph: Edge-to-Vertex Dual
![Page 22: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/22.jpg)
• Two nodes in line graph are connected if corresponding edges in primal graph share a node.
• Edge weights in primal graph become node weights in line graph.
Line Graph: Edge-to-Vertex Dual
![Page 23: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/23.jpg)
Tree
![Page 24: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/24.jpg)
Line Graph
![Page 25: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/25.jpg)
Line Graph: Edge-to-Vertex Dual
• Two nodes in line graph are connected if corresponding edges in primal graph share a node.
• Edge weights in primal graph become node weights in line graph.
![Page 26: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/26.jpg)
Learning Graph Algorithms as Single Player Games
Action
State
Reward
![Page 27: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/27.jpg)
Action
State
Reward
State
![Page 28: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/28.jpg)
Action
State
Reward
our state now is learning an algorithm,that has a state, which includes a graph, etc.
Food for Thought: Learning to Learn to Learn..
![Page 29: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/29.jpg)
Actions
• Add/remove nodes/edges.
![Page 30: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/30.jpg)
Reward
• Objective function
![Page 31: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/31.jpg)
Action
State
Reward
A
B
C
Unified Framework
![Page 32: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/32.jpg)
Machine Learning for Combinatorial Optimization
• Rapidly growing field• Leading architecture
– Outer loop: RL / search– Inner loop: GNN’s
![Page 33: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/33.jpg)
RL
Model-basedModel-free
Supervised
PolicyValue Actor-Critic Given Model Learn Model
ML for CO
ML Approaches for NP-Hard Combinatorial Optimization Problems
![Page 34: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/34.jpg)
Generalization on Graphs
1. From small to large graphs2. Between different types of random graphs3. From random to real-world graphs
![Page 35: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/35.jpg)
• From small to large random regular graphs• Training on 100 node graphs• Testing on 100/250/500/750/1000 node graphs
Generalization on Graphs
![Page 36: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/36.jpg)
• From small to large random regular graphs• Trained on graphs with 100 nodes, tested on 250 nodes
Generalization on Graphs
![Page 37: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/37.jpg)
Generalization on Graphs• From random graphs to real world graphs• Trained on random Euclidean graphs with 100 nodes
![Page 38: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/38.jpg)
• From small random graphs to large real world graphs• Trained on random Euclidean graphs with 100 nodes
Generalization on Graphs
![Page 39: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/39.jpg)
Conclusions
• Approximation with linear running time complexity and optimality gaps close to 1
• Generalizations on graphs
• Unified framework for approximating combinatorial optimization problems over graphs
![Page 40: Learning to Solve Combinatorial Optimization Problems on](https://reader030.vdocuments.us/reader030/viewer/2022012506/6182552cda63f97a244bb039/html5/thumbnails/40.jpg)
Learning to Solve Combinatorial Optimization Problems on
Real-World Graphs in Linear Time
IddoDrori
BrandonKates
WilliamSickinger
AnantKharkar
BrendaDietrich
DavidWilliamson
MadeleineUdell
1 2 3
1,2,3 233 2 22QiangMa
3 SuwenGe
3 EdenDolev
3