csc 282: design & analysis of efficient algorithms graph algorithms shortest path algorithms...
TRANSCRIPT
![Page 1: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/1.jpg)
CSC 282: Design & Analysis of Efficient Algorithms
Graph AlgorithmsShortest Path Algorithms
Fall 2013
![Page 2: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/2.jpg)
Path Finding Problems
Many problems in computer science correspond to searching for paths in a graph from a given start node
• Route planning• Packet-switching• VLSI layout• 6-degrees of Kevin Bacon• Program synthesis • Speech recognition
![Page 3: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/3.jpg)
Application: Robot Planning
A huge graph may be implicitly specified by rules for generating it on-the-fly
Blocks world: • vertex = relative positions of all blocks• edge = robot arm stacks one block
stack(blue,red)
stack(green,red)
stack(green,blue)
stack(blue,table)
stack(green,blue)
![Page 4: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/4.jpg)
Iterative Depth-First Search Path Finding
procedure DFS_Path(G, s)stack: open
for each u in V: u.visited = falseopen.push(s)while not open.empty():
u = open.pop()for each (u,v) in E:
if not v.visited thenv.prev = uopen.push(v)
end
![Page 5: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/5.jpg)
Iterative Breadth-First Search Path Finding
procedure BFS_Path(G, s)queue: open
for each u in V: u.visited = falseopen.enqueue(s)while not open.empty():
u = open.dequeue()for each (u,v) in E:
if not v.visited thenv.prev = uopen.enqueue(v)
end
![Page 6: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/6.jpg)
Example DFS Path Finding
Seattle
San FranciscoDallas
Salt Lake City
Rochester
![Page 7: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/7.jpg)
Example DFS Path Finding
Seattle
San FranciscoDallas
Salt Lake City
Rochester
![Page 8: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/8.jpg)
Example DFS Path Finding
Seattle
San FranciscoDallas
Salt Lake City
Rochester
![Page 9: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/9.jpg)
DFS BFS
O(|G|)Shortest paths?
O(|G|)Shortest paths?
![Page 10: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/10.jpg)
DFS BFS
O(|G|)Shortest paths? NO
O(|G|)Shortest paths? YES
![Page 11: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/11.jpg)
Lengths on Edges
Path length: the number of edges in the pathPath cost: the sum of the costs of each edge
Seattle
San FranciscoDallas
Chicago
Salt Lake City
3.5
2 2
2.5
3
22.5
2.5
![Page 12: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/12.jpg)
Edsger Wybe Dijkstra (1930-2002)
Invented concepts of structured programming, synchronization, weakest precondition, and "semaphores" for controlling computer processes. The Oxford English Dictionary cites his use of the words "vector" and "stack" in a computing context.
Believed programming should be taught without computers1972 Turing Award“In their capacity as a tool, computers will be but a ripple on the
surface of our culture. In their capacity as intellectual challenge, they are without precedent in the cultural history of mankind.”
![Page 13: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/13.jpg)
Dijkstras Algorithm for Single Source Shortest Path
Similar to breadth-first search, but uses a heap instead of a queue:
• Always select (expand) the vertex that has a lowest-cost path to the start vertex
Correctly handles the case where the shortest path to a vertex is not the one with fewest edges
![Page 14: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/14.jpg)
Dijkstra’s Algorithm
procedure Dijkstra(G, s)heap: open
for each u in V: u.dist = infinitys.dist = 0for each u in V: open.insert(u, u.dist)while not open.empty():
u = open.deleteMin()for each (u,v) in E:
if v.dist > u.dist + length(u,v) then
v.dist = u.dist + length(u,v) v.prev = uopen.decreaseKey(v)
end
![Page 15: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/15.jpg)
Demo
http://www.unf.edu/~wkloster/foundations/DijkstraApplet/DijkstraApplet.htm
![Page 16: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/16.jpg)
Analyzing Dijkstra’s Algorithm
To create the heap, insert each nodeThis happens |V| times
Once a vertex is removed from the head, the cost of the shortest path to that node is known
This happens |V| timesWhile a vertex is still in the heap, another
shorter path to it might still be foundThis happens at most |E| times
![Page 17: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/17.jpg)
Run Time Analysis
|V|O(insert) + |V|O(delete) + |E|O(decreaseKey)
Binary heap:|V|O(log|V|) + |V|O(log|V|) + |E|O(log|V|)= O((|V|+|E|) log|V|)
Array:|V|O(1) + |V|O(|V|) + |E|O(1)= O(|V|2)
![Page 18: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/18.jpg)
Finding Paths in Very Large Graphs
• It is expensive to find optimal paths in large graphs using BFS or Dijkstra’s algorithm
• If we only care about paths to a goal node, when should we stop?
• If the graph is generated “on the fly” as it is searched, how do we avoid needing to generate all nodes when initializing the heap?
![Page 19: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/19.jpg)
Dijkstra’s Algorithm with Goal
procedure Dijkstra(G, s, g)heap: open
s.dist = 0open.insert(s, s.dist)while not open.empty():
u = open.deleteMin()if u = g then returnfor each (u,v) in E: // might create v
if v is newly created then v.dist = infinity
if v.dist > u.dist + length(u,v) thenv.dist = u.dist + length(u,v) v.prev = uif open.contains(v) then
open.decreaseKey(v, v.dist)else open.insert(v, v.dist)
end
![Page 20: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/20.jpg)
Finding Paths in Very Large Graphs
• It is expensive to find optimal paths in large graphs using BFS or Dijkstra’s algorithm
• If we only care about paths to a goal node, when should we stop?
• If the graph is generated “on the fly” as it is searched, how do we avoid needing to generate all nodes when initializing the heap?
• How can we use search heuristics?
![Page 21: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/21.jpg)
Best-First Search
The Manhattan distance is an estimate of the distance to the goal
• It is a search heuristic• It is optimistic (never overestimates)
Best-First Search• Order nodes in priority to minimize
estimated distance to the goalCompare: BFS / Dijkstra
• Order nodes in priority to minimize distance from the start
![Page 22: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/22.jpg)
Example
52nd St
51st St
50th St
10th A
ve
9th A
ve
8th A
ve
7th A
ve
6th A
ve
5th A
ve
4th A
ve
3rd A
ve
2nd A
ve
S
G
53nd St
Plan a route from 9th & 50th to 3rd & 51st
![Page 23: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/23.jpg)
Example
52nd St
51st St
50th St
10th A
ve
9th A
ve
8th A
ve
7th A
ve
6th A
ve
5th A
ve
4th A
ve
3rd A
ve
2nd A
ve
S
G
53nd St
Plan a route from 9th & 50th to 3rd & 51st
![Page 24: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/24.jpg)
Best First Search
procedure best_first(G, s, g)heap: open
open.insert(s, heuristic(s,g))while not open.empty():
u = open.deleteMin()u.visited = trueif u = g then returnfor each (u,v) in E:
if v is newly created then v.visited = false
if not v.visited thenv.prev = uopen.insert(v, heuristic(v,g))
end
![Page 25: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/25.jpg)
Non-Optimality of Best-First
52nd St
51st St
50th St
10th A
ve
9th A
ve
8th A
ve
7th A
ve
6th A
ve
5th A
ve
4th A
ve
3rd A
ve
2nd A
ve
S G
53nd St
Path found by Best-first
Shortest Path
![Page 26: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/26.jpg)
Improving Best-First
Best-first is often tremendously faster than BFS/Dijkstra, but might stop with a non-optimal solution
How can it be modified to be (almost) as fast, but guaranteed to find optimal solutions?
A* - Hart, Nilsson, Raphael 1968• One of the first significant algorithms
developed in AI• Widely used in many applications
![Page 27: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/27.jpg)
A*
Exactly like Best-first search, but using a different criteria for the priority queue:
minimize (distance from start) + (estimated distance to goal)
priority f(n) = g(n) + h(n)f(n) = priority of a nodeg(n) = true distance from starth(n) = heuristic distance to goal
![Page 28: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/28.jpg)
A*
procedure aStar(G, s, g)heap: open
s.dist = 0open.insert(s, s.dist+heuristic(s,g))while not open.empty():
u = open.deleteMin()if u = g then returnfor each (u,v) in E:
if v is newly created then v.dist = infinityif v.dist > u.dist + length(u,v) then
v.dist = u.dist + length(u,v) v.prev = uif open.contains(v) then
open.decreaseKey(v, v.dist+heuristic(v,g))
else open.insert(v, v.dist+heuristic(v,g))end
![Page 29: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/29.jpg)
A* in Action
52nd St
51st St
50th St
10th A
ve
9th A
ve
8th A
ve
7th A
ve
6th A
ve
5th A
ve
4th A
ve
3rd A
ve
2nd A
ve
S G
53nd St
h=6+2
H=1+7
h=7+3
![Page 30: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/30.jpg)
Applications of A*: Planning
A huge graph may be implicitly specified by rules for generating it on-the-fly
Blocks world: • vertex = relative positions of all blocks• edge = robot arm stacks one block
stack(blue,red)
stack(green,red)
stack(green,blue)
stack(blue,table)
stack(green,blue)
![Page 31: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/31.jpg)
Blocks World
Blocks world: • distance = number of stacks to perform• heuristic lower bound = number of blocks
out of place (on wrong thing)
# out of place = 1, true distance to goal = 3
![Page 32: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/32.jpg)
Demo
http://www.cs.rochester.edu/u/kautz/MazesOriginal/search_algorithm_demo.htm
![Page 33: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/33.jpg)
Negative Lengths on Edges
Suppose we allow negative lengths on edges?
Dijkstra: Can fail to find shortest path
A*: Can fail to terminate if there is a negative cycle
![Page 34: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/34.jpg)
Bellman-Ford
procedure BellmanFord(G, s)
for each u in V: u.dist = infinitys.dist = 0repeat |V|-1 times:
for each (u,v) in E:if v.dist > u.dist + length(u,v) then
v.dist = u.dist + length(u,v) v.prev = u
end
Run time:
![Page 35: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/35.jpg)
Bellman-Ford
procedure aStar(G, s)
for each u in V: u.dist = infinitys.dist = 0repeat |V|-1 times:
for each (u,v) in E:if v.dist > u.dist + length(u,v) then
v.dist = u.dist + length(u,v) v.prev = u
end
Run time: O(|V||E|)
![Page 36: CSC 282: Design & Analysis of Efficient Algorithms Graph Algorithms Shortest Path Algorithms Fall 2013](https://reader030.vdocuments.us/reader030/viewer/2022012321/56649ccf5503460f9499b668/html5/thumbnails/36.jpg)
Summary: Graph Search
Depth First• Little memory required• Might find non-optimal path
Breadth First • Much memory required• Always finds optimal path
Dijskstra’s Short Path Algorithm• Like BFS for weighted graphs
Best First• Can visit fewer nodes• Might find non-optimal path
A*• Can visit fewer nodes than BFS or Dijkstra• Optimal if heuristic has certain common properties
Bellman-Ford• Handles negative edges• Not as efficient as Dijsktra for positive only graphs