part8 graphs
TRANSCRIPT
-
7/29/2019 Part8 Graphs
1/123
CSE 326: Data Structures
Part 8
Graphs
Henry KautzAutumn Quarter 2002
-
7/29/2019 Part8 Graphs
2/123
Outline
Graphs (TO DO: READ WEISS CH 9)
Graph Data Structures
Graph Properties
Topological Sort
Graph Traversals
Depth First Search
Breadth First Search Iterative Deepening Depth First
Shortest Path Problem
Dijkstras Algorithm
-
7/29/2019 Part8 Graphs
3/123
Graph ADT
Graphs are a formalism for representingrelationships between objects
a graph G is represented asG = (V, E)
Vis a set of vertices E is a set of edges
operations include:
iterating over vertices iterating over edges
iterating over vertices adjacent to a specific vertex
asking whether an edge exists connected two vertices
Han
Leia
Luke
V = {Han, Leia, Luke}
E = {(Luke, Leia),
(Han, Leia),
(Leia, Han)}
-
7/29/2019 Part8 Graphs
4/123
What Graph is THIS?
-
7/29/2019 Part8 Graphs
5/123
ReferralWeb
(co-authorship in scientific papers)
-
7/29/2019 Part8 Graphs
6/123
Biological Function Semantic Network
-
7/29/2019 Part8 Graphs
7/123
Graph Representation 1:
Adjacency MatrixA |V| x |V| array in which an element (u, v)
is true if and only if there is an edge from uto v
Han
Leia
Luke
Han Luke Leia
Han
Luke
LeiaRuntime:
iterate over vertices
iterate ever edges
iterate edges adj. to vertex
edge exists?Space requirements:
-
7/29/2019 Part8 Graphs
8/123
Graph Representation 2:
Adjacency ListA |V|-ary list (array) in which each entry stores a
list (linked list) of all adjacent vertices
Han
Leia
LukeHan
Luke
Leia
space requirements:
Runtime:
iterate over vertices
iterate ever edges
iterate edges adj. to vertex
edge exists?
-
7/29/2019 Part8 Graphs
9/123
Directed vs. Undirected Graphs
Han
Leia
Luke
Han
Leia
Luke
In directedgraphs, edges have a specific direction:
In undirectedgraphs, they dont (edges are two-way):
Vertices u and v are adjacentif(u, v) E
-
7/29/2019 Part8 Graphs
10/123
Graph Density
A sparse graph has O(|V|) edges
A dense graph has (|V|2) edges
Anything in between is either sparsish or densy depending on the context.
-
7/29/2019 Part8 Graphs
11/123
Weighted Graphs
20
30
35
60
Mukilteo
Edmonds
Seattle
Bremerton
Bainbridge
Kingston
Clinton
There may be more
information in the graph as well.
Each edge has an associated weight or cost.
-
7/29/2019 Part8 Graphs
12/123
Paths and CyclesApath is a list of vertices {v1, v2, , vn} such
that (vi, vi+1) E for all 0 i < n.
A cycle is a path that begins and ends at the samenode.
Seattle
San Francisco
Dallas
Chicago
Salt Lake City
p = {Seattle, Salt Lake City, Chicago, Dallas, San Francisco, Seattle}
-
7/29/2019 Part8 Graphs
13/123
Path Length and Cost
Path length: the number of edges in the pathPath cost: the sum of the costs of each edge
Seattle
San Francisco
Dallas
Chicago
Salt Lake City
3.5
2 2
2.5
3
22.5
2.5
length(p) = 5 cost(p) = 11.5
-
7/29/2019 Part8 Graphs
14/123
ConnectivityUndirected graphs are connectedif there is a path between
any two vertices
Directed graphs are strongly connectedif there is a path from
any one vertex to any other
Directed graphs are weakly connectedif there is a path
between any two vertices, ignoring direction
A complete graph has an edge between every pair of vertices
-
7/29/2019 Part8 Graphs
15/123
Trees as Graphs
Every tree is a graph with
some restrictions:
the tree is directed there are no cycles (directed
or undirected)
there is a directed path from
the root to every node
A
B
D E
C
F
HG
JI
BAD!
-
7/29/2019 Part8 Graphs
16/123
Directed Acyclic Graphs (DAGs)
DAGs are directed
graphs with no
cycles.
main()
add()
access()
mult()
read()
Trees DAGs Graphs
if program call
graph is a DAG,
then allprocedure calls
can be in-lined
-
7/29/2019 Part8 Graphs
17/123
Application of DAGs:
Representing Partial Orders
check in
airport
call
taxi
taxi to
airport
reserve
flight
pack
bagstake
flight
locate
gate
-
7/29/2019 Part8 Graphs
18/123
Topological Sort
Given a graph, G = (V, E), output all the verticesinVsuch that no vertex is output before any other
vertex with an edge to it.
check in
airportcall
taxi
taxi toairport
reserve
flight
pack
bags
take
flightlocate
gate
-
7/29/2019 Part8 Graphs
19/123
Topo-Sort Take One
Label each vertexs in-degree (# of inbound edges)
While there are vertices remaining
Pick a vertex with in-degree of zero and output it
Reduce the in-degree of all vertices adjacent to it
Remove it from the list of vertices
runtime:
-
7/29/2019 Part8 Graphs
20/123
Topo-Sort Take Two
Label each vertexs in-degree
Initialize a queue (or stack)to contain all in-degree zero
verticesWhile there are vertices remaining in the queue
Remove a vertex v with in-degree of zero and output it
Reduce the in-degree of all vertices adjacent to vPut any of these with new in-degree zero on the queue
runtime:
-
7/29/2019 Part8 Graphs
21/123
Recall: Tree Traversals
a
i
d
h j
b
f
k l
ec
g
a b f g k c d h i l j e
-
7/29/2019 Part8 Graphs
22/123
Depth-First Search Pre/Post/Inorder traversals are examples of
depth-first search
Nodes are visited deeply on the left-most branches
before any nodes are visited on the right-most branches Visiting the right branches deeply before the left would still be
depth-first! Crucial idea is go deep first!
Difference in pre/post/in-order is how some computation (e.g.
printing) is done at current node relative to the recursive calls
In DFS the nodes being worked on are kept ona stack
-
7/29/2019 Part8 Graphs
23/123
Iterative Version DFS
Pre-order Traversal
Push root on a Stack
Repeat until Stack is empty:Pop a node
Process it
Push its children on the Stack
-
7/29/2019 Part8 Graphs
24/123
Level-Order Tree Traversal
Consider task of traversing tree level by level fromtop to
bottom (alphabetic order)
Is this also DFS? a
i
d
h j
b
f
k l
ec
g
-
7/29/2019 Part8 Graphs
25/123
Breadth-First Search
No! Level-order traversal is an example ofBreadth-FirstSearch
BFS characteristics
Nodes being worked on maintained in a FIFO Queue, not a stack Iterative style procedures often easier to design than recursive
procedures
Put root in a Queue
Repeat until Queue is empty:
Dequeue a node
Process it
Add its children to queue
-
7/29/2019 Part8 Graphs
26/123
QUEUE
a
b c d e
c d e f g
d e f g
e f g h i j
f g h i j
g h i j
h i j k
i j kj k l
k l
l
a
i
d
h j
b
f
k l
ec
g
-
7/29/2019 Part8 Graphs
27/123
Graph Traversals
Depth first search and breadth first search also work forarbitrary (directed or undirected) graphs
Must mark visited vertices so you do not go into an infinite
loop!
Either can be used to determine connectivity:
Is there a path between two given vertices?
Is the graph (weakly) connected?
Important difference: Breadth-first search always findsa shortest path from the start vertex to any other (for
unweighted graphs)
Depth first search may not!
-
7/29/2019 Part8 Graphs
28/123
Demos on Web Page
DFSBFS
-
7/29/2019 Part8 Graphs
29/123
Is BFS the Hands Down Winner?
Depth-first search Simple to implement (implicit or explict stack)
Does not always find shortest paths
Must be careful to mark visited vertices, or you could
go into an infinite loop if there is a cycle Breadth-first search
Simple to implement (queue)
Always finds shortest paths
Marking visited nodes can improve efficiency, but even
without doing so search is guaranteed to terminate
-
7/29/2019 Part8 Graphs
30/123
Space Requirements
Consider space required by the stack or queue
Suppose
G is known to be at distance dfrom S
Each vertex n has kout-edges
There are no (undirected or directed) cycles
BFS queue will grow to size kd
Will simultaneously contain all nodes that are at
distance d(once last vertex at distance d-1 is expanded)
For k=10, d=15, size is 1,000,000,000,000,000
-
7/29/2019 Part8 Graphs
31/123
DFS Space Requirements
Consider DFS, where we limit the depth of the searchto d
Force a backtrack at d+1
When visiting a node n at depth d, stack will contain
(at most) k-1 siblings ofn parent ofn
siblings of parent ofn
grandparent ofn
siblings of grandparent ofn
DFS queue grows at most to size dk
For k=10, d=15, size is 150
Compare with BFS 1,000,000,000,000,000
-
7/29/2019 Part8 Graphs
32/123
Conclusion
For very large graphsDFS is hugely more
memory efficient, ifwe know the distance to the
goal vertex! But suppose we dont know d. What is the
(obvious) strategy?
-
7/29/2019 Part8 Graphs
33/123
Iterative Deepening DFS
IterativeDeepeningDFS(vertex s, g){
for (i=1;true;i++)
if DFS(i, s, g) return;
}
// Also need to keep track of path found
bool DFS(int limit, vertex s, g){
if (s==g) return true;
if (limit--
-
7/29/2019 Part8 Graphs
34/123
Analysis of Iterative Deepening
Even without marking nodes as visited,
iterative-deepening DFS never goes into an
infinite loop For very large graphs, memory cost of keeping track of
visited vertices may make marking prohibitive
Work performed with limit < actual distance to G
is wastedbut the wasted work is usually smallcompared to amount of work done during the last
iteration
-
7/29/2019 Part8 Graphs
35/123
Asymptotic Analysis
There are pathological graphs for which
iterative deepening is bad:
S G
n=d
2
Iterative Deepening DFS =
1 2 3 ... ( )
BFS = ( )
n O n
O n
-
7/29/2019 Part8 Graphs
36/123
A Better Case
Suppose each vertex n has kout-edges, no cycles
Bounded DFS to level i reaches ki vertices
Iterative Deepening DFS(d) =
1
( )
BFS = ( )
d
i d
i
d
k O k
O k
ignore low order terms!
-
7/29/2019 Part8 Graphs
37/123
(More) Conclusions
To find a shortest path between two nodes in a
unweighted graph, use either BFS or Iterated DFS
If the graph is large, Iterated DFS typically uses
much less memory
Later well learn about heuristic search algorithms,
which use additional knowledge about the problem
domain to reduce the number of vertices visited
-
7/29/2019 Part8 Graphs
38/123
Single Source, Shortest Path for
Weighted GraphsGiven a graph G = (V, E) with edge costs c(e),
and a vertex s V, find the shortest (lowest cost)
path from s to every vertex inV
Graph may be directed or undirected
Graph may or may not contain cycles
Weights may be all positive or not
What is the problem if graph contains cycleswhose total cost is negative?
-
7/29/2019 Part8 Graphs
39/123
The Trouble with
Negative Weighted Cycles
A B
C D
E
2
101-5
2
-
7/29/2019 Part8 Graphs
40/123
Edsger Wybe Dijkstra
(1930-2002)
Invented concepts of structured programming,synchronization, weakest precondition, and "semaphores"
for controlling computer processes. The Oxford EnglishDictionary cites his use of the words "vector" and "stack" ina computing context.
Believed programming should be taught without computers
1972 Turing Award
In their capacity as a tool, computers will be but a ripple onthe surface of our culture. In their capacity as intellectualchallenge, they are without precedent in the cultural historyof mankind.
-
7/29/2019 Part8 Graphs
41/123
Dijkstras Algorithm for
Single Source Shortest Path Classic algorithm for solving shortest path in
weighted graphs (with onlypositive edge weights)
Similar to breadth-first search, but uses a priorityqueue instead of a FIFO queue:
Always select (expand) the vertex that has a lowest-costpath to the start vertex
a kind of greedy algorithm
Correctly handles the case where the lowest-cost(shortest) path to a vertex is not the one withfewest edges
-
7/29/2019 Part8 Graphs
42/123
Pseudocode for Dijkstra
Initialize the cost of each vertex to cost[s] = 0;
heap.insert(s);
While (! heap.empty())
n = heap.deleteMin()For (each vertex a which is adjacent to n along edge e)
if (cost[n] + edge_cost[e] < cost[a]) then
cost [a] = cost[n] + edge_cost[e]
previous_on_path_to[a] = n;if (a is in the heap) then heap.decreaseKey(a)
else heap.insert(a)
-
7/29/2019 Part8 Graphs
43/123
Important Features
Once a vertex is removed from the head, the cost
of the shortest path to that node is known
While a vertex is still in the heap, another shorterpath to it might still be found
The shortest path itselffrom s to any node a can
be found by following the pointers stored in
previous_on_path_to[a]
-
7/29/2019 Part8 Graphs
44/123
Dijkstras Algorithm in Action
A
C
B
D
F H
G
E
2 2 3
21
1
410
8
11
94
2
7
vertex known cost
A
B
C
D
E
F
G
H
-
7/29/2019 Part8 Graphs
45/123
Demo
Dijkstras
-
7/29/2019 Part8 Graphs
46/123
Data Structures
for Dijkstras AlgorithmSelect the unknown node with the lowest cost
findMin/deleteMin
as cost = min(as old cost, )
decreaseKey
|V| times:
|E| times:
runtime: O(|E| log |V|)
O(log |V|)
O(log |V|)
-
7/29/2019 Part8 Graphs
47/123
CSE 326: Data Structures
Lecture 8.B
Heuristic Graph Search
Henry Kautz
Winter Quarter 2002
-
7/29/2019 Part8 Graphs
48/123
Homework Hint - Problem 4
You can turn in a final version of your answer toproblem 4 without penalty on Wednesday.
1
1 2 1 1
1
2
1 2
final mod in case sum is
Let ( ... ) be the interpretation of a bit string
as a bin
( ) mod ( mod
ary numbe
mod ) mod
( ( mod )) mod ( )
r. Then:
mod
( ... ) 2 ( .
k
k k k k k k
k k k
a b p a p b p p
c a p
p
b b b b
p ca p
b b b b b b b
1.. )b
-
7/29/2019 Part8 Graphs
49/123
Outline
Best First Search
A* Search
Example: Plan Synthesis
This material is NOT in Weiss, but is important
for both the programming project and the finalexam!
-
7/29/2019 Part8 Graphs
50/123
Huge Graphs
Consider some really hugegraphs
All cities and towns in the World Atlas
All stars in the Galaxy
All ways 10 blocks can be stacked
Huh???
http://images.google.com/imgres?imgurl=skyserver.fnal.gov/images/sdss-float-galaxy-1.gif&imgrefurl=http://skyserver.fnal.gov/&h=160&w=160&prev=/images%3Fq%3Dgalaxy%26svnum%3D10%26hl%3Denhttp://images.google.com/imgres?imgurl=www.sgi.com/sales/images/earth.jpg&imgrefurl=http://www.sgi.com/sales/solutions_finance/&h=148&w=150&prev=/images%3Fq%3Dearth%26svnum%3D10%26hl%3Den -
7/29/2019 Part8 Graphs
51/123
Implicitly Generated Graphs
A huge graph may be implicitly specified by rules forgenerating it on-the-fly
Blocks world: vertex = relative positions of all blocks
edge = robot arm stacks one block
stack(blue,red)
stack(green,red)
stack(green,blue)
stack(blue,table)
stack(green,blue)
-
7/29/2019 Part8 Graphs
52/123
Blocks World
Source = initial state of the blocks
Goal = desired state of the blocks
Path source to goal = sequence of actions(program) for robot arm!
n blocks nn vertices
10 blocks
10 billion vertices!
-
7/29/2019 Part8 Graphs
53/123
Problem: Branching Factor
Cannot search such huge graphs exhaustively.
Suppose we know that goal is only dsteps away.
Dijkstras algorithm is basically breadth-firstsearch (modified to handle arc weights)
Breadth-first search (or for weighted graphs,
Dijkstras algorithm) If out-degree of each node
is 10, potentially visits 10dvertices 10 step plan = 10 billion vertices visited!
-
7/29/2019 Part8 Graphs
54/123
An Easier Case
Suppose you live in Manhattan; what do you do?
52nd St
51st St
50th St
10thAve
9thAve
8thAve
7thAve
6thAve
5thAve
4thAve
3rdAve
2ndAve
S
G
-
7/29/2019 Part8 Graphs
55/123
Best-First Search
TheManhattan distance ( x+ y) is an estimate
of the distance to the goal
a heuristic value
Best-First Search
Order nodes in priority to minimize estimated distance
to the goal h(n)
Compare: BFS / Dijkstra Order nodes in priority to minimize distance from the
start
-
7/29/2019 Part8 Graphs
56/123
Best First in Action
Suppose you live in Manhattan; what do you do?
52nd St
51st St
50th St
10thAve
9thAve
8thAve
7thAve
6thAve
5thAve
4thAve
3rdAve
2ndAve
S
G
-
7/29/2019 Part8 Graphs
57/123
Problem 1: Led Astray
Eventually will expand vertex to get back on the
right track
52nd St
51st St
50th St
10thAve
9thAve
8thAve
7thAve
6thAve
5thAve
4thAve
3rdAve
2ndAve
SG
-
7/29/2019 Part8 Graphs
58/123
Problem 2: Optimality
With Best-First Search, are you guaranteeda
shortest path is found when
goal is first seen?
when goal is removed from priority queue (as with
Dijkstra?)
-
7/29/2019 Part8 Graphs
59/123
Sub-Optimal Solution No! Goal is by definition at distance 0: will be
removed from priority queue immediately, even if
a shorter path exists!
52nd St
51st St
9thAve
8thAve
7thAve
6thAve
5thAve
4thAve
S
G
(5 blocks)
h=2
h=1h=4
h=5
-
7/29/2019 Part8 Graphs
60/123
Synergy?
Dijkstra / Breadth First guaranteed to find optimal
solution
Best First often visitsfar fewervertices, but maynot provide optimal solution
Can we get the best of both?
-
7/29/2019 Part8 Graphs
61/123
A* (A star)
Order vertices in priority queue to minimize
(distance from start) + (estimated distance to goal)
f(n) = g(n) + h(n)
f(n) = priority of a nodeg(n) = true distance from start
h(n) = heuristic distance to goal
-
7/29/2019 Part8 Graphs
62/123
Optimality Suppose the estimated distance (h) is
always less than or equal to the true distance to the
goal
heuristic is a lower bound on true distance
Then: when the goal is removed from the priority
queue, we are guaranteed to have found a shortest
path!
-
7/29/2019 Part8 Graphs
63/123
Problem 2 Revisited
52nd St
51st St
9thAve
8thAve
7thAve
6thAve
5thAve
4thAve
S
G
(5 blocks)
50th St
vertex g(n) h(n) f(n)
52nd & 9th 0 5 5
-
7/29/2019 Part8 Graphs
64/123
Problem 2 Revisited
52nd St
51st St
9thAve
8thAve
7thAve
6thAve
5thAve
4thAve
S
G
(5 blocks)
50th St
vertex g(n) h(n) f(n)
52nd & 4th 5 2 7
51st & 9th 1 4 5
-
7/29/2019 Part8 Graphs
65/123
Problem 2 Revisited
52nd St
51st St
9thAve
8thAve
7thAve
6thAve
5thAve
4thAve
S
G
(5 blocks)
50th St
vertex g(n) h(n) f(n)
52nd & 4th 5 2 7
51st & 8th 2 3 5
50th & 9th 2 5 7
-
7/29/2019 Part8 Graphs
66/123
Problem 2 Revisited
52nd St
51st St
9thAve
8thAve
7thAve
6thAve
5thAve
4thAve
S
G
(5 blocks)
50th St
vertex g(n) h(n) f(n)
52nd & 4th 5 2 7
51st & 7th 3 2 5
50th & 9th 2 5 7
50th & 8th 3 4 7
-
7/29/2019 Part8 Graphs
67/123
Problem 2 Revisited
52nd St
51st St
9thAve
8thAve
7thAve
6thAve
5thAve
4thAve
S
G
(5 blocks)
50th St
vertex g(n) h(n) f(n)
52nd & 4th 5 2 7
51st & 6th 4 1 5
50th & 9th 2 5 7
50th & 8th 3 4 7
50th
& 7th
4 3 7
-
7/29/2019 Part8 Graphs
68/123
Problem 2 Revisited
52nd St
51st St
9thAve
8thAve
7thAve
6thAve
5thAve
4thAve
S
G
(5 blocks)
50th St
vertex g(n) h(n) f(n)
52nd & 4th 5 2 7
51st & 5th 5 0 5
50th & 9th 2 5 7
50th & 8th 3 4 7
50th
& 7th
4 3 7
-
7/29/2019 Part8 Graphs
69/123
Problem 2 Revisited
52nd St
51st St
9thAve
8thAve
7thAve
6thAve
5thAve
4thAve
S
G
(5 blocks)
50th St
vertex g(n) h(n) f(n)
52nd & 4th 5 2 7
50th & 9th 2 5 7
50th & 8th 3 4 7
50th & 7th 4 3 7
DONE!
-
7/29/2019 Part8 Graphs
70/123
What Would Dijkstra Have
Done?
52nd St
51st St
9thAve
8thAv
e
7thAv
e
6thAv
e
5thAv
e
4thAv
e
S
G
(5 blocks)
50th St
49th St
48th St
47th St
Proof of A* Optimality
-
7/29/2019 Part8 Graphs
71/123
Proof of A* Optimality
A* terminates when G is popped from the heap.
Suppose G is popped but the path found isnt optimal:
priority(G) > optimal path length c
Let P be an optimal path from S to G, and let N be the lastvertex on that path that has been visited but not yet popped.
There must be such an N, otherwise the optimal path would have beenfound.
priority(N) = g(N) + h(N) c
So N should have popped before G can pop. Contradiction.
S
N
G
non-optimal path to G
portion of optimal
path found so far
undiscovered portion
of shortest path
-
7/29/2019 Part8 Graphs
72/123
What About Those Blocks?
Distance to goal is not always physical distance
Blocks world:
distance = number of stacks to perform heuristic lower bound = number of blocks out of place
# out of place = 2, true distance to goal = 3
3-Blocks State
S G h
-
7/29/2019 Part8 Graphs
73/123
Space GraphABC
h=2
C
AB
h=3
B
AC
h=2
A
BC
h=1
C
BA
h=3
A
CB
h=2
B
CA
h=1
B
C
A
h=3
C
B
A
h=3
C
A
B
h=3
A
C
B
h=3
B
A
C
h=2
A
B
C
h=0
start goal
3-BlocksBest First
-
7/29/2019 Part8 Graphs
74/123
Solution ABC
h=2
C
AB
h=3
B
AC
h=2
A
BC
h=1
C
BA
h=3
A
CB
h=2
B
CA
h=1
B
C
A
h=3
C
B
A
h=3
C
A
B
h=3
A
C
B
h=3
B
A
C
h=2
A
B
C
h=0
start goal
3-Blocks BFS
Solution
-
7/29/2019 Part8 Graphs
75/123
SolutionABC
h=2
C
AB
h=3
B
AC
h=2
A
BC
h=1
C
BA
h=3
A
CB
h=2
B
CA
h=1
B
C
A
h=3
C
B
A
h=3
C
A
B
h=3
A
C
B
h=3
B
A
C
h=2
A
B
C
h=0
expanded, but not
in solution
start goal
3-Blocks A*
Solution
-
7/29/2019 Part8 Graphs
76/123
SolutionABC
h=2
C
AB
h=3
B
AC
h=2
A
BC
h=1
C
BA
h=3
A
CB
h=2
B
CA
h=1
B
C
A
h=3
C
B
A
h=3
C
A
B
h=3
A
C
B
h=3
B
A
C
h=2
A
B
C
h=0
expanded, but not
in solution
start goal
-
7/29/2019 Part8 Graphs
77/123
Other Real-World Applications
Routing findingcomputer networks, airlineroute planning
VLSI layoutcell layout and channel routing
Production planningjust in time optimization
Protein sequence alignment
Many other NP-Hard problems
A class of problems for which no exact polynomialtime algorithms existso heuristic search is the bestwe can hope for
-
7/29/2019 Part8 Graphs
78/123
Coming Up
Other graph problems
Connected components
Spanning tree
-
7/29/2019 Part8 Graphs
79/123
CSE 326: Data Structures
Part 8.C
Spanning Trees and More
Henry Kautz
Autumn Quarter 2002
-
7/29/2019 Part8 Graphs
80/123
Today
Incremental hashing
MazeRunner project
Longest Path? Finding Connected Components
Application to machine vision
Finding Minimum Spanning Trees
Yet another use for union/find
Incremental Hashing
-
7/29/2019 Part8 Graphs
81/123
Incremental Hashing
11 1
2 1 1
2 2
11 1
1
1 1
11
1 11
1
1 1
1
2
1
( ... ) % %
( ..
%
%
% %
. ) %
n nn i n i
n i n i
i i
nn
n
nn i
nn i
n ii
nn n i
i
i
nn i
n
n
i
i
in
i
h a a c a p a c a p
a c a p
a c
h a a c a p
c a c a
ca p
a p c
a
c cp a
c
a
11 11
%
( ... )
%
%n
nn
p
a c a c a p
p
h a
20 15
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-
7/29/2019 Part8 Graphs
82/123
Maze Runner+ + + + + + + + + + + + + + + + + + + + +
|* |
+ + + + + + + + + + + + + + + + + + + + +
| | | | | | | | | | | | | | | | | | | | |
+-+-+-+-+ +-+ +-+ +-+ +-+-+ +-+-+-+-+-+-+
| | | | | | | | | | | |
+-+-+-+-+-+-+-+-+-+-+-+ + + + + + + +-+ +
|X | | | | | |
+-+ + +-+-+ +-+-+-+ +-+ +-+ +-+-+-+-+-+-+
| | | | | | | | | | | | | | | | | |
+ + + + + + + + + +-+ + + +-+ + + +-+ +-+
| | | | | | | | | | | | | | |
+-+-+ + + + + + + + + + + + + +-+ + + + +
| | | | | | | | | |
+ + + +-+ + + + + + + + + + + + + + + + +
| | | | | | | | | | | | | | |
+ + + + + + + + + + + + + + +-+-+-+-+ +-+
| | | | | | | | | | | | | | | |
+ + + + + + + + + + +-+ +-+-+ + + +-+-+ +
| | | | | | | | | | | | | | | |
+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+
| | | | | | | | | | | | |
+ + + + + + + + + + +-+ +-+-+-+-+ +-+-+-+
| | | | | | | | | | | | | |
+ + + + +-+ +-+ + + + + +-+ + +-+ + + + +
| | | | | | | | | | | |
+ + +-+-+-+-+ +-+ +-+-+-+ +-+-+ +-+ +-+ +
| | | | | | | | | | | | | | | | | |
+ + + + + + + + + + + + + + + + + + +-+ +
| | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
DFS, iterated DFS, BFS,
best-first, A*
Crufty old C++ code from
fresh clean Java code
Win fame and glory by
writing a nice real-time maze
visualizer
-
7/29/2019 Part8 Graphs
83/123
Java Note
Java lacks enumerated constants
enum {DOG, CAT, MOUSE} animal;
animal a = DOG;
Static constants not type-safe
static final int DOG = 1;
static final int CAT = 2;
static final int BLUE = 1;
int favoriteColor = DOG;
-
7/29/2019 Part8 Graphs
84/123
Amazing Java Trick
public final class Animal {
private Animal() {}
public static final Animal DOG = new Animal();
public static final Animal CAT = new Animal();
}
public final class Color {
private Color() {}
public static final Animal BLUE = new Color();
}
Animal x = DOG;
Animal x = BLUE; // Gives compile-time error!
-
7/29/2019 Part8 Graphs
85/123
Longest Path Problem
Given a graph G=(V,E) and vertices s, t
Find a longest simple path (no repeating vertices)
from s to t. Does reverse Dijkstra work?
Dijkstra
-
7/29/2019 Part8 Graphs
86/123
Dijkstra
Initialize the cost of each vertex to
cost[s] = 0;
heap.insert(s);
While (! heap.empty())
n = heap.deleteMin()For (each vertex a which is adjacent to n along edge e)
if (cost[n] + edge_cost[e] < cost[a]) then
cost [a] = cost[n] + edge_cost[e]
previous_on_path_to[a] = n;if (a is in the heap) then heap.decreaseKey(a)
else heap.insert(a)
Reverse Dijkstra
-
7/29/2019 Part8 Graphs
87/123
Reverse Dijkstra
Initialize the cost of each vertex to
cost[s] = 0;
heap.insert(s);
While (! heap.empty())
n = heap.deleteMax()For (each vertex a which is adjacent to n along edge e)
if (cost[n] + edge_cost[e] > cost[a]) then
cost [a] = cost[n] + edge_cost[e]
previous_on_path_to[a] = n;if (a is in the heap) then heap.increaseKey(a)
else heap.insert(a)
-
7/29/2019 Part8 Graphs
88/123
Does it Work?
s
ta
b
3
1
5
6
-
7/29/2019 Part8 Graphs
89/123
Problem
No clear stopping condition!
How many times could a vertex be inserted in the
priority queue?
Exponential!
Not a good algorithm!
Is the better one?
-
7/29/2019 Part8 Graphs
90/123
Counting Connected Components
Initialize the cost of each vertex to
Num_cc = 0
While there are vertices of cost {
Pick an arbitrary such vertex S, set its cost to 0Find paths from S
Num_cc ++ }
-
7/29/2019 Part8 Graphs
91/123
Using DFS
Set each vertex to unvisitedNum_cc = 0
While there are unvisited vertices {
Pick an arbitrary such vertex S
Perform DFS from S, marking vertices as visited
Num_cc ++ }
Complexity = O(|V|+|E|)
-
7/29/2019 Part8 Graphs
92/123
Using Union / Find
Put each node in its own equivalence classNum_cc = 0
For each edge E =
Union(x,y)Return number of equivalence classes
Complexity =
-
7/29/2019 Part8 Graphs
93/123
Using Union / Find
Put each node in its own equivalence classNum_cc = 0
For each edge E =
Union(x,y)Return number of equivalence classes
Complexity = O(|V|+|E| ack(|E|,|V|))
-
7/29/2019 Part8 Graphs
94/123
Machine Vision: Blob Finding
-
7/29/2019 Part8 Graphs
95/123
Machine Vision: Blob Finding
1
2
34
5
-
7/29/2019 Part8 Graphs
96/123
Blob Finding
Matrix can be considered an efficient
representation of a graph with a very regular
structure
Cell = vertex
Adjacent cells of same color = edge between
vertices
Blob finding = finding connected components
-
7/29/2019 Part8 Graphs
97/123
Tradeoffs
Both DFS and Union/Find approaches are
(essentially) O(|E|+|V|) = O(|E|) for binary images
For each component, DFS (recursive labeling)
can move all over the imageentire image must
be in main memory
Better in practice: row-by-row processing
localizes accesses to memory typically 1-2 orders of magnitude faster!
High Level Blob Labeling
-
7/29/2019 Part8 Graphs
98/123
High-Level Blob-Labeling
Scan through image left/right and top/bottom
If a cell is same color as (connected to) cell to
right or below, then union them
Give the same blob number to cells in each
equivalence class
Blob Labeling Algorithm
-
7/29/2019 Part8 Graphs
99/123
Blob-Labeling Algorithm
Put each cell in its own equivalence classFor each cell
if color[x,y] == color[x+1,y] then
Union( , )
if color[x,y] == color[x,y+1] thenUnion( , )
label = 0
For each root
blobnum[x,y] = ++ label;For each cell
blobnum[x,y] = blobnum( Find() )
-
7/29/2019 Part8 Graphs
100/123
Spanning tree: a subset of the edges from a connected graphthat
touches all vertices in the graph (spans the graph)
forms a tree (is connected and contains no cycles)
Minimum spanning tree: the spanning tree with the least totaledge cost.
Spanning Tree
4 7
1 5
9
2
Applications of Minimal
-
7/29/2019 Part8 Graphs
101/123
Applications of Minimal
Spanning Trees Communication networks
VLSI design
Transportation systems
Kruskals Algorithm for
http://images.google.com/imgres?imgurl=www.metropla.net/am/nyrk/nyc-map-centre.gif&imgrefurl=http://www.metropla.net/am/nyrk/nyc.htm&h=716&w=748&prev=/images%3Fq%3Dnyc%2Bsubway%26start%3D40%26svnum%3D10%26hl%3Den%26sa%3DNhttp://www.gel.ulaval.ca/~vision/vlsi/gifs/vlsi.gif -
7/29/2019 Part8 Graphs
102/123
Kruskal s Algorithm for
Minimum Spanning TreesA greedy algorithm:
Initialize all vertices to unconnectedWhile there are still unmarked edges
Pick a lowest cost edge e = (u, v) and mark it
Ifu and v are not already connected, add e to the
minimum spanning tree and connect u and v
Sound familiar?
(Think maze generation.)
-
7/29/2019 Part8 Graphs
103/123
Kruskals Algorithm in Action (1/5)
A
C
B
D
F H
G
E
2 2 3
2
1
4
10
8
1
94
2
7
-
7/29/2019 Part8 Graphs
104/123
Kruskals Algorithm in Action (2/5)
A
C
B
D
F H
G
E
2 2 3
2
1
4
10
8
1
94
2
7
-
7/29/2019 Part8 Graphs
105/123
Kruskals Algorithm in Action (3/5)
A
C
B
D
F H
G
E
2 2 3
2
1
4
10
8
1
94
2
7
-
7/29/2019 Part8 Graphs
106/123
Kruskals Algorithm in Action (4/5)
A
C
B
D
F H
G
E
2 2 3
2
1
4
10
8
1
94
2
7
-
7/29/2019 Part8 Graphs
107/123
Kruskals Algorithm Completed (5/5)
A
C
B
D
F H
G
E
2 2 3
2
1
4
10
8
1
94
2
7
Why Greediness Works
-
7/29/2019 Part8 Graphs
108/123
yProof by contradictionthat Kruskals finds a minimum
spanning tree:
Assume another spanning tree has lower costthanKruskals.
Pick an edge e1 = (u, v)in that tree thats notinKruskals.
Consider the point in Kruskals algorithm where us setand vs set were about to be connected. Kruskal selectedsome edge to connect them: call it e2 .
But, e2 must have at most the same cost as e1 (otherwiseKruskal would have selected it instead).
So, swap e2 for e1 (at worst keeping the cost the same)
Repeat until the tree is identicalto Kruskals, where thecost is the same or lower than the original cost:contradiction!
Data Structures
-
7/29/2019 Part8 Graphs
109/123
Data Structures
for Kruskals AlgorithmPick the lowest cost edge
findMin/deleteMin
Ifu and v are not already connected
connect u and v.
union
|E| times:
|E| times:
runtime:
Once:
Initialize heap of edges
buildHeap
|E| + |E| log |E| + |E| ack(|E|,|V|)
Data Structures
-
7/29/2019 Part8 Graphs
110/123
Data Structures
for Kruskals AlgorithmPick the lowest cost edge
findMin/deleteMin
Ifu and v are not already connected
connect u and v.
union
|E| times:
|E| times:
runtime:
Once:
Initialize heap of edges
buildHeap
|E| + |E| log |E| + |E| ack(|E|,|V|) = O(|E|log|E|)
-
7/29/2019 Part8 Graphs
111/123
Prims Algorithm
Can also find Minimum Spanning Trees using avariation of Dijkstras algorithm:
Pick a initial node
Until graph is connected:
Choose edge (u,v) which is of minimum costamong edges where u is in tree but v is not
Add (u,v) to the tree Same greedy proof, same asymptotic
complexity
-
7/29/2019 Part8 Graphs
112/123
Coming Up
Application: Sentence Disambiguation
All-pairs Shortest Paths
NP-Complete Problems Advanced topics
Quad trees
Randomized algorithms
Sentence Disambiguation
-
7/29/2019 Part8 Graphs
113/123
Sentence Disambiguation
A person types a message on their cell phonekeypad. Each button can stand for three different
letter (e.g. 1 is a, b, or c), but the person does not
explicitly indicate which letter is meant. (Words are
separated by blanksthe 0 key.)
Problem: How can the system determine what
sentence was typed?
My Nokia cell phone does this! How can this problem be cast as a shortest-path
problem?
-
7/29/2019 Part8 Graphs
114/123
Sentence Disambiguation as
-
7/29/2019 Part8 Graphs
115/123
Sentence Disambiguation as
Shortest PathIdea: Possible words are vertices
Directed edge between adjacent possible words
Weight on edge from W1 to W2 is probability thatW2 appears adjacent to W1 Probabilities over what?! Some large archive (corpus)
of text
Word bi-gram model
Find the most probable path through the graph
-
7/29/2019 Part8 Graphs
116/123
W11
W11W3
1
W41
W21
W12
W22
W13
W23
W33
W43
-
7/29/2019 Part8 Graphs
117/123
Technical Concerns
Isnt most probable actually longest (most
heavily weighted) path?!
Shouldnt we be multiplying probabilities, not
adding them?!
1 2 3 1 2 1 3 2 3(# #) ( | #) ( | ) ( | ) (# | )P w w w P w P w w P w w P w
-
7/29/2019 Part8 Graphs
118/123
Logs to the Rescue
Make weight on edge fromW1 to W2 be
- log P(W2 | W1)
Logs of probabilities are always negativenumbers, so take negative logs
The lower the probability, the larger the negative
log! So this is shortest path
Adding logs is the same as multiplying theunderlying quantities
hi k b
-
7/29/2019 Part8 Graphs
119/123
To Think About
This really works in practice99% accuracy!
Cell phone memory is limitedhow can we use aslittle storage as possible?
How can the system customize itself to a user?
Q i
-
7/29/2019 Part8 Graphs
120/123
Question
Which graph algorithm is asymptotically
better:
(|V||E|log|V|)
(|V|3)
All P i Sh P h
-
7/29/2019 Part8 Graphs
121/123
All Pairs Shortest Path
Suppose you want to compute the length of the
shortest paths between all pairs of vertices in a
graph
Run Dijkstras algorithm (with priority queue)
repeatedly, starting with each node in the graph:
Complexity in terms of V when graph is dense:
Dynamic Programming
-
7/29/2019 Part8 Graphs
122/123
Dynamic Programming
Approach
,
, ,
1, 2
, , 1, , 1, ,
,
1, ,
Note that path for
distance from to that u
either does not use ,
or merges the paths
ses
only ,..., as intermediates
min{ , }
and
k i j
k i j i j
k
k i j
k
i
k i j k i k k k j
k k j
D v v
v v v
D D D D
D v
v v v v
Floyd-Warshall Algorithm
-
7/29/2019 Part8 Graphs
123/123
y g
// C adjacency matrix representation of graph
// C[i][j] = weighted edge i->j or if none// D computed distances
FW(int n, int C [][], int D [][]){
for (i = 0; i < N; i++){
for (j = 0; j < N; j++)
D[i][j] = C[i][j];
D[i][i] = 0.0;
}
for (k = 0; k < N; k++)
for (i = 0; i < N; i++)
for (j = 0; j < N; j++)
if (D[i][k] + D[k][j] < D[i][j])
Run time =
How could we
compute the paths?