1 ee 355 unit 14 a-star algorithm & heaps/priority queues mark redekopp

1

EE 355 Unit 14A-Star Algorithm &

Heaps/Priority QueuesMark Redekopp

2

ALGORITHM HIGHLIGHTA* Search Algorithm

3

Search Methods

• Many systems require searching for goal states– Path Planning

• Roomba Vacuum• Mapquest/Google Maps• Games!!

– Optimization Problems• Find the optimal solution to a problem with many

constraints

4

Search Applied to 8-Tile Game• 8-Tile Puzzle

– 3x3 grid with one blank space– With a series of moves, get the tiles in sequential

order– Goal state:

1 2

3 4 5

6 7 8

PA4 Goal State

1 2 3

4 5 6

7 8

Goal State for these slides

5

Search Methods

• Brute-Force Search: When you don’t know where the answer is, just search all possibilities until you find it.

• Heuristic Search: A heuristic is a “rule of thumb”. An example is in a chess game, to decide which move to make, count the values of the pieces left for your opponent. Use that value to “score” the possible moves you can make.– Heuristics are not perfect measures, they are quick

computations to give an approximation (e.g. may not take into account “delayed gratification” or “setting up an opponent”)

6

Brute Force Search

• Brute Force Search Tree– Generate all

possible moves– Explore each move

despite its proximity to the goal node

1 24 8 37 6 5

1 24 8 37 6 5

1 24 8 37 6 5

1 8 24 37 6 5

1 24 8 37 6 5

1 24 8 37 6 5

1 2 34 8 57 6

1 2 34 87 6 5

1 2 34 87 6 5

1 2 34 8 57 6

1 2 34 57 8 6

W S

W S E

7

Heuristics

• Heuristics are “scores” of how close a state is to the goal (usually, lower = better)

• These scores must be easy to compute (i.e. simpler than solving the problem)

• Heuristics can usually be developed by simplifying the constraints on a problem

• Heuristics for 8-tile puzzle– # of tiles out of place

• Simplified problem: If we could just pick a tile up and put it in its correct place

– Total x-, y- distance of each tile from its correct location (Manhattan distance)

• Simplified problem if tiles could stack on top of each other / hop over each other

1 8 3

4 5 6

2 7

1 8 3

4 5 6

2 7

# of Tiles out of Place = 3

Total x-/y- distance = 6

8

Heuristic Search

• Heuristic Search Tree– Use total x-/y-

distance (Manhattan distance) heuristic

– Explore the lowest scored states

1 24 8 37 6 5

1 24 8 37 6 5

1 24 8 37 6 5

1 2 34 8 57 6

1 2 34 5 67 8

1 2 34 87 6 5

1 2 34 87 6 5

1 2 34 8 57 6

1 2 34 57 8 6

1 2 34 57 8 6

H=6

H=7 H=5

H=6 H=6 H=4

H=3

H=2

H=1

Goal

1 2 34 87 6 5

H=5

9

Caution About Heuristics

• Heuristics are just estimates and thus could be wrong

• Sometimes pursuing lowest heuristic score leads to a less-than optimal solution or even no solution

• Solution– Take # of moves from start (depth)

into account

H=2

Start

H=1

H=1

H=1

H=1

H=1

H=1

…

Goal

H=1

10

A-star Algorithm• Use a new metric to decide

which state to explore/expand• Define

– h = heuristic score (same as always)

– g = number of moves from start it took to get to current state

– f = g + h• As we explore states and their

successors, assign each state its f-score and always explore the state with lowest f-score

• Heuristics should always underestimate the distance to the goal– If they do, A* guarantees optimal

solutions

g=1,h=2f=3

Start

g=1,h=1f=2

g=2,h=1f=3

g=3,h=1f=4

Goal

g=2,h=1f=3

11

A-Star Algorithm• Maintain 2 lists

– Open list = Nodes to be explored (chosen from)

– Closed list = Nodes already explored (already chosen)

• Pseudocodeopen_list.push(Start State)while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. Generate successors/neighbors of s, compute their f values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

g=1,h=2f=3

Start

g=1,h=1f=2

g=2,h=1f=3

g=3,h=1f=4

Goal

g=2,h=1f=3

12

Data Structures & Ordering• We’ve seen some abstract data

structures– Queue (first-in, first-out access order)

• Deque works nicely

• There are others…– Stack (last-in, first-out access order)

• Vector push_back / pop_back

– Trees

• A new one…priority queues (heaps)– Order of access depends on value– Items arrive in some arbitrary order– When removing an item, we always want

the minimum or maximum valued item– To implement efficiently use heap data

structure

remove_min = 21

46 78 35 67

original heap46 21 78 35 67

0 1 2 3 4

remove_min = 35

46 78 67

13

Path-Planning w/ A* Algorithm• Find optimal path from S to G using A*

– Use heuristic of Manhattan (x-/y-) distance

S

G

open_list.push(Start State)while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

Closed List

Open List

**If implementing this for a programming assignment, please see the slide at the end about alternate closed-list implementation

14



S

G

Closed List

Open Listg=0,h=6,f=6


**If implementing this for a programming assignment, please see the slide at the end about alternate closed-list implementation

15



g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List


16



g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

g=2,h=6,f=8

g=2,h=4,f=6


17



g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

g=2,h=6,f=8

g=2,h=4,f=6


18



g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

g=2,h=6,f=8

g=2,h=4,f=6

g=1,h=5,f=6

g=2,h=6,f=8


19



g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

g=2,h=6,f=8

g=2,h=4,f=6

g=2,h=6,f=8

g=3,h=7,f=10

g=3,h=5,f=8


20



g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

g=2,h=6,f=8

g=2,h=4,f=6

g=2,h=6,f=8

g=3,h=7,f=10

g=3,h=5,f=8

g=2,h=8,f=10

g=2,h=8,f=10

g=2,h=8,f=10

g=3,h=7,f=10

g=3,h=5,f=8

g=4,h=6,f=10

g=4,h=4,f=8


21



g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

g=2,h=6,f=8

g=2,h=4,f=6

g=2,h=6,f=8

g=3,h=7,f=10

g=3,h=5,f=8

g=2,h=8,f=10

g=2,h=8,f=10

g=2,h=8,f=10

g=3,h=7,f=10

g=3,h=5,f=8

g=4,h=6,f=10

g=4,h=4,f=8

g=5,h=5,f=10

g=5,h=5,f=10

g=5,h=3,f=8

g=1,h=7,f=8

g=1,h=7,f=8

g=2,h=6,f=8

g=1,h=7,f=8

g=1,h=7,f=8

g=2,h=6,f=8


22



g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

g=2,h=6,f=8

g=2,h=4,f=6

g=2,h=6,f=8

g=3,h=7,f=10

g=3,h=5,f=8

g=2,h=8,f=10

g=2,h=8,f=10

g=2,h=8,f=10

g=3,h=7,f=10

g=3,h=5,f=8

g=4,h=6,f=10

g=4,h=4,f=8

g=5,h=5,f=10

g=5,h=5,f=10

g=5,h=3,f=8

g=6,h=4,f=10

g=6,h=2,f=8

g=1,h=7,f=8

g=1,h=7,f=8

g=2,h=6,f=8


23



g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

g=2,h=6,f=8

g=2,h=4,f=6

g=2,h=6,f=8

g=3,h=7,f=10

g=3,h=5,f=8

g=2,h=8,f=10

g=2,h=8,f=10

g=2,h=8,f=10

g=3,h=7,f=10

g=3,h=5,f=8

g=4,h=6,f=10

g=4,h=4,f=8

g=5,h=5,f=10

g=5,h=5,f=10

g=5,h=3,f=8

g=6,h=4,f=10

g=6,h=2,f=8

g=7,h=3,f=10

g=7,h=1,f=8

g=1,h=7,f=8

g=1,h=7,f=8

g=2,h=6,f=8


24



g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

g=2,h=6,f=8

g=2,h=4,f=6

g=2,h=6,f=8

g=3,h=7,f=10

g=3,h=5,f=8

g=2,h=8,f=10

g=2,h=8,f=10

g=2,h=8,f=10

g=3,h=7,f=10

g=3,h=5,f=8

g=4,h=6,f=10

g=4,h=4,f=8

g=5,h=5,f=10

g=5,h=5,f=10

g=5,h=3,f=8

g=6,h=4,f=10

g=6,h=2,f=8

g=7,h=3,f=10

g=7,h=1,f=8

g=8,h=2,f=10

g=8,h=0,f=8

g=1,h=7,f=8

g=1,h=7,f=8

g=2,h=6,f=8


25



g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

g=2,h=6,f=8

g=2,h=4,f=6

g=2,h=6,f=8

g=3,h=7,f=10

g=3,h=5,f=8

g=2,h=8,f=10

g=2,h=8,f=10

g=2,h=8,f=10

g=3,h=7,f=10

g=3,h=5,f=8

g=4,h=6,f=10

g=4,h=4,f=8

g=5,h=5,f=10

g=5,h=5,f=10

g=5,h=3,f=8

g=6,h=4,f=10

g=6,h=2,f=8

g=7,h=3,f=10

g=7,h=1,f=8

g=8,h=2,f=10

g=8,h=0,f=8

g=1,h=7,f=8

g=1,h=7,f=8

g=2,h=6,f=8


26



g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

g=2,h=6,f=8

g=2,h=4,f=6

g=2,h=6,f=8

g=3,h=7,f=10

g=3,h=5,f=8

g=2,h=8,f=10

g=2,h=8,f=10

g=2,h=8,f=10

g=3,h=7,f=10

g=3,h=5,f=8

g=4,h=6,f=10

g=4,h=4,f=8

g=5,h=5,f=10

g=5,h=5,f=10

g=5,h=3,f=8

g=6,h=4,f=10

g=6,h=2,f=8

g=7,h=3,f=10

g=7,h=1,f=8

g=8,h=2,f=10

g=8,h=0,f=8

g=1,h=7,f=8

g=1,h=7,f=8

g=2,h=6,f=8


27

A* and BFS• BFS explores all nodes at a shorter distance

from the start (i.e. g value)

g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

28

A* and BFS• BFS explores all nodes at a shorter distance

from the start (i.e. g value)

g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

g=2,h=8,f=10

g=2,h=8,f=10

g=2,h=8,f=10

g=2,h=6,f=8

g=2,h=6,f=8

g=2,h=4,f=6

29

A* and BFS• BFS is A* using just the g value to choose

which item to select and expand

g=1,h=7,f=8

S

G

g=1,h=5,f=6

g=1,h=5,f=6

g=1,h=7,f=8

Closed List

Open List

g=2,h=8,f=10

g=2,h=8,f=10

g=2,h=8,f=10

g=2,h=6,f=8

g=2,h=6,f=8

g=2,h=4,f=6

30

Implementation Note• When the distance to a node/state/successor (i.e. g value) is

uniform, we can greedily add a state to the closed-list at the same time as we add it to the open-list

open_list.push(Start State)while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. Generate successors/neighbors of s, compute their f values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

open_list.push(Start State)Closed_list.push(Start State)while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 3a. if s = goal node then trace path back to start; STOP! 3b. Generate successors/neighbors of s, compute their f values, and add them to open_list and closed_list if they are not in the closed_list

Non-uniform g-values Uniform g-values

1 24 8 37 6 5

g=0,H=6

1 24 8 37 6 5

…

g=k,H=6

The first occurrence of a board has to be on the shortest path

to the solution

31

DATA STRUCTURE HIGHLIGHTHeaps

32

Heap Data Structure

• Can think of heap as a full binary tree with the property that every parent is less-than (if min-heap) or greater-than (if max-heap) both children– But no ordering property between children

• Minimum/Maximum value is always the top element

Min-Heap

7

9 18

19 35 14 10

28 39 36 43 16 25

33

Heap Operations

• Push: Add a new item to the heap and modify heap as necessary

• Pop: Remove min/max item and modify heap as necessary

• Top: Returns min/max• To create a heap from an unordered

array/vector takes O(n*log2n) time while push/pop takes O(log2n)

34

Push Heap

• Add item to first free location at bottom of tree

• Recursively promote it up until a parent is less than the current item

7

9 18

19 35 14 10

28 39 36 43 16 25

7

8 18

19 35 14 9

28 39 36 43 16 258 810

Push_Heap(8)1

2 3

4 5 6 7

8 9 10 11 12 13 14

35

Pop Heap

• Takes last (greatest) node puts it in the top location and then recursively swaps it for the smallest child until it is in its right place

7

9 18

19 35 14 10

28 39 36 43 16 25

9

10 18

19 35 14

7

28 39 36 43 16

Original

9 25

9 18

19 35 14 10

28 39 36 43 16

25

1

2 3

4 5 6 7

8 9 10 11 12 13

1

2 3

4 5 6 7

8 9 10 11 12

36

Practice

7

21

35 26 24

50294336

18

19

3928

1

2 3

4 5 6 7

89 10 11 12 13

7

9

35 14 10

36

18

19

3928

1

2 3

4 5 6 7

8 9 10

Push(11)

Pop()

4

8

35 26 24

36

17

19

3928

1

2 3

4 5 6

9 10

Pop()

7

7

21

35 26 24

50294336

18

19

3928

1

2 3

4 5 6 7

9 10 11 12 13

Push(23)

37

Array/Vector Storage for Heap• Binary tree that is full (i.e. only the lowest-level contains empty

locations and items added left to right) can be modeled as an array (let’s say it starts at index 1) where:

– Parent(i) = i/2– Left_child(p) = 2*p– Right_child(p) = 2*p + 1

7

9 18

19 35 14 10

28 39 36 43 16 17

em 7 18 9 19

0 1 2 3 4

35 14 10 28 39

5 6 7 8 9

36 43 16 17

10 11 12 13

parent(5) = 5/2 = 2

Left_child(5) = 2*5 = 10

Right_child(5) = 2*5+1 = 11

38

STL Priority Queue• Implements a max-heap by

default• Operations:

– push(new_item)– pop(): removes but does not

return top item– top() return top item (item at

back/end of the container)– size()– empty()

• http://www.cplusplus.com/reference/stl/priority_queue/push/

// priority_queue::push/pop #include <iostream> #include <queue>

using namespace std;

int main () { priority_queue<int> mypq; mypq.push(30); mypq.push(100); mypq.push(25); mypq.push(40); cout << "Popping out elements..."; while (!mypq.empty()) { cout<< " " << mypq.top(); mypq.pop(); } cout<< endl; return 0; }

Code here will print100 40 30 25

39

STL Priority Queue Template• Template that allows type of element, container class, and comparison

operation for ordering to be provided• First template parameter should be type of element stored• Second template parameter should be vector<type_of_elem>• Third template parameters should be comparison function object/class that

will define the order from first to last in the container

// priority_queue::push/pop #include <iostream> #include <queue>

using namespace std;

intmain () {priority_queue<int, vector<int>, greater<int>> mypq;mypq.push(30); mypq.push(100); mypq.push(25);cout<< "Popping out elements...";while (!mypq.empty()) { cout<< " " << mypq.top(); mypq.pop(); } } Code here will print

25, 30, 100

'greater' will yield a min-heap 'less' will yield a max-heap

40

STL Priority Queue Template• For user defined classes,

must implement operator<() for max-heap

or operator>() for min-heap

• Code here will pop in order:

– Jane– Charlie– Bill

// priority_queue::push/pop #include <iostream> #include <queue> #include <string>using namespace std;

class Item { public: int score; string name;

Item(int s, string n) { score = s; name = n;} bool operator>(const Item &lhs, const Item &rhs) const { if(lhs.score > rhs.score) return true; else return false; }};

int main () { priority_queue<Item, vector<Item>, greater<Item> > mypq; Item i1(25,”Bill”); mypq.push(i1); Item i2(5,”Jane”); mypq.push(i2); Item i3(10,”Charlie”); mypq.push(i3); cout<< "Popping out elements..."; while (!mypq.empty()) { cout<< " " << mypq.top().name; mypq.pop(); } }

1 ee 355 unit 14 a-star algorithm & heaps/priority queues mark redekopp

Documents

account h

goal12348765 h

search algorithm

fvalue state

y distance

lowest heuristic score

lowest fscoreheuristics

listsopen list