Slide 1

A Survey of Techniques for Designing I/O-Efficient AlgorithmS.Fahimeh MoosaviFall 1389Basic TechniquesScanning -N/B I/Os while linear scanning the whole array.

Sorting -O((N/B)logM/B N/B) I/Os.

2Simulation of Parallel Algorithms in External Memory3PRAM [Parallel Random Access Machine]

p processors, each with local memoryEach processor has unique id in range 0-(p-1)Shared memory reads and writes

4

At each unit of time, a processor is either active or idle (depending on id)At each time step, all processors may execute different instructions on different data. Note: any pair of processor Pi Pj can communicate in constant time! Pi writes the message in cell x at time t Pj reads the message in cell x at time t+1

Measures of performance: 1. Running time.2. Amount of work it performs.

5PRAM-algorithm A :Uses: N processorO(N) spaceRun time: O(T(N))

Assumption: every computation step of a processor consists of a constant number of write/read accesses to shared memory.

6Simulation one step of algorithm A Scan the list of processor context (read accesses read requests).Sort the resulting list of read request by the memory locations they access.Scan the sorted list of read request and memory representation.Sort the list of read request again, by the issuing processor.Scan the sorted list of read request and the list of processor context to transfer the requested operands to each processor.

7O(1) scans of list of processor context.O(1) scans of the representation of the shared memory .A constant number of times scanning and sorting the list of read/write request.All this lists have size O(N).

Consequence: Simulation one step of algorithm A takes O(sort(N)) I/Os.

Theorem 3.2. A PRAM algorithm that uses N processors and O(N) space and runs in time T(N) can be simulated in O(T(N).sort(N)) I/Os.8Time-Forward Processing9Evaluating a DAG GL: an assignment of labels L(v) to the vertices of G.

Goal: compute another labelling S of the vertices of G so that for every vertex vG, S(v) be computed from L(v) and S(u1), ... , S(uk) (u1, , uk: in-neighbors of v).

10Expression-tree evaluationInput: a binary tree T whose leaves store real number and internal vertices store binary operation.If v is leaf then val(v)=number stored at v.If v is internal vertex with label o, left child x, right child y then val(v) = val(x) o val(y).

11

Evaluate a DAG G I/O-efficientlyTwo assumption: the vertices of G have to be stored in topologically sorted order.

label S(v) has to computable from labels L(v) and S(u1),..., S(uk) in O(sort(k)) I/Os.12 Insertion and deletemin operations on Q (priority queue) be performed in O((1/B).log(|E|/B)M/B).

Total number of priority queue operations: O(|E|) (Every edge inserted into and deleted from Q exactly once).

Consequenc: all updates of priority queue takes O(sort(|E|)) I/Os.

13Note: Vertex set + adjacency lists scanned: O(scan(|V| + |E|)) I/Os.Computation labels S(v) from L(v) and S(u1),..., S(uk), for all vG, takes O(sort(|E|)).

Theorem 3.3. given a DAG G=(V,E) whose vertices are stored in topologically sorted order, graph G can be evaluated in O(sort(|V|+|E|)) I/Os, provided that the computation of the label of every vertex vG can be carried out in O(sort(deg-(v))) I/Os, where deg-(v) is the in-degree of vertex v.14Greedy Graph Algorithm 15A vertex labelling algorithm A call:

single-pass: if it compute the desired labelling of vertices of the graph by visiting every vertex exactly once.

local: if label L(v) can be computed in O(sort(k)) I/Os from labels L(u1),...,L(uk), where u1,...,uk the neighbors of v whose labels are computed before L(v).

Presortable: if there is an algorithm that take O(sort(|V|+|E|)) I/Os to compute an order of the vertices of the graph so that A produces a correct result if it visits the vertices of the graph in this order.16Main Problems at Make Algorithm A I/O-efficient determine an order in which algorithm A should visit the vertices of graph.

devise a mechanism that provides every vertex v with the labels of its previously visited neighbors.17Theorem 3.4. Every graph problem P that can be solved by a presortable local single-pass vertex labelling algorithm can be solved in O(sort(|V|+(|E|)) I/Os.

Proof: A: presortable local single-pass vertex labelling algorithm.L: labelling of vertices of a graph G=(V,E).A: an algorithm that takes O(sort(|V|+(E)) I/Os to compute an order of the vertices of G (numbering the vertices)G : a derived DAG from G by directing every edge from the with smaller number to the vertex with larger number.

Hence, labelling L can be computed using time-forward processing.18Computing a Maximal Independent SetIn internal memory: Process the vertices in an arbitrary order. When a vertex vV is visited, add it to S if none of its neighbors is in S.

Translate into a labelling problem:Xs :V{0,1}If vS then Xs(v)=1,If vS then Xs(v)=0.

Theorem 3.5. Given an undirected graph G=(V,E), a maximal independent set of G can be found in O(sort(|V|+(E)) I/Os and linear space.

19Coloring Graphs of Bounded DegreeIn internal memory: Process the vertices in an arbitrary order. When a vertex vV is visited, assign a color c(v) {1, , +1} to vertex v that has not been assigned to any neighbor of v.

Theorem 3.6. Given an undirected graph G=(V,E) whose vertices have degree at most , a (+1)-coloring of G can be found in O(sort(|V|+(|E|)) I/Os and linear space.

20210123456-choose an arbitrary order.

Application: coloring Graphs of bounded degreeG with degree 3 to be colored with 3+1 colors

22Application: coloring Graphs of bounded degreeG with degree 3 to be colored with 3+1 colors-choose an arbitrary order.-assign the color to the vertex that has not been chosen by its neighbors.012345623-choose an arbitrary order.-assign the color to the vertex that has not been chosen by its neighbors. To choose a color for a vertex, order all colors of its neighbors and assign the first unused color to it.023456Application: coloring Graphs of bounded degreeG with degree 3 to be colored with 3+1 colors240123456Application: coloring Graphs of bounded degreeG with degree 3 to be colored with 3+1 colors-choose an arbitrary order.-assign the color to the vertex that has not been chosen by its neighbors. To choose a color for a vertex, order all colors of its neighbors and assign the first unused color to it.250123456-choose an arbitrary order.-assign the color to the vertex that has not been chosen by its neighbors. To choose a color for a vertex, order all colors of its neighbors and assign the first unused color to it.Application: coloring Graphs of bounded degreeG with degree 3 to be colored with 3+1 colors260123456-choose an arbitrary order.-assign the color to the vertex that has not been chosen by its neighbors. To choose a color for a vertex, order all colors of its neighbors and assign the first unused color to it.Application: coloring Graphs of bounded degreeG with degree 3 to be colored with 3+1 colorsList Ranking and the Euler Tour Technique27List RankingList ranking problem: computing distance from head of linked list L to xi, for every vertex of L (the number of edges on the path from head of L to xi).

Solving in internal memory:Starting at the head of the list, follow successor pointers and number the vertices of the list from 0 to N-1 in the order they are visited.

28Generalization of the List Ranking(prefix product)Input:: {x1 ,,xN} X : XX XOutput:(xi) For each vertex xi of L such that(x(1))=(x(1))(x(i))= (x(i-1)) (x(i)) (1< i N)Where =[1,N] [1,N] is a permutation,And x(1) is the head of L,And succ(x(i))=x(i+1).

29I/O complexity:O(sort(N))

ExampleList ranking:

Generalization:0123453152313491114153030Internal memory algorithm is not I/O-efficient123456789101112131415161591326101434781112151615913261014347811121516159132610143478111215161256910131434781112151612569101314347811121516The internal memory algorithm spends W(N) I/Os in the worst case.3131An Efficient List Ranking AlgorithmFind an independent set I of L so that |I|=(N).Remove elements of I from L. for every element x I with predecessor y and successor z in L let succ(y)=z. The label of x multiplied with the label of z, and result assigned to z.Apply this algorithm recursively to the compressed list.Compute the ranks of elements in I by multiplying their labels with the ranks of their predecessors in L.

32Example333152313174341115315231349111415I/O-ComplexityEvery step, except the recursive invocation, takes O(sort(N)) I/Os.

Total I/O-complexity: (N)=(cN)+O(sort(N)) (0