ELSEVIER Microprocessing and Microprogramming 40 (19941 57 64

Microprocessing and Microprogramming

Z Efficient message routing in -network

Hong Shen* School O[ Computing and Irtformation Technology, Gr(ffith Unil,ersity, Nathan. QLD 41 l I. du,wralia

Abstract

The ~[-network is a high performance interconnection network constructed by the recursive expansion method on the basis of the Petersen graph. As its cost (degree × diameter), a main performance measure for static interconnection networks, is very low in comparison with other known networks of the same size, a, ga~ becomes an attractive configuration for high performance interconnection networks. This paper describes how message routing in a , ~ network can be efficiently carried out. We first show a node addressing scheme for the network, and then present a simple and efficient routing algorithm for message routing in the network. Our algorithm realizes point-to-point message routing without the need of maintaining a routing table at each node, and has a time complexity of Oflog n,) for a ~[-network containing n~ = 10 ~+~ nodes, where r >~ O.

Key words: lnterconnection network; Cost; Performance: Recursive expansion: Message routing; Algorithm: Time complexity

1. Introduction

Interconnect ion network is a fundamental part in a mult iprocessor system [I , 3]. The ~',Z-network is a high performance interconnection network constructed by the recursive expansion method [4] on the basis of the Petersen graph [2]. It contains 10 r+l nodes for r~>0, and has a cost 2 4 r + 12 (degree 6, diameter 4r + 2) for 0 ~< r ~< 10 which is much lower than the torus and hypercube and is

comparable with the cube-connected-cycles of the same size [5]. The network possesses also other desirable features such as regular topology, high scalability.

Message routing is a central problem to the ap- plication of an interconnection network and is also a sensitive problem which must be considered by the network designer. A good (interconnection) net- work should facilitate message routing and make a simple and efficient routing possible. An efficient

*Previously with Department of Computer Science, A, bo Akademi University, Finland. Email: hong @ cit.gn.edu.au, fax: (07) 8755051.

0165-6074/94/$7.00 c~-) 1994 Elsevier Science B.V. All rights reserved. SSDI 0165-6074(93)E0021-M

58 H. Shen/ Microprocessing and Microprogramming 40 (1994) 57-64

routing algorithm for a given network will make better use of the network and increase the overall performance of the multiprocessor system built on the network.

In this paper, we will address the problem of message routing in the ~ - n e t w o r k . We will first propose a node addressing scheme for ~ , and then present a simple and efficient routing algorithm with time complexity O(logn,) for point-to-point message routing in a ~ - n e t w o r k of size 10 ~+~, r~>0.

2. The ~-network

Recursive expansion (RE for short) is a method for systematically constructing high-performance interconnection networks [4]. Based on two chosen small networks, unit Gu and frame G I , the RE method constructs G~+ 1 by replacing each node in a Gy-copy (i.e. a copy of Gy) with a Gi-copy and

each edge in the Gfcopy with a set of inter-unit connections for i = 0, 1 . . . . . where Go = Gu, until the desired size network is obtained.

The ~ - n e t w o r k is constructed by taking the Petersen graph, denoted by ~ , as both the unit and frame and applying the RE method to them [-5].

In ~ , the pivot-set sequence PSS(sk~ [4] for min- imizing the diameter of ~ in ~ (unit) is used. Let the nodes in ~ be addressed from 0 to 9 as shown in Fig. l(a). Pivot set PI + ti- 1) rood 10 in the unit, which accepts the new edges for the inter-unit connections during the ith-phase expansion, is defined by P l + ( i - 1 ) m o d l O = {node~i-l)modl0}, 1 ~< i ~ r, as depicted in Fig. 1 (a).

The ~ - n e t w o r k (r/> 0) is constructed by the following algorithm:

Algorithm ~ ( ~ , r). • Let ~ = ~. • Copy ~ into ~ , replace each node in ~ with

Z a ~0-copy (all copies are disjoint) and replace

" i

(a) P i v o t se t P i ={~Le.t_ I l , O < i < l l

PI

PI

(b)

Fig. 1. (a) Node addressing and pivot sets in ~; (b) configuration of s~ z.

H. Shen/ Microprocessing and Microprogramming 40 (1994) 57 64 59

each edge in ~ z with an inter-unit connection between two adjacent z . ~o-coples, thus get the first-phase expanded network ~Zl consisting of

Z a set of ~0-coples together with a set of inter-unit connections.

• A~ain copy ~ into ~z, replace each node in z

~2 with a ~ - c o p y (all copies are disjoint) and each edge with an inter-unit connection set be- tween two adjacent x . ~l-Coples, thus get the sec- ond-phase expanded network ~ consisting of

' v

a set of~@l-copies together with a set of inter-unit connections.

• Repeat the above expanding until ~ z is ob- tained: copy ~ into ~z , replace each node in J7'~ z with a ~ z 1-copy constructed in the (r - 1)th expansion (all copies are disjoint) and each edge with an inter-unit connection set between two adjacent ~ z l-copies.

The size (n~), degree (d~), diameter (k~) and cost (c,) is given by the following equations [5] for r ) l ( n o = 10, d o = 3 a n d k o = 2 b y ~ ) :

n~= 10 ~ +1 d r = 6 , k ~ = 4 r + 2 , c ~ = 2 4 r + 12.

As showing an example of ~ f , Fig. 1 (b) depicts z

the configuration of the ~ - n e t w o r k .

3. Node addressing in the ~-network

As ~ is constructed on the basis of the Petersen graph, we need, first of all, to find an appropriate node addressing scheme for the Petersen graph, such that a simple node addressing scheme for ~ , which will facilitate message routing in ~ , can be directly derived.

The Petersen graph can be virtually divided into two parts: the outer pentagon and the inner penta- gon. We index the nodes on the outer pentagon by even numbers: 0, 2, 4, 6, 8, and nodes on the inner pentagon by odd numbers: 1, 3, 5, 7, 9, as shown in Fig. l(a).

Thus the Petersen graph can be described by the following lemma whose correctness is trivial:

Lemma 1. Given 10 nodes indexed from 0 to 9, the Petersen graph can be constructed hy connecting node i to its three adjacent nodes, Al(i), A2(i) and A 3 (i), given by the following.formulae:

{ t l i+ l ) m o d l O ( f i is even,

A~(i )= i + 9 )mod lO if i is odd;

( i + 2 ) m o d l 0 (f i is even,

Aa(i)= ( i + 4 ) m o d l 0 (f i is odd;

( i + 8 ) m o d l O (f i is et~en, A3(i)= ( i + 6 ) m o d l 0 ([" i is odd.

If Ae(i) = i', we call that node i connects to node i' via edge e (number) outgoing from i, where 1 ~< e ~< 3 and 0 ~< i, i'~< 9. Thus each node has three (outgoing) edges numbered by 1, 2 and 3 con- necting to its three neighbors.

Now considering the ~Z-network, we propose the following addressing scheme on the basis of the addressing scheme for the Petersen graph.

The ~ - n e t w o r k uses the Petersen graph, ~ , as its frame and is constructed by r-phases of expan- sions of the ~Z-network for i - 0, 1 . . . . . r - - 1, where ~ z = ~ is the unit, and the ith-phase expan- sion is simply a replacement of each node in a frame-copy with a ,~i ~-copy and a replacement of each edge in the frame-copy with a set of inter- unit connections, 1 ~< i ~< r. If a ~ ~-copy is "map- ped' onto node j of the frame-copy in the ith-phase expansion, we say that all nodes in the ~ ~-copy have positionj (number) in ,~_,~, where 1 ~< i ~< r and 0 ~< j ~ 9. Thus each node in ~ z will have r posi- tions in the ~ z ~ z 1-copy . . . . . . ~ - c o p y contain- ing the node for the r phases of expansions, plus its own position (address) in the unit-copy .~' to which it belongs, which brings r + 1 positions in total. This shows that using a (r + l)-tuple (array), a , a~ ~ . . . . . ao), where ai is the position number of a node in the ith-phase expansion, we can

60 H. Shen/Microprocessing and Microprogramming 40 (1994) 5~64

one- to-one m a p all 10 r+l nodes in ~ z to the ( r + 1)-tuple addresses f rom (0, 0 . . . . ,0) to (9, 9 . . . . . 9), r ~> 1, and thus leads to the following lemma:

Lemma 2. The address of each node in ~ z is unique- ly given by (at, ar 1 . . . . . a0), where al is the position of the node in the ~ - c o p y containing it, that is, the address of the node in the frame-copy on which the ~ - 1-copy containing it is mapped, 1 <.% i <~ r, and ao is the address of the node in its unit-copy.

mainta ins a rout ing table which keeps the shor- test paths f rom it to each of the other nodes in ~ , and routes the message to a dest ination along a shortest pa th kept in this table. Since ~ has only 10 nodes and diameter 2, mainta in ing a routing table at each node in ~ is acceptable in practice. Nevertheless, we are going to devise a routing algo- r i thm without using any routing table at each node.

Facil i tated by the node addressing scheme, a s t ra ightforward a lgor i thm for message rout ing in

can be described as follows:

Algorithm R O U T - ~ ( i , i', msg). {*Send msg from node i to node i' in the Petersen graph ~.*}

s := i, t := i'; {*Set the source node s and destination node t.*} while s # t do

if s is even then if t = (s + 1) m o d 10 v (s + 5 )mod 10 v (s + 7 )mod 10

then Send msg via edge 1 to A l(s), s := A l(s); if t = (s + 2)rood 10 v (s + 4)rood 10 v (s + 3)rood 10

then Send msg via edge 2 to A2(s), s := Az(s); if t = (s + 8 )mod 10 v (s + 9)rood 10 v (s + 6)rood 10

then Send msg via edge 3 to A3(s), s := A3(s); if s is odd then

if t = (s + 9)rood 10 v (s + 1)mod 10 v (s + 7)rood 10 then Send msg via edge 1 to AI(S), s := AI(S);

if t = (s + 4 )mod 10 v (s + 3)rood 10 v (s + 8 )mod 10 then Send msg via edge 2 to A2(s), s := A2(s);

if t = (s + 6 )mod 10 v (s + 2 )mod 10 v (s + 5)rood 10 then Send msg via edge 3 to A3(s), s := A3(s)

end.

4. The routing algorithm

Based on the above node addressing scheme, a simple a lgor i thm to realize efficient message rout- ing in the ~ r z can be devised.

Similar to node addressing, as the basis of mess- age rout ing in ~ z , we first need to find an algo- r i thm for message rout ing in the Petersen graph.

A trivial way to realize message rout ing in ~ is with the help of a rout ing table: each node in

The correctness of the a lgor i thm can be clearly seen by listing all the shortest paths f rom node i via edge 1, 2 and 3 to all other nodes in the cases of i is even and odd respectively.

Since the d iameter of ~ is 2 and Algor i thm sends the message always by a shortest path, the

a lgor i thm runs in constant t ime (while-loop is ex- ecuted at most twice).

Now we consider message routing in ~ . Assume that a message, msg, is to be sent from source node

H. Shen/Microprocessing and Microprogramming 40 (1994) 57 64 61

(it, ir 1 . . . . . io) to destination node (i'. i; 1 . . . . . i~), where O<<.ik, i'k<~9, O < ~ k < ~ r and r>~0. The whole routing process can be divided into three steps as follows:

Step l (Intra-unit routin9). If the source and the destination, node ( i , , i r -~ . . . . . io) and node (i'r, i'r_ 1 . . . . . i~), belong to the same H-copy,

then rout the ms(/ by Algorithm R O U T - 3 ~. Otherwise find the smallest m, 1 ~< m ~< r, such that the source and destination reside in two different ~ z _ ~ - c o p i e s of the same 3 ~ - c o p y which is called the most significant subnetwork for routing ms9 from the source to the destina- tion [4], that is, find the smallest m such that ik = i'k for m < k ~< r and i,~ ¢ i~,; and then route

. s • -

i

s s

S • l

i

s "

s: Source node address.

t: Destination node acktrcss.

(') ~ - I :The ~--I-copy ~ g node s. (t) ~ 1

(0 ~ -1 : The ~--1 -copy c, oraair~g node t.

Pm: Pivot set used for the m-th phase inter-unit connection.

Fig. 2. Message routing in 3~.

62 H. S h e n / M i c r o p r o c e s s i n g a n d M i c r o p r o g r a m m i n g 40 ( 1 9 9 4 ) 57~54

msg from (i~,i~ ~ . . . . . i0) to (i,,i~ a . . . . . i ~ , ( m - 1)modlO), the node in pivot set Pl+(m-1)modlO, in the same ~ - c o p y (unit). For simplicity and without loss of generality, we as- sume that m ~< 10, so that ( m - 1) m o d l O = m - 1.

S tep 3 ( l n t r a - : ~ ~ rout ing) . Route m s g from node (i . . . . . . ira+l, i~,, ira-1 . . . . . ia, m - 1) to the des-

. . , " . . . . . - 1), tination, ( i , , . i,, + 1, l m, l m - 1 . . . . , t o, m where ik = i'k for m < k ~< r. This can be com- pleted by repeating the above procedure at most m - 1 times.

S t e p 2 ( I n t e r - : ~ 1routing). Route msg from node ( i , , i , - 1 . . . . . i l , m - 1 ) to node (i . . . . . . i,,+~, i~,, i,,_ 1 . . . . . il, m - 1). Since all : ~ ~-copies are connected in the topo logy of the frame ( : ) , and each pair of corresponding units ( : ) in two adjacent ~ 1-copies are connected by an inter- unit connect ion at the node, ( . . . . m - 1 ) , in pivot set P,, of each unit, there is obviously

Figure 2 demonstrates how message routing in : ' ~ is realized as the above three steps.

As Step 1 routes the message in the unit and Step 2 routes the message in the frame (each node is replaced by a : ~ _ ~-copy), and both the unit and frame are : , these two steps can thus be accomplished by simply calling Algorithm R O U T - : . The following algorithm realizes message routing in : '~ , for r/> 0.

Algori thm R O U T - : ~ ((it, i r a . . . . . i0), ( i : , r :_ a . . . . . i~), msg). {* Send msg f r o m source node (it, i t -1 . . . . . io) to de s t i na t i on m o d e (i: , i: 1 . . . . . i~)

in ~ z , w h e r e 0 <~ ik, i'k <, 9, 0 <~ k <~ r, r >~ 0.*} s := (it, it_ 1 . . . . . io), t := (i'r, i',_ l . . . . . i~); {* E q u i v a l e n t to Sk := ik and tk := i~, for 0 <<, k <~ r, w h e r e

s = (sr, sr 1 . . . . . So) a n d t = (t , , t r -1 . . . . . to).*} m : = r; while m ~> 0

if (s., ~ t , .) /x (m = 0) then call R O U T - ~ ( s , t, msg);

if (s., ~ t.,) /x (m > 0) then t : = (i . . . . . . ia, m - 1); {*ik = i ' k for m < k <<, r.*} call R O U T - ~ (s, t, msg); { * l n t r a - u n i t rou t ing .*}

s : = ( i . . . . . . i l , m - 1); t : = ( i . . . . . . i , , + x , i ' , , , i , , - x . . . . . i a , m - 1); call R O U T - ~ (s, t, msg); { * I n t e r - ~ Z ~ - 1 rou t ing .*} s := (i . . . . . . ira+a, i~,, im-x . . . . . il, m - 1); t : = (i',, i ',-a . . . . . i~); {*ik = i 'k for m < k <~ r.*}

m : = m - - 1 end.

a path from node ( i , , i r 1 . . . . . i x , m - 1 ) to (i . . . . . . i , . + l , i ' , . , i , , 1 . . . . . i x , m - 1) with length at most 2 (the diameter of the frame).

It is trivial that the time complexity of the above algori thm is O(r) = O(log nr), since the while-loop is executed at most r + 1 times and Algori thm

H. Shen/' Microprocessing and Microproeramming 40 ~1994 ) 57 64 63

ROUT<a/takes only constant time. Thus we have the following theorem:

Theorem. Message routing between any pair of nodes in ~ of size n~ can be accomplished in O(log n~) time, where n~ = 10 ~+l, r >~ 0.

5. Concluding remarks

The ~ - n e t w o r k is a high performance intercon- nection network constructed by the recursive ex- pansion method on the basis of the Petersen graph. As this network has the features of low cost, regular topology and high scalability, it provides a promis- ing configuration for multiprocessor systems. In this paper, we have shown that, based on an appro- priate node addressing scheme, message routing in the ~ - n e t w o r k can be simply and efficiently realiz- ed, thus proved another feature of this network. The proposed routing algorithm has a time com- plexity of O(log n~) for a ~ar~-network of size n~= 10' + l wherer>~0.

Our algorithm is for the ~ - n e t w o r k using pivot set sequence PSS¢k~ for minimizing the diameter [5]. As an alternative, , ~ can also be constructed by using pivot set sequence PPS¢d~ for the purpose of minimizing the degree. Without increasing the or- der of the time complexity as shown in [4], our algorithm can be easily modified to suit the latter.

I n ~ ~ using pivot set sequence PSS~k~, since inter- connections during the ith-phase expansion are as- signed to node¢~-l~mod 10(l ~ i~< r), all inter-unit communication between two ~_~-copies in ~ must go through node,_ ~1 mod 10. This may re- sult in heavy traffic at node,_ l~moa ~0 of unit-copy o~ in one ~ 1-copy when all the 10 nodes of

send messages simultaneously to a unit-copy in another ~ ~-copy. One way to solve this problem is to include 3 nodes in each pivot set: P~ ~ti 1)rood 1 0 = {node(/ l ) m o d 1 0 , nodeimod 1 0 ,

node.~ I ) m o d 1 0 "~ 1 ~< i <~ r, each node accepting J~

only one inter-unit connection for the correspond- ing phase expansion, which can be viewed as a special case of using pivot set sequence PSSI~ ~ and will complicate the routing algorithm [4].

Moreover, since node, ~mod 10 in ~ is the only pivot node that controls the communication to outside in ~ ( 1 ~< i ~< r) for all nodes in ,~, a failed node of ~ in one , ~ 1-copy will isolate all the remaining 9 nodes of ~ from the nodes in all other ~a~ l-copies. This can be remedied by the same way as above: including three nodes in each pivot set so that when one node is failed the communication to

~ Z other ;a~i., 1-copies for the remaining nine nodes in the same ~ can be realised via other nodes in the pivot set. As our routing algorithm employs the procedure of shortest-path routing in :# (ROUT- ~), another problem may arise when a failed node appears: any lnter-2/~ ~ communication whose shortest path goes through the failed node will be broken. This can be overcome by simply modifying ROUT-a5 a to work on ~ with broken edges such that an alternative path can be built between any pair of nodes in ~ if the shortest path is broken due to failed node(s).

Acknowledgments

The author wishes to thank anonymous referees for their valuable comments and suggestions, espe- cially for their suggestions of adding discussions on network traffic and fault-tolerance.

References

[1] G.S. Almasi and A. Gottlieb, Highly Parallel Computinq (Benjamin/Cummings, Menlo Park, CA, 1988}.

[2] L.W. Beineke and R.J. Wilson, Selected Topics in Graph Theory (2) [Academic Press, New York, 1983).

[3] T. Feng, A survey of interconnection networks, Computer 14(12} (1981} 12 27.

64 H. Shen / Microprocessing and Microprogramming 40 (1994) 57~64

[4] H. Shen and R.-J. Back, Construction of large-size intercon- nection networks with high performance, Microprocessing and Microprogramming 35 (1992) 545-554.

[5] H. Shen, A high performance interconnection network for multiprocessor systems, to appear in Parallel Computing.

Hong Shen is currently a Lecturer at the School of Computing and Information Technology, Griffith University, Austra- lia. He received the B.S. degree from Beij- ing University of Iron & Steel Techno- logy, China in 1982, M.S. degree from the University of Science and Technology of China in 1987, Ph.Lic and Ph.D. degrees from ,~bo Akademi University, Finland, in 1990 and 1991 respectively, all in Computer Science. He became an Assist- ant Professor at. the Department of Com- puter Science, Abo Akademi University,

m 1991, and joined Griflith University in 1992. His main re- search interests include parallel algorithms, parallel and distrib- uted computing and parallel computer architectures. He has over 30 technical publications in the above areas.

Dr. Shen is a member of the Association for Computing Machinery and the IEEE Computer Society.