martinet z 1994
TRANSCRIPT
-
7/25/2019 Martinet z 1994
1/16
Pergamon
Neural Networks, Vol. 7, No. 3, pp. 507-522, 1994
Copyright 1994 ElsevierScience Ltd
Printed in the USA. All rights reserved
0893-6080/94 6.00 + .00
C O N T R I B U T E D R T I C L E
Topology Representing etworks
THOMA S MARTIN ETZ 1'2 AND K_LAUS SCHULTEN l
tUniversityof Illinoisand 2Siemens AG
Received 4 March
1993;
accepted
20
September
1993 )
Abstract--A
Hebbian adap ta t ion ru le w i th w inner - take -a l l l i ke compe t i t ion i s in t roduced . I t i s shown tha t th i s
compe t i t i ve Hebb ian ru le for ms so -ca l led De lau nay t r iangulat ions, wh ich p lay an imp or tan t ro le in computa t iona l
geom etry o r e f f ic ien tly so lv ing prox imi ty prob lems . Given a se t o f neura l un i t s i, i = 1 . . . . N , the synap t ic we igh ts
o f wh ich can be in terpre ted as po in ters w i, i = 1 . . . . N in n , the compe t i t i ve Hebb ian ru le leads to a connec tiv i ty
s t ruc ture be tween the un i t s i tha t corresponds to the De lau nay t r iangu la t ion o f the se t o f po in ters w i . Such compe t i t i ve
Heb bian rule develops connec tions C o > O) betw een neura l units i , j with neighbo ring receptive f ie lds Voronoi
po lygons) V i , V j , whereas be tween a l l o ther un i t s i, j n o connec t ions evolve Ci j = 0 ) . Co mb ined wi th a procedure
tha t d i s t r ibu tes the po in ters w i over a g iven f ea tu re m ani fo ld M, for ex ample , a subm ani fo ld M C n , the compe t i t i ve
Hebbian ru le prov ides a nove l approach to the prob lem o f cons t ruct ing topo logy preserv ing f ea tu re map s and rep-
resenting intricately s tructure d manifolds. Th e compe ti t ive Hebbian ru le connects only neu ral units , the receptive
f i e lds Vorono i po lygons) V i , Vy o f wh ich are ad jacen t on the g iven mani fo ld M . Th is l eads to a connec t iv i ty s t ruc ture
tha t de f ines a per fec t l y topo logy preserv ing ma p and for ms a d i screte, pa th preserv ing represen ta t ion o f M, a l so in
cases where M has an intricate topology. This m ak es this novel approach part icularly useful in al l applications
where ne ighborhood re la t ions have to be exp lo i t ed or the shap e and topo logy o f subma ni fo lds have to be taken in to
account.
Keywords--Proximity problems, Delaunay triangulation, Vbronoi polyhedron, Hebb rule, Winner-take-all com-
petition, Topologypreserving feature map, Topology epresentation, Path preservation, Path planning.
1. DELAUNAY TRIANGULATION AND
PROXIMITY PRO LEMS
In a number of information processing tasks the prob-
lem that has to be solved efficiently can be reduced to
a geometric problem that deals with the proximi ty of
points in a metric space. The most promine nt example
of such a
p r o x i m i t y p r o b l e m
is the
n e a r e s t - n e i g h b o r
or,
more generally, the
k - n e a r e s t - n e i g h b o r s e a r c h :
given N
points in a metric space, which is (are) the nearest (k
nearest) neighbor (s) to a new given query point (Duda
& Hart 1973). This
b e s t m a t c h r e t r ie v a l
has to be per-
formed in classification and in terpolation tasks with
applications in areas ranging from speech and image
processing over robotics to efficient storage and transfer
Acknowledgements: We than k Benoi t Noel and Phi lippe Dalger
for pointing out th e connection between the competitive Hebb rule
and second-order Voronoi polyhedra and for formulating Theorem
1. This work has been support ed by the Carver Charitabl e Trust, the
BMFT under Grant No. 01IN102A7, and by a fellowship of the
olkswagen
Foundation to T.M.
Requests for reprints should be sent t o Klaus Schulten, Beckman-
Institute and Depart ment of Physics, University of Illinois at Urb ana-
Champaign, 405 North Mathews Ave., Urbana, IL 61801.
5 7
of data (Makhoul, Roucos, & Gish, 1985; Kohonen,
M/ikisara, & Saram~iki, 1984; Naylor & Li, 1988; Gray,
1984; Nasrabadi & King, 1988; Nasrabadi & Feng,
1988; Ritter & Schulten, 1986). Another example of
a proximity problem is the construction o f the
E u c l i d -
e a n m i n i m u m s p a n n i n g tr ee :
given Npoi nts in a metric
space, what is the graph of minimum total length whose
vertices are the given points (Kruskal, 1956; Prim,
1957; Dijkstra, 1959). Constructing the Euclidean
minimum spanning tree is a common task in appli-
cations requir ing optimally designed networks, for ex-
ample, communication systems that have minimal in-
terconnection length. Other applications of the Euclid-
ean mi nimum spanning tree are in clustering (Gower
& Ross, 1969; Zahn , 1971 ), patt ern recognition (Os-
teen & Lin, 1974), and in searching for (approximate )
solutions of the
t r av e l in g s a l e s m a n p r o b l e m R o s e n -
krantz, Stearns, & Lewis, 1974). A third example is
the
t r i a n g u l a t i o n p r o b l e m :
given N points in a plane,
connect them by nonintersecting straight lines so that
every region inside the convex hull of the N points is a
triangle. The triangulation problem occurs in the finite-
element method (Strang & Fix, 1973) and in function
interpolation on the basis of N data points, where the
-
7/25/2019 Martinet z 1994
2/16
508
7 .
Martinetz and K Schulten
f u n c t i o n s u r f a c e is a p p r o x i m a t e d b y a n e t w o r k o f tr i -
angu la r face ts (Georg e , 1971 ) . A com p rehens iv e ove r -
v i e w o f t h e a b o v e a n d f u r t h e r p r o x i m i t y p r o b l e m s c a n
b e f o u n d i n P r e p a r a t a a n d S h a m o s ( 1 98 5 ) .
A p o w e r f u l s t r u c t u r e f r o m c o m p u t a t i o n a l g e o m e t r y
that solves or , a t leas t , yie lds a s tart ing point for eff i -
c ien t ly so lv ing the above and o the r p rox im i ty p rob lem s
is the so-cal led
V o r o n o i d i a g r a m
and i t s dua l , the
D e -
l a u n a y t r ia n g u l a t i o n .
The Voro noi d iagra m CVs of a s e t
S = { wl . . . . . wx } o f po in t s wi E ~D i s g iven by N D-
d i m e n s i o n a l p o l y h e d r a , t h e V o r o n o i p o l y h e d r a V i ,
which a re de f ined a s fo l lows : the Voronoi po lyhedron
l~ of a po in t w~ E S i s g iven by the s e t o f po in t s v E
,~D tha t a re c lose r to w, tha n to a ny o the r wj @ S , j #
i , that is ,
J , = { v ~ ~ 1 I l v - w , II - < I t v -
w j l l
= 1 . . . . N }
i = 1 . . . . N . ( 1 )
The V oronoi po lyh edra p rov ide a com ple te pa r t i t ion ing
of the em bed ding space ~D, tha t i s, ~D = U U_l
V i .
In
t h e c o n t e x t o f
w i n n e r - t a k e - a l l
t y p e n e u r a l n e t w o r k s th e
Voro noi po lyhe dro n V~ is often cal led the recept ive f ie ld
of neura l u n i t i , w i th wi be ing the syna p t ic we igh t vec tor
of th i s neura l u n i t . F or a l l inp u t v ec tors v E Vi the
e l e m e n t i i s t h e b e s t m a t c h i n g u n i t o f t h e n e t w o r k a n d
i s e m p l o y e d t o r e p r e s e n t ( e n c o d e ) t h e se i n p u t p a t t e r n s
v . F igure l a i l lus t ra te s the de f in i t ion o f Vi by show ing
the V oronoi d iagram , tha t i s, a ll the Vo ronoi po lygons ,
o f a s am p le s e t o f po in t s in a p lane . In ~2 the V oronoi
d i a g r a m f o r m s a g r a p h , t h e v e r ti c e s o f w h i c h a r e g i v e n
by all the v ~ 9~2 t h a t a r e s i m u l t a n e o u s l y e l e m e n t o f
th ree Voronoi po lygons Vi . The edges a re g iven by a l l
t h e v E ~ 2 t h a t a r e s i m u l t a n e o u s l y e l e m e n t o f t w o V o -
r o n o i p o l y g o n s . T h e V o r o n o i d i a g r a m a s t h e s e t o f
a l l
V o r o n o i p o l y h e d r a c o n t a i n s a l l t h e p r o x i m i t y i n f o r -
m a t i o n a b o u t t h e s e t o f p o i n t s S .
T h e s t r a ig h t l i n e d u a l o f th e V o r o n o i d i a g r a m i s t h e
so-cal led
D e l a u n a y t r i a n g u l a t i o n
(De la unay , 1934 ) . In
a p l a n e , t h e D e l a u n a y t r i a n g u l a t i o n i s o b t a i n e d i f w e
con nec t a l l pa i r s wi , wj E S , the Voro noi po lygon s V~,
I ~ o f w h i c h s h a r e a n e d g e. I n g e n e r a l , f o r e m b e d d i n g
s p a ce s ~ D o f a r b i t r a r y d i m e n s i o n D , t h e D e l a u n a y
t r i angu la t ion ~0s o f a s e t S = { Wl . . . . . WN } of po in t s
(a)
(b)
F I G U R E 1 . ( a ) T h e V o r o n o i d i a g r a m o f a s e t o f p o i n t s. ( b ) T h e
D e l a u n a y t r ia n g u l a t i o n t h a t c o r r e s p o n d s t o t h e V o r o n o i d i a g r a m
i n ( a ) .
Wi ~ ~ j ~ D i s de f ined by the g raph who se ve r t i ces a re the
w i a n d w h o s e
a d j a c e n c y m a t r i x A A i j ~ { O 1 } i j =
1 . . . N ca r r i e s the va lue one i f f Vi fq ~ 4 : ~ . Two
ver ti ces wi , wj a re connec ted by an edge i f f the i r Voronoi
po ly hed ra V~, ~ a re ad jacen t . An i l lus t ra t ion o f the
p lana r case is g iven in F igure 1b , w h e r e t h e D e l a u n a y
t r i a n g u l a t i o n t h a t c o r r e s p o n d s t o t h e V o r o n o i d i a g r a m
of F igure l a i s shown. In a p lane , each edge o f the
D e l a u n a y t r i a n g u l a t i o n c o r r e s p o n d s t o a n e d g e o f t h e
V o r o n o i d i a g r a m , a n d t h e t w o c o r r e s p o n d i n g e d g e s a re
a lways pe rpe ndicu la r to each o the r .
A n u m b e r o f t h e o r e m s a b o u t p r o p e r t i e s o f t h e V o -
r o n o i d i a g r a m a n d t h e D e l a u n a y t r i a n g u l a t i o n a r e
kn ow n (see , e .g. , Prep arata & Sha mo s, 1 985). However,
m o s t o f t h e m a r e v a l i d o r a t le a s t c a n b e p r o v e n o n l y
in the p lana r case , fo r D = 2 . In h ighe r -d im ens iona l
em bed ding spaces ~D, D > 2 , on ly l i t tl e i s kn ow n so
fa r . One reason i s tha t on ly fo r D = 2 the Voronoi
d i a g r a m a n d t h e D e l a u n a y t r i a n g u l a t i o n a r e p l a n a r
g r a p h s a n d , t h e r e f o r e , o n l y f o r D = 2 E u l e r ' s f o r m u l a
can be ap p l ied (Bol lob~s , 1979 ) . Eu le r ' s fo rm ula p ro-
v i d e s t h e i m p o r t a n t r e s u l t t h a t i n t h e p l a n a r c a s e t h e
n u m b e r o f ed g e s o f th e V o r o n o i d i a g r a m a s w e ll a s o f
t h e D e l a u n a y t r i a n g u l a t i o n d o es n o t ex c e e d 3 N - 6
a n d , h e n c e , t h e V o r o n o i d i a g r a m a n d t h e D e l a u n a y
t r i a n g u l a t i o n c a n b e s t o r e d i n o n l y l i n e a r s p a c e ( l i n e a r
in the nu m b er o f ver t i ces N) . F ur the r , due to th i s re su l t ,
b o t h s t r u c t u r e s a r e t r a n s f o r m a b l e i n t o e a c h o t h e r i n
on ly l inea r t im e .
C o n s t r u c t i n g t h e D e l a u n a y t r i a n g u l a t i o n i n a p r e -
proces s ing s tage y ie lds a s t a r t ing po in t fo r e f f i c ien t ly
s o l v in g p r o x i m i t y p r o b l e m s . I t c a n b e s h o w n t h a t i f t h e
D e l a u n a y t r i a n g u l a t i o n o f a g i v en s e t o f p o i n t s S i s
k n o w n , t h e a b o v e s t a t ed a n d o t h e r p r o x i m i t y p r o b l e m s
c a n b e s o l v e d w i t h a t m o s t l i n e a r l y i n c r e a s i n g c o m -
p u t a t i o n a l e f f o r t . T h e t r i a n g u l a t i o n p r o b l e m , f o r e x -
am ple , i s obv ious ly a l ready so lved wi th the cons t ru c t ion
o f th e D e l a u n a y t r i a n g u la t i o n a n d d o e s n o t n e e d f u r t h e r
c o m p u t a t i o n , l T h e c o m p u t a t i o n t i m e n e e d e d f o r f i n d -
i n g t h e E u c l i d e a n m i n i m u m s p a n n i n g t r e e is r e d u c e d
s ign i f i can t ly because the edges o f the E uc l idea n m in i -
m u m s p a n n i n g t r e e a r e a s u b s e t o f th e e d g e s o f t h e
D e l a u n a y t r ia n g u l a t i o n ( S h a m o s , 1 97 8 ) . K n o w i n g th e
D e l a u n a y t r i a n g u l a t i o n , i t o n l y r e q u i re s O ( N ) i n s t e a d
o f O ( N l o g N ) t i m e f o r i ts c o n s t r u c t i o n . T h e n e a r e s t-
n e i g h b o r a n d k - n e a r e s t - n e i g h b o r s e a r c h c a n b e p e r -
f o r m e d i n o n l y O (l o g N ) i n s t e a d o f O ( N ) t i m e b y e x -
p l o i ti n g t h e D e l a u n a y t r i a n g u l a t i o n ( K n u t h , 1 9 7 3 ) .
I n t h e f o l lo w i n g s e c t io n w e w i l l i n t r o d u c e a H e b b i a n
a d a p t a t i o n r u l e t h a t , w i t h i n a n e t w o r k o f n e u r a l u n i t s ,
y i e l d s i n t e r n e u r a l c o n n e c t i o n s c o r r e s p o n d i n g t o t h e
l S o l v i n g t h e t r i a n g u l a t i o n p r o b l e m b y m e a n s o f t h e D e l a u n a y
t r i a n g u l a t i o n h a s a d v a n t a g e s p a r t ic u l a r l y i n f u n c t i o n i n t e r p o l a t i o n .
W h e n a f u n c t i o n i s a p p r o x i m a t e d p i e c e w i s e - l i n e a r o v e r t h e f a c e t s o f
a t r i a n g u l a t i o n , t h e D e l a u n a y t r i a n g u l a t i o n y i e l d s a s m a l l e r w o r s t c a s e
e r r o r t h a n a n y o t he r t ri a n g u la t i o n ( O m o h u n d r o , 1 9 9 0 ) .
-
7/25/2019 Martinet z 1994
3/16
Topology Representing Networks 509
D e l a u n a y t r i a n g u l a t i o n . I n S e c t i o n 3 w e w i l l e x p l a i n
t h e c o n c e p t o f t o p o l o g y p r e se r v i n g f e a t u r e m a p s . W e
w i l l i n t r o d u c e t h e t e r m
m a s k e d V o ro n oi p o l y h e d r o n
a n d s h o w h o w t h e s e m a s k e d V o r o n o i p o l y h e d r a p r o v id e
a r i g o r o u s d e f i n i t i o n f o r th e t e r m s
n e i g h b o r h o o d a n d
t o p o l o g y p r e s e r v i n g m a p s .
T h i s l e a d s t o
i n d u c e d D e -
l a u n a y t r i a n g u l a t i o n s
a s t h e g r a p h i c a l s t r u c t u r e t h a t
p r o v i d e s p e r f e c t t o p o l o g y p r e s e r v a t i o n . W e w i l l s h o w
t h a t t h e H e b b i a n a d a p t a t i o n r u l e o f t h e f o l l o w i n g s ec -
t i o n f o r m s i n d u c e d D e l a u n a y t r i a n g u l a t io n s a n d , h e n c e ,
i s a b le t o f o r m p e r f e c t ly t o p o l o g y p r e se r v i n g m a p s a l s o
o f t o p o l o g i c a l l y i n t r i c a t e l y s t r u c t u r e d m a n i f o l d s , f o r
e x a m p l e , o f m a n i fo l d s t h a t a r e d i s c o n n e c t e d a n d / o r
p iecewise 0 -, 1 -, 2- , e t c ., d im en s iona l . In S ec t ion 4 w e
w i l l s h o w t h a t t h e i n d u c e d D e l a u n a y t r i a n g u l a t i o n s
p r o v i d e d b y t h e H e b b i a n a d a p t a t i o n r u l e y i e l d p a t h
p r e s e r v in g r e p r e s e n t a t i o n s a n d a l l o w o n e t o d e s c r ib e ,
c la s si fy , a n d p l a n p a t h s o n m a n i f o l d s . T h e s e i n d u c e d
De launa y t r i angula t ions , which re f l ec t the topo log y and ,
a t t h e s a m e t i m e , f o r m p e r f e c t ly t o p o l o g y p re s e r v i n g
m a p s a n d p a t h p r e s e r v i n g r e p r e s e n t a t io n s o f g i v en
m ani fo lds , we ca l l t o p o l o g y r e p r e s e n t i n g n e t w o r k s
( T R N ) . I n t h e l a st se c t i o n w e w i ll p r e s e n t a c o m p a c t
a l g o r i t h m , d e r i v e d f r o m t h e H e b b i a n a d a p t a t i o n r u l e
o f t h e n e x t s e c t io n , w h i c h c o n s t r u c t s T R N s .
2 . H E B B I A N A D A P T A TI O N R U L E F O R M S
D E L A U N A Y T R I A N G U L A T I O N S
I n t h e f o l lo w i n g w e a s s u m e a s e t o f n e u r a l u n i t s i , i =
1 . . . . N t h a t c a n d e v el o p
l a t e ra l connec t i ons
b e t w e e n
e a c h o t h e r . A n e u r a l u n i t c o n n e c t s i t s e l f w i t h a n o t h e r
u n i t b y d e v e l o p i n g a
s y n a p t i c l i n k
t o t h i s u n i t . T h e
l a t e r a l c o n n e c t i o n s a r e d e s c r i b e d b y a c o n n e c t i o n
s t r e n g t h m a t r i x C w i t h e l e m e n t s
C o E ~ .
The la rge r
a m a t r i x e l e m e n t C 0 , t h e s t r o n g e r i s t h e s y n a p t i c l i n k
f r o m u n i t i t o u n i t j . O n l y i f C o > 0 , w e r e g a r d n e u r a l
u n i t i a s b e i n g c o n n e c t e d w i t h u n i t j . I f C 0 = 0 , n e u r a l
un i t i i s
n o t
c o n n e c t e d w i t h u n i t j . N e g a t i v e v al u e s fo r
C o
do no t a r i s e .
T h e b a s i c p r in c i p l e t h a t g o v e r n s t h e c h a n g e o f i n -
t e r n e u r a l c o n n e c t i o n s t r e n g th , a t l e a s t in p a r t s o f t h e
h i p p o c a m p u s ( K e l s o , G a n o n g , & B r o w n , 1 9 8 6 ) , w a s
f i rs t f o r m u l a t e d b y H e b b ( 1 9 4 9 ) . A c c o r d i n g to H e b b ' s
p o s t u l a t e , a p r e s y n a p t i c u n i t i in c r e a s e s t h e s t r e n g t h o f
i t s s y n a p t ic l i n k t o a p o s t s y n a p t i c u n i t j i f b o t h u n i t s
a re co ncu r ren t ly ac t ive , tha t i s, i f bo th ac t iv i t i e s do cor -
r e la t e . A v a r i e t y o f q u a n t i t a t i v e f o r m u l a t i o n s o f t h i s
c o n j u n c t i v e m e c h a n i s m h a v e b e e n p r o p o s e d , f o r e x -
a m p l e , f o r m o d e l i n g P a v l o v i a n c o n d i t i o n i n g ( G r o s s -
be rg , 1974) , m o tor l ea rn ing (Marr , 1969) , o r a s soc ia -
t iv e m e m o r y ( P a l m , 1 98 2 ; K o h o n e n , O j a , & L e h t i o ,
1 9 8 5 ). I n i t s s im p l e s t m a t h e m a t i c a l f o r m u l a t i o n H e b b ' s
r u l e i s d e s c r i b e d b y t h e e q u a t i o n
Co oc y i . yj, (2 )
i n w h i c h t h e c h a n g e o f t h e s t r e n g t h C o o f t h e s y n a p t i c
l i n k f r o m u n i t i t o u n i t j i s l i n e a r l y p r o p o r t i o n a l t o t h e
p r e s y n a p t i c a c t i v i t y
Yi
a n d t o t h e p o s t s y n a p t i c a c t i v i ty
yj (see , e .g. , Coo per, 1973 ). T he qu an ti t ie s Yi , i = 1,
. . . . N d e n o t e t h e o u t p u t a c t iv i t ie s o f t h e n e u r a l u n i t s .
M o r e r e a li s ti c a n d d e t a i l e d m a t h e m a t i c a l d e s c r i p t i o n s
o f H e b b ' s p o s t u l a t e t a k e t e m p o r a l a s p e c t s a n d t h e s t o -
c h a s t ic n a t u r e o f t h e o u t p u t a c t iv i t ie s o f n e u r o n s i n t o
a c c o u n t . A l s o , it m a y b e n e c e s s a r y t o a d d a d e c a y t e r m
t o k e e p t h e s t r e n g t h o f t h e s y n a p t i c l i n k s b o u n d e d .
D i f f e r e n t n e u r a l n e t w o r k m o d e l s h a v e e m p l o y e d d if -
f e r e n t r e a li z a t io n s o f H e b b ' s p o s t u l a t e , d e p e n d i n g o n
t h e a d a p t a t i o n r u l e s r e q u i r e d t o a c h i e v e t h e d e s i r e d
i n f o r m a t i o n p r o c e s s in g t as k . W e w i l l e m p l o y t h e H e b b
r u l e i n a f o r m t h a t i n c o r p o r a t e s t h e n o v e l a s p e c t o f
c o m p e t i t i o n
a m o n g t h e s y n o p t i c l i n k s. W e a s s u m e t h a t
to each neura l u n i t i a we igh t vec tor w~ E ~ t is assigned.
F u r t h e r , w e a s s u m e t h a t e a c h n e u r a l u n i t i , i = 1 . . . .
N r e c e iv e s t h e s a m e e x t e r n a l i n p u t p a t t e r n s v E ~ o .
T h e w e i g h t v e c t o r w i d e t e r m i n e s t h e c e n t e r o f t h e r e -
cep t ive f i e ld o f un i t i in the s ense tha t w i th the recep t ion
o f a n i n p u t p a t t e r n v t h e o u t p u t a c t i v it y y ; o f u n i t i i s
the larger the c loser i ts w
is
to V. In m a them at ica l t e rm s ,
w e a s s u m e t h a t
Y i = R (
I I v - w i l l )
i s v a li d , w i t h R ( . )
b e i n g a p o s i t i v e a n d c o n t i n u o u s l y m o n o t o n i c a l l y d e -
c r e a s in g f u n c t i o n ( e . g ., a G a u s s i a n ) .
I f w e a p p l y t h e H e b b r u l e i n t h e s i m p l e f o r m g i v e n
b y e q n ( 2 ) , w e o b t a i n f o r t h e c h a n g e o f t h e s t r e n g t h o f
t h e l a t e r a l c o n n e c t i o n s
C o
w i t h t h e p r e s e n t a t i o n o f a n
i n p u t p a t t e r n v
A f o o c R l l v - - w , l l ) . R l l v - w i l l ) . 3 )
Asym pto t ica l ly , tha t i s , wi th the s equen t ia l p re sen ta t ion
o f m a n y i n p u t p a t t e r n s v ~ 3 to , t h e
Co's are
d e t e r m i n e d
b y t h e i n t e g ra l o f e q n ( 3 ) o v e r t h e g i v e n p a t t e r n d i s t r i -
b u t i o n P ( v ) . I f w e a s s u m e a p a t t e r n d i s t r i b u t i o n t h a t
i s h o m o g e n e o u s o v e r ~ u , w e o b t a i n
A C i j ( t ~ o o) oc f ~ D
g(l lv - wil l)" g(l lv - wil l) dv. (4)
H e n c e , e m p l o y i n g t h e H e b b r u l e i n t h e s i m p l e f o r m a s
g i v e n i n e q n ( 2 ) y i e l d s t h e r a t h e r t r i v i a l r e s u l t t h a t e a c h
n e u r a l u n i t i de v e lo p s c o n n e c t i o n s t o a l l t h e o t h e r u n i t s
j ~ i , wi th l a te ra l con nec t ion s t reng ths C o.t h a t a r e s i m -
p ly p ropo r t ion a l to the ov e r laps o f the recep t ive f i e lds
R ( l l v -
w ; l l )
a n d R ( l l v -
w i l l ) .
T h e s t re n g t h o f t h e
s y n a p t i c li n k b e t w e e n t w o u n i t s i a n d j i s s i m p l y m o n o -
t o n i c a ll y a n d c o n t i n u o u s l y d e c r e a si n g w i t h t h e d i s t an c e
b e t w e e n w i a n d w j .
A s i n m a n y s y s t e m s g o v e r n e d b y s e l f - o r g a n i z i n g
p r o c es s es , t h e c o n n e c t i v i t y p a t t e r n t h a t e v o l v es o n t h e
s e t o f n e u r a l u n i t s b e c o m e s s i g n i f i ca n t l y m o r e s t r u c -
t u r e d i f w e i n t r o d u c e c o m p e t i t i o n . I n a w i n n e r - t a k e -
a l l n e t w o r k , f o r e x a m p l e , t h e u n i t s c o m p e t e w i t h e a c h
othe r based on the i r ou tpu t ac t iv i ti e s , wh ich f ina l ly l eads
t o a n a d a p t a t i o n o n l y o f t h e w e i g h ts o f th e u n i t w i t h
the h ighes t ou t pu t ac t iv i ty ( s ee , e. g ., Gros sbe rg , 1976 ) .
-
7/25/2019 Martinet z 1994
4/16
5 1 0 T . M a r t i n e t z a n d K . S c h u l t e n
W i t h o u t c o m p e t i t i o n , a l l t h e u n i t s w o u l d b e h a v e a l i k e
a n d n o s p e c i a l iz a t i o n o f t h e u n i t s , a s i t is c h a r a c t e r i s t ic
f o r w i n n e r - t a k e - a l l n e t w o r k s , w o u l d e v o l v e .
A n a l o g t o t h e c o m p e t i t i o n a m o n g t h e u n i t s in a w i n -
n e r -t a k e- a ll n e t w o r k w e i n t r o d u c e c o m p e t i t i o n a m o n g
t h e s y n a p t i c l in k s . I n s t e a d o f b e i n g b a s e d o n t h e o u t p u t
ac t i v i t i e s o f t he neu r a l u n i t s i t s e l f, a s i n a w i n ne r - t ak e -
a ll n e tw o r k , i n o u r m o d e l t h e c o m p e t i t io n a m o n g t h e
s y n a p t i c l i n k s i s d e t e r m i n e d b y w h a t w e w a n t t o c a l l
t h e c o r r e l a t e d o u t p u t a c t i v i t ie s Y 0, t h e c o r r e l a t i o n s o f
t h e o u t p u t a c t iv i ti e s o f al l p a ir s o f p r e - a n d p o s t s y n a p t i c
u n i t s . I n t h e q u a n t i t a t i v e f o r m u l a t i o n g i v e n b e lo w , t h e
c o r r e l a t e d o u t p u t a c t i v i t i e s a r e d e t e r m i n e d b y
Y~j =
y g . y j ,
a c c o r d i n g t o t h e H e b b r u l e ( 2 ) . K e e p i n g th e
a n a l o g y t o w i n n e r - t a k e - a l l n e t w o r k s , w i t h t h e p r e s e n -
t a t i o n o f a n i n p u t p a t t e r n v w e o n l y m o d i f y t h e s y n a p t i c
l i nk i - j who se ac t i v i t y Y o = Y i Y j i s h i ghes t . I ns t ead
o f c h a n g i ng t h e c o n n e c t i o n s t re n g t h s
C o
a c c o r d i n g t o
t h e H e b b r u l e ( 2 ) , i n t h e f o l lo w i n g w e w i ll e m p l o y a
w i n n e r - t a k e - a l l o r c o m p e t i t i v e v e rs i o n o f e q n ( 2 ) , d e -
t e r m i n e d b y
A C ~ j O C { o i . Y j
i f
y i . y j > - y k . y t
ot he r wi se .
V k , l = 1 . . . . N
( 5 )
W e w i ll s h o w t h a t , i n s t e a d o f c o n n e c t i n g e a c h u n i t
w i t h a ll th e o t h e r u n i t s , t h e c o m p e t i t i v e H e b b r u l e ( 5 )
f o r m s a c o n n e c t i v i t y s t r u c t u r e a m o n g t h e n e u r a l u n i t s
i , i = 1 . . . . N t h a t c o r r e s p o n d s t o t h e D e l a u n a y t r ia n -
g u l a t i o n o f t h e w e i g h t v e c t o rs w~ . . . . . W N. M o r e p r e -
c is el y, we wi l l show t ha t i f we p r esen t s equen t i a l l y i npu t
p a t t e r n s v w i t h a d i s tr i b u t i o n P ( v ) t h a t h a s s u p p o r t ( i s
n o n z e r o ) e v e r y w h e r e o n ~ D , t h e n t h e e l e m e n t s
C o
o f
t h e c o n n e c t i o n s t r e n g th m a t r i x C o b e y a s y m p t o t i c a l ly
O [Ci j ( t - -~
oo)] =
A o i , j = 1 . . . . N
( 6 )
w h e r e 0 ( . ) i s t h e H e a v y s i d e s t e p f u n c t i o n a n d w h e r e
A o a r e t h e e l e m e n t s o f t h e a d j a c e n c y m a t r i x A o f th e
D e l a u n a y t r i a n g u l a t i o n o f t h e p o i n t s w ~ . . . . . W u , f o r
w h i c h
i f
Vi f -) Vj # ~ ( Vi ,
V j a r e a d j a c e n t )
i f
V, f ] V j = Z ( V i , V j
a r e n o t a d j a c e n t )
( 7 )
i s v a l id . V g, ~ a g a i n d e n o t e th e V o r o n o i p o l y h e d r a o f
w~, w , t h a t i s , i n a w i nn e r - t ake - a l l ne t w or k t he r ecep t i ve
f i e lds o f t he un i t s i , j .
B e f o r e w e p r o v e e q n ( 6 ) , w e f i rs t s h o w h o w t h e c o m -
p e t it iv e H e b b r u l e f o r c o n s t r u c t in g t h e D e l a u n a y t r ia n -
g u l a t io n c a n b e f o r m u l a t e d i n a v e r y c o m p a c t a l g o -
r i t h m . T h e c o r r e l a t e d a c t i v i t y
Y o = Y i Y j
i s h i ghes t i f
i ( o r j ) d e n o t e s t h e n e u r a l u n i t w i t h t h e l a r g e st o u t p u t
a n d j ( o r i , r e s p e c t i v e l y ) d e n o t e s t h e u n i t w i t h t h e s e c -
o n d l a r g e s t o u t p u t . B e c a u s e
Y o = Y j i
i s va l i d , t he r e a r e
a l w a y s t w o
w i n n i n g
l in k s , t h e s t r e n g th s o f w h i c h a r e
c h a n g e d b y t h e s a m e a m o u n t , t h a t i s, t h e s t r e n g t h o f
t he l i nk f r om i t o j a s we l l a s o f t he l i nk f r om j t o i .
H e n c e , t h e c o n n e c t i o n s t r e n g t h m a t r i x C o i s s y m m e t r i c ,
a n d i f w e s a y " u n i t i is c o n n e c t e d w i t h u n i t j , " t h i s
a l w a y s i m p l i e s t h a t u n i t j i s a l s o c o n n e c t e d w i t h u n i t
i . B e c a u s e w e a r e i n t e r e s t e d i n t h e g r a p h d e f i n e d b y
t h e c o n n e c t i o n s b e t w e e n t h e u n i t s , a n d b e c a u s e t h i s
g r a p h is c o m p l e t e l y d e t e r m i n e d b y t h e a d j a c e n c y m a -
t r i x A i j = O C i j ) , i n t h e f o l l o w i n g a l g o r i t h m w e w i l l n o t
c a r e a b o u t t h e a b s o l u t e v a l u e s o f th e
C~j's,
b u t w i ll o n l y
r e g i s te r w h e t h e r a Ci j b e c o m e s n o n z e r o .
T h e p r o c e d u r e f o r c o n s t ru c t i n g t h e c o n n e c t i o n s b e -
t w e e n t h e u n i t s i , j ( i , j = 1 . . . . N ) c a n n o w b e f o r -
mul a t ed a s f o l l ows :
( i ) S e t a l l c o n n e c t i o n s t r e n g th s
C~j
t o z e r o ;
( i i ) P r e s e n t a n i n p u t p a t t e r n v @ ~ z~ w i t h p r o b a b i l i t y
P ( v ) ;
( i i i ) D e t e r m i n e u n i t io f o r w h i c h
IIv - w~oll -< IIv - will v j = l . . . . . N
a n d u n i t i l f o r w h i c h
IIv - w,, II -< IIv - w, II Vj # io, j = 1 . . . . N;
( iv ) I fCioi, = 0, se t Cioi > 0, that i s , co nn ec t io an d i , .
I f C~oi > 0 , l eave C~ oi,, unc hang ed . Co nt i n ue w i t h
( i i ) .
I n F i g u r e 2 a l g o r i t h m ( i ) - ( i v ) i s i l l u s t r a t e d . T w o
u n i t s i , j b e c o m e c o n n e c t e d i f f t h e i r V o r o n o i p o l y h e d r a
V~, ~ a r e ad j acen t . T o show r i gor o us l y t ha t t he ad j a -
c e n c y m a t r i x A ,j = 0 ( C u ) o f t h e c o n n e c t i v i t y s t r u c t u r e
b e c o m e s e q u i v a l e n t to t h e a d j a c e n c y m a t r i x o f t h e D e -
l a u n a y t r i a n g u l a t i o n ~ gs o f th e s e t o f p o i n t s S = ( w ~,
F I G U R E 2 . I ll u s t r a ti o n o f th e u p d a t e r u l e f o r th e c o n n e c t i o n s
b e t w e e n t h e n e u r a l u n i t s . E a c h t i m e a n i n p u t p a t t e r n v i s p r e -
s e n t e d w i t h in t h e s h a d e d a r e a , w h i c h d e p i c t s t h e V o r o n o i p o l y -
g o n V l o f u n i t i , a c o n n e c t i o n f r o m i t o t h e u n i t j w i t h i ts w e i g h t
v e c t o r w s e c o n d c l o s e s t t o th e i n p u t p a t t e r n is e s t a b l i s h e d .
T h e n u m b e r s 1 . . . . 6 d e n o t e t h e s u b re g i o n s w i th i n t h e s h a d e d
a r e a f o r w h i c h th e r e s p e c t i v e n e ig h b o r i n g u n i t i s s e c o n d c l o s e s t
t o a n i n p u t s i g n a l v E V I . U n i t i d e v e l o p s c o n n e c t i o n s o n l y t o
t h o s e u n i t s, th e V o r e n o i p o l y g o n s o f w h i c h s h a r e a n e d g e w i th
i t s o w n V o r o n o i p o l y g o n .
-
7/25/2019 Martinet z 1994
5/16
Topology Representing Networks 511
. . . . w u ) , w e i n t r o d u c e t h e
s e c o n d - o r d e r V o r o n o i p o l y -
h e d r a V o , i , j = 1 . . . . N .
T h e s e c o n d - o r d e r V o r o n o i
p o l y h e d r o n V0 is given b y a l l th e v G ~RD for wh ich wi
a n d w j a r e t h e t w o c l o s e s t p o i n t s o f S ; t h a t i s,
V o
is
d e f i n e d b y
vo
= {v e ~111v - will -< IIv - wkll A IIv - will
-< IIv - Wkll
V k + i , j } .
(8)
AS 1I,-, als o Vo f o r m s a c o n v e x p o l y h e d r o n . W e s e e f r o m
e q n ( 8 ) t h a t a l g o r i t h m ( i ) - ( i v ) c o n n e c t s t w o u n i t s i,
j on ly i f K , j J~ i s va l id . We wi l l p rove th a t
V o q: $25
i s v a l i d i f f t h e c o r r e s p o n d i n g f i r st - o r d er V o r o n o i p o l y -
hed ra Vi , Vj a re ad jace n t , tha t i s , i f f
V i f l V j ~ ~ .
T h e n , i n c a se f z P ( v ) d v 0 h o l d s f o r e a c h VU 4= ~5 ,
. ~ t J . . .
t h e c o n n e c t io n s g e n er a t ed b y a l g o r i th m ( 1 ) - ( w ) f o r m
t h e D e l a u n a y t r i a n g u l a t i o n o f t h e p o i n t s w~ . . . . . W N.2
THEOREM 1.
F o r a s e t
S = { w~ . . . . w~}
o f p o i n t s w i
E ~ D t h e r e l a t i o n
V/
f l ~ ,~=~ ~ j ~ .~
(9)
i s v a l i d f o r e a c h p a i r i , j . V i d e n o t e s t h e f i r s t - o r d e r
V o r o n oi p o ly h e d r o n o f p o i n t
wi ,
a n d V o. d e n o t e s t h e s e c -
o n d - o r d e r V o r o no i p o l y h e d r o n o f th e p o i n t s w i , w j .
P r o o f .
If V/f-) Vj 4: O is val id, there is a v E ~ o w ith
v ~ V i
an d v E Vj. Th en w e obta in [Iv - wil l = [Iv -
wi l l -< I l v - w~ ll fo r al l Wk E S a nd , the re fore , v ~
Vo,
that is ,
V i i 4 : ~ ,
is val id.
I f
Vii ~ ~
is val id, there is a v E ~tD f o r w h i c h t h e
p o i n t s w i a n d w j a r e t h e t w o n e a r e s t n e i g h b o r s . W i t h o u t
los s o f gene ra l i ty we as sum e th a t w~ i s the nea res t
ne ighbor . Because fo r each u ~ ~ the po in t wj i s e i the r
t h e n e a r e s t o r t h e s e c o n d n e a r e s t n e i g h b o r o f u , a n d
for u = v the p o in t wi i s c loses t and fo r u = wj the po in t
w~ is c loses t to u, there is a u* G V ~ for wh ich [[u* -
w i l l - - I l u *
- wjl l is val id. Hen ce, we ob tai n u* ~ 1I,.
a n d u * ~ ~ , a n d , t h e r e f o r e , u * ~
Vi f l V j ,
that is , Vj
fq Vj # ~5, is valid .
In the fo l lowing we wi l l cons ide r pa t t e rn d i s t r ibu t ions
P ( v ) t h a t h a v e s u p p o r t n o t o n t h e e n t i r e e m b e d d i n g
s p a c e ~ D , b u t o n l y o n a s u b m a n i f o l d M . I n t h e s e ca s es ,
f o r s o m e
V o 4= ~
t h e i n t e g r a l
f v o
P ( v ) d v m i g h t v a n i sh ,
wi th the re su l t tha t the edge i - j wi l l no t be e s tab l i shed
b y t h e c o m p e t i t i v e H e b b r u l e . I n t h e s e c as e s, t h e c o m -
p e t i t i v e H e b b r u l e d o e s n o t f o r m t h e e n t i r e D e l a u n a y
t r i a n g u l a t i o n , b u t o n l y s u b g r a p h s o f it . W e w i l l s h o w
t h a t t h e s e s u b g r a p h s d e f i n e t o p o l o g y p r e se r v i n g m a p s .
3 . D E L A U N A Y T R I A N G U L A T I O N A N D
T O P O L O G Y P R E S E R V I N G M A P S
T h e d e f i n i t i o n o f t h e D e l a u n a y t r i a n g u l a t i o n a s b e i n g
t h e g r a p h t h a t c o n n e c t s t h o s e p o i n t s w i , w j t h a t h a v e
2 The following heo rem was formulated ogether with Ph ilippe
Dalger and B ennoit Noel (Dalger et a l., 1 992).
ad jace n t Voronoi po lyhe dra V, , Vj m ak es th i s s t ruc tu re
i d e a l ly s u it e d f o r a p r o x i m i t y p r o b l e m o f a d i f f e re n t
t y p e t h a n t h e o n e s m e n t i o n e d i n t h e f i r s t s e c t i o n ,
n a m e l y , f o r t h e f o r m a t i o n o f s o - c a l l e d t o p o l o g y p r e -
se rv ing fea tu re m aps . Topology prese rv ing fea tu re m ap s
p l a y a n i m p o r t a n t r o le a s c o m p o n e n t s i n a v a r i e ty o f
n a t u r a l a s w e l l a s a r t i f i c i a l n e u r a l i n f o r m a t i o n p r o -
ces s ing sys tem s (Knudsen , de Lac , & Es te r ly , 1987;
Kohonen , 1989; R i t t e r , Mar t ine tz , & S chul ten , 1992) .
B y p r o je c t i n g i n p u t p a t t e r n s o n t o a n e t w o r k o f n e u r a l
u n i t s s u c h t h a t s i m i l a r p a t t e r n s a r e p r o j e c t e d o n t o u n i t s
a d j a c e n t i n t h e n e t w o r k a n d , v i c e v er sa , s u c h t h a t u n i t s
a d j a c e n t i n t h e n e t w o r k c o d e s i m i l a r p a t t e r n s , a r e p -
r e s e n t a t i o n o f t h e i n p u t p a t t e r n s i s a c h i e v e d t h a t i n
pos tproces s ing s tages a l lows one to exp lo i t the s im i la r i ty
r e l a t io n s o f th e i n p u t p a t t e r n s . E x a m p l e s o f t o p o l o g y
p r e s e r v in g f e a t u r e m a p s i n t h e n e r v o u s s y s t e m a r e t h e
r e t i n o t o p i c m a p i n t h e v i s u a l c o r t e x ( H u b e l & W i e s e l,
1 97 4; B l a sd e l & S a l a m a , 1 9 8 6 ) , t h e m a p p i n g f r o m t h e
b o d y s u r f ac e o n t o t h e s o m a t o s e n s o r y c o r t e x ( K a a s e t
a l. , 1 9 7 9 ) , o r t h e t o n o t o p i c m a p s i n t h e a u d i t o r y c o r t e x
( S u g a & O ' N e i ll , 1 9 7 9 ) . A s c o m p o n e n t s o f a rt i fi c i a l
n e u r a l i n f o r m a t i o n p r o c e s s i n g s y s t em s , t o p o l o g y p r e-
s e r v i n g f e a tu r e m a p s h a v e b e e n a p p l i e d s u c c e ss f u l ly in
s p e e c h p r o ce s s i n g ( K o h o n e n e t a l ., 1 98 4; K o h o n e n ,
1990; Naylor & L i , 1988; Bran dt , Beh m e , & S t rube ,
1991 ) , im age proces s ing (Nas r abad i & F eng , 19 88) ,
a n d r o b o t i c s ( R i t t e r & S c h u l t e n , 1 9 86 ; M a r t i n e t z , R i t -
t e r , & S chul ten , 1990) .
A t o p o l o g y p r es e r v i n g f e a tu r e m a p i s d e t e r m i n e d b y
a m a p p i n g f r o m a m a n i f o l d M _ ~ D o n t o t h e v e r ti c e s
( n e u r a l u n i t s ) i , i = 1 . . . . N o f a g r a p h ( n e t w o r k ) G .
T h e m a p p i n g f r o m M t o G i s d e t e r m i n e d b y p o i n t e r s
wi E ~D, i = 1 . . . . N a t t ach ed to the ve r t i ces i . A
f e a t u r e v e c to r v E M i s m a p p e d t o t h e v e r t e x i * ( v ) ,
the p o in te r wi . ( , ) o f wh ich i s c losest to v ; th a t i s , v i s
m a p p e d t o t h e v e r te x i * ( v ) w h o s e V o r o n o i p o l y h e d r o n
V i . ( , )
e n c l o se s v . T h e m a p p i n g ~ i s c o m p l e t e l y d e t e r-
m i n e d b y t h e p o in t e r se t S = { w l . . . . . W N } , a n d w e
c a n w r i t e
s :M ~ - - ,G , v E M - - - , i * ( v ) @ G
(10)
w h e r e i * ( v ) i s d e t e r m i n e d b y
IIw,.~v) - vii ~ II w /- vii v i E G. (11 )
T h e m a p p i n g ~ s f r o m M t o G i s n e i g h b o r h o o d p r e -
se rv ing , i f s im i la r fe a tu re vec tors , tha t i s , v ' s tha t a re
c l o se w i t h i n t h e m a n i f o l d M , a r e m a p p e d t o v e r t ic e s i
t h a t a r e c l o s e w i t h i n t h e g r a p h . T h i s r e q u i r e s t h a t
p o i n t e r s w i , w j t h a t a r e n e i g h b o r i n g o n t h e f e a t u r e
m a n i f o l d a r e a s s i g n ed t o v e r t i c es i , j t h a t a r e a d j a c e n t
i n t h e g r a p h . 3
T h e i n ve r se m a p p i n g ~ 1 f r o m th e g ra p h G o n t o
t h e m a n i f o l d M i s d e t e r m i n e d b y
~ s l : G ~ - ' ~ M , i E G ~ - ' ~ w i E M
(12)
-
7/25/2019 Martinet z 1994
6/16
512 T Martinetz and K Schuhen
(a) (b) (c)
F I G U R E 3 . A s q u a r e -s h a p e d m a n i f o l d M m a p p e d o n t o t h r e e
d i f fe r e n t g r a p h s G . Ea c h o f t h e g r a p h s c o n s is t s o f n in e v e r t ic e s
i , a n d t h e a s s o c i a t e d p o i n t e r s w l a r e d i s t r ib u t e d r e g u l a rl y o v e r
t h e s q u a r e . T h e t h r e e d i f f e re n t f e a t u r e m a p s a r e v i s u a l i z e d b y
m a r k in g t h e p o i n t e r p o s it io n s w l o n t h e m a n i f o l d M b y d o t s a n d
c o n n e c t i n g b y l i n e s t h o s e p o i n t e r p o s i ti o n s w ~ , w j t h e c o r r e -
s p o n d i n g v e r t ic e s i , j o f w h i c h a r e a d j a c e n t i n t h e g r a p h G . I n
( a ) t h e g r a p h i s a s t ri n g t h a t o n l y a l l o w s a n e i g h b o r h o o d p r e -
s e r vi n g i n v e r s e m a p p i n g f r o m G t o M , b u t n o t a n e i g h b o r h o o d
p r e s e rv in g m a p p i n g f r o m M t o G . A d j a c e n t v e r t i c e s a r e a s -
s i g n e d t o n e i g h b o r in g l o c a ti o n s , b u t n o t a ll t h e p o i n t e r s w ~ t h a t
a r e n e i g h b o r i n g o n M b e l o n g t o a d j a c e n t v e r ti c e s . W i t h t h i s
g r a p h i c a l s t r u c tu r e i t i s n o t p o s s i b l e t o fo r m a t o p o l o g y p r e -
s e rv in g m a p o f M . I n ( b ) t h e g r a p h G h a s a t h r e e - d i m e n s i o n a l
s t ru c t u re . I n t h i s c a s e t h e m a p p i n g f r o m M t o G , b u t n o t t h e
i n v e r s e m a p p i n g f r o m G t o M , i s n e i g h b o r h o o d p r e s e r vi n g .
P o i n te r s t h a t a r e n e ig h b o r i n g o n M a r e a s s i g n e d t o a d j a c e n t
v e r ti c e s , b u t n o t a l l th e v e r t i c e s t h a t a r e a d j a c e n t i n G a r e a s -
s i g n e d t o n e i g h b o ri n g l o c a ti o n s o n M . H e n c e , a s i n ( a ) , t h e
g i v e n g r a p h i c a l s t r u c tu r e d o e s n o t a l l o w o n e t o fo r m a t o p o lo g y
p r e se r vi n g m a p o f M . O n l y i n ( c ) w h e r e t h e g r a p h G i s a s q u a r e
l a t t ic e a n d m a t c h e s t h e t o p o l o g y o f M , b o t h th e m a p p i n g f r o m
M t o G a n d t h e i n v e r s e m a p p i n g f ro m G t o M a r e n e i g h b o r h o o d
p r e s e r v in g . On ly in t h is c a s e d o e s G f o r m a t o p o lo g y p r e s e r v in g
m a p o f M .
( w e a s s u m e t h a t a l l p o i n t e r s w , , i = 1 . . . . N , l i e o n
M ) . A n a l o g t o n e i g h b o r h o o d p r e s e rv a t i o n o f ~ s ,
n e i g h b o r h o o d p r e s e r v a t i o n o f t h e i n ve r se m a p p i n g
s ~ i s g iven i f po i n t e r s w i , w j o f ad j a cen t ve r t i ces i , j
a r e n e i g h b o r i n g o n t h e m a n i f o l d M .
T he g r aph G wi t h i t s ve r t i ces i a s s i gned t o t he l o -
c a t i o n s w i f o r m s a t o p o l o g y p r e s e r v i n g m a p o f M , i f
t he m app i ng ~ i' s f r o m M t o G as we l l a s t he i nve r se
m a p p i n g ~ f r o m G t o M i s n e i g h b o r h o o d p re s e rv i n g .
O n l y t h e n G f o r m s a m a p o n w h i c h a d j a c e n t l o c a t io n s
( v e r t ic e s ) c o r r e s p o n d t o f e a t u r e s n e i g h b o r i n g o n M a n d ,
v i c e v e r s a , w h i c h r e p r e s e n t s f e a t u r e s n e i g h b o r i n g o n
M b y a d j a c e n t l o c a t i o n s .
T h e q u e s t i o n a r i s es w h i c h t y p e o f g r a p h G c a n p r o -
v i d e a t o p o l o g y p re s e r v in g m a p o f a g iv e n m a n i f o l d M .
C r u c i a l i s G ' s c o n n e c t i v i t y s t r u c t u r e o r t o p o l o g y c o m -
p a r e d t o t h e t o p o l o g y o f M . T h i s i s d e m o n s t r a t e d s c he -
3 A t t h i s p o i n t w e l e a v e th e d e f i n i t i o n o f " a d j a c e n c y o f p o i n t e r s
w i o n a m a n i f o l d M " t o t h e r e a d e r ' s i n t u i ti o n , l i ke i n m o s t c o n t r i -
b u t i o n s o n t o p o l o g y p r e s e r v i n g f e a t u r e m a p s . A r i g o r o u s d e f i n i t i o n
b a s e d o n V o r o n o i p o l y h e d r a i s g i v e n in t h e n e x t s e c t i o n .
m a t i c a l l y i n F i g u r e 3 . I n e a c h o f t h e t h r e e c a s e s t h a t
a r e d e p i c t e d th e m a n i f o l d M i s s i m p l y a s q u a r e , a n d
t h e g r a p h G c o n s i s ts o f n i n e v e r t i c e s i, t h e a s s o c i a t e d
p o i n t e r s w i o f w h i c h a r e d i s t r i b u t e d r e g u l a r l y o v e r t h e
s q u a r e - s h a p e d m a n i f o l d . I n F i g u r e 3 a t h e g r a p h G h a s
a o n e - d i m e n s i o n a l t o p o l o g y a n d c o n s is t s o f n i n e v e r t i ce s
a r r a n g e d a s a st r in g , c o m p a r e d t o t h e t w o - d i m e n s i o n a l
m a n i f o l d M . I n t h i s c a s e G c a n n o t f o r m a t o p o l o g y
p r e s e r v i n g m a p o f M . A l l i t c a n a c h i e v e is a n e i g h b o r -
h o o d p r e s e r v i n g i n v er s e m a p p i n g ~ - 1 f r o m G t o M .
I n F i g u r e 3 b w e h a v e t h e o p p o s i t e c a s e . T h e d i m e n -
s i o n a l it y o f G is h i g h e r t h a n t h e d i m e n s i o n a l i t y o f M .
T h e r e s u lt is th a t a n e i g h b o r h o o d p r e s e r v i n g m a p p i n g
f r o m M t o G is p o s s ib l e , b u t n o t a n e i g h b o r h o o d
p r e s e r v i n g i n v e r s e m a p p i n g ~ f r o m G t o M . H e n c e ,
a s in ( a ) , G i s n o t a b l e t o f o r m a t o p o l o g y p r e s e r v i n g
m a p o f M . I n F i g u r e 3 c t h e t o p o l o g y o f t h e g r a p h G i s
a s q u a r e l a t ti c e a n d c o r r e s p o n d s t o t h e t o p o l o g y o f t h e
m a n i f o l d M . O n l y i n t h is c a s e d o e s G f o r m a t o p o l o g y
p r e s e rv i n g m a p o f M ; t h a t i s, t h e m a p p i n g f r o m M
t o G a s w e ll a s th e i n v e r s e m a p p i n g ~ - 1 f r o m G t o M
i s n e i g h b o r h o o d p r e s e r v i n g .
A n u m b e r o f n e u r a l n e t w o r k m o d e l s f o r a d a p t iv e l y
f o r m i n g t o p o l o g y p r e se r v i n g f e a tu r e m a p s h a v e b e e n
p r o p o s e d ( W i l l s h a w & y o n d e r M a l s b u r g , 1 9 7 6 ; y o n
d e r M a l s b u r g & W i ll s ha w , 1 9 7 7 ; T a k e u c h i & A m a r i ,
1 97 9 ; K o h o n e n , 1 9 8 2 a, b; D u r b i n & M i t c h i s o n , 1 9 9 0 ) .
A m o d e l t h a t p r o v i d es a v e r y c o m p a c t p r o c e d u r e a n d ,
t he r e f o r e , has f ound w i despr ead app l i ca t i on i n a r t if i c ia l
n e u r a l i n f o r m a t i o n p r o c e s s i n g s y s t e m s i s K o h o n e n ' s
s e l f- o r g a n i zi n g f e a t u r e m a p ( K o h o n e n , 1 9 8 2 a , b , 1 9 8 9 ,
1 9 9 0 ) . T h i s a l g o r i t h m r e q u i r e s t h a t o n e f i r s t c h o o s e s
a g r a p h G , u s u a l l y a o n e - , t w o - , o r t h r e e - d i m e n s i o n a l
l a t t i c e . I n a s u b s e q u e n t a d a p t a t i o n p r o c e d u r e , t h e
p o i n t e r s W g a r e d i s t r i b u t e d o v e r t h e g i v e n f e a t u r e m a n -
i f o ld M i n s u c h a w a y, t h a t ( i ) p o i n t e r s l ie o n l y o n M ,
a n d ( i i ) p o i n t e r s o f v e r ti c e s a d j a c e n t i n G a r e a s s i g n e d
t o l o c a ti o n s cl o se o n M . H e n c e , K o h o n e n ' s a l g o r i t h m
t r i e s t o f o r m a n e i g h b o r h o o d p r e s e r v i n g i n v e r s e m a p -
p i n g -~ f r o m G t o M , b u t n o t n e c e s s a r i l y a n e i g h -
b o r h o o d p r e s e r v in g m a p p i n g f r o m M t o G . T o o b t a i n
a t o p o l o g y p r e s e r v i n g m a p , t h e t o p o l o g i c a l s t r u c t u r e o f
t h e p r e s e t g r a p h G h a s t o m a t c h t h e t o p o l o g i c a l s t r u c -
t u r e o f t h e g i v en m a n i f o l d M .
F i g u r e 4 i l l u s t ra t e s t h r e e e x a m p l e s o f d i f f e r e n t m a p s
t h a t e v o l v e w i t h K o h o n e n ' s a l g o r it h m , d e p e n d i n g o n
t h e d e g r e e o f m i s m a t c h b e t w e e n t h e t o p o l o g i c a l s tr u c -
t u r e o f th e p r e s e t g ra p h G a n d t h e g i v en m a n i f o l d M .
I n a l l t h r e e e x a m p l e s , t h e m a n i f o l d M i s a s u b m a n i f o l d
o f ~ 2 . I n F i gur es 4a , b , t he ma ni f o l d M aga i n i s a squa r e ,
l i ke i n F i gur e 3 . I n F i gur e 4c t he mani f o l d M i s d i s -
c o n n e c t e d a n d p i ec e w is e o n e - a n d t w o - d i m e n s i o n a l . I n
F i g u r e 4 a t h e p r e s e t g r a p h G i s a t w o - d i m e n s i o n a l l a t t ic e
a n d , h e n c e , t h e t o p o l o g y o f t h e g i v e n m a n i f o l d a n d t h e
g r a p h d o m a t c h . A s w e c a n s e e , i n th i s ca s e K o h o n e n ' s
s e l f- o r g a n i z in g f e a t u r e m a p a l g o r i t h m p r o v i d e s a t o -
p o l o g y p r e s e r v i n g m a p . I n F i g u r e 4 b t h e t o p o l o g y o f
-
7/25/2019 Martinet z 1994
7/16
Topology Represen t ing Networks 5 3
(a) (b) (c)
F I G U R E 4 . M a p s o f m a n i f o l d s f o rm e d b y K o h o n e n s s e l f- o r -
g a n i z i n g f e a t u r e m a p a l g o r i t h m . I n ( a ) t h e g r a p h G , a s q u a r e
l a t ti c e , h a s a t o p o l o g y t h a t c o r r e s p o n d s t o t h e t o p o l o g y o f t h e
s q u a r e - s h a p e d f e a t u r e m a n i f o l d M . H e n c e , t h e m a p t h a t e v o l v e s
i s to p o l o g y p r e s e r v i n g . I n ( b ) t h e g i v e n m a n i f o l d M a g a i n i s a
s q u a r e ; h o w e v e r , t h is t i m e t h e g r a p h i s a s t ri n g a n d d o e s n o t
c o r r e s p o n d t o p o l o g i c a l l y to M . W i t h t h i s g r a p h i t is n o t p o s s i b l e
t o a c h i e v e a t o p o l o g y p r e s e r v in g m a p , a n d t h e K o h o n e n a l -
g o r i t h m c a n o n l y fo r m a n e i g h b o r h o o d p r e s e r v in g i n v e r s e
m a p p i n g ~ - 1 f r o m G t o M , b u t n o t a n e i g h b o r h o o d p r e s e r v i n g
m a p p i n g (]D f r o m M t o G . P a r t ( c ) s h o w s a n e x a m p l e o f v e r y
h i g h t o p o l o g i c a l m i s m a t c h b e t w e e n M a n d G . T h e g i v e n m a n -
i f o ld M i s d i s c o n n e c t e d a n d c o n s i s t s o f a t w o - d i m e n s i o n a l ( a
r e c t a n g l e a n d a s q u a r e ) a n d a o n e - d i m e n s i o n a l ( a l i n e a n d a
c i r c l e ) s u b m a n i f o l d ; t h e g r a p h G i s a s q u a r e - la t t i c e . A g a i n n o
t o p o l o g y p r e s e r v a t io n i s p o s s i b l e . I n t h is c a s e t h e K o h o n e n
a l g o r it h m n e i t h e r a c h i e v e s a n e i g h b o r h o o d p r e s e rv i n g m a p p i n g
f r o m M t o G n o r a n e i g h b o r h o o d p r e s e r v i n g i n v e r s e m a p p i n g
(]D 1 f r o m G to M .
the given manifold and of the graph do no t match. The
graph is a one-dimensional string of vertices, in contrast
to the two-dimensional feature manifold. As shown
schematically in Figure 3a, because of this mismatch
topology preservation is not possible, and Kohonen's
algorithm can only provide a neighborhood preserving
inverse mapping ~-1 from G to M. Figure 4c finally
shows an example where the topological structure of
the graph G and the given manifold M mismatch to an
extent that neither a neighborhood preserving inverse
mapping ~-t from G to M nor a neighborhood pre-
serving mapping from M to G can be achieved by
Kohonen's algorithm. The graph G is again a two-di-
mensional lattice. The given manifold M, however, has
a relatively intrica te topological structure. I t consists of
a combination of two-dimensional and one-dimen-
sional partially disconnected submanifolds. The two-
dimensional submanifold is formed by a rectangle and
a square. The one-dimensional submanifold consists of
a circle and a connecting line between the rectangle
and the circle. Figure 4c shows that in this case some
vertices that are adjacent in G are assigned to locations
remote on the manifold M, and, vice versa, feature vec-
tors adjacent on Mm ay be mapped onto vertices remote
in G. Obviously, the topology preservation of the map
is highly disturbed.
In Figure 3 we have demonstrated schematically hat
topology preserving maps are only possible if the to-
pological structure of the graph G matches the topo-
logical structure of he given manifold M. The Kohonen
algorithm starts from a prespecified graph G. The three
examples of Figure 4 demonstrate that Kohonen 's al-
gorithm tries to form a neighborhood preserving inverse
mapping from G to M and indeed succeeds in con-
structing a topology preserving map, if the topological
structure o f the preset graph G matches the topological
structure o f the manifold M. In cases, however, where
it is not possible to a p r io r i determine an appropriate
graph G, for example, in cases where the topological
structure of M is not known
a p r io r i
or is too compli-
cated to be specified, Kohonen's algorithm necessarily
fails in providing perfectly topology preserving maps.
An application where the topological structure o f M
is not known and, at the same time, a perfectly topology
preserving map of M is essential for optimal perfor-
mance, is in speech recognit ion as described in Brandt
et al. (1991). In their word recognition scheme each
word is described by a sequence of 19-dimensional fea-
ture vectors, with each feature vector describing the
frequency spectrum at a different time step. Hence, a
word is a trajectory in a 19-dimensional input space,
given typically by a sequence o f 40-50 feature vectors.
The feature vectors that occur with the spoken words
form a lower-dimensional submanifold M within the
19-dimensional input space. By constructing a topology
preserving map of the manifold M, as it was first de-
scribed by Kohonen et al. (1984), each word can be
represented by a trajectory within the graph G, that is,
by a sequence o f a c t i v e nodes. Interestingly, it turns out
that a classification of the words based on their trajec-
tories in G yields a significantly ncreased performance
compared to a classification based on their trajectories
on M. To obtain optimal performance in this word
recognition scheme, the topological structure o f G has
to match the topology of the feature manifold M. This
manifold, however, has a topological struc ture that is
complicated and not known. Its average dimension, for
example, seems to lie between two and three (Bauer &
Pawelzik, 1992).
Another application where only a neighborhood
preserving mapping from M to G together with a
neighborhood preserving inverse mapping from G to
M, that is, a perfectly topology preserving map of M,
provides optimal results is in representing and learning
input-o utput relations with topology preserving maps,
for example, in robotics (Ritter & Schulten, 1986;
Martinetz et al., 1990). For representing input-output
relations, to each vertex i of G an output quantity ai is
assigned. The ai's determine the output o f the map for
a given input vector v. It can be shown that a robust
and fast learning procedure for the ai's can only be
obtained if the graph G preserves the neighborhood
relations of the input manifold, which then allows a
concerted adaptation of the output quantities of adja-
cent vertices (Ritter et al., 1992). Only if adjacent ver-
tices are assigned to locations neighboring on the input
manifold, the learning process will be successful. The
learning procedure is optimized if, also, all the pointers
-
7/25/2019 Martinet z 1994
8/16
5 1 4 I . M a r t i n e t z a n d K . S c h u l t e n
W t h a t a r e n e i g h b o r i n g o n t h e i n p u t m a n i f o l d c o r r e -
s p o n d t o a d j a c e n t v e r t ic e s. T h i s i s v al i d i f f t h e m a p o f
t h e i n p u t m a n i f o l d i s p e r fe c t l y t o p o l o g y p r e se r v i n g. I n
m a n y a p p l i c a t i o n s , h o w e v e r , t h e i n p u t m a n i f o l d i s a
s u b m a n i f o l d o f a h i g h - d i m e n s i o n a l i n p u t s p a ce a n d m a y
n e i t h e r b e k n o w n a p r i o r y n o r t o p o l o g ic a l l y s im p l e
e n o u g h f o r p r e sp e c i f y in g a c o r r e s p o n d i n g l y s t r u c t u r e d
graph .
F o r t h e a p p l i c a t i o n s d e s c r i b e d a b o v e i t w o u l d b e
h i g h l y d e s ir a b le t o h a v e a p r o c e d u r e t h a t a d a p t s t h e
t o p o l o g y o f t h e g r a p h G t o t h e t o p o l o g y o f t h e g i v e n
m a n i f o l d M o r , a t l e as t , t o h a v e a m e a n s b y w h i c h o n e
c a n d e c i d e w h e t h e r a c h o s e n g r a p h i s a p p r o p r i a t e f o r
fo rm in g a topo logy prese rv ing m ap of a g iven m ani fo ld .
A n a p p r o a c h o f t h e f i rs t k i n d h a s b e e n p r o p o s e d b y
K o h o n e n a n d c o w o r k e rs ( K a n g a s, K o h o n e n , & L a a k -
s o n e n , 1 9 9 0 ) . T h e i d e a o f t h i s a p p r o a c h i s ( i ) t o d e -
t e r m i n e t h e m i n i m u m s p a n n i n g t re e b e tw e e n t h e p o i n ts
w~ a t d i f fe ren t s t ages o f the i r adap ta t ion p roces s and
( i i ) t o t a k e th e r e s u l t in g m i n i m u m s p a n n i n g t r e e a s th e
graph G tha t i s approp r ia te a t th e re spec t ive st age . Be-
c a u s e t h e w ~ 's a t t h e e n d o f t h e a d a p t a t i o n p r o c e d u r e
a r e d i s t r i b u t e d o n l y o v e r t h e m a n i f o l d M , t h e f i n a l
m i n i m u m s p a n n i n g t r e e w i l l r e f l e c t a t l e a s t s o m e o f
t h e t o p o l o gi c a l p r o p e r t ie s o f M . T h e d e f i n i t i o n o f t h e
m i n i m u m s p a n n i n g t r e e , h ow e ve r, o n l y r e q u i r es a d j a -
cen t ve r t i ces i to be a s s igned to ne ighbor ing loca t ions
w~ , no t v ice ve rsa . P o in te rs wi tha t a re ad jacen t on M
m ight ve ry we l l be long to rem ote ve r t i ces . The re fore ,
excep t fo r spec ia l cases , th i s approach i s no t ab le to
prov ide a pe r fec t ly topo logy prese rv ing m ap . In gene ra l ,
o n l y t h e i n v e rs e m a p p i n g ~ - 1 f r o m G t o M b u t n o t t h e
m a ppin g ~ I, f rom M to G wi l l be ne igh bor hoo d pre -
serving.
A n o t h e r a p p r o a c h o f t h e f i rs t k i n d h a s b e e n i n t r o -
d u c e d b y F r i t z k e ( 1 99 1 ) . H i s a p p r o a c h e m p l o y s t w o -
d i m e n s i o n a l , t r i a n g u l a r c e l l s t r u c t u r e s t h a t a r e d i s t r ib -
u t e d o v e r t h e m a n i f o l d M . B y s e le c ti v e ly a d d i n g a n d
rem o ving ce l ls , based on a h eur i s t i c c r i t e r ion tha t t akes
a ce l l' s ave rage desc r ip t ion e r ro r in t o acc oun t , a g raph
i s f o r m e d t h a t r e s e m b l e s t h e s h a p e o f th e m a n i f o l d M .
A n a p p r o a c h o f t h e s e c o n d k i n d f o r o b t a i n i n g a n
a p p r o p r i a t e g r a p h G h a s b e e n p r o p o s e d b y B a u e r a n d
P a w e l z i k ( 1 9 9 2 ) . T h e y e m p l o y t h e s o - c a l l e d t o p o -
g r a p h i c p r o d u c t a s a m e a n s t o d e t e r m i n e t h e d e g r e e t o
which a pa r t i cu la r g raph ica l s t ruc tu re i s ab le to p rese rve
t h e n e i g h b o r h o o d r e l a t i o n s o f a g i v en m a n i f o l d . B y
tes t ing , fo r exam ple , one - , two- , o r th ree -d im ens iona l
l a t t ices , one can f ina l ly choose the one th a t p rese rves
t h e n e i g h b o r h o o d r e l a ti o n s b e st .
In the nex t s ec t ion we wi l l in t roduce a nove l ap-
p r o a c h t o t h e p r o b l e m o f c o n s t r u c t i n g t o p o l o g y p r e -
s e r v in g m a p s , a n a p p r o a c h o f th e f i r st k i n d b a s e d o n
t h e D e l a u n a y t r i a n g u l a t i o n . B e c a u s e t h e D e l a u n a y
t r i angula t ion connec t s those ve r t i ces i the a s s igned
p o i n t e r s w i o f w h i c h a r e a d j a c e n t b y h a v i n g n e ig h b o r i n g
V o r o n o i p o l y h e d r a V i , th i s s t ruc tu re i s idea l ly su i t ed
f o r t h i s p u r p o se . T h e a p p r o a c h t a k e n i s o p p o s i te t o t h a t
o f K o h o n e n ' s s e l f - o rg a n i z in g f ea t u r e m a p . N o t t h e
graph G bu t the po in te rs w~ a re p respec i f i ed , fo r ex-
a m p l e , b y u s i n g a v e c t o r q u a n t i z a t i o n p r o c e d u r e t h a t
d i s t r ibu te s them over M . S ubsequent ly , the approp r ia te
e d g e s b e tw e e n t h e w ~ 's a r e c o n s t r u c t e d f o r f o r m i n g a
graph G tha t de f ines a pe r fec t ly topo logy prese rv ing
m a p o f t h e g i ve n m a n i f o l d M .
3 1 Definit ion of Topology Preserving M aps
I n t h e p r e v i o u s s e c t i o n th e d e f i n i t i o n o f n e i g h b o r h o o d
p r e s e r v a t io n o f a m a p p i n g ~ f r o m M t o G o r o f a n
inve rse m a ppin g ~ l f rom G to M was , l ike in m o s t
con t r ibu t ions on topo logy prese rv ing fea tu re m aps , no t
r i g o ro u s . T h e p r o b l e m i s t h a t a d j a c e n c y o f v e r t ic e s i
in a g raph G i s c lea r ly de f ined , a p rope r de f in i t ion fo r
ad jacency o f po in te rs w~ on M , however, i s no t obv ious .
T h i s i s w h y t h e d e f i n i t io n o f n e i g h b o r h o o d p r e s e r v at i o n
o f f e a t u r e m a p s h a s u s u a l l y b e e n l e f t to t h e r e a d e r ' s
i n t u i t i o n .
An excep t ion i s the t r iv ia l one -d im ens iona l case .
Obvious ly , two po in t s w~ , wj in ~ t a re ne ighbor ing i f
t h e r e i s n o p o i n t w k i n b e t w e e n . E x p r e s s e d i n t e r m s o f
Voronoi po lyhedra , an equ iva len t de f in i t ion i s : two
poin t s w~ wj in ~ a re ne igh bor ing i f the i r Voro noi po ly-
hedr a a re ad jacen t , tha t i s , i f V~ f ') ~ ~ . In these
t e r m s a g e n e r a l i z a t i o n t o h i g h e r - d i m e n s i o n a l e m b e d -
d ing spaces ~o i s s t ra igh t fo rw ard . However , because we
a r e i n t e r e s t e d i n a d e f i n i t i o n o f n e i g h b o r h o o d o f p o i n t s
o n a m a n i f o l d M ,
w e i n t r o d u c e t h e
m a s k e d F o r o n o i
p o l y h e d r o n . T h e m a s k e d V o r o n o i p o l y h e d r o n F IM) is
the pa r t o f V~ tha t i s a l so pa r t o f M, tha t i s , V IM) =
F~ f~ M . T he sup e rsc r ip t ind ica te s the dep enden ce o f
t h e m a s k e d V o r o n o i p o l y h e d r o n o n t h e g i v e n m a n i f o l d
M . B y u s in g th e n e i g h b o r h o o d o f th e m a s k e d V o r o n o i
V M) V~ M)
p o l y h e d r a _ i , i n s t e a d o f t h e n e i g h b o r h o o d o f
t h e V o r o n o i p o l y h e d r a V i, V j f o r d e t e r m i n i n g n e i g h -
b o r h o o d o f p o i n t s w ~, w j o n M , w e e n s u r e t h a t t w o
poin t s wi , wj a re ca l l ed a d j a c e n t o n M o n l y i f t h e y d o
n o t b e l o n g t o d i s c o n n e c t e d r e g i o n s o f M . T h i s l e a d s to
the fo l lowing de f in i t ion :
DEFINITION 1.
L e t M ~_ .~ D b e a g i v e n m a n i f o l d a n d S
= {w~ . . . . . WN} b e a s e t o f p o i n t s w ~ E M . T h e V o ro no i
p o l y h e d r a o f S a r e d e n o t e d b y V ~ , i = 1 . . . . N . T w o
p o i n t s w ~ , w j ~ M ~_ ~ D a r e a d j a c e n t o n M i f t h e i r
M) M)
m a s k e d V o r o n o i p o l y h e d r a V ~ = Vi f q M , V ~ =
V . . . . M) M) j
0 M a r e a d ) a c e n t , t h a t i s , i J V ~ a n d V . s h a r e
a n e l e m e n t v E M o r , e q u i v a l e n t l y , i f V ~ M) f ~ v J M )
; ~ i s v a l i d .
E a c h m a s k e d o r o n o i p o l y h e d r o n i s a p a r t o f t h e m a n -
i f o ld M , a n d , i n s t e ad o f f o r m i n g a c o m p l e t e p a r t i t i o n i n g
o f th e e m b e d d i n g s p a c e ~ D l ik e t h e V o r o n o i p o l y h e d r a ,
t h e m a s k e d V o r o n o i p o l y h e d r a f o r m a c o m p l e t e
p a r t i t i o n i n g o n l y o f t h e m a n i f o l d M , t h a t i s , M - -
v (M) is valid.
N-I -- i
A n a l o g t o t h e d e f i n i t i o n o f t h e D e l a u n a y t r i a n g u -
la t ion ~gs , which i s based on the Voronoi po lyhedra ,
-
7/25/2019 Martinet z 1994
9/16
Topology Representing Networks 515
we define the
induced De la una y t r iangula t ion ~9 sM)
through the masked Voronoi polyhedra:
DEFINITION 2.
L e t M ~_ ~o be a g iven man i fo ld and S
= {wz . . . . . WN}
b e a s e t o f p o i n t s w i E M . T h e i n d u c e d
Delaunay tr iangulat ion ~9~sM) o f S , g iven M , i s de te r -
m i n e d b y t h e g r ap h t h a t c o n n e c ts t w o p o i n t s
wz, wj/ff
their m as ke d Voronoi polyh edra V ~M), V~.M) are ad-
j a c e n t , t h a t i s , b y t h e g r a p h w h o s e a d ja c e n c y m a t r i x
A , A o E { O, 1 } , i , j = 1 . . . . N has the proper ty
A~j = 1 =~ V
-
7/25/2019 Martinet z 1994
10/16
5 16 7". Ma rtinetz an d K. Schulten
m a p p i n g f r o m M t o G n o r t h e in v e rs e m a p p i n g f r o m
G to M i s n e i g h b o r h o o d p r e s e rv i n g . I n F i g u r e 5 d t h e
g r a p h G i s t h e i n d u c e d D e l a u n a y t r ia n g u l a t i o n o f t h e
s e t o f p o i n t s w i a n d i s a g a i n a s u b g r a p h o f t h e D e l a u n a y
t r ia n g u l a t i o n i n ( a ) . W i t h t h e in d u c e d D e l a u n a y t r i a n -
g u l a t i o n t h e m a p p i n g f r o m M t o G a s w el l a s t h e i n v e rs e
m a p p i n g f r o m G t o M i s n e i g h b o r h o o d p r e s er v i n g . T w o
v e r t ic e s a r e c o n n e c t e d b y a n e d g e i f f t h e i r m a s k e d V o -
r o n o i p o l y g o n s a r e a d j a c e n t . O n l y i n F i g u r e 5 d d o e s
t h e g r a p h G f o r m a p e r f e c tl y t o p o l o g y p r e s e r v in g m a p
o f t h e g i ve n m a n i f o l d M .
I t i s n o t s u r p r i s i n g t h a t t h e D e l a u n a y t r i a n g u l a t i o n
a n d t h e m i n i m u m s p a n n i n g t r e e in g e n e ra l d o n o t d e -
f i n e a p e r f e c t l y t o p o l o g y p r e s e r v i n g m a p o f M . B o t h
g r a p h s a r e d e t e r m i n e d o n l y b y t h e s e t o f p o in t s S =
{ w l . . . . . W N } . T h e t o p o l o g y o f th e m a n i f o l d M i n f lu -
e n c e s b o t h g r a p h i c a l s t r u c t u r e s o n l y v e r y i n d i r e c t l y .
O n l y t h r o u g h t h e c o n d i t i o n t h a t e a c h w i l ie s o n M d o e s
t h e g i v e n m a n i f o l d s h a p e t h e D e l a u n a y t r i a n g u l a t i o n
o r t h e m i n i m u m s p a n n i n g t r e e . T h i s i s d i f f e re n t fo r t h e
i n d u c e d D e l a u n a y t r ia n g u l a ti o n . T h e i n d u c e d D e l a u -
n a y t r ia n g u l a t io n i s d e f i n e d th r o u g h t h e a d j a c e n c y o f
t he m aske d Vo r ono i po l yh edr a V ~M ) = Vj 71 M . Be-
c a u s e e a c h m a s k e d V o r o n o i p o l y h e d r o n i s c o m p l e t e l y
a p a r t o f M , t h e a d j a c e n c y o f t h e s e p a r t s c a r r i e s im -
m e d i a t e i n f o r m a t i o n a b o u t t h e t o p o lo g i c al s t ru c t u r e o f
t h e m a n i f o l d M .
T h e a b o v e c o n s i d e r a t i o n s a n d d e f i n i t io n s l e a d t o th e
f o l lo w i n g t h e o r e m :
THEOREM 2. L e t G be a graph (ne t work ) w i t h ver t i ces"
( n e u r a l u n i ts ) i , i = 1 . . . . N a n d e d g e s ( c o n n e c t i o n s )
de f i ned b y an ad j acen cy m at r i x A , A o E { O , 1 } . L e t
M c_ ~ D b e a g i v e n m a n i f o l d o f a D - d i m e n s i o n a l
e m b e d d i n g s p a c e a n d S = {W l . . . . . W N} b e a s e t o f
po i n t e r s w i E M each o f wh i ch i s a t t ac hed t o a ver t ex
i o f t h egr ap h G . T heg rap h G w i t h i t s vert ices i as s i gned
t o t he l oca t i ons w~ on M f or m s a per f ec t ly t opo l ogy pre -
s e r v i n g m a p o f M , i f f t h e g r a p h G i s th e i n d u c e d D e -
lau na y t r iang ulat ion 2D (sM) o f S .
P roo f . T h e p r o o f is s t r a ig h t f o r w a r d . I f G i s t h e i n d u c e d
D e l a u n a y t r i a n g u l a t i o n ) (sM ) o f S , ve r t i ces i , j t ha t a r e
a d j a c e n t i n G ( A 0 = 1 ) a re a s s i g n e d t o m a s k e d V o r o n o i
p o l y h e d r a V IM), V } M) t h a t a r e n e ig h b o r i n g o n M , a n d ,
V(M) V~M ) ha t
i c e v e r sa , m a s k e d V o r o n o i p o l y h e d r a _ i ,
a r e n e i g h b o r i n g o n M b e l o n g t o v e r t i c e s i , j t h a t a r e
a d j a c e n t in G . T h e n t h e m a p p i n g q ' s f r o m M t o G a s
w e l l a s t h e i n v e r s e m a p p i n g q , ~ l f r o m G t o M a r e
n e x g h b o r h o o d p r es e r v in g , a n d , h e n c e , t h e i n d u c e d D e -
" Z ~M r m
a u n a y t r i a n g u l a t i o n s o f S f o s a p e r f e c t l y t o -
p o l o g y p r es e r v in g m a p o f M .
I f t h e g r a p h G f o r m s a p e r f e c t l y t o p o l o g y p r e s e r v i n g
m a p o f M , t h e n th e m a p p i n g ~ s f r o m M to G as we l l
a s t h e i n v e r s e m a p p i n g ~ f r o m G t o M a r e n e i g h -
b o r h o o d p r e s e r v i n g . T h i s i s v a l i d i f v e r t i c e s i , j t h a t a r e
a d j a c e n t i n G (A ~j = 1 ) a r e a s s i g n e d t o m a s k e d V o r o n o i
p o l y h e d r a V IM), V~M ) t h a t a r e n e i g h b o r i n g o n M , a n d
(M) (g)
i f m a s k e d V o r o n o i p o l y h e d r a V~ , V j t h a t a r e
n e i g h b o r i n g o n M b e l o n g t o v e r t ic e s i, j t h a t a r e a d -
j a c e n t i n G . A c c o r d i n g t o t h e d e f i n i t i o n , G i s t h e i n -
d u c e d D e l a u n a y tr i a n g u l a t i o n )( sM) o f S .
3.2 . Competitive Heb bian Rule Forms Perfect
Topology Preserving Maps
T h e c o m p e t i t i v e H e b b r u l e c o n s t r u c t s t h e f u l l D e -
l a u n a y t r i a n g u l a t i o n o f a s e t o f p o i n t s W l . . . . , W N o n l y
i f e a c h V o r o n o i p o l y h e d r o n o f s e c o n d - o r d e r V0 is, at
l eas t pa r t i a ll y , cove r ed by t he dens i t y d i s t r i bu t i on P ( v ) .
I f w e d e f i n e t h e m a n i f o l d M a s b e in g t h e m a n i f o l d o f
3~D o n w h i c h P ( v ) i s n o n z e r o , t w o u n i ts i , j b e c o m e
co nn ec ted i f /V~j 71 M 4= ~ . H e n c e , i f t h e m a n i f o ld M
f o r m s a s u b m a n i f o ld t h a t d o e s n o t c o v e r e a c h V o r o n o i
p o l y h e d r o n o f s e c o n d o r d er , th e D e l a u n a y t r i a n g u l a t i o n
w i ll e v o l v e o n l y p a r t l y . I f th e d i s t r i b u t i o n o f t h e p o i n t s
wi i s dense on M i n a s ense we wi l l de f i ne be l ow, t he
p a r t o f t h e D e l a u n a y t r ia n g u l a t io n t h a t i s f o r m e d b y
t h e c o m p e t i t iv e H e b b r u l e is th e i n d u c e d D e l a u n a y
t r i a n g u l a t i o n a n d , h e n c e , p r o v i d e s a p e r fe c t l y t o p o l o g y
p r e s e rv i n g m a p o f M .
DEFINITION 6. L e t S = { Wl . . . . . W N } b e a s e t o / p o i n t s
w i t ha t are d i s t r i bu t ed over a g iven m an i f o l d M c_ ~ " .
T h e d is t r ib u t i o n o f t h e p o i n t s w i E M , i = 1 . . . . N i s
d e n s e o n M , i f f o r e a c h v E M t h e t ri a n g l e A(v, W,o,
w ~, ) f o r m e d b y t h e p o i n t w~ t ha t i s c l osest t o v , t he po i n t
w ~, t ha t i s s econd c l oses t t o v , an d v i t s e l f l i es co mpl e t e l y
o n M , t h a t is , i fA(v, w~ ' Wil
~
M is val id.
A d i s t r i b u t i o n o f p o i n t s w i is d e n s e o n M a c c o r d i n g
t o t h e a b o v e d e f i n i t io n , i f t h e d i s t r i b u t i o n i s d e n s e c o m -
p a r e d t o t h e s t r u c t u r a l d e t a i l s o f M . A d e n s e d i s t r i -
bu t i o n o f t he po i n t s w i r e so lves t he t opo l og i ca l s t r uc t u r e
o f M . I f f o r e a c h s a m p l e p o i n t v E M t h e r e i s a c l o s e st
p o i n t w i0 and a s eco nd c l oses t po i n t w~Ls u c h , t h a t t h e
t r i ang l e A( v , W ~o, w i, ) l ie s com pl e t e l y on M , t he d i s t r i -
b u t i o n o f th e w ~ ' s i s d e n s e o n M a c c o r d i n g t o D e f i n i t io n
6 . I f t h e d i s t r i b u t i o n is h o m o g e n e o u s , t h e d i s t r i b u t i o n
b e c o m e s d e n s e s i m p l y b y i n c r e a si n g th e n u m b e r N o f
points w~.
W i t h D e f i n i t i o n 6 w e o b t a i n t h e m a i n t h e o r e m o f
t h i s s ec t i on :
THEOREM 3. L e t S = { w~ . . . . . W N} b e a s e t o / p o i n t s
w i t h a t a r e d i s tr i b u t e d o v e r a g i ve n m a n i f o l d M . I f t h e
d i s t ri b u t io n o f t h e p o i n t s w i E M i s d e n s e o n M , t h e n
t h e g r a p h G t h a t i s o r m e d b y t h e c o m p e t i ti v e H e b b r u l e
i s t he i nduced De l aunay t r i angu l a t i on ) ( sM) o f S and ,
h e n c e , f o r m s a p e rf e c tl y t o p o l o g y p r e s e r v i n g m a p o f M .
P roo f . A n a l o g t o T h e o r e m 1 w e p r o v e th e a b o v e t h e o -
r e m b y s h o w i n g th a t
vC~t) " (M) 4: ~23 ,: , v M ) -~ S2~ ( 16)
0 Vj - -u
M)
is valid , w ith V ~j = V o (-1 M a s t h e m a s k e d V o r o n o i
p o l y h e d r o n o f s e c o n d o r d e r.
If v-"~M)i V/ _iVM) 4= ~ i s val id, the re i s a v E M wi th
v E Vi
an d v ~ ~ . Th en we obt ain I[v - w~[[ = [ Iv -
wi ll -< [Iv - wk[[ for a l l wk E S , and , th ere fore , v
V ~y) is v alid.
-
7/25/2019 Martinet z 1994
11/16
Topology Representing Networks
v M) q= ~ i s v a l id , t h e r e i s a v * ~ V ~ ) w i t h
f - i j
a ( v * , w i , w j ) __ M , b e c a u s e t h e d i s t r i b u t i o n o f t h e
p o i n t e r s w ] . . . . . W N i s d e n s e o n M . F o r e a c h v E
V {/jM) he po i n t s w i and wj a r e t he t wo nea r es t ne i ghbor s .
W i t h o u t l o s s o f g e n e r a li t y w e a s s u m e t h a t f o r v * t h e
p o i n t w i is t h e n e a r e s t n e i g h b o r . B e c a u s e f o r e a c h u E
v * w j t h e p o i n t w j i s e i t h e r t h e n e a r e s t o r t h e s e c o n d
n e a r e s t n e i g h b o r o f u , a n d b e c a u s e f o r u = v * t h e p o i n t
w i is c l oses t an d f o r u = w j the p o i n t w j is c loses t t o u ,
t he r e i s a u* E v*w j f o r wh i ch [ [u* - w i ll = I I n * -
w i l l
i s v a l i d . H e n c e , w e o b t a i n u * E Vi , u* E Vj, and
u* E A( v* , w i , w j ) c_ M , wh i ch y i e l ds M f q ( V i fq V j )
. V M ) : ~ .
~: ~ or , equ ivale nt ly, V IM ) A _ j
W i t h t h e t h e o r e m a b o v e w e h av e s h o w n t h a t t h e
c o m p e t i ti v e H e b b r u l e f o r m s p e r f e c t ly to p o l o g y p r e -
s e r v i n g m a p s , s u p p o s e d t h e d i s t r i b u t i o n o f t h e p o i n t s
w i i s d e n s e o n t h e g i v e n m a n i f o l d . I n F i g u r e s 6 a n d 7
w e s h o w t w o s i m u l a t i o n e x a m p l e s . I n F i g u r e 6 t h e
m a n i f o l d M c o n s i s ts o f a t h r e e - d i m e n s i o n a l ( r i g h t p a r -
a l l e l e p ip e d ) , a t w o - d i m e n s i o n a l ( r e c t a n g l e ) , a n d a o n e -
d i m e n s i o n a l ( c i rc l e a n d c o n n e c t i n g l i n e ) s u b m a n i f o l d .
I n it ia l ly , th e p o i n t e r s w i a r e d i s t r i b u t e d r a n d o m l y w i t h i n
t h e s p a c e t h a t e m b e d s M , t h a t i s , w i E M i s n o t v a l i d
n e c e s s a ri ly . T o o b t a i n w i E M f o r e a c h i , i = 1 . . . .
N t h e p o i n t e r s a r e d i s t r ib u t e d b y e m p l o y i n g t h e n e u r a l
g a s a l g o r i t h m . T h e n e u r a l g a s a l g o r i t h m i s a n e f fi c ie n t ,
p a t t e r n d r i v e n v e c t o r q u a n t i z a t i o n p r o c e d u r e a n d l e a d s
t o a h o m o g e n e o u s d i s t ri b u t i o n o f t h e p o i n t e r s wi o n M
( M a r t i n e t z & S c h u l t e n , 1 9 9 1 ; M a r t i n e t z , B e r k o v i c h , &
S c h u l t e n , 1 9 9 3 ) .
S i m u l t a n e o u s l y t o d i s t r i b u t i n g t h e p o i n t e r s t h e c o n -
n e c t i o n s a r e f o r m e d . W i t h e a c h p r e s e n t a t i o n o f a n i n p u t
p a t t e r n v ~ M a c y c le o f t h e c o m p e t i t i v e H e b b r u l e a s
w e l l a s o f t h e n e u r a l g a s a l g o r i t h m i s p e r f o r m e d . F i g u r e
6 shows t he i n i t ia l s t a te , t he ne t w or k a f t e r 5000 , 10 ,000 ,
1 5 , 0 0 0 , 2 5 , 0 0 0 , a n d a t t h e f i n a l s t a t e a f t e r 4 0 , 0 0 0 a d -
a p t a t i o n s t e p s . T h e p o i n t e r s w i c h a n g e t h e i r l o c a t i o n s
s l o w ly b u t p e r m a n e n t l y a n d , t h e r e f o r e , p o i n t e r s t h a t
a r e n e i g h b o r i n g a t a n e a r l y s ta g e o f t h e a d a p t a t i o n p r o -
c e d u r e m i g h t n o t b e n e i g h b o r i n g a n y m o r e a t a m o r e
a d v a n c e d s ta g e. T h o s e c o n n e c t i o n s t h a t w e r e f o r m e d
i n t h e e a r l y s t ag e a n d a r e n o t v a l i d a n y m o r e i n a l a t e r
s t ag e s h o u l d d i e o u t . F o r t h a t r e a s o n t h e c o m p e t i t i v e
H e b b r u l e n o t o n l y e s ta b l i s h e s b u t a l s o r e f r e s h e s c o n -
n e c t io n s . C o n n e c t i o n s t h a t h a v e n o t b e e n r e f r e sh e d f o r
a w h i l e d i e o u t a n d a r e r e m o v e d . T h e a l g o r i t h m t h a t
r e s u lt s f r o m c o m b i n i n g t h e c o m p e t i ti v e H e b b r u l e w i t h
t h e n e u r a l g a s a l g o r i t h m i s l i st e d i n S e c t i o n 5 .
A t t h e e n d o f t h e a d a p t a t i o n p r o c e d u r e , a f t e r t h e
r a n d o m p r e s e n t a t i o n o f 4 0 , 0 0 0 i n p m p a t t e r n s v E M ,
t h e c o n n e c t i v i t y s t r u c t u r e c o r r e s p o n d s t o t h e i n d u c e d
D e l a u n a y t r i a n g u l a ti o n o f t h e f i n al p o i n t e r d i s t ri b u t i o n
a n d f o r m s a p e r f e c t ly t o p o l o g y p re s e r v i n g m a p t h a t r e -
f le c t s t h e d i m e n s i o n a l i t y a n d t o p o l o g i c a l s t r u c t u r e o f
t h e g i v en m a n i f o l d M .
I n F i g u r e 7 t h e m a n i f o l d M i s a t o r u s . I n t h i s s i m -
u l a t io n e x a m p l e t h e p o i n t e r s w e r e d i s tr i b u t e d o v e r M
i n a p r e p r o c e s s i n g s t ag e , a g a i n b y t h e n e u r a l g a s a lg o -
r i t h m . A f t e r h a v i n g d i s t r i b u t e d t h e p o i n t e r s , t h e c o n -
517
F I G U R E 6 . T h e c o m p e t i t iv e H e b b r u l e t o g e t h e r w i t h t h e n e u r a l
g a s a l g o r i t h m f o r m i n g a t o p o l o g y p r e s e r v i n g m a p o f a t o p o -
l o g i c al l y h e t e r o g e n e o u s l y s t r u c t u re d m a n i f o ld . T h e g i v e n m a n -
i f ol d M c o n s i s t s o f a t h r e e - d i m e n s i o n a l ( r i g h t p a r a l l e l e p i p e d ) ,
a t w o - d im e n s i o n a l ( r e c t a n g l e ) , a n d a o n e - d i m e n s i o n a l ( c i r c le
a n d c o n n e c t i n g l i n e ) s u b s e t . T h e n e u r a l g a s a l g o r i t h m a s a n
e f f i c i e n t i n p u t d r i v e n v e c t o r q u a n t i