clustering of the self-organizing time map
TRANSCRIPT
Neurocomputing 121 (2013) 317–327
Contents lists available at ScienceDirect
Neurocomputing
0925-23http://d
n CorrE-m
journal homepage: www.elsevier.com/locate/neucom
Letters
Clustering of the Self-Organizing Time Map
Peter Sarlin n, Zhiyuan YaoÅbo Akademi University, Department of Information Technologies, Turku Centre for Computer Science, Joukahaisenkatu 3-5, 20520 Turku, Finland
a r t i c l e i n f o
Article history:Received 19 November 2012Received in revised form15 March 2013Accepted 29 April 2013
Communicated by B. HammerThis paper combines the SOTM with clustering and illustrates the usefulness of second-level clustering
Available online 21 May 2013
Keywords:Self-Organizing Time MapCluster analysisVisual dynamic clustering
12/$ - see front matter & 2013 Elsevier B.V. Ax.doi.org/10.1016/j.neucom.2013.04.007
esponding author. Tel. +358 2 215 4670.ail address: [email protected] (P. Sarlin).
a b s t r a c t
This paper extends the use of recently introduced Self-Organizing Time Map (SOTM) by pairing it withclassical cluster analysis. The SOTM is an adaptation of the Self-Organizing Map for visualizing dynamicsin cluster structures. While enabling visual dynamic clustering of temporal and cross-sectional patterns,the stand-alone SOTM lacks means for objectively representing temporal changes in cluster structures.
for representing changes in cluster structures in an easily interpretable format. This provides means foridentification of changing, emerging and disappearing clusters over time. Experiments are performed ontwo toy datasets and two real-world datasets. The first real-world application explores evolutiondynamics of European banks before and during the global financial crisis. Not surprisingly, the resultsindicate a build-up of risks and vulnerabilities throughout the European banking sector prior to the startof the crisis. The second application identifies the cyclicality of currency crises through changes in themost vulnerable clusters.
& 2013 Elsevier B.V. All rights reserved.
1. Introduction
The Self-Organizing TimeMap (SOTM) [1] is a recently introducedmethod for visual dynamic clustering based upon Kohonen's Self-Organizing Map (SOM) [2]. The SOTM can directly be related to theapproach of evolutionary clustering [3], which concerns processingtemporal data by producing a sequence of clustering solutions. Aneffective evolutionary clustering aims to achieve a balance betweenclustering results being faithful to current data and comparable withthe previous clustering result. In this vein, Chakrabarti et al. [3]illustrate that the usefulness of such an approach is fourfold: (i)consistency (i.e., familiarity with previous clustering), (ii) noiseremoval (i.e., a historical consistent clustering increases robustness),(iii) smoothing (i.e., a smooth view of transitions), and (iv) clustercorrespondence (i.e., relation to historical context). The SOTM is avisual approach to evolutionary clustering by providing means to alow-dimensional representation of all three dimensions of data: (i)cross-sectional, (ii) temporal, and (iii) multivariate. This is a commonsetting in many domains, such as financial tasks with data for severalentities, reference periods and explanatory variables. While Chakra-barti et al. [3] propose a framework and measures for determining anoptimal evolutionary clustering, they do not aim at visualizing thecross-sectional cluster structures and temporal trends in an easilyinterpretable format. The SOTM is essentially a series of one-dimensional SOMs on data ordered in consequent time units and
ll rights reserved.
represents both time and data topology on a two dimensional grid ofreference vectors. Hence, the topology preservation of the SOTMpreserves data topology on the vertical direction and time topologyon the horizontal direction.
Since the introduction of the approach, the SOTM has beenapplied to temporal customer segmentation by Yao et al. [4] and toprovide an abstraction of the ongoing global financial crisis bySarlin [5]. These case studies have, however, shown the inherentneed for guidance in assessing cluster structures and their tem-poral evolvement. Previous works [1,4,5] have paired the SOTMwith a number of visualization aids. The distance structure of aSOTM has been illustrated using U-matrix [6] visualizations,where a color code between all neighboring units indicates theiraverage distance. An adapted version of the cluster coloring byKaski et al. [7] based upon Sammon's mapping [8] has been usedto illustrate the distance structure using perceptual differences incolors (see [1,4,5]). The very standard feature planes have beenused to represent the distribution of values of individual inputs onthe SOTM (see [1,4,5]). To quantify changes in cluster structuresover time, a measure coined structural change has been introduced(see [1,4,5]). Yet, these approaches lack means for objectivelyrepresenting temporal changes in the multivariate clusterstructures.
With stationary data, rows of the SOTM would representsimilar data at different points in time. One ambiguity with theSOTM, and its visualization aids, is, however, that the rows ofunits, when data are non-stationary, may represent data ofdifferent characteristics. Hence, while the stand-alone SOTMillustrates temporal changes, it is difficult to judge whether or
P. Sarlin, Z. Yao / Neurocomputing 121 (2013) 317–327318
not units in a single row are members in a particular cluster,especially when changes in cluster structures are substantial. Thepreviously provided visualizations of multivariate distances on theSOTM with cluster coloring and the U-matrix do not provide anobjective means for identification of changes in cluster structures.In the SOM literature, a common approach to grouping units of aSOM is so-called two-level clustering (e.g. [9,10]), where a SOM,agglomerative hierarchical clustering and partitioning clusteringhave been used to cluster the SOM units. Similarly, the SOTM couldnot only be utilized as a stand-alone dynamic clustering technique,but its output could also be used as input for a subsequent second-level clustering. The second-level clustering of the SOTM, whilebeing conceptually similar, differs from two-level SOMs by cluster-ing both units representing cross-sectional data, as well as unitsrepresenting distinct points in time. Thus, a second-level cluster-ing of the SOTM should be treated as an alternative for a roughlabeling of units and for identification of changing, emerging anddisappearing clusters over time.
The novelty of this paper is to provide means to visualizeevolving cluster structures. We use three types of experiments todemonstrate the usefulness of combining the SOTM with asecond-level clustering. Due to the lack of a comparable evaluationfunction, the paper is short of a quantitative comparison to othermethods. Instead, we illustrate the functioning, output and useful-ness of combining the SOTM with a second-level clustering on anartificial toy dataset with expected patterns. First, we exemplifyhow stationary data may be labeled by the rows of the SOTM, andconfirm it with a second-level clustering. Second, we illustratehow non-stationary data can be processed and show how chan-ging, emerging and disappearing clusters can be identified. Themain rationale for the artificial examples is to illustrate howevolving data can be represented with a second-level clusteringof the SOTM. Also, this illustrates the interpretation of the SOTMand the second-level clusters, as well as defines the types ofdynamics in cluster structures.
The SOTM with a second-level clustering is also applied in tworeal-world settings: gradual changes of European banks andcyclical behavior of currency-crisis indicators. First, we illustratehow the SOTM can be used for visual dynamic clustering offinancial ratios of European banks, and use a second-level cluster-ing to highlight gradual changes in cluster structures over time.While mostly illustrating changes in cluster structures, we can alsoobserve the emergence of new clusters and the disappearance ofold. Second, we use the SOTM to assess cyclical changes in clusterstructures of currency crisis indicators. A center of interest is onillustrating the temporal emergence of clusters, or changes in theirsize, where we mainly assess states vulnerable to a crisis. Thisfollows from the common notion of reoccurring macro-financialvulnerabilities prior to crises that are then triggered by a widerange of unforeseeable events (see e.g. the work by Reinhart andRogoff [11,12]).
This paper is structured as follows. Section 2 introduces theSOTM and clustering methods, as well as techniques for evaluatingthem. In Section 3, we illustrate the rationale behind clustering ofthe SOTM with two simple toy examples. Section 4 applies theSOTM in two real-world settings by assessing changes in clusterstructures in the European banks and indicators of currency crises.Finally, Section 5 concludes by presenting key findings.
2. Literature review
The SOM paradigm [2] has its basis in clustering via vectorquantization and projection via neighborhood preservation, butprovides only a static snapshot of data structures. To introducetime or dynamics, the SOM literature has provided a wide range of
adaptations. Following the categorization of [1], there are fourtypes of approaches for temporal processing in SOMs: (i) integrat-ing time into the standard SOM through pre-processing and post-processing; (ii) modifying the activation and learning rule; (iii)modifying the network topology and; (iv) combinatorialapproaches of standard SOMs and other visualization techniques.The following two paragraphs decompose the discussion of thegroups by first focusing on the approaches and then on their aims,where the latter are explicitly compared to the SOTM. In this vein,the literature review herein strives to illustrate the inherentdifferences in approaches and aims when paying regard to time.
The first type of techniques deal with the inclusion of time toSOMs through processing before or after training. For example,Chappelier and Grumbach [13] used tapped delay to form a vectorof time-shifted samples that is subsequently fed into the SOM.Although this approach provides a convenient way for introducingtime into SOMs and is suitable for tasks such as time-seriesclustering, the SOM does not exploit the sequential informationof successive time points. Moreover, it is difficult to determine theoptimal length of the delay. The post-processing approach firsttrains the SOM with no embedding of temporality during thelearning process, where after superimposed information is used toillustrate temporal patterns, such as trajectories [14] to show statetransitions of individual data. However, trajectories can only beused on a limited set of data in order not to clutter the display. Thesecond type of techniques modify the learning or activation rulefor processing data in a supervised manner. The Hypermapstructure [15] modifies learning by decomposing the input vectorsinto a past and a future vector, of which the past is used formatching and the future part is only included in the update. Therationale for preferring the SOM over alternative predictive meth-ods is the inherent local modeling property and topology pre-servation of units that enhances interpretability of dynamics (seee.g., [16]). Moreover, the activation is adapted by, for instance,constraining matching to the neighborhood of the previous BMU[17].The third type of techniques deal with time by modifying thenetwork topology through feedback connections. The feedbackSOMs have their basis in the seminal Temporal Kohonen Map(TKM) [18] that performs leaky integration to the outputs of theSOM. While the TKM can memorize the magnitude of the outputof the units, it cannot capture the direction of errors. TheRecurrent SOM (RSOM) [19] improves the TKM by moving theleaky integration from the output units to the input vectors. Thistype of techniques have advantage over the first type of techni-ques (i.e., pre-processing of the SOM) in that they do not requirethe user to specify the length of the time series. The fourth, andlast, type of approaches combine SOMs and various visualizationtechniques with the aim of better spatiotemporal visualization. Forinstance, Guo et al. [20] integrate SOMs, reorderable matrices andparallel coordinate plots in order to better visualize multivariate,spatial and temporal patterns.
With the objective of a visual approach to the above discussedevolutionary clustering [3], the herein reviewed SOM-basedapproaches come with inherently different aims. The extensions,while dealing with temporal transitions and dependencies andmainly focusing on time-series prediction or clustering, are notdirectly applicable for evolutionary clustering in general and visualdynamic clustering in particular. The aims of the first group ofextensions are twofold: to cluster time-series patterns in data or tovisualize temporal transitions of only limited number of samples.One example of embedding time beyond the standard tapped-delay approach is the integration of a dissimilarity measure into astandard SOM for comparing time sequences and clusteringtrajectories [21]. Yet, the approach lacks a focus on visualizinghow cluster structures evolve over time (i.e., between two con-secutive time points). The second group of extensions simply aims
P. Sarlin, Z. Yao / Neurocomputing 121 (2013) 317–327 319
at prediction, including both time-series forecasts or categoricalclassifications, but obviously also exhibit the visualization proper-ties of standard SOMs, as well as the possibility to make use oftrajectories. For instance, Sarlin and Marghescu [22] apply thestandard SOM to data with a temporal and cross-sectional dimen-sion, but include time through both decomposed input vectors forprediction and trajectories for exploring dynamics of individualdata. They do not, however, illustrate dynamics beyond the task ofprediction and individual trajectories. While the third group isbroad in nature, the aims of the methods eventually boil down toan attempt to generalize self-organization to time. With an aim oftime-series prediction, feedback SOMs can be related to ARMA-type forecasting models. In comparison to more traditionalapproaches, such as windowing techniques to split data intoshorter vectors, feedback SOMs exhibit the advantage of keepingtrack of the temporal context. Yet, this task is inherently differentfrom that of visualizing dynamics in multivariate cross-sectionaldata. The fourth group aims at improving the visualization cap-abilities of the SOM by combinations with other techniques. Theapproaches in this group, such as Guo et al. [20], focus more onvisualization aids for addressing the spatial dimension, as well asits evolution, whereas dynamics of cluster structures are largelyneglected.
Until recently, comparing standard two-dimensional SOMs foreach time unit, as proposed in [23–25], has been the best availablemethod for assessing dynamics in cross-sectional structures.Denny et al. [24,25] enhance interpretability for comparing dataat two points in time by applying specific initializations andvisualizations to two-dimensional SOMs. The method, whileindeed having merit for detailed comparisons of two datasets,involves complex comparisons of two-dimensional SOMs, has thedrawback of possible instability in orientation and does not enableanalysis of a large number of periods. Visualizing temporalchanges in data structures in a manner that disentangles timefrom cross-sectional structures has thus to be dealt with differ-ently. With the aim of complementing the four properties ofevolutionary clustering by means of a visual approach, thisparticular issue is addressed by the recently introduced SOTM[1], in which no restriction is set on the number of analyzedperiods. Yet, the visualization approaches used in combinationwith the SOTM lack means for objectively representing temporalchanges in the cluster structures (i.e., feature planes, clustercoloring and structural change [1,4,5]). This motivates not onlyto use the SOTM as a representation of temporal changes in cross-sectional data, but also as an input for a subsequent second-levelclustering. Thus, the clustering of the SOTM should be treated as ameans for identifying how clusters change, emerge and disappearover time.
3. Methodology
This section presents the SOTM and stand-alone clusteringmethods, as well as techniques for quantitative evaluation ofclustering results.
3.1. Self-Organizing Time Map
The SOTM [1] uses the capabilities of the classical SOM forabstraction of changes in data structures over time. To observe thecross-sectional structures of the dataset for each time unit t(where t¼1, 2,…, T), the SOTM uses one-dimensional SOMs pertime unit t. More precisely, the SOTM maps from the input spacexj(t)∈Ω(t) (where j¼1, 2,…, N(t)), with a probability densityfunction p(x,t), onto a one-dimensional array A(t) of output unitsmi(t) (where i¼1, 2,…, M). For t¼1 the first principal component
of principal component analysis (PCA) is used for initializing A(t)in that the reference vectors of the units for t¼1 are determinedfrom the subspace spanned by the first principal component onthe input data Ω(t1) (i.e., the eigenvector of the first principalcomponent). The SOTM uses short-term memory to preserve theorientation between consecutive arrays A(t). The short-termmemory refers to reference vectors of A(t) being initialized bythose of A(t−1). Adjustment to temporal changes is achieved byperforming a batch update per time t. The arrays A(t) are thenturned into a timeline by arranging them in an ascending order oftime t. Hence, one can observe twofold topology preservation: thevertical direction preserves data topology and the horizontalpreserves time topology. Matching is performed by min||x(t)−mi(t)|| and the batch update by
miðtÞ ¼ ∑NðtÞ
j ¼ 1hicðtÞxjðtÞ= ∑
NðtÞ
j ¼ 1hicðtÞ
where c is a best-matching unit (BMU) and the neighborhoodhicðtÞ is defined as a Gaussian function restricted to verticalrelations. For a comparable timeline, the neighborhood is constantover time. The functioning of the SOTM can be summarized asfollows.
t¼1
initialize A(t) using PCA on Ω(t)apply the batch update to A(t) using Ω(t)while toT
t¼t+1initialize A(t) using mi(t) of A(t−1)apply the batch update to A(t) using Ω(t)
endorder the one-dimensional arrays A(t) in an ascending order oftime t.
The final step of the above algorithm assures, and clarifies, thatthe SOTM has two interpretable dimensions: the horizontal axisrepresents time topology and the vertical data topology.
In this paper, we represent the multidimensionality of theSOTM using feature planes, as is common in the SOM literature.They show the distribution of values for each variable on theSOTM grid and enable assessing the temporal evolution of cross-sectional distributions, as the timeline below the figure representsthe time dimension in data (see e.g. Fig. 2). The feature planes areproduced using a ColorBrewer's [26] scale, where variation of ablue hue occurs in luminance and light to dark represent low tohigh values. On each feature plane, the differences of the unitsalong the vertical direction show differences in cross-sections,while differences along the horizontal direction show differencesover time. This allows a twofold interpretation of patterns: (i)cross-sectional relations and distributions and (ii) changes incross-sectional patterns over time.
Characteristics of a SOTM can be quantified by a number oftime-restricted quality measures from the SOM paradigm. Wefocus in this paper on quantization error εqe(t), distortion measureεdm(t), and topographic error εte(t) (for details see [1])
εqe ¼1T
∑T
t ¼ 1
1NðtÞ ∑
NðtÞ
j ¼ 1jjxjðtÞ−mcðjÞðtÞjj
εdm ¼ 1T
∑T
t ¼ 1
1NðtÞ
1MðtÞ ∑
NðtÞ
j ¼ 1∑MðtÞ
i ¼ 1hicðjÞðtÞjjxðtÞj−miðtÞjj2
εte ¼ 1T
∑T
t ¼ 1
1NðtÞ ∑
NðtÞ
j ¼ 1uðxjðtÞÞ
P. Sarlin, Z. Yao / Neurocomputing 121 (2013) 317–327320
where u(xj(t)) is the average proportion of xj(t)∈Ω(t) for which firstand second BMUs (within A(t)) are non-adjacent units.
3.2. Cluster analysis
The SOTM, which is essentially a series of one-dimensionalSOMs on data ordered in consequent time units, represents timeand data topology on a two dimensional grid. While visualizationaids may facilitate in interpreting structures of the SOTM, e.g.feature planes and cluster coloring, a large number of dimensionsand units in a SOTM complicate our ability to perceive structuresin data, not to mention large temporal changes. Thus, we turn to asecond-level clustering for identifying changing, emerging anddisappearing clusters on the SOTM. More precisely, the outputunits produced by the SOTM are used as an input for classicalcluster analysis. Then, on the SOTM, we illustrate clusters with aqualitative color scheme from ColorBrewer [26], where clustersare differentiated in hue contrast with nearly constant saturationand lightness.
3.2.1. Clustering techniquesClustering is a class of techniques that partition data into
groups attempting to minimize intra-cluster similarity and max-imize inter-cluster similarity. The existing clustering algorithmscan be generally classified into the following categories: partition-ing clustering, hierarchical clustering, density clustering, gridclustering and model-based clustering. Since the original datahave been reduced by the SOTM, the general problems of compu-tational cost and uncertainty of the clustering result caused byoutliers and noisy data are decreased. In this study, agglomerativehierarchical clustering will be used to group the SOTM units. Thekey motivation for using hierarchical clustering, rather than forexample k-means clustering, is that this enables us to circumvent adefinite choice of number of clusters K. Instead it provides us themeans to explore the clustering results with varying K.
Agglomerative hierarchical clustering starts by treating eachunit of the SOTM as a separate cluster (K¼MT) and mergesiteratively clusters with the shortest distance until all units aremerged (K¼1). Merging can be performed using several defini-tions of distance, e.g. single-linkage, complete-linkage, averagelinkage or Ward's method. In single-linkage, the distance betweentwo clusters is determined by the distance of the closest pair ofobservations in the two clusters. In complete-linkage, the distanceis based upon the distance of the most distant pair. Unlike single-linkage and complete linkage that form clusters based on singleobservation pairs, average-linkage focuses on the inter-clusterdistance by averaging the distances of all pairs of the two clusters.On the contrary, Ward's method attempts to minimize the var-iance of the merged cluster. As hierarchical clustering methods arewell-known, readers interested in details are referred to [27] forfurther details. Clusters of single-linkage tend to take the form oflong chains and other irregular shapes with little homogeneity,whereas complete-linkage has been shown to oftentimes beinefficient in separating data [28,29]. In seminal literature, Ward'smethod was shown to be accurate by outperforming otherhierarchical methods [29–31]. We experimented with single,complete and average-linkage and Ward's hierarchical clustering,but found the results to be qualitatively similar. Hence, supportedby the previous literature, we report in this paper only the resultswith the Ward's method. We use the following Ward's [32]criterion for merging clusters:
dkl ¼nknl
nk þ nlck−cl j2;
������
where k and l represent two clusters, nk and nl the number ofreference vectors mi in the clusters k and l, and ||ck–cl||
2 the squaredEuclidean distance between the cluster centers of clusters k and l.
This divides the SOTM units into a number of clusters, but weneed precise definitions of the changes to properly assess howcluster structures evolve. Hence, we define the three types ofdynamics in cluster structures as follows: (i) a cluster disappearswhen one or more units are a member of it in time t and none is int+1, (ii) a cluster emerges when no unit is a member of it in time tand one or more are in t+1, and (iii) a cluster changes when the(positive) number of units being a member of it in time t and t+1differ.
3.2.2. Validation measuresThe clustering validity is assessed by the Dunn index [33] and
the Silhouette coefficient [34], both of which take into accountcluster compactness and cluster separation [35]. For both mea-sures, the higher the value is, the better the observations areclustered.
The Dunn index is defined as the ratio of the smallest inter-cluster distance to the largest intra-cluster distance. The Dunnindex is computed as
DK ¼ min1 ≤l ≤K
min1≤k≤Kl≠k
dðCl;CkÞmax1 ≤h ≤K
fðd′ðChÞÞg
8<:
9=;
8>>>>><>>>>>:
9>>>>>=>>>>>;
where K is the number of clusters, d(Cl,Ck) is the distance betweenclusters l and k (inter-cluster distance) and d′(Ch) is the maximumdistance between observations in cluster h (intra-cluster distance).
For each observation i, its Silhouette coefficient is defined as
Si ¼bi−ai
maxðbi; aiÞwhere ai is the average distance between i and all other observa-tions in the same cluster and bi is the average distance between iand the observations in its nearest cluster. The Silhouette coeffi-cient for a clustering solution is simply the average of theSilhouette coefficient of all observations.
4. Some experiments on toy data
This section introduces second-level clustering of the SOTMwith experiments on an artificial toy dataset. The experiments ondata with expected patterns illustrate the usefulness of combiningclustering techniques with the SOTM.
4.1. Toy data
We generate a toy dataset that represents a three-dimensionalcube with one dimension for time, one for cross-sectional entities,and one for input variables. We follow the procedure used in [1],but generate data for the purposes of this paper. The expectedpatterns in data are adjusted by five weights w1–5 that steerrandomized shocks on four levels: group-specific (g), time-specific (t), variable-specific (r) and common (j) properties.Group-level differences represent clusters in data, time-levelproperties temporal trends and group-specific and commonshocks general noise. The data x are generated as follows:
xðr; g; j; tÞ ¼ Eðr; g; tÞ þw4ðr; gÞe4ðr; tÞ þw5ðr; gÞe5ðr; j; tÞand group-specific trends by
Eðr; g; tÞ ¼w1ðrÞe1ðgÞ þw2ðrÞe2ðgÞt þw3ðrÞe3ðg; tÞ
P. Sarlin, Z. Yao / Neurocomputing 121 (2013) 317–327 321
where e1,3–5∼N(0,1) are drawn from a normal distribution and e2∼U(0,1) from a uniform distribution, and weights w1–3 are specifiedby the user for adjusting properties in data. The rationale fordrawing e2 from a uniform rather than a normal distribution is tohave larger variation in the group-specific slopes. Finally, eachvariable xj is transformed into [0,1] through a logistic sigmoidalfunction.
The properties in data are as follows: group-specific intercepts(w1), group-specific slopes over time (w2), magnitude of group-specific random shocks (w3), magnitude of time-specific commonshocks (w4) and magnitude of general common shocks (w5). Fig. 1reports the used weights w1–5 for generating the four variableswith five groups of 20 cross-sectional entities over 10 periods,where the color coding illustrates the groups. Particular character-istics of the below four variables are obviously that the time seriesof x1–2 are stationary and those of x3–4 are non-stationary. Whilewe may not fulfill all conditions of stationarity, e.g. constantvariance and autocorrelation, the aim of these data is to havetwo variables with a somewhat constant data structure and twowith a time-varying and converging structure. That is, the time-varying data are generated such that parts of the groups convergewhereas others diverge. Evident changes of x3–4 in Fig. 1 are, forinstance, that the red group diverges from the purple group andthe blue converges to the green group.
4.2. Toy examples with the SOTM
This section presents two toy examples of the SOTM: one withstationary and one with non-stationary data. The key focus is onillustrating the usefulness of second-level clustering of the SOTM.We use first the stationary data x1–2 as inputs for a SOTM and thenthe non-stationary data x3–4. We specify the SOTM to have 5�10units, where 5 vertical units represent data topology and 10
10.0
0.2
0.4
0.6
0.8
1.0
Time t
0.0
0.2
0.4
0.6
0.8
1.0
0
0
0
0
0
1
x1
w1 = 2 w2 = 0 w3 = 0w4 = 0 w5 = 0.2
w1 = -2 w2 = 0 w3 = 0w4 = 0 w5 = 0.2
x2
2 3 4 5 6 7 8 9 10 1Time t
2 3 4 5 6 7 8 9 10
Fig. 1. Generated artificial toy dataset. Notes: The figure reports the used weights w1–5
illustrates the 5 groups, x axis represents time t and y axis values. (For interpretation of tof this article.)
x1
0.20.40.60.8
1 3 4 5 6 7 8 9 10
0.20.40.60.8
2 1 3 42
Fig. 2. A SOTM applied to toy data with no temporal variation. Notes: The two first gridshows cluster memberships using color coding. Vertical color scales on the left of featurcolor in this figure legend, the reader is referred to the web version of this article.)
horizontal units time topology. The number of units (or clusters)on the vertical axis is set to equal the number of generated groupsin data, while the number of units on the horizontal axis is fixedby the number of time units T. When choosing the final specifica-tion of the SOTM, quality measures introduced in Section 3.1 areused. As the main focus ought to be on topographic accuracy, wechoose a neighborhood s that leads to minimum quantizationerror given no topographic errors. Topographic error is stressed asthe interpretation of a SOTM relies heavily on topology preserva-tion, not least the time dimension.
To the trained SOTM, we apply a second-level Ward's clusteringto the units of the SOTM. As the number of generated groups ispre-defined, we do not compare the performance of different Kusing clustering validation measures. On the stationary variablesx1–2, we obviously set K equal to the number of created groups. Inthe non-stationary case, we use the same K to better illustrate thedifference to the stationary case. However, exploring different Kwould be useful for illustrating properties of the data. Fig. 2 showsfeature planes and cluster memberships for a SOTM on thestationary data x1–2. It is worth noting that the coloring of theclusters follows the coloring of the groups in Fig. 1. Indeed, one canobserve that both the two feature planes and the cluster member-ships are constant over time. This illustrates that stationary datamay be labeled by the rows of the SOTM. Fig. 3, on the other hand,shows feature planes for a SOTM on the non-stationary data x3–4.The feature planes clearly depict the increasing and decreasingtrends as well as differences in densities in the structure. This isalso reflected in the cluster memberships. The feature of approx-imating the probability density functions of data p(x,t) leads to adirect interpretation of memberships; the denser the part of thedata space Ω(t), the higher is the number of units in that location.The convergence of the green, blue and orange groups anddivergence of the red from the purple group (as shown in Fig. 1)are shown as the vertical units in a second-level cluster increase
.0
.2
.4
.6
.8
.0
0.0
0.2
0.4
0.6
0.8
1.0
w1 = 1 w2 = 0.5 w3 = 0.2w4 = 0.1 w5 = 0.1
w1 = -2.1 w2 = 0.5 w3 = 0.2w4 = 0.1 w5 = 0.1
x3 x4
1Time t
2 3 4 5 6 7 8 9 10 1Time t
2 3 4 5 6 7 8 9 10
for generating the x1–4 with 100 entities over 10 periods, where the color codinghe references to color in this figure legend, the reader is referred to the web version
x2 Cluster memberships
12345
5 6 7 8 9 10 1 3 4 5 6 7 8 9 102
s are feature planes and represent the distribution of values of x1–2. The final gride planes link a cluster number to each color. (For interpretation of the references to
x3
0.20.40.60.8
1 3 4 5 6 7 8 9 10
x4
0.20.40.60.8
Cluster memberships
12345
2 1 3 4 5 6 7 8 9 102 1 3 4 5 6 7 8 9 102
Fig. 3. A SOTM applied to toy data with temporal variation. Notes: See notes to Fig. 2. (For interpretation of the references to color in this figure, the reader is referred to theweb version of this article.)
P. Sarlin, Z. Yao / Neurocomputing 121 (2013) 317–327322
and decrease. That is, in Fig. 3, we can observe that cluster 2 (blueunits) disappears in period 7 and that cluster 1 (red units) emergesfrom cluster 4 (purple units) in period 3, as well as cluster 3 (greenunits) and 4 (purple units) change from one vertical unit to twoand from two units to one, respectively.
5. Applications in real-world settings
This section presents two applications of the SOTM, as well asits second-level clustering, in financial settings. The first applica-tion focuses on gradual changes in clusters of European banksand the second on the cyclical changes in indicators of currencycrises.
5.1. An application to European banks
We present in this section a full application of the SOTM, aswell as its second-level clustering, to a dataset of financial ratios ofEuropean banks. The dataset has previously been used in [36,37],in particular for financial benchmarking of high-dimensional dataon a low-dimensional display of the SOM. These models do,however, lack means for assessing changes in cluster structures.The dataset consists of a large number of financial ratios ofEuropean banks measuring asset and capital quality, as well asoperations and liquidity ratios. The data have been retrieved fromBankscope of Bureau van Dijk. The sample comes from all 27 EUcountries, including total 855 banks spanning from 1992 to 2008,with in total 9654 bank-year observations.
5.1.1. A SOTM on European banksThe SOTM is used for an abstraction of a dataset of financial
ratios of European banks before and during the global financialcrisis of 2007–2008. We use a set of 20 financial ratios as trainingvariables, and associate two ratios to the model (Tier 1 and TotalCapital Ratio).1 Association of variables is performed by notincluding them in training but only computing averages for thedata per unit of the SOTM. These particular variables are asso-ciated, due to them being important while having a large numberof missing values that would impair training. The model architec-ture is set to 8�17 units. The 17 horizontal units represent thetime dimension and the 8 vertical units represent cross-sectionalstructures per time point t. The units on the time dimension spanperiods before and during the crisis in 2007–2008 (1992–2008)and equals the number of time units T, while the number of unitsat each point in time is determined based upon its descriptivevalue. It is important to note that the SOTM, likewise the SOM, isnot restricted to treat each unit as an individual cluster. Due to theproperty of approximating probability density functions p(x,t),
1 Tier 1 capital is a ratio used by regulators in assessing a bank's financialstrength and measures a bank's core equity capital to its total risk-weighted assets.The Total Capital Ratio also comprises Tier 2 capital, and provides thus a broadermeasure of the capital base by also including supplementary capital, such asundisclosed and revaluation reserves, loan-loss provisions and subordinated debt.
only the dense locations in the data at time t tend to attract unitsat time t. Quality measures are again used when choosing the finalspecification of the SOTM. For a SOTM with 8�17 units, we chosea neighborhood radius s¼2.0, as it has the highest quantizationaccuracies and no topographic errors (see Fig. 4).
As mentioned in Section 2, we represent the multidimension-ality of the SOTM using feature planes. On feature planes,differences between units along the vertical direction showsdifferences in cross-sections and differences along the horizontaldirection changes over time. This enables assessing two types ofpatterns: (i) cross-sectional relations and distributions and (ii)changes in patterns over time. Below, we summarize some of thepatterns that the feature planes in Fig. 5 reveal. First, while beingtime-varying, one can roughly divide the vertical dimension intodifferent types of banks: the higher the poorer and the lower thebetter. The upper part represents banks with somewhat poorerasset quality, i.e. loan loss provisions (cf. Fig. 6(a)); significantlypoorer capital ratios, i.e. a low share of equity and capital funds (cf.Fig. 6(b–g)), a high share of subordinated debt (cf. Fig. 6(h));poorer operations ratios, i.e. low interest and other than operatingincome (cf. Fig. 6(i–k)), low non-interest expenses (cf. Fig. 6(l)),low pre-tax income (cf. Fig. 6(m)), high non-operating expenses(cf. Fig. 6(n)) and low return on assets and equity (cf. Fig. 6(o–p)),and poorer liquidity ratios, i.e. low interbank ratio (cf. Fig. 6(q)) andmixed results for loan ratios (cf. Fig. 6(r–t)). The loan ratios are notonly mixed as the upper part is characterized by both the highestand lowest values, but also as there is a wave of decreased liquidityin the latter part of the SOTM. The share of low non-interestexpenses to assets (cf. Fig. 6(l)) of the poor banks is somewhatcontrary to their general performance, as it can be considered ameasure of the operating and overhead expenses of a bank relativeto the assets invested. Similarly, then the lower part representswell-performing banks with better asset quality, operations, andliquidity ratios.
Second, to assess temporal changes, one has to observe changeson the horizontal direction. We will divide these into gradual andabrupt changes. Some gradual changes are increases in equity andcapital funds ratios (cf. Fig. 6(b–g)) and subordinated debt (cf.Fig. 6(h)), and decreases in interest income (cf. Fig. 6(i–j)) and non-interest expenses (cf. Fig. 6(l)). The abrupt changes includesignificant decreases of interest income (cf. Fig. 6(i–j)) and theinterbank ratio (cf. Fig. 6(q)) in 1994, and increases in loan ratios(cf. Fig. 6(r–t)) for the mid and lower part of the map in 2004. In2008, we can observe significant changes for several ratios:increases in loan loss provisions (cf. Fig. 6(a)), subordinated debt(cf. Fig. 6(h)), non-interest expenses (cf. Fig. 6(l)), non-operatingexpenses (cf. Fig. 6(n)) and loan ratios (cf. Fig. 6(r–)), and decreasesin other than operation income (cf. Fig. 6(k)), pre-tax income (cf.Fig. 6(m)), returns (cf. Fig. 6(o–p)), and interbank (cf. Fig. 6(q)), Tier1 and 2 ratios (cf. Fig. 6(u–v)). To sum up, while we can observedirect negative effects of the financial crisis in 2008, several ratiosindicate a boom or building up of risks and vulnerabilities duringthe pre-crisis period. However, we can neither objectively labelunits nor identify changing, emerging or disappearing clustersover time.
0.10.00.20.40.60.81.01.21.4
ε q e
/ εte
/ ε d
m
Quantization error εqe (σ) Topographic error εte (σ) Distortion measure εdm (σ)
Radius of the neighborhood σ0.3 0.5 0.7 0.9 1.1 1.3 1.5 1.7 1.9 2.1 2.3 2.5 2.7 2.9
Fig. 4. Quality measures of the SOTM over radius of the neighborhood.
0.4
0.5
0.6
0.7
1992
0.20.30.40.50.60.70.8
0.20.30.40.50.60.70.8
0.20.30.40.50.60.70.8
0.2
0.4
0.6
0.8
0.20.30.40.50.60.70.8
0.2
0.4
0.6
0.8
0.4
0.5
0.6
0.7
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
0.2
0.3
0.4
0.5
0.6
0.7
0.2
0.3
0.4
0.5
0.6
0.7
0.2
0.4
0.6
0.8
0.2
0.3
0.4
0.5
0.6
0.7
0.30.40.50.60.70.8
0.3
0.4
0.5
0.6
0.4
0.5
0.6
0.4
0.5
0.6
0.7
0.4
0.5
0.6
0.7
0.3
0.4
0.5
0.6
0.7
0.8
1015
2025
3035
10152025303540
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
Loan Loss Provisions / Interest Revenue Equity / Loans Equity / Deposits Equity / Liabilities
Capital Funds / Tot Assets Capital Funds / Deposits Capital Funds / Liabilities Subordinated Debt / Capital Funds
Net Interest Margin Interest Revenue / Assets Other Income / Assets Non Interest Expenses / Assets
Pre-Tax Income / Assets Non-Operating Items & Tax / Assets Return On Assets (ROA) Return On Equity (ROE)
Interbank Ratio Loans / Assets Loans / Deposits Loans / Deposits & Borrowings
Tier 1 Capital Ratio Total Capital Ratio
Fig. 5. Feature planes of the bank SOTM.
30.0
0.1
0.2
0.3
0.4
0.5
Number of clusters
SilhouetteDunn
4 5 6 7 8 9 10
Fig. 6. Cluster validation of the second-level clustering of the bank SOTM.
P. Sarlin, Z. Yao / Neurocomputing 121 (2013) 317–327 323
1
2
3
1992
3 Clusters
1
2
3
4
4 Clusters
1
2
3
4
5
5 Clusters
1
2
3
4
5
6
6 Clusters
1234567
7 Clusters
12345678
8 Clusters
12345678
9 Clusters
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
1992
1994
1996
1998
2000
2002
2004
2006
2008
Fig. 7. Cluster membership planes of the bank SOTM. Notes: Cluster memberships are shown using color coding. The vertical color scale on the left of each grid links a clusternumber to each color. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
P. Sarlin, Z. Yao / Neurocomputing 121 (2013) 317–327324
5.1.2. Clustering of the bank SOTMWe apply a second-level clustering to the units of the trained
bank SOTM. As we do not have a pre-defined number of groups inthese data, we use cluster validation measures for evaluating theclustering solutions with different K. We do, however, still vary Kto explore the structures in the dataset, as is commonly done withhierarchical clustering methods. Fig. 6 shows the Dunn index andSilhouette coefficient for K¼3, 4,…, 10. While Dunn index indi-cates that K¼7, 8 is optimal and the Silhouette coefficientindicates that K¼9 is optimal, a general view of both measuresshows only minor differences between different K. The smalldifferences in cluster validation measures motivate to explorethe hierarchical process of agglomerating clusters on the SOTM.The cluster membership planes in Fig. 7 illustrate how clusters areagglomerated when increasing K from 3 to 9. To facilitate theinterpretation of the agglomeration process for an increase in K,we set the colors from the ColorBrewer [26] scheme to be constantfor all clusters except for the splitted one, where one new color isintroduced.
While agglomeration proceeds by decreasing K, we summarizethe process of the SOTM clustering from low to high K forillustrational purposes. First, the 3- and 4-cluster solutions mainlyshow cross-sectional differences in data. However, in the 4-clustersolution, gradual changes show that clusters 2 and 4 (blue andpurple) move and merge upwards. This represents improvedboom-like conditions in the entire European banking sector thatmay have lead to an overheating and build-up of risks andvulnerabilities prior to the peak of the crisis. Second, the 5-cluster solution starts to reveal the abrupt change in the condi-tions during the peak of the crisis in 2008. The downwardmovement of cluster memberships of clusters 1 and 2 (red andblue) shows that bank performance decreased significantly as aresult of the global financial crisis. Third, the 6-cluster solutionillustrates a horizontal agglomeration, or loss of one cluster andemergence of another, where cluster 2 (blue) in the 5-cluster
solution consists of two horizontally separate clusters 2 and 6(blue and yellow) in the 6-cluster solution. Fourth, the 7-, 8- and9-cluster solutions actually represent a decrease in bank perfor-mance prior to the crisis in 2008. The 9-cluster solution revealsthat from 2005 onwards clusters 6 and 8 include units on a lowerrow in addition to only horizontal units. In addition, some clustersre-emerge after disappearing, while some clusters emerge todisappear some periods later. As confirmed by the feature planesin Fig. 5(r–t), these patterns to a large extent can be derived fromchanges in loan ratios. That is, the liquidity of banks startsdecreasing prior to the peak of crisis in 2008. The 10-clustersolution, or above, starts to include detail to the extent thatinterpretation is hindered.
5.2. An application to currency crises
This section turns the attention to cyclical changes in indicatorsof currency crises. We present an application of a SOTM, with itssecond-level clustering, to indicators commonly used to describeand explain the vulnerabilities to a currency crisis. These indica-tors were originally introduced to crisis monitoring in a predictivemodel of the International Monetary Fund by Berg and Pattillo[38]. In [22], a currency crisis map based upon similar data and theSOM was built for visual temporal and cross-sectional surveillanceof risks and vulnerabilities. Again, while holding some promise forpredicting vulnerabilities and visualizing trajectories of econo-mies, these applications have not involved any assessment oftemporal changes in cross-sectional cluster structures. The datasetconsists of five monthly indicators for 23 emerging marketeconomies from 1986:1 to 1996:12 within total 2916 country-month observations: foreign reserve loss, export loss, realexchange-rate overvaluation relative to trend, current accountdeficit relative to GDP, and short-term debt to reserves. To controlfor cross-country differences, we transform each indicator into itscountry-specific percentile distributions. In addition, we use an
P. Sarlin, Z. Yao / Neurocomputing 121 (2013) 317–327 325
exchange market pressure index to date crises, as defined in [38]:a crisis occurs if the sum of a weighted average of monthlypercentage depreciation in the currency and monthly percentagedeclines in reserves exceeds its mean by more than three standarddeviations. Using the crisis occurrences, we define an observationto be in a vulnerability state, or pre-crisis period, if it experienced acrisis within 24 months.
5.2.1. A SOTM on currency crisesThe SOTM is used for an abstraction of macro-financial indica-
tors prior to the Asian financial crisis in 1997. In addition to a set oftraining variables, we associate a class variable to the SOTM,namely the occurrence of pre-crisis periods. The rationale forassociating the class variables is that it enables us to locate risksand vulnerabilities on the SOTM. Model architecture is set to6�11 units. The 11 horizontal units represent the time dimensionwith annual frequency, as monthly data had to be pooled to amore granular solution. Quality measures are again the basis for
304050607080
1986
30
40
50
60
70
203040506070
20
40
60
80
1988 1990 1992 1994 1996 1986 1988 1990
1986 1988 1990 1992 1994 1996 1986 1988 1990
Reserve loss Expo
Current account deficit Short-t
Fig. 8. Feature planes o
1
2
3
1986
3 clusters
1
2
3
4
4 clust
123456
6 clusters
1234567
7 clust1988 1990 1992 1994 1996 1986 1988 1990
1986 1988 1990 1992 1994 1996 1986 1988 1990
Fig. 9. Cluster membership planes of the crisis SOTM. Notes: Cluster memberships are shonumber to each color. (For interpretation of the references to color in this figure legend
choosing a final specification of the SOTM; however, for brevitythey are not reported here. With the highest quantization accura-cies and no topographic errors, we chose a neighborhood radiuss¼2.6.
In this application we will only briefly discuss the properties ofthe SOTM, and rather turn the focus to the clustering results. Still,we provide a short discussion on the interpretation of the crisisSOTM. High values of the pre-crisis class variable throughout thelower part of the SOTM illustrates the location of vulnerable states(cf. Fig. 8(f)). In fact, the pre-crisis variable shows that pre-crises,and thus crises, occur in three waves, which is also confirmed bythe class frequency: one in 1986, one in 1992 and one in 1996. Theindicators are transformed such that a univariate increase leads,on average, to increased risks. Accordingly, reserve loss takeslarger values on the bottom part of the map, whereas there is noclear time trend (cf. Fig. 8(a)). While export loss also shows thatincreases lead to larger vulnerabilities, it illustrates clear increasesduring the most vulnerable times (cf. Fig. 8(b)). Exchange-rateovervaluation again increases with vulnerabilities, but shows a
203040506070
00.10.20.30.40.50.6
1992 1994 1996 1986 1988 1990 1992 1994 1996
1992 1994 1996 1986 1988 1990 1992 1994 1996
rt loss Currency overvaluation
erm debt Pre-crisis
f the crisis SOTM.
ers
1
2
3
4
5
5 clusters
ers
12345678
8 clusters1992 1994 1996 1986 1988 1990 1992 1994 1996
1992 1994 1996 1986 1988 1990 1992 1994 1996
wn using color coding. The vertical color scale on the left of each grid links a cluster, the reader is referred to the web version of this article.)
P. Sarlin, Z. Yao / Neurocomputing 121 (2013) 317–327326
remarkably increasing trend towards the latter part of the sample(cf. Fig. 8(c)). The current account deficits exhibit similar behavior(cf. Fig. 8(d)). Short-term debt, while decreasing with vulnerabil-ities, have an increasing relationship with vulnerabilities towardsthe end of the sample (cf. Fig. 8(e)). However, the short-term debthas, overall, decreased over time.
5.2.2. Clustering of the crisis SOTMThe second-level clustering of the SOTM better reveals the
cyclical nature of the indicators. In the following, rather thandiscussing all details of cluster structures, the main focus is onchanges in the clusters with the highest risks, namely the lowerpart of the SOTM. Hence, we do not report the cluster validationmeasures here, but rather turn directly to the agglomerationprocess. As above and shown in Fig. 9, we apply Ward's hierarch-ical clustering, this time ranging from a 3 to 8-cluster solution dueto fewer units. Already the model with 3 clusters illustrates aninteresting behavior of cluster 3 (green), the high-risk cluster.Cluster 3 exhibits the following changes: it is in the first period a 2-unit solution and decreases then to only include 1 unit, whereasthe second wave of vulnerability attracts 2 units and the thirdwave further increases the solution to include 3 units. While thesolutions up to 7 clusters provide information on cluster structuresin the less risky parts of the SOTM, e.g., changing, emerging anddisappearing clusters, the 8-cluster solution decouples a part fromthe risky cluster (cluster 3). This part (cluster 8) may be inter-preted as even more risky, not the least due to its profile as per thepre-crisis variable (cf. Fig. 8(f)) and the fact that a unit abovecluster 8 is a member of cluster 3.
6. Conclusion
In this paper, we have combined the SOTM with classicalcluster analysis. The rationale for clustering the SOTM is twofold:(i) it provides an objective means for labeling units and (ii) itenables identification of changing, emerging and disappearingclusters over time. We have used two sorts of experiments forillustrating the rationale behind and usefulness of second-levelclustering of the SOTM. First, we used simple toy data fordemonstrating the need for clustering of the SOTM. Second, wepresent applications in two real-world settings: financial ratios ofEuropean banks and indicators of currency crisis. Not surprisingly,the results indicate a build-up of risks and vulnerabilities through-out the European banking sector prior to the start of the crisis. Thesecond application identifies the cyclicality of currency crisesthrough increases in the size of the most vulnerable clusters priorto times of crisis. Thus, we have illustrated with four experimentsthe usefulness of a second-level clustering of the SOTM for betterrevealing the temporal evolution of cluster structures. In addition,the two real-world applications have further confirmed theusefulness of the SOTM in financial settings. This opens the doorfor a wide range of applications to illustrate dynamics in customerdata, firm data, macroeconomic data, financial data, processmonitoring data, etc.
Acknowledgments
The authors are grateful to Barbro Back, Barbara Hammer, andtwo anonymous reviewers for insightful comments and discus-sions, as well as the Academy of Finland (Grant no. 127592) forfinancial support.
References
[1] P. Sarlin, Self-organizing time map: an abstraction of temporal multivariatepatterns, Neurocomputing 99 (1) (2013) 496–508.
[2] T. Kohonen, Self-organized formation of topologically correct feature maps,Biol. Cybern. 43 (1982) 59–69.
[3] D. Chakrabarti, R. Kumar, A. Tomkins, Evolutionary clustering, in: Proceedingsof the 12th ACM SIGKDD International Conference on Knowledge Discoveryand Data Mining, ACM, Philadelphia, PA, USA, 2006, pp. 554–560.
[4] Z. Yao, P. Sarlin, T. Eklund, B. Back, Temporal customer segmentationusing the self-organizing time map, in: Proceedings of the InternationalConference on Information Visualisation (IV 2012), Montpellier, France,pp. 234–240.
[5] P. Sarlin, Decomposing the global financial crisis: a self-organizing time map,Pattern Recognition Letters, http://dx.doi.org/10.1016/j.patrec.2013.03.017,forthcoming.
[6] A. Ultsch, H.P. Siemon, Kohonen's self-organizing feature maps for exploratorydata analysis, in: Proceedings of the International Conference on NeuralNetworks (ICNN 1990), Dordrecht, the Netherlands, 1990, pp. 305–308.
[7] S. Kaski, J. Venna, T. Kohonen, Coloring that reveals cluster structures inmultivariate data, Aust. J. Intelligent Inf. Process. Syst. 6 (2000) 82–88.
[8] J.W. Sammon Jr., A nonlinear mapping for data structure analysis, IEEE Trans.Comput. 18 (1969) 401–409.
[9] J. Kangas, Time-delayed self-organizing maps, in: Proceedings of the Interna-tional Joint Conference on Neural Networks (IJCNN 1990), San Diego, USA,1990, pp. 331–336.
[10] J. Vesanto, E. Alhoniemi, Clustering of the self-organizing map, IEEE Trans.Neural Networks 11 (3) (2000) 586–600.
[11] C.M. Reinhart, K.S. Rogoff, Is the 2007 US sub-prime financial crisis sodifferent? An international historical comparison, Am. Econ. Review 98 (2)(2008) 339–344.
[12] C.M. Reinhart, K.S. Rogoff, The aftermath of financial crises, Am. Econ. Rev. 99(2) (2009) 466–472.
[13] J. Chappelier, A. Grumbach, A Kohonen map for temporal sequences, in:Proceedings of the Neural Networks and Their Application, NEURAP'96,IUSPIM, 1996, pp. 104–110.
[14] T. Kohonen, The “Neural” Phonetic Typewriter, Computer 21 (3) (1988) 11–22.[15] T. Kohonen, The hypermap architecture, in: T. Kohonen, K. Mäkisara, O. Simula,
J. Kangas (Eds.), Artificial Neural Networks, vol. II, Elsevier, Amsterdam,Netherlands, 1991, pp. 1357–1360.
[16] G. Barreto, Time series prediction with the self-organizing map: a review,Perspectives of Neural-Symbolic Integration, 135–158.
[17] J. Kangas, Temporal knowledge in locations of activations in a self-organizingmap, in: I. Aleksander, J. Taylor (Eds.), Artificial Neural Networks, vol. 2, North-Holland, Amsterdam, 1992, pp. 117–120.
[18] G.J. Chappell, J.G. Taylor, The temporal Kohonen map, Neural Networks 6(1993) 441–445.
[19] M. Varsta, J. Heikkonen, J. Lampinen, J.R. Millán, Temporal Kohonen map andthe recurrent self-organizing map: analytical and experimental comparison,Neural Process. Lett. 13 (2001) 237–251.
[20] D. Guo, J. Chen, A.M. MacEachren, K. Liao, A visualization system for space-time and multivariate patterns (VIS-STAMP), IEEE Trans. Visualization Com-put. Graphics 12 (6) (2006) 1461–1474.
[21] P. D’Urso, L.De Giovanni, Temporal self-organizing maps for telecommunica-tions market segmentation, Neurocomputing 71 (13) (2008) 2880–2892.
[22] P. Sarlin, D. Marghescu, Visual predictions of currency crises using self-organizing maps, Intell. Syst. Account. Finance Manag. 18 (1) (2011) 15–38.
[23] B. Back, K. Sere, H. Vanharanta, Managing complexity in large data bases usingself-organizing maps, Acc. Manage. Inf. Technol. 8 (4) (1998) 191–210.
[24] Denny, W. Graham, P. Christen, Visualizing temporal cluster changesusing relative density self-organizing maps, Knowl. Inf. Syst. 25 (2) (2010)281–302.
[25] Denny, D.M. Squire, Visualization of cluster changes by comparing self-organizing maps, in: Proceedings of the Pacific-Asia Conference on KnowledgeDiscovery and Data Mining (PAKDD 2005), Springer, Berlin, 2005, pp. 410–419.
[26] M. Harrower, C. Brewer, Colorbrewer.org: an online tool for selecting colourschemes for maps, Cartographic J. 40 (1) (2003) 27–37.
[27] P.J. Rousseeuw, L. Kaufman, Finding groups in data: an introduction to clusteranalysis, Wiley-Interscience, New York, 1990.
[28] P. Hansen, M. Delattre, Complete-link cluster analysis by graph coloring, J. Am.Stat. Assoc. 73 (362) (1978) 397–403.
[29] R.K. Blashfield, Mixture model tests of cluster analysis: accuracy of fouragglomerative hierarchical methods, Psychol. Bull. 83 (3) (1976) 377.
[30] F.K. Kuiper, L. Fisher, A Monte Carlo comparison of six clustering procedures,Biometrics 31 (3) (1975) 777–783.
[31] R. Mojena, Hierarchical grouping methods and stopping rules: an evaluation,Comput. J. 20 (4) (1977) 359–363.
[32] J.H. Ward Jr., Hierarchical grouping to optimize an objective function, J. Am.Stat. Assoc. 58 (301) (1963) 236–244.
[33] J.C. Dunn, A fuzzy relative of the ISODATA process and its use in detectingcompact well-separated clusters, J. Cybern. 3 (3) (1973) 32–57.
[34] P.J. Rousseeuw, Silhouettes: a graphical aid to the interpretation and valida-tion of cluster analysis, J. Comput. Appl. Math. 20 (1987) 53–65.
[35] R.M. Cormack, A review of classification, J. R. Stat. Soc. Ser. A (Gen.) 134 (3)(1971) 321–367.
P. Sarlin, Z. Yao / Neurocomputing 121 (2013) 317–327 327
[36] P. Sarlin, T. Eklund, Financial performance analysis of European banks using afuzzified self-organizing map, Int. J. Knowl.-Based Intelligent Eng. Syst. (2013).(forthcoming).
[37] P. Sarlin, Z. Yao, T. Eklund, A framework for state transitions on the self-organizing map: some temporal financial applications, Intelligent Syst. Acc.Finance Manage. 19 (3) (2012) 189–203.
[38] A. Berg, C. Pattillo, What caused the Asian crises: an early warning systemapproach, Econ. Notes 28 (3) (1999) 285–334.
Peter Sarlin obtained his M.Sc. degree in 2009 fromÅbo Akademi University. At the same university, he iscurrently pursuing a Ph.D. degree in information sys-tems. Peter has also been a regular visitor at centralbanks, such as the European Central Bank and the Bankof Finland. His recent work has focused on financialstability surveillance with computational intelligence,in particular data and dimension reduction, and incomputational finance and economics in general.Further information can be obtained from here:research.it.abo.fi/personnel/psarlin.
Zhiyuan Yao started his studies for a doctoral degree inInformation Systems at Åbo Akademi University in2009. His research interests are in visual dynamicclustering and the application of data mining techni-ques in Customer Relationship Management in general.Further information can be obtained from here:research.it.abo.fi/personnel/zyao.