courtesy of prof. shixia liu @tsinghua...
TRANSCRIPT
Courtesy of Prof. Shixia Liu @Tsinghua University
Outline
• Introduction
• Classification of Techniques – Table
– Scatter Plot Matrices
– Projections
– Parallel Coordinates
• Summary
Motivation
• Real world data contain multiple dimensions
Multivariate/Multidimensional Data Visualization• Multivariate data visualization is a specific type of
information visualization that deals with multivariate/multidimensional data
• The data to be visualized are of high dimensionality in which the correlations between these many attributes are of interest
Dimensionality
• Refers to the number of attributes that presents in the data – 1: one-dimensional 1D / univariate
– 2: two-dimensional 2D/ bivaraite
– 3: three-dimensional 3D / trivariate
– ≥3: multidimensional / hypervarite / multivariate
• Boundary between high and low dimensionality not clear, generally high dimensionality has >4 variables
Terminology
Dimensions
Variables
Multidimensional Dimensionality of the independent dimensions
Multivariate Dimensionality of the dependent variables
Outline
• Introduction
• Classification of Techniques – Projections
– Parallel Coordinates
– Table
– Scatter Plot Matrices
• Summary
Classification of Techniques • Projection
• Parallel Coordinates Plot
• Table
• Scatter Plot Matrix
• What if we have too many dimensions?
• A intuitive way is to project to low dimension space
• Linear projections
• Nonlinear projections
A projection (X -> Y) maps points {x1, x2, …, xm} in an n-dimensional space into a p-dimensional space as {y1, y2, …, ym} (p << n) while preserving distance measures of data items.
Classification
• Linear projection – Example: PCA (principal
component analysis)
• Non-linear projection – Example: t-SNE (t-distributed
stochastic neighbor embedding)
PCA
• Seeks a space of lower dimensionality (magenta)
• Such that the orthogonal projection of the data points (red) onto this subspace maximizes the variance of the projected points (green)
Maximizes Variance
• To begin with, consider the projection onto a one-dimensional space
• The direction of this space
• Variance
• How to maximize this?
Trick:
Maximizes Variance (cont’d)•
Eigenvalue
One Example
Extension to M-dimension
• Define additional principal components in an incremental fashion (details refer to Chapter 12 in Patter Recognition and Machine Learning)
• Conclusion of M dimension:
• The M eigenvectors u1,...,uM of the data covariance matrix S corresponding to the M largest eigenvalues λ1 ,...,λM
Covariance Matrix
Covariance
Fit an n-d Ellipsoid to the Data
T-SNE
T-SNE
• Particularly well-suited for embedding high-dimensional data into a space of two or three dimensions, which can then be visualized in a scatter plot
Major Goal • t-Distributed stochastic neighbor embedding (t-
SNE) minimizes the divergence between two distributions: a distribution that measures pairwise similarities of the input objects and a distribution that measures pairwise similarities of the corresponding low-dimensional points in the embedding.
Two Main Stages
• First, t-SNE constructs a probability distribution over pairs of high-dimensional objects – Similar objects have a high probability of being picked
– Dissimilar points have an extremely small probability of being picked
Example – Step 1
Two Main Stages (cont’d)• Second, t-SNE defines a probability distribution
over the points in the low-dimensional map – Similar to the one in high-dimensional space
– Minimizes the Kullback–Leibler divergence between the two distributions with respect to the locations of the points in the map.
Heavy-tailed student-t distribution
Example: Step Two
Example: Step Two
Before optimization
Example: Final Result
Student t-distribution Gaussian distribution
The t-Student distribution • The volume of the N-dimensional ball of radius r
scales is
• When N is large, if we pick random points uniformly in the ball, most points will be close to the surface, and very few will be near the center.
The t-Student distribution • If the same Gaussian distribution is used for the
low dimensional map points， not enough space is available in low dimensional space – The crowding problem
• Use a t-Student with one degree of freedom (or Cauchy) distribution instead for the map points. – Has a much heavier tail than the Gaussian distribution,
which compensates the original imbalance.
Comparison
The Distribution Model • Probability model for high-dimensional data points
• Probability model for low-dimensional map points
• The different between two distributions
The Solution • To minimize this score, we perform a gradient
descent. The gradient can be computed analytically:
• Update yi iteratively
One Example
Example: MNIST
• Hand written digit (0-9)
Package • Laurens van der Maate https://lvdmaaten.github.io/tsne/
– L.J.P. van der Maaten. Accelerating t-SNE using Tree-Based Algorithms. Journal of Machine Learning Research 15(Oct):3221-3245, 2014.
– L.J.P. van der Maaten and G.E. Hinton. Visualizing Non-Metric Similarities in Multiple Maps. Machine Learning 87(1):33-55, 2012.
– L.J.P. van der Maaten. Learning a Parametric Embedding by Preserving Local Structure. In Proceedings of the Twelfth International Conference on Artificial Intelligence & Statistics (AI-STATS), JMLR W&CP 5:384-391, 2009. PDF
– L.J.P. van der Maaten and G.E. Hinton. Visualizing High-Dimensional Data Using t-SNE. Journal of Machine Learning Research 9(Nov):2579-2605, 2008.
Comparison • PCA, MDS
– Linear technique
– Keep the low-dimensional representations of dissimilar data points far apart
• t-SNE – Non-linear technique
– Capture much of the local structure of the high-dimensional data very well, while also revealing global structure such as the presence of clusters at multiple scales.
Comparison
• Inselberg， "Multidimensional detective" (parallel coordiantes), 1997
Parallel Coordinates: Visual Design
• Dimensions as parallel axes
• Data items as line segments
• Intersections on the axes indicates the values of the corresponding attributes
dim1 dim2 dim3 dimn……
Min: 0
Max: 1
0.8
0.6
0.8
0.30.25
Parallel Coordinates: Pros and Cons
!"Correlations among attributes studied by spotting the locations of the intersection points
!"Effective for revealing data distributions and functional dependencies
#"Visual clutter due to limited space available for each parallel axis
#"Axes packed very closely when dimensionality is high
• Clustering and filtering approaches
• Dimension reordering approaches
• Visual enhancement approaches
Out5d dataset (5 dimensions, 16384 data items)
Star Coordinates
• Scatterplots for higher dimensions: attribute as axis on a circle, data item as point
• Change the length of axis $ alters contribution of attribute
• Change the direction of axis $ angles not equal, adjusts correlations between attributes
!"Useful for gaining insight into hierarchically clustered datasets and for multi-factor analysis for decision-making
Table Lens
• Represents rows as data items and columns as attributes
• Each column viewed as histogram or plot • Information along rows or columns interrelated !"Uses the familiar concept “table”
The table lens: merging graphical and symbolic representations in an interactive focus+ context visualization for tabular information
Scatterplot Matrix • Scatterplot: 2 attributes
projected along the x- and y-axis
• Collection of scatterplots is organized in a matrix
!"Straightforward%
#"Important patterns in higher dimensions barely recognized
#"Chaotic when number of data items too large
Outline
• Introduction
• Classification of Techniques – Table
– Scatter Plot Matrices
– Projections
– Parallel Coordinates
– Pixel-Oriented Techniques
– Iconography
• Summary
Visualizations Advantages Disadvantages
Clear visual patterns1. Obscured semantics 2. Loss of information 3. Visual Clutter
Clear visual patterns Visual Clutter
Uses the familiar concept “table”
Support limited numbers of dimensions
Simple1. Visual clutter 2. Unclear patterns
Further Reading
• Survey – Dos Santos, Selan, and Ken Brodlie. "Gaining
understanding of multivariate and multidimensional data through visualization." Computers & Graphics28.3 (2004): 311-325.
• Website – http://www.sci.utah.edu/~shusenl/highDimSurvey/
website/
Further Reading
• Evaluation – Rubio-Sánchez, Manuel, et al. "A comparative study
between RadViz and Star Coordinates." IEEE transactions on visualization and computer graphics 22.1 (2016): 619-628.
References
• Rao, Ramana, and Stuart K. Card. "The table lens: merging graphical and symbolic representations in an interactive focus+ context visualization for tabular information." Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 1994.
• Gratzl, Samuel, et al. "Lineup: Visual analysis of multi-attribute rankings."IEEE transactions on visualization and computer graphics 19.12 (2013): 2277-2286.
• van Wijk, Jarke J., and Robert van Liere. "HyperSlice: visualization of scalar functions of many variables." Proceedings of the 4th conference on Visualization'93. IEEE Computer Society, 1993.
• Kim, Hannah, et al. "InterAxis: Steering Scatterplot Axes via Observation-Level Interaction." IEEE transactions on visualization and computer graphics22.1 (2016): 131-140.
References
• Maaten, Laurens van der, and Geoffrey Hinton. "Visualizing data using t-SNE." Journal of Machine Learning Research 9.Nov (2008): 2579-2605.
• Zhou, Hong, et al. "Visual clustering in parallel coordinates." Computer Graphics Forum. Vol. 27. No. 3., 2008.
• Ferdosi, Bilkis J., and Jos BTM Roerdink. "Visualizing High‐
Dimensional Structures by Dimension Ordering and Filtering using Subspace Analysis."Computer Graphics Forum. Vol. 30. No. 3, 2011.
• Novotny, Matej, and Helwig Hauser. "Outlier-preserving focus+ context visualization in parallel coordinates." IEEE Transactions on Visualization and Computer Graphics 12.5 (2006): 893-900.
References
• Keim, Daniel A., and H-P. Kriegel. "Visualization techniques for mining large databases: A comparison." IEEE Transactions on knowledge and data engineering 8.6 (1996): 923-938.