1. Even More on Dimensions
Professor Douglas W. Raiford
What to do with all those dimensions: MDS

2. Multidimensional scaling (MDS)
MDS is a special case of ordination
Ordination orders objects that are characterized by values on multiple variables
MDS
Multidimensional scaling (MDS)
Pattern Recognition: Multidimensional Analysis
9/17/2018

3. Classical Multidimensional Scaling
AKA
Principal Coordinates Analysis (PCoA)
Torgerson Scaling or Torgerson–Gower scaling
Classical Multidimensional Scaling
Pattern Recognition: Multidimensional Analysis
9/17/2018

4. PCA: uses covariance matrix
Based upon distance between instances instead of covariance between columns
Between columns
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
setosa
setosa
setosa
setosa
setosa
setosa
setosa
setosa
setosa
setosa
setosa
setosa
setosa
setosa
setosa .
MDS: uses distance matrix
Distance Between Rows
Rows vs. Columns
Pattern Recognition: Multidimensional Analysis
9/17/2018

5. points = cmdscale(dist(irisData)) plot(points,col=myColors, main = "Iris Data Using MDS", xlab = "Eigen 1", ylab = "Eigen 2",pch = 20) legend("topright", levels(iris[,5]), col = c("red","green","blue"), pch = 20)
Simple Example in R
Pattern Recognition: Multidimensional Analysis
9/17/2018

6. How does it work Distance Matrix instead of a covariance matrix
And don’t even bother projecting original data
Just returns the first two Eigenvectors
Why are there 150 entries in each Eigenvector???
> points
[,1] [,2]
[1,]
[2,]
[3,]
[4,]
[5,]
[6,]
[7,]
[8,]
[9,]
[10,] .
Eigen’s taken of distance matrix which is 150 x 150
Think of as a factorization, in two d’s get greatest axis and orthogonal axis, can use to get area (pi r squared, but two diff r’s)
How does it work
Pattern Recognition: Multidimensional Analysis
9/17/2018

7. d = dist(irisData) x = as
d = dist(irisData) x = as.matrix(d) x = x^2 x = dblcen(x)#normalizes, puts centroid at origin e <- eigen(-x/2, symmetric = TRUE) ev <- e$values[seq_len(k)] evec <- e$vectors[, seq_len(k), drop = FALSE] plot(e$vectors[,c(1,2)],col = myColors, main = "Iris Data Using MDS", xlab = "Eigen 1",ylab = "Eigen 2")
Implementation in R
Pattern Recognition: Multidimensional Analysis
9/17/2018

8. When would you … What if your data had 100 rows but 20,000 dimensions?
20k
100 rows
When would you …
Pattern Recognition: Multidimensional Analysis
9/17/2018

9. When would you … What if your data had 20,000 rows but 100 dimensions?
100 dims
What if your data had 20,000 rows but 100 dimensions?
20k
When would you …
Pattern Recognition: Multidimensional Analysis
9/17/2018

10. The end
Pattern Recognition: Multidimensional Analysis
9/17/2018

11. Pattern Recognition: Multidimensional Analysis
9/17/2018

12. More versions Metric multidimensional scaling
A superset of classical MDS that generalizes the optimization procedure to a variety of loss functions and input matrices of known distances with weights and so on. A useful loss function in this context is called stress, which is often minimized using a procedure called stress majorization.
Apparantly this is classic according to
Same site mentions Individual Difference Scaling: indscal()
Non-metric multidimensional scaling
In contrast to metric MDS, non-metric MDS finds both a non-parametric monotonic relationship between the dissimilarities in the item-item matrix and the Euclidean distances between items, and the location of each item in the low-dimensional space. The relationship is typically found using isotonic regression.
Louis Guttman's smallest space analysis (SSA) is an example of a non-metric MDS procedure.
Look at fit <- isoMDS(d, k=2) # k is the number of dim
Generalized multidimensional scaling
An extension of metric multidimensional scaling, in which the target space is an arbitrary smooth non-Euclidean space. In case when the dissimilarities are distances on a surface and the target space is another surface, GMDS allows finding the minimum-distortion embedding of one surface into another.[2]
More versions
Pattern Recognition: Multidimensional Analysis
9/17/2018