1. Data Intensive and Cloud Computing Matrices and Arrays Lecture 9

2. Matrices and Arrays in Python: NumPy
We’ve already seen lists – single-dimensional data with a single type
More generically, we may want to have matrices (as in PageRank) or multidimensional arrays
This leads to numpy – support for numeric operations in Python
A building block for Pandas
And for SciPy, which is scientific programming support

3. What Do We Use Matrices / Multi-Dimensional Arrays for?
Graph adjacency matrices, and variants thereof
Rows of readings of sets of numeric values (e.g., a suite of sensors)
Machine learning features (later in the semester!)
Images, volumes, numeric state values in a multidimensional space

4. 5 Important Concepts from Linear Algebra
Help! My matrix is too big to analyze by inspection. I should compute:
Frobenius Norm
Rank
Determinant
Singular Vectors / Singular Values
Eigenvectors / Eigenvalues

5. Basic concepts Vector in Rn is an ordered set of n real numbers.
e.g. v = (1,6,3,4) is in R4
A column vector:
A row vector:
m-by-n matrix is an object in Rmxn with m rows and n columns, each entry filled with a (typically) real number:

6. Basic concepts
Vector norms: A norm of a vector ||x|| is informally a measure of the “length” of the vector.
Common norms: L1, L2 (Euclidean)
Linfinity

7. Basic concepts
Use lower case letters for vectors. The elements are referred by xi.
Vector dot (inner) product:
Vector outer product:
If u•v=0, ||u||2 != 0, ||v||2 != 0  u and v are orthogonal
If u•v=0, ||u||2 = 1, ||v||2 = 1  u and v are orthonormal

8. Basic concepts e.g. Matrix product:
Use upper case letters for matrices. The elements are referred by Ai,j.
Matrix product:
e.g.

9. Special matrices diagonal upper-triangular lower-triangular
tri-diagonal
I (identity matrix)

10. Basic concepts e.g. Transpose: You can think of it as
“flipping” the rows and columns OR
“reflecting” vector/matrix on line
e.g.

11. Linear independence e.g.
A set of vectors is linearly independent if none of them can be written as a linear combination of the others.
Vectors v1,…,vk are linearly independent if c1v1+…+ckvk = 0 implies c1=…=ck=0
e.g.
(u,v)=(0,0), i.e. the columns are linearly independent.
x3 = −2x1 + x2

12. Span of a vector space e.g.
If all vectors in a vector space may be expressed as linear combinations of a set of vectors v1,…,vk, then v1,…,vk spans the space.
The cardinality of this set is the dimension of the vector space.
A basis is a maximal set of linearly independent vectors and a minimal set of spanning vectors of a vector space
(0,0,1)
(0,1,0)
(1,0,0)
e.g.

13. Determinants A determinant of a matrix A is denoted by |A|.
The determinant of a 22 matrix:
The determinant of a 33 matrix:

14. The determinant of an nn matrix:
The minor of aij, denoted by Aij, is the matrix after removing row i and column j.
The determinant of an nn matrix:
The general expression for the determinant of an nn matrix:

15. Example: Matrix Determinant
with the first row and their minors:

16. with the second column and their minors:
Since |A|=0, A is a singular matrix; that is the inverse of A does not exist.

17. If a matrix A has a zero determinant, then A is a singular matrix; that is, the inverse of A does not exist.

18. Rank of a Matrix rank(A) (the rank of a m-by-n matrix A) is
The maximal number of linearly independent columns
=The maximal number of linearly independent rows
=The dimension of col(A)
=The dimension of row(A)
If A is n by m, then
rank(A)<= min(m,n)
If n=rank(A), then A has full row rank
If m=rank(A), then A has full column rank

19. Rank– Many Equivalent Definitions
A matrix of r rows and c columns is said to be of order r by c. If it is a square matrix, r by r, then the matrix is of order r.
The rank of a matrix equals the order of highest-order nonsingular submatrix.

20. Example 1: Rank of Matrix
3 square submatrices:
Each of these has a determinant of 0, so the rank is less than 2. Thus the rank of R is 1.

21. Example 2: Rank of Matrix
Since |A|=0, the rank is not 3. The following submatrix has a nonzero determinant:
Thus, the rank of A is 2.

22. Compute Rank of Large Matrices
Theorem 1
Row-Equivalent Matrices
Row-equivalent matrices have the same rank.
We call a matrix A1 row-equivalent to a matrix A2 if A1 can be obtained from A2 by (finitely many!) elementary row operations.

23. EXAMPLE: Computing Rank

24. EXAMPLE Computing Rank (continued)
The last matrix is in row-echelon form and has two nonzero rows. Hence rank A = 2.

25. Inverse of a matrix
Inverse of a square matrix A, denoted by A-1 is the unique matrix s.t.
AA-1 =A-1A=I (identity matrix)
If A-1 and B-1 exist, then
(AB)-1 = B-1A-1,
(AT)-1 = (A-1)T
For orthonormal matrices
For diagonal matrices

26. Python Support for Matrices

27. Frobenius Norm
A metric for overall “value” of the matrix (sum of squared values).
numpy.linalg.norm

28. Rank The number of nonzero rows after a row reduction.
Yet another definition
numpy.linalg.matrix_rank

29. Determinant
A metric describing the overall “scaling factor” of the matrix. How much would multiplying by this matrix stretch the unit box?
numpy.linalg.det

30. Singular vectors / Singular Values
Start with the vector that gets stretched most by this matrix
(and how much). Then, find independent others in descending order.
numpy.linalg.svd

31. Eigenvectors / Eigenvalues
The special vectors that this matrix can only scale (and by how much).
numpy.linalg.eig

32. NumPy - Details

33. What is NumPy?
NumPy is the fundamental package needed for scientific computing with Python. It contains:
a powerful N-dimensional array object
basic linear algebra functions
basic Fourier transforms
sophisticated random number capabilities
tools for integrating Fortran code
tools for integrating C/C++ code

34. NumPy documentation Official documentation The NumPy book Example list
The NumPy book
Example list

35. The ndarray data structure
NumPy adds a new data structure to Python – the ndarray
An N-dimensional array is a homogeneous collection of “items” indexed using N integers
Defined by:
the shape of the array, and
the kind of item the array is composed of

36. Numpy – ndarray attributes
ndarray.ndim
the number of axes (dimensions) of the array i.e. the rank.
ndarray.shape
the dimensions of the array. This is a tuple of integers indicating the size of the array in each dimension. For a matrix with n rows and m columns, shape will be (n,m). The length of the shape tuple is therefore the rank, or number of dimensions, ndim.
ndarray.size
the total number of elements of the array, equal to the product of the elements of shape.
ndarray.dtype
an object describing the type of the elements in the array. One can create or specify dtype's using standard Python types. NumPy provides many, for example bool_, character, int_, int8, int16, int32, int64, float_, float8, float16, float32, float64, complex_, complex64, object_.
ndarray.itemsize
the size in bytes of each element of the array. E.g. for elements of type float64, itemsize is 8 (=64/8), while complex32 has itemsize 4 (=32/8) (equivalent to ndarray.dtype.itemsize).
ndarray.data
the buffer containing the actual elements of the array. Normally, we won't need to use this attribute because we will access the elements in an array using indexing facilities.

37. Array shape ndarrays are rectangular
The shape of the array is a tuple of N integers (one for each dimension)

38. Array item types
Every ndarray is a homogeneous collection of exactly the same data-type
every item takes up the same size block of memory
each block of memory in the array is interpreted in exactly the same way

39. Arrays and Matrices in NumPy
3 wide x 2 high

40. Simple Creation (Borrowed from Matlab)

41. Uninitialized Matrices – Fast But Full of Garbage

42. Numpy – arrays, matrices
For two dimensional arrays NumPy defined a special matrix class in module matrix. Objects are created either with matrix() or mat() or converted from an array with method asmatrix().
>>> import numpy
>>> m = numpy.mat([[1,2],[3,4]]) or
>>> a = numpy.array([[1,2],[3,4]])
>>> m = numpy.mat(a)
>>> m = numpy.asmatrix(a)
Note that the statement m = mat(a) creates a copy of array 'a'.
Changing values in 'a' will not affect 'm'.
On the other hand, method m = asmatrix(a) returns a new reference to the same data.
Changing values in 'a' will affect matrix 'm'. 42

43. An Example of Using a Graph Adjacency Matrix

44. Graph Adjacency Matrices in Numpy1
Graph G 2 3 1

45. Recall the PageRank Linear Algebra formulation
Create an m x m matrix M to capture links:
M(i, j) = 1 / nj if page i is pointed to by page j and page j has nj outgoing links = otherwise
Initialize all PageRanks to 1, multiply by M repeatedly until all values converge:
Officially computes principal eigenvector via power iteration

46. PageRank Example in Python
0.5 1
Google g y a
0.15
= 0.85 * +
Amazon
Yahoo

47. PageRank in Spark? Spark has a Matrix construct built over RDDs
The challenge: how distribute/shard the matrix?
RowMatrix – arbitrarily distributed rows. e.g., machine learning features, document vectors, …
IndexedRowMatrix – rows have a numeric (long) key
CoordinateMatrix – cells are (row, column, value)
BlockMatrix – matrix broken into “tiles” with coordinates
In general: these don’t offer strong advantages over using DataFrames with edge lists... so you are better off doing PageRank using joins!

48. A Sketch of PageRank with Joins
Given dataframe edge(from, to)
Compute a dataframe transfer_weight(node, ratio) for each node
Initialize pagerank(node,score) for each node
for n iterations:
Compute weight_from_edge(from, to, weight) for each node, given existing pagerank(from, score) and transfer_weight(from, ratio)
Sum up weight_from_edge(from, to, weight) for each to – this is the new PageRank for the next iteration

49. Broader Matrix Manipulation
University of Pennsylvania

50. Matrices Have “Bulk Operations” too
Addition and multiplication with scalars
Simple arithmetic over arrays / matrices (add, multiply)
Linear algebra operations – multiply, transpose, …
“Slicing”
Many of these are key building blocks for scientific computing

51. Array Math
Basic mathematical functions operate elementwise on arrays, and are available both as operator overloads and as functions:

52. Array Math – Cont’d

53. Numpy – matrices Operator *, dot(), and multiply():
For array, '*' means element-wise multiplication, and the dot() function is used for matrix multiplication.
For matrix, '*'means matrix multiplication, and the multiply() function is used for element-wise multiplication.
Handling of higher-rank arrays (rank > 2)
array objects can have rank > 2.
matrix objects always have exactly rank 2.
Convenience attributes
array has a .T attribute, which returns the transpose of the data.
matrix also has .H, .I, and .A attributes, which return the conjugate transpose, inverse, and asarray() of the matrix, respectively.
Convenience constructor
The array constructor takes (nested) Python sequences as initializers. As in array([[1,2,3],[4,5,6]]).
The matrix constructor additionally takes a convenient string initializer. As in
matrix("[1 2 3; 4 5 6]") 53

54. Multiplication

55. Computations on Arrays

56. Aggregation over Arrays
Standard “averaging” measures – arr.mean(), median(), mode()
Distributional info – std() (standard deviation) and var() (variance)
min(), max()
argmin(), argmax()

57. Printing Arrays
When you print an array, NumPy displays it in a similar way to nested lists, but with the following layout:
the last axis is printed from left to right,
the second-to-last is printed from top to bottom,
the rest are also printed from top to bottom, with each slice separated from the next by an empty line.

59. Shape Manipulation Changing the shape of an array
An array has a shape given by the number of elements along each axis:

60. Changing Shape
The shape of an array can be changed with various commands. The following commands return a modified array, but do not change the original array:

61. Reshape versus Resize
The reshape function returns its argument with a modified shape, whereas the resize method modifies the array itself:

62. Slicing
Similar to Python lists, numpy arrays can be sliced. Since arrays may be multidimensional, you must specify a slice for each dimension of the array:

63. Slicing – cont’d
When fewer indices are provided than the number of axes, the missing indices are considered complete slices:

64. An Example Application: Image Processing
How does an image get represented?

65. Array “Slicing” – Projection or Cropping

66. Beware: Numpy Slices Are NOT Copies
By default, a “slice” points to the data in the original array!
More efficient, but it means you need to be careful about changing the slice
i.e., if I change values in crop_face, I also change face
If you’re going to modify the values, call
array.copy()

67. Selecting Parts of an Array in Numpy

68. Recall How Rows Were Selected in Pandas

69. Selecting by a Boolean List

70. A Test on a DataFrame Produces a Boolean Series (List)

71. Row Selection in an Array (And Populating a Random Array)

73. “Fancy Indexing” (Selecting by Index)
Sometimes I want to choose a list of array components in a particular order…
University of Pennsylvania

74. Arithmetic on a Numpy Array Exploits All RGB Values 0  255

75. Common Operations: Unary and Binary Functions on Elements
abs
sqrt
square
exp
log
ceil
floor …
add
subtract
multiply
power
mod
greater
dot

76. Transposition – Swapping Dimensions

77. Putting It Together

78. Randomly populating values
Create array with random values from distributions:
np.random.randn() – normal distribution with mean 0, stdev 1
binomial
normal / Gaussian
beta
chisquare
gamma

79. Linear Algebra Numpy treats arrays as matrices and supports:
np.dot(A, B) – matrix multiply, also in some versions of Python, B
Determinant (det), eigenvalues + eigenvectors (eig)
Singular value decomposition (svd)
And many more!

80. A few examples

82. Broadcasting
Work with arrays of different shapes when performing arithmetic operations Example: add a constant vector to each row of a matrix

83. Broadcasting – cont’d
Using broadcasting

84. Broadcasting Rules
If the arrays do not have the same rank, prepend the shape of the lower rank array with 1’s until both shapes have the same length.
The two arrays are said to be compatible in a dimension if they have the same size in the dimension, or if one of the arrays has size 1 in that dimension.
The arrays can be broadcast together if they are compatible in all dimensions.
After broadcasting, each array behaves as if it had shape equal to the elementwise maximum of shapes of the two input arrays.
In any dimension where one array had size 1 and the other array had size greater than 1, the first array behaves as if it were copied along that dimension

85. Recap and Take-aways
NumPy (and SciPy) provide powerful support for arrays and matrices
Bulk operators across the various elements, different kinds of iteration, …
For some kinds of array computations, there is a natural mapping to Spark- style distributed computation
... but for others we don’t get much speedup
Thus, we can do general-purpose matrix computation on a single machine, and specialized matrix computation in Spark
Sometimes it makes as much sense to encode the matrix data as a DataFrame as a Spark matrix...