A compression-boosting transform for 2D data Qiaofeng Yang Stefano Lonardi University of California, Riverside

Haifa Stringology Research Workshop 2005 Problem Given a matrix A over a binary alphabet Find an invertible transform T such that T(A) is more “compressible” than A Idea: try a reordering of the rows and columns of A

Haifa Stringology Research Workshop 2005 Applications Lossless compression of binary matrices (i.e., binary digital images) Evaluation of the randomness of a graph (represented by its adjacency matrix)

Haifa Stringology Research Workshop 2005 Compression-boosting transform Design an invertible transform that improves compression The transform must expose the “patterns” in the matrix [Storer and Helfgott 97] find uniform blocks to compress images (lossless)

Haifa Stringology Research Workshop 2005 Redundancy in binary matrices Look for uniform submatrices A uniform submatrix is a submatrix induced by a subset of rows and columns (i.e., not necessarily contiguous) solely composed by the same symbol (either 0 or 1)

Haifa Stringology Research Workshop 2005 Outline of the direct transform

Haifa Stringology Research Workshop 2005 Direct transform Find the largest uniform submatrix in A Reorder the rows and the columns such that the uniform submatrix is moved to the upper-left corner Recursively apply the transform on the rest of the matrix Stop when the partition produces a matrix which is smaller than r x c (r,c predetermined thresholds)

Haifa Stringology Research Workshop 2005 Does it help? Maybe … Each uniform submatrix can be represented by one bit and a list of its rows/columns If a matrix can be decomposed in a small number of uniform submatrices, the compression should improve

Haifa Stringology Research Workshop 2005 Does it help? Maybe not… Typically, in order to get good compression of 2D data (e.g., digital images) one has to exploit dependencies between adjacent rows and columns The reordering can “break” local dependencies between adjacent rows or columns, therefore could affecting negatively the compression

Haifa Stringology Research Workshop 2005 Complexity of the direct transform The computation cost depends on the complexity of finding the largest uniform submatrix This problem is a special case of the biclustering problem

Haifa Stringology Research Workshop 2005 Some related works Hartigan, ‘72 Aggarwal et al., SIGMOD’99 Cheng & Church, ISMB’00 Wang et al., SIGMOD’02 Ben-Dor et al., RECOMB’02 Tanay et al., ISMB’02 Procopiuc et al., SIGMOD’02 Murali & Kasif, PSB’03 Sheng et al., ECCB’03 Mishra et al., COLT’03 Lonardi et al., CPM’04

Haifa Stringology Research Workshop 2005 Problem definition L ARGEST U NIFORM S UBMATRIX Instance: A matrix Question: Find a row selection R and a column selection C such that A (R, C) is uniform and |R||C| is maximized This problem is also called Maximum Edge Biclique, and is computationally hard

Haifa Stringology Research Workshop 2005 Randomized search

Haifa Stringology Research Workshop 2005 Randomized search (step 1) Select a random subset S of size k uniformly from the set of columns {1,2,…, m}

Haifa Stringology Research Workshop 2005 Randomized search (step 2) For all the subset U of S, check whether string 1 |U| (resp., 0 |U| ) appears at least r times and record the rows R 1 (resp., R 0 ) in which it occurs 1

Haifa Stringology Research Workshop 2005 Randomized search (step 3) Given R 1 (resp., R 0 ) select the set of columns C which read 1 (resp., 0) Check whether

Haifa Stringology Research Workshop 2005 Randomized search (step 4) Save the solutions (R 1,C) and (R 0,C) and repeat step 1 to 4 for t iterations

Haifa Stringology Research Workshop 2005 Parameters Projection size k Column threshold c Row threshold r Number of iterations t See [Lonardi ‘04] for details on how to choose k, r and c

Haifa Stringology Research Workshop 2005 Selecting the number of iterations t We can miss a solution in two cases –S completely misses C* –when S overlaps C*, but the string 11…1 (00…0) selected by the algorithm also appears in a row outside R*

Haifa Stringology Research Workshop 2005 Selecting the number of iterations t The probability of missing the solution in one iteration is which is

Haifa Stringology Research Workshop 2005 Selecting the number of iterations t Given the probability of missing the solution in t iterations to be smaller than ε For the experiments, we fixed the number of iterations to t=1,000 and t=10,000

Haifa Stringology Research Workshop 2005 Recursive decomposition Input: binary matrix A, row/column thresholds r/c, iterations t (or error ε) Run L ARGEST U NIFORM S UBMATRIX on A Reorder A and decompose A into four smaller matrices U,a,b,c where is U is the uniform submatrix Should we recursively apply the transform to (a,b,c), (a+c,b) or (a,b+c) ?

Haifa Stringology Research Workshop 2005 Outline of the direct transform

Haifa Stringology Research Workshop 2005 Refined transform

Haifa Stringology Research Workshop 2005 Is (a+c,b) better than (a,b+c)? Let us call A 1, A 2, B 1, B 2 the area of the uniform submatrix found in a+c, b, a, and b+c respectively Choose decomposition (a+c,b) if –S UM : A 1 +A 2 >B 1 +B 2 –M AX : max{A 1,A 2 }>max{B 1,B 2 } –I NDIV : A 1 >B 2

Haifa Stringology Research Workshop 2005 Implementation Each uniform submatrix can be represented by one bit Non-decomposable matrices are saved in row-major order The content of both types of matrices is saved in a file called strings

Haifa Stringology Research Workshop 2005 Implementation Row and column indices for uniform and non-decomposable matrices are saved in a file called index For each set of rows/column indices the first index is saved as is, while the other are saved as difference between adjacent indices

Haifa Stringology Research Workshop 2005 Implementation The number of rows and columns for uniform and non-decomposable matrices is saved in a file called length The files string, index and length allow one to invert the transform Note that the complexity of the inverse transform is linear in the size of the matrix

Haifa Stringology Research Workshop 2005 Experiments on synthetic data We generated datasets composed by four random 256x256 binary matrices In each matrix of a dataset we embedded 1, 2, 3 and 4 uniform submatrices of size 64x64 For each matrix we compared the compression size before and after the transform

Haifa Stringology Research Workshop 2005 Experiments on synthetic data Matrix i is a random 256x256 binary matrix and contains i uniform 64x64 submatrices. Parameters are r=c=10, t=10,000

Haifa Stringology Research Workshop 2005 Experiments on synthetic data Failure in recovering all the embedded submatrices is typically due to the recursive partitioning If the partitioning happens to split an embedded submatrix, there is no hope in recovering it

Haifa Stringology Research Workshop 2005 Experiments on compressing images

Haifa Stringology Research Workshop 2005 Compressing images

Haifa Stringology Research Workshop 2005 Experiments on images Parameters are r=c=60, t=10,000

Haifa Stringology Research Workshop 2005 Experiments on images How does the performance depend on –the partitioning strategy? –the row and column thresholds? –the number of iterations? The next graphs are about the image bird

Haifa Stringology Research Workshop 2005 Size as a function of decomposition strategy

Haifa Stringology Research Workshop 2005 Average area of uniform submatrices

Haifa Stringology Research Workshop 2005 Number of uniform submatrices

Haifa Stringology Research Workshop 2005 Proportion of uniform submatrices

Haifa Stringology Research Workshop 2005 Final compressed size

Haifa Stringology Research Workshop 2005 Findings Proposed a transform to boost compression of 2D binary data The direct transform is hard, but a randomized algorithm can be used The inverse transform is very fast The transform boosts compression

Haifa Stringology Research Workshop 2005 Randomness of a graph Problem: determine whether a graph G is “random” or not (and how much) Use the Kolmogorov complexity Use the compressibility of the adjacency matrix of G to bound the Kolmogorov complexity of G

Haifa Stringology Research Workshop 2005

The general problem Biclustering is the problem of finding a partition of the vectors and a subset of the dimensions such that the projections along those directions of the vectors in each cluster are close to one another The problem requires to cluster the vectors and the dimensions simultaneously, thus the name “biclustering”

Haifa Stringology Research Workshop 2005 Applications of biclustering Collaborative filtering and recommender systems Finding web communities Discovery association rules in databases Gene expression analysis …

Haifa Stringology Research Workshop 2005 Pseudo-code