Presentation is loading. Please wait.

Presentation is loading. Please wait.

Mining Biological Data Jiong Yang, Ph. D. Visiting Assistant Professor UIUC

Similar presentations


Presentation on theme: "Mining Biological Data Jiong Yang, Ph. D. Visiting Assistant Professor UIUC"— Presentation transcript:

1 Mining Biological Data Jiong Yang, Ph. D. Visiting Assistant Professor UIUC

2 Data is Everywhere

3 Data Mining is a Powerful Tool Computational Biology E-Commerce Intrusion Detection Multimedia Processing Unstructured Data... Data Data Mining Knowledge

4 Biological Data Bio-informatics have become one of the most important applications in data mining.  DNA sequences  Protein sequences  Protein folding  Microarray data  ……

5 Outline Approximate sequential pattern mining Coherent cluster: clustering by pattern similarity in a large data set

6 Frequent Patterns Model  A set of sequences of symbols. a1,a2,a4 a2,a3,a5 a1,a4,a5,a6,a7  If a pattern occurs more than a certain number of times, then this pattern is considered important. a1,a4 Widely studied  Frequent itemset mining: Agarwal and Srikant (IBM Almaden)  FP growth: Han (UIUC)  Stream data: Motwani (Stanford)  …

7 Apriori Property Widely used in data mining field It holds for the support metrics All patterns form a lattice.  (a, b, d) is a super-pattern of (a, d) and it is a sub- pattern of (a, b, c, d).  Support metric defines a partial order on the lattice. Support(a, b, d) <= min{Support(b, d), Support(a, d), Support(a, b) }  Level-wise search algorithm can be used

8 Shortcomings Require exact match and fail to recognize possible substitution among symbols  Protein may mutate without change of its functionality.  A sensor may make some mistakes  Different web pages may have similar contents.  A word may have many synonyms. How can the symbol substitution be modeled

9 Compatibility Matrix d1d1 d2d2 d5d5 d3d3 d4d4 d1d1 d2d2 d5d5 d3d3 d4d4 observed true Compatibility matrix of 5 symbols

10 Compatibility Matrix The compatibility matrix serves as a bridge between the observation and the underlying substance.  Each observed symbol is interpreted as an occurrence of a set of symbols with various probabilities.  An observed symbol combination is treated as an occurrence of a set of patterns with various degrees. Obtain the compatibility matrix through  empirical study  domain expert

11 Match A new metric, match, is then proposed to quantify the importance of a pattern.  The match of a pattern P in a subsequence s (with the same length) is defined as the conditional probability Prob(P| s).  The match of a pattern P in a sequence S is defined as the maximal match of P in every distinct subsequence in S. A dynamic programming technique is used to compute the match of P in a sequence S

12 Match M(d 1 d 2 …d i, S 1 S 2 …S j ) is the maximum of M(d 1 s 2 …d i, S 1 S 2 …S j-1 ) and M(d 1 d 2 …d i-1,S 1 S 2 …S j-1 ) x C(d i, S j )  The match of a pattern P in a set of sequence is defined as the sum of the pattern P with each sequence.  A pattern is called a frequent pattern if its match exceeds a user-specified threshold min_match. S p S p max d1d3d4d1 d2 S p

13 Challenges Previous work focuses on short patterns. Long patterns require a large number of scans through the input sequence.  Expensive I/O cost Performance vs. Accuracy Probabilistic Approach

14 Chernoff Bound Let X be a random variable whose range is R. Suppose that we have n independent observations of X and the observed mean is . The Chernoff bound states that, with probability (1-  ), the true mean of X is at least  - , where With probability (1-  ), the true value of X is at most  + .

15 Approach Three-stage approach to mine patterns with length l:  Finding Match of Individual Symbols and Take a Sample set of sequences  Pattern Discovery on Samples  Ambiguous Patterns Determination Pattern Discovery on Samples  Sample size: depending on memory size  Based on the samples, three types of patterns are determined.

16 Approach Frequent pattern if match is greater than (min_match +  ) Ambiguous pattern if match is between (min_match -  ) and (min_match +  ). Infrequent pattern otherwise;

17 Ambiguous Patterns  Too many  Border collapse We have the negative and positive borders of significant patterns. Our goal is to collapse the border as fast as possible.

18 Ambiguous Patterns (d 1 ) (d 1,d 2 ) (d 1,d 3 )(d 1,d 4 ) (d 1,d 5 ) (d 1,d 2,d 3 )(d 1,d 2,d 4 )(d 1,d 2,d 5 )(d 1,d 3,d 4 )(d 1,d 3,d 5 ) (d 1,d 2,d 3,d 4 ) (d 1,d 2,d 3,d 5 )(d 1,d 2,d 4,d 5 ) (d 1,d 3,d 4,d 5 ) (d 1,d 2,d 3,d 4,d 5 ) (d 1,d 4,d 5 )

19 Ambiguous Patterns (d 1 ) (d 1,d 2,)(d 1,d 3 )(d 1,d 4 )(d 1,d 5 ) (d 1,d 2,d 3 )(d 1,d 2,d 4 )(d 1,d 2,d 5 )(d 1,d 3,d 4 )(d 1,d 3,d 5 )(d 1,d 4,d 5 ) (d 1,d 2,d 3,d 4 )(d 1,d 2,d 3,d 5 )(d 1,d 2,d 4,d 5 )(d 1,d 3,d 4,d 5 ) (d 1,d 2,d 3,d 4,d 5 ) frequent infrequent

20 Effects of 1-  With Border Collapse Without Border Collapse

21 Approximate Pattern Mining Reference:  Mining long sequential patterns in a noisy environment, Proceeding of ACM SIGMOD International Conference on Management of Data (SIGMOD), pp , Other Work  Periodic Patterns (KDD2000, ICDM2001)  Statistically significant Patterns (KDD2001, ICDM 2002)

22 Outline Approximate sequential pattern mining Coherent cluster: clustering by pattern similarity in a large data set

23 Coherent Cluster In many applications, data can be of very high dimensionality.  Gene expression data Dozens to hundreds conditions/samples  Customer evaluation Thousands or more merchants Objective: discover peer groups d ij attributes objects oioi ajaj o1o a1a1...

24 17 conditions 40 genes

25 Coherent Cluster

26 40 genes

27 Coherent Cluster Co-regulated genes

28 Coherent Cluster Observations: 1. If mapped to points in high dimensional space, they may not be close to each other. Bias exists universally. 2. Only a subset of objects and a subset of attributes may participate. 3. Need to accommodate some degree of noise. Solution: subspace cluster, bicluster, coherent cluster

29 Subspace cluster CLICK: Argawal et al IBM Almaden  Find a subset of dimensions and a subset of objects such that the distance between the objects on the subset of dimensions is close.  The clusters may overlap Proclus: Aggawal et al IBM T. J. Watson  Do not allow overlap

30 Bicluster Developed in 2000 by Cheung and Church Using mean squared error residual After discovering one cluster, replace the cluster with random data and find another Not efficient and not accurate

31 Coherent Cluster Coherent cluster  Subspace clustering  Measure distance on mutual bias pair-wise disparity  For a 2  2 (sub)matrix consisting of objects {x, y} and attributes {a, b} x y ab d xa d ya d xb d yb x y ab attribute mutual bias of attribute a mutual bias of attribute b

32 Coherent Cluster  A 2  2 (sub)matrix is a  -coherent cluster if its D value is less than or equal to .  An m  n matrix X is a  -coherent cluster if every 2  2 submatrix of X is  -coherent cluster. A  -coherent cluster is a maximum  -coherent cluster if it is not a submatrix of any other  -coherent cluster.  Objective: given a data matrix and a threshold , find all maximum  -coherent clusters.

33 Coherent Cluster Challenges:  Finding subspace clustering based on distance itself is already a difficult task due to the curse of dimensionality. The (sub)set of objects and the (sub)set of attributes that form a cluster are unknown in advance and may not be adjacent to each other in the data matrix.  The actual values of the objects in a coherent cluster may be far apart from each other. Each object or attribute in a coherent cluster may bear some relative bias (that are unknown in advance) and such bias may be local to the coherent cluster.

34 Coherent Cluster Compute the maximum coherent attribute sets for each pair of objects Construct the lexicographical tree Post-order traverse the tree to find maximum coherent clusters Compute the maximum coherent object sets for each pair of attributes Two way pruning

35 Coherent Cluster Observation: Given a pair of objects {o 1, o 2 } and a (sub)set of attributes {a 1, a 2, …, a k }, the 2  k submatrix is a  -coherent cluster iff, for every attribute a i, the mutual bias (d o1ai – d o2ai ) does not differ from each other by more than . a1a1 a2a2 a3a3 a4a4 a5a o1o1 o2o2  [2, 3.5] If  = 1.5, then {a 1,a 2,a 3,a 4,a 5 } is a coherent attribute set (CAS) of (o 1,o 2 ).

36 a1a1 a2a2 a3a3 a4a4 a5a r1r1 r2r2 Coherent Cluster Strategy: find the maximum coherent attribute sets for each pair of objects with respect to the given threshold .  = r1r1 r2r2 a2a2 2 a3a3 3.5 a4a4 2 a5a5 2.5 a1a1 3 1 The maximum coherent attribute sets define the search space for maximum coherent clusters.

37 Two Way Pruning a0a1a2 o0142 o1255 o2365 o o (o0,o2) →(a0,a1,a2) (o1,o2) →(a0,a1,a2) (a0,a1) →(o0,o1,o2) (a0,a2) →(o1,o2,o3) (a1,a2) →(o1,o2,o4) (a1,a2) →(o0,o2,o4) (o0,o2) →(a0,a1,a2) (o1,o2) →(a0,a1,a2) (a0,a1) →(o0,o1,o2) (a0,a2) →(o1,o2,o3) (a1,a2) →(o1,o2,o4) (a1,a2) →(o0,o2,o4) delta=1 nc =3 nr = 3 MCAS MCOS

38 Coherent Cluster High expressive power  The coherent cluster can capture many interesting and meaningful patterns overlooked by previous clustering methods. Efficient and highly scalable Wide applications  Gene expression analysis  Collaborative filtering traditional clustering coherent clustering

39 Coherent Cluster References:  Delta-cluster: capturing subspace correlation in a large data set, Proceedings of the 18th IEEE International Conference on Data Engineering (ICDE), pp ,  Clustering by pattern similarity in large data sets, Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD), pp ,  Enhanced biclustering on expression data, Proceedings of the IEEE bio- informatics and bioengineering (BIBE), Other Work  STING (VLDB1997)  STING+ (ICDE1999, TKDE 2000)  CLUSEQ (CSB2002, ICDE2003)  Cluster Streams (ICDE2003)

40 Remarks Similarity measure  Powerful in capturing high order statistics and dependencies  Efficient in computation  Robust to noise Clustering algorithm  High accuracy  High adaptability  High scalability  High reliability


Download ppt "Mining Biological Data Jiong Yang, Ph. D. Visiting Assistant Professor UIUC"

Similar presentations


Ads by Google