1. Visualization of TV Space TVX(  )=TVX(x 33 ) TVX(x 15 ) 1 2 3 4 5 X Y TV 1 2 3 4 5

2. Proof that graph of TVX is a steep hyper-parabola centered at the mean, µ=(  x  X x i )/|X| Let f(c) = TVX(c)   x  X (x-c)o(x-c) =  x  X  i=1..n (x i - c i ) 2 =  x  X  i=1..n (x i ) 2 - 2*  x in X  i=1..n x i *c i +  x  X  i=1..n c i 2 This is clearly parabolic in each dimension, c i (fixing all other dimensions)  f/  c k =  x  X -2(x k - c k ) =0 iff  x  X x k =  x  X c k = |X|c k iff c k = (  x  X x k )/|X| = µ We can say more about the shape of the hyper-parabolic graph of f. Since  f/  c k =  x  X -2(x k - c k ) = -2  x  X x k + 2  x  X c k = 2  x  X c k - 2|X|μ k = 2|X|(c k -μ k ) we see that on each dimensional slice the parabola has the same shape, since the parabola in xy centered at (x 0,y 0 ) has the form y - y 0 = a (x - x 0 ) 2 and a obviously = f(x 0 +1) -f(x 0 ) we note y' =2a(x-x 0 ), so in our case a=|X| a very large number (steep parabola) and x 0 = µ k Since the slope of the graph is |X|, if one wants, roughly, an  -radius-contour (hyper-circular) centered at a, one needs to take the pre-image of the  |X|-interval about TVX(a), f -1 ( TVX(a)-  |X|, TVX(a)+  |X| )

3. 1 2 3 4 5 X1X1 x2x2 1 2 3 4 5 a Proof that graph of IPX is a steep hyper-plane  to µ = (  x  X x i )/|X| Inner Product functional: IPX(c) =  x  X x  c =  x  X  i=1..n x i c i = =  i=1..n c i  x  X x i =  i=1..n c i |X|µ i =  i=1..n c i |X|µ i = |X|  i=1..n c i µ i = |X| c  µ, so IPX(c) = |X| |µ| |c| cosθ where θ is the angle between c and µ. We can use any form of these equivalent formulas, depending upon which one sheds the most light on the issue we are concerned with. The blue one tell us what that the graph is extremely steep vertically (slight change in length of c causes a tremendous change in IPX(c) ) and that the contour(IPX,a,r) about a point, a, is a linear slice perpendicular to µ and also tells us how to choose the interval radius on the IPX axis so that the contour has radius, r. Red version guides to efficient preprocessing. The steepness of the graph is evident from  f/  c k = |X|µ k or the gradient,  f = |X| |µ µ

4. Proof that graph of X a is a 45 o hyper-plane nearly  to µ = (  x  X x i )/|X|  a  DomainA 1 ..  DomainA n, projection onto a, X a (x) =x  a =  i=1..n x i *a i is a functional whose graph is a hyperplane at a 45 angle with a. Contour(X a,X,b,r) is a linear (n-1)- dimensional hyper-bar through b perpendicular to a. X i (x) = x i is just X e i which also has planar graphs and have linear hyper-slice (n-1 dimensional) contours perpendicular to their coordinate basis vector, e i. X a is just as easily calculated as TV (easier!), but which ones? All of them? That's impractical! One could process each X i though. To classify all s in S, we could first cluster S based on some notion of closeness (isotropic clusters), then take the cluster means as representatives of the entire cluster, classify those cluster means individually (giving the same class assignment to all other points in that cluster, addressing the curse of cardinality of S), or we can classify each s in S individually. In either case we the classify s in S as follows: For unclassified sample (mean of cluster or just any sample), s, find a set of epsilon contours (from TVX, IPX, X i' s ) that reduce the candidate near neighbor set to a manageable size. Select the candidate near neighbors that are Euclidean close enough (or the closest k of them). Let those selected near nbrs vote with Gaussian Radial Basis (RDF) weighted votes. Done!

5. TVX(a)- ε TVX(a)+ε IPX(a)+ ε IPX(a)- ε X 2 (a)+ ε X 2 (a)- ε µ To prune halos with a small number of contours (fewer that the n+1 contours: {TV; X i, i=1..n}, e.g., with 2 contours, use the X a-µ -contour or use just a few X i -contours corresponding to the largest coordinates of a-µ X a-μ (a)+ ε X a-μ (a)-ε halos, but small no halo, not too large X 1 (a+ ε X 1 (a- ε no halo,but very large a

6. ContourTVX(a,r) Contours of TVX, IPX, X a, X a-μ X y (a) 1 2 3 4 5 X Y 1 2 3 4 5 a ContourX y (a,r) ContourX x (a,r) 1 2 3 4 5 X Y 1 2 3 4 5 a μ μ ContourIPX(c,r) ContourX b (a,r) 1 2 3 4 5 X Y 1 2 3 4 5 a μ a-μ b ContourX a-μ (a,r) c Note: ContourTVX(a,r) = Contour(TV,X,a,r), etc. Γ Γ

7. How about higher dimensions? ContourX y (a,r) outside surface of ContourTVX(a,r) two (n-1)-D hypersurfaces (isobars) bounding ContourX a (a,r) outside surface of (n-1)D surface, ContourTVX(a,r)

8. Type-0 P-trees 101..0101..0 101..1101..1 100..1100..1 100..0100..0 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 100..1100..1 101..1101..1 101..1101..1 111..1111..1 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 001..1001..1 Ex.: dimension=1 fanout=2 dimensoin =2 depth=5 NumberOfPotentialLeaves=NOPL=32=fanout depth Leaf Existence Array, LA 31302928232221201514131211983210 1 101..0101..0 101..1101..1 100..1100..1 100..0100..0 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 001..1001..1 0 Type0: pure0 leaves omitted. pure1 s tagged in Purity Field. 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 Purity Field (PF) The upper inode levels are not productive. Indicate which leaves are either mixed or pure1 in a Leaf Existence Array (LA) with Purity Field (PF) or Leaf Existence Map (LM) and Purity Map (PM) Only the mixed leaves that get stored. Leaves are bit vectors, any or all of which could be compressed the same way (i.e., this 2-level structure can be nested to more levels). The LM is a "existential smoothing" of the Ptree (tells us precisely which leaves contain at least 1 1-bit). If we next enough, the LMs give us multiple smoothing levels. Needless to say, I prefer the LM/PM approach. The LM/PF may be clearer 0 1 1 1 1 0 0 0 0 1 1 0 1 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 Leaf Existence Map, LM (size=NOPL) 101..0101..0 101..1101..1 100..1100..1 100..0100..0 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 001..1001..1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 Purity Map (PM) Leaf Map (LM) 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 leaf length

9. Type-1 P-trees 1 1 1 1 1 0 0 0 0 1 1 0 1 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 Physical Structure: Leaf Exists Map or LM (size=NOPL=32) 101..0101..0 101..1101..1 100..1100..1 100..0100..0 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 001..1001..1 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 Purity Map (PM) Leaf Map (LM) 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Physical Structure: Leaf Existence Array or LA 31302928232221201514131211983210 1 101..0101..0 101..1101..1 100..1100..1 100..0100..0 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 001..1001..1 0 Type 0 means pure0 leaves omitted, pure1 s switched on 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 Purity Field The Type-0 Ptree above, can be expressed as Type-1: Using either a Leaf Existence Array (LA) with Pure Field (PF) or a Leaf Existence Map (LM) and a Purity Map (PM) 4,5,6,7 Physical Structure: Leaf Exists Array or LA 31302928 23 24,25,26,27 222120 15 16,17,18,19 141312 10 983210 1 101..0101..0 101..1101..1 100..1100..1 100..0100..0 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 101..1101..1 100..1100..1 101..1101..1 001..1001..1 1 Type1 means pure1 omitted, pure0 s tagged in Purity Field. 0 0 0 0 0 0 1 0 0 0 0 00 0 0 0 0 0 Purity Field (PF) 1 11 011

10. 0p2 How should Ptree be stored? 0p4 0p1 0p6 1p3 1p5 1 111 Leaf Map 1 1 Pure Map 1 1 1 Leaf Map 2 1 Pure Map 2 1 1 Leaf Map 4 1 Pure Map 4 1 1 Leaf Map 5 Pure Map 5 1 1 Leaf Map 3 Pure Map 3 1 1 1 Leaf Map 6 1 Pure Map 6 1 The tempting way to store these structures is to cluster by Ptree (horizontally across the rows of this cube above). But since those leaves almost never get ANDed with oneanother (except 1-time preprocessing) better to cluster by Leaf Offset or LM position ? ( vertically down this cube) since these are precisely the bit vectors that get ANDed together. If there is good compression (not too many mixed leaves per Ptree), then storing each Leaf Offset (vertical slice of the cube) on a page (or extent), would mean that only that page would need to be brought in when an actual AND is called for (and prefetching is straight forward). The collection of LMs and PMs could be stored separately on one extent, since they're processed separately (before) the leaves (or processed as smoothings). Type bit impure leaves leaf length 101000101000011

11. P-tree operation: COMPLEMENT Flip the Type Bit and complement the Leaves ( That's all! ) If the structure is nested, complement a leaf by flipping its TypeBit and complementing its leaves (leaves of the leaves), etc. 111..1111..1 101..1101..1 100..1100..1 111..1111..1 000..1000..1 100..0100..0 1817 11676 1 19 10 1. 1 10 0. 1 00 0. 1 10 0. 0 0 000..0000..0 010..0010..0 011..0011..0 000..0000..0 111..0111..0 0111.10111.1 1817 11676 1 19 1 010. 0 011. 0 111. 0 011 1. 1

12. P-tree operation: AND 0p2 0p4 0p1 0p6 1p3 1p5 1 111 0-Leaf Map 1 1 0Pure Map 1 1 1 0-Leaf Map 2 1 0Pure Map 2 1 1 0-Leaf Map 4 1 0Pure Map 4 1 1 1-Leaf Map 5 1Pure Map 5 1 1 1-Leaf Map 3 1Pure Map 3 1 1 1 0-Leaf Map 6 1 0Pure Map 6 1 Assumemixed leaves are clustered by Leaf Offset ( vertically down the cube), and the collection of LMs (and PMs) are stored separately on one additional extent. 1. AND all 0-LM s --> A 2. scan l-to-r across A for next 1 bit, if that position in any 1-PM=1, then GOTO 2 else fetch & AND nonpure leaves --> B; GOTO 2 3. A forms the LM of the result and the B s are the nonpure leaves. E.g., p1 ^ p3 ^ p6 1. 1001100001^ 1000001001^ 1000010000= 1000000000 0 1 2 3 4 5 6 7 8 9 positions 101000101000011101000111110011 101000000000011 101000101000000 111100101000011 111000000000011 000000000011011 000011000000001 000011000010101 2. pos=1, PM 3 (1)=0 so fetch & AND p1 101000101000011 p3 111100101000011 p6 000011000010101 res 000000000000001 3. Result Ptree: 0-Leaf_Map: 1000000000 0-Pure_Map: 0000000000 impure leaves: 000000000000001 root-count = 1 In ASM, is there an operation, AND and COUNT? to count 1-bits as they are produced? Even better: ^{0LM} ^{1PM'}  0LM (result is always type0) Fetch & AND leaves corresp. to 1-bits in 0LM. Set Purity Map.

13. K 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U 5 6 120000000000000000000000001011110100111010101001111111111123120000000000000000000000001011110100111010101001111111111123 1300000000000000000000000001111111000000000000000000000000713000000000000000000000000011111110000000000000000000000007 110011001100110011010101010111111111001111111110010100101034110011001100110011010101010111111111001111111110010100101034 1001010101010101011100110001010100010100000000101000011001221001010101010101011100110001010100010100000000101000011001222 230000000000000000000000000111110011111111111111111000000021230000000000000000000000000111110011111111111111111000000021 210000000011111111000011111111111011111110000111110111100035210000000011111111000011111111111011111110000111110111100035 200000111100001111001100111000111111111001100111111100011033200000111100001111001100111000111111111001100111111100011033 220000000000000000111111111111111011111111111000001111111134220000000000000000111111111111111011111111111000001111111134 LeafSize=8, NOPL=7 LeafOff=0; 11 10 20 001100110101010100001111 0-p13 01111111 0-p12 10111101 00111010 10100111 1-p11 00110011 01010101 01111111 11001111 11111001 01001010 0-p10 01010101 11001100 01010100 01010000 00001010 00011001 0-p23 01111100 10000000 1-p22 11111110 11100000 1-p21 00001111 11111110 00011111 01111000 0-p20 00001111 00110011 10001111 11111001 10011111 11000110 13 0 1 0 12 0 1 11 1 10 1 23 0 1 22 1 0 1 0 1 0 21 1 0 1 20 1 12 0 1 23 0 1 0 22 1 0 21 1 0 LeafOff=1; 11 10 20 001100110101010100001111 LeafOff=2; 11 10 21 20 01010101110011000000111100110011 LeafOff=3; 13 12 11 10 23 22 21 20 0111111110111101 011111110101010001111100111111101111111010001111 LeafOff=4; 12 11 10 21 20 001110101100111101010000 1111111011111001 LeafOff=5; 12 11 10 22 21 20 101001111111100100001010 111000000001111110011111 LeafOff=6; 11 10 23 21 20 010010100001100110000000 0111100011000110 Leaf Maps (red are type-1) 13 0 1 0 12 0 1 11 1 10 1 23 0 1 22 1 0 1 0 1 0 21 1 0 1 20 1 Purity Maps 12 0 1 23 0 1 0 22 1 0 21 1 0 13 00000000 01111111 00000000 12 00000000 10111101 00111010 10100111 11111111 11 00110011 01010101 01111111 11001111 11111001 01001010 10 01010101 11001100 01010100 01010000 00001010 00011001 23 00000000 01111100 11111111 10000000 22 00000000 11111111 11111110 11111111 11100000 11111111 21 00000000 11111111 00001111 11111110 00011111 01111000 20 00001111 00110011 10001111 11111001 10011111 11000110

14. lo0; 11 10 20 001100110101010100001111 13 0 1 0 12 0 1 11 1 10 1 23 0 1 22 1 0 1 0 1 0 21 1 0 1 20 1 lo1; 11 10 20 001100110101010100001111 lo2; 11 10 21 20 01010101110011000000111100110011 lo3; 13 12 11 10 23 22 21 20 0111111110111101011111110101010001111100111111101111111010001111 lo4; 12 11 10 21 20 0011101011001111010100001111111011111001 lo5; 12 11 10 22 21 20 101001111111100100001010111000000001111110011111 lo6; 11 10 23 21 20 0100101000011001100000000111100011000110 LeafMaps; (red=type1) P13^p12 13 0 1 0 12 0 1 0LM 0 1 0 lo3; 13 12 01111111 10111101 00111101 rc=5 P11^p10 ^{0LM} ^{1PM'}  0LM Fetch & AND lo=0,1,2,3,4,5,6 LM 1 10 1 PureMaps; 12 0 1 23 0 1 0 22 1 0 21 1 0 lo=0; 11 10 00110011 01010101  00010001 rc=2 lo=1; 11 10 00110011 01010101  00010001 rc=2 lo=2; 11 10 01010101 11001100  01000100 rc=2 lo=3; 11 10 01111111 01010100  01010100 rc=3 lo=4; 11 10 11001111 01010000  01000000 rc=1 lo=5; 11 10 11111001 00001010  00001000 rc=1 lo=6; 11 10 01001010 00011001  00001000 rc=1 Total rc=12 ^{0LM} ^{1PM'}  0LM Fetch & AND lo=3

15. lo0; 11 10 20 001100110101010100001111 13 0 1 0 12 0 1 11 1 10 1 23 0 1 22 1 0 1 0 1 0 21 1 0 1 20 1 lo1; 11 10 20 001100110101010100001111 lo2; 11 10 21 20 01010101110011000000111100110011 lo3; 13 12 11 10 23 22 21 20 0111111110111101011111110101010001111100111111101111111010001111 lo4; 12 11 10 21 20 0011101011001111010100001111111011111001 lo5; 12 11 10 22 21 20 101001111111100100001010111000000001111110011111 lo6; 11 10 23 21 20 0100101000011001100000000111100011000110 LeafMaps; (red=type1) P22^p21 ^{0LM} ^{1PM'}  0LM Fetch & AND lo of 0LM 1-bit positions (i.e., 2,3,4,5,6) for P21, p22 (those that exits) LM 0 1 PureMaps; 12 0 1 23 0 1 0 22 1 0 21 1 0 22' 0 1 21' 0 1 lo2;21 00001111 lo3; 22 11111110 21 11111110 lo4; 21 11111110 lo5; 22 11100000 21 00011111 lo6; 21 01111000 00001111 rc=4 11111110 rc=7 00000000 rc=0 01111000 rc=4 Total rc=22

16. lo0; 11 10 20 001100110101010100001111 13 0 1 0 12 0 1 11 1 10 1 23 0 1 22 1 0 1 0 1 0 21 1 0 1 20 1 lo1; 11 10 20 001100110101010100001111 lo2; 11 10 21 20 01010101110011000000111100110011 lo3; 13 12 11 10 23 22 21 20 0111111110111101011111110101010001111100111111101111111010001111 lo4; 12 11 10 21 20 0011101011001111010100001111111011111001 lo5; 12 11 10 22 21 20 101001111111100100001010111000000001111110011111 lo6; 11 10 23 21 20 0100101000011001100000000111100011000110 LeafMaps; (red=type1) P22^p21^p13^p12 ^{0LM} ^{1PM'}  0LM Fetch & AND lo of 0LM 1-bit positions (i.e., lo3) for 21, 22 13, 12 (those that exits) LM 0 1 0 PureMaps; 12 0 1 23 0 1 0 22 1 0 21 1 0 22' 0 1 21' 0 1 13 0 1 0 12 0 1 lo3; 13 12 22 21 01111111 10111101 11111110 00111100 rc=4

17. lo0; 11 10 20 001100110101010100001111 13 0 1 0 12 0 1 11 1 10 1 23 0 1 22 1 0 1 0 1 0 21 1 0 1 20 1 lo1; 11 10 20 001100110101010100001111 lo2; 11 10 21 20 01010101110011000000111100110011 lo3; 13 12 11 10 23 22 21 20 0111111110111101011111110101010001111100111111101111111010001111 lo4; 12 11 10 21 20 0011101011001111010100001111111011111001 lo5; 12 11 10 22 21 20 101001111111100100001010111000000001111110011111 lo6; 11 10 23 21 20 0100101000011001100000000111100011000110 LeafMaps; (red=type1) P22^p21 ^{0LM} ^{1PM'}  0LM Fetch & AND lo of 0LM 1-bit positions (i.e., 2,3,4,5,6) for P21, p22 (those that exits) LM 0 1 PureMaps; 12 0 1 23 0 1 0 22 1 0 21 1 0 22' 0 1 21' 0 1 lo2;21 00001111 lo3; 22 11111110 21 11111110 lo4; 21 11111110 lo5; 22 11100000 21 00011111 lo6; 21 01111000 00001111 rc=4 11111110 rc=7 00000000 rc=0 01111000 rc=4 Total rc=22

18. Vertical Data Assistant (VDA) is a windows (or windows CE) application that can data mine massive datasets efficiently? (note that the a separate application can be built to convert an store properly. I am changing my thinking a bit on DataMIME and the whole idea of competing in the "big iron" community with a data mining system. There is no way to win there (too many players with too much money - Google, Microsoft, all the Bioinformatics and Drug development companies....). It occurred to me, as I was revising Masum's slides, that other than the size of the datasets, our methods and modules are getting surprisingly simple and compact. Developing DataMIME on big hardware may be a mistake, if no one uses it! Maybe it is wrong-headed? Maybe the killer app is a completely portable, tiny client system (on a desktop/laptop/PDA) that can do scalable data mining for "anyone, anytime, anywhere" provided their data is capture into a consistent universally format. If this would work, it could eventually be scaled up to supercomputers and as a Grid App, but first, let's do a tiny version (tiny in terms of system requirements, but not in terms of DM power). DataSURG software development (e.g., SMILEY, DataMIME) weren't too successful. The one software system has been used is TM-Lab, a small app on small PCs and requiring data be in a simple format (BSQ). I now favor a Windows and Windows CE Utility Suite approach. Each Utility does one thing we and can be invoked in a GUI drag and drop mode. (e.g., An AND operation pulls the specified Ptrees from the specified folder and returns either just the rootcount or drops the resulting "derived Ptree" back in the folder and returns its name. Each platform (PDA to supercomputer) will get an appropriate subset of the suite.

19. Body Area Network (BAN) and Network is the Computer (NITC) technologies may eventually converge to produce the next "sea change" technology. Imagine nano-computers, massive storage (e.g., 4G thumb drives), a wireless networks and applications such as real-time health monitors (blood_emzynes/body_temperature/blood_sugar/coronary_flow...); PDA apps (schedulers/reminders/auto_thought_recorders, DNA/FingerPrint/FaceGeo/IrisScan realtime name_recall/restraining_order_enforcement/homeland_security); Environmental sensors (eyes_in_back_of_head/sense_nearby_explosives/viruses_sensing...). All that data will have to be data mined for exceptions. Exception mining (i.e., classification into 2 classes, exceptional_situation or normal_situation) is what DataSURG does better than anyone in the world (add "ego monitoring" to VDA apps ;-) Our approach can be implemented on tiny processing platforms. Of course the datasets are large, so they must be broken up and compressed and store in an extremely simple and universal (no variations!) format. So simple that Windows CE can handle them and so that Joe Public can understand them. After all, simplicity and consistency made the Relational Model successful!

20. K 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V 5 7 12000000000000000000000000101111010011101010100111111111111241200000000000000000000000010111101001110101010011111111111124 130000000000000000000000000111111100000000000000000000000007130000000000000000000000000111111100000000000000000000000007 11001100110011001101010101011111111100111111111001010010101351100110011001100110101010101111111110011111111100101001010135 10010101010101010111001100010101000101000000001010000110011231001010101010101011100110001010100010100000000101000011001123 23000000000000000000000000011111001111111111111111100000000222300000000000000000000000001111100111111111111111110000000022 21000000001111111100001111111111101111111000011111011110000352100000000111111110000111111111110111111100001111101111000035 20000011110000111100110011100011111111100110011111110001100332000001111000011110011001110001111111110011001111111000110033 22000000000000000011111111111111101111111111100000111111111352200000000000000001111111111111110111111111110000011111111135 d e f g 9 a b c 5 6 7 8 1 2 3 4 n o l m j k h i t u r s v w x y z A B C D E N F G H I J K L M q p O P Q R S T U V 0000 0001 0010 0011 0100 0101 0110 01111000 1001 1010 1011 1100 1101 1110 1111 1111 1110 1101 1100 1011 1010 1001 1000 0111 0110 0101 0100 0011 0010 0001 0000 µ i = (  x  X x i ) / |X| =(1/57)  x  k 2 k x i, (1/57)  k 2 k  x x i,k =(1/57)  k 2 k rcP i,k for i=1 =(1/57)*(7*2 3 +24*2 2 +35*2 1 +23*2 0 ) =4.3 for i=2 =(1/57)*(22*2 3 +35*2 2 +35*2 1 +33*2 0 ) =7.35 µ Created a bit map for each {ring(f,Y,k)} k=1..K, of each Y={x}, one would have all reverse-kNN information and much more! ring(f,Y,k) s give up distance information, so {ring(f,Y,(kr, (k+1)r])} k=0..R holds more info, but makes it more difficult to computer reverse-kNN sets (though more useful sets can easily be calculated to do a better job of whatever reverse-kNN sets were to be used for anyway).

21. K 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V 5 7 12000000000000000000000000101111010011101010100111111111111241200000000000000000000000010111101001110101010011111111111124 130000000000000000000000000111111100000000000000000000000007130000000000000000000000000111111100000000000000000000000007 11001100110011001101010101011111111100111111111001010010101351100110011001100110101010101111111110011111111100101001010135 10010101010101010111001100010101000101000000001010000110011231001010101010101011100110001010100010100000000101000011001123 23000000000000000000000000011111001111111111111111100000000222300000000000000000000000001111100111111111111111110000000022 21000000001111111100001111111111101111111000011111011110000352100000000111111110000111111111110111111100001111101111000035 20000011110000111100110011100011111111100110011111110001100332000001111000011110011001110001111111110011001111111000110033 22000000000000000011111111111111101111111111100000111111111352200000000000000001111111111111110111111111110000011111111135 d e f g 9 a b c 5 6 7 8 1 2 3 4 n o l m j k h i t u r s v w x y z A B C D E N F G H I J K L M q p O P Q R S T U V 0000 0001 0010 0011 0100 0101 0110 01111000 1001 1010 1011 1100 1101 1110 1111 1111 1110 1101 1100 1011 1010 1001 1000 0111 0110 0101 0100 0011 0010 0001 0000 µ ring(a,d*2 k-1,d*2 k ) k=0... tell a great story on neighbors. (here, d=1) We note that the ring(a,d*(k-1),d*k) tell an even greater story but they may have higher cost? r a,0 =skin(a,1) r a,k =ring(a,2 k-1,2 k ) r f,a,k =ring(f,f -1 a,2 k-1,2 k ) 100100010001000100100010000000000000000000000000000000000006100100010001000100100010000000000000000000000000000000000006 11001000100010001000010001000000001000010101010000000000000111100100010001000100001000100000000100001010101000000000000011 12000100010001000101000100100000000110000000001100101001000141200010001000100010100010010000000011000000000110010100100014 22000000000000000000000000100000000011000000000110101101010102200000000000000000000000010000000001100000000011010110101010 20101010101010101000110011000000001000010101010000000000000172010101010101010100011001100000000100001010101000000000000017 21000100010001000101000100000000000100000000001000000000000 821000100010001000101000100000000000100000000001000000000000 8 r x 1,1,k 32000000000000000000000000000000000001101010100011010100110113200000000000000000000000000000000000110101010001101010011011 30010101011001010111001100000000000100000000001000000000000143001010101100101011100110000000000010000000000100000000000014 31100010001000100000100010100000000010000000000100101001000123110001000100010000010001010000000001000000000010010100100012

22. K 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V 5 7 d e f g 9 a b c 5 6 7 8 1 2 3 4 n o l m j k h i t u r s v w x y z A B C D E N F G H I J K L M q p O P Q R S T U V 0000 0001 0010 0011 0100 0101 0110 01111000 1001 1010 1011 1100 1101 1110 1111 1111 1110 1101 1100 1011 1010 1001 1000 0111 0110 0101 0100 0011 0010 0001 0000 µ 40001000100010001000010001100000001010010101010100101001000174000100010001000100001000110000000101001010101010010100100017 41010001000100010010001000000000000001000000000010000100010104101000100010001001000100000000000000100000000001000010001010 42100010001000100000100010000000000000101010100001010010101154210001000100010000010001000000000000010101010000101001010115 S211001100110011001010101000000000000000000000000000001000114S211001100110011001010101000000000000000000000000000001000114 S000010001000100010100010000000000010100000000101000010001012S000010001000100010100010000000000010100000000101000010001012 S100100010001000100001000100000000100011111111000101000010018S100100010001000100001000100000000100011111111000101000010018 U200000000000000000000000000000000000110101010001101010011011U200000000000000000000000000000000000110101010001101010011011 U000000000000000000000000010000000001010101010010111100110013U000000000000000000000000010000000001010101010010111100110013 U100010001000100010100010000000000010000000000100000001000110U100010001000100010100010000000000010000000000100000001000110

23. K 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V 5 7 d e f g 9 a b c 5 6 7 8 1 2 3 4 n o l m j k h i t u r s v w x y z A B C D E N F G H I J K L M q p O P Q R S T U V 0000 0001 0010 0011 0100 0101 0110 01111000 1001 1010 1011 1100 1101 1110 1111 1111 1110 1101 1100 1011 1010 1001 1000 0111 0110 0101 0100 0011 0010 0001 0000 µ T0000000000000000000000000000000000001000000000010000110011 6T0000000000000000000000000000000000001000000000010000110011 6 T1000000000000000000000000100000000010000000000100101001000 6T1000000000000000000000000100000000010000000000100101001000 6 T200110011001100110101010100000010110001010101100000000000020T200110011001100110101010100000010110001010101100000000000020 V200010001000100010100010011000010011000000000110010100100016V200010001000100010100010011000010011000000000110010100100016 V0000000000000000000000000000000000000101010100001010000100 7V0000000000000000000000000000000000000101010100001010000100 7 V1000000000000000000000000000000000001000000000010000100010 4V1000000000000000000000000000000000001000000000010000100010 4 v200000000000000000000000000101001000010101010000101001010112v200000000000000000000000000101001000010101010000101001010112 v0000000000000000000000000010000000000000000000000000000000 1v0000000000000000000000000010000000000000000000000000000000 1 v1000000000000000000000000000000000000000000000000000000000 0v1000000000000000000000000000000000000000000000000000000000 0

24. K 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V 5 7 d e f g 9 a b c 5 6 7 8 1 2 3 4 n o l m j k h i t u r s v w x y z A B C D E N F G H I J K L M q p O P Q R S T U V 0000 0001 0010 0011 0100 0101 0110 01111000 1001 1010 1011 1100 1101 1110 1111 1111 1110 1101 1100 1011 1010 1001 1000 0111 0110 0101 0100 0011 0010 0001 0000 q0000000000000000000000000000000100000000000000000000000000 1q0000000000000000000000000000000100000000000000000000000000 1 q1000000000000000000000000000000000000000000000000000000000 0q1000000000000000000000000000000000000000000000000000000000 0 q2000000000000000000000000001111010000000000000000000010001 7q2000000000000000000000000001111010000000000000000000010001 7 w2000000000000000000000000010000100000000000000000000000000 2w2000000000000000000000000010000100000000000000000000000000 2 w0000000000000000000000000000101000000000000000000000000000 2w0000000000000000000000000000101000000000000000000000000000 2 w1000000000000000000000000000000000000000000000000000000000 0w1000000000000000000000000000000000000000000000000000000000 0 s2000000000000000000000000010000000000000000000000000000000 1s2000000000000000000000000010000000000000000000000000000000 1 s0000000000000000000000000001010010000000000000000000000000 3s0000000000000000000000000001010010000000000000000000000000 3 s1000000000000000000000000000000000000000000000000000000000 0s1000000000000000000000000000000000000000000000000000000000 0

25. K 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V 5 7 d e f g 9 a b c 5 6 7 8 1 2 3 4 n o l m j k h i t u r s v w x y z A B C D E N F G H I J K L M q p O P Q R S T U V 0000 0001 0010 0011 0100 0101 0110 01111000 1001 1010 1011 1100 1101 1110 1111 1111 1110 1101 1100 1011 1010 1001 1000 0111 0110 0101 0100 0011 0010 0001 0000 10000011110000000000000000000000010000000000000000000000000 510000011110000000000000000000000010000000000000000000000000 5 11000000001111000000000000000000000000000000000000000000000 411000000001111000000000000000000000000000000000000000000000 4 12000000000000111111000000000000000000000000000000000000011 812000000000000111111000000000000000000000000000000000000011 8 52000000000000000011110000000000000000000000000000000001111 852000000000000000011110000000000000000000000000000000001111 8 50111100001111000000000000000000000000000000000000000000000 850111100001111000000000000000000000000000000000000000000000 8 51000000000000111100000000000000000000000000000000000000000 451000000000000111100000000000000000000000000000000000000000 4 92000000000000000000111100000000000000000000000000001111100 992000000000000000000111100000000000000000000000000001111100 9 90000011110000111100000000000000010000000000000000000000000 990000011110000111100000000000000010000000000000000000000000 9 91111100000000000011000000000000000000000000000000000000011 891111100000000000011000000000000000000000000000000000000011 8

26. K 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V 5 7 d e f g 9 a b c 5 6 7 8 1 2 3 4 n o l m j k h i t u r s v w x y z A B C D E N F G H I J K L M q p O P Q R S T U V 0000 0001 0010 0011 0100 0101 0110 01111000 1001 1010 1011 1100 1101 1110 1111 1111 1110 1101 1100 1011 1010 1001 1000 0111 0110 0101 0100 0011 0010 0001 0000 d0000000001111000011000000000000000000000000000000000000011 8d0000000001111000011000000000000000000000000000000000000011 8 d1000011110000000000110000000000010000000000000000000001100 9d1000011110000000000110000000000010000000000000000000001100 9 d211110000000000000000111110000010000000000000000001111000014d211110000000000000000111110000010000000000000000001111000014 h211111111000000000000001110000011000000000000000001000000014h211111111000000000000001110000011000000000000000001000000014 h0000000000000111100110000000000000000000000000000000001100 8h0000000000000111100110000000000000000000000000000000001100 8 h1000000001111000000001100000000000000000000000000001110000 9h1000000001111000000001100000000000000000000000000001110000 9 j2000011111111000000000000000000010000000000000000000000000 9j2000011111111000000000000000000010000000000000000000000000 9 j0000000000000000011001100000000000000000000000000001110011 9j0000000000000000011001100000000000000000000000000001110011 90 j1000000000000111100000011100000100000000000000000010000000 9j1000000000000111100000011100000100000000000000000010000000 9

27. K 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V 5 7 d e f g 9 a b c 5 6 7 8 1 2 3 4 n o l m j k h i t u r s v w x y z A B C D E N F G H I J K L M q p O P Q R S T U V 0000 0001 0010 0011 0100 0101 0110 01111000 1001 1010 1011 1100 1101 1110 1111 1111 1110 1101 1100 1011 1010 1001 1000 0111 0110 0101 0100 0011 0010 0001 0000 l0000000000000000000110011100000100000000000000000010001100 9l0000000000000000000110011100000100000000000000000010001100 9 l1000000000000000011000000000000000000000000000000000000011 4l1000000000000000011000000000000000000000000000000000000011 4 l2000000001111111100000000000000000000000000000000000000000 8l2000000001111111100000000000000000000000000000000000000000 8 n200000000000011111100000000000000000000000001111100000001113n200000000000011111100000000000000000000000001111100000001113 n0000000000000000000001100000000000000000000000000001110000 5n0000000000000000000001100000000000000000000000000001110000 5 n1000000000000000000110000000000000000000000000000000001100 4n1000000000000000000110000000000000000000000000000000001100 4

28. K 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V 5 7 d e f g 9 a b c 5 6 7 8 1 2 3 4 n o l m j k h i t u r s v w x y z A B C D E N F G H I J K L M q p O P Q R S T U V 0000 0001 0010 0011 0100 0101 0110 01111000 1001 1010 1011 1100 1101 1110 1111 1111 1110 1101 1100 1011 1010 1001 1000 0111 0110 0101 0100 0011 0010 0001 0000 I0000000000000000000000000000000000000000001100000000000000 2I0000000000000000000000000000000000000000001100000000000000 2 I1000000000000000000000000000000000000000110000000100000000 3I1000000000000000000000000000000000000000110000000100000000 3 I200000000000000000000001111111110111111100000000001000000017I200000000000000000000001111111110111111100000000001000000017 G2000000000000000000000000000011001111100000000000000000000 7G2000000000000000000000000000011001111100000000000000000000 7 G0000000000000000000000000000000000000000110011111100000000 8G0000000000000000000000000000000000000000110011111100000000 8 G1000000000000000000000000011100000000011000000000000000000 5G1000000000000000000000000011100000000011000000000000000000 5 E2000000000000000000000000000000000000000000000000000000000 0E2000000000000000000000000000000000000000000000000000000000 0 E0000000000000000000000000011100000000011001100000000000000 7E0000000000000000000000000011100000000011001100000000000000 7 E100000000000000000000000000001100111110000001111100000000012E100000000000000000000000000001100111110000001111100000000012

29. K 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V 5 7 d e f g 9 a b c 5 6 7 8 1 2 3 4 n o l m j k h i t u r s v w x y z A B C D E N F G H I J K L M q p O P Q R S T U V 0000 0001 0010 0011 0100 0101 0110 01111000 1001 1010 1011 1100 1101 1110 1111 1111 1110 1101 1100 1011 1010 1001 1000 0111 0110 0101 0100 0011 0010 0001 0000 C000000000000000000000000000001100111110011000000010000000010C000000000000000000000000000001100111110011000000010000000010 C1000000000000000000000000000000000000000001100000000000000 2C1000000000000000000000000000000000000000001100000000000000 2 C2000000000000000000000000000000000000000000011111000000000 5C2000000000000000000000000000000000000000000011111000000000 5 x2000000000000000000000000000000000000000001111111000000000 7x2000000000000000000000000000000000000000001111111000000000 7 x0000000000000000000000000011100000000011000000000000000000 5x0000000000000000000000000011100000000011000000000000000000 5 x1000000000000000000000000000000000000000110000000100000000 3x1000000000000000000000000000000000000000110000000100000000 3 µ

30. K 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V 5 7 d e f g 9 a b c 5 6 7 8 1 2 3 4 n o l m j k h i t u r s v w x y z A B C D E N F G H I J K L M q p O P Q R S T U V 0000 0001 0010 0011 0100 0101 0110 01111000 1001 1010 1011 1100 1101 1110 1111 1111 1110 1101 1100 1011 1010 1001 1000 0111 0110 0101 0100 0011 0010 0001 0000 µ x TVX gap  2415 13 p 2428 97 P 2525 21 Q 2546 38 O 2584 34 m 2618 110 o 2728 8 S 2736 156 T 2892 38 R 2930 106 k 3036 46 U 3082 64 l 3146 8 i 3154 26 K 3180 21 L 3201 72 J 3273 63 M 3336 130 V 3466 14 I 3480 4 n 3484 109 g 3593 89 h 3682 110 j 3792 8 f 3800 9 H 3809 144 G 3953 168 e 4121 25 c 4146 94 N 4240 24 v 4264 89 b 4353 43 F 4396 144 E 4540 16 d 4556 118 a 4674 139 8 4813 207 7 5020 77 D 5097 12 9 5109 132 C 5241 100 6 5341 253 4 5594 162 z 5756 20 5 5776 1 A 5777 24 3 5801 48 y 5849 63 B 5912 144 x 6056 66 2 6122 435 1 6557 925 q 74822799 r10281 815 t11096 125 w11221 221 s11442 815 u12257 000000000000000000000000100000000000000000000000000000000 1000000000000000000000000100000000000000000000000000000000 1 TV-countours bounded by isobar gaps of at least 57*radial_distance_from_  =(7.4, 4.3) m000000000000000000000100000000000000000000000000011100000 4m000000000000000000000100000000000000000000000000011100000 4 T000000000000000000000001000000000000000000000000000001100 3T000000000000000000000001000000000000000000000000000001100 3 M000000000000000001011000000000000000000000001111000010010 9M000000000000000001011000000000000000000000001111000010010 9 57*1 57*2 57*3 57*4 H000000000000001110100010000000000000000000110000000000001 8H000000000000001110100010000000000000000000110000000000001 8 57*5 F000000000011010000000000000000100000000011000000100000000 7F000000000011010000000000000000100000000011000000100000000 7 57*6 C000000111100100000000000000000000000011100000000000000000 8C000000111100100000000000000000000000011100000000000000000 8 57*7 x001111000000000000000000000000001111100000000000000000000 9x001111000000000000000000000000001111100000000000000000000 9 1110000000000000000000000000000000000000000000000000000000 21110000000000000000000000000000000000000000000000000000000 2 57*8 000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000 0 57*9 q000000000000000000000000000000000000000000000000000000000 1q000000000000000000000000000000000000000000000000000000000 1 57*11 s000000000000000000000000001110010000000000000000000000000 4s000000000000000000000000001110010000000000000000000000000 4 57*14 u000000000000000000000000000001000000000000000000000000000 1u000000000000000000000000000001000000000000000000000000000 1 57*15

31. d e f g 9 a b c 5 6 7 8 1 2 3 4 n o l m j k h i t u r s v w x y z A B C D E N F G H I J K L M q p O P Q R S T U V 0000 0001 0010 0011 0100 0101 0110 01111000 1001 1010 1011 1100 1101 1110 1111 1111 1110 1101 1100 1011 1010 1001 1000 0111 0110 0101 0100 0011 0010 0001 0000 µ  =(7.4, 4.3) We note taking a = (  1, 0) and a = (0,  2 ) the 4 resulting TV-X a -contours nearly partition a thick ring. Thickening the Xa-contours even more, gives better coverage without increasing the neighborhood much. Does this hold true in higher dimensions? Do we need to consider other diagonals, e.g., (0,0..,  i,0,..,  j,0,..,0) etc. ?

32. FAML Vector Space Classification using sorting (ala SMART-TV) Given T=R(A 1,…,A n,C) and X=R[A 1,…,A n ] 1.Create and store the 2-column n tables, X i (x,f i (x)) sorted on f i (x) 1.equivalently, [ala 4 th NF] decompose R(A 1..A n,C,A f 1..A f m ) into T, X 1,..., X m sorted on f i (x). 2.equiv, create secondary indexes, X i for each derived attribute, A f i on R(A 1..A n, C, A f 1..A f m ) 3.Store 3-col, X i (x,f i (x),x.C) so votes are handy (# coding classes so bitwidth = log 2 |C| )? 2.For an unclassified sample, a, calculate {f i (a)} and retrieve the {cskin(f i (a),r)} [or cskin(f i (a),k)]. 3.Form contour(f i,cskin(f i (a),r)) and intersect them, to get a candidate near neighbor set. 4.For every candidate, x, in that set, if d(x,a)<threshold, tally RBF-weighted vote of x. Sorting is expensive, (creating indexes) even tho it is 1-time activity which can be amortized over all classifications, it may be too expensive for very large data sets. (If SMART-TV is essentially "creating indexes on derived attributes") FAML Vector Space Classification using P-trees (includes PINE) To address curse of cardinality (mostly sorting) we use P-tree technology on derived attributes: 1.Create basic P-trees for each derived attribute, A f i 2.Using EIN technology, create P-tree mask for the contour(f i,cskin(a,r)), AND them 3.For every x in the resulting mask, if d(x,a)<threshold, tally RBF-weighted vote of x. Next we examine some specific functionals (dually, derived attributes). First some notation: A functional, f:X  R where Dom(R)={0,1} (binary) is called a predicate. The derived attribute, A f will have bitwidth=1 and is called a derived map. When starting with a functional, f:X  R, the dual derived attribute will be denoted, A f. When starting with a derived attribute, A, the dual functional will be denoted f A :X  R.

33. Coordinate Projection X i (x) =x i, is just X e i (X i 's define L ∞ -nbrhds in EIN-PINE technology). IPX(a) =  x  X xoa =  x  X X a (x) is TAPP with ContourIPX(a,r) linear hyper-bar  to µ) FAML Classification Given R(A 1..A n,C)=Training Space X=R[A 1..A n ]=Feature Space (since R,X same key, we'll use X.C) Functionals, {f i :dom(A 1 ) ..  dom(A n )  Reals} (e.g., TV; IP; Projection onto a, X a ) Define: Contour(f i, X, a, r) = {x  X  :|f i (x)-f i (a)|<r} (polar). Isobar(f i, X, a, r) = {x  X  :|f i (x)-f i (a)|=r}. ContourNbrhd(X,a,r) =  i Contour(f i,X,a,r). Basic Contour classification algorithm:  x  ContourNbrhd, if dist(a,x)< ε, then tally x.C-vote as a [weighted] vote for a.C. TVX(  )=TVX(x 33 ) 1 2 3 4 5 X Y TVX 1 2 3 4 5 a Contour(TV,X,a,r) r  b Contour(X a,X,b,r) TVX(a)

34. FAML Classification (early version)  x  X and  f, compute f(X). Store Table(f i ) consisting of (x, f i (x))-tuples sorted on f i. (or basic Ptrees of derived attributes?)  unclassified, a: Calculate {f i (a)} subset of i's. (No calculation is required for X i (a)=a i ), Retrieve portion of Table(f i ) at f i (a), NBR(f i (a)). (Count/radius based? Different k/ r for each f i ?) Construct contours, f -1 (NBR(f i (a)). Prune halos (by intersecting other contours until |  Contour(X,f i,a,r)|<threshold. Prune to VoterSet, by checking d(a,x).  x close enough, cast x s vote using weighted (Gaussian Radial Basis) vote. (Note that the purple part could be done in bulk on the set of all unclassified samples, S, by clustering S by distance, then apply Purple part to each component rather). FAML Clustering? One pervasive use of clustering is for class identification or class generation, that is, to identify sets of highly similar objects that might form the classes in a training set and for which subsequent samples can be classified against. These generated classes can be isotropic or density based (in either case subsequent classification can correctly based on near neighbors.) In fact, if class generation is done exclusively for the purpose of creating a training set for subsequent classification, then isotropic clustering will always suffice (that is, the round pre-clusters do not have to be joined into arbitrarily shaped clusters since the near neighbor set will determine the class assignment anyway (independent of the other round pre-clusters that might or might not get connected up to that one). Assume a [large] data set of historical tuples and that we want to use them to classify subsequent tuples we may receive from, say, a data stream. First we want to generate classes with historical data set thru isotropic clustering (from a recent window of the stream or the whole stream so far). Then we want to use that "classed" dataset as a training set for near neighbor classification. We can partition into core, core nbr, border, and noise points ala Rana's method) using functional near nbr sets (intersecting all or just some of the n+2 functional contours) instead of epsilon-nbrhds. Then the border pts can be attached to the best (closest/ most overlapping) core cluster. Then classify according to overlap with these clusters???

35. POLAR Neighborhoods Construct contours, f -1 (NN) to prune halos on NN until |NN|<threshold, revisited. First cluster the unclassified sample set, S, Identify dense angular contours in S-μ, Acontour(a-μ,θ); Identify dense radial contours in S-μ, Rcontour(a-μ,r) ( TVXcontour(a,r) ) Intersect them and pick out the dense Polar Neighborhood, PN(a-μ,r,θ) (no halos) For dense PN s in S-μ roughly classify the entire PN (pruned by Euclidean dist. to be isotropic) by classifying one representative! Then classify the rest of S, one at a time. a-μ θ The Polar (r,θ)-Neighborhood at a, PN(a,r,θ) =AContour(a,θ)  RContour(a,r) is an interesting gridding for grid based clustering. Can we easily construct this partition? Acontour(a,θ)={x  X |  a x <θ}={x  X | x/|x|  a/|a| < cosθ } = {x  X | x  a<|x||a|cosθ)}. Acontour(a-μ,θ)={x  X |  x a-μ <θ}={x  X | x/|x|  (a-μ)/|a-μ|<cosθ} a-μ

36. Non-uniform Polar Griddings  a  DomA 1 ..  DomA n, projection onto a, X a (x)=x  a=  i=1..n x i *a i Contour(X a,X,b,r) is a linear (n-1)-dimensional hyper-bar through b perpendicular to a. Contour(TV,X,a,r)  Contour(X a,X,a,θ) is an approximation of PN(a,r,θ). X a (x) values are easier to calculated than TVX(x)c a r θ Consider a very non-uniform gridding (for finding Dense Cores in clustering or for finding Near Neighbor Sets): Determine the densest rings (about μ) Within dense rings determine directions, a, such that the green nbrhds are dense and extract them. Treat the remainder of x one point at a time. 0 r 2r 3r Non-uniform Parallel Griddings  dimension, e i, partition into r-slices (actually, we should use j-low griddings?) Starting on one side, determine the count in each r-slice by determining the count in the first one (one < inequality). P-tree mask it (for j-low griddings this is already determined). Do the same for the next one and then AND off the first.... As in previous slide, determine the dense cells (j-low cells) and treat the remaining points as one large (but sparse) partition. Each partition is P-tree masked.

37. APPENDIX (slides that may be unnecessary) No halos! But what is the functional, X a-  ? The functional is clearly projection onto the a direction or f a-  (x) = x o (a-  )/|a-  |. Is this PrePreprocessing Compliant? (Taufik Abidin PreProcessing Compliant)? I think (from the chapter in the book) we can conclude that it’s quickly computed. 1 2 3 4 5 X Y 1 2 3 4 5 a ContourX a (a- ,r) ContourTVX(a,r) Xa = X  {g(x,a)|x  X} has mean(Xa) = a so IPXaContours are  a a x x-a (x-a)  a/|a| (x-a)  a/|a| - (x-a) g(x,a)=x+2[(x-a)  a/|a| - (x-a)]