Look-up problem IP address did we see the IP address before?

Hashing + chaining use IP address as an index linked list IP address hash function index

How to choose a hash function? x  [0,1] x   x.n  depends on the distribution of the data interpret x as a number x  x mod n IP-addresses n=256 bad n=257 good?

Universal hash functions choose a hash function “randomly” n = number of entries in the hash table U = the universe h: U  {0,...,n-1} a hash function

Universal hash functions choose a hash function “randomly” a set of hash functions H is universal if  x,y  U and random h  H P ( h(x) = h(y) )  1/n n = number of entries in the hash table U = the universe h: U  {0,...,n-1} a hash function

Universal hash functions a set of hash functions H is universal if  x,y  U and random h  H P ( h(x) = h(y) )  1/n For IP addresses choose a 1,a 2,a 3,a 4  {0,1,...,256} (x 1,x 2,x 3,x 4 )  a 1 x 1 +a 2 x 2 +a 3 x 3 +a 4 x 4 mod 257

Perfect hashing Goal: worst-case O(1) search space used O(m) static set of elements

Perfect hashing Goal: worst-case O(1) search space used O(m) static set of elements n = m 2 i.e., space used  (m 2 ) H = family of universal hash functions  hash function h  H with no collision

Perfect hashing Goal: worst-case O(1) search space used O(m) n = m H = family of universal hash functions  h  H such that  x i 2 = O(m) x 1,...,x n the number of elements that map to 1,2,...,n

Perfect hashing Goal: worst-case O(1) search space used O(m) n = m H = family of universal hash functions  h  H such that  x i 2 = O(m) x 1,...,x n the number of elements that map to 1,2,...,n secondary hash table of size x i 2

Bloom filter n-bits of storage Goal: store an m element subset of IP addresses IP address HASH 000

Bloom filter - insert n-bits of storage IP address HASH 111 INSERT(x) for i from 1 to k do A(h i (x))  1

Bloom filter – member n-bits of storage IP address HASH 111 MEMBER(x) for i from 1 to k do if A(h i (x))=0 then return FALSE return TRUE

Bloom filter – member MEMBER(x) for i from 1 to k do if A(h i (x))=0 then return FALSE return TRUE sometimes gives false positive answer error parameter: false positive probability

Bloom filter – analysis error parameter: false positive probability m = number of items to be stored n = number of bits of storage k = number of hash functions

Bloom filter – analysis error parameter: false positive probability m = number of items to be stored n = number of bits of storage k = number of hash functions p = fraction of the bits filled p  e -km/n

Bloom filter – analysis error parameter: false positive probability m = number of items to be stored n = number of bits of storage k = number of hash functions p = fraction of the bits filled false positive probability (1-p) k p  e -km/n

Bloom filter – analysis error parameter: false positive probability m = number of items to be stored n = number of bits of storage k = number of hash functions optimal k  0.7 m/n false positive rate  m/n