Data Structures and Algorithms

Data Structures and Algorithms
Course slides: Hashing

Data Structures for Sets
Many applications deal with sets. Compilers have symbol tables (set of vars, classes) Dictionary is a set of words. Routers have sets of forwarding rules. Web servers have set of clients, etc. A set is a collection of members No repetition of members Members themselves can be sets 2

Data Structures for Sets
Examples Set of first 5 natural numbers: {1,2,3,4,5} {x | x is a positive integer and x < 100} {x | x is a CA driver with > 10 years of driving experience and 0 accidents in the last 3 years} 3

Set Operations Unary operation: min, max, sort, makenull, …
Binary operations Member Set Order (=, <, >) Find, insert, delete, split, … Union, intersection, difference, equal, … Unary operation: min, max, sort, makenull, … 4

Observations Set + Operations define an ADT.
A set + insert, delete, find A set + ordering Multiple sets + union, insert, delete Multiple sets + merge Etc. Depending on type of members and choice of operations, different implementations can have different asymptotic complexity. 5

Dictionary ADTs Maintain a set of items with distinct keys with:
find (k): find item with key k insert (x): insert item x into the dictionary remove (k): delete item with key k Where do we use them: Symbol tables for compiler Customer records (access by name) Games (positions, configurations) Spell checkers Peer to Peer systems (access songs by name), etc. 6

Naïve Implementations
The simplest possible scheme to implement a dictionary is “log file” or “audit trail”. Maintain the elements in a linked list, with insertions occuring at the head. The search and delete operations require searching the entire list in the worst-case. Insertion is O(1), but find and delete are O(n). A sorted array does not help, even with ordered keys. The search becomes fast, but insert/delete take O(n). 7

Hash Tables: Intuition
Hashing is function that maps each key to a location in memory. A key’s location does not depend on other elements, and does not change after insertion. unlike a sorted list A good hash function should be easy to compute. With such a hash function, the dictionary operations can be implemented in O(1) time 8

Let us denote the set of all possible key values (i.e., the universe of keys) used in a dictionary application by U. Suppose an application requires a dictionary in which elements are assigned keys from the set of small natural numbers. That is, U ⊂ Z+ and ⏐U⏐ is relatively small. If no two elements have the same key, then this dictionary can be implemented by storing its elements in the array T[0, ... , ⏐U⏐ - 1]. This implementation is referred to as a direct-access table since each of the requisite DICTIONARY ADT operations - Search, Insert, and Delete - can always be performed in Θ(1) time by using a given key value to index directly into T, as shown: 9

The obvious shortcoming associated with direct-access tables is that the set U rarely has such "nice" properties. In practice, ⏐U⏐ can be quite large. This will lead to wasted memory if the number of elements actually stored in the table is small relative to ⏐U⏐. Furthermore, it may be difficult to ensure that all keys are unique. Finally, a specific application may require that the key values be real numbers, or some symbols which cannot be used directly to index into the table. An effective alternative to direct-access tables are hash tables. A hash table is a sequentially mapped data structure that is similar to a direct- access table in that both attempt to make use of the random- access capability afforded by sequential mapping. 10

11

Hash Tables All search structures so far Assume I have a function
Relied on a comparison operation Performance O(n) or O( log n) Assume I have a function f ( key ) ® integer ie one that maps a key to an integer What performance might I expect now?

Hash Tables - Structure
Simplest case: Assume items have integer keys in the range 1 .. m Use the value of the key itself to select a slot in a direct access table in which to store the item To search for an item with key, k, just look in slot k If there’s an item there, you’ve found it If the tag is 0, it’s missing. Constant time, O(1)

Hashing : the basic idea
Map key values to hash table addresses keys -> hash table address This applies to find, insert, and remove Usually: integers -> {0, 1, 2, …, Hsize-1} Typical example: f(n) = n mod Hsize Non-numeric keys converted to numbers For example, strings converted to numbers as Sum of ASCII values First three characters 14

Hashing : the basic idea
Perm # (mod 9) 9 10 20 39 4 14 8 Student Records 15

Hash Tables - Choosing the Hash Function
Uniform hashing Ideal hash function P(k) = probability that a key, k, occurs If there are m slots in our hash table, a uniform hashing function, h(k), would ensure: or, in plain English, the number of keys that map to each slot is equal S P(k) = k | h(k) = 0 S P(k) = k | h(k) = 1 k | h(k) = m-1 1 m Read as sum over all k such that h(k) = 0

Hash Tables - A Uniform Hash Function
If the keys are integers randomly distributed in [ 0 , r ), then is a uniform hash function Most hashing functions can be made to map the keys to [ 0 , r ) for some r, eg adding the ASCII codes for characters mod 255 will give values in [ 0, 256 ) or [ 0, 255 ] Replace + by xor same range without the mod operation h(k) = mk r Read as 0 £ k < r

Hash Tables - Reducing the range to [ 0, m )
We’ve mapped the keys to a range of integers £ k < r Now we must reduce this range to [ 0, m ) where m is a reasonable size for the hash table Strategies Division - use a mod function Multiplication Universal hashing

Division Use a mod function h(k) = k mod m Choice of m? Powers of 2 are generally not good! h(k) = k mod 2n selects last n bits of k All combinations are not generally equally likely Prime numbers close to 2n seem to be good choices eg want ~4000 entry table, choose m = 4093 k mod 28 selects these bits

Multiplication method Multiply the key by constant, A, 0 < A < 1 Extract the fractional part of the product ( kA - ëkAû ) Multiply this by m h(k) = ëm * ( kA - ëkAû )û Now m is not critical and a power of 2 can be chosen So this procedure is fast on a typical digital computer Set m = 2p Multiply k (w bits) by ëA•2wû ç 2w bit product Extract p most significant bits of lower half A = ½(Ö5 -1) seems to be a good choice (see Knuth)

Universal Hashing A determined “adversary” can always find a set of data that will defeat any hash function Hash all keys to same slot ç O(n) search Select the hash function randomly (at run time) from a set of hash functions Reduced probability of poor performance Set of functions, H, which map keys to [ 0, m ) H, is universal, if for each pair of keys, x and y, the number of functions, h Ì H, for which h(x) = h(y) is |H |/m

Hash Tables - Reducing the range to ( 0, m ]
Universal Hashing A determined “adversary” can always find a set of data that will defeat any hash function Hash all keys to same slot ç O(n) search Select the hash function randomly (at run time) from a set of hash functions Functions are selected at run time Each run can give different results Even with the same data Good average performance obtainable

Hash Tables - Reducing the range to ( 0, m ]
Universal Hashing Can we design a set of universal hash functions? Quite easily Key, x = x0, x1, x2, ...., xr Choose a = <a0, a1, a2, ...., ar> a is a sequence of elements chosen randomly from { 0, m-1 } ha(x) = S aixi mod m There are mr+1 sequences a, so there are mr+1 functions, ha(x) Theorem The ha form a set of universal hash functions Proof: See Cormen

Hash Tables - Constraints
Keys must be unique Keys must lie in a small range For storage efficiency, keys must be dense in the range If they’re sparse (lots of gaps between values), a lot of space is used to obtain speed Space for speed trade-off

Hash Tables - Relaxing the constraints
Keys must be unique Construct a linked list of duplicates “attached” to each slot If a search can be satisfied by any item with key, k, performance is still O(1) but If the item has some other distinguishing feature which must be matched, we get O(nmax) where nmax is the largest number of duplicates - or length of the longest chain

Hash Tables - Relaxing the constraints
Keys are integers Need a hash function h( key ) ® integer ie one that maps a key to an integer Applying this function to the key produces an address If h maps each key to a unique integer in the range 0 .. m-1 then search is O(1)

Hash Tables - Hash functions
Form of the hash function Example - using an n-character key int hash( char *s, int n ) { int sum = 0; while( n-- ) sum = sum + *s++; return sum % 256; } returns a value in xor function is also commonly used sum = sum ^ *s++; But any function that generates integers in 0..m-1 for some suitable (not too large) m will do As long as the hash function itself is O(1) !

Hash Tables - Collisions
Hash function With this hash function int hash( char *s, int n ) { int sum = 0; while( n-- ) sum = sum + *s++; return sum % 256; } hash( “AB”, 2 ) and hash( “BA”, 2 ) return the same value! This is called a collision A variety of techniques are used for resolving collisions

Hash Tables - Collision handling
Collisions Occur when the hash function maps two different keys to the same address The table must be able to recognise and resolve this Recognise Store the actual key with the item in the hash table Compute the address k = h( key ) Check for a hit if ( table[k].key == key ) then hit else try next entry Resolution Variety of techniques

Hash Tables - Linked lists
Collisions - Resolution Linked list attached to each primary table slot h(i) == h(i1) h(k) == h(k1) == h(k2) Searching for i1 Calculate h(i1) Item in table, i, doesn’t match Follow linked list to i1 If NULL found, key isn’t in table

Hash Tables - Overflow area
Linked list constructed in special area of table called overflow area h(k) == h(j) k stored first Adding j Calculate h(j) Find k Get first slot in overflow area Put j in it k’s pointer points to this slot Searching - same as linked list

Hash Tables - Re-hashing
Use a second hash function Many variations General term: re-hashing h(k) == h(j) k stored first Adding j Calculate h(j) Find k Repeat until we find an empty slot Calculate h’(j) Put j in it Searching - Use h(x), then h’(x) h’(x) - second hash function

Hash Tables - Re-hash functions
The re-hash function Many variations Linear probing h’(x) is +1 Go to the next slot until you find one empty Can lead to bad clustering Re-hash keys fill in gaps between other keys and exacerbate the collision problem

Hash Tables - Re-hash functions
The re-hash function Many variations Quadratic probing h’(x) is c i2 on the ith probe Avoids primary clustering Secondary clustering occurs All keys which collide on h(x) follow the same sequence First a = h(j) = h(k) Then a + c, a + 4c, a + 9c, .... Secondary clustering generally less of a problem

Hash Tables - Collision Resolution Summary
Chaining Unlimited number of elements Unlimited number of collisions Overhead of multiple linked lists Re-hashing Fast re-hashing Fast access through use of main table space Maximum number of elements must be known Multiple collisions become probable Overflow area Fast access Collisions don't use primary table space Two parameters which govern performance need to be estimated

Hash Tables - Collision Resolution Summary
Re-hashing Fast re-hashing Fast access through use of main table space Maximum number of elements must be known Multiple collisions become probable Overflow area Fast access Collisions don't use primary table space Two parameters which govern performance need to be estimated

Hash Tables - Summary so far ...
Potential O(1) search time If a suitable function h(key) ® integer can be found Space for speed trade-off “Full” hash tables don’t work (more later!) Collisions Inevitable Hash function reduces amount of information in key Various resolution strategies Linked lists Overflow areas Re-hash functions Linear probing h’ is +1 Quadratic probing h’ is +ci2 Any other hash function! or even sequence of functions!

Hashing: Choose a hash function h; it also determines the hash table size. Given an item x with key k, put x at location h(k). To find if x is in the set, check location h(k). What to do if more than one keys hash to the same value. This is called collision. We will discuss two methods to handle collision: Separate chaining Open addressing 38

Separate chaining 14 42 29 20 1 36 56 23 16 24 31 17 7 2 3 4 5 6 8 9 10 Maintain a list of all elements that hash to the same value Search -- using the hash function to determine which list to traverse Insert/deletion–once the “bucket” is found through Hash, insert and delete are list operations find(k,e) HashVal = Hash(k,Hsize); if (TheList[HashVal].Search(k,e)) then return true; else return false; Hash function is x mod 11 class HashTable { …… private: unsigned int Hsize; List<E,K> *TheList; …… 39

Insertion: insert 53 53 = 4 x 11 + 9 53 mod 11 = 9 14 42 29 20 1 36 56
23 16 24 53 17 7 2 3 4 5 6 8 9 10 31 14 42 29 20 1 36 56 23 16 24 31 17 7 2 3 4 5 6 8 9 10 40

Analysis of Hashing with Chaining
Worst case All keys hash into the same bucket a single linked list. insert, delete, find take O(n) time. Average case Keys are uniformly distributed into buckets O(1+N/B): N is the number of elements in a hash table, B is the number of buckets. If N = O(B), then O(1) time per operation. N/B is called the load factor of the hash table. 41

1 2 3 4 5 6 7 8 9 10 42 14 16 24 31 28 Open addressing If collision happens, alternative cells are tried until an empty cell is found. Linear probing : Try next available position 42

Linear Probing (insert 12)
12 = 1 x 12 mod 11 = 1 1 2 3 4 5 6 7 8 9 10 42 14 16 24 31 28 1 2 3 4 5 6 7 8 9 10 42 14 16 24 31 28 12 43

Search with linear probing (Search 15)
15 = 1 x 15 mod 11 = 4 1 2 3 4 5 6 7 8 9 10 42 14 16 24 31 28 12 NOT FOUND ! 44

Deletion in Hashing with Linear Probing
Since empty buckets are used to terminate search, standard deletion does not work. One simple idea is to not delete, but mark. Insert: put item in first empty or marked bucket. Search: Continue past marked buckets. Delete: just mark the bucket as deleted. Advantage: Easy and correct. Disadvantage: table can become full with dead items. 45

Deletion with linear probing: LAZY (Delete 9)
9 = 0 x 9 mod 11 = 9 1 2 3 4 5 6 7 8 9 10 42 14 16 24 31 28 12 1 2 3 4 5 6 7 8 9 10 42 D 14 16 24 31 28 12 FOUND ! 46

Eager Deletion: fill holes
Remove and find replacement: Fill in the hole for later searches remove(j) { i = j; empty[i] = true; i = (i + 1) % D; // candidate for swapping while ((not empty[i]) and i!=j) { r = Hash(ht[i]); // where should it go without collision? // can we still find it based on the rehashing strategy? if not ((j<r<=i) or (i<j<r) or (r<=i<j)) then break; // yes find it from rehashing, swap i = (i + 1) % D; // no, cannot find it from rehashing } if (i!=j and not empty[i]) then { ht[j] = ht[i]; remove(i); 47

Eager Deletion Analysis (cont.)
If not full After deletion, there will be at least two holes Elements that are affected by the new hole are Initial hashed location is cyclically before the new hole Location after linear probing is in between the new hole and the next hole in the search order Elements are movable to fill the hole Initial hashed location Initial hashed location Location after linear probing Next hole in the search order New hole Next hole in the search order 48

Eager Deletion Analysis (cont.)
The important thing is to make sure that if a replacement (i) is swapped into deleted (j), we can still find that element. How can we not find it? If the original hashed position (r) is circularly in between deleted and the replacement j r i i r Will not find i past the empty green slot! i j r r i j Will find i j i r i r 49

Quadratic Probing Solves the clustering problem in Linear Probing
Check H(x) If collision occurs check H(x) + 1 If collision occurs check H(x) + 4 If collision occurs check H(x) + 9 If collision occurs check H(x) + 16 ... H(x) + i2 50

Quadratic Probing (insert 12)
12 = 1 x 12 mod 11 = 1 1 2 3 4 5 6 7 8 9 10 42 14 16 24 31 28 1 2 3 4 5 6 7 8 9 10 42 14 16 24 31 28 12 51

Double Hashing When collision occurs use a second hash function
Hash2 (x) = R – (x mod R) R: greatest prime number smaller than table-size Inserting 12 H2(x) = 7 – (x mod 7) = 7 – (12 mod 7) = 2 Check H(x) If collision occurs check H(x) + 2 If collision occurs check H(x) + 4 If collision occurs check H(x) + 6 If collision occurs check H(x) + 8 H(x) + i * H2(x) 52

Double Hashing (insert 12)
12 = 1 x 12 mod 11 = 1 7 –12 mod 7 = 2 1 2 3 4 5 6 7 8 9 10 42 14 16 24 31 28 1 2 3 4 5 6 7 8 9 10 42 14 16 24 31 28 12 53

Rehashing If table gets too full, operations will take too long.
Build another table, twice as big (and prime). Next prime number after 11 x 2 is 23 Insert every element again to this table Rehash after a percentage of the table becomes full (70% for example) 54

Good and Bad Hashing Functions
Hash using the wrong key Age of a student Hash using limited information First letter of last names (a lot of A’s, few Z’s) Hash functions choices : keys evenly distributed in the hash table Even distribution guaranteed by “randomness” No expectation of outcomes Cannot design input patterns to defeat randomness 55

Examples of Hashing Function
B=100, N=100, keys = A0, A1, …, A99 Hashing(A12) = (Ascii(A)+Ascii(1)+Ascii(2)) / B H(A18)=H(A27)=H(A36)=H(A45) … Theoretically, N(1+N/B)= 200 In reality, 395 steps are needed because of collision How to fix it? Hashing(A12) = (Ascii(A)*22+Ascii(1)*2+Ascci(2))/B H(A12)!=H(A21) Examples: numerical keys Use X2 and take middle numbers 56

Collision Frequency Birthdays or the von Mises paradox
There are 365 days in a normal year Birthdays on the same day unlikely? How many people do I need before “it’s an even bet” (ie the probability is > 50%) that two have the same birthday? View the days of the year as the slots in a hash table the “birthday function” as mapping people to slots Answering von Mises’ question answers the question about the probability of collisions in a hash table

Distinct Birthdays Let Q(n) = probability that n people have distinct birthdays Q(1) = 1 With two people, the 2nd has only 364 “free” birthdays The 3rd has only 363, and so on: Q(2) = Q(1) * 364 365 Q(n) = Q(1) * 364 365 365-n+1 * * … *

Coincident Birthdays Probability of having two identical birthdays
P(n) = 1 - Q(n) P(23) = 0.507 With 23 entries, table is only 23/365 = 6.3% full!

Hash Tables - Load factor
n = number of items Collisions are very probable! Table load factor must be kept low Detailed analyses of the average chain length (or number of comparisons/search) are available Separate chaining linked lists attached to each slot gives best performance but uses more space! m = number of slots  n m

Hash Tables - General Design
Choose the table size Large tables reduce the probability of collisions! Table size, m n items Collision probability a = n / m Choose a table organisation Does the collection keep growing? Linked lists ( but consider a tree!) Size relatively static? Overflow area or Re-hash Choose a hash function ....

Hash Tables - General Design
Choose a hash function A simple (and fast) one may well be fine ... Read your text for some ideas! Check the hash function against your data Fixed data Try various h, m until the maximum collision chain is acceptable Known performance Changing data Choose some representative data Try various h, m until collision chain is OK Usually predictable performance

Hash Tables - Review If you can meet the constraints
Hash Tables will generally give good performance O(1) search Like radix sort, they rely on calculating an address from a key But, unlike radix sort, relatively easy to get good performance with a little experimentation not advisable for unknown data collection size relatively static memory management is actually simpler All memory is pre-allocated!

Collision Functions Hi(x)= (H(x)+i) mod B
Linear pobing Hi(x)= (H(x)+ci) mod B (c>1) Linear probing with step-size = c Hi(x)= (H(x)+i2) mod B Quadratic probing Hi(x)= (H(x)+ i * H2(x)) mod B 64

Analysis of Open Hashing
Effort of one Insert? Intuitively – that depends on how full the hash is Effort of an average Insert? Effort to fill the Bucket to a certain capacity? Intuitively – accumulated efforts in inserts Effort to search an item (both successful and unsuccessful)? Effort to delete an item (both successful and unsuccessful)? Same effort for successful search and delete? Same effort for unsuccessful search and delete? 65

Picking a Hash Function
In practice, hash functions often use the bit representation of values. e.g. compute the binary representation of the characters in the search key and return the sum modulus the number of buckets. e.g. distribute first names over 59 buckets. Use ASCII values of the letters e.g. John ( = 399), 399 modulo 59 = bucket 45. In practice schemes are more complicated than the above. See the big white algorithm book for more details (Cormen, Leiserson, Rivest). For a hash table the idea is that there will be n entries. n = the number of actual values. the hope is that each of the n values maps to a different index (no collisions). For DB hash indexing the index is divided into buckets. each bucket maps to a disk page (which maps to a disk block). The hash function maps index entries to buckets with the intent that no bucket overflows.

Hash Function Problems
Poor distribution of values to buckets. This can happen because the hash function is either not random or not uniform. Solution: Change the hash function. Skewed data distribution. The distribution of search key values is skewed. That is, there is not a uniform distribution of values – many incidences of some values and few of others. Solution: There is no way this can be addressed by changing the hash function. If this is a problem then a hash index may not be a good choice. Collisions Overflow may be caused by inserts of data entries with the same hash value (but different search key values). Solution: Static hashing does not address this. Extendible hashing and linear hashing deal with collisions to some extent.

Static Hashing Hash function Primary pages Overflow pages Performance
A uniform and random hash function h evenly maps search key values to a primary page. With N buckets the hash function maps a search key value to buckets 0 to N-1. Primary pages The number of buckets (pages) is pre-determined. Each bucket is a disk page. Overflow pages As the DB grows primary pages become full. Additional data is placed in overflow pages chained to the primary pages. Finding a value involves searching these pages. Performance Generally a search requires one disk access. Insert or delete require two disk accesses. The number of primary pages is fixed. The structure becomes inefficient as the file grows (because of the overflow pages). It wastes space if the file shrinks. Re-hashing removes overflow until the DB grows again. This process is time-consuming and the index is inaccessible during it.

Extendible Hashing The efficiency of static hashing declines as the file grows. Overflow pages increase search time. One solution would be to use a range of hash functions based on a bit value and double the number of buckets (and the function range) whenever an overflow page is needed. Such a reorganization is expensive. Is it possible to make local changes? Use a directory of pointers to buckets. Double the directory size when required. Only split the pages that have overflowed. The directory need only consist of an array of pointers to pages so is relatively compact. The array index represents the value computed by the hash function. At any time the array size is determined by how many bits of the hash result are being used. Usually the last d (least significant) bits are used.

Basic Structure 64 16 00 1 17 5 01 10 11 6 31 15 The array index is the last two bits of the hash value. Note that the values in the cells represents the hash value (not the search key value). Assume that three records fit on a page (so each bucket is a disc page). There are only four pages of data and none of the pages is full – only two bits are required as an index.

Inserting Values 64 16 00 1 17 5 01 9 10 11 6 14 31 15 insert Insert 14 and 9 into the index shown in the previous example. 14 fits but inserting 9 causes the page to overflow. Double the directory and split the overflowing bucket (01) into two. Distribute the entries based on the last three digits of the hash value. Directory pointers to the existing buckets are added for the other three new three digit hash values.

Inserting Values Example
64 16 1 17 9 000 001 010 6 14 011 100 101 31 15 110 111 5 new bucket Note how the directory has doubled in size but only one new bucket has been created. The directory is small (each entry consists of a value and a pointer). New index pages (buckets) are kept to a minimum.

Keeping Track of Bits After the directory is doubled some pages will be referenced by two directory entries. If the referenced pages become full, subsequent insertions will require a new page to be allocated but will not require doubling the directory. If pages that are referenced by only one directory entry overflow the directory will have to be doubled again. Contrast inserting 4 and 12 (x00) into the example with inserting 25 (x01). How do we keep track of whether or not an insert requires that the directory is doubled? Record the global depth of the hashed file. The number of bits required for the directory. Also record the local depth of each page. The number of bits needed to reference a particular page.

Local and Global Depth local depth global depth 3 2 64 16 3 1 17 9 000
001 010 6 14 011 100 101 31 15 110 111 5 global depth local depth 3

Summary Create initial hash file and directory. Insert values
Each directory entry points to a different bucket. All local depths are equal to the global depth. Insert values If no overflow occurs insert entry and finish. If the bucket overflows compare the local depth of the bucket with the global depth. If the local depth is less than the global depth create a new bucket and distribute the entries with no change to the directory. Increment the local depth of the split buckets. If the local depth is the same as the global depth double the directory, and split the bucket. Increment the local depth of the split buckets and the global depth of the directory. Delete values If the deletion empties the bucket it can be merged and the local depth decremented. If the local depth of all buckets is less than the global depth the directory can be halved. In practice this is often not done.

Performance Performance is identical to static hashing if the directory fits in memory. If the directory does not fit in memory an additional disk access is required. It is likely that the directory will fit in memory. Collisions and Performance Collisions at low global depth are dealt with by doubling the directory, and the range of the hash function, and making local index changes. Many collisions will result in producing a large directory (that might not fit in memory). Overflow pages If many entries have the same hash value across the entire range of bits overflow pages have to be allocated, reducing efficiency. This is because splitting a bucket (and doubling the directory) would cause another split if all the entries map the same bucket after splitting. This can occur if the hash function is poor. Collisions will occur if the distribution of values is skewed.

Linear Hashing Another dynamic system.
Like extendible hashing insertions and deletions are efficiently accommodated. Unlike extendible hashing a directory is not required. Collisions may result in chains of overflow pages. Linear hashing, like extendible hashing uses a family of hash functions. Each function’s range is twice that of its predecessor. Pages are split when overflows occur – but not necessarily the page with the overflow. Splitting occurs in turn, in a round robin fashion. When all the pages at one level (the current hash function) have been split a new level is applied. Splitting occurs gradually Primary pages are allocated consecutively.

Levels of Linear Hashing
Initial Stage. The initial level distributes entries into N0 buckets. Call the hash function to perform this h0. Splitting buckets. If a bucket overflows its primary page is chained to an overflow page (increasing the bucket’s size). Also when a bucket overflows some bucket is split. The first bucket to be split is the first bucket in the file (not necessarily the bucket that overflows). The next bucket to be split is the second bucket in the file … and so on until the Nth. has been split. When buckets are split their entries (including those in overflow pages) are distributed using h1. To access split buckets the next level hash function (h1) is applied. h1 maps entries to 2N0 (or N1)buckets. Level progression. Once all Ni buckets of the current level (i) are split the hash function hi is replaced by hi+1. The splitting process starts again at the first bucket and hi+2 is applied to find entries in split buckets.

Linear Hashing Example
next 64 36 1 17 5 6 31 15 The example above shows the index level equal to 0 where N0 equals 4 (three entries fit on a page). h0 maps index entries to one of four buckets. Given the initial page of the file the appropriate primary page can be determined by using an offset. i.e. initial page + h0(search key value) In the above example only h0 is used and no buckets have been split. Now consider what happens when 9 is inserted (which will not fit in the second bucket). Note that next indicates which bucket is to split next.

Linear Hashing Example 2
next 64 h0 1 17 5 9 6 31 15 36 The page indicated by next is split (the first one). Next is incremented. An overflow page is chained to the primary page to contain the inserted value. If h0 maps a value from zero to next – 1 (just the first page in this case) h1 must be used to where to insert the new entry. Note how the new page falls naturally into the sequence as the fifth page.

Linear Hashing Example 3
2 1 h1 next3 64 8 32 16 17 9 h0 next1 10 18 6 14 next2 11 31 15 7 36 5 13 - 23 Assume inserts of 8, 7, 18, 14, 111, 32, 162, 10, 13, 233 After the 2nd. split the base level is 1 (N1 = 8), use h1. Subsequent splits will use h2 for inserts between the first bucket and next- 1.

Comparing Linear and Extendible Hashing
Differences Because buckets are split in turn linear hashing does not need a directory. Extendible hashing may lead to better use of space because the overflowing bucket is always the one that is split. In particular, linear hashing does not deal elegantly with collisions in that long overflow chains may develop. Collisions with extendible hashing lead to a large directory. Similarities Doubling the directory and moving to the next level of hash function have the same effect: the range of buckets is doubled. The hash functions used by the two schemes may be the same. i.e. they can both use a bit translation of the search key value modulus the number of buckets required.

Issues: What do we lose? Operations that require ordering are inefficient FindMax: O(n) O(log n) Balanced binary tree FindMin: O(n) O(log n) Balanced binary tree PrintSorted: O(n log n) O(n) Balanced binary tree What do we gain? Insert: O(1) O(log n) Balanced binary tree Delete: O(1) O(log n) Balanced binary tree Find: O(1) O(log n) Balanced binary tree How to handle Collision? Separate chaining Open addressing 83

Interface Main methods: Goal: methods are O(1)! (ususally)
Void Put(Object) Object Get(Object) … returns null if not i … Remove(Object) Goal: methods are O(1)! (ususally) Implementation details HashTable: the storage bin hashfunction(object): tells where object should go collision resolution strategy: what to do when two objects “hash” to same location. In Java, all objects have default int hashcode(), but better to define your own. Except for strings. String hashing in Java is good.

HashFunctions Goal: map objects into table so distribution is uniform
Tricky to do. Examples for string s product ascii codes, then mod tablesize nearly always even, so bad sum ascii codes, then mod tablesize may be too small shift bits in ascii code java allows this with << and >> Java does a good job with Strings

Example Problem Suppose we are storing numeric id’s of customers, maybe 100,000 We want to check if a person is delinquent, usually less than 400. Use an array of size 1000, the delinquents. Put id in at id mod tableSize. Clearly fast for getting, removing But what happens if entries collide?

Separate Chaining Array of linked lists
The hash function determines which list to search May or may keep individual lists in sorted order Problems: needs a very good hash function, which may not exist worse case: O(n) extra-space for links Another approach: Open Addressing everything goes into the array, somehow several approaches: linear, quadratic, double, rehashing

Linear Probing Store information (or prts to objects) in array
When inserting an object, if location filled, find first unfilled position. I.e look at hi(x)+f(i) where f(i)= i; When getting an object, start at hash addresses, and do linear search till find object or a hole. primary clustering blocks of filled cells occur Harder to insert than find existing element Load factor =lf = percent of array filled Expected probes for insertion: 1/2(1+1/(1-lf)^2)) successful search: 1/2(1+1/(1-lf))

Expected number of probes

Quadratic Probing Idea: f(i) = i^2 (or some other quadratic function)
Problem: If table is more than 1/2 full, no quarantee of finding any space! Theorem: if table is less than 1/2 full, and table size is prime, then an element can be inserted. Good: Quadratic probing eliminates primary clustering Quadratic probing has secondary clustering (minor) if hash to same addresses, then probe sequence will be the same

Proof of theorem Theorem: The first P/2 probes are distinct.
Suppose not. Then there are i and j <P/2 that hash to same place So h(x)+i^2 = h(y)+j^2 and h(x) = h(y). So i^2 = j^2 mod P (i+j)*(i-j) = 0 mod P Since P is prime and i and j are less than P/2 then i+j and i-j are less than P and P factors. Contradiction

Double Hashing Goal: spreading out the probe sequence
f(i) = i*hash2(x), where hash2 is another hash function Dangerous: can be very bad. Also may not eliminate any problems In best case, it’s great

Rehashing All methods degrade when table becomes too full
Simpliest solution: create new table, twice as large rehash everything O(N), so not happy if often With quadratic probing, rehash when table 1/2 full

Extendible Hashing: Uses secondary storage
Suppose data does not fit in main memory Goal: Reduce number of disks accesses. Suppose N records to store and M records fit in a disk block Result: 2 disk accesses for find (~4 for insert) Let D be max number of bits so 2^D < M. This is for root or directory (a disk block) Algo: hash on first D bits, yields ptr to disk block Expected number of leaves: (N/M) log 2 Expected directory size: O(N^(1+1/M) / M) Theoretically difficult, more details for implementation

Applications Compilers: keep track of variables and scope
Graph Theory: associate id with name (general) Game Playing: E.G. in chess, keep track of positions already considered and evaluated (which may be expensive) Spelling Checker: At least to check that word is right. But how to suggest correct word Lexicon/book indices

HashSets vs HashMaps HashSets store objects
supports adding and removing in constant time HashMaps store a pair (key,object) this is an implementation of a Map HashMaps are more useful and standard HashMaps main methods are: put(Object key, Object value) get(Object key) remove(Object key) All done in expected O(1) time.

Lexicon Example Inputs: text file (N) + content word file (the keys) (M) Ouput: content words in order, with page numbers Algo: Define entry = (content word, linked list of integers) Initially, list is empty for each word. Step 1: Read content word file and Make HashMap of content word, empty list Step 2: Read text file and check if work in HashMap; if in, add to page number, else continue. Step 3: Use the iterator method to now walk thru the HashMap and put it into a sortable container.

Lexicon Example Complexity: Dumb Algorithm
step 1: O(M), M number of content words step 2: O(N), N word file size step 3: O(M log M) max. So O(max(N, M log M)) Dumb Algorithm Sort content words O(Mlog M) (balanced tree) Look up each word in Content Word tree and update O(N*logM) Total complexity: O(N log M) N = 500*2000 =1,000,000 and M = 1000 Smart algo: 1,000,000; dumb algo: 1,000,000*10.

Memoization Recursive Fibonacci:
fib(n) = if (n<2) return 1 else return fib(n-1)+fib(n-2) Use hashing to store intermediate results Hashtable ht; fib(n) = Entry e = (Entry)ht.get(n); if (e != null) return e.answer; else if (n<2) return 1; else ans = fib(n-1)+fib(n-2); ht.put(n,ans); return ans;

Appendix: Hashing for Databases

Contents Static Hashing Dynamic Hashing Comparisons File Organization
Properties of the Hash Function Bucket Overflow Indices Dynamic Hashing Underlying Data Structure Querying and Updating Comparisons Other types of hashing Ordered Indexing vs. Hashing

Static Hashing Hashing provides a means for accessing data without the use of an index structure. Data is addressed on disk by computing a function on a search key instead.

File organization A bucket in a hash file is unit of storage (typically a disk block) that can hold one or more records. The hash function, h, is a function from the set of all search-keys, K, to the set of all bucket addresses, B. Insertion, deletion, and lookup are done in constant time.

Querying and Updates To insert a record into the structure compute the hash value h(Ki), and place the record in the bucket address returned. For lookup operations, compute the hash value as above and search each record in the bucket for the specific record. To delete simply lookup and remove.

Properties of the Hash Function
The distribution should be uniform. An ideal hash function should assign the same number of records in each bucket. The distribution should be random. Regardless of the actual search-keys, the each bucket has the same number of records on average Hash values should not depend on any ordering or the search-keys

Bucket Overflow How does bucket overflow occur?
Not enough buckets to handle data A few buckets have considerably more records then others. This is referred to as skew. Multiple records have the same hash value Non-uniform hash function distribution.

Solutions Provide more buckets then are needed. Overflow chaining
If a bucket is full, link another bucket to it. Repeat as necessary. The system must then check overflow buckets for querying and updates. This is known as closed hashing.

Alternatives Open hashing Compute more hash functions.
The number of buckets is fixed Overflow is handled by using the next bucket in cyclic order that has space. This is known as linear probing. Compute more hash functions. Note: Closed hashing is preferred in database systems.

Indices A hash index organizes the search keys, with their pointers, into a hash file. Hash indices never primary even though they provide direct access.

Example of Hash Index

Dynamic Hashing More effective then static hashing when the database grows or shrinks Extendable hashing splits and coalesces buckets appropriately with the database size. i.e. buckets are added and deleted on demand.

The Hash Function Typically produces a large number of values, uniformly and randomly. Only part of the value is used depending on the size of the database.

Data Structure Hash indices are typically a prefix of the entire hash value. More then one consecutive index can point to the same bucket. The indices have the same hash prefix which can be shorter then the length of the index.

General Extendable Hash Structure
In this structure, i2 = i3 = i, whereas i1 = i – 1

Queries and Updates Lookup Take the first i bits of the hash value.
Following the corresponding entry in the bucket address table. Look in the bucket.

Queries and Updates (Cont’d)
Insertion Follow lookup procedure If the bucket has space, add the record. If not…

Insertion (Cont’d) Case 1: i = ij
Use an additional bit in the hash value This doubles the size of the bucket address table. Makes two entries in the table point to the full bucket. Allocate a new bucket, z. Set ij and iz to i Point the second entry to the new bucket Rehash the old bucket Repeat insertion attempt

Insertion (cont’d) Case 2: i > ij Allocate a new bucket, z
Add 1 to ij, set ij and iz to this new value Put half of the entries in the first bucket and half in the other Rehash records in bucket j Reattempt insertion

Insertion (Finally) If all the records in the bucket have the same search value, simply use overflow buckets as seen in static hashing.

Use of Extendable Hash Structure: Example
Initial Hash structure, bucket size = 2

Example (Cont.) Hash structure after insertion of one Brighton and two Downtown records

Hash structure after insertion of Mianus record
Example (Cont.) Hash structure after insertion of Mianus record

Hash structure after insertion of three Perryridge records
Example (Cont.) Hash structure after insertion of three Perryridge records

Example (Cont.) Hash structure after insertion of Redwood and Round Hill records

Comparison to Other Hashing Methods
Advantage: performance does not decrease as the database size increases Space is conserved by adding and removing as necessary Disadvantage: additional level of indirection for operations Complex implementation

Ordered Indexing vs. Hashing
Hashing is less efficient if queries to the database include ranges as opposed to specific values. In cases where ranges are infrequent hashing provides faster insertion, deletion, and lookup then ordered indexing.

Data Structures and Algorithms

Similar presentations

Presentation on theme: "Data Structures and Algorithms"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Data Structures and Algorithms

Similar presentations

Presentation on theme: "Data Structures and Algorithms"— Presentation transcript:

Similar presentations

About project

Feedback