1. Online Algorithms Advanced Seminar A Supervisor: Matya Katz Ran Taig, Achiya Elyasaf December, 2009

2. 2 Overview-part A  Introduction What is Online Algorithm? Evaluating the Online Algorithm The Online Paging Problem & Algorithm Deterministic Online Algorithms For Paging CA ≥ k Theorem  Adversary Models Randomize Online Paging Algorithm Oblivious Adversary Adaptive Adversary

3. 3 Introduction What is Online Algorithm?  So far, all algorithms received their entire inputs at one time  Online algorithms (OA), receive and process the input in partial amounts.  A sequence of requests are received and the OA must service each request before it receives the next one.

4. 4 Introduction What is Online Algorithm? (Cont.)  In servicing each request, a several alternatives with an associated cost are possible.  The alternative chosen may influence the costs of alternatives on future requests.  Examples: data-structuring resource-allocation in operating systems finance distributed computing

5. 5 Introduction Evaluating the Online Algorithm IIt is often meaningless to have an absolute performance measure for an algorithm. The algorithm can be forced to incur an unbounded cost by appropriately choosing the input sequence IIt is difficult, if not impossible, to perform a comparison of competing strategies. QQ: How to evaluate the algorithm? AA: We compare the total cost of the OA on a sequence of requests, to the total cost of an offline algorithm WWe refer to such an analysis as a competitive analysis

6. 6 Introduction The Paging Problem Some definitions: Memory item - a page of virtual memory Cache memory – a fast memory of size of k memory items Main memory - a slower memory that can potentially hold an infinite number of items A paging algorithm - decides which k items to retain in the cache at each point in time We have a sequence of requests, each of which specifies a memory item. A hit – the requested item is in the cache. There is no cost A miss – the item must be fetched from the main memory. There is a ‘unit’ cost and one of the k items in the cache must be evicted Paging – The replacement of one page with another in the cache is called paging or page fault

7. 7 Introduction The Paging Algorithm  The cost measure is the number of misses on a sequence of requests  This cost depends on the algorithm that decides which k items to retain in the cache at each point in time  When a page fault happens, the paging algorithm invoke an eviction rule for deciding which item currently in the cache should be evicted to make room for the new item  Intuitively, items that will be requested again in the near future should not be evicted

8. 8 Introduction Offline Algorithm  The offline algorithm (aka the MIN algorithm): on a miss, evict that item whose next request occurs furthest in the future  The worst-case number of misses on a request sequence of length N is N/k.

9. 9 Introduction Deterministic Online Algorithms For Paging  Least Recently Used (LRU): evict the item in the cache whose most recent request occurred furthest in the past.  First-in, First-out (FIFO): evict the item that has been in the cache for the longest period  Least Frequently Used (LFU): evict the item in the cache that has been requested least often

10. 10 Introduction Deterministic Online Algorithms For Paging (Cont.)  Let p 1, p 2,...,p n be a request sequence presented to an online paging algorithm A  Let f A (p 1, p 2,...,p n ) denote the number of times A misses on p 1, p 2,...,p n  Let f 0 (p 1, p 2,...,p n ) denote the minimum number of misses on p 1, p 2,...,p n

11. 11  A deterministic online paging algorithm A is said to be C-Competitive if there exists a constant b such that on every sequence of requests p 1, p 2,...,p n : f A (p 1, p 2,...,p n ) - C∙f 0 (p 1, p 2,...,p n ) ≤ b where the constant b must be independent of N but may depend on k  Competitiveness measures the performance of an OA in terms of the worst-case ratio of its cost to that of the optimal offline algorithm Introduction Deterministic Online Algorithms For Paging (Cont.)

12. 12 Introduction Deterministic Online Algorithms For Paging (Cont.)  The competitiveness coefficient of A, denoted C A, is the infimum of C such that A is C-competitive  Since the worst case of offline algorithm is N/k, no deterministic online paging algorithm has competitiveness coefficient smaller than k  LRU, FIFO are known to be k-competitive  No deterministic online paging algorithm has competitiveness coefficient smaller than k (proof will follow soon…)  Thereby, LRU and FIFO are optimal deterministic OA

13. 13 Introduction Paging Algorithm - Formally  A paging algorithm consists of an automaton with a finite set S of states  The automaton response is defined by function  The cache after the request is serviced must include the requested item curr. statecurr. cachenew itemcurr. statecurr. cache

14. 14 Introduction C A ≥ k Theorem Let A be a deterministic online algorithm for paging then C A ≥ k Proof: Initialization -  Both A & the offline algorithm are managing different caches for the same request sequence  They both has the same k-items in the cache  First request is to an item not in either cache, and the algorithms incur a miss  Let S be the set of k+1 items consisting of the initially k items together with the new item  From then on, every request is for the unique item in S not in A's cache Thus A misses on every request

15. 15 Introduction C A ≥ k Theorem (cont.)  A round is a maximal sequence of requests in which at most k distinct items are requested; each of these items may be requested any number of times and in any order  A round ends when, after k distinct items have been requested, a new item p is requested, and p then becomes the first request of the next round  Since the round contains at least k requests and A misses on every one of them, it misses at least k times during the round

16. 16 Introduction C A ≥ k Theorem (cont.)  There is an offline algorithm that misses only once during a round, in fact on the first request of the round  We denote p as first request of the following round  When the offline algorithm misses on the first request, it evicts p and thereby ensures that there are no further misses in that round (as expected from a MIN algorithm)

17. 17  The offline algorithm knows A’s initial cache, the entire request sequence in advance and the identity of p for every round  At the end of each round, both the online algorithm and the offline algorithm have the same set of items in their caches  This construction can be repeated many times, proving that there are arbitrarily long sequences on which A has k times as many misses as the offline algorithm. Introduction C A ≥ k Theorem (cont.)

18. 18 Introduction Conclusions  The proof uses only the fact that the OA doesn’t know future requests  Thus the lower bound applies to any deterministic OA without any regard for its use of computational resources such as time or space  This is a negative result for the online algorithms  The offline algorithm may be view as an adversary who is not only managing a cache, but is also generating the request sequence

19. 19 Overview-part A  Introduction What is Online Algorithm? Evaluating the Online Algorithm The Online Paging Problem & Algorithm Deterministic Online Algorithms For Paging CA ≥ k Theorem  Adversary Models Randomize Online Paging Algorithm Oblivious Adversary Adaptive Adversary

20. 20 Adversary Models Randomize Online Paging Algorithm  The Adversary Models, where in collusion with a reference algorithm that is the yardstick against which the competitiveness of the given online algorithm is being measured  The adversary's goal is to increase the cost to the given online algorithm, while keeping it down for the reference algorithm

21. 21 Adversary Models Randomize Online Paging Algorithm  2 definitions: R- a randomize online paging algorithm (=ROA) f R (p 1, p 2,...,p n ) – a random variable, denotes the number of times that R misses on the sequence  Evaluating the ROA We still study the behavior of R when the sequence of requests is generated by an adversary However, there is no longer a unique notion of an "adversary" for a randomized online algorithm

22. 22 Adversary Models Oblivious Adversary  The weakest adversary knows R in advance, but has no knowledge of the random choices made by R  This adversary calculates the “worst case” request sequence for R, regardless of the actual execution of R  The fixed cost of this sequence is not a random variable and is denoted by f 0 (p 1, p 2,...,p n )  We call such an adversary an oblivious adversary, reflecting that the adversary is oblivious to the random choices made by R

23. 23  R is C-competitive against the oblivious adversary if for every sequence of requests p 1, p 2,...,p n : for a constant b independent of N.  The oblivious competitiveness coefficient of R, denoted by, is the infimum of C such that R is C-competitive Adversary Models Oblivious Adversary

24. 24  The Adaptive Adversary chooses p i+1 after observing the responses of the ROA to p 1, p 2,...,p i  To define the cost of the optimal algorithm it might help to think of the adaptive adversary and the optimal algorithm as working in collusion Adversary Models Adaptive Adversary

25. 25  First scenario: The adversary generates the optimal strategy adaptively by learning p 1, p 2,...,p i We refer to this as the adaptive offline adversary The request sequence is a random sequence. Thus, f 0 (p 1, p 2,...,p n ) and f R (p 1, p 2,...,p n ) are random variables Adversary Models Adaptive Adversary

26. 26  Second scenario: The adversary works as before, but also required to concurrently manage a cache online Meaning, the adversary generates p i+1 based on the responses of R to p 1,p 2,...,p i, and immediately exhibits its own response to p i+1 Again both f 0 (p 1, p 2,...,p n ) and f R (p 1, p 2,...,p n ) are random variables We refer to such an adversary as an adaptive online adversary Adversary Models Adaptive Adversary

27. 27  We say that R is C-competitive against the adaptive offline adversary if for a constant b independent of N:  The adaptive offline competitiveness coefficient of R,, is the infimum of C such that R is C-competitive  Likewise, is the adaptive online competitiveness coefficient of R Adversary Models Adaptive Adversary

28. 28  Clearly, we can define the following proportion –  Let be the lowest oblivious competitive coefficient of any randomized paging algorithm  Similarly we define  Finally we define to be the lowest competitive coefficient of any deterministic paging algorithm  Then we have Adversary Models Adaptive Adversary

29. 29 Overview-part B  Introduction Paging against an oblivious – Definitions. The players on this section – What is the goal? Yao’s minimax theorem.  Proof of a Theorem on C R : Preparations. Behavior of the offline algorithm. Behavior of a deterministic algorithm. Results.  The marker algorithm Description. Proof of it’s competitiveness coefficient.  Summary

30. 30 Paging against an oblivious adversary  Remember that in the proof of the result : C A ≥ k we based our analysis on the fact that the request sequence was determined at each step according to the behavior (the cache contents) of the deterministic online algorithm  When talking about randomized algorithm it’s intuitive that this ability of the offline algorithm won’t be so helpful

31. 31 Our goal now  We now want to prove that against an oblivious adversary a randomized online algorithm can do much better in terms of competitiveness against the result (K) we saw when we talked about deterministic online algorithm.  We will see that under the above assumptions. where is the k’th harmonic number which is clearly smaller then K.

32. 32 So… who against who?  Our players now are an oblivious adversary which specifies all the request sequence in advance and never changing it later on  Then, an optimal offline algorithm is activated on this sequence and announcing it’s result  Only then – the sequence is given to the random online algorithm we want to test – each request at a time.

33. 33 A reminder : Yao’s MiniMax principle  Yao's minimax principle states that given any arbitrarily chosen input distribution P: the expected cost of the optimal deterministic algorithm under this distribution is a lower bound on the expected cost of the optimal randomized (Las Vegas) algorithm.

34. 34 How to use this principle for our purposes?  The principle gives us the ability to deal only with deterministic algorithms in order to give a lower bound for the randomized algorithm.  All we need to do is to choose a probability distribution for the inputs and then give a lower bound for the best (in terms of expected cost) deterministic online algorithm under this distribution.

35. 35 Overview-part B  Introduction Paging against an oblivious – Definitions. The players on this section – What is the goal? Yao’s minimax theorem.  Proof of a Theorem on C R : Preparations. Behavior of the offline algorithm. Behavior of a deterministic algorithm. Results.  The marker algorithm Description. Proof of it’s competitiveness coefficient.  Summary

36. 36 Some preparations:  The probability distribution on the inputs can be taught as the probability to choose the i’th memory item on the sequence – this probability can depend in the older requests on the chosen sequence;  Given a deterministic online paging algorithm A we define it’s competitiveness under a distribution ρ to be the infimum of C such that:

37. 37 Cont.  Formally – the MiniMax principle gives us:  Meaning of the left member: The competitiveness co efficient of the best random algorithm (Vs. an oblivious adversary).  Meaning of the Right member: The competitiveness coefficient of the best deterministic online Alg. Under a “worst-case” input distribution probability.

38. 38  Since we are looking for a lower bound we can look at a very simple case  We will look on a world where there are only K+1 memory items: I = {I 1, ……….,I k+1 }  Since the cache can consist at most K items at once only one of the k + 1 items is out at each step  So, we can say the algorithm decides which of the items will be out at each step. Theorem: Let R be a randomized algorithm for paging then

39. 39 Choosing a “worst-case” distribution  First we should give P – an input probability distribution : - The sequences to be chosen will be of length N – we assume N>>K. 1. Choose the first request p 1 uniformly from all the items in I. 2. Choose the i’th request p i uniformly from the set : I\{p i-1 }. i>1 Intuitive explanation: We never request an item that we are sure about it’s status – this is why this is a worst-case distribution.

40. 40 Demonstration I4I4 12345678910 I = {I 1,I 2,I 3, I 4,I 5,I 6,I 7,I 8,I 9,I 10,I 11 } current choice set = {I 1,I 2,I 3, I 5,I 6,I 7,I 8,I 9,I 10,I 11 } I4I4 I5I5 12345678910

41. 41 Now we need to find a lower bound for A – an optimal det’ online algorithm  We will use again the notion of rounds that Achiya has mentioned: The first round starts with the first request and ends when, in the first time all the k+1 items in I has been requested at least once. The next round starts with the next request and ends just before the request to the (k+1)th distinct item since the start of the current round. All next rounds acts the same.

42. 42 How the offline algorithm behaves on each round?  The offline algorithm will use MIN on each round – the result will be that the item left out will be the one requested only at the last request of the round  This means that the offline algorithm will miss only once during a round – at the last request of a round  when it adds the missed item to the cache it puts out the last requested item of the next round

43. 43 Demonstration I = {I 1,I 2,I 3,I4} Requests= {I1,I2,I1,I4,I3,I2,I4,I1,I3,I1,I3,I2,I4,I2,I4,I3} I1I2I4 123 321 I2I3 321 I4I2 321 I1I2I3 321 I 321 I4I1I2 321 I4I

44. 44 How often does the offline algorithm miss?  We saw that the answer to that question depends the length of each round ( since it misses once in a round).  We can ask an equivalent question: what is the expected number of random choices we need to do until we choose all items in a group of order K+1?  Let’s look at it as a random walk on a complete graph with k+1 vertices – the expected number of steps to achieve full cover of that graph was approved to be : K*H k

45. 45 How a deterministic online algorithm behaves on each round?  Since the request sequence is random we can’t expect any regularity in the sequence.  Since the policy of the known deterministic algorithms is based on some kind of regularity we can’t expect any of these policies to be better then the others.  So – the probability for a miss is totally random.

46. 46 Cont.  the probability that any request falls on the item that A leaves out at same time-step is exactly 1/k  explanation: remember there is no item that will be requested twice consecutively so the next request is for one of the other k items that A might have choose to leave out.  Given the expected length of a round we conclude that the expected number of misses by A during a round will be: (K*H k )*(1/k) = H k

47. 47 Let’s finish the proof  We found that the any of the deterministic online algorithms misses (expected) H k times the number of (expected) misses by an optimal offline algorithm  This result is under the worst case input distribution we defined  So we found H k to be the competitiveness coefficient of A and by the MinMax principle we conclude this is also the competitiveness coefficient of an optimal randomized algorithm for our problem

48. 48 Overview-part B  Introduction Paging against an oblivious – Definitions. The players on this section – What is the goal? Yao’s minimax theorem.  Proof of a Theorem on C R : Preparations. Behavior of the offline algorithm. Behavior of a deterministic algorithm. Results.  The marker algorithm Description. Proof of it’s competitiveness coefficient.  Summary

49. 49 The Marker Algorithm  After we proved that there are randomized online algorithms with a competitiveness coefficient of H k we will now see an example for a randomized online algorithm with a very close achievement.  The algorithm require K bits (marker bits) – one for each cache location  At The beginning all these bits are set to 0.

50. 50 The Algorithm 1. Given a memory request: 1.1 if the requested Item is currently in cache – mark it’s location (set mark bit to 1). 1.2 else bring the Item into memory and evict an item by the following policy: 1.2.1 choose uniformly at random an unmarked memory location. 1.2.2 mark this location, replace the item in it with the currently requested item. 2. If all memory locations are marked - unmark all locations when the next request is given.

51. 51 Theorem : The Marker Algorithm is 2*H k - competitive  We will again use the notion of rounds – about the same definition as earlier :  A round starts with Pi that is not in the cache and ends with Pj where : P i,P i+1,……….P j,P j+1 where j+1 is the smallest integer s.t the above list consists k+1 distinct items.  As we will demonstrate each round ends with all the locations marked.  Each round begins with an item that is not currently in the cache.

52. 52 Demonstration I = {I 1,I 2,I 3,I 4 } Requests= {I1,I2,I1,I4,I3,I2,I4,….I1,I3,I1,I4,I2,I4,I2,I3} I1 123 321 I4I1 321 I4I3 321 I2I3I1 321321 I2I3I4 321 I 321 I2I1 321 I2 321 I4I2I1 321 I4I2 321 I4I2I1 321 I4I2 321 I3I4 321 I I1 123 321 I4I1 321 I4I3 321 I4I3I2 321321 I4I3I2 321 I 321 I3I4 321 I

53. 53 Some definitions  We will divide all the items requested during a round to 2 groups:  Stale item is an item that is currently unmarked but was marked during the last round – meaning: he was requested during the last round but not during this round  clean item is an item that is not stale and/or marked – meaning it wasn’t requested during the last and current rounds  Let l be the number of clean items requested during each round.

54. 54 Some more definitions :  S 0 is the set of k items currently in the offline algorithm cache  S m is the set of k items currently in the Marker algorithm cache  D I = |S 0 /S M | at the beginning of a round  D F = |S 0 /S M | at the end of a round  M 0 = the number of misses incurred by the offline algorithm cache during a round.

55. 55 Observetions:  Note that each of the distinct items is either clean or stale at the beginning of a round.  M0≥ l – D I the number of misses by the offline algorithm is at least as the number of the items that wasn’t requested during the last round (l) and are not in it’s cache at the beginning of the round.  on the other end, all The k items in S m at the end of a round are items that were requested during the last round so we can conclude that the offline algorithm missed at least D F misses in the last round since these D F items are not in Sm!

56. 56 How many time the offline algorithm miss during all rounds?  By the last observation we have:  Note that D F of the k’th round equals D I of the (k+1)’th round so they are telescoping when summing the above lower bound on all rounds.  We can use amortization and we charge each round with l/2 misses for the offline algorithm.  Our mistake range is ±2k – size of the 2 cache memories.

57. 57 How many time the Marker algorithm miss during all rounds?  First of all by definition, all the clean items are missed during each round  For the k-l requests to stale items we should consider the probability that the item requested is not in the cache and this depends directly on the number of clean items requested before him in the round  See the board for calculation of this probability and the expected number of misses.

58. 58 So, let’s finish the proof:  We saw the Marker algorithm misses at most twice the best result possible (achieved by the offline algorithm:  Note that these of you interested can find a more sophisticated algorithm that actually achieves Hk competitiveness coefficient.

59. 59 Overview-part B  Introduction Paging against an oblivious – Definitions. The players on this section – What is the goal? Yao’s minimax theorem.  Proof of a Theorem on C R : Preparations. Behavior of the offline algorithm. Behavior of a deterministic algorithm. Results.  The marker algorithm Description. Proof of it’s competitiveness coefficient.  Summary

60. 60 Some more interesting insights/summary  One should ask himself if it was a fair game since we gave the randomized algorithm some more comfortable conditions comparing to these of the deterministic algorithm  Well the all advantage of the randomized algorithm lies on the fact that it’s actions are unexpected  This way we prevent the ability to predict the behavior of the algorithm and affect it.

61. 61 Cont.  This is why it’s intuitive to think on an oblivious adversary when talking on randomized online algorithms which hides the status of memory and the eviction policy at each step  At the same time, it’s intuitive that on application where it is important to prevent outsiders from affecting the algorithm we won’t use a deterministic algorithm which is totally expectable  Application for these kind of algorithms with changing demands can be found in finance, robot navigation, short paths in graph, task systems etc.

62. 62 Further results  For these of you interested: Further results shows much less impressive achievements of the randomized algorithms against the smarter adversaries presented by Achiya.  competitive coefficient against the online adaptive algorithm : k  competitive coefficient against the offine adaptive algorithm : k*H k

63. 63 THE END