Dissemination of Dynamic Data on the Internet Krithi Ramamritham Pavan Deolasee Amol Katkar Ankur Panchbudhe Prashant Shenoy Dec 2000

Overview of Presentation. Dynamic Data Temporal Coherency Cache Consistency Push and Pull Approaches Combining Push and Pull Dynamically choosing Push or Pull Conclusions

Dynamic Data Dissemination Data which changes rapidly & unpredictably. e.g Sports scores, traffic or weather data. Coherency requirements Nature of the item User tolerances

How dynamic data is dealt with? Functions of a Proxy Exploits user specified coherency. Maintains desired temporal coherency. Server Proxy User Push Push Pull

Temporal Coherency Each cached item must be periodically refreshed. For highly dynamic data Difficult to maintain cache consistency. Heavy network overload. Server load.

How temporal coherency is achieved. User is not interested in every change happening at the source. User specifies a temporal coherency requirement c for each cached item. Proxy depending on value of c can use push or pull approach to maintain coherency.

Cache Consistency. Fidelity of the data Degree to which coherency needs of a user are met. Normally, the problem of cache consistency is resolved by 2 approaches Client-driven Server-driven

Cache Consistency Client-driven Polling each time Adaptive TTR(time to refresh) Server-driven Invalidates cache entries Updates proxy cache In case of dynamic data, Cannot deliver fidelity with optimum resource utilization

The Pull Approach Each data item is assigned a certain TTR & until that time all requests are satisfied from the cache. Proxy issues a get request to the server. Periodic Server Push Proxy User Periodic Pull Server Pull Proxy Push User Aperiodic

Pull Approach. Periodic Pull Proxy periodically polls the server Obtains data with a high frequency Disadvantages Very high network overhead User may miss some changes Not suitable if rate of change is varying

Pull Approach(Contd..) Aperiodic pull TTR decreases dynamically when a data item starts changing rapidly Increases when a hot file becomes cold Adaptive TTR takes into account Rapid changes that have occurred so far Recent changes to the polled data

Push Approach Server can push data either periodically or aperiodically Periodic Push Disseminates data based on demand for data item All data items get divided into frequency bands Disadvantages Low fidelity, Wastage of bandwidth

Push Approach(Contd..) Aperiodic Push Proxy registers with the server Server uses tcr to determine if data item is to be pushed. Server maintains state information as list of proxies,tcr and last update to each proxy Combines requests with identical tcr into a single request Disadvantages Limits scalability, not resilient to failures.

Push v/s Pull Communication Overhead Pull- Larger load on the network Push-Large message overhead Computational Overhead High load due to too much polling or monitoring Space Overhead Maintains c value, pushed value, state with open connection

Push v/s Pull Resiliency State of server is lost in case of server failures In client failures, resources assigned must be reclaimed Scalability With upper bound on the number of sockets and state space available, servers become scalable It arises due to excessive server computation and resources allocated

Combining Push and Pull Leases Contracts given to a lease holder over some property Server informs the client about any changes during the lease period Adaptive Leases Dynamically adjust the lease duration Gives strong cache consistency

Combining Push and Pull Server tries to predict when a client is going to poll next If it knows that a client is going to miss some change it pushes the data to the client In this approach the performance in terms of fidelity and resiliency can be controlled

Dynamically choosing Push v/s Pull If resources are plentiful, every client is given a push connection As clients increase, some clients are shifted to pull mode and scalability is ensured Clients are assigned priorities by Temporal coherency requirement Fidelity requirement Network bandwidth available

Conclusions It is a priori difficult to determine whether a push or pull based approach is to be employed. Combination of Push and Pull is used. Currently work is going on in determining range of applicability of new algorithms for disseminating web data