1 Data Mining at work Krithi Ramamritham

2 Dynamics of Web Data Dynamically created Web Pages -- using scripting languages Ad Component Headline Component Headline Component Headline Component Headline Component Personalized Component Navigation Component

3 1. What to deliver? Page content may be based on queries on dynamically changing data – e.g., sports scores, stock prices, environment type of access device time and location of access/user Existing sites may contain new information New sites (URLs) may come into being

4 2. How to deliver? Data sources Proxies /caches End-hosts servers sensors wired host mobile host Network

5 Keep Data Up-to-date Update Mumbai temperature every 2 degrees The proxy obtains data from the source(s) | U(t) - S(t) | <= 2Maintains | U(t) - S(t) | <= 2 Source S(t) Proxy / DB P(t) User U(t)

6 When to poll the source? After a specific interval Server Proxy User Pull Based on temporal data mining – time series analysis – and prediction of when change will exceed 2 degrees

7 Where to do the work? Diverse client devices –Differ in hardware, software, network connectivity, form factor Web content needs to be tailored for each client type Each response depends not only on the requested URL but also on the capabilities of the client

8 Transcoding Conversion of one data version to another –Decreasing Image Quality (JPEG quality level) and size - “convert” utility in Linux –Summarizing text transcode => Info extraction/ retrieval/ classification

9 Who should transcode? 1.Download desired version from server 2.Transcode higher version locally Factors influencing decision –Transcoding Complexity –Proxy-server network connection –Load on proxy (Multiple Linear) Regression Predict based on a (linear) model of overheads

10 What is new on the Web? How is the monsoon progressing? Time series analysis: Change prediction, pattern mining

‘Bhav Puchiye’ Interface for Bhav Puchiye

Inverted Pyramid Interfaces Inverted pyramid approach Conclusion Findings Discussions Conclusion Discussions Findings Background & related Information

Bhav Poochiye Pricing Module developed for selected commodities for selected markets for selected areas DEMO

14 Building Usage Profiles Estimate access probabilities based on: Current user/community navigational patterns over site contents (in the form of click streams) Historical user/community access patterns over site contents (in the form of association rules) Cluster needs based on location, income/age of user, time-of-day

15 Data Mining From data to information to knowledge to money!