Presentation is loading. Please wait.

Presentation is loading. Please wait.

TiVo Suggestions: Predicting Viewer Affinity Using Collaborative Filtering Kamal Ali – TiVo, Yahoo Wijnand van Stam, TiVo.

Similar presentations


Presentation on theme: "TiVo Suggestions: Predicting Viewer Affinity Using Collaborative Filtering Kamal Ali – TiVo, Yahoo Wijnand van Stam, TiVo."— Presentation transcript:

1 TiVo Suggestions: Predicting Viewer Affinity Using Collaborative Filtering Kamal Ali – TiVo, Yahoo Wijnand van Stam, TiVo

2 Outline l What is TiVo ? l Why Suggestions? l Collaborative filtering background l TiVo collaborative filtering data cycle l Server-side learning l Previous Work l Contributions

3 Contributions l Large fielded system wLarge number (3M) of users wLong-lived interaction w user: >90 t/user w10^8 ratings over 300K shows wVery large in user-hours l Distributed architecture zServer: Throttle-able zClients do bulk of work l Privacy-preservation z Privacy and distributed goals aligned z No persistent memory of user on server

4 What is TiVo ? l TiVo = set-top TV box + program-guide service l Pause & rewind live TV l Linux OS l Viewers can rate shows l Suggestions l Q4 1999

5 Why Suggestions? l Connect users to shows theyll like l Predict degree to which viewer will like TV show l Produces ranked list of upcoming shows l Records shows if disk space is available

6 Filtering Background Recommendation Systems Content-based: use intrinsic features such as genre, cast, director, writers, age, channel-type,… Collaborative filtering: use other peoples ratings Combined, Cascaded

7 Content isnt sufficient l Genres are few l Text length is small

8 Data cycle Thumbs Profile on TiVo Client box Random ID generated for profile and stored on server 1: Collecting Feedback: Thumbs up/dn Recorded 2. TiVo calls server uploads entire anonymized profile Correlation pairs on server 3. Server- side learning Correlation pairs on client 4. Download pairs during some client-initiated calls Rated shows in sorted order 5. Use correlations and Thumbs profile to rate shows

9 Collaborative Filtering Model k Nearest Neighbor over other rated correlated shows Use Pair-wise Pearson correlation Adjusted correlation for low support Use weighted linear combination

10 1. Collecting Feedback l Explicit: wThumbs up, down: l Implicit: wUser-initiated recording thumbs

11 2. Privacy and Data Upload l TiVo calls server daily l Entire profile uploaded and given temp id l Server deletes old profiles: sliding window

12 3.1 Server-side scaling l 300,000 unique shows /week l 10^11 pairs of shows l 3M users l Average of 90 thumbs / user: > 10^8 thumbs (ratings) l Ratings are sparse in the pair space l Dont need to predict for very unpopular pairs

13 3.2 Server-side Learning l Building pair-wise item/item correlations on server l Use simple Pearson pair-wise correlation z7 ratings levels per show [-3 … +3] wOnly need to maintain 7 * 7 array of counts per pair l Efficient: CPU, memory l Compute r-to-z transform to computer confidence interval zSupport-penalized degree of correlation: lower bound of confidence-interval zDistinguishes r = 0.8 for S=10 versus S=1000

14 3.3 Throttled Server-side Architecture Log Collector 1 Boxes K Log Collector m Boxes 100K(m-1).. 100K m By-series Counter 1 Series 0..30K By-series Counter n Series 30k(n-1)..30kn 1: By-series-pair Counter and Correlations Calc. P: By-series-pair Counter and Correlations Calc. Transmit correlation pairs to TiVo Clients

15 3.4 Server-side throttling l min_single (150) l min_pair (100) l Throttle-able: zMore HW available zIncreasing TiVo population zGo deeper into distribution tail

16 Details l Pearson r l Weighted average l r-to-z transform (Fisher) l Standard: Lower bound of confidence interval:

17 4. Download to clients w 28K pairs sent to client (320kb) w Correl. between old shows dont change fast w New Shows: want to do it faster

18 5. Client-side processing l Ratings must not cause video glitching! l 2am: TiVo re-rates all shows l Collab: k-nearest neighbor l Content-based: Naïve Bayes

19 Previous Work l User-user or item-item - Sarwar et al l Form of model wk-nearest neighbor, wBayes nets (Breese et al.), wFactor Analysis (Canny) wSimilarity/distance function wPearson (subsumes cosine) wTFIDF corrections (Salton et al.) wUser amplification l Combination functions: k-NN, Bayes nets.. l Evaluation Criteria: MAE, Spearman rank correl.

20 Contributions l Large fielded system wLarge number (3M) of users wLong-lived interaction w user: >90 t/user w10^8 ratings over 300K shows wVery large in user-hours l Distributed architecture zServer: Throttle-able zClients do actual suggestion calculations l Privacy-preservation z Privacy and distributed goals aligned z No persistent memory of user on server


Download ppt "TiVo Suggestions: Predicting Viewer Affinity Using Collaborative Filtering Kamal Ali – TiVo, Yahoo Wijnand van Stam, TiVo."

Similar presentations


Ads by Google