1. Scalable Maximum Margin Factorization by Active Riemannian Subspace search
Yan Yan, Mingkui Tan, Ivor W. Tsang, Yi Yang,
Chengqi Zhang and Qinfeng Shi
QCIS, University of Technology, Sydney
ACVT, The University

2. Outline
Introduction
The Proposed Model
Experiments
Conclusion

3. Collaborative filtering for recommendation systems
Goal
Recover missing ratings by low-rank matrix completion
Real world applications
Recommend TV shows/movies on Netflix
Recommend artists/music tracks on Xiami
Recommend products on Taobao…
Data that can be used
Partially observed rating data from users on items
A specific output of recommendation systems
The predicted ranking scores of users on unseen items

4. A user/item rating matrix on movies
Figure: An example of a user/item rating matrix on movies

5. Problem setup of matrix completion
Reconstruct the rating matrix X with a low-rank constraint
Y is the observed matrix
The problem is NP-hard
Approach: matrix factorization

6. Matrix factorization approach
Figure: Matrix factorization

7. Challenges Real world rating data are in discrete values
Maximum margin matrix factorization
Existing methods usually requires repetitive SVDs
Our optimization avoids repetitive SVDs and applies cheaper QR
The latent variable r is usually unknown and can be different among various datasets
A automatic method to detect the rank

8. Maximum margin matrix factorization (M3F)
Hinge loss: appropriate for discrete rating data in real world
M3F for binary values (-1/+1)
From binary values to ordinal values
Suppose
Introduce L+1 thresholds

9. Maximum margin matrix factorization (M3F)
M3F for ordinal values

10. Maximum margin matrix factorization (M3F)
Figure: M3F loss for discrete ordinal values

11. Formulation

12. Differential Geometry of Fixed-rank Matrices

13. Differential Geometry of Fixed-rank Matrices
Figure: Gradient descent on Riemannian manifold

14. Differential Geometry of Fixed-rank Matrices

15. Differential Geometry of Fixed-rank Matrices
Figure: Gradient descent on Riemannian manifold

16. Differential Geometry of Fixed-rank Matrices
Retraction
Retraction can be cheaply calculated without SVD in

17. Line Search on Riemannian Manifold
Figure: Gradient descent on Riemannian manifold

18. BNRCG: Block-wise nonlinear Riemannian conjugate gradient descent for M3F

19. Active Riemannian subspace search for M3F: ARSS-M3F

20. Active Riemannian subspace search for M3F: ARSS-M3F
Step 1: Increase the rank.

21. Active Riemannian subspace search for M3F: ARSS-M3F
Step 2: Update X and thresholds.

22. Experiments Data sets # users # items # ratings Binary-syn 1,000 All
Ordinal-syn-small
Ordinal-syn-large
20,000
Movielens 1M
6,040
3,952
1,000,209
Movielens 10M
71,567
10,681
10,000,054
Netflix
480,189
17,770
100,480,507
Yahoo! Music Track 1
1,000,990
624,961
262,810,175

23. The sensitivity of the regularization parameter experiment

24. The convergence behavior experiment

25. RMSE and consumed time on the synthetic datasets

26. RMSE and consumed time on Movielens 1M and Movielens 10M

27. RMSE and consumed time on Netix and Yahoo Music

28. Conclusion
Two challenges in M3F: scalability and latent factor detection
BNRCG addresses the scalability problem by exploiting Riemannian geometry
ARSS-M3F applies an efficient and simple method to detect the latent factor
Extensive experiments demonstrate the proposed method can provide competitive performance.

29. Thank you!