1. Optimization Problem Based on L2,1-norms
Xiaohong Chen

2. Outline
Efficient and robust feature selection via joint l2,1-norm minimzation
Robust and discriminative distance for multi-instance learning
Its application…

3. Outline
Efficient and robust feature selection via joint l2,1-norm minimization
Robust and discriminative distance for multi-instance learning
Its application…

4. Efficient and robust feature selection via joint l2,1-norm minimzation

5. Robust Feature Selection Based on l21-norm
Given training data {x1, x2,…, xn} and the associated class labels {y1,y2,…, yn}
Least square regression solves the following optimizaiton problem to obtain the projection matrix W
Add a regularization R(W) to the robust version of LS,

6. Robust Feature Selection Based on l21-norm
Possible regularizations
Ridge regularization
Lasso regularization
Penalize all c regression coefficients corresponding to a single feature as a whole

7. Robust Feature Selection Based on l21-norm

8. Robust Feature Selection Based on l21-norm
Denote
(14)

9. Robust Feature Selection Based on l21-norm
Then we have
(19)

10. The iterative algorithm to solve problem (14)
Theorem1: The algorithm will monotonically decrease the objective
of the problem in Eq.(14) in each iteration, and converge to the global
optimum of the problem.

11. Proof of theorem1 u

12. Proof of theorem1

13. (1)
(2)
(1)+(2)

14. Outline
Efficient and robust feature selection via joint l2,1-norm minimization
Robust and discriminative distance for multi-instance learning
Its application…

15. Robust and discriminative distance for multi-instance learning

16. Multi-instance learning
多示例学习中，训练集由若干个具有概念标记的包(bag)组成，
每个包包含若干个没有概念标记的示例。若一个包中至少有
一个正例，则该包被标记为正(positive)，若一个包中所以示
例都是反例，则该包被标记为反(negative),通过对训练包的学
习，希望学习系统尽可能正确地对训练集之外的包的概念标
记进行预测。

17. The illustration of MIL

18. Notations Given N training bags and K conceptual classes.
Each bag contains a number of instances
Given the class memberships of the input data, denoted as

19. Notations
First, we represent every class as a super-bag that comprises the
instances of all its training , where

20. Objective to learn class specific distance metrics
For a given class, Ck,, we solve the following optimization problem:

21. Algorithm and its analysis

22. Algorithm and its analysis

23. Algorithm and its analysis

24. Algorithm and its analysis
On the other hand,

25. Algorithm and its analysis

26. Algorithm and its analysis

27. Algorithm and its analysis
Therefore, the objective value of the problem of (6) is decreased in
each iteration till convergences.

28. Outline
Efficient and robust feature selection via joint l2,1-norm minimzation
Robust and discriminative distance for multi-instance learning
Its application…

29. Its application
For example:

30. Reference
[1]F.Nie, D.Xu, X.Cai, and C.Ding. Efficient and robust feature selection via
joint l2,1-norm minimzation. NIPS 2010.
[2] H.Wang, F.Nie and H.Huang. Robust and discriminative distance for multi-
instance learning, CVPR 2012:

31. Thanks! Q&A