Download presentation

Presentation is loading. Please wait.

Published byAshleigh Hench Modified over 4 years ago

1
Regularized Least-Squares

2
Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints

3
Regularized Least-Squares Why regularization? We have seen that

4
Regularized Least-Squares Why regularization? We have seen that But what happens if the system is almost dependent? –The solution becomes very sensitive to the data –Poor conditioning

5
Regularized Least-Squares The 1-dimensional case The 1-dimensional normal equation

6
Regularized Least-Squares The 1-dimensional case The 1-dimensional normal equation

7
Regularized Least-Squares The 1-dimensional case The 1-dimensional normal equation

8
Regularized Least-Squares Why regularization Contradiction between data and model

9
Regularized Least-Squares A more interesting example: scattered data interpolation

10
Regularized Least-Squares “True” curve

11
Regularized Least-Squares Radial basis functions

12
Regularized Least-Squares Radial basis functions

13
Regularized Least-Squares Rbf are popular Modeling –J. C. Carr, R. K. Beatson, J. B. Cherrie, T. J. Mitchell,W. R. Fright, B. C. McCallum, and T. R. Evans. Reconstruction and representation of 3d objects with radial basis functions. In Proceedings of ACM SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, pages 67–76, August 2001. –G. Turk and J. F. O’Brien. Modelling with implicit surfaces that interpolate. ACM Transactions on Graphics, 21(4):855–873, October 2002. Animation –J. Noh and U. Neumann. Expression cloning. In Proceedings of ACMSIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, pages 277–288, August 2001. –F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D. H. Salesin. Synthesizing realistic facial expressions from photographs. In Proceedings of SIGGRAPH 98, Computer Graphics Proceedings, Annual Conference Series, pages 75–84, July 1998.

14
Regularized Least-Squares Radial basis functions At every point

15
Regularized Least-Squares Radial basis functions At every point Solve the least-squares problem

16
Regularized Least-Squares Radial basis functions At every point Solve the least-squares problem

17
Regularized Least-Squares Rbf results

18
Regularized Least-Squares p i 0 close to p i 1

19
Regularized Least-Squares Radial basis functions At every point Solve the least-squares problem

20
Regularized Least-Squares Radial basis functions At every point Solve the least-squares problem If p i 0 close to p i 1, A is near singular

21
Regularized Least-Squares p i 0 close to p i 1

22
Regularized Least-Squares p i 0 close to p i 1

23
Regularized Least-Squares Rbf results with noise

24
Regularized Least-Squares Rbf results with noise

25
Regularized Least-Squares The Singular Value Decomposition Every matrix A ( nxm ) can be decomposed into: –where U is an nxn orthogonal matrix V is an mxm orthogonal matrix D is an nxm diagonal matrix

26
Regularized Least-Squares The Singular Value Decomposition Every matrix A ( nxm ) can be decomposed into: –where U is an nxn orthogonal matrix V is an mxm orthogonal matrix D is an nxm diagonal matrix

27
Regularized Least-Squares Geometric interpretation

28
Regularized Least-Squares Solving with the SVD

29
Regularized Least-Squares Solving with the SVD

30
Regularized Least-Squares Solving with the SVD

31
Regularized Least-Squares Solving with the SVD

32
Regularized Least-Squares Solving with the SVD

33
Regularized Least-Squares A is nearly rank defficient

34
Regularized Least-Squares A is nearly rank defficient

35
Regularized Least-Squares A is nearly rank defficient

36
Regularized Least-Squares A is nearly rank defficient =>some of the are close to 0

37
Regularized Least-Squares A is nearly rank defficient =>some of the are close to 0 =>some of the are close to

38
Regularized Least-Squares A is nearly rank defficient =>some of the are close to 0 =>some of the are close to Problem with

39
Regularized Least-Squares A is nearly rank defficient =>some of the are close to 0 =>some of the are close to Problem with Truncate the SVD

40
Regularized Least-Squares p i 0 close to p i 1

41
Regularized Least-Squares Rbf fit with truncated SVD

42
Regularized Least-Squares Rbf results with noise

43
Regularized Least-Squares Rbf fit with truncated SVD

44
Regularized Least-Squares Choosing cutoff value k The first k such as

45
Regularized Least-Squares Example: inverse skinning “Skinning Mesh Animations”, James and Twigg, siggraph Skinning

46
Regularized Least-Squares Example: inverse skinning “Skinning Mesh Animations”, James and Twigg, siggraph Skinning ?

47
Regularized Least-Squares Example: inverse skinning “Skinning Mesh Animations”, James and Twigg, siggraph Skinning

48
Regularized Least-Squares Example: inverse skinning “Skinning Mesh Animations”, James and Twigg, siggraph Skinning Inverse skinning –Let be a set of pairs of geometry and skeleton configurations

49
Regularized Least-Squares Example: inverse skinning “Skinning Mesh Animations”, James and Twigg, siggraph Skinning Inverse skinning –Let be a set of pairs of geometry and skeleton configurations

50
Regularized Least-Squares Example: inverse skinning “Skinning Mesh Animations”, James and Twigg, siggraph Skinning Inverse skinning –Let be a set of pairs of geometry and skeleton configurations

51
Regularized Least-Squares Example: inverse skinning “Skinning Mesh Animations”, James and Twigg, siggraph Skinning Inverse skinning –Let be a set of pairs of geometry and skeleton configurations

52
Regularized Least-Squares “Skinning Mesh Animations”, James and Twigg, siggraph

53
Regularized Least-Squares Problem with the TSVD We have to compute the SVD of A, and O() process: impractical for large marices Little control over regularization

54
Regularized Least-Squares Damped least-squares Replace by where is a scalar and L is a matrix

55
Regularized Least-Squares Damped least-squares Replace by where is a scalar and L is a matrix The solution verifies

56
Regularized Least-Squares Examples of L DiagonalDifferential Limit scaleEnforce smoothness

57
Regularized Least-Squares Rbf results with noise

58
Regularized Least-Squares

61
Example: “Least-Squares Meshes”, Sorkin and Cohen-Or, siggaph Reconstruct a mesh given –Control points –Connectivity (planar mesh)

62
Regularized Least-Squares Example: “Least-Squares Meshes”, Sorkin and Cohen-Or, siggaph Reconstruct a mesh given –Control points –Connectivity (planar mesh) Smooth reconstruction

63
Regularized Least-Squares Example: “Least-Squares Meshes”, Sorkin and Cohen-Or, siggaph Reconstruct a mesh given –Control points –Connectivity (planar mesh) Smooth reconstruction In matrix form

64
Regularized Least-Squares Reconstruction Minimize reconstruction error where

65
Regularized Least-Squares “Least-Squares Meshes”, Sorkine and Cohen-Or, siggraph

66
Regularized Least-Squares Quadratic constraints Solve or

67
Regularized Least-Squares Quadratic constraints Solve or

68
Regularized Least-Squares Example

69
Regularized Least-Squares Example

70
Regularized Least-Squares Example

71
Regularized Least-Squares Discussion If, there is no solution (since there is no x for which )

72
Regularized Least-Squares Discussion If, there is no solution (since there is no x for which ) If, the solution exists and is unique

73
Regularized Least-Squares Discussion If, there is no solution (since there is no x for which ) If, the solution exists and is unique –Either the solution of is in the feasible set

74
Regularized Least-Squares Discussion If, there is no solution (since there is no x for which ) If, the solution exists and is unique –Either the solution of is in the feasible set –or the solution is at the boundary Solve

75
Regularized Least-Squares Discussion Solve where is a Lagrange multiplier

76
Regularized Least-Squares Conclusion TSVD really useful if you need an SVD

77
Regularized Least-Squares Conclusion TSVD really useful if you need an SVD Regularization constrains the solution: –Value, differential operator, other properties –Soft (damping) or hard constraint (quadratic) –Linear or non-linear

78
Regularized Least-Squares Conclusion TSVD really useful if you need an SVD Regularization constrains the solution: –Value, differential operator, other properties –Soft (damping) or hard constraint (quadratic) –Linear or non-linear Danger of over-damping or constraining

79
Regularized Least-Squares Example: inverse kinematic Problem: solve for joint angles given end-effector positions

80
Regularized Least-Squares Example: inverse kinematic Problem: solve for joint angles given end-effector positions ?

81
Regularized Least-Squares Example: inverse kinematic Problem: solve for joint angles given end-effector positions

82
Regularized Least-Squares Example: inverse kinematic Problem: solve for joint angles given end-effector positions

Similar presentations

OK

CS246 Linear Algebra Review. A Brief Review of Linear Algebra Vector and a list of numbers Addition Scalar multiplication Dot product Dot product as a.

CS246 Linear Algebra Review. A Brief Review of Linear Algebra Vector and a list of numbers Addition Scalar multiplication Dot product Dot product as a.

© 2018 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google