Download presentation

1
**Methods For Nonlinear Least-Square Problems**

Jinxiang Chai

2
**Applications Inverse kinematics Physically-based animation**

Data-driven motion synthesis Many other problems in graphics, vision, machine learning, robotics, etc.

3
Problem Definition Most optimization problem can be formulated as a nonlinear least squares problem Where , i=1,…,m are given functions, and m>=n

4
Data Fitting

5
Data Fitting

6
Inverse Kinematics Find the joint angles θ that minimizes the distance between the character position and user specified position θ2 θ2 l2 l1 θ1 C=(c1,c2) Base (0,0)

7
**Global Minimum vs. Local Minimum**

Finding the global minimum for nonlinear functions is very hard Finding the local minimum is much easier

8
Assumptions The cost function F is differentiable and so smooth that the following Taylor expansion is valid,

9
Gradient Descent Objective function: Which direction is optimal?

10
Gradient Descent Which direction is optimal?

11
**Gradient Descent A first-order optimization algorithm.**

To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient of the function at the current point.

12
Gradient Descent Initialize k=0, choose x0 While k<kmax

13
**Newton’s Method Quadratic approximation**

What’s the minimum solution of the quadratic approximation

14
Newton’s Method High dimensional case: What’s the optimal direction?

15
Newton’s Method Initialize k=0, choose x0 While k<kmax

16
Newton’s Method Finding the inverse of the Hessian matrix is often expensive Approximation methods are often used - conjugate gradient method - quasi-newton method

17
Comparison Newton’s method vs. Gradient descent

18
Gauss-Newton Methods Often used to solve non-linear least squares problems. Define We have

19
Gauss-Newton Method In general, we want to minimize a sum of squared function values

20
Gauss-Newton Method In general, we want to minimize a sum of squared function values Unlike Newton’s method, second derivatives are not required.

21
Gauss-Newton Method In general, we want to minimize a sum of squared function values

22
Gauss-Newton Method In general, we want to minimize a sum of squared function values Quadratic function

23
Gauss-Newton Method In general, we want to minimize a sum of squared function values Quadratic function

24
Gauss-Newton Method In general, we want to minimize a sum of squared function values Quadratic function

25
Gauss-Newton Method In general, we want to minimize a sum of squared function values Quadratic function

26
Gauss-Newton Method Initialize k=0, choose x0 While k<kmax

27
Gauss-Newton Method In general, we want to minimize a sum of squared function values Any Problem? Quadratic function

28
Gauss-Newton Method In general, we want to minimize a sum of squared function values Any Problem? Quadratic function

29
Gauss-Newton Method In general, we want to minimize a sum of squared function values Any Problem? Quadratic function Solution might not be unique!

30
Gauss-Newton Method In general, we want to minimize a sum of squared function values Any Problem? Quadratic function Add regularization term!

31
**Levenberg-Marquardt Method**

In general, we want to minimize a sum of squared function values Any Problem?

32
**Levenberg-Marquardt Method**

In general, we want to minimize a sum of squared function values Any Problem? Quadratic function Add regularization term!

33
**Levenberg-Marquardt Method**

In general, we want to minimize a sum of squared function values Any Problem? Quadratic function Add regularization term!

34
**Levenberg-Marquardt Method**

Initialize k=0, choose x0 While k<kmax

35
Stopping Criteria Criterion 1: reach the number of iteration specified by the user K>kmax

36
Stopping Criteria Criterion 1: reach the number of iteration specified by the user Criterion 2: when the current function value is smaller than a user-specified threshold K>kmax F(xk)<σuser

37
Stopping Criteria Criterion 1: reach the number of iteration specified by the user Criterion 2: when the current function value is smaller than a user-specified threshold Criterion 3: when the change of function value is smaller than a user specified threshold K>kmax F(xk)<σuser ||F(xk)-F(xk-1)||<εuser

38
**Levmar Library Implementation of the Levenberg-Marquardt algorithm**

39
**Constrained Nonlinear Optimization**

Finding the minimum value while satisfying some constraints

Similar presentations

© 2020 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google