Nonlinear Optimal Control

Collaborators: Joel Burdick, Matanya Horowitz, Elis Stefansson

In the study of nonlinear optimal control, the value function that produces an optimal controller is described by the Hamilton-Jacobi-Bellman (HJB) equation, a nonlinear partial differential equation (PDE). Being nonlinear, this PDE is difficult to solve. Nonetheless, under a mild assumption, the associated HJB equation can be transformed into a linear PDE under a log transformation of the value function. The quantity that the linear PDE describes is called the desirability function. Once the PDE is linear, multiple tools in the applied mathematics community can now be used to solve it.

One method that I work on is to relax the HJB PDE and solve it using a sum of squares (SOS) program iteratively. In each iteration, the solution becomes closer to the true optimal solution. This method provides analytical guarantees on the system performance using the controller obtained from solving the relaxed HJB PDE. The following figures show the improved approximation of the desirability function.

Nonlinear Optimal Control Nonlinear Optimal Control Nonlinear Optimal Control Nonlinear Optimal Control

Apart from the SOS based method, I also work on another method using the CANDECOMP/PARAFAC tensor decomposition to represent and solve the high dimensional linear HJB PDE. A drawback of the SOS based method is that it succumbs to the curse of dimensionality. However, this tensor decomposition based method scales linearly with dimension. A MATLAB tool called Sequential Alternating Least Squares (SeALS) was developed together with a SURF student, Elis Stefansson, to implement this technique.

The next step is to find a common ground for both the SeALS and the SOS based technique to solve for the optimal controller in high dimensions. The number of monomials in the SOS technique grows exponentially with dimension; however, the SOS technique provides a performance guarantees that SeALS does not provide.

Another related on going work is to use the tensor decomposition technique for nonlinear estimation. Stay tuned for more.

Software:

  1. E. Stefansson, and Y. P. Leong, “Sequential Alternating Least Squares (SeALS)”, available here with the user’s guide.

Publications:

  1. E. Stefansson, and Y. P. Leong, “Sequential Alternating Least Squares (SeALS),” in IEEE Int. Conf. on Intelligent Robots and Systems (IROS), 2016, pp. 3757–3764. (long version)
  2. Y. P. Leong, M. B. Horowitz and J. W. Burdick, “Linearly solvable stochastic control Lyapunov functions,” SIAM Journal on Control and Optimization, vol. 54, no. 6, pp. 3106–3125, 2016.
  3. Y. P. Leong, M. B. Horowitz, and J. W. Burdick, “Suboptimal stabilizing controllers for linearly solvable system,” in IEEE Int. Conf. on Decision and Control (CDC), 2015, pp. 7157–7164.