Iterative Learning Control

Iterative learning control (ILC) is a control technique that is useful when you want to improve the performance of systems that execute repeated operations, starting at the same initial operating condition. This method uses frequent measurements in the form of the error trajectory from the previous batch run to update the control signal for the subsequent batch run. Applications of ILC include many practical industrial systems in manufacturing, robotics, and chemical processing, where mass production on an assembly line entails repetition. Therefore, use ILC when you have a repetitive task or repetitive disturbances and want to use knowledge from previous iteration to improve next iteration.

A general ILC update law is of the following form ([1] and [2]):

$u_{k+1}(t)=Q(q)[u_k(t)+L(q)e_k(t+1)].$

Here, Q is a low-pass filter that removes control chatter and L is a learning function. k represents the kth iteration. There are multiple ways to design the learning function.

To implement ILC in Simulink^®, use the Iterative Learning Control block. Using this block, you can implement model-free or model-based ILC.

Model-Free ILC

Model-free ILC does not require prior knowledge of the system dynamics and uses proportional-derivative (PD) error feedback to update the control history. This method is applicable only to SISO systems.

The model-free ILC update law is:

$$u_{k+1}(t)=Q(q)\left(u_k(t)+{\gamma }_p{e}_k(t+1)+{\gamma }_d(e_k(t+1)-e_k(t))\right).$$

Here, γ_p and γ_d are the proportion and derivative gains, respectively. Depending on the choice of gains, you can implement P-type, D-type, or PD-type ILC. The gains γ_p and γ_d determine how well ILC learns between iterations. If ILC gains are too big, it might make the closed-loop system unstable (robustness). If ILC gains are too small, it might lead to slower convergence (performance). With properly tuned ILC gains, you can achieve the nominal asymptotic convergence of the tracking error to zero.

Model-Based ILC

This method is a more general form of ILC and is applicable to SISO and MIMO LTI state-space systems of the following form:

$$\begin{array}{l}\begin{array}{cc}{x}_k(t+1)=A{x}_k(t)+B{u}_k(t),& x(0)=x_0,\end{array}\\ y_k(t)=C{x}_k(t).\end{array}$$

In this method, you design a learning function L based on the plant input-output matrix G. From the plant dynamics, you can define the input-output relationship Y_k = GU_k + d. Here:

$$\begin{array}{l}\begin{array}{cc}{Y}_k=\left[\begin{array}{c}y(t+1)\\ y(t+2)\\ ⋮\\ y(t+N)\end{array}\right],& U_k=\left[\begin{array}{c}u(t)\\ u(t+2)\\ ⋮\\ u(t+N-1)\end{array}\right]\end{array}\\ G=\left[\begin{array}{cccc}CB& 0& \cdots & 0\\ CAB& CB& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ C{A}^{N-1}B& C{A}^{N-2}B& \cdots & CB\end{array}\right],\\ d=\left[\begin{array}{cccc}CA{x}_0& C{A}^2{x}_0& \cdots & C{A}^N{x}_0\end{array}\right].\end{array}$$

Additionally, you can write the error dynamics as:

$$e_{k+1}=(I-GL)e_k.$$

The Iterative Learning Control block provides two types of model-based ILC: gradient based and inverse-model based.

Gradient-Based ILC Law

The gradient-based ILC uses the transpose of input-output matrix in the learning function L = γG^T. Therefore, the ILC control law becomes:

$u_{k+1}(t)=Q(q)[u_k(t)+\gamma G^T{e}_k(t+1)]$

The error dynamics are:

$$e_{k+1}=(I-\gamma G{G}^T)e_k$$

Here, γ is the ILC gain. The error convergence depends on the choice of gain. This method guarantees error convergence with |1 – γGG^T| < 1. Therefore, for fastest convergence, you can specify γ = 1/|G²|.

Inverse-Model-Based ILC Law

The inverse-model-based ILC uses the inverse of input-output matrix in the learning function L = γG^-1. Therefore, the ILC control law becomes:

$u_{k+1}(t)=Q(q)[u_k(t)+\gamma G^{-1}{e}_k(t+1)].$

When G matrix is not square, the block uses a pseudoinverse instead.

The error dynamics are:

$$e_{k+1}=(I-\gamma G{G}^{-1})e_k$$

This method guarantees error convergence with |1 – γ| < 1. Therefore, specify a positive scalar less than 2 to achieve convergence.

References

[1] Bristow, Douglas A., Marina Tharayil, and Andrew G. Alleyne. “A Survey of Iterative Learning Control.” IEEE Control Systems 26, no. 3 (June 2006): 96–114. https://doi.org/10.1109/MCS.2006.1636313.

[2] Gunnarsson, Svante, and Mikael Norrlöf. A Short Introduction to Iterative Learning Control. Linköping University Electronic Press, 1997.

[3] Hätönen, J., T.J. Harte, D.H. Owens, J. Ratcliffe, P. Lewin, and E. Rogers. “Discrete-Time Arimoto ILC-Algorithm Revisited.” IFAC Proceedings Volumes 37, no. 12 (August 2004): 541–46.

[4] Lee, Jay H., Kwang S. Lee, and Won C. Kim. “Model-Based Iterative Learning Control with a Quadratic Criterion for Time-Varying Linear Systems.” Automatica 36, no. 5 (May 1, 2000): 641–57.

[5] Harte, T. J., J. Hätönen, and D. H. Owens *. “Discrete-Time Inverse Model-Based Iterative Learning Control: Stability, Monotonicity and Robustness.” International Journal of Control 78, no. 8 (May 20, 2005): 577–86.

[6] Zhang, Yueqing, Bing Chu, and Zhan Shu. “A Preliminary Study on the Relationship Between Iterative Learning Control and Reinforcement Learning⁎.” IFAC-PapersOnLine, 13th IFAC Workshop on Adaptive and Learning Control Systems ALCOS 2019, 52, no. 29 (January 1, 2019): 314–19. https://doi.org/10.1016/j.ifacol.2019.12.669.