Modern Control: Optimal, Robust, and Adaptive

Look back at what the previous twenty-nine chapters have built. Chapters 2 to 5 turned hardware into transfer functions. Chapters 6 to 9 measured what a loop does in time. Chapters 10 to 18 answered the question of whether it stays bounded at all, from the Routh array to the Nyquist criterion to the margins that quantify how much abuse a loop can take. Chapters 19 to 22 used those tools to design: PID actions, tuning rules, lead and lag networks. Chapters 23 to 26 rebuilt the whole subject in state space, where pole placement and observers gave complete freedom over the closed-loop dynamics. Chapters 27 and 28 moved the controller into a computer, and Chapter 29 admitted that real plants are not linear.

It is a powerful toolkit, and it has three holes in it.

The first is that every method requires a judgement it cannot justify. Chapter 26 showed that if a system is controllable, its closed-loop poles can be placed anywhere. Anywhere is too much freedom. Nothing in the pole-placement algorithm tells you where to put them; push them far left and the gains become enormous, the actuator saturates, and the sensor noise is amplified into the control signal. The same is true of the compensator breakpoints of Chapter 21 and the PID gains of Chapter 20 — the rules of thumb work, but they are rules of thumb. And in a plant with three inputs and five outputs there is no single loop to shape at all.

The second hole is that the model is wrong. It was wrong from Chapter 2 onwards: we linearised, we ignored the flexible modes above the frequency band of interest, we took the motor's inductance as negligible, we assumed the load inertia was the value on the datasheet. Gain and phase margin (Chapter 17) are the classical acknowledgement of this — they say how much pure gain error or pure phase lag the loop tolerates. But real errors are neither pure gain nor pure phase, and a system can have \(12\) dB of gain margin and \(50^{\circ}\) of phase margin and still be destabilised by a small simultaneous perturbation of both.

The third hole is that the plant does not hold still. An aircraft's dynamics change by an order of magnitude between sea level and cruise altitude. A robot arm's inertia depends on what it is carrying. A distillation column drifts as its catalyst ages. A controller designed for one operating point is, at another, a controller designed for the wrong plant.

Three holes, three answers. Optimal control replaces the arbitrary choice with a cost to minimise. Robust control replaces the single nominal model with a set of models the controller must handle. Adaptive control lets the controller's own parameters move when the plant's do. This chapter takes each in turn.

Start with the state model of Chapter 23, with the state accessible and no noise for the moment:

We want to drive \(x\) to the origin. Pole placement asks "where shall the poles go?"; optimal control asks instead "what would a good trajectory look like, numerically?" and then finds the input that produces the best one. To make that question answerable we need a number attached to each candidate trajectory — a cost functional.

Two things are bad: the state being away from the origin, and the control effort needed to bring it back. A large tracking error is bad; so is an actuator that slams to its limits. Any sensible cost must therefore contain both, and must weigh one against the other. The quadratic form does exactly this:

Each requirement earns its place. The integrand must be non-negative, or the "minimum" could run to \(-\infty\), which forces \(Q\succeq 0\) and \(R\succeq 0\). \(R\) must be strictly positive definite, not merely semi-definite: if some direction of \(u\) were free of charge, the optimiser would use infinite control in that direction and the problem would have no solution. Stabilizability of \((A,B)\) is needed so that some input gives a finite cost — an uncontrollable unstable mode makes \(J\) infinite whatever we do. Detectability of \((A,Q^{1/2})\) stops the optimiser from ignoring an unstable mode that \(Q\) happens not to penalise; without it, the "optimal" loop can be unstable and still cheap.

The quadratic choice is not arbitrary. It is the only cost that is (i) symmetric in sign — overshooting by \(2\) costs the same as undershooting by \(2\); (ii) smooth, so the calculus works; and (iii) scale-consistent, in that doubling the initial state quadruples the cost, which matches the linearity of the plant. It is also the cost that makes the answer come out linear, and that is the fact everything else in this section rests on.

The minimisation is over an infinite-dimensional set — every possible function \(u(t)\) on \([0,\infty)\). That sounds hopeless, but a single algebraic trick collapses it. Suppose there exists a symmetric matrix \(P\) satisfying

Take any input \(u(t)\) whatsoever, and watch what the quadratic form \(V(x)=x^{T}Px\) does along the resulting trajectory:

Now add the cost integrand to both sides and use the ARE to replace \(A^{T}P+PA\) by \(-Q+PBR^{-1}B^{T}P\). The \(x^{T}Qx\) terms cancel exactly:

The right-hand side is a perfect square. Writing \(K = R^{-1}B^{T}P\), expand \((u+Kx)^{T}R(u+Kx)\) and compare term by term: \(u^{T}Ru\), then \(2u^{T}RKx = 2u^{T}B^{T}Px\), then \(x^{T}K^{T}RKx = x^{T}PBR^{-1}B^{T}Px\). They match. So

Integrate from \(0\) to \(\infty\). The total derivative telescopes to \(x_0^{T}Px_0 - x^{T}(\infty)Px(\infty)\), and any input worth considering drives the state to zero, so the second term vanishes:

Read that last line carefully, because it settles the entire problem in one stroke. The first term does not depend on \(u\) at all — it is fixed by the initial condition. The second term is an integral of a non-negative quantity. The cost is therefore minimised by making that integral zero, which happens if and only if \(u(t) = -Kx(t)\) for all \(t\). There is no need to search over functions: the optimal input is a constant-gain state feedback, and its cost is known before the trajectory is ever computed.

Three consequences deserve emphasis. First, the closed-loop system \(\dot x = (A-BK)x\) is guaranteed stable — this is not something to check afterwards with a Routh array, it follows from the construction, since a finite \(J\) forces \(x\to 0\). Second, \(V(x)=x^{T}Px\) is a Lyapunov function for the closed loop, so the whole apparatus of Chapter 29 applies directly; the Riccati equation is precisely the condition that makes the optimal cost a valid Lyapunov function. Third, \(K\) is a full state feedback of exactly the form Chapter 26 introduced — LQR does not invent a new controller structure, it removes the arbitrariness in choosing the gains.

Over a finite horizon \([0,t_f]\) the same argument goes through with \(P\) time-varying, giving the differential Riccati equation \(-\dot P = A^{T}P+PA-PBR^{-1}B^{T}P+Q\) integrated backwards from a terminal weight \(P(t_f)=S\). The gain \(K(t)=R^{-1}B^{T}P(t)\) is then time-varying, but away from the final instant it settles to the constant value above. The infinite-horizon problem is the practically important one, and it is the only one solved in this chapter.

The Riccati equation removed the arbitrariness from the gains and moved it into \(Q\) and \(R\). That is real progress — there are usually far fewer meaningful entries in \(Q\) and \(R\) than there are gains, and their meaning is physical rather than geometric — but the weights still have to be chosen.

The standard starting point is Bryson's rule: make each weight the reciprocal of the square of the largest acceptable value of the corresponding variable,

This normalises the problem: every term in the integrand is then of order one when its variable sits at its limit, so the cost adds like with like even when the states are an angle in radians and a current in amperes. It is a starting point, not an answer, and the design proceeds by iterating from it.

Beyond that, the useful picture is obtained by writing \(R=\rho R_0\) and sweeping the single scalar \(\rho\). This makes the trade-off one-dimensional.

The expensive-control limit is worth pausing on, because it is exactly what one would want a lazy controller to do: if using the actuator is costly, leave the modes that are already stable entirely alone, and spend effort only on the modes that would otherwise blow up — reflecting each unstable pole \(p\) to \(-p\), the cheapest possible repair.

Now the more remarkable property. Nothing in the derivation mentioned gain margin or phase margin, yet the optimal loop has both, automatically, and the same amounts every time.

The inequality follows from a return-difference identity obtained by substituting the ARE into \((1+L(-s))R(1+L(s))\); the result is \(R\) plus a term that is non-negative on the imaginary axis, so \(|1+L(j\omega)|^{2}\ge 1\). What matters here is not the algebra but the picture, and Chapter 16 already taught us to read it.

Chapter 17 defined the gain margin as the factor by which \(|L|\) may be scaled before the plot reaches \(-1\), and the phase margin as the rotation it tolerates. Both are distances to the critical point, and the inequality says that distance is never less than one. Scale the loop gain up by any factor \(k>1\) and the plot moves outward, away from \(-1\); scale it down and the plot shrinks towards the origin, which is itself at distance \(1\) from the critical point, so no crossing occurs until \(k\) falls to \(\tfrac12\). Rotating the plot about the origin, the phase at which a point of magnitude \(1\) first reaches the disc boundary is \(60^{\circ}\), by the equilateral triangle formed by the origin, the point \(-1\), and the intersection of the unit circles about each.

The regulator of Section 30-3 needs the whole state vector. Chapter 26 solved that with an observer, \(\dot{\hat x} = A\hat x + Bu + L(y-C\hat x)\), and left the same open question: where should the observer poles go? Fast observation tracks the true state quickly but amplifies measurement noise; slow observation is smooth but lags. There is a genuine optimum, and finding it requires saying what the noise actually is.

Here \(w\) is process noise — the wind gust, the load torque ripple, everything the model omits — and \(v\) is measurement noise from the sensor. Both are taken as zero-mean white noise with covariances \(W = E[ww^{T}]\succeq 0\) and \(V=E[vv^{T}]\succ 0\). The estimation problem is to choose \(L\) so that the steady-state error covariance \(E[(x-\hat x)(x-\hat x)^{T}]\) is as small as possible.

The answer is a second Riccati equation — and, remarkably, it is the same equation with the roles of the matrices transposed.

The correspondence is exact, and it is worth tabulating because it means one piece of software and one piece of understanding serve both problems.

The ratio \(W/V\) plays for the filter exactly the role that \(Q/R\) plays for the regulator. Trust the model and distrust the sensor (small \(W\), large \(V\)) and \(L\) becomes small: the filter mostly integrates its own model and ignores the noisy measurement. Distrust the model and trust the sensor and \(L\) grows: the estimate is dragged hard towards \(y\). This is the single most useful intuition about Kalman filtering, and it is why the filter is often tuned in practice by adjusting \(W\) with \(V\) fixed at the sensor's measured noise floor.

Putting the two halves together gives the Linear Quadratic Gaussian controller: estimate the state optimally, then feed it back optimally, as if it were the true state.

That "as if" is licensed by the separation principle of Chapter 26: with \(u=-K\hat x\), the closed-loop eigenvalues are the union of the eigenvalues of \(A-BK\) and those of \(A-LC\). The regulator and the estimator can be designed independently and joined without either spoiling the other's pole locations. Here the principle is stronger still — the optimal output-feedback controller for the quadratic cost with Gaussian noise really is the optimal estimator followed by the optimal regulator, and no cleverer combined design exists.

Robust control begins with a change of object. Everywhere in this book so far, "the plant" has meant one transfer function \(G(s)\). Replace it with a set:

\(G(s)\) is the nominal model. \(\Delta\) is any stable transfer function of size at most one — completely unknown in shape, phase, and order. \(W(s)\) is the uncertainty weight, a known, stable, minimum-phase function whose magnitude \(|W(j\omega)|\) is the relative size of the modelling error at frequency \(\omega\). The design question is no longer "is the closed loop stable?" but "is the closed loop stable for every member of \(\mathcal{G}\)?"

The weight is where physical knowledge enters. Typically \(|W(j\omega)|\) is small at low frequency — a few percent, reflecting parameter tolerance — and rises through unity somewhere near the frequency at which the neglected dynamics live. Above that frequency the model is worthless and \(|W|>1\); the relative error can exceed \(100\%\) because the true phase is unknown. In practice one identifies the plant at several operating points, plots \(|G_p(j\omega)/G(j\omega)-1|\) for each, and fits a low-order \(W\) that covers the envelope.

Now the tool that turns this into a test. Consider any feedback interconnection of a known stable \(M(s)\) with an unknown stable \(\Delta(s)\).

The intuition is the one Chapter 10 used for stability in the first place. A signal injected into the loop is multiplied by \(M\), then by \(\Delta\), then returns. If the product of the two gains is less than one at every frequency, each circuit of the loop shrinks the signal, so it dies away and no self-sustaining oscillation can form. If the product can exceed one, some \(\Delta\) — a suitable phase shift at the offending frequency — closes the loop with a gain of exactly \(-1\), and the system is on the verge of instability. The strength of the theorem is the "for all": it is not conservative merely to be safe, it is exactly the condition, because among all \(\Delta\) of that size one of them will be the worst.

To use it, rearrange the uncertain loop so that \(\Delta\) is pulled out into its own block, and identify what \(\Delta\) sees. With the controller \(C(s)\), nominal loop gain \(L=GC\), and multiplicative uncertainty at the plant output, the transfer function from the output of \(\Delta\) back to its input is \(-WL/(1+L) = -WT\), where \(T\) is the complementary sensitivity of Chapter 18. The small-gain theorem then gives the central result of robust stability.

Two closed-loop transfer functions govern everything, and Chapter 18 has already met them both. For the standard unity-feedback loop with \(L=GC\),

\(S\) is the transfer function from reference to error and from output disturbance to output — small \(|S|\) means good tracking and good disturbance rejection. \(T\) is the transfer function from reference to output and from sensor noise to output — small \(|T|\) means noise rejection and, by Section 30-6, robustness to unmodelled dynamics. And the identity \(S+T=1\) holds at every single frequency, algebraically, for every possible controller. It is not a design limitation that better engineering could remove; it is arithmetic.

So performance and robustness are in direct competition at each frequency, and the design consists of choosing where to spend which. Below the crossover, make \(|S|\) small and accept \(|T|\approx 1\). Above it, make \(|T|\) small and accept \(|S|\approx 1\). The interesting question is what happens in between, and there a second, deeper constraint appears.

Read the stable case first. The integral of \(\ln|S|\) — that is, of the sensitivity in decibels — over all frequencies is exactly zero. Every decibel of sensitivity reduction bought in one frequency band must be paid for, decibel for radian, by sensitivity amplification in another. Disturbance rejection is not created by feedback; it is moved.

This is the waterbed effect, and it explains a great deal of what earlier chapters observed empirically. It is why the integral action of Chapter 19, which drives \(|S|\) to zero at DC, always produces a peak in the closed-loop response somewhere near crossover. It is why raising the gain in Chapter 9 to shrink the steady-state error degraded the transient. And when the plant is open-loop unstable, the right-hand side is strictly positive: an unstable pole imposes a mandatory quota of sensitivity amplification that must be absorbed somewhere. Instability is expensive, and the further right the pole, the higher the bill.

The peak of \(|S|\) has a name and a direct classical meaning. \(M_s = \lVert S\rVert_\infty\) is the reciprocal of the shortest distance from the Nyquist plot of \(L\) to the critical point \(-1\), because \(|S(j\omega)|=1/|1+L(j\omega)|\) and \(|1+L|\) is that distance. So a single number bounds both classical margins at once.

With \(S\) and \(T\) understood, \(H_\infty\) design writes the whole specification as one norm. Choose a performance weight \(W_1(s)\) — large at low frequency, where good rejection is demanded — and the uncertainty weight \(W_2(s)=W(s)\) of Section 30-6. Then:

The last row is the demanding one, and it is achievable only if the two weights overlap little — which is another way of saying that a plant can be controlled tightly only over the band in which it is known accurately. The mixed-sensitivity problem stacks the first two into a single objective,

and a value below \(1\) means both specifications are met. The problem is solved numerically by two Riccati equations of exactly the family met in Sections 30-3 and 30-5, iterated over a scalar \(\gamma\) until the smallest achievable norm is found. What matters for a first course is not that algorithm but the shift in what the engineer supplies: not gains, not pole locations, but frequency-domain templates saying how much error is acceptable where, and how badly the model is known where. Those are questions a plant engineer can actually answer.

Robust control survives uncertainty by being conservative: it designs one fixed controller good enough for the whole model set, and pays for the guarantee with performance at the nominal plant. When the uncertainty is large — an aircraft across its flight envelope, a manipulator across its payload range — that price becomes unacceptable. Adaptive control takes the other route: keep the controller sharp, and change it as the plant changes.

The simplest form is barely more than bookkeeping. Gain scheduling measures an external variable that indexes the operating point — dynamic pressure, tank level, joint angle — designs a controller at each of several such points by any method in this book, and interpolates between them as the variable moves. It is honest, it is overwhelmingly the most used adaptive scheme in industry, and it has one severe limitation: it is open-loop with respect to the adaptation. Nothing checks whether the scheduled controller is actually working. It also relies on the scheduling variable moving slowly compared with the loop dynamics, since the stability of each frozen design says nothing about the stability of the transitions — this is exactly the frozen-parameter fallacy Chapter 29 warned about with linearisation.

The genuine adaptive schemes close that loop. Two structures dominate. In a self-tuning regulator, a recursive least-squares estimator identifies the plant parameters online, and a design equation — pole placement from Chapter 26, or a PID rule from Chapter 20 — recomputes the controller each sample from the current estimates, treating them as if they were correct. That last step is the certainty equivalence assumption. In model-reference adaptive control (MRAC), no explicit plant model is identified at all; instead the controller parameters are driven directly by the mismatch between the plant output and the output of a reference model that encodes the desired behaviour.

The historically first adaptation law is the MIT rule, a gradient descent on the squared model-following error \(J(\theta)=\tfrac12 e^{2}\) with \(e=y-y_m\):

The partial derivative \(\partial e/\partial\theta\) is the sensitivity derivative, and it is the practical difficulty: computing it requires the very plant parameters we do not know. The classical escape is to approximate it using the reference model in place of the plant, which is exact when the parameters have converged and merely approximate on the way there. The rule is intuitive, easy to implement, and — this is the point — carries no stability proof at all. It is a gradient method applied to a dynamic system, and for large \(\gamma\) it can and does diverge.

The repair is to design the adaptation law from a Lyapunov function rather than from a gradient, which is the strategy Chapter 29 established for nonlinear systems generally. Take a first-order plant with both parameters unknown, and a reference model specifying the desired behaviour:

If \(a\) and \(b\) were known, the ideal parameters would follow by matching coefficients: \(b\theta_1^{*}=b_m\) and \(b\theta_2^{*}+a=a_m\). Substituting the control law into the plant and subtracting the model gives the error dynamics in terms of how far the parameters are from those ideals. Writing \(\varphi_1=b\theta_1-b_m\) and \(\varphi_2=b\theta_2+a-a_m\),

Now choose a Lyapunov candidate that charges for both the tracking error and the parameter error (taking \(b>0\) with known sign, which is the standard minimal assumption):

The two bracketed terms are the only ones whose sign is unknown, and both contain a quantity we are free to choose. Cancel them.

Notice what this guarantees and what it does not. It guarantees that the output converges. It does not guarantee that \(\theta_1\to\theta_1^{*}\) and \(\theta_2\to\theta_2^{*}\), because \(\dot V=0\) whenever \(e=0\), regardless of the parameter errors. The parameters converge only if the reference signal is persistently exciting — rich enough in frequency content to distinguish every parameter. A constant setpoint excites one frequency and can identify one combination; a plant with two unknowns fed a constant \(r\) will track perfectly with permanently wrong parameters, and then behave badly the moment the setpoint changes.

That subtlety is the gateway to the practical failure modes of adaptive control, and any engineer deploying one must know them. Without persistent excitation the estimates drift slowly along the directions the data cannot see, until a disturbance or an unmodelled high-frequency mode kicks the loop into a burst of oscillation. Rohrs' celebrated 1982 counterexamples showed that an ideal MRAC, provably stable for the modelled plant, could be destabilised by an entirely reasonable unmodelled fast pole — the adaptive analogue of Doyle's LQG result, and evidence for the same underlying lesson. The standard defences are \(\sigma\)-modification or \(e\)-modification, which add a leakage term to the adaptation law so the parameters decay towards a nominal value in the absence of information; a dead zone that freezes adaptation when the error is within the noise; and projection, which confines \(\theta\) to a set known a priori to contain the truth.

Three questions, three answers. Which controller? The one minimising \(\int (x^{T}Qx+u^{T}Ru)\,dt\), obtained from a Riccati equation, guaranteed stable and — for full state feedback — guaranteed \(60^{\circ}\) of phase margin. How wrong may the model be? As wrong as \(|W(j\omega)|\) allows, provided \(\lVert WT\rVert_\infty<1\); and no controller escapes \(S+T=1\) or Bode's integral, so the design is always an allocation of a fixed budget rather than an elimination of error. What if the plant moves? Then the controller moves too, along an adaptation law built from a Lyapunov function rather than a gradient, converging in output always and in parameters only when the command is rich enough to identify them.

Solve the Riccati equations by hand where they are scalar or \(2\times2\); for the frequency-domain problems, locate the peak of the relevant magnitude before computing anything else. Difficulty rises down the list.