Optimization in Engineering: Applications and Case Studies

Every solver in this book expects the same three ingredients, and supplying them is the real work of applied optimization. Decision variables are the quantities you control — the knobs to be set. The objective function is the single scalar you want to make as good as possible. The constraints are the rules the solution must obey: physical laws, resource limits, safety margins, logical requirements. Naming these three, precisely, converts an engineering problem into a mathematical one.

The skill is deciding what belongs where, and it is genuinely hard. A quantity you first think of as fixed may actually be a decision; a goal you first write as a constraint ("efficiency at least 90%") may be better as an objective ("maximise efficiency") or vice versa; and the choice of variables silently fixes which methods are even available. Good formulation is iterative — model, solve, look at the answer, discover it is absurd, and find the missing constraint that makes it absurd.

With the model written, its structure dictates the method — and using the wrong one is the most common and most expensive mistake in applied optimization. A convex problem thrown at a genetic algorithm wastes thousands of evaluations reaching an answer that KKT would give in closed form; a rugged black-box handed to a gradient method converges confidently to the nearest bad local optimum. The map below is the course in one table.

A power utility runs several generators, each with a different fuel-cost curve, and must together meet the system demand \(D\) at every instant. Economic load dispatch asks how to split the demand among them at least total cost. It is the daily bread of power-system operation, and it is a constrained nonlinear program straight out of Part 5.

Each generator's cost is well modelled as a quadratic in its output \(P_i\): \(C_i(P_i) = a_i + b_i P_i + c_i P_i^2\). The formulation is then compact — minimise total cost subject to meeting demand and respecting each unit's capacity:

Ignore the capacity limits for a moment and attach a Lagrange multiplier \(\lambda\) to the demand equality. Stationarity of the Lagrangian gives \(\partial C_i/\partial P_i = \lambda\) for every generator — the celebrated equal incremental cost criterion. At the optimum, every running unit operates where its marginal cost equals a common system value \(\lambda\); if one unit's marginal cost were lower than another's, shifting output toward it would cut total cost, so optimality demands they be equal.

The capacity limits are handled by the KKT conditions of Chapter 21. A unit whose unconstrained solution exceeds \(P_i^{\max}\) is simply fixed at that bound — its limit constraint becomes active, carrying its own multiplier — and the remaining demand is redispatched among the rest. Examples 1 and 2 work both cases numerically, and the answer confirms the physical intuition: the cheapest units at the margin are loaded hardest, up to their limits.

An investor allocates capital across assets with uncertain returns. Harry Markowitz's insight — worth a Nobel Prize — was that the right objective is not return alone but the trade-off between expected return and risk, with risk measured as the variance of the portfolio. It is simultaneously a quadratic program (Chapter 22) and a multi-objective problem (Chapter 27), and it shows those two threads of the course meeting.

Let \(\mathbf{w}\) be the fraction of capital in each asset, \(\boldsymbol{\mu}\) the expected returns, and \(\Sigma\) the covariance matrix. Portfolio return is \(\boldsymbol{\mu}^{\mathsf T}\mathbf{w}\) and portfolio variance is the quadratic form \(\mathbf{w}^{\mathsf T}\Sigma\mathbf{w}\). Minimising risk for a required return is a QP:

Sweeping the required return \(R\) traces the efficient frontier — the Pareto front of this two-objective problem, each point a portfolio that gives the least risk for its level of return. The frontier's most celebrated feature is diversification: because assets are imperfectly correlated, a blend can have lower variance than any single asset in it, as Example 3 shows numerically. Splitting risk across uncorrelated bets is, mathematically, the same convexity that makes the frontier bulge toward the ideal corner of low-risk, high-return.

Many engineering choices are not "how much" but "whether": build this substation or not, select this project or not, open this warehouse or not. Such yes/no decisions are binary variables, and the resulting models are integer programs from Part 4 — where, as Chapter 16 warned, rounding a continuous solution is not a reliable shortcut.

Capital budgeting is the archetype. Given a set of projects, each with a cost and a net present value, and a fixed budget, choose the subset that maximises total NPV without exceeding the budget. With \(x_j \in \{0,1\}\) indicating whether project \(j\) is chosen, this is exactly the \(0\)–\(1\) knapsack problem:

It is tempting to rank projects by their NPV-per-cost ratio and greedily fill the budget — the continuous (LP-relaxation) instinct. Example 4 shows it failing: the greedy pick leaves a better combination on the table, because a slightly less efficient project can fit the budget with a partner that the greedy leader crowds out. Recovering the true optimum needs the systematic search of branch and bound, which is precisely why integer programming is its own discipline and not a footnote to linear programming.

The three case studies so far had exploitable structure — quadratic costs, convex variance, linear constraints. Some engineering problems have none. When the objective is computed by a simulation — a finite-element stress solver, a circuit simulator, a fluid-dynamics run — there is no formula, no gradient, and often a rugged, multi-modal landscape riddled with local optima. This is the native territory of the metaheuristics of Part 6.

Two everyday examples make it concrete. Tuning a PID controller means choosing three gains to minimise a performance index (overshoot, settling time, steady-state error) that can only be measured by running the closed loop — a black box in three variables, routinely tuned by PSO or a GA. Designing a truss means choosing member sizes to minimise weight subject to stress and deflection limits computed by a structural solver; with discrete catalogue sizes and a nonlinear analysis, metaheuristics or specialised structural-optimization methods take over. In each, the engineer trades the guarantee of optimality for the ability to optimize something that has no equation at all — a trade worth making only when the structure genuinely is not there.

A handful of mistakes account for most failed optimization projects, and none of them are about the solver.

Step back, and the whole course tells one story. It opened with a single idea — choose the decision variables that make an objective as good as possible while respecting the constraints — and everything since has been that idea meeting a new complication. Linearity gave us the simplex method and duality; the shadow prices of duality became the multipliers of Lagrange, then the KKT conditions of nonlinear programming, then the \(\lambda\) of economic dispatch. Integrality forced branch and bound; multiple objectives forced the Pareto front; ruggedness forced the metaheuristics; randomness forced recourse and the Bellman equation; and an adversary forced the minimax theorem — which turned out to be duality wearing a different hat.

That recurrence is the deepest lesson of the book. These are not thirty separate techniques but one principle refracted through thirty kinds of difficulty, and the mark of a skilled engineer is seeing which refraction a given problem calls for. The methods will keep evolving — better solvers, learned heuristics, new hybrids — but the discipline of stating clearly what you want, what you can control, and what you must respect, and then choosing the least powerful tool that fits, will not. That discipline is what this course has really been about. Go build something optimal with it.

These problems ask you to do the two things this chapter is about: formulate, and choose a method. For each, state the variables, objective and constraints explicitly, and name the technique before solving. Difficulty rises down the list.