Single-Variable Classical Optimization

Key Definitions

• Stationary point: Where \( f'(x) = 0 \) (necessary condition for an extremum).
• Local minimum: \( f(x^*) \leq f(x) \) in some neighborhood around \( x^* \).
• Global minimum: \( f(x^*) \leq f(x) \) for all \( x \) in the domain.

Types of Extrema in Single-Variable Functions

Consider the one-sided derivative at a point \( x^* \):

\[ \lim_{h \to 0} \frac{f(x^* + h) - f(x^*)}{h} = m^+ \text{ (positive) or } m^- \text{ (negative)} \]

• Derivative undefined at \( x^* \): Possible extremum or cusp.
• Stationary (inflection) point: Where \( f'(x^*) = 0 \) but not an extremum.

First-Order Necessary Condition

If \( f \) is differentiable at \( x^* \) and \( x^* \) is an extremum, then \( f'(x^*) = 0 \).

Second-Order Sufficient Condition

If \( f'(x^*) = 0 \) and:

• \( f''(x^*) > 0 \): Local minimum.
• \( f''(x^*) < 0 \): Local maximum.
• \( f''(x^*) = 0 \): Test fails (use higher-order derivatives).

Theorem: General Sufficient Condition

Let \( f'(x^*) = f''(x^*) = \cdots = f^{(n-1)}(x^*) = 0 \), but \( f^{(n)}(x^*) \neq 0 \). Then:

• If \( n \) is even and \( f^{(n)}(x^*) > 0 \): Relative minimum.
• If \( n \) is even and \( f^{(n)}(x^*) < 0 \): Relative maximum.
• If \( n \) is odd: Inflection point (no extremum).

Practical Implications

• Second-order test: Sufficient for most engineering problems.
• Higher-order tests: Needed for degenerate cases where lower derivatives vanish.

Example

Find and classify all extrema of \( f(x) = 12x^5 - 45x^4 + 40x^3 + 5 \).