Chapter 4: Partial Differentiation
Section 4.8: Unconstrained Optimization
Essentials
Critical points for a function of several variables are points where the function or at least one of its first-partial derivatives is undefined, or where the first-partial derivatives vanish simultaneously.
Points where the first-partial derivatives of f vanish simultaneously are found as solutions of the equations contained in ∇f=0.
Table 4.8.1 contains a statement of the Second-Derivative test for a critical point P that is a solution of ∇fx,y=0.
TP=fxxfyy−fxy2x=a|f(x)P
TP>0 and {fxxP>0⇒fP local minimumfxxP<0⇒fP local maximum
TP<0⇒P is a saddle point
TP=0⇒test fails and no conclusion can be drawn
Table 4.8.1 Second-Derivative test for fx,y
A point P is a saddle point if every neighborhood of P contains points at which f>fP and points at which f<fP. In other words, the tangent plane at P intersects the surface at P. Such a point is a stationary point, but not an extreme point.
The Second-Derivative test stated in Table 4.8.1 is a special case of a more general test that extends to functions of more than two variables. This generalization, stated in Table 4.8.2, is based on the Routh-Hurwitz criterion for quadratic forms. (Some authors, including this one, see the test as based on Sylvester's Law of Inertia.)
HP is the Hessian for fx1,…,xn evaluated at P
S is the sequence 1,Q1,…,Qn, where Qk≠0 is the kth principal minor of HP
Signs of S strictly alternate ⇒P is a local maximum
Signs of S are all the same ⇒P is a local minimum
Signs of S neither alternate nor are all the same ⇒P stationary, but not extreme
At least one Qk=0 ⇒test fails and no conclusion can be drawn
Table 4.8.2 Generalized Second-Derivative test
The principal minor Qk is the determinant of the k×k submatrix of HP whose main diagonal coincides with the main diagonal of HP, which starts with the 1,1-element of HP, and whose last row and column are the kth row and column of HP.
Table 4.8.3 provides some insight into the generalized Second-Derivative test stated in Table 4.8.2.
In any form of the Second-Derivative test, the tested function is expanded in a Taylor series about the point P. Since the first-derivative terms for a function of several variables are zero, the function near P is approximately fP+12XTHP X, where X=R−P, R is the position vector to the general point, and XT is a row vector, the transpose of X.
The values of f near P then depend on whether the quadratic form that comprises the second-derivative terms is always positive, always negative, or sometimes positive and sometimes negative for R near P. If the symmetric matrix HP is positive definite, the quadratic form is always positive near P. If it is negative definite, the form is always negative near P. If it is both positive and negative near P, the form is indefinite. Thus, the Routh-Hurwitz and Sylvester criteria essentially determine if the quadratic form associated with the Hessian is positive definite, negative definite, or indefinite.
If the quadratic form is positive definite, then it adds to the value of fP, so fP is a local minimum.
If the quadratic form is negative definite, then it subtracts from the value of fP, so fP is a local maximum.
If the quadratic form is indefinite, then is both adds to and subtracts from the value of fP, so fP is a stationary, but not an extreme, point.
Table 4.8.3 Remarks on the theoretical framework for the Second-Derivative test
Examples
Example 4.8.1
Find and classify the critical (i.e., stationary) points for fx,y=7−3 x2−5 y2+6 x−9 y+4.
Example 4.8.2
Find and classify the critical (i.e., stationary) points for fx,y=x y−x2−y2−2 x−2 y+4.
Example 4.8.3
Find and classify the critical (i.e., stationary) points for fx,y=x y+2 x−3 y+1.
Example 4.8.4
Find and classify the critical (i.e., stationary) points for fx,y=x3−y3−3 x y+4.
Example 4.8.5
Find and classify the critical (i.e., stationary) points for fx,y=2 x y−3 x4−5 y4+2−y+7.
Example 4.8.6
Find and classify the critical (i.e., stationary) points for x2−2⁢x+8+y2−4⁢y+z2+2⁢z.
Example 4.8.7
Line L1 passes through the point 1,2,3 and has direction V1=2 i−5 j+4 k. Line L2 passes through the point 3,−1,2 and has direction V2=3 i+7 j−6 k. Show that the lines are skew, and find the minimum distance between them. Hint: Parametrize each line with a different parameter and minimize the square of the distance between an arbitrary point on each line.
Example 4.8.8
Find the minimum distance between the point 1,−2,3 and the plane 5 x+3 y+2 z=7.
Example 4.8.9
If both U=3 i−5 j+7 k and V=6 i+j−13 k are bound to the origin, project U onto V. Hint: Find the minimum distance from the tip of U to the line along V.
Example 4.8.10
Choose a and b so that the line y=a x+b minimizes S=∑k=1na xk+b−yk2, the sum of squares of the deviations from the points 1,5.3,3,8.8,5,12.5,6,15.4 to the line. Such a line is called the "least-squares" line.
Example 4.8.11
Obtain formulas for a and b so that the line y=a x+b minimizes S=∑k=1na xk+b−yk2, the sum of squares of the deviations from the points xk,yk,k=1,…,n, to the line.
Example 4.8.12
By minimizing the sum of squares of deviations S=∑k=15fxk−yk2 between fx=a x2+b x+c and the points 1,3,2,8,3,15,4,30,5,38, obtain the best least-squares quadratic fit to the data.
<< Previous Section Table of Contents Next Section >>
© Maplesoft, a division of Waterloo Maple Inc., 2024. All rights reserved. This product is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation.
For more information on Maplesoft products and services, visit www.maplesoft.com
Download Help Document