Chapter 4: Partial Differentiation
Section 4.8: Unconstrained Optimization
Example 4.8.11
Obtain formulas for a and b so that the line y=a x+b minimizes S=∑k=1na xk+b−yk2, the sum of squares of the deviations from the points xk,yk,k=1,…,n, to the line.
Solution
Mathematical Solution
The optimal values of the parameters in the least-squares line y=a x+b are given in a compact form by the expressions
a=Σ x y−n x&conjugate0; y&conjugate0;Σ x2−n x&conjugate0;2 and b=y&conjugate0; Σ x2− x&conjugate0; Σ x yΣ x2−n x&conjugate0;2
where x&conjugate0;=1n∑k=1nxk, y&conjugate0;=1n ∑k=1nyk, Σ x2=∑k=1nxk2, Σ x y=∑k=1nxk yk.
These results are obtained by differentiating S=∑k=1na xk+b−yk2 with respect to a and b to form the equations
∂S∂a=∑k=1n2⁢a⁢xk2+2⁢b⁢xk−2⁢xk⁢yk=0 and ∂S∂b=2⁢n⁢b+∑k=1n2⁢a⁢xk−2⁢yk=0
which become
n x&conjugate0;nΣ x2n x&conjugate0;ab=n y&conjugate0;Σ x y
An application of Cramer's rule then gives
a=|n y&conjugate0;nΣ x yn x&conjugate0;||n x&conjugate0;nΣ x2n x&conjugate0;| = n2 x&conjugate0; y&conjugate0; − n Σ x yn2 x&conjugate0;2−n Σ x2=Σ x y−n x&conjugate0; y&conjugate0;Σ x2−n x&conjugate0;2
and
b=|n x&conjugate0;n y&conjugate0;Σ x2Σ x y||n x&conjugate0;nΣ x2n x&conjugate0;| = n x&conjugate0; Σ x y−n y&conjugate0; Σ x2n2 x&conjugate0;2−n Σ x2 = y&conjugate0; Σ x2− x&conjugate0; Σ x yΣ x2−n x&conjugate0;2
Maple Solution - Interactive
Define S, the sum of squares of deviations
Expression palette: Summation template
Context Panel: Assign name
S=∑k=1na xk+b−yk2→assign
Form and solve the equations ∂S∂a=∂S∂b=0
Calculus palette: Partial-differentiation operator Press the Enter key.
Context Panel: Solve≻Solve for Variables≻a,b
∂∂ a S=0,∂∂ b S=0
∑k=1n2⁢a⁢xk2+2⁢b⁢xk−2⁢xk⁢yk=0,2⁢n⁢b+∑k=1n2⁢a⁢xk−2⁢yk=0
→solve (specified)
a=n⁢∑k=1nxk⁢yk−∑k=1nxk⁢∑k=1nykn⁢∑k=1nxk2−∑k=1nxk2,b=−∑k=1nxk⁢yk⁢∑k=1nxk−∑k=1nxk2⁢∑k=1nykn⁢∑k=1nxk2−∑k=1nxk2
Maple Solution - Coded
Use the Sum command to obtain S.
S≔Suma xk+b−yk2,k=1..n:
Apply the value command to the result of applying the solve command to the set of equations obtained with the diff command.
valuesolvediffS,a=0,diffS,b=0,a,b
<< Previous Example Section 4.8 Next Example >>
© Maplesoft, a division of Waterloo Maple Inc., 2024. All rights reserved. This product is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation.
For more information on Maplesoft products and services, visit www.maplesoft.com
Download Help Document