Chapter 4: Partial Differentiation
Section 4.8: Unconstrained Optimization
Example 4.8.10
Choose a and b so that the line y=a x+b minimizes S=∑k=1na xk+b−yk2, the sum of squares of the deviations from the points 1,5.3,3,8.8,5,12.5,6,15.4 to the line. Such a line is called the "least-squares" line.
Solution
Mathematical Solution
Figure 4.8.10(a) shows the least-squares line in red, and the four given data points in green.
The deviation of the data point xk,yk from the graph of the function fx is measured vertically, and is given by the difference fxk−yk. The sum of the squares of deviations of the given data points from the line y=a x+b is given by
use plots in module() local p1,p2,p3: p1:=pointplot([1,3,5,6],[5.3,8.8,12.5,15.4],symbol=solidcircle,symbolsize=15,color=green); p2:=plot(1.972881356*x+3.101694915,x=0..7,color=red); p3:=display(p1,p2,scaling=constrained); print(p3); end module: end use:
Figure 4.8.10(a) Data and least-squares line
S=∑k=14a xk+b−yk2 = 71.a2+30.ab−373.2a+4.b2−84.0 b+498.94
To minimize S, solve the equations ∂S∂a=∂S∂b=0 for a=1.972881356 and b=3.101694915.
The least-squares line is then y=1.972881356x+3.101694915.
Maple Solution - Interactive
In Table 4.8.10(a), the required least-squares line is constructed from first principles. The first step is to decide an appropriate form for the data, either a list of abscissas and ordinates, or a list of data points.
Define lists of abscissas and ordinates
Context Panel: Assign Name
X=1,3,5,6→assign
Y=5.3,8.8,12.5,15.4→assign
Define S, the sum of squares of deviations
Expression palette: Summation template
S=∑k=14a Xk+b−Yk2→assign
Minimize S
Calculus palette: Partial-differentiation template
Context Panel: Solve≻Solve
Context Panel: Assign to a Name≻Q
∂∂ a S=0,∂∂ b S=0
142a+30b−373.2=0,30a+8b−84.0=0
→solve
a=1.972881356,b=3.101694915
→assign to a name
Q
Obtain the least-squares line y=a x+b
Expression palette: Evaluation template Press the Enter key.
y=a x+bx=a|f(x)Q
y=1.972881356x+3.101694915
Table 4.8.10(a) Construction of least-squares line from first principles.
Table 4.8.10(b) illustrates how the Context Panel provides for the immediate construction of the least-squares line. However, the data must appear as a list-of-lists for this to work.
Write the generic data point. Context Panel: Sequence≻k (from 1 to 4)
Context Panel: Conversions≻To List
Context Panel: Curve Fitting≻Least Squares
Xk,Yk→sequence w.r.t. k1,5.3,3,8.8,5,12.5,6,15.4→to list1,5.3,3,8.8,5,12.5,6,15.4→least squares curve fit3.10169491525423+1.97288135593220x
Table 4.8.10(b) Least-squares line via the Context Panel
There are several other tools in Maple for obtaining a least-squares line. Table 4.8.10(c) illustrates the Assistant, accessed through the Context Panel launched on a sequence of the lists of abscissas and ordinates for the data points. Figure 4.8.10(b) shows the state of the Assistant after the Plot button in the Least-Squares section, and the Default button to the right of "Vertical range" have been clicked. The expression for the least-squares line appears under the graph, and either this expression or the graph can be chosen as the return when the "Done" button is clicked.
The Curve-Fitting Assistant is no longer listed in Tools≻Assistants. It is now found in Tools≻Tutors≻Statistics≻Curve Fitting.
Launch the Curve-Fitting Assistant from the Context Panel
Context Panel: Evaluate and Display Inline
Context Panel: Curve Fitting≻Interactive Curve Fitting
X,Y = 1,3,5,6,5.3,8.8,12.5,15.4
Figure 4.8.10(b) Least-squares line by Curve-Fitting Assistant
Table 4.8.10(c) Least-squares line via the Curve-Fitting Assistant
Table 4.8.10(d) illustrates the task template. In addition to returning the least-squares line and the minimum value of S, the sum of squares of deviations, the template draws a graph of the data points and the least-squares line annotated with small squares near the data points. The area of a square is in proportion to the square of the deviation at that point.
Tools≻Tasks≻Browse: Linear Algebra≻Visualizations≻Least Squares Plot
Least Squares Plot
Data Entry
Data can be entered in one of the following formats:
List of lists (either 2-D or 3-D)
Two lists of coordinates (for 2-D)
Three lists of coordinates (for 3-D)
Fitting Function
If the fitting function is linear (line in 2-D or plane in 3-D), a function need not be entered.
If a formula for a curve or surface (linear in the unknown parameters) is optionally entered, variables must be explicitly declared, below.
Line or PlaneOther
Variables
The default line will use the variables x,y.
The default plane will use the variables x,y,z.
Optionally, these defaults can be overridden.
If a fitting function has been given explicitly, variables must be declared as a sequence or list.
Click the icon below to generate the Least Squares Plot.
Fitting curve: 3.102+1.973*x Least squares error: .7274 Maximum error: .4661
Table 4.8.10(d) The Least-Squares Plot task template
Maple Solution - Coded
Solution from first principles:
Enter the data
Define lists of the abscissas and ordinates.
X,Y≔1,3,5,6,5.3,8.8,12.5,15.4:
Obtain S, the sum of squares of the deviations
Apply the sum command.
S≔suma Xk+b−Yk2,k=1..4:
Use the diff command to form equations and the solve command to solve them for a and b.
Q≔solvediffS,a=0,diffS,b=0,a,b:
Insert the optimal values of a and b into the equation y=a x+b
Use the eval command.
evaly=a x+b,Q
To obtain just the least-squares line:
Apply the LeastSquares command from the CurveFitting package.
CurveFitting:-LeastSquaresX,Y,x
3.10169491525423+1.97288135593220x
To obtain a figure like the one in Table 4.8.10(d), use the LeastSquaresPlot command from the Student LinearAlgebra package. See Table 4.8.10(e).
Student:-LinearAlgebra:-LeastSquaresPlotX,Y,x,boxoptions=color=magenta,pointoptions=symbol=solidcircle,symbolsize=15
Table 4.8.10(e) Least-squares line via the LeastSquaresPlot command
<< Previous Example Section 4.8 Next Example >>
© Maplesoft, a division of Waterloo Maple Inc., 2024. All rights reserved. This product is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation.
For more information on Maplesoft products and services, visit www.maplesoft.com
Download Help Document
