Maximum Likelihood Estimation
Introduction
Theory
Load Packages
Generate Data Set of Random Numbers
Maximizing the Likelihood Function
Histogram
The likelihood function describes how closely a probability distribution describes a data set. This application will demonstrate how to:
Generate a data set according to a Weibull distribution with a specified scale and shape parameter.
Backsolve the scale and shape parameters for this data set by maximizing the likelihood function using the Global Optimization Toolbox.
Compare the resulting probability distribution against a histogram of the data set.
The Weibull probability distribution has the following density function:
Pt | α, β = β⁢tβ−1⁢ⅇ−tαβαβ,
where α and β are the scale and shape parameters. The likelihood that a data set fits this distribution is:
∏i=1nPti | α, β
Taking the log of this gives:
ln∏i=1nPti | α, β
Hence we derive the following expression describing the likelihood function:
∑i=1nlnPti | α, β
restart:
withStatistics:withGlobalOptimization:withplots:
Number of random numbers to generate
n≔10000:
Define the desired scale and shape parameters for the data set.
α≔73:β≔17:
Generate a sample of random numbers according to a Weibull distribution
Data:=SampleRandomVariableWeibull⁡α,β,n:
The probability density of a Weibull distribution
P≔t,α, β→β⁢tβ−1⁢ⅇ−tαβαβ:
The likelihood function
maxLike:=α,β→addlnPDatai,α,β,i=1..n:
Find the values of α and β that maximize the likelihood function with the Global Optimization Toolbox
results:=GlobalSolve⁡maxLike⁡alpha_ML,beta_ML,alpha_ML=1..100,beta_ML=1..100,maximize2;
results:=beta_ML=17.1301969320734,alpha_ML=72.9747976582517
α__GOT≔rhsselecthas,results,alpha_ML1; β__GOT≔rhsselect⁡has,results,beta_ML1;
α__GOT:=72.9747976582517
β__GOT:=17.1301969320734
These values match those used to generate the data set.
You can also match moments to find the shape and scale parameters
fsolveMomentData,1=MomentWeibulla,b,1,MomentData,2=MomentWeibulla,b,2
a=72.98206445,b=17.00542373
plotopts:=axesfont=Calibri,labelfont=Calibri,labels=t,Probability Density,labeldirections=horizontal,vertical:p1≔HistogramData,color=Gainsboro,plotopts:p2:=plot⁡Pt,α,β,t=sort⁡Data1..sort⁡Datan,color=Black,legend=Sample Parameters,plotopts:p3:=plot⁡P⁡t,α__GOT,β__GOT,t=sort⁡Data1..sort⁡Datan,color=Red,legend=Estimated Parameters,plotopts:display⁡p1,p2,p3,size=800,golden
In the above plot, the red line corresponds to the estimated parameters based on the sample population and the black line corresponds to the desired sample parameters.
Download Help Document