Statistics Estimation
Estimation is the process of determining a numerical value for one or more parameters of a population from a set of data samples. The Statistics package provides a framework for performing maximum likelihood estimation and other techniques used in statistical estimation.
1 Maximum Likelihood Estimation
Maximum likelihood estimation applies the concept of the likelihood function - an equation that considers the probability of a set of recorded observations given one or more unknown parameters. The maximum value that this function attains when mapped over the unknown parameter is of particular importance, as it indicates the "most likely" value for that parameter.
The following is an example problem that demonstrates a method for finding the maximum likelihood estimate of the scale parameter c in a Pareto distribution.
restartwithStatistics:
Generate a sample from the Pareto distribution that will act as the provided data.
S:=SamplePareto5,7,1000:
Compute the likelihood function of a Pareto distribution with unknown scale parameter, assuming only one sample has been provided.
LikelihoodPareto5,c,A,samplesize=1
{0A1<5c5cA1−c−15≤A1
Compute the log likelihood function, assuming four samples have been provided.
LogLikelihoodPareto5,c,A,samplesize=4
4ln5c+lncA1−c−1+lncA2−c−1+lncA3−c−1+lncA4−c−1
Compute the score (derivative of the log likelihood function), under the same considerations.
ScorePareto5,c,A,samplesize=4
4ln5+4c−lnA1−lnA2−lnA3−lnA4
Compute the score, instead applying the generated sample data.
F:=ScorePareto5,c,S
F:=1000ln5−1744.554810+1000c
Solve for c, such that the score is equal to zero.
fsolveF=0
7.400998824
Calculate the maximum likelihood estimate (this should give us the same result).
M:=MaximumLikelihoodEstimatePareto5,c,S
M:=7.400998143
Compute the likelihood ratio statistic.
L:=LikelihoodRatioStatisticPareto5,c,S
L:=2003.229898−2000.lnc+270.233800c
The likelihood ratio statistic is generally distributed as a ChiSquare random variable with 1 degree of freedom. Using this and the fact that the confidence interval bounds the maximum likelihood estimate, we can determine an approximate 95% confidence interval for the maximum likelihood estimate.
θ:=QuantileChiSquare1,0.95
θ:=3.841456066
CI:=fsolveL=θ,c=−∞..M..fsolveL=θ,c=M..∞
CI:=6.951716117..7.869234199
2 Method of Moments
The method of moments is used to determine the parameters of a probability distribution function by comparing the moments of a sample to the moments of a probability distribution. In order to apply the method of moments, apply the following steps.
1. Calculate the first n moments of the fitting distribution, where n is the number of free indeterminates in the fitting distribution. For example, to perform a fit using the Normal distribution, which has parameters mu and sigma, calculate the first 2 moments.
2. Calculate the first n moments of the data sample.
3. Equate the equations generated by steps 1 and 2 and solve for the indeterminates. Assuming that the equations created in step 3 can be solved, these steps provide you with an estimate to the values of the parameters of the fitting distribution. As an example, consider an estimate of the parameters of a Beta distribution, with parameters ν and ω:
Generate a sample from a Beta distribution to act as the provided data.
S:=Sample'Β'5,7,10000:
1. Calculate the first and second moments of the Beta distribution with free parameters ν and ω.
DistMoments:=Moment'Β'ν,ω,1,Moment'Β'ν,ω,2
DistMoments:=νν+ω,νν+1ν+ων+ω+1
2. Calculate the first and second moments of the provided data.
DataMoments:=MomentS,1,MomentS,2
DataMoments:=0.4164527455,0.1921293371
3. Equate the moments and solve for ν and ω.
solveEquateDistMoments,DataMoments
ν=4.996676373,ω=7.001506919
Return to Index for Example Worksheets
Download Help Document
