Maximum Likelihood Estimation in R: Understanding with Normal and Exponential Distributions

Name

Rajiv Chandra

Published on 4/30/2023

(opens in a new tab) (opens in a new tab) (opens in a new tab) (opens in a new tab) (opens in a new tab) (opens in a new tab)

In statistical theory, one of the most ubiquitous estimation methods is the Maximum Likelihood Estimation (MLE). It is a powerful tool, frequently used in fields as diverse as machine learning, econometrics, and biostatistics. This article will walk you through how to perform MLE in R, focusing on the Normal and Exponential distributions. We will use the stats4 package in R, particularly the mle function.

Want to quickly create Data Visualisation from Python Pandas Dataframe with No code?

PyGWalker is a Python library for Exploratory Data Analysis with Visualization. PyGWalker (opens in a new tab) can simplify your Jupyter Notebook data analysis and data visualization workflow, by turning your pandas dataframe (and polars dataframe) into a Tableau-style User Interface for visual exploration.

(opens in a new tab)

Introduction to Maximum Likelihood Estimation

Maximum Likelihood Estimation is a method used to estimate the parameters of a statistical model. Given a set of data and a statistical model, MLE finds the parameters that maximize the likelihood of observing the given data.

Maximum Likelihood Estimation for Normal Distribution

The normal distribution, also known as Gaussian distribution, is widely used due to its mathematical properties. Let's look at how we can perform MLE for the normal distribution.

Theory

The normal distribution is defined by two parameters: mean (µ) and standard deviation (σ). The goal of MLE is to find the µ and σ that maximize the likelihood of our observed data.

R Maximum Likelihood Estimation

Let's illustrate this with a code snippet in R. We will use the mle function from the stats4 package.

# loading the required package
library(stats4)

# defining the likelihood function for a normal distribution
ll <- function(mu, sigma) {
    -sum(dnorm(data, mean = mu, sd = sigma, log = TRUE))
}

# performing MLE
data = rnorm(100, 5, 2) # generates 100 observations from a normal distribution with µ=5, σ=2
fit = mle(ll, start = list(mu = mean(data), sigma = sd(data)))

# displaying the results
summary(fit)

The output will provide you the MLEs for µ and σ.

Maximum Likelihood Estimation for Exponential Distribution

Moving onto the exponential distribution, this distribution is primarily used to model the time between the occurrence of events in a Poisson point process.

Theory

The exponential distribution is defined by one parameter: the rate (λ). The objective of MLE in the exponential distribution is to find the λ that maximizes the likelihood of our observed data.

Maximum Likelihood Estimation R

Let's illustrate this with another code snippet.

# defining the likelihood function for an exponential distribution
ll_exponential <- function(lambda) {
    -sum(dexp(data_exp, rate = lambda, log = TRUE))
}

# performing MLE
data_exp = rexp(100, 0.5) # generates 100 observations from an exponential distribution with λ=0.5
fit_exp = mle(ll_exponential, start = list(lambda = 1/mean(data_exp)))

# displaying the results
summary(fit_exp)

The output will provide you the MLE for λ.

Conclusion

MLE is a powerful estimation technique in the realm of statistical modeling. This article guided you through how to perform maximum likelihood estimation in R for normal and exponential distributions. Remember, MLE is not restricted to these distributions.