Fitting Distributions

Fitting distributions is the skill that lets you summarize a pile of data with a few numbers, and it is the foundation for everything you will learn next. When you fit a distribution, you are building a simple, compact description of where your data lives and how spread out it is. That description is what lets you make predictions, detect anomalies, and understand uncertainty. This concept unlocks further study in probability and statistics: you will need it for sampling distributions, for understanding how estimates bounce around, for concentration inequalities that bound how far data can stray from expectations, and for Bayesian methods where you update beliefs with data. Concretely, after this you will be able to look at a column of measurements—wait times, incomes, errors—and say 'these follow a pattern, here are the parameters', which is the first step in any serious data analysis or machine learning work.

Think of a distribution as a smooth curve that describes how often different values occur in your data. Some values happen a lot, some rarely. The curve shows that. Fitting a distribution means picking one of these curves—from a family of curves with a named shape—and adjusting its knobs so it matches your data as closely as possible.

Why do you need a curve at all? Because your data is just a list of numbers, but the underlying process that produced those numbers follows a pattern. For example, if you measure the time between buses arriving at a stop, you might see that short waits are common and long waits become rarer. A distribution can capture that shape with just one or two numbers, instead of listing every single wait time.

The knobs on the curve are called parameters. A simple family like the normal distribution has two parameters: the mean (where the curve is centered) and the standard deviation (how spread out it is). Other families have different parameters that control shape in different ways.

Fitting is not guessing—it is a systematic way of choosing the parameters so that the curve fits the data as well as possible. There are different methods for doing this, and the most common is called maximum likelihood. The idea is simple: choose the parameters that make the observed data most likely to have happened. If you have a hypothesis about how the data was generated, you want the hypothesis that would most plausibly produce what you saw.

But fitting is not the end. After you fit, you must check whether the curve is actually a good fit. This is called goodness-of-fit testing. It is the difference between assuming a distribution fits and checking that it truly does. A curve might fit the middle of your data well but miss the tails, or it might be completely wrong for some other reason. So you always look at diagnostic plots and run tests to see if the fit is adequate, not just assume it.

Imagine you are a tailor. A customer walks in wanting a suit, and you have a rack of suit patterns—these are the families of distributions. One pattern is for slim fit, another for relaxed fit, another for tall people. Each pattern is just a template with adjustable measurements: shoulder width, sleeve length, inseam. Those measurements are the parameters.

The customer's measurements are your data. Fitting the suit means taking one pattern and adjusting its measurements so the suit matches the person as closely as possible. You don't start from scratch—you start from a pattern that broadly matches the person's build, and then you tweak until it looks right.

But here is the catch: a suit that fits in the shoulders but is too tight in the hips is a bad fit, even if the pattern was the right family. So you must check the fit, not just assume it. You ask the customer to move around, you look at how the fabric drapes, you pin and adjust. That is goodness-of-fit—you test the fit, not just assume it.

The analogy stops working when you consider that distribution families are much more rigid than suit patterns. A suit pattern can be adjusted continuously to fit almost any body, but a distribution family is a fixed mathematical shape. If your data comes from a process that is fundamentally different from the family you chose, no amount of parameter tweaking will make it fit well. A normal distribution, for example, is symmetric and bell-shaped; if your data is heavily skewed (like income data, where a few people earn many times the median), a normal fit will be poor no matter what mean and standard deviation you pick. That is why choosing the right family matters, and why you always check the fit afterward.

Fitting a distribution means choosing a specific parametric family (a named curve with adjustable parameters) and estimating those parameters from observed data using a method like maximum likelihood, then verifying that the chosen family adequately describes the data through goodness-of-fit diagnostics.

This concept is your entry point into probability theory. You already know that data has patterns—now you will learn how to express those patterns as mathematical curves. This connects directly to several neighboring topics you have notes on. First, it leads naturally to Sampling Distributions: once you have fitted a distribution to a sample, the sampling distribution tells you how the estimated parameters would vary if you repeated the sampling process many times. Second, it connects to Concentration Inequalities: these give you bounds on how far a sample statistic can deviate from the true value, which is useful when checking whether your fitted parameters are reliable. Third, it connects to the Empirical CDF: that is a nonparametric way of estimating the distribution directly from data, which you can compare against your fitted parametric curve to see how well it matches. Finally, in later topics like Bayesian Methods, you will use fitted distributions as priors or likelihoods. This concept sits between having raw data and being able to make probabilistic statements about it.