Bayesian Approaches in Machine Learning

From point estimates to distributions over hypotheses.

Bayes' rule as the engine

Bayesian learning updates beliefs about parameters or models after observing data:

posterior = likelihood × prior / evidence.

Instead of committing to one best parameter vector, we maintain uncertainty and integrate over possibilities.

Why this matters

• Uncertainty calibration for high-stakes decisions.
• Better behavior in small-data settings via informative priors.
• Natural protection against overfitting through posterior averaging.

Core Bayesian methods

• Conjugate models: analytically tractable updates.
• MCMC: sample from complex posteriors.
• Variational inference: optimize a simpler approximate posterior.
• Bayesian neural networks: distributions over weights.

Decision-theoretic layer

Bayesian pipelines separate inference (what is true?) from decisions (what should we do?). Utility-aware decisions can be made from posterior predictive distributions using expected risk minimization.

Takeaway: Bayesian ML frames learning as uncertainty-aware inference, not only error minimization.