Bayes’ rule and the independence assumption
For a class c and observed features x₁, …, xₚ:
P(c | x₁, …, xₚ) ∝ P(c) × P(x₁, …, xₚ | c)
Naive Bayes approximates the joint likelihood as a product:
P(c | x₁, …, xₚ) ∝ P(c) × ∏ᵢ P(xᵢ | c)
The features need not be independent in general; the assumption is conditional on the class. It is often false—for example, “New” and “York” are related words—but the approximation can still rank classes usefully. The class prior matters: common classes start with more probability before the features are considered.
Work through a prediction
In the classic weather example, the observed features are Sunny, Cool and Strong wind. Without smoothing, the training counts give:
Score(Yes) = 9/14 × 2/9 × 3/9 × 3/9 = 1/63
Score(No) = 5/14 × 3/5 × 1/5 × 3/5 = 9/350
These are unnormalised scores: each is the class prior multiplied by the three class-conditional likelihoods. The No score is larger, so the model predicts No. Dividing each score by their sum would produce normalised probabilities. The visualiser uses Laplace smoothing to avoid zero likelihoods, so its displayed values differ from this unsmoothed hand calculation.
What the assumption buys, and what it costs
The product turns a large joint-probability problem into separate per-feature counts. This makes training and prediction fast, even with many features and relatively few examples. It can work well for text classification, where word features are dependent but their counts still carry useful class information.
Naive Bayes variants match different feature types: Multinomial Naive Bayes is common for non-negative counts, Bernoulli Naive Bayes for binary indicators, and Gaussian Naive Bayes for continuous features modelled with class-specific normal distributions. Choosing the wrong variant or preprocessing can make the likelihoods inappropriate.
The independence approximation can produce probabilities that are too extreme, especially when correlated features repeat the same evidence. A strong class prediction is not proof that the probability is calibrated. If probability accuracy matters, evaluate a reliability curve and consider calibration. For a model that can learn feature interactions directly, compare with logistic regression.
Sources and next steps
See scikit-learn’s Naive Bayes guide for the variants and their assumptions. The weather counts above illustrate how priors and likelihoods combine; your own application should validate both classification performance and probability quality.