vizlearn

Machine Learning / Labels

Hard vs Soft Labelling

Machine LearningBy Ashish JangraUpdated 26 October 2026

Categorical vs. Probabilistic labels. Pick a tweet to see how data representation impacts nuance.

Select Input

Hard Label

Negative 0%

Neutral 0%

Positive 0%

Vector: [0, 0, 0]

Soft Label

Negative 0%

Neutral 0%

Positive 0%

Vector: [0, 0, 0]

Hard labelling picks one winner. It’s simple but rigid.

Soft labelling maps a probability distribution. It allows the model to learn uncertainty and class relationships, making it more robust against noisy or ambiguous data.

Overview

For three classes, a hard label for class two is ([0,1,0]). A soft label such as ([0.1,0.7,0.2]) assigns a target share to each class; its values sum to 1. Hard labels are appropriate when examples have a clear agreed answer. Soft labels are useful when ambiguity or disagreement is part of the task.

A worked example

Suppose five of seven clinicians mark a scan positive and two mark it negative. A hard majority vote records only “positive.” A soft target of ([2/7,5/7]) retains the disagreement. It does not prove the scan has a (5/7) chance of disease; it records how this group labeled it. The interpretation depends on how the targets were made.

Soft targets can also come from label smoothing, which moves a small amount of probability away from the one-hot class, or knowledge distillation, which trains a smaller model against a teacher’s class distribution. These methods can regularize training, but gains depend on the data and model and should be evaluated on held-out examples.

What changes during training

With cross-entropy, a hard target penalizes probability assigned away from the labelled class. A soft target distributes that penalty across classes. This can discourage extreme confidence and preserve relationships among classes. It cannot correct an inaccurate target distribution, and soft labels do not automatically produce calibrated probabilities.

Question	Hard labels	Soft labels
Target	One class	Distribution over classes
Best fit	Clear, agreed categories	Ambiguous cases, multiple annotators, or teacher outputs
Information kept	Chosen class	Relative target support across classes
Main risk	Throws away disagreement	Can encode noisy or poorly calibrated targets

When to choose each

Use hard labels when annotation is reliable and the task expects a single category. Keep annotator votes or confidence distributions when disagreement is meaningful. For label smoothing or distillation, compare against the hard-label baseline with the same data split and metrics; avoid tuning against the test set.

Sources and next steps

Szegedy et al., Rethinking the Inception Architecture for Computer Vision, discusses label smoothing.
Hinton, Vinyals, and Dean, Distilling the Knowledge in a Neural Network, introduces teacher-student distillation.
Continue with one-hot encoding and probability calibration.

Recall check

0 of 3

Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.

What does this module say about “What each label represents”?
What does this module say about “A worked example”?
What does this module say about “What changes during training”?

Cheat sheet