If KLDivLoss becomes negative, first check whether its input is a log probability and its target is normalized. If the value is smaller by a class-count factor, inspect the reduction. A probability supplied where a log probability was expected may still produce a finite scalar, but that scalar is no longer the intended divergence.

For “What belongs in KLDivLoss input and target, and why do mean and batchmean differ?”, one binary-distribution example separates direction, representation and denominator. 中文版.

Write the mathematical direction beside the call

Let P be the target distribution and Q the model distribution. The intended quantity is D(P∥Q)=ΣᵢPᵢ(log Pᵢ−log Qᵢ). It is not a symmetric distance: swapping P and Q generally changes the answer. The worked examples use natural logarithms and strictly positive distributions whose rows each sum to one.

The PyTorch 2.8 definition expects input in log space, with a probability target by default or a log-space target when log_target=True. Consequently D(P∥Q) corresponds to input=log Q and target=P. Reading mathematical notation left to right into the argument list reverses that intended correspondence.

With P=(0.8,0.2) and Q=(0.5,0.5), the sum is 0.8log1.6+0.2log0.4, approximately 0.192745. In the opposite direction D(Q∥P) is approximately 0.223144. Both are legitimate quantities, but encode different objectives. Choosing whichever happens to be smaller does not establish the direction required by a training design.

Why a probability supplied as a log probability goes negative

If the probabilities 0.5 are supplied directly as input, the pointwise expression becomes ΣPᵢ(log Pᵢ−Qᵢ). Its value here is about −1.000402. The log Q term has disappeared, so the nonnegativity guarantee for a properly defined KL divergence no longer applies. This is a representation error, not a model that performs better than a perfect match.

Nonnegativity concerns the complete sum over the probability support, not every class contribution. In the correct example the second term, 0.2log0.4, is already negative; only the complete sum is nonnegative. Negative entries from reduction=’none’ therefore do not by themselves identify a broken call.

Also separate a very small negative floating-point discrepancy from a large semantic error. Direction, normalization, support and representation must be established before selecting a numerical tolerance. The roughly negative-one counterexample exposes a conceptual mistake. It does not prescribe a universal threshold for every training task or data type.

Two rows and two classes expose the denominator

Build a 2×2 batch whose first row uses the distributions above and whose second row has P=Q=(0.5,0.5). Summing every class contribution gives S≈0.192745. Averaging per sample gives S/2≈0.0963724; averaging all four elements gives S/4≈0.0481862. They differ by the class count, two.

batchmean divides the total by the first dimension, while mean averages all elements. For shape [B,C] with one distribution per row, the former is the average per-sample divergence. For [B,T,C] with one distribution per time step, batchmean still divides only by B. It averages each sample’s sum across time rather than automatically giving a per-token average.

Choose the unit of measurement explicitly for a sequence task. With padding, one possible objective is to obtain pointwise terms, sum over classes, mask invalid tokens, then divide the valid-token sum by the valid-token count. Skip that objective or use an explicitly defined empty-batch policy when no valid token remains. Define what to do when that count is zero rather than hiding an empty batch behind an arbitrary denominator. This is an objective-design choice, not something a reduction name can infer.

log_target changes representation, not direction

The default pointwise form is Pᵢ(log Pᵢ−log Qᵢ). If the target contains aᵢ=log Pᵢ, the same term is exp(aᵢ)(aᵢ−log Qᵢ). Equality requires a to be the natural log of that probability. log_target=True neither exchanges the KL direction nor automatically normalizes raw logits.

When model outputs are logits, log_softmax along the intended class axis provides log probabilities. A wrong axis can leave the shape legal while changing what is normalized. A target supplied by another model also needs an explicit distribution-construction rule; an arbitrary floating-point array is not a normalized soft label merely because it has the right dimensions.

The strictly positive examples avoid zero-probability boundaries. Real P=0 or Q=0 cases involve zero contributions or infinite divergences under their respective conditions. The teaching program’s ordinary log calls must not be applied directly to zero. Adding epsilon also changes the objective rather than automatically preserving its interpretation. Actual support and numerical handling require separate validation.

Nine checks separate three classes of mistake

The program verifies the correct value, identical distributions, direction asymmetry, equivalence of log-target representation, the negative result from a wrong input space, and both denominators. It also rejects one unnormalized input. It uses standard-library math, without running PyTorch or testing backward gradients.

# Positive normalized distributions; standard-library math only.
from math import log, exp, isclose

def kl(p, q):
    if len(p) != len(q) or not p:
        raise ValueError('shape')
    if min(p+q) <= 0 or not isclose(sum(p),1) or not isclose(sum(q),1):
        raise ValueError('positive normalized probabilities required')
    return sum(a*(log(a)-log(b)) for a,b in zip(p,q))

p=(0.8,0.2); q=(0.5,0.5)
d=kl(p,q)
assert isclose(d,0.19274475702175753)
assert d > 0 and isclose(kl(p,p),0)
assert not isclose(d,kl(q,p))
log_input=tuple(map(log,q))
log_target=tuple(map(log,p))
by_log=sum(exp(a)*(a-b) for a,b in zip(log_target,log_input))
assert isclose(by_log,d)
wrong=sum(a*(log(a)-b) for a,b in zip(p,q))
assert wrong < 0
B,C=2,2
summed=d+kl((0.5,0.5),(0.5,0.5))
assert isclose(summed/B, d/2)
assert isclose(summed/(B*C), d/4)
assert isclose((summed/B)/(summed/(B*C)), C)
try:
    kl((0.8,0.8),q)
except ValueError:
    pass
else:
    raise AssertionError('unnormalized target accepted')
print('9 KL arithmetic checks passed')

The validator deliberately accepts only strictly positive normalized vectors. Rejecting zeros limits this example; it is not a claim that KL is undefined for every distribution containing a zero. Encoding assumptions in validation keeps the demonstration from silently expanding to cases its implementation does not handle.

Retain both numerator and denominator when comparing losses

A debugging record should state D(P∥Q), the representation of both arguments, the class axis, the per-distribution sum and the final denominator. If experiments differ by a constant factor, align those definitions before interpreting model quality or changing a learning rate. Class count, sequence length and padding fraction can all expose a hidden denominator choice.

The earlier ignore_index denominator analysis similarly starts by deciding which positions participate. KL additionally requires an explicit distribution direction and log-space contract. Reusing another loss function’s intuition about mean leaves these extra requirements unchecked.

If these numerical comparisons help diagnose a negative KL value or a loss-scaling mismatch, voluntary support through the section below is welcome. Separately recording direction, representation and denominator remains useful when the next distillation objective is introduced.

Sources and backfill note

KLDivLoss · log_softmax

Checked against versioned PyTorch 2.8 definitions available before the planned October20 19:00 Beijing slot. This backfill retains that article date; actual public release time is recorded separately. Derivations and diagrams are original. Standard-library code ran locally, without a PyTorch integration test or device benchmark.