The same predictions can produce macro-F1 of 0.474 or 0.316. Include a class with no true or predicted examples, replace its undefined score with zero, and the average changes. The model has not changed; the evaluation question has. Before comparing F1, fix the label set, averaging rule, absent-class policy, and aggregation scope.

阅读中文版. This article covers single-label multiclass classification: each example has exactly one true label and one predicted label. All numbers come from a constructed example checked with the Python standard library. No model was trained and scikit-learn was not executed.

Ten predictions, three different summaries

Suppose a test set contains nine examples of A and one of B. A classifier predicts A for all ten. The labels are anonymous teaching categories, with no implied business costs. Accuracy is nine out of ten, or 0.9, while the classifier misses every B. The evaluation decision is whether examples or classes should receive equal weight. An averaging option expresses that decision; its name does not make it inherently better.

Treat each class as positive in turn. For A, TP=9, FP=1, FN=0, so its F1 is 18/19. For B, TP=FP=0 and FN=1, so its F1 is zero. We use F1=2TP/(2TP+FP+FN) directly. Working from counts avoids introducing extra zero-denominator branches by calculating precision and recall first.

Macro averages the two class scores equally: (18/19+0)/2=9/19, approximately 0.474. Weighted F1 uses true support, giving (9×18/19+1×0)/10=81/95, approximately 0.853. Micro first adds class counts, then divides: 18/(18+1+1)=0.9. Each calculation is valid, but they emphasize different things. The scikit-learn 1.7.2 F1 reference specifies these averaging and label-selection options.

An absent class can change the average

Now define the task as three classes, A, B, and C. This batch contains neither true C examples nor C predictions. For C, TP, FP, and FN are all zero, so F1 is undefined. Assign zero and macro becomes (18/19+0+0)/3=6/19, approximately 0.316. Exclude C from the average and it returns to 0.474. Assign one instead and it becomes 37/57, approximately 0.649. A reporting policy alone creates a substantial difference.

This does not establish that absent classes should always receive zero. A report using a fixed task vocabulary and that convention should show support alongside the score. C has no evaluation evidence here. Its substituted zero does not demonstrate that the model cannot recognize C. A fixed vocabulary stabilizes the report structure; it does not manufacture evidence for a missing class. Conversely, excluding undefined scores requires reporting which classes were excluded and how many remained, otherwise weekly reports or slices can silently use different denominators.

Distinguish absence from zero true support. If there is no true C but the model incorrectly predicts C once, C has an FP. Its F1 denominator is nonzero and its score is a defined zero. Dropping it merely because its support is zero discards a real error. A class absent from the ground truth is not necessarily absent from every relevant count.

Also distinguish data-inferred labels from an explicit label configuration. A report using observed labels cannot automatically be compared with another using a fixed three-class vocabulary. Our teaching implementation makes the vocabulary explicit and rejects truth or prediction labels outside it. That strict validation is a deliberate local policy, not a claim that the implementation reproduces every library parameter.

When micro-F1 equals accuracy

Under this article’s conditions—single-label predictions, all classes included, and no sample weights—let N be the number of examples and K the number correctly predicted. Each correct prediction contributes one TP. Every mistake contributes one FP to the predicted class and one FN to the true class. Therefore total TP=K and FP=FN=N−K, giving micro-F1=2K/[2K+2(N−K)]=K/N, exactly accuracy.

The proof shows the boundary as well as the result. For example, when reporting only A, a B-to-C mistake might not enter the selected counts at all. With only a subset of labels selected, both ends of a mistake might no longer enter the totals. In multilabel classification, an example can contribute multiple label decisions, so the same counting argument does not apply directly. Do not encode “micro always equals accuracy” as a general evaluation rule. Establish what an example and a label decision mean first.

Weighted F1 is close to micro in this example, but is not another name for it. It averages nonlinear class ratios after calculating them. Class supports alone do not determine how errors flow between classes, so they cannot generally convert weighted F1 into accuracy. Keep the class results if the aggregate is meant to be explainable.

Across batches, retain counts until the end

Consider a separate three-example set split into two batches. The first has two A examples predicted correctly; the second has one B predicted as A. Their micro scores are one and zero. An unweighted mean gives 0.5, while pooling all three examples gives two correct predictions out of three, hence micro-F1 of 2/3.

Weighting these particular batch micro scores by batch size recovers the pooled result because micro equals accuracy under the stated conditions. That special case does not justify weighting every kind of batch F1 by batch size. Macro and weighted F1 contain class-level ratios whose denominators change between batches.

A robust aggregation design makes each batch or worker return TP, FP, FN, and support in a fixed class order. Add those counts element by element, then calculate class scores and the desired average once. Do not apply an absent-class policy within every batch and average the resulting macro scores: a class can be present in one batch and absent in another, changing which terms and weights entered each intermediate result.

Check the contract with 19 assertions

The following standard-library program ran on Python 3.15.0 and passed 19 assertions. It checks the displayed scores, absent-class conventions, a predicted-only class, batch aggregation, perfect predictions, and rejection of empty inputs, unequal lengths, unknown labels, and duplicate or empty class configurations. Here absent=None excludes undefined class scores; it is not an example of passing None to scikit-learn.

from math import isclose
import sys

def report(truth, pred, labels, absent=0):
    if not truth or len(truth) != len(pred):
        raise ValueError('nonempty aligned inputs required')
    if not labels or len(set(labels)) != len(labels):
        raise ValueError('unique fixed labels required')
    if absent not in (0, 1, None):
        raise ValueError('absent must be 0, 1 or None')
    allowed = set(labels)
    if not set(truth + pred) <= allowed:
        raise ValueError('unknown label')
    counts = []
    for c in labels:
        tp = sum(t == c and p == c for t,p in zip(truth,pred))
        fp = sum(t != c and p == c for t,p in zip(truth,pred))
        fn = sum(t == c and p != c for t,p in zip(truth,pred))
        counts.append((tp,fp,fn,truth.count(c)))
    scores = [2*t/(2*t+f+n) if 2*t+f+n else absent
              for t,f,n,s in counts]
    defined = [x for x in scores if x is not None]
    macro = sum(defined)/len(defined) if defined else None
    weighted = sum((x or 0)*s for x,(_,_,_,s) in zip(scores,counts))/len(truth)
    tp = sum(x[0] for x in counts)
    fp = sum(x[1] for x in counts)
    fn = sum(x[2] for x in counts)
    micro = 2*tp/(2*tp+fp+fn)
    return counts,scores,macro,weighted,micro

checks=0
def check(ok):
    global checks
    assert ok
    checks+=1
truth=['A']*9+['B']; pred=['A']*10
r=report(truth,pred,['A','B'])
check(r[0]==[(9,1,0,9),(0,0,1,1)])
check(isclose(r[1][0],18/19) and r[1][1]==0)
check(isclose(r[2],9/19))
check(isclose(r[3],81/95))
check(isclose(r[4],.9))
r3=report(truth,pred,['A','B','C'])
check(isclose(r3[2],6/19))
check(isclose(r3[3],r[3]) and isclose(r3[4],r[4]))
check(isclose(report(truth,pred,['A','B','C'],None)[2],r[2]))
check(isclose(report(truth,pred,['A','B','C'],1)[2],37/57))
# A zero-support class predicted once still has an FP; it is defined.
a=report(['A'],['C'],['A','B','C'],None)
check(a[1]==[0,None,0] and a[2]==0)
s1=report(['A','A'],['A','A'],['A','B'])
s2=report(['B'],['A'],['A','B'])
pool=report(['A','A','B'],['A','A','A'],['A','B'])
check((s1[4]+s2[4])/2==.5)
check(isclose(pool[4],2/3))
check(isclose(pool[4],sum(t==p for t,p in zip(['A','A','B'],['A','A','A']))/3))
check(isclose(report(truth,truth,['A','B'])[2],1))
for args in [([],[],['A']),(['A'],[],['A']),(['A'],['X'],['A']),(['A'],['A'],['A','A']),(['A'],['A'],[])]:
    try: report(*args)
    except ValueError: check(True)
    else: raise AssertionError('invalid input accepted')
print(checks,'checks passed; Python',sys.version.split()[0])
print('AB macro, weighted, micro:',r[2:])
print('ABC macro:',r3[2],'; shard micro mean .5 vs pooled',pool[4])

For auditability, this implementation scans the examples for each class, taking roughly O(NC) time. It is not a throughput benchmark. A large-scale implementation can accumulate the same counts in one pass without changing their meaning. The code does not implement multilabel targets, sample weights, or scoring only selected labels. Library documentation establishes interface policy; this local execution establishes the arithmetic of the explicitly restricted example.

Compare evaluation rules before comparing models

A reproducible result should bind its data snapshot and filtering conditions, label mapping and order, single-label or multilabel setting, absent-class convention, averaging method, class supports and counts, and aggregation scope. If the model emits probabilities, fix the rule that converts them into predictions too. A threshold or mapping change can alter a score without a model improvement.

If rare classes matter, put their recall, precision, and support beside macro-F1 and inspect where their mistakes go. If you want an estimate weighted by real traffic, first check whether the test distribution represents actual use. These ten constructed examples do not justify a business threshold or establish anything about a trained model’s quality.

The example is useful as a regression fixture for an evaluation pipeline: fixed inputs and expected counts can catch accidental switches from macro to weighted, silent removal of absent classes, or premature averaging of batch scores. Passing it establishes consistency of the evaluation rules, not validation of a real classifier. Related denominator issues appear in RAG recall and NDCG’s ideal ranking, although classification F1 should not be substituted for ranking quality.

Sources: the F1 reference and 1.7 release history. Version 1.7.2 dates to September 2025, before the intended article date. This article was backfilled on October 25, 2026, retaining its intended October 23, 2026, 23:30 Beijing article date.