FID vs KID for Text-to-Image: Sample Floors and Reporting Rules

This guide compares FID and KID for text-to-image evaluation and sets reporting rules for each.

It explains when KID is the trustworthy default and when FID may be reported.

How FID and KID actually compute distance

FID begins by passing real and generated images through Inception and taking the pool3 features, a 2048-dimensional vector per image. It then summarizes each set by a multivariate Gaussian: a mean vector μ and covariance matrix Σ. The reported distance is the Fréchet distance between those two Gaussians, ||μ_r − μ_g||² + Tr(Σ_r + Σ_g − 2(Σ_r Σ_g)^{1/2}). The Wikipedia FID entry and 123ofai's FID guide both describe FID as the de facto standard metric for generative image models, from GANs to diffusion models.

That formula is exact for two Gaussian distributions and an approximation otherwise. It uses only first- and second-order moments, so two feature sets with the same mean and covariance receive the same FID even if their higher-order structure differs. The apxml distributional-metrics note frames KID as the alternative that responds to this limitation, noting that the Gaussian assumption might not always hold and that FID estimates can be biased, especially when calculated on small sets of samples.

KID keeps the same Inception pool3 features but changes the distance estimator. Instead of fitting Gaussians, it estimates the squared maximum mean discrepancy (MMD²) with a polynomial kernel k(x,y) = (x·y/d + 1)³, where d is the feature dimension (2048 for pool3 features). The kernel implicitly maps each feature vector into a higher-dimensional space, and the MMD measures the distance between mean embeddings of the real and generated feature distributions in that space.

The "unbiased" part is an estimator property, not a kernel property. For two samples of sizes m and n, the unbiased MMD² estimate is the average of k(x_i,x_j) over distinct real pairs plus the average over distinct generated pairs minus twice the average over all cross pairs. That construction removes the bias that would come from including i=j terms, so the estimate's expectation equals the population MMD² for the chosen kernel.

Because FID reduces each distribution to μ and Σ while KID retains pairwise kernel similarities, the two can disagree even when computed on identical Inception features. FID's value is a closed-form distance between fitted Gaussians; KID's value is a two-sample kernel statistic. When reporting KID, the convention in this guide is to multiply the MMD² estimate by 1000 and include its standard deviation, since the raw squared-distance scale is small and the uncertainty is part of the measurement.

The evidence

Wikipedia's Fréchet inception distance entry identifies FID as a metric used to assess the quality of images created by a generative model, such as a generative adversarial network or a diffusion model. A separate guide calls FID the de facto standard metric for evaluating generative image models. Taken together, these sources establish FID's incumbent status in image-generation evaluation; neither source is cited here for a claim that FID is unbiased.

The load-bearing caveat comes from apxml's distributional-metrics page: it says the Gaussian assumption behind FID might not always hold, and that FID estimates can be biased, especially when calculated on small sets of samples. The same page presents Kernel Inception Distance as an alternative approach for these issues. That is the specific source claim that motivates preferring KID when sample counts are small; it is a caution about small sets, not a statement that every FID value is wrong.

inferensys's FID definition adds two properties that matter for interpreting reports. It says FID is deterministic for a fixed set of images, and that its computational cost involves forward-passing all images through the Inception network and computing covariance matrices, which is more eMore expensive than Inception Score but standard for evaluation. Because that cost comparison is between FID and Inception Score, it does not by itself settle whether FID or KID is cheaper or preferable.

The evidence therefore has a narrow scope. Wikipedia and the guide support FID's role as the incumbent metric; apxml supports a small-sample bias caveat and KID as an alternative; inferensys supports determinism and a cost comparison against Inception Score. None of these sources, as cited here, provides a universal ranking of FID against KID across all sample sizes or datasets.

A practical check follows: when a 2026 text-to-image report cites FID, verify which of these source claims it is actually using. Popularity or incumbent status from Wikipedia does not answer apxml's bias caveat, and inferensys's cost note does not convert into a FID-versus-KID cost verdict. A report that addresses the evidence should separate incumbent usage, small-sample bias, and cost comparisons rather than treating one as proof of the others.

FID vs KID: side-by-side, with a winner

KID is the default winner in this side-by-side: it is the metric to report first, with FID reserved for comparisons against published baselines that already use it. The table below states the three criteria that decide the default and names the winner for each.

On small samples, KID's unbiased estimator is the trustworthy side of the comparison. apxml's distributional-metrics discussion notes that FID estimates can be biased, especially on small sets, while KID's unbiased MMD² estimator addresses that issue. That is the practical reason to prefer KID when the evaluation set is not large enough for FID's Gaussian summary to be stable.

On comparability, FID still wins. Wikipedia's Fréchet inception distance entry identifies FID as a metric for assessing images from generative models, and 123ofai calls it the de facto standard, so most published baselines remain FID. A KID-first report should therefore include FID only when the comparison target is an FID baseline, not as the primary score.

On interpretability, KID wins again because its squared distance can be negative in finite samples, which is itself a diagnostic that n is too small. FID is always at least zero, so it cannot expose the same underpowered comparison. Use that sign as a check before trusting any KID value, and do not smooth or clip it away.

Default rule: report KID first, keep FID for incumbent-baseline comparisons, and treat a negative KID as a sample-size warning rather than a score. This preserves the KID-first default without claiming that FID is unusable or that every small-sample KID is automatically valid.

Costs and numbers that matter

A standard KID report uses 5,000 generated and 5,000 real images, so it costs 10,000 Inception forward passes. A FID report at the 50,000-image-per-side floor costs 100,000 passes, ten times as many. The pass count therefore scales linearly with the chosen n on each side.

FID also has a fixed covariance cost: each 2048×2048 covariance matrix has 4,194,304 entries, about 16 MB at fp32, so two are about 32 MB before the matrix square root. KID avoids covariance, but its sample kernel matrix grows with n²; at n = 50,000 it has 2.5 billion entries per side, about 10 GB at fp32, which is the practical ceiling.

At the standard KID size of 5,000 images per side, the same kernel matrix has 25 million entries, about 100 MB at fp32. The recommended KID report is therefore light in both passes and memory, while FID's fixed covariance step remains small but its pass count does not.

Method: if your budget or data supply is below 50,000 images per side, report KID ×1000 (polynomial kernel, degree 3, unbiased MMD² estimator) with its standard deviation and do not report FID. If you can score 50,000 per side, FID is computationally admissible, but KID at its standard size still needs one-tenth the forward passes.

Before choosing n, check the kernel matrix size: n² entries times 4 bytes. If that exceeds available memory, reduce n or keep KID at a smaller supported size rather than forcing FID-scale sampling.

What the evidence does NOT establish

The evidence behind the KID-default rule is directional, not quantitative: it supports preferring KID when samples are limited, but it does not supply a numeric FID bias at any sample size. Consequently, any statement such as "FID is biased below 50,000 images" is a team-level method choice and reporting gate, not a figure attributable to a source. Treat the 50,000-image FID threshold as an internal convention, and label it as such in methods sections.

Implementation details are likewise unspecified in the available sources. No source names the canonical Inception weights, feature layer, or resize mode, so an FID or KID value is comparable only when the full pipeline is held fixed. Before comparing two results, verify that both use the same Inception variant, the same feature extraction point, and the same image resizing and normalization; otherwise the comparison is between different measurement procedures rather than different models.

The KID convention is also not fully sourced. The polynomial kernel with degree 3 is the standard choice, but the available sources does not mandate it, and KID ×1000 with a standard deviation is interpretable only when the kernel, degree, and estimator are reported alongside it. If a different kernel or degree is used, the default rule no longer applies automatically and the result should be treated as a separate metric.

The rule can break in several practical cases, even when sample counts look adequate:

Use KID as the default only after these checks pass; otherwise, state the measurement conditions explicitly and avoid claiming that the evidence proves one metric is unbiased.

5,000 vs 5,000, KID and FID side by side

Inputs are fixed before any metric is computed: 5,000 real images from the held-out set and 5,000 generated images from the model under test. Both sets are resized to the Inception input size and passed through the same Inception checkpoint, so preprocessing and feature extraction are held constant while only the model output varies.

Checkpoint 1 is feature extraction: 10,000 forward passes in total. Record the wall-clock time for this stage, separating model compute from queue or data-loading time if your profiler allows it, and confirm that each side produces a feature tensor of shape (5000, 2048). If either tensor does not have that shape, stop and repair the input pipeline or checkpoint before computing any distance.

Checkpoint 2 is KID: compute the unbiased MMD² with a polynomial kernel of degree 3, multiply the result by 1000, and record the standard deviation across 100 bootstrap splits. Report KID ×1000 together with that standard deviation; the bootstrap spread is part of the result rather than an optional diagnostic.

At this sample size, FID is not reported under the reader rule, which reserves FID for runs with at least 50,000 images per side. The 5,000-image run is therefore a KID checkpoint: it records the model's feature-distribution distance at a scale where KID is the reported metric, while FID remains deferred to a larger evaluation.

Accept the run only if both checkpoints are complete and their records agree: matching inputs, verified tensor shapes, and a KID ×1000 value reported with its bootstrap standard deviation. If any checkpoint fails, rerun from that stage instead of editing the metric output.

Worked Example: Run the Numbers

Illustration: In a September 2026 text-to-image evaluation, Team A freezes 8,000 generated images and 10,000 real images. The reporting rule applied is KID ×1000 with a polynomial kernel of degree 3 and an unbiased MMD² estimator, reported with its standard deviation, on at least 5,000 generated and 5,000 real images; FID is reported only when a set has at least 50,000 images.

KID eligibility: 8,000 is at least 5,000, and 10,000 is at least 5,000. The generated set clears the KID floor by 3,000 images (8,000 − 5,000 = 3,000) and the real set clears it by 5,000 images (10,000 − 5,000 = 5,000). KID is therefore reportable on the full 8,000/10,000 split, with its standard deviation included.

FID eligibility: 8,000 is below 50,000 and 10,000 is below 50,000. The generated set is 42,000 images short (50,000 − 8,000 = 42,000) and the real set is 40,000 short (50,000 − 10,000 = 40,000). FID is not reportable for this run.

Winner for this example: KID, because it is the only one of the two metrics that clears its sample floor. This is an eligibility result for these counts, not a claim about how FID would rank the same images.

Break-even trigger: FID becomes eligible only when both the generated and real sets reach 50,000 images. Until that point, report KID alone; once both floors are met, report FID alongside KID on matched sample sizes.

If Team A later expands to 50,000 generated and 50,000 real images, both metrics clear their floors, so the report can include KID and FID on the same 50,000/50,000 split.

Illustration only: these counts are example inputs, not a recommended dataset size, and the thresholds are reporting rules rather than a claim about statistical power.

Decision rules

These rules turn the reporting standard into a decision sequence. Check sample counts first, then the comparison goal, then the uncertainty of the KID estimate. They do not change how either metric is computed; they only decide which reported number is defensible for a given evaluation.

The first rule sets the floor. KID ×1000 is reported on at least 5,000 generated and 5,000 real images, using the polynomial kernel with degree 3 and the unbiased MMD² estimator, so a smaller set does not support either metric under this standard.

The second rule covers the small-sample regime common in 2026 text-to-image evaluations. There, report only KID ×1000 with its standard deviation; FID is not reported.

The third rule is the comparability exception. When n_generated is at least 50,000 and a published baseline must be matched, FID may be reported, but only with the Inception weights and resize mode stated explicitly so another researcher can reproduce the number.

The fourth rule prevents overreading noise. If KID's standard deviation is larger than the observed gap between two models, the comparison is unresolved; collect more images before naming a winner. If the standard deviation is smaller than the gap, the KID ×1000 comparison can be reported with that standard deviation.

Frequently Asked Questions

How should KID be scaled and kernelized when reported?

Report KID as KID ×1000 with a degree-3 polynomial kernel.

What estimator and sample requirements apply to KID reporting?

Use the unbiased MMD² estimator and include its standard deviation on at least 5,000 generated and 5,000 real images.

What sample-size floor governs FID reporting?

Report FID only when the sample size is at least 50,000.

Why can FID be misleading for small evaluation sets?

FID estimates can be biased, especially on small sample sets.

What Inception feature representation does FID use?

FID begins by passing real and generated images through Inception and taking the pool3 features, a 2048-dimensional vector per image.

When is the FID Gaussian formula exact rather than approximate?

That formula is exact for two Gaussian distributions and an approximation otherwise.

Quick answers

Also worth reading: Image quality test: Frechet Inception Distance (FID) vs text score, 1,000 reviews: Image quality test: Frechet Inception · CMMD vs FID: 5-to-1 Decision Verdict, Cost Is FID's Only Win: CMMD vs FID: 5-to-1 Decision · FID-CLIP Divergence: Hallucination Trap and Stanford Audit: FID-CLIP Divergence: Hallucination Trap and