Score Stability

The contract

Scores are byte-identical from version to version. Thresholds are calibrated against real shoots with calibrate_blur.py; a score that silently drifted by a few percent would invalidate every calibration in the wild without anyone noticing. Score stability is therefore treated as a hard interface: the golden CSV reports checked by Golden Tests must stay byte-identical across any refactor or optimization.

Hand-rolled by design

The pipeline implements the classic variance-of-Laplacian recipe with OpenCV’s exact numeric semantics, which the original scores were validated against. Because “roughly the same” is not good enough, the image operations replicate those semantics instead of calling off-the-shelf crates, which use different weights, rounding or border conventions:

• BT.601 grayscale weights (0.299 / 0.587 / 0.114, rounded like OpenCV’s cvtColor) for non-JPEG sources.
• Area-averaging resize replicating INTER_AREA, with truncated (not rounded) target dimensions.
• BORDER_REFLECT_101 borders (gfedcb|abcdefgh|gfedcba) in both the Gaussian pre-blur and the Laplacian — OpenCV’s default border mode.
• OpenCV’s automatic Gaussian kernel size for 8-bit images: round(6σ + 1), forced odd.
• The same 3×3 Laplacian kernel as OpenCV’s Laplacian with its default aperture.
• Population variance (not sample variance), with identical integer tile boundaries.

Replacing any of these with a library call, or changing a rounding or truncation, shifts every score. Do not “simplify” them. Optimizations are welcome, but they must reproduce the reference operations exactly: every fast path in the code is backed by an exact-equivalence unit test against a reference implementation kept in the tests module of src/lib.rs, and the whole pipeline is backed by the golden CSVs.

One deliberate departure

One scoring change was accepted, in July 2026: JPEG sources score on the JPEG’s own luminance plane instead of BT.601 recomputed over decoded RGB.

For a YCbCr JPEG, the Y plane is the BT.601 luminance the encoder computed from the original pixels; recomputing it after chroma upsampling and the RGB round trip only adds two lossy conversions and a double rounding. Decoding Y directly is both closer to the source signal and much faster.

Measured on the reference corpus before adoption:

Changing the scores on purpose

A deliberate scoring change is exceptional and follows a fixed procedure:

1. Produce a candidate CSV of the reference corpus with the change applied.

2. Measure the drift:

   cargo run --release --example drift_report -- tests/assets/golden/default.csv candidate.csv
   

   The report prints per-file deltas, the Spearman rank correlation (does the new score still order photos the same way?), verdict flips at the threshold, and the equivalent threshold suggested by the median score ratio.

3. Announce the change with those numbers — users may need to adjust thresholds.

4. Regenerate the golden references (Golden Tests) so the new behavior becomes the contract.