Results and reporting

Each generated config owns a unique results_dir. The training entry point stores enough information to audit individual folds and to build a common benchmark table.

Run artifacts

Artifact	Contents
.run.lock	Advisory process lock and last trainer PID; prevents concurrent writers in one experiment directory
config.json	Effective configuration, including command-line seed override
fold<K>_best.pt	Best validation checkpoint when checkpointing is active
fold<K>_predictions.csv	Slide and case IDs, label, predicted class, and per-class probabilities when batch metadata is available
metrics.json	Fold metrics and validation loss for one experiment
logs/<method>/	TensorBoard training and validation loss events
interpretability/fold<K>/<slide-id>/	Optional raw patch-score CSVs, rendered PNGs, and a checkpoint/provenance manifest

Generated benchmark aggregation additionally writes:

• fold_results.csv, one normalized row per completed fold;
• aggregate_results.csv, mean, standard deviation, and observed fold count for every metric and experiment signature.

TensorBoard logs are grouped by method name and are convenient for monitoring, but the experiment-specific results directory is the archival source of truth.

Reported metrics

The common training loop computes the same classification metrics for every method:

Metric	Interpretation
Accuracy	Fraction of correct predictions
Balanced accuracy	Mean recall across classes
Macro F1	Unweighted mean of per-class F1 scores
AUROC OVR	Binary AUROC or macro one-vs-rest AUROC; null when undefined
NLL	Mean negative log-likelihood of the true class
ECE	Ten-bin expected calibration error
Per-class recall	Recall indexed by the frozen numeric class order

Slide-level metrics are always emitted. Patient-level metrics are emitted when the loader returns case IDs for every prediction; per-slide probabilities are averaged within each case before patient-level scoring.

For TCGA-BRCA and the other class-imbalanced common benchmarks, slide-level balanced accuracy is the primary accuracy measure. Raw accuracy is retained for completeness, but can be close to the majority-class prevalence even when minority-class recall has collapsed. The trainer therefore prints balanced accuracy first and labels ordinary accuracy as raw accuracy.

Reading aggregate tables

Never compare only the mean column. A valid comparison should match on:

• cohort and shot count;
• feature and resolution signatures;
• fold count;
• prompt provenance and encoder checkpoint;
• slide-level versus patient-level metric namespace.

aggregate_results.csv includes the observed folds count. A row with fewer folds than the protocol requested is incomplete, even if its mean is high.

Post-processing reports

Two read-only analysis commands extend the fold aggregates without rerunning a model. scripts/statistical_report.py validates the current run fingerprint and saved out-of-fold predictions before producing patient-bootstrap confidence intervals, AUPRC/Brier/calibration tables, collapse diagnostics, and optional patient-paired method differences. scripts/efficiency_report.py joins the same current identities to launch reports, Slurm accounting, trainer parameter records, artifact sizes, and optional checkpoint profiles before identifying broad and encoder-controlled performance/cost Pareto fronts.

See Statistical analysis and Efficiency and Pareto reporting for their exact scopes and commands. Missing resource measurements remain blank; neither report estimates or silently substitutes unavailable evidence.

Minimum reporting checklist

For every result table, state:

1. dataset version, task labels, and patient/slide counts;
2. shot definition, number of folds, seed, and split provenance;
3. method variant and whether it is an upstream reproduction or framework extension;
4. patch/slide encoder checkpoint, feature space, magnification, and tile size;
5. prompt source and whether prompts were optimized;
6. mean and standard deviation across the same completed folds;
7. balanced accuracy as the primary measure for imbalanced cohorts, plus raw accuracy and per-class recall;
8. any failed, skipped, or unavailable runs.

For an interpretability figure, additionally retain manifest.json and report the exact score type, target class, scale, feature space, magnification, patch size, and whether the run is native or an encoder extension. Use the raw CSV for quantitative analysis; percentile clipping changes only the PNG. See Patch-level interpretability for the complete contract.