INVEX: Label-Free Pathology Foundation Model Ranking via Invariance and Expressiveness

The Key Idea

Evaluating pathology foundation models purely intrinsically, without labels or fine-tuning.

Central Hypothesis

Robust to Variations: A strong pathology representation must handle variations that don't change the underlying tissue morphology.
Arbitrary Orientations: Histopathology images inherently exhibit arbitrary orientations due to tissue prep and scanning, but their biological semantics remain unchanged.
Consistent Embeddings: An effective model should produce consistent (invariant) embeddings for rotated views of the exact same tissue.
Discriminative Information: While being rotation-invariant, the model must maintain sufficient expressiveness (discriminative power) to successfully distinguish different biological structures.

Invariance x Expressiveness

0° Rotation

Invariant Representation

Rotating a tissue patch does not alter its underlying morphology or diagnostic content. Consequently, an ideal pathology representation should assign nearly identical embeddings to rotated versions of the same tissue patch.

Invariance x Expressiveness

Collapsed (Poor Expressiveness)

Separated (High Expressiveness)

Expressiveness ensures the model does not collapse all data points together. It must preserve enough discriminative information to clearly separate different biological structures across the manifold.

Theoretical Goals vs Empirical Reality

Theoretical Trade-off

Expressiveness

Invariance

Representation
Collapse

Rotation
Sensitive

Ideal (INVEX)

INVEX

Empirical Rankings

Expressiveness

Invariance

Encoder 1
(Rank 3)

Encoder 2
(Rank 2)

Encoder 3
(Rank 1)

22 Evaluated Foundation Models

INVEX systematically audits the top pathology foundation models in the field across different pretraining objectives and architectures.

Encoder	Backbone	Training	Dim
`UNI-v1`	ViT-L/16	DINOv2	1024
`UNI-v2`	ViT-H/14	DINOv2	1536
`Virchow`	ViT-H/14	DINOv2	2560
`Virchow2`	ViT-H/14	DINOv2	2560
`GigaPath`	ViT-g/14	DINOv2	1536
`H-Optimus-0`	ViT-g/14	DINOv2-style	1536
`H-Optimus-1`	ViT-g/14	DINOv2-style	1536
`Phikon`	ViT-B/16	iBOT	768
`Phikon-v2`	ViT-L/16	DINOv2	1024
`Hibou-L`	ViT-L/14	DINOv2	1024
`GPFM`	ViT-L	Distillation	1024
`H0-mini`	ViT-B/14	Distillation	768
`Midnight-12k`	ViT-L/14	DINOv2	1536
`OpenMidnight`	ViT-L/14	DINOv2	1536
`Kaiko-ViT-B/8`	ViT-B/8	DINO	768
`Kaiko-ViT-B/16`	ViT-B/16	DINO	768
`Kaiko-ViT-S/8`	ViT-S/8	DINO	384
`Kaiko-ViT-S/16`	ViT-S/16	DINO	384
`Kaiko-ViT-L/14`	ViT-L/14	DINOv2	1024
`Lunit-ViT-S/8`	ViT-S/8	DINO	384
`GenBio-PathFM`	ViT-L	SSL	4608
`CTransPath`	CNN+Swin-T	SRCL Contrastive	768

16 Patch-Level Benchmark Datasets

A rigorous collection of public cohorts spanning multiple organs, magnifications, and downstream task classes.

Composition of the benchmark across eight anatomical sites (lung, breast, stomach, lymph node, colon, colorectum, kidney, and prostate) together with a pan-cancer cohort, illustrating the included datasets and their corresponding organs.

Dataset	Source / Institution	Description	Link
Tumor Detection
LC25000	University of South Florida / TampaPath	Created from a small collection of pathology images that were extensively augmented to produce 25,000 images across five tissue classes.	Link
BreakHis	Federal University of Paraná (UFPR), Brazil	Histopathology microscopy dataset collected from 82 patients at four magnifications (40×, 100×, 200×, and 400×), with expert-confirmed benign and malignant diagnoses.	Link
PCAM (PatchCamelyon)	CAMELYON16 Grand Challenge (Radboud UMC and UMC Utrecht)	96×96 image patches extracted from whole-slide images with binary labels indicating metastatic tissue.	Link
DigestPath	DigestPath Grand Challenge (2019)	Public benchmark consisting of pathology images collected for gastrointestinal pathology tasks, including benign and malignant lesion classification.	Link
Molecular Status Prediction
Kather-MSI-STAD	University Hospital Heidelberg	Clinical cohort assembled from TCGA and additional institutional data for predicting microsatellite instability directly from H&E slides.	Link
Kather-MSI-CRC	University Hospital Heidelberg	Clinical and TCGA-derived colorectal cancer cohort developed for microsatellite instability prediction from histopathology.	Link
Immune / TIL Assessment
CCRCC-Lymphocyte	TCGA-KIRC derived research dataset	Image patches extracted from TCGA whole-slide images and annotated for lymphocyte infiltration assessment.	Link
TCGA-TILs	The Cancer Genome Atlas (TCGA)	Pan-cancer whole-slide image collection with tumor-infiltrating lymphocyte (TIL) annotations for immune profiling studies.	Link
Tissue Typing
WSSS4LUAD	WSSS4LUAD Grand Challenge; Guangdong Provincial People's Hospital (GDPH) and TCGA	Weakly supervised tissue segmentation benchmark. Training uses patch-level labels, while validation and test pixel-level annotations were generated through a pathologist-in-the-loop annotation pipeline.	Link
CHAOYANG	Beijing Chaoyang Hospital	Clinical pathology image dataset manually annotated by expert pathologists for tissue classification.	Link
NCT-CRC-HE-100K	National Center for Tumor Diseases (NCT) Biobank and University Medical Center Mannheim	100,000 manually annotated tissue patches extracted from 86 whole-slide images and categorized into nine tissue classes.	Link
NCT-CRC-VAL-HE-7K	National Center for Tumor Diseases (NCT), Heidelberg	Independent validation cohort created using the same extraction and annotation protocol as NCT-CRC-HE-100K.	Link
CCRCC-Tissue	TCGA-KIRC derived research dataset	Whole-slide image patches extracted from TCGA and manually assigned tissue-type labels including tumor, stroma, and necrosis.	Link
Tumor Subtyping / Grading
MHIST	Dartmouth-Hitchcock Medical Center	Curated clinical dataset with expert-consensus annotations distinguishing hyperplastic polyps from sessile serrated adenomas.	Link
SICAPv2	Hospital de Braga and University of Minho, Portugal	Expert-annotated prostate biopsy whole-slide images with Gleason grading for automated pathology analysis.	Link

Dataset Classes

Dataset	Class Labels	Description
Tumor Detection
LC25000-LUNG	Adenocarcinoma (ACA), Benign Lung (N), Squamous Cell Carcinoma (SCC)	Task: Lung tissue classification N: Benign lung parenchyma ACA: Glandular epithelial carcinoma SCC: Squamous epithelial carcinoma.
LC25000-COLON	Adenocarcinoma (ACA), Benign Colon (N)	Task: Benign vs. malignant colon tissue N: Normal colonic mucosa ACA: Invasive colorectal adenocarcinoma.
BreakHis	Benign, Malignant	Task: Breast tumor classification Benign: Non-invasive breast lesions Malignant: Invasive breast carcinomas.
PCAM	Normal, Tumor	Task: Lymph node metastasis detection Normal: Healthy lymphoid tissue Tumor: Metastatic breast carcinoma.
DigestPath	Non-cancerous, Cancerous	Task: Digestive-system cancer detection Non-cancerous: Benign tissue Cancerous: Malignant epithelial tissue.
Molecular Status Prediction
Kather-MSI-STAD	MSS, MSI	Task: MSI prediction in gastric cancer MSI: Deficient DNA mismatch repair (dMMR) MSS: Intact mismatch repair.
Kather-MSI-CRC	MSS, MSI	Task: MSI prediction in colorectal cancer MSI: Associated with immunotherapy response MSS: Microsatellite stable tumors.
Immune / TIL Assessment
CCRCC-Lymphocyte	Blood, Lymphocyte, Tumor	Task: Immune composition analysis Blood: Blood-rich regions Lymphocyte: Immune-cell infiltrates Tumor: Clear cell renal carcinoma.
TCGA-TILs	Negative, Positive	Task: Tumor-infiltrating lymphocyte (TIL) detection Positive: Detectable immune infiltration Negative: Minimal or absent TILs.
Tissue Typing
WSSS4LUAD	Normal, Stroma, Tumor	Task: Lung tissue segmentation Normal: Pulmonary parenchyma Stroma: Connective tissue Tumor: Malignant epithelium.
CHAOYANG	Adenocarcinoma, Adenoma, Serrated, Normal	Task: Colorectal tissue classification Adenoma/Serrated: Premalignant lesions Adenocarcinoma: Malignant tissue Normal: Healthy mucosa.
NCT-CRC-HE-100K	ADI, BACK, DEB, LYM, MUC, MUS, NORM, STR, TUM	Task: Nine-class colorectal tissue recognition Classes: Adipose, background, debris, lymphocytes, mucus, muscle, normal mucosa, stroma, and tumor epithelium.
NCT-CRC-VAL-HE-7K	ADI, BACK, DEB, LYM, MUC, MUS, NORM, STR, TUM	Task: Independent validation cohort Classes: Same nine tissue categories as NCT-CRC-HE-100K.
CCRCC-Tissue	Empty, Cancer, Normal, Other, Stroma, Blood	Task: Renal tissue classification Classes: Background, tumor, normal kidney, stromal, blood, and miscellaneous tissue.
Tumor Subtyping / Grading
MHIST	Hyperplastic Polyp (HP), Sessile Serrated Adenoma (SSA)	Task: Colorectal polyp classification HP: Benign hyperplastic polyp SSA: Premalignant serrated lesion.
SICAPv2	Normal, Gleason Grade 3 (GG3), Gleason Grade 4 (GG4), Gleason Grade 5 (GG5)	Task: Prostate cancer grading GG3--GG5: Increasing Gleason grade and tumor aggressiveness Normal: Non-neoplastic prostate tissue.

Robustness analyses

Additional Results

We test whether INVEX's agreement with downstream performance depends on a single dataset, a particular task mix, or one encoder subfamily. Unless noted otherwise, values are Spearman correlations between the label-free INVEX ranking and the downstream oracle across the 22 visual encoders.

16/16 leave-one-dataset-out runs remain positively correlated

0.642–0.788 linear-oracle correlation after omitting one dataset

0.610–0.722 kNN-oracle correlation after omitting one dataset

Leave-one-dataset-out robustness

We recompute the mean downstream oracle after removing each benchmark dataset in turn. The full-benchmark correlations are $\rho=0.749$ for the linear oracle and $\rho=0.683$ for the $k$NN oracle. Across all 16 omissions, correlations remain positive and close to the headline result, showing that no single dataset determines the INVEX ranking.

Dataset omitted	Linear oracle	kNN oracle
None (full benchmark)	0.749	0.683
BreaKHis	0.709	0.658
CCRCC-Lymphocyte	0.642	0.685
CCRCC-Tissue	0.743	0.685
CHAOYANG	0.746	0.656
Kather-MSI-CRC	0.747	0.617
Kather-MSI-STAD	0.727	0.722
LC25000-COLON	0.743	0.671
LC25000-LUNG	0.744	0.684
MHIST	0.752	0.658
NCT-CRC	0.749	0.683
NCT-CRC-VAL-HE-7K	0.739	0.673
PCam	0.741	0.610
SICAPv2	0.788	0.719
TCGA-TILs	0.766	0.689
WSSS4LUAD	0.761	0.676
DigestPath	0.772	0.689

Interpretation

The association is not being created by one unusually influential benchmark. The largest linear decrease occurs when CCRCC-Lymphocyte is omitted ($0.749\rightarrow0.642$), while the largest $k$NN decrease occurs without PCam ($0.683\rightarrow0.610$); both correlations nevertheless remain substantial. Omitting SICAPv2 increases both coefficients, suggesting that some tasks introduce mild ranking tension rather than artificially producing the result. The overall relationship therefore reflects evidence shared across heterogeneous datasets.

Reliability and saturated-task analysis

Datasets are ordered by their across-encoder oracle-AUC spread. “Separated first” adds high-spread tasks, which distinguish models clearly; “saturated first” adds low-spread tasks, on which most encoders perform similarly. INVEX is already predictive with a few separated tasks, whereas saturated tasks provide a weak or unstable oracle until more diverse datasets are included. This isolates criterion reliability from sample count.

Datasets used	Separated first	Saturated first
2	0.799	0.650
4	0.837	0.108
6	0.778	−0.059
8	0.771	−0.008
10	0.764	0.462
12	0.749	0.401
14	0.749	0.598
16	0.749	0.749

Interpretation

Task informativeness matters more than task count alone. Just two well-separated datasets yield $\rho=0.799$, and the correlation remains between $0.749$ and $0.837$ as additional informative tasks are added. In contrast, six to eight saturated datasets give correlations near zero because their downstream scores barely distinguish the encoders. The two orderings converge at all 16 datasets by construction. Thus, INVEX agrees most clearly with downstream evaluation when the oracle itself contains enough performance variation to define a reliable rank.

Within-family and leave-one-family-out stability

We next restrict the analysis to the 20 pathology self-supervised encoders. Within-family correlations test whether INVEX preserves rankings among related architectures or training objectives; the small groups should be interpreted cautiously. Leave-one-subfamily-out analysis is the stronger stress test: most exclusions preserve a positive relationship, although estimates vary when removing a large family leaves a small or less diverse cohort.

Within-family correlation

INVEX versus each oracle inside the named subfamily.

Subfamily	n	Linear	kNN
ViT-S	3	0.500	0.500
ViT-B	4	0.400	0.400
ViT-L	7	0.679	0.321
ViT-g/H	6	−0.200	0.143
DINOv2	12	0.671	0.706
DINO	5	0.000	0.800

Leave-one-subfamily-out correlation

INVEX after removing every encoder in the named subfamily.

Removed	n	Linear	kNN
None	20	0.689	0.620
ViT-S	17	0.721	0.620
ViT-B	16	0.688	0.571
ViT-L	13	0.577	0.753
ViT-g/H	14	0.468	0.182
DINOv2	8	0.429	0.833
DINO	15	0.704	0.700

Interpretation. The DINOv2 subset ($n=12$) provides the most informative within-family estimate and remains predictive for both oracles ($\rho=0.671$ linear; $0.706$ $k$NN). Every leave-one-subfamily-out coefficient remains positive, so no single family is solely responsible for the relationship. However, removing ViT-g/H reduces the $k$NN correlation from $0.620$ to $0.182$, while removing DINOv2 leaves only eight encoders and produces divergent oracle correlations ($0.429$ versus $0.833$). These results support broad family robustness, but not complete architecture independence; estimates for the smallest subgroups should be treated as descriptive rather than definitive.