PGVL-Gym
A configuration-first framework for systematic, reproducible evaluation of whole-slide pathology vision-language models.
The benchmark contract¶
PGVL-Gym turns every experiment into the same explicit contract: a method adapter, a dataset protocol, and traceable feature provenance. The result is a comparison you can explain, reproduce, and extend.
Configure the task
Labels, prompts, folds, shots, resolutions, and paths stay in readable protocol files—not hidden in training code.
Configuration guideDeclare provenance
Every cached tensor records its encoder, feature space, level, and resolution so incompatible assets fail early.
Feature architectureCompare fairly
Shared seeds, patient-disjoint folds, identical shot counts, and aligned reporting keep the scoreboard honest.
Benchmark protocolsAudit patch evidence
Checkpoint-bound heatmaps retain native score semantics, exact level-0 coordinates, target class, and run provenance.
Interpretability workflowFrom feature store to fair result¶
FOCUS, MAPLE, MSCPT, HiVE-MIL, MI-VisionShot, Libra-MIL, DyKo, MGPATH, HIPSS, MUSE, PathPT, ViLa-MIL, CoD-MIL, TOP, SLIP, WSI-FiVE, ConVLM, SLDPC, and composite variants all enter through the same registry. The generated protocols currently cover TCGA-NSCLC, TCGA-BRCA, TCGA-RCC, CAMELYON16, and UBC-OCEAN.
Model fidelity is part of fairness
The shared interface standardizes loading, validation, and orchestration. It never silently redesigns a paper architecture. Fixed and allowlisted encoder boundaries remain explicit—even when another encoder has the same output dimension.
Evidence semantics are part of fidelity
Eleven adapters expose audited patch-level quantities. PGVL-Gym retains their exact meanings rather than labeling every visualization as attention; unsupported methods refuse generation until a faithful mapping exists.
Start with a validated configuration.
Build a smoke-tested run before committing expensive GPU hours.