Method support matrix¶
Every adapter declares its input level and encoder swap policy. This table describes architectural compatibility; it does not imply that every required feature or checkpoint is currently present.
| Method | Input contract | Swap policy | Supported encoder boundary |
|---|---|---|---|
| Composite | Composite patch/text pipeline | Capability | Paired tile-text and text encoding capabilities |
| FOCUS | Single high-resolution patch bag | Allowlist | Native CONCH; explicit paired ports for CONCH, KEEP, MUSK, PLIP, or QuiltNet-B-16 |
| ViLa-MIL | Dual-scale patch bags | Allowlist | Native CLIP-RN50; explicit paired ports for CONCH, KEEP, MUSK, PLIP, or QuiltNet-B-16 |
| CoD-MIL | Dual-scale patch bags plus map | Precomputed | CLIP-RN50, PLIP, or QuiltNet-B-16 extension; the upstream QuiltNet-B-32 tensor is audit-only, and prompt/patch banks must share one exact verified feature space |
| MAPLE | Dual-scale patch bags | Allowlist | PLIP or HF CLIP ViT-B shared features, or their named 768-wide vision-preprojection boundary followed by the same encoder's frozen native visual projection |
| MSCPT | Dual-scale patch bags | Allowlist | PLIP, HF CLIP ViT-B, CONCH |
| PathPT | Patch bag | Allowlist | PLIP, CONCH, KEEP, MUSK |
| TOP | Patch bag | Allowlist | Native CLIP-RN50; explicit paired ports for CONCH, KEEP, MUSK, PLIP, or QuiltNet-B-16 |
| SLIP | Patch bag | Allowlist | CLIP ViT-B, CLIP-RN50, PLIP, BiomedCLIP |
| WSI-FiVE | Precomputed patch sequence; native train-time answer bank | Precomputed | Native: DSMIL SimCLR-ResNet18 512-wide. BioClinicalBERT 20x feature extensions: CONCH v1, QuiltNet-B-16, PLIP, KEEP, CONCH v1.5, CLIP-RN50, and MUSK. Separate paired-tower ablations use exact frozen CONCH v1, QuiltNet-B-16, CLIP-RN50, PLIP, KEEP, or MUSK text towers with trainable native soft prompts and fusion; PLIP restores its frozen native visual projection. Paired CLIP-RN50 uses its native 77-token EOT-preserving truncation for overlength answer candidates. All extensions remain partial. |
| MUSE | Patch bag | Capability | Any declared patch space with a registered text encoder and learned visual adapter. Native MUSE keeps the 16-token CONCH learner; explicit text-tower extensions use frozen text features plus a trainable feature-space context delta and must carry encoder_extension provenance. |
| ConVLM | Local attribute-conditioned patch-bag reconstruction | Precomputed | Any declared patch bags plus metadata-bound attribute vectors; not the released raw-image training path |
| SLDPC | Slide embedding | Capability | Soft-prompt and text-encoding prompt tower; slide source declared separately. TITAN retains its native projector, while a different tower requires an explicit linear/MLP slide adapter and extension provenance. OpenAI CLIP-RN50 and CLIP ViT-B/16 are the first built-in non-TITAN promptable towers exposed through this interface. |
| HiVE-MIL | Coordinate-aligned 5x/20x hierarchy ([N,D] + [N,16,D]) |
Fixed | CONCH v1 shared features plus the same native CONCH soft-prompt text tower |
| MI-VisionShot | Single-scale patch bag; training-free support prototypes | Fixed | PLIP 20x features and the matching PLIP text tower/native visual projection |
| Libra-MIL | Single-scale patch bag plus instance and bag prompt banks | Allowlist | Native CONCH; exact paired CONCH/KEEP/MUSK/PLIP/QuiltNet re-encoding requires extension provenance |
| DyKo | Single-scale patch bag plus released concept tensor | Allowlist | Native TITAN/CONCH-v1.5; paired CONCH/KEEP/MUSK/PLIP/QuiltNet ports require feature context plus a learned bridge from the fixed TITAN concept bank |
| MGPATH | Coordinate-bearing dual-scale patch bags and spatial graphs | Allowlist | Native PLIP preprojection; paired CONCH/KEEP/MUSK/QuiltNet ports consume final shared features and replace token prompt views with feature-space views |
| HIPSS | Coordinate-grouped region/patch hierarchy | Allowlist | Native CONCH block-level SSF; paired CONCH/KEEP/MUSK/PLIP/QuiltNet ports replace it with final-feature affine SSF |
Query the live contracts at any time:
python scripts/list_backbone_compatibility.py
python scripts/list_backbone_compatibility.py --json
python scripts/list_encoder_swaps.py
python scripts/list_encoder_swaps.py --method muse --json
The backbone report describes each adapter's primary runtime boundary. The role-aware encoder-swap report is stricter: it lists feature and prompt encoders separately, names the implemented alignment modes, and marks methods that can accept a capability-compatible registered plugin. A paired-only row never permits a cross-checkpoint feature/text combination merely because both tensors have the same width.
Patch-level interpretability¶
Interpretability is an adapter capability, not a claim that every model emits
the same kind of attention. scripts/generate_heatmap.py preserves the native
score name, exact feature source, level-0 coordinate frame, checkpoint hash,
and selected class in a JSON/CSV/PNG artifact set.
| Method | Audited patch signal | Class-specific | Scales |
|---|---|---|---|
| FOCUS | Class-query cross-attention after selection/compression | Yes | Active high-resolution bag |
| ViLa-MIL | Class-prompt context cross-attention, excluding learned prototype keys | Yes | Low and high |
| CoD-MIL | Predicted-class prompt evidence used for high-scale selection | Predicted class only | Low |
| PathPT | Native patch-class probability (upstream_patch_ssl only) |
Yes | Patch bag |
| TOP | Mean instance-prototype pooling attention for attention-producing pooling variants | No | Patch bag |
| SLIP | Patch-to-tissue-to-slide routing evidence | Yes | Patch bag |
| MUSE | Label-free class-semantic expert-routing attention | Yes | Patch bag |
| Libra-MIL | Multimodal transport pooling attention | No | Patch bag |
| DyKo | Visual prompt-to-patch cross-attention | Yes when prompt rows align with classes | Patch bag |
| MGPATH | Learned-image-center pooling attention | No | Low and high |
| HIPSS | Region attention multiplied by within-region patch attention | No; follows native guidance rule | Patch bag |
MAPLE, MSCPT, WSI-FiVE, ConVLM, SLDPC, HiVE-MIL, MI-VisionShot, and Composite currently refuse heatmap generation because their registered path does not yet expose an audited patch-aligned native score. A future gradient or perturbation map must be registered as an explicit post-hoc extension, not relabeled as paper-native attention.
Opt-in paired-VLM architecture ports¶
The standard protocol rows remain native and were not rewritten. A new row is
generated only when its authored experiment sets encoder_extension: true,
uses one exact paired feature/text checkpoint, and selects the registered
strategy below. These are architecture extensions and must not be pooled with
native results.
| Method | Preserved computation | Explicit replacement |
|---|---|---|
| FOCUS | token selection, spatial compression, cross-attention, classifier, loss | CONCH 16-token learner -> shared zero-initialized final-feature context |
| ViLa-MIL | low/high ordering, centers, dual-scale attention and loss | RN50 16-token learner -> shared final-feature context |
| TOP | instance/bag roles, prototype routing, pooling objective and learning rates | RN50 token contexts -> two role-specific feature contexts; width-dependent layers resize to the paired space |
| MGPATH | spatial graph, 64 centers, four views, aggregation and Sinkhorn score | PLIP preprojection/head -> final shared bags; four token views -> four feature-context views |
| Libra-MIL | both prompt roles, visual/text prototypes, attention, SOT, loss and optimizer | only prompt re-encoding and shared width change; no learned cross-space bridge |
| DyKo | WAKI, clustering/retrieval, dual attention and structural consistency | TITAN token context -> paired feature context; fixed 1000x768 TITAN concepts -> trainable linear bridge |
| HIPSS | region/WSI hierarchy, relevance guidance, attention and loss | affine sites inside CONCH blocks -> identity-initialized affine adaptation of final text features |
Every port computes in the paired tower's final shared feature space. PLIP's
available 768-wide vision-preprojection caches first pass through that exact
checkpoint's frozen native 768-to-512 visual projection; QuiltNet-B-16 and the
other registered stores are already in their final shared spaces. Vision
features from KEEP cannot be crossed with MUSK text, and a private tensor with
the same width as KEEP is rejected by feature_space_id.
QuiltNet-B-16 bags are complete at 5x/10x/20x with 224px tiles for TCGA-NSCLC, CAMELYON16, and UBC-OCEAN. TCGA-RCC has complete 10x bags but still needs 5x and 20x; TCGA-BRCA needs all three scales. The combined CAMELYON16+17 cohort still needs its current 20x/256px geometry for all three source partitions. Until an exact store is registered and validated, its QuiltNet port remains unavailable rather than substituting another encoder, scale, or patch grid.
Protocol experiment variants¶
The standard protocols expand 19 method families into separately named, provenance-bearing variants:
| Family | Registered variants |
|---|---|
| PathPT | CONCH, KEEP, and MUSK 20x rows where those exact shared stores exist. PLIP remains architecturally supported but is not emitted because the available 768-wide vision-preprojection bags do not satisfy PathPT's 512-wide shared-feature boundary. |
| MUSE | Native CONCH prompting plus independent cached-feature/CONCH-text rows; paired CLIP-RN50, PLIP, QuiltNet-B-16, KEEP, and MUSK text-tower extensions are registered only where the exact feature store and checkpoint exist. The combined CAMELYON16+17 CONCH 20x/256px condition remains separate. |
| FOCUS | Native CONCH 20x plus paired KEEP/MUSK extensions where available, PLIP on NSCLC/BRCA/RCC, and QuiltNet-B-16 on NSCLC/CAMELYON16/UBC-OCEAN; the accepted low-scale argument remains unused. PLIP restores its frozen native visual projection. The combined CAMELYON16+17 row retains the native bank and discloses 20x/256px geometry and patient-grouped folds. |
| MSCPT | CONCH 10x/20x and 5x/20x |
| MAPLE | TCGA-NSCLC PLIP 10x/20x and 5x/20x release-code variants, plus a separately labeled 5x/10x paper-text recipe; cached 768-wide bags are projected by PLIP's frozen native visual head into its 512-wide shared space |
| ViLa-MIL | Native CLIP-RN50 5x/10x plus paired CONCH 5x/10x on every registered task, KEEP/MUSK where both scales are complete, QuiltNet-B-16 on NSCLC/CAMELYON16/UBC-OCEAN, and PLIP on NSCLC/BRCA. RCC has neither a PLIP pair nor a complete QuiltNet 5x/10x pair, so no substitute is generated. |
| CoD-MIL | CLIP-RN50 10x/20x plus cross-scale map in the standard protocol; CLIP-RN50 uses native-scale text features while PLIP/QuiltNet use their released unit-scale convention. PLIP/QuiltNet are supported when matching dual-scale bags, maps, and verified prompt tensors are supplied |
| TOP | CLIP-RN50 native rows plus separately named CONCH/KEEP/MUSK ports; PLIP is registered on NSCLC/BRCA/RCC and QuiltNet-B-16 on NSCLC/CAMELYON16/RCC/UBC-OCEAN. RCC's PLIP row is explicitly 20x because no 10x PLIP store exists, while its QuiltNet row uses the complete 10x store. |
| SLIP | CLIP-RN50 10x fidelity-oriented TCGA rows; separately named NSCLC and BRCA PLIP 10x extensions restore 768-wide cached features through PLIP's frozen native visual projection |
| WSI-FiVE | NSCLC native DSMIL 512-d; seven BioClinicalBERT cached-VLM feature extensions; six paired-text-tower ablations (CONCH v1, QuiltNet-B-16, CLIP-RN50, PLIP, KEEP, MUSK). CAMELYON keeps the released DSMIL transfer row and adds BioClinicalBERT feature swaps for CONCH, CLIP-RN50, and KEEP; paired CAMELYON towers are intentionally absent because the released transfer checkpoint is BioClinicalBERT-specific. |
| SLDPC | Native TITAN 20x slide embeddings/prompt tower plus a separately named CLIP-RN50 prompt-tower extension with an explicit trainable slide-to-text linear projection |
| ConVLM | NSCLC local QuiltNet-B-16 20x patch-bag reconstruction with runtime-encoded audited generated attributes and a 512-patch cap; no upstream bank or att_splits.mat is released, so the row is partial |
| HiVE-MIL | CONCH v1 5x/20x on TCGA-BRCA, NSCLC, and RCC; hierarchy compiled from Trident coordinates and therefore explicitly partial |
| MI-VisionShot | PLIP 20x on TCGA-RCC; separate released-code unnormalized and paper-Eq.-2.2 per-patch-L2 rows, both with top-200 support prototypes and label-free BGAP inference; current tile/fold/cohort differences are explicitly partial |
| Libra-MIL | Native CONCH v1 plus paired KEEP and PLIP 20x rows on TCGA-RCC; all retain 46 text prototypes, 10 visual prototypes, SOT fusion, and three class queries |
| DyKo | Native CONCH-v1.5 20x bags plus TITAN text/concept knowledge on NSCLC/RCC; paired CONCH/KEEP ports are registered on both, MUSK and QuiltNet-B-16 on NSCLC, and PLIP on both, with an explicit learned bridge from the fixed TITAN concept bank |
| MGPATH | Paper-authoritative PLIP 5x/10x on BRCA/NSCLC plus paired CONCH on both and paired KEEP/MUSK/QuiltNet-B-16 on NSCLC; every port retains four views, coordinate graphs, 64 centers, and Sinkhorn scoring |
| HIPSS | Native coordinate-grouped CONCH bags on CAMELYON16, NSCLC, and UBC-OCEAN plus paired KEEP and QuiltNet-B-16 on all three, and MUSK/PLIP on NSCLC; ports use disclosed final-feature affine SSF rather than claiming upstream CONCH-block SSF |
| Composite | CLIP-RN50 10x classname baseline |
FOCUS prompt fidelity is upstream for TCGA-NSCLC, UBC-OCEAN, CAMELYON16, and the combined CAMELYON16+17 matrix, whose native positional CSVs are byte-exact copies. TCGA-BRCA and TCGA-RCC use generated native-format extensions because the release contains no matching banks. UBC's released file order is explicitly rebound to benchmark order. The combined row is still partial due to its 898-slide operational universe, 20x/256px feature geometry, and frozen patient-disjoint rather than released random slide-level folds.
ViLa-MIL prompt fidelity is task-specific: TCGA-NSCLC and TCGA-RCC select
byte-exact released native CSVs, while TCGA-BRCA, UBC-OCEAN, and CAMELYON16 use
generated native-format task extensions. The RCC copy preserves upstream's
CRCC text while explicitly binding that positional slot to CHRCC.
The runtime corrects a released prompt-layout defect: the 16 learned context
embeddings and the end-of-text index are now derived from the same tokenized
sequence. This correction and the mandatory 5x/10x scale recipe make
implementation fidelity partial, independently of prompt-bank provenance.
MSCPT's native NSCLC, RCC, and UBC-OCEAN conditions bind two distinct upstream
assets: the multiscale disease descriptions used by its prompt learners and the
50-set selector ensemble used to rank the low-magnification patch bag. The
feature-only adapter restores both text roles but cannot recreate the deep
visual prompting of selected RGB patches from final cached features. UBC's
separate mscpt_raw_rgb condition restores that native visual-prompt path from
50 manifest-ordered selector crops per slide; the deterministic crop compiler
and local patient-grouped folds remain explicitly partial. BRCA
IDC/ILC and CAMELYON16 are generated task extensions without a task-matched
upstream selector bank.
WSI-FiVE native NSCLC uses three hash-bound roles: six upstream questions in a
derived container, a complete 1,021-case answer bank containing 912 upstream
answers, 27 disclosed blank-cell completions, and 82 conservative cohort
extensions generated by openai:gpt-5.6, plus two upstream evaluation
descriptions in a derived container. The generated rows are not paper-authored.
An audit with CLIP's actual BPE tokenizer found 183 of the 1,021 full answer
candidates exceed its fixed 77-token window (maximum 144). The separately named
paired CLIP-RN50 extension therefore records clip_eot_truncate_77_v1 and uses
the checkpoint's native truncate=True path, which restores EOT at the final
position. This policy does not alter BioClinicalBERT, other paired towers, or
any candidate that already fits CLIP's context.
RCC and UBC use generated questions plus classname comparison and are explicitly
simplified conditions, not upstream prompt banks.
WSI-FiVE CAMELYON16 instead uses the released normal/tumor class bank and
does not load per-case report answers. Its released CAM configuration repeats
the lung-specific question list; PGVL-Gym preserves that exact wording and
records the issue rather than substituting a generated task bank.
Architectural compatibility is not implementation fidelity. Generated configs
and result tables separately carry implementation_provenance and
upstream_fidelity; partial integrations must not be presented as upstream
reproductions.
PathPT generated configs run the native patch-supervision mode: training-fold
prompt selection, synthetic Normal for subtype tasks, PatchSSLoss, and
Normal-excluding patch voting. Its older mean-patch slide-CE integration is
available as simplified_slide_ce but reports partial fidelity. CAMELYON is
also partial because its normal-vs-tumour slide labels require a disclosed
binary adaptation; prompt-bank provenance is an independent field.
All upstream-reference upstream_patch_ssl configurations use 20x patch
features. Other magnifications remain supported as explicitly labeled partial
experimental extensions. The epoch-10 pseudo-label loss follows the released
family-specific policy: enabled for CONCH and KEEP, disabled for PLIP and MUSK
because upstream reports unstable training for those two encoders.
What “supported” means¶
- Capability: any registered encoder bundle with the required operations may be used.
- Allowlist: only explicitly named architecture branches are implemented.
- Fixed: the paper implementation owns a specific representation boundary.
- Precomputed: runtime behavior assumes cached tensors from a declared feature space and may require auxiliary files.
Feature dimensions never establish compatibility on their own. A 512-wide CONCH tensor is not interchangeable with a 512-wide PLIP tensor, and an ImageNet ResNet representation is not an OpenAI CLIP-RN50 representation.
Readiness layers¶
Use three separate questions:
- Is the method/encoder pairing architecturally supported?
- Is the generated config valid under that contract?
- Are all metadata, features, prompts, weights, and auxiliaries available?
The compatibility command answers the first. config_audit.csv answers the
second. run_matrix.csv and feature coverage answer the third.