Camelyon¶
20,400 breast lymph-node tiles, labelled tumour or normal, contributed by two medical
centers (RUMC and UMCU) and scored at k = 11. Scored entirely
outside TCGA, so no encoder holds an in-distribution advantage from its pretraining corpus
— the cleanest of the three cohorts to read, and the most discriminating:
7 pathology encoders score below zero, meaning their
typical neighbourhood is closer to a different-biology tile from the same center than to
a same-biology tile from another.
Model |
bio bacc |
conf bacc |
|
|
|
F(0) |
LTM₁₀ |
support |
|---|---|---|---|---|---|---|---|---|
RudolfV-2-S |
0.984 |
0.823 |
0.932 |
0.940 |
0.32 |
0.047 |
-0.02 |
58.8% |
Mascaret |
0.981 |
0.849 |
0.909 |
0.914 |
0.29 |
0.041 |
-0.02 |
72.7% |
RudolfV-2 |
0.989 |
0.915 |
0.895 |
0.903 |
0.24 |
0.065 |
-0.04 |
38.3% |
RudolfV-2-B |
0.987 |
0.921 |
0.878 |
0.888 |
0.24 |
0.071 |
-0.05 |
38.3% |
Virchow2 |
0.988 |
0.958 |
0.806 |
0.823 |
0.20 |
0.129 |
-0.11 |
31.4% |
CONCH |
0.971 |
0.956 |
0.662 |
0.626 |
0.20 |
0.225 |
-0.20 |
35.9% |
GenBio-PathFM |
0.983 |
0.928 |
0.842 |
0.850 |
0.19 |
0.092 |
-0.07 |
38.3% |
CONCHv1.5 |
0.971 |
0.915 |
0.774 |
0.763 |
0.19 |
0.174 |
-0.14 |
46.3% |
H0-mini |
0.969 |
0.927 |
0.741 |
0.718 |
0.17 |
0.180 |
-0.16 |
38.7% |
Virchow |
0.980 |
0.946 |
0.751 |
0.708 |
0.16 |
0.221 |
-0.18 |
26.4% |
Phaet |
0.967 |
0.943 |
0.708 |
0.686 |
0.11 |
0.219 |
-0.18 |
48.4% |
Midnight-12k |
0.976 |
0.984 |
0.478 |
0.408 |
0.11 |
0.354 |
-0.35 |
19.8% |
H-optimus-1 |
0.986 |
0.978 |
0.664 |
0.677 |
0.08 |
0.219 |
-0.14 |
17.2% |
DINOv2-B † |
0.919 |
0.912 |
0.561 |
0.507 |
0.05 |
0.345 |
-0.18 |
68.0% |
H-optimus-0 |
0.982 |
0.966 |
0.659 |
0.652 |
0.05 |
0.315 |
-0.15 |
23.7% |
UNI2-h |
0.986 |
0.987 |
0.515 |
0.548 |
0.04 |
0.370 |
-0.21 |
13.0% |
MUSK |
0.958 |
0.983 |
0.366 |
0.297 |
0.04 |
0.403 |
-0.22 |
28.0% |
mSTAR |
0.979 |
0.984 |
0.460 |
0.434 |
0.02 |
0.418 |
-0.18 |
18.0% |
Prov-GigaPath |
0.979 |
0.991 |
0.375 |
0.369 |
0.01 |
0.470 |
-0.19 |
14.4% |
UNI |
0.982 |
0.999 |
0.108 |
0.092 |
-0.03 |
0.651 |
-0.22 |
9.6% |
Hibou-B |
0.973 |
0.999 |
0.057 |
0.041 |
-0.09 |
0.737 |
-0.36 |
13.4% |
GPFM |
0.955 |
0.999 |
0.034 |
0.017 |
-0.10 |
0.753 |
-0.36 |
20.9% |
Phikon |
0.955 |
1.000 |
0.009 |
0.004 |
-0.20 |
0.905 |
-0.48 |
16.6% |
Phikon-v2 |
0.954 |
1.000 |
0.019 |
0.008 |
-0.21 |
0.932 |
-0.50 |
16.9% |
Prost40M |
0.926 |
1.000 |
0.015 |
0.002 |
-0.32 |
0.922 |
-0.64 |
27.2% |
Hibou-L |
0.971 |
1.000 |
0.013 |
0.001 |
-0.44 |
0.993 |
-0.66 |
12.1% |
Read the support column carefully here. Two biological classes across two centers is a
sparse neighbourhood: no encoder’s support fraction clears
73%, and the floor is
10%. A high RI over that little evidence is not the
same claim as one over TCGA-4×4’s near-total support — the same two
indices, resting on very different amounts of evidence.
The two rankings, on this cohort alone:
Median CRoMa against tail severity LTM₁₀ on Camelyon. Better is up and to the
right; ringed points are undominated on both axes and named, and the shaded region is
dominated on both. Hover or tab to any point to name it with its two values. The
natural-image control is excluded — the frontier is a pathology-only claim.
The distribution explorer¶
The same explorer as the aggregate page’s, pinned to Camelyon. Click a row to move the detail, drag across the detail curve to count the samples in any range, and pick a second encoder under Compare with to overlay its shape.
The shape says more than the median. Virchow2 and CONCH sit within
0.002 of each other on median CRoMa —
indistinguishable on that column alone — while CONCH carries
1.7× the confounder-dominant mass
and 1.9× the tail severity.
Overlay the two above to see it.
Shortcut susceptibility¶
Shortcut susceptibility for every encoder on this cohort, in domain (ID) and out of
domain (OOD); the natural-image control sits last. Rows are ranked by Change at V = 1,
the normalized change at maximum confounding, because nIPD averages over the whole
range: an early gain there can pay for a late collapse, so a curve ending at chance can
outrank one that never moved. Rows ending at or below -0.900 are marked ≈ chance,
where none of the above-chance margin survives.
Bold marks the leading ranked encoder in each column where higher is better, so a
column that disagrees with the ranking shows it at a glance. Each caption reports
Spearman ρ, the rank correlation between the CRoMa and nIPD columns: how closely
the two order the encoders the same way.
Shortcut susceptibility defines the measure and holds the interactive explorer.
Model |
Median CRoMa (m=5) |
Change at |
|
Baseline balanced accuracy |
|---|---|---|---|---|
RudolfV-2-S |
0.324 |
-0.074 |
-0.008 |
0.986 |
Mascaret |
0.285 |
-0.087 |
-0.007 |
0.974 |
RudolfV-2 |
0.243 |
-0.116 |
-0.020 |
0.991 |
RudolfV-2-B |
0.235 |
-0.127 |
-0.020 |
0.988 |
Virchow |
0.155 |
-0.228 |
-0.028 |
0.982 |
Virchow2 |
0.199 |
-0.234 |
-0.041 |
0.987 |
CONCHv1.5 |
0.187 |
-0.250 |
-0.032 |
0.969 |
H0-mini |
0.167 |
-0.296 |
-0.036 |
0.980 |
GenBio-PathFM |
0.191 |
-0.304 |
-0.044 |
0.986 |
CONCH |
0.196 |
-0.367 |
-0.043 |
0.971 |
Phaet |
0.111 |
-0.378 |
-0.064 |
0.971 |
H-optimus-1 |
0.082 |
-0.417 |
-0.072 |
0.991 |
Midnight-12k |
0.108 |
-0.436 |
-0.075 |
0.978 |
UNI2-h |
0.045 |
-0.446 |
-0.084 |
0.986 |
MUSK |
0.042 |
-0.548 |
-0.087 |
0.964 |
Prov-GigaPath |
0.009 |
-0.647 |
-0.121 |
0.981 |
H-optimus-0 |
0.045 |
-0.650 |
-0.110 |
0.982 |
mSTAR |
0.022 |
-0.714 |
-0.153 |
0.984 |
GPFM |
-0.104 |
-0.772 |
-0.158 |
0.968 |
Phikon |
-0.196 |
-0.859 |
-0.147 |
0.960 |
UNI |
-0.034 |
-0.884 |
-0.176 |
0.983 |
Hibou-B |
-0.089 |
-0.895 |
-0.193 |
0.967 |
Prost40M |
-0.318 |
-0.910 ≈ chance |
-0.145 |
0.927 |
Hibou-L |
-0.443 |
-0.949 ≈ chance |
-0.183 |
0.958 |
Phikon-v2 |
-0.208 |
-0.971 ≈ chance |
-0.199 |
0.970 |
DINOv2-B † |
0.050 |
-0.730 |
-0.111 |
0.936 |
Model |
Median CRoMa (m=5) |
Change at |
|
Baseline balanced accuracy |
|---|---|---|---|---|
RudolfV-2 |
0.243 |
-0.002 |
0.001 |
0.996 |
RudolfV-2-B |
0.235 |
-0.002 |
-0.000 |
0.996 |
RudolfV-2-S |
0.324 |
-0.015 |
0.001 |
0.996 |
Virchow2 |
0.199 |
-0.027 |
-0.006 |
0.995 |
Mascaret |
0.285 |
-0.038 |
-0.003 |
0.988 |
Midnight-12k |
0.108 |
-0.042 |
0.002 |
0.975 |
Virchow |
0.155 |
-0.045 |
0.000 |
0.984 |
H-optimus-0 |
0.045 |
-0.079 |
-0.008 |
0.983 |
GenBio-PathFM |
0.191 |
-0.115 |
-0.012 |
0.994 |
H0-mini |
0.167 |
-0.117 |
-0.007 |
0.986 |
CONCHv1.5 |
0.187 |
-0.125 |
-0.012 |
0.979 |
H-optimus-1 |
0.082 |
-0.163 |
-0.020 |
0.995 |
Prov-GigaPath |
0.009 |
-0.202 |
-0.022 |
0.983 |
UNI2-h |
0.045 |
-0.203 |
-0.012 |
0.985 |
Phaet |
0.111 |
-0.207 |
-0.020 |
0.977 |
CONCH |
0.196 |
-0.232 |
-0.018 |
0.983 |
MUSK |
0.042 |
-0.322 |
-0.047 |
0.976 |
Phikon |
-0.196 |
-0.327 |
-0.056 |
0.955 |
mSTAR |
0.022 |
-0.328 |
-0.053 |
0.986 |
UNI |
-0.034 |
-0.336 |
-0.037 |
0.985 |
Hibou-B |
-0.089 |
-0.354 |
-0.091 |
0.964 |
Hibou-L |
-0.443 |
-0.369 |
-0.011 |
0.944 |
GPFM |
-0.104 |
-0.381 |
-0.045 |
0.966 |
Prost40M |
-0.318 |
-0.538 |
-0.047 |
0.894 |
Phikon-v2 |
-0.208 |
-0.744 |
-0.171 |
0.962 |
DINOv2-B † |
0.050 |
-0.759 |
-0.139 |
0.940 |