API

Use the Python API to extract features once, train downstream models, and inspect results. Start with Getting started for the complete pipeline; the recipes here cover custom orchestration.

Main building blocks

Page	Focus
Dataset and splits	CSV manifest schema and fold assignment rules
Preprocessing	Tissue segmentation and slide tiling at a given spacing
Encoders	Feature extraction backends
Aggregators	MIL pooling and bag-level aggregation
Tasks	Prediction heads and metric contracts
Evaluation	Metric contracts, subgroup analysis, predictions and evaluations
Training	Optimization behavior and training defaults

Extract once, cache, and reuse features across experiments

Extract a shared feature source, then train each downstream model in its own run directory. This example uses uni2; see Encoders for model access requirements.

from dataclasses import replace

import soma
from soma import Cohort, ExperimentConfig, ImageManifest, Pipeline
from soma.config import CacheOptions, CohortConfig, LearnerConfig, RunConfig, TrainerConfig

# The cohort holds identity, targets and folds; the image manifest holds the paths.
cohort = Cohort.from_csv("dataset.csv", "splits.csv", targets=["label"])
images = ImageManifest.from_csv("dataset.csv")
cache = CacheOptions(enabled=True, root_dir="shared/feature_cache")

# Features enter through soma.extract: it reads the image manifest and returns the source.
features = soma.extract(images, encoder="uni2", cache=cache, output_dir="output")

def experiment(aggregator: dict) -> ExperimentConfig:
    return ExperimentConfig(
        run=RunConfig(output_dir="output"),
        cohort=CohortConfig(records="dataset.csv", splits="splits.csv"),
        learner=LearnerConfig(
            "mil", {"aggregator": aggregator, "head": "binary_classification"}
        ),
        trainer=TrainerConfig(max_epochs=50),
    )

abmil = experiment({"name": "abmil", "params": {"hidden_dim": 256}})
clam = experiment({"name": "clam_sb", "params": {"hidden_dim": 256, "attn_dim": 128}})

abmil_result = Pipeline(abmil, source=features).run()
clam_result = Pipeline(clam, source=features).run()

Each config trains in its own run directory under run.output_dir; the source is attached, never re-extracted.

soma.extract is the single way features enter soma; a Pipeline without a source extracts through it too. It is also the namespace of soma.extract.live_encoder().

soma.extract.extract(images, *, encoder, cache, annotations=None, unit='slide', rois=None, execution=None, extraction=None, dense=None, spacing_policy='strict', output_dir)

Extract (or reuse from the feature cache) encoder’s features of images.

Parameters:

• images (ImageManifest) – The samples’ pixels: one row per whole slide (unit="slide") or per pre-cropped image (unit="tile"). The cache keys each sample on its row.

• encoder (EncoderLeaf | str) – A slide2vec encoder name or {name, params} leaf.

• cache (CacheOptions) – The feature and tiling cache policy (CacheOptions(enabled=False) writes under output_dir only).

• annotations (AnnotationManifest | None) – The annotation rasters ROI sampling reads (label_mask_path); needed only to sample ROIs from whole slides.

• unit (Literal['slide', 'tile']) – What one row is: a whole slide or a pre-cropped image.

• rois (RoiSet | None) – The ROIs of an earlier annotation-sampled extraction (its source’s rois): their grids are encoded again, without sampling. Their label crops were stored at the read spacing and window they were sampled at, so pass the same extraction, dense and spacing_policy; another geometry is refused.

• execution (ExecutionOptions | None) – slide2vec’s ExecutionOptions (batch size, precision, GPUs).

• extraction (PreprocessingConfig | None) – slide2vec’s PreprocessingConfig (tiling, segmentation, masks); unset fields come from the encoder’s recommendation.

• dense (DenseOptions | None) – slide2vec’s DenseOptions: given, the features are token grids read and encoded at it; else one tensor per sample.

• spacing_policy (str) – How a grid run reads a sample coarser than dense.spacing_um (native_if_coarser reads it at its own spacing).

• output_dir (str | Path) – Where the run’s tiling, extraction record and uncached features go.

Returns:

A SetSource (CachedSetSource) or, with dense or rois, a GridSource (CachedGridSource). An annotation-sampled grid source is keyed by ROI id and carries the RoiSet as rois.

class soma.extract.RoiSet(entries, *, sampling=None)

Bases: object

The ROIs annotation sampling drew from a cohort’s whole slides.

An annotation-sampled soma.extract() returns a grid source keyed by ROI id whose rois is this set. It names each ROI’s parent sample (children, for soma.data.Cohort.derived()) and serves what is read per ROI: the label maps of the mask crops sampling stored (label_maps()) and each ROI’s window of its slide (pixels()). Pass it back as soma.extract(..., rois=...) to encode the same ROIs again (another encoder or recipe) without sampling.

sampling records the extraction the ROIs were sampled (and their mask crops stored) at, as soma.extract.cache.roi_sampling_signature gives it; a replay at another one is refused. A set built without it is not checked.

The source can be reused across experiments as long as the upstream dataset, preprocessing, and encoder settings do not change; Caching gives the rules.

Evaluation settings and attention heatmaps

Both live in the evaluation section. See Evaluation for metric contracts and Run outputs for the saved attention scores and overlays:

from soma.config import EvaluationConfig

evaluation = EvaluationConfig(
    metrics=["auroc", "balanced_accuracy", "f1"],
    subgroups=["center", "grade"],
    attention_heatmaps={"cmap": "coolwarm", "alpha": 0.5},
)

result = Pipeline(replace(abmil, evaluation=evaluation), source=features).run()

Dense features over given images

Dense extraction is an extraction setting, not a manifest kind: dense asks for token grids and unit="tile" says each row is a pre-cropped image. The grid recipe is slide2vec’s DenseOptions (read spacing, supervision size, padding, window, feature kind); batch size and precision are slide2vec’s ExecutionOptions:

from slide2vec import DenseOptions, ExecutionOptions

import soma
from soma import ImageManifest
from soma.config import CacheOptions

grids = soma.extract(
    ImageManifest.from_csv("segmentation.csv"),
    encoder="phikon",
    unit="tile",
    dense=DenseOptions(spacing_um=0.5, target_size=224),
    execution=ExecutionOptions(batch_size=64),
    cache=CacheOptions(enabled=True, root_dir="shared/feature_cache"),
    output_dir="output/dense",
)

grid = grids.load("roi_001")  # float32: (feature_dim, grid_h, grid_w)
geometry = grids.geometry("roi_001")

Annotation-sampled whole slides

A table of whole-slide image and annotation paths becomes an annotation-sampled grid extraction when the extraction carries an annotation vocabulary (hs2p’s MaskSamplingConfig) and a grid runs over whole slides. The grids are keyed by ROI id; their rois name each ROI’s slide, from which the slide cohort derives the ROI cohort (each ROI inherits its slide’s splits):

from hs2p import MaskSamplingConfig, SegmentationConfig, TilingConfig
from slide2vec import DenseOptions, PreprocessingConfig

import soma
from soma import AnnotationManifest, Cohort, ImageManifest
from soma.config import CacheOptions

grids = soma.extract(
    ImageManifest.from_csv("slides.csv"),
    annotations=AnnotationManifest.from_csv("slides.csv"),
    encoder="phikon",
    dense=DenseOptions(spacing_um=0.5, target_size=512),
    extraction=PreprocessingConfig(
        tiling=TilingConfig(
            requested_spacing_um=0.5, requested_tile_size_px=512, tolerance=0.05, overlap=0.0
        ),
        segmentation=SegmentationConfig(method="hsv"),
        masks=MaskSamplingConfig(
            pixel_mapping={"background": 0, "tumor": 1},
            min_coverage={"tumor": 0.1},
            strategy="joint",
            output_mode="merged",
        ),
    ),
    cache=CacheOptions(enabled=True, root_dir="shared/feature_cache"),
    output_dir="output/annotation-rois",
)

rois = Cohort.from_csv("slides.csv", "splits.csv").derived(grids.rois.children)
label_maps = grids.rois.label_maps(size=(512, 512))  # the stored mask crops

Slides with no sampled ROI have no child, so they leave the ROI cohort; output/annotation-rois/extraction_provenance.json lists them.

Score and compare saved predictions

Each test split’s predictions_<split>.parquet can be scored again, with other metrics or subgroups, and compared across runs. Given two single-fold runs over the same cohort:

from soma import evaluate
from soma.evaluation import Predictions, compare

targets = cohort.targets("test")
abmil = Predictions.from_parquet(abmil_result.run_dir / "predictions_test.parquet")
clam = Predictions.from_parquet(clam_result.run_dir / "predictions_test.parquet")

evaluation = evaluate(abmil, targets, metrics=["auroc", "f1"], subgroups=["center"])
p_values = compare([abmil, clam], targets, "auroc")

compare gives the best set 1.0 and every other set the p-value of a paired permutation test against it. See Evaluation for the scoring rules and Benchmarking for leaderboards across many runs.

soma.components() names the component kinds (aggregators, heads, decoders, encoders, optimizers, …); soma.components("aggregators") lists one kind’s registered components, each with its params and their defaults. See A custom component through the registry for adding your own.

Segmentation and detection building blocks

The verbs soma.learn.loaders, soma.fit, soma.predict and soma.evaluate also cover feature grids. The grid shape walks through a complete run; these are the parts:

Object	Role
soma.data.GridSource	Cached feature grids, one (D, h, w) grid per sample
soma.data.TileImageSource	Pixels of pre-cropped tiles for the live path; TileImageSource.from_manifest(image_manifest, size=...)
soma.data.LabelMapSource, soma.data.CachedLabelMapSource	Segmentation targets: label masks, or the ROI mask crops stored by annotation sampling
soma.data.PointSource	Detection targets; class_remap and drop fold in the class scheme
soma.extract.live_encoder(), soma.extract.FrozenEncoder	A parameter-free frozen encoder stage that re-encodes pixels at every step
soma.models.DenseModel	encoder (optional) → adaptor (optional) → decoder → head
soma.learn.DenseLearner, soma.learn.GridBatch	The grid learner and its batch (cached features or live pixels, plus raw targets)
soma.learn.SegmentationAugment	Joint image and label-mask or point augmentation for the live path
soma.evaluation.LabelMapPredictions, soma.evaluation.PointPredictions	What soma.predict returns for a segmentation or detection head
soma.evaluation.sweep_threshold()	Picks a detection score threshold on tune predictions
soma.evaluation.SaveOverlays, soma.evaluation.SaveProbabilities, soma.evaluation.SaveHeatmaps	predict callbacks that write overlays, softmax maps and heatmaps
soma.testing.FakeEncoder	A weight-free encoder stage for tests

Every target source has subset(sample_ids) to restrict it to one split.

For whole-image segmentation after a run, load each fold’s bundle with soma.load() and pass the list to SlidingWindowSegmentationPredictor.from_bundles(bundles, encoder=...), which slides the fold ensemble over an image. A bundle trained live carries its frozen encoder stage; for a run trained on cached grids, pass the stage that extracted them, soma.pipeline.build.live_encoder_for(plan) with the run’s Plan (soma.Plan.from_yaml(run_dir / "plan.yaml")).

Task-free representation evaluation

Evaluate frozen tile embeddings with the fixed CRoMa v1 protocol by leaving out the learner and setting evaluation.representation. Save this as a YAML config and run it with soma run config.yaml:

cohort:
  records: dataset.csv
  splits: splits.csv
features:
  encoder:
    name: phikon
evaluation:
  representation:
    confounder_column: medical_center
    split: test
    evaluation_design: all
    m: 5
    alpha: 0.10

Selected dataset rows require non-empty label, group_id (a metadata column naming the non-independence group), and medical_center columns. The selected split must occur in exactly one fold; cross-validation is unsupported. The cohort must provide enough same- and other-confounder neighbours for m=5; CRoMa raises if any selected sample lacks them.

Representation runs return metrics in result.summary with an empty fold_results list. They fit no task head, so they write no predictions and no bundle. Provenance records the installed CRoMa version and encoder configuration, including output variant. It does not include checkpoint hashes or an upstream fingerprint, so these records alone cannot establish byte-identical weights or preprocessing. See CRoMa for the registered cohorts and metrics.

Reproduce a packaged benchmark programmatically

Every registered benchmark is a Python object, so the soma reproduce flow is available from code, with results directly comparable to the CLI’s. The benchmarks live in the soma.benchmark package, which import soma does not import: importing it (here through from soma.benchmark import get_benchmark) registers them, after which soma.components("benchmarks") lists them:

import statistics

import soma
from soma.benchmark import get_benchmark
from soma.pipeline import Pipeline

[entry["name"] for entry in soma.components("benchmarks")]  # [..., "eva/bach", "hest/IDC", "ocelot"]
benchmark = get_benchmark("eva/bach")

manifest = benchmark.curate("/path/to/eva/bach", "runs/eva-bach/curated")

measured = []
for seed in benchmark.canonical_seeds:
    seed_root = f"runs/eva-bach/seed_{seed}"
    config = benchmark.build_config(
        encoder="uni2",                 # the axis a benchmark varies
        dataset_csv=manifest.dataset_csv,
        splits_csv=manifest.splits_csv,
        output_root=seed_root,
        seed=seed,
        # share one feature cache across seeds (extraction is seed-independent)
        overrides={"cache": {"enabled": True, "root_dir": "runs/eva-bach/feature_cache"}},
    )
    result = Pipeline(config).run()
    metrics = benchmark.score(result.run_dir)
    measured.append(metrics[benchmark.primary_metric])

print(statistics.fmean(measured))

benchmark.expected(encoder="uni2") returns the packaged reference rows to compare against, and benchmark.score(run_dir) alone re-scores an existing run without retraining (the --from-run-dir fast path).

The one-call equivalent is soma.benchmark.run_benchmark, the importable orchestration behind soma reproduce itself: the canonical-seed loop, the reference-row tolerance status, provenance stamping (git commit, slide2vec/croma versions), and the results-ledger append used by the CLI. Its keywords mirror the CLI flags, plus results_root so an external repository can append MeasuredRow rows to its own committed ledger instead of the packaged one (soma/benchmark/<protocol>/results.csv):

from soma.benchmark import run_benchmark

run_benchmark(
    "eva/bach",
    encoder="uni2",
    raw_root="/path/to/eva/bach",
    output_root="runs/eva-bach",
    record=True,
    # host the results ledger outside the soma checkout:
    # appends to <results_root>/eva.csv with full provenance
    results_root="/path/to/leaderboard-repo/results",
)

Run an external benchmark specification

A project outside soma can own its benchmark protocol as a typed soma.benchmark.BenchmarkSpec — a config builder, the canonical seed set, and a scorer — and hand it to soma.benchmark.run_benchmark_spec for execution. soma runs every canonical seed with one shared feature cache and returns the aggregated BenchmarkRunResult together with the per-seed evidence roots, without touching the registry, reference tables, or ledger.

Runnable demonstrations live in examples/ (examples/README.md):

• examples/external_benchmark_spec.py — build and execute a BenchmarkSpec;

• examples/fixed_step_training.py — fixed optimizer-step training budgets (TrainerConfig(max_steps=N));

• examples/aggregator_resolution.py — resolve one fixed MIL recipe for tile- and slide-level encoders with soma.models.encoders.resolve_aggregator;

• examples/portable_identity.py — manifest identity is portable across storage roots.