Aggregators

Aggregators combine tile features into a bag-level representation for MIL. Start with the simplest preset that matches the task, then tune only the knobs you need.

Where aggregators sit. A frozen tile encoder turns a slide into a bag of features; a trained MIL aggregator (or, alternatively, a frozen slide-level foundation model) pools that bag into one slide-level vector for the task head.

The shared base classes are soma.aggregators.base.Aggregator and soma.aggregators.base.AggregatorOutput.

class soma.aggregators.base.Aggregator(*args, **kwargs)

Bases: ABC, Module

Abstract base class for MIL aggregators.

Consumes a bag of tile features (B, N, D) and produces a slide-level representation (B, D_out) wrapped in an AggregatorOutput.

class soma.aggregators.base.AggregatorOutput(bag_representation, tile_attention=None, auxiliary=None)

Bases: object

Structured output from an aggregator.

bag_representation

Slide-level representation, shape (B, D_out).

tile_attention

Per-tile attention weights, shape (B, N). None for non-attention methods (e.g. MeanPool, MaxPool).

auxiliary

Optional dict of auxiliary tensors for training-time losses (e.g. instance logits for DSMIL, embeddings for CLAM).

Aggregator Zoo

Preset	Description	Notes
mean_pool	Mean over all tile features.
max_pool	Element-wise max over tile features.
abmil	Gated attention pooling for slide-level aggregation.	Ilse et al., 2018
clam_sb	Single-branch CLAM with instance-level supervision.	Lu et al., 2021
clam_mb	Multi-branch CLAM with one attention branch per class.	Lu et al., 2021
dsmil	Dual-stream MIL with a critical-instance query.	Li et al., 2021
dtfdmil	Two-tier MIL with pseudo-bag distillation.	Zhang et al., 2022
transmil	Transformer-based MIL with Nystrom attention.	Shao et al., 2021
hipt	Hierarchical image pyramid transformer.	Chen et al., 2022

Aggregator details

The short notes below explain what each aggregator is for. The class docstrings show the full constructor signatures and parameter descriptions.

ABMIL

abmil applies gated attention pooling and returns tile-level attention scores for interpretability and heatmap generation.

class soma.aggregators.mil.abmil.ABMIL(input_dim, hidden_dim=128, activation='tanh', gated=True, dropout=0.25)

Bases: Aggregator

Attention-Based MIL aggregator.

Uses gated attention pooling to aggregate tile features into a slide-level representation, with per-tile attention weights for interpretability and heatmap generation.

Parameters:

• input_dim (int) – Feature dimension of input tiles.

• hidden_dim (int) – Attention bottleneck dimension.

• activation (str) – Activation function (‘tanh’, ‘relu’, ‘gelu’).

• gated (bool) – If True, use gated attention.

• dropout (float) – Dropout rate applied before attention.

CLAM-SB

clam_sb is the single-branch CLAM preset.

It supports binary, multiclass, ordinal, and single-target regression tasks, and can mix bag-level and instance-level supervision.

class soma.aggregators.mil.clam.CLAM_SB(input_dim, hidden_dim=512, attn_dim=256, gated=True, dropout=0.0, k_sample=8, n_classes=2, inst_loss='ce', use_negative_class_instance_loss=False, bag_weight=0.7, instance_loss_mode=None, low_attention_weight=0.1, topk_target_weight=1.0)

Bases: _CLAMBase

Single-branch CLAM aggregator.

CLAM-MB

clam_mb is the multi-branch CLAM preset.

It is classification-only and creates one attention branch per class.

class soma.aggregators.mil.clam.CLAM_MB(input_dim, hidden_dim=512, attn_dim=256, gated=True, dropout=0.0, k_sample=8, n_classes=2, inst_loss='ce', use_negative_class_instance_loss=False, bag_weight=0.7)

Bases: _CLAMBase

Multi-branch CLAM aggregator.

DSMIL

dsmil first scores instances to find a critical tile, then performs query-key attention against that tile to build the bag representation.

class soma.aggregators.mil.dsmil.DSMIL(input_dim, att_dim=128, nonlinear_q=False, nonlinear_v=False, dropout=0.0)

Bases: Aggregator

Dual-Stream MIL aggregator.

Uses an instance classifier to identify a critical instance, then computes attention via query-key matching with the critical instance.

Parameters:

• input_dim (int) – Feature dimension of input tiles.

• att_dim (int) – Attention/query dimension.

• nonlinear_q (bool) – If True, use nonlinear query projection.

• nonlinear_v (bool) – If True, use nonlinear value projection.

• dropout (float) – Dropout rate for value projection (when nonlinear_v=True).

DTFDMIL

dtfdmil partitions a bag into pseudo-bags, distills features from the first tier, then aggregates the distilled set a second time.

class soma.aggregators.mil.dtfdmil.DTFDMIL(input_dim, hidden_dim=128, n_groups=8, distill_mode='maxmin', dropout=0.25)

Bases: Aggregator

Double-Tier Feature Distillation MIL aggregator.

Parameters:

• input_dim (int) – Feature dimension of input tiles.

• hidden_dim (int) – Attention bottleneck dimension.

• n_groups (int) – Number of pseudo-bags to partition into.

• distill_mode (str) – Feature distillation mode (‘maxmin’, ‘max’, ‘afs’).

• dropout (float) – Dropout rate applied before tier-1 attention.

TransMIL

transmil uses Nystromformer-style self-attention with pyramid positional encoding.

class soma.aggregators.mil.transmil.TransMIL(input_dim, att_dim=512, n_layers=2, n_heads=4, n_landmarks=None, pinv_iterations=6, dropout=0.0, use_mlp=False)

Bases: Aggregator

Transformer-based Correlated MIL aggregator.

Uses Nystromformer layers with PPEG positional encoding and a learnable class token to aggregate tile features.

Parameters:

• input_dim (int) – Feature dimension of input tiles.

• att_dim (int) – Transformer embedding dimension.

• n_layers (int) – Number of Nystromformer layers (must be >= 2).

• n_heads (int) – Number of attention heads.

• n_landmarks (int | None) – Landmarks for Nystrom approximation (default: att_dim//2).

• pinv_iterations (int) – Pseudo-inverse iterations.

• dropout (float) – Dropout rate.

• use_mlp (bool) – Whether to use MLP blocks in transformer layers.

HIPT

hipt first aggregates tiles within regions, then aggregates regions into a slide-level representation. This preset assumes hierarchical tiling in the preprocessing pipeline.

class soma.aggregators.mil.hipt.HIPT(input_dim, region_size, patch_size, embed_dim_region=192, embed_dim_slide=192, num_heads=6, dropout=0.25)

Bases: Aggregator

Hierarchical Image Pyramid Transformer aggregator (Chen et al., 2022).

Two-level hierarchy: a region-level ViT aggregates P tile features per region, then a global transformer + gated attention pools M region embeddings into a slide-level representation.

Features are stored either flat as (B, N, D) where N = M × P, or natively hierarchical as (B, M, P, D) where P = (region_size / patch_size)². HIPT reshapes internally.

Parameters:

• input_dim (int) – Feature dimension of input tiles (auto-resolved from FeatureStore).

• region_size (int) – Region pixel size (e.g. 4096). Required.

• patch_size (int) – Subtile pixel size within region (e.g. 256). Required.

• embed_dim_region (int) – Region ViT output dimension.

• embed_dim_slide (int) – Global transformer / output dimension.

• num_heads (int) – Attention heads in region ViT.

• dropout (float) – Dropout rate.

Notes

• clam_sb is the only CLAM preset that supports regression and ordinal classification.

• clam_mb is classification-only and emits one branch per class.

• hipt requires hierarchical tiling; set region_tile_multiple in soma.config.PreprocessingConfig to control how many tiles fit inside a region.

• The task head ultimately determines the valid loss and metric pairing.

Discovery helper

Use soma.list_aggregators() to inspect the registered aggregator names from code when you are wiring configs or building a UI.