Struct BatchPolicy

Struct Documentation

struct BatchPolicy

One batchability policy shared by the single-term optimizer and the runtime batched evaluator (make_evaluator, Task A3). All predicates default empty.

Public Functions

inline std::function<bool(Index const&)> is_batchable_index() const

Derived “batchable in any role”: the union of the two building-block predicates. This is never a settable field &#8212; it is computed from is_batchable_contracted_index and is_batchable_external_index. The runtime batched evaluator’s accept predicate is this union (a mode is accepted at runtime if it is batchable in either role); the factorizer’s role filters instead consume the individual building blocks. The building blocks default-decline, so both are always callable here.

Public Members

SEQUANT_DESIGNATED_INIT_ONLY
std::function<bool(Index const&)> is_batchable_contracted_index = [](Index const&) { return false; }

Spaces batchable in the contracted role: a mode of such a space is batchable where it is summed. Companion to is_batchable_external_index (the external role). Splitting batchability by role lets a caller admit a space only where batching it is meaningful &#8212; e.g. a space batchable only as an external spectator contributes none of its contracted occurrences to the optimizer’s 2^m search. Building block; the derived “batchable in any role” query is is_batchable_index(). Defaults to decline every index; a caller opts spaces in explicitly.

std::function<bool(Index const&)> is_batchable_external_index = [](Index const&) { return false; }

Spaces batchable in the external role: a mode of such a space is batchable where it is open on the term root (a spectator carried to the result), not where it is contracted. Building block; declared adjacent to its contracted companion. Defaults to decline every index; a caller that wants external batching sets this predicate explicitly (there is no fallback to the contracted role).

std::function<std::size_t(Index const&)> batch_target_size = {}

Per-index per-batch slice size (in elements) for a batchable index &#8212; an UPPER BOUND, not a goal. Both the single-term optimizer and the runtime batched evaluator treat it as a ceiling: the realized whole-tile batch is rounded down to a tile multiple and never exceeds this value, except the one-tile floor (a lone tile larger than the target forms its own batch).

std::function<bool(Tensor const&)> is_volatile_leaf = {}
bool batch_spectator_indices = false

If true, an external/spectator index &#8212; open on the whole network’s result yet contracted at no node &#8212; is eligible for batching; its per-slice size comes from batch_target_size(ix) like any batchable index. Default false = no spectator batching. Necessary but not sufficient: the DP opens externals per node, and only where peak_threshold is finite &#8212; the gate inside PeakBatchedModel::relax is exactly batch_spectator_indices && std::isfinite(peak_threshold) (optimize/cost_model.hpp), with no objective condition and no post-DP placement pass. Both batched objectives therefore admit spectator axes; an infinite budget admits none.

bool persistent_only = false

If true, restrict batching to persistent (amplitude-independent) subtrees, declining to batch any subtree that contains a volatile leaf. If false (the default), batch ACROSS THE BOARD: slicing the batch axis shrinks any intermediate carrying it regardless of volatility (footprint objective) and leaves flops unchanged, so the persistence gate would only ever raise the modelled/realized peak. Set true to recover the persistent-only behavior (amortizes the per-replay partition + relaxed-screening cost over many reuses, at the price of a higher peak for volatile intermediates). Read identically by the single-term optimizer and the runtime evaluator.

double accumulation_factor = 0.0

Footprint multiplier for the in-flight batch contribution that co-resides with a batch-accumulated intermediate (K += contribution). 0 = ignore (default); ~1 = full contribution materialized; backend-specific (TA’s eager tile accumulation lowers it, multiple in-flight Summa steps raise it ~30%). Read by the single-term optimizer’s PeakBatchedModel to price the accumulator + contribution co-residency of a node that contracts a batchable index.

BatchScheduler scheduler = BatchScheduler::forest_descent

Selects between the two runtime execution models (BatchScheduler above). Consulted by the sequant::evaluate(Nodes const&, BatchPolicy const&, ...) driver overload (ordered_executor.hpp) to select the driver. Default forest_descent selects the forest-descent evaluator.

double peak_threshold = std::numeric_limits<double>::infinity()

Peak-memory budget in bytes. It is a feasibility ceiling under both batched objectives, and it is the single knob that turns batching on: PeakBatchedModel::relax opens neither a contracted nor an external loop unless std::isfinite(peak_threshold), so the default +infinity means no batching at all.

  • space-first (DenseSpaceTimeBatched): among the frontier points whose modeled byte peak is <= peak_threshold, minimize flops, ties broken by lower peak; fall back to global min-peak (best effort) when none fit.

  • time-first (DenseTimeSpaceBatched): among the frontier points whose modeled byte peak is <= peak_threshold, minimize flops, ties broken toward the least-sliced realization (nsl) and then lower peak &#8212; so a schedule is not sliced for free below the ceiling. When nothing fits, the fallback keeps the perf-first character: global min flops, ties by min peak (accepting the overage).

See PeakBatchedModel::select_root (optimize/cost_model.hpp) and the as-built design, doc/dev/specs/2026-09-12-batched-array-dag-eval-as-built.md section 4.4.