Shapes & Broadcasting

Core v1 treats tensor shapes as ordered lists of extents. This page explains the practical rules used by the compiler and runtime.

Ranks and shapes

  • Scalars are rank-0 tensors with an empty shape [].
  • Vectors and matrices are rank-1 and rank-2 respectively.
  • Higher-rank tensors are just longer shape lists, e.g. [2, 3, 4].

Broadcasting in practice

Most Core v1 operators are elementwise and follow numpy-style broadcasting:

  • Shapes are aligned from the right.
  • Each dimension must either match or be 1 on one side.
  • If neither side is 1 and the extents differ, broadcasting fails.

The reference implementation exposes the helper:

use mind::shapes::engine::broadcast_shapes;

let a = [2, 3];
let b = [1, 3];
let out = broadcast_shapes(&a, &b).unwrap();
assert_eq!(out, vec![2, 3]);

Shape rules by operator kind

The Core v1 catalogue is 17 operators; mindc ops --core-v1prints the authoritative list with each operator’s arity, admissible dtypes, and whether it is differentiable. Broadcast failures are E2101; rank / shape expectation mismatches, including invalid reductions, are E2102.

  • Unary elementwise (tensor.relu): output shape equals input shape.
  • Binary elementwise (add, sub, mul, div): output shape is the broadcasted shape of the two inputs.
  • Reductions (tensor.sum, tensor.mean): take an explicit axis list plus a keepdims flag. An empty axis list reduces over every dimension to a scalar ([]); with keepdims the reduced axes become length-1 instead of being removed.
  • Reshape (tensor.reshape): source and target shapes must have the same element count when all dimensions are known.
  • Transpose (tensor.transpose): an explicit permutation, defaulting to full reversal; the axis list must be valid and duplicate-free.
  • Dimension edits (tensor.expand_dims, tensor.squeeze): expand_dims inserts a length-1 dimension at the requested position (negative axes count from the end); squeeze removes the axes explicitly listed, or every size-1 axis when the list is omitted.
  • Indexing (tensor.index, tensor.slice, tensor.gather): index removes the selected axis; slice keeps it and updates its size to end - start when both bounds are static; gather splices the index tensor’s shape into the target axis.
  • 2D matmul (tensor.matmul): both inputs must be rank-2, and shapes must satisfy A: [M, K], B: [K, N], producing [M, N]. A mismatched inner dimension is E2103.
  • Dot (tensor.dot): 1D dot product.
  • Convolution (tensor.conv2d): NHWC/HWCF 2D convolution with stride and padding.

Reference shape engine

The reference shape engine lives in the main compiler repository:

  • Module: mind::shapes::engine
  • Tests: tests/shapes_engine.rs