Recurrent depth in Z1T:
a quality–storage trade-off.

TSUs, energy-based models, and Z1T

A TSU or thermodynamic sampling unit is hardware built to draw samples from programmable probability distributions. Extropic’s probabilistic circuits use physical fluctuations as a computational resource. Coupled probabilistic bits, or pbits, fluctuate between binary states; their programmed interactions shape which joint states occur more often. Their outputs are determined by allowing physics to settle the chip into a stable configuration over multiple samples.

An energy-based model (EBM) assigns an energy score to each possible configuration. Lower-energy configurations are more probable: p(x) ∝ exp(−E(x)). Think of a landscape with valleys: learning shapes the landscape, and sampling explores it. If the model is trained effectively, finding the lowest-energy regions is equivalent to generating a useful output. Compared to regular computers, TSUs can cost much less input energy to arrive at an answer, but it can only run energy-based models which are currently incompatible with traditional LLMs.

Z1T is a sparse language-model architecture. It is Extropic’s family of transformer-like models, designed with its Z1 probabilistic hardware in mind. It is similar to a traditional transformer but uses sparse layers that give each output only a small number of input connections, which allow it to run on a TSU. Its gated convolutional attention combines information across the sequence.

Recurrence and looping

In a conventional non-looping model, a deeper model usually both stores more weights and performs more computation. Weight sharing lets us separate those choices: a small stored core can be applied repeatedly. Recurrence here means repeating a stack of residual blocks within the model, not carrying a persistent hidden state across tokens.

Why flashing still matters

A TSU’s advantage is in drawing samples, not in loading a program. Sampling can cost much less energy than the same inference on a GPU. Flashing the weights onto the chip—setting the couplings that define the distribution is still a conventional write. That step looks more like a GPU memory transfer than a thermodynamic computation in terms of energy cost.

If the model is too large to fit on the chip, it must be flashed in sections for each output. Recurrence is one way to get more applied depth from a smaller stored core, which is why the quality cost of sharing those weights is worth measuring.

I built on Extropic’s Z1T software recipe and compared three architectures.

U6 versus R6×2 isolates the benefit of extra computation at fixed unique body capacity. R6×2 versus U12 measures the cost of sharing at fixed applied depth. Each architecture was trained separately at its evaluation recurrence count; this is not a test of adding loops to an already-trained model.

Experiment results

I ran six training seeds per architecture—18 confirmation runs—on an archived 40,000-document OpenWebText subset. Each run received 254,164,992 target-token presentations, including repeated exposures. Width was 384, context length 256, and sparse body fan-in c = 4.

At the same unique body capacity, R6×2 improved over U6 by 2.35% perplexity. The mean negative log-likelihood (NLL) difference was −0.0237 nats/token, with a 97.5% paired interval of [−0.0273, −0.0202]. Reusing the weights for a second pass helped.

At the same applied depth, R6×2 had 0.81% higher perplexity than U12. The mean NLL difference was +0.0081 nats/token, with a 97.5% paired interval of [+0.0023, +0.0139]. The deeper untied model was better, but required nearly twice the unique body parameters.

So this is a trade-off, not lossless compression or a claim of equivalent quality. The intervals describe training-seed variability on a fixed holdout, not uncertainty across datasets.

• Software, not silicon: these runs used H100 GPUs and float32 model arrays. Fan-in four is sparsity, not 4-bit precision. No TSU energy, latency, or silicon area was measured.
• A limited regime: one model width, one context length, and an OpenWebText subset—not a full-corpus benchmark or a scaling law.
• Not a reasoning result: this measures next-token prediction, not latent reasoning, adaptive halting, or persistent memory. Cached decoding was not evaluated.

Inspect the result

The research repository contains the model changes, training configurations, checksummed metrics for all 18 confirmation runs, analysis scripts, and manuscript source. Full-state checkpoints are not included. The analysis and figures can be regenerated without a GPU; after cloning the repo:

uv sync --frozen --python 3.12 --extra dev --extra prepare
JAX_PLATFORMS=cpu uv run python scripts/analyze_claim_a.py
JAX_PLATFORMS=cpu uv run python scripts/make_paper_figures.py

The next question is a deployment question: under a realistic resident-capacity constraint, when do avoided program loads outweigh the costs of shared execution? This study gives that question a measured quality cost to work with.