PREVIEW BUILD · IMAGES + INTERACTIVE FIGURES EMBEDDED · NOT THE LW SOURCE

The Drifting Frame and the Program Atlas: What Deep Computation Actually Is

Post 8, the finale of the Legible Networks series. Sources: the writeups The Drifting Frame and The Program Atlas in the explainable_ai repo, with their follow-up instruments (frame kinematics, checkpoint archaeology, the deviation lens, the canalization and cross-run tests). This is where the exact instrument built across posts 1–3 gets run to the end of its rope — and what it reports is a negative result so total it becomes a positive theory. Interactive figures continue the series numbering.

Epistemic status: the conservation and cancellation results are closed-form arithmetic on an exactly row-stochastic network, now replicated across two architectures (post- and pre-activation trunks), two games, and two board sizes, with rate constants agreeing to the second decimal. The frame results replicate across a conv-ResNet and a transformer. The atlas results are anchored on one chess network but their key claims carry independent checks: an exact-solver ground-truth leg (KPvK), a lobotomy-invariance test, a cross-training-run replication, and a five-domain generalization of the closing law. Where a claim rests on one net or one corpus, we say so.

The puzzle

The unit-net geometry was built for one instrument above all: conserved path mass. On an exactly row-stochastic skeleton, every neuron's unit budget splits exactly over its inputs, so "how much of this output came through that unit" has a well-defined, comparable, certifiable answer. Posts 1–3 built the geometry; this post runs the instrument to its logical end on a superhuman chess network and asks the question the whole series was pointed at:

What does exact attribution say about a deep network?

The answer, in three measurements: almost nothing, almost nowhere — for a reason with rate constants — and the shape of the nothing turns out to be a theory of what deep computation is instead.

A convenient miracle to start: Leela T70's trunk needs no transplant at all. Every one of its 2,688 conv rows sits at positive row-sum 1.0000 — min = median = max, to 1.5×10⁻⁷ — because its conversion pipeline had already folded the scale debt.¹ The precondition for exact bookkeeping holds natively, verified at joint KL 0.0000 and −3 Elo in arena play.

1. Evaporation: where the positive mass goes

Track backward positive path mass from the trunk tip toward the input, on 256 real positions. Two sinks, one dominant:

The padding sink is real but minor (~8%/layer: kernel taps that read off-board). The gates are the story: realized positive mass decays at ~0.25× per convolution, identically across game phases — there are no purely-positive chains of active units of any depth. And this constant is a law, not a quirk: replicated on KataGo's 19×19 pre-activation trunk, the per-conv gate factor comes out 0.250 — same number, different game, different architecture, different board size.

2. Effective depth: the visible computation is 1–2 blocks deep

Include the skip connections (excluded above by convention) and apportion each residual summation between branch and skip by measured magnitude share; charge each branch traversal its measured gated retention (ρ ≈ 0.03–0.06); a generating function over blocks then yields the exact distribution of tip-visible evidence by number of conv-blocks traversed. Result: median 1–2 blocks of 10 (P(≤2) = 0.69–0.91), because a conv hop costs ~25× a skip hop. The range spans two apportioning conventions, both shown in the figure: "full" splits each summation by the raw magnitudes as they stand, while "beta-free" first excludes the squeeze-excitation layers' broadcast bias constants (their β terms) from the split — those are position-independent constants that inflate the branch's apparent magnitude share roughly 2× without carrying any per-position signal, so the beta-free row is the more conservative reading. The conclusion is the same under either bookkeeping; conventions like this bite, which is why we always run both. This is Veit et al.'s shallow-ensembles picture — deep ResNets behave like ensembles of short paths — upgraded from lesion study to closed-form arithmetic on a certified skeleton. (On KataGo it's more extreme: skip-dominated summations put 84% of tip-visible mass at depth zero.)

The complementary magnitude fact keeps this from being misread as "the network is shallow": the branches dominate every summation's magnitude (skip share ≤ 0.41 in T70). The blocks rewrite the stream at every step; the ledger-visible remnant merely rides it. Deep computation is happening — the positive ledger just can't see it. Which raises the question §3 answers terminally: could any extension of the books see it?

3. Total cancellation: the terminal statement

Extend the ledger to signed mass — backward flow through a positive weight preserves support sign, through a negative weight flips it, both gated. One convolution below the tip, the cancellation coefficient |net|/gross is 0.121. By five convolutions: 0.000, to float64 precision, with the flipped-support share pinned at exactly one-half — every unit more than a couple of convs from the tip supports the output through positive and negative path families in perfect balance. Replicated on KataGo: 0.115 at one hop (T70: 0.121), identically zero by four convs. Two architectures agree on how fast the signed books balance, to the third decimal of the first hop.

Sit with what this means. No static path algebra — positive, signed, skip-inclusive, however refined — carries net information at depth in these networks. The function is not in the paths; it is in the data-dependent selection among perfectly balanced path families. Attribution must be data-conditional or it is nothing. This is the mechanistic ground under a decade of practical folklore about why weights-only saliency methods disappoint — here it's not folklore, it's a rate constant.

4. Persistence: information stays; encodings rotate

So what is the deep stack doing, if not accumulating attributable support? Decode nameable features at every block. Material (the piece-value balance) stays decodable across the full trunk — R² 0.99 → 0.91 — while its decoder direction rotates to orthogonality: adjacent-block cosines ~0.67, compounding to 0.04 end-to-end. Castling rights: same. Mobility is visibly computed mid-trunk (R² rises 0.74 → 0.89).

And the bridge measurement corrected our own earlier framing. Post 5 described Qwen's bracket-depth counter as "built by layer 4 and carried to 24" — but only its flat R² had been measured. Its direction rotates just as completely: cos(L4, L24) = 0.055. Flat R² masks rotation, in both architectures — a conv-ResNet and a transformer. The artifact check that makes the rotation real rather than probe non-uniqueness: naive cross-layer probe transfer fails outright (a decoder fit at layer 4, applied at layer 12, scores R² = −1.9 — worse than predicting the mean).

5. The drifting frame

Is that rotation per-feature re-encoding (each feature individually rewritten — computation) or a global frame drift (one coordinate system turning — kinematics)? The discriminating test: fit an unsupervised linear alignment between layer states — knowing nothing about any feature — and transport the probe through it.

Alignment restores transfer to 96–99% of the refit ceiling, in both architectures, with a pure rotation recovering most of it. One blind-fit map serves all features at once (material, mobility, castling, piece count — 97–100% of each ceiling through the same matrix); the maps compose (nine chained adjacent-pair maps ≈ the direct map); and the drift is steady — adjacent Procrustes rotations average 45.5° ± 8° per block against a 90° random baseline, a constant-rate turn that fully scrambles absolute directions in 2–4 blocks, exactly matching the cosine decays.

The law: information persists; the frame rotates; positions within the frame hold.

🕹️ Interactive Figure 13 — The rotating frame. A feature direction turning ~45° per block, a probe frozen at its fit layer, and the transfer R² collapsing as depth accumulates — then the align first toggle, which co-rotates the probe through the blind-fit map and holds the read at ceiling across the whole stack. The practical moral for anyone who has ever applied a layer-k probe at layer k±4: it was never valid, even at flat R², and the fix is one cheap unsupervised map.

Three follow-ups turn "frame" from metaphor into measured kinematics — and each lands on a series theme:

• The motion is data-dependent and the features are passengers. An alignment map fit on endgames transfers poorly to openings (0.975 → 0.711): there is no single weight-level drift, only the average of gate-conditioned dynamics. Carried features are passengers: their own directions turn at essentially the frame's bulk rate — 48°/block for material and castling against the frame's 45.5°/block, a difference inside the jitter — while mobility, the one feature actively computed mid-trunk, rotates measurably faster (55°/block): being worked on adds motion beyond the carriage. And the frame is mostly slack: a deliberate 45° rotation at block 5 costs the policy almost nothing (KL 0.0045; −18 Elo with a confidence interval touching zero). Wander in the carrying; work in the variation.
• The frame is congenital. Checkpoint archaeology (the T70 training run recovered from the Wayback Machine) shows align-then-transfer already works at 0.99 of ceiling at a near-birth checkpoint that agrees with the final policy 8% of the time. The frame is a property of architecture-plus-data, not a trained achievement; its rotation rate matures early (26° → 45°/block by mid-run); and — post 6's law, found in a third place — the representations were never the point: material decodes at R² 0.96 from the birth checkpoint (higher than the final net's 0.91). Representations are cheap from step one. What training builds, expensively and late, is the play.
• The deep blocks keep moving longest — per-block similarity to the final geometry is depth-ordered at every checkpoint. The trunk freezes bottom-up.

6. The deviation lens

If the mean flow is passive carriage and the work rides in the input-conditioned variation, then the variation is the thing to read. Define, per block pair and position: δ(x) = z_next(x) − A·z(x) — what this block did to this position beyond its average transport. Four measurements:

• Deviations are a third of the stream, at every depth (‖δ‖/‖z‖ = 0.35–0.47, flat) — per-block computation isn't concentrated anywhere special.
• The mean flow is information-neutral — measured, not assumed: features decoded from the transported state match the source layer to three decimals (the data-processing inequality with an empirical signature).
• The increment lives in the deviation. Where mobility's decodability jumps (b5→b6), decoding it from δ alone peaks; material — carried, never incremented — decodes from δ at R² 0.00 at every pair. And the deviation out-reads the raw state on the newly-computed part: R² 0.400 vs 0.242 at the hottest pair — subtracting predictable carriage is a matched filter; the carried bulk is, for reading the increment, noise.
• ‖δ(x)‖ is a computation-intensity meter: partialing out piece count, it correlates most with being in check, queens on, and hanging pieces. The network works hardest, in this literal per-position sense, exactly where the position is tactically hot.²

Then the causal pricing, because correlational lenses invite the obvious objection. At block 6's output: emit the mean-flow prediction instead of the true state (zero-δ — carriage without this position's work); emit the mean flow plus another position's deviation (swap-δ — right statistics, wrong content); or add unstructured noise of exactly ‖δ‖:

The predicted ordering exactly: content beats statistics beats distance. Deleting one block's per-position work changes the played move in 41% of positions and costs a quarter-thousand Elo; a norm-preserving rotation of the entire state costs approximately nothing. The frame is slack; the deviation is load-bearing — as a causal statement with a dose curve. (One meta-lesson for the field's favorite metric: Elo-per-KL runs ~7,400 for the structured deletion vs ~4,000 for noise — KL under-prices structured damage.)

7. The program atlas: counting a network's situations

The frame paper ends in a content-addressed-dispatch picture: per-position Jacobians are near-orthogonal — every position runs what amounts to its own linear program, assembled by gates, and gates are set by states. If that's right, then the state is literally the address of the program the position will run, and plain cosine similarity between two positions' states should predict the distance between their operators — no Jacobians needed.

It does: Spearman −0.878 against measured Jacobian-probe distance, while every board-level similarity fails (piece-placement agreement −0.11; game phase +0.48; partial all of them out and the state kernel holds at −0.84 to −0.88). The demonstration pairs need no statistics: the dynamically-nearest positions with low board overlap share zero squares — same program, different corners of the board. Computational neighborhood is activation geometry, not board geometry. One forward pass reads a position's program address; surveys become affordable.

The endgame survey. 1,600 endgames spanning 154 material signatures, clustered by the kernel, clusters then certified as dynamics clusters (within-cluster operator distance vs between: AUC 0.90). The 154 materials collapse to six programs — and the programs ignore material (adjusted Rand index³ vs the material partition: 0.08). What organizes them is the verdict, from the mover's seat:

Three details carry it. The mirror pairs: KQvK and KvKQ — one material class — land in opposite programs; the program is the role, not the inventory. The contested class: the one genuinely mixed-verdict program is the rook endings — the class chess lore singles out as hardest gets its own routine with honest uncertainty, while dozens of decided materials share routines. Discovery, not construction: the clusters are built from mid-trunk states alone; the value head enters only as post-hoc description — and four of six emerge with near-pure verdicts anyway. The kernel-space embedding's dominant geometry is literally a win→draw→loss horseshoe. Programs emerge by block 3, are richest at blocks 4–6, and coarsen toward pure verdict by block 9, as the value head approaches.

The middlegame is a continuum — with a chart. Same instrument on 1,600 middlegames: no discrete programs (silhouettes decay with k), only a development gradient, verdict-blind. But clusterless is not structureless: human positional structures — fianchetto setups, locked centers, isolated queen's pawns — are readable as regions of the same geometry at 0.81–0.93 balanced accuracy. Verdicts are clusters; structures are regions — human chess theory's taxonomy is, measurably, a chart of the network's continuum.⁵

The map is scaffold — two remarkable invariances. First, canalization: recompute the kernel from the Elo-priced damaged variants of §6, and even the −237-Elo lobotomy keeps 75% of every position's top-5 neighbors (CKA⁴ 0.970) — better than the intact network's own drift across three blocks of depth (63%). Deleting a quarter-thousand Elo of computation moves the situation map less than walking three blocks inside the healthy net: the neighborhoods are carried by shared content, while the per-position computation — idiosyncratic by the operator results — cancels out of pairwise similarity. Second, cross-run replication: an independently trained network from a different Leela era (T60: different architecture, different inputs, agreeing with T70 on only 54% of moves) reproduces T70's middlegame neighborhoods at top-5 overlap 0.637 — statistically indistinguishable from T70's own internal depth drift — and replicates the six-program partition at ARI 0.475. The situation map is the task's, seen through the architecture class: individual training runs carve their different play into the same geometry, which survives both lobotomy within a net and replacement of the net entirely.

The future is not encoded — and that's calibration. Do middlegames secretly carry their endgame destiny? No, twice: destination-conditioned middlegame clusters match random controls, and 5-NN prediction of the eventual program from the middlegame state scores below base rate. The crystallization curve makes the timing exact: destiny-visibility lift is zero at every distance beyond eight plies and appears only in the final bin — +0.13 at 1–8 plies out. Commitment is a phase transition at the door. Under the strong reading this is not myopia but calibration: between near-equal players the destination genuinely isn't determined sixteen plies out, and a representation claiming otherwise would be wrong.

Sub-programs against exact truth. For one material class small enough to solve exactly — KPvK, all ~400k states, retrograde solver validated on textbook cases (correcting one author's misremembered "textbook fact" along the way) — the kernel resolves sub-programs within the class: won-KPvK vs drawn-KPvK separate at 90.7% where board geometry sits at base rate, and the network visibly solves the pawn race through its depth — easy cases early, opposition theory last, the final increment entering through the deviation channel, exactly as §6's framework predicts.

The objective clause. Run the per-block survey on four more domains and the law is forced into its correct general form. A language model's Dyck geometry organizes by recency (last token, then the token before — a suffix trie, ARI 0.75–0.95) with the perfectly-represented automaton variance-subordinate everywhere — until the final layer, where recency collapses and the behavioral partition rises: the LM's last layer coarsens toward its verdict, and its verdict is the next token. KataGo's verdict readability steps exactly at its two global-pooling blocks (+0.10 at block 7, 3× any other block — emergence tied to a named architectural event). ResNet18's class organization rises monotonically to ARI 0.79 with the silhouette-optimal k landing at 11 ≈ its 10 classes — the geometry discovers the class count at the depth where the verdict is due. And natural text behaves like chess with the verdict renamed: kernel neighborhoods read the model's next token at 12× base rate exactly where continuation entropy is low — an LM's "endgames" are its low-entropy contexts. The law, final form:

Cluster geometry tracks the epistemic state of the training objective's output — the verdict for value-trained game networks, the next-token decision for language models — discrete programs where it is settled, graded structure where it is being computed, continuum where it is genuinely open. Everything else the network represents is present, readable as regions or by probes, and never dominant.

🕹️ Interactive Figure 14 — The wall and the gradient. How far ahead of commitment is the future readable from the state? Two measured curves, one slider. Chess: a wall — zero destiny information beyond eight plies, then a step (+0.13), because between equal players the future genuinely isn't determined yet. Language: a gradient with a long tail — the settled token reads at 5.5× base rate one token out and still 2.6× base sixteen tokens out, decaying smoothly, because single-author text telegraphs its destinations topically where adversarial games hide theirs. For the question this rehearses — when is a model's upcoming content readable before emission? — language's answer is: gradually, substantially, and by simple nearest neighbors.

Coda: the series, and what we'd ask of you

The arc in five sentences. A weight-budget geometry makes path mass exact, and optimization methods sort by the information channel they consume (post 1). Models in the geometry made interpretability's flagship lens exact, its averaged approximation measurable (3% vs 91% by regime), and task-reasoners carvable with certificates (post 2) — and pretrained superhuman networks turned out to be transplantable into the geometry exactly, without a gradient step (post 3). Pointed at real systems, the exact instruments found a certified representational hole behind a famous adversarial exploit (post 4), pristine latent structure rationed by consumer gain rather than existence (posts 5–6), and audit methodology that deflates ~98% of "drive" findings while measuring when search actually helps (post 7). And when the attribution ledger itself reported zero — totally, with cross-architecture rate constants — the zero had a shape (post 8): information persists, the frame rotates, the work rides in the deviations, and the network's states are the addresses of the programs it runs, organized by exactly one thing — how settled the objective's answer is.

Running beneath all of it, one law surfaced independently four times — in a distilled toy's dead units, a language model's unread stack, a Go program's silent safety wire, and a chess network's birth checkpoint: representations are cheap; what training rations is consumers, and what consumers are rationed by is gain. If we had to compress the series to one sentence for the alignment discourse, it's that behavioral evaluation measures the gain schedule, not the knowledge — in both directions.

What we'd most like from LessWrong: adversarial replication. The single-substrate results (the atlas's chess anchor, the elicitation specifics of post 5, the gain-ledger cohorts) are the ones we're least entitled to generalize. The repos are public, every headline number has a named script, most experiments run in minutes-to-an-hour on one GPU, and this series has now retracted or demoted five of its own findings when the kill-tests said so — we would genuinely rather you find the sixth than have the first five be the last. The comment section is open; the verifier is running.