INTERACTIVE FIGURE 6The denominator law — one knob is both precision and sparsity
row length L24bit budget b8
A legal row is a point on the probability simplex; storing it as integer numerators over a shared
denominator 2b makes it exact fixed-point arithmetic — the numerators sum to 2b by
construction, so there is no float error to accumulate, ever. But integers can't be split: by pigeonhole, once
2b < L (or merely once the mass is spread thin) some synapses must round to zero — the rust ticks. Drag b
down and watch precision and sparsification move together: they are the same knob. This is why 10-bit quantization of
the LM student was simultaneously lossless and 5× sparsifying, and why tolerance to the denominator is a
redundancy meter for the underlying network.