INTERACTIVE FIGURE 4The renorm spike — why carving is deletion plus forced renormalization
A · one P row under pruning — survivors must re-sum to 1.0
edges kept12
B · four rows, one shared budget — who decides where it goes?
total edge budget16of 48
Legality forces every surviving P row to renormalize back to Σ = 1, so pruning an entry doesn't just remove
its contribution — it inflates every survivor by 1/m (m = the mass that was kept). Panel A: drag the slider to
one surviving edge and that synapse becomes a full-strength wire; on the fixed input shown, the neuron's activation
jumps far above anything downstream weights were trained to receive. Panel B is why the textbook saliency score goes
below random: a global sort lets one hot row hoard the budget while the others starve down to their single forced
entry and spike (rust bars = renorm factor). Balanced allocation, same budget, keeps every factor near 1. Starved rows
aren't lossy — they're toxic.