Researchers Propose Closure Validation Method to Confirm Attention Head Circuits in Neural Networks

A new preprint from arXiv challenges a common shortcut in neural network interpretability research: using co-activation clustering to identify functional circuits in transformer attention heads. The authors adapted a sparse-autoencoder clustering recipe to attention heads but replaced reconstruction-based validation with causal ablation and a 'closure test,' which checks whether ablating a discovered community causes measurably greater damage than ablating a matched random control. Across Pythia 1B and OLMo 1B — two dense models at the 1-billion-parameter scale — the discovered communities passed closure across two input distributions, lending some support to the clustering approach in those settings. However, in OLMoE-1B-7B, a Mixture-of-Experts model where route-conditional clustering was applied, the statistically detectable signal did not survive closure: ablation improved loss, the opposite of what a genuine circuit would produce. The authors also tracked circuit metrics across training checkpoints, finding that attention-target selectivity and participation ratio decouple from function in both directions, meaning neither metric reliably tracks whether a group of heads is actually doing functional work. The study concludes that co-activation clustering is best understood as a circuit proposal mechanism, and that closure via causal ablation is the necessary — and currently underused — step that separates proposals from confirmed circuits.