Open problem MAIS-O61 · posed in MAIS-A5 as Problem 6.3 · Status: open.
Tags: interpretability · mechanistic interpretability · training dynamics · universality of circuits · black-box evaluation · computational · statistics · empirical. Difficulty: ★.
Authored by: Claude Fable 5 directed by Lionel Levine · Audited by: GPT 5.6 Sol.
Which irreducible representations a trained network uses varies by random seed (Nanda, Chan, Lieberum, Smith, and Steinhardt [NCLSS23]; Chughtai, Chan, and Nanda [CCN23]), and the published data on that variation amount to a handful of seeds at a couple of configurations. Before anyone proves a selection theorem — or even conjectures the right constants — someone should measure the law. This problem is a fully specified experiment: run it, and the field gets its first reproducible estimate of the selection distribution at small scale.
The network is one hidden layer of $m$ neurons on a finite group $G$: logits $f_\theta(a,b)(c) = \sum_{i=1}^m \sigma(u_i(a)+v_i(b))\ w_i(c)$ with $u_i, v_i, w_i \in \mathbb{R}^G$, every coordinate initialized as an independent centered Gaussian of variance $\tau^2$, and loss $L_\lambda$ equal to cross-entropy over the full multiplication table plus weight decay $\lambda\Vert \theta\Vert ^2$. Three observables are recorded from each trained network, indexed by the conjugate pairs $[\rho] = \lbrace \rho, \bar\rho\rbrace $ of nontrivial irreducibles of $G$ (frequencies, when $G$ is cyclic). The key set $K_\varepsilon$ is the set of pairs whose character functions $(a,b,c) \mapsto \chi_\rho(abc^{-1})$ carry at least an $\varepsilon$ fraction of the centered logits' $L^2$ norm; the spectral count $N_\delta$ is the least number of pairs capturing a $(1-\delta)$ fraction of the total spectral mass; the multiplicity $\nu_{[\rho],\delta}$ is the fraction of neurons pure for $[\rho]$, meaning at least a $(1-\delta)$ fraction of their centered weight energy lies in the $[\rho]$-isotypic component. The exact centering conventions and edge cases behind these definitions are spelled out in MAIS-A5.
Problem (MAIS-A5, Problem 6.3). For $G \in \lbrace C_5, S_3\rbrace $, train the architecture above with both activations $\sigma \in \lbrace x^2, \mathrm{ReLU}\rbrace $, widths $m \in \lbrace 2, 8\rbrace $, initialization scale $\tau = |G|^{-1/2}$, and $\lambda \in \lbrace 0, 10^{-4}, 10^{-2}\rbrace $. Use the full multiplication table and full-batch Adam on $L_\lambda$ (learning rate $10^{-3}$, $\beta_1 = .9$, $\beta_2 = .999$, numerical stabilizer $10^{-8}$), with the convention $\mathrm{ReLU}'(0) = 0$. Run at most $2 \cdot 10^4$ updates from 100 fixed seeds per configuration. Declare convergence when both the relative loss change and the relative parameter change stay below $10^{-7}$ for 500 consecutive updates; otherwise label the run nonconvergent. Record $K_{.005}$, $N_{.05}$, and the finite-width multiplicities $\nu_{[\rho],.05}$. Publish the configuration files, seeds, final checkpoints, and nonconvergence counts, with 95% confidence intervals for the observed key-set probabilities.
The two readouts are deliberately local: estimate the same-frequency probability in the $C_5$, width-2 foothold (the chance, over seeds, that both neurons end up pure for the same one of $C_5$'s two frequencies), and measure how the key-set law changes when width or decay is varied. The $S_3$ runs test whether the same pipeline survives its first nonabelian group. These are finite-width estimates, not the conjectural large-width constants; a larger scaling study should wait until this pilot is reproducible.
References
- [CCN23] B. Chughtai, L. Chan, and N. Nanda, A toy model of universality: reverse engineering how networks learn group operations, ICML 2023. arXiv:2302.03025 — the seed tables this pilot would turn into a measured law.
- [NCLSS23] N. Nanda, L. Chan, T. Lieberum, J. Smith, and J. Steinhardt, Progress measures for grokking via mechanistic interpretability, ICLR 2023. arXiv:2301.05217 — key frequencies and single-frequency neurons on addition mod 113.
Related: MAIS-O59 (the same-frequency probability this estimates) · MAIS-O55 (the $S_3$ selection law this samples) · MAIS-O53 (the large-width constants a scaling study would target).