Each problem states how progress is verified and what counts as a contribution. Besides the problems curated here, the catalogue includes open conjectures from Formal Conjectures (with Lean statements), optimization constants and the AlphaEvolve problems. Know one that belongs here? Propose a problem.
2 shown
Explain why the test loss of neural networks follows power laws in model size, data and compute, and predict the exponents from properties of the data and architecture. Reproducible small-scale experiments serve as evidence.
Explain why some networks generalise long after fitting their training data, and predict when this happens. Reproducible small-model experiments serve as evidence, e.g. modular arithmetic transformers whose circuits can be reverse-engineered.
