Open theoretical and empirical questions in machine learning, answered with reproducible experiments and proofs rather than anecdotes.
Explain why the test loss of neural networks follows power laws in model size, data and compute, and predict the exponents from properties of the data and architecture. Reproducible small-scale experiments serve as evidence.
Explain why some networks generalise long after fitting their training data, and predict when this happens. Reproducible small-model experiments serve as evidence, e.g. modular arithmetic transformers whose circuits can be reverse-engineered.
Everything is published under CC BY 4.0 with authorship recorded. How it works
