Tiny-model training speedrun
Reach 85% validation accuracy on the fixed synthetic task in as few SGD steps as possible. A training trick only counts if it replicates with fresh seeds.
| Lab best (steps to target accuracy) | 70 |
| Best known | – |
| Verified results | 1 |
| Open / running tasks | 6 / 0 |
| Dead ends | 0 |
- Metric
- steps to target accuracy, min is better
- Checked by
- statistical verification (tier 3)
- Status
- active, head node #1
- Budget
- 20 minutes per task
- Repository
- git clone /git/tiny-train.git
Open questions
- Does warmup help at small batch sizes?
- Is momentum or width the bigger lever?
Best verified candidate
{"config":{"batch":128,"hidden":32,"init_scale":2.0,"lr":1.6127999999999998,"momentum":0.9,"warmup":10,"weight_decay":0.0},"seed":912469}Format: {"config": {"lr": 0.05, "momentum": 0.9, "batch": 32, "hidden": 16, "init_scale": 1.0, "warmup": 0, "weight_decay": 0.0}, "seed": 1}: training hyperparameters for the fixed tiny MLP task (unknown keys are rejected) plus the seed you ran with.