Q Labs proposes training quadrillion-parameter AI models on fixed datasets using multi-epoch training and ensembling—7,000x larger than current Chinchilla approach—to close 10-million-fold efficiency gap with human brains.

Quadrillion-parameter transformers and how to train them – Q

Q Labs Research Contents Summary 1 Introduction 2 Parameters are all you need 3 Larger models for more epochs 4 (Fast) Ensembling as a scaling axis 5 Conclusion References TL;DR $$L = \underbrace{E\vphantom{\left(\frac{N}{U^{\gamma}}\right)^{\kappa}}}_{\text{irreducible}} + \underbrace{\frac{A}{N^{\alpha}}\vphantom{\left(\frac{N}{U^{\gamma}}\right)^{\kappa}}}_{\text{small model}} + \underbrace{\frac{B}{(UR)^{\beta}}\vphantom{\left(\frac{N}{U^{\gamma}}\right)^{\kappa}}}_{\text{tokens seen}} + \un...