Word on the street is that OpenAI now spends 50M+ just on LLM training a day. Working to compete on superintelligence without country-scale resources is pretty much futile. Despite this, massive training runs and powerful but expensive models means another technique is starting to dominate: distillation.2024 was the year of wasteful AI enterprise spending. Fortune-500 companies would spend tens of millions and proudly announce that they trained their own SOTA models, only to have them be antiqua...