AuthorsJiaming SongLinqi ZhouMarch 11, 2025Generated samples on ImageNet-256×256 using 8 steps.Fully unlocking the potential of rich multi-modal dataThere is a growing sentiment in the AI community that generative pre-training is reaching a limit. However, we argue that these limits are not due to a lack of data itself, but rather a stagnation in algorithmic innovation. The field remains dominated by just two paradigms since around mid-2020: autoregressive models for discrete signals and diffusi...