LegoGPT generates a LEGO structure from a user-provided text prompt in an end-to-end manner. Notably, our generated LEGO structure is physically stable and buildable. Abstract We introduce LegoGPT, the first approach for generating physically stable LEGO brick models from text prompts. To achieve this, we construct a large-scale, physically stable dataset of LEGO designs, along with their associated captions, and train an autoregressive large language model to predict the next brick to add via n...