6 minute read In this post I’ll look at how LLMs performed on Advent of Code 2024. Surprisingly, they didn’t perform nearly as well as I’d expect, which we’ll explore in this post. Here are the main results: Intro I recently wrote about my experience with the 2024 advent of code challenge. Part of my challenge this year was not to use any LLMs, which I got really used to using in the last few years. This was mostly a learning exercise for me to brush up on and learn more about leetcode-style pro...