Your LLM Doesn't Write Correct Code. It Writes Plausible Code.
One of the simplest tests you can run on a database:Doing a primary key lookup on 100 rows.SQLite takes 0.09 ms. An LLM-generated Rust rewrite takes 1,815.43 ms.It’s not a misplaced comma! The rewrite is 20,171 times slower on one of the most basic database operations.The thing is though: The code compiles. It passes all its tests. It reads and writes the correct SQLite file format. Its README claims MVCC concurrent writers, file compatibility, and a drop-in C API. On first glance it reads like ...
Read more at blog.katanaquant.com