TL;DR In our quest to generate effective code suggestions with LLMs at Qodo Merge, an AI-powered tool for automated pull request analysis and feedback, we discovered that prioritization isn’t enough. Initially we tried telling the model to prioritize finding bugs and problems in pull requests, while still allowing lower-priority suggestions for style, code smells, and best practices. We discovered that this approach was ineffective, and despite the explicit prioritization, the model got overwhel...