News Score: Score the News, Sort the News, Rewrite the Headlines

PhD Knowledge Not Required: A Reasoning Challenge for Large Language Models

View PDF HTML (experimental) Abstract:Existing benchmarks for frontier models often test specialized, ``PhD-level'' knowledge that is difficult for non-experts to grasp. In contrast, we present a benchmark based on the NPR Sunday Puzzle Challenge that requires only general knowledge. Our benchmark is challenging for both humans and models, however correct solutions are easy to verify, and models' mistakes are easy to spot. Our work reveals capability gaps that are not evident in existing benchma...

Read more at arxiv.org

© News Score  score the news, sort the news, rewrite the headlines