Study finds 24 leading language models fail to distinguish belief from knowledge; GPT-4o accuracy drops from 98% to 64% on false belief tasks, raising concerns for use in healthcare, law and journalism.

Language models cannot reliably distinguish belief from knowledge and fact

Data availability The KaBLE dataset introduced in this study is publicly available via Hugging Face Datasets at https://huggingface.co/datasets/turingmachine/kable (ref. 11). An online leaderboard tracking model performance on the dataset is available at https://huggingface.co/spaces/vinid/kable-leaderboard. Code availability The full code for reproducing our results is available via Zenodo at https://doi.org/10.5281/zenodo.15249480 (ref. 10). It is also available via GitHub at https://github.co...