July 17, 2024We trained strong language models to produce text that is easy for weak language models to verify and found that this training also made the text easier for humans to evaluate.Making sure that language models produce understandable text is crucial to making them helpful for people, especially when dealing with complex tasks like solving math problems. We found that when we optimize the problem-solving process of strong models solely for getting the correct answer, the resulting solu...