- ChatGPT might be able to write lines of code, but it's often full of errors.
- OpenAI thinks it's found a solution to the problem: CriticGPT.
OpenAI is starting to get serious about the hunt for bugs lurking in code generated by ChatGPT.
Since its release, the AI chatbot has impressed the developer community with its ability to produce code in programming languages such as Python and Ruby. Yet it's also given developers reason to be dubious: code produced by ChatGPT is often full of mistakes.
A study published in August 2023 by researchers at Purdue University found ChatGPT's responses to questions about code in the developer forum Stack Overflow were wrong 52% of the time, after assessing it for "correctness, consistency, comprehensiveness and conciseness."
What's worse, the researchers found, was that the mistakes were often tough to identify: ChatGPT's seemingly "articulate" responses made it difficult to spot the errors.
OpenAI seems to have recognized the problem, and is responding with a new solution: CriticGPT.
The new model, revealed by the startup last week, has been built to "catch errors in ChatGPT's code output." In OpenAI's telling, the tool based on its GPT-4 model sounds like it could be a huge help to developers.
"We found that when people get help from CriticGPT to review ChatGPT code they outperform those without help 60% of the time," the company said in a blog post.
'Fewer hallucinated bugs'
To begin, OpenAI is limiting CriticGPT's access to AI trainers. In practice, that means humans whose job involves reviewing answers produced by ChatGPT — through reinforcement learning from human feedback (RLHF) — will get assistance from CriticGPT to assess for accuracy.
OpenAI says CriticGPT will offer humans AI that "augments their skills, resulting in more comprehensive critiques than when people work alone" in reviewing ChatGPT answers and "fewer hallucinated bugs than when the model works alone."
Their hope is that this will boost a review process becoming increasingly tricky. OpenAI has acknowledged that the more advanced its AI models become, the harder it can be for AI trainers "to spot inaccuracies when they do occur."
The bigger problem this can lead to, the company said, is increased difficulty in aligning models with their intended objectives "as they gradually become more knowledgeable than any person that could provide feedback."
A future where models produced by AI companies actually become more knowledgeable than a person using them is not yet in sight, but AI researchers focused on safety have been busy thinking about how to keep such models in check to ensure they pose no threats.
Such researchers include Jan Leike, who quit OpenAI in May over safety concerns and happened to be one of several coauthors on the paper explaining how CriticGPT works.
Very exciting that this is out now (from my time at OpenAI):
— Jan Leike (@janleike) June 27, 2024
We trained an LLM critic to find bugs in code, and this helps humans find flaws on real-world production tasks that they would have missed otherwise.
A promising sign for scalable oversight!https://t.co/e6CiXXoCeG pic.twitter.com/EJ6OSfUN9p
OpenAI admits there are some limitations to CriticGPT.
For now, it's only handling answers from ChatGPT "that are quite short." AI models are also still susceptible to hallucinations that can be missed by AI trainers, OpenAI said.
Still, Sam Altman's company seems keen to boost its chatbot's coding chops by trying to catch its errors. CriticGPT clearly has a long way to go, but shows that OpenAI is at least trying to tackle the problem.