Researchers have detected an apparent disadvantage of smarter chatbots. Although ai models unsurprisingly become more accurate as they advance, they are also more likely to (wrongly) answer questions that are beyond their capabilities rather than saying, “I don't know.” And the humans who cause them are more likely to take their confident hallucinations at face value, creating a confident misinformation effect.
“Nowadays they are responding to almost everything,” said José Hernández-Orallo, a professor at the Polytechnic University of Valencia, Spain. said Nature. “And that means more right, but also more wrong.” Hernández-Orallo, the project leader, worked on the study with colleagues from the Valencian Institute for Research in artificial intelligence in Spain.
The team studied three LLM families, including OpenAI's GPT series, Meta's LLaMA, and open source BLOOM. They tested the first versions of each model and moved on to larger and more advanced ones, but not to today's most advanced ones. For example, the team started with OpenAI's relatively primitive GPT-3 ada model and tested iterations that led to GPT-4, which arrived in March 2023. The four-month GPT-4o was not included in the study, nor was the new o1 preview. I would like to know if the trend continues with the latest models.
The researchers tested each model on thousands of questions about “arithmetic, anagrams, geography, and science.” They also questioned the ai models about their ability to transform information, such as alphabetizing a list. The team ranked their prompts based on perceived difficulty.
The data showed that the proportion of chatbots answering incorrectly (rather than avoiding questions altogether) increased as the models grew. So, ai is a bit like a teacher who, as he masters more subjects, increasingly believes that he has the golden answers for all of them.
To complicate matters further, humans activate chatbots and read their responses. The researchers tasked volunteers with rating the accuracy of the ai robots' responses and found that they “incorrectly classified inaccurate responses as accurate with surprising frequency.” The range of incorrect answers that volunteers falsely perceived as correct generally fell between 10 and 40 percent.
“Humans are not capable of supervising these models,” concluded Hernández-Orallo.
The research team recommends that ai developers start improving the performance of easy questions and programming chatbots to refuse to answer complex questions. “We need humans to understand, 'I can use it in this area and I shouldn't use it in that area,'” Hernandez-Orallo said. Nature.
It's a well-intentioned suggestion that might make sense in an ideal world. But there's a good chance ai companies will do it. Chatbots that most often say “I don't know” would likely be perceived as less advanced or valuable, leading to less usage and less money for the companies that make and sell them. So instead we get warnings in fine print that “ChatGPT may make errors” and “Gemini may display inaccurate information.”
That leaves us to avoid believing and spreading mind-blowing misinformation that could harm ourselves or others. For accuracy, check your damn chatbot's responses, for God's sake.
you can read the complete team study in Nature.
