What makes chatbots “freak” or say the wrong thing?

In today’s AI newsletter, the third of a five part seriesI discuss some of the ways that chatbots can fail.

A few hours after the bulletin was published yesterday, a group of AI experts and technology leaders, including Elon Musk, urged AI labs to stop work on their most advanced systems, warning that they pose “profound risks to society and humanity”.

The group called for a six-month pause on systems more powerful than GPT-4, introduced this month by OpenAI, which Musk co-founded. A pause would allow time to implement “shared security protocols,” the group said in an open letter. “If such a pause cannot be quickly enacted, governments should step in and institute a moratorium.”

Many experts disagree on the seriousness of the risks cited in the letter, and we’ll explore some of them later this week. But a number of AI mishaps have already surfaced. I’ll spend today’s newsletter explaining how they happen.

In early February, Google introduced a new chatbot, Bard, which answered questions about the James Webb Space Telescope. There was just one problem: One of the bot’s claims, that the telescope had captured the first images of a planet outside our solar system, was completely false.

Bots like Bard and OpenAI’s ChatGPT deliver information with unnerving dexterity. But they also tell plausible falsehoods or do things that are downright creepy, like insisting they’re in love with New York Times journalists.

How is that possible?

Internet garbage and hallucinations

In the past, technology companies carefully defined how software was supposed to behave, one line of code at a time. Now, they are designing chatbots and other technologies that learn skills on their own, identifying statistical patterns in vast amounts of data.

Much of this data comes from sites like Wikipedia and Reddit. The Internet is full of useful information, from historical facts to medical advice. But it’s also full of falsehoods, hate speech, and other rubbish. Chatbots absorb everything, including explicit and implicit ones inclination of the text they absorb.

And because of the amazing way they mix and match what they’ve learned to generate entirely new text, they often create compelling language that’s completely wrong or doesn’t exist in their training data. AI researchers call this tendency to invent things a “hallucination”, which may include irrelevant, nonsensical or factually incorrect answers.

We’re already seeing the real-world consequences of AI hallucination. Stack Overflow, a question and answer site for programmers, temporarily banned users from sending responses generated with ChatGPTbecause the chatbot made it too easy to send plausible but incorrect answers.

“These systems live in a world of language,” said Melanie Mitchell, an AI researcher at the Santa Fe Institute. “That world gives them some clues about what is true and what is not true, but the language they learn from does not it is based on reality. They don’t necessarily know if what they are generating is true or false.”

(When we asked Bing for examples of chatbot hallucinations, it actually hallucinated the answer.)

Think of chatbots like jazz musicians. They can digest vast amounts of information, such as every song ever written, and then critique the results. They have the ability to put ideas together in surprising and creative ways. But they also hit wrong notes with absolute confidence.

It’s not just them, it’s us

Sometimes the wild card is not the software. They are the humans.

We are prone to seeing patterns that don’t really exist, and assuming human-like traits and emotions into non-human entities. This is known as anthropomorphism. When a dog makes eye contact with us, we tend to assume that he is smarter than he really is. This is how our minds work.

And when a computer starts to put words together like we do, we get the wrong impression that it can reason, understand, and express emotion. We can also behave unpredictably. (Last year, Google placed an engineer on paid leave after dismissing his claim that his AI was sentient. He was later fired.)

The longer the conversation goes on, the more influence a large language model has on what it says. Kevin’s infamous conversation with Bing is a particularly good example. After a while, a chatbot can begin to reflect your thoughts and goals, according to researchers like AI pioneer Terry Sejnowski. If you ask him to get creepy, he gets creepy.

He compared the technology with the Mirror of Erised, a mystical artifact in the Harry Potter novels and movies. “It provides what you’re looking for, what you want, expect or desire,” Dr. Sejnowski said. “Because both the human and the LLMs mirror each other, over time they will tend towards a common conceptual state.”

Can they fix it?

Companies like Google, Microsoft, and OpenAI are working to solve these problems.

OpenAI worked to refine the chatbot using feedback from human testers. Using a technique called reinforcement learning, the system gained a better understanding of what it should and shouldn’t do.

Microsoft, for its part, has limited the duration of conversations with its Bing chatbot. It is also patching vulnerabilities that intrepid users have identified. But fixing every setback is difficult, if not impossible.

So yes, if you’re smart, you can probably persuade these systems to do things that are offensive or creepy. Bad actors can do it too: The concern among many experts is that these bots will allow Internet scammers, unscrupulous marketers, and hostile nation states to spread false information and cause other kinds of trouble.

a big thing

While using these chatbots, stay skeptical. See them for what they really are.

They are not sentient or conscious. They are smart in some ways, but dumb in others. Remember that they can be wrong. Remember that they can make things up.

But on the plus side, there are plenty of other things these systems are great for. Kevin will have more on that tomorrow.

Your homework

Ask ChatGPT either Bing to explain something you already know a lot about. Are the answers accurate?

If you get interesting answers, right or wrong, you can share them in the comments.

Proof

Question 1 of 3

How do large language models generate text?

Begin the quiz by choosing your answer.

Glossary

Hallucination: A well-known phenomenon in large language models, in which the system provides an objectively incorrect, irrelevant, or meaningless answer, due to limitations in its training data and architecture.

Inclination: A type of error that can occur in a large language model if its output is biased by the model’s training data. For example, a model may associate specific traits or professions with a certain race or gender, leading to inaccurate predictions and offensive responses.

Anthropomorphism: The tendency of people to attribute human-like qualities or characteristics to an AI chatbot. For example, you may assume that you are kind or cruel based on your responses, even though you are not capable of emotions, or you may believe that the AI ​​is sentient because it is so good at mimicking human language.

Click here for more glossary terms.