Jeremy Price was curious to see if new ai chatbots, including ChatGPT, were biased on issues of race and class, so he designed an unusual experiment to find out.
Price, an associate professor of technology, innovation, and pedagogy in urban education at Indiana University, turned to three leading chatbots — ChatGPT, Claude, and Google Bard, now called Gemini — and asked them to tell him a story about two people who met and learned about each other, with details like the people’s names and context. He then shared the stories with experts on race and class and asked them to code them for signs of bias.
I was hoping to find some, as chatbots are trained on large volumes of data extracted from the Internet, which reflect the demographics of our society.
“The data that is fed into the chatbot and the way society says learning should be seems very superficial,” he says. “It is a mirror of our society.”
Her broader idea, however, is to experiment with creating tools and strategies that help guide these chatbots to reduce biases based on race, class and gender. One possibility, she says, is to develop an additional chatbot that reviews a response from, say, ChatGPT, before sending it to a user to reconsider whether it contains bias.
“You can put another agent over its shoulder,” he says, “so that while it’s generating the text, it will stop the language model and say, ‘Well, wait a second. Is what you’re about to post biased? Is it going to be beneficial and useful to the people you’re chatting with? ’ And if the answer is yes, then it will continue to post it. If the answer is no, then it will have to redo it to make it do that.”
He hopes these tools can help people become more aware of their own biases and try to counteract them.
And without such interventions, he worries that ai could reinforce or even exacerbate the problems.
“We should continue to use generative ai,” he says. “But we have to be very careful and vigilant as we move forward in this area.”
Hear the full story of Price’s work and findings on this week’s EdSurge Podcast.
Listen to the episode on Spotify, Apple Podcastsor in the player below.
