Ryan Meuth believes there is huge potential for ai tutors. The professor at the School of Computing and Augmented Intelligence at Arizona State University's Fulton Schools of Engineering is particularly interested in the potential for ai to help the university's online students taking condensed seven-week courses. and a half.
“These are students who have jobs, families and lives,” Meuth says, noting that they are often doing schoolwork late at night or early in the morning, when there may not be anyone around to answer their questions. “They are the students who most need support at the worst times of the day to receive live support.”
Meuth believes the right ai tutor can help these students. “You can give them immediate feedback and assistance, so they can continue to move forward on that compressed timeline,” he says.
To that end, Meuth and Arizona State University have partnered with Wiley, a publishing and educational company, to create a computer science tutor that can help students who get stuck while working in Wiley's zyBooks coding lab, that ASU uses for classes.
The ai tutor, like any good ai tutor, is designed to help students find the answer instead of providing it to them. But creating a tool that actually accomplishes this is easier said than done. So rather than launching this school-wide, Meuth and the Wiley team are conducting several pilot studies and improving the tutor as they go to ensure its long-term effectiveness.
Here's a look at what they've learned so far.
<h2 id="an-ai-tutor-but-not-a-chatbot-3″>An ai tutor but not a chatbot
When we imagine an ai tutor, many of us imagine a ChatGPT-style chatbot interface through which the student can ask any question and the ai will respond with a generative response. However, that's not what's happening with this ai tutor from zyBooks.
“One of the keys to the design is that it's not an open text field,” says Lyssa Vanderbeek, vice president of educational software at the Wiley group. If a student is struggling with a problem, the tutor will appear and provide six different input options, for example offering to give the student a hint or provide an alternative explanation.
This gives the ai tutor a more controlled interface and can keep them focused on helping students solve a problem rather than revealing the answer, Meuth adds. “No matter what kind of protections we put in place as a driving cue, the students, if they have the ability to converse with the object, will trick it into giving them the answer,” Meuth says.
<h2 id="teaching-an-ai-tutor-3″>Teach an ai Tutor
During an initial pilot study with 150 students this fall, the ai tutor was not particularly effective at first. At first it didn't give the most useful clues to students, so it had to be programmed to give better clues, according to Vanderbeek.
“We've really identified the highest-value suggestions,” Vanderbeek says. “Because if a student is stuck, there might be ten things you could suggest, but really, what are the top two things that will get the student out of the plateau?”
He adds, “Through the feedback loop we've had with Ryan and his team, we've been able to hone the tutor so they can better identify the most valuable thing to say to the student.”
For his part, Meuth also noticed that the tutor tended to oscillate between two extremes: giving the answer or giving a clue that was too vague to be useful. He says Wiley's team figured out how to tune into that middle sweet spot, where it's doable, but it's not the answer.
After these improvements, a second group of 350 students began using the ai tutor this fall and are using it ten times more frequently so far. “Students come back to it more often and the responses are very much geared toward helping them get over the barrier,” Meuth says.
Next phase of research
Despite being satisfied with how the ai tutor works, Meuth and the Wiley team are still studying how to best use it.
“We just got approval to start the next experiment, and it will be a study with a control group,” Meuth says. One group will have access to the tutor and another will not. Meuth and colleagues will compare academic performance between the two groups of students and will also assess the self-efficacy of students' attitudes toward programming. They want to know if the students who use the tutor are more or less confident.
Although Meuth loves the way this ai tutor works, he is dedicated to using it with students in a thoughtful, research-backed way. “We worry about things like students becoming too dependent on aid or things like that,” he says. “That's why we want to make sure that the value that students get from using the tutor contributes in a healthy way.”
