ai tutors have enormous potential. In theory, enthusiasts hope they can provide students with individualized tutoring on a large scale with the same benefits a student could get from a human tutor. In practice, as with almost everything related to education, it is more complicated.
ai tutors have been helpful in some cases, but in others they have seemingly hindered student success. I had the opportunity to interview several experts at the forefront of research on ai tutors, and the consensus is that, used in the right way in the right context, these can probably be useful, but we are still learning how to do it. use technology better.
As a teacher, here are four studies I've turned to to try to understand the strange and still new world of ai tutors. I think any teacher exploring the use of ai tutors or curious about using one in class would benefit from becoming familiar with these findings.
1. A math tutor who hindered your success
In a study of nearly 1,000 students, researchers at the University of Pennsylvania compared the results of students who worked with a GPT-4-powered tutor with those who did not on a math exam. Students with access to the ai tutor performed worse on average than those without.
“We found that generative ai could harm learning because students potentially use it as an answer machine, rather than a tool that supports learning,” Alp Sungu, one of the paper's co-authors, told me.
This is an important study for teachers to keep in mind because it serves as a warning. It clearly demonstrates that greater access to ai does not always equal greater success. However, even Sungu cautioned against taking the findings too far. Study does not show that using ai tutors hurts overall student success, just that relying on one can.
2. Some Bright Points for Chatbot Tutors
Yu Zhonggen, a professor of Foreign Studies at Beijing Language and Culture University, led a team that analyzed 24 studies. compare how interacting with chatbots and ChatGPT-style tutors influenced students.
“Overall, the study found that ai chatbots had a significant positive effect on students' learning outcomes,” Zhonggen told me. “Specifically, ai chatbots were found to improve students' learning motivation, performance, self-efficacy, interests, and perceived value of learning. Additionally, ai chatbots could be useful in alleviating student anxiety.”
However, the full story was a little murkier. These positive impacts were only observed in university students; younger students did not see statistically significant improvements. Furthermore, the positive impact for college students seemed to disappear over time.
<h2 id="3-a-clear-win-for-ai-tutor-interaction-3″>3. A clear victory for tutor interaction with ai
A recent study led by Ying Xu, a professor at the Harvard Graduate School of Education, found ai characters can help young students learn.
For the study, Xu partnered with PBS Kids to conduct a study with more than 200 children ages 4 to 7. The students were divided into three groups of 80 students. All students were shown the PBS KIDS science program, Elinor wonders whywhich is aimed at preschool and early primary school children. A group of students has just watched the program. Another group of students watched the show and received help from an ai-assisted version of Elinor, who is a curious cartoon rabbit. This ai version of Elinor encouraged children to answer questions and offered advice if they got it wrong. The ai-assisted students performed the best and answered most questions correctly, Xu told me.
However, interestingly this study did not use generative ai. Instead, Elinor's ai version chose from pre-selected and vetted answers to help prompt students. I find this approach interesting because it avoids some of the problems that generative ai tutors can have with hallucinating and simply providing unhelpful answers. That said, my feeling is that to realize the full potential of ai tutors, we will need to figure out how to leverage generative ai effectively.
<h2 id="4-ai-helping-human-tutors-3″>4. ai helps human tutors
Given how varied some of the research on human and ai mentoring has been, I'm particularly interested in an artificial intelligence tool designed at Stanford called Tutor CoPilot.
Instead of replacing human tutors, this tool is designed to help them do the work they do more efficiently by training them on different questions they might ask a student, and more. At first glance, this sounds overly optimistic to me, as I generally don't believe in the “ai will help humans do the job they do better” argument. But in this case, that seems to be exactly what is happening.
In a study of the tool, nearly 1,000 students worked with 900 tutors. Students who worked with tutors and used Tutor CoPilot were 4% more likely to master a topic after one session. And students who worked with the lowest-rated tutors saw the most significant benefits, as ai helped compensate.
I am delighted with these results, particularly because a human connection occurs during tutoring that can help the emotional well-being of the student, and this tool helps with that. Furthermore, it suggests to me that there are many creative potential uses for ai tutoring that will be discovered in the future.
Ultimately, we are still in the early days of ai; more successes are likely to occur in mentoring.
