Today there is a wave of new educational technology products hitting the market, and teachers and professors are increasingly creating educational videos and other materials for their classes. But there is often one group that is left out of the design process: students.
“Many educational products are never shown to students until they have already been designed,” said Elliott Hedman, a consultant who works with educational technology companies, in a talk this month at the SXSW EDU festival.
Today, most major consumer technology and media products undergo extensive testing, he noted, but he says many adults are okay with a much lower standard for anything designed for kids.
“The budget for an episode of Game of Thrones was 15 million dollars. It was probably worth it, it was a very good episode,” he joked. “Sometimes they hire me to write reading stories for children in classrooms and they pay me $80. “That is the amount we are willing to pay for a child’s experience in the classroom.”
Perhaps an extreme example, Hedman admits, but he says the lack of testing really shows up when students receive the materials. While visiting the school's computer labs for a research project, she says, “I discovered that many kids drop out of the educational program and start watching YouTube for an hour.”
In an interview after his presentation, Hedman, who has a Ph.D. from the MIT Media Lab and has been working on educational materials design for more than a decade, said it's not that edtech companies don't test. But he says the testing they do is often ineffective or too limited, such as relying primarily on family members of company employees, who are “well above average in school and very white.”
He notes that there are no good incentives for edtech companies to spend time and effort conducting more detailed testing with students. “They are selling to the government, to the administration, to the district,” she says. “They are not selling to the child: the child has no purchasing power. Children are never really listened to and teachers are rarely listened to. Then they throw it in the classroom and then you ask, 'Did the scores go up?'”
And he argues that conducting more user testing with students during the design process is different from conducting research on the effectiveness of an educational technology product, something a growing number of educational leaders have requested in recent years.
“I'm not saying that effectiveness research is a bad thing,” he says, but he says it can “get in the way” of doing design research, since most research on whether a teaching approach works or not They do not focus on making changes that correct any failure.
He says the ideal approach is to present an educational technology product or learning material in front of a diverse group of students (without a researcher in the room) and video record how they use the tool. Designers should then make small improvements based on what they learn, she says, and continue to do so iteratively throughout the development process. In that way, it's more of a co-creation with students rather than adults building something and assuming it will work for kids.
Experts have also called for more teachers and educators to participate in the development of educational technology products.
Instructors as designers
Increasingly, it is instructors themselves who create learning materials, such as a short video lesson if they are trying a “flipped classroom” approach in which students learn some basic material at home so that the Class time can be used for more interactive activities. . And some experts say that too often students are left out of the design process for these materials, too.
“The challenge with having materials that are not good user experiences is that they make learning difficult,” says Kayla B. McNabb, assistant director of teaching and learning engagement at Virginia tech. “If you want students to do well in their course, then you must reduce the barriers that prevent them from participating in the learning experience.
McNabb, who earned a Ph.D. in rhetoric and composition from Virginia tech, co-authored a paper In 2021, calls on educators to do more to incorporate user testing when designing learning materials.
In many cases, what instructors learn by, for example, playing a short sample of a video they made for a class with some students, is not about the content but about the user experience, McNabb told EdSurge in an interview. . And often the problems posed by students can be easy to solve and can also be applied to other projects.
“Maybe the transition to video is too jarring,” he says. “In that case, let's put a different header image to make it clearer.”
McNabb says a major challenge at universities is that many instructors have never received training in how to teach, much less how to design learning materials. But she says that even spending a small amount of time sharing a portion of the course materials with students for input can help the instructor make changes that will greatly improve impact.
“The first thing people can do is ask for feedback,” he says, noting that any point in the development process can work. “Any time is better than never.”
And if instructors can't find students to test their materials on, he suggests asking a colleague to test it or even present it at a conference. “Any feedback is better than none,” she argues.
Times of changes
The need to test products may become even more important as companies rush to add new ai features to their products, Hedman says.
“The way children use ai tools is very different than we think,” he says. “I have to remind companies: they have never shown that to a child. “It will be very important as companies rush to launch ai-enabled products.”
But he affirms that doing so will mean a change in the culture of many companies.
“In an ideal world, I would like the student voice to be front and center in every edtech company,” he says.
One way to address the challenge may be to increase collaboration between university researchers and the edtech industry, argued Yu-Chen Chiu, a user design researcher, in edtech/bridging-the-gap-between-academia-and-the-edtech-industry-ecbaccbe20a2″ target=”_blank” rel=”noopener nofollow”>a recent post on Medium. The goal, he maintains, is “to develop impactful products that are not only 'effective' but also 'usable' and 'desirable'.”
Engaging students is also important now that they have more ways than ever to find information online or use ai chatbots. That means teachers should focus more on teaching skills and ways of thinking in their discipline, rather than spending so much time on specific content or details that could change quickly or that students can find elsewhere.
Basically, he maintains, the rise of ai will mean that many instructors will need to update the materials they created in the past, and he hopes that more educators will involve students in that redesign process.
“That doesn't mean you have to drop everything and get a master's degree in instructional design,” McNabb says. “It means you have to think clearly about the user experience throughout the course.”
