Choosing Wisely: Lessons for Leaders in AI Integration

The path to their collective goal is equally short: continuous research, customer feedback, and growth.

And they have grown, they have. Since we spoke with Malkin when he became CEO seven years ago, Carnegie has added 500 new employees, 500 part-time tutors, four new adaptive ai products, countless research projects and a new headquarters in Canada.

Today, after seven years of attending classrooms, speaking with school leaders, a pandemic, and the nationwide leap toward ai-mania, Malkin has unique insights to share about how schools can choose ai tools wisely, how Carnegie fits into the picture of ai and how ai can be a source of positive reinforcement for students.

EdSurge: What differentiates Carnegie Learning's approach to ai from other educational technology offerings?

Malkin: Our origin story is that of an ai-powered product initially launched by Carnegie Mellon University through Carnegie Learning. Twenty-five years ago, Carnegie Mellon University created the first ai-powered adaptive tutor for teaching high school mathematics, MATHia, which remains one of our flagship products.

Carnegie Learning was ahead of its time with that first version of ai. Of course, the technology was not as advanced as today, but it was still a product powered by artificial intelligence and adaptive learning. When the generative ai revolution arrived, we were well positioned to step forward as a leader, take that ai knowledge and apply it in a way that directly supported students and teachers.

We have people who understand the technology behind ai. We have researchers who have studied it for 25 years. ai was already part of Carnegie Learning's DNA, and now we're moving fast and furious to integrate it into products in new and impactful ways.

Drawing on such deep experience, what do you think school leaders and teachers can keep in mind when choosing effective ai tools?

There are already many technology solutions in education, but only a few demonstrate real value, and ai is no different. Anyone can create a technology product focused on education, but not everyone can create a technology product influenced by science learning and research that truly makes a difference.

Everyone has access to a large language model (LLM) that will allow you to create an educational application that will appear to look the same in many companies in the sector. Only those companies that understand cognitive models and learning sciences and have data to add something substantial to that LLM will provide a significantly differentiated product.

At Carnegie Learning, for example, we are thoughtful and purposeful about integrating generative ai into our products as an enhancement to the work we already do, and that is critical.

We are integrating it into our curriculum. It is not just another tool that educators and students access. It is a tool integrated into the Carnegie Learning ecosystem that gives educators and students one more arrow in their quiver to help them solve a challenge or motivate them to learn more.

Schools should consider those things when adopting ai tools. What is the real value of the tool for students and teachers? Is it thoughtful, purposeful, and most importantly, research-based?

Obviously, research is vital at Carnegie. What does that look like in practice?

We have a great research team that constantly investigate the effectiveness of our products and carries out a system of continuous improvement so that our solutions are increasingly better.

Its spirit is relentless questioning and exploration.

Our research team constantly challenges itself: let's find out if there are biases in this mathematical problem, let's test it in different communities and find out which language resonates positively to improve the results. How can we better understand students' misconceptions? Are there patterns in the types of errors students make? This is essential.

They are never satisfied with the status quo and are very intentional about introducing product and content improvements that make a difference.

The Carnegie research team is also involved in large-scale studies such as the Gold Standard RAND Study funded by the U.S. Department of Education and smaller studies with districts and schools. Data collection and independent validation, and I emphasize the independent assessment, which is unique, is essential to the credibility of our products. All of that informs our processes to make our products more effective. That's a key part of our goal.

How do you think ai supports that goal of giving all students access to equitable and personalized learning?

If we use this technology the right way, it can give students more of what they want (what I remember wanting as a student): more engagement, empowerment, and context. Today's students deserve it.

While it is great for students to understand the theory, they must also understand the practical applications of their learning. “What can I do with this knowledge? How can I take it beyond the classroom?

We spent a lot of time sitting in classrooms and observing; There is no reason why curriculum and technology cannot be inspiring for all students. technology like adaptive ai can help greatly through personalization. And the more we can do that, the more successful we will be in igniting the passion within each student.

I struggled as a math student in high school and remember sitting in the classroom as the class progressed while I was still discussing concepts that prevented me from moving faster. It's a difficult place to be and a horrible feeling. I am motivated to help all students, but especially students who need additional support. Carnegie Learning products, especially our ai-powered solutions, are well-positioned to help students stay on grade level at all times.

If we can help students achieve ever greater results, we will have accomplished something amazing. That's really what we're doing here.