The rise of ai promises new solutions to long-standing challenges. It also introduces some challenges of its own. In addition to concerns about privacy, bias and trustworthiness, ai is generating a flood of new products in a wide range of sectors, including education. As options pile up, districts and schools struggle to identify effective solutions amid clever marketing and bold promises.
As a member of the Network LEARNA federally funded initiative dedicated to supporting the development and expansion of quality educational products and programs, I have collaborated with researchers, developers, professionals, and educational leaders across the country. Over the years, our team has gained unique insights into why some products succeed, why others fail, and what districts and schools can consider when selecting new tools and programs.
One lesson we've learned is that the current wave of ai-powered educational technology isn't all that different from the products and programs we're used to. Some products are promising; others fail. Although the options are more abundant and the technology is advanced, schools must continue to be diligent in their selection processes. Based on our work and our conversations with leaders in this space, here are some important questions we should ask ourselves as we look for an edtech solution in the age of ai.
What are you doing?
Effective educational technology has never been designed, and never should, to replace human relationships with students. In response to the rise of ai programs in schools, policymakers in states like California and Minnesota and organizations like the National Education Association are pushing to ensure educators remain at the center of education. Quality educational technology, whether ai-powered or not, should work to improve educational effectiveness and efficiency.
A key differentiator that decision makers can consider is between student-facing ai, with which students interact directly, and products and programs designed for professionals, administrators and other staff. Both uses require unique considerations. For example, for products aimed at students, it is essential that developers use guardrails to prevent bias, protect privacy, and ensure trustworthiness. For administrative applications, considerations will likely focus more on whether educational technology increases efficiency while leveraging the expertise of humans.
Do you have a solid evidence base?
The most critical factor when selecting educational technology is its empirical basis. Is there research to back up your claims? If so, how reliable is that research? As we have discovered in our work, these can be difficult questions to answer.
He Every Student Succeeds Act (ESSA) Evidence Levels can provide a useful framework for evaluating educational technology, describing the different degrees of research that can underlie a product. However, simply meeting an ESSA level of evidence does not guarantee effectiveness. Products or programs that meet the lowest level of ESSA, for example, can only be based on strategies or practices supported by evidence. The products themselves may have never been tested. Researchers Mary Bratsch-Hines and Heather Aiken, leaders behind the TRI-Reading app, expanded on the importance of a comprehensive assessment in a recent episode of The SRI Homeroom Podcast.
“Sometimes people may claim that they are following the science of reading simply because they are covering the five elements of the National Reading Panel… But how they packaged it, we don't necessarily know if the program as a whole will work.” — Mary Bratsch-Hines, senior director of research and evaluation at the Lastinger Center for Learning at the University of Florida.
Educational technology that meets the highest levels of ESSA evidence, Levels 1 and 2, has been evaluated more rigorously and findings are cited in repositories such as the What Clearinghouse works.
However, a lack of evidence does not necessarily mean that a product is ineffective. Some educational technologies, particularly in the age of ai, may be too new to boast an extensive research base. In these cases, products should at least present a compelling case and satisfy a priority need.
Schools may also consider compiling their own set of tests. Examine developer reports, talk to peers in similar contexts, and if you decide to move forward, pilot to generate real, localized evidence. H. Alix Gallagher, director of strategic partnerships at Policy Analysis for California Education (PACE), recently wrote about the pilot process for the LEARN Network blog.
Does it really meet your needs?
While evidence is crucial, it is not the only factor that can influence the success or failure of a new product or program. Schools and districts should also carefully consider their goals. Is there a specific challenge you're trying to address or a set of outcomes you're hoping to improve? Take some time to research your needs, talk to team members, and make sure you fully understand the scope of your challenges and their root causes.
Some products claim to offer broad, widespread benefits and improvements for schools, teachers, and students. in our conversations with educational leadersHowever, we have found that teachers are often unlikely to adopt solutions to problems if they do not perceive them as meaningful. Ensuring that the technology addresses a recognized need is critical to a successful implementation.
Does it fit the context of your school?
Educational technology only works if it can be implemented effectively. Products can fail if they are too cumbersome, don't fit into staff workflows, or don't align with existing programs. Examine your current systems, staff capabilities, and capacity to determine if a product is a good fit.
For example, the developers of A2i, an impactful, broad-scale tool designed to improve literacy outcomes for young students, attributed its success in part to its Integration with various learning management systems. Implementation can be affected when a product requires a radical departure from established processes, duties, or expectations.
Districts and schools can also consider the unique needs of their community. Are there any structural, cultural, or environmental factors that may limit some members' access to a new product or limit its effectiveness for the community as a whole? Ensuring that the product adapts to your specific environment is crucial to your success.
How was the product developed?
Effective educational technology is often the result of extensive discussions, collaborations, reviews, and iterations involving a wide range of stakeholders. At LEARN Network, we encourage researchers and developers to involve school and district leaders, educators, and community members in the design phase of their work. Products that lack sufficient inputs may face unforeseen challenges during their implementation.
in our recent webinar Focused on schools and rural communities, a panel of researchers, practitioners, and educational leaders warned against “drive-through approaches” to product design, encouraging developers to employ a more inclusive, community-centered development approach.
Do you prioritize equity?
Educational technology must be designed to meet the needs of all students, fostering growth and equity through education. When products or programs are not relevant or accessible to members of the communities they serve, they may cause harm unintentionally.
The rise of ai-driven educational technology offers districts and schools a unique opportunity to pursue new tools and programs that are accessible, equitable, and responsive to the needs of diverse learners. in a recent blog from the LEARN networkWe spoke with leading voices from the Strategic Education Research Partnership Institute (SERP) and SRI about this opportunity and some potential paths forward.
An effective method to center equity in the educational technology search process is to elevate student voice. Students are self-aware, highly technologically competent, and have unique experiences and perspectives on learning. By involving students in their pilot and procurement processes, districts and schools can help ensure they are adopting technology that serves all members of their community.
Decision makers can also prioritize student voice in their pursuit of educational technology, considering products that have been designed and developed in collaboration with students. LEARN Network researcher Ela Joshi expanded on the value of student voice in this recent podcast.
artificial intelligence has the potential to drive effective new tools and approaches, reducing the burden on schools, promoting equity and inclusion, and helping students overcome long-standing barriers. However, as we are seeing in other sectors, the letters “ai” do not always indicate quality.
In your search for the next generation of educational technology solutions, we must all avoid the flash and continue to focus on the fundamentals. By prioritizing evidence, understanding specific needs, ensuring contextual fit, examining how products are developed, emphasizing equity, and including diverse voices in search and selection processes, educational leaders can navigate the complexities of educational technology driven by ai and finding products that actually lead to better results.
The information reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through grant R305N220012 to SRI International. The opinions expressed are those of the authors and do not represent the views of the Institute or the U.S. Department of Education.
