We can’t keep guessing whether AI helps students learn

On September 2, New York City and Los Angeles, the country's two largest school systems, halted the use of generative AI tools in nearly every student's device for a one-year trial. This decision came after parents voiced concerns about the untested nature of these tools and their potential impact on student learning. New York City Schools Chancellor Kamar Samuels emphasized the importance of focusing on evidence rather than merely embracing new technology.

However, the question remains: where will the necessary evidence come from? Teachers, school leaders, parents, and students are making critical decisions about AI use without dependable information on its effectiveness, suitability, and conditions for implementation. Despite 60% of teachers utilizing AI in their work, 82% claim to have received no formal guidance on its usage. Only half of middle and high schools have AI policies, and just 6% of teachers in those schools deem them clear.

This lack of clarity is evident in public sentiment as well. Thirty-one percent of Gen Z respondents now perceive AI negatively, a significant increase from 22% the previous year. Additionally, 80% of respondents believe that relying on AI may hinder their learning. Nonetheless, banning AI in schools won't erase its presence from students' lives.

Recent surveys indicate that 91% of teens ages 13 to 15 use AI for schoolwork or homework, and with 21% of adults already employing AI at work, students must adapt to the increasing role of AI in their future careers.

It is crucial for school district leaders, educators, and families to understand when AI enhances student learning and when it detracts from it. However, crafting effective policies, training teachers, and making informed purchasing decisions become increasingly challenging when the evidence base is so limited. To address this issue, Congress and the federal administration have an opportunity to build upon existing infrastructure and generate the evidence necessary for safe and confident advancements in student learning.

The Alliance for Learning Innovation, in collaboration with AI researchers, practitioners, and a coalition of over 150 members, has developed federal recommendations in this regard. The primary suggestions include passing the NEED Act, which would establish a national center at the Institute of Education Sciences (IES) to develop and test innovative teaching and learning approaches, similar to the Pentagon's Defense Advanced Research Projects Agency (DARPA).

The center would provide expert program managers who can swiftly allocate funds to promising ideas, invest more upon successful outcomes, and shift focus when necessary.

The federal government should also support school districts across the nation in conducting AI pilots in select classrooms to evaluate the tools' effectiveness tailored to their students' needs. Furthermore, a coordinated national network of testbeds, combining real classrooms with synthetic environments that mimic students' AI usage, would allow researchers to test AI tools across diverse schools and communities.

This approach would yield practical evidence about AI's performance in real-world teaching scenarios, as well as provide clearer indications for developers and investors regarding which tools warrant scaling.

Measuring student learning outcomes is another crucial aspect that current evaluations often fail to address. Evaluations frequently struggle to differentiate between a student's work and AI-generated outputs. Federally funded AI studies should assess how students perform with a tool, then evaluate them again on the same topics later without the tool's assistance.

The American Psychological Association has called for this type of rigorous evaluation, emphasizing that AI-powered learning tools should be judged by the students' performance after they discontinue using them. This approach aligns with the Department of Education's guidance and emerging outcomes-based contracting models, where vendor payments are contingent on results.

Lastly, providing school districts with a uniform evaluation tool is essential. Currently, districts lack a clear definition of what constitutes safe, effective, and ethical AI tools. The National Institute of Standards and Technology (NIST) has already published a voluntary framework for AI risk management that organizations across industries employ.

The IES should collaborate with NIST to create an education-specific version of this framework, grounded in the EDSAFE AI Alliance's SAFE Benchmarks. This standardized evaluation tool would enable states and districts to assess AI tools based on their impact on learning, safety, privacy, and transparency, prior to their integration into classrooms.

In conclusion, the growing adoption of AI in education necessitates a robust evidence base to ensure that these tools enhance student learning. By investing in our federal research infrastructure now, we can establish a trustworthy evidence base nationwide, empowering students to utilize AI tools that have been proven effective when they enter classrooms in Fall 2027.