A Large RCT of ALTER-Math – AI Powered Learning by Teachers to Enhance and Renovate Middle School Math Learning

This proposed project evaluates to what extent an AI-powered teachable agent can enhance middle-school math learning by turning students into the AI’s teachers through a large-scale randomized controlled trial. The primary research question is: How effectively does the AI-powered teachable agent engage students and improve mathematical achievement? We hypothesize that students assigned to ALTER-Math will outperform controls on state math assessments, show higher interest in mathematics, and exhibit stronger engagement (time-on-task and persistence), with equitable outcomes for historically underserved students; we also expect effects to generalize beyond Math Nation contexts. The intervention embeds a multimodal AI teachable agent (students explain, correct, and guide the agent across text, tables, graphs, and drawings) into regular coursework, with teachers receiving PD and weekly implementation across a 28-week term. We will run a three-level cluster randomized trial at the school level with 50 schools, 150 teachers, and 7,500 students (50% low-income). Primary outcomes are performance on Florida’s FAST assessment (PM1 baseline to PM3 post), student interest in mathematics (Student Interest in Mathematics Scale), and log-based engagement indicators; subgroup analyses will test differential effects by FRPM, race/ethnicity, and EL status.

This project is supported through the Learning Engineering Virtual Institute (LEVI), in collaboration with J-PAL North America, which builds off of a previous pilot project.