Objective: To evaluate the impact of Rori, an AI-powered math tutor accessible via WhatsApp, on math performance of students in Ghana.
Methods:
- Randomized controlled trial with approximately 500 students in grades 3-9 from 11 schools in Ghana
- Treatment group used Rori for two 30-minute sessions weekly during study hall
- Control group continued regular math instruction
- Math assessments given at baseline and after 8 months
- Difference-in-difference analysis to measure impact
Key Findings:
- Treatment group showed significantly higher math growth scores (p < 0.001)
- Effect size of 0.36, considered large for educational interventions
- Improvement equivalent to approximately an extra year of learning
- Marginal cost of providing Rori is approximately $5 per student
Implications:
- Chat-based AI tutoring could offer a cost-effective, scalable approach to enhancing learning outcomes in low-resource settings
- Potential for widespread implementation due to high mobile phone usage in West Africa
- Could be a viable alternative to expensive personal computers and high-speed internet for educational technology in LMICs
Limitations:
- Results only report on year 1 of the intervention
- Potential for unobserved differences between treatment and control groups
- Assessment design may have understated growth for higher-grade students
Future Directions:
- Examine the relationship between time spent using Rori and improved math ability
- Investigate optimal usage time and potential interference with other subjects
- Employ more diverse and comprehensive assessment tools
- Study Rori's impact in other schools, countries, and home-based settings
Title and Authors: "Effective and Scalable Math Support: Experimental Evidence on the Impact of an AI-Math Tutor in Ghana" by Owen Henkel, Hannah Horne-Robinson, Nessie Kozhakhmetova, and Amanda Lee
Published On: February 2024 (based on the arXiv naming convention) Published By: arXiv (preprint)