Artificial intelligence in education (AIEd) has a very large positive effect on learning achievement, but its effectiveness varies based on factors like educational field, level, mode, duration, and geographical location.

Objective: To investigate how AI tools impact learning achievement across different educational contexts and to understand the moderating role of various factors including field of education, level of education, learning mode, intervention duration, and geographical distribution.

Methods:

• Meta-analysis and research synthesis of 85 quantitative studies with 10,469 total participants
• Statistical analyses using ANOVA, correlation analysis, multiple regression, and random forest regression
• Semi-structured interviews with 50 participants
• Use of Hedges' g to calculate effect sizes
• Publication bias assessment using funnel plots and Egger's regression test

Key Findings:

• Overall very large positive effect of AIEd on learning achievement (g=1.10, p<0.001)
• Chatbots showed very large effect (g=1.31), while Intelligent Tutoring Systems (g=1.07) and personalized learning systems (g=0.76) showed large effects
• AIEd had the strongest effect in primary education (huge effect, g=2.72) and higher education (very large effect, g=1.15)
• Online learning showed better results (very large effect, g=1.29) than blended learning (medium effect, g=0.55)
• Shorter interventions (1 week to 1 month) were more effective than longer ones
• Geographical differences were significant, with strongest effects in Africa and Europe

Implications:

• AIEd can significantly enhance learning outcomes when properly implemented
• Educational institutions should consider various moderating factors when implementing AIEd
• Teacher training programs need to incorporate AIEd competencies
• Cross-country collaboration is needed to standardize effective AIEd implementation
• The findings can help achieve UN's Sustainable Development Goal 4 (quality education)

Limitations:

• Study limited to English-language publications
• Limited coverage of geographical regions (no studies from South America)
• Potential publication bias
• Focus on quantitative metrics may miss qualitative aspects of learning
• Did not consider all possible moderating variables

Future Directions:

• Include more databases and non-English publications
• Investigate additional moderating variables
• Examine long-term effects of AIEd on learning
• Study the role of teacher experience and student individual differences
• Research AIEd implementation in underrepresented geographical regions

Title and Authors: "Investigating the effect of artificial intelligence in education (AIEd) on learning achievement: A meta-analysis and research synthesis" by Ahmed Tlili, Khitam Saqer, Soheil Salha, and Ronghuai Huang

Published On: January 3, 2025

Published By: Information Development (SAGE)