Objective: The main goal of this meta-analysis was to examine the overall effectiveness of artificial intelligence (AI) in enhancing second language (L2) learning and to analyze specific factors that influence this effectiveness.

Methods: The researchers conducted a comprehensive meta-analysis of 15 studies involving a total of 2,156 participants, generating 53 effect sizes. The study employed a bare-bones meta-analysis method using Cohen's d to calculate effect sizes. The analysis used a random-effects model to account for true effect size variations across studies. Six potential moderators were examined: type of AI-assisted interactions (automated feedback, personalization, and intelligent tutoring), type of assessment (productive vs. receptive skills), type of language skills (listening, speaking, writing, grammar, and vocabulary), context of AI-assisted learning (in-class vs. out-of-class), type of technology (ICALL vs. IMALL), and participant grade level (K-12 vs. college).

Key Findings:

• AI-assisted L2 learning demonstrated a large positive effect (d = 1.167) compared to traditional learning without AI assistance.
• Type of AI-assisted interactions was not a significant moderator, although intelligent tutoring showed the highest effect size (d = 1.507), followed by personalization (d = 0.665) and automated feedback (d = 0.268).
• AI-assisted learning was significantly more beneficial for developing receptive skills (d = 2.011) than productive skills (d = 0.909).
• Among language skills, vocabulary learning showed the largest effect size (d = 2.210), significantly higher than listening, speaking, writing, and grammar skills.
• In-class AI-assisted learning (d = 1.834) was significantly more effective than out-of-class learning (d = 0.523).
• Intelligent Mobile-Assisted Language Learning (IMALL) (d = 1.611) demonstrated significantly higher effectiveness than Intelligent Computer-Assisted Language Learning (ICALL) (d = 0.861).
• There was no significant difference in the effectiveness of AI-assisted learning between K-12 students and college learners, suggesting AI benefits both age groups similarly.

Implications: The findings contribute significantly to understanding how AI can enhance L2 learning, offering important insights for instructors and AI application designers. The study highlights the potential of AI to support different aspects of language acquisition, particularly for vocabulary building and receptive skills. The results suggest that integrating AI technologies into classroom instruction is more beneficial than solely recommending them for out-of-class study. Additionally, the effectiveness of IMALL indicates the value of developing mobile-friendly AI applications for L2 learning. The study also demonstrates that AI technologies can effectively support learners across different age groups through adaptability and personalization.

Limitations: The meta-analysis has several limitations. First, it only included experimental and quasi-experimental studies comparing groups with AI assistance to control groups without such assistance, excluding other research designs like within-group designs and qualitative studies. The funnel plot analysis suggested possible publication bias in the included studies. Additionally, many factors that impact L2 learning, such as linguistic distance and intentional versus incidental learning, were not considered in this analysis. The limited number of studies examining certain moderators (particularly automated feedback) may have affected the accuracy of some effect size estimates.

Future Directions: The researchers recommend that future studies employ more rigorous research methods, including pre-tests, to increase the validity and reliability of results. They suggest considering linguistic distance (participants' native language) when examining the effectiveness of specific AI functions for L2 learning. They also encourage researchers to provide more detailed information about methodologies and participants to facilitate a deeper understanding of moderating factors. Future research could benefit from investigating additional moderators that may influence AI's effectiveness in L2 learning.

Title and Authors: "A meta-analysis examining AI-assisted L2 learning" by Guanyao Xu, Aiqing Yu, and Lin Liu.

Published On: March 18, 2025

Published By: International Review of Applied Linguistics in Language Teaching (IRAL)