AI-based interactive scaffolding improves secondary students' vocabulary achievement, goal setting, self-evaluation, and motivation in informal digital learning of English, with less decline in self-regulated learning skills compared to fixed scaffolding.

Objective: The main goal of this study was to investigate the effects of AI-based interactive scaffolding (AIIS) on secondary students' speaking performance, goal setting, self-evaluation, and motivation in informal digital learning of English (IDLE), and to examine how these self-regulated learning skills develop over time.

Methods: The researchers conducted a quasi-experimental study with 60 seventh-grade students (ages 12-13) from a middle school in Southern China. Students were divided into two groups: a treatment group (TG) that interacted with AI-based interactive scaffolding and a control group (CG) that observed fixed scaffolding without interaction. The experiment was conducted over four weeks with ten sessions covering five topics: family, clothes, emotion, music, and food. Each topic was covered in two consecutive sessions. Students in both groups learned independently using an AI conversational agent developed by LAIX on a mobile app without teacher supervision. Pre-tests and post-tests measured speaking performance, goal setting, self-evaluation, and motivation. Additional mid-tests after the second topic assessed changes in self-regulated learning skills. Speaking performance was evaluated across four sub-dimensions: fluency, vocabulary, grammar, and pronunciation.

Key Findings:

• No significant difference was found in overall speaking performance between the groups, but the treatment group showed significantly better performance in vocabulary.
• The treatment group demonstrated significantly higher goal setting scores compared to the control group.
• While differences in self-evaluation and motivation were not statistically significant, positive effect sizes were observed (0.43 for self-evaluation and 0.22 for motivation).
• Both groups showed similar developmental trajectories of decline and rise in goal setting, self-evaluation, and motivation (decreasing in the middle of the experiment and increasing at the end).
• The treatment group experienced less decline in these self-regulated learning skills during the middle phase of the experiment compared to the control group.
• The frequency of scaffolding provided by AI decreased over time, aligning with the principles of fading and transfer of responsibility in scaffolding theory.
• Instructing (31.68%) and further questioning (19.29%) were the most commonly used scaffolding means, with cognitive activities (56.11%) being the most frequent scaffolding intention.

Implications: The study provides evidence that AI-based interactive scaffolding can effectively support secondary students' vocabulary learning and self-regulated learning skills in informal digital learning environments. The findings extend the application of the integrative framework for scaffolding strategies into AI-based interactive scaffolding and reveal the fading and transfer of responsibility characteristics of AIIS. The research offers pedagogical implications for using AI tools to support independent language learning outside the classroom, suggesting that secondary students can effectively interact with AIIS to learn speaking in IDLE.

Limitations: The limitations include a small sample size, which may restrict the generalizability of the results. The study only measured immediate effects with no delayed post-test to assess long-term impacts. Additionally, the unexpected outbreak of COVID-19 at the end of the intervention may have affected learning progress as students transitioned from classroom instruction to distance learning. The study also focused only on seventh-graders, limiting insights into how AIIS might affect students at different developmental stages.

Future Directions: The researchers suggest future studies should use larger sample sizes and conduct longer-term experimental designs with delayed post-tests to reveal longer-term effects of AIIS. They recommend investigating the influence of formal schooling environments on self-regulated language learning with AIIS in IDLE, examining the effects of AIIS on different age groups within K-12 education, exploring how individual differences shape students' interactions with AIIS, and further investigating the relationship between task features, learner characteristics, and the fading of scaffolding in AIIS.

Title and Authors: "The effects of artificial intelligence-based interactive scaffolding on secondary students' speaking performance, goal setting, self-evaluation, and motivation in informal digital learning of English" by Feifei Wang, Xiaohua Zhou, Kangxin Li, Alan C. K. Cheung, and Ming Tian.

Published On: March 4, 2025

Published By: Interactive Learning Environments (Taylor & Francis Group)