Instructional videos significantly enhance student science learning in digital game-based environments, while GenAI-supported chatbots alone show no main learning effects but can reduce cognitive load when combined with videos.

Objective: This study investigated the individual and combined effects of two scaffolding tools in a digital game-based learning (DGBL) environment for secondary science education: instructional videos providing systematic content explanations and a GenAI-supported chatbot offering adaptive, contextual guidance during gameplay.

Methods: The researchers employed a 2×2 factorial experimental design with 160 seventh-grade students (aged 12-13) from Taiwan, randomly assigned to four groups: video only, GenAI chatbot only, video + GenAI chatbot, and control. Students played "Summon of Magicrystal," an adventure-simulation game focused on Newton's Laws of Motion, during two 45-minute sessions totaling 90 minutes. The study measured science learning outcomes using pre-post knowledge tests, cognitive load through validated surveys, in-game performance metrics, and game behaviors through sequential trace data analysis. The GenAI chatbot was powered by ChatGPT 3.5-turbo and provided contextual, adaptive support without giving direct answers, while instructional videos offered structured explanations of physics concepts relevant to game activities.

Key Findings:

• Instructional videos demonstrated a significant positive main effect on science learning outcomes with a large effect size (F(1,153)=55.64, p<0.001, partial η²=0.27), with both video groups significantly outperforming non-video groups.
• The GenAI chatbot alone showed no significant main effects on science learning, cognitive load, or overall game performance.
• A significant interaction effect emerged for extraneous cognitive load (F(1,153)=6.75, p=0.01, partial η²=0.04), where the combination of videos and chatbot resulted in the lowest extraneous load.
• GenAI groups made significantly fewer game attempts, suggesting potential novelty effects or behavioral fragmentation.
• Sequential analysis revealed that the GenAI-only group displayed the most fragmented inquiry behaviors, while the video + GenAI group showed more cohesive and integrated game behaviors compared to GenAI alone.

Implications: This research provides crucial evidence for educators implementing scaffolding in DGBL environments. Instructional videos serve as highly effective "just-in-time teachers," providing systematic knowledge delivery without increasing cognitive burden. The findings suggest that standalone GenAI chatbots may not automatically enhance learning for secondary students, highlighting the importance of self-regulated learning skills for effective AI interaction. However, combining structured scaffolds (videos) with adaptive ones (GenAI chatbots) can create synergistic effects that reduce cognitive load and promote more coherent learning behaviors. This has significant implications for designing hybrid scaffolding approaches that leverage both systematic content delivery and personalized adaptive support.

Limitations: The study faced several important constraints including the short 90-minute intervention duration, which may have been insufficient for students to develop effective chatbot interaction skills. The novelty effect of GenAI technology could have influenced student behavior and engagement patterns. The focus on seventh-grade students in Taiwan limits generalizability across age groups and cultural contexts. The text-based chatbot modality might not represent the full potential of multimodal AI assistance. Additionally, the lack of a true control group without any scaffolding tools and the potential for testing effects using identical pre-post assessments present methodological considerations.

Future Directions: Research should explore longer intervention periods to allow familiarity development and mitigate novelty effects. Investigating the role of self-regulated learning skills in GenAI effectiveness, particularly through embedded SRL support within chatbots, represents a crucial research avenue. Studies should examine multimodal chatbot capabilities beyond text-based interaction and explore optimal combinations of structured and adaptive scaffolding across different educational levels. Cross-cultural replication studies and investigation of subject-specific effectiveness would enhance generalizability. Additionally, research should focus on developing adaptive models that balance engagement with preventing over-reliance on AI tools.

Title and Authors: "Instructional Video and GenAI-Supported Chatbot in Digital Game-Based Learning: Influences on Science Learning, Cognitive Load and Game Behaviours" by Kun Huang from the University of Kentucky and Ching-Huei Chen from National Sun Yat-sen University, Taiwan.

Published On: July 6, 2025 (accepted), with final publication in 2025

Published By: Journal of Computer Assisted Learning, published by John Wiley & Sons Ltd.

This study makes significant contributions to our understanding of how different scaffolding approaches function in technology-enhanced learning environments. The research demonstrates that while GenAI chatbots represent promising educational technology, their effectiveness depends heavily on implementation context and learner characteristics. The finding that instructional videos consistently enhance learning while GenAI chatbots require careful integration suggests that educators should prioritize proven scaffolding methods while thoughtfully incorporating emerging AI technologies. The study's emphasis on behavioral analysis through sequential trace data provides valuable insights into how students actually interact with different support tools, revealing that the combination of structured and adaptive scaffolds can create more coherent learning experiences than either approach alone.