Middle school students using AI-enhanced Extended Reality (XR) simulations demonstrated significantly higher engagement, motivation, and perceived learning quality compared to students using XR with traditional teacher support alone, while both groups achieved substantial communication skill gains—suggesting AI-XR integration prepares students more effectively for workforce demands where employers increasingly prioritize generative AI experience.
Objective: This study aimed to determine whether AI-assisted Extended Reality tools would improve student communication skills more effectively than teacher-assisted XR tools alone, and to understand student preferences for these technologies to inform how educators can better prepare students for future workforce demands. The research specifically examined whether combining AI chatbot support with XR simulations would enhance middle school students' expressive, receptive, and pragmatic communication abilities compared to human educator support with XR, while exploring implications for both K-12 education and military training contexts where communication proficiency is critical.
Methods: The research employed a rigorous randomized controlled trial design with 100 middle school students (ages 10-13) and 10 educators from public, private, and charter schools across rural, suburban, and urban locations in Kansas, New Mexico, Virginia, and North Carolina. Participants were randomly assigned to two matched groups of 50 students each. Both groups completed 30 Extended Reality simulations designed to teach and assess communication and pragmatic language skills delivered via iPads or Chromebooks from September 2022 to April 2023. The control group (Group 1) communicated with human educators after XR simulations to ask questions and create narratives for skill generalization, while the experimental group (Group 2) communicated with an AI chatbot within MagicSchoolAI for the same purposes. The study used two well-validated assessment instruments: the Clinical Evaluation of Language Fundamentals, Fifth Edition (CELF-5) Pragmatic Profile measuring expressive, receptive, and pragmatic communication as pre-post standardized assessment, and the Computer Simulation Evaluation Scale for Students (CSES-S) measuring student attitudes toward learning quality, engagement, and motivation on a five-point Likert scale. Procedural fidelity was monitored in 35% of professional development sessions through direct observational recording by a Research Assistant using scripts, checklists, and notation forms, calculating percentage of correct implementation behaviors. The intervention was embedded within students' normal social skills instruction time using existing school technology.
Key Findings: A one-way Analysis of Variance (ANOVA) revealed that all students made statistically significant gains in communication skills, F(1,98) = 24.89, p < .001, with a large effect size (eta-squared = .20), with no statistical significance between groups on the CELF-5 Pragmatic Profile outcomes. However, students in the AI-assisted group reported significantly more positive attitudes (M = 4.11) compared to the teacher-supported group (M = 3.19) on the CSES-S total score. Individual item analysis revealed statistically significant improvements for the AI-assisted group across three critical domains. In learning outcomes, students reported substantially higher understanding and skill acquisition, with statements like "Working with the AI helped me learn effectively" showing statistical significance at p < .001. Regarding quality perceptions, AI-created narratives and feedback were perceived as significantly higher quality and more relevant, with feedback items demonstrating p < .001 significance. For engagement and motivation, students were significantly more motivated by and engaged with AI-enhanced simulations than traditional instruction, with motivational items reaching p < .001 significance levels. Notably, the CELF-5 Pragmatic Profile measuring Expressive and Receptive Communication showed significant gains in how students express themselves and understand others in both XR-using groups, though the AI-enhanced group demonstrated slightly increased gains compared to the teacher-only group. An interesting demographic finding revealed that over 15% of participating middle school students mentioned military employment and service within their stated career aspirations, including roles as pilots, medics, and in legal fields, underscoring the relevance of communication training for future military contexts.
Implications: The findings carry profound implications for preparing students for workforce demands in an AI-driven economy. The 2024 Cengage Group Employability Report indicates that 62% of employers believe hires should have generative AI knowledge, with 58% more likely to interview candidates with AI experience—trends directly aligned with this study's demonstration that students engage more effectively with AI-enhanced learning tools. The research suggests that aligning instructional methods with both student preferences and employer expectations simultaneously fulfills educational goals and critical workforce readiness demands. For military education specifically, the implications are substantial given that military environments operate under high-stakes conditions where instructional quality, adaptability, and communication efficiency directly influence operational readiness. The study demonstrates several military-relevant applications including reduced instructor burden through AI automation of grading, lesson planning, and content generation, allowing military educators to reallocate time toward live debriefing, mentorship, and personalized coaching; adaptive instruction capabilities where AI tailors scenarios based on user behavior and performance, providing skill-specific remediation or advanced challenges appropriate for diverse military roles; and embedded communication training where AI-integrated XR simulations reinforce communication protocols, team cohesion, and behavioral expectations during joint operations. The integration approach proved practical and non-disruptive, as the entire intervention was delivered over Zoom using students' existing Chromebooks and iPads, embedded within their normal social skills instruction time without requiring significant policy or practice changes—a model readily transferable to military training contexts. The strong student preference for AI-supported narrative creation after XR learning suggests that military personnel training may benefit similarly from XR training with AI feedback for developing critical communication skills and situational understanding essential for complex operational environments.
Limitations: Several limitations contextualize the study's findings and conclusions. First, the sample size of 100 students across 10 schools, while adequate for statistical analysis, represents a relatively modest scale that may limit generalizability across broader and more diverse student populations, particularly in military educational contexts not directly studied. Second, the study duration of approximately seven months (September 2022 to April 2023) captured short-term outcomes but could not assess long-term retention of communication skills or sustained engagement with AI-XR tools over multiple years—a consideration particularly important for understanding career-long skill development in military contexts. Third, the geographic concentration in four U.S. states (Kansas, New Mexico, Virginia, North Carolina) may not represent educational contexts in other regions, international settings, or specific military training facilities with unique technological infrastructure or cultural norms. Fourth, while both groups included students with various diagnosed disabilities (ADHD, ASD, OCD, depression, learning disabilities, anxiety, intellectual disabilities), the relatively small numbers within specific diagnostic categories limited ability to conduct subgroup analyses examining how AI-XR effectiveness varies across different disability types—relevant for understanding applications in military populations with diverse cognitive and learning profiles. Fifth, the study utilized specific technology platforms (MagicSchoolAI chatbot, existing XR simulations) whose features, capabilities, and limitations may not generalize to other AI or XR systems, and rapid technological advancement means these specific tools may quickly become outdated or superseded by more capable alternatives. Sixth, procedural fidelity monitoring in 35% of sessions, while methodologically sound, means that implementation variation in the remaining 65% of sessions could have influenced outcomes in unmeasured ways. Seventh, reliance on self-reported attitude measures (CSES-S) introduces potential social desirability bias where students might respond according to perceived expectations rather than genuine preferences. Finally, the study focused specifically on communication skill development and did not examine other critical competencies like decision-making under pressure, tactical reasoning, or technical skill acquisition that would be essential in military training contexts, limiting conclusions about broader AI-XR applications.
Future Directions: The research identifies several crucial directions for advancing knowledge and practice. First, longitudinal research is essential to understand whether the engagement and learning advantages observed in this study persist over time, examining how sustained AI-XR use over multiple years affects long-term communication competency, skill transfer to real-world contexts, and career readiness—particularly important for understanding developmental trajectories in military education where skills must be retained and applied throughout careers. Second, field validation in authentic military training environments is urgently needed to refine these tools and determine how findings from middle school educational settings translate to high-stakes military contexts with different performance demands, stress levels, and operational complexities. Third, research examining optimal AI-human instructor ratios and timing is necessary to understand when AI-facilitated practice, dialogue, and question-answering is most appropriate versus when human teachers and peer interaction should be prioritized—helping establish evidence-based guidelines for balancing automated support with human mentorship to foster resilience, adaptability, and clear communication without over-relying on technology. Fourth, investigation of how XR will extend to new military roles such as Joint Terminal Attack Controllers, drone pilots, and other emerging specializations requiring sophisticated communication and coordination skills would expand understanding of AI-XR applications across diverse military occupational specialties. Fifth, research on incorporating AI-facilitated narratives after XR experiences to build empathy, communication, and team-based decision-making competencies represents a promising direction, particularly examining whether practice with AI reduces performance anxiety or fear of disappointing educators, potentially enabling more authentic skill development. Sixth, studies examining how AI's real-time analytical capabilities can provide instructors with deeper insights into trainee performance—including communication style effectiveness, decision-making trends under pressure, and group cohesion dynamics—would enhance after-action review processes and targeted intervention development. Seventh, cross-cultural and international research is needed to understand how AI-XR tools function across diverse military contexts globally, examining cultural differences in technology acceptance, communication norms, and training approaches. Finally, research addressing ethical implementation, data privacy protection, algorithmic transparency, and equity of access remains critical to ensure AI-XR tools enhance rather than undermine educational quality, with particular attention to establishing clear standards and best practices that balance innovation with learner protection in both civilian and military educational contexts.
Title and Authors: "Assessment and Communication through AI-Enhanced Extended Reality Simulations: Educating the Future Workforce" by Maggie Mosher, Lisa Dieker, and Amber Rowland, University of Kansas.
Published On: October 2025 (Preprint)
Published By: ResearchGate