Using ChatGPT as a virtual mentor for K-12 students learning science improved student grades by 30% and provided both functional and emotional support when teachers were unavailable.

Objective: To evaluate the real impact of using ChatGPT as a virtual mentor for K-12 students learning science, within a blended learning framework that combines in-person teaching with AI-assisted independent learning.

Methods:

• Systematic assessment of ChatGPT's performance through a 52-question test covering K-12 chemistry and physics
• Quasi-experimental study with control and experimental groups (23 students total)
• Student grades analyzed before and after ChatGPT intervention
• Student perceptions measured using 5-point Likert scale surveys
• Evaluation conducted over multiple academic terms using both GPT-3.5 and GPT-4 models

Key Findings:

• ChatGPT demonstrated high accuracy in K-12 science (9.3/10 with GPT-3.5, improved to 9.7/10 with GPT-4)
• Experimental group showed 30% grade improvement compared to control group
• Student perception of ChatGPT as a useful educational tool increased from 37% to 70% after extended use
• AI provided both functional (academic) and emotional support for students lacking access to help at home
• 90% of students in experimental group showed improved performance regardless of initial proficiency level

Implications:

• ChatGPT can effectively supplement traditional teaching when teachers are unavailable
• The tool helps democratize access to educational support
• Can provide both academic and emotional support for students
• Particularly beneficial for students without access to tutoring or homework help at home
• Promotes student autonomy and self-efficacy

Limitations:

• Small sample size (23 students from one school)
• Non-randomized experimental design
• Potential Hawthorne effect (subjects aware of being studied)
• Some mathematical hallucinations in ChatGPT's responses
• Limited ability to handle image inputs/outputs

Future Directions:

• Conduct larger-scale randomized studies across multiple schools/regions
• Evaluate impact across different subjects and educational levels
• Assess long-term effects on student learning and development
• Study impact on teachers and teaching methodologies
• Investigate applications for students with learning difficulties

Title and Authors: "Use of ChatGPT as a Virtual Mentor on K-12 Students Learning Science in the Fourth Industrial Revolution" by Rafael Castañeda et al.

Published On: December 5, 2024

Published By: Knowledge (MDPI journal)