An active learning intervention that exposes students to ChatGPT's limitations can significantly reduce their behavioral intention to use it inappropriately for academic writing tasks.

Objective: To determine whether students can be dissuaded from inappropriate use of generative AI in academic writing through an active learning intervention that demonstrates ChatGPT's limitations.

Methods:

• Conducted at Noroff University College with 173 undergraduate computing students
• Used a multi-stage intervention incorporating active learning approaches
• Measured changes in students' perceptions using Technology Acceptance Model (TAM)
• Collected data through pre- and post-intervention surveys (N=159 pre, N=95 post)
• Used McNemar-Bowker test to analyze changes in student perceptions
• Evaluated seven key characteristics of academic writing

Key Findings:

• Significant decrease in students' intention to use ChatGPT for finding academic references (56.46% to 15.39%)
• Reduced student confidence in ChatGPT's critical insight capabilities (48.33% to 20.88%)
• Students maintained positive views of ChatGPT for improving writing structure and conciseness
• Statistically significant changes in behavioral intentions for 6 out of 7 academic writing characteristics
• English as second language students found particular value in ChatGPT for improving formal writing tone
• Students developed more nuanced understanding of ChatGPT's limitations and appropriate use cases

Implications:

• Demonstrates effectiveness of active learning interventions in addressing AI over-reliance
• Shows importance of allowing students to discover AI limitations through hands-on experience
• Suggests need for holistic approach combining ethical guidelines with practical technology training
• Indicates value of focusing on appropriate rather than prohibited use of AI tools
• Highlights importance of considering language proficiency in AI tool adoption

Limitations:

• Single student cohort at one institution
• Focus only on ChatGPT 3.5
• Potential bias from participant dropout between surveys
• Lack of detailed demographic data about participants
• Limited long-term follow-up on intervention effects

Future Directions:

• Test intervention effectiveness with newer AI models
• Conduct longitudinal studies to assess lasting impact
• Develop similar interventions for other subject areas
• Investigate role of language proficiency in AI tool usage
• Create frameworks for appropriate AI integration in education

Title and Authors: "Addressing the use of generative AI in academic writing" by Johan van Niekerk, Petrus M.J. Delport, and Iain Sutherland

Published On: December 12, 2024

Published By: Computers and Education: Artificial Intelligence (Elsevier)