Here's a detailed summary of the research article:

Final Statement: Students with high AI literacy demonstrate better academic writing performance and more effective prompt patterns when using GenAI compared to students with low AI literacy.

Objective: The study aimed to analyze differences in students' prompt patterns when interacting with GenAI on academic writing tasks and examine how these patterns relate to writing performance based on students' AI literacy levels.

Methods:

• 19 international students from a Sino-British university participated in the study
• Students were divided into high and low AI literacy groups based on a 17-item survey
• Students completed academic writing tasks using a ChatGPT4-embedded writing system
• Analysis included content analysis of student-GenAI chat histories, categorical data analysis of interviews, and evaluation of writing performance by five experts
• Used Lag-Sequential Analysis and Gephi 0.10.1 for visualizing pattern differences
• Writing performance was assessed using a 5-Likert scaled evaluation rubric

Key Findings:

• High AI literacy students showed:
  • More descriptive, context-based prompts
  • Collaborative interaction approaches
  • Better writing performance across content, structure, and expression
  • More sophisticated use of AI for metacognitive activities
• Low AI literacy students demonstrated:
  • More general prompts
  • Student-directed approaches
  • Limited engagement with AI
  • Less sophisticated use of AI tools

Implications:

• The study suggests the need for:
  • Designing educational GenAI systems that facilitate diverse cognitive engagements
  • Guiding students toward interactive engagement with AI tools
  • Developing students' AI literacy to enhance their learning outcomes
  • Integrating AI literacy into educational frameworks

Limitations:

• Small sample size (19 students)
• Focus on only two levels of AI literacy (high/low)
• Limited to specific writing task types
• Analysis focused only on student prompts, not the complete dialogue

Future Directions:

• Investigate additional student characteristics (study areas, gender, nationality)
• Develop more fine-grained groupings of AI literacy levels
• Explore various topics and interdisciplinary tasks
• Analyze complete dialogues between students and GenAI
• Examine long-term impacts of AI literacy on learning outcomes

Title and Authors: "Students' prompt patterns and its effects in AI-assisted academic writing: Focusing on students' level of AI literacy" by Jinhee Kim, Seongryeong Yu, Sang-Soog Lee & Rita Detrick

Published On: January 24, 2025

Published By: Journal of Research on Technology in Education

Note: While this study was published in 2025, it represents important research in understanding how AI literacy levels affect student interactions with generative AI tools in academic contexts. The findings suggest that developing AI literacy is crucial for effective educational outcomes when using AI-assisted learning tools.