Semantic features extracted from students' small group conversations during AI literacy lessons significantly predict their interest in learning AI, while social knowledge construction characteristics do not show significant correlation.

Objective: To examine how students' small group conversations during AI literacy lessons predict their situational interest in AI learning, focusing on identifying conversational characteristics that indicate student engagement.

Methods:

• Study conducted in a high school cognitive science class
• 17 students (15 females, 2 males, ages 16-18)
• Two one-hour sessions
• Students worked in five small groups
• Data collection through:
  • Video recordings of group conversations
  • Transcript analysis using LIWC (Linguistic Inquiry and Word Count)
  • Interaction analysis model for social knowledge construction
  • 5-point Likert scale situational interest surveys
• Analysis of three feature sets:
  • Basic features (sentence count, word length, etc.)
  • Annotation features (knowledge sharing, testing, etc.)
  • LIWC features (AI focus, creativity, innovation)

Key Findings:

• LIWC features significantly improved prediction of student interest
• Model with LIWC features showed highest adjusted R-square (0.74)
• Positive correlations found between interest and:
  • AI-related content
  • Creativity/innovation discussions
  • Analytical conversations
• Social construction of knowledge showed no significant relationship
• Basic conversational features alone were poor predictors

Implications:

• Semantic features can serve as indicators of student interest
• Analysis could lead to development of real-time feedback tools
• Findings support importance of content-focused discussions
• Results could help improve AI literacy education effectiveness

Limitations:

• Small sample size (14 unique students)
• Limited to two sessions
• Gender imbalance in participant group
• Focus on single school/class context
• Preliminary nature of analysis

Future Directions:

• Incorporation of multimodal features
• Development of classroom analytics tools
• Analysis of paralinguistic cues
• Integration of smartpen data
• Exploration of cognitive and social process dynamics

Title and Authors: "Predicting Students' Interest from Small Group Conversational Characteristics: Insights from an AI Literacy Education with High School Students" by Shenghua Zha, Lujie Karen Chen, Woei Hung, Na Gong, Pamela Moore, and Bethany Klemetsrud

Published On: To be presented February 26-March 1, 2025

Published By: Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 2 (SIGCSE TS 2025)

The study provides valuable insights into how conversational analysis can predict student interest in AI education, suggesting potential tools for improving teaching effectiveness in this emerging field.