Reflective AI use and student engagement in AI-supported programming: technology acceptance, programming self-efficacy, and behavioral trace evidence.

INTRODUCTION

AI coding assistants are increasingly used in programming education, but willingness to use them does not show whether students evaluate or learn from AI suggestions. Grounded in technology acceptance, self-regulated learning, and self-efficacy theory, this study examined reflective AI use and student engagement.

METHODS

The Reflective AI Acceptance Integration Framework (RAIAF) organized three independent public datasets: a survey structural equation model (N = 131), double human coding of 66 nonblank Critical Engagement responses from a Copilot field study, and complementary programming-behavior prediction using DTA logs (N = 1, 423; 40% calendar-window sample N = 1, 122). The DTA dataset did not measure AI use and was used only to test whether recovery-related programming traces added predictive information.

RESULTS

In the MLR survey model, technology acceptance predicted reflective use (β = 0.694, p < 0.001) and engagement (β = 0.694, p < 0.001); reflective use predicted programming self-efficacy (β = 0.519, p < 0.001) and engagement (β = 0.273, p = 0.034). None of the hypothesized indirect effects was significant. Initial text-coder agreement was limited (linear-weighted κ = 0.421; ordinal α = 0.495), and consensus-coded reflective use did not significantly predict productive engagement (b = 0.149, p = 0.202). In DTA, adding recovery features increased held-out final-success Macro-F1 from 0.725 to 0.743, but average gains in repeated and 30%-50% window analyses were negligible. Activity-defined engagement was already captured by overlapping activity indicators.

DISCUSSION: The findings distinguish acceptance from reflective use but do not establish a general mediation mechanism. Reflective use was associated with self-efficacy and engagement in the survey, whereas its text and behavioral evidence was exploratory, small, and context dependent.