Human or Machine? EFL Students’ Trust Calibration in AI-Generated English Content:A Study at the Faculty of Languages, Surman

Abstract

Generative artificial intelligence (GenAI) is increasingly used by university EFL students for language learning and academic tasks, yet fluent output may be inaccurate. This study examined self-reported trust and verification practices and students’ judgments of selected AI-generated English content among undergraduates at the Faculty of Languages, Surman, University of Sabratha. A quantitative descriptive-correlational design was applied to questionnaire responses from 38 students. The instrument included background questions, ten five-point Likert items, five evaluation tasks with confidence and verification-intention responses, and an optional open-ended item. Descriptive statistics, Cronbach’s alpha, and Spearman’s rank correlations were used. Mean self-reported trust was 3.36/5 and verification was 3.74/5. Students correctly judged 62.1% of task responses under the primary answer key. Use frequency was not significantly associated with trust (ρ = .279, p = .090), whereas verification was associated with accuracy (ρ = .473, p = .003), as was confidence (ρ = .648, p < .001). Findings are preliminary given the small sample, modest trust-scale internal consistency, and ambiguity in one evaluation item. In exploratory HC3 regression, confidence remained associated with accuracy (β = .51, p = .005), while reported verification did not uniquely reach p < .05 after confidence was included (p = .063). After Holm correction across 28 exploratory pairs, only confidence–accuracy and awareness–adaptive-trust correlations remained significant. The study highlights the value of teaching verification and interpreting trust relative to output reliability.