Developing Higher Education Students’ Artificial Intelligence (AI) Literacy for Academic Writing: A Pilot Study in A First-Year Seminar

Abstract

As generative AI tools like ChatGPT become increasingly integrated into higher education contexts, the need to equip students with critical AI literacy has become urgent. This pilot study investigates how a four-lesson instructional module shaped undergraduate students’ understanding of the ethical use, potentials, and limitations of generative AI tools, particularly in relation to academic reading and writing. Implemented in a First-Year Seminar (FYS) course at a U.S. university, the study employed a mixed-methods design, drawing on pre- and post-module AI Literacy Surveys (n=7), a post-semester instructor interview, and the researcher’s reflection journal. Findings revealed a notable shift in students’ awareness of ethical considerations, collaborative engagement with AI, and development of authorial voice when scaffolded through structured activities. However, these gains did not consistently transfer to independent academic tasks, pointing to the limitations of short-term interventions and the importance of integrating AI literacy instruction within foundational academic writing curricula. As an exploratory pilot study, the findings provide preliminary insight into the pedagogical potentials and constraints of early AI literacy intervention for college-level academic writing, suggesting the need for sustained, curriculum-embedded approaches to cultivate critical and ethical AI engagement among higher education students.