RAGMan is an LLM-powered tutoring system that can support a variety of course-specific and homework-specific AI tutors. RAGMan leverages Retrieval Augmented Generation (RAG), as well as strict instructions, to ensure the alignment of the AI tutors’ responses. By using RAGMan’s AI tutors, students receive assistance with their specific homework assignments without directly obtaining solutions, while also having the ability to ask general programming-related questions. RAGMan was deployed as an optional resource in an introductory programming course with an enrollment of 455 students. It was configured as a set of five homework-specific AI tutors. This paper describes the interactions the students had with the AI tutors, the students’ feedback, and a comparative grade analysis. Overall, about half of the students engaged with the AI tutors, and the vast majority of the interactions were legitimate homework questions. When students posed questions within the intended scope, the AI tutors delivered accurate responses 98% of the time. Within the students used AI tutors, 78% reported that the tutors helped their learning. Beyond AI tutors’ ability to provide valuable suggestions, students reported appreciating them for fostering a safe learning environment free from judgment.
Sat 7 DecDisplayed time zone: (UTC) Coordinated Universal Time change
15:00 - 16:30 | |||
15:00 30mPaper | Integrating AI Tutors in a Programming Course Conference Iris Ma University of California, Irvine, Alberto Krone-Martins University of California, Irvine, Crista Lopes University of California, Irvine | ||
15:30 30mPaper | Integrating Natural Language Prompting Tasks in Introductory Programming Courses Conference Chris Kerslake Simon Fraser University, Paul Denny The University of Auckland, David Smith University of Illinois at Urbana-Champaign, James Prather Abilene Christian University, Juho Leinonen Aalto University, Andrew Luxton-Reilly The University of Auckland, Stephen MacNeil Temple University | ||
16:00 30mPaper | Synthetic Students: A Comparative Study of Bug Distribution Between Large Language Models and Computing Students Conference Stephen MacNeil Temple University, Magdalena Rogalska Temple University, Juho Leinonen Aalto University, Paul Denny The University of Auckland, Arto Hellas Aalto University, Xandria Crosland Western Governors University |