SIGCSE Virtual 2024
Thu 5 - Sun 8 December 2024
Sat 7 Dec 2024 15:00 - 15:30 at Track 1 - Papers 1: AI (1)

RAGMan is an LLM-powered tutoring system that can support a variety of course-specific and homework-specific AI tutors. RAGMan leverages Retrieval Augmented Generation (RAG), as well as strict instructions, to ensure the alignment of the AI tutors’ responses. By using RAGMan’s AI tutors, students receive assistance with their specific homework assignments without directly obtaining solutions, while also having the ability to ask general programming-related questions. RAGMan was deployed as an optional resource in an introductory programming course with an enrollment of 455 students. It was configured as a set of five homework-specific AI tutors. This paper describes the interactions the students had with the AI tutors, the students’ feedback, and a comparative grade analysis. Overall, about half of the students engaged with the AI tutors, and the vast majority of the interactions were legitimate homework questions. When students posed questions within the intended scope, the AI tutors delivered accurate responses 98% of the time. Within the students used AI tutors, 78% reported that the tutors helped their learning. Beyond AI tutors’ ability to provide valuable suggestions, students reported appreciating them for fostering a safe learning environment free from judgment.

Sat 7 Dec

Displayed time zone: (UTC) Coordinated Universal Time change

15:00 - 16:30
Papers 1: AI (1)Conference at Track 1
15:00
30m
Paper
Integrating AI Tutors in a Programming Course
Conference
Iris Ma University of California, Irvine, Alberto Krone-Martins University of California, Irvine, Crista Lopes University of California, Irvine
15:30
30m
Paper
Integrating Natural Language Prompting Tasks in Introductory Programming Courses
Conference
Chris Kerslake Simon Fraser University, Paul Denny The University of Auckland, David Smith University of Illinois at Urbana-Champaign, James Prather Abilene Christian University, Juho Leinonen Aalto University, Andrew Luxton-Reilly The University of Auckland, Stephen MacNeil Temple University
16:00
30m
Paper
Synthetic Students: A Comparative Study of Bug Distribution Between Large Language Models and Computing Students
Conference
Stephen MacNeil Temple University, Magdalena Rogalska Temple University, Juho Leinonen Aalto University, Paul Denny The University of Auckland, Arto Hellas Aalto University, Xandria Crosland Western Governors University