Florida State University Professor Develops Innovative Solution to Detect AI Cheating on Multiple-Choice Exams
In the realm of academia, the rise of generative artificial intelligence has sparked concerns about its potential for facilitating academic dishonesty. This technology, which allows computers to generate human-like text, has raised questions about its impact on student learning and assessment. While much attention has been focused on the use of AI for writing term papers and admissions essays, the potential for AI cheating on multiple-choice exams has largely gone unnoticed.
Kenneth Hanson, a professor at Florida State University, became intrigued by the idea of AI cheating after conducting research on the outcomes of in-person versus online exams. When a peer reviewer raised the question of how generative AI like ChatGPT could influence exam results, Hanson teamed up with Ben Sorenson, a machine-learning engineer at FSU, to investigate further. Their collaboration led to a groundbreaking study that shed light on the use of AI in cheating on multiple-choice tests.
The Impact of Generative AI on Cheating
Hanson’s research revealed that most cheating is a result of barriers to access, where students feel compelled to seek alternative means to succeed. Generative AI, such as ChatGPT, has made the process of answering multiple-choice questions faster and more efficient. However, the accuracy of the generated answers has come into question.
By collecting student responses from five semesters’ worth of exams and inputting them into ChatGPT 3.5, Hanson and his team discovered patterns specific to the AI’s responses. Interestingly, ChatGPT answered nearly every “difficult” test question correctly, while answering nearly every “easy” question incorrectly. This finding highlighted the disconnect between how students approach problems and how AI generates answers.
The Limitations of AI in Creating Multiple-Choice Tests
While ChatGPT excelled in answering multiple-choice questions, its ability to create practice tests was less impressive. A study published by the National Library of Medicine found that when researchers used ChatGPT to generate multiple-choice exams, only one-third of the questions had correct answers. The majority of the answers were incorrect, with little explanation provided for why they were chosen.
To cheat on a multiple-choice exam using AI, a student would need to input the questions and possible answers into ChatGPT using a device like a phone. Without proper proctoring measures in place, the student could easily copy and paste the answers into their browser. However, Victor Lee, a faculty lead of AI and education at Stanford University, believes that this process may be too cumbersome for students seeking quick solutions.
The Feasibility of Detecting AI Cheating
Despite the high accuracy rate of Hanson’s method for detecting AI cheating, he does not believe it is a practical solution for individual professors to implement. Running answers through his program multiple times can be time-consuming and may not yield significant results. Moreover, research suggests that only a small fraction of students resort to cheating with AI, indicating that the effort to catch them may not be worthwhile.
Looking ahead, Hanson envisions his detection method being used on a larger scale by proctoring companies like Data Recognition Corporation and ACT. These companies have the resources and data to implement such technology effectively. While ACT has stated that it is not currently using generative AI detection, Hanson remains optimistic about the potential for broader adoption in the future.
The Future of AI in Education
As concerns over AI cheating continue to linger, the focus on adapting educational practices to coexist with this technology has become increasingly important. Lee notes that universities are beginning to implement AI-focused policies to address these concerns, while also exploring new ways to incorporate AI into the educational experience. Adapting to these changes may require effort, but it is essential for ensuring academic integrity in the digital age.
In conclusion, Hanson’s innovative solution to detect AI cheating on multiple-choice exams has opened up new possibilities for addressing academic dishonesty. While the use of generative AI in cheating poses challenges, it also presents opportunities for educators to evolve their assessment practices. By staying vigilant and proactive, institutions can navigate the complexities of AI in education and uphold the integrity of learning and evaluation processes.