교사가 C 줬는데 AI는 A?…논리 이상해도 ‘특정 단어' 많으면 만점

South Korea's National Education Commission (National Education Commission) is currently considering the introduction of essay and argumentative writing questions for the nationwide high school entrance exams. However, as the country debates the use of artificial intelligence (AI) for grading, a recent report by the Korean Education Evaluation Institute has highlighted the limitations of AI grading systems.

In the report titled "Research on the Application of Artificial Intelligence Models for Automatic Grading of Written Expression," it was found that AI grading models, when applied to five core subjects such as Korean, social studies, mathematics, science, and technology, display certain shortcomings.

One example provided in the report involved a high school essay graded by a teacher at a C grade, which the AI system classified as the top grade, A. The teacher noted that the essay contained inconsistent tone of address and failure to accurately understand the meaning of the cited sources, resulting in a lower grade. Yet, AI deemed the lengthy essay and the use of specific vocabulary to be high-scoring.

The report also mentioned similar cases in the science subject, where incorrect explanations of principles were recognized as correct due to the presence of specific keywords.

Moreover, the AI grading system showed sensitivity to factors like sentence length, connecting words, and intonation, while human teachers emphasized the accuracy of technical concepts and logical reasoning. The grading committee, led by Ms. Yeom Seok, Minister of Education for AI Strategy, emphasized that AI grading should be used as a supplementary tool to assist teachers in the evaluation process, rather than replacing them entirely.

The AI grading system comprises four stages, with the final stage involving human review of the AI's scores and their justifications. The committee also suggested measures to address security concerns, such as separating student answers and personal information, and ensuring that student answers are not used for external AI learning.