독파모 2차 평가 앞두고 ‘벤치맥싱’ 논란…문제 풀이 학습해 점수 높였나?

As foreign AI companies have proposed offering "benchmarks" (performance metrics) to domestic participants in the second evaluation of South Korea's AI Foundation Model project, called 독파모, questions have arisen about potential collaboration with some of these international firms. Industry sources reported that companies such as AfterQuery, an AI learning specialist from the United States, offered to provide data and post-learning techniques to domestic developers participating in the 독파모 project at the International Machine Learning Conference (ICML) 2026 in Seoul last month.

Following this, the Ministry of Science, ICT & Future Planning plans to select one of the four teams that made it to the final round for dismantlement based on a combination of benchmark assessments, expert evaluations, and usability testing among end-users. The company leading the push for benchmark optimization is AfterQuery, which was established in Silicon Valley last year and specializes in creating data for AI learning and supporting large language models (LLMs) through post-learning.

The term "benchmarking" (Benchmaxxing) has gained attention in the industry for optimizing models to achieve higher scores on specific benchmark tests. While all four participating companies deny any involvement in such collaborations, concerns have been raised based on technical reports submitted ahead of the evaluation process.

The primary concern is whether certain benchmark assessments led to excessive model tuning and potential data contamination. AI model development principles dictate separating learning data from performance measurement data, and mixing test data with learning data could lead to overfitting, where the AI model may excel in tests but fail to generalize to real-world applications.

The Ministry of Science, ICT & Future Planning clarified that evaluations will take into account not just benchmark scores but also usability and real-world applicability.