Your eval set is probably in your training set — here's how to check in ten minutes
You fine-tune a model, run your benchmark, and the score jumps six points. Before you write that up, there's one question worth ten minutes: how many of those benchmark examples were in the training data? If the answer is "some", part of that six points is a measurement of memory rather than capability — and there is no way to separate the two after the fact. This is train/test contamination.…
When fine-tuning a model, it's common to observe a significant improvement in benchmark scores. However, before publishing these results, it's crucial to consider the possibility of contamination from the training set. This happens when some of the benchmark examples were present in the training data, which can inflate the scores and create a false impression of the model's performance.
This phenomenon, known as train/test contamination, is a frequent yet often overlooked issue that can lead to inaccurate conclusions and misguided decisions.
There are three levels of overlap that can contribute to this problem. The first is an exact match, where byte-for-byte comparison reveals identical examples. The second level involves normalized comparison, which accounts for various text formats, such as different casing, punctuation, whitespace, and markdown wrappers. This method is still highly accurate and can catch more matches than the exact approach.
The third level is near-duplicate detection, which identifies records that share most of their content but not all. This method uses shingles (overlapping word n-grams) and calculates the Jaccard index to estimate similarity, providing a more nuanced assessment of overlaps.
It's essential to report contamination rates accurately, considering the fraction of evaluation records with at least one match in the training set. Different levels of detection should be reported separately to provide a clear understanding of the contamination issue. By performing these checks, researchers can ensure their results are reliable, avoid overstating model capabilities, and make informed decisions based on accurate data.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
