DeepSeek’s cost-saving AI may lead to inaccuracy in model performance: research
New research has exposed a potential flaw in the memory-saving technology underpinning DeepSeek’s models, marking a rare challenge to a breakthrough that has helped the Chinese artificial intelligence lab close the gap with US rivals OpenAI and Anthropic with its low-cost approach. The findings were published late last month in a paper led by researchers from ByteDance and co-authors affiliated…
A recent study has revealed a potential flaw in DeepSeek’s memory-saving technology, which has been instrumental in the Chinese AI lab’s efforts to rival US counterparts like OpenAI and Anthropic. Published in a paper led by ByteDance researchers and co-authors from Princeton, Stanford, and UC Berkeley, the findings suggest that the cost-cutting approach may compromise DeepSeek’s model reliability.
According to the study, DeepSeek's compression technique, designed to reduce memory requirements for processing long inputs, can lead to a 40 percentage point decrease in retrieval accuracy. This occurs because the method groups and compresses information before storing it, potentially making certain details harder to recover depending on their position within the compressed segments.
The issue was observed in DeepSeek-V4, V4.1, and all pre-trained compressed models. For instance, inserting irrelevant text before computer code can negatively impact the model's ability to accurately retrieve the correct answer. While the researchers acknowledged their findings were based on controlled experiments and limited coding examples, the issue underscores the trade-off between efficiency and reliability in AI development, particularly for applications requiring accurate information retrieval from lengthy documents or software code.
Written by urgent.news from SCMP Tech's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
