V4-Flash vs. V4-Pro: DeepSeek promised better and cheaper. It’s true, but not how I expected.

DeepSeek, the Chinese AI lab, recently refreshed its V4-Flash model, moving it out of preview. In a comparison with the V4-Pro model, DeepSeek's documentation claimed that the refreshed Flash model outperformed the V4-Pro preview on coding and agentic benchmarks. However, the pricing structure raised questions about this claim. V4-Flash costs $0.14 per million input tokens and $0.28 per million output, while V4-Pro costs $0.435 and $0.87 respectively.

This price difference is nearly one-third for both input and output. Intrigued by this pricing discrepancy, I decided to test both models against the same three professional coding tasks to determine their true capabilities.

For each test, I recorded time from OpenCode's session timer, API requests, and cost from DeepSeek's usage dashboard. To ensure a fair comparison, I built separate clones of the repository for each model, started a fresh session for every test, and used identical prompts. The three tests I ran were as follows.

Test 1: The Bug Fix

The prompt asked the model to fix a bug where importing Rich crashes on macOS due to a permission error raised by os.getcwd(). Both models identified the root cause and provided identical fixes, using a broader exception handler and adding regression tests. The only difference was where each model placed its test – Flash created a new file, while Pro added its test to the existing suite, adhering to the project's conventions.

Flash completed the task in 59 seconds with 6 API requests and 119.9K tokens, costing less than a cent. Pro took 1 minute 1 second, 13 requests, and 159.2K tokens, costing about a cent. Despite providing the same correct solution, Pro made twice the API calls and used 75% more tokens to arrive at the answer.

Test 2: The Feature Build

The prompt asked the model to add lstrip() and strip() methods to Rich's Text class, preserving styles while trimming leading whitespace. Both models successfully completed the task, adding tests covering plain text, styled text, and all-whitespace text. The interesting aspect was how each model approached the problem. Pro delegated initial codebase exploration to a subagent that made 13 tool calls, accidentally introducing a bug in its initial implementation.

However, Pro's tests caught the bug and fixed it in one iteration. Flash, on the other hand, simply read the files and wrote the code. Both models passed all eight edge cases, but Pro required nearly four times the API calls and 75% more tokens to achieve the same result. Despite the identical code quality, Flash was more efficient in this test.

Test 3: The Performance Job

The prompt asked the model to write a benchmark script measuring the rendering time of a Table with 5,000 rows and 6 columns in a string console. Both models were tasked with finding the baseline time, profiling the dominant hot path, and optimizing the rendering process. The models' performance in this test was not explicitly compared in the source material, so it is impossible to determine which model performed better in this specific scenario.

In conclusion, while DeepSeek's V4-Flash model proved to be more cost-effective and efficient in the bug fix and feature build tests, the V4-Pro model showed similar performance in the feature build test. The performance job test could not be evaluated due to the lack of explicit comparison data. Overall, the pricing structure of DeepSeek's models suggests that the V4-Flash model offers better value for money, particularly for less complex tasks.