I tested DeepSeek V4 Pro 0813 on 16 Hack The Box challenges
I’ve been benchmarking LLMs on Hack The Box challenges to see how well they can actually find and exploit vulnerabilities, rather than relying only on standard coding benchmarks.
I recently tested the new DeepSeek V4 Pro 0813 and compared it with the previous 0423 version.
The overall result was fairly close: 38.8% vs 36.2%, with 9 challenges solved instead of 8. Given the small test set and normal run-to-run variation, I’d consider that basically a draw.
What surprised me more was the efficiency improvement. The median run dropped from 62.5 to 12.5 steps, from 1.94M to 0.25M tokens, and from $0.35 to $0.20 per challenge.
It still struggled with the harder challenges, but the reduction in token usage and steps compared with the previous version was pretty significant.
Full results and methodology are in my write-up here: