6B Tokens on Writing Pipeline - Worth it
I have a research -> writing -> qa agentic pipeline that I've been working on, I actually created it to be used by local Gemma 4 12B, since it's output token heavy. Local AI handles it well, in fact I could have used it. But since the release of Deepseek V4 Flash I thought why not adapt it for DeepSeek, it's a smarter model and it's slightly more thorough at research. Gemma is good too but it tends to rely on snippets whereas with DeepSeek it's really interested in sucking up as much info as possible before it writes. Try as I may, you can't teach a smaller model to be more 'thorough' - at least not with context engineering.
I've just run it 4,100 times, each written piece was between 1,800 and 2,500 words and I thought I'd share my cost. Total was 6B tokens at a pretty decent cache hit. It ended up being about 120mil output tokens to write all 4,100 pieces. Each run averaged between 40-80k tokens in total, maybe a bit more. I was able to almost halve my token usage by having Kimi K3 optimise the pipeline and I used a number of custom Pi add-ons to facilitate caching of documents for research as well as various scraping tasks. I also used Exa.ai for search (another $50-60 or so in API cost there).
Concurrently I also ran a few tests to see if r/Neuralwatt would be cheaper for inference using their energy pricing. They were not cheaper, in fact they were about double the price with energy, 250 pieces for US $8 versus $4 on Deepseek. Neuralwatt's pricing indicator says that the pipeline would have been cheaper at $6 for 250 if I had used token pricing. So that's interesting.
Anyway thought I'd share. It's excellent value, and given that I was able to halve my token usage by optimizing the pipeline, I'm not that scared of them doubling the price, or even tripling the price.
If they quadruple the price I think they lose to local AI inference for these types of workloads. But writing based on research is actually something local AI does really well. It also wasn't time sensitive. It was convenient for me to be able to produce these in a batch now, but I could have had them backgrounded for weeks without affecting the outcome.