u/Effective-Giraffe655

Working with deepseek-v4-flash-0731 with prompt_cache_retention: 24h on the chat completions endpoint, some requests pass and work, others - 400 with the field not supported. The irony is - it is the same agent session, so request #1-7 are ok, request #8 fails, request #9 is ok, request #10 fails, and so on.

What's going on?

Hermes is used in my family, and we use Venice as provider, but there was no proper integration, and my personal patchset got over 30+ patches and became inconvenient to maintain, so I refactored some of the patches into reusable plugins.

Hope this will be useful ;)

Hi there,

Trying to use `e2ee-deepseek-v4-flash` with Hermes Agent with a patchset to integrate Venice caching (`prompt_cache_key` with session id for routing hints, `prompt_cache_retention` with `24h`) into all API requests, and found out that within the same session, *the same turn* cache hit rate bounces from 95+% to 0% and then back to high hit numbers, example from logs:

```

2026-07-15 15:23:16,482 INFO [20260715_150122_2f35f29d] agent.conversation_loop: cache: model=e2ee-deepseek-v4-flash cached=68,864/71,463 (96% hit) written=0 in=2,599 out=1,130

2026-07-15 15:23:36,790 INFO [20260715_150122_2f35f29d] agent.conversation_loop: cache: model=e2ee-deepseek-v4-flash cached=71,424/74,949 (95% hit) written=0 in=3,525 out=2,106

2026-07-15 15:23:55,938 INFO [20260715_150122_2f35f29d] agent.conversation_loop: cache: model=e2ee-deepseek-v4-flash cached=0/75,544 (0% hit) written=0 in=75,544 out=1,179

2026-07-15 15:24:05,395 INFO [20260715_150122_2f35f29d] agent.conversation_loop: cache: model=e2ee-deepseek-v4-flash cached=75,520/76,724 (98% hit) written=0 in=1,204 out=743

2026-07-15 15:24:08,055 INFO [20260715_150122_2f35f29d] agent.conversation_loop: cache: model=e2ee-deepseek-v4-flash cached=76,544/78,645 (97% hit) written=0 in=2,101 out=105

2026-07-15 15:24:29,043 INFO [20260715_150122_2f35f29d] agent.conversation_loop: cache: model=e2ee-deepseek-v4-flash cached=78,592/82,155 (95% hit) written=0 in=3,563 out=2,366

2026-07-15 15:24:46,941 INFO [20260715_150122_2f35f29d] agent.conversation_loop: cache: model=e2ee-deepseek-v4-flash cached=0/83,690 (0% hit) written=0 in=83,690 out=1,782

2026-07-15 15:24:58,977 INFO [20260715_150122_2f35f29d] agent.conversation_loop: cache: model=e2ee-deepseek-v4-flash cached=83,456/85,545 (97% hit) written=0 in=2,089 out=642

```

My first suspicion was the Hermes' approach - harness opens a fresh HTTP client and connection on every tool call (tool-heavy turns often come with 0% cache hit mid-turn), but the response header `x-venice-version` shows the same value across all the requests, so supposedly all requests should hit the same server, and have proper caching.

Could anybody tell whether that's a known problem with this model specifically, and suggest any fixes?

PS: I can share the patch set if needed

Hi there,

Tried to use `xiaomi-mimo-v2-5` in some local pet projects, but it's almost unusable with constant 429 failures about the model being currently overloaded.

Any plans to increase throughput for this model?