How do you scale Kafka consumers in C++ without blocking threads?

We have a C++ microservice that churns through a decent stream of data from Kafka. We use standard librdkafka, but during load spikes it blocks our OS worker threads with its internal locks. As a result, the overall service latency skyrockets.

Has anyone managed to beat this without writing hacky separate thread pools?

reddit.com
u/Sudden-Association47 — 1 month ago