fixing token latency in sequential agent loops (Parallel Tool Calling fix) and this method worked for me well.
If you are building multi-step agent loops, you have probably run into the bottleneck where your agent waits for Tool A to finish completely before it even initiates Tool B—even when the two actions don't depend on each other. Sequential execution absolutely kills the user experience. Here is a quick architectural fix using Python's asyncio to force parallel tool execution inside your agent orchestration loop:
The slow way: Sequential execution
import asyncio
async def sequential_run(): result_a = await call_tool_a() # Waits 2.5 seconds result_b = await call_tool_b() # Waits 2.0 seconds return [result_a, result_b] # Total time: 4.5 seconds
The fast way: Parallel execution
import asyncio
async def parallel_run(): # Dispatches both tool calls concurrently results = await asyncio.gather( call_tool_a(), call_tool_b() ) return results # Total time: ~2.5 seconds (bound by the slowest tool)
When you parse your LLM's tool_calls JSON array, do not just loop through them with a standard for loop. Map them into an async gather block instead. This drops your total execution latency down to the speed of your single slowest tool, rather than stacking the response times of every single tool combined and also please let me know any errors and corrections in this code.🤗