Copilot Studio Multi-Agent Setup (9 Agents, ~60 Documents) Causing 2-Minute Response Times. Is This Normal?
Hi everyone,
I'm working on a multi-agent architecture in Copilot Studio and I'm trying to understand whether the performance I'm seeing is expected or if there are optimization opportunities.
Current Setup
- Using Copilot Studio (New Experience).
- All agents are running on GPT-5.5.
- I have approximately 90 documents distributed across 9 specialized agents.
- Documents include:
- PPTX
- DOCX
- XLSX
- Markdown files
- Other documentation formats
- Each agent is responsible for a specific domain/topic and only contains the documents relevant to that area.
- There is a single orchestrator agent that routes questions to the appropriate specialized agents.
- The orchestrator is the agent shared with end users.
- For each agent, I've also created summaries describing the purpose of each document and the scope of the agent's knowledge.
Problem
Most user questions are taking around 2 minutes to receive a response.
The responses are generally correct, but the latency feels too high for a production-like experience.
My Questions
- Is this latency mainly related to the model having to search and interpret a large number of documents at runtime?
- Can having many agents behind a single orchestrator significantly impact response time?
- Does the file type matter? For example, are PPTX/XLSX documents typically slower to process than Markdown or text-based documentation?
- Could poorly optimized agent instructions cause the orchestrator to spend too much time deciding which agent should answer?
- Are there any recommended best practices for reducing latency in multi-agent RAG scenarios in Copilot Studio?
- Has anyone seen substantial improvements by consolidating agents or restructuring their knowledge sources?
Additional Context
One thing I'm wondering is whether I'm over-segmenting the knowledge base. My original goal was to improve answer quality by separating content into specialized agents, but I'm now questioning whether the orchestration overhead is outweighing the benefits.
I'd be interested in hearing how others structure:
- Number of agents
- Number of documents per agent
- Use of orchestrators
- Prompt/instruction design
- Knowledge source organization
Has anyone faced similar response times and found an effective optimization strategy?
Thanks!