One search query is usually not enough for a reliable AI answer
A common RAG pattern is to send the user’s exact question to a search API, retrieve five results, and pass them to an LLM. It works for simple lookups but becomes unreliable when the question contains several claims or requires different types of evidence.
A better approach is to decompose the question before searching.
For example:
“Did Company X’s revenue grow because of higher prices or customer growth, and is that trend continuing?”
Instead of one broad query, generate smaller searches:
- Company X latest revenue growth
- Company X pricing changes
- Company X customer count
- Company X latest guidance
- Company X investor relations earnings release
Then combine and deduplicate the results before reranking them:
queries = decompose(user_question)
results = []
for query in queries:
results.extend(search_api(query, limit=5))
unique_results = deduplicate(results, key="canonical_url")
ranked = rerank(user_question, unique_results)
context = ranked[:8]
A few practical improvements:
- Add a date filter for time-sensitive questions.
- Search primary domains separately when authoritative evidence matters.
- Deduplicate syndicated articles by content, not just URL.
- Keep at least one result per sub-question before global reranking.
- Make the final answer identify which claims lack supporting evidence.
This adds more API calls, but it usually produces better coverage and makes missing evidence easier to detect. It also works regardless of whether retrieval comes from Exa, Tavily, Brave, Serper, or another provider.
For people running search-backed agents, has query decomposition improved answer quality enough to justify the added latency and cost?
