How would you build a low-latency chatbot for Money transaction data?
I have Money transaction data in postgres DB, with fields like merchant, MCC, amount, date, card last four digits, and location.
I want users to ask questions like:
- “How much did I spend on restaurants last month?”
- “Show transactions in Dubai on my card ending 4242.”
- “How does my spending compare with last month?”
Low end-to-end latency is the main requirement, while keeping filtering and calculations accurate.
I’m considering using a small LLM to turn questions into structured queries, executing them against the database, and rendering results without a second LLM call.
Has anyone built something similar? What architecture and models worked for you, and what actual latency—especially p95—did you achieve?