u/not_apply_yet

Hey everyone, one of the founding engineers at Yappi here 👍

When we first started building our Shopify AI chatbot, we did what most devs do: we spun up a simple LangChain pipeline, hooked it up to GPT 4.0, threw a basic RAG system on top of our product catalog database, and called it a day.

It worked... okay-ish. But we quickly hit a wall. In e-commerce, generic chatbots don't drive conversions. A standard RAG pipeline will spit out a product description, but it doesn't know WHEN to recommend a product, HOW to handle buying hesitation (objections), or how to capture real-time storefront behavior (e.g., a customer lingering on a checkout policy page).

So, we tore it all down. Over the last few months, we've built a highly distributed, multi-agent orchestration mesh powered by 4 custom-fine-tuned Small Language Models (SLMs) and a real-time behavioral telemetry loop.

Here is a look under the hood at our complete production pipeline:

https://preview.redd.it/qt67ygfmm0bh1.png?width=640&format=png&auto=webp&s=92a48a889a8e59e2b024b3492e2f52c0d3a357c7

Real-Time Behavioral Signal Processing

Unlike standard chat systems where the LLM only sees the TEXT the user types, our orchestrator receives a continuous event stream of the customer’s browser behavior.

We monitor:

Dwell time on specific product images
Scroll velocity and depth on sizing charts
Cart events (additions, removals, quantity changes)
Click trails showing what category pages they navigated prior to launching the chat
These behavioral signals are ingested via a low-latency queue, normalized into numerical feature vectors, and injected directly into the prompt context window as specialized temporal state tokens. If a user says "Is this in stock?" while hovering on a red hoodie, the system already knows they mean the Red Hoodie in Size L without needing to ask.

The Multi-Agent Orchestration Layer

At the heart of the system is the Coordinator Agent. Written as a highly optimized state machine, the Coordinator controls the multi-stage reasoning chains (using a modified ReAct/Chain-of-Thought approach). It doesn't try to answer the query itself. Instead, it decides how to route the user's input and store state across our cluster of specialized, task-oriented Small Language Models (SLMs).

By using smaller, hyper-specialized models instead of one massive monolithic model, we reduce latency down to double-digit milliseconds and keep inference costs manageable.

Our 4 Proprietary Task-Specific SLMs

We fine-tuned four separate open-weights base models on our proprietary Shopify interaction dataset. Each model does exactly one thing exceptionally well:

SLM-Intent (3.8B parameters): Determines user intent at a granular level. It distinguishes standard queries ("Where is my order?") from buying signals ("Will this arrive before Friday?") and support inquiries ("How do I return this?"
SLM-Match (1.5B parameters): Maps vague natural language queries to our shop graph database. If a user asks for "something breezy for a beach wedding," SLM-Match translates this semantic concept into specific product tags, fabrics (linen, cotton), and SKUs
SLM-Objection (7B Mixture-of-Experts): Fine-tuned specifically for sales objection handling. When a customer expresses hesitation ("this is too expensive" or "I'm worried it won't fit"), this model selects the optimal negotiation strategy (e.g., highlighting warranty, suggesting a discount code, or offering a split-payment option)
SLM-Upsell (1.5B parameters): Analyzes the current cart value, customer session history, and sentiment data to calculate the exact millisecond to trigger an upsell or cross-sell recommendation. If triggered too early, it annoys the user; if too late, they checkout. This model predicts the highest probability conversion window

Custom hybrid retrieval pipeline

To make sure our models never hallucinate product details (which is a fast track to getting a merchant sued), we bypassed traditional out-of-the-box vector databases.

We designed a split-path hybrid retrieval flow that enforces strict catalog validation constraints prior to generating a response:

https://preview.redd.it/i9kl754pm0bh1.png?width=640&format=png&auto=webp&s=9d9bf7f9276a954fcd5d59497acb2a75277b699b

Dense Retrieval: We embed the merchant's catalog using a custom-fine-tuned embedding model that understands e-commerce terminology
Sparse Retrieval (BM25): We run lexical searches to capture exact model numbers, colors, and sizes
Graph Constraint Solver: The results of the dense and sparse searches are merged, then fed into a constraint solver which queries the Shopify Admin API in real-time to check stock levels, active price adjustments, and checkout rules
Cross-Encoder Reranker: A final cross-attention step reranks the validated products using the telemetry feature vectors from the user's active session
Multi-Stage Reasoning & Synthesis

Once the retrieval pipeline returns the valid catalog data, the Coordinator routes all inputs (Behavior features, Intent class, Objection strategy, Product matches, and Context) into a final synthesis agent. This agent generates the response stream using a dynamic, self-correcting logic chain. Before the response is sent back to the customer over a persistent WebSocket connection, a safety filter (built on a custom LLaMA Guard wrapper) checks the output for price mismatches, policy compliance, and toxicity.

Why we built it this way (and why it matters):

It would have been 100x easier to just build a standard wrapper around GPT 4.0. But e-commerce conversion rates live and die by latency and relevance. Monolithic models are too slow, too generic, and lack the real-time context of e-commerce user behavior. By distributing the workload across tiny, specialized SLMs and coordinating them through a lightning-fast event bus, we've managed to achieve sub-second response times while driving actual double-digit conversion rate increases for our merchants.

Would love to hear from other folks building in the e-commerce AI space!

* How are you all handling zero-shot product matching when catalogs update in real-time?

* Are you seeing better results with fine-tuned SLMs vs heavily prompted frontier LLMs?

Why do some posts have the promote button but some dont?

Deep Dive: How we built a multi-agent orchestration mesh with 4 task-specific SLMs to power our Shopify AI agent

Roast my store — Good product, low prices, but $40-50 CPA is killing us