u/Adventurous_Club_495

I’m curious if anyone here is actually using small/local language models as part of agent systems.

Not necessarily as the main “brain” of the agent, but for specific parts of the workflow, like routing, classification, extraction, summarization, tool selection, validation, memory cleanup, or simple decision steps.

I keep thinking that a lot of agent flows probably don’t need a large model for every single step. Some parts feel like they could be handled by a smaller fine-tuned model, especially when the task is narrow and repetitive.

Has anyone tried this in production or in a serious project?
What parts of the agent pipeline worked well with an SLM, and where did you still need a larger model?

I’d love to hear real examples, even small ones.

I’m trying to understand where small/local language models will actually be useful in the next few years.

Not as a full replacement for Claude, Gemini or GPT for general reasoning/coding, but as specialized models inside real applications.

For example, I can imagine SLMs being useful for things like:

intent routing
document classification
transaction categorization
JSON formatting/validation
simple tool-calling
customer support triage
data extraction from repetitive text
agents doing repeated internal steps

But I’m curious what people here think.

What types of applications do you think will actually move from frontier models to small fine-tuned/local models?

And what would make that transition easier: better models, better fine-tuning tools, easier deployment, lower inference cost, or something else?

Has anyone here used SLMs inside agent workflows?

What kind of applications will actually run on small language models?