u/Proof-Possibility-54

Hi everyone!

My Spring AI course is now available on JetBrains Academy.

The course is designed around practical, real-world tasks completed directly in IntelliJ IDEA using the JetBrains Academy plugin. The project, dependencies, and configuration are already prepared, so you can focus on learning Spring AI and writing code instead of spending time on setup.

I honestly wish I’d had this kind of learning experience when I was starting out: clear tasks, a ready-to-use project, and immediate feedback—all inside the same IDE used for professional development.

I’d be glad to hear your feedback, especially which Spring AI topics or practical use cases you’d like to see covered next.

Course link: https://academy.jetbrains.com/course/32882

Spring AI structured output: how to make a model correct itself

​

If you use an LLM for something else than just a free-form chatting, you might probably want it to return data in a structured form, e.g. JSON

​

Spring AI allows to \[soft\] force a model to do that. But sometimes LLM fails to do that. The simplier the model is, the more chances that it will fail. Morover, any such a failure could be devided into 2 categories: incorrect schema and correct schema with incorrect data burned in. For example, some fields of the desired schema are missing. Or all fields are present, but field type, for example, is incorrect or a required filed missies a value

​

​

​

The POC i built validates not only schema as such, but also field types and ranges (e.g. Min-Max, NotNull, etc.) using validation package **spring-boot-starter-validation**

​

If any of the checks doesn't pass, this is feded back to model:

​

prompt = """

Your previous response was invalid.

Problem(s): %s

Your previous output was:

%s

Return corrected JSON that fixes these problems and matches the

schema exactly. Output ONLY JSON, no prose.

%s

""".formatted(lastError, lastOutput, format);

​

So we give a feedback to the model, not just asking to redo/re-think. By providing a detailed feedback we increase chances that the next reply will satisfy our expectations

​

As always, all code is available in the github repo: [https://github.com/DmitryFinashkin/spring-ai\](https://github.com/DmitryFinashkin/spring-ai)

​

You might also like to watch a detailed video walk-through on YouTube: [https://youtu.be/59kcnTLVu0Q\](https://youtu.be/59kcnTLVu0Q)

Last Friday I shared what I learned from swapping providers in a Spring AI app. This Friday: what happens when you stop swapping and start routing dynamically.

Same project as previous videos. Same ChatClient code. The only new piece is a dispatcher that looks at each request before choosing which provider handles it.

The pattern in one method:

public RoutedResponse route(String prompt) {

RoutingDecision decision = router.route(prompt);

ChatClient client = (decision.tier() == ModelTier.LOCAL)

? localClient

: cloudClient;

ChatResponse response = client.prompt(prompt).call().chatResponse();

long[] tokens = extractTokens(response, prompt, text);

tracker.record(decision, tokens[0], tokens[1]);

return new RoutedResponse(decision, text);

}

Two ChatClient beans — one autoconfigured against LM Studio (local), one explicit @Configuration for Anthropic (cloud). Spring's qualifier mechanism handles disambiguation. The dispatch is a ternary expression.

The router itself is intentionally simple — length check + keyword check. Not embeddings, not a classifier model. Just transparent rules you can debug by reading the code.

Result from the demo: 10 code review requests, 7 routed local, 3 routed cloud. Routed total $0.25 vs all-cloud baseline $0.48 — 48% lower, with identical-quality answers on the easy questions (verified by side-by-side comparison).

The data point worth flagging: those 7 routed-away queries would have cost ~$0.23 collectively on cloud, almost matching the $0.25 from the 3 cloud queries. The cheap-individually queries collectively rival the expensive ones. Routing the long tail away from cloud is where the real savings come from, not avoiding premium prices on premium queries.

A few practical notes that aren't obvious until you actually ship this:

1. Anthropic's API requires max_tokens on every request. Without it, Spring AI's default truncates Opus responses mid-sentence. Set it explicitly to 4096 on cloud options.

2. Claude Opus regularly takes 15-45 seconds per call. Spring AI's underlying Reactor Netty client has a default response timeout shorter than that. You'll see ReadTimeoutException in the Spring log if you don't extend it. Custom RestClient.Builder with responseTimeout(Duration.ofSeconds(300)).

3. Don't refactor the original endpoints. The /chat, /review, /chat-with-tools endpoints from Model Switching keep running on the autoconfigured local ChatClient unchanged. The new multi-model controller lives in a separate package under /ai. Less surgery, less narrative debt.

Recorded the full walkthrough including a live cost dashboard demo: https://youtu.be/ziMzlY9Szvs

Has anyone here implemented cost-based routing in production? Curious how teams are handling the "is this request hard enough to escalate" decision — keyword rules, embeddings, confidence scoring, or something else.

As always, latest code in the repo: https://github.com/DmitryFinashkin/spring-ai

Built my first AI app entirely in Java using Spring AI — no Python involved

I've been experimenting with Spring AI (the official Spring project for AI integration) and was surprised how little code it takes to get something working.

The whole setup is one Maven dependency and a few lines of YAML config. From there I built three things on top of the same project:

• A simple chat endpoint using ChatClient — literally prompt(), call(), content()
• Structured output that maps AI responses directly to Java records (no JSON parsing)
• Tool calling where the AI invokes Java methods to get real data

The tool calling part was the most interesting — you annotate a method with @Tool and Spring AI handles the function-calling protocol with the model. The AI decides when to call your code and uses the result in its response.

I recorded the whole process if anyone wants to see the code in action: https://youtu.be/SiPq1i_0YgY

Anyone else using Spring AI in production or side projects? Curious what use cases people are finding beyond chat endpoints.