Couple of days ago I shared what I learned about Spring AI's chat memory. Today, here's what happened when I swapped the model behind it entirely.

Same Spring AI app. Same Java code. Same ChatClient, same @Tool annotations, same BeanOutputConverter for structured output. The only thing that changed: which model handled the requests.

OpenAI (GPT-4o) → Anthropic Claude Opus 4→ local Gemma 4 2B running through LM Studio.

The OpenAI → Claude switch was expected to work. Swap the starter dependency, update the config block, ship. Spring AI's provider abstraction is designed for this.

The local Gemma 4 2B switch was the interesting part. Same Anthropic starter dependency, just pointed at localhost:1234:

spring:

application:

name: spring-ai

ai:

anthropic:

api-key: ${LM_STUDIO_API_KEY}

base-url: http://127.0.0.1:1234

chat:

options:

model: google/gemma-4-e2b

memory:

repository:

jdbc:

initialize-schema: always

That's the entire config delta. LM Studio implements the Anthropic protocol, so Spring AI treats it as just another Anthropic-compatible endpoint. No separate "spring-ai-local" starter. No conditional Java code paths.

What I didn't expect — the 2B local model handled:

- Chat with memory (the same ChatMemoryAdvisor + JDBC repository setup from yesterday's post)

- Structured JSON output matching strict schemas

- Tool calling with proper parameter dispatch

- Code review (correctly identified a == vs .equals() bug in a real Java example)

Quality wasn't quite GPT-4o level, but it was meaningful enough that for what's probably 70% of business AI use cases — classification, summarization, structured extraction, simple agent loops — this would work in production. With zero per-request cost and full offline operation.

Recorded a walkthrough showing all three providers running the same demos (chat, memory, structured output, tool calling, code review) if you prefer video: https://youtu.be/lW0FMjDUzik

Repo with code: https://github.com/DmitryFinashkin/spring-ai

Has anyone here shipped multi-provider Spring AI in production yet? Curious how teams are handling provider routing — cost-based, latency-based, quality fallback, regional compliance — and what failure modes you're watching for.

At Devoxx, Dr. Mark Pollack, Christian Tzolov, Josh Long, and James Ward explore how to integrate modern AI into Java applications using the powerful Spring AI framework. The team demonstrates how developers can leverage existing enterprise systems to build sophisticated AI-driven solutions without leaving the familiar Spring ecosystem.

Key Takeaways:

• Spring AI provides a portable abstraction layer for connecting to diverse LLMs, including Amazon Bedrock.
• Developers can implement semantic search and RAG patterns using existing data stores like PostgreSQL.
• The Model Context Protocol (MCP) enables standardized, bidirectional communication between AI agents and tools.
• Structured output support allows developers to receive type-safe JSON directly from AI models.
• The new Spring AI Agent project introduces portable SDKs for managing complex, goal-oriented AI tasks.

This session bridges the gap between traditional enterprise development and the new agentic AI paradigm, proving that Java developers are perfectly positioned to lead the next wave of AI engineering.

Watch the full presentation to see these powerful integrations in action!

Spring AI’s ChatClient Fluent API is designed to make building conversational applications smoother and more intuitive. Baeldung’s article dives into how developers can leverage this API to integrate AI-driven chat features directly into Spring projects, with a focus on flexibility and clean design.

Key takeaways:

• Provides a fluent, chainable API for building chat interactions.
• Simplifies integration of AI models into Spring applications.
• Supports structured prompts and responses for better control.
• Enables customization of conversation flow with minimal boilerplate.
• Designed to be developer-friendly, reducing complexity in setup.
• Encourages clean, maintainable code for AI-driven features.

In short, this API helps developers bring conversational AI into their Spring apps without the usual headaches of complex configuration. It’s a practical step forward for anyone working with AI in Java-based projects.

In this Spring AI recipe, we dive into how to keep ChatClient clean and modular by using ChatClientCustomizer. Instead of stuffing every feature into a single config, we compose behaviors —such as tools, prompts, and conditions—separately, making the system easier to extend and maintain.

Key takeaways:

• Add tools (like weather lookups) without bloating the core client
• Use customizers to inject prompts or behaviors independently
• Keep concerns separated for clarity and reusability
• Toggle features on/off via application.properties with @ConditionalOnProperty
• Gain flexibility through environment variables or config servers
• Treat customizers as a design pattern, not just convenience

In short, this approach makes advanced agentic scenarios manageable and future-proof.

In this tutorial, Dan Vega explores how to effectively communicate with Large Language Models (LLMs) using Spring AI. While many view Spring AI simply as an API abstraction, Dan demonstrates that the real challenge lies in mastering prompt design to guide AI behavior and achieve precise, reliable outputs in your Java applications.

Key Takeaways

• Prompts are foundational inputs that dictate the style and quality of AI output.
• Move beyond simple strings by using Prompt Templates for dynamic, reusable logic.
• Leverage the User role for specific queries and the System role to enforce behavioral constraints.
• Use System messages to define an AI's persona, such as an exclusive dad-joke generator.
• Externalize your prompt templates into .st files to keep your code clean and manageable.
• Always instruct the model to state if it lacks an answer to prevent hallucinations.

Mastering these techniques is essential for moving from basic prototypes to robust, production-ready AI tools.

Hello community,

I'm very interested to learn how have you been using Spring AI.

What have you built with Spring AI? Was it a simple chat application, an assistant, an agent?

Was it a side project or did you deploy it in production?

Share your experience with us in the comment sections. Post the Github link if it is a public project!

Thank you!

In this tutorial, TechyTacos explores how Java developers can effectively generate JSON output using Spring AI. The video provides a practical walkthrough for moving beyond simple text-based responses toward structured, reliable data integration.

Key Takeaways

• Prompt engineering alone often results in text that mimics JSON but lacks true programmatic structure.
• BeanOutputParser is the recommended tool for mapping AI responses directly to Java records.
• Defining a Java record serves as the schema for the expected output data structure.
• Developers must include formatting instructions in the prompt template to ensure consistent parsing.
• Setting the output parser on the template ensures the AI adheres to the requested schema.
• Using the .parse() method converts the raw AI response into a type-safe object ready for use.

By implementing the BeanOutputParser, you ensure your application receives valid, type-safe JSON.

"When working with Spring AI, it’s often useful to have a quick way to experiment — trying prompts, testing tools, or observing how conversational behavior evolves.

Instead of writing one-off test code, a better approach is to create a simple interactive chat loop around ChatClient.

This recipe shows how to build a lightweight textual UI and extend it with conversational memory."

👇
https://thetalkingapp.medium.com/spring-ai-recipe-building-a-text-based-chat-loop-around-chatclient-444573e4106c

Hello community!

This video does a fantastic job of walking through the fundamentals of Spring AI, showing how you can integrate OpenAI models directly into your Java applications.

Here’s what it covers:

• Setting up the environment: A step-by-step guide on configuring pom.xml with the necessary repositories and dependencies.
• Building the App: How to set up your Controller and Service layers effectively.
• Hands-on Examples:
  • Generating creative content like jokes using AIClient and PromptTemplate.
  • Parsing responses into structured JSON for more complex application needs.

It’s a perfect entry point if you're tired of switching to Python just to experiment with LLMs. Check it out if you're looking to modernize your stack!

This article by Spring AI developers introduces the fundamental concepts of Spring AI, including ChatClient, ChatMemory, system prompts, vector stores, and more.

In this article, we learn how to build a simple conversation application with SpringAI by using the `ChatClient` interface to establish communication with an AI model and a `MessageChatMemoryAdvisor` object to keep the conversational context.

In this talk, Yuri Bezsonov present how to leverage SpringAI to build agentic AI systems. In his presentation, he shows:

• How agentic AI solutions evolved from simple assistants to full autonomous agents
• How the agent loop enables an agent to perform complex, multi-step and reasoning actions

After introducing these fundamentals, then he proceeds to a practical application of agentic AI implementation with RAG and MCP tools integration.

AI is now an ubiquitous technology and with it, a whole new field emerges as well: the AI engineering.

However, as more and more people decides to start (or shift to) an AI career, the term drifts to be overused. After all, an AI engineer is someone who specializes in using AI or building AI?

Honestly, I don't think there's a right answer. Many people and companies will just come out with their own definitions and stick to it.

So, how to build an AI career?

In my opinion, more important that pursue a specific title or role, is to understand the different type of works involved in AI development and what needs to be done to get in those positions.

The way I see the AI landscape, in a very simplified way, is defined by two opposite positions:
* the traditional AI researcher: someone with a PhD degree in computer science that publish papers on new algorithms and architectures
* the heavy AI user: some who is high skilled in applying different AI tools on many different fields, like administration, medicine, law, marketing and many others

and between these two positions, there's a whole variety of professional that works close to on extreme or the other. Professional who are focused on implementing algorithms and tools, model building, building data ingestion pipelines, systems integration and many others...

So I would like to know from those who are willing to build an AI career. Which kind of position do you identify with? I'd love to know the thoughts from the members of our community in our comments.

So I deployed an AI agent for a client a few months ago.

It worked. Like technically it worked fine. But every time someone asked it something company specific, past decisions, internal policies, how they'd handled a situation before, it just had nothing. It would hallucinate or give a generic answer or ask for context that should've already been there.

The fix everyone reaches for is stuffing everything into the system prompt. Which works until it doesn't. You hit context limits, it gets stale, and you're manually maintaining a document that nobody trusts.

I'm a CS freshman and I've been building something on the side for about 3 weeks called Lore. Institutional memory as an API. You point it at your Slack or Notion or docs, it extracts decisions your team has made, builds judgment rules from patterns, and your agents can query it at runtime before they respond.

So instead of the agent being a smart day-one hire, it actually starts with company context.

The architecture is the part I'm most interested in getting feedback on. A few things under the hood:

• R3Mem style multi-level memory, episodic events roll up into semantic patterns which roll up into rules. Inspired by the paper.
• GAAMA style concept nodes with dynamic taxonomy so the graph isn't just static categories, it evolves as the company's language evolves
• Bi-temporal modeling so you always know what the company believed at a given point in time, not just what's true now. Policy changed in February? The agent knows not to apply the old rule to new queries.
• Causal event nodes so decisions aren't just stored, they're linked to what caused them and what they caused downstream
• Semantic deduplication so you don't end up with 40 slightly different versions of the same decision
• Confidence scoring on every extracted decision so agents know how much to trust what they're retrieving

Still pre-launch. Haven't had a real user touch it yet. Before I go find one I wanted to ask people who've actually built agents in production:

1. Is this a real pain or do you solve it some other way?
2. What data source would matter most to you, Slack, Notion, email, something else?
3. What would it take for you to actually trust the extracted rules enough to let an agent act on them?

Honest answers only. Happy to go deep on any part of the architecture if anyone's curious.

https://preview.redd.it/wo8hzusyiqzg1.png?width=1669&format=png&auto=webp&s=1948d300666e5f38881e88bc4b31d7122cb613b2

At this presentation at Spring I/O conference, Thomas Vitale presents different RAG architectures that you can build using SpringAI.

What I liked at this presentation is that it presents different approaches of data flows, pipelines architectures, storing and indexing mechanisms that you can apply to build your RAG pipeline. And all of them supported by SpringAI.

Enjoy!