Spring AI's New Agents Project: What Java Teams Actually Get, and Whether to Adopt It Now

Every few months a framework announcement lands in my feed with the word "agents" in it, and my first reaction is the same: what does this change for the service I am paged for at 3 a.m.? This month it is Spring AI's turn. The announced plan is a dedicated agents project, shipped as a Spring Projects experimental module in November 2026, with a possible path into Spring AI 3.0 around mid-2027. Those dates come from the project's own announced timeline, and I would treat the mid-2027 part as an intention, not a promise.

I run backends where a wrong action costs real money, so I read this announcement as an engineer deciding what to put on a roadmap. Here is my breakdown, starting from zero and ending in code.

Part 1: What is an agent, in plain words?

A large language model (LLM) is a program that predicts text. Ask it a question and it answers from what it learned in training. It cannot check your bank balance or restart a server, because it has no hands.

An agent gives it hands. The idea is a loop with four steps:

1. You describe a goal and list the actions the model may request. Each action is called a tool, and in Java a tool is just a method you expose, such as getAccountBalance(accountId).
2. The model replies, either with a final answer or with a request like "please run getAccountBalance for account 42."
3. Your application runs that method and sends the result back.
4. The model reads the result and decides again. This repeats until it answers or something stops it.

That is the entire trick. The word "agentic" simply means a system where the model chooses the next step instead of a programmer hard-coding the sequence. A fixed workflow says "do A, then B, then C." An agent says "here is the goal; pick from A, B and C as you see fit."

Three problems show up the moment you do this for real, and the rest of this post is about them:

• Orchestration: who owns the loop, and how do you stop it from running forever?
• Memory: the model forgets everything between calls, so something must store the conversation and decide what to keep.
• Guardrails: the model can request a harmful or nonsensical action, so something must check it before it runs.

If you remember only that, you can read any agent framework announcement and ask the right questions.

Part 2: What Spring AI already gave you

Spring AI is the Spring team's library for talking to LLMs from Java. Its main entry point, ChatClient, sends prompts to a model and returns responses. Around it sits the advisor concept: an advisor is a piece of code that wraps each model call, much like a servlet filter wraps an HTTP request. Advisors can add stored conversation history, inject retrieved documents, log, or block.

Before 2.0, tool calling lived mostly inside the model integration. The framework ran the loop in the background, and you saw little of it. That was convenient until you wanted to count iterations, trace each step, or intervene in the middle.

Part 3: What 2.0 changed, and why I cared

In 2.0 the agent loop moved into the advisor chain. In practice the loop became a visible, composable step that you can place relative to other advisors, rather than a hidden internal behaviour. The advisor that owns this is ToolCallingAdvisor.

This sounds like plumbing, but it changes the design in a way I find useful. When the loop is an advisor, everything else that already works as an advisor can sit inside or outside it:

• Put a logging or tracing advisor outside the loop and you see one span per user request.
• Put one inside the loop and you see one span per model round trip, which is where cost and latency actually live.
• Put a memory advisor so that every iteration is saved, or only the final answer.
• Put a guardrail advisor inside, so each requested tool call is vetted before execution rather than after the fact.

Placement is a decision you make on purpose now. In my experience, most agent incidents are ordering bugs, such as a safety check running once per request when it needed to run once per tool call. Making that ordering explicit removes a whole family of surprises. If you want the observability side of this in depth, I covered the span structure I use in distributed tracing with OpenTelemetry on Spring Boot.

Part 4: What 2.1 fixes

Two items from the 2.1 line matter for production teams.

The tool-call cap. An agent that can call tools can also call them endlessly. A confused model might keep re-requesting the same lookup, or bounce between two tools. Every iteration costs tokens, time and possibly money through the downstream API. A configurable ceiling on tool-call rounds turns an unbounded loop into a bounded one. This is not exciting, but it is the first thing I would demand from any loop, and it belongs in the framework because a cap implemented in application code tends to get skipped under deadline pressure.

The OpenAI Responses model. OpenAI's newer Responses API differs from the older chat completions format in how it represents tool calls and conversation state. Support for that model means Spring AI can talk to it natively instead of forcing it through the older shape. For you, the practical effect is fewer translation hacks if your provider routes through that API, and a cleaner mapping of tool-call items into advisor-visible objects.

I am deliberately keeping my claims here narrow. Read the 2.1 release notes for the exact property names before you code against them, because I do not want you copying a setting name from a blog post.

Part 5: The new agents project

Now to the headline. The agents project is announced as a separate module, not a new version of the core library. Being experimental has a precise meaning in Spring land: interfaces may change between releases, and nothing is promised about long-term compatibility. Graduation into Spring AI 3.0 is the stated direction, and that is exactly why it is worth watching but risky to bet on today.

What it aims to package is the stuff teams keep rewriting: a higher-level agent abstraction, memory handling, and a place for guardrail and orchestration patterns. If you have used LangGraph in Python, or LangChain4j as a Java alternative, the goal will feel familiar. The difference is the Spring flavour: auto-configuration, properties, Micrometer metrics, and the dependency injection model you already run.

Part 6: Adopt now or hand-roll?

My recommendation, stated plainly: keep the loop on ChatClient plus ToolCallingAdvisor for anything that touches money or compliance, and try the experimental agents module in a side project or internal tool. My reasoning:

• The advisor chain in 2.x is a stable seam. If the agents project graduates, you can migrate by swapping a thin layer, because your tools, your memory store and your guardrails are plain classes with no framework base class.
• Experimental modules can rename the thing you built on. In a payments path, an API break is an incident, and the cost of a rewrite lands on the team, not on the framework authors.
• Hand-rolling does not mean writing the loop yourself. It means owning the three concerns (cap, memory, guardrails) using the advisor and tool primitives, and letting the framework run the mechanics.

When would I flip the answer? If the agents module reaches a release with a documented stability promise, if it exposes the same advisor hooks I rely on, and if a pilot shows it removes code instead of hiding it. Until then I treat it as reading material and a spike.

Part 7: Running it on Spring Boot 4 with virtual threads

This is where my fintech background shapes the advice. An agent request is slow by nature: each model round trip can take seconds, and one user request may contain several. That is a lot of waiting on the network, which is precisely the work virtual threads handle well. A virtual thread is a lightweight thread managed by the JVM, so thousands of them can sit blocked on I/O without exhausting an operating-system thread pool.

On Spring Boot 4 with Java 21 or later, I enable them and let each agent request block naturally, which keeps the loop code as readable synchronous Java. I wrote up the pitfalls I hit, including pinning and pool sizing, in virtual threads in Spring Boot production. Two agent-specific warnings apply:

• Blocking is cheap, concurrency is not free. Ten thousand parked virtual threads are fine, but ten thousand simultaneous calls into your model provider will hit rate limits. Put a semaphore or a bulkhead around the model client, not around threads.
• Tool methods run on whatever thread the loop uses. If a tool holds a database connection, your connection pool becomes the real limit, so size it with agent fan-out in mind.

If your agents need GPUs for local models, scheduling is a separate headache that I covered in Kubernetes Dynamic Resource Allocation for AI microservices.

Part 8: A hand-rolled loop with the three concerns owned

Here is the shape I use. The cap, the memory and the guardrail are each a small piece I control. Treat it as a sketch; confirm builder method names against the release you run.

@Configuration
class AgentConfig {

    @Bean
    ChatClient agentClient(ChatClient.Builder builder,
                           ChatMemory memory,
                           ToolGuard guard,
                           AccountTools tools) {
        return builder
            .defaultSystem("""
                You assist with read-only account questions.
                Never request a tool unless the user asked for that data.
                """)
            .defaultTools(tools)
            .defaultAdvisors(
                MessageChatMemoryAdvisor.builder(memory).build(),
                new ToolCallingAdvisor(),   // owns the loop
                guard                       // vets each round
            )
            .build();
    }
}

@Component
class ToolGuard implements CallAdvisor {

    private final Set<String> allowed = Set.of("getBalance", "listTransactions");
    private final MeterRegistry metrics;

    ToolGuard(MeterRegistry metrics) { this.metrics = metrics; }

    @Override
    public ChatClientResponse adviseCall(ChatClientRequest req, CallAdvisorChain chain) {
        var response = chain.nextCall(req);
        response.chatResponse().getResult().getOutput().getToolCalls()
            .forEach(call -> {
                if (!allowed.contains(call.name())) {
                    metrics.counter("agent.tool.denied", "tool", call.name()).increment();
                    throw new ToolDeniedException(call.name());
                }
            });
        return response;
    }

    @Override public String getName() { return "toolGuard"; }
    @Override public int getOrder() { return 100; }
}

Three things are worth noticing. First, the allow-list is deny-by-default, so a model that invents a tool name is stopped and counted. Second, the guard emits a metric, because a rising denial rate is an early signal that a prompt or a model version drifted. Third, the tools are read-only by design. Anything that writes, such as initiating a transfer, I route through a separate human-confirmation step and the patterns from transactional outbox and idempotent consumers, so a retried model decision cannot send money twice.

For the loop cap, I set the framework's tool-call limit where the release provides it, and keep a wall-clock deadline in the request handler as a second fence. Two independent limits is the point: one counts rounds, the other counts seconds.

Part 9: Memory and agentic RAG

Memory is the quiet failure source. A naive approach stores every message forever, which inflates token cost and leaks old context into new answers. My rules:

• Key memory by conversation id, never by user id alone, or two chats mix.
• Keep a windowed history (the last N messages) and summarise older turns.
• Store tool results with care. A balance fetched ten minutes ago is stale, and a model that trusts stale memory will state it confidently.

Retrieval-augmented generation (RAG) means fetching relevant documents and showing them to the model so it answers from your data. Agentic RAG lets the model decide when to retrieve and what to search for, as one more tool. It is more flexible than fixed retrieval, and also easier to misuse, because the model might skip retrieval and answer from imagination. I make retrieval a tool the guard always permits, and I log whether a final answer cited any retrieved source.

If you expose your own services to agents rather than embedding them, the Model Context Protocol (MCP), a standard way for a model to discover and call tools over a network, is the cleaner route. I walked through that setup in exposing a Java backend to AI agents with an MCP server. It also keeps your tool contract separate from any one agent framework, which makes a later migration cheaper.

Part 10: The guardrails I will not skip

For services where mistakes are expensive, I treat these as non-negotiable, whatever framework runs the loop:

1. Deny-by-default tools, as above.
2. Read and write separation: agents get read tools; writes need explicit confirmation.
3. Per-request budgets for tokens, rounds and wall-clock time.
4. Full tracing of prompts, tool arguments and results, with sensitive fields masked. The approach I use is in my post on observability guardrails for agentic AI backends.
5. Replayability: store enough to rerun a failed conversation. I learned this from building market-data systems, where deterministic replay is how you debug anything, and the same idea transfers.

What I will actually do

On my own projects I am keeping the production agent paths on ChatClient and ToolCallingAdvisor, and I am building one internal, read-only assistant on the experimental agents module as a spike, so that I learn its rough edges before anyone's money depends on it. I will compare lines of code, trace clarity and how many concerns it hides from me. If you want to see how I approach systems like this across real deployments, my background is on the experience section of this site, and the projects section shows what I run today.

Your next step this week: pick one agent path you already run, add a hard tool-call cap and a deny-by-default tool allow-list to it, and ship a counter for denied calls. That single change makes any future framework migration safer, and it takes an afternoon.

Related Articles

• Kubernetes Dynamic Resource Allocation for GPU-backed Spring Boot AI microservices
• Observability guardrails for agentic AI backends on Spring Boot
• Exposing a Java backend to AI agents with a Spring AI MCP server