[javaone 2026] Building Agents with Spring AI and Amazon Bedrock

Posted on by
Reply

Speakers: Josh Long & James Ward

See the live blog table of contents

Repo: live coded at session

https://github.com/joshlong-attic/2026-03-18-javaone-bootiful-spring-boot-ai

General

  • Agent – call LLM in a loop and give it tools so it can do things.

Spring Intiailzr pulls

  • Spring MVC
  • vector database (for RAG)
  • Amazon Bedrock (embeddings for RAG)
  • Amazon Bedrock Converse (for chat)
  • Spring Boot Actuator
  • MCP client

Random stuff

  • UnzipAndOpen.java – made an alias to run quickly
  • Postgres – pgvector – postgres with vector support

My take

I wasn’t sure what to expect from this. I put it on my calendar thinking it was more like a hands on lab (I also prepared my hack session immediately after in that format). I was warned the day before that most “hack session” presenters did mostly speaking. I saw this a similar presentation at the NYJavaSIG, but different emphasis.\. So I came both to get a feel for how I want to handle mine and to see what the space was like [hack sessions are in a tent where we had lunch]. Also luckily I was warned the tent gets hot so I changed into shorts before my time in the tent. (While it was hot for lunch, it wasn’t as noticeable because less time)

[javaone 2026] Developing an Asynchronous Application with Virtual Threads and Structured Concurrency

Posted on by
Reply

Speakers: Jose Paumarrd & Ana-Maria Mihalceanu

See the live blog table of contents

Structured concurrency pattern. Note tasks end by autoclosable block complete

try (var scope - StructuredTaskScope.open()) {

  var subtask = scope.fork(lambda)

scope.join()

var result = subtask.get();

}

Scopes

  • A scope can spawn other scope
  • Closing the parent scope closes the child scopes

Lab

https://github.com/JosePaumard/2026_JavaOne-Loom-lab/blob/Step-00_Initial-application/JavaOne-Loom-Lab.md

My take

This feels like streams where it is going to take me a few tries to “get” it. Luckily, I have time since structured concurrency is still in preview. I did ask my question about ScopedValues being useful without structured concurrency. You can use them but you need a ScopedValue.where() call for each thread as far as I understand which is limited in usefulness.

[javaone2026] Running GPU-Accelerated AI Inference from Java at Uber Scale

Posted on by
Reply

Speakers: Baojun Liu & Anshuman Mishra

See the live blog table of contents

Michelangelo

  • Uber’s unified ML platform
  • 20K models trained/month
  • 5.3 models in prod

Java

  • Online prediction is all Java
  • High concurrency orchestration
  • Business logic management
  • Production ecosystem integration

Spark ML pipeline

  • C++ is ML runtime and Math libraries
  • Java manages threads, coordinates execution, handles back pressure

Scaling

  • Model sizes have increased from 100K to 800G
  • Traffic growth about 30% per year

GPU/CPU

  • GPU cluster – 20x cost reduction and order of magnitude less instances
  • NVIDIA Triton inference server (C++)
  • GPU can handle 10x-100x more requests
  • Dynamic latency – trades latency for throughput
  • Scale down instance count for large models to save memory and up for high queries per second
  • One GPU to multiple CPUs. CPU bottleneck – cold start, error spikes, memory pressure

Techniques

  • Profiling
  • Send dummy data to avoid cold start
  • Tune GC memory and pause time
  • Frequent efficient collections
  • Panama
  • Virtual threads
  • Upgrade Java newer version to reduce CPU use

Other notes

  • Java is not the bottleneck, bad architecture is

My take

The beginning was the same slide as the keynote. Ok to review though. It got new quickly and it was nice to see what Uber is doing. Good questions from the audience