Skip to content

Add Streaming Capabiity Tutorial#19274

Open
igordayen wants to merge 3 commits into
eugenp:masterfrom
igordayen:streaming-with-reasoning
Open

Add Streaming Capabiity Tutorial#19274
igordayen wants to merge 3 commits into
eugenp:masterfrom
igordayen:streaming-with-reasoning

Conversation

@igordayen

@igordayen igordayen commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Integration tests for Embabel Agent streaming with OpenAI and Anthropic, covering structured
object streaming, text-instructed thinking, and tool-loop reasoning.

OpenAI (StreamingWithThinkingIntegrationTest)

  • whenStreaming_thenReceivesParkingRecommendation — basic structured streaming, single object received via reactive pipeline
  • whenStreamingWithInstructedThinking_thenReceivesReasoningBlocks — text-instructed thinking via <think> format; reasoning events emitted per line, tags present or stripped depending on chunking
  • whenStreamingMultipleScenarios_thenReceivesRecommendationPerScenario — collection streaming, one object per
    scenario streamed incrementally

Anthropic (StreamingWithThinkingAndToolingIntegrationTest)

  • whenStreamingWithThinking_thenReceivesReasoningAndRecommendation — thinking with token budget (< max_tokens=8192), no tools; multiple reasoning events emitted one per output line
  • whenStreamingWithThinkingAndTooling_thenReceivesRecommendationAndReasoning — tool-loop streaming with three parking tools; reasoning surfaced only from the final LLM iteration after all tool calls complete

Notes

  • Anthropic budget_tokens must be strictly less than max_tokens (8192 for claude-sonnet-4-5)
  • Intermediate reasoning between tool calls is not surfaced to the subscriber as per Spring AI Streaming infra
  • PR is work-in-progress; need to move common data structures to common module
  • switched to 1.0.0 (stable release)

@igordayen

Copy link
Copy Markdown
Contributor Author

At a glance, there is a very high amount of duplication between the StreamingWithThinkingAndToolingIntegrationTest and StreamingWithThinkingIntegrationTest, which is not something I'd usually encourage unless there's a good reason for it. ==> CONSOLIDATED

I can see one is a embabel-streaming-anthropic example and another embabel-streaming-openai.

As I understand, the focus of this article is about Embabel's streaming capabilities and not so much about vendor specific integration (e.g. OpenAI vs Anthrophic) - so I'd lean towards picking one vendor to use to demonstrate the capabilities. ==> USING ANTHROPIC

If there's something very different about the way each vendor can be used, perhaps a dedicated section just for those differences would make sense, but even then it probably sounds outside the scope of this particular article if the main goal is to demonstrate the common features of Embabel Streaming, rather than the vendor specific differences. ==> AGREE

I can also see AI was used to generate these examples, which is allowed, but particularly for a new library like Embabel I'd expect that you would personally have more expertise on how to best demonstrate and test the library. Leaning too much on AI here might lead to excessively verbose/duplicative generated examples compared to what you might achieve if you wrote them more directly. ==> REMOVED Boilerplate code

On the system under test vs test harness separation - as with the previous article we did, each test here unfortunately has the system under test tightly mixed together with the test harness, making it hard to see which parts are "how to use Embabel" vs "how we tested Embabel here". We also seem to be exercising things like "Streaming should not produce errors" and "Stream should complete successfully" which seem to be being demonstrated incidentally rather than for any particular purpose or use case. I would lean towards focusing on demonstrating a real use case rather than synthetic coverage. I'd keep each test focused on demonstrating a particular goal the user is trying to achieve, and stick to proving that goal is met, rather than a general test that exercises coverage. This may be a consequence of you sourcing your examples from an existing test code repository - the goal in that repository is to prove the library works as it is supposed to and achieve high test coverage - the goal when writing an article is a bit different, we want to walk the reader through some real world examples they may encounter, and prove those work.==> I would like to keep doOnError and doOnComplete just for a single test to explain reader that those callbacks are available for development if needed

On the testing approach - I can see we are again relying on log statements as the main way to demonstrate the capabilities. If possible, I'd prefer to move more of the testing to be assertion-based, so we can prove what is happening with a green build, rather than by a human looking at log lines. I appreciate this might be a challenge if anything is non-deterministic here, but I'd hope at least some properties of the expected output can be specified. Also note the guideline about using AssertJ for any complex assertions (e.g. hasSize). ==> ADDRESSED: added more asserts on domain object

I think I would recommend you ditch the "Example got patterned after" approach altogether and instead create a fresh set of examples just for this article. I think this would really help make the examples flow more coherently for the reader and get us to a better result overall. ==> REMOVED, its all brand new for Baeldung

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant