Skip to content

feat: carry reasoning across turns with per-backend replay policy#1374

Open
AngeloDanducci wants to merge 2 commits into
generative-computing:mainfrom
AngeloDanducci:ad-1201
Open

feat: carry reasoning across turns with per-backend replay policy#1374
AngeloDanducci wants to merge 2 commits into
generative-computing:mainfrom
AngeloDanducci:ad-1201

Conversation

@AngeloDanducci

@AngeloDanducci AngeloDanducci commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Pull Request

Issue

Fixes #1201

Description

Carry reasoning across turns with per-backend replay policy for better thinking handling.

The policy is currently provider-agnostic (provider is unused, documented as a reserved hook). The divergent contracts from #1201 table — DeepSeek hard-400 without tools, Anthropic signed blocks — are not implemented here.

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code was added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

  • AI coding assistants used

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

  • Component
  • Requirement
  • Sampling Strategy
  • Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

Signed-off-by: AngeloDanducci <angelo.danducci.ii@ibm.com>
@github-actions github-actions Bot added the enhancement New feature or request label Jul 9, 2026
@AngeloDanducci AngeloDanducci marked this pull request as ready for review July 13, 2026 07:14
@AngeloDanducci AngeloDanducci requested a review from a team as a code owner July 13, 2026 07:14
@AngeloDanducci AngeloDanducci requested a review from ajbozarth July 13, 2026 07:14

@ajbozarth ajbozarth left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the full diff and traced the capture → _parse → policy → serialize path. Solid, conservative change — no correctness blockers. All 72 new/changed tests pass locally. Action items below.

Before merge

  • Complete the live-model verification. The e2e test (test_reasoning_replay_e2e.py) is env-gated and CI-skipped, so nothing in CI exercises reasoning_content/thinking against a real endpoint. The PR body's own "still needs verification" note stands — this is the one thing that would catch a provider rejecting the injected wire key.
  • Tighten the PR body to say this lands the consensus rule only. The title says "per-backend replay policy" and provider is a parameter, but the policy is currently provider-agnostic (provider is unused, documented as a reserved hook). The divergent contracts from #1201's table — DeepSeek hard-400 without tools, Anthropic signed blocks — are not implemented here. State that explicitly so reviewers/users don't assume DeepSeek/Anthropic are covered.

Non-blocking

  • HF chat path is silently uncovered (see inline comment). Functionally fine given HF's capture gap, but a one-liner would prevent a future reader assuming coverage.

Verified separately

  • should_replay_reasoning returns exactly len(messages) flags, so the zip in each backend never truncates.
  • The adjacency check correctly maps to "this turn issued a tool call" — _acall_tools appends ToolMessage(role="tool") right after the tool-issuing turn.
  • Default replay_reasoning=False preserves historical behavior.
  • Real OpenAI Chat Completions never populates thinking, so the non-standard key is never sent there.

Comment thread mellea/backends/huggingface.py
Signed-off-by: AngeloDanducci <angelo.danducci.ii@ibm.com>
@AngeloDanducci

Copy link
Copy Markdown
Contributor Author

Complete the live-model verification. - Live model verification was completed, just forgot to remove the line from the PR description when converting from draft.

@AngeloDanducci AngeloDanducci requested a review from ajbozarth July 13, 2026 18:19
@ajbozarth

Copy link
Copy Markdown
Contributor

This LGTM as far as Claude reviews go , but I'd prefer to defer final approval to @jakelorocco after he returns since this is a decently sized change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Better handling of reasoning output from thinking models

2 participants