Skip to content

fix: avoid double-counting logprobs in streamed chat completions - #2444

Open
dfedoryshchev wants to merge 1 commit into
openai:mainfrom
dfedoryshchev:fix/logprobs-double-count
Open

fix: avoid double-counting logprobs in streamed chat completions#2444
dfedoryshchev wants to merge 1 commit into
openai:mainfrom
dfedoryshchev:fix/logprobs-double-count

Conversation

@dfedoryshchev

@dfedoryshchev dfedoryshchev commented Aug 21, 2026

Copy link
Copy Markdown
Contributor
  • I understand that this repository is auto-generated and my pull request may not be merged

Changes being requested

#accumulateChatCompletion seeds a new choice with the chunk's logprobs by reference, so on that same iteration choice.logprobs === logprobs and choice.logprobs.content.push(...content) pushes the array into itself. One streamed token arrives as two entries in finalChatCompletion().choices[0].logprobs.content and in the logprobs.content.done payload.

The initialise branch then does choice.logprobs = { ...logprobs }, one level deep, so the snapshot keeps the caller's array and the next chunk's push appends into the chunk the caller passed in.

Current tests miss both: the fixture's first logprobs-bearing chunk is {"content":[],"refusal":null}, so push(...[]) is a no-op, and nothing streams two chunks that each carry logprobs.

Fix: seed logprobs: null, and copy the arrays when initialising.

Additional context & links

  • Three added tests cover the content path, the refusal path and cross-chunk accumulation; all three fail on main.
  • npx vitest run --config vitest.config.mts tests/lib/ChatCompletionStream.test.ts -> 38 passed. oxlint and oxfmt --check pass.

@dfedoryshchev
dfedoryshchev requested a review from a team as a code owner August 21, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant