fix(buffers): recheck disk buffer progress before waiting

[fernandol-nvidia]

Summary

Fix two disk_v2 reader coordination failures:

• A transient EOF after a writer notification could leave published data unread until another write or shutdown.
• Dropping the final published frame as corrupt could make the reader retry continuously and consume CPU.

This is related to #23605, which reports events remaining in a disk buffer until restart, and #23456, which tracks a rare reader wake-up failure in vector-buffers. The race fixed here is reproducible on current master and the latest nightly. These issues cover broader symptoms; this PR addresses one concrete reader-side cause.

The reader now uses a runtime-only published byte boundary for the active writer file. It retries while published bytes remain and waits once those bytes have been consumed. The boundary is reset on file rollover and rebuilt during recovery.

This does not change configuration or the on-disk format.

How did you test this PR?

• Added deterministic coverage for transient EOF, corrupt-frame recovery, and visible but unpublished bytes.
• cargo test -p vector-buffers
• cargo clippy -p vector-buffers --all-targets --all-features -- -D warnings
• cargo build -p vector-buffers --release
• make check-changelog-fragments
• make check-markdown

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes. A changelog fragment is included.
• No. A maintainer will apply the no-changelog label to this PR.

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[datadog-vectordotdev]

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 0659219 | Docs | Datadog PR Page | Give us feedback!}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ba0bd6f40a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[fernandol-nvidia]

I have read the CLA Document and I hereby sign the CLA