Skip to content

Fix eval_start_step logic with eval_interval#4585

Open
zxhe-sean wants to merge 1 commit into
mainfrom
zxhe/eval_start_step
Open

Fix eval_start_step logic with eval_interval#4585
zxhe-sean wants to merge 1 commit into
mainfrom
zxhe/eval_start_step

Conversation

@zxhe-sean

@zxhe-sean zxhe-sean commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Description

This PR resolves the logic conflict and confusion when both eval_start_step and eval_interval are specified.

  • Why is this change being made:
    Previously, when users set both eval_start_step and eval_interval, the evaluation logic had conflicting behaviors—either waiting until the end of the first full interval before starting or misaligning the initial evaluation step with subsequent intervals.
  • The problem being solved and relevant context:
    Standardizing the behavior so that evaluation explicitly starts at eval_start_step and subsequently follows eval_interval steps ((step - eval_start_step) % eval_interval == 0).
    • Users have a clean, intuitive way to decide when evaluation first starts and how frequently it repeats.
    • If users want the legacy behavior (e.g., eval_interval=10 evaluating at steps 9, 19, 29...), they can achieve it under this unified model by setting eval_start_step=9.

FIXES: b/535641549

Tests

Tested locally

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@zxhe-sean
zxhe-sean force-pushed the zxhe/eval_start_step branch 4 times, most recently from fc17d60 to 35ede7a Compare July 24, 2026 00:57
@zxhe-sean
zxhe-sean force-pushed the zxhe/eval_start_step branch from 35ede7a to 25702cd Compare July 24, 2026 01:00
@zxhe-sean zxhe-sean changed the title Fix eval_start_step logic and set default to -1 Fix eval_start_step logic with eval_interval Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant