fix: make LLM review resilient to individual question failures#550
Merged
fix: make LLM review resilient to individual question failures#550
Conversation
- Increase LLM call timeout from 20s to 90s to handle large code contexts - Remove redundant outer retry loop in AskLLMAboutCode (inner callLLMWithRetry already handles retries) - Skip failed questions instead of aborting the entire review - Report OK status when LLM review completes without concerns
academo
commented
Mar 26, 2026
| } | ||
| lastErr = nil | ||
| break | ||
| agenticAnswers, err := agenticClient.CallLLM(c.ctx, []string{userPrompt}, absCodePath) |
Collaborator
Author
There was a problem hiding this comment.
the main change here is removing the for retries := 3; retries > 0; retries-- loop
this was retrying 3 times but CallLLM already has retrying handling internally (3 times)
tolzhabayev
approved these changes
Mar 26, 2026
github-project-automation
bot
moved this from 🔬 In review
to 🚀 Shipped
in Grafana Catalog Team
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The LLM review was silently dropping all results when any single question failed. Two issues:
AskLLMAboutCodehad a retry loop wrappingCallLLM, which itself already retries internally viacallLLMWithRetryso a failing question would fire up to 9 API calls before giving up, then discard all previously successful answerscontext deadline exceedederrors on AnthropicNow failed questions are skipped and processing continues, the redundant outer retry is removed, and the timeout is bumped to 90s. Also adds an OK report when LLM review completes clean.