Skip to content

Fix scenario runner event loop freeze and initial_unpark timeout#292

Open
deepmasq wants to merge 1 commit intomainfrom
fix/scenario-runner-event-loop-freeze
Open

Fix scenario runner event loop freeze and initial_unpark timeout#292
deepmasq wants to merge 1 commit intomainfrom
fix/scenario-runner-event-loop-freeze

Conversation

@deepmasq
Copy link
Copy Markdown
Contributor

@deepmasq deepmasq commented Apr 7, 2026

Summary

  • Event loop freeze fix: In scenario mode (running_test_scenario), tool calls are now routed through scenario_generate_tool_result_via_model as fire-and-forget instead of going through local tool handlers. The local handlers use gql/aiohttp sessions whose cleanup (__aexit__) blocks the event loop, causing the scenario runner to hang indefinitely after the first tool call.

  • Initial unpark timeout: _completed_initial_unpark now has a 30-second timeout. Previously, scheduler events (kanban sync, etc.) kept did_anything=True on every unpark cycle, so the flag never got set to True, blocking the scenario runner forever in the "WAIT for bot to complete initial unpark" loop.

  • Background cloudtool_post_result: Moved cloudtool_post_result in _local_tool_call to an asyncio.create_task background task to prevent blocking the unpark loop.

  • Fire-and-forget pattern in ckit_scenario: scenario_generate_tool_result_via_model now wraps the GQL mutation in asyncio.create_task and immediately raises AlreadyFakedResult, so the caller never awaits the HTTP response.

Test plan

  • Ran 10 Karen scenarios on staging with grok-4-1-fast-reasoning -- 8/10 passed (2 failures were model quality issues, not infra)
  • Verify no regressions in non-scenario bot execution (cloudtool_post_result background task change)
  • Verify version bump 1.2.196 deploys cleanly

🤖 Generated with Claude Code

@deepmasq deepmasq force-pushed the fix/scenario-runner-event-loop-freeze branch 2 times, most recently from 9f72e99 to 80dd5ee Compare April 7, 2026 13:36
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@deepmasq deepmasq force-pushed the fix/scenario-runner-event-loop-freeze branch from 80dd5ee to bcb2e6b Compare April 7, 2026 13:50
@deepmasq deepmasq marked this pull request as ready for review April 7, 2026 14:40
@deepmasq deepmasq requested a review from olegklimov April 7, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants