Skip to content

feat: add flush timeout#687

Open
marandaneto wants to merge 3 commits into
mainfrom
fix/flush-timeout-79
Open

feat: add flush timeout#687
marandaneto wants to merge 3 commits into
mainfrom
fix/flush-timeout-79

Conversation

@marandaneto

@marandaneto marandaneto commented Jun 19, 2026

Copy link
Copy Markdown
Member

💡 Motivation and Context

Fixes #79.

flush() previously waited indefinitely on the internal queue, which could block short-lived Celery/serverless workloads if the queue never drained. This adds a bounded wait by default while preserving the old behavior with timeout_seconds=None.

💚 How did you test it?

  • uv run --extra test pytest posthog/test/test_client.py posthog/test/test_module.py -q
  • uv run --extra test pytest posthog/test/test_client.py::TestClient::test_shutdown posthog/test/integrations/test_celery_integration.py -q
  • uv run --extra test pytest posthog/test/test_client.py::TestClient::test_flush_timeout_returns_when_queue_does_not_drain posthog/test/test_client.py::TestClient::test_flush_logs_and_returns_on_unexpected_error posthog/test/test_client.py::TestClient::test_shutdown_flushes_without_timeout -q
  • uv run --extra dev ruff check posthog/client.py posthog/__init__.py posthog/test/test_client.py posthog/test/test_module.py
  • uv run --extra dev mypy posthog/client.py posthog/__init__.py --config-file mypy.ini
  • uv run --extra dev make public_api_check

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran sampo add to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Implemented with the pi coding agent. The change is intentionally small: flush() now accepts timeout_seconds with a 10 second default, uses the queue condition variable to avoid indefinite blocking, logs on timeout, and keeps indefinite waiting available via timeout_seconds=None. Review feedback was addressed by making flush() defensive, preserving shutdown()'s indefinite flush behavior, marking the release as a minor feature, and updating the public API snapshot.

@marandaneto marandaneto self-assigned this Jun 19, 2026
Comment thread .sampo/changesets/steadfast-baroness-vainamoinen.md Outdated
Comment thread posthog/client.py Outdated
@greptile-apps

greptile-apps Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Comments Outside Diff (1)

  1. posthog/client.py, line 1496-1497 (link)

    P1 shutdown() now silently drops events: it calls self.flush() with no argument, which picks up the new 10-second default. shutdown()'s documented contract is "Flush all messages… to avoid data loss," but if consumers haven't finished within 10 seconds the remaining events are silently discarded. Pass timeout_seconds=None here so shutdown() keeps its original "wait indefinitely" guarantee, while direct flush() callers still get the bounded timeout.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: posthog/client.py
    Line: 1496-1497
    
    Comment:
    `shutdown()` now silently drops events: it calls `self.flush()` with no argument, which picks up the new 10-second default. `shutdown()`'s documented contract is "Flush **all** messages… to avoid data loss," but if consumers haven't finished within 10 seconds the remaining events are silently discarded. Pass `timeout_seconds=None` here so `shutdown()` keeps its original "wait indefinitely" guarantee, while direct `flush()` callers still get the bounded timeout.
    
    
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
posthog/client.py:1496-1497
`shutdown()` now silently drops events: it calls `self.flush()` with no argument, which picks up the new 10-second default. `shutdown()`'s documented contract is "Flush **all** messages… to avoid data loss," but if consumers haven't finished within 10 seconds the remaining events are silently discarded. Pass `timeout_seconds=None` here so `shutdown()` keeps its original "wait indefinitely" guarantee, while direct `flush()` callers still get the bounded timeout.

```suggestion
        self.flush(timeout_seconds=None)
        self.join()
```

### Issue 2 of 2
posthog/test/test_client.py:144-157
Potential race condition in new test: a live consumer thread may dequeue and call `task_done()` on the manually-inserted item before `flush()` is called, making the queue appear empty and causing `assertFalse(client.queue.empty())` to fail intermittently. Consider using `threads=0` (or the equivalent config to disable background consumers) when constructing this test client so the queue item is guaranteed to stay unprocessed for the duration of the timeout check.

Reviews (1): Last reviewed commit: "fix: add flush timeout" | Re-trigger Greptile

Comment thread posthog/test/test_client.py
@marandaneto marandaneto changed the title fix: add flush timeout feat: add flush timeout Jun 19, 2026
@marandaneto

Copy link
Copy Markdown
Member Author

Addressed Greptile feedback in 7d3a956: shutdown() now calls flush(timeout_seconds=None) to preserve its indefinite wait contract, and the timeout test now uses thread=0.

@github-actions

github-actions Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

posthog-python Compliance Report

Date: 2026-06-19 13:55:27 UTC
Duration: 530071ms

✅ All Tests Passed!

45/45 tests passed


Capture Tests

29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 516ms
Format Validation.Event Has Uuid 10007ms
Format Validation.Event Has Lib Properties 10006ms
Format Validation.Distinct Id Is String 10006ms
Format Validation.Token Is Present 10006ms
Format Validation.Custom Properties Preserved 10006ms
Format Validation.Event Has Timestamp 10005ms
Retry Behavior.Retries On 503 18017ms
Retry Behavior.Does Not Retry On 400 12002ms
Retry Behavior.Does Not Retry On 401 10006ms
Retry Behavior.Respects Retry After Header 16011ms
Retry Behavior.Implements Backoff 30018ms
Retry Behavior.Retries On 500 13006ms
Retry Behavior.Retries On 502 16008ms
Retry Behavior.Retries On 504 16009ms
Retry Behavior.Max Retries Respected 30017ms
Deduplication.Generates Unique Uuids 7000ms
Deduplication.Preserves Uuid On Retry 16014ms
Deduplication.Preserves Uuid And Timestamp On Retry 23016ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 16001ms
Deduplication.No Duplicate Events In Batch 10004ms
Deduplication.Different Events Have Different Uuids 10005ms
Compression.Sends Gzip When Enabled 10006ms
Batch Format.Uses Proper Batch Structure 10006ms
Batch Format.Flush With No Events Sends Nothing 5005ms
Batch Format.Multiple Events Batched Together 10004ms
Error Handling.Does Not Retry On 403 12008ms
Error Handling.Does Not Retry On 413 10006ms
Error Handling.Retries On 408 14012ms

Feature_Flags Tests

16/16 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 9501ms
Request Payload.Flags Request Uses V2 Query Param 10006ms
Request Payload.Flags Request Hits Flags Path Not Decide 10006ms
Request Payload.Flags Request Omits Authorization Header 10006ms
Request Payload.Token In Flags Body Matches Init 10006ms
Request Payload.Groups Round Trip 10006ms
Request Payload.Groups Default To Empty Object 10006ms
Request Payload.Person Properties Distinct Id Auto Populated When Caller Omits It 10006ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 10006ms
Request Payload.Disable Geoip Omitted Defaults To False 10006ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 10006ms
Request Lifecycle.No Flags Request On Init Alone 5003ms
Request Lifecycle.No Flags Request On Normal Capture 10506ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 9510ms
Request Lifecycle.Mock Response Value Is Returned To Caller 10003ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 10509ms

@marandaneto marandaneto marked this pull request as ready for review June 19, 2026 13:48
@marandaneto marandaneto requested a review from a team as a code owner June 19, 2026 13:48
@greptile-apps

greptile-apps Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Reviews (2): Last reviewed commit: "update public api snapshot" | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a 10 second max timeout to flush requests

1 participant