fix(flags): parse locally-evaluated flag payloads in evaluate_flags() (sdk-specs local-feature-flag-evaluator)

[posthog]

💡 Motivation and Context

Compliance gap against the cross-SDK contracts in PostHog/sdk-specs.

openspec/specs/local-feature-flag-evaluator/spec.md — Behavior #8 "Resolve payload from the chosen value", Scenario "Evaluator resolves payload from the matched value":

AND local feature flag definitions include a multivariate flag "checkout" with variant "blue" and payload: | copy | new |
WHEN local feature flag "checkout" is evaluated for distinct id "user-123"
THEN the local evaluation payload should include: | copy | new |

openspec/specs/get-feature-flag-payload/spec.md says the same thing about the read path:

Parse serialized payloads if necessary. Some SDKs store payloads as JSON strings in cache and parse them on read before returning them.

What was out of compliance. In Client.evaluate_flags(), the remote branch JSON-decodes metadata.payload before building the _EvaluatedFlagRecord, but the local branch stored the value straight from the flag definition — where payloads live as JSON strings. So the payload type of the same flag depended on where it happened to resolve:

flags = posthog.evaluate_flags("user-123")
flags.get_flag_payload("checkout")
# poller resolved it locally  -> '{"copy": "new"}'   (str)
# fell back to /flags         -> {"copy": "new"}     (dict)

The rest of the SDK already treats "payload" as the decoded value — FeatureFlagResult.from_value_and_payload / from_flag_details both json.loads, so get_feature_flag_result(...).payload returns a dict for the very same locally-evaluated flag that evaluate_flags(...).get_flag_payload(...) returned a string for. The same raw string also leaked into $feature_flag_payload on $feature_flag_called events for locally-evaluated flags, while the remote path sent the decoded value (the existing test at posthog/test/test_feature_flag_called_minimization.py notes "The evaluate_flags path JSON-parses the payload before attaching it" — which held only for remote).

How this fixes it. Both branches now route through one _parse_flag_payload() helper, so a decoded payload is what callers see regardless of evaluation source. The helper preserves the remote branch's existing semantics exactly: an empty string becomes None, a string that isn't valid JSON is passed through unchanged, and non-string values (a number or an already-structured payload) are untouched.

Behavior change / compatibility risk. evaluate_flags(...).get_flag_payload(key) and $feature_flag_payload now return the decoded payload for locally-evaluated flags instead of a JSON string. Anyone who called json.loads() on that result will need to drop it — but only for flags that resolved locally, since the same code already received a decoded value on the remote path, so no caller could have relied on the string shape reliably. No public API, signature, config, or event-name changes.

Deliberately scoped to the evaluate_flags() snapshot. get_all_flags_and_payloads() returns JSON strings on both its local and remote paths, so it is internally consistent today; changing it is a wider, unambiguously breaking change and belongs in its own discussion.

💚 How did you test it?

Added TestEvaluateFlagsLocalPayloads in posthog/test/test_evaluate_flags.py, covering local evaluation of a multivariate and a boolean flag (decoded payload, no /flags request), a non-JSON payload string (passed through), and the $feature_flag_payload property on $feature_flag_called.

Verified the two payload assertions fail without the fix (AssertionError: '{"copy": "new"}' != {'copy': 'new'}) and pass with it. Also ran test_evaluate_flags.py, test_feature_flags.py, test_feature_flag.py, test_feature_flag_result.py, test_feature_flag_called_minimization.py, test_feature_flag_has_experiment.py, test_types.py, test_client.py, test_module.py, test_contexts.py and posthog/test/features — all passing. ruff format --check, ruff check, mypy, and check_public_api.py are clean (no public API surface change).

📝 Checklist

• I reviewed the submitted code.
• I added tests to verify the changes.
• I updated the docs if needed.
• No breaking change or entry added to the changelog.

If releasing new changes

• Ran sampo add to generate a changeset file

🤖 Agent context

Autonomy: Fully autonomous

Opened by the scheduled "SDK Spec Compliance Enforcer" loop, which audits posthog-python against the contracts in PostHog/sdk-specs and files one focused fix per run. This run checked the server-SDK-applicable requirements across ~40 specs (capture/exception, batching + HTTP/retry, flag getters, flag internals, identity, logs/traces/tracing-headers) using Claude Code with read-only search plus targeted runs of the SDK against mocked transports.

This finding was chosen over the other candidates because it is narrow, verifiable against two spec files, and internally corroborated: the SDK's own FeatureFlagResult and the remote branch of the same function already decode payloads, so this is the recommended non-deprecated API regressing against a convention the rest of the codebase keeps. Candidates deliberately not filed: the group-flag-without-group-context "conclusive False" behavior (the code comment marks it intentional and it is shared with other server SDKs — needs a cross-SDK decision), get_all_flags_and_payloads() payload strings (wider breaking change), and missing input validation on alias() / group_identify() (real but separate; would drop events users send today).

Agent-authored — please review rather than rubber-stamp, especially the compatibility call on returning decoded payloads.

---

Created with PostHog Code

[github-actions]

posthog-python Compliance Report

Date: 2026-08-04 08:04:12 UTC
Duration: 338573ms

✅ All Tests Passed!

111/111 tests passed

---

Capture_V1 Tests

✅ 94/94 tests passed

View Details

Test	Status	Duration
Endpoint And Method.Targets V1 Endpoint	✅	514ms
Endpoint And Method.Does Not Use Legacy Endpoints	✅	1007ms
Required Headers.Has Authorization Bearer Header	✅	1007ms
Required Headers.Has Content Type Json	✅	1007ms
Required Headers.Has Posthog Sdk Info Format	✅	1007ms
Required Headers.Has Posthog Attempt Header	✅	1007ms
Required Headers.Has Posthog Request Id	✅	1007ms
Required Headers.Has Posthog Request Timestamp	✅	1007ms
Required Headers.Has User Agent	✅	1007ms
Body Format.Body Has Created At And Batch	✅	1007ms
Body Format.No Api Key In Body	✅	1007ms
Body Format.No Sent At In Body	✅	1007ms
Event Format.Event Has Required Root Fields	✅	1007ms
Event Format.Event Uuid Is Valid	✅	1006ms
Event Format.Event Timestamp Is Rfc3339	✅	1007ms
Event Format.Distinct Id Is String	✅	1007ms
Event Format.Distinct Id At Root Not Properties	✅	1007ms
Event Format.Custom Properties Preserved	✅	1006ms
Event Format.Set Properties Preserved	✅	1007ms
Event Format.Set Once Properties Preserved	✅	1008ms
Event Format.Groups Properties Preserved	✅	1007ms
Event Format.Sdk Generates Uuid If Not Provided	✅	1008ms
Event Format.Event Has Required Root Fields Batch	✅	1009ms
Event Format.Event Uuid Is Valid Batch	✅	1010ms
Event Format.Event Timestamp Is Rfc3339 Batch	✅	1010ms
Event Format.Distinct Id Is String Batch	✅	1009ms
Event Format.Distinct Id At Root Not Properties Batch	✅	1010ms
Event Format.Custom Properties Preserved Batch	✅	1010ms
Event Format.Set Properties Preserved Batch	✅	1009ms
Event Format.Set Once Properties Preserved Batch	✅	1010ms
Event Format.Groups Properties Preserved Batch	✅	1009ms
Event Format.Sdk Generates Uuid If Not Provided Batch	✅	1010ms
Batch Behavior.Multiple Events In Single Batch	✅	1506ms
Batch Behavior.Batch Envelope Smoke	✅	1011ms
Batch Behavior.Flush With No Events Sends Nothing	✅	1004ms
Batch Behavior.Flush At Triggers Batch	✅	1506ms
Batch Behavior.Created At Reflects Batch Creation Time	✅	1009ms
Deduplication.Generates Unique Uuids	✅	1506ms
Deduplication.Different Events Same Content Different Uuids	✅	1506ms
Deduplication.Preserves Uuid On Retry	✅	7514ms
Deduplication.Preserves Timestamp On Retry	✅	7508ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	7511ms
Deduplication.No Duplicate Events In Batch	✅	1503ms
Header Behavior On Retry.Attempt Header Starts At One	✅	1007ms
Header Behavior On Retry.Attempt Header Increments On Retry	✅	14521ms
Header Behavior On Retry.Request Id Preserved On Retry	✅	7506ms
Header Behavior On Retry.Different Requests Have Different Request Ids	✅	3512ms
Header Behavior On Retry.Request Timestamp Changes On Retry	✅	7513ms
Response Format Validation.Success Response Has Uuid Keyed Results	✅	1003ms
Response Format Validation.Success Response Has Ok For Each Event	✅	1505ms
Response Format Validation.Success No Retry After When All Ok	✅	1506ms
Response Format Validation.Success Retry After Present When Retry Events	✅	2509ms
Response Format Validation.Success No Retry After When Drop Only	✅	1506ms
Response Format Validation.Response Echoes Request Id	✅	1007ms
Retry Behavior.Retries On 408	✅	7509ms
Retry Behavior.Retries On 500	✅	7514ms
Retry Behavior.Retries On 503	✅	9513ms
Retry Behavior.Retries On 504	✅	7506ms
Retry Behavior.Retryable Errors Have Retry After	✅	4511ms
Retry Behavior.Respects Retry After On Retryable Error	✅	12509ms
Retry Behavior.Does Not Retry On 400	✅	3509ms
Retry Behavior.Does Not Retry On 401	✅	3506ms
Retry Behavior.Does Not Retry On 402	✅	3505ms
Retry Behavior.Does Not Retry On 413	✅	3507ms
Retry Behavior.Does Not Retry On 415	✅	3507ms
Retry Behavior.Non Retryable Errors Have No Retry After	✅	3506ms
Retry Behavior.Implements Backoff	✅	23524ms
Retry Behavior.Max Retries Respected	✅	23513ms
Partial Batch Handling.Handles 200 Full Success	✅	3008ms
Partial Batch Handling.Handles 200 With All Ok	✅	4509ms
Partial Batch Handling.Does Not Retry Dropped Events	✅	4507ms
Partial Batch Handling.Does Not Retry Limited Events	✅	4508ms
Partial Batch Handling.Prunes Ok Events On Partial Retry	✅	7510ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry	✅	7510ms
Partial Batch Handling.Retries Only Retry Events From Partial	✅	7509ms
Partial Batch Handling.Partial Retry Preserves Uuids	✅	7511ms
Partial Batch Handling.Partial Retry Attempt Header Increments	✅	7510ms
Partial Batch Handling.Partial Retry Request Id Preserved	✅	7510ms
Partial Batch Handling.Respects Retry After On Partial	✅	9506ms
Partial Batch Handling.Unknown Result Treated As Terminal	✅	4509ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry	✅	4506ms
Compression.Sends Gzip Content Encoding	✅	1006ms
Compression.No Content Encoding When Disabled	✅	1007ms
Compression.Compressed Body Is Decompressible	✅	1008ms
Error Handling.Does Not Retry On Unknown 4Xx	✅	3507ms
Event Options.Cookieless Mode Override	✅	1006ms
Event Options.Disable Skew Correction Override	✅	1006ms
Event Options.Process Person Profile Override	✅	1008ms
Event Options.Product Tour Id Override	✅	1007ms
Event Options.Unset Options Omitted	✅	1007ms
Event Options.Options Override In Batch	✅	1010ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties	✅	1006ms
Geoip And Historical Migration.Historical Migration Set In Body	✅	1007ms
Geoip And Historical Migration.Historical Migration Absent By Default	✅	1007ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details

Test	Status	Duration
Request Payload.Request With Person Properties Device Id	✅	1005ms
Request Payload.Flags Request Uses V2 Query Param	✅	1006ms
Request Payload.Flags Request Hits Flags Path Not Decide	✅	1005ms
Request Payload.Flags Request Omits Authorization Header	✅	1006ms
Request Payload.Token In Flags Body Matches Init	✅	1005ms
Request Payload.Groups Round Trip	✅	1006ms
Request Payload.Groups Default To Empty Object	✅	1005ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False	✅	1006ms
Request Payload.Disable Geoip Omitted Defaults To False	✅	1005ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key	✅	1006ms
Request Lifecycle.No Flags Request On Init Alone	✅	502ms
Request Lifecycle.No Flags Request On Normal Capture	✅	1506ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests	✅	1008ms
Request Lifecycle.Mock Response Value Is Returned To Caller	✅	1003ms
Retry Behavior.Retries Flags On 502	✅	1005ms
Retry Behavior.Retries Flags On 504	✅	1006ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event	✅	1507ms