Skip to content

feat(dotnet): enable FFE span-enrichment parametric tests#7152

Draft
leoromanovsky wants to merge 2 commits into
mainfrom
leo.romanovsky/ffe-enrichment-enable-dotnet
Draft

feat(dotnet): enable FFE span-enrichment parametric tests#7152
leoromanovsky wants to merge 2 commits into
mainfrom
leo.romanovsky/ffe-enrichment-enable-dotnet

Conversation

@leoromanovsky

Copy link
Copy Markdown
Contributor

Motivation

Fan out the FFE APM feature-flag span-enrichment contract (shipped in Node.js via
dd-trace-js#8343, frozen) to the .NET server SDK by enabling the existing frozen
parametric suite tests/parametric/test_ffe/test_span_enrichment.py for dotnet. With the
DD_EXPERIMENTAL_FLAGGING_PROVIDER_SPAN_ENRICHMENT_ENABLED gate on, each root APM span carries
ffe_flags_enc / ffe_subjects_enc / ffe_runtime_defaults, encoded identically across SDKs so
backend/Trino decode and these shared assertions stay in parity.

SDK PR: DataDog/dd-trace-dotnet#8795.

Changes

.NET-scoped only:

  • manifests/dotnet.yml: flip tests/parametric/test_ffe/test_span_enrichment.py from
    missing_feature to '>=3.36.0' so the suite runs and must pass.
  • utils/build/docker/dotnet/parametric/Endpoints/ApmTestApi.cs: add /ffe/start and
    /ffe/evaluate. /ffe/evaluate re-activates the caller-supplied root span
    (StartActive("ffe.evaluate", { Parent = storedSpan.Context, FinishOnClose = true })) so the
    OpenFeature Finally hook accumulates onto the test's real root span — multiple evaluations
    aggregate serial IDs onto one root, and child-span evaluations propagate to the root
    (chunk-level tagging). The transient eval span finishes on scope dispose so the trace flushes.
  • utils/build/docker/dotnet/parametric/ApmTestApi.csproj: reference OpenFeature 2.3.0 +
    Datadog.FeatureFlags.OpenFeature 2.3.0.
  • utils/build/docker/dotnet/parametric/Dockerfile + nuget.config: ship the enrichment-aware
    Datadog.FeatureFlags.OpenFeature 2.3.0 nupkg in the build context and order the local nuget
    source first.

Decisions

  • Why the local nupkg: the nuget.org-published Datadog.FeatureFlags.OpenFeature 2.3.0
    predates span enrichment — it has neither SpanEnrichmentHook nor the
    FeatureFlagsSdk.AccumulateSpanEnrichment CallTarget stub — so restoring it builds a parametric
    app that attaches no ffe_* tags (every assertion fails with zero tags). The enrichment-aware
    build is the same 2.3.0 version, so the local nuget source is ordered first to win the
    version-tie. Once an enrichment-bearing Datadog.FeatureFlags.OpenFeature is published to
    nuget.org, the shipped nupkg + local-source ordering can be dropped.
  • Span re-activation, not a new trace: an earlier shape forked a fresh trace per evaluate,
    landing enrichment on a throwaway root; Parent = storedSpan.Context ensures aggregation onto the
    test-created root.
  • Runtime tracer: the ffe_* write happens in the auto-instrumentation tracer
    (datadog-dotnet-apm-3.46.0) via CallTarget, so the compile-time Datadog.Trace reference stays
    the published *.

Results

Validated locally on datadog-dotnet-apm-3.46.0 (arm64):

TEST_LIBRARY=dotnet ./run.sh PARAMETRIC -k span_enrichment
Library: dotnet@3.46.0
tests/parametric/test_ffe/test_span_enrichment.py ..................   [100%]
===================== 18 passed, 2561 deselected in 57.27s =====================

Enable tests/parametric/test_ffe/test_span_enrichment.py for .NET (flip
manifest from missing_feature to >=3.36.0) and wire the parametric app to
the Datadog OpenFeature provider:

- ApmTestApi.cs: add /ffe/start + /ffe/evaluate; /ffe/evaluate re-activates
  the caller-supplied root span (Parent = stored span context) so multiple
  evaluations aggregate ffe_* tags onto the test's real root span and child
  spans propagate to it.
- ApmTestApi.csproj: reference OpenFeature 2.3.0 + Datadog.FeatureFlags.OpenFeature 2.3.0.
- Ship the enrichment-aware Datadog.FeatureFlags.OpenFeature 2.3.0 nupkg in
  the build context and order the local nuget source first; the nuget.org
  2.3.0 predates span enrichment and would attach no ffe_* tags.

Validated locally: TEST_LIBRARY=dotnet ./run.sh PARAMETRIC -k span_enrichment
=> 18 passed (dotnet@3.46.0).
@github-actions

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

utils/build/docker/dotnet/parametric/Datadog.FeatureFlags.OpenFeature.2.3.0.nupkg  @DataDog/apm-dotnet @DataDog/asm-dotnet @DataDog/system-tests-core
manifests/dotnet.yml                                                    @DataDog/apm-dotnet @DataDog/asm-dotnet
utils/build/docker/dotnet/parametric/ApmTestApi.csproj                  @DataDog/apm-dotnet @DataDog/asm-dotnet @DataDog/system-tests-core
utils/build/docker/dotnet/parametric/Dockerfile                         @DataDog/apm-dotnet @DataDog/asm-dotnet @DataDog/system-tests-core
utils/build/docker/dotnet/parametric/Endpoints/ApmTestApi.cs            @DataDog/apm-dotnet @DataDog/asm-dotnet @DataDog/system-tests-core
utils/build/docker/dotnet/parametric/nuget.config                       @DataDog/apm-dotnet @DataDog/asm-dotnet @DataDog/system-tests-core

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jun 16, 2026

Copy link
Copy Markdown

Pipelines  Tests

Fix all issues with BitsAI

⚠️ Warnings

🚦 4 Pipeline jobs failed

Testing the test | System Tests (dotnet, dev) / parametric / parametric (2)   View in Datadog   GitHub Actions

🧪 10 Tests failed

tests.parametric.test_ffe.test_span_enrichment.Test_Span_Enrichment_Child_Span_Propagation.test_child_span_flag_evaluation_propagates_to_root[library_env0, parametric-dotnet] from system_tests_suite   View in Datadog
AssertionError: ffe_flags_enc not found in root span meta: ['runtime-id', 'language', '_dd.base_service', '_dd.svc_src']
assert 'ffe_flags_enc' in {'_dd.base_service': 'ApmTestApi', '_dd.svc_src': 'm', 'language': 'dotnet', 'runtime-id': '3cc6a8b3-c61a-4ba2-886a-7444af78e05c'}

self = <tests.parametric.test_ffe.test_span_enrichment.Test_Span_Enrichment_Child_Span_Propagation object at 0x7f4ee290a690>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f4ead689580>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f4eadd82600>

    @parametrize("library_env", [{**DEFAULT_ENVVARS}])
    def test_child_span_flag_evaluation_propagates_to_root(
        self, test_agent: TestAgentAPI, test_library: APMLibrary
...
tests.parametric.test_ffe.test_span_enrichment.Test_Span_Enrichment_Default_Fallback.test_ffe_runtime_defaults_value_truncated_at_64_chars[library_env0, parametric-dotnet] from system_tests_suite   View in Datadog
AssertionError: ffe_runtime_defaults not found in span meta: ['runtime-id', 'language', '_dd.base_service', '_dd.svc_src']
assert 'ffe_runtime_defaults' in {'_dd.base_service': 'ApmTestApi', '_dd.svc_src': 'm', 'language': 'dotnet', 'runtime-id': '39a1aa06-6c14-44fc-8c32-006b66e22e29'}

self = <tests.parametric.test_ffe.test_span_enrichment.Test_Span_Enrichment_Default_Fallback object at 0x7fdd59d7a420>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7fdd282d7e30>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7fdd278a18e0>

    @parametrize("library_env", [{**DEFAULT_ENVVARS}])
    def test_ffe_runtime_defaults_value_truncated_at_64_chars(
        self, test_agent: TestAgentAPI, test_library: APMLibrary
...
View all 10 test failures

Testing the test | System Tests (dotnet, prod) / parametric / parametric (1)   View in Datadog   GitHub Actions

🧪 10 Tests failed

tests.parametric.test_ffe.test_span_enrichment.Test_Span_Enrichment_Child_Span_Propagation.test_child_span_flag_evaluation_propagates_to_root[library_env0, parametric-dotnet] from system_tests_suite   View in Datadog
AssertionError: ffe_flags_enc not found in root span meta: ['runtime-id', 'language', '_dd.base_service', '_dd.svc_src']
assert 'ffe_flags_enc' in {'_dd.base_service': 'ApmTestApi', '_dd.svc_src': 'm', 'language': 'dotnet', 'runtime-id': 'e22608ae-b848-4416-87cf-85b4b462ddf9'}

self = <tests.parametric.test_ffe.test_span_enrichment.Test_Span_Enrichment_Child_Span_Propagation object at 0x7f448bf7a150>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f448ba65a30>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f448badd0d0>

    @parametrize("library_env", [{**DEFAULT_ENVVARS}])
    def test_child_span_flag_evaluation_propagates_to_root(
        self, test_agent: TestAgentAPI, test_library: APMLibrary
...
tests.parametric.test_ffe.test_span_enrichment.Test_Span_Enrichment_Default_Fallback.test_ffe_runtime_defaults_value_truncated_at_64_chars[library_env0, parametric-dotnet] from system_tests_suite   View in Datadog
AssertionError: ffe_runtime_defaults not found in span meta: ['runtime-id', 'language', '_dd.base_service', '_dd.svc_src']
assert 'ffe_runtime_defaults' in {'_dd.base_service': 'ApmTestApi', '_dd.svc_src': 'm', 'language': 'dotnet', 'runtime-id': 'e3cd67b4-6f48-4640-b254-4f3bce58e054'}

self = <tests.parametric.test_ffe.test_span_enrichment.Test_Span_Enrichment_Default_Fallback object at 0x7f16c0b4de50>
test_agent = <utils.docker_fixtures._test_agent.TestAgentAPI object at 0x7f1690734140>
test_library = <utils.docker_fixtures._test_clients._test_client_parametric.ParametricTestClientApi object at 0x7f1690aa65d0>

    @parametrize("library_env", [{**DEFAULT_ENVVARS}])
    def test_ffe_runtime_defaults_value_truncated_at_64_chars(
        self, test_agent: TestAgentAPI, test_library: APMLibrary
...
View all 10 test failures

DataDog/system-tests | compute_pipeline   View in Datadog   GitLab

View all 4 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 952cbf7 | Docs | Datadog PR Page | Give us feedback!

POST /ffe/evaluate re-activates the caller's root span by starting a transient
"ffe.evaluate" child scope. It was created with FinishOnClose=false, so disposing the
scope never finished that child; the trace kept a pending open span, the tracer never
flushed it, and the test agent received zero traces (all 13 span-enrichment cases failed
with "Number (1) of traces not available"). Finish the transient child on dispose — the
stored root is unaffected and is still closed later by /trace/span/finish.

Also refresh the staged Datadog.FeatureFlags.OpenFeature 2.3.0 nupkg to the build carrying
the object/structure runtime-default fix (dd-trace-dotnet#8795) so the parametric client
matches the SDK under test.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant