Add Config json schema generation by NimrodAvni78 · Pull Request #1017 · open-telemetry/opentelemetry-ebpf-instrumentation

NimrodAvni78 · 2025-12-16T13:49:41Z

Fixes #973
Resolve #53

Automatically generates json schema for config struct, with enum values, deprecated warnings and comments as descriptions

also validates in CI that json schema is the same

codecov · 2025-12-16T13:59:54Z

Codecov Report

❌ Patch coverage is 0% with 145 lines in your changes missing coverage. Please review.
✅ Project coverage is 1.92%. Comparing base (ede9606) to head (c2ba4db).

Files with missing lines	Patch %	Lines
pkg/config/ebpf_tracer.go	0.00%	26 Missing ⚠️
pkg/appolly/services/attr_glob.go	0.00%	19 Missing ⚠️
pkg/appolly/services/attr_regex.go	0.00%	19 Missing ⚠️
pkg/obi/network_cfg.go	0.00%	15 Missing ⚠️
pkg/appolly/services/export.go	0.00%	14 Missing ⚠️
pkg/export/feature.go	0.00%	14 Missing ⚠️
pkg/obi/config.go	0.00%	14 Missing ⚠️
pkg/appolly/services/criteria.go	0.00%	7 Missing ⚠️
pkg/appolly/services/samplerconfig.go	0.00%	6 Missing ⚠️
pkg/export/otel/metrics.go	0.00%	3 Missing ⚠️
... and 5 more

❗ There is a different number of reports uploaded between BASE (ede9606) and HEAD (c2ba4db). Click for more details.

HEAD has 8 uploads less than BASE

Flag BASE (ede9606) HEAD (c2ba4db)

unittests 1 0

integration-test 9 2

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #1017       +/-   ##
==========================================
- Coverage   43.52%   1.92%   -41.61%     
==========================================
  Files         305     240       -65     
  Lines       32864   28168     -4696     
==========================================
- Hits        14305     542    -13763     
- Misses      17639   27571     +9932     
+ Partials      920      55      -865

Flag	Coverage Δ
integration-test	`0.00% <0.00%> (-21.69%)`	⬇️
integration-test-arm	`0.00% <0.00%> (ø)`
k8s-integration-test	`2.35% <0.00%> (-0.02%)`	⬇️
oats-test	`0.00% <0.00%> (ø)`
unittests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

mariomac

Good initiative!

mariomac · 2025-12-16T15:19:41Z

+	ContextPropagationDisabledText  = "disabled"
+	ContextPropagationAllText       = "all"
+	ContextPropagationHeadersText   = "headers"
+	ContextPropagationHTTPText      = "http"
+	ContextPropagationTCPText       = "tcp"
+	ContextPropagationIPOptionsText = "ip"


It might be a bit confusing that some "ContextPropagation..." variables are integers and others are strings.

I'd rename these variables to StrContextPropagation, for example.

Also, make them private if they are going to be only used here.

+1 to making private.

marctc · 2025-12-16T16:38:19Z

@@ -0,0 +1,363 @@
+// Copyright The OpenTelemetry Authors


I wonder if this belongs to this repo, as it looks pretty generic. Are there any project ins the open-telemetry ecosystem doing that? Maybe its worth to move this somewhere later if that can be reused?

i couldn't find it in projects like otel-collector, maybe we can make it something that can be reused
im not familiar with how other otel projects manage schema.
maybe @MrAlias you have more insight?

itsCheithanya · 2025-12-17T13:26:31Z

@NimrodAvni78 Looks great! I see the generated schema does list the accepted values for fields that are validated against a specific set of allowed values when those are defined as constants or enums, but it does not do so when the validation is performed via a method on a struct.

For example, Discovery.Instrument.Metadata should only accept values like:

k8s_namespace
k8s_pod_name
k8s_deployment_name
k8s_replicaset_name
k8s_daemonset_name
k8s_statefulset_name
k8s_job_name
k8s_cronjob_name
k8s_owner_name
k8s_container_name

(as defined in attr_regex.go (https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/blob/v0.3.0/pkg/appolly/services/attr_regex.go#L19) ), but the schema does not currently reflect this.

Similarly, if a struct field is validated by a method (for example, a custom Validate() or IsValid() method that restricts accepted values), it would be very helpful if the schema could include those accepted values (e.g., as an enum or with a description), so users and tooling know what is valid.

NimrodAvni78 · 2025-12-17T14:40:45Z

@itsCheithanya yeah you are right, generally the more we can embed into the typing of the config we can generate it automatically, and there are other types here which still missing their valid options in the schema. alternatively we can also add some manual implementation of JsonSchema for the target type and provided the schema ourself (like i did with ContextPropagationMode.

this can also be improved as we go, since refactoring big parts of the config is a lot of work

for this example of attr_regex i will try to see which approach works best and add it

itsCheithanya · 2025-12-17T18:25:40Z

@NimrodAvni78 https://github.com/itsCheithanya/opentelemetry-ebpf-instrumentation/blob/f5edac2114cad288ee315019ddee3e15a62ed677/pkg/appolly/services/attr_glob.go check this if it can be be of any help

NimrodAvni78 · 2025-12-18T10:48:55Z

thanks @itsCheithanya
i did something really similar and didnt notice this comment :)
i think this now describes the metadata better, also added some other improvments like adding the environment variable matching the config + adding annotations to other fields

NimrodAvni78 · 2025-12-18T11:10:14Z

also noticed the field is actually inlined and not under another field called metadata, so i also fixed that

MrAlias · 2026-01-28T23:21:02Z

This is missing tests demonstrating schema generation correctness.

will add some testing

NimrodAvni78 · 2026-01-29T09:30:05Z

if a struct field is validated by a method (for example, a custom Validate() or IsValid() method that restricts accepted values), it would be very helpful if the schema could include those accepted values (e.g., as an enum or with a description), so users and tooling know what is valid.

+1 to this.

Since validate is a function with custom logic its sometimes really hard to automatically infer from it what are the accepted values, i did some manual work on some complex structs that added custom JsonSchema implementation to reflect the validation process, but unfortunately i dont think we can do some mapping automatically. ideally what we need is to model our typing to only accept valid values (tried to do for most types), so then validation and schema will be aligned.

…son-schema # Conflicts: # go.mod # go.sum

antonjim-te

some comments/questions :)

antonjim-te · 2026-02-09T08:28:26Z

+        }
+      ],
+      "title": "Agent Type Interface",
+      "description": "Specifies which interface should the agent pick the IP address from in order to report it in the AgentIP field on each flow. Accepted values are: external, local, or name:\u003cinterface name\u003e (e.g. name:eth0)."


the generated JSON currently escapes < / > in descriptions (e.g., name:\u003cinterface name\u003e).

Can we try to avoid HTML escaping? or were they expected?

antonjim-te · 2026-02-09T08:33:57Z

+              "tcp"
+            ]
+          },
+          "type": "array",


it is not parsed as an array, it is parsed a comma-separated

parts := strings.Split(str, ",")

is something wrong here?

antonjim-te · 2026-02-09T08:58:17Z


 	// Convenience aliases
-	ContextPropagationAll = ContextPropagationHeaders | ContextPropagationTCP
+	ContextPropagationAll         = ContextPropagationHeaders | ContextPropagationTCP


is it expected that All does not contain IP in the description it says "All contains all the methods"

yes its intentional to be removed by default
see #1001

…son-schema # Conflicts: # Makefile # pkg/export/otel/otelcfg/config_metrics.go # pkg/export/otel/otelcfg/config_traces.go

codecov · 2026-02-17T16:33:47Z

Codecov Report

❌ Patch coverage is 51.32924% with 238 lines in your changes missing coverage. Please review.
✅ Project coverage is 43.72%. Comparing base (155505c) to head (68cd9c5).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
cmd/obi-schema/main.go	62.79%	111 Missing and 17 partials ⚠️
pkg/config/ebpf_tracer.go	19.23%	21 Missing ⚠️
pkg/obi/network_cfg.go	0.00%	15 Missing ⚠️
pkg/appolly/services/export.go	0.00%	14 Missing ⚠️
pkg/export/feature.go	0.00%	14 Missing ⚠️
pkg/obi/config.go	0.00%	14 Missing ⚠️
pkg/appolly/services/attr_glob.go	52.63%	9 Missing ⚠️
pkg/appolly/services/attr_regex.go	52.63%	9 Missing ⚠️
pkg/appolly/services/criteria.go	0.00%	7 Missing ⚠️
pkg/internal/transform/route/harvest/harvester.go	33.33%	2 Missing ⚠️
... and 5 more

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1017       +/-   ##
===========================================
+ Coverage   19.54%   43.72%   +24.18%     
===========================================
  Files         242      308       +66     
  Lines       28138    33426     +5288     
===========================================
+ Hits         5499    14615     +9116     
+ Misses      21987    17872     -4115     
- Partials      652      939      +287

Flag	Coverage Δ
integration-test	`21.78% <1.60%> (+0.07%)`	⬆️
integration-test-arm	`0.00% <0.00%> (ø)`
integration-test-vm-x86_64-5.15.152	`?`
integration-test-vm-x86_64-6.10.6	`?`
k8s-integration-test	`2.34% <0.00%> (-0.02%)`	⬇️
oats-test	`0.00% <0.00%> (ø)`
unittests	`44.55% <53.30%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…son-schema

…son-schema # Conflicts: # pkg/appolly/services/criteria.go

NimrodAvni78 added 8 commits December 15, 2025 18:13

config schema generation

fc4f4da

add ci validation and make command

3c6e393

update schema

788ea8a

remove default annotation referencesa

1cdb8bc

fmt

5a884f1

add deperecated annotation

1c745e8

remove beyla related schema description

f4658a5

PR changes

4427cfd

NimrodAvni78 requested a review from a team as a code owner December 16, 2025 13:49

NimrodAvni78 mentioned this pull request Dec 16, 2025

added config template that OBI accepts #974

Closed

1 task

mariomac reviewed Dec 16, 2025

View reviewed changes

marctc reviewed Dec 16, 2025

View reviewed changes

NimrodAvni78 added 4 commits December 18, 2025 11:55

fix metadata map and regex + glob

8a393a1

take field name from yaml annotation instead of json

e4caca4

add x-env-var to specify environment variable from annotation

995cd09

network config json schema

0a6d9d8

NimrodAvni78 added 2 commits December 18, 2025 12:59

fix naming to StrContextPropagation

d5da03d

support inline

404cce3

NimrodAvni78 added 6 commits December 18, 2025 14:13

support port enum

9d82ef5

sampler name options

c56ca4a

more jsonschema annotations

134d1d1

fix AgentIpFace errors

6df86dd

even more annotations

1116924

fmt

612fd82

MrAlias reviewed Jan 28, 2026

View reviewed changes

Comment thread docs/config-schema.json

MrAlias reviewed Jan 28, 2026

View reviewed changes

NimrodAvni78 added 2 commits January 31, 2026 19:02

add more schema types

20271cc

improve schema generation + tests

53c606c

NimrodAvni78 force-pushed the nimrodavni78/config-json-schema branch 2 times, most recently from 841da66 to 53c606c Compare February 1, 2026 10:54

update notices

d1d4366

NimrodAvni78 force-pushed the nimrodavni78/config-json-schema branch from 70a8add to d1d4366 Compare February 1, 2026 12:25

update notices

4d2f0d8

NimrodAvni78 force-pushed the nimrodavni78/config-json-schema branch from 6a87721 to 4d2f0d8 Compare February 2, 2026 12:10

NimrodAvni78 requested a review from MrAlias February 2, 2026 13:23

NimrodAvni78 added 2 commits February 8, 2026 10:58

Merge remote-tracking branch 'origin/main' into nimrodavni78/config-j…

7a4c35a

…son-schema # Conflicts: # go.mod # go.sum

generate schema

4adeb9d

antonjim-te approved these changes Feb 9, 2026

View reviewed changes

NimrodAvni78 added 3 commits February 17, 2026 13:02

PR fixes

071ebe5

Merge remote-tracking branch 'origin/main' into nimrodavni78/config-j…

27dacd7

…son-schema # Conflicts: # Makefile # pkg/export/otel/otelcfg/config_metrics.go # pkg/export/otel/otelcfg/config_traces.go

fix misspell

c2ba4db

generate schema

ae83662

MrAlias approved these changes Feb 18, 2026

View reviewed changes

NimrodAvni78 added 5 commits February 19, 2026 14:06

Merge remote-tracking branch 'origin/main' into nimrodavni78/config-j…

0b8051a

…son-schema

add pre commit hook

a156b14

fix comments

f5fba93

Merge remote-tracking branch 'origin/main' into nimrodavni78/config-j…

eb94527

…son-schema # Conflicts: # pkg/appolly/services/criteria.go

remove newlines from description

68cd9c5

MrAlias merged commit b905642 into open-telemetry:main Feb 20, 2026
52 checks passed

MrAlias added this to the v0.6.0 milestone Feb 23, 2026

MrAlias mentioned this pull request Mar 5, 2026

Release v0.6.0 #1478

Merged

MrAlias mentioned this pull request Mar 16, 2026

Use Go configuration validator/reader from JSON Schema #280

Closed

Conversation

NimrodAvni78 commented Dec 16, 2025 • edited by MrAlias Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mariomac left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

itsCheithanya commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NimrodAvni78 commented Dec 17, 2025

Uh oh!

itsCheithanya commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NimrodAvni78 commented Dec 18, 2025

Uh oh!

NimrodAvni78 commented Dec 18, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NimrodAvni78 commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

antonjim-te left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

NimrodAvni78 commented Dec 16, 2025 •

edited by MrAlias

Loading

codecov Bot commented Dec 16, 2025 •

edited

Loading

itsCheithanya commented Dec 17, 2025 •

edited

Loading

itsCheithanya commented Dec 17, 2025 •

edited

Loading

NimrodAvni78 commented Jan 29, 2026 •

edited

Loading

codecov Bot commented Feb 17, 2026 •

edited

Loading