velero: add container-based config discovery support

[vitkyrka]

What does this PR do?

Adds container-based config discovery support to velero.

Since this is the first integration which uses Kind for its e2e tests gaining config discovery support, we also add support for config discovery E2E testing with Kind.

The new datadog_checks_dev/datadog_checks/dev/kube_discovery.py module provides Kubernetes Autodiscovery e2e helpers, similar in purpose to the Docker discovery helpers.

Kubernetes discovery cannot use the normal dd_agent_check() flow directly. That fixture is useful when the test can run a check from the test host against a static config, but Kubernetes Autodiscovery depends on the Agent running inside the cluster: it needs Kubernetes RBAC, kubelet pod/node metadata, the mounted auto_conf.yaml, and network access from an Agent pod to workload pod IPs.

The helper therefore sets up a lightweight in-cluster Agent target:

• setup_discovery_agent deploys a sleeping, RBAC-scoped Agent pod in the kind cluster.
• The pod gets the minimum ClusterRole needed to resolve node and pod data, an IPC secret, and the integration's auto_conf.yaml mounted under conf.d/<check>.d.
• The local integration checkout is installed into the pod's embedded Python so the e2e test exercises the code from this PR.

Tests then run one-shot commands inside that pod with kubectl exec instead of starting a long-lived Agent:

• run_discovery_check_kubernetes is the Kubernetes equivalent of dd_agent_check_discovery.
• assert_all_discovery_candidates_stable_kubernetes is the Kubernetes equivalent of assert_all_discovery_candidates_stable: it copies each generated candidate config into the Agent pod and runs agent check --config-file there. That also verifies the generated config works from inside the cluster, where the Agent reaches the workload by pod IP.

This PR also adds a new discovery strategy that will be re-used by other Kubernetes integrations which will get the metrics ports from named ports instead of having to probe all ports of the container.

Motivation

https://datadoghq.atlassian.net/browse/DSCVR-524.

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
• If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

[datadog-datadog-prod-us1-2]

 

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 95.93%
• Overall Coverage: 88.47% (+0.13%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: e183434 | Docs | Datadog PR Page | Give us feedback!}

[vitkyrka]

• velero: add container-based config discovery support #24507 : 6 dependent PRs (#24459 , #24506 , #24512 and 3 others) 👈 (View in Graphite)
• Add discovery port name helper #24555
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ff084545e2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8daa754b16

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Bump Velero's base dependency for named-port discovery

When Velero is installed outside this source tree, the resolver may satisfy this with datadog-checks-base 37.41.0 or 37.42.0, but the generated discovery module imports candidate_ports_by_name (velero/datadog_checks/velero/config_models/discovery.py:15), which was added by the unreleased base changelog entry in the parent commit. In that environment Autodiscovery fails with ImportError before producing any Velero configs; require the first base release that contains the named-port helper.

Useful? React with 👍 / 👎.

[vitkyrka]

Not sure how to handle this, the datadog-checks-base with the helper is unreleased as of writing.

[chatgpt-codex-connector]

Declare cryptography for the new dev helper

When setup_discovery_agent() runs from a clean datadog-checks-dev test/install environment, this function imports cryptography, but datadog_checks_dev/pyproject.toml does not declare that dependency and the ddev hatch collector skips installing datadog-checks-base for the dev package. That makes the new datadog_checks_dev tests and any standalone use of the helper fail with ModuleNotFoundError before Kubernetes setup starts; add the dependency or avoid requiring it here.

Useful? React with 👍 / 👎.

[vitkyrka]

[Claude] Checked this: datadog_checks_dev's own hatch test env does install datadog-checks-base[deps] (which pins cryptography==48.0.1), so cryptography is available there. The collector's is_dev_package check (in ddev/src/ddev/plugin/external/hatch/environment_collector.py) refers to the ddev CLI package itself, not datadog_checks_dev — the latter is tracked separately as is_test_package and isn't excluded from the base-package install. Ran ddev --no-interactive test datadog_checks_dev -- -k test_kube_discovery to confirm: all 30 tests pass, including the two that call through setup_discovery_agent() → generate_ipc_cert_pem() → from cryptography import x509 unmocked. So no ModuleNotFoundError in the supported ddev-managed environment; leaving this as-is.

Review from joepeeples is dismissed. Related teams and files:

• documentation
  • velero/assets/configuration/spec.yaml
  • velero/datadog_checks/velero/data/auto_conf.yaml

[dd-octo-sts]

Validation Report

All 21 validations passed.

Show details

Validation	Description	Status
agent-reqs	Verify check versions match the Agent requirements file	✅
ci	Validate CI configuration and code coverage settings	✅
codeowners	Validate every integration has a CODEOWNERS entry	✅
config	Validate default configuration files against spec.yaml	✅
dep	Verify dependency pins are consistent and Agent-compatible	✅
http	Validate integrations use the HTTP wrapper correctly	✅
imports	Validate check imports do not use deprecated modules	✅
integration-style	Validate check code style conventions	✅
jmx-metrics	Validate JMX metrics definition files and config	✅
labeler	Validate PR labeler config matches integration directories	✅
legacy-signature	Validate no integration uses the legacy Agent check signature	✅
license-headers	Validate Python files have proper license headers	✅
licenses	Validate third-party license attribution list	✅
metadata	Validate metadata.csv metric definitions	✅
models	Validate configuration data models match spec.yaml	✅
openmetrics	Validate OpenMetrics integrations disable the metric limit	✅
package	Validate Python package metadata and naming	✅
qa-label	Validate the pull request declares whether it needs QA for the next Agent release	✅
readmes	Validate README files have required sections	✅
saved-views	Validate saved view JSON file structure and fields	✅
version	Validate version consistency between package and changelog	✅

View full run