Skip to content

filter_kubernetes: Plug SEGV occurence on enabling Use_Pod_Association#12153

Open
cosmo0920 wants to merge 3 commits into
masterfrom
cosmo0920-plug-segv-on-k8s-filter
Open

filter_kubernetes: Plug SEGV occurence on enabling Use_Pod_Association#12153
cosmo0920 wants to merge 3 commits into
masterfrom
cosmo0920-plug-segv-on-k8s-filter

Conversation

@cosmo0920

@cosmo0920 cosmo0920 commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

The Kubernetes filter now:

  • Keeps refresh thread, mutex, condition variable, and event loop state per filter instance in kube_conf.h.
  • Uses flb_time_get() for portable refresh deadlines in kubernetes.c.
  • Gracefully signals and joins each worker before destroying its TLS/upstream context in kubernetes.c.

This prevents two instances from sharing an overwritten context and concurrently destroying the same TLS state.

Verification:

  • Focused regression test: passed.
  • Regression test repeated 10 times: all passed.
  • ctest --test-dir build -R '^flb-rt-filter_kubernetes$' --output-on-failure: passed in 119.90 seconds.
  • Valgrind was not run because it is unavailable on this Windows host.
  • Working tree is clean.

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • Bug Fixes
    • Improved Kubernetes pod-to-service association refresh by using a shutdown-aware wait loop (avoids hard sleeps).
    • Added graceful shutdown for background pod-association updates, including proper signaling, thread join, and cleanup.
    • Improved reliability and isolation when multiple Kubernetes filter instances run concurrently.
  • Tests
    • Added a runtime test covering multiple Kubernetes filter instances with shared pod-association settings, ensuring they can start and stop cleanly.

Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 77d746d2-d523-4591-954d-d6bb8a284d4c

📥 Commits

Reviewing files that changed from the base of the PR and between 8067921 and e318fdb.

📒 Files selected for processing (1)
  • plugins/filter_kubernetes/kubernetes.c
🚧 Files skipped from review as they are similar to previous changes (1)
  • plugins/filter_kubernetes/kubernetes.c

📝 Walkthrough

Walkthrough

The Kubernetes filter moves AWS pod-to-service association synchronization into each filter context, adds shutdown-aware timed refreshes, and joins workers during exit. A runtime test now exercises two pod-association filter instances sharing a cache directory.

Changes

Pod association worker lifecycle

Layer / File(s) Summary
Context synchronization contract
plugins/filter_kubernetes/kube_conf.h
Adds pthread support and stores mutex, condition variable, worker, lifecycle flags, shutdown state, and event-loop state in struct flb_kube.
Worker refresh and startup
plugins/filter_kubernetes/kubernetes.c
Reworks pod-service refresh to use context-owned synchronization and timed waits, and initializes and starts the worker per filter context.
Graceful shutdown and multi-instance validation
plugins/filter_kubernetes/kubernetes.c, tests/runtime/filter_kubernetes.c
Signals and joins the worker before cleanup, and adds a runtime test covering two Kubernetes filter instances with pod association enabled.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant FilterInit
  participant PodServiceWorker
  participant PodServiceMap
  participant FilterExit
  FilterInit->>PodServiceWorker: create worker with filter context
  PodServiceWorker->>PodServiceMap: fetch pod-service map
  PodServiceWorker->>PodServiceWorker: wait on refresh condition
  FilterExit->>PodServiceWorker: signal shutdown
  PodServiceWorker-->>FilterExit: join worker
Loading

Possibly related PRs

Suggested reviewers: edsiper

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main change: fixing a segmentation fault when Use_Pod_Association is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch cosmo0920-plug-segv-on-k8s-filter

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/filter_kubernetes/kubernetes.c`:
- Around line 281-294: Update the mutex and condition initialization failure
paths in the filter initialization function to call flb_kube_conf_destroy(ctx)
before returning -1, matching the existing regex-init cleanup path; retain mutex
destruction before cleanup on condition initialization failure.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 1c125796-822c-4277-afb2-a440d9191399

📥 Commits

Reviewing files that changed from the base of the PR and between cb7256c and 8067921.

⛔ Files ignored due to path filters (1)
  • tests/runtime/data/kubernetes/pod-service.map is excluded by !**/*.map
📒 Files selected for processing (3)
  • plugins/filter_kubernetes/kube_conf.h
  • plugins/filter_kubernetes/kubernetes.c
  • tests/runtime/filter_kubernetes.c

Comment thread plugins/filter_kubernetes/kubernetes.c
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant