Skip to content

fix(sandbox): undo tmpfs overlay when setns fails during SPIFFE mount namespace setup#2184

Open
zanetworker wants to merge 1 commit into
NVIDIA:mainfrom
zanetworker:fix/spiffe-mount-namespace-setns-eperm
Open

fix(sandbox): undo tmpfs overlay when setns fails during SPIFFE mount namespace setup#2184
zanetworker wants to merge 1 commit into
NVIDIA:mainfrom
zanetworker:fix/spiffe-mount-namespace-setns-eperm

Conversation

@zanetworker

Copy link
Copy Markdown
Contributor

Summary

  • When setns(CLONE_NEWNS) fails with EPERM (common on OpenShift and hardened K8s without CAP_SYS_ADMIN), create_supervisor_identity_mount_namespace leaves an empty tmpfs mounted over the SPIFFE workload API socket directory. Since PID 1 is stuck in the new mount namespace, the socket becomes invisible to all processes in the container, even though the CSI volume is healthy.
  • This fix calls umount2(MNT_DETACH) in the setns() error path to remove the tmpfs overlay before returning, keeping the SPIFFE socket accessible.
  • Also wires up the prepare_supervisor_identity_mount_namespace_from_env call in main.rs and makes it gracefully degrade (warn + continue) instead of aborting the supervisor on error.

Related Issue

N/A (discovered during SPIFFE + token grant integration testing on OpenShift)

Changes

  • crates/openshell-supervisor-process/src/process.rs:
    • create_supervisor_identity_mount_namespace: on setns() failure, umount2(target, MNT_DETACH) before returning error
    • prepare_supervisor_identity_mount_namespace_from_env: catch errors, log warning, set namespace to None, continue
  • crates/openshell-sandbox/src/main.rs: call prepare_supervisor_identity_mount_namespace_from_env() before tracing setup

Testing

  • Built patched supervisor binary (static musl, cross-compiled from macOS via cargo-zigbuild)
  • Deployed as custom supervisor image on OpenShift 4.x cluster with SPIRE CSI driver
  • Confirmed supervisor logs show: "undid tmpfs overlay so SPIFFE socket remains accessible"
  • Confirmed /spiffe-workload-api/spire-agent.sock is visible inside sandbox container
  • Confirmed sandbox proxy allows traffic to provider endpoints (401 from protected service, not 403 policy_denied)
  • All existing tests pass (cargo test --workspace)

Checklist

  • Conventional commit format
  • cargo fmt clean
  • cargo clippy clean
  • All tests pass
  • Signed off (DCO)

… namespace setup

On OpenShift and other container runtimes that do not grant
CAP_SYS_ADMIN, setns(CLONE_NEWNS) fails with EPERM after
create_supervisor_identity_mount_namespace has already mounted
an empty tmpfs over the SPIFFE workload API socket directory.
PID 1 remains stuck in the new mount namespace where the tmpfs
hides the socket, making it invisible to all processes.

Fix: when setns() fails, call umount2(MNT_DETACH) to remove
the tmpfs overlay before returning the error, so the SPIFFE
socket stays accessible even without namespace isolation.

Also wire up prepare_supervisor_identity_mount_namespace_from_env
in main.rs and make it gracefully degrade on error instead of
aborting the supervisor.

Signed-off-by: Adel Zaalouk <azaalouk@redhat.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 8, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@TaylorMutch

Copy link
Copy Markdown
Collaborator

/ok to test bec39d9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants