shim: fetch sandbox stats without subprocesses

[laotoutou]

Background

The containerd shim implements each Runsc.Stats request by starting a new runsc events --stats process. Kubernetes CRI stats collection calls this path once per sandbox container. On dense nodes, a single ListPodSandboxStats request therefore creates hundreds of short-lived processes, and recurring kubelet and observability scrapes turn that into sustained fork/exec and context-switch overhead even when the sandboxes are idle.

In a controlled deployment with 443 gVisor sandboxes on an 8-vCPU node, the existing path averaged 72.80% host CPU and repeatedly saturated the node. Client-side pacing reduced peak concurrency, but it did not reduce the number of stats subprocesses and made steady CPU worse. This makes the subprocess boundary, rather than request fan-out alone, the useful optimization point.

Related production investigation: sandbox0-ai/sandbox0#704

How it works

This change keeps the existing shim and CRI contracts, but makes Runsc.Stats use the sandbox control RPC directly:

• Read the existing runsc container state while holding its advisory state lock.
• Resolve and cache immutable sandbox-lifetime metadata: the control socket and host cgroup.
• Connect to the sandbox control server and call containerManager.Event, the same RPC used by runsc events --stats.
• Apply the host-cgroup CPU correction used by runsc's existing container event path.
• Invalidate cached metadata after a direct-path failure.
• Fall back to the current runsc events --stats CLI implementation when the direct path is unavailable or fails.

The fallback keeps compatibility with unexpected state layouts or runtime conditions while removing recurring subprocess creation from the normal path.

Validation

• bazel test //pkg/shim/v1/runsccmd:runsccmd_test
• Focused tests cover direct-provider success, CLI fallback, state discovery and locking, and host-cgroup CPU apportionment.
• A 444-sandbox ACK validation reduced steady host CPU from the 72.80% baseline to 35.63%.
• A production comparison used the same fixed worker and matched 424/424 ready sandboxes for both 180-second windows, with 406 active plus 18 idle sandboxes, metrics-server absent, and ctld metrics enabled. The direct path reduced average host CPU from 60.68% to 16.70%, CPU p95 from 100% to 36.70%, and forks from 271.30/s to 17.18/s.
• The production process trace observed 11,254 runsc events --stats execs in the baseline window and zero in the direct-path window.
• The existing CRI stats contract and metric fields remained available through the normal collection path.

Assisted-by: OpenAI Codex

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[EtiennePerot]

@zkoopmans Don't we already have something for this in flight?

[zkoopmans]

Yes, we have this in flight for all commands and it will take a minute to develop and roll out. This CL will be redundant when that happens.