fix(vpa): align maxAllowed with the recommender's 64Mi rounding quantum

[duckhawk]

Description

Raises every hardcoded maxAllowed in the storage-oriented VPA templates to a multiple of 64Mi, and switches helm_lib_module_controller_manifests from updateMode: "Initial" to "InPlaceOrRecreate". minAllowed is left untouched everywhere.

Template	Container	Before	After
_csi_controller.tpl	provisioner, attacher, resizer, syncer, snapshotter	20m / 50Mi	50m / 128Mi
	livenessprobe	20m / 50Mi	25m / 64Mi
	controller	20m / 100Mi	100m / 256Mi
_csi_node.tpl	node-driver-registrar	25m / 50Mi	25m / 64Mi
	node	25m / 50Mi	100m / 128Mi
_resources_management.tpl	kube-rbac-proxy	20m / 25Mi	25m / 64Mi
_module_controller.tpl	controller (default)	200m / 100Mi	200m / 256Mi
	webhooks (default)	20m / 100Mi	25m / 128Mi

controllerMaxCpu, controllerMaxMemory, webhooksMaxCpu and webhooksMaxMemory remain overridable per module; only their defaults move.

This restarts pods of critical cluster components. With InPlaceOrRecreate, the controller/webhooks deployments of every module built on helm_lib_module_controller_manifests will have their requests resized for the first time. Where in-place resize is unavailable, the pod is recreated instead — once, but across all CSI modules at the same time.

Why do we need it, and what problem does it solve?

The recommender runs with --round-memory-bytes=67108864 (modules/302-vertical-pod-autoscaler/templates/recommender/deployment.yaml, introduced in deckhouse#18932 replacing the deprecated --humanize-memory). It therefore never emits a memory recommendation below 64Mi and always rounds up to a multiple of it.

Every maxAllowed.memory below 64Mi was consequently binding on every iteration: target was pinned to the ceiling forever and memory autoscaling was effectively off. Values that are not a multiple of 64Mi wasted the remainder — a 100Mi ceiling exposes only the ladder {64Mi, 100Mi}.

The same recommender splits --pod-recommendation-min-cpu-millicores=25 evenly across the containers of a pod, so a maxAllowed.cpu below 25m is always binding for a single-container pod.

Measured across four clusters (VPA uncappedTarget plus max_over_time(...[7d]) from Prometheus), the old ceilings also sat below real demand:

Container	7-day peak	Old ceiling
resizer	66–95Mi	50Mi
node	84Mi / 70m (csi-huawei)	50Mi / 25m
provisioner, attacher	68–73Mi	50Mi
kube-rbac-proxy	39Mi	25Mi
controller (CSI, cephfsplugin)	175Mi	100Mi
controller (module, csi-netapp)	168Mi	100Mi

Roughly a hundred container policies across the storage modules were capped this way, against ten limited by their floor.

Raising a ceiling reserves nothing: container requests are taken from minAllowed, and nothing in Deckhouse reads maxAllowed for capacity planning.

updateMode: "Initial" applies a recommendation only when a pod is created, so the tuning above would have stayed inert for helm_lib_module_controller_manifests consumers until an unrelated restart. The two CSI templates already used InPlaceOrRecreate; this makes the third consistent with them.

What is the expected result?

For a module rendering these defines with vertical-pod-autoscaler-crd enabled:

• Every spec.resourcePolicy.containerPolicies[*].maxAllowed.memory is a multiple of 64Mi, and no maxAllowed.cpu is below 25m.
• spec.updatePolicy.updateMode of the controller VPA is InPlaceOrRecreate.
• minAllowed, and therefore spec.template.spec.containers[*].resources.requests, are unchanged — the diff must not touch container requests.
• Over the following hours, status.recommendation.containerRecommendations[*].target.memory stops sitting exactly on maxAllowed for the sidecars and starts tracking uncappedTarget (64Mi for idle containers, 128Mi for resizer).

Checklist

• The code is covered by unit tests.
• e2e tests passed.
• Documentation updated according to the changes.
• Changes were tested in the Kubernetes cluster manually.