[Feature]: Support arm64 for gpu-operator containers

[cyanidium]

Suggestion Description

gpu-operator should be able to run on arm64 control plane nodes.

I understand there might be dependencies for the workloads/kmm on amd64, but the gpu-operator container is very straightforward and runs fine on arm64. It is also the only image that seems to need to run on the control plane. Supporting arm64 is useful for multi-arch clusters with no amd64 control planes.

It seems like this already works with some simple changes:

sed -i 's/amd64/arm64/g' Makefile
sed -i 's/amd64/arm64/g' Dockerfile
sed -i 's/amd64/arm64/g' Dockerfile.build
make docker-build

This generates a working image that I've got running in my test cluster.

There seems to be two ways to implement this:

1. Add build steps for dedicated arm64 containers/tags (could copy the existing files and find/replace, but would result in a lot of duplication), or
2. Implement multi-platform builds (a bit more work needed to get the Makefile changes in place)

Operating System

No response

GPU

No response

ROCm Component

No response

[yansun1996]

Hi @cyanidium thanks for raising this feature request. As of now feel free to fork and rebuild the image based on the platform you're using.

Could you please also add more details on the use case you described Supporting arm64 is useful for multi-arch clusters with no amd64 control planes. ? It would be helpful for our team to understand:

1. what is the overall cluster topology on your side ? (what are all platforms making a multi-arch cluster, control plane and worker node)
2. how is the cluster scale ?
3. which GPU operator / operand components need the extra arch image other than amd64

As you can see when we are doing a new operator release with supporting new feature we also release the operand as well, like device plugin, node labeller, metrics exporter, test runner, device config manager.

The team needs to know more information about the multi-arch cluster you mentioned then discuss internally about the roadmap.

[cyanidium]

These are small on-prem clusters for customers, and a growing subset want to be able to run AI workloads within these small clusters. We usually deploy 3 low cost control plane nodes using arm64 boards like the Rock Pi4, then 3-8 amd64 worker nodes with specs to match the customer requirements. Deploying these small, low cost control plane nodes saves rack space and money.

The only container that must run on the control plane is the gpu-operator image. It's entirely possible that the controller-manager doesn't need to be on the control plane, which is how Intel's GPU plugin works, in which case the simplest fix is to remove the nodeAffinity linked to above.

[yansun1996]

sure thanks @cyanidium , so as of now you can either:

1. build arm64 image based on your forked change
   or

2. modify the entire part under

   gpu-operator/helm-charts-k8s/values.yaml

   Line 275 in 5187ea8

   affinity:

   and our controller manager deployment will pick it up in its YAML manifest

   gpu-operator/helm-charts-k8s/templates/deployment.yaml

   Lines 28 to 31 in 5187ea8

   	{{- with .Values.controllerManager.affinity }}
   	affinity:
   	{{- toYaml . | nindent 8 }}
   	{{- end }}

we'll discuss internally to decide whether / when to support the additional arch based image release, this may not be decided very fast in a short period. More info is always welcome to amend details on the use case and potential deployment opportunities, then we can have more evidence to prioritize resources for this.