Spark Connect Server/Exec pods are not updated #2784
Description
What happened?
- [*] ✋ I have searched the open/closed issues and my issue is not listed.
When updating the SparkConnect Resource with a newer image, the operator does not restart the respective -(server|exec) pods.
Reproduction Code
[] Deploy SparkOperator
[] Deploy a SparkConnect resource
[] Wait for pods to start/stabilize
[] Patch the SparkConnect resource with a different image
Expected behavior
We would expect Spark Operator to restart pods with the new container Specs
Actual behavior
Spark Connect exec/server pods do not get updated
Environment & Versions
- Kubernetes Version: EKS/RKE2
- Spark Operator Version: 2.3.0 + 2.4.0
- Apache Spark Version: Spark 4
Additional context
The operator logs:
2025-12-19T10:29:40.025Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"dev"}, "namespace": "dev", "name": "spark-connect", "reconcileID": "a6350e13-e79e-4adf-8092-98679c30aa9a"}
2025-12-19T10:29:40.034Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"dev"}, "namespace": "dev", "name": "spark-connect", "reconcileID": "c4ec8b94-5e6f-4777-a10f-d730921101fc"}
2025-12-19T11:08:43.657Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"stefb"}, "namespace": "stefb", "name": "spark-connect", "reconcileID": "0d510659-01ec-4921-aac4-3cc65993da4b"}
2025-12-19T11:08:43.657Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"baltrus"}, "namespace": "baltrus", "name": "spark-connect", "reconcileID": "90d2453a-d6ac-48a5-8451-ebbe611608aa"}
2025-12-19T11:08:43.657Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"dev"}, "namespace": "dev", "name": "spark-connect", "reconcileID": "9d1919e8-0eef-43be-ae3e-bc7ef7c430c5"}
2025-12-19T11:26:28.983Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"dev"}, "namespace": "dev", "name": "spark-connect", "reconcileID": "1a1c29a0-7ded-4a3e-8f8b-ad8e675be38e"}
2025-12-19T11:26:28.992Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"dev"}, "namespace": "dev", "name": "spark-connect", "reconcileID": "fc02423a-3599-493f-89cb-02ed8f50c4e8"}
2025-12-19T11:32:43.552Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"dev"}, "namespace": "dev", "name": "spark-connect", "reconcileID": "626b1c3b-a4c9-4376-a6bb-ae9643cb423e"}
2025-12-19T11:32:43.552Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"stefb"}, "namespace": "stefb", "name": "spark-connect", "reconcileID": "e67abbe4-756e-48e8-bd5b-481bd9bdb268"}
2025-12-19T11:32:43.553Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"baltrus"}, "namespace": "baltrus", "name": "spark-connect", "reconcileID": "4602debb-96df-47c8-922e-8dad0cc9a34e"}
2025-12-19T11:41:32.969Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"dev"}, "namespace": "dev", "name": "spark-connect", "reconcileID": "85ebcb73-7ba7-4d52-9286-e6a957a1b627"}
2025-12-19T11:41:32.978Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"dev"}, "namespace": "dev", "name": "spark-connect", "reconcileID": "043d78d7-48e0-4783-ae1b-8a3856411be6"}
2025-12-19T11:55:17.152Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"dev"}, "namespace": "dev", "name": "spark-connect", "reconcileID": "d7bbae41-e408-48ab-a330-4acc259345e3"}
2025-12-19T11:55:17.164Z INFO sparkconnect/reconciler.go:195 Reconciling SparkConnect {"controller": "sparkconnect", "controllerGroup": "sparkoperator.k8s.io", "controllerKind": "SparkConnect", "SparkConnect": {"name":"spark-connect","namespace":"dev"}, "namespace": "dev", "name": "spark-connect", "reconcileID": "da25f929-f332-4ada-9f95-435be38c6a2c"}
Impacted by this bug?
Give it a 👍 We prioritize the issues with most 👍