ArgoCD set replicas of old replicaSet to 1 during deployment of new version

[jacob-braaten]

We are working on moving to using ArgoCD for our deployments, previously we were using Helm. We use KEDA to autoscale our services, so we do not define a replicas field in the deployment manifest.

The upgrade involved a new image version, so there a rolling update was triggered. When this happened ArgoCD set the replicaCount of the old replicaSet to 1. There is a kubernetes event log that says Scaled down to 1 from 200. This was without waiting for new pods to spin up. This led to many errors being thrown until the service could scale.

The diff for the deployment only showed adding the ArgoCD annotations and changing the versions. Does anyone have ideas as to what could have caused this?

I thought it might have been that since our Deployment manifest doesn't supply a replicas field the default value of 1 that kubernetes assumed overrode the value in the deployment, but I have not been able to replicate that scenario.

[anntnzrb]

When ArgoCD syncs a Deployment without a replicas field, Kubernetes defaults to 1, overriding KEDA's scaled count.

Fix: Configure ArgoCD to ignore the replicas field:

spec:
  ignoreDifferences:
    - group: apps
      kind: Deployment
      jsonPointers:
        - /spec/replicas
  syncPolicy:
    syncOptions:
      - RespectIgnoreDifferences=true

This prevents ArgoCD from managing replica counts, allowing KEDA to maintain full control during deployments.

[jacob-braaten]

I had thought this was the case, but I was not able to get it to reproduce to prove this was the case. I'll keep trying to get it to reproduce, I may have setup my test case incorrectly because from what I can tell this is the only logical reason.

Thank you for the quick reply.

[anntnzrb]

When your Deployment manifest omits replicas, Kubernetes defaults to 1 during sync, overriding KEDA's scaled count. Configure ArgoCD to ignore the replicas field:

spec:
  ignoreDifferences:
  - group: apps
    kind: Deployment
    jsonPointers:
      - /spec/replicas
  syncPolicy:
    syncOptions:
      - RespectIgnoreDifferences=true

The RespectIgnoreDifferences=true is critical - it applies the ignore rule during sync, not just diff calculation. This lets KEDA maintain control over replicas during rolling updates.

[jacob-braaten]

Updating in case others come across this. The issue was related to what @anntnzrb said above. There is an added wrinkle though. In the past we had gone from using helm template | kubectl apply -f - to using helm upgrade however, the kubectl.kubernetes.io/last-applied-config annotation was still on the resources from the last release before moving to helm upgrade. This resulted in ArgoCD thinking it needed to update the replicas field to empty which cause kubernetes to default to 1. Leading to the scaling issue.

I have verified that setting RespectIgnoreDifferences=true correctly updates that annotation to have the correct number of replicas.