Start every investigation with the same three commands:
argocd app get shop-api-prod # conditions, sync result, resource list
argocd app diff shop-api-prod # what exactly differs
kubectl -n argocd logs deploy/argocd-repo-server --tail=50 # render errors
1. OutOfSync that never goes away
Another controller (HPA, a mutating webhook, an operator, cert-manager injecting a CA bundle) changes fields Argo CD also manages. Tell Argo CD to ignore those fields:
spec:
ignoreDifferences:
- group: apps
kind: Deployment
jsonPointers: [/spec/replicas]
- group: admissionregistration.k8s.io
kind: MutatingWebhookConfiguration
jqPathExpressions: ['.webhooks[]?.clientConfig.caBundle']
syncPolicy:
syncOptions: [RespectIgnoreDifferences=true] # also skip these fields when syncing
Use it narrowly — ignoring a field also hides real drift there. System-wide rules go in argocd-cm under resource.customizations.ignoreDifferences.<group>_<kind>.
2. A sync that hangs or fails
| Message / symptom | Cause | Fix |
|---|---|---|
| Waiting for a PreSync hook | The Job never completes | argocd app terminate-op, fix the Job, add activeDeadlineSeconds |
admission webhook ... denied the request |
A policy engine (Kyverno, PSA) rejects an object | Fix the manifest; check the policy report |
the server could not find the requested resource |
A CRD is not installed yet | Install the CRD in an earlier wave, or SkipDryRunOnMissingResource=true |
exceeded quota |
ResourceQuota in the namespace | Lower requests or raise the quota |
metadata.annotations: Too long |
A very large object with client-side apply | ServerSideApply=true |
3. An app that will not delete
argocd app delete waits while the finalizer resources-finalizer.argocd.argoproj.io removes child objects. If a child is stuck (its own finalizer, a namespace terminating), find it with argocd app resources and fix that. Removing the Application's finalizer by hand leaves orphaned workloads — only do it on purpose.
4. Unknown status and render errors
ComparisonError: rpc error ... authentication required→ repo credentials expired or deploy key removed (argocd repo listshows the status).helm template ... failed/kustomize build ... failed→ reproduce locally with the same tool and version; the repo-server log shows the full error.- Very large monorepos → raise
reposerver.parallelism.limit, use webhooks instead of short polling, and point Applications at small paths.
5. Performance at scale
Hundreds of apps across clusters: shard the application controller (controller.sharding.algorithm, more replicas), increase --status-processors / --operation-processors, and exclude noisy resource kinds you do not manage (resource.exclusions).
A short runbook
- Read conditions in
argocd app get— the message is usually precise. argocd app difffor OutOfSync,kubectl describe/ events for Degraded.- Check the component log that owns the step: repo server (render), application controller (compare/sync), server (login/RBAC).
- Fix in Git when the cause is a manifest; fix in Argo CD config (also in Git) when it is a setting.