A Kubernetes upgrade is a coordinated change to the control plane, nodes, APIs, add-ons, storage integrations, and workloads. This reusable Kubernetes upgrade checklist helps you assess compatibility, prepare backups, stage the change, validate services, and respond safely if the upgrade does not go as planned.
Overview
Upgrade work should begin with an inventory, not a command. Record the current Kubernetes version, target version, cluster type, node pools, operating systems, container runtime, installed controllers, admission webhooks, storage drivers, ingress components, monitoring agents, and critical workloads. Managed Kubernetes services may separate control-plane upgrades from node-pool upgrades, while self-managed clusters require a broader plan for control-plane components, certificates, bootstrapping, and infrastructure dependencies.
Use the target platform's supported-version documentation as the authority for exact upgrade order, version availability, command syntax, and rollback limitations. Kubernetes version-skew guidance can affect which kubelets, clients, controllers, and extensions may run together. Treat the following table as a planning model rather than a substitute for the version policy of your Kubernetes distribution.
| Area | What to verify | Evidence to collect |
|---|---|---|
| Control plane | Current and target versions, upgrade order, API-server access, and provider maintenance behavior | Cluster inventory and provider or distribution runbook |
| Nodes | Kubelet compatibility, operating-system images, runtime support, capacity, and replacement strategy | Node list, capacity report, and node-pool plan |
| APIs | Removed or deprecated API versions used by manifests, charts, controllers, or custom resources | Rendered manifests and API usage scan |
| Add-ons | Ingress, DNS, CNI, CSI, autoscaling, policy, observability, and admission components | Compatibility matrix and tested versions |
| Workloads | Readiness, replicas, probes, disruption budgets, scheduling rules, and capacity needs | Workload health report and deployment test results |
| Data | Persistent volumes, backup coverage, restore procedure, and application-level recovery | Recent backup confirmation and restore evidence |
Checklist by scenario
Before any cluster upgrade
- Define the target version, maintenance window, owners, communication channel, and stop conditions.
- Export or otherwise preserve important cluster configuration, including namespaces, workloads, RBAC, network policies, custom resources, and admission configuration.
- Confirm that critical data has a usable backup. A backup is only meaningful if the team knows how to restore it and has considered application consistency.
- Check cluster health: nodes should be ready, system pods should be stable, and there should be no unresolved scheduling, storage, DNS, or networking failures.
- Review resource headroom. Draining a node requires enough capacity to place its pods elsewhere, including temporary overhead from replacement nodes.
- Search source repositories, Helm values, GitOps configuration, and generated manifests for deprecated API versions and version-specific fields.
- Check every add-on separately. CNI, CSI, ingress, policy, autoscaling, metrics, logging, and service-mesh components can have independent compatibility requirements.
- Confirm access to the cluster, cloud account, infrastructure code, image registry, secrets system, and emergency support path.
When upgrading a managed cluster
- Read the provider's procedure for control-plane upgrades, node-pool upgrades, surge capacity, automatic replacement, maintenance windows, and rollback limitations.
- Determine whether the provider changes the control plane first and whether existing node pools remain supported during the transition.
- Plan node-pool replacement or in-place maintenance as a separate operation when the platform recommends it.
- Review provider-managed add-ons and identify which components your team owns.
- Test the application in a staging cluster that resembles production, including policies, storage classes, ingress, identity, and autoscaling behavior.
When upgrading a self-managed cluster
- Document the control-plane topology, quorum dependencies, certificates, bootstrap configuration, etcd protection, and recovery process.
- Validate the documented upgrade order for control-plane components, kubelets, and command-line clients.
- Back up the cluster state using the method appropriate to your installation and verify that the backup can be located and restored.
- Upgrade one change domain at a time, recording commands, results, warnings, and configuration differences.
- Keep an out-of-band access path available for nodes and control-plane hosts if normal cluster access fails.
During node maintenance
- Inspect the node before acting:
kubectl get nodes -o wideandkubectl describe node <name>. - Check placement constraints, replica counts, topology spread rules, and PodDisruptionBudgets before draining.
- Cordon the node, then drain it using the flags and behavior appropriate to your workload. Review the dry-run output and understand how local storage, unmanaged pods, and daemon sets will be handled.
- Watch pending pods, replacement replicas, events, readiness probes, and application metrics while the node is unavailable.
- Uncordon only after the node is healthy and the intended kubelet, runtime, and operating-system versions are confirmed.
For maintenance-related availability planning, see the Kubernetes PodDisruptionBudget guide. If the release process uses staged traffic, a canary, or blue-green deployment, pair the cluster checklist with the canary and blue-green deployment comparison.
What to double-check after the upgrade
Do not define success as “the upgrade command completed.” Validate the platform in layers, beginning with infrastructure and ending with user-visible behavior.
- Cluster access: Confirm API access, authentication, authorization, context selection, and expected client behavior.
- Nodes: Check readiness, conditions, allocatable capacity, taints, labels, runtime status, and version reporting.
- System services: Inspect DNS, networking, proxy or dataplane components, storage controllers, admission webhooks, metrics, and autoscaling controllers.
- Workloads: Review deployments, stateful sets, daemon sets, jobs, and cron jobs. Look for pending, restarting, unavailable, or unexpectedly recreated pods.
- Storage: Confirm that existing claims remain attached and that a controlled read/write test succeeds where appropriate.
- Networking: Test service discovery, ingress routes, TLS termination, network policies, external dependencies, and health checks.
- Observability: Check logs, metrics, traces, alerts, dashboards, and recording rules. A monitoring gap can hide a failed upgrade.
- Application paths: Run smoke tests for authentication, reads, writes, background processing, queues, and critical integrations.
Record before-and-after results rather than relying on memory. This creates a useful baseline for incident response and makes it easier to distinguish upgrade effects from unrelated changes.
Recovery and Kubernetes upgrade troubleshooting
Write the recovery plan before the maintenance window. Decide who can pause the upgrade, restore workloads, revert infrastructure changes, or contact the platform provider. In many environments, a completed control-plane upgrade cannot simply be rolled back. Recovery may instead mean restoring data, replacing a node pool, reverting application manifests, disabling an incompatible add-on, or moving traffic to another environment.
When a failure appears, preserve evidence first: timestamps, events, controller logs, node conditions, provider activity records, and the exact commands or automation output. Then classify the problem:
- Scheduling failure: inspect capacity, taints, affinity, topology rules, and disruption budgets.
- API failure: inspect removed API versions, webhook availability, authentication, and client-server compatibility.
- Networking failure: inspect CNI health, routes, DNS, policies, load balancers, and security controls.
- Storage failure: inspect CSI components, volume attachment, permissions, topology, and provider events.
- Application failure: compare image, configuration, probe, secret, and dependency changes with the pre-upgrade baseline.
Avoid making several corrective changes at once. Stabilize the platform, capture evidence, apply the smallest reversible change, and rerun the validation checks. For broader deployment recovery planning, the CI/CD pipeline failure triage playbook provides a complementary incident workflow.
Common mistakes
- Checking only the control-plane version: Node pools, clients, add-ons, and manifests may need separate review.
- Relying on a backup without a restore test: Confirm that backups are complete, accessible, and usable for the failure scenarios you care about.
- Ignoring disruption policy: A strict PodDisruptionBudget can intentionally block draining; weakening it temporarily should be an explicit risk decision.
- Assuming staging proves production safety: Differences in traffic, data volume, identity, storage, and topology can expose production-only problems.
- Updating add-ons as an afterthought: Controllers and webhooks can affect every namespace and may fail before application pods do.
- Starting during an active incident: Delay planned maintenance when the baseline is already unstable unless the upgrade is the agreed remediation.
- Leaving temporary changes behind: Remove emergency tolerations, relaxed policies, extra capacity, debug settings, and temporary access after validation.
When to revisit this checklist
Review this Kubernetes upgrade checklist before every control-plane upgrade, node-pool replacement, platform migration, or major add-on change. Revisit it during seasonal planning cycles, before a maintenance window, after a provider changes its upgrade workflow, and whenever the cluster's storage, networking, identity, or observability architecture changes.
Refresh the version-skew notes, deprecated API examples, provider commands, add-on compatibility records, backup procedure, and rollback assumptions whenever Kubernetes or the surrounding platform changes. After each upgrade, hold a short review: record what was checked, which warnings appeared, which steps took longer than expected, and what should become an automated preflight test. The next upgrade should use that record as its starting point.
Practical next step: copy the checklist into your team's runbook, add the exact platform-specific commands and owners, and require a completed preflight plus post-upgrade validation record before closing the maintenance task.