Upgrade to NKP 2.18, Part 1: Prerequisites, Artifacts and the Registry Push

Before upgrading, read what 2.18 actually changes, verified on a live cluster. This first part of the upgrade series is everything that happens before the first nkp upgrade command: the gates to check, the artifacts to stage, and the registry push. We upgraded our own air-gapped management cluster (3 control plane + 6 workers, Ultimate, Nutanix AHV) with this exact procedure; the warnings come from that run.

What you are about to do#

An NKP 2.17 to 2.18 upgrade is four phases, in strict order: Kommander and platform apps on the management cluster, platform apps per workspace, the management cluster's Kubernetes/node upgrade, then the workload clusters. Plan half a day for the whole thing. The registry push depends on your link and is resumable. Every phase is resumable; none of it requires downtime for workloads that tolerate node drains.

Step 1: check the compatibility gates#

Checkpoint: every row of the table passes before you continue.

On the OS image: the pre-built nkp-rocky-*-release-cis-* images are CIS-hardened. If you build your own with nkp create image, hardening is only applied when you set cis: true in the build overrides.

Step 2: verify the cluster is ready#

Run these the day before, not the morning of:

# 1. Everything green? Fix or consciously remove every line of output.
kubectl get hr -A --no-headers | awk '$4!="True"'
kubectl get kustomizations -A --no-headers | awk '$4!="True"'

# 2. PDB audit: every hit below WILL freeze your node roll (Part 4).
kubectl get pdb -A | awk '$4=="0"'

# 3. Deprecation audit, against the LIVE cluster (UI-created RBAC is invisible in git):
kubectl get clusterrolebindings -o wide | grep kommander-federated-edit

# 4. Backup: a tested restore, not an installed chart. Why: the platform phase rewrites
#    cluster state (new CRDs, adopted objects, RBAC) hours before any node is touched -
#    an etcd snapshot is your restore point if that goes wrong. Minimum:
kubectl -n kube-system exec etcd-<cp-node> -- etcdctl \
  --cacert /etc/kubernetes/pki/etcd/ca.crt \
  --cert /etc/kubernetes/pki/etcd/server.crt \
  --key /etc/kubernetes/pki/etcd/server.key \
  snapshot save /var/lib/etcd/pre-218-upgrade.db

Two more checks, if they apply to your setup:

• Custom config overrides inventory. List every *-overrides ConfigMap you ever created, and flag anything touching Loki storage: the automated v2 to v3 migration has a narrow input contract and will stop your platform upgrade (Part 3 is entirely about this).
• GitOps-managed clusters: set prune: false on the Flux Kustomizations that own cluster manifests, now, before the Kommander upgrade (Part 5 explains why this ordering is not optional). Ours already ran with it; verify yours: kubectl get kustomization <name> -n <ns> -o jsonpath='{.spec.prune}' must return false.

Step 3: stage the artifacts#

Three downloads from the Nutanix portal:

1. The air-gapped bundle nkp-air-gapped-bundle_v2.18.0_linux_amd64.tar.gz, extracted:

nkp-v2.18.0/
  cli/                                              # the 2.18 nkp binary
  container-images/
    konvoy-image-bundle-v2.18.0.tar                 # cluster / Kubernetes images
    kommander-image-bundle-v2.18.0.tar              # platform app images
    nutanix-product-catalog-v2.18.0-airgapped.tar   # NEW: NDK catalog images
  application-repositories/
    kommander-applications-v2.18.0.tar.gz           # platform app helm charts
  konvoy-bootstrap-image-v2.18.0.tar
  image-artifacts/

The nutanix-product-catalog tar is new in 2.18: NDK ships in the default bundle, so your catalog gets NDK without a separate download.

2. The 2.18 CLI, installed alongside the old one:

sudo cp /usr/local/bin/nkp /usr/local/bin/nkp217     # keep 2.17!
sudo cp ./cli/nkp /usr/local/bin/nkp

Keeping the 2.17 binary is required, not optional. The 2.18 CLI cannot delete pre-2.18 clusters (NCN-113770) and refuses several commands against pre-2.18 management clusters (NCN-115429). Until your last cluster is upgraded, you need both.

3. The OS image, uploaded to Prism Central. Ours: nkp-rocky-9.7-release-cis-1.35.2-20260626061237.qcow2. These images are hardened by Nutanix (CIS); we will publish an analysis of what that hardening actually contains in the coming days.

Step 4: push the bundles to your registry#

From the extracted bundle root, one command pushes all three image bundles:

nkp push bundle \
  --bundle ./container-images/konvoy-image-bundle-v2.18.0.tar,./container-images/kommander-image-bundle-v2.18.0.tar,./container-images/nutanix-product-catalog-v2.18.0-airgapped.tar \
  --to-registry=https://10.12.52.11/nkp \
  --to-registry-username=admin \
  --to-registry-password='***'

Notes from our push:

• The 2.18 images land in the same registry project as the 2.17 ones. Nothing needs removing first; the running cluster keeps pulling its 2.17 images until each component is upgraded past them.
• Sizes, measured: the bundle download is ~29 GB, and extracting it needs the same again (the two image tars alone are 16 GB and 6.3 GB). Plan roughly 60 GB free on the machine doing the push, before the images even reach the registry's own storage.
• It is just over 400 images across the three bundles (the CLI shows per-bundle counters: Pushing bundled images [==> 266/266]).
• The push is resumable: already-pushed layers are skipped, so an interrupted run just gets re-run.
• If your registry uses a self-signed or private CA, add --to-registry-ca-cert-file <your-registry-ca.crt>.

If something bites#

Ready checklist#

Start Part 2 only when every box is checked:

• All HelmReleases and Kustomizations green (or consciously removed)
• PDB audit list empty, or every entry has a planned intervention
• Backup verified (etcd snapshot taken, restore path known)
• nkp version shows 2.18.0 and nkp217 version shows 2.17.x
• All three image bundles pushed to the registry
• The 1.35.2 OS image visible in Prism Central
• GitOps only: prune: false set on cluster Kustomizations

Next: Part 2, the platform upgrade - nkp upgrade kommander, the workspaces, and what to do when the CLI times out.