Skip to main content

Upgrade and rollback Rancher environment

The Rancher package provided by Adeptia also allows you to upgrade the deployed components to the desired supported versions.

Upgrading a component​

To upgrade the components, follow the steps given below:

  1. Enter the version of the component to which you want to upgrade in vars.yaml file under default directory.
  2. Run the following command to upgrade the component:

Code

sudo ./adeptia-rancher.sh --tag=upgrade-<name of the component>

Wherein <name of the component> can be any valid component name, for example, rke2, rancher.

TagDescription
--tag=upgrade-platformUpgrades RKE2 → Envoy Gateway → Rancher → LAN Volume Provisioner. After this, you have a working K8s cluster with ingress + storage but no app.
--tag=upgrade-dependenciesUpgrades RabbitMQ Operator and KEDA on top of the platform.
--tag=upgrade-applicationUpgrades Adeptia Automate. If mcp_enabled: true, also upgrades the MCP add-on. Re-renders Envoy routes so the app is reachable via your domain.
--tag=upgrade-allUpgrades all three layers end-to-end.
Warningupgrade-platform takes a pre-upgrade etcd snapshot, and writes /var/lib/rancher/rke2/server/upgrade-state.json. It refuses to run if that state file already exists or if the cluster is already on the target version. This state file also helps you to roll back to the previous version of the component if required.

Rolling back RKE2​

If your upgrade somehow fails, you can roll back the RKE2 component only. Run the command below to roll back:

Code

sudo ./adeptia-rancher.sh --tag=rollback-rke2

RKE2 Upgrade and Rollback Workflow​

The upgrade-rke2 workflow automatically creates a pre-upgrade etcd snapshot and records cluster metadata to enable a full rollback if required. The rollback-rke2 workflow uses this information to restore the cluster to its exact pre-upgrade state.

Files Used by the Upgrade and Rollback Workflows​

ETCD Snapshot​

Location

/var/lib/rancher/rke2/server/db/snapshots/pre-upgrade-<epoch>.db

Purpose

An etcd snapshot taken immediately before the upgrade begins. The snapshot filename includes a unique epoch timestamp to ensure snapshots are never overwritten and can accumulate across multiple upgrades.

Upgrade State File​

Location

/var/lib/rancher/rke2/server/upgrade-state.json

Purpose

Stores rollback metadata required by the rollback workflow, including:

  • Pre-upgrade RKE2 version
  • Rancher version
  • cert-manager version
  • Snapshot filename and path
  • Snapshot timestamp
  • Per-node copy of /etc/rancher/rke2/config.yaml

Temporary Snapshot Backup Directory​

Location

/tmp/rke2-snapshots-backup/

Purpose

Used temporarily during the rollback process. A backup of the snapshots directory is stored here before the cluster reset operation. The directory is automatically removed after rollback completes.

Upgrade-rke2 Workflow​

The upgrade workflow performs the following steps:

  1. Reads the current:

    • rke2_version
    • rancher_version
    • cert_manager_version

  2. Creates an etcd snapshot named:

pre-upgrade-<epoch>

and stores it in the standard RKE2 snapshots directory. 3. Collects each node's:

/etc/rancher/rke2/config.yaml

file and writes the following information to:

/var/lib/rancher/rke2/server/upgrade-state.json
  • Component versions
  • Snapshot location
  • Snapshot timestamp
  • Per-node RKE2 configuration
  1. Performs a rolling, node-by-node upgrade across the cluster.

Rollback-rke2 Workflow​

The rollback workflow performs the following steps:

  1. Reads:
/var/lib/rancher/rke2/server/upgrade-state.json

The workflow aborts if the file is missing. 2. Verifies that the snapshot referenced by snapshot_path exists.

The workflow aborts if the snapshot cannot be found. 3. Performs cluster restoration:

  • Backs up the snapshots directory to:
/tmp/rke2-snapshots-backup/
  • Executes:
rke2 server --cluster-reset --cluster-reset-restore-path=<snapshot>
  • Restores the original snapshots directory.

  1. Reapplies the recorded pre-upgrade versions of:

    • RKE2
    • Rancher
    • cert-manager

  2. Restores each node's original:

/etc/rancher/rke2/config.yaml

from the saved configuration snapshot.

Safety Checks in upgrade-rke2​

The upgrade workflow will refuse to start under either of the following conditions.

Existing Upgrade State File​

If the following file already exists:

/var/lib/rancher/rke2/server/upgrade-state.json

the workflow aborts to prevent overwriting rollback metadata from a previous upgrade.

If you are certain that the cluster remains in its pre-upgrade state and want to initiate a new upgrade, move the existing state file:

Code

sudo mv /var/lib/rancher/rke2/server/upgrade-state.json /var/lib/rancher/rke2/server/upgrade-state.json.bak.$(date +%s)

Cluster Already Running the Target Version​

The workflow aborts if the cluster is already running the specified target rke2_version.

Capturing rollback metadata in this scenario would record the post-upgrade state as the rollback target, making rollback ineffective.

Ensure that rke2_version is set to the version you intend to upgrade to.

Operator Notes

  • The snapshot file and upgrade-state.json are stored only on the primary server.
  • Regularly back up the following locations to protect against primary-server failures:
/var/lib/rancher/rke2/server/upgrade-state.json 
/var/lib/rancher/rke2/server/db/snapshots/