Rolling back a deployment

Runway tags every deployed manifest bundle with its version in your deployment project’s container registry:

• deploy:staging-<version> and deploy:production-<version> are kept for each deploy.
• deploy:staging and deploy:production are the tags that Flux watches.

A rollback moves the live tag (deploy:staging / deploy:production) back onto an existing versioned bundle. Nothing is re-rendered, so the cluster receives exactly the manifests that were deployed at that version.

• Your service uses Runway v2 (Kubernetes with Fairway).
• Your service runs runwayctl v4.73.1 or later. See Find your runwayctl version.
• At least one deploy has completed since upgrading. Versioned tags only exist for deploys made with v4.73.1 or later, so you cannot roll back to an older version.
• You have the Maintainer role on the service’s deployment project.

1. Open your service’s deployment project: gitlab.com/gitlab-com/gl-infra/platform/runway/deployments/<service>.

2. Go to Build → Pipelines → Run pipeline and select the main branch.

3. Add these variables:

   Variable	Value
   RUNWAY_VERSION	The runwayctl version your service runs (for example v4.73.1).
   USE_FAIRWAY_CHART	true
   DEPLOYMENT_VERSION_OVERRIDE	The version to restore, without the leading v (for example 1.1.205).

4. Run the pipeline. The 📤 Upload Resource Manifests job moves the live tags, and Flux then reconciles the cluster to the older bundle.

To find the versions you can roll back to, open Deploy → Container Registry in the deployment project and look at the staging-* and production-* tags.

Open the latest pipeline of your service project and look at the log of the 📝 [<service-id>] Runway environment job. It prints a RUNWAY_VERSION= line near the end.

If you use the release-platform component, the runwayctl version is bundled with the component version (see Step 4) and is not set in your .gitlab-ci.yml, so the pipeline log is the way to check it. To get a newer runwayctl, upgrade the component version, which Renovate does for you.

• The rollback runs for both staging and production. If your service sets runway_manual_production_deploy: true on the release-platform component, the production job waits for a manual play.
• A rollback must be started manually or through the API. Pipelines triggered by your service project are rejected when DEPLOYMENT_VERSION_OVERRIDE is set.
• If the version has no versioned bundle, or the registry call fails, the job fails and prints the error.
• Versioned bundles (staging-<version>, production-<version>) older than 90 days are removed by a weekly registry cleanup. The 10 most recent versioned bundles are always kept, even if they are older, so a service that rarely deploys still keeps some history. deploy:staging and deploy:production are never removed.
• The next normal deploy replaces the rolled-back version, so revert or fix the change that caused the problem as well.