Canary releases

A canary release puts a new version beside the stable one, exposes it to a small share of traffic, watches it, and either widens the exposure step by step until it is promoted or rolls it back. @zuke/canary builds that whole chain from one settings lambda — and every way the rollout can go wrong runs one rollback.

A rollout in one declaration

canary(...) is a component: assign it to a field and its targets appear under that field's name. The rest of the build depends on them like any other target — here, ship runs only once the release is promoted.

import { Build, run, target } from "jsr:@zuke/core";
import { canary, httpProbe, prometheus } from "jsr:@zuke/canary";

class Deploy extends Build {
  rollout = canary((c) =>
    c.platform(cloudRun) // anything implementing CanaryPlatform
      .steps(10, 25, 50) // exposure after each step; promotion is the 100 %
      .bake("10m") // per step; .bakeStep(3, "30m") overrides one
      .analysis(
        httpProbe((h) => h.url(HEALTH_URL).samples(30).maxFailures(0)),
        prometheus((p) => p.url(PROMETHEUS).query(ERROR_RATIO).max(0.01)),
      )
      .approval("canary-approved") // optional gate before promotion
      .approvalTimeout("24h") // and how long it waits
      .lock((l) => l.lockKey("deploy", "api").withTtl("24h"))
  );

  ship = target().dependsOn(this.rollout.promote).executes(() => {});
}

await run(Deploy);

A failed analysis, a failed step, an approval that times out, and zuke cancel all end in the same place: rollout.abort, which returns every user to the stable version.

The targets

A target only exists when it has something to do: a step with no bake has no bake, and a canary with no analyses has no analyze. A canary with no steps — on a release-channel platform, for example — gets a single rollout.soak phase when it bakes or analyses.

Requires a state store

Platforms

The engine never talks to Cloud Run, Kubernetes or anything else directly. It calls a CanaryPlatform, and the adapters for specific platforms live in their tool's wrapper package. Each adapter's lambda runs on every call, so it may read resolved parameters.

Cloud Run

import { cloudRunCanary } from "jsr:@zuke/gcloud";

c.platform(
  cloudRunCanary((r) =>
    r.service("api").region("europe-west1").image(this.image.value)
      .gcloud((g) => g.project("my-project"))
  ),
)

stage deploys the image as a new revision with no traffic and the canary tag; each step moves the tag's share with update-traffic; promote sends everything to the latest revision after checking it is still the staged one.

Kubernetes

import { kubectlCanary } from "jsr:@zuke/kubectl";

c.platform(
  kubectlCanary((k) =>
    k.stable("api").canary("api-canary").container("api")
      .image(this.image.value).replicas(10).namespace("prod")
      .kubectl((s) => s.context("prod"))
  ),
)

Give the two Deployments a shared label that one Service selects. A Service balances connections, not requests, so with keep-alive or HTTP/2 clients the real share can differ from the replica share — use a traffic platform when you need an exact request split. Suspend any HorizontalPodAutoscaler or GitOps self-heal that owns either Deployment during a rollout.

Helm

import { helmCanary } from "jsr:@zuke/helm";

c.platform(
  helmCanary((h) =>
    h.chart("./charts/api").stableRelease("api").namespace("prod")
      .image(this.tag.value).replicas(10).values("values-prod.yaml")
      .helm((s) => s.kubeContext("prod"))
  ),
)

The candidate runs as <stable>-canary unless you name it with .canaryRelease(...). A rollback returns the stable release to the revision stage recorded and uninstalls the canary.

Docker Compose

import { dockerComposeCanary } from "jsr:@zuke/docker-compose";

c.platform(
  dockerComposeCanary((d) =>
    d.service("app").canaryService("app-canary").replicas(4)
      .image(this.image.value)
      .stableImageVariable("APP_IMAGE")
      .canaryImageVariable("APP_CANARY_IMAGE")
      .compose((s) => s.file("compose.yml").projectName("shop"))
  ),
)

Compose has no command that sets a service's image, so each service's image: must be a whole variable that the platform sets on every command it runs.

Every adapter records what it staged, and later calls refuse — changing nothing — when the configuration no longer matches. The full canary guide lists each adapter's exact commands, the checks it runs, and how to recover from a refusal.

Your own platform

Until an adapter exists for yours, a platform is a plain object in the build:

import { Build, run, target } from "jsr:@zuke/core";
import { $ } from "jsr:@zuke/core/shell";
import { canary, type CanaryPlatform, httpProbe } from "jsr:@zuke/canary";

// Two Docker Compose services behind a weighted proxy, scripted by hand.
const compose: CanaryPlatform = {
  exposure: "replicas",
  describe: () => "compose service api",
  async stage(ctx) {
    await $`docker compose up -d --scale api-canary=1 api-canary`;
    await ctx.state.set({ stable: "api", candidate: "api-canary" });
  },
  async expose(percent) {
    const canaries = Math.round(percent / 25);
    await $`docker compose up -d --scale api-canary=${canaries} api-canary`;
    return canaries * 25;
  },
  async promote() {
    await $`docker compose up -d api`;
    await $`docker compose rm -sf api-canary`;
  },
  async abort() {
    // Needs no state: the stable service is fixed by configuration.
    await $`docker compose rm -sf api-canary`;
  },
};

class Deploy extends Build {
  rollout = canary((c) =>
    c.platform(compose)
      .steps(25, 50)
      .bake("5m")
      .analysis(httpProbe((h) => h.url("http://localhost:8080/healthz")))
  );
  ship = target().dependsOn(this.rollout.promote).executes(() => {});
}

await run(Deploy);

Bakes: inline or durable

By default a bake is inline: the bake target sleeps in the process while the analyses run every .analysisInterval(...) (default "1m"). The first failing check stops the bake at once and rolls back; zuke cancel or Ctrl-C ends it too. An inline bake is a timer, so it can be at most about 24 days.

.durable() makes each bake a gate instead. The run suspends at it, so no process is alive during the bake, and zuke resume --check reopens it once the time is up. A durable bake runs no in-flight analyses, since nothing is there to run them; each step's final analyze still runs in whichever process resumes the run. Use durable bakes when a bake is long, or when the CI job that started the rollout should not stay alive for it.

rollout = canary((c) =>
  c.platform(cloudRun)
    .steps(10, 25, 50)
    .bake("6h")
    .durable() // each bake suspends the run; no process stays alive
);

# from a cron: reopen every parked gate whose time is up
zuke resume --check

Analyses

An analysis is anything with a validate(ctx) that throws to fail, so any core Validation works unchanged — an @zuke/ai review included. A canary-aware one also reads where the rollout stands: ctx.step, ctx.requested and ctx.exposure. The package ships three, all over plain HTTP:

For anything beyond bounds on one instant query — a quantile you reduce yourself, a range, a rule's state, the firing alerts — read it with PrometheusTasks from @zuke/prometheus, the same typed client prometheus(...) queries through, and judge it in a metricThreshold(...) reader. PrometheusTasks.value returns the one number an instant query answered, and fails clearly on an empty or multi-sample result.

import { metricThreshold } from "jsr:@zuke/canary";
import { PrometheusTasks } from "jsr:@zuke/prometheus";

metricThreshold((m) =>
  m.name("p99 seconds").max(0.5).read(async () =>
    PrometheusTasks.value(
      await PrometheusTasks.query((q) => q.url(PROMETHEUS).query(P99_LATENCY)),
    )
  )
)

Failure messages pass through the run's redactor, so a secret parameter in a URL or a query is masked before it reaches a log. A ratio query over a window with no traffic is 0/0, which Prometheus reports as NaN — and a NaN sample fails. Guard such a query so quiet means healthy.

.connection(...) hands the rest of the connection to @zuke/prometheus's own settings, so a canary reads a managed Prometheus with the cloud's own credentials — no gcloud, az or aws CLI on the runner:

// Amazon Managed Service for Prometheus, signed with SigV4.
prometheus((p) =>
  p.name("error ratio")
    .url("https://aps-workspaces.us-east-1.amazonaws.com/workspaces/ws-1234")
    .query(ERROR_RATIO).max(0.01)
    .connection((c) => c.sigv4((a) => a.region("us-east-1")))
)

// Google Cloud Managed Service for Prometheus, through Application Default
// Credentials; c.azure() does the same for Azure Monitor.
prometheus((p) =>
  p.name("error ratio")
    .url("https://monitoring.googleapis.com/v1/projects/my-project/location/global/prometheus")
    .query(ERROR_RATIO).max(0.01)
    .connection((c) => c.google())
)

The approval gate

.approval("canary-approved") adds rollout.approve before the promotion: the run suspends until the signal arrives, and .approvalTimeout("24h") bounds the wait — a timeout rolls the canary back.

zuke resume <run-id> --signal canary-approved

The lock

.lock((l) => l.lockKey("deploy", "api").withTtl("24h")) takes a core lock held from staging until the run settles, across every bake and approval wait, so a second rollout of the same service is refused until the first is promoted or rolled back. Set the TTL to cover the longest the run can stay parked: it is how long a parked rollout keeps the lock with no process alive.

Rolling back

There is one rollback, rollout.abort, and every path reaches it:

Two cases deliberately do not roll back:

• Another rollout lost the lock. It fails before its stage body runs, so it touches nothing; the candidate belongs to the rollout holding the lock.
• The release was already promoted. A cancellation later in the same run leaves the promoted release alone instead of returning traffic to a version promotion already retired.