Skip to content

App Releases

changie gives each first-party app under apps/ its version and its changelog. Two apps use it: dns-control and hl-assets-uploader. A pull request that changes an app adds a changelog fragment for that app. A fragment is a small YAML file that holds one changelog entry. A release is a pull request that collects the fragments of one app into a new version.

File What it holds Written by
.changie.yaml The two projects, the kinds and the version bump of each kind A person
.changes/unreleased/*.yaml One fragment for each entry that no release holds yet changie new
.changes/<app>/v<version>.md The notes of one released version scripts/release.sh
apps/<app>/CHANGELOG.md All versions of the app, newest first scripts/release.sh
apps/<app>/VERSION The version that the image and the binaries carry scripts/release.sh

Do not edit apps/<app>/VERSION, apps/<app>/CHANGELOG.md or a file under .changes/<app>/ by hand. changie finds the current version in .changes/<app>/, and changie merge writes the two app files from it. A hand edit makes the files disagree.

Both apps are below version 1.0.0. Semantic versioning lets a 0.x version break compatibility in a minor version, so no kind bumps the major version.

Kind Use it for Bump
Added A new feature minor
Changed A change to existing behavior minor
Deprecated A feature that a later version removes minor
Removed A removed feature minor
Fixed A bug fix patch
Security A fix for a vulnerability patch
Dependencies A dependency update patch
Internal A build or test change that does not change the shipped software none

A release takes the largest bump of its fragments. An Internal fragment appears in the notes and bumps nothing, so a release needs at least one fragment of another kind.

  1. Run this command from the repository root. Replace <app>, <kind> and <text>:

    Terminal window
    changie new --projects <app> --kind <kind> --body "<text>"
  2. Commit the new file under .changes/unreleased/ with the change.

If the pull request changes both apps, write --projects dns-control,hl-assets-uploader. changie then writes one fragment for each app. Write the body as one sentence that a reader of the changelog understands without the pull request.

A change to a Markdown file under apps/<app>/ needs no fragment.

The pr-gates member changelog-fragment runs tools/pr-gates/changelog-fragment.sh. The member fails a pull request that changes files under apps/<app>/ and adds no fragment for <app>. A release pull request needs no fragment, because it adds .changes/<app>/v<version>.md.

If the member fails, read its log. Each FAIL line names an app. Add a fragment for each named app, as in Add a fragment to a pull request.

To run the check before you push, run this from the repository root:

Terminal window
tools/pr-gates/changelog-fragment.sh origin/main

The script reads the commits of your branch. It does not read uncommitted files.

An operator or an agent opens the release pull request. Only an operator merges it.

  1. Make sure that the working tree of your checkout is clean.
  2. Run git fetch origin and then git switch --detach origin/main.
  3. Run scripts/release.sh --dry-run <app>. It prints the release branch and the next version.
  4. Run scripts/release.sh <app>. It opens the release pull request.
  5. Read the pull request. Its body holds the notes of the new version.
  6. Merge the pull request.

The script runs changie batch auto and changie merge for the app. It commits the result on the branch release/<app>-v<version>, pushes the branch and opens the pull request with gh. The fragments of the other app stay in .changes/unreleased/.

If the script prints no unreleased changes found for automatic bumping, main holds no fragment for the app that bumps the version. There is nothing to release.

If the script prints changie has <app>/v<a>, but apps/<app>/VERSION holds <b>, VERSION changed outside changie. If version <b> is published, add .changes/<app>/v<b>.md with a heading ## v<b> - <date> and one line of notes. Then run changie merge and merge that change before the release.

When a release pull request merges, the release workflow (.github/workflows/release.yml) finds the new file .changes/<app>/v<version>.md and pushes the tag <app>/v<version> on the merge commit. If the tag exists on another commit, the run fails. If a re-run finds the tag on the same commit, it continues.

For dns-control, the same run builds the release with GoReleaser Pro from apps/dns-control/.goreleaser.yaml:

  • The image ghcr.io/fzymgc-house/dns-control:<version> for amd64 and arm64, also tagged latest, signed with cosign keyless.
  • The archive dns-node-worker_v<version>_linux_arm64.tar.gz and checksums.txt.
  • The notes from .changes/dns-control/v<version>.md.

Before GoReleaser starts, the job makes sure that the tag <version> does not exist in GHCR. If the tag exists, the run fails, because a published version never changes. A re-run of a run that published <version> also fails at this step.

GoReleaser creates a draft release. The workflow pulls the image and makes sure that it reports the version. Only then does it publish the release. A merge that is not a release pull request publishes nothing for dns-control.

If a run fails after GoReleaser pushed the image, a re-run cannot finish the release. Finish it by hand. The first command must print <version>:

Terminal window
docker run --rm --pull always ghcr.io/fzymgc-house/dns-control:<version> -version
gh release edit dns-control/v<version> --draft=false

For hl-assets-uploader, the same run calls build-hl-assets-uploader.yaml, because the image links libheif with cgo and keeps its Dockerfile build. Its check job stops the run before any push if the version differs from apps/hl-assets-uploader/VERSION or the tag <version> exists in GHCR. Each architecture then builds natively, and the Dockerfile runs the Go suite and the HEIC smoke test. The publish job merges both images into ghcr.io/fzymgc-house/hl-assets-uploader:<version> and latest, and pulls the image to make sure that it reports the version. The summary of that job shows the index digest. Within five minutes, Argo CD Image Updater opens a pull request that moves the pin of hl-assets to the new version. A merge that is not a release pull request publishes nothing for either app.

To make sure that a dns-control release is complete, run:

Terminal window
gh release view dns-control/v<version> --json assets --jq '[.assets[].name]'
cosign verify ghcr.io/fzymgc-house/dns-control:<version> \
--certificate-identity 'https://github.com/fzymgc-house/selfhosted-cluster/.github/workflows/release.yml@refs/heads/main' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com

The first command lists the tarball and checksums.txt. The second command exits 0.

The release workflow reads the license key from the Actions secret GORELEASER_KEY. Vault holds the same key at secret/fzymgc-house/cluster/goreleaser, field license_key. If the license lapses, the release job fails before it publishes anything.

To change the key, run this on a host that has vault and gh logged in:

Terminal window
read -rs -p 'GoReleaser Pro key: ' GR_KEY && echo
vault kv put secret/fzymgc-house/cluster/goreleaser license_key="$GR_KEY"
printf '%s' "$GR_KEY" | gh secret set GORELEASER_KEY --repo fzymgc-house/selfhosted-cluster
unset GR_KEY

Argo CD Image Updater moves the one pin of dns-control. Within five minutes of a release, it opens one pull request that sets newTag in argocd/app-configs/dns-control-image/kustomization.yaml. That Component pins both the Deployment of dns-control and the webhook of external-dns. The pull request requests a review from seanb4t, and it does not merge without a person, because dns-control is the only writer of pdns.

  1. Make sure that the signature of the new version is valid. Use the cosign verify command in The image and its pin.
  2. If the release needs a new configuration key or a changed variable, push that change to the branch of the updater pull request. Do this before the merge.
  3. If the release changes the node worker, open a pull request that sets dns_node_worker_version in ansible/roles/dns-node-worker/defaults/main.yml to the new version.
  4. In a quiet window, merge the updater pull request. ArgoCD rolls dns-control and the webhook of external-dns in the same sync. external-dns retries the webhook at its next interval of one minute.
  5. If you opened the pull request of step 3, merge it.

Step 2 exists because dns-control refuses a configuration key that it does not know. A new key therefore cannot merge before the version that reads it. Reloader restarts the pod on a change of its ConfigMap, and the old version then stops at start. Image Updater leaves a commit that you push to its branch alone. It finds its pin on the branch and pushes nothing.

If you changed dns_node_worker_version, converge the node worker on one resolver node per window. Run these two commands for one node, and then for the other node in a later window:

Terminal window
scripts/dns-playbook.sh --limit <node> --tags dns-node-worker
scripts/dns-playbook.sh --limit <node> --tags dns-node-worker-verify

The second command makes sure that the installed worker is the pinned release.