Skip to content
fzymgc-house Cluster Docs

GitHub Token Operations

Guide to GitHub token creation and management for cluster services.

Quick Reference

Section titled “Quick Reference”
Property Value
Actions Runner Token Path secret/fzymgc-house/cluster/github
Token Key actions_runner_token
Runner Namespace arc-runners (controller and listener: arc-systems)
Runner Label fzymgc-house-cluster-runners

Token Types

Section titled “Token Types”
Type Use Case Expiry Security
Fine-grained PAT Service automation Configurable Higher (recommended)
Classic PAT Legacy integrations Configurable Lower
GitHub App Org-level automation No expiry Highest

Required Tokens

Section titled “Required Tokens”

Actions Runner Controller

Section titled “Actions Runner Controller”

Repository access for self-hosted runners.

  • Scope: repo, workflow
  • Stored: secret/fzymgc-house/cluster/github

Successful registration does not guarantee that a runner can receive jobs. On 2026-09-07, GitHub rejected the deployed runner with Runner version v2.330.0 is deprecated and cannot receive messages, while ARC’s controller and listener remained healthy. This is an image-version failure, not a reason to rotate the registration token.

The custom image’s default version is ARG RUNNER_VERSION in images/actions-runner/Dockerfile. The Build Actions Runner Image workflow uses that default unless its manual input overrides it. After a successful multi-architecture build, update the image pin in argocd/app-configs/arc-runners/values.yaml through a PR. ArgoCD deploys the change; confirm that a queued job starts on the new runner version. Building an image alone does not update the cluster.

ArgoCD

Section titled “ArgoCD”

Repository access for GitOps sync.

  • Scope: Repository read
  • Stored: secret/fzymgc-house/cluster/argocd

HCP Terraform

Section titled “HCP Terraform”

VCS integration for speculative plans.

  • Type: GitHub App
  • Configuration: HCP Terraform settings

Token Creation

Section titled “Token Creation”

Option 1: Personal Access Token (Classic)

Section titled “Option 1: Personal Access Token (Classic)”

  1. Navigate to GitHub Settings

  2. Generate New Token

    • Click “Generate new token” > “Generate new token (classic)”
    • Note: Give it a descriptive name like actions-runner-controller-selfhosted-cluster
    • Expiration: Recommended: 90 days (you’ll need to rotate it)

  3. Select Scopes

    For repository-level runners, select these scopes:

    • repo (Full control of private repositories)
    • workflow (Update GitHub Action workflows)

    Important: These are the ONLY two scopes needed.

  4. Generate and Copy Token

    • Click “Generate token” at the bottom
    • IMPORTANT: Copy the token immediately - you won’t see it again
    • Token format: ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Section titled “Option 2: Fine-Grained Personal Access Token (Recommended)”

  1. Navigate to Fine-Grained Tokens

  2. Configure Token

    • Token name: actions-runner-controller-selfhosted
    • Expiration: 90 days (recommended)
    • Description: Self-hosted GitHub Actions runner
    • Resource owner: fzymgc-house

  3. Repository Access

    • Select: Only select repositories
    • Choose: fzymgc-house/selfhosted-cluster

  4. Permissions

    Under “Repository permissions”:

    • Actions: Read and write
    • Contents: Read-only
    • Metadata: Read-only (automatically selected)
    • Workflows: Read and write

  5. Generate and Copy Token

    • Click “Generate token”
    • Copy the token: github_pat_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Store Token in Vault

Section titled “Store Token in Vault”
Terminal window
# Make sure you're authenticated to Vault
vault token lookup


# Store the token
vault kv put secret/fzymgc-house/cluster/github \
  actions_runner_token="ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"


# Verify it was stored
vault kv get secret/fzymgc-house/cluster/github


# Get the actual token value (if needed for debugging)
vault kv get -field=windmill_actions_runner_token secret/fzymgc-house/cluster/github

Token Rotation

Section titled “Token Rotation”

Since tokens expire, you’ll need to rotate them periodically:

  1. Generate a new token following the same steps

  2. Update Vault:

    Terminal window
    vault kv patch secret/fzymgc-house/cluster/github \
      actions_runner_token="<new-token>"

  3. ExternalSecret will automatically sync the new token

  4. Runner pods will automatically use the new token

Verify Runner Deployment

Section titled “Verify Runner Deployment”

After storing the token:

Terminal window
# Check if ExternalSecret synced the token
kubectl --context fzymgc-house get externalsecret github-token -n actions-runner-system
kubectl --context fzymgc-house get secret github-token -n actions-runner-system


# Check controller deployment
kubectl --context fzymgc-house get pods -n actions-runner-system


# Verify runner registered with GitHub
kubectl --context fzymgc-house get runnerdeployment -n actions-runner-system
kubectl --context fzymgc-house describe runnerdeployment -n actions-runner-system

Verify on GitHub

Section titled “Verify on GitHub”

Check that the runner appears in GitHub:

  1. Go to: https://github.com/fzymgc-house/selfhosted-cluster/settings/actions/runners
  2. You should see a runner listed with label: selfhosted-cluster
  3. Status should show as “Idle” (green)

GitHub App: fzymgc-terraform (Terraform write identity)

Section titled “GitHub App: fzymgc-terraform (Terraform write identity)”

Unlike everything above, this credential is a GitHub App, not a token. It is Terraform’s provider auth for github_repository_ruleset.main_branch_protection in tf/core-services — the resource that manages branch protection on main. Vault is its only home: Terraform reads it ephemerally at plan time and never writes it, so the App is created, installed and seeded by hand, once, following the five steps below.

Do this before merging the Terraform PR that adds the ruleset resource. The provider’s Vault read fires during plan, so the secret must already exist and be readable or the plan fails outright.

Both prerequisites, in order — the plan needs the secret and the permission to read it:

  1. This runbook seeds secret/fzymgc-house/infrastructure/github-terraform.
  2. tf/vault must have applied the terraform-core-services-admin grant (main-cluster-vault has auto_apply = true; it applied ~50s after merge when this was first done). Until it has, the ephemeral read 403s — and the failure presents as a missing secret rather than a missing grant, which is the wrong place to start debugging.

rg 'run_trigger' across tf/ returns nothing, so a single merge touching both tf/vault and tf/core-services queues two runs with no ordering guarantee between them. Land the grant alone and first.

  1. Create the App

    • Go to: https://github.com/organizations/fzymgc-house/settings/apps/new
    • Or: GitHub > fzymgc-house org > Settings > Developer settings > GitHub Apps > New GitHub App
    • GitHub App name: fzymgc-terraform
    • Homepage URL: the repository URL is fine; nothing consumes it
    • Uncheck Active under “Webhook” — Terraform never receives callbacks
    • Record the App slug and the Client ID shown on the App’s settings page. The Client ID — not the numeric App ID — is what Vault stores and what the provider sends as the JWT iss claim

  2. Set repository permissions

    • Under “Repository permissions”, set Administration: Read & write
    • Set no other permission. Metadata: Read-only is selected automatically and is expected
    • Do not soften this to Read-only. GitHub’s permissions reference places all four ruleset endpoints — including GET /repos/{owner}/{repo}/rulesets — under repository Administration at write access. A read-only App 403s on the import’s GET, and the failure reads like an installation-scoping mistake rather than a permission one. This App does not only read the ruleset; it applies it.

  3. Install on one repository

    • From the App’s settings page: “Install App” > fzymgc-house
    • Choose Only select repositories, and select fzymgc-house/selfhosted-cluster only
    • This App holds write authority over the protection that gates every other change in this repository; widening the installation re-creates the exact blast radius the org-wide Renovate App was rejected for
    • Record the installation ID — the trailing number in the URL you land on: https://github.com/organizations/fzymgc-house/settings/installations/<installation-id>

  4. Generate a private key

    • On the App’s settings page: “Private keys” > Generate a private key
    • The .pem downloads once and GitHub never shows it again. Losing it means generating a new key and re-writing Vault — not recreating the App
    • Leave the file where it landed for step 5, then delete it

  5. Write the three keys to Vault

    Terminal window
    # Make sure you're authenticated to Vault
    vault token lookup
    

    PEM="$HOME/Downloads/fzymgc-terraform.<date>.private-key.pem"
    

    # `private_key=@<file>` makes the Vault CLI read the PEM from the file. Never paste the key
    # as an inline literal — that puts it in shell history and in the process table.
    vault kv put secret/fzymgc-house/infrastructure/github-terraform \
      client_id="<client-id-from-step-1>" \
      installation_id="<installation-id-from-step-3>" \
      private_key=@"$PEM"
    

    # Verify with METADATA ONLY. Do not `vault kv get` this path — that prints the private key.
    vault kv metadata get secret/fzymgc-house/infrastructure/github-terraform
    

    # Destroy the local copy immediately.
    rm -f "$PEM"

    Expected: current_version 1, destroyed false. The key must not reach an HCP Terraform workspace variable, a PR body, a commit, or a shell-history-visible inline argument. The terraform-core-services-admin policy in tf/vault grants the workspace read on this path and nothing more — see the registry row in Secrets Reference.

    That sentence is about one policy, not about the path. admin and infrastructure-developer (humans) also reach this secret through broad globs. The cluster does not: the path is outside cluster/, the only prefix the cluster-wide vault store may read, so no ExternalSecret can copy the key. Keep it outside cluster/ (#1980).

Troubleshooting

Section titled “Troubleshooting”

Token Not Working

Section titled “Token Not Working”
Terminal window
# Check controller logs
kubectl --context fzymgc-house logs -n actions-runner-system \
  -l app.kubernetes.io/name=actions-runner-controller --tail=100


# Common errors:
# - "401 Unauthorized": Token invalid or expired
# - "403 Forbidden": Insufficient permissions
# - "404 Not Found": Repository access not granted

Runner Not Appearing in GitHub

Section titled “Runner Not Appearing in GitHub”
  1. Check token scopes: Must have repo and workflow (classic) or equivalent fine-grained permissions
  2. Verify repository access: Token must have access to fzymgc-house/selfhosted-cluster
  3. Check controller status: kubectl get pods -n actions-runner-system
  4. Review logs: Look for authentication errors in controller logs

ExternalSecret Not Syncing

Section titled “ExternalSecret Not Syncing”
Terminal window
# Check ExternalSecret status
kubectl --context fzymgc-house describe externalsecret github-token -n actions-runner-system


# Common issues:
# - Vault path wrong: Should be secret/fzymgc-house/cluster/github
# - Vault key wrong: Should be actions_runner_token
# - ClusterSecretStore not configured: Check 'vault' ClusterSecretStore exists

403 Resource not accessible by integration (GitHub App auth)

Section titled “403 Resource not accessible by integration (GitHub App auth)”

Two distinct causes produce this identical message from the github Terraform provider. Check them in this order:

  1. The App has Administration: Read-only. All four ruleset endpoints — the two GETs included — sit under repository Administration at write access, so a read-only App 403s even on a plan that only reads. Fix: set Administration: Read & write on the App, then accept the permission change on the installation.
  2. The provider block omits owner. Under App auth the provider cannot infer the owner from an installation token the way it can from a PAT, and returns this same 403. Fix: set owner = "fzymgc-house" in the provider "github" block.

If both are correct, confirm the installation actually covers fzymgc-house/selfhosted-cluster: an App installed with “Only select repositories” and the wrong repository selected fails the same way.

Token Comparison

Section titled “Token Comparison”
Feature Classic PAT Fine-Grained PAT
Scope All repos user has access to Specific repositories only
Permissions Broad (repo, workflow) Granular (Actions, Workflows, etc.)
Expiration Custom (max 1 year) Custom (max 1 year)
Security Lower Higher (recommended)
Setup Simpler More complex

Recommendation: Use Fine-Grained PAT for better security.

Security Best Practices

Section titled “Security Best Practices”
  • Token Storage: Never commit tokens to Git. Always use Vault.
  • Token Scope: Use minimum required scopes. Fine-grained tokens are more secure.
  • Token Expiration: Set reasonable expiration (90 days recommended).
  • Token Rotation: Have a process to rotate before expiration.
  • Access Control: Limit who can access the Vault secret.
  • Rotate immediately if compromised

See Also

Section titled “See Also”