Mend Renovate CE — Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Migrate this repo (and, later, other orgs) off the Mend-hosted Renovate cloud app onto a self-hosted Mend Renovate Community Edition (CE) server running in-cluster via GitOps.

Architecture: An ArgoCD Application (app-of-apps template under argocd/cluster-app/templates/) renders the official mend-renovate-ce Helm chart (Source A) plus in-repo cluster wiring (Source B: ExternalSecret, NetworkPolicy, optional IngressRoute). CE authenticates as one public GitHub App installed per-org, runs both the autodiscovery scheduler and the webhook handler (public /webhook via the Cloudflare-tunnel webhook_services map), and stores its rebuildable job/registry cache in SQLite on a Longhorn PVC.

Tech Stack: ArgoCD, Helm (mend-renovate-ce chart v14.6.x), Kubernetes (k3s), Vault + External Secrets Operator, Cilium NetworkPolicy, Traefik IngressRoute, Terraform (Cloudflare tunnel via HCP Terraform), Longhorn.

Design bead: hl-2ctz · Spec: docs/engineering/specs/2026-06-14-renovate-ce-design.md

Conventions for this plan

This is an infrastructure/GitOps plan: the TDD red/green loop maps to artifact → validate → reconcile → verify. “Tests” are lint gates (yamllint, rumdl), render checks (helm template, kubectl --dry-run, argocd app diff), terraform plan, and live health/behavior probes after sync. Commit after each task (jj commit -m '…').

Secret-key correction (supersedes the spec’s extraEnvFromSecrets/UPPERCASE note): the chart provides a native renovate.existingSecret knob expecting a Secret whose keys are the camelCase mendRnv* names (mendRnvLicenseKey, mendRnvGithubAppId, mendRnvGithubAppKey, mendRnvWebhookSecret, mendRnvServerApiSecret, optional githubComToken). This plan uses existingSecret; the ExternalSecret therefore templates camelCase keys. Source: helm show values mend-renovate-ce-ee/mend-renovate-ce (chart 14.6.x).

File structure

Path	Responsibility	Action
`argocd/cluster-app/templates/renovate.yaml`	The ArgoCD `Application` (multi-source: chart + app-config)	Create
`argocd/app-configs/renovate/kustomization.yaml`	Kustomize entrypoint for Source B cluster wiring	Create
`argocd/app-configs/renovate/external-secret.yaml`	Vault → `renovate-app` Secret (camelCase `mendRnv*` keys)	Create
`argocd/app-configs/renovate/networkpolicy.yaml`	Ingress from cloudflared; egress DNS/GitHub/registries	Create
`argocd/app-configs/renovate/ingressroute.yaml`	Optional internal Traefik route to `/health` + admin API	Create
`tf/cloudflare/variables.tf`	Add `renovate` entry to `webhook_services` map	Modify
`.github/renovate.json`	Existing repo config — preserved unchanged	(none)
Vault `fzymgc-house/cluster/renovate`	License key, GitHub App private key, webhook secret, API secret	Create (out-of-band)

Naming locked for determinism: fullnameOverride: renovate-ce → Service renovate-ce in namespace renovate → cluster DNS renovate-ce.renovate.svc.cluster.local (Service port 80 → container 8080); public webhook host renovate-wh.fzymgc.net (derived ${svc}-wh.fzymgc.net in tf/cloudflare/tunnel.tf).

Task 1: Provision the GitHub App and license key (out-of-band)

Files: none (GitHub UI + Mend registration). Produces the credential values consumed in Task 2.

Step 1: Create a public GitHub App

In GitHub → Settings → Developer settings → GitHub Apps → New GitHub App:

Name: e.g. fzymgc-renovate (the bot identity becomes fzymgc-renovate[bot]).
Homepage URL: https://cluster-docs.docs.fzymgc.house.
Webhook: Active, URL https://renovate-wh.fzymgc.net/webhook, secret = a generated random string (save it for Task 2).
Permissions (least-privilege self-hosted Renovate): Repository → Contents: Read & write; Pull requests: Read & write; Issues: Read & write; Metadata: Read; Workflows: Read & write (needed when bumping .github/workflows); Administration: Read (branch-protection checks). Organization → none.
Subscribe to events: push, pull_request, installation, installation_repositories, issue_comment (dependency-dashboard checkbox), check_run/check_suite (automerge gating).
“Where can this app be installed?”: Any account (public — required for multi-org per D4).
Step 2: Generate and download the App private key (PEM)

In the App settings → Private keys → Generate a private key. Save the downloaded .pem (used in Task 2). Note the App ID (top of the App settings page).

Step 3: Obtain the free unlimited CE license key

Register at https://www.mend.io/renovate-community/ for a free unrestricted-repo license key (D6). Save the returned key string.

Step 4: Verify

App page shows App ID, a webhook configured to renovate-wh.fzymgc.net/webhook, and one private key listed. Do not install the App on any org yet — installation happens at cutover (Task 9) to avoid the cloud app + CE double-managing repos.

Step 5: Commit — nothing to commit (no repo files changed).

Task 2: Store credentials in Vault + update policy (out-of-band)

Files: Vault path fzymgc-house/cluster/renovate; Vault policy for the cluster ExternalSecrets role.

Step 1: Write the secret to Vault

# App ID is NON-secret (per spec + fovea): it goes in the Application valuesObject
# (Task 7), NOT in Vault. Only true secrets are stored here.
vault kv put fzymgc-house/cluster/renovate \
  mend_license_key="<license-key-from-task-1>" \
  github_app_private_key=@/path/to/downloaded.pem \
  github_webhook_secret="<webhook-secret-from-task-1>" \
  server_api_secret="$(openssl rand -hex 32)"

Step 2: Grant the ExternalSecrets Vault role read on the new path

Add the path to the cluster secrets policy (mirror an existing fzymgc-house/cluster/<app> grant; see docs/reference/secrets.md and tf/ Vault policy files):

path "fzymgc-house/data/cluster/renovate" { capabilities = ["read"] }
path "fzymgc-house/metadata/cluster/renovate" { capabilities = ["read"] }

Apply via the Vault policy workflow used for other paths (Terraform tf/vault or the documented policy update).

Step 3: Verify

vault kv get fzymgc-house/cluster/renovate

Expected: all five keys present; github_app_private_key shows a PEM block.

Step 4: Commit — commit only the Vault policy change if it is file-managed (Terraform). The secret values themselves are never committed.

Task 3: ExternalSecret (Source B)

Files:

Create: argocd/app-configs/renovate/external-secret.yaml
Step 1: Write the ExternalSecret (camelCase keys for existingSecret)

---
# Renovate CE credentials. Keys are the chart's camelCase `mendRnv*` names so the
# Helm chart can consume them via `renovate.existingSecret: renovate-app`.
# Mirrors the fovea Vault→ExternalSecret pattern (refreshPolicy: Periodic).
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: renovate-app
  namespace: renovate
spec:
  refreshPolicy: Periodic
  refreshInterval: 5m
  secretStoreRef:
    name: vault
    kind: ClusterSecretStore
  target:
    name: renovate-app
    creationPolicy: Owner
    deletionPolicy: Delete
    template:
      type: Opaque
      data:
        # App ID is non-secret → set in valuesObject (Task 7), not here.
        mendRnvLicenseKey: "{{ .mend_license_key }}"
        mendRnvGithubAppKey: "{{ .github_app_private_key }}"
        mendRnvWebhookSecret: "{{ .github_webhook_secret }}"
        mendRnvServerApiSecret: "{{ .server_api_secret }}"
  data:
    - secretKey: mend_license_key
      remoteRef: { key: fzymgc-house/cluster/renovate, property: mend_license_key }
    - secretKey: github_app_private_key
      remoteRef: { key: fzymgc-house/cluster/renovate, property: github_app_private_key }
    - secretKey: github_webhook_secret
      remoteRef: { key: fzymgc-house/cluster/renovate, property: github_webhook_secret }
    - secretKey: server_api_secret
      remoteRef: { key: fzymgc-house/cluster/renovate, property: server_api_secret }

Step 2: Lint

Run: yamllint argocd/app-configs/renovate/external-secret.yaml Expected: no errors.

Step 3: Commit

jj commit -m "feat(renovate): ExternalSecret for CE credentials [hl-2ctz]"

Task 4: NetworkPolicy (Source B)

Files:

Create: argocd/app-configs/renovate/networkpolicy.yaml
Step 1: Write the NetworkPolicy

Ingress: only the cloudflared namespace may reach the webhook port (container 8080). Egress: DNS, plus HTTPS to GitHub + package registries (kept broad to 443 since Renovate fans out to many registries; tighten later if desired). The cloudflared namespace is cloudflared (the app-config dir is cloudflared-main, but argocd/app-configs/cloudflared-main/namespace.yaml declares name: cloudflared).

---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: renovate-ce
  namespace: renovate
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: mend-renovate-ce
  policyTypes: [Ingress, Egress]
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: cloudflared
      ports:
        - protocol: TCP
          port: 8080
  egress:
    - ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53
    - ports:
        - protocol: TCP
          port: 443
    - ports:
        - protocol: TCP
          port: 6443  # k8s API (ExternalSecrets / chart health), if required

Step 2: Validate the pod selector matches the chart’s pod labels

Run: helm template renovate-ce mend-renovate-ce-ee/mend-renovate-ce --set fullnameOverride=renovate-ce | rg -A2 'labels:' | rg 'app.kubernetes.io/name' Expected: app.kubernetes.io/name: mend-renovate-ce (adjust the podSelector if the chart emits a different label).

Step 3: Lint

Run: yamllint argocd/app-configs/renovate/networkpolicy.yaml Expected: no errors.

Step 4: Commit

jj commit -m "feat(renovate): NetworkPolicy (cloudflared ingress, egress to GitHub/registries) [hl-2ctz]"

Task 5: Optional internal IngressRoute (Source B)

Files:

Create: argocd/app-configs/renovate/ingressroute.yaml

Include only if internal /health + admin-API access is wanted day-one (spec open question). Skip otherwise.

Step 1: Write the IngressRoute (mirrors argocd/app-configs/octopus/ingress.yaml)

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: renovate
  namespace: renovate
  annotations:
    router-hosts.fzymgc.house/enabled: "true"
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`renovate.fzymgc.house`)
      kind: Rule
      services:
        - name: renovate-ce
          port: 80
  tls:
    secretName: renovate-tls

Step 2: Lint + commit

Run: yamllint argocd/app-configs/renovate/ingressroute.yaml jj commit -m "feat(renovate): internal IngressRoute for health/admin [hl-2ctz]"

Task 6: Kustomization (Source B entrypoint)

Files:

Create: argocd/app-configs/renovate/kustomization.yaml
Step 1: Write the kustomization (mirrors argocd/app-configs/fovea/kustomization.yaml)

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: renovate

# Source B of the multi-source renovate Application. Workloads are rendered by the
# mend-renovate-ce Helm chart (Source A). This path holds only the cluster wiring
# the chart doesn't own. The namespace is created by the Application's
# CreateNamespace=true syncOption (matching fovea) — no namespace.yaml here.
resources:
  - external-secret.yaml
  - networkpolicy.yaml
  # - ingressroute.yaml   # uncomment if Task 5 included

Step 2: Validate the kustomize build

Run: kubectl kustomize argocd/app-configs/renovate Expected: renders the ExternalSecret + NetworkPolicy (+ IngressRoute) with namespace: renovate.

Step 3: Commit

jj commit -m "feat(renovate): kustomization for app-config wiring [hl-2ctz]"

Task 7: ArgoCD Application (Source A chart + Source B wiring)

Files:

Create: argocd/cluster-app/templates/renovate.yaml
Step 1: Write the Application (multi-source, mirrors argocd/cluster-app/templates/fovea.yaml)

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: renovate
  namespace: argocd
  # Self-hosted Mend Renovate CE. Multi-source: A = mend-renovate-ce Helm chart
  # (workloads), B = app-config cluster wiring (ExternalSecret, NetworkPolicy).
  # finalizer ensures a clean prune cascade (fovea/agent-memory precedent).
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: core-services
  sources:
    # Source A — workloads from the Mend Renovate CE Helm chart.
    - repoURL: https://mend.github.io/renovate-ce-ee
      chart: mend-renovate-ce
      targetRevision: 14.6.2  # pin; Renovate will PR bumps via this repo's helm manager
      helm:
        releaseName: renovate-ce
        valuesObject:
          fullnameOverride: renovate-ce
          image:
            repository: ghcr.io/mend/renovate-ce
            tag: "14.6.0"
            useFull: true            # full image: all package-manager toolchains
            pullPolicy: IfNotPresent
          renovate:
            existingSecret: renovate-app   # camelCase mendRnv* keys (Task 3)
            mendRnvAcceptTos: "y"          # Mend ToS consent
            mendRnvPlatform: github
            mendRnvGithubAppId: "<APP_ID>" # quoted: Helm coerces bare ints (fovea note)
            mendRnvApiEnabled: "true"      # admin/system APIs (mendRnvAdminApiEnabled is deprecated)
            mendRnvAutoDiscoverFilter: "fzymgc-house/*"  # scope; widen per-org at rollout
            logFormat: json                # ClickStack/Vector ingestion
          cachePersistence:
            enabled: true                  # PVC + auto SQLite persistence
            storageClass: longhorn
            size: 1Gi
            accessModes: [ReadWriteOnce]
          postgresql:
            enabled: false                 # SQLite (D5)
          service:
            type: ClusterIP
            ports:
              http: 80
          podSecurityContext:
            fsGroup: 12021                 # SQLite uid/rwx footgun on the PVC
            runAsUser: 12021
            runAsGroup: 12021
          resources:
            requests: { cpu: 100m, memory: 512Mi }
            limits: { memory: 2Gi }
    # Source B — cluster wiring that stays in this repo.
    - repoURL: https://github.com/fzymgc-house/selfhosted-cluster
      targetRevision: HEAD
      path: argocd/app-configs/renovate
  destination:
    server: https://kubernetes.default.svc
    namespace: renovate
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - ServerSideApply=true
      - CreateNamespace=true
  ignoreDifferences:
    - kind: ExternalSecret
      group: external-secrets.io
      jqPathExpressions:
        - .spec.data[].remoteRef.conversionStrategy
        - .spec.data[].remoteRef.decodingStrategy
        - .spec.data[].remoteRef.metadataPolicy

Step 2: Render the chart with these values (catch template errors early)

Run:

helm template renovate-ce mend-renovate-ce-ee/mend-renovate-ce \
  --version 14.6.2 \
  --set fullnameOverride=renovate-ce \
  --set renovate.existingSecret=renovate-app \
  --set renovate.mendRnvAcceptTos=y \
  --set renovate.mendRnvPlatform=github \
  --set cachePersistence.enabled=true --set cachePersistence.storageClass=longhorn \
  --set postgresql.enabled=false | rg -n 'kind:|name:|fsGroup|existingSecret|PersistentVolumeClaim|port'

Expected: a Deployment, Service (port 80), and PVC render; the Service name is renovate-ce; no template errors.

Step 3: Lint

Run: yamllint argocd/cluster-app/templates/renovate.yaml Expected: no errors. (If cluster-app lints as a Helm template dir, validate with helm lint argocd/cluster-app instead.)

Step 4: Commit

jj commit -m "feat(renovate): ArgoCD Application for self-hosted CE [hl-2ctz]"

Task 8: Exclude the `renovate` namespace from Velero backups

Files:

Modify: argocd/app-configs/velero/backup-schedule.yaml

Per the cluster’s exclude-only backup strategy (argocd/CLAUDE.md), stateless workloads whose storage is rebuildable must be excluded. CE’s PVC is a rebuildable cache (job queue + installation registry, reconstructed from GitHub), so the renovate namespace belongs in excludedNamespaces.

Step 1: Add renovate to the exclusion list

In excludedNamespaces, under a telemetry/ephemeral-style grouping:

    - renovate            # Stateless-ish: SQLite cache is rebuildable from GitHub

Step 2: Lint + commit

Run: yamllint argocd/app-configs/velero/backup-schedule.yaml jj commit -m "feat(renovate): exclude renovate ns from velero (rebuildable cache) [hl-2ctz]"

Task 9: Cloudflare tunnel webhook entry (Terraform)

Files:

Modify: tf/cloudflare/variables.tf (the webhook_services default map)
Step 1: Add the renovate entry

In the webhook_services variable default block, after the fovea entry:

    renovate = {
      # Renovate CE self-hosted — GitHub App webhook.
      # GitHub POSTs to /webhook, forwarded as-is to the renovate-ce Service (:80 -> :8080).
      service_url = "http://renovate-ce.renovate.svc.cluster.local"
    }

This auto-creates renovate-wh.fzymgc.net (DNS record ${each.key}-wh) routing to the tunnel; origin_server_name derives from service_url (renovate-ce.renovate.svc.cluster.local).

Step 2: Validate

Run: terraform -chdir=tf/cloudflare fmt -check && terraform -chdir=tf/cloudflare validate Expected: formatted, valid. (Plan runs as a speculative plan on the PR.)

Step 3: Commit

jj commit -m "feat(renovate): expose CE webhook via cloudflare tunnel [hl-2ctz]"

Apply is GitOps: the main-cluster-cloudflare HCP TF workspace (auto_apply = true) applies on PR merge. Review the speculative plan on the PR (production-facing module — docs/operations/hcp-terraform.md). tf/cloudflare/MANUAL_APPLY.md is break-glass only.

Task 10: Cutover (out-of-band, after the above merges and CE is healthy)

Files: none in this plan (GitHub UI actions); .github/renovate.json stays unchanged.

Prerequisite checks (spec Impl step 0): confirm the current runner is the Mend cloud app; confirm the self-hosted App can satisfy main branch protection (write + required checks); env-var names re-verified.

Step 1: Install the self-hosted App on this repo

Install fzymgc-renovate on fzymgc-house/selfhosted-cluster (selected-repos scope for the first cutover).

Step 2: Remove the cloud app from this repo (atomic with Step 1)

Uninstall the Mend-hosted Renovate cloud app from this repo so the two engines never double-manage it (spec “Dual management” risk).

Step 3: Close stale cloud-bot PRs

Close any open renovate[bot] PRs so CE recreates them under fzymgc-renovate[bot] (chosen handover option A; ignorePrAuthor stays false).

Step 4: Verify CE adopts the repo

Trigger a sync (push, webhook, or mendRnvCronAppSync). CE should pick up the existing .github/renovate.json, refresh the dependency dashboard under the new bot, and open update PRs. Confirm platformAutomerge lands a minor/patch PR through branch protection.

Step 5: Commit — none (no repo files changed).

Task 11: End-to-end verification

Files: none.

Step 1: Pod health

Run: kubectl -n renovate get pods && kubectl -n renovate logs deploy/renovate-ce --tail=50 Expected: pod Running/ready; logs show license accepted, GitHub App auth OK, autodiscovery starting; no SQLite permission errors (validates fsGroup).

Step 2: Health endpoint

Run: kubectl -n renovate exec deploy/renovate-ce -- wget -qO- localhost:8080/health (or via the internal IngressRoute if Task 5 was included). Expected: healthy response.

Step 3: Webhook round-trip

Push a trivial commit (or redeliver a webhook from the App’s Advanced page). Logs should show an inbound /webhook event enqueuing a job. Confirms the Cloudflare tunnel route + HMAC secret.

Step 4: Autodiscovery + registry

Logs/admin API show the installation enumerated and this repo in the registry; SQLite cache populated (cachePersistence).

Step 5: Telemetry

Confirm CE logs land in ClickStack (default.otel_logs, service ≈ renovate) via the cluster’s Vector/OTel pipeline.

Step 6: Commit — none.

Spec coverage check

Spec element	Task
CE via official Helm chart + ArgoCD (D1, D2)	7
Multi-org one public App (D4)	1, 9
Scheduler + webhook (D3)	7 (scheduler config), 9 (webhook route)
SQLite on PVC (D5)	7 (`cachePersistence`, `postgresql.enabled=false`)
Free unlimited license (D6)	1, 2
Auto-onboarding (D7)	7 (chart defaults; `mendRnvAutoDiscoverFilter` scope)
Migrate, preserve existing config (D8)	10 (cutover), `.github/renovate.json` untouched
Vault/ExternalSecret (fovea pattern)	2, 3
NetworkPolicy	4
Internal health/admin (optional)	5
Velero exclusion (rebuildable cache)	8
Cloudflare webhook (GitOps apply)	9
Bot-identity handover (`ignorePrAuthor` false, close+recreate)	10

Notes / deferred

Org-wide rollout (other orgs/repos off the cloud app) is a deliberate follow-up; this plan cuts over only this repo.
githubComToken (optional changelog-fetch PAT to dodge rate limits) can be added to Vault + existingSecret later if changelog lookups hit limits.
autodiscoverFilter scope starts at fzymgc-house/*; widen per-org at rollout.
Global config.js / onboarding preset (spec Impl step 6) is deferred to org-wide rollout. This plan migrates only this repo, which already has .github/renovate.json and is therefore not re-onboarded — so no global preset is needed for it. The chosen bot-identity handover keeps ignorePrAuthor: false, which is also the chart/Renovate default, so no explicit setting is required for this cutover. When other orgs/repos are onboarded later, add a global config.js (org defaults: onboarding: true, onboardingConfig, explicit ignorePrAuthor: false) via renovate.extraEnvVars/a mounted config at that time.

Mend Renovate CE — Implementation Plan