Skip to content

Environment Setup

How to set up a workstation for work on this repository.

  • A workstation on macOS or Linux, with Git and a terminal
  • Homebrew on macOS, or the distribution’s package manager on Linux
  • A path to the cluster network. A trusted-Main address (192.168.16.0/20, wired Main 192.168.20.0/22 or MainWireless 192.168.24.0/22) reaches the nodes on 192.168.20.0/24 and, through the gateway, the service VIPs on the Lab VLAN. The tailnet reaches the same addresses from anywhere.
Tool Purpose Install
mise Installs the tools that mise.toml pins brew install mise
kubectl Kubernetes CLI brew install kubectl
kubelogin OIDC login for kubectl brew install kubelogin
vault Vault CLI brew install vault
gh GitHub CLI brew install gh
terraform Local runs of the cluster-bootstrap workspace mise install
uv Runs the pinned Python and Ansible mise install
hk Git hooks and lint checks mise install
rumdl, yamlfmt, shellcheck, cocogitto, actionlint The linters that the hooks and the tasks run mise install
ripgrep Search, as rg mise install
go, node, pnpm, buf Builds and tests of the applications under apps/ mise install

mise.toml pins one version of each tool that a task, a hook or a workflow step calls. mise.lock holds the checksum of each tool for each platform. CI installs the same versions from the same files. The Python tools, such as yamllint, ruff and ansible-lint, come from uv.lock.

Every command in this repository that runs Python or Ansible goes through uv. Do not use pip.

Terminal window
brew install mise kubectl kubelogin vault gh

Add mise activate to the configuration of your shell. In this repository, the shell then uses the versions that mise.toml pins.

Terminal window
echo 'eval "$(mise activate zsh)"' >> ~/.zshrc # zsh
echo 'eval "$(mise activate bash)"' >> ~/.bashrc # bash
echo 'mise activate fish | source' >> ~/.config/fish/config.fish # fish

Clone the repository, install the pinned tools and install the hooks:

Terminal window
git clone git@github.com:fzymgc-house/selfhosted-cluster.git
cd selfhosted-cluster
mise trust
mise install
hk install --mise

mise trust allows mise to read the mise.toml of this repository. mise install installs the versions that mise.lock records.

When the shell enters the repository, mise deps runs uv sync --quiet --frozen --extra dev. It runs the sync only when uv.lock or pyproject.toml changed or .venv is missing. mise then activates .venv, puts scripts/ on PATH, and sets VAULT_ADDR and SOPS_AGE_KEY_FILE.

hk.pkl holds the hooks. On commit they lint YAML, Markdown, Python, shell, Terraform and Ansible, and cocogitto checks the commit message. hk check --all runs the same checks on every file, and hk fix applies the fixes.

mise tasks ls --all prints every task. The tasks cover the Ansible gate fixtures and molecule scenarios, the builds and tests of dns-control and hl-assets-uploader, the Temporal gateway policy tests and the hl-assets verification suite. Run a task as mise //<directory>:<task>, for example mise //apps/dns-control:test. Inside a directory that has tasks, mise :<task> is enough.

setup-venv.sh creates .venv with uv from pyproject.toml, which pins Python 3.13 and Ansible. Run Ansible through uv run:

Terminal window
./setup-venv.sh
cd ansible
uv run ansible-playbook -i inventory/hosts.yml k3s-playbook.yml --list-tags

A Claude Code cloud session runs in a new VM that Anthropic hosts, with a new clone of this repository. The SessionStart hook .claude/hooks/cloud-session-setup.sh does the steps of Repository in that VM. It runs mise install --locked, mise deps --monorepo and hk install --mise. The hook does nothing unless CLAUDE_CODE_REMOTE is true, so a session on a workstation skips it.

The hook needs mise in the VM. The setup script of the cloud environment installs mise and the global tools. Without mise, the hook prints a message and stops. The hook runs mise without GITHUB_TOKEN and GH_TOKEN. The token of the session covers this repository only, and GitHub answers 404 when mise sends it for a file of another repository. Each step writes its output to cloud-session-setup.log in the temporary directory of the VM.

mise sets VAULT_ADDR in this repository.

Terminal window
vault login -method=oidc

The login opens a browser for Keycloak. scripts/vault-helper.sh status prints the connection and the token state, and scripts/vault-helper.sh reads and writes paths under secret/fzymgc-house/infrastructure/.

fnox.toml names one secret, TFE_TOKEN, from secret/fzymgc-house/cluster/hcp-terraform. It reaches only a command that fnox starts, never the shell itself (env = "exec" in fnox.toml). Tools and agents that run in the shell therefore do not inherit it. Start a command that needs it through fnox exec:

Terminal window
fnox exec -- terraform -chdir=tf/hcp-terraform plan # TFE_TOKEN for the tfe provider
fnox exec -- bash # a shell for a runbook that reads $TFE_TOKEN

TFE_TOKEN can start runs, because HCP Terraform has no read-only team token. Exit a shell that fnox exec -- bash started when you finish with it.

fnox does not read ~/.vault-token. The provider runs vault print token, which prints the token that the last vault login wrote, so a new login needs no change to the configuration.

The Firecrawl and Exa MCP servers need no local key. They are HTTP endpoints on the LiteLLM gateway, which holds the provider keys.

To use your own HCP Terraform token instead of the cluster token, keep it at secret/users/<name>/terraform and create fnox.local.toml at the root of the repository. Git ignores that file.

[providers.me]
type = "vault"
address = "https://vault.fzymgc.house"
path = "secret/users/<name>"
credential_command = "vault print token"
[secrets]
TFE_TOKEN = { provider = "me", value = "terraform/api_token" }

Kubeconfig files live under ~/.kube/configs/. Every documented kubectl command names the context fzymgc-house. Cluster Access describes how to create the OIDC file and how to fetch the admin file.

Terminal window
mkdir -p ~/.kube/configs
export KUBECONFIG=~/.kube/configs/fzymgc-house-oidc.yml
Terminal window
gh auth login
Terminal window
scripts/vault-helper.sh status
kubectl --context fzymgc-house get nodes # 8 nodes: 3 control plane, 5 workers
terraform version # 1.15, from mise.toml
cd ansible && uv run ansible --version && cd ..
gh auth status
Directory Purpose
ansible/ Node, resolver, heimdall and NAS configuration
tf/ Terraform, one directory per HCP Terraform workspace
argocd/ Kubernetes manifests, synced by ArgoCD
apps/ Application source that this repository builds
tools/ Lint, gate and verification tooling
scripts/ Operator scripts
docs/ This documentation
website/ The Starlight site that renders docs/