Skip to content

Life agent

The life agent is one personal Hermes agent. Its design, images and Hermes configuration are in github.com/seanb4t/life-agent. This repository holds its Kubernetes manifests, and ArgoCD deploys them to the namespace life-agent in two applications:

Application Manifests Contents
life-agent (sync wave 6) argocd/app-configs/life-agent/ the namespace, both volumes, the sandbox, both proxies, the GHCR pull secret and the token generator of the GitHub App
life-agent-gateway (sync wave 7) argocd/app-configs/life-agent-gateway/ the gateway Deployment, its webhook Service, its ExternalSecret, its network policy and the PreSync snapshot hook

The life agent has three parts: the Hermes gateway, the shell sandbox and two egress proxies. Sean speaks to it in one Discord channel.

Deployment Image Accepts connections from Can connect to
smokescreen-gateway ghcr.io/seanb4t/life-agent-smokescreen pods with app.kubernetes.io/name: life-agent-gateway, on 4750 kube-dns, and the internet on 443
smokescreen-sandbox ghcr.io/seanb4t/life-agent-smokescreen pods with app.kubernetes.io/name: life-agent-sandbox, on 4750 kube-dns, and the internet on 443

Each proxy is a Stripe smokescreen HTTP CONNECT proxy with a hostname ACL. The image carries the ACL and the instance config of each proxy:

  • smokescreen-gateway permits only the Discord domains and fal.media with its subdomains (enforce).
  • smokescreen-sandbox permits only the hosts in config/smokescreen/sandbox-acl.yaml of seanb4t/life-agent, and rejects each other host with HTTP 407 (enforce). The rejection names the skill sandbox-egress.

A CiliumNetworkPolicy lets each proxy accept connections only from its own life agent pod. The policy excludes 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 and 100.64.0.0/10 from the internet rule. smokescreen also denies private, CGNAT and non-unicast addresses by default.

The Deployment gateway operates the Hermes gateway from the image ghcr.io/seanb4t/life-agent-hermes. The image carries the Hermes configuration and the SOUL of the life agent from config/ of seanb4t/life-agent.

Item Value
Process UID 10000, without s6. shareProcessNamespace: true makes the pause container PID 1
Init container init_home.py writes the image value of each top-level key of config.yaml into the home volume and keeps the other keys, such as command_allowlist. It copies SOUL.md, saves each replaced file in /opt/data/config-history/ (newest 20), writes the SSH files, removes the Secret-owned variables from .env, and links /opt/data/cron/output to /shared/workspace/cron-output, so that the sandbox reads the cron reports at /workspace/cron-output. The init container mounts life-agent-shared at /shared for that link
Home volume life-agent-home (longhorn-encrypted, ReadWriteOnce) at /opt/data
Shared volume life-agent-shared at /shared, in the init container and in the gateway. workspace/cron-output/ holds the cron reports
Workspace an empty, read-only emptyDir at /workspace. Hermes finds the path of terminal.cwd on the gateway, but loads no context files from it. The workspace of the agent is /workspace on the sandbox
Secret life-agent-gateway: the Discord token and IDs, the SSH client key and the sandbox host key from fzymgc-house/cluster/life-agent, the LiteLLM key from fzymgc-house/cluster/litellm-keys/life-agent, the engram static token (engram_token) from fzymgc-house/cluster/life-agent, and the webhook secret from fzymgc-house/cluster/life-agent/github-app
Accepts connections from the cloudflared namespace, on 8644, through the Service gateway-webhook. The probes run curl inside the container against 127.0.0.1:8642/health
Webhook https://life-agent-wh.fzymgc.net/webhooks/github, the route of the GitHub App. WEBHOOK_ENABLED turns the platform on, and WEBHOOK_SECRET is the HMAC secret. The notices go to the Discord channel in DISCORD_HOME_CHANNEL. See GitHub App
Can connect to kube-dns, LiteLLM on 4000, the sandbox on 2222, and smokescreen-gateway on 4750
Models Claude Sonnet 5.5 for chat and cron, Claude Haiku 5.5 for the auxiliary tasks and Claude Opus 5.5 for /model, by their openrouter-zdr/anthropic/ names. See the life-agent section of LiteLLM
MCP servers context7, exa, firecrawl, fal, deepwiki and engram, at http://litellm.litellm.svc.cluster.local:4000/<server>/mcp. The key goes in x-litellm-api-key. The engram route also takes the engram static token of the life agent in Authorization. The access group mcp-life-agent in tf/litellm/access_groups.tf grants the six routes

The top-level keys of config/config.yaml in seanb4t/life-agent are the guardrails. A change to a guardrail is a pull request there and a new release. At each start, the init container writes the guardrail values of the image, so an edit of a guardrail in the pod lasts until the next start. Hermes keeps each other key, such as command_allowlist, after a restart. A key that leaves config/config.yaml keeps its last value. To retire such a key, remove it from /opt/data/config.yaml.

The images come from the images workflow of seanb4t/life-agent. The workflow builds and signs an image only when one of its inputs changes. The sandbox and the proxies follow the moving tag main. The gateway follows calendar release tags of the form YYYYMMDD.NNN.

Image Pin Updater
life-agent-sandbox newTag: main and digest in argocd/app-configs/life-agent/kustomization.yaml Argo CD Image Updater
life-agent-smokescreen newTag: main and digest in argocd/app-configs/life-agent/kustomization.yaml, for both proxies Argo CD Image Updater
life-agent-hermes newTag in argocd/app-configs/life-agent-gateway/kustomization.yaml, for the init container and the gateway Argo CD Image Updater

Argo CD Image Updater reads GHCR every two minutes. The workflow image-updater-automerge.yml turns on auto-merge for its pull requests on life-agent and life-agent-gateway (Argo CD Image Updater).

  1. A pull request that changes an input of the sandbox or a proxy merges in seanb4t/life-agent.
  2. The images workflow publishes the image with the tags main-<short SHA> and main.
  3. Within five minutes, Argo CD Image Updater opens a pull request that sets the new digest in argocd/app-configs/life-agent/kustomization.yaml. GitHub merges it after pr-gates and Octopus Review pass.
  4. ArgoCD syncs life-agent, and the changed pods restart. The snapshot hook does not run.
  1. A release pull request (mise run release) merges in seanb4t/life-agent.
  2. The release workflow tags the merge commit, makes the GitHub release, and gives life-agent-hermes the version tag with crane tag. The digest and its signature do not change. If the version tag exists in GHCR, the workflow fails before it tags.
  3. Within five minutes, Argo CD Image Updater opens a pull request that sets the new version as newTag in argocd/app-configs/life-agent-gateway/kustomization.yaml. GitHub merges it after pr-gates and Octopus Review pass.
  4. ArgoCD syncs life-agent-gateway. The snapshot hook runs first, and then the gateway pod restarts.

If no gateway input changed after the previous release, the release gives no image the version tag, and nothing deploys.

Before each sync of life-agent-gateway, the PreSync Job life-agent-presync-snapshot does these steps:

  1. It makes a VolumeSnapshot of life-agent-home with the class longhorn-snap, a Longhorn snapshot on the volume and not a backup.
  2. It waits until the snapshot shows readyToUse: true, for five minutes at most.
  3. It removes its own snapshots except the newest seven.

If the snapshot is not ready in time, the Job fails and the sync stops. The gateway then continues to operate the earlier image. The Job runs its script in ghcr.io/fzymgc-house/k8s-utils, with kubectl from an image volume of registry.k8s.io/kubectl. Its network policy permits egress only to kube-dns and the Kubernetes API.

To read the result of the last hook:

Terminal window
kubectl --context fzymgc-house -n life-agent logs job/life-agent-presync-snapshot
kubectl --context fzymgc-house -n life-agent get volumesnapshot -l app.kubernetes.io/name=life-agent-presync-snapshot --sort-by=.metadata.creationTimestamp

If the hook fails, read its log. Then delete the Job, and sync life-agent-gateway again in the ArgoCD UI.

The snapshot is crash-consistent: the gateway operates while Longhorn takes it. state.db is SQLite with a write-ahead log, and it recovers from such a copy.

Each sync of life-agent-gateway makes a new snapshot. Thus select the snapshot by its time: use the newest snapshot that is older than the bad release.

  1. Find the snapshot and its Longhorn snapshot handle:

    Terminal window
    kubectl --context fzymgc-house -n life-agent get volumesnapshot -l app.kubernetes.io/name=life-agent-presync-snapshot --sort-by=.metadata.creationTimestamp
    kubectl --context fzymgc-house get volumesnapshotcontent "$(kubectl --context fzymgc-house -n life-agent get volumesnapshot <name> -o jsonpath='{.status.boundVolumeSnapshotContentName}')" -o jsonpath='{.status.snapshotHandle}'

    The handle ends with the name of the Longhorn snapshot.

  2. Stop the gateway. Merge a pull request that sets replicas: 0 in argocd/app-configs/life-agent-gateway/gateway.yaml. ArgoCD self-heal reverts a manual scale.

  3. Make sure that the gateway pod is gone and that the Longhorn volume of life-agent-home is detached.

  4. In the Longhorn UI, attach the volume to a node in maintenance mode.

  5. Revert the volume to the Longhorn snapshot from step 1.

  6. Detach the volume.

  7. Merge a pull request that sets replicas: 1 again.

  8. Make sure that the gateway is ready and answers in Discord.

The reliable rollback is a revert in seanb4t/life-agent:

  • For a stateless image, revert the change and merge the revert. The workflow publishes a new main digest, and Argo CD Image Updater deploys it.
  • For the gateway, revert the change, merge the revert and make a new release.

A faster rollback of a stateless image is a hold in this repository. A revert of the updater pull request is not enough, because Image Updater opens the same update again at the next poll. Make both changes in one pull request:

  1. Remove the entry of the image from argocd/app-configs/argocd-image-updater/life-agent.yaml.
  2. Set the earlier digest of the image in argocd/app-configs/life-agent/kustomization.yaml. The history of that file holds each earlier digest.

To end the hold, add the entry back in a later pull request. Image Updater then moves the pin to the newest main digest.

A faster rollback of the gateway is a hold in this repository. Add the bad version to ignoreTags of the image in argocd/app-configs/argocd-image-updater/life-agent-gateway.yaml, through a pull request (Hold a version). Image Updater then opens a pull request that moves the pin back to the newest version that is not held. If the new version changed the state, restore the snapshot of the earlier release. A revert of the updater pull request is not a hold, because Image Updater opens the same update again at the next poll.

The Deployment sandbox operates sshd as UID 10000 on port 2222, from the image ghcr.io/seanb4t/life-agent-sandbox. Each command that the model of the life agent gives runs in this pod, through the Hermes SSH terminal backend.

Item Value
Accepts connections from pods with app.kubernetes.io/name: life-agent-gateway, on 2222
Can connect to kube-dns, and smokescreen-sandbox on 4750
Secrets life-agent-sandbox-ssh: the SSH host key and the public key of the gateway, from fzymgc-house/cluster/life-agent. life-agent-github-tokens: one installation token of the GitHub App for each installation owner, read-only at /run/github/<owner>. The volume is optional, so the sandbox starts without the tokens
Shared volume life-agent-shared (Longhorn, ReadWriteMany): skills/ read-only at /skills, workspace/ at /workspace
Session variables the ConfigMap life-agent-sandbox-sshd sets them in each SSH session

The ConfigMap life-agent-sandbox-sshd sets these variables in each SSH session:

Variable Value
HTTPS_PROXY, HTTP_PROXY, http_proxy http://smokescreen-sandbox:4750
NO_PROXY the cluster domain, the cluster range and the loopback addresses
MISE_DATA_DIR, MISE_STATE_DIR /workspace/.mise/data and /workspace/.mise/state
MISE_CACHE_DIR /tmp/mise-cache, on the emptyDir of the pod. A lock file of the npm backend of mise breaks on the NFS volume, so the cache is not on /workspace
UV_PYTHON_INSTALL_DIR /workspace/.uv/python
GIT_AUTHOR_NAME, GIT_COMMITTER_NAME seanb4t-jarvis[bot], the bot of the GitHub App
GIT_AUTHOR_EMAIL, GIT_COMMITTER_EMAIL 341099872+seanb4t-jarvis[bot]@users.noreply.github.com

curl reads only the lowercase http_proxy for plain HTTP. The mise data and state directories and the uv directory are on the shared volume, so the tools of a repository stay after a restart of the sandbox pod. The mise cache does not stay, and mise downloads a tool again when it is absent.

sshd reads the drop-in file only when the pod starts. If you change the ConfigMap, restart the sandbox:

Terminal window
kubectl --context fzymgc-house -n life-agent rollout restart deployment/sandbox

life-agent-shared is the first ReadWriteMany volume in the cluster. Longhorn serves it through an NFS share-manager pod in longhorn-system. When the share-manager fails over, I/O on the volume stops for tens of seconds.

The sandbox has no service account token, and no secret other than its SSH host key, the public key of the gateway and the installation tokens of the GitHub App.

The life agent opens pull requests and issues through the GitHub App seanb4t-jarvis (App ID 5276997), which Sean owns. Its manifest, its permissions and the rules for an installed repository are in config/github/ and docs/decisions/github-app-for-the-agent.md of seanb4t/life-agent. The App has these permissions: contents, pull_requests and issues (write), and checks, statuses, actions and metadata (read). It is installed on the account seanb4t (installation 170356430) and on the organization fzymgc-house (installation 170374569), each with selected repositories.

Item Value
Vault path cluster/life-agent/github-app: app_id, installation_id, private_key and webhook_secret. The operator writes it
Private key the ExternalSecret life-agent-github-app copies private_key into the Secret life-agent-github-app. Only the generator reads it
Token generators the GithubAccessToken generators life-agent-github-seanb4t (installation 170356430) and life-agent-github-fzymgc-house (installation 170374569) make an installation token each with the permissions of the App
Tokens the ExternalSecrets life-agent-github-tokens and life-agent-github-tokens-fzymgc-house write the token into the key seanb4t or fzymgc-house of their Secret every 30 minutes. A token expires after one hour
Mount the sandbox projects both Secrets read-only into /run/github, one file for each installation owner, without subPath, so the kubelet updates the files after each renewal. Each source is optional, so the sandbox starts without a token
Webhook GitHub sends the events of each installed repository to https://life-agent-wh.fzymgc.net/webhooks/github. The tunnel entry life-agent of webhook_services in tf/cloudflare/variables.tf forwards only that path to the Service gateway-webhook on 8644. Hermes checks the signature with WEBHOOK_SECRET

The tunnel has no access policy for the webhook host, so the gateway checks the HMAC signature of each request. The network policy of the gateway admits only the cloudflared namespace on 8644.

The App can push a branch and call the merge endpoint in each installed repository. A repository joins the installation only when its default branch needs an approving review, when the App cannot bypass that rule, and when no workflow gives a secret or a write permission to branch code.

  1. In a clone of seanb4t/life-agent, with gh logged in as Sean, make sure that the repository passes the check:

    Terminal window
    mise run github:check <owner>/<repo>

    If the task fails, correct the rules or the workflows of the repository first. The task names each problem.

  2. Open the installation page https://github.com/settings/installations/170356430, and add the repository to the selected repositories.

A repository of fzymgc-house joins the installation 170374569 on the organization page https://github.com/organizations/fzymgc-house/settings/installations/170374569. A third account needs a new installation, a third generator with its installation ID, a third ExternalSecret, and a third source of the projected volume github-tokens of the sandbox. fzymgc-house/selfhosted-cluster passes the check since its ruleset needs one approving review and the ruleset image-tags-by-admins restricts the creation of the tags headroom-otel-*, nas-otel-* and lightningstream-* to the admin role. The release workflow pushes tags of the form <app>/v<version>, which the ruleset does not cover. The build-* workflows run on a tag push with a write permission, so an open tag creation would let the App start them.

Terminal window
kubectl --context fzymgc-house -n life-agent get externalsecret life-agent-github-app life-agent-github-tokens life-agent-github-tokens-fzymgc-house
kubectl --context fzymgc-house -n life-agent exec deploy/sandbox -c sshd -- ls -l /run/github

The ExternalSecrets show SecretSynced, and /run/github has the files seanb4t and fzymgc-house. When a token fails, git and gh in the sandbox get the status 401, and the ExternalSecret life-agent-github-tokens shows the error of the generator.

To rotate the private key, generate a new key on the settings page of the App, write it with vault kv patch secret/fzymgc-house/cluster/life-agent/github-app private_key=@<file>, and remove the old key on the settings page after the next refresh. To rotate the webhook secret, set the new secret on the settings page and in Vault with the same command, then restart the gateway. To remove all access at once, suspend or uninstall the App on GitHub. Each token stops working at once.

Each proxy writes one JSON line, CANONICAL-PROXY-DECISION, for each request:

Terminal window
kubectl --context fzymgc-house -n life-agent logs deploy/smokescreen-gateway | rg CANONICAL-PROXY-DECISION
Field Meaning
requested_host The host and port that the client asked for.
allow true if the proxy connected, false if it denied the request.
decision_reason Why the proxy decided.
enforce_would_deny In report mode, true for a host that is not on the list.

ClickStack collects the same lines from the pod logs.

The ACLs are in config/smokescreen/ of seanb4t/life-agent. To change an ACL, merge the change in seanb4t/life-agent. The workflow publishes a new life-agent-smokescreen image, and Argo CD Image Updater deploys it to both proxies. See Stateless images.

The workflow summary prints a cosign verify command for each image that it publishes. The cluster does not examine the signature at this time.

After a sync, make sure that each part is ready:

Terminal window
kubectl --context fzymgc-house -n life-agent get deploy gateway sandbox smokescreen-gateway smokescreen-sandbox
kubectl --context fzymgc-house -n life-agent logs deploy/smokescreen-gateway | head -5
kubectl --context fzymgc-house -n life-agent logs deploy/sandbox | head -3

The four Deployments show 1/1. The proxy logs show the listen address 0.0.0.0:4750 and no error. The sandbox log shows Server listening on 0.0.0.0 port 2222.

After a sync of life-agent-gateway, make sure that the newest hook snapshot shows READYTOUSE true:

Terminal window
kubectl --context fzymgc-house -n life-agent get volumesnapshot -l app.kubernetes.io/name=life-agent-presync-snapshot --sort-by=.metadata.creationTimestamp