Life agent
The life agent is one personal Hermes agent. Its design, images and Hermes configuration are in
github.com/seanb4t/life-agent. This repository holds its Kubernetes manifests, and ArgoCD
deploys them to the namespace life-agent in two applications:
| Application | Manifests | Contents |
|---|---|---|
life-agent (sync wave 6) |
argocd/app-configs/life-agent/ |
the namespace, both volumes, the sandbox, both proxies, the GHCR pull secret and the token generator of the GitHub App |
life-agent-gateway (sync wave 7) |
argocd/app-configs/life-agent-gateway/ |
the gateway Deployment, its webhook Service, its ExternalSecret, its network policy and the PreSync snapshot hook |
The life agent has three parts: the Hermes gateway, the shell sandbox and two egress proxies. Sean speaks to it in one Discord channel.
What exists
Section titled “What exists”| Deployment | Image | Accepts connections from | Can connect to |
|---|---|---|---|
smokescreen-gateway |
ghcr.io/seanb4t/life-agent-smokescreen |
pods with app.kubernetes.io/name: life-agent-gateway, on 4750 |
kube-dns, and the internet on 443 |
smokescreen-sandbox |
ghcr.io/seanb4t/life-agent-smokescreen |
pods with app.kubernetes.io/name: life-agent-sandbox, on 4750 |
kube-dns, and the internet on 443 |
Each proxy is a Stripe smokescreen HTTP CONNECT proxy with a hostname ACL. The image carries the ACL and the instance config of each proxy:
smokescreen-gatewaypermits only the Discord domains andfal.mediawith its subdomains (enforce).smokescreen-sandboxpermits only the hosts inconfig/smokescreen/sandbox-acl.yamlofseanb4t/life-agent, and rejects each other host with HTTP 407 (enforce). The rejection names the skillsandbox-egress.
A CiliumNetworkPolicy lets each proxy accept connections only from its own life agent pod. The
policy excludes 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 and 100.64.0.0/10 from the
internet rule. smokescreen also denies private, CGNAT and non-unicast addresses by default.
Hermes gateway
Section titled “Hermes gateway”The Deployment gateway operates the Hermes gateway from the image
ghcr.io/seanb4t/life-agent-hermes. The image carries the Hermes configuration and the SOUL of the
life agent from config/ of seanb4t/life-agent.
| Item | Value |
|---|---|
| Process | UID 10000, without s6. shareProcessNamespace: true makes the pause container PID 1 |
| Init container | init_home.py writes the image value of each top-level key of config.yaml into the home volume and keeps the other keys, such as command_allowlist. It copies SOUL.md, saves each replaced file in /opt/data/config-history/ (newest 20), writes the SSH files, removes the Secret-owned variables from .env, and links /opt/data/cron/output to /shared/workspace/cron-output, so that the sandbox reads the cron reports at /workspace/cron-output. The init container mounts life-agent-shared at /shared for that link |
| Home volume | life-agent-home (longhorn-encrypted, ReadWriteOnce) at /opt/data |
| Shared volume | life-agent-shared at /shared, in the init container and in the gateway. workspace/cron-output/ holds the cron reports |
| Workspace | an empty, read-only emptyDir at /workspace. Hermes finds the path of terminal.cwd on the gateway, but loads no context files from it. The workspace of the agent is /workspace on the sandbox |
| Secret | life-agent-gateway: the Discord token and IDs, the SSH client key and the sandbox host key from fzymgc-house/cluster/life-agent, the LiteLLM key from fzymgc-house/cluster/litellm-keys/life-agent, the engram static token (engram_token) from fzymgc-house/cluster/life-agent, and the webhook secret from fzymgc-house/cluster/life-agent/github-app |
| Accepts connections from | the cloudflared namespace, on 8644, through the Service gateway-webhook. The probes run curl inside the container against 127.0.0.1:8642/health |
| Webhook | https://life-agent-wh.fzymgc.net/webhooks/github, the route of the GitHub App. WEBHOOK_ENABLED turns the platform on, and WEBHOOK_SECRET is the HMAC secret. The notices go to the Discord channel in DISCORD_HOME_CHANNEL. See GitHub App |
| Can connect to | kube-dns, LiteLLM on 4000, the sandbox on 2222, and smokescreen-gateway on 4750 |
| Models | Claude Sonnet 5.5 for chat and cron, Claude Haiku 5.5 for the auxiliary tasks and Claude Opus 5.5 for /model, by their openrouter-zdr/anthropic/ names. See the life-agent section of LiteLLM |
| MCP servers | context7, exa, firecrawl, fal, deepwiki and engram, at http://litellm.litellm.svc.cluster.local:4000/<server>/mcp. The key goes in x-litellm-api-key. The engram route also takes the engram static token of the life agent in Authorization. The access group mcp-life-agent in tf/litellm/access_groups.tf grants the six routes |
The top-level keys of config/config.yaml in seanb4t/life-agent are the guardrails. A change to a
guardrail is a pull request there and a new release. At each start, the init container writes the
guardrail values of the image, so an edit of a guardrail in the pod lasts until the next start.
Hermes keeps each other key, such as command_allowlist, after a restart. A key that leaves
config/config.yaml keeps its last value. To retire such a key, remove it from
/opt/data/config.yaml.
Image rollout
Section titled “Image rollout”The images come from the images workflow of seanb4t/life-agent. The workflow builds and signs
an image only when one of its inputs changes. The sandbox and the proxies follow the moving tag
main. The gateway follows calendar release tags of the form YYYYMMDD.NNN.
| Image | Pin | Updater |
|---|---|---|
life-agent-sandbox |
newTag: main and digest in argocd/app-configs/life-agent/kustomization.yaml |
Argo CD Image Updater |
life-agent-smokescreen |
newTag: main and digest in argocd/app-configs/life-agent/kustomization.yaml, for both proxies |
Argo CD Image Updater |
life-agent-hermes |
newTag in argocd/app-configs/life-agent-gateway/kustomization.yaml, for the init container and the gateway |
Argo CD Image Updater |
Argo CD Image Updater reads GHCR every two minutes. The workflow image-updater-automerge.yml turns
on auto-merge for its pull requests on life-agent and life-agent-gateway
(Argo CD Image Updater).
Stateless images
Section titled “Stateless images”- A pull request that changes an input of the sandbox or a proxy merges in
seanb4t/life-agent. - The images workflow publishes the image with the tags
main-<short SHA>andmain. - Within five minutes, Argo CD Image Updater opens a pull request that sets the new
digestinargocd/app-configs/life-agent/kustomization.yaml. GitHub merges it afterpr-gatesandOctopus Reviewpass. - ArgoCD syncs
life-agent, and the changed pods restart. The snapshot hook does not run.
Gateway release
Section titled “Gateway release”- A release pull request (
mise run release) merges inseanb4t/life-agent. - The release workflow tags the merge commit, makes the GitHub release, and gives
life-agent-hermesthe version tag withcrane tag. The digest and its signature do not change. If the version tag exists in GHCR, the workflow fails before it tags. - Within five minutes, Argo CD Image Updater opens a pull request that sets the new version as
newTaginargocd/app-configs/life-agent-gateway/kustomization.yaml. GitHub merges it afterpr-gatesandOctopus Reviewpass. - ArgoCD syncs
life-agent-gateway. The snapshot hook runs first, and then the gateway pod restarts.
If no gateway input changed after the previous release, the release gives no image the version tag, and nothing deploys.
Snapshot hook
Section titled “Snapshot hook”Before each sync of life-agent-gateway, the PreSync Job life-agent-presync-snapshot does these
steps:
- It makes a VolumeSnapshot of
life-agent-homewith the classlonghorn-snap, a Longhorn snapshot on the volume and not a backup. - It waits until the snapshot shows
readyToUse: true, for five minutes at most. - It removes its own snapshots except the newest seven.
If the snapshot is not ready in time, the Job fails and the sync stops. The gateway then continues
to operate the earlier image. The Job runs its script in ghcr.io/fzymgc-house/k8s-utils, with
kubectl from an image volume of registry.k8s.io/kubectl. Its network policy permits egress only
to kube-dns and the Kubernetes API.
To read the result of the last hook:
kubectl --context fzymgc-house -n life-agent logs job/life-agent-presync-snapshotkubectl --context fzymgc-house -n life-agent get volumesnapshot -l app.kubernetes.io/name=life-agent-presync-snapshot --sort-by=.metadata.creationTimestampIf the hook fails, read its log. Then delete the Job, and sync life-agent-gateway again in the
ArgoCD UI.
The snapshot is crash-consistent: the gateway operates while Longhorn takes it. state.db is
SQLite with a write-ahead log, and it recovers from such a copy.
Restore a snapshot
Section titled “Restore a snapshot”Each sync of life-agent-gateway makes a new snapshot. Thus select the snapshot by its time: use
the newest snapshot that is older than the bad release.
-
Find the snapshot and its Longhorn snapshot handle:
Terminal window kubectl --context fzymgc-house -n life-agent get volumesnapshot -l app.kubernetes.io/name=life-agent-presync-snapshot --sort-by=.metadata.creationTimestampkubectl --context fzymgc-house get volumesnapshotcontent "$(kubectl --context fzymgc-house -n life-agent get volumesnapshot <name> -o jsonpath='{.status.boundVolumeSnapshotContentName}')" -o jsonpath='{.status.snapshotHandle}'The handle ends with the name of the Longhorn snapshot.
-
Stop the gateway. Merge a pull request that sets
replicas: 0inargocd/app-configs/life-agent-gateway/gateway.yaml. ArgoCD self-heal reverts a manual scale. -
Make sure that the gateway pod is gone and that the Longhorn volume of
life-agent-homeis detached. -
In the Longhorn UI, attach the volume to a node in maintenance mode.
-
Revert the volume to the Longhorn snapshot from step 1.
-
Detach the volume.
-
Merge a pull request that sets
replicas: 1again. -
Make sure that the gateway is ready and answers in Discord.
Roll back an image
Section titled “Roll back an image”The reliable rollback is a revert in seanb4t/life-agent:
- For a stateless image, revert the change and merge the revert. The workflow publishes a new
maindigest, and Argo CD Image Updater deploys it. - For the gateway, revert the change, merge the revert and make a new release.
A faster rollback of a stateless image is a hold in this repository. A revert of the updater pull request is not enough, because Image Updater opens the same update again at the next poll. Make both changes in one pull request:
- Remove the entry of the image from
argocd/app-configs/argocd-image-updater/life-agent.yaml. - Set the earlier
digestof the image inargocd/app-configs/life-agent/kustomization.yaml. The history of that file holds each earlier digest.
To end the hold, add the entry back in a later pull request. Image Updater then moves the pin to
the newest main digest.
A faster rollback of the gateway is a hold in this repository. Add the bad version to ignoreTags
of the image in argocd/app-configs/argocd-image-updater/life-agent-gateway.yaml, through a pull
request (Hold a version). Image Updater then
opens a pull request that moves the pin back to the newest version that is not held. If the new
version changed the state, restore the snapshot of the earlier release. A revert of the updater pull
request is not a hold, because Image Updater opens the same update again at the next poll.
Shell sandbox
Section titled “Shell sandbox”The Deployment sandbox operates sshd as UID 10000 on port 2222, from the image
ghcr.io/seanb4t/life-agent-sandbox. Each command that the model of the life agent gives runs in
this pod, through the Hermes SSH terminal backend.
| Item | Value |
|---|---|
| Accepts connections from | pods with app.kubernetes.io/name: life-agent-gateway, on 2222 |
| Can connect to | kube-dns, and smokescreen-sandbox on 4750 |
| Secrets | life-agent-sandbox-ssh: the SSH host key and the public key of the gateway, from fzymgc-house/cluster/life-agent. life-agent-github-tokens: one installation token of the GitHub App for each installation owner, read-only at /run/github/<owner>. The volume is optional, so the sandbox starts without the tokens |
| Shared volume | life-agent-shared (Longhorn, ReadWriteMany): skills/ read-only at /skills, workspace/ at /workspace |
| Session variables | the ConfigMap life-agent-sandbox-sshd sets them in each SSH session |
The ConfigMap life-agent-sandbox-sshd sets these variables in each SSH session:
| Variable | Value |
|---|---|
HTTPS_PROXY, HTTP_PROXY, http_proxy |
http://smokescreen-sandbox:4750 |
NO_PROXY |
the cluster domain, the cluster range and the loopback addresses |
MISE_DATA_DIR, MISE_STATE_DIR |
/workspace/.mise/data and /workspace/.mise/state |
MISE_CACHE_DIR |
/tmp/mise-cache, on the emptyDir of the pod. A lock file of the npm backend of mise breaks on the NFS volume, so the cache is not on /workspace |
UV_PYTHON_INSTALL_DIR |
/workspace/.uv/python |
GIT_AUTHOR_NAME, GIT_COMMITTER_NAME |
seanb4t-jarvis[bot], the bot of the GitHub App |
GIT_AUTHOR_EMAIL, GIT_COMMITTER_EMAIL |
341099872+seanb4t-jarvis[bot]@users.noreply.github.com |
curl reads only the lowercase http_proxy for plain HTTP. The mise data and state directories
and the uv directory are on the shared volume, so the tools of a repository stay after a restart of
the sandbox pod. The mise cache does not stay, and mise downloads a tool again when it is absent.
sshd reads the drop-in file only when the pod starts. If you change the ConfigMap, restart the
sandbox:
kubectl --context fzymgc-house -n life-agent rollout restart deployment/sandboxlife-agent-shared is the first ReadWriteMany volume in the cluster. Longhorn serves it through
an NFS share-manager pod in longhorn-system. When the share-manager fails over, I/O on the volume
stops for tens of seconds.
The sandbox has no service account token, and no secret other than its SSH host key, the public key of the gateway and the installation tokens of the GitHub App.
GitHub App
Section titled “GitHub App”The life agent opens pull requests and issues through the GitHub App seanb4t-jarvis (App ID
5276997), which Sean owns. Its manifest, its permissions and the rules for an installed repository
are in config/github/ and docs/decisions/github-app-for-the-agent.md of seanb4t/life-agent.
The App has these permissions: contents, pull_requests and issues (write), and checks,
statuses, actions and metadata (read). It is installed on the account seanb4t (installation
170356430) and on the organization fzymgc-house (installation 170374569), each with selected
repositories.
| Item | Value |
|---|---|
| Vault path | cluster/life-agent/github-app: app_id, installation_id, private_key and webhook_secret. The operator writes it |
| Private key | the ExternalSecret life-agent-github-app copies private_key into the Secret life-agent-github-app. Only the generator reads it |
| Token generators | the GithubAccessToken generators life-agent-github-seanb4t (installation 170356430) and life-agent-github-fzymgc-house (installation 170374569) make an installation token each with the permissions of the App |
| Tokens | the ExternalSecrets life-agent-github-tokens and life-agent-github-tokens-fzymgc-house write the token into the key seanb4t or fzymgc-house of their Secret every 30 minutes. A token expires after one hour |
| Mount | the sandbox projects both Secrets read-only into /run/github, one file for each installation owner, without subPath, so the kubelet updates the files after each renewal. Each source is optional, so the sandbox starts without a token |
| Webhook | GitHub sends the events of each installed repository to https://life-agent-wh.fzymgc.net/webhooks/github. The tunnel entry life-agent of webhook_services in tf/cloudflare/variables.tf forwards only that path to the Service gateway-webhook on 8644. Hermes checks the signature with WEBHOOK_SECRET |
The tunnel has no access policy for the webhook host, so the gateway checks the HMAC signature of
each request. The network policy of the gateway admits only the cloudflared namespace on 8644.
Add a repository to the installation
Section titled “Add a repository to the installation”The App can push a branch and call the merge endpoint in each installed repository. A repository joins the installation only when its default branch needs an approving review, when the App cannot bypass that rule, and when no workflow gives a secret or a write permission to branch code.
-
In a clone of
seanb4t/life-agent, withghlogged in as Sean, make sure that the repository passes the check:Terminal window mise run github:check <owner>/<repo>If the task fails, correct the rules or the workflows of the repository first. The task names each problem.
-
Open the installation page
https://github.com/settings/installations/170356430, and add the repository to the selected repositories.
A repository of fzymgc-house joins the installation 170374569 on the organization page
https://github.com/organizations/fzymgc-house/settings/installations/170374569. A third account
needs a new installation, a third generator with its installation ID, a third ExternalSecret, and a
third source of the projected volume github-tokens of the sandbox. fzymgc-house/selfhosted-cluster passes the
check since its ruleset needs one approving review and the ruleset image-tags-by-admins
restricts the creation of the tags headroom-otel-*, nas-otel-* and lightningstream-* to the
admin role. The release workflow pushes tags of the form <app>/v<version>, which the ruleset does
not cover. The build-* workflows
run on a tag push with a write permission, so an open tag creation would let the App start them.
Examine the tokens
Section titled “Examine the tokens”kubectl --context fzymgc-house -n life-agent get externalsecret life-agent-github-app life-agent-github-tokens life-agent-github-tokens-fzymgc-housekubectl --context fzymgc-house -n life-agent exec deploy/sandbox -c sshd -- ls -l /run/githubThe ExternalSecrets show SecretSynced, and /run/github has the files seanb4t and fzymgc-house. When a token
fails, git and gh in the sandbox get the status 401, and the ExternalSecret
life-agent-github-tokens shows the error of the generator.
Rotate the credentials
Section titled “Rotate the credentials”To rotate the private key, generate a new key on the settings page of the App, write it with
vault kv patch secret/fzymgc-house/cluster/life-agent/github-app private_key=@<file>, and remove
the old key on the settings page after the next refresh. To rotate the webhook secret, set the new
secret on the settings page and in Vault with the same command, then restart the gateway. To
remove all access at once, suspend or uninstall the App on GitHub. Each token stops working at
once.
Read the decisions of a proxy
Section titled “Read the decisions of a proxy”Each proxy writes one JSON line, CANONICAL-PROXY-DECISION, for each request:
kubectl --context fzymgc-house -n life-agent logs deploy/smokescreen-gateway | rg CANONICAL-PROXY-DECISION| Field | Meaning |
|---|---|
requested_host |
The host and port that the client asked for. |
allow |
true if the proxy connected, false if it denied the request. |
decision_reason |
Why the proxy decided. |
enforce_would_deny |
In report mode, true for a host that is not on the list. |
ClickStack collects the same lines from the pod logs.
Change an ACL
Section titled “Change an ACL”The ACLs are in config/smokescreen/ of seanb4t/life-agent. To change an ACL, merge the change
in seanb4t/life-agent. The workflow publishes a new life-agent-smokescreen image, and Argo CD
Image Updater deploys it to both proxies. See Stateless images.
The workflow summary prints a cosign verify command for each image that it publishes. The cluster
does not examine the signature at this time.
After a sync, make sure that each part is ready:
kubectl --context fzymgc-house -n life-agent get deploy gateway sandbox smokescreen-gateway smokescreen-sandboxkubectl --context fzymgc-house -n life-agent logs deploy/smokescreen-gateway | head -5kubectl --context fzymgc-house -n life-agent logs deploy/sandbox | head -3The four Deployments show 1/1. The proxy logs show the listen address 0.0.0.0:4750 and no
error. The sandbox log shows Server listening on 0.0.0.0 port 2222.
After a sync of life-agent-gateway, make sure that the newest hook snapshot shows
READYTOUSE true:
kubectl --context fzymgc-house -n life-agent get volumesnapshot -l app.kubernetes.io/name=life-agent-presync-snapshot --sort-by=.metadata.creationTimestamp