Skip to content
fzymgc-house Cluster Docs

Clawdbot Kubernetes Deployment Design

Date: 2026-01-26 Status: Approved Author: Claude (Sonnet 4.5)

Overview

Section titled “Overview”

This document describes the design for deploying clawdbot (https://clawd.bot) - an open-source AI assistant - to the fzymgc-house Kubernetes cluster. The deployment focuses on the gateway component, which handles messaging platform integrations and remote CLI connections.

Background

Section titled “Background”

Clawdbot is a self-hosted AI assistant that provides:

  • Persistent conversation memory and context
  • Proactive capabilities (reminders, briefings, alerts)
  • Integration with messaging platforms (Discord, Slack, WhatsApp, etc.)
  • Full computer access for automation and task execution
  • CLI and gateway modes for flexible interaction

The gateway mode runs as a persistent service, while the CLI mode will be run externally (from local machines) connecting to the gateway.

Architecture

Section titled “Architecture”

Overall Design

Section titled “Overall Design”

The clawdbot deployment consists of:

  • StatefulSet (1 replica): Runs clawdbot-gateway container with stable network identity
  • PersistentVolumeClaims (2 volumes): Config and workspace directories using Longhorn storage
  • Service (ClusterIP with Tailscale): Exposes gateway via Tailscale operator
  • ExternalSecret: Pulls credentials from Vault for Claude API access

Data Flow:

  1. Tailscale operator creates a device proxying to the ClusterIP service
  2. External CLI clients connect to clawdbot.tailnet-name.ts.net via Tailscale
  3. Gateway maintains persistent state in Longhorn PVCs (conversation history, skills, memory)
  4. Gateway communicates with Claude API using credentials from Vault

Key Design Decisions:

  • Tailscale operator over sidecar: Uses existing operator, cleaner integration, automatic DNS
  • StatefulSet over Deployment: Stable pod identity, ordered startup/shutdown for stateful AI
  • Longhorn retention policy: Prevents accidental data loss if StatefulSet is deleted
  • Single replica: Clawdbot is stateful and not designed for horizontal scaling

Component Details

Section titled “Component Details”

1. Storage & Persistence

Section titled “1. Storage & Persistence”

Clawdbot requires two persistent volumes:

Config Volume (~/.clawdbot):

  • Purpose: Configuration, conversation memory, skills, user preferences
  • Size: 5Gi
  • StorageClass: longhorn-encrypted (encrypted at rest, 2 replicas, Delete reclaim policy)
  • Access Mode: ReadWriteOnce
  • Mount Path: /home/node/.clawdbot

Workspace Volume (~/clawd):

  • Purpose: Working directory for operations, temporary files, downloads
  • Size: 20Gi
  • StorageClass: longhorn-encrypted (encrypted at rest, 2 replicas, Delete reclaim policy)
  • Access Mode: ReadWriteOnce
  • Mount Path: /home/node/clawd

Note on Storage Class Choice: Using longhorn-encrypted to match cluster patterns for sensitive data (see Vault and NATS deployments). Clawdbot conversation memory contains private interactions and should be encrypted at rest.

Encryption & Redundancy:

  • Encryption: LUKS encryption via longhorn-crypto-config secret in longhorn-system namespace
  • Replicas: 2-replica redundancy (survives single node failure)
  • Volume Expansion: Enabled (can grow without recreation)
  • Binding Mode: WaitForFirstConsumer (optimized pod scheduling)

Data Protection Strategy:

  • Reclaim Policy: Delete (PVCs are removed if StatefulSet is deleted)
  • Backup: Velero automatically backs up the clawdbot namespace every 24 hours (encrypted backups)
  • Recovery: Use velero restore create --from-backup <backup-name> to recover from accidental deletion
  • Risk Mitigation: Always use kubectl delete sts clawdbot --cascade=orphan if you need to delete/recreate the StatefulSet without losing data

Alternative Consideration: If you prefer explicit retention, manually change the reclaim policy to Retain before risky operations:

Terminal window
kubectl patch pv <pv-name> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'

StatefulSet Volume Pattern: Volumes are defined using volumeClaimTemplates in the StatefulSet spec (not separate PVC files). This is the standard Kubernetes pattern for StatefulSets, which automatically creates and manages PVCs for each pod:

apiVersion: apps/v1
kind: StatefulSet
spec:
  volumeClaimTemplates:
    - metadata:
        name: config
      spec:
        accessModes: [ "ReadWriteOnce" ]
        storageClassName: longhorn-encrypted
        resources:
          requests:
            storage: 5Gi
    - metadata:
        name: workspace
      spec:
        accessModes: [ "ReadWriteOnce" ]
        storageClassName: longhorn-encrypted
        resources:
          requests:
            storage: 20Gi

Backup Strategy:

  • Velero automatically backs up the clawdbot namespace (not in exclusion list)
  • Backs up PVCs, volume snapshots, and all Kubernetes resources
  • Recovery: Delete/recreate StatefulSet reattaches to existing PVCs

2. Secrets Management

Section titled “2. Secrets Management”

Required Secrets:

  • CLAWDBOT_GATEWAY_TOKEN - Gateway API authentication
  • CLAUDE_AI_SESSION_KEY - Claude API session key

Note: CLAUDE_WEB_SESSION_KEY and CLAUDE_WEB_COOKIE were originally planned but are not required for gateway operation.

Vault Configuration:

  • Path: fzymgc-house/cluster/clawdbot
  • Properties: gateway-token, claude-ai-session-key

ExternalSecret:

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: clawdbot-secrets
  namespace: clawdbot
spec:
  refreshInterval: 15m
  secretStoreRef:
    name: vault
    kind: ClusterSecretStore
  target:
    name: clawdbot-secrets
    creationPolicy: Owner
  data:
    - secretKey: CLAWDBOT_GATEWAY_TOKEN
      remoteRef:
        key: fzymgc-house/cluster/clawdbot
        property: gateway-token
    - secretKey: CLAUDE_AI_SESSION_KEY
      remoteRef:
        key: fzymgc-house/cluster/clawdbot
        property: claude-ai-session-key

Terraform Requirements:

Two Terraform files are required in the tf/vault/ directory:

File: tf/vault/policy-clawdbot.tf

# Vault policy for clawdbot
resource "vault_policy" "clawdbot" {
  name = "clawdbot"


  policy = <<EOT
# Allow clawdbot to read its secrets from Vault
path "secret/data/fzymgc-house/cluster/clawdbot" {
  capabilities = ["read"]
}


# Required for ExternalSecrets to verify secret existence
path "secret/metadata/fzymgc-house/cluster/clawdbot" {
  capabilities = ["read"]
}
EOT
}

File: tf/vault/k8s-clawdbot.tf

# Kubernetes auth role binding for clawdbot
resource "vault_kubernetes_auth_backend_role" "clawdbot" {
  backend                          = vault_auth_backend.kubernetes.path
  role_name                        = "clawdbot"
  bound_service_account_names      = ["clawdbot"]
  bound_service_account_namespaces = ["clawdbot"]
  audience                         = "https://kubernetes.default.svc.cluster.local"
  token_ttl                        = 3600
  token_policies                   = ["default", vault_policy.clawdbot.name]
}

Service Account:

  • Name: clawdbot
  • Namespace: clawdbot
  • Must exist before ExternalSecret can sync (defined in namespace.yaml)
  • automountServiceAccountToken: false (clawdbot doesn’t need K8s API access; ExternalSecrets operator handles Vault auth)

Note: The service account should be created in namespace.yaml alongside the namespace definition, following Kubernetes best practices.

3. Container Configuration

Section titled “3. Container Configuration”

Image:

  • ghcr.io/clawdbot/clawdbot:2026.1.24-1 (pinned release)
  • Pull Policy: IfNotPresent

Security Context:

securityContext:
  runAsUser: 1000
  runAsGroup: 1000
  fsGroup: 1000
  runAsNonRoot: true
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL

Command:

command:
  - node
  - dist/index.js
  - gateway
  - --bind
  - "0.0.0.0"
  - --port
  - "18789"

Environment Variables:

  • HOME: /home/node
  • TERM: xterm-256color
  • Secrets mounted from ExternalSecret

Resources:

  • Requests: CPU 500m, Memory 1Gi
  • Limits: CPU 2000m, Memory 2Gi

Rationale: Conservative estimates for initial deployment. The gateway process is lightweight (Node.js), but conversation history and session state require adequate memory. CPU burst capacity (2000m) allows for handling multiple concurrent connections during peak usage.

Health Checks:

  • Liveness: TCP :18789, initialDelay 30s, period 30s
  • Readiness: TCP :18789, initialDelay 10s, period 10s

Probe Choice: TCP probes are used instead of HTTP because the gateway service doesn’t expose dedicated health endpoints. TCP port checks are sufficient to verify the gateway process is listening and accepting connections.

Version Updates:

  • Pin to specific release tag in manifest (currently v2026.1.24)
  • Update cadence: Review upstream releases monthly, update when bug fixes or needed features are available
  • Update process: Test new version locally, create PR with updated image tag, merge triggers ArgoCD sync

4. Networking & Tailscale

Section titled “4. Networking & Tailscale”

Service Configuration:

apiVersion: v1
kind: Service
metadata:
  name: clawdbot-gateway
  namespace: clawdbot
  annotations:
    tailscale.com/expose: "true"
    tailscale.com/hostname: "clawdbot"
    tailscale.com/tags: "tag:k8s-operator"
spec:
  type: ClusterIP
  ports:
    - name: gateway
      port: 18789
      targetPort: 18789
    - name: bridge
      port: 18790
      targetPort: 18790

Tailscale Integration:

  1. Operator detects tailscale.com/expose: "true" annotation
  2. Creates Tailscale device named clawdbot
  3. Device proxies traffic to ClusterIP service
  4. Accessible at clawdbot.your-tailnet.ts.net

Network Access:

  • Gateway: clawdbot.your-tailnet.ts.net:18789 (primary API endpoint)
  • Bridge: clawdbot.your-tailnet.ts.net:18790 (WebSocket/streaming connections)
  • No public exposure (Tailscale-only)
  • Automatic MagicDNS registration

Port Usage:

  • 18789 (gateway): REST API for command execution and configuration (required for core functionality)
  • 18790 (bridge): WebSocket bridge for real-time updates and streaming responses (required for interactive CLI features)

Note: Both ports are required for full clawdbot functionality. The bridge port (18790) is used by the CLI client for bidirectional communication and streaming responses. Without it, interactive features like real-time command output and progress updates will not work.

Security:

  • Zero-trust network via Tailscale
  • No public ingress required
  • Existing Tailscale ACLs apply
  • Encrypted WireGuard tunnels

Deployment Strategy

Section titled “Deployment Strategy”

ArgoCD Integration

Section titled “ArgoCD Integration”
  • Manifests: argocd/app-configs/clawdbot/
  • Application: argocd/cluster-app/clawdbot.yaml
  • Sync Policy: Manual initially, auto after validation
  • Health Monitoring: StatefulSet rollout, pod readiness

Deployment Workflow

Section titled “Deployment Workflow”
  1. Create Terraform Vault policy (tf/vault/policy-clawdbot.tf)
  2. Add secrets to Vault path fzymgc-house/cluster/clawdbot
  3. Merge Kubernetes manifests to main branch
  4. ArgoCD syncs deployment
  5. Tailscale operator provisions device
  6. StatefulSet creates pod with PVCs

Implementation Phases

Section titled “Implementation Phases”

Phase 1: Vault Setup

  • Create Terraform policy definition
  • Apply Vault configuration
  • Store secrets in Vault KV

Phase 2: Kubernetes Manifests

  • Namespace definition
  • ExternalSecret resource
  • StatefulSet with PVCs
  • Service with Tailscale annotations

Phase 3: ArgoCD Application

  • Application definition
  • Sync policy configuration
  • Health check setup

Phase 4: Validation

  • Verify Tailscale device creation
  • Test connectivity from Mac
  • Validate conversation persistence
  • Test pod restart recovery

Monitoring & Operations

Section titled “Monitoring & Operations”

Observability

Section titled “Observability”
  • Logs: kubectl logs, Loki/Grafana integration
  • Metrics: Kubernetes resource metrics via Prometheus (CPU, memory, network)
  • Health: StatefulSet status, pod readiness, PVC binding
  • Tailscale: Device status in admin console

Note on Application Metrics: Clawdbot does not currently expose Prometheus metrics endpoints. If application-level metrics (request latency, error rates, active connections) become needed, consider adding a metrics exporter in a future enhancement.

Operational Procedures

Section titled “Operational Procedures”

Updates:

  1. Update image tag in manifest
  2. Create PR with changes
  3. Merge to main
  4. ArgoCD syncs new version

Rollback:

  • ArgoCD rollback to previous manifest version
  • StatefulSet rolls back to previous image
  • PVCs retain data throughout

Recovery:

  • Delete StatefulSet (PVCs remain)
  • Recreate StatefulSet
  • Pods reattach to existing volumes
  • State fully preserved

Scaling:

  • Single replica only (stateful design)
  • Not horizontally scalable

Testing Procedures

Section titled “Testing Procedures”

  1. Verify Tailscale device registration:

  2. Test connectivity:

    • Gateway port: nc -zv clawdbot.your-tailnet.ts.net 18789
    • Bridge port: nc -zv clawdbot.your-tailnet.ts.net 18790

  3. Verify application functionality:

    • Connect CLI client, verify persistence across requests
    • Test conversation history retention

  4. Test recovery:

    • Restart pod: kubectl delete pod -n clawdbot -l app=clawdbot
    • Verify state recovery from PVC
    • Confirm Tailscale reconnects automatically

File Structure

Section titled “File Structure”
argocd/app-configs/clawdbot/
├── kustomization.yaml          # Lists all resources below
├── namespace.yaml               # Namespace and ServiceAccount
├── external-secret.yaml         # Vault secret sync
├── statefulset.yaml             # StatefulSet with volumeClaimTemplates (no separate PVC files)
└── service.yaml                 # ClusterIP service with Tailscale annotations


argocd/cluster-app/
└── clawdbot.yaml                # ArgoCD Application definition


tf/vault/
├── policy-clawdbot.tf           # Vault policy for secret access
└── k8s-clawdbot.tf              # Kubernetes auth role binding


docs/operations/
└── clawdbot.md                  # Operational runbook (to be created in Phase 4)

kustomization.yaml example:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - namespace.yaml
  - external-secret.yaml
  - statefulset.yaml
  - service.yaml

Note on PVCs: PersistentVolumeClaims are NOT separate files. They are defined inline in the StatefulSet via volumeClaimTemplates, which is the standard Kubernetes pattern for StatefulSets. Kubernetes automatically creates and manages PVCs for each pod replica.

Security Considerations

Section titled “Security Considerations”
  • Secrets never committed to Git (Vault-backed)
  • Automatic secret sync via ExternalSecret refresh (15m interval)
  • Container runs as non-root user (uid 1000)
  • Network access restricted to Tailscale
  • Vault audit trail for secret access

Secret Rotation

Section titled “Secret Rotation”

ExternalSecrets Refresh:

  • ExternalSecrets operator polls Vault every 15 minutes for secret changes
  • When secrets are updated in Vault, ExternalSecrets automatically updates the Kubernetes Secret
  • Pod restart is required for applications to pick up new secret values

Rotation Procedure:

  1. Update secret values in Vault at secret/fzymgc-house/cluster/clawdbot
  2. Wait up to 15 minutes for ExternalSecrets to sync (or force sync via annotation change)
  3. Restart the StatefulSet: kubectl rollout restart statefulset/clawdbot -n clawdbot
  4. Verify new secrets are loaded: Check pod logs for successful authentication

Zero-Downtime Rotation: Not currently supported due to single replica design. Brief service interruption (30-60s) during pod restart is expected.

Additional Security Measures

Section titled “Additional Security Measures”
  • Backup encryption via Velero/Longhorn
  • Service account token scoped to namespace only

Future Enhancements

Section titled “Future Enhancements”
  • Grafana dashboard for clawdbot metrics
  • Loki-based log aggregation and alerts
  • Automated secret rotation workflow
  • Integration with additional messaging platforms
  • Multi-region gateway deployment (if needed)

References

Section titled “References”