HostAgentics Docs

Contabo Runtime Integration

Last updated: 2026-08-09

This is an internal operational document. Customer-facing surfaces use the term **HostAgentics Cloud**; the provider name appears publicly only in the legally required subprocessor disclosure.

1. Runtime topology

The HostAgentics control plane may remain on its independent application platform, but customer OpenClaw, Hermes, and n8n workloads run exclusively on Contabo infrastructure.

  • One `infrastructure_shards` row maps to one Contabo Cloud VPS.
  • A configured shard accepts one runtime by default (`CONTABO_SHARD_CAPACITY=1`). The exact product ID, RAM, and vCPU count have no hidden defaults and must match the current account catalog. The worker reserves the largest Resource Boost (2 vCPU / 4 GiB) at allocation time, so a later paid boost cannot overcommit the host. Higher densities require a larger VPS and successful load testing.
  • Each customer runtime receives a private Docker network, a main container, dedicated bind-mounted storage, and root-only environment files.
  • n8n also receives a dedicated PostgreSQL container and volume on the same private runtime network. Other customer networks cannot resolve or reach it.
  • Only the main container joins the shared edge network. Caddy is the only process exposing ports 80/443 on the host and reverse-proxies each branded hostname to its explicit container port.
  • No runtime mounts the Docker socket, host filesystem, worker SSH key, provider credentials, or another runtime's volume.

    • 2. Contabo API contract

    The implementation uses Contabo's official REST API at `https://api.contabo.com/v1`:

  • OAuth token: password grant (`client_id`, `client_secret`, API username/password), or `client_credentials` only when the API client has service-account access.
  • Create/list/get compute instances and cancel their monthly subscription through the dedicated

    • `/compute/instances/{id}/cancel` endpoint. Contabo does not expose `DELETE` for an instance.

  • `sshKeys` contains numeric Secrets API IDs, not raw or Base64 public keys.
  • `userData` carries the non-secret cloud-init bootstrap.
  • `x-request-id` is a new UUID for every API request.

    • Reference: <https://api.contabo.com/>

    The provider reconciles a deterministic shard display name before instance creation, so a worker crash after the external API call does not buy a duplicate VPS on retry.

    3. Host bootstrap and SSH trust

    `packages/provider-contabo/src/bootstrap.ts` generates valid `#cloud-config` YAML. It installs a Base64-encoded, mode-0700 bootstrap script through `write_files` and executes it with `runcmd`; a raw shell script is rejected by Contabo's user-data validator. The script installs Docker, restic, rsync, Caddy's container service, unattended security upgrades, and a deny-by-default firewall. SSH password authentication and interactive authentication are disabled; root is key-only.

    The control channel does not use blind trust-on-first-use. `CONTABO_SSH_HOST_KEY_SEED` derives a unique Ed25519 server identity from the deterministic shard name. Cloud-init installs that identity, and every worker connection verifies the exact SSH wire-format public key. Before cloud-init replaces the distro key and writes the readiness marker, connections fail closed and are retried.

    `CONTABO_CONTROL_PLANE_CIDRS` must contain only the public egress CIDRs of API/worker hosts. Production validation also requires a digest-pinned `CONTABO_CADDY_IMAGE`.

    Runtime secrets are deliberately excluded from provider-visible cloud-init. They are delivered only after verified SSH is available and stored in mode-0600 files.

    4. Provisioning mapping

    | Provider-core operation | Contabo implementation |

    | ----------------------- | --------------------------------------------------------------------------------------------- |

    | `createProject` | Reconcile/create a Contabo VPS with cloud-init |

    | `createEnvironment` | Poll instance state, verified SSH, and bootstrap readiness |

    | `createService` | Create root-only metadata/env files and the private runtime network |

    | `createVolume` | Create one dedicated host directory and record its explicit mount path |

    | `setVariables` | Atomic mode-0600 env-file replacement/merge over verified SSH |

    | `assignDomain` | Upsert Cloudflare A record, write Caddy site, reload edge proxy |

    | `deployService` | Pull pinned image, apply CPU/RAM/PID/log limits, mount only declared volumes, start container |

    | lifecycle/logs/metrics | Docker inspect/stats/logs through verified SSH |

    | `updateServiceSource` | Persist a new tested digest; deployment recreates only that container |

    | `deleteService` | Delete DNS/routes/container/config/volume only for the validated opaque service ID |

    | `deleteProject` | Schedule the VPS cancellation for its next monthly contract date |

    Provider IDs are opaque (`ctbhost:*`, `ctbsvc:*`, `ctbvol:*`, `ctbdep:*`) and validated before they influence a command or filesystem path. They never leave internal tables.

    5. Domains and TLS

    Cloudflare credentials must be restricted to DNS edits for the HostAgentics zone. Each runtime hostname gets an A record pointing to its shard. `CLOUDFLARE_RUNTIME_PROXY=false` is the conservative default; enable proxying only after end-to-end WebSocket, streaming, upload-size, and TLS-mode tests.

    Caddy obtains and renews origin certificates, forwards WebSockets/streaming automatically, and routes only to containers attached to `hostagentics-edge`. Hermes gets two hostnames/ports (dashboard and authenticated API); n8n and OpenClaw get one.

    6. Backups and restores

    Whole-VPS snapshots are not used for customer backups: on a shared shard they would combine tenants and a rollback would affect unrelated runtimes. Instead:

  • Every volume is backed up separately with restic to a dedicated Contabo Object Storage bucket.
  • Restic client-side encryption uses `CONTABO_RESTIC_PASSWORD`; S3 credentials are sent only inside verified SSH sessions and are absent from cloud-init and runtime containers.
  • Snapshots carry both a volume tag and a durable operation tag. Retries find the prior snapshot instead of creating duplicates.
  • Before restore/delete, the provider verifies that the requested snapshot has the expected volume tag.
  • Restore stops only the volume's container, restores through a unique temporary directory, atomically synchronizes the volume, cleans up, and restarts only that container.
  • Retention deletion uses `restic forget --prune`, serialized by a host lock.

    • 7. Monitoring and limits

    CPU and memory come from Docker stats; storage comes from exact dedicated volume paths. Public edge egress is read as exact byte counters from Docker's local API, mapped to the container interface attached to `hostagentics-edge`, and accumulated per billing period across container restarts. Internal n8n↔PostgreSQL traffic is therefore not charged against a customer's public-transfer allowance. The control plane stores observed values and applies the existing 70/85/95/100-percent warning and block rules.

    Host-level monitors must additionally alert on CPU steal, disk latency, memory pressure, Docker/Caddy health, restic failures, and shard reachability. A shard with degraded health must set `accepting_new_runtimes=false` before evacuation.

    8. Required live acceptance test

    Automated tests mock provider and SSH boundaries; they do not spend money or prove a specific account's catalog, quota, firewall, or object-storage policy. Before enabling production checkout, use disposable credentials and run:

    1. List products/images and set an exact Ubuntu image UUID available in the account. Copy the selected product ID and its actual RAM/vCPU values into the three shard settings; never reuse an old friendly plan name or example ID. A product shown by the marketing catalog is not sufficient: confirm that the Compute API has an offer for the provider's one-month (`period: 1`) contract, because catalog-only IDs can be rejected at creation time.

    2. Create one disposable EU shard and verify its deterministic SSH host key and CIDR firewall.

    3. Provision one runtime of each kind; verify n8n database isolation, OpenClaw auth, Hermes dashboard/API auth, WebSockets, uploads, and streaming.

    4. Create a backup for every volume, destroy test data, restore, and verify application-level records/memories/workflows.

    5. Restart containers and the VPS; confirm persistent data, transfer-counter continuity, Caddy recovery, and health reporting.

    6. Run CPU-steal, disk, network, and concurrent-runtime load tests. Increase `CONTABO_SHARD_CAPACITY` only when the configured VPS can sustain the reserved 2 vCPU / 4 GiB per runtime plus the host reserve.

    7. Delete the runtimes; verify DNS, restic snapshots, containers, env files, and volume paths are gone. Cancel the disposable VPS and Object Storage, then verify the returned `cancelDate`. Empty and delete every disposable S3 bucket before cancellation.

    Production rollout is blocked until this checklist passes with real credentials.

    The live harness supports both a pre-existing staging shard and an explicitly disposable host. The disposable mode requires `CONTABO_SMOKE_CREATE_HOST=disposable` together with `CONTABO_SMOKE_TEST_CONFIRM=create-and-delete-disposable`; it creates a billable VPS and attempts to cancel that VPS in a `finally` block even after a failed test. It also refuses unpinned Caddy images or an empty SSH CIDR allowlist. Run it only from an isolated staging account.

    Contabo cancellations are contractual, not immediate deletion. A one-month VPS or Object Storage can remain visible and usable until its verified `cancelDate`; a same-day cancellation request may be rejected. Cleanup is complete only when disposable DNS records, buckets, objects, snapshots, and temporary Secrets API entries are gone and both billable resources show the intended future cancellation date. Do not mistake a still-`running` VPS with a populated `cancelDate` for an uncancelled orphan.