Skip to content

Install Argo CD on lith-1 and open the ports monitoring needs - #2

Merged
TakGN merged 7 commits into
mainfrom
argocd-monitoring
Oct 13, 2026
Merged

TakGN merged 7 commits into
mainfrom
argocd-monitoring

Conversation

@TakGN

@TakGN TakGN commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator

Companion to ipni/infra#153 (the monitoring stack). Adds platform.yml, which prepares the boxes for it.

What it does

  • roles/argocd installs Argo CD on argocd_hub (lith-1) once, with the argo-cd Helm chart and the release name ipni/infra uses, and creates the root Application. From then on Argo CD manages its own release from ipni/infra; this role never touches it again.
  • Every run, it:
    • opens 6443 on the other boxes to lith-1's address only (the one exception to the closed API server, documented in group_vars and AGENTS.md) and creates the argocd-manager account there,
    • registers each box with Argo CD,
    • installs Argo CD's age key (vault_argocd_age_key), which decrypts the SOPS-encrypted Secrets in ipni/infra,
    • and, while ipni/infra is private, its deploy key (vault_argocd_repo_ssh_key, optional).
  • roles/monitoring opens 8443 on lith-1 (Traefik for monitoring.ipni.io, argocd.ipni.io, ingest.monitoring.ipni.io) to Cloudflare and the other boxes, and 8000 on sing-1 (Healthchecks, the hub's dead-man's switch) to lith-1. It also installs the Cloudflare DNS token in monitoring for the hub's certificate, from the vault entry the WSS and pinning certificates already use.
  • roles/ufw_allowlist opens a port to an exact list of sources and removes rules for sources dropped from it (the same approach as the route-origin Cloudflare allowlist).
  • Reserves ports 8000 and 8443, and CI syntax-checks platform.yml.

It restarts nothing that serves traffic, so it runs every box in one batch: the hub registers the others using what they produced earlier in the same run.

Before running it

  • Add vault_argocd_age_key (and, while ipni/infra is private, vault_argocd_repo_ssh_key) to group_vars/ipfs_nodes/vault.yml; see the new README section "Argo CD and monitoring".
  • ipni/infra#153 merged, and the ipni.io DNS records in place.

Checked

ansible-playbook --syntax-check on all six playbooks, yamllint, ansible-lint (production profile, 0 failures). Not run against the boxes yet: platform.yml --check --diff first.

🤖 Generated with Claude Code

platform.yml prepares the boxes for the monitoring stack in ipni/infra:

- roles/argocd installs Argo CD on argocd_hub once, with the argo-cd Helm
  chart and the release name ipni/infra uses, and creates the root
  Application. From then on Argo CD manages its own release from ipni/infra.
  Every run it registers the boxes, and installs Argo CD's age key (for the
  SOPS-encrypted Secrets in ipni/infra) and, while that repo is private, its
  deploy key, both from the vault.
- On the other boxes it opens 6443 to lith-1's address only, the one
  exception to the closed API server, and creates the account Argo CD uses.
- roles/monitoring opens 8443 on lith-1 to Cloudflare and the other boxes,
  and 8000 on sing-1 to lith-1 (Healthchecks), and installs the Cloudflare
  token for the monitoring certificate.
- roles/ufw_allowlist opens a port to an exact source list and removes rules
  for sources that are dropped from it.

Ports 8000 and 8443 are reserved, and CI syntax-checks platform.yml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TakGN and others added 6 commits October 12, 2026 19:12
.github/workflows/platform.yml dry-runs platform.yml (--check --diff) on pull
requests and runs it on merge to main, once a required reviewer approves the
`production` environment; it can also be started by hand, check or apply,
with a --limit. It is the only playbook that runs from CI.

CI has no vault files, so scripts/ci-platform.sh turns repository secrets
into platform.yml's three vault variables (passed from a file, never on the
command line), signs in with CI's own SSH key (keys/github-actions.pub, now
in admin_pubkey_file so the hardening role keeps it) and only connects to the
host keys pinned in inventory/known_hosts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
admin_pubkey_file was ~/.ssh/id_ed25519.pub: the key of whoever runs the
playbook. The hardening role installs it exclusively, so the next run would
have removed every other operator's key. The keys now on the boxes (takgn,
nymd) are committed as keys/*.pub and listed with CI's, so the result is the
same whoever runs it. Rendered with the role's own template: it keeps both
existing keys, adds CI's, and removes nothing else.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
So the boxes stay reachable when no operator key works. 1Password generated
the key in the IPFS account's Compass vault; only its public half is here
(keys/break-glass.pub, in admin_pubkey_file). Installed on all three boxes
and tested: it logs in as ipni with sudo, with only that key offered and the
host keys pinned. A hardening run now installs exactly the keys on the boxes.
AGENTS.md explains when and how to use it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
op rejects the ':' in the item's title in a secret reference ("invalid character in secret reference"), and the CLI cannot rename SSH Key items. The ID works and survives a rename; tested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
byo's key (a hardware key, one of those on github.com/byo.keys) is now in
keys/byo.pub and admin_pubkey_file, and installed on all three boxes. A
hardening run installs exactly the five keys now on them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first --check run in CI failed on lith-1: with Argo CD not installed, the
argocd namespace does not exist yet, so the API server rejected the dry-run
create of Argo CD's age key ("namespaces \"argocd\" not found"). The
monitoring role's Cloudflare token Secret would have hit the same thing.
Under --check, both now skip their Secrets while the namespace is missing and
say so, as route_origin does for its TLS secret. A real run is unchanged.

Dry-run against the three boxes from an operator's machine: failed=0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@TakGN
TakGN merged commit 99c17a1 into main Oct 13, 2026
10 checks passed
TakGN added a commit that referenced this pull request Oct 13, 2026
Dry runs and applies shared one concurrency group, and GitHub keeps only one
pending run per group: pushing a pull request branch cancelled the apply of
the #2 merge while it waited for approval. Each job now has its own group:
dry runs per branch (a newer push replaces the older), applies in
platform-apply (never cancelled by a pull request).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant