Skip to content

A component with healthCheck type none always fails its deploy #96

Closed

Description

@shipmill-romamo

Found deploying Abecra staging (customer zero) with Cloudfall 0.6.0.

component.schema.json accepts healthCheck: {type: none} for a component that serves no HTTP (a queue worker), but engine/ansible/roles/cloudfall_deploy/tasks/main.yml ("Gate the release on its declared health check") always runs ansible.builtin.uri with the declared scheme, port and path. With type: none those are undefined, the task fails, and the first release of the component is stopped: "Release ... failed its health check and was stopped because no previous release exists".

Proposal

  • type: none: skip the HTTP gate; instead check the systemd unit is active and stays active for a short settle time (for example 10 s), so a worker that crashes on start still fails the deploy
  • A test in the deploy role for a component with type: none

Acceptance

  • A component with healthCheck: {type: none} whose unit stays up deploys
  • One whose unit exits within the settle time fails the deploy and is rolled back

Workaround in use: the worker's health check points at the API of the same release.

Activity

  1. shipmill-romamo commented on Oct 20, 2026

    @shipmill-romamo
    ContributorAuthor

    Triage: implement.

    Confirmed on main (348ed73): component.schema.json accepts healthCheck: {type: none}, but "Gate the release on its declared health check" in engine/ansible/roles/cloudfall_deploy/tasks/main.yml always runs ansible.builtin.uri with healthCheck.scheme, port and path, which a none check doesn't have. The task fails, the rescue runs, and a first release is stopped. The schema promises something the role breaks, so this is a bug fix.

    The plan:

    • For type: none, skip the HTTP gate and check the systemd unit instead: it must be active right after the restart and still active after a fixed settle time (10 s), so a worker that crashes on start still fails and rolls back
    • type: http keeps today's tasks unchanged
    • No schema or receipt shape change: the receipt's existing health field records that the unit gate passed
    • Tests in the deploy role for a none component whose unit stays up, and one whose unit exits inside the settle time

    The settle time is fixed, not a new field; a per-component setting can follow if 10 s turns out wrong for someone. I'll link the PR here.

  2. shipmill-romamo commented on Oct 20, 2026

    @shipmill-romamo
    ContributorAuthor

    The fix is in #107. For healthCheck: {type: none}, the gate skips the HTTP request and checks the unit: it must be active right after the restart, still active 10 s later, and its NRestarts count can't go up in between. That way a worker that crash-loops under Restart=on-failure still fails the gate. Any failure goes through the same rescue as an HTTP check. type: http is unchanged.

    Trade-offs: every type: none deploy now waits 10 s at the gate. The settle time is a role default (cloudfall_deploy_unit_settle_seconds), not a schema field. restart, rollback and the health probe still use the shared gate with no settle wait. Nothing has run against a live systemd yet. This issue closes when #107 merges.

  3. shipmill-romamo commented on Oct 20, 2026

    @shipmill-romamo
    ContributorAuthor

    Released in v0.8.0rc1 (pre-release). uv tool install --prerelease allow cloudfall==0.8.0rc1
    A plain install gets it with the next stable release.
    If you filed this: react 👍 on this comment once it works for you, or reply if it doesn't.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions