fix(ci): recover K3s forwarding and gate test boot readiness

.agents/skills/watch-github-actions/SKILL.md

@@ -144,6 +144,13 @@ the `release-tag-v1` qualification profile. Failed qualification prevents stable
 publication but still allows pre-release artifacts to publish with the failure
 recorded.
 
+For tmachine K3s conformance failures, download the job's
+`tmachine-diagnostics-*` artifact. It contains diagnostics fetched before the VM
+exits: forwarder restart counts and start-limit settings, K3s and forwarder
+journals, listener state, gateway Pod identity/readiness, endpoint state, and
+current/previous gateway logs. A readiness failure means scenarios did not run;
+use the collector output to identify the failing layer before rerunning.
+
 View logs for a specific run:
 
 ```bash

.github/workflows/integration-runner.yml

@@ -86,3 +86,12 @@ jobs:
           INSTALLER: ${{ matrix.installer }}
           TESTSUITE: ${{ matrix.testsuite }}
         run: nix run .#tmachine -- test "${ENVIRONMENT}" "${INSTALLER}" "${TESTSUITE}"
+
+      - name: Upload tmachine diagnostics
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
+        with:
+          name: tmachine-diagnostics-${{ inputs.category }}-${{ matrix.environment }}-${{ matrix.installer }}-${{ matrix.testsuite }}
+          path: artifacts/tmachine-diagnostics
+          if-no-files-found: ignore
+          retention-days: 7

CI.md

@@ -105,6 +105,28 @@ compatibility baseline for the v1beta1 Sandbox API. It does not track the local
 K3s development default, currently v1.0.3. OpenShell also supports v0.4.6 through
 its v1alpha1 fallback, so v0.5.0 is not the overall minimum supported version.
 
+### K3s boot readiness and diagnostics
+
+The K3s installer checks readiness during provisioning and again after the automated
+test VM boots, including boots from cached installer disks. The checks wait for the
+API, nodes, and gateway StatefulSet, then require `openshell status --output json`
+to report the registered `tmachine` gateway as connected through the forwarder. Readiness
+failures stop the suite before its scenarios run; scenario timeouts are unchanged.
+The loopback forwarder retries every five seconds without a systemd start limit
+so it can recover after temporary K3s API failures.
+
+Interactive `shell` suites skip test-boot preparation so you can inspect a gateway
+that failed during boot. They still run the installer checks when provisioning.
+
+Installation, readiness, and conformance failures collect bounded K3s and
+forwarder journals, service state, listener state, Pod identity and container
+status, endpoints, events, and current and previous gateway logs before the VM
+exits. The collector omits Secrets, kubeconfigs, environment dumps, and full Pod
+specifications and redacts common credential fields. Local runs save these to
+`artifacts/tmachine-diagnostics`; CI uploads them as `tmachine-diagnostics-*`
+artifacts even when the test step fails. Check these alongside the Ansible error
+to distinguish forwarding failures from gateway restarts or readiness failures.
+
 ### Run only the policy advisor conformance tests
 
 Manually dispatch `Integration Tests` on the candidate branch with an

tests/ansible/playbooks/conformance/cli.yaml

@@ -106,6 +106,11 @@
         var: openshell_gateway_logs.stdout_lines
       when: conformance_result.rc != 0
 
+    - name: Collect K3s conformance diagnostics
+      ansible.builtin.include_role:
+        name: openshell_k3s_diagnostics
+      when: conformance_result.rc != 0
+
     - name: Require OpenShell conformance success
       ansible.builtin.assert:
         that:

tests/ansible/playbooks/openshell-k3s-ready.yaml

@@ -0,0 +1,9 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+---
+- name: Check OpenShell K3s readiness after boot
+  hosts: all
+  gather_facts: false
+  roles:
+    - openshell_k3s_ready