Repository navigation
test(mesh): wait for the Docker servers to serve, and install Galaxy collections without the cache - #192
Merged
Conversation
…st's run The role's first start of a Docker server returns once Compose has started the container, before its API server serves. native_server ran the native host's play straight after, and its version check, which reads the cluster through every server and refuses when none answers, found all three still starting and refused. The scenario now waits until every server's /readyz answers and every node is Ready, and logs whether the race was live.
ansible-core's _call_galaxy writes an empty cache entry before it sends a request and fills it only on success. A request it retries (429, 502, 520) therefore finds that unexpired entry on the retry and fails the install with "Missing expected 'results' in ansible-galaxy cache", which a single transient Galaxy error turned into a failed regrade job. Every install that reaches a Galaxy server, in the workflows and bootstrap.sh, now passes --no-cache, and a unit test checks each one and reproduces the cache behaviour through ansible-core's own call.
Mearman
marked this pull request as ready for review
November 3, 2026 13:48
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins. |
|
🎉 This PR is included in version 2.26.2 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two mesh flakes from run 41641946042.
Mesh (native_server)(job 125937402944): the role's first start of a Docker server returns once Compose has started the container, not once its API server serves. The scenario ran the native host's play straight after (servers' run ended 13:11:58, native refusal 13:12:08), and the version check reads the cluster through every server and refuses when none answers: node1 and node2 answeredServiceUnavailable, node3 had no kubeconfig yet. The refusal is right, so the fix is in the harness: native_server now waits until every Docker server's/readyzanswers and every node is Ready before the native play, and logs whether the race was live on that run. The other scenarios already runassert_cluster(nodes Ready through node 1) before any second play, and the version check only needs one server to answer, so they don't have this race.Mesh (regrade)(job 125937403105): not concurrency. The job runs a singleansible-galaxyon a fresh runner._call_galaxyinansible/galaxy/api.pywrites an empty cache entry before it sends a request and only fills it on success. It is retried on 429, 502 and 520, so after one transient Galaxy error the retry finds the unexpired empty entry and raisesMissing expected 'results' in ansible-galaxy cache. Every install that reaches a Galaxy server (mesh, litestream, arc-upgrade, helm-apply, bootstrap.sh) now passes--no-cache.tests/unit/test_galaxy_installs.pyfails if any install drops the flag (checked by removing it from mesh-integration.yml), and reproduces the bug through ansible-core's own_call_galaxy: with the cache, one 502 then a failure with no second request; without it, two requests and success.The failing main run at 4a6daa8 (2026-10-22, regrade) is a different cause:
fleet-regradelisted nodes through node 3's kubeconfig before the recreated etcd-only container had written it. That was fixed by e052424 the same day, and main's mesh runs have passed since. #189 (host_network direct path via the runner's public endpoint) is a Tailscale endpoint choice, not a readiness race, so this doesn't fix it.