Mark XRs not ready while waiting, and replicas until each object is - #467
Merged
Merged
Conversation
negz
changed the title
compose-model-replica marked every composed Object other than its workloads ready as soon as it was observed. That included one that had failed to create, such as a picker Service rejected for an invalid name, and the endpoint picker's Deployment before it was Available, though its Object has a CEL readiness query for exactly that and Envoy AI Gateway fails closed without a picker. Either way the replica reported Ready and compose-model-deployment put it into rotation. This relays each Object's own Ready condition instead, as its readiness policy decides. Signed-off-by: Nic Cope <nicc@rk0n.org>
Crossplane counts an XR ready when every resource composed for it is ready, so one with nothing composed, or only resources marked ready, reports Ready. A ModelReplica waiting for its InferenceCluster, a ModelCache with no cluster to stage onto or waiting for its auth token, a ModelDeployment with no InferenceClusters, an InferenceCluster waiting for its InferenceClasses and a GKECluster waiting for its ProviderConfig all reported Ready while saying they were waiting. Signed-off-by: Nic Cope <nicc@rk0n.org>
negz
changed the title
negz
changed the title
haarchri
approved these changes
Sep 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of your changes
Broken by #101, which marked a ModelReplica's non-workload Objects ready as soon as it observed them. #142 then gave the endpoint picker a readiness query that marking ignored.
compose-model-replica marked every composed Object other than its workloads ready once observed. That included one that failed to create, such as a picker Service the API server rejected for an invalid name, and the picker's Deployment before it was Available, though Envoy AI Gateway fails closed without a picker whatever the pool's
failureModesays. Either way the replica reported Ready and compose-model-deployment put it into rotation. It now relays each Object's own Ready condition, as its readiness policy decides. UnderSuccessfulCreatethat catches an Object that never applied, but not a later update the API server rejects.Separately, Crossplane counts an XR Ready when every composed resource is ready, so an XR with nothing composed is Ready. A ModelReplica waiting for its InferenceCluster, a ModelCache with no cluster to stage onto or waiting for its auth token, a ModelDeployment with no InferenceClusters, an InferenceCluster waiting for its InferenceClasses and a GKECluster waiting for its ProviderConfig all reported
Ready=Truebeside a condition or event saying they were waiting. Each now marks itself not ready where it reports the wait.On EKS, before the change, a ModelDeployment and a ModelCache that matched no cluster were
Ready=True, and a replica whose picker Service had failed to create was Ready with a ModelEndpoint. After it, the first two wereReady=False, and with the picker's Deployment made unavailable the replica wentReady=FalsewithUnready resources: eppand its ModelEndpoint was withdrawn, then both came back once the picker did.I have:
nix flake check(or./nix.sh flake check) and made sure it passes.git commit -s.