Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions api/v1/clusterobjectset_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ const (
ClusterObjectSetReasonObjectsNotReady = "ObjectsNotReady"
ClusterObjectSetReasonSuperseded = "Superseded"
ClusterObjectSetReasonObjectCollision = "ObjectCollision"
ClusterObjectSetReasonRetryableError = "RetryableError"
ClusterObjectSetReasonValidationFailure = "ValidationFailure"
ClusterObjectSetReasonAllObjectsReady = "AllObjectsReady"
)
Expand Down Expand Up @@ -506,10 +505,12 @@ type ClusterObjectSetStatus struct {
// - False with reason ObjectCollision: one or more objects are controlled by another owner, or are unowned while collision protection prevents adoption, so this revision cannot take ownership of them.
// - False with reason ValidationFailure: the revision failed a pre-apply check (an invalid revision/phase object or probe, preflight validation, a mutable referenced Secret, or changed referenced content) and requires manual intervention.
// - False with reason ProgressDeadlineExceeded: the revision did not roll out within spec.progressDeadlineMinutes.
// - False with reason RetryableError: the revision encountered a transient error that may resolve on a subsequent reconciliation.
// - False with reason Archiving: the revision is being archived and its owned objects are still being torn down.
// - False with reason Archived: the revision has been archived and its objects have been torn down.
//
// A transient internal error leaves the Ready condition unchanged and is retried
// with backoff; it is not reported as a distinct reason.
//
// Rollout completion is recorded separately by status.completedAt.
//
// +listType=map
Expand Down
4 changes: 3 additions & 1 deletion applyconfigurations/api/v1/clusterobjectsetstatus.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion docs/draft/concepts/clusterobjectsets.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,11 +221,12 @@ Indicates whether all objects in the revision have been successfully rolled out
| False | `Superseded` | One or more objects have been adopted by a newer revision of the object set |
| False | `ObjectCollision` | One or more objects are controlled by another owner, or are unowned and collision protection prevents adoption |
| False | `ValidationFailure` | A pre-apply check failed (an invalid revision/phase object or probe, preflight validation, a mutable referenced Secret, or changed referenced content); requires manual intervention |
| Unknown | `RetryableError` | Error prevented probe observation |
| False | `Archiving` | The revision is being archived; owned objects are still being torn down |
| False | `Archived` | Objects torn down after archival |
| Unknown | `Migrated` | Migrated from existing release; probes not yet observed |

A transient internal error (e.g. an API read failure) leaves the `Ready` condition unchanged and is retried with backoff; it is not reported as a distinct reason, so on a first reconcile the condition may remain absent (`Unknown`).

### Succeeded

A terminal condition set once the rollout completes. It persists even if the revision later becomes unavailable, marking that this version was successfully deployed at least once.
Expand Down
4 changes: 2 additions & 2 deletions docs/draft/concepts/large-bundle-support.md
Original file line number Diff line number Diff line change
Expand Up @@ -416,8 +416,8 @@ The reconciler distinguishes two failure modes:

- A referenced **Secret that does not exist yet** — for example, informer cache lag
after creation, or a partially completed creation sequence — is treated as a
transient error. The ClusterObjectSet reports `Ready=False, Reason=RetryableError`
and the controller retries with backoff.
transient error. The ClusterObjectSet leaves its `Ready` condition unchanged and the
controller returns the error so it retries with backoff.
- **Invalid referenced content** — a missing key, a malformed manifest, or content
that has changed since first rollout — is a validation failure. The ClusterObjectSet
reports `Ready=False, Reason=ValidationFailure`. Because referenced Secrets are not
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -564,10 +564,12 @@ spec:
- False with reason ObjectCollision: one or more objects are controlled by another owner, or are unowned while collision protection prevents adoption, so this revision cannot take ownership of them.
- False with reason ValidationFailure: the revision failed a pre-apply check (an invalid revision/phase object or probe, preflight validation, a mutable referenced Secret, or changed referenced content) and requires manual intervention.
- False with reason ProgressDeadlineExceeded: the revision did not roll out within spec.progressDeadlineMinutes.
- False with reason RetryableError: the revision encountered a transient error that may resolve on a subsequent reconciliation.
- False with reason Archiving: the revision is being archived and its owned objects are still being torn down.
- False with reason Archived: the revision has been archived and its objects have been torn down.

A transient internal error leaves the Ready condition unchanged and is retried
with backoff; it is not reported as a distinct reason.

Rollout completion is recorded separately by status.completedAt.
items:
description: Condition contains details for one aspect of the current
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -149,10 +149,9 @@ func (c *ClusterObjectSetReconciler) reconcile(ctx context.Context, cos *ocv1.Cl
markAsNotReady(cos, ocv1.ClusterObjectSetReasonValidationFailure, err.Error())
return ctrl.Result{RequeueAfter: validationRetryInterval}, nil
}
// A Secret read error is transient; surface it as retryable and requeue with backoff.
werr := fmt.Errorf("verifying referenced Secrets: %v", err)
setRetryableErrorConditions(cos, werr.Error(), isDeadlineExceeded)
return ctrl.Result{}, werr
// A Secret read error is transient; leave the Ready condition unchanged and
// return the error for rate-limited backoff.
return ctrl.Result{}, fmt.Errorf("verifying referenced Secrets: %v", err)
}

phases, currentPhases, opts, err := c.buildBoxcutterPhases(ctx, cos)
Expand All @@ -174,7 +173,6 @@ func (c *ClusterObjectSetReconciler) reconcile(ctx context.Context, cos *ocv1.Cl
markAsNotReady(cos, ocv1.ClusterObjectSetReasonValidationFailure, err.Error())
return ctrl.Result{}, nil
}
setRetryableErrorConditions(cos, err.Error(), isDeadlineExceeded)
return ctrl.Result{}, fmt.Errorf("converting to boxcutter revision: %v", err)
}

Expand All @@ -191,7 +189,6 @@ func (c *ClusterObjectSetReconciler) reconcile(ctx context.Context, cos *ocv1.Cl

revisionEngine, err := c.RevisionEngineFactory.CreateRevisionEngine(ctx, cos)
if err != nil {
setRetryableErrorConditions(cos, err.Error(), isDeadlineExceeded)
return ctrl.Result{}, fmt.Errorf("failed to create revision engine: %v", err)
}

Expand All @@ -205,7 +202,6 @@ func (c *ClusterObjectSetReconciler) reconcile(ctx context.Context, cos *ocv1.Cl

if cos.Spec.LifecycleState == ocv1.ClusterObjectSetLifecycleStateArchived {
if err := c.TrackingCache.Free(ctx, cos); err != nil {
markAsNotReady(cos, ocv1.ClusterObjectSetReasonRetryableError, err.Error())
return ctrl.Result{}, fmt.Errorf("error stopping informers: %v", err)
}
return c.archive(ctx, revisionEngine, cos, revision)
Expand All @@ -216,9 +212,7 @@ func (c *ClusterObjectSetReconciler) reconcile(ctx context.Context, cos *ocv1.Cl
}

if err := c.establishWatch(ctx, cos, revision); err != nil {
werr := fmt.Errorf("establish watch: %v", err)
setRetryableErrorConditions(cos, werr.Error(), isDeadlineExceeded)
return ctrl.Result{}, werr
return ctrl.Result{}, fmt.Errorf("establish watch: %v", err)
}

rres, err := revisionEngine.Reconcile(ctx, revision, opts...)
Expand All @@ -227,7 +221,6 @@ func (c *ClusterObjectSetReconciler) reconcile(ctx context.Context, cos *ocv1.Cl
// Log detailed reconcile reports only in debug mode (V(1)) to reduce verbosity.
l.V(1).Info("reconcile report", "report", rres.String())
}
setRetryableErrorConditions(cos, err.Error(), isDeadlineExceeded)
return ctrl.Result{}, fmt.Errorf("revision reconcile: %v", err)
}

Expand Down Expand Up @@ -389,7 +382,6 @@ func (c *ClusterObjectSetReconciler) reconcile(ctx context.Context, cos *ocv1.Cl

func (c *ClusterObjectSetReconciler) delete(ctx context.Context, cos *ocv1.ClusterObjectSet) (ctrl.Result, error) {
if err := c.TrackingCache.Free(ctx, cos); err != nil {
markAsNotReady(cos, ocv1.ClusterObjectSetReasonRetryableError, err.Error())
return ctrl.Result{}, fmt.Errorf("error stopping informers: %v", err)
}
if err := c.removeFinalizer(ctx, cos, clusterObjectSetTeardownFinalizer); err != nil {
Expand All @@ -401,9 +393,9 @@ func (c *ClusterObjectSetReconciler) delete(ctx context.Context, cos *ocv1.Clust
func (c *ClusterObjectSetReconciler) archive(ctx context.Context, revisionEngine RevisionEngine, cos *ocv1.ClusterObjectSet, revision boxcutter.RevisionBuilder) (ctrl.Result, error) {
tdres, err := revisionEngine.Teardown(ctx, revision, machinerytypes.WithObserveAfterIncomplete{})
if err != nil {
err = fmt.Errorf("error archiving revision: %v", err)
setRetryableErrorConditions(cos, err.Error(), false)
return ctrl.Result{}, err
// Transient teardown error: leave the Ready condition unchanged and return
// the error for rate-limited backoff.
return ctrl.Result{}, fmt.Errorf("error archiving revision: %v", err)
}
if tdres != nil && !tdres.IsComplete() {
markAsArchiving(cos)
Expand Down Expand Up @@ -769,10 +761,6 @@ func setReadyWithDeadline(cos *ocv1.ClusterObjectSet, status metav1.ConditionSta
})
}

func setRetryableErrorConditions(cos *ocv1.ClusterObjectSet, message string, isDeadlineExceeded bool) {
setReadyWithDeadline(cos, metav1.ConditionFalse, ocv1.ClusterObjectSetReasonRetryableError, message, isDeadlineExceeded)
}

func markAsReady(cos *ocv1.ClusterObjectSet, reason, message string) bool {
return meta.SetStatusCondition(&cos.Status.Conditions, metav1.Condition{
Type: ocv1.ClusterObjectSetTypeReady,
Expand Down
Loading
Loading