Skip to content

migrate mcm-provider from openstack to stackit - #272

Open
aniruddha2000 wants to merge 29 commits into
mainfrom
migrate-mcm
Open

aniruddha2000 wants to merge 29 commits into
mainfrom
migrate-mcm

Conversation

@aniruddha2000

@aniruddha2000 aniruddha2000 commented Aug 26, 2026

Copy link
Copy Markdown
Member

How to categorize this PR?

/kind enhancement

What this PR does / why we need it:

This PR replaces mcm provider from openstack to stackit. Replaces openstack provider ID with stackit provider ID and add stackit.cloud/migrated-machine: true annotation to machines and puts annotation stackit.cloud/machine-controller-manager-migrated: true to workers once the migration is done.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Breaking changes:

@ske-prow ske-prow Bot added the kind/enhancement Enhancement, improvement, extension label Aug 26, 2026
@ske-prow

ske-prow Bot commented Aug 26, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign aniruddha2000 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ske-prow ske-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Aug 26, 2026
@ske-prow ske-prow Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 28, 2026
@ske-prow ske-prow Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 31, 2026
@aniruddha2000

Copy link
Copy Markdown
Member Author

/retest-required

@aniruddha2000

Copy link
Copy Markdown
Member Author

/retest

3 similar comments
@aniruddha2000

Copy link
Copy Markdown
Member Author

/retest

@aniruddha2000

Copy link
Copy Markdown
Member Author

/retest

@aniruddha2000

Copy link
Copy Markdown
Member Author

/retest

@aniruddha2000

Copy link
Copy Markdown
Member Author

/test pull-gardener-extension-provider-stackit-dev-artifacts

@aniruddha2000

Copy link
Copy Markdown
Member Author

/test all

@aniruddha2000

Copy link
Copy Markdown
Member Author

/test pull-gardener-extension-provider-stackit-integration-tests

Comment thread pkg/controller/worker/machines.go Outdated
Comment thread pkg/controller/worker/machines.go Outdated
Comment thread pkg/controller/worker/machines.go Outdated
Comment thread pkg/controller/worker/machines.go Outdated
Comment thread pkg/controller/worker/machines.go Outdated
Comment thread pkg/controller/worker/machines.go Outdated
Comment thread pkg/controller/worker/machines.go Outdated
Comment thread pkg/controller/worker/machines.go Outdated
Comment thread pkg/controller/worker/machines.go Outdated
Comment thread pkg/controller/worker/machines.go Outdated
Comment thread pkg/controller/worker/machines.go Outdated
Comment thread pkg/controller/worker/machines.go

@stackit-ske-bot stackit-ske-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SKE Code Review

Architectural Feedback

  • Inverted Migration Transaction Flow & Redundant Patch Overhead: In pkg/controller/worker/machines.go#L456-L499, each machine undergoes three separate Kubernetes API Patch calls (shouldMigrateMachineAnnotation + migratedMachineAnnotation, then Spec.ProviderID, then removing shouldMigrateMachineAnnotation). This transaction complexity and ephemeral annotation were introduced because Spec.ProviderID is updated in Kubernetes before confirming that the STACKIT IaaS UpdateServer call succeeded. Reversing the sequence—executing UpdateServer (an idempotent operation) on STACKIT IaaS first, and then executing a single atomic Kubernetes Patch updating both Spec.ProviderID and migratedMachineAnnotation—eliminates the ephemeral shouldMigrateMachineAnnotation, reduces Kubernetes API patch traffic by 66% during node migrations, and avoids leaving machine resources in a state where Spec.ProviderID points to STACKIT while the remote server labels were never updated.
  • Unconditional IaaS Client Initialization in WorkerDelegate: In pkg/controller/worker/actuator.go#L76-L80, stackitClient.IaaS(ctx, d.seedClient, worker.Spec.SecretRef) is called unconditionally on every Worker reconciliation. Because w.iaaSClient is exclusively used by migrateMachines (which only executes when feature.MigrateStackitMachineControllerManager(cluster) is enabled and the worker has not yet migrated), Shoot clusters that do not use MCM migration or OpenStack clusters whose secret lacks STACKIT service account credentials (saKeyJSON) will fail secret credential parsing in stackit.GetCredentialsFromSecretRef, breaking Worker reconciliation. The IaaS client should only be instantiated when feature.MigrateStackitMachineControllerManager(cluster) && worker.Annotations[workerMigratedAnnotation] != "true", or lazily on demand inside migrateMachines.

Findings & Feedback

All findings and concrete recommendations have been provided as inline code suggestions above.

Verdict

Comment

Comment thread pkg/controller/worker/machines.go Outdated
Comment thread pkg/controller/worker/machines.go
Comment thread pkg/controller/worker/machines.go
Comment thread pkg/controller/worker/machines.go Outdated
Comment thread pkg/controller/worker/machines.go
Comment thread pkg/controller/worker/machines.go Outdated
if err != nil {
return fmt.Errorf("migrateMachines: %w", err)
}
if m.Spec.ProviderID == "" {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we cannot just error out here.
we have to answer the following questions:
can the provider id even be empty? if yes, in which cases? what is the state of the server if the id is empty?
what happens when we return an error here if the id is empty, this will result in a "failed migration" right? is this the expected behaviour?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aniruddha2000 why have you resolved this comment?
as discussed, we should no error out here, just remove these 3 lines.
if the providerID is empty, the machine will get cleaned up by the mcm anyways after some time

@aniruddha2000 aniruddha2000 Sep 18, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:) whats wrong with my git, i deleted it and pushed a commit bellow this comment, but somehow it did not! sorrry! :(

commit also says "revert provider id is empty review"

Comment thread pkg/controller/worker/machines.go Outdated
Comment on lines +82 to +85
err = w.migrateMachines(ctx)
if err != nil {
return err
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
err = w.migrateMachines(ctx)
if err != nil {
return err
}
if err := w.migrateMachines(ctx); err != nil {
return err
}

Comment thread pkg/controller/worker/machines.go Outdated
if err != nil {
return fmt.Errorf("migrateMachines: %w", err)
}
if m.Spec.ProviderID == "" {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aniruddha2000 why have you resolved this comment?
as discussed, we should no error out here, just remove these 3 lines.
if the providerID is empty, the machine will get cleaned up by the mcm anyways after some time

return fmt.Errorf("cannot migrate machine %s: providerID is empty (provisioning in progress)", m.Name)
}

if m.Spec.ProviderID != "" {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a comment above on why empty providerID is also fine (this is stated in the ADR), so the next person who reads that code knows that we intentionally skip them

m.Annotations = make(map[string]string)
}
m.Annotations[shouldMigrateMachineAnnotation] = "true"
m.Annotations[migratedMachineAnnotation] = "true"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a comment what this annotation does, and that we need it for cleaning up the nics (stated in the ADR)

Comment thread pkg/controller/worker/machines.go Outdated
Comment on lines +487 to +490
// TODO refine labels
"mcm.gardener.cloud/machine": m.Name,
"mcm.gardener.cloud/machineclass": m.Spec.Class.Name,
"mcm.gardener.cloud/role": "node",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// TODO refine labels
"mcm.gardener.cloud/machine": m.Name,
"mcm.gardener.cloud/machineclass": m.Spec.Class.Name,
"mcm.gardener.cloud/role": "node",
"kubernetes.io/machine": m.Name,
"kubernetes.io/machineclass": m.Spec.Class.Name,
"kubernetes.io/role": "node",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK these are the labels that the stackit-mcm are using.
double check that, when using the stackit mcm, creating a new machine, what labels are present on this machine?

Comment thread pkg/controller/worker/machines.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/enhancement Enhancement, improvement, extension size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants