OCPBUGS-87249: on AWS, select associated IPv6 CIDR block for egress IP subnet - #228
Conversation
|
@jechen0648: This pull request references Jira Issue OCPBUGS-87249, which is invalid:
Comment The bug has been updated to refer to the pull request using the external bug tracker. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
Summary by CodeRabbit
WalkthroughThe AWS ChangesAWS subnet CIDR parsing
Estimated code review effort: 2 (Simple) | ~10 minutes Mergeability Score: ⚪ Minimal · up to This localized change selects the active AWS IPv6 subnet association and tightens subnet handling; no actionable merge-blocking risk remains beyond normal checks and review. Suggested reviewers: 🚥 Pre-merge checks | ✅ 15✅ Passed checks (15 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai full review |
✅ Action performedFull review finished. |
1568129 to
b71c43e
Compare
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
pkg/cloudprovider/aws.go (1)
268-277: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winHandle zero subnet results before indexing.
Line 268 only rejects multiple results, so Line 277 can panic when
DescribeSubnetsreturns zero subnets. This also conflicts with the PR objective to tighten the guard to exactly one subnet.Proposed fix
- if len(describeOutput.Subnets) > 1 { + if len(describeOutput.Subnets) != 1 { subnetId := "<nil>" if networkInterface.SubnetId != nil { subnetId = *networkInterface.SubnetId } - return nil, nil, fmt.Errorf("error: multiple subnets found for the subnet ID: %s", subnetId) + return nil, nil, fmt.Errorf("error: expected exactly one subnet for subnet ID %s, got %d", subnetId, len(describeOutput.Subnets)) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pkg/cloudprovider/aws.go` around lines 268 - 277, The subnet lookup in aws.go only checks for multiple results in the DescribeSubnets flow, so it can still panic when zero subnets are returned before using subnet := describeOutput.Subnets[0]. Update the guard around the DescribeSubnets result in the same block to require exactly one subnet, handling the zero-subnet case with an error before indexing and keeping the existing error path for multiple results. Use the existing networkInterface.SubnetId, describeOutput.Subnets, and subnet variables to locate and adjust the logic.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@pkg/cloudprovider/aws.go`:
- Around line 268-277: The subnet lookup in aws.go only checks for multiple
results in the DescribeSubnets flow, so it can still panic when zero subnets are
returned before using subnet := describeOutput.Subnets[0]. Update the guard
around the DescribeSubnets result in the same block to require exactly one
subnet, handling the zero-subnet case with an error before indexing and keeping
the existing error path for multiple results. Use the existing
networkInterface.SubnetId, describeOutput.Subnets, and subnet variables to
locate and adjust the logic.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 85fa1e5c-85d3-4b26-9ac5-1f81e3d4ad6b
📒 Files selected for processing (1)
pkg/cloudprovider/aws.go
|
/assign @arghosh93 and @pperiyasamy |
|
@tssurya: GitHub didn't allow me to assign the following users: and. Note that only openshift members with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
@arghosh93 I pre-merge tested by fix by building pre-merged image using this PR + openshift/ovn-kubernetes#3269 which is same as u/s PR: ovn-kubernetes/ovn-kubernetes#6648, tests were performed on dualstack AWS for both IPv4 primary and IPv6 primary flavors. |
arghosh93
left a comment
There was a problem hiding this comment.
Only one NIT cmment and mostly looks god to me. I will approve based on your answer.
| subnet := describeOutput.Subnets[0] | ||
| if subnet.CidrBlock != nil && *subnet.CidrBlock != "" { | ||
| _, subnet, err := net.ParseCIDR(*subnet.CidrBlock) | ||
| _, v4Net, err := net.ParseCIDR(*subnet.CidrBlock) |
There was a problem hiding this comment.
Why do you have to change the variable name here?
There was a problem hiding this comment.
this subnet variable is used for v4 and v6 below, it's like one variable used for two different ipstack types, it is easy to get confused , that is why I changed the variable name. But I can revert if you prefer minimal change
|
/payload-job-with-prs periodic-ci-openshift-release-main-nightly-5.0-e2e-aws-ovn-installer-dualstack-ipv6-primary-techpreview openshift/ovn-kubernetes#3269 |
|
/payload-job-with-prs periodic-ci-openshift-release-main-nightly-5.0-e2e-aws-ovn-installer-dualstack-ipv6-primary-techpreview openshift/ovn-kubernetes#3269 |
|
@sadasu: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/91d917a0-8087-11f1-886f-7fdbe052f05d-0 |
I do not think so, |
you mentions a subnet can accumulate multiple entries over lifetime, do you mean lifetime of the node ? I assume associated ipv6 cidr never changes. right ? because cncc needs to populate |
|
/payload-job-with-prs periodic-ci-openshift-release-main-nightly-5.0-e2e-aws-ovn-installer-dualstack-ipv6-primary-techpreview openshift/ovn-kubernetes#3269 |
|
/payload-job-with-prs periodic-ci-openshift-release-main-nightly-5.0-e2e-aws-ovn-installer-dualstack-ipv6-primary-techpreview openshift/ovn-kubernetes#3269 |
|
@sadasu: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/aaa99e80-90d6-11f1-9bba-3026d0a99078-0 |
| } | ||
| v6Subnet = subnet | ||
| v6Subnet = v6Net | ||
| break |
There was a problem hiding this comment.
If this method is called only for dualstack installations, we should warn when we don't find any IPv6 subnet that is associated. For DualStack IPv6 primary, that would be an error and IPv4Primary that would be a warning. Is that captured anywhere?
There was a problem hiding this comment.
getSubnet() will be called for any IP stack type AWS, if Ipv6CidrBlockAssociationSet is not empty, it just processes and finds the first associated IPv6 CidrBlock
You're right that on dual-stack clusters, a missing associated IPv6 subnet may warrant a warning or error depending on Infrastructure.status.platformStatus.aws.ipFamily (DualStackIPv6Primary vs DualStackIPv4Primary). That logic is not implemented today — CNCC does not read cluster IP family config.
I think that's out of scope for this PR, which only fixes selecting the correct associated IPv6 CIDR when multiple associations exist. How about to open a follow-up for IP-stack-aware validation?
|
/jira refresh |
|
@sadasu: This pull request references Jira Issue OCPBUGS-87249, which is valid. 3 validation(s) were run on this bug
No GitHub users were found matching the public email listed for the QA contact in Jira (jechen+ocp-jechen@redhat.com), skipping review request. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
I agree that the bug is hard to reproduce. The results from https://pr-payload-tests.ci.openshift.org/runs/ci/aaa99e80-90d6-11f1-9bba-3026d0a99078-0 show there are no regressions from this. |
…P subnet On dualstack AWS clusters, a subnet's Ipv6CidrBlockAssociationSet can contain multiple entries when an IPv6 CIDR block has been replaced (e.g., a stale 'disassociated' entry followed by the current 'associated' one). The previous code unconditionally picked [0], which could return the wrong CIDR or no CIDR at all, causing the egress-ipconfig annotation to lack the IPv6 subnet. OVN-Kubernetes would then have no node eligible to host the IPv6 egress IP. Fix by iterating through all associations and selecting the first one in 'associated' state. Signed-off-by: Jean Chen <jechen@redhat.com>
b71c43e to
67353ad
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
|
/lgtm |
|
Scheduling required tests: |
|
/verified by pre-merge testing by @jechen0648 |
|
@jechen0648: This PR has been marked as verified by DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@pperiyasamy @kyrtapz, @tssurya This seems ready for approval. |
|
/pipeline required |
|
Scheduling required tests: |
|
@jechen0648: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: arghosh93, jechen0648, pperiyasamy, sadasu The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@jechen0648: Jira Issue OCPBUGS-87249: Some pull requests linked via external trackers have merged: The following pull request, linked via external tracker, has not merged:
All associated pull requests must be merged or unlinked from the Jira bug in order for it to move to the next state. Once unlinked, request a bug refresh with Jira Issue OCPBUGS-87249 has not been moved to the MODIFIED state. This PR is marked as verified. If the remaining PRs listed above are marked as verified before merging, the issue will automatically be moved to VERIFIED after all of the changes from the PRs are available in an accepted nightly payload. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
On dualstack IPv4-primary AWS clusters, egressIPv6 addresses are never assigned to egress nodes even though the nodes report non-zero IPv6 capacity in their cloud.network.openshift.io/egress-ipconfig annotation.
The root cause is in getSubnet (pkg/cloudprovider/aws.go). AWS represents an IPv6 CIDR block association on a subnet through Ipv6CidrBlockAssociationSet, and each entry carries a state (associating, associated, disassociated, etc.). A subnet can accumulate multiple entries over its lifetime — for example, after an IPv6 CIDR block is replaced, the old entry stays in the list as disassociated ahead of the new associated entry.
The previous code unconditionally picked Ipv6CidrBlockAssociationSet[0] without checking state, if the first entry is stale (disassociated), one of two things goes wrong:
In both cases, no CloudPrivateIPConfig is ever created for the IPv6 EgressIP, so CNCC is never asked to assign it.
Fix
Iterate through all entries in Ipv6CidrBlockAssociationSet and select the first one whose state is associated.
Two minor cleanups are included in the same change:
The empty-subnet guard is tightened from len > 1 to len != 1 so that a DescribeSubnets response returning zero subnets is also caught.
The IPv4 parsing block is cleaned up to avoid shadowing the outer subnet variable (_, v4Net, err instead of _, subnet, err).
Related
A companion fix in [openshift/ovn-kubernetes](openshift/ovn-kubernetes#3269) ensures OVN-Kubernetes re-evaluates unassigned EgressIPs when the cloud.network.openshift.io/egress-ipconfig annotation changes, so that any cluster where the annotation was previously incorrect (e.g., missing IPv6 subnet) gets healed automatically once CNCC updates it.
Testing
Pre-merge testing was performed with openshift/ovn-kubernetes#3269 on dualstack AWS IPv4 primary and dualstack AWS IPv6 primary cluster
Regression testing with openshift/ovn-kubernetes#3269 was performed on dualstack BM cluster