EndpointSelector resource for creating EndpointSlices without Service

[keithmattix]

Enhancement Description

Today, any user/controller that wants a set of EndpointSlices for a set of pods matching a label selector, they have 2 options:

1. Create a Service
2. Create and update the EndpointSlice manually as pods matching that selector spin up and down

The latter tends to be avoided by controllers when possible since there is a real impact to scalability when you update endpoints that rapidly across a large cluster. The Kubernetes control plane is already capable of doing this for Service resources, so it stands to reason that this functionality become available to arbitrary consumers as well. Thus, many controllers end up settling for option 1, creating a Service whenever they need EndpointSlices. However, this is not always desirable because Service comes with a bunch of other stuff that you may not want (e.g. a DNS hostname, a frontendIP, etc.). One recent example of this complexity is the InferencePool resource in the Gateway API inference extension. That resource needs to describe a pool of inference endpoints in the cluster; however, using Service to do so would inevitably result in users calling the InferencePool using its hostname/cluster VIP, resulting in kube-proxy load balancing that negates all of the performance benefits of the endpoint picker architecture. And due to only have the aforementioned two options for endpont generation available to them, GAIE implementations end up bearing the complexity of making custom load balancing work.

This KEP is meant to serve as the root of the parallel Gateway API GEP describing this functionality. The idea is that that GEP would be permanently experimental in order to give implementations (and ecosystem subprojects like Kube-Agentic-Networking and WG AI Gateway) the ability to prototype and get feedback before this KEP becomes GA.

This would also give us the opportunity to address some longstanding feature requests like kubernetes/kubernetes#62795 and kubernetes/kubernetes#48528.

/ccing some interested folks: @mikemorris @robscott @LiorLieberman @bowei @aojea @danwinship @youngnick @kflynn @rikatz

• One-line enhancement description (can be used as a release note): Introduce a new resource to allow creating EndpointSlices from selectors without having to create a Service resource
• Kubernetes Enhancement Proposal: KEP-6116 (alpha) : EndpointSelector resource for selecting pod endpoints without requiring a Service #6127
• Discussion Link: Here's the recording of me presenting this slide deck on the functionality described in the KEP.
• PRs by stage and milestone:
  • Alpha - v1.38
    • KEP (k/enhancements) update PR(s):
    • Code (k/k) update PR(s):
    • Docs (k/website) update PR(s):

[keithmattix]

/sig network

[aojea]

Without a review of the proposal and its details, I just want to highlight some of the constraints this has regarding core k/k features and dependencies.

k/k cannot take dependencies on external APIs (like Gateway API or its extensions), since we have strict policies around release cycles and backports. If a core API is designed primarily to unblock and depend on an out-of-tree project, and we create a tightly coupling, our release cycle can create deadlocks. So a core resource needs a standalone justification entirely to be used in core Kubernetes.

Sometime parts of the k/k codebase are occasionally exported. However, the internal Golang API is not stable and can change between versions, because it follows the internal lifecycle of the project. Since it is impossible to guarantee a contract, we avoid to build "generic" controllers in core just so external consumers can import them or try to extend the golang API surface for features out of core.

[keithmattix]

@aojea Both points make sense. I think the ideal scenario from the Gateway API side is that the endpointslice controller internals could take a configurable group and kind so that the Gateway API version of the resource could be read as well during the KEP's maturation. We figured there would be barriers there so worst case, gateway-api could host the controller for the experimental resource, potentially importing pieces of endpointslice controller that it needed. It's still WIP, but a high level draft of the KEP motivations and proposal is at #6127

[szuecs]

@aojea just FYI likely all ingress controllers will use service only to get endpointslices.

In general I do not see why users should call the cluster internal service DNS/VIP, if they want proper inference handling @keithmattix
If they would delete their deployment without --cascade=orphan they would delete all their pods. Errors can happen...

[keithmattix]

@szuecs that's exactly my point re: inference (and ingress fwiw). InferencePool exists because GAIE maintainers didn't want users to accidentally put kube proxy in the mix to nullify the benefits of the EPP

[szuecs]

@keithmattix without much thinking: why do you suggest EndpointSelector and not service type "selector" that does not add ClusterIP VIP, but everything will work normally and you can use the label selector in endpointslices to service as before.
For me it sounds like the thing you want with less headache on the sig-network part of controllers.

I answer myself with a cite from the KEP:

Furthermore, the highly coupled nature of Service has made it difficult to extend its label selection functionality to support more complex selector semantics (for example, set-based matchExpressions) without breaking backwards compatibility with existing Service objects that use an empty selector to indicate "manual" mode. This has been a longstanding community request (see kubernetes/kubernetes#48528 and kubernetes/kubernetes#62795) that has been deferred due to the complexity of adding new selector fields to Service without breaking existing objects.

[keithmattix]

@szuecs great question! I've got 3 answers, one of which you found in the KEP already 🙂

1. We already have a semantic for Services that don't have a clusterIP: headless services. And there's already a bunch of conventions around DNS and how controllers interpret them.
2. I'm hoping to drive towards a more decomposed model of Service so that the ecosystem has more atomic and robust building blocks for creating networking experiences on Kubernetes. Maybe one day these different fields get exposed on Service (e.g. spec.frontend and spec.backend) but we could only get there if we have the roles and scope of each field scoped down.

[aojea]

What I'm trying to indicate is that EndpointSelector needs to be a core object

apiVersion: networking.k8s.io/v1alpha1
kind: EndpointSelector

we can not take a dependency on an external object in core

[keithmattix]

@aojea completely agree; that's the intention. This resource 100% belongs in core long term; the only reason for the gateway api version of the resource is for early experimentation.