Repository navigation
fix(substrate): scope actors and templates to the watched namespaces - #3261
danielle-mustillo wants to merge 9 commits into
Conversation
…entory Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
A kagent that watches only its own namespace listed every tenant's actors and ActorTemplates from a shared Substrate, and served a foreign atespace or namespace when a caller named one. Resolve one scope from the watched namespaces for every Substrate read. An empty request means the watched namespaces, and a namespace or atespace outside them is refused with PermissionDenied. A cluster-wide install still sees everything. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
…refusal wording A controller watching tenant-a and tenant-c must never page tenant-b actors to the caller, must ask ate-api only for its own atespaces when it builds the summary, and must hide a worker that reports no namespace. The refusal message must say the namespace is not watched, since the gRPC layer sends only the message. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
…workers The summary for a controller watching several namespaces asked ate-api for every atespace and filtered the rows itself, so ate-api served every tenant's actors and templates to it. It now walks actors and drains templates once per atespace in scope. ListSubstrateActors still reads one upstream page across all atespaces and filters it, because one upstream token cannot span several requests. A worker that reports no namespace now belongs only to a cluster-wide scope, and the refusal message names the namespace that is not watched, since the gRPC layer sends only the message. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
EItanya
left a comment
There was a problem hiding this comment.
🤖 AI-generated review. I looked through them to make sure they weren't crazy and they all seem worth looking at.
Thanks for the fix! I found one correctness issue and two follow-ups:
-
Can we deduplicate
observedNamespacesonce when constructing the service?watch_namespacesdoes not require unique values, andresolveSubstrateScopepreserves duplicates.walkActorsandsubstrateActorTemplatesthen query each entry, so["tenant-a", "tenant-a"]reports every actor and template twice. I reproduced this with one actor and one template: the summary returnedActorCount == 2and two template entries. A regression test should include non-adjacent duplicates as well. -
The public protobuf comments for
GetSubstrateSummaryRequest.atespaceandListSubstrateActorsRequest.atespacestill say that an empty value lists all atespaces. With this change, it means the watched namespaces unless the controller is cluster-wide, and an explicitly unwatched atespace returnsPermissionDenied. Can we update the source comments and regenerate so clients don't interpret a scoped result as cluster-wide inventory? -
TestSubstrateTenantIsolationuses an empty Kubernetes client, so it doesn't exercise worker-pool isolation even though the PR applies the scope to every Substrate read. Can we seed in-scope and foreign pools and assert the exact namespace/name pairs for scoped, multi-namespace, and cluster-wide summaries?
The isolation test built its fake cluster with no WorkerPools, so it never checked that the summary lists only pools in the watched namespaces. It also never listed a namespace twice in watchNamespaces, which counted tenant-a's actors, templates and pools twice. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
watchNamespaces allows duplicates, and every Substrate read asks ate-api and Kubernetes once per watched namespace, so a duplicate entry doubled the summary's actors, templates and worker pools. Drop duplicates once when the service is built, keeping the configured order. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
GetSubstrateSummaryRequest.atespace and ListSubstrateActorsRequest.atespace still said an empty value lists all atespaces. It now means the watched namespaces unless the controller is cluster-wide, and an unwatched atespace returns PermissionDenied. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
|
Thanks @EItanya. All three were real. Fixed in 3 commits:
|
Sorting before Compact removes duplicates that are not adjacent. ListNamespaces already sorts its output, so the order change is not visible to callers. Verified with go test ./core/internal/service/system, including the case that watches [tenant-a, tenant-c, tenant-a]. Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
…strate-atespace-scope-indep Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io> # Conflicts: # go/core/internal/service/system/service.go
Why
When a kagent install watches only its own namespace and shares a Substrate with other installs, the Substrate API listed every install's actors and actor templates. It also served another tenant's atespace or namespace when a caller named it. That second path leaked workers too. Before #2771, actors and templates were filtered by watched namespace. The paging rewrite in #2771 dropped that filter for actors and templates and kept it only for workers.
What changed
PermissionDeniedwith a message that says so.Scope
ListSubstrateActors,ListSubstrateWorkers, andGetSubstrateSummary. No API change.atespaceproto comments now say an empty value means the watched namespaces. A namespace listed twice inwatchNamespacesis counted once.Tradeoffs
ListSubstrateActorsmust pass ate-api's page token through unchanged. So with two or more watched namespaces it reads all atespaces and filters each page. A page can be empty and still have a next-page token, as the worker list already allows. A composite token would avoid this, but it changes the token contract.Blast Radius
Only installs that watch some namespaces change. They see less, and requests outside their scope are refused. A cluster-wide install behaves as before.
Verification
ListSubstrateActors {}returned["tenant-a","tenant-b"]before and["tenant-a"]after.GetSubstrateSummary {atespace: tenant-b}returned tenant-b's templates before andPermissionDeniedafter.ListSubstrateWorkers {namespace: tenant-b}returned tenant-b's worker before andPermissionDeniedafter.ListSubstrateActorsmakes the multi-namespace test fail.go test ./core/internal/service/...passes, andmake lintreports 0 issues.