Add network policy examples to restrict executors to needed frontend port

charts/sourcegraph-executor/CHANGELOG.md

@@ -3,6 +3,7 @@
 <!-- START CHANGELOG -->
 ## Unreleased
 
+* Added a `network-policy` example to the `sourcegraph-executor-k8s` and `sourcegraph-executor-dind` charts, with `CiliumNetworkPolicy` resources which limit Executor and job pods to the frontend API.
 * Added new chart `sourcegraph-executor-k8s` to deploy Sourcegraph executors that use Kubernetes jobs.
 * **BREAKING:** Renamed `sourcegraph-executor` chart to `sourcegraph-executor-dind` to indicate these are Docker in Docker executors. To update to newer versions of this chart, ensure the new Chart name is used.
 - **BREAKING:** The `securityContext` field in the `sourcegraph-executor-k8s` chart is now deprecated. Use `containerSecurityContext` or `podSecurityContext` instead. The `privileged` field has been moved to `containerSecurityContext`. To update to newer versions of this chart, ensure the new fields are used and the deprecated `securityContext` field is removed.

charts/sourcegraph-executor/dind/examples/network-policy/CiliumNetworkPolicy.yaml

@@ -0,0 +1,48 @@
+# Limit Executor pods to the Sourcegraph frontend API. See README.md
+#
+# Before applying, set io.kubernetes.pod.namespace below to the namespace Sourcegraph is deployed in
+#
+# Apply in the Executor's namespace. Docker-in-Docker jobs run as containers inside the Executor pod,
+# so they share its network path and this one policy covers both the Executor and its jobs
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: sourcegraph-executor-frontend-only
+spec:
+  description: Limit Executor pods to the sourcegraph-frontend API
+  # Keep this a deny-only policy; do not turn on default-deny egress
+  enableDefaultDeny:
+    egress: false
+  endpointSelector:
+    matchLabels:
+      app.kubernetes.io/component: executor
+  egressDeny:
+    # Every Sourcegraph pod except sourcegraph-frontend
+    - toEndpoints:
+        - matchExpressions:
+            - key: io.kubernetes.pod.namespace
+              operator: In
+              values:
+                - sourcegraph
+            - key: deploy
+              operator: In
+              values:
+                - sourcegraph
+            - key: app
+              operator: NotIn
+              values:
+                - sourcegraph-frontend
+    # sourcegraph-frontend's internal (3090) and debug (6060) ports
+    - toEndpoints:
+        - matchLabels:
+            io.kubernetes.pod.namespace: sourcegraph
+            app: sourcegraph-frontend
+      toPorts:
+        - ports:
+            - port: "3090"
+              protocol: TCP
+            - port: "6060"
+              protocol: TCP
+    # Cloud instance metadata service, so jobs cannot read node credentials
+    - toCIDRSet:
+        - cidr: 169.254.169.254/32

charts/sourcegraph-executor/dind/examples/network-policy/README.md

@@ -0,0 +1,27 @@
+# Network Policy: limit Executors to the Sourcegraph frontend API
+
+## Why
+
+- Executors run untrusted code from Batch Changes and auto-indexing jobs
+- Kubernetes allows all pod-to-pod traffic by default, even across namespaces
+- Executors only need the frontend API, so limit them to that
+
+## What it does
+
+`CiliumNetworkPolicy.yaml` contains one deny-only `CiliumNetworkPolicy` for the Executor pods. Docker-in-Docker jobs run as containers inside the Executor pod and share its network path, so this one policy covers both the Executor and its jobs. It denies egress to:
+
+- Every Sourcegraph pod (`deploy=sourcegraph`) except `sourcegraph-frontend`
+- `sourcegraph-frontend`'s internal (3090) and debug (6060) ports
+- The cloud instance metadata service (`169.254.169.254`), so jobs cannot read node credentials
+
+All other egress (DNS, code hosts, package registries, the frontend API) is unchanged. Cilium deny rules take precedence over any allow rules, and `enableDefaultDeny.egress: false` keeps this from becoming a new default-deny policy.
+
+## How to use
+
+- Set `io.kubernetes.pod.namespace` to the namespace Sourcegraph is deployed in
+- Apply in the Executor's namespace, ex. `kubectl apply -n sourcegraph-executor -f CiliumNetworkPolicy.yaml`
+- Verify from a job that `gitserver` is unreachable and `sourcegraph-frontend` still works
+
+## Other CNIs
+
+For clusters without Cilium, use the ingress-side native `NetworkPolicy` example in [`charts/sourcegraph/examples/network-policy`](../../../../sourcegraph/examples/network-policy), which is applied to the Sourcegraph release instead

charts/sourcegraph-executor/k8s/examples/network-policy/CiliumNetworkPolicy.yaml

@@ -0,0 +1,92 @@
+# Limit Executor pods to the Sourcegraph frontend API. See README.md
+#
+# Before applying, set io.kubernetes.pod.namespace below to the namespace Sourcegraph is deployed in
+
+# Apply in the Executor controller's namespace
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: sourcegraph-executor-frontend-only
+spec:
+  description: Limit Executor pods to the sourcegraph-frontend API
+  # Keep this a deny-only policy; do not turn on default-deny egress
+  enableDefaultDeny:
+    egress: false
+  endpointSelector:
+    matchLabels:
+      app.kubernetes.io/component: executor
+  egressDeny:
+    # Every Sourcegraph pod except sourcegraph-frontend
+    - toEndpoints:
+        - matchExpressions:
+            - key: io.kubernetes.pod.namespace
+              operator: In
+              values:
+                - sourcegraph
+            - key: deploy
+              operator: In
+              values:
+                - sourcegraph
+            - key: app
+              operator: NotIn
+              values:
+                - sourcegraph-frontend
+    # sourcegraph-frontend's internal (3090) and debug (6060) ports
+    - toEndpoints:
+        - matchLabels:
+            io.kubernetes.pod.namespace: sourcegraph
+            app: sourcegraph-frontend
+      toPorts:
+        - ports:
+            - port: "3090"
+              protocol: TCP
+            - port: "6060"
+              protocol: TCP
+    # Cloud instance metadata service, so job pods cannot read node credentials
+    - toCIDRSet:
+        - cidr: 169.254.169.254/32
+
+---
+# Apply in the Executor jobs' namespace (executor.namespace in values.yaml)
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: sourcegraph-executor-jobs-frontend-only
+spec:
+  description: Limit Executor job pods to the sourcegraph-frontend API
+  enableDefaultDeny:
+    egress: false
+  endpointSelector:
+    matchExpressions:
+      # Do not match on sourcegraph/queue; multi-queue Executors can create jobs with an empty queue label
+      - key: sourcegraph/job-id
+        operator: Exists
+      - key: sourcegraph/run-id
+        operator: Exists
+  egressDeny:
+    - toEndpoints:
+        - matchExpressions:
+            - key: io.kubernetes.pod.namespace
+              operator: In
+              values:
+                - sourcegraph
+            - key: deploy
+              operator: In
+              values:
+                - sourcegraph
+            - key: app
+              operator: NotIn
+              values:
+                - sourcegraph-frontend
+    - toEndpoints:
+        - matchLabels:
+            io.kubernetes.pod.namespace: sourcegraph
+            app: sourcegraph-frontend
+      toPorts:
+        - ports:
+            - port: "3090"
+              protocol: TCP
+            - port: "6060"
+              protocol: TCP
+    - toCIDRSet:
+        - cidr: 169.254.169.254/32

charts/sourcegraph-executor/k8s/examples/network-policy/README.md

@@ -0,0 +1,27 @@
+# Network Policy: limit Executors to the Sourcegraph frontend API
+
+## Why
+
+- Executors run untrusted code from Batch Changes and auto-indexing jobs
+- Kubernetes allows all pod-to-pod traffic by default, even across namespaces
+- Executors only need the frontend API, so limit them to that
+
+## What it does
+
+`CiliumNetworkPolicy.yaml` contains two deny-only `CiliumNetworkPolicy` resources, one for the Executor controller pods and one for the job pods they create. Each denies egress to:
+
+- Every Sourcegraph pod (`deploy=sourcegraph`) except `sourcegraph-frontend`
+- `sourcegraph-frontend`'s internal (3090) and debug (6060) ports
+- The cloud instance metadata service (`169.254.169.254`), so jobs cannot read node credentials
+
+All other egress (DNS, code hosts, package registries, the frontend API) is unchanged. Cilium deny rules take precedence over any allow rules, and `enableDefaultDeny.egress: false` keeps this from becoming a new default-deny policy.
+
+## How to use
+
+- Set `io.kubernetes.pod.namespace` in both policies to the namespace Sourcegraph is deployed in
+- Apply the first policy in the Executor controller's namespace, and the second in the jobs' namespace (`executor.namespace`), ex. `kubectl apply -n sourcegraph-executor -f CiliumNetworkPolicy.yaml`
+- Verify from an Executor job pod that `gitserver` is unreachable and `sourcegraph-frontend` still works
+
+## Other CNIs
+
+For clusters without Cilium, use the ingress-side native `NetworkPolicy` example in [`charts/sourcegraph/examples/network-policy`](../../../../sourcegraph/examples/network-policy), which is applied to the Sourcegraph release instead

charts/sourcegraph/CHANGELOG.md

@@ -8,6 +8,7 @@ Use `**BREAKING**:` to denote a breaking change
 
 ## Unreleased
 
+- Added a `network-policy` example, which limits Executor and Executor job pods to the frontend API
 - Corrected the external object storage examples to configure the shared store for frontend, worker, precise code intel, syntactic code intel, gitserver, and searcher, including credentials or workload service accounts as required.
 - Removed the unused application ports from the precise and syntactic code intel worker Deployments and Services; health checks and Prometheus metrics continue to use the debug server on port 6060.
 - Added `grafana.initContainers`, so a native sidecar (`restartPolicy: Always`) can be started before `grafana`, e.g. to write a datasource credential file that `datasources.yml` reads with `$__file{}`

charts/sourcegraph/examples/network-policy/README.md

@@ -0,0 +1,28 @@
+# Network Policy: limit Executors to the Sourcegraph frontend API
+
+## Why
+
+- Executors run untrusted code from Batch Changes and auto-indexing jobs
+- Kubernetes allows all pod-to-pod traffic by default, even across namespaces
+- Executors only need the frontend API, so limit them to that
+
+## What it does
+
+`override.yaml` adds two `NetworkPolicy` resources to the Sourcegraph namespace via `extraResources`:
+
+- `sourcegraph-executor-frontend-only`: only allows ingress to Sourcegraph pods from pods in the same namespace which are not Executors (`app.kubernetes.io/component=executor`) or Executor jobs (`sourcegraph/job-id`, `sourcegraph/run-id`)
+- `sourcegraph-frontend-allow-http`: allows the authenticated frontend API (port `http`, 3080) from anywhere, so ingress controllers, users, and Executors keep working
+
+## How to use
+
+- Confirm your CNI enforces `NetworkPolicy` (most do; Cilium, Calico, and OpenShift SDN all do)
+- Add the contents of `override.yaml` to your Helm values override file, or pass it as an extra `-f override.yaml`
+- If pods outside the Sourcegraph namespace need to reach Sourcegraph pods (ex. a cluster-wide Prometheus), uncomment `namespaceSelector: {}`
+- Upgrade the release, then verify from an Executor job pod that `gitserver` is unreachable and `sourcegraph-frontend` still works
+
+## Cilium
+
+If your cluster uses Cilium and this native policy does not take effect, use the egress-side `CiliumNetworkPolicy` example for your Executor chart instead:
+
+- [`charts/sourcegraph-executor/k8s/examples/network-policy`](../../../sourcegraph-executor/k8s/examples/network-policy)
+- [`charts/sourcegraph-executor/dind/examples/network-policy`](../../../sourcegraph-executor/dind/examples/network-policy)

charts/sourcegraph/examples/network-policy/override.yaml

@@ -0,0 +1,52 @@
+# Limit Executor pods to the Sourcegraph frontend API. See README.md
+extraResources:
+  # Only allow ingress to Sourcegraph pods from pods which are not Executors or Executor jobs
+  - apiVersion: networking.k8s.io/v1
+    kind: NetworkPolicy
+    metadata:
+      name: sourcegraph-executor-frontend-only
+      labels:
+        deploy: sourcegraph
+    spec:
+      # Every pod deployed by this chart carries this label
+      podSelector:
+        matchLabels:
+          deploy: sourcegraph
+      policyTypes:
+        - Ingress
+      ingress:
+        - from:
+            - podSelector:
+                matchExpressions:
+                  # Executor pods
+                  - key: app.kubernetes.io/component
+                    operator: NotIn
+                    values:
+                      - executor
+                  # Executor job pods
+                  - key: sourcegraph/job-id
+                    operator: DoesNotExist
+                  - key: sourcegraph/run-id
+                    operator: DoesNotExist
+              # Without a namespaceSelector, only pods in the Sourcegraph namespace are allowed
+              # Uncomment to also allow non-Executor pods from every namespace, ex. a cluster-wide Prometheus
+              # namespaceSelector: {}
+
+  # Allow the authenticated frontend HTTP API from anywhere: ingress controllers, users, and Executors
+  # http-internal (3090) and http-debug (6060) stay covered by the policy above
+  - apiVersion: networking.k8s.io/v1
+    kind: NetworkPolicy
+    metadata:
+      name: sourcegraph-frontend-allow-http
+      labels:
+        deploy: sourcegraph
+    spec:
+      podSelector:
+        matchLabels:
+          app: sourcegraph-frontend
+      policyTypes:
+        - Ingress
+      ingress:
+        - ports:
+            - protocol: TCP
+              port: http