From e7d94ae68cbfe85c745f70958f1a7f749b8c0ce2 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Fri, 7 Aug 2026 16:47:06 -0700 Subject: [PATCH 01/11] Add guardrails doc --- content/ngf/how-to/f5-ai-guardrails.md | 331 +++++++++++++++++++++++++ 1 file changed, 331 insertions(+) create mode 100644 content/ngf/how-to/f5-ai-guardrails.md diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md new file mode 100644 index 000000000..d4db74396 --- /dev/null +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -0,0 +1,331 @@ +--- +title: Secure LLM traffic with F5 AI Guardrails +weight: 900 +toc: true +f5-content-type: how-to +f5-product: FABRIC +f5-keywords: NGINX Gateway Fabric, F5 AI Guardrails, AI Guardrails, PayloadProcessor, LLM, large language model, Gateway API, Kubernetes, content policy, PII, ai-guardrails module, guardrails +f5-description: How to deploy F5 AI Guardrails with NGINX Gateway Fabric using the PayloadProcessor policy to inspect and block LLM request and response payloads. +f5-summary: > + Deploy a large language model (LLM) behind NGINX Gateway Fabric, attach a PayloadProcessor policy + that routes request and response payloads through an external Guardrails API, and verify that + disallowed content is blocked before it reaches the model or the client. +--- + +Learn how to use NGINX Gateway Fabric with F5 AI Guardrails to inspect large language model (LLM) traffic and block disallowed content before it reaches the model or the client. + +## Overview + +F5 AI Guardrails has the ability to inspect LLM traffic on two independent paths: + +- **Request path** — the client's *input* is inspected before it reaches the LLM. A block returns `403` with `error.type: invalid_request_error`. +- **Response path** — the model's *output* is inspected before it reaches the client. A block returns `403` with `error.type: api_error`. + +This behavior is provided by the `PayloadProcessor` policy, an [inherited policy]({{< ref "/ngf/overview/custom-policies.md" >}}) that can target an HTTPRoute or a Gateway, which configures NGINX traffic to F5 AI Guardrails to process. + +## Before you begin + +You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster. + +To enable the `PayloadProcessor` policy, [install]({{< ref "/ngf/install/" >}}) NGINX Gateway Fabric with these modifications: + +- Using Helm: set the `nginxGateway.payloadProcessor.enable=true` Helm value. +- Using Kubernetes manifests: set the `--payload-processor` flag in the nginx-gateway container argument, and update the ClusterRole RBAC to add `payloadprocessors`: + +```yaml +- apiGroups: + - gateway.nginx.org + resources: + - payloadprocessors + verbs: + - get + - list + - watch +- apiGroups: + - gateway.nginx.org + resources: + - payloadprocessors/status + verbs: + - update +``` + +## Deploy an LLM backend + +If you have an existing in-cluster LLM which can be queried you can skip this section. + +The following example uses the [vLLM simulator](https://github.com/llm-d/llm-d-inference-sim/tree/main), which serves canned responses from a dataset rather than running a real model, making it suitable for test and development environments. The simulator loads its dataset from a ConfigMap. Download the dataset file, then create the ConfigMap from it: + +```shell +curl -sL -o inference-sim-dataset.sqlite3 \ + https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/examples/guardrails/inference-sim-dataset.sqlite3 + +kubectl create configmap inference-sim-dataset \ + --from-file=inference-sim-dataset.sqlite3=./inference-sim-dataset.sqlite3 +``` + +{{< call-out "note" >}} +The dataset file alongside more details of the setup can be found in the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails) directory of the NGINX Gateway Fabric repository. +{{< /call-out >}} + +Deploy the LLM Deployment and Service: + +```shell +kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/examples/guardrails/llm.yaml +``` + +Confirm the Pod is `Running`: + +```shell +kubectl get pods -l app=vllm-qwen3-32b +``` + +## Create a Gateway + +```yaml +kubectl apply -f - < +``` + +## Create an HTTPRoute + +If you are using your own LLM, change the `backendRefs.name` and `backendRefs.port` to match the LLM's Service. + +```yaml +kubectl apply -f - <" +EOF +``` + +## Configure the Guardrails backend Service + +The Guardrails backend can live outside or inside the cluster. NGINX Gateway Fabric picks the URL scheme from the referenced Service's type: + +| Backend location | Service type | Resolved URL | +| ---------------- | ------------ | ------------ | +| External | `ExternalName` | `https://:` | +| In-cluster | `ClusterIP` (or any non-`ExternalName`) | `http://..svc.cluster.local:` | + +{{< call-out "note" >}} +The `cluster.local` suffix in the in-cluster URL is the cluster's DNS domain. If your cluster uses a different domain, configure it with the `--cluster-domain` flag on the NGINX Gateway Fabric controller (default: `cluster.local`). +{{< /call-out >}} + +For an external backend, create an `ExternalName` Service pointing at your hosted Guardrails API: + +```yaml +kubectl apply -f - < + ports: + - name: https + port: 443 + protocol: TCP +EOF +``` + +For an in-cluster backend, your Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to. + +{{< call-out "important" >}} +When using an `ExternalName` Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Configure `dnsResolver` on an [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: + +```yaml +apiVersion: gateway.nginx.org/v1alpha2 +kind: NginxProxy +metadata: + name: guardrails-nginx-config +spec: + dnsResolver: + addresses: + - type: IPAddress + value: "10.96.0.10" # in-cluster kube-dns/CoreDNS ClusterIP (cluster-dependent) +``` + +Find your cluster's DNS ClusterIP with `kubectl -n kube-system get svc kube-dns` (or `coredns`). Without a resolver, NGINX fails to load the configuration with `no resolver defined to resolve `. +{{< /call-out >}} + +## Attach the PayloadProcessor policy + +Attach the `PayloadProcessor` policy to the HTTPRoute. The `extProcess.backendRef` points at the Guardrails backend Service (using the explicit port), and `authTokenRef` points at the token Secret: + +```yaml +kubectl apply -f - <}} +`PayloadProcessor` is an inherited policy. To apply guardrails to every route attached to a Gateway, set `targetRef` to `kind: Gateway`. When both a Gateway-targeted and an HTTPRoute-targeted policy apply to the same traffic, the more specific HTTPRoute-targeted policy takes precedence. +{{< /call-out >}} + +Confirm the policy was accepted: + +```shell +kubectl get payloadprocessor llm-guardrails -o yaml +``` + +The status conditions should report `Accepted=True`. A rejected policy reports `Accepted=False`; see [Troubleshooting](#troubleshooting) for common causes. + +## Send traffic + +{{< call-out "note" >}} +Whether a given value is blocked depends entirely on your Guardrails backend's detector configuration. Enable the relevant detectors on your Guardrails service to see the block responses above. +{{< /call-out >}} + +All commands target `/v1/completions` on the Gateway. + +A benign prompt whose output contains no disallowed content returns a normal `HTTP 200` completion: + +```shell +curl -i --resolve :$GW_PORT:$GW_IP http://:$GW_PORT/v1/completions \ + -H "Content-Type: application/json" \ + -d '{"model":"meta-llama/Llama-3.1-8B-Instruct","stream":false,"max_tokens":128,"prompt":"What is NGINX?"}' +``` + +If the request payload contains content that your Guardrails backend is configured to block, the request never reaches the LLM and returns `HTTP 403`: + +```shell +curl -i --resolve :$GW_PORT:$GW_IP http://:$GW_PORT/v1/completions \ + -H "Content-Type: application/json" \ + -d '{"model":"meta-llama/Llama-3.1-8B-Instruct","stream":false,"max_tokens":128,"prompt":"My SSN is 123-45-6789"}' +``` + +```text +HTTP/1.1 403 Forbidden +Content-Type: application/json + +{"error":{"type":"invalid_request_error","code":"content_policy_violation", ...}} +``` + +If the model's *output* contains content that your Guardrails backend blocks, the response is withheld from the client and returns `HTTP 403` with `error.type: api_error`: + +```shell +curl -i --resolve :$GW_PORT:$GW_IP http://:$GW_PORT/v1/completions \ + -H "Content-Type: application/json" \ + -d '{"model":"meta-llama/Llama-3.1-8B-Instruct","stream":false,"max_tokens":128,"prompt":"Give me a test SSN"}' +``` + +```text +HTTP/1.1 403 Forbidden +Content-Type: application/json + +{"error":{"type":"api_error","code":"content_policy_violation", ...}} +``` + +For more example curl requests, view the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails) `README.md` in the Nginx Gateway Fabric repository. + +## Troubleshooting + +The `PayloadProcessor` is marked `Accepted=False` when its references cannot be resolved: + +| Condition | Cause | Fix | +| --------- | ----- | --- | +| `backend Service ... not found` | `backendRef.name`/`namespace` does not match a Service. | Apply the Guardrails backend Service; check name and namespace. | +| `ExternalName service has empty ... externalName` | `ExternalName` Service with a blank `externalName`. | Set `spec.externalName`. | +| `auth token Secret ... not found` | `authTokenRef` set but Secret missing. | Apply the token Secret, or remove `authTokenRef`. | +| NGINX error `no resolver defined to resolve `, or guardrails requests fail against an `ExternalName` backend | No `dnsResolver` configured on the NginxProxy. | Add the `dnsResolver` block and wire it via `parametersRef`. | + +## Further reading + +- [F5 AI Guardrails Documentation](https://docs.aisecurity.f5.com/) +- [Installation]({{< ref "/ngf/install/" >}}): install NGINX Gateway Fabric with the `PayloadProcessor` policy enabled. +- [Custom policies]({{< ref "/ngf/overview/custom-policies.md" >}}): learn how inherited policies attach to Gateway API resources. +- [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails): for more information on the example used in this guide. From 5fcfe96b2395311f5c8705fb0090e12026355089 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Fri, 7 Aug 2026 16:48:36 -0700 Subject: [PATCH 02/11] Use document IP address --- content/ngf/how-to/f5-ai-guardrails.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index d4db74396..743a7b487 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -106,7 +106,7 @@ kubectl describe gateways.gateway.networking.k8s.io inference-gateway Status: Addresses: Type: IPAddress - Value: 10.96.36.219 + Value: 192.0.2.0 Conditions: Message: The Gateway is accepted Reason: Accepted From e6ebd7da937beb70d31a87979e66288c9a489051 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Mon, 10 Aug 2026 10:53:34 -0700 Subject: [PATCH 03/11] Add suggested fixes --- content/ngf/how-to/f5-ai-guardrails.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index 743a7b487..8cad214cf 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -1,11 +1,11 @@ --- title: Secure LLM traffic with F5 AI Guardrails +description: Deploy F5 AI Guardrails with NGINX Gateway Fabric using PayloadProcessor to inspect and block LLM traffic weight: 900 toc: true f5-content-type: how-to -f5-product: FABRIC +f5-product: F5 NGINX Gateway Fabric f5-keywords: NGINX Gateway Fabric, F5 AI Guardrails, AI Guardrails, PayloadProcessor, LLM, large language model, Gateway API, Kubernetes, content policy, PII, ai-guardrails module, guardrails -f5-description: How to deploy F5 AI Guardrails with NGINX Gateway Fabric using the PayloadProcessor policy to inspect and block LLM request and response payloads. f5-summary: > Deploy a large language model (LLM) behind NGINX Gateway Fabric, attach a PayloadProcessor policy that routes request and response payloads through an external Guardrails API, and verify that @@ -16,10 +16,10 @@ Learn how to use NGINX Gateway Fabric with F5 AI Guardrails to inspect large lan ## Overview -F5 AI Guardrails has the ability to inspect LLM traffic on two independent paths: +F5 AI Guardrails can inspect LLM traffic on two independent paths: -- **Request path** — the client's *input* is inspected before it reaches the LLM. A block returns `403` with `error.type: invalid_request_error`. -- **Response path** — the model's *output* is inspected before it reaches the client. A block returns `403` with `error.type: api_error`. +- **Prompts** — the client's *input* is inspected before it reaches the LLM. A block returns `403` with `error.type: invalid_request_error`. +- **Responses** — the model's *output* is inspected before it reaches the client. A block returns `403` with `error.type: api_error`. This behavior is provided by the `PayloadProcessor` policy, an [inherited policy]({{< ref "/ngf/overview/custom-policies.md" >}}) that can target an HTTPRoute or a Gateway, which configures NGINX traffic to F5 AI Guardrails to process. @@ -204,10 +204,10 @@ spec: EOF ``` -For an in-cluster backend, your Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to. +For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to. {{< call-out "important" >}} -When using an `ExternalName` Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Configure `dnsResolver` on an [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: +When using an `ExternalName` AI Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Configure `dnsResolver` on an [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: ```yaml apiVersion: gateway.nginx.org/v1alpha2 @@ -310,7 +310,7 @@ Content-Type: application/json {"error":{"type":"api_error","code":"content_policy_violation", ...}} ``` -For more example curl requests, view the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails) `README.md` in the Nginx Gateway Fabric repository. +For more example curl requests, view the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails) `README.md` in the NGINX Gateway Fabric repository. ## Troubleshooting From 4fd0da3f8bc5155a7d31179ba6094c74bd97afd9 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Mon, 10 Aug 2026 22:08:32 -0700 Subject: [PATCH 04/11] Small fix to further reading link --- content/ngf/how-to/f5-ai-guardrails.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index 8cad214cf..9ac744851 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -325,7 +325,7 @@ The `PayloadProcessor` is marked `Accepted=False` when its references cannot be ## Further reading -- [F5 AI Guardrails Documentation](https://docs.aisecurity.f5.com/) +- [Scan streaming in AI Security](https://docs.aisecurity.f5.com/api-docs/scan-request-streaming.html) - [Installation]({{< ref "/ngf/install/" >}}): install NGINX Gateway Fabric with the `PayloadProcessor` policy enabled. - [Custom policies]({{< ref "/ngf/overview/custom-policies.md" >}}): learn how inherited policies attach to Gateway API resources. - [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails): for more information on the example used in this guide. From fdaaee1cc3a80d436f9e5dfc8c0ba8d96ec6e2dc Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Tue, 11 Aug 2026 10:41:20 -0700 Subject: [PATCH 05/11] Add various outputs for shell commands --- content/ngf/how-to/f5-ai-guardrails.md | 73 ++++++++++++++++++++++++-- 1 file changed, 68 insertions(+), 5 deletions(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index 9ac744851..6f3f7b788 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -73,10 +73,15 @@ Deploy the LLM Deployment and Service: kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/examples/guardrails/llm.yaml ``` -Confirm the Pod is `Running`: +Confirm the Pod is `Ready`: ```shell -kubectl get pods -l app=vllm-qwen3-32b +kubectl get deployment vllm-qwen3-32b +``` + +```text +NAME READY UP-TO-DATE AVAILABLE AGE +vllm-qwen3-32b 1/1 1 1 6m13s ``` ## Create a Gateway @@ -157,6 +162,23 @@ Confirm that the HTTPRoute status conditions include `Accepted=True` and `Resolv kubectl describe httproute llm-route ``` +```text +Conditions: + Last Transition Time: 2026-08-11T17:29:19Z + Message: The Route is accepted + Observed Generation: 1 + Reason: Accepted + Status: True + Type: Accepted + Last Transition Time: 2026-08-11T17:29:19Z + Message: All references are resolved + Observed Generation: 1 + Reason: ResolvedRefs + Status: True + Type: ResolvedRefs + Controller Name: gateway.nginx.org/nginx-gateway-controller +``` + ## Create the authentication token Secret Create the Secret with your Guardrails API token under the `token` key. The Secret must live in the same namespace as the `PayloadProcessor`: @@ -259,11 +281,28 @@ EOF Confirm the policy was accepted: ```shell -kubectl get payloadprocessor llm-guardrails -o yaml +kubectl describe payloadprocessor llm-guardrails ``` The status conditions should report `Accepted=True`. A rejected policy reports `Accepted=False`; see [Troubleshooting](#troubleshooting) for common causes. +```text +Conditions: + Last Transition Time: 2026-08-11T17:32:52Z + Message: The Policy is accepted + Observed Generation: 1 + Reason: Accepted + Status: True + Type: Accepted + Last Transition Time: 2026-08-11T17:32:52Z + Message: Policy is programmed in the data plane + Observed Generation: 1 + Reason: Programmed + Status: True + Type: Programmed + Controller Name: gateway.nginx.org/nginx-gateway-controller +``` + ## Send traffic {{< call-out "note" >}} @@ -280,6 +319,20 @@ curl -i --resolve :$GW_PORT:$GW_IP http://:$GW_PORT:$GW_IP http://:$GW_PORT:$GW_IP http://}}/examples/guardrails) `README.md` in the NGINX Gateway Fabric repository. From 9c39ea1d87b1853d8a4f626cd5a47f397258cade Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Tue, 11 Aug 2026 11:11:27 -0700 Subject: [PATCH 06/11] Refactor ordering of sections --- content/ngf/how-to/f5-ai-guardrails.md | 180 +++++++++++++------------ 1 file changed, 91 insertions(+), 89 deletions(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index 6f3f7b788..558e9f378 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -27,28 +27,6 @@ This behavior is provided by the `PayloadProcessor` policy, an [inherited policy You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster. -To enable the `PayloadProcessor` policy, [install]({{< ref "/ngf/install/" >}}) NGINX Gateway Fabric with these modifications: - -- Using Helm: set the `nginxGateway.payloadProcessor.enable=true` Helm value. -- Using Kubernetes manifests: set the `--payload-processor` flag in the nginx-gateway container argument, and update the ClusterRole RBAC to add `payloadprocessors`: - -```yaml -- apiGroups: - - gateway.nginx.org - resources: - - payloadprocessors - verbs: - - get - - list - - watch -- apiGroups: - - gateway.nginx.org - resources: - - payloadprocessors/status - verbs: - - update -``` - ## Deploy an LLM backend If you have an existing in-cluster LLM which can be queried you can skip this section. @@ -84,6 +62,97 @@ NAME READY UP-TO-DATE AVAILABLE AGE vllm-qwen3-32b 1/1 1 1 6m13s ``` +## Create the authentication token Secret + +Create the Secret with your Guardrails API token under the `token` key. The Secret must live in the same namespace as the `PayloadProcessor`: + +```yaml +kubectl apply -f - <" +EOF +``` + +## Configure the Guardrails backend Service + +The Guardrails backend can live outside or inside the cluster. NGINX Gateway Fabric picks the URL scheme from the referenced Service's type: + +| Backend location | Service type | Resolved URL | +| ---------------- | ------------ | ------------ | +| External | `ExternalName` | `https://:` | +| In-cluster | `ClusterIP` (or any non-`ExternalName`) | `http://..svc.cluster.local:` | + +{{< call-out "note" >}} +The `cluster.local` suffix in the in-cluster URL is the cluster's DNS domain. If your cluster uses a different domain, configure it with the `--cluster-domain` flag or `clusterDomain` Helm value when deploying NGINX Gateway Fabric (default: `cluster.local`). +{{< /call-out >}} + +For an external backend, create an `ExternalName` Service pointing at your hosted Guardrails API: + +```yaml +kubectl apply -f - < + ports: + - name: https + port: 443 + protocol: TCP +EOF +``` + +For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to. + +{{< call-out "important" >}} +When using an `ExternalName` AI Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Either edit the NginxProxy which gets created when you deploy NGINX Gateway Fabric, or configure `dnsResolver` on an new [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: + +```yaml +apiVersion: gateway.nginx.org/v1alpha2 +kind: NginxProxy +metadata: + name: guardrails-nginx-config +spec: + dnsResolver: + addresses: + - type: IPAddress + value: "10.96.0.10" # in-cluster kube-dns/CoreDNS ClusterIP (cluster-dependent) +``` + +Find your cluster's DNS ClusterIP with `kubectl -n kube-system get svc kube-dns` (or `coredns`). Without a resolver, NGINX fails to load the configuration with `no resolver defined to resolve `. +{{< /call-out >}} + +## Deploy NGINX Gateway Fabric + +[Install]({{< ref "/ngf/install/" >}}) NGINX Gateway Fabric with the `PayloadProcessor` policy enabled: + +- Using Helm: set the `nginxGateway.payloadProcessor.enable=true` Helm value. +- Using Kubernetes manifests: set the `--payload-processor` flag in the nginx-gateway container argument, and update the ClusterRole RBAC to add `payloadprocessors`: + +```yaml +- apiGroups: + - gateway.nginx.org + resources: + - payloadprocessors + verbs: + - get + - list + - watch +- apiGroups: + - gateway.nginx.org + resources: + - payloadprocessors/status + verbs: + - update +``` + ## Create a Gateway ```yaml @@ -179,73 +248,6 @@ Conditions: Controller Name: gateway.nginx.org/nginx-gateway-controller ``` -## Create the authentication token Secret - -Create the Secret with your Guardrails API token under the `token` key. The Secret must live in the same namespace as the `PayloadProcessor`: - -```yaml -kubectl apply -f - <" -EOF -``` - -## Configure the Guardrails backend Service - -The Guardrails backend can live outside or inside the cluster. NGINX Gateway Fabric picks the URL scheme from the referenced Service's type: - -| Backend location | Service type | Resolved URL | -| ---------------- | ------------ | ------------ | -| External | `ExternalName` | `https://:` | -| In-cluster | `ClusterIP` (or any non-`ExternalName`) | `http://..svc.cluster.local:` | - -{{< call-out "note" >}} -The `cluster.local` suffix in the in-cluster URL is the cluster's DNS domain. If your cluster uses a different domain, configure it with the `--cluster-domain` flag on the NGINX Gateway Fabric controller (default: `cluster.local`). -{{< /call-out >}} - -For an external backend, create an `ExternalName` Service pointing at your hosted Guardrails API: - -```yaml -kubectl apply -f - < - ports: - - name: https - port: 443 - protocol: TCP -EOF -``` - -For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to. - -{{< call-out "important" >}} -When using an `ExternalName` AI Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Configure `dnsResolver` on an [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: - -```yaml -apiVersion: gateway.nginx.org/v1alpha2 -kind: NginxProxy -metadata: - name: guardrails-nginx-config -spec: - dnsResolver: - addresses: - - type: IPAddress - value: "10.96.0.10" # in-cluster kube-dns/CoreDNS ClusterIP (cluster-dependent) -``` - -Find your cluster's DNS ClusterIP with `kubectl -n kube-system get svc kube-dns` (or `coredns`). Without a resolver, NGINX fails to load the configuration with `no resolver defined to resolve `. -{{< /call-out >}} - ## Attach the PayloadProcessor policy Attach the `PayloadProcessor` policy to the HTTPRoute. The `extProcess.backendRef` points at the Guardrails backend Service (using the explicit port), and `authTokenRef` points at the token Secret: From bb956ddedf489ae16a7ad8446c255cf3960473d7 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Tue, 11 Aug 2026 11:15:39 -0700 Subject: [PATCH 07/11] Add cli flag documentation --- content/ngf/reference/cli-help.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/ngf/reference/cli-help.md b/content/ngf/reference/cli-help.md index f70b108d3..d00e65ffa 100644 --- a/content/ngf/reference/cli-help.md +++ b/content/ngf/reference/cli-help.md @@ -63,6 +63,8 @@ This command runs the NGINX Gateway Fabric control plane. | _endpoint-picker-disable-tls_ | _bool_ | Disables TLS when connecting to the EndpointPicker. Set to true only for development/testing or when using a service mesh for encryption. (Default: `false`) | | _endpoint-picker-tls-skip-verify_ | _bool_ | Disables server certificate verification when connecting to the EndpointPicker, if TLS is enabled. REQUIRED: Must be true until Gateway API Inference Extension EndpointPicker supports mounting certificates. (Default `true`) | | _watch-namespaces_ | _list_ | Comma-separated list of namespaces to watch for resources. If not set, all namespaces are watched. The controller's own namespace is always included. | +| _payload-processor_ | _bool_ | Enable the PayloadProcessor API. PayloadProcessors enable declarative, ordered processing of HTTP request and response payloads by attaching to a Gateway or HTTPRoute, and are used to implement features such as Guardrails for AI workloads. | +| _cluster-domain_ | _string_ | The DNS domain of your Kubernetes cluster. | ## Sleep From 50c648371cb1698b0960f0545887752418478080 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Tue, 11 Aug 2026 12:45:54 -0700 Subject: [PATCH 08/11] Add fixes for grammar --- content/ngf/how-to/f5-ai-guardrails.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index 558e9f378..74a2d6e4d 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -21,7 +21,7 @@ F5 AI Guardrails can inspect LLM traffic on two independent paths: - **Prompts** — the client's *input* is inspected before it reaches the LLM. A block returns `403` with `error.type: invalid_request_error`. - **Responses** — the model's *output* is inspected before it reaches the client. A block returns `403` with `error.type: api_error`. -This behavior is provided by the `PayloadProcessor` policy, an [inherited policy]({{< ref "/ngf/overview/custom-policies.md" >}}) that can target an HTTPRoute or a Gateway, which configures NGINX traffic to F5 AI Guardrails to process. +This behavior is provided by the `PayloadProcessor` policy, an [inherited policy]({{< ref "/ngf/overview/custom-policies.md" >}}) that can target an HTTPRoute or a Gateway, which offloads NGINX traffic to F5 AI Guardrails to inspect. ## Before you begin @@ -42,7 +42,7 @@ kubectl create configmap inference-sim-dataset \ ``` {{< call-out "note" >}} -The dataset file alongside more details of the setup can be found in the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails) directory of the NGINX Gateway Fabric repository. +The dataset file, alongside more details of the setup, can be found in the [`examples/guardrails`](https://github.com/nginx/nginx-gateway-fabric/tree/v{{< version-ngf >}}/examples/guardrails) directory of the NGINX Gateway Fabric repository. {{< /call-out >}} Deploy the LLM Deployment and Service: @@ -112,7 +112,7 @@ EOF For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to. {{< call-out "important" >}} -When using an `ExternalName` AI Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Either edit the NginxProxy which gets created when you deploy NGINX Gateway Fabric, or configure `dnsResolver` on an new [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: +When using an `ExternalName` AI Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Either edit the NginxProxy which gets created when you deploy NGINX Gateway Fabric, or configure `dnsResolver` on a new [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: ```yaml apiVersion: gateway.nginx.org/v1alpha2 From e22d06a4a68c7fbe4efca0fa7aec2e8407a62647 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Tue, 11 Aug 2026 22:06:08 -0700 Subject: [PATCH 09/11] Add PayloadProcessor to custom-policies doc --- content/ngf/overview/custom-policies.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/ngf/overview/custom-policies.md b/content/ngf/overview/custom-policies.md index 62600c3fc..3c26df44e 100644 --- a/content/ngf/overview/custom-policies.md +++ b/content/ngf/overview/custom-policies.md @@ -23,6 +23,7 @@ The following table summarizes NGINX Gateway Fabric custom policies: | [SnippetsPolicy]({{< ref "/ngf/traffic-management/snippets.md" >}}) | Inject custom NGINX configuration snippets | Direct | Gateway | Yes | Yes | v1alpha1 | | [UpstreamSettingsPolicy]({{< ref "/ngf/traffic-management/upstream-settings.md" >}}) | Configure upstream load balancing and connection behavior | Direct | Service | Yes | Yes | v1alpha1 | | [WAFPolicy]({{< ref "/ngf/waf-integration/overview.md" >}}) | Apply F5 WAF for NGINX protection (separate add-on to NGINX Plus) | Inherited | Gateway, HTTPRoute, GRPCRoute | Yes | No | v1alpha1 | +| [PayloadProcessor]({{< ref "/ngf/how-to/f5-ai-guardrails.md" >}}) | Secure LLM traffic with F5 AI Guardrails | Inherited | Gateway, HTTPRoute | No | No | v1alpha1 | {{< call-out "important" >}} If attaching a Policy to a Route, that Route must not share a hostname:port/path combination with any other Route that is not referenced by the same Policy. If it does, the Policy will be rejected. This is because the Policy would end up affecting other Routes that it is not attached to. From 5b34f11c00b8600754e293891391142d7b51a1fa Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Tue, 11 Aug 2026 22:17:11 -0700 Subject: [PATCH 10/11] Use tabs for external/internal guardrails backend configs --- content/ngf/how-to/f5-ai-guardrails.md | 31 ++++++++++++++++++++++++-- 1 file changed, 29 insertions(+), 2 deletions(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index 74a2d6e4d..62d48b08d 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -91,7 +91,11 @@ The Guardrails backend can live outside or inside the cluster. NGINX Gateway Fab The `cluster.local` suffix in the in-cluster URL is the cluster's DNS domain. If your cluster uses a different domain, configure it with the `--cluster-domain` flag or `clusterDomain` Helm value when deploying NGINX Gateway Fabric (default: `cluster.local`). {{< /call-out >}} -For an external backend, create an `ExternalName` Service pointing at your hosted Guardrails API: +{{}} + +{{%tab name="External"%}} + +To configure a Guardrails backend Service which is external, create an `ExternalName` Service pointing at your hosted Guardrails API: ```yaml kubectl apply -f - <}} {{< call-out "important" >}} When using an `ExternalName` AI Guardrails backend, you **must** configure a DNS `resolver` so NGINX can resolve the external hostname at request time. Either edit the NginxProxy which gets created when you deploy NGINX Gateway Fabric, or configure `dnsResolver` on a new [NginxProxy]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) resource and attach it to the Gateway via `spec.infrastructure.parametersRef`: From 01e184bc7f5c69fb6d27908a1bf0ec5df434ba77 Mon Sep 17 00:00:00 2001 From: Ben Jee Date: Tue, 11 Aug 2026 22:25:44 -0700 Subject: [PATCH 11/11] Add some additional links to official docs --- content/ngf/how-to/f5-ai-guardrails.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/content/ngf/how-to/f5-ai-guardrails.md b/content/ngf/how-to/f5-ai-guardrails.md index 62d48b08d..078cb3122 100644 --- a/content/ngf/how-to/f5-ai-guardrails.md +++ b/content/ngf/how-to/f5-ai-guardrails.md @@ -21,11 +21,13 @@ F5 AI Guardrails can inspect LLM traffic on two independent paths: - **Prompts** — the client's *input* is inspected before it reaches the LLM. A block returns `403` with `error.type: invalid_request_error`. - **Responses** — the model's *output* is inspected before it reaches the client. A block returns `403` with `error.type: api_error`. -This behavior is provided by the `PayloadProcessor` policy, an [inherited policy]({{< ref "/ngf/overview/custom-policies.md" >}}) that can target an HTTPRoute or a Gateway, which offloads NGINX traffic to F5 AI Guardrails to inspect. +You can set up prompts and responses as described in [Prompts and scans in AI Security](https://docs.aisecurity.f5.com/api-docs/prompts-scans.html), through the F5 AI Guardrails dashboard. + +To connect NGINX Gateway Fabric with your configured F5 AI Guardrails, use the `PayloadProcessor` policy, an [inherited policy]({{< ref "/ngf/overview/custom-policies.md" >}}) that can target an HTTPRoute or a Gateway. The `PayloadProcessor` configures NGINX to offload traffic to F5 AI Guardrails to inspect. ## Before you begin -You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster. +You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster. View the official [F5 AI Guardrails](https://docs.aisecurity.f5.com/) docs to learn more. ## Deploy an LLM backend