NGF: Add guardrails doc by bjee19 · Pull Request #2191 · nginx/documentation

bjee19 · 2026-08-07T23:48:56Z

Add document for NGF's integration with F5 AI Guardrails.

Checklist

Before sharing this pull request, I completed the following checklist:

I read the Contributing guidelines
My branch adheres to the Git conventions
My content changes adhere to the F5 Technical Writing Style Guide
If my changes involve potentially sensitive information¹, I have assessed the possible impact
I have waited to ensure my changes pass tests, and addressed any discovered issues

Potentially sensitive information includes personally identify information (PII), authentication credentials, and live URLs. Refer to the style guide for guidance about placeholder content. ↩

mjang

Some of my comments depend on whether / where we refer to F5 AI Guardrails or NGF guardrails

mjang · 2026-08-10T13:47:16Z

+
+## Before you begin
+
+You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster.


We have many F5 AI Guardrails API endpoints

I think the right endpoint might be: https://docs.aisecurity.f5.com/operations/get_prompts.html ?

Oh yea those are endpoints/paths on a specific F5 AI Guardrails service. I guess I was talking about the F5 AI Guardrails service. such as us1.calypsoai.app / eu1.calypsoai.app, or a self hosted endpoint.

When a doc says:

"You need an F5 AI Guardrails API endpoint to inspect payloads."

I think: "Which endpoint"? I made my best guess.

Oh yea those are endpoints/paths on a specific F5 AI Guardrails service. I guess I was talking about the F5 AI Guardrails service. such as us1.calypsoai.app / eu1.calypsoai.app, or a self hosted endpoint.

As you can see from the link, you can include the server in the URL. The (changeable) default shown in the AI Security API doc is us1.calypsoai.app.

So do you think I should give the example us1.calypsoai.app to help clarify what endpoint means?

because that endpoint can be different per user. they should/would know the answer to "which endpoint?"

Actually the base URL depends on the deployment -- which varies per customer (not user?)

us1.calypsoai.app or eu1.calypsoai.app would correspond to a SaaS install. For on prem, the base URL depends on the deploying customer. Ref: https://docs.aisecurity.f5.com/integrations/_base_vars.html

Sorry yea the base url can vary per customer, how would you suggest this portion be changed to account for that?

I differentiate between BASE_URL and endpoint.

BASE_URL = something like us1.calypsoai.app

Endpoint = what's added to the end of the BASE_URL

Based on what I see, I'm guessing it's based on the scans endpoint

With examples such as https://docs.aisecurity.f5.com/api-docs/scan-request-streaming.html#scan-a-request-body-before-you-send-it

Suggested change

You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster.

To inspect payloads, you can use the F5 AI Guardrails [scans](https://docs.aisecurity.f5.com/operations/get_scans.html) endpoint. Specifically, this can be an F5 hosted service or a service running inside your cluster.

With the `scans/` endpoint, the server URL varies depending on whether you've deployed AI Security on your system, or use the AI Security SaaS deployment. For more information, see our description of [BASE_VARS](https://docs.aisecurity.f5.com/integrations/_base_vars.html).

Yea, though currently NGF doesn't allow for any other endpoint besides the scans endpoint. We don't allow the user to configure any endpoint and it almost feels like an implementation detail that is unnecessary to bring up.

I'd rather hide that part of the implementation and perhaps add a small line just saying something like "NGINX Gateway Fabric uses the scans/ endpoint when communicating with the F5 AI Guardrails API for both requests and responses (and link to the docs just so they can look into it if they want)" than insinuate that the user can configure this endpoint.

mjang · 2026-08-10T13:52:36Z

+The `cluster.local` suffix in the in-cluster URL is the cluster's DNS domain. If your cluster uses a different domain, configure it with the `--cluster-domain` flag on the NGINX Gateway Fabric controller (default: `cluster.local`).
+{{< /call-out >}}
+
+For an external backend, create an `ExternalName` Service pointing at your hosted Guardrails API:


Question: with the Guardrails API, are we referring to some F5 API? (That may change some of my comments)

If it isn't an F5 API, we need to figure out how to clarify:

What's a NGF guardrail

What's used for F5 AI Guardrails

Guardrails API is referring to this product https://docs.aisecurity.f5.com/api-docs/getting-started-defend.html. and this https://docs.aisecurity.f5.com/api-docs/prompts-scans.html.

There is no NGF guardrails.

But in that section I don't put the full "F5 AI Guardrails" because they are self-hosted, meaning they aren't sending requests to https://us1.calypsoai.app/, they are sending to their own hosted version of F5 AI Guardrails. I can adjust if necessary.

I've added a suggstion earlier (and will add one for the end of the page)

mjang

I've suggested a few other changes. Thanks for clarifying how NGF integrates

mjang · 2026-08-10T21:12:01Z

+
+## Before you begin
+
+You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster.


I differentiate between BASE_URL and endpoint.

BASE_URL = something like us1.calypsoai.app

Endpoint = what's added to the end of the BASE_URL

Based on what I see, I'm guessing it's based on the scans endpoint

With examples such as https://docs.aisecurity.f5.com/api-docs/scan-request-streaming.html#scan-a-request-body-before-you-send-it

Suggested change

You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster.

To inspect payloads, you can use the F5 AI Guardrails [scans](https://docs.aisecurity.f5.com/operations/get_scans.html) endpoint. Specifically, this can be an F5 hosted service or a service running inside your cluster.

With the `scans/` endpoint, the server URL varies depending on whether you've deployed AI Security on your system, or use the AI Security SaaS deployment. For more information, see our description of [BASE_VARS](https://docs.aisecurity.f5.com/integrations/_base_vars.html).

mjang · 2026-08-10T21:31:34Z

+The `cluster.local` suffix in the in-cluster URL is the cluster's DNS domain. If your cluster uses a different domain, configure it with the `--cluster-domain` flag on the NGINX Gateway Fabric controller (default: `cluster.local`).
+{{< /call-out >}}
+
+For an external backend, create an `ExternalName` Service pointing at your hosted Guardrails API:


I've added a suggstion earlier (and will add one for the end of the page)

mjang · 2026-08-10T21:35:21Z

I'm going to follow up with the AI Security PM on Tuesday Aug 11 (we have a weekly meeting).

shaun-nx · 2026-08-11T13:56:20Z

+{{< call-out "note" >}}
+The `cluster.local` suffix in the in-cluster URL is the cluster's DNS domain. If your cluster uses a different domain, configure it with the `--cluster-domain` flag on the NGINX Gateway Fabric controller (default: `cluster.local`).
+{{< /call-out >}}


I wonder if we should make a reference to this section in the "Before you begin" section. If a user is following this document, they would have first installed NGF with nginxGateway.payloadProcessor.enable=true, then by the time they get to this step, the realize they need to set their own cluster domain, and then need to upgrade/reinstall NGF again.

It might be fine though as I think about it. What do you think?

I just don't think they should install NGF in the before you begin section.

I personally don't think its a big enough issue to make the change. Though it would be solved if we don't install NGF in the before you begin section.

@sjberman did you have a place in mind to install NGF? Technically we could Deploy the LLM, create the gateway, secret, guardrails backend Service before deploying NGF.

That sounds fine to me. Maybe do the gateway after installing NGF

sjberman · 2026-08-11T18:27:24Z

+
+## Before you begin
+
+You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster.


Are there Guardrails installation/usage docs? Feels like we should link out to them.

added a small link. It's difficult since guardrails doesn't have guides on using the dashboard, which is what I think users would be using.

sjberman · 2026-08-11T18:28:34Z

+For an external backend, create an `ExternalName` Service pointing at your hosted Guardrails API:
+
+```yaml
+kubectl apply -f - <<EOF
+apiVersion: v1
+kind: Service
+metadata:
+  name: guardrails-api
+spec:
+  type: ExternalName
+  externalName: <GUARDRAILS_API_HOSTNAME>
+  ports:
+  - name: https
+    port: 443
+    protocol: TCP
+EOF
+```
+
+For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to.


could we use tabs for the external/internal differing configs? Kind of nice, concise way to display multiple options in a doc, similar to our installation docs with OSS/Plus

JTorreG · 2026-08-12T13:42:28Z

+
+{{%tab name="External"%}}
+
+To configure a Guardrails backend Service which is external, create an `ExternalName` Service pointing at your hosted Guardrails API:


should this not be...

Suggested change

To configure a Guardrails backend Service which is external, create an `ExternalName` Service pointing at your hosted Guardrails API:

To configure a AI Guardrails backend Service which is external, create an `ExternalName` Service pointing at your hosted Guardrails API:

?
or the backend service is just "Guardrails" ?

JTorreG · 2026-08-12T13:42:50Z

+
+{{%tab name="In-cluster"%}}
+
+For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to, otherwise create a Service configured to expose your guardrail backends:


Suggested change

For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to, otherwise create a Service configured to expose your guardrail backends:

For an in-cluster backend, your AI Guardrails backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to, otherwise create a Service configured to expose your guardrail backends:

See also my other question

If it's the name of a product of service, it needs to be consistent (AI Guardrails) across the document. If it's not a product or service it needs to be lowercase...

JTorreG · 2026-08-12T13:44:20Z

+## Send traffic
+
+{{< call-out "note" >}}
+Whether a given value is blocked depends entirely on your Guardrails backend's detector configuration. Enable the relevant detectors on your Guardrails service to see the block responses above.


Suggested change

Whether a given value is blocked depends entirely on your Guardrails backend's detector configuration. Enable the relevant detectors on your Guardrails service to see the block responses above.

Whether a given value is blocked depends entirely on your AI Guardrails backend's detector configuration. Enable the relevant detectors on your Guardrails service to see the block responses above.

JTorreG · 2026-08-12T13:44:33Z

+```
+
+
+If the request payload contains content that your Guardrails backend is configured to block, the request never reaches the LLM and returns `HTTP 403`:


Suggested change

If the request payload contains content that your Guardrails backend is configured to block, the request never reaches the LLM and returns `HTTP 403`:

If the request payload contains content that your AI Guardrails backend is configured to block, the request never reaches the LLM and returns `HTTP 403`:

JTorreG · 2026-08-12T13:44:43Z

+{"error":{"code":"content_policy_violation","message":"Request blocked by guardrails policy.","param":null,"type":"invalid_request_error"}
+```
+
+If the model's *output* contains content that your Guardrails backend blocks, the response is withheld from the client and returns `HTTP 403` with `error.type: api_error`:


Suggested change

If the model's *output* contains content that your Guardrails backend blocks, the response is withheld from the client and returns `HTTP 403` with `error.type: api_error`:

If the model's *output* contains content that your AI Guardrails backend blocks, the response is withheld from the client and returns `HTTP 403` with `error.type: api_error`:

JTorreG

approved with nits

bjee19 · 2026-08-12T16:44:38Z

btw, waiting on some ongoing changes in nginx/nginx-gateway-fabric#5697 before merging. Will merge after that PR is merged.

mjang

Approving based on @JTorreG 's comments / suggestions

bjee19 added 2 commits August 7, 2026 16:47

Add guardrails doc

e7d94ae

Use document IP address

5fcfe96

bjee19 requested a review from a team as a code owner August 7, 2026 23:48

github-actions Bot added documentation Improvements or additions to documentation product/ngf Issues related to NGINX Gateway Fabric labels Aug 7, 2026

bjee19 requested a review from a team August 7, 2026 23:49

mjang reviewed Aug 10, 2026

View reviewed changes

Add suggested fixes

e6ebd7d

mjang reviewed Aug 10, 2026

View reviewed changes

Small fix to further reading link

4fd0da3

shaun-nx reviewed Aug 11, 2026

View reviewed changes

Comment thread content/ngf/how-to/f5-ai-guardrails.md

shaun-nx reviewed Aug 11, 2026

View reviewed changes

Comment thread content/ngf/how-to/f5-ai-guardrails.md

shaun-nx reviewed Aug 11, 2026

View reviewed changes

Comment thread content/ngf/how-to/f5-ai-guardrails.md

shaun-nx reviewed Aug 11, 2026

View reviewed changes

Comment thread content/ngf/how-to/f5-ai-guardrails.md

bjee19 added 3 commits August 11, 2026 10:41

Add various outputs for shell commands

fdaaee1

Refactor ordering of sections

9c39ea1

Add cli flag documentation

bb956dd

bjee19 requested review from mjang, shaun-nx and sjberman August 11, 2026 18:18

sjberman reviewed Aug 11, 2026

View reviewed changes

bjee19 added 4 commits August 11, 2026 12:45

Add fixes for grammar

50c6483

Add PayloadProcessor to custom-policies doc

e22d06a

Use tabs for external/internal guardrails backend configs

5b34f11

Add some additional links to official docs

01e184b

JTorreG reviewed Aug 12, 2026

View reviewed changes

JTorreG approved these changes Aug 12, 2026

View reviewed changes

sjberman approved these changes Aug 12, 2026

View reviewed changes

shaun-nx approved these changes Aug 12, 2026

View reviewed changes

mjang approved these changes Aug 12, 2026

View reviewed changes


		## Before you begin

		You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster.

-You need an F5 AI Guardrails API endpoint to inspect payloads. This can be an F5 hosted service or a service running inside your cluster.
+To inspect payloads, you can use the F5 AI Guardrails [scans](https://docs.aisecurity.f5.com/operations/get_scans.html) endpoint. Specifically, this can be an F5 hosted service or a service running inside your cluster.
+With the `scans/` endpoint, the server URL varies depending on whether you've deployed AI Security on your system, or use the AI Security SaaS deployment. For more information, see our description of [BASE_VARS](https://docs.aisecurity.f5.com/integrations/_base_vars.html).


		{{%tab name="External"%}}

		To configure a Guardrails backend Service which is external, create an `ExternalName` Service pointing at your hosted Guardrails API:

	To configure a Guardrails backend Service which is external, create an `ExternalName` Service pointing at your hosted Guardrails API:
	To configure a AI Guardrails backend Service which is external, create an `ExternalName` Service pointing at your hosted Guardrails API:


		{{%tab name="In-cluster"%}}

		For an in-cluster backend, your AI Guardrail backend pods will most likely have an existing Service which you can point the PayloadProcessor backendRef to, otherwise create a Service configured to expose your guardrail backends:

	Whether a given value is blocked depends entirely on your Guardrails backend's detector configuration. Enable the relevant detectors on your Guardrails service to see the block responses above.
	Whether a given value is blocked depends entirely on your AI Guardrails backend's detector configuration. Enable the relevant detectors on your Guardrails service to see the block responses above.

		```


		If the request payload contains content that your Guardrails backend is configured to block, the request never reaches the LLM and returns `HTTP 403`:

	If the model's output contains content that your Guardrails backend blocks, the response is withheld from the client and returns `HTTP 403` with `error.type: api_error`:
	If the model's output contains content that your AI Guardrails backend blocks, the response is withheld from the client and returns `HTTP 403` with `error.type: api_error`:

Uh oh!

Conversation

bjee19 commented Aug 7, 2026

Checklist

Footnotes

Uh oh!

mjang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mjang left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mjang commented Aug 10, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sjberman Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sjberman Aug 11, 2026 •

edited

Loading