Content Guardrails

GuardrailPolicy evaluates request or response payloads for an AIServiceBackend. Rules can use local regular expressions, native integrations with external content-safety providers, or any custom guardrail service that implements a small HTTP contract.

Apply a regex guardrail

apiVersion: aigateway.envoyproxy.io/v1beta1
kind: GuardrailPolicy
metadata:
  name: pii-guardrails
spec:
  maxRequestBodyBytes: 10485760
  maxResponseBodyBytes: 10485760
  targetRefs:
    - group: aigateway.envoyproxy.io
      kind: AIServiceBackend
      name: openai
  rules:
    - name: block-sensitive-input
      phase: Request
      provider:
        type: Regex
        pattern: "(?i)password|social security number"
        action: Block
        message: Request contains sensitive content

A matched rule returns HTTP 403 with error type GuardrailViolation. Rules are scoped to the generated backends that reference the targeted AIServiceBackend.

Rules support three actions:

• Block rejects matching traffic.
• Monitor records matching traffic without blocking or changing it.
• Mask replaces detected text. Regex and Presidio use maskReplacement; Bedrock uses transformed output returned by the provider; HTTP uses the returned replacement or masks returned findings with maskReplacement. Azure Text Analysis does not support Mask.

When multiple policies target one backend, policies are evaluated in namespace/name order and rules retain declaration order. The first Block result stops evaluation.

External providers

Presidio

Presidio calls the analyzer POST /analyze endpoint. scoreThresholdPercent accepts values from 0 through 100. Authentication is optional.

provider:
  type: Presidio
  timeoutSeconds: 5
  failureMode: FailClosed
  presidio:
    endpoint: http://presidio-analyzer.presidio.svc.cluster.local:3000
    language: en
    scoreThresholdPercent: 70
    apiKeySecretRef:
      name: presidio-key

When configured, the Secret must contain an apiKey entry, sent as a bearer token.

AWS Bedrock Guardrails

Bedrock uses the ApplyGuardrail API and SigV4 signing. By default, the ext-proc uses the standard AWS credential chain, including IRSA and EKS Pod Identity.

provider:
  type: Bedrock
  bedrock:
    region: us-east-1
    guardrailIdentifier: my-guardrail
    guardrailVersion: "1"

For static credentials, set credentialsSecretRef to a Secret whose credentials entry contains an AWS shared credentials file. endpoint can override the public Bedrock runtime endpoint for a private endpoint.

Azure AI Content Safety

provider:
  type: AzureContentSafety
  azureContentSafety:
    endpoint: https://my-resource.cognitiveservices.azure.com
    apiVersion: "2024-09-01"
    severityThreshold: 4
    apiKeySecretRef:
      name: azure-content-safety-key

The referenced Secret must contain an apiKey entry.

Custom HTTP guardrails

Use the HTTP provider to integrate a guardrail service that has no native integration, such as an in-house classifier. The gateway calls the service over HTTP instead of running local executables.

provider:
  type: HTTP
  action: Mask
  timeoutSeconds: 5
  failureMode: FailClosed
  http:
    endpoint: http://custom-guardrail.guardrails.svc.cluster.local:8080
    path: /analyze
    apiKeySecretRef:
      name: custom-guardrail-key

path defaults to /analyze. When apiKeySecretRef is set, the Secret must contain an apiKey entry, sent as a bearer token.

Request

For each extracted text fragment, the gateway sends POST {endpoint}{path}:

{
  "text": "some user input",
  "context": {
    "stage": "input"
  }
}

context.stage is input for Request rules and output for Response rules. Services should ignore unknown fields, because more context may be added later.

Response

The service must return HTTP 2xx with:

{
  "action": "allow",
  "findings": [
    {
      "type": "PII",
      "start": 10,
      "end": 20,
      "score": 0.92
    }
  ]
}

Field	Required	Description
action	yes	allow, block, or modify.
findings	no	Detected spans. start and end are Unicode code point offsets into text, with end exclusive. type and score are informational.
replacement	no	Replacement text for the whole fragment, used with modify.

The gateway interprets the response as follows:

Service action	Result
allow	The rule does not match.
block	The rule matches. Findings, if present, are masked with maskReplacement for Mask rules.
modify	The rule matches. replacement is used when present; otherwise findings are masked with maskReplacement.

The rule's action decides what happens on a match: Block rejects the request or response, Monitor records it, and Mask rewrites the fragment. A Mask rule fails when the service reports a match without a replacement or valid findings, so unmasked content is never forwarded.

Non-2xx responses, invalid JSON, and missing or unknown actions are provider errors handled by failureMode.

A minimal service that blocks prompts containing a keyword could look like this:

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class Context(BaseModel):
    stage: str

class Request(BaseModel):
    text: str
    context: Context

@app.post("/analyze")
def analyze(req: Request):
    start = req.text.lower().find("confidential")
    if start < 0:
        return {"action": "allow"}
    return {
        "action": "block",
        "findings": [{"type": "KEYWORD", "start": start, "end": start + len("confidential"), "score": 1.0}],
    }

Python string indexes are Unicode code points, which matches the offsets expected by the gateway.

Failure behavior

External providers default to failureMode: FailClosed. Provider errors, missing Secrets, and revoked credentials prevent unchecked traffic. Use FailOpen only when availability is more important than enforcement:

provider:
  type: Presidio
  failureMode: FailOpen
  timeoutSeconds: 3
  presidio:
    endpoint: http://presidio-analyzer.presidio.svc.cluster.local:3000

Credential Secret changes automatically requeue the policy and regenerate affected gateway configuration.

Streaming responses

When an applicable response guardrail exists, the gateway keeps the upstream response buffered until evaluation completes. This prevents unsafe content from being partially delivered before a blocking or masking decision and gives Monitor rules a complete payload. Routes without response guardrails retain normal streaming behavior.

Request and response limits default to 10 MiB and can be configured independently with maxRequestBodyBytes and maxResponseBodyBytes, up to 50 MiB. An oversized payload follows the rule's failure mode.

Observability

The ext-proc emits structured block and provider-failure logs without payload content. It also records guardrail.evaluation span events and the counter:

aigateway.guardrail.evaluation.count

The counter attributes are aigateway.guardrail.phase (Request or Response) and aigateway.guardrail.result (allowed, blocked, or error). With the Prometheus exporter, dots are converted to underscores.

More examples

See examples/guardrails for local and external-provider manifests.

Optional live-provider tests

Presidio is tested against its official analyzer image with Testcontainers. The test runs automatically when Docker is available and skips otherwise:

go test ./internal/guardrails -run '^TestPresidioEvaluatorContainer$' -v

The Azure and Bedrock live tests are disabled unless all required environment variables for a provider are set:

• Presidio managed/external deployment: TEST_PRESIDIO_ENDPOINT, TEST_PRESIDIO_BLOCKED_TEXT, and optionally TEST_PRESIDIO_API_KEY.
• Azure: TEST_AZURE_CONTENT_SAFETY_ENDPOINT, TEST_AZURE_CONTENT_SAFETY_API_KEY, and TEST_AZURE_CONTENT_SAFETY_BLOCKED_TEXT.
• Bedrock: TEST_AWS_BEDROCK_GUARDRAIL_REGION, TEST_AWS_BEDROCK_GUARDRAIL_ID, TEST_AWS_BEDROCK_GUARDRAIL_VERSION, and TEST_AWS_BEDROCK_GUARDRAIL_BLOCKED_TEXT. AWS credentials use the standard credential chain.

Run them with:

go test ./internal/guardrails -run '^TestLive' -v