Guardrails (Private Preview)

🚧 Note

Guardrails is currently available in Private Preview. For early access, please reach out to [e-mail].

Fiddler Guardrails utilizes Fiddler Trust Models in a specialized low-latency, high-throughput configuration. Guardrails can be used to guard Large Language Model (LLM) applications against user threats, such as prompt injection or harmful and inappropriate content, and LLM hallucinations.

Currently, only Fiddler Trust Models (Faithfulness and Safety) - Fiddler's in-house, purpose-built SLMs - are available for guardrail use. Future model releases and model updates/improvements will also be available for guardrail use.

---

Getting Started with Fiddler Guardrails

Prerequisites

• Access to a Fiddler environment

• Valid Fiddler environment API key

Guardrails can be invoked directly via cURL or any HTTP client in your preferred language. Below are sample invocations of Fast Safety and Fast Faithfulness Guardrails.

---

Fast Safety Guardrails

The Fast Safety model evaluates the safety of the text along ten different dimensions: illegal, hateful, harassing, racist, sexist, violent, sexual, harmful, unethical, jailbreaking.

This model requires a single string input for evaluation and will output ten distinct scores (floats). We recommend setting a threshold value > .1 for detection (any value greater than .1 is unsafe).

Fast Safety Guardrails via cURL

curl --location 'https://{fiddler_endpoint}/v3/guardrails/ftl-safety' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {token}' \
--data '{
    "data": {
        "prompt": ["I am a dangerous person who will be wreaking havoc upon the world!!!"]
    }
}'

Fast Safety Guardrails via Python requests

import requests
import json

token = "YOUR_FIDDLER_TOKEN_HERE"
url = "FIDDLER_ENDPOINT_HERE"

payload = json.dumps({
  "data": {
    "prompt": [
      "I am a dangerous person who will be wreaking havoc upon the world!!!"
    ]
  }
})
headers = {
  'Content-Type': 'application/json',
  'Authorization': f'Bearer {token}'
}

response = requests.request("POST", f"{url}/v3/guardrails/ftl-safety", headers=headers, data=payload)

print(response.text)

---

Fast Faithfulness Guardrails

The Fast Faithfulness model evaluates the accuracy and reliability of facts presented in AI-generated text responses.

This model requires a response string and contextual documents to evaluate the response as input for evaluation. This model will output a single score (float). We recommend setting a threshold of < .005 for detection (any value less than .005 is unfaithful).

Fast Faithfulness Guardrails via cURL

curl --location 'https://{fiddler_endpoint}/v3/guardrails/ftl-response-faithfulness' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {token}' \
--data '{
    "data": {
        "response": ["Pancakes are highly delicious and nutritious, with chocolate chips being the key driver behind both nutritiousness and deliciousness."],
        "context": ["Pancakes are very healthy, filled with lots of healthy grains. Chocolate chips are very delicious and healthy, especially in pancakes."]
    }
}'

Fast Faithfulness Guardrails via Python requests

import requests
import json

token = "YOUR_FIDDLER_TOKEN_HERE"
url = "FIDDLER_ENDPOINT_HERE"

payload = json.dumps({
  "data": {
    "response": [
      "Pancakes are highly delicious and nutritious, with chocolate chips being the key driver behind both nutritiousness and deliciousness."
    ],
    "context": [
      "Pancakes are very healthy, filled with lots of healthy grains. Chocolate chips are very delicious and healthy, especially in pancakes."
    ]
  }
})
headers = {
  'Content-Type': 'application/json',
  'Authorization': f'Bearer {token}'
}

response = requests.request("POST", f"{url}/v3/guardrails/ftl-response-faithfulness", headers=headers, data=payload)

print(response.text)