Rules and Guards

Ensuring the safety of your community is essential for building trust and loyalty among your users. When people feel secure and welcome, they are more likely to engage with your service. While UGC Guard’s reporting feature allows users to flag unwanted content, proactive protection is even more effective. Why wait for reports when you can prevent harmful content at the source?

With UGC Guard’s Rules and Guards feature, you can implement robust safeguards that analyze and filter content as it’s submitted. This enables you to intervene before any user is exposed to inappropriate material.

Rules

Rules determine how UGC Guard evaluates the content you send. There are two types: deterministic and non-deterministic rules. Rules are defined at the organization level, making them reusable across all your modules.

Deterministic Rules

Deterministic rules always produce the same result for a given input. They operate using either a blacklist or whitelist and will break once a specified threshold is reached.

• With a blacklist, the threshold is met when the number or percentage of prohibited words in the content exceeds your limit.
• With a whitelist, the threshold is reached when the number or percentage of undefined words surpasses your set value.

The percentage (referred to as density) is calculated by dividing the number of relevant words by the total word count.

Non-deterministic Rules

WARNING

Using non-deterministic rules may incur additional costs with your AI provider (self-hosted) or with UGC Guard (SaaS).

Non-deterministic rules may yield different results for the same input. They leverage large language models (LLMs) such as ChatGPT or Mistral, and are defined using customizable prompts. You can use template tags in your prompts to tailor the LLM’s behavior to your specific needs.

These rules return a severity score from 1 to 5, and will break if the score reaches 3 or higher.

Guards

Guards are collections of rules that are executed together. They are defined at the module level.

Defining Guards

TIP

Always run deterministic rules first. Only proceed to non-deterministic rules if the deterministic ones do not break.

You can configure Guards by selecting the rules and specifying their order. Each Guard receives a unique URL, which you can use to submit content for evaluation.

Using Guards

To use a Guard, send a POST request to its dedicated endpoint, including your module secret in the request header as X-Module-Secret. You can find each Guard’s unique URL in the user interface. Guard endpoints follow the same structure as the magic report endpoint, making it easy to adapt your existing reporting code for moderation purposes.

When you submit content to a Guard, your request is placed in UGC Guard’s task queue and processed as soon as a worker is available. This means there may be a short delay before you receive a response. The enqueue endpoint returns an evaluation ID, which you can use to check the status of your request.

To track progress, periodically query UGC Guard using the evaluation ID. The response includes an ongoing property. If True, the evaluation is still in progress. Once ongoing becomes False, you can check the passed property to determine whether the content passed moderation or if the guard intervened.

TIP

If you’ve already integrated UGC Guard’s reporting feature, you can reuse your content conversion logic for Guard evaluations.

package.py

"""
This is an example using the UGC Guard python package. 
It automatically uploads (non-proxied) MediaFiles when
calling the guard_evaluation endpoint.

If you do not want to use the UGC Guard Python Package, 
have a look at `request.py` which is manually calling the endpoints.
"""

from ugc_guard_python.ugc_guard_python import GuardClient
from ugc_guard_python.wrapper.content_wrapper import (
    ContentWrapper, TextBody, ProxiedMultiMultiMediaBody,ReportContent
)
from time import sleep

organization_id: str = "YOUR_ORGANIZATION_ID"
module_id: str = "YOUR_MODULE_ID"
module_secret: str = "YOUR_MODULE_SECRET"
guard_id: str = "YOUR_GUARD_ID"
guard_client = GuardClient(organization_id)

response = guard_client.enqueue_guard(
    guard_id,
    content=ContentWrapper(
        content=ReportContent(
            body=ProxiedMultiMultiMediaBody(
                body="This is an example text with an image. Media is only validated by non-deterministic Rules.",
                urls=[
                    "https://picsum.photos/200/300"
                ]
            ),
            additional_data={
                "description": "Any extra data is not validated by the Guard"
            },
            unique_partner_id="-1"
        ),   
    ),
    context=[
        ContentWrapper(
            content=ReportContent(
                body=TextBody(
                    body="This is some context text."
                ),
                unique_partner_id="-2"
            )
        )
    ]
)

if response:
    evaluation_id = response.id # The ID of the created evaluation
    wait_period = 1 # Seconds
    while True:
      sleep(wait_period)
      status_response = guard_client.get_guard_evaluation(evaluation_id, module_secret)
      ongoing = status_response.guard_evaluation.ongoing
      if ongoing:
        wait_period = min(wait_period * 2, 30)  # Cap wait time to avoid long delays
        continue
      # Moderation result: True if content passed, False if blocked
      passed = status_response.guard_evaluation.passed
      break

request.py

import requests
from time import sleep

# Submit content to the Guard for evaluation
response = requests.post(
  "https://api.ugc-guard.com/guards/{guard_id}/enqueue",
  headers={
    "Content-Type": "application/json",
    "X-Module-Secret": "{module_secret}"
  },
  json={
    "content": {
      "body_type": "image",
      "body": "This is an example text with an image. Media is only validated by non-deterministic Rules.",
      "media_identifiers": [
        "https://picsum.photos/200/300"
      ],
      "extra_data": {
        "description": "Any extra data is not validated by the Guard"
      },
      "creator_id": "-1"
    },
    "context": [
      {
        "body_type": "text",
        "body": "This is some context",
        "creator_id": "-1"
      }
    ]
  }
)

if response.status_code == 200:
  evaluation_id = response.json().get("id", None)
  if evaluation_id:
    wait_period = 1
    while True:
      sleep(wait_period)
      status_response = requests.get(
        f"https://api.ugc-guard.com/guards/results/{evaluation_id}",
        headers={
          "Content-Type": "application/json",
          "X-Module-Secret": "{module_secret}"
        }
      )
      result = status_response.json().get("guard_evaluation", {})
      if result.get("ongoing", True):
        wait_period = min(wait_period * 2, 30)  # Cap wait time to avoid long delays
        continue
      # Moderation result: True if content passed, False if blocked
      passed = result.get("passed", False)
      break

TIP

It's recommended not to block users while waiting for Guard evaluation. Allow users to proceed, and if the Guard later intervenes, quarantine the content and notify the user that their submission was not accepted.