Examples
Content Moderation

Content Moderation

This example demonstrates how to build an autonomous content moderation system using NudgeLang. The system can analyze content, detect violations, and take appropriate actions.

Overview

The Content Moderation system is designed to:

  • Analyze content for violations
  • Detect inappropriate material
  • Enforce community guidelines
  • Take automated actions
  • Escalate when necessary
  • Maintain audit logs

Implementation 

name: content_moderation
version: 1.0.0
description: Autonomous content moderation system

states:
  - id: start
    type: initial
    next: analyze_content
  
  - id: analyze_content
    type: llm
    model: analyzer
    prompt: |
      Analyze this content:
      {content}
      
      Consider:
      1. Content type
      2. Context
      3. Language
      4. Media elements      
    input:
      content: "${input.content}"
    next: check_violations
  
  - id: check_violations
    type: llm
    model: violation_checker
    prompt: |
      Check for policy violations in:
      {content}
      
      Evaluate against:
      - Community guidelines
      - Content policies
      - Legal requirements
      - Platform rules      
    input:
      content: "${input.content}"
    next: assess_severity
  
  - id: assess_severity
    type: llm
    model: severity_assessor
    prompt: |
      Assess violation severity:
      
      Content: {content}
      Violations: {violations}
      
      Consider:
      - Policy type
      - Violation extent
      - User history
      - Content impact      
    input:
      content: "${input.content}"
      violations: "${previous.output}"
    next: determine_action
  
  - id: determine_action
    type: llm
    model: action_planner
    prompt: |
      Determine appropriate action:
      
      Content: {content}
      Violations: {violations}
      Severity: {severity}
      
      Possible actions:
      - Allow
      - Flag for review
      - Remove content
      - Suspend user
      - Ban user      
    input:
      content: "${input.content}"
      violations: "${states.check_violations.output}"
      severity: "${previous.output}"
    next: execute_action
  
  - id: execute_action
    type: tool
    tool: "${previous.output.action}"
    parameters:
      content_id: "${input.content_id}"
      user_id: "${input.user_id}"
      reason: "${previous.output.reason}"
      action: "${previous.output.action}"
    next: log_action
  
  - id: log_action
    type: tool
    tool: create_audit_log
    parameters:
      content_id: "${input.content_id}"
      user_id: "${input.user_id}"
      action: "${states.determine_action.output.action}"
      reason: "${states.determine_action.output.reason}"
      violations: "${states.check_violations.output}"
      severity: "${states.assess_severity.output}"
    next: check_appeal
  
  - id: check_appeal
    type: llm
    model: appeal_checker
    prompt: |
      Check if appeal is needed:
      
      Action: {action}
      Severity: {severity}
      User History: {history}      
    input:
      action: "${states.determine_action.output.action}"
      severity: "${states.assess_severity.output}"
      history: "${context.user_history}"
    transitions:
      - when: "${output.needs_appeal}"
        next: notify_user
      - next: end
  
  - id: notify_user
    type: tool
    tool: send_notification
    parameters:
      user_id: "${input.user_id}"
      message: "${previous.output.notification}"
      action: "${states.determine_action.output.action}"
    next: end
  
  - id: end
    type: output
    value: "${context.moderation_result}"

Key Features

  1. Content Analysis

    • Identifies content type
    • Analyzes context
    • Evaluates language
    • Processes media

  2. Violation Detection

    • Checks guidelines
    • Verifies policies
    • Ensures compliance
    • Monitors rules

  3. Severity Assessment

    • Evaluates violations
    • Considers history
    • Assesses impact
    • Determines risk

  4. Action Management

    • Takes appropriate actions
    • Maintains audit logs
    • Handles appeals
    • Notifies users

  5. Policy Enforcement

    • Applies guidelines
    • Enforces rules
    • Maintains consistency
    • Ensures fairness

Best Practices

  1. Content Analysis

    • Use multiple models
    • Consider context
    • Handle different formats
    • Process media effectively

  2. Policy Enforcement

    • Define clear rules
    • Update guidelines
    • Maintain consistency
    • Document decisions

  3. Action Management

    • Take appropriate actions
    • Log all decisions
    • Handle appeals
    • Notify users

  4. Performance Monitoring

    • Track accuracy
    • Measure response time
    • Monitor false positives
    • Analyze patterns

Common Use Cases

  1. Social Media Moderation

    • Post analysis
    • Comment filtering
    • User management
    • Policy enforcement

  2. Forum Moderation

    • Thread monitoring
    • User behavior
    • Content quality
    • Community guidelines

  3. Content Platform

    • Media review
    • Policy compliance
    • User management
    • Quality control

  4. E-commerce

    • Product listings
    • Review moderation
    • User feedback
    • Policy enforcement

Next Steps

Last updated on
Code Review Assistant