Routing

Routing

Routing is a pattern where an agent determines the appropriate path or handler for a given input based on classification or decision-making.

Basic Structure

states:
  - id: start
    type: initial
    next: router
  
  - id: router
    type: llm
    model: classifier
    prompt: |
      Determine which department should handle this query:
      
      {query}
      
      Choose from: sales, support, billing      
    input:
      query: "${input.message}"
    transitions:
      - when: "${output === 'sales'}"
        next: sales_handler
      - when: "${output === 'support'}"
        next: support_handler
      - when: "${output === 'billing'}"
        next: billing_handler
      - next: general_handler

Common Use Cases

1. Customer Support Routing

states:
  - id: start
    type: initial
    next: classify_request
  
  - id: classify_request
    type: llm
    model: classifier
    prompt: |
      Classify this customer request into one of these categories:
      - technical_issue
      - billing_inquiry
      - product_info
      - complaint
      - general_question
      
      Request: {request}      
    input:
      request: "${input.message}"
    transitions:
      - when: "${output === 'technical_issue'}"
        next: technical_support
      - when: "${output === 'billing_inquiry'}"
        next: billing_support
      - when: "${output === 'product_info'}"
        next: product_support
      - when: "${output === 'complaint'}"
        next: complaint_handler
      - next: general_support

2. Content Moderation

states:
  - id: start
    type: initial
    next: moderate_content
  
  - id: moderate_content
    type: llm
    model: moderator
    prompt: |
      Analyze this content for moderation:
      
      {content}
      
      Classify as:
      - safe
      - inappropriate
      - spam
      - needs_review      
    input:
      content: "${input.content}"
    transitions:
      - when: "${output === 'safe'}"
        next: approve_content
      - when: "${output === 'inappropriate'}"
        next: reject_content
      - when: "${output === 'spam'}"
        next: mark_spam
      - next: human_review

3. Document Processing

states:
  - id: start
    type: initial
    next: classify_document
  
  - id: classify_document
    type: llm
    model: classifier
    prompt: |
      Classify this document:
      
      {document}
      
      Categories:
      - invoice
      - contract
      - report
      - other      
    input:
      document: "${input.document}"
    transitions:
      - when: "${output === 'invoice'}"
        next: process_invoice
      - when: "${output === 'contract'}"
        next: process_contract
      - when: "${output === 'report'}"
        next: process_report
      - next: general_processing

Advanced Patterns

1. Multi-level Routing

states:
  - id: start
    type: initial
    next: first_level_router
  
  - id: first_level_router
    type: llm
    model: classifier
    prompt: "Classify into: technical, business, other"
    input:
      query: "${input.message}"
    transitions:
      - when: "${output === 'technical'}"
        next: technical_router
      - when: "${output === 'business'}"
        next: business_router
      - next: general_handler
  
  - id: technical_router
    type: llm
    model: classifier
    prompt: "Classify technical issue into: bug, feature, performance"
    input:
      query: "${input.message}"
    transitions:
      - when: "${output === 'bug'}"
        next: bug_handler
      - when: "${output === 'feature'}"
        next: feature_handler
      - when: "${output === 'performance'}"
        next: performance_handler
      - next: general_technical

2. Confidence-based Routing

states:
  - id: start
    type: initial
    next: classify_with_confidence
  
  - id: classify_with_confidence
    type: llm
    model: classifier
    prompt: |
      Classify this input and provide confidence score (0-1):
      
      {input}
      
      Respond in JSON format:
      {
        "category": "category_name",
        "confidence": 0.95
      }      
    input:
      input: "${input.message}"
    transitions:
      - when: "${output.confidence >= 0.9}"
        next: "${output.category}_handler"
      - when: "${output.confidence >= 0.7}"
        next: "${output.category}_handler_with_review"
      - next: human_review

3. Context-aware Routing

states:
  - id: start
    type: initial
    next: context_router
  
  - id: context_router
    type: llm
    model: classifier
    prompt: |
      Route this request based on context:
      
      Request: {request}
      User Type: {user_type}
      Previous Interactions: {history}
      
      Choose from: premium_support, standard_support, basic_support      
    input:
      request: "${input.message}"
      user_type: "${context.user.type}"
      history: "${context.interaction_history}"
    transitions:
      - when: "${output === 'premium_support'}"
        next: premium_handler
      - when: "${output === 'standard_support'}"
        next: standard_handler
      - next: basic_handler

Best Practices

  1. Clear Categories: Define distinct, non-overlapping categories
  2. Confidence Thresholds: Set appropriate confidence thresholds
  3. Fallback Routes: Always provide a default route
  4. Context Usage: Consider relevant context in routing decisions
  5. Error Handling: Handle classification errors gracefully
  6. Monitoring: Track routing decisions and outcomes

Common Pitfalls

  1. Ambiguous Categories: Overlapping or unclear categories
  2. Missing Fallbacks: No default route for unclassified cases
  3. Context Ignorance: Not considering relevant context
  4. Over-complexity: Too many routing levels or categories
  5. Inadequate Testing: Not testing all possible routes

Next Steps

Last updated on