Routing
Routing
Routing is a pattern where an agent determines the appropriate path or handler for a given input based on classification or decision-making.
Basic Structure
states:
- id: start
type: initial
next: router
- id: router
type: llm
model: classifier
prompt: |
Determine which department should handle this query:
{query}
Choose from: sales, support, billing
input:
query: "${input.message}"
transitions:
- when: "${output === 'sales'}"
next: sales_handler
- when: "${output === 'support'}"
next: support_handler
- when: "${output === 'billing'}"
next: billing_handler
- next: general_handlerCommon Use Cases
1. Customer Support Routing
states:
- id: start
type: initial
next: classify_request
- id: classify_request
type: llm
model: classifier
prompt: |
Classify this customer request into one of these categories:
- technical_issue
- billing_inquiry
- product_info
- complaint
- general_question
Request: {request}
input:
request: "${input.message}"
transitions:
- when: "${output === 'technical_issue'}"
next: technical_support
- when: "${output === 'billing_inquiry'}"
next: billing_support
- when: "${output === 'product_info'}"
next: product_support
- when: "${output === 'complaint'}"
next: complaint_handler
- next: general_support2. Content Moderation
states:
- id: start
type: initial
next: moderate_content
- id: moderate_content
type: llm
model: moderator
prompt: |
Analyze this content for moderation:
{content}
Classify as:
- safe
- inappropriate
- spam
- needs_review
input:
content: "${input.content}"
transitions:
- when: "${output === 'safe'}"
next: approve_content
- when: "${output === 'inappropriate'}"
next: reject_content
- when: "${output === 'spam'}"
next: mark_spam
- next: human_review3. Document Processing
states:
- id: start
type: initial
next: classify_document
- id: classify_document
type: llm
model: classifier
prompt: |
Classify this document:
{document}
Categories:
- invoice
- contract
- report
- other
input:
document: "${input.document}"
transitions:
- when: "${output === 'invoice'}"
next: process_invoice
- when: "${output === 'contract'}"
next: process_contract
- when: "${output === 'report'}"
next: process_report
- next: general_processingAdvanced Patterns
1. Multi-level Routing
states:
- id: start
type: initial
next: first_level_router
- id: first_level_router
type: llm
model: classifier
prompt: "Classify into: technical, business, other"
input:
query: "${input.message}"
transitions:
- when: "${output === 'technical'}"
next: technical_router
- when: "${output === 'business'}"
next: business_router
- next: general_handler
- id: technical_router
type: llm
model: classifier
prompt: "Classify technical issue into: bug, feature, performance"
input:
query: "${input.message}"
transitions:
- when: "${output === 'bug'}"
next: bug_handler
- when: "${output === 'feature'}"
next: feature_handler
- when: "${output === 'performance'}"
next: performance_handler
- next: general_technical2. Confidence-based Routing
states:
- id: start
type: initial
next: classify_with_confidence
- id: classify_with_confidence
type: llm
model: classifier
prompt: |
Classify this input and provide confidence score (0-1):
{input}
Respond in JSON format:
{
"category": "category_name",
"confidence": 0.95
}
input:
input: "${input.message}"
transitions:
- when: "${output.confidence >= 0.9}"
next: "${output.category}_handler"
- when: "${output.confidence >= 0.7}"
next: "${output.category}_handler_with_review"
- next: human_review3. Context-aware Routing
states:
- id: start
type: initial
next: context_router
- id: context_router
type: llm
model: classifier
prompt: |
Route this request based on context:
Request: {request}
User Type: {user_type}
Previous Interactions: {history}
Choose from: premium_support, standard_support, basic_support
input:
request: "${input.message}"
user_type: "${context.user.type}"
history: "${context.interaction_history}"
transitions:
- when: "${output === 'premium_support'}"
next: premium_handler
- when: "${output === 'standard_support'}"
next: standard_handler
- next: basic_handlerBest Practices
- Clear Categories: Define distinct, non-overlapping categories
- Confidence Thresholds: Set appropriate confidence thresholds
- Fallback Routes: Always provide a default route
- Context Usage: Consider relevant context in routing decisions
- Error Handling: Handle classification errors gracefully
- Monitoring: Track routing decisions and outcomes
Common Pitfalls
- Ambiguous Categories: Overlapping or unclear categories
- Missing Fallbacks: No default route for unclassified cases
- Context Ignorance: Not considering relevant context
- Over-complexity: Too many routing levels or categories
- Inadequate Testing: Not testing all possible routes
Next Steps
- Learn about Parallelization
- Explore Orchestrator-Workers
- Read Best Practices
Last updated on