Auto-capture all LLM calls with OpenTelemetry instrumentation.Auto-tracing only works with supported LLM SDKs (OpenAI, Anthropic, etc.) - not raw HTTP requests. If you’re using an OpenAI-compatible API like OpenRouter, LiteLLM, or a self-hosted model, use the OpenAI SDK with a custom base_url:
from openai import OpenAI
# OpenRouter, LiteLLM, vLLM, etc.
client = OpenAI(
base_url="https://openrouter.ai/api/v1", # or your provider's URL
api_key="your-provider-key"
)
# Now this call will be auto-traced!
response = client.chat.completions.create(model="gpt-4o", messages=[...])
Automatic Tracing
# Step 1: Import and init Fallom FIRST
import fallom
fallom.init()
# Step 2: Import OpenAI AFTER init
from openai import OpenAI
client = OpenAI()
# Set session context
fallom.trace.set_session("my-agent", session_id)
# Step 3: All LLM calls automatically traced with:
# - Model, tokens, latency
# - Prompts and completions
# - Your config_key and session_id
response = client.chat.completions.create(model="gpt-4o", messages=[...])
Custom Metrics
Record business metrics that OTEL can’t capture automatically:from fallom import trace
# Record custom metrics for this session
trace.span({
"outlier_score": 0.8,
"user_satisfaction": 4,
"conversion": True
})
# Or explicitly specify session (for batch jobs)
trace.span(
{"outlier_score": 0.8},
config_key="my-agent",
session_id="user123-convo456"
)
Multiple A/B Tests in One Workflow
If you have multiple LLM calls and only want to A/B test some of them:import fallom
fallom.init(api_key="your-api-key")
# Set default context for all tracing
fallom.trace.set_session("my-agent", session_id)
# ... regular LLM calls (traced as "my-agent") ...
# A/B test a specific call - models.get() auto-updates context
model = models.get("summarizer-test", session_id, fallback="gpt-4o")
result = summarizer.run(model=model)
# Reset context back to default
fallom.trace.set_session("my-agent", session_id)
# ... more regular LLM calls (traced as "my-agent") ...
