LangChain

Fallom integrates with LangChain to trace all your chain and agent LLM calls automatically using the callback handler pattern.

Get your API key from the dashboard.

Installation

Python
TypeScript

pip install "fallom[langchain]" langchain-openai

npm install @fallom/trace langchain @langchain/openai

Quick Start

Python
TypeScript

import fallom
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

# Initialize Fallom
fallom.init(api_key="your-api-key")

# Create a session and get the callback handler
session = fallom.session(
    config_key="langchain-app",
    session_id="user-123-conversation-456",
)
handler = session.langchain_callback()

# Pass the handler to your LangChain components
llm = ChatOpenAI(model="gpt-4o", callbacks=[handler])
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    ("user", "{input}")
])

chain = prompt | llm

# All LLM calls in the chain are automatically traced!
response = chain.invoke({"input": "What is the capital of France?"})

import fallom from "@fallom/trace";
import { ChatOpenAI } from "@langchain/openai";
import { ChatPromptTemplate } from "@langchain/core/prompts";

// Initialize Fallom once at app startup
await fallom.init({ apiKey: "your-api-key" });

// Create a session for this conversation
const session = fallom.session({
  configKey: "langchain-app",
  sessionId: sessionId,
});

// Wrap OpenAI for tracing
const model = session.wrapOpenAI(new ChatOpenAI({ model: "gpt-4o" }));

const prompt = ChatPromptTemplate.fromMessages([
  ["system", "You are a helpful assistant."],
  ["user", "{input}"]
]);

const chain = prompt.pipe(model);

// All LLM calls in the chain are traced
const response = await chain.invoke({ input: "What is the capital of France?" });

How It Works

Fallom uses LangChain’s callback system to automatically capture:

Event	Captured Data
LLM calls	Model, messages, tokens, latency, response
Chain executions	Inputs, outputs, duration
Tool calls	Tool name, inputs, outputs
Agent actions	Actions taken, reasoning
Retriever queries	Query, retrieved documents
Errors	Error messages, stack traces

All events are linked together with parent-child relationships, giving you a complete trace of your LangChain application.

Alternative: Direct Handler Creation

You can also create the callback handler directly without a session:

import fallom
from fallom.trace.wrappers.langchain import FallomCallbackHandler
from langchain_openai import ChatOpenAI

fallom.init(api_key="your-api-key")

# Create handler directly with full control
handler = FallomCallbackHandler(
    config_key="my-app",
    session_id="session-123",
    customer_id="user-456",        # optional: for user analytics
    metadata={"env": "production"}, # optional: custom metadata
    tags=["langchain", "prod"],     # optional: for filtering
)

llm = ChatOpenAI(model="gpt-4o", callbacks=[handler])
response = llm.invoke("Hello!")

Model A/B Testing with LangChain

Test different models in your LangChain applications:

Python
TypeScript

import fallom
from fallom import models
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

fallom.init(api_key="your-api-key")

session_id = "user-123-conversation-456"

# Get assigned model for this session (sticky assignment)
model_id = models.get("langchain-app", session_id, fallback="gpt-4o")

# Create session and handler
session = fallom.session(config_key="langchain-app", session_id=session_id)
handler = session.langchain_callback()

# Use the A/B tested model
llm = ChatOpenAI(model=model_id, callbacks=[handler])
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    ("user", "{input}")
])

chain = prompt | llm
response = chain.invoke({"input": "Summarize this document"})

import fallom from "@fallom/trace";
import { ChatOpenAI } from "@langchain/openai";
import { ChatPromptTemplate } from "@langchain/core/prompts";

await fallom.init({ apiKey: "your-api-key" });

const session = fallom.session({
  configKey: "langchain-app",
  sessionId: sessionId,
});

// Get assigned model for this session
const modelId = await session.getModel({ fallback: "gpt-4o" });

const model = new ChatOpenAI({ model: modelId });
const prompt = ChatPromptTemplate.fromMessages([
  ["system", "You are a helpful assistant."],
  ["user", "{input}"]
]);

const chain = prompt.pipe(model);
const response = await chain.invoke({ input: "Summarize this document" });

LangChain Agents with Fallom

Trace your LangChain agents including all tool calls:

Python
TypeScript

import fallom
from langchain_openai import ChatOpenAI
from langchain.agents import create_react_agent, AgentExecutor
from langchain_core.prompts import ChatPromptTemplate
from langchain_community.tools import DuckDuckGoSearchRun

fallom.init(api_key="your-api-key")

session = fallom.session(
    config_key="langchain-agent",
    session_id="agent-session-123",
)
handler = session.langchain_callback()

# Pass handler to both the LLM and the executor
llm = ChatOpenAI(model="gpt-4o", callbacks=[handler])
tools = [DuckDuckGoSearchRun()]

agent = create_react_agent(llm, tools, prompt)
agent_executor = AgentExecutor(
    agent=agent,
    tools=tools,
    callbacks=[handler],  # Also trace tool executions
    verbose=True
)

# All LLM calls, tool calls, and agent actions are traced
response = agent_executor.invoke({"input": "What's the latest news about AI?"})

import fallom from "@fallom/trace";
import { ChatOpenAI } from "@langchain/openai";
import { createReactAgent, AgentExecutor } from "langchain/agents";

await fallom.init({ apiKey: "your-api-key" });

const session = fallom.session({
  configKey: "langchain-agent",
  sessionId: sessionId,
});

const model = new ChatOpenAI({ model: "gpt-4o" });

// Create and run the agent - all LLM calls are traced
const agent = await createReactAgent({ llm: model, tools, prompt });
const agentExecutor = new AgentExecutor({ agent, tools });

const response = await agentExecutor.invoke({
  input: "What's the latest news about AI?"
});

RAG Applications

Trace retrieval-augmented generation pipelines:

import fallom
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough

fallom.init(api_key="your-api-key")

session = fallom.session(config_key="rag-app", session_id="session-123")
handler = session.langchain_callback()

# Your vector store
vectorstore = FAISS.from_texts(texts, OpenAIEmbeddings())
retriever = vectorstore.as_retriever()

llm = ChatOpenAI(model="gpt-4o", callbacks=[handler])

prompt = ChatPromptTemplate.from_template("""
Answer based on the context:
{context}

Question: {question}
""")

# Build RAG chain
rag_chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | llm
)

# Retriever queries and LLM calls are all traced
response = rag_chain.invoke(
    "What is the refund policy?",
    config={"callbacks": [handler]}  # Pass handler to trace retriever
)

Prompt Management with LangChain

Use Fallom’s managed prompts with LangChain:

Python
TypeScript

import fallom
from fallom import prompts
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

fallom.init(api_key="your-api-key")

# Get managed prompt from Fallom
prompt_config = prompts.get("assistant-prompt", variables={
    "persona": "helpful assistant"
})

session = fallom.session(config_key="langchain-app", session_id="session-123")
handler = session.langchain_callback()

llm = ChatOpenAI(model="gpt-4o", callbacks=[handler])
prompt = ChatPromptTemplate.from_messages([
    ("system", prompt_config.system),
    ("user", "{input}")
])

chain = prompt | llm
# Traces are automatically tagged with prompt_key and prompt_version
response = chain.invoke({"input": "Help me write an email"})

import fallom, { prompts } from "@fallom/trace";
import { ChatOpenAI } from "@langchain/openai";
import { ChatPromptTemplate } from "@langchain/core/prompts";

await fallom.init({ apiKey: "your-api-key" });

// Get managed prompt
const promptConfig = await prompts.get("assistant-prompt", {
  variables: { persona: "helpful assistant" }
});

const session = fallom.session({
  configKey: "langchain-app",
  sessionId: sessionId,
});

const model = new ChatOpenAI({ model: "gpt-4o" });
const prompt = ChatPromptTemplate.fromMessages([
  ["system", promptConfig.system],
  ["user", "{input}"]
]);

const chain = prompt.pipe(model);
const response = await chain.invoke({ input: "Help me write an email" });

Streaming Support

The callback handler works with streaming responses:

import fallom
from langchain_openai import ChatOpenAI

fallom.init(api_key="your-api-key")

session = fallom.session(config_key="streaming-app", session_id="session-123")
handler = session.langchain_callback()

llm = ChatOpenAI(model="gpt-4o", streaming=True, callbacks=[handler])

# Stream responses - trace is sent when stream completes
for chunk in llm.stream("Tell me a story"):
    print(chunk.content, end="", flush=True)

Best Practices

Use one handler per conversation

Create a new callback handler for each user conversation or request to properly group traces:

def handle_message(user_id: str, conversation_id: str, message: str):
    session = fallom.session(
        config_key="chat-app",
        session_id=conversation_id,
        customer_id=user_id,
    )
    handler = session.langchain_callback()

    llm = ChatOpenAI(model="gpt-4o", callbacks=[handler])
    return llm.invoke(message)

Pass handler to all components

For complete tracing, pass the handler to all LangChain components:

handler = session.langchain_callback()

# Pass to LLM
llm = ChatOpenAI(callbacks=[handler])

# Pass to chains via invoke config
chain.invoke(input, config={"callbacks": [handler]})

# Pass to agent executor
executor = AgentExecutor(agent=agent, tools=tools, callbacks=[handler])

Add metadata for filtering

Use metadata and tags to organize traces in the dashboard:

session = fallom.session(
    config_key="my-app",
    session_id="session-123",
    customer_id="user-456",
    metadata={
        "environment": "production",
        "feature": "summarization",
        "user_plan": "enterprise",
    },
    tags=["production", "langchain", "gpt-4o"],
)

Next Steps

Model A/B Testing

Learn more about running model experiments.

Prompt Management

Manage and A/B test your prompts.

Tracing Guide

Deep dive into tracing concepts.

View Dashboard

See your LangChain traces and analytics.

Getting Started

Features

TypeScript Integrations

Python Integrations

AI Coding Tools

No-Code Integrations

Installation

Quick Start

How It Works

Alternative: Direct Handler Creation

Model A/B Testing with LangChain

LangChain Agents with Fallom

RAG Applications

Prompt Management with LangChain

Streaming Support

Best Practices

Next Steps

Model A/B Testing

Prompt Management

Tracing Guide

View Dashboard

​Installation

​Quick Start

​How It Works

​Alternative: Direct Handler Creation

​Model A/B Testing with LangChain

​LangChain Agents with Fallom

​RAG Applications

​Prompt Management with LangChain

​Streaming Support

​Best Practices

​Next Steps

Model A/B Testing

Prompt Management

Tracing Guide

View Dashboard

Installation

Quick Start

How It Works

Alternative: Direct Handler Creation

Model A/B Testing with LangChain

LangChain Agents with Fallom

RAG Applications

Prompt Management with LangChain

Streaming Support

Best Practices

Next Steps