Testing LangChain and LangGraph Workflows: A Developer's Playbook

TL;DR

Mock LLM calls with deterministic responses for unit tests Use snapshot testing for prompt/chain output validation Test graph state transitions with explicit state objects and assertions Integration tests should sample real LLM calls and verify quality Implement runnable hooks to inspect intermediate chain steps Monitor drift in production using continuous evaluation

Part 1: Unit Testing LangChain Chains

The fundamental problem: LangChain chains depend on external APIs. You can't test in isolation without losing the ability to verify behavior. The solution: structured mocking at multiple levels.

Mocking LLM Responses

Start with the simplest case, a single chain that calls an LLM. You want to verify that your prompt is correctly structured, that the output parsing works, and that your business logic is sound. You don't care about the LLM's quality yet.

from langchain.llms.fake import FakeListLLM
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain
import pytest

def test_summarization_chain():
 # Create a fake LLM that returns predefined responses
 fake_responses = [
 "This is a summary of the input text.",
 "Another summary response."
 ]
 fake_llm = FakeListLLM(responses=fake_responses)

 prompt = PromptTemplate(
 input_variables=["text"],
 template="Summarize: {text}"
 )

 chain = LLMChain(llm=fake_llm, prompt=prompt)

 result = chain.run(text="Long document...")
 assert "summary" in result.lower()
 assert len(result) > 10

This tests your chain logic without hitting any API. But there's a catch: FakeListLLM doesn't validate that your prompt is reasonable. You're testing the happy path. For production readiness, you need more.

Testing Prompt Quality

The prompt is the bridge between your code and the LLM. It needs to be clear, specific, and resilient to variations. Use snapshot testing to catch unexpected changes.

def test_prompt_template_structure():
 prompt = PromptTemplate(
 input_variables=["context", "question"],
 template="""Given the context below, answer the question concisely.

Context: {context}

Question: {question}

Answer:"""
 )

 # Verify the prompt structure doesn't accidentally change
 formatted = prompt.format(
 context="Earth orbits the sun",
 question="What does Earth orbit?"
 )

 # Use snapshot assertion (pytest-snapshot or similar)
 assert formatted == snapshot
 # This catches if someone modifies the template unexpectedly

Snapshot tests are underrated. They prevent silent failures where prompt changes degrade quality without triggering test failures.

Part 2: Integration Testing with Sampled Real Calls

Unit tests catch structural issues. Integration tests verify actual quality. The challenge: calling real LLMs is slow and expensive. The solution: sample calls and verify output characteristics.

Sampling Real LLM Calls

Run your chain against a real LLM on a subset of test cases. Use async to parallelize. Cache responses to avoid re-running expensive calls.

import asyncio
from langchain.llms import ChatOpenAI
from langchain.prompts import ChatPromptTemplate

async def test_chain_quality_sampling():
 llm = ChatOpenAI(model="gpt-4-turbo", temperature=0)

 prompt = ChatPromptTemplate.from_template(
 "Classify sentiment: {text}"
 )
 chain = prompt | llm

 test_cases = [
 ("I love this product!", "positive"),
 ("Terrible experience", "negative"),
 ("It's okay", "neutral"),
 ]

 results = await asyncio.gather(*[
 chain.ainvoke({"text": text})
 for text, _ in test_cases
 ])

 for (text, expected), result in zip(test_cases, results):
 # Verify output contains expected sentiment
 assert expected in result.content.lower()

This runs actual LLM calls but only on a small sample. You get real quality signals without testing every case every time.

Output Validation Patterns

LLM outputs are non-deterministic. You can't test for exact string equality. Instead, validate characteristics.

def validate_summary_output(output: str, min_length: int = 50):
 """Verify summary quality without exact string matching"""
 assert len(output) >= min_length, "Summary too short"
 assert len(output) < 500, "Summary too long"

 # Check for common markers of good summaries
 sentences = output.split('.')
 assert len(sentences) >= 2, "Summary too brief"
 assert len(sentences) <= 10, "Summary too verbose"

 # Verify no hallucinations (basic check)
 forbidden_phrases = ["I don't know", "unclear", "cannot determine"]
 for phrase in forbidden_phrases:
 assert phrase not in output.lower()

 return True

This validates that output meets quality criteria without being brittle to exact wording.

Part 3: Testing LangGraph State Transitions

LangGraph is where things get complex. Your graph has multiple nodes, branching logic, and state that evolves. Testing becomes about verifying that state transitions work correctly.

Explicit State Objects

Define your graph state as a typed class. This makes testing straightforward because you can assert on state properties.

from typing import TypedDict, List
from langgraph.graph import StateGraph, START, END

class ResearchState(TypedDict):
 query: str
 sources: List[str]
 research_results: str
 final_answer: str

def test_graph_state_transitions():
 # Create a mock graph that transitions state
 graph_builder = StateGraph(ResearchState)

 def node_search(state: ResearchState) -> ResearchState:
 state["sources"] = ["source1", "source2"]
 return state

 def node_analyze(state: ResearchState) -> ResearchState:
 state["research_results"] = f"Analyzed {len(state['sources'])} sources"
 return state

 graph_builder.add_node("search", node_search)
 graph_builder.add_node("analyze", node_analyze)

 graph_builder.add_edge(START, "search")
 graph_builder.add_edge("search", "analyze")
 graph_builder.add_edge("analyze", END)

 graph = graph_builder.compile()

 initial_state = {"query": "test query", "sources": [],
 "research_results": "", "final_answer": ""}

 result = graph.invoke(initial_state)

 # Verify state transitions
 assert len(result["sources"]) == 2
 assert "Analyzed" in result["research_results"]

Typed state objects make your tests declarative. The types double as documentation and catch errors early.

Testing Branching Logic

Graphs often have conditional edges that route to different nodes. Test each branch explicitly.

def test_conditional_routing():
 class ConditionalState(TypedDict):
 input: str
 route: str

 def decide_route(state: ConditionalState) -> str:
 if "urgent" in state["input"].lower():
 return "urgent_path"
 return "standard_path"

 def process_urgent(state: ConditionalState) -> ConditionalState:
 state["route"] = "urgent"
 return state

 def process_standard(state: ConditionalState) -> ConditionalState:
 state["route"] = "standard"
 return state

 graph_builder = StateGraph(ConditionalState)
 graph_builder.add_node("route", lambda x: x)
 graph_builder.add_node("urgent_path", process_urgent)
 graph_builder.add_node("standard_path", process_standard)

 graph_builder.add_edge(START, "route")
 graph_builder.add_conditional_edges(
 "route",
 decide_route,
 {"urgent_path": "urgent_path", "standard_path": "standard_path"}
 )
 graph_builder.add_edge("urgent_path", END)
 graph_builder.add_edge("standard_path", END)

 graph = graph_builder.compile()

 # Test urgent path
 result_urgent = graph.invoke({"input": "URGENT issue", "route": ""})
 assert result_urgent["route"] == "urgent"

 # Test standard path
 result_standard = graph.invoke({"input": "regular issue", "route": ""})
 assert result_standard["route"] == "standard"

Conditional edges are common sources of bugs. Test each condition explicitly. Don't rely on "it'll probably work in production."

Part 4: Mocking LLM Calls in Graphs

Your graph probably has nodes that call LLMs. Mock these for fast iteration, sample real calls for quality validation.

from unittest.mock import patch, AsyncMock

def test_graph_with_mocked_llm():
 # Define your graph as normal
 graph_builder = StateGraph(ResearchState)

 def research_node(state: ResearchState):
 # This node would normally call an LLM
 # We'll mock it instead
 state["research_results"] = "Mocked research output"
 return state

 graph_builder.add_node("research", research_node)
 graph_builder.add_edge(START, "research")
 graph_builder.add_edge("research", END)

 graph = graph_builder.compile()

 # Even with mocking, verify state flows correctly
 result = graph.invoke({
 "query": "test",
 "sources": [],
 "research_results": "",
 "final_answer": ""
 })

 assert "Mocked" in result["research_results"]

Part 5: Testing Memory and Retrieval

Many LangChain applications use memory (conversation history) or retrieval (RAG). These add complexity. Test them explicitly.

Memory Testing

Memory should persist across chain invocations and format correctly for the LLM.

from langchain.memory import ConversationBufferMemory

def test_conversation_memory():
 memory = ConversationBufferMemory()

 # Simulate conversation turns
 memory.save_context({"input": "Hello"}, {"output": "Hi there!"})
 memory.save_context({"input": "What's your name?"},
 {"output": "I'm Claude"})

 # Verify memory retrieves correct history
 history = memory.load_memory_variables({})["history"]
 assert "Hello" in history
 assert "What's your name?" in history
 assert "I'm Claude" in history

 # Verify format is LLM-friendly
 assert "Human:" in history
 assert "AI:" in history

Memory formatting matters. An incorrectly formatted memory string confuses the LLM. Always verify the exact format.

Retrieval Testing

Test that your retriever returns relevant documents and that you're using them correctly.

from langchain_community.vectorstores import FAISS
from langchain.embeddings.fake import FakeEmbeddings

def test_rag_retrieval():
 # Create a fake vector store for testing
 embeddings = FakeEmbeddings(size=10)

 documents = [
 {"page_content": "The capital of France is Paris", "metadata": {"source": "wiki"}},
 {"page_content": "The Eiffel Tower is in Paris", "metadata": {"source": "wiki"}},
 ]

 # Use FAISS for testing (no real embeddings needed)
 vectorstore = FAISS.from_documents(documents, embeddings)

 # Test retrieval
 results = vectorstore.similarity_search("Where is the Eiffel Tower?", k=2)
 assert len(results) >= 1
 assert "Paris" in results[0].page_content

Part 6: Common LangChain Testing Pitfalls

We've seen teams make these mistakes repeatedly. Learn from them.

Pitfall 1: Not Testing Prompt Escaping

User input goes into your prompts. What if input contains curly braces? Quotes? Test injection.

def test_prompt_injection_safety():
 prompt = PromptTemplate(
 input_variables=["user_input"],
 template="User says: {user_input}"
 )

 # This could break the prompt
 malicious_input = "Hello {fake_variable}"

 # Verify it doesn't break
 result = prompt.format(user_input=malicious_input)
 assert malicious_input in result

Pitfall 2: Ignoring Temperature in Tests

Tests should set temperature to 0 for deterministic behavior. Production can use higher temperatures.

def test_deterministic_output():
 # ALWAYS use temperature=0 in tests
 llm = ChatOpenAI(model="gpt-4-turbo", temperature=0)

 # This should produce identical output on repeated calls
 response1 = llm.invoke("Say hello")
 response2 = llm.invoke("Say hello")

 assert response1.content == response2.content

Pitfall 3: Not Testing Timeout Behavior

Real LLM calls timeout. Your code should handle this gracefully.

import asyncio
import pytest

async def test_timeout_handling():
 llm = ChatOpenAI(model="gpt-4-turbo", request_timeout=0.001)

 # This should timeout
 with pytest.raises(asyncio.TimeoutError):
 await llm.ainvoke("Some prompt")

Part 7: Continuous Quality Monitoring

Your tests pass today. What about tomorrow? LLMs drift. Your data changes. Production quality degrades silently without monitoring.

Automated Evaluation

Run periodic evaluations on production data. Compare against baselines.

def evaluate_chain_quality_periodic():
 """Run this as a scheduled job"""
 import requests

 # Sample recent inputs from production
 recent_inputs = get_production_inputs(limit=100)

 results = []
 for input_data in recent_inputs:
 output = chain.invoke(input_data)
 # Use an evaluator LLM to score quality
 score = evaluate_output(output)
 results.append(score)

 avg_score = sum(results) / len(results)

 # Alert if quality drops below baseline
 baseline = 0.85
 if avg_score < baseline:
 alert(f"Chain quality degraded: {avg_score}")

 return avg_score

"If you're not monitoring your chain quality in production, you're flying blind."

Bringing It All Together

Testing LangChain and LangGraph isn't about achieving 100% coverage. It's about strategic coverage: unit tests for logic, integration tests for quality, production monitoring for drift. Use mocks where they save you time and money. Use real LLM calls where they validate quality.

The teams shipping the highest-quality LLM applications aren't the ones with the most tests. They're the ones with the right tests, in the right places, validating the right things.