Skip to main content

5.4 Parallel & Branching Chains

AI-Generated Content

AI-generated content may contain errors. Always verify against official sources.

5.4 Parallel & Branching Chains

Key Concepts: Fan-out/fan-in · Conditional routing · Aspect-based decomposition

Official Docs: LangChain RunnableParallel · LangChain Routing

Fan-Out / Fan-In with RunnableParallel​

When independent sub-tasks can run simultaneously, use RunnableParallel to execute them concurrently and merge the results.

Input ─┬── Step A (sentiment)   ─┐
      ├── Step B (entities)    ─┤─→ Merge → Output
      └── Step C (category)    ─┘
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser, JsonOutputParser
from langchain_core.runnables import RunnableParallel

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

sentiment_chain = (
    ChatPromptTemplate.from_template("What is the sentiment of this review? One word: {review}")
    | llm | StrOutputParser()
)

entity_chain = (
    ChatPromptTemplate.from_template(
        "Extract named entities from this review as JSON list: {review}"
    )
    | llm | JsonOutputParser()
)

category_chain = (
    ChatPromptTemplate.from_template(
        "Classify this review category (product/service/delivery): {review}"
    )
    | llm | StrOutputParser()
)

# Run all three in parallel — latency ≈ max(A, B, C), not A + B + C
analysis = RunnableParallel(
    sentiment=sentiment_chain,
    entities=entity_chain,
    category=category_chain,
)

result = analysis.invoke({"review": "Absolutely love the new phone, but delivery took 2 weeks!"})
print(result)
# {"sentiment": "positive", "entities": ["phone"], "category": "product"}

Conditional Routing​

Route the input to different chains based on a condition:

from langchain_core.runnables import RunnableLambda

code_chain = (
    ChatPromptTemplate.from_template("Answer this coding question with a working Python example: {input}")
    | llm | StrOutputParser()
)

general_chain = (
    ChatPromptTemplate.from_template("Answer this general question concisely: {input}")
    | llm | StrOutputParser()
)

def route(info: dict) -> str:
    question = info["input"].lower()
    if any(kw in question for kw in ["python", "code", "function", "error", "debug"]):
        return "code"
    return "general"

router = RunnableLambda(route)

full_chain = {
    "input": lambda x: x["input"],
    "route": router,
} | RunnableLambda(
    lambda x: code_chain.invoke({"input": x["input"]})
    if x["route"] == "code"
    else general_chain.invoke({"input": x["input"]})
)

print(full_chain.invoke({"input": "How do I sort a list in Python?"}))
print(full_chain.invoke({"input": "What is the capital of Japan?"}))

Aspect-Based Decomposition Pattern​

Analyse multiple independent aspects of the same input simultaneously:

aspects = ["grammar", "clarity", "tone", "factual accuracy"]

def make_aspect_chain(aspect: str):
    return (
        ChatPromptTemplate.from_template(
            f"Rate the {{aspect}} of this text from 1-10 and explain in one sentence.\n\nText: {{text}}"
        ).partial(aspect=aspect)
        | llm | StrOutputParser()
    )

aspect_analysis = RunnableParallel(
    **{aspect: make_aspect_chain(aspect) for aspect in aspects}
)

result = aspect_analysis.invoke({
    "text": "The whether was grate today, lots of sun shine everywhere."
})
for aspect, feedback in result.items():
    print(f"\n{aspect.upper()}:\n{feedback}")

Common Mistakes​

Common Mistakes
  1. Using sequential chains for independent tasks — if Steps A, B, C don’t depend on each other, running them sequentially triples your latency. Use RunnableParallel.
  2. Not handling routing edge cases — always have a default/fallback route for inputs that don’t match any condition.
  3. Shared mutable state — parallel branches should be stateless functions. Avoid writing to shared variables from parallel branches.

Quick Quiz​

Test Your Understanding

Q1. What is the latency of 3 parallel chains (taking 2s, 3s, 4s respectively) vs the same chains run sequentially?
A1. Parallel: ~4s (max). Sequential: ~9s (sum). Parallel is 2.25× faster in this example.

Q2. What LangChain class runs multiple Runnables concurrently on the same input?
A2. RunnableParallel.

Q3. What is aspect-based decomposition?
A3. Analysing multiple independent dimensions of the same input simultaneously using parallel chains — each chain evaluates one aspect.

Student Exercise​

Exercise 5.4 — Parallel essay analyser
Build a RunnableParallel chain that analyses a student essay along 4 dimensions simultaneously: (1) grammar score (1-10), (2) argument strength (1-10), (3) evidence quality (1-10), (4) suggested improvements (list of 3 bullets). Combine results into a final feedback JSON.

Further Reading​

Next → 5.5 Multi-Stage Generation Pattern