Shipping LangGraph pipelines without a full MLOps stack

Building an agent that calls an LLM once is easy. Building one that coordinates multiple steps — retrieve context, call tools, branch on results, retry on failure — needs orchestration. The question is how much infrastructure that orchestration requires.

Sequential chains aren't enough

The simplest pattern is a chain of functions: fetch data, pass to LLM, parse output, call next function. This breaks when you need:

• Branching — different paths based on LLM output or tool results
• Retries — re-run a step with modified input after failure
• Human-in-the-loop — pause for approval before executing an action
• State persistence — resume a pipeline after a crash or timeout

Function chains can hack around these with flags and global state, but the result is unmaintainable within a few weeks.

LangGraph state machines

LangGraph models workflows as directed graphs where each node is a step and edges define transitions. State is explicit — a typed object passed between nodes, not scattered across closures.

This makes debugging straightforward: when a pipeline fails at step 4, you inspect the state object at step 3's output. No archaeology through call stacks.

For the AI Pipeline Toolkit, I use LangGraph for:

• Webhook-triggered pipelines that fan out into parallel retrieval + inference steps
• Conditional routing based on classification results
• Checkpointing state for long-running jobs

FastAPI as ingress

Pipelines need triggers. FastAPI provides async webhook endpoints that accept external events, validate payloads, and dispatch jobs without blocking the response.

The pattern: webhook returns 202 immediately, pipeline runs async in the background, results are written to a store or callback URL. Simple, debuggable, no message queue required for low-to-medium volume.

Local inference during development

Cloud LLM APIs make development expensive and slow — every test run costs tokens and adds network latency. Ollama integration lets pipelines run against local models during development, with automatic fallback to cloud providers in production.

This isn't about avoiding cloud APIs entirely. It's about removing friction from the inner loop. You test pipeline logic locally, then flip a config flag for production inference.

What I deliberately didn't build

• Kubernetes operators for pipeline scaling
• A custom DAG UI for non-technical users
• Multi-tenant pipeline isolation

Teams that need those things should use a proper MLOps platform. This toolkit targets teams with a handful of pipelines, a FastAPI process, and a need to ship this quarter.

The shipping checklist

Before calling a LangGraph pipeline production-ready:

1. State schema is typed and validated — no dict soup
2. Each node has isolated error handling — failures don't corrupt state
3. Checkpoints exist for pipelines longer than 30 seconds
4. Inference provider is configurable — local for dev, cloud for prod
5. Webhook payloads are validated at ingress — reject garbage early

Orchestration should be boring infrastructure. The interesting work is in the agent logic, not the plumbing.