Document intelligence·Case study

Multi-AgentDocumentIntelligenceSystem

Q: How does prompt caching reduce LLM cost by roughly 50%?

Multiple agents in the pipeline share overlapping context, such as the same document chunks and system instructions. Caching that shared context means it is not resent and re-billed on every agent call, which is what drove the roughly 50% reduction in LLM cost.

A production-ready multi-agent platform that analyzes contracts, extracts key clauses, identifies risks, and generates structured summaries through collaborative AI agents coordinated with LangGraph.

~50%LLM cost reduction

5Specialized agents

SSEReal-time streaming

Overview

Enterprise organizations spend significant time reviewing contracts, legal agreements, and business documents manually. Traditional LLM-based chatbots often struggle with long documents, structured outputs, and complex reasoning across multiple sections. The goal of this project was to build a production-ready multi-agent platform capable of analyzing contracts, extracting key clauses, identifying risks, and generating structured summaries through collaborative AI agents.

Problem

Legal and procurement teams manually review lengthy contracts, leading to slow turnaround times and inconsistent analysis. Existing chatbot-style solutions often:

Lose context in long documents
Produce inconsistent outputs
Cannot reason through multiple document sections simultaneously

Solution

I designed a multi-agent architecture where specialized AI agents collaborate to complete different stages of document analysis. Instead of relying on a single LLM prompt, the workflow distributes responsibilities across multiple autonomous agents coordinated through LangGraph. Each agent focuses on a dedicated task such as:

Document understanding
Clause extraction
Risk identification
Summary generation
Response validation

The final output is streamed back to the user in real time with structured JSON responses.

Architecture

PDF Upload
    |
    v
  Parser
    |
    v
 Chunking
    |
    v
 Retriever
    |
    v
Planner Agent
    |
    +-----------+-----------+
    v           v           v
Clause Agent  Risk Agent  Summary Agent
    |           |           |
    +-----------+-----------+
                |
                v
           Aggregator
                |
                v
        Structured Output
                |
                v
            Frontend

1

Parsing & retrieval

Ingestion

stage_1_output

The uploaded PDF is parsed, split into chunks, and indexed by a retriever so downstream agents can pull only the relevant context for a given task instead of reasoning over the entire document at once.

2

Specialized analysis agents

Parallel execution

stage_2_output

A planner agent dispatches retrieved context to three specialized agents running in parallel, each focused on a single dedicated task:

Clause AgentExtracts key clauses from the document

Risk AgentIdentifies risk signals within extracted clauses

Summary AgentGenerates a structured summary of the findings

3

Aggregation & streaming

Response

stage_3_output

An aggregator merges the outputs from all three agents, a validation step checks the result against a defined schema, and the structured JSON response is streamed back to the frontend in real time using Server-Sent Events.

Key features

Multi-agent orchestrationStreaming responses (Server-Sent Events)Prompt caching for cost optimizationStructured outputs with PydanticLangSmith tracingConversation memoryModular agent architecture

Tech stack

LangGraphOpenAI GPT-4oFastAPINext.jsTypeScriptLangSmithPydantic

Results

LLM cost reduction

Reduced LLM cost by approximately 50%

Achieved through prompt caching across the multi-agent pipeline.

Real-time streaming responses

Enabled using Server-Sent Events for a noticeably faster-feeling user experience.

Improved maintainability

Achieved by separating reasoning into specialized agents instead of one monolithic prompt.

Frequently asked questions

Why use a multi-agent architecture instead of a single LLM prompt for contract analysis?

Single-prompt chatbot approaches lose context in long documents, produce inconsistent outputs, and cannot reason through multiple sections simultaneously. A multi-agent architecture distributes responsibility across specialized agents, so each one focuses on a single task instead of one generalist prompt trying to do everything at once.

How does LangGraph coordinate the agents in this system?

A planner agent routes the parsed and retrieved document context to the clause extraction, risk identification, and summary generation agents. LangGraph coordinates these as nodes in a graph, and an aggregator node merges their outputs into a single validated, structured response.

How does prompt caching reduce LLM cost by roughly 50%?

Multiple agents in the pipeline share overlapping context, like the same document chunks and system instructions. Caching that shared context means it isn't resent and re-billed on every agent call, which is what drove the roughly 50% reduction in LLM cost.

Why stream responses instead of returning one completed analysis?

Server-Sent Events stream each agent's output as it completes rather than making the user wait for the entire pipeline to finish before seeing anything. For longer documents, this significantly improves perceived responsiveness.

What does structured output with Pydantic provide over freeform text responses?

Each agent's output, and the final aggregated response, is validated against a defined schema before it reaches the frontend. Downstream systems can rely on consistent fields instead of parsing freeform text.

Let's Talk ← All case studies