LangChain vs LlamaIndex vs Claude Agent SDK: How to Choose

LangChain is a general framework for LLM applications. Its current agent entry point is create_agent, which takes a model (a "provider:model" string or a model object), tools (plain Python callables work), and a system prompt, and returns an agent you invoke with messages. Customisation goes through middleware: prebuilt pieces include model and tool retries, PII handling, human-in-the-loop approval, and summarisation. Conversation state persists through a checkpointer and a thread_id. Its strengths are provider portability and explicit control over the steps an agent takes.

LlamaIndex is built for connecting models to data. Its core objects are readers (load documents from files and services), indexes such as VectorStoreIndex (chunk, embed, store), and query engines (retrieve and synthesise an answer). Agents such as FunctionAgent sit on top and can use a query engine as one of their tools. Its strengths are ingestion and retrieval: the part of RAG where most quality problems live.

The Claude Agent SDK exposes the loop behind Claude Code. You call query() with a prompt and options, and Claude reads files, runs commands, edits code, and calls MCP tools until the task is done, while you stream messages. Built-in tools already work, and Claude Code's permission rules, permission modes, hooks, and MCP support carry over. Its strength is agents that act on a codebase or system; its constraint is that it runs Claude models (directly or through Bedrock, Google Cloud, or Foundry).

LangChain (Python):

1from langchain.agents import create_agent2 3def get_weather(city: str) -> str:4    """Get the current weather for a city."""5    return f"It is sunny in {city}."6 7agent = create_agent(8    model="anthropic:<model>",9    tools=[get_weather],10    system_prompt="You are a concise assistant.",11)12result = agent.invoke({"messages": [{"role": "user", "content": "Weather in Oslo?"}]})

LlamaIndex (Python), with retrieval as a tool:

1from llama_index.core import VectorStoreIndex, SimpleDirectoryReader2from llama_index.core.agent.workflow import FunctionAgent3from llama_index.llms.openai import OpenAI4 5documents = SimpleDirectoryReader("data").load_data()6query_engine = VectorStoreIndex.from_documents(documents).as_query_engine()7 8async def search_documents(query: str) -> str:9    """Answer questions about the documents in ./data."""10    return str(await query_engine.aquery(query))11 12agent = FunctionAgent(tools=[search_documents], llm=OpenAI(model="gpt-4o-mini"),13                      system_prompt="Answer from the documents and cite them.")

Claude Agent SDK (TypeScript):

1import { query } from "@anthropic-ai/claude-agent-sdk";2 3for await (const message of query({4  prompt: "Find the failing test in this repo and fix it.",5  options: { allowedTools: ["Read", "Grep", "Edit", "Bash"], permissionMode: "acceptEdits" },6})) {7  if (message.type === "result") console.log(message.subtype);8}

Note what each example has to provide. The LangChain and LlamaIndex agents only know the tools you write; the Agent SDK agent can already read, search, edit, and run commands, which is powerful and also the reason it needs tighter permissions.

Each has a characteristic way of going wrong. With LangChain, it is distance from the model call: several layers between your code and the request make it harder to see the exact prompt and tool definitions, so tracing (for example with LangSmith) stops being optional. With LlamaIndex, it is retrieval quality: the agent answers confidently from the wrong chunks, and the fix is in chunking, metadata, and re-ranking, measured separately from the final answer. With the Agent SDK, it is reach: an agent that can edit files and run commands will do so, so permission rules, sandboxing, and a narrow tool list are part of the design, not an afterthought.

Version churn affects all three. LangChain's agent API changed substantially with create_agent; LlamaIndex moved agents to a workflow-based design; the Agent SDK tracks Claude Code releases. Pin versions and read the changelog before upgrading.

Many systems combine them at a clean boundary. LlamaIndex owns ingestion and the index; the agent, built with whichever framework fits, sees retrieval as a single tool:

1# retrieval.py - LlamaIndex only2from llama_index.core import VectorStoreIndex, SimpleDirectoryReader3_engine = VectorStoreIndex.from_documents(SimpleDirectoryReader("docs").load_data()).as_query_engine()4 5async def search_docs(query: str) -> str:6    """Search the internal docs. Use for product or policy questions; not for code."""7    response = await _engine.aquery(query)8    return str(response)

That function can be registered as a LangChain tool, a LlamaIndex agent tool, or, wrapped in an MCP server, a tool for the Claude Agent SDK and Claude Code. The boundary keeps responsibilities separate: retrieval can be evaluated and tuned on its own (did the right passages come back?), while agent behaviour is evaluated end to end (did it search when it should, and use the answer correctly?).

Keep the tool's output short and cited. Long retrieved passages fill the agent's context, and uncited answers make it impossible to tell retrieval errors from reasoning errors.

Use a folder of 20 internal documents and one question set of ten questions, half answerable from the documents.

1. Build the LlamaIndex index and a search_docs function; check that the top results contain the answer for each answerable question.
2. Register search_docs with a LangChain create_agent agent and run the ten questions.
3. Wrap search_docs in a small MCP server, give it to the Claude Agent SDK with allowedTools limited to that tool, and run the same questions.
4. For each run, record correct answers, unnecessary searches, and confident answers to unanswerable questions.
5. Read one full trace from each framework and note how easy it was to see the exact prompt and tool calls.

Pass criterion: retrieval is measured separately from answers, all three agents share one retrieval function, and you can state which framework you would ship and why, based on the numbers you recorded.