AI Stack Map: 12 Tools Across Six Lanes
A support ticket that needs docs, a tool call, and a model reply does not need twelve equal "frameworks." It needs an orchestration lane, a retrieval lane, a serving lane, and a place to store embeddings. Popular roundups flatten those jobs into one grid and call FAISS, Ollama, and XGBoost the same kind of thing.
This guide is for engineers choosing a stack for a production AI feature. You will map twelve widely used tools into six lanes, follow one support request through the path, see short code for the hot spots, and leave with a decision rule—including when not to adopt another framework. Product surfaces reflect July 2026 defaults; re-check official docs before a bake-off. For RAG variants see Classic vs Graph vs Agentic RAG; for vector stores see Vector database map.
Six lanes, not twelve peers
Treat the slide's twelve names as ingredients on different shelves. Orchestration (LangGraph, CrewAI, Microsoft Agent Framework) decides steps, state, and human approval. RAG / data frameworks (Haystack, LlamaIndex) connect documents, indexes, and query engines. Inference runtimes (vLLM, Ollama) serve or host model weights. Vector indexes and databases (FAISS, ChromaDB, Pinecone/Qdrant/Weaviate) store embeddings and run similarity search. Classical ML (XGBoost) still wins on tabular prediction. Model hubs (Hugging Face Transformers) load pretrained weights for NLP, vision, and audio.
Calling all of them "frameworks" is the first error. FAISS is a similarity-search library. Ollama is a local model runner. Transformers is a model API and hub client. Mixing those layers makes every comparison dishonest: you cannot rank "best for RAG" between vLLM and LlamaIndex because they solve different jobs.
Quick reference
- Lane first, logo second — pick the job (orchestrate / retrieve / serve / index) before the product.
- One support request can touch four lanes: orchestrator → retriever → vector store → LLM server.
- You rarely need every name on the slide in one service.
- Link satellites for depth: agentic AI, RAG comparison, vector DB guide, LLMOps stack.
Remember this
FAISS, Ollama, and Transformers sit on different shelves — a similarity-search library, a local model runner, a hub client — so "best for RAG" between vLLM and LlamaIndex is a comparison across jobs, not a real ranking.
Orchestration: LangGraph, CrewAI, Microsoft Agent Framework
LangGraph models agent work as a stateful graph: nodes mutate shared state, edges can loop, and you can pause for human approval. That fits long workflows where retries and branching matter more than a single prompt. CrewAI leans on role-playing crews—researcher, writer, reviewer—with task delegation; it is often faster to sketch multi-agent collaboration, with less explicit graph control. Microsoft Agent Framework is Microsoft's production-oriented SDK for agents and multi-agent workflows in Python and .NET, positioned as the successor path for teams that used AutoGen and Semantic Kernel. Prefer it when you need Microsoft/Azure ecosystem integration and long-term support commitments—not because every greenfield team must adopt it.
Orchestrators own control flow and tool trust. They do not replace retrieval quality or model serving. Failure mode: an agent retries a mutating tool without idempotency keys and double-charges a customer. Put approvals and idempotency at the tool boundary before you celebrate multi-agent demos.
Quick reference
- LangGraph — cyclic/conditional graphs, checkpointing, human-in-the-loop.
- CrewAI — role + task collaboration; good for structured multi-agent drafts.
- Microsoft Agent Framework — Python/.NET agents & workflows; enterprise/Azure-friendly successor path for AutoGen/SK users.
- Skip a heavy orchestrator when one prompt + tools + tests already meet the SLA.
- Trust boundary: tools that write money or PII need authz, audit, and idempotency.
Remember this
An orchestrator owns control flow and tool trust, not retrieval quality or model serving — an agent that retries a mutating tool without an idempotency key is what turns a demo into a double charge.
RAG and data: Haystack and LlamaIndex
Haystack is an end-to-end framework for document pipelines: ingest, clean, embed, retrieve, generate, evaluate. Its strength is modular pipelines you can test as components. LlamaIndex emphasizes connecting LLMs to private data—connectors, indexes, query engines, and agents over those indexes. Both sit above a vector store and a model endpoint; neither is a serving engine.
Mechanism that matters: retrieval quality dominates generation style. If chunking, metadata filters, or hybrid search are wrong, a prettier agent loop still hallucinates. Failure mode: the pipeline returns top-k by cosine similarity only, misses keyword entities (order IDs), and the model invents a tracking number. Add hybrid search or structured lookup for identifiers, and evaluate faithfulness on a fixed ticket set before shipping.
Quick reference
- Haystack — production RAG/search pipelines with modular components.
- LlamaIndex — data connectors, indexes, query engines, agent-over-data patterns.
- Both need a vector/keyword backend and an LLM runtime underneath.
- Evaluate with a frozen question set: faithfulness, latency, cost per query.
- Depth: RAG comparison.
Remember this
Retrieval quality dominates generation style — top-k cosine similarity alone misses exact identifiers like order IDs, and a prettier agent loop on top of that gap still invents a tracking number.
Serving and local inference: vLLM and Ollama
vLLM is a high-throughput inference engine for serving LLMs—continuous batching, efficient KV-cache use, OpenAI-compatible APIs in common deployments. Choose it when many concurrent requests hit the same model and GPU utilization matters. Ollama optimizes for local developer experience: pull a model, run offline, talk to a simple CLI/API. It is not a drop-in replacement for a multi-tenant GPU cluster.
Failure mode: teams "productionize" Ollama behind a public URL with no auth, quotas, or model allow-list. Treat local runners as trusted-dev or private-network tools unless you add the same gateway controls you would put in front of any model API. Performance claims depend on GPU, quantization, sequence length, and concurrency—never a universal "faster than X" ranking.
Quick reference
- vLLM — throughput-oriented GPU serving for shared model endpoints.
- Ollama — local/offline iteration; wide model library; simple API.
- Put TLS, auth, rate limits, and model allow-lists in front of any remote server.
- Measure tokens/sec and p95 latency under your concurrency—not slide adjectives.
Remember this
Ollama behind a public URL with no auth or quotas is a local dev tool masquerading as a production endpoint — the same gateway controls that front any model API belong in front of it too.
Vector search: FAISS, Chroma, and managed stores
FAISS (Facebook AI Similarity Search) is a library for dense similarity search and clustering—often in-process or beside a custom index service. It is not a full database with multi-tenant auth and ops. ChromaDB is a lightweight embedding store aimed at developers building RAG apps quickly (collections, metadata filters). Pinecone, Qdrant, and Weaviate are production-oriented vector databases (managed and/or self-hosted options differ by product) with APIs for upsert, filter, and scale-out search.
They are not peers of LangGraph. An orchestrator calls a vector API. Failure mode: embedding model A indexes the corpus, model B queries it, cosine scores look confident, answers are nonsense. Pin embedding model + dimension in config and fail closed on mismatch. For product comparisons and ops trade-offs, use the dedicated vector database guide.
Quick reference
- FAISS — library for fast similarity search; you own serving and durability.
- ChromaDB — developer-friendly store for RAG prototypes and small apps.
- Pinecone / Qdrant / Weaviate — vector DB products for scalable semantic search (ops models differ).
- Always version the embedding model with the index.
- Skip a managed vector DB when pgvector on an existing Postgres already meets scale and ops.
Remember this
One index queried with an embedding model different from the one that built it returns confident cosine scores on nonsense — pin the embedding model and dimension in config and fail closed on mismatch.
XGBoost, Transformers, and how to choose
XGBoost remains a default for tabular classification and regression: gradient boosting with strong baselines on structured features. It is not an LLM framework; put it beside your AI stack when the label is churn, fraud, or ranking on rows—not when the input is free text that needs retrieval. Hugging Face Transformers is the standard way to load and run pretrained models across NLP, vision, and audio, with a huge hub of weights and tokenizers. Use it when you fine-tune or host open weights; use a hosted API when you want someone else to run the GPUs.
Decision rule: start from the request. Need multi-step tools and approvals → orchestrator. Need private docs in the answer → RAG framework + vector store + embedding pin. Need tokens under load → serving engine. Need a local laptop loop → Ollama. Need row-level prediction → XGBoost (or similar). Need a base model file → Transformers/hub. Prefer the smallest lane set that passes a frozen eval. For platform tooling beyond these twelve names, see LLMOps stack and What is agentic AI.
Official source notes (checked July 2026): orchestration and RAG claims track langchain-ai.github.io/langgraph, docs.crewai.com, learn.microsoft.com/agent-framework, docs.haystack.deepset.ai, and docs.llamaindex.ai; serving and storage claims track docs.vllm.ai, docs.ollama.com, faiss.ai, and docs.trychroma.com; ML/hub boundaries track xgboost.readthedocs.io and huggingface.co/docs/transformers. These sources establish each lane, not a universal product ranking; re-check preview status and defaults before adoption.
Quick reference
- XGBoost — structured/tabular ML; not a substitute for RAG or agents.
- Transformers — pretrained model access and fine-tune loops; hub + PyTorch/TF ecosystem.
- When not to add a framework: a single LLM API call + SQL already solves the ticket.
- Compare options under the same workload, embedding model, and eval set.
- Security: never expose local model ports; scope tool credentials per tenant.
Remember this
The request decides the lane — row-level prediction wants XGBoost, not retrieval; free text needing grounded answers wants RAG, not a tabular model — pick the smallest lane set that still passes a frozen eval.
Key takeaway
The useful skill is not memorizing twelve logos. It is routing a real request through orchestration, retrieval, serving, and storage without pretending those jobs are interchangeable. Practice (30 minutes): pick one internal FAQ. Pin an embedding model; index ten docs in Chroma or pgvector; serve answers through Ollama or a hosted LLM; wrap the flow in a tiny state machine that returns no_sources on empty retrieval. Record faithfulness on five questions and one failure (wrong embedding model or empty index). Ship the write-up of what you would add next—orchestrator, hybrid search, or GPU serving—only if the eval demands it.
Related Articles
Explore this topic