AI Agents MCP System Design Observability RAG

9 AI Concepts for Production Systems in 2026

Core Concept LearningJuly 16, 20266 min readUpdated July 21, 2026

A prompt is only one part of a production AI system. Engineers also need vocabulary for execution loops, tool connections, model access, cost, evaluation, safety, and diagnosis. A gateway cannot repair a weak eval set, and MCP cannot decide whether a task should be agentic.

For engineers moving an LLM feature beyond a demo, this map places nine controls around one bug-triage flow. You will be able to locate a failure, run a bounded tool loop, and decide which production control to add next; use What Is Agentic AI? and MCP vs A2A vs ACP for focused depth. Product and protocol examples are current as of July 2026; implementations and defaults may change.

High-level: nine concepts in four lanes

Learning path

Follow these checkpoints in order — each builds on the last.

Every concept sits in one of four lanes — runtime, platform, quality, strategy — and a v1 bug-triage system needs only one loop, two tools, and twenty golden tickets from that map, not all nine boxes filled.
Decide → Perform → Track → Optimize without a step cap is a chatbot that burns tokens forever — the limit belongs inside the loop, not bolted on after the first runaway bill.
MCP standardizes how a host discovers and calls tools across several AI hosts — a single fixed integration is still simpler as a typed function or a direct REST client.
A subagent earns its own slot only when it needs its own loop, context, or evaluated outcome — an ordinary tool call renamed "agent" just hides the orchestrator's real job list.
One gateway centralizing auth, limits, and logs is what turns a scattered set of API keys into a single point for failover, routing, and a kill-switch when a provider degrades.
Both a cache hit and a cache miss land on the bill — an agent that pastes the whole repo into every prompt turns that fact into a budget incident, not a rounding error.
Without evals every prompt edit is a guess, and without guards a jailbreak or a PII leak becomes the incident — ship both before scaling traffic, and log every block so a filter doesn't hide a real product bug.
Tracing one report through loop → MCP tool → guard → draft ticket is what turns "why did it loop?" from an unanswerable question into a specific decision, tool, or trust boundary to fix.
A twenty-step failure is undebuggable without traces, logs, and metrics — and the Bitter Lesson says bet the core on general models plus compute, keeping custom rules only as policy edges that don't have to scale.

Map the nine concepts

Split the nine into four lanes. Runtime: agentic loops, MCP, subagents. Platform: AI gateway, inference economics. Quality: evals, guardrails. Truth: observability, plus the Bitter Lesson as a strategy filter.

You do not need all nine on day one. For bug triage, start with one loop, two tools, input/output guards, and twenty golden tickets. Add a gateway when spend or keys get messy. Add deep traces when you cannot explain a bad run.

Mid-level build order: loop → tools → safety → visibility

Quick reference

Runtime: Decide → Perform → Track → Optimize until done.
Platform: one gateway for many models and apps.
Quality: test continuously; fail closed when unsafe.
Strategy: prefer general models + compute over brittle handcrafted rules.

Remember this

Every concept sits in one of four lanes — runtime, platform, quality, strategy — and a v1 bug-triage system needs only one loop, two tools, and twenty golden tickets from that map, not all nine boxes filled.

Agentic loops

An agentic loop improves through cycles: Decide → Perform → Track → Optimize, then repeat until done or a limit fires. The triage agent decides to search docs, performs the search, tracks whether the issue matches a known bug, then optimizes the next action (ask for logs vs open a ticket).

Without a loop you have a chatbot. With a loop and no caps you burn tokens forever. Always set max steps, cost budgets, and human escalation.

Agentic loop: Decide → Perform → Track → Optimize

Quick reference

Track means logs/traces — not “it felt done.”
Cap iterations before the first demo.
When to use: multi-step goals with tools.
When to skip: single-shot classify or rewrite tasks.

Pasteable bounded loop — TypeScript

1type Action =2  | { kind: "search_docs"; query: string }3  | { kind: "finish"; answer: string };4 5const plan: Action[] = [6  { kind: "search_docs", query: "ECONNRESET upload" },7  { kind: "finish", answer: "Check proxy timeout and retry logs." },8];9 10async function runTool(action: Action): Promise<string> {11  if (action.kind !== "search_docs") throw new Error("tool_not_allowed");12  if (!action.query.trim()) throw new Error("invalid_tool_arguments");13  return "Doc match: inspect proxy read timeout.";14}15 16async function runAgent(maxSteps = 3) {17  const observations: string[] = [];18  for (let step = 0; step < maxSteps; step++) {19    const action = plan[step];20    if (!action) throw new Error("planner_returned_no_action");21    if (action.kind === "finish") return { answer: action.answer, observations };22    try {23      observations.push(await runTool(action));24    } catch (error) {25      observations.push(`tool_error:${String(error)}`);26    }27  }28  throw new Error("max_steps_exceeded");29}30 31runAgent().then(console.log).catch(console.error);

Verify success and stop behavior

1# Save the sample as agent-loop.ts2npx tsx agent-loop.ts3# Expected: one docs observation and one final answer4 5# Then change maxSteps to 16# Expected: Error: max_steps_exceeded

Remember this

Decide → Perform → Track → Optimize without a step cap is a chatbot that burns tokens forever — the limit belongs inside the loop, not bolted on after the first runaway bill.

MCP (Model Context Protocol)

MCP standardizes how an AI host connects to servers that expose tools, resources, and prompts. In the triage example, the host can discover a docs-search tool and invoke it through an MCP client while the server enforces access to the underlying system.

MCP is not an agent-to-agent task protocol and does not replace an application’s ordinary HTTP API. When to use: several AI hosts need a reusable capability boundary. When to skip: one fixed integration where a typed function or direct API client is simpler. The focused protocol comparison covers the wire placement.

MCP bridges one agent to many tools

Quick reference

One protocol → many tools with clear scopes.
Keep REST for product clients; add MCP for agents.
Auth still matters — MCP is not a free pass to production data.
Build one domain MCP server before adopting five frameworks.

MCPGmailGitHubDatabaseBrowser

Tool contract exposed by an MCP server

1{2  "name": "search_docs",3  "description": "Search support docs visible to this tenant",4  "inputSchema": {5    "type": "object",6    "properties": {7      "query": { "type": "string", "minLength": 3 }8    },9    "required": ["query"],10    "additionalProperties": false11  }12}

Host handles tool errors as data

1const result = await mcpClient.callTool({2  name: "search_docs",3  arguments: { query: ticket.summary },4});5 6if (result.isError) {7  trace.record("tool_failed", { tool: "search_docs" });8  return { status: "needs_human", reason: "docs_unavailable" };9}10 11const text = result.content12  .filter((item) => item.type === "text")13  .map((item) => item.text)14  .join("\n");

Remember this

MCP standardizes how a host discovers and calls tools across several AI hosts — a single fixed integration is still simpler as a typed function or a direct REST client.

Subagents and multi-agent systems

Subagents are specialists under an orchestrator. One may retrieve docs, one draft the ticket, and one check severity before a combined result returns. They help when work has distinct expertise or permissions; they hurt when ordinary tool calls are renamed as vague “agents.”

Start with one orchestrator and typed tools. Promote a tool-sized responsibility to a subagent only when it needs its own loop, context, policy, or independently evaluated outcome. The canonical anatomy stays in What Is Agentic AI?.

Orchestrator coordinates specialist subagents

Quick reference

Orchestrator plans; subagents execute narrow jobs.
Define merge rules when specialists disagree.
Prefer clear APIs between agents over open chat.
Related: What Is Agentic AI and AI Agent Project Structure.

Remember this

A subagent earns its own slot only when it needs its own loop, context, or evaluated outcome — an ordinary tool call renamed "agent" just hides the orchestrator's real job list.

AI gateway

An AI gateway sits between apps and model providers. Bug-triage, chat, and batch jobs all hit one place for auth, rate limits, and logs, then route to OpenAI, Anthropic, or open models.

When to use: several services call models, or you need kill-switches and routing. When to skip: one app, one key, early prototype.

AI gateway: apps → policy → many providers

Quick reference

Centralize keys, limits, and request logs.
Route easy tasks to cheaper models.
Failover and shadow traffic become possible.
Version the gateway like any critical API.

OpenAIAnthropicOpen Models

Application calls one gateway contract

1const response = await fetch("https://ai.internal/v1/generate", {2  method: "POST",3  headers: {4    "content-type": "application/json",5    authorization: `Bearer ${process.env.AI_GATEWAY_TOKEN}`,6  },7  body: JSON.stringify({8    task: "triage",9    input: ticket.summary,10    requestId: ticket.id,11  }),12});13 14if (!response.ok) {15  throw new Error(`gateway_failed:${response.status}`);16}17const result = await response.json();

Gateway makes provider failure explicit

1try {2  return await callPrimaryModel(request);3} catch (error) {4  metrics.increment("provider_error", { provider: "primary" });5  if (!request.allowFailover) {6    return problem(503, "model_provider_unavailable");7  }8  try {9    return await callFallbackModel(request);10  } catch {11    return problem(502, "all_model_routes_failed");12  }13}

Remember this

One gateway centralizing auth, limits, and logs is what turns a scattered set of API keys into a single point for failover, routing, and a kill-switch when a provider degrades.

Inference economics

Inference economics is simple: tokens cost money. A request either hits cache (cheaper) or misses (full compute) — both land on the bill. The triage agent that pastes the whole repo into every prompt will bankrupt you.

Track cost per successful ticket draft, shrink context, cache where providers allow, and route easy classification to smaller models.

Low-level cost zoom: cache hit vs miss → bill

Quick reference

Measure cost per successful task, not only $/1k tokens.
Retrieval beats stuffing the entire codebase.
Budgets and alerts belong next to latency SLOs.
When to care: any feature past a weekend demo.

Remember this

Both a cache hit and a cache miss land on the bill — an agent that pastes the whole repo into every prompt turns that fact into a budget incident, not a rounding error.

Evals and guardrails

Evals run golden bug reports through the system and score pass/fail. Guardrails screen the user prompt, then check the model’s reply before it opens a ticket.

Ship both. Without evals, prompt edits are guesses. Without guards, a jailbreak or PII leak becomes your incident. Log every block so silent filters do not hide product bugs.

Quality: eval pipeline + guardrail pipeline

Quick reference

Golden sets beat “looks good in chat.”
Eval when prompts, models, or retrieval change.
Input screen → model → response checks → safe reply.
Fail closed on irreversible tools (delete, mass email).

Remember this

Without evals every prompt edit is a guess, and without guards a jailbreak or a PII leak becomes the incident — ship both before scaling traffic, and log every block so a filter doesn't hide a real product bug.

Follow one bug report through the system

Zoom in. A user pastes a crash log. The loop decides to search docs through an MCP tool, observes a weak match, asks for the app version, and then opens a draft ticket. An input guard rejects instructions embedded in an untrusted log, while a trace records the tool sequence, token use, latency, and terminal status.

That path turns the vocabulary into diagnostic questions: which decision repeated, which tool failed, what context crossed a trust boundary, and did the final ticket pass the eval rubric?

Small-scale zoom: one crash-log triage

Quick reference

High-level map = which boxes exist.
This path = what happens on one ticket.
Missing traces make “why did it loop?” unanswerable.
Practice: write this path for your own agent before adding subagents.

Remember this

Tracing one report through loop → MCP tool → guard → draft ticket is what turns "why did it loop?" from an unanswerable question into a specific decision, tool, or trust boundary to fix.

Observability and the Bitter Lesson

Observability means agent runs emit traces, logs, and metrics into panels you actually watch. Without them you cannot debug a twenty-step failure.

The Bitter Lesson: systems that bet on general models + compute beat brittle custom rules that do not scale. Keep thin rules for hard policy; bet the core on learning, then wrap with evals and guards.

Observability signals + Bitter Lesson paths

Quick reference

Traces = step path; logs = events; metrics = cost/latency/quality.
Custom rules → hard to maintain → weak scaling.
Models + compute → improve over time → stronger results.
Use rules as policy edges, not as the whole brain.

Remember this

A twenty-step failure is undebuggable without traces, logs, and metrics — and the Bitter Lesson says bet the core on general models plus compute, keeping custom rules only as policy edges that don't have to scale.

Key takeaway

The nine concepts — loops, MCP, multi-agent, gateways, inference economics, evals, guardrails, observability, and the Bitter Lesson — describe different production concerns. Locate the failed lane before choosing a framework.

Practice (30 min): paste the agent-loop.ts sample from Agentic loops and run npx tsx agent-loop.ts. Verify one tool observation and one final answer, then set maxSteps to 1 and verify max_steps_exceeded. Add one unsafe ticket string, one guard that blocks it, and a trace object containing requestId, steps, latencyMs, and cost; the exercise passes only when all three terminal states—success, blocked, and capped—are observable.