AI & LLM Glossary: 60+ Essential Terms, Mechanisms & Architecture

Modern AI systems rest on fundamental architectural abstractions. At the core is Artificial Intelligence (AI)—technology enabling computers to perform tasks requiring human intelligence. Machine Learning (ML) enables systems to learn patterns from data rather than following static hardcoded rules. Deep Learning uses multi-layer neural networks for complex processing, while Generative AI creates new content across text, code, audio, and video.

The Transformer & Model Scaling

• LLM (Large Language Model): General-purpose foundation models (e.g. GPT-4, Claude 3.5, Llama 3) pre-trained on massive text datasets.
• SLM (Small Language Model): Compact, high-speed models (e.g. Phi-3, Gemma 2B) optimized for edge execution and low-latency task completion.
• Transformer: The foundational neural network architecture utilizing self-attention mechanisms to process tokens in parallel.
• Parameters: The learned numerical weights of a neural network, serving as a rough proxy for model capacity and hardware requirements.
• Scaling Laws: Empirical observations showing model performance predictably improves as compute, dataset size, and parameter count scale exponentially.
• Frontier Model: The state-of-the-art models representing the boundary of intelligence and capability at any given moment.

Executing a trained model is called Inference. Modern inference engines optimize response latency, time-to-first-token (TTFT), and throughput.

Tokens, Context Windows & Reasoning Runtimes

• Token: The atomic unit of text processed by an LLM (~4 characters or 0.75 words in English).
• Context Window: The maximum token length an LLM can evaluate in a single request (now regularly 128K to 1M+ tokens).
• Reasoning / Thinking Models: Models that execute internal chain-of-thought tokens prior to outputting a final answer (e.g. OpenAI o3, DeepSeek R1).
• Test-Time Compute: Allocating extra inference compute to allow a model to 'think longer' on complex logic problems.
• Effort Levels: User-configurable dials controlling the depth of test-time compute thinking tokens.
• Temperature: Randomness hyperparameter governing token selection probability (0.0 = deterministic; 0.7+ = creative).
• Context Engineering: The discipline of curating context window payload structures, replacing static 'prompt engineering'.

An AI Agent is an autonomous system designed to act toward goals rather than merely responding to single prompts. Agents break goals into plans, call tools, inspect results, and self-correct in execution loops. For container boundaries, see our Claude Code Sandboxing Guide.

Tool Calling, MCP & Swarm Topologies

• Tool Calling / Function Calling: Enabling models to execute external APIs, SQL queries, or local code by emitting structured JSON schema parameters.
• MCP (Model Context Protocol): The open standard created by Anthropic that standardizes how AI assistants connect to local/remote tools and data sources.
• Connectors: Production MCP server plugins linking assistants to enterprise applications (Slack, Jira, GitHub, PostgreSQL).
• Skills: Packaged workflow instructions teaching agents repeatable multi-step tasks.
• Subagents: Specialized child agents spawned by a primary lead agent to execute parallel subtasks in isolated context scratchpads.
• Sandbox: An isolated ephemeral container (e.g. Firecracker, Sprites, Docker) where an agent safely executes bash commands or code.
• Human in the Loop (HITL): Required human confirmation gates before executing high-risk or irreversible operations.

Retrieval-Augmented Generation (RAG) grounds LLMs by fetching relevant documents from external knowledge repositories prior to answer generation. This eliminates stale static cutoffs and eliminates hallucinations.

The RAG Data Pipeline

• Chunking: Splitting raw documents into smaller text passages prior to embedding generation.
• Chunk Overlap: Retaining a small overlapping text segment between adjacent chunks to prevent context fragmentation.
• Embeddings: Dense numerical vector representations of text capturing semantic meaning in high-dimensional vector space.
• Vector Database: Specialized databases (Pinecone, Qdrant, pgvector) optimized for nearest-neighbor similarity search.
• Retriever: The engine fetching top-K candidate chunks using cosine similarity or Euclidean distance.
• Reranking: Secondary scoring models (e.g., Cohere Rerank) that re-order candidate chunks by precise relevance to the prompt.
• Hybrid Search: Combining keyword search (BM25) with vector similarity search for optimal recall and precision.
• Graph RAG: RAG leveraging knowledge graph relationships across entities for multi-hop reasoning.

Adapting general foundation models to specific enterprise domains requires targeted fine-tuning and quantization techniques.

PEFT, LoRA & Quantization

• Fine-Tuning: Further training a pre-trained base model on domain-specific datasets.
• Instruction Tuning: Fine-tuning a base model on instruction-response pairs to improve prompt following.
• LoRA (Low-Rank Adaptation): A Parameter-Efficient Fine-Tuning (PEFT) technique that freezes base weights and trains low-rank adapter matrices.
• QLoRA: Combining LoRA with 4-bit NormalFloat quantization to fine-tune 70B models on a single GPU.
• Quantization: Converting high-precision floating point weights (FP16/BF16) to lower precision (INT8/INT4) to reduce VRAM requirements.
• Distillation: Training a smaller 'student' model to emulate the output distribution of a larger 'teacher' model.
• Synthetic Data: AI-generated training data used to train, evaluate, or align downstream models.

Deploying AI in production demands resilient security controls, continuous evaluation, and a clear-eyed evaluation of market hype.

Alignment, Security & LLMOps

• AI Alignment: Ensuring model outputs conform to intended human goals, ethics, and safety rules.
• RLHF / RLAIF: Reinforcement Learning from Human (or AI) Feedback to optimize model response preferences.
• DPO (Direct Preference Optimization): A streamlined preference optimization algorithm operating directly on preference pairs without a reward model.
• Constitutional AI: Anthropic's method of training models against a written constitution of safety principles.
• Prompt Injection: A security attack where untrusted user input overrides system prompt instructions.
• Jailbreak: An adversarial attempt to bypass a model's safety and policy guardrails.
• Guardrails / Safeguards: Real-time input/output filters (e.g. NeMo Guardrails) ensuring responses stay safe and compliant.
• LLMOps: Processes and infrastructure for deploying, evaluating, tracing, and monitoring production LLM applications.
• Evals / Benchmarks: Automated testing suites (SWE-bench, MMLU, GPQA) measuring model performance.
• Model Drift: Degradation of model accuracy over time as real-world input distributions shift.

🚨 The Buzzword Hype Radar: What to View with Skepticism

1. 'AGI is Imminent': Marketing claims predicting Artificial General Intelligence within months often conflate benchmark optimization with true general reasoning. 2. 'Reasoning' Marketing Labels: Every new model update labeled 'reasoning' may simply be executing test-time compute tokens or basic chain-of-thought prompts. 3. 'Agentic' Applied to Simple Loops: Adding a single while loop around an API call does not constitute a full agentic architecture. 4. '10x Engineer Replacement': AI tools augment developer velocity, but autonomous systems still require human architectural direction, code review, and domain validation. 5. Unpublished Parameter Counts: Skepticism is warranted when vendor claims omit model parameter sizes, training token counts, or benchmark contamination checks.