AI & LLM Fundamentals: Models, Tokens & Context Windows

Artificial Intelligence (AI) encompasses systems that simulate human intelligence. Machine Learning (ML) is a subset of AI where algorithms learn patterns from data rather than following hardcoded rules. Deep Learning uses multi-layer neural networks for complex tasks like image recognition and natural language understanding.

Large Language Models (LLMs) are deep learning models trained on vast text corpora to predict next tokens. Small Language Models (SLMs) deliver lower latency and cost for focused tasks on edge devices. Foundation Models are general-purpose pre-trained base models (e.g. GPT-4, Claude 3.5, Llama 3) that adapt to downstream applications. Multimodal AI processes multiple modalities simultaneously—text, vision, audio, and video.

Text is processed as tokens—sub-word units averaging ~4 characters in English. The Context Window defines the maximum token capacity an LLM can evaluate in a single request. Temperature controls output randomness: low values (0.0–0.2) yield deterministic output, while high values (0.7–1.0) increase variation. When LLMs generate ungrounded or invented facts, it is called a Hallucination. Grounding connects responses to verified data sources, and Guardrails enforce safety, correctness, and business rules.