The Evolution of LLM APIs: OpenAI vs Anthropic vs Google

Every major provider's API went through a recognizable arc. Generation one was a single prompt string with a single completion string back — useful for text generation, unworkable for anything needing memory or structure. Generation two added a message array with roles (system/user/assistant) and, shortly after, function/tool declarations the model could ask the caller to invoke. Generation three, which is where the three current APIs sit today, adds typed structured outputs (a JSON schema the response must conform to, not just a hopeful prompt instruction) and native multi-step tool loops with explicit reasoning controls.

The useful takeaway is not "the newest generation is best" — it's that the generations are not interchangeable, and code written against generation one (a single prompt string, no roles) silently breaks trust boundaries once you add tools, because there is no role separation to keep a tool result from being mistaken for a user instruction. Prompt injection through an unstructured tool result is a direct consequence of skipping the message-role generation; see prompt injection in RAG and tool-using agents.

OpenAI's Chat Completions API set the pattern most of the industry copied: a messages array with roles, a tools array of function schemas, and a tool_calls field the model populates when it wants to invoke one. The newer Responses API keeps that shape but adds a persistent previous_response_id chain for multi-turn state and built-in tool types (web search, code execution) the caller doesn't have to implement.

For the order-lookup assistant, Chat Completions requires you to manage the message history yourself and re-send it every call. The Responses API can carry state server-side via the response ID chain, trading a stateless request for one less thing your backend has to store — a real operational trade-off, not just an API preference.

Anthropic's Messages API uses the same role-based shape but keeps the system prompt as a separate top-level field rather than a message with role: "system", and returns tool calls as typed content blocks (type: "tool_use") inside the assistant message rather than a separate tool_calls array. Functionally equivalent to OpenAI's approach, but the field paths differ enough that naive code ports break silently — reading message.tool_calls on an Anthropic response returns undefined, not an error, which is the worse failure mode.

Google's Gemini API took a different structural choice: contents is a list of turns, each with parts that can mix text, inline data (images), and function calls in the same array — reflecting Gemini's multimodal-first design. A tool call arrives as a functionCall part alongside any text parts in the same turn, rather than a dedicated field.

Given three real, incompatible wire formats, the decision that matters is what to abstract behind your own interface versus what to leave provider-specific. Abstract the concepts that are stable across generation three: message turns, tool declarations, and a normalized "tool call requested" event your application logic reacts to. Do not abstract away provider-specific reasoning controls, context-caching behavior, or rate-limit semantics — those differ enough in practice that a leaky abstraction there produces bugs that only show up under load.

The realistic failure to design against: a team builds one internal LlmClient interface, ships it, and later discovers a parallel-tool-call response from one provider silently drops all but the first call because the adapter was written against a single-call assumption from an earlier integration. The fix is treating "can this turn contain more than one tool call" as a contract question to verify per provider, not an assumption to copy from the first one you integrated.