The Complete Map of an LLM System

A production LLM system is a chain of contracts, not a model wrapped in a chat box.

The computation layer

Text is tokenized, embedded, and transformed into next-token distributions. The context window bounds the working material and decoding selects a continuation. This layer explains capability and constraints, but it does not supply application state, permissions, or truth.

The application layer

Messages and prompts define the current task. Structured outputs make responses parseable. Tools provide authoritative computation and action while the application retains authorization. State and memory live outside the model and are selected into context deliberately.

The knowledge layer

Ingestion creates versioned, permission-aware chunks. Retrieval generates candidates, hybrid signals and reranking improve relevance, and grounded generation binds claims to evidence. Retrieval quality and answer quality must be measured separately.

The control layer

Evaluation defines acceptable behavior. Observability connects model, retrieval, tool, cost, and user outcomes. Security establishes trust boundaries and least privilege. Reliability engineering sets timeouts, retry budgets, fallbacks, and service targets.

A practical design order

Begin with the user decision or task, define success cases and risks, choose the smallest sufficient model workflow, build typed boundaries, then add retrieval, tools, memory, and adaptation only when evidence justifies them. Measure cost per successful task, not model calls in isolation.

Decision checklist

1. Name the user outcome and unacceptable failures.
2. Identify the layer where the observed problem originates.
3. Choose the smallest mechanism that directly addresses that problem.
4. Define representative evaluation cases and operational budgets.
5. Preserve source, model, prompt, data, and release versions.
6. Require explicit approval before changing public behavior.