v0.1 · a conjecture, perpetually exposed to refutation
From User Intent User Interface What actually happens between a user pressing Enter on a prompt and a rendered result landing back on their screen. Four orthogonal projections of one system: the journey a request takes (Axis B), the layers it is built on (Axis A), the concerns that cut across them (Axis C), and where every runtime artifact came from (Axis D).
Click any node to zoom in. The journey is the spine; every phase drills into the layers it exercises.
Axis B · Lifecycle The Journey — one request, Φ0 → Φ17 Follow a single prompt from intent to interface. The loop (Φ3 → Φ13) repeats every turn; tool calls can recurse into the whole machine again. Click a phase to see the layers it touches.
User Intent a thought, underspecified
Φ0 Intent formation L0 Φ1 Input capture L1.b Φ2 Prompt preprocessing L1.b L2.j ↺ agent loop — repeats Φ3 → Φ13, appending context each turn
loop Φ3 Context assembly L2.b Φ4 Request construction & tokenization L2.i L3.1 L3.4 Φ5 Transport to engine L3.3 C9 Φ6 Admission · scheduling · batching L4.a Φ7 Prefill L5.d L6 L7 L8 Φ8 Decode loop L4.c L4.b L5 L7 Φ9 Stream back L3.3 L2.i Φ10 Incremental parse L2.i.3 ↺ recurses Φ11 Tool dispatch & execution L2.c L2.f Φ12 Tool-result capture L2.c.4 loop Φ13 Loop decision L2.a L2.d L2.e Φ14 Final synthesis L2.a.4 Φ15 Output rendering L1.c Φ16 State persistence L2.h Φ17 Side effects on the world world User Interface a rendered result
Axis A · Vertical layers The Stack — what is built on what Human at the top, silicon at the bottom. Each layer consumes the interface below it and exposes one above. Click to expand; the zoom is the recursion.
L0 Human / Intent top boundary The human and the intent we model the interface to — not the psychology of it. 5 parts ▸ L0.1 Goal / intent L0.2 Mental model of the tool's capabilities & limits L0.3 Prompt as the encoding of intent L0.4 Gulf of execution / evaluation L0.5 Human cognition, prompting skill
L1 Interface / Client UI in → UI out Where intent enters as text and the result leaves as a rendered interface. 7 parts ▸ L1.a Modality & shell L1.b Input subsystem L1.c Output / rendering L1.d Interaction control L1.e Session UX L1.f Local config surface L1.g OS integration
L2 Agent / Harness / Orchestration the brain stem The largest layer — the agent runtime. Distinct responsibilities, each its own subsystem. 11 parts ▸ L2.a 5 › Control loop / runtime L2.b 10 › Context / prompt assembly L2.c 6 › Tooling subsystem L2.d 5 › Planning & decomposition L2.e 7 › Multi-agent / sub-agent orchestration L2.f 7 › External-capability protocol: MCP L2.g 6 › Permissions / safety / sandboxing L2.h 5 › State & persistence L2.i 7 › Model-interaction management L2.j 4 › Extensibility L2.k Instrumentation hooks
L3 Model-serving interface / API harness ↔ model contract The contract between the harness and the model: protocol, auth, tokenization, tool-calling. 12 parts ▸ L3.1 API protocol L3.2 Auth L3.3 Streaming transport L3.4 Tokenization at the boundary L3.5 Function/tool-calling protocol L3.6 Structured output / JSON mode L3.7 Prompt-caching protocol L3.8 Sampling params exposed L3.9 Rate limits / quotas / usage metering / billing L3.10 Safety/moderation classifiers L3.11 Provider gateway / proxy L3.12 Versioning
L4 Inference serving / runtime the engine The engine that actually runs the model: scheduling, KV memory, decoding, parallelism, quant. 9 parts ▸ L4.a Request lifecycle in engine L4.b Memory management L4.c 6 › Decoding / generation strategies L4.d Parallelism / distribution L4.e Runtime quantization & optimization L4.f Adapter / multi-model serving L4.g Engines (instances) L4.h Serving control plane L4.i Model loading & formats
L5 Model / neural network the LLM as a mathematical object The transformer itself: architecture, weights, encoded capabilities, inference-time numerics. 4 parts ▸ L5.a 8 › Architecture (forward-pass structure) L5.b Parameters / weights L5.c Capabilities encoded in weights L5.d Inference-time computation (numerics)
L6 Compute framework / backend how tensors get computed Tensor frameworks, graph compilers, compute APIs and device runtimes. 6 parts ▸ L6.1 Tensor framework L6.2 Graph capture/compile L6.3 Compute API / programming model L6.4 Device runtime L6.5 Alt inference runtimes L6.6 Dispatch
L7 Kernel / numerical primitives the actual math on the metal The GPU kernels: GEMM, attention, norms, sampling, collectives — and how they're authored. 12 parts ▸ L7.1 GEMM/matmul L7.2 Attention kernels L7.3 Norm kernels L7.4 Activation kernels L7.5 Elementwise/pointwise; reductions L7.6 Sampling kernels L7.7 Quant/dequant kernels L7.8 Collective-comm kernels L7.9 Kernel libraries L7.10 Kernel authoring L7.11 Optimizations L7.12 Memory-hierarchy use
L8 Hardware / silicon bottom boundary The accelerators, memory, interconnect and datacenter substrate the FLOPs run on. 8 parts ▸ L8.1 Accelerators L8.2 Compute units L8.3 Memory L8.4 Interconnect (intra-node) L8.5 Interconnect (inter-node) L8.6 Host L8.7 Datacenter substrate L8.8 Semiconductor microarch → transistor → lithography → physics
Axis C · Cross-cutting planes The Concerns that span every layer The systematically under-enumerated stuff — threaded through the whole stack rather than living in any one layer.
C1 Observability & telemetry structured logs; tracing (spans across agent steps + tool calls); metrics (latency, TTFT/TPOT, throughput, tokens, cost); LLM tracing (LangSmith, Langfuse, OpenLLMetry/OTel, Phoenix); replay/debug; dashboards C2 Evaluation & quality offline evals (golden sets); prompt unit/regression tests; LLM-as-judge; rubric grading; eval harnesses; online eval / A-B; guardrail checks; benchmarks (SWE-bench, HumanEval, Aider Polyglot); eval-driven dev C3 Security & trust prompt-injection / jailbreak defense; tool sandboxing; secrets mgmt; supply-chain provenance; data-exfiltration prevention; MCP-server trust; output sanitization; authN/authZ; multi-tenant isolation; red-teaming; audit trails C4 Cost / FinOps / efficiency token accounting; model routing for cost; cache-for-cost; batch-for-throughput; quant-for-cost; budget enforcement; cost attribution C5 Caching (multi-level) provider prompt/context cache (L3.7); engine KV cache (L4.b); engine prefix cache; semantic/response cache; tool-result cache; embedding cache; HTTP cache — spans L2/L3/L4 C6 State, memory & knowledge short-term (context window); working (scratchpad); long-term (vector DB / memory store / KG); episodic vs semantic; RAG (chunking, embedding, HNSW/IVF, pgvector, reranking); write/update/forget policies; project knowledge (CLAUDE.md); citations C7 Configuration & reproducibility layered config; env mgmt; model/version pinning; determinism caveat (inference non-deterministic from batching + floating-point even with fixed seed); prompt versioning; experiment tracking C8 Concurrency, reliability & lifecycle async/parallel exec; rate-limit & backpressure; retries/idempotency; circuit breakers; graceful degradation/fallbacks; durable execution / crash recovery; timeouts; cancellation C9 Networking & transport literal pipes: HTTP/2, SSE, WebSocket, gRPC, TLS, load balancers, CDNs, API gateways — between client ↔ API ↔ engine C10 Deployment & infra / control plane containerization; orchestration (K8s); autoscaling; GPU scheduling; model registry/serving infra; CI/CD for models & prompts; IaC; multi-region — collapses to process management for local tools
Axis D · Provenance Where every runtime artifact came from Build-time pipelines. Lets the drill-downs terminate honestly: weights, tokenizer, engine, kernels, harness — each has an origin.
D1 Model production (training) produces: weights (L5.b) D2 Tokenizer production produces: tokenizer (L3.4): corpus, algorithm (BPE/Unigram), vocab size, special tokens, merges D3 Inference-engine & kernel production produces: engine binaries (L4.g), kernels (L7): kernel authoring/tuning, compiler toolchains, per-GPU engine compilation (TensorRT), benchmarking D4 Harness / tool production produces: the agent software (L2): prompt engineering, tool design, system-prompt authoring, eval suites, the harness codebase D5 Distribution & packaging model hubs, registries, package managers, container images, release pipelines