Month archive
Everything published in July 2026.
The 7 articles published in July 2026 cover Inference Systems and Evaluation and Security. They were published from July 3 to 24, 2026, and are listed newest first.
Articles from July 2026
Inference SystemsTransformer Inference, From HTTP Request to the Next Token
A systems-level tour of prefill, decode, batching, memory movement, and the scheduling decisions that determine real-world LLM latency.
Inference SystemsKV Cache Engineering: Pages, Prefix Reuse, and Memory Pressure
How to size, allocate, reuse, compress, and evict the state that makes autoregressive decoding practical.
Inference SystemsSemantic Caching Without Serving the Wrong Answer Faster
A production design for similarity keys, freshness, authorization, invalidation, and the economics that determine when semantic reuse is safe.
Inference SystemsSpeculative Decoding Without Hand-Waving
Draft models, acceptance math, tree proposals, and the operational details that decide whether speculation makes serving faster.
Evaluation and SecuritySynthetic Evaluation Data That Finds Real Failures
How to generate, filter, diversify, and maintain synthetic cases without turning an evaluation suite into a mirror of its generator.
Inference SystemsThe LLM Gateway Is a Policy Engine, Not a Proxy
Designing the routing, budgets, identity, resilience, and evidence layer between products and a changing model portfolio.
Evaluation and SecurityPrivacy-Preserving AI Is a Dataflow Architecture
Purpose limitation, minimization, isolation, retention, redaction, and verifiable deletion across retrieval, models, tools, traces, and feedback loops.