Engineering the Context Window
Retrieval, memory, and the working set a model actually reasons over
The context window is the one surface you fully control, so most problems blamed on model quality are context-assembly problems.
Retrieval, memory, compaction and ordering are usually built by different people at different times, and then the result is judged as though the model were the variable. This book treats them as a single system for deciding what the model sees: an evidence pipeline feeding a budgeted, ordered window, with provenance, freshness and access control surviving all the way into the prompt. It also takes the long-context argument seriously — long windows did kill lazy retrieval — and gives you the tests that show which parts of your own pipeline are now redundant.
Written for: Teams whose RAG system plateaued and who are being told long context makes it obsolete.
No chapter is published yet. The full contents below is the finished outline; chapters go live as they are written, and none of them will sit behind a signup.
What it makes operable
Outcomes, not topic coverage.
- 01Decide retrieval versus long context on corpus size, freshness, access control and cost
- 02Run lexical and vector retrieval together and fuse them by rank rather than score
- 03Assemble a window under an explicit token budget, with decisive evidence out of the middle
- 04Write a memory policy that holds up when the evidence is untrusted or poisoned
Complete contents
12 chapters, in order.
Each chapter settles one decision, so it can be read on its own. Read in sequence to build the argument the book is making.
- 01In writing
Context Is the Only Thing You Control
The surface you own, as against the weights you do not.
- 02In writing
What Long Context Fixed, and What It Did Not
Mid-window accuracy loss and cost per query, measured on your own corpus.
- 03In writing
Retrieval as an Evidence Pipeline
The stages, contracts and failure points between a query and a supported claim.
- 04In writing
Lexical and Vector Retrieval, Together
Why exact identifiers still need BM25, and how to fuse two ranked lists.
- 05In writing
Rerankers and the Cost of Being Sure
Spending cross-encoder time only where it changes the evidence set.
- 06In writing
Chunking Is a Document-Modelling Problem
Segmenting on structure so a claim keeps the qualifier that bounds it.
- 07In writing
Assembling the Window: Order, Budget, Compaction
A token budget per section, an ordering rule, and compaction you can reverse.
- 08In writing
Memory: What to Write, What to Forget
A write policy for durable facts, with provenance and a TTL on every record.
- 09In writing
Provenance, Freshness, and Access Control Inside Context
Keeping who-may-see-this and how-old-is-this attached to the evidence itself.
- 10In writing
Poisoned Memory and Untrusted Evidence
Treating retrieved text, including your own memory, as attacker-controlled input.
- 11In writing
Measuring Context Quality Independently of the Model
Scoring the window so a bad answer can be blamed on the right stage.
- 12In writing
Migrating a RAG System Into a Context System
The order of operations for rebuilding a plateaued pipeline in place.
Evidence
What this book is arguing against.
Third-party sources behind the book's premise. Every figure in them belongs to the party that published it and is attributed to them in the text.
- RAG vs Long Context: What the 2026 Data Shows
Wire · 2026-04-08 · Industry report · reported
- Context Engineering: A Practical Guide for AI Agents (2026)
Sourcegraph · 2026-05-28 · Vendor engineering · reported
- Lost in the Middle: How Language Models Use Long Contexts
Stanford University · 2023-07-06 · Research paper · verified
- Reciprocal rank fusion
Elastic · Official documentation · verified
Continue through the library
Adjacent operating manuals.
Proof Before Production
The demo passes because someone chose the inputs. The product fails quietly because production did not. This book builds the apparatus that closes the gap: a definition of done for open-ended output, a first dataset drawn from real traffic, metrics separated into outcome, trajectory and system, a judge calibrated against human annotation that carries its own error bar, and a suite with the authority to fail a deploy. It also draws the line where the OpenTelemetry GenAI conventions stop and your own evaluation data has to start.
Read the full book
The Agent Trust Boundary
Assume the injection succeeds. That single assumption reorganises the whole problem: security stops being a matter of instruction wording and becomes a question of identity, authority and containment. This book draws the boundary — who the agent is, what it may call, where its code runs, what it may reach, what it may remember — and places enforcement outside the model, at the gateway, the tool or the destination system. It ends with the evidence an auditor asks for, because from 2 August 2026 some teams have to produce it.
Read the full book