Issue 1322 Sept 2026
What a rehearsed rollback actually costs
FrameworkThe quarterly rollback drill checklist
[Principal, Engineering] · 2 min
Weekly Wisdom
Notes on Foundations.
- One signed essay.
- One thing we saw inside a client engagement, anonymised.
- One framework you can use on Monday.
- Three things worth a senior leader's time this week.
Under 1,200 words, every Tuesday.
- 9 issues issues in the archive
- Two minutes a week
- No fluff
The archive
Every issue, with the framework it gave you.
Nobody scans subject lines. Everybody scans frameworks — so every issue carries the one thing it hands you, and that is what the archive is indexed by.
- Issue 1429 Sept 2026
The eval set is the architecture
FrameworkThe 7-question AI readiness audit
[Principal, Engineering] · 2 min
- Issue 1215 Sept 2026
Your first AI hire is not a researcher
FrameworkThe first-five-hires sequence
[Principal, Strategy] · 2 min
- Issue 118 Sept 2026
The demo that answered eleven questions
FrameworkGolden-set coverage in four passes
[Senior Engineer] · 2 min
- Issue 101 Sept 2026
Trust is a latency budget
FrameworkThe p95-to-adoption map
[Senior Designer] · 2 min
- Issue 925 Aug 2026
Classifying against the EU AI Act at design time
FrameworkThe risk-tier decision tree
[Principal] · 2 min
- Issue 818 Aug 2026
Why we moved retrieval off a managed service
FrameworkThe vector-store cost model
[Senior Engineer] · 2 min
- Issue 711 Aug 2026
The twelve things a clinician does daily
FrameworkTask-frequency interface audit
[Senior Designer] · 2 min
- Issue 64 Aug 2026
No pyramid, and what it costs us
FrameworkThe senior-only staffing model
[Founder] · 2 min
This week, in full
The eval set is the architecture.
Issue 14 · 29 September 2026 · by [Principal, Engineering] · 2 min
The framework in this issue
The 7-question AI readiness audit
Founder's Note[Founder]
Every system we have been asked to rescue this year was on its second build, and in every case the first one failed for the same reason: nobody could tell when the model was wrong. Not because it was hard to measure, but because measuring it was never anybody's job.
The uncomfortable version of this argument is that an eval set is an architectural decision rather than a testing one. Where it lives, who owns it, and whether it can block a release are all choices you make in week one and cannot meaningfully revisit in month nine — because by then the system is shaped around not knowing.
Engineer's Notebook[Senior Engineer]
A thing we got wrong and fixed this month: our golden set was written entirely by engineers, which meant it measured the failure modes engineers had thought of. The client's claims team found four more in an afternoon, two of which were the expensive kind.
The fix was procedural rather than technical. The eval set now has a named owner in the business, and adding a case to it does not require a pull request. Coverage went up forty per cent in three weeks, and none of that was our work.
Designer's Margin[Senior Designer]
We removed a confidence score from an interface this month and adoption went up. The score was accurate; it was also uninterpretable — users had no way to convert 0.78 into a decision, so they either ignored it or over-trusted it, and which one depended on the user rather than on the number.
What replaced it was a sentence naming the two sources the answer came from. Same information budget, and now it points at something the user can check.
Strategist's Memo[Principal, Strategy]
The procurement conversation has shifted from "can you build it" to "can you show me it still works in eighteen months", and most vendors cannot answer the second question because they never instrumented the first.
If you are buying this year, the question worth asking is not about the model. It is: what blocks your release, who owns that gate, and when did it last fire? A vendor who can answer with a number has run a real pipeline. A vendor who describes a process has not.
If that was worth two minutes, the next one arrives Tuesday.
The format
The same four parts, every week.
So you can skim straight to the one you care about, and so we cannot quietly turn it into whatever we felt like writing.
Founder's Note
the founders
One argument, first person, signed.
Engineer's Notebook
rotating engineers
Something specific that was learned in production this month.
Designer's Margin
rotating designers
A design decision and what it cost.
Strategist's Memo
rotating strategists
What the market did, and what it means for an operator.
Who writes it
A rotation, with its real numbers.
- [Principal, Engineering]
[Principal, Engineering]
2 issues written
- [Principal, Strategy]
[Principal, Strategy]
1 issue written
- [Senior Engineer]
[Senior Engineer]
2 issues written
- [Senior Designer]
[Senior Designer]
2 issues written
- [Principal]
[Principal]
1 issue written
- [Founder]
[Founder]
1 issue written
The numbers are uneven because the rotation is real. We publish them rather than claiming an even split, because an even split would mean we were assigning issues to fit a diagram.
Subscribe
No recruitment, no product announcements, no webinar invites. One email, Tuesdays, and an unsubscribe link that works on the first click.
We will never sell the list, and we do not run it through a third-party tracker that tells us what you opened.
The archive
Every issue, at a permanent address.
Nothing here expires and nothing lives only in an inbox. Search by subject, by framework, or by who wrote it.
9 of 9 issuesof 9 issues
Showing 9 of 9 issues.
- No. 14The eval set is the architectureThe 7-question AI readiness audit[Principal, Engineering] · 2 min29 Sept 2026
- No. 13What a rehearsed rollback actually costsThe quarterly rollback drill checklist[Principal, Engineering] · 2 min22 Sept 2026
- No. 12Your first AI hire is not a researcherThe first-five-hires sequence[Principal, Strategy] · 2 min15 Sept 2026
- No. 11The demo that answered eleven questionsGolden-set coverage in four passes[Senior Engineer] · 2 min8 Sept 2026
- No. 10Trust is a latency budgetThe p95-to-adoption map[Senior Designer] · 2 min1 Sept 2026
- No. 9Classifying against the EU AI Act at design timeThe risk-tier decision tree[Principal] · 2 min25 Aug 2026
- No. 8Why we moved retrieval off a managed serviceThe vector-store cost model[Senior Engineer] · 2 min18 Aug 2026
- No. 7The twelve things a clinician does dailyTask-frequency interface audit[Senior Designer] · 2 min11 Aug 2026
- No. 6No pyramid, and what it costs usThe senior-only staffing model[Founder] · 2 min4 Aug 2026
Issue pages are being migrated to permanent URLs. Until that lands, this week’s issue is readable in full above.