Prototype Production
From prototype to production — without the second rewrite.
build 4,182 · green · deployed 6m ago · p95 312ms
We build AI-native systems that hold up under real traffic, real auditors and real on-call rotations. Observability from day one. Evals in CI. Nothing ships without a rollback path.
- p95 latency budgets
- drift detection
- eval gates in CI
- Delivered inside your controls
The second rewrite
Most AI projects are built twice. We charge for one.
The pattern is always the same. A team ships a working prototype in six weeks, the demo lands, leadership funds it — and then it turns out the prototype has no auth model, no evaluation harness, no cost ceiling, and a vector index rebuilt by hand every Tuesday. The second build takes four times longer than the first, and it is paid for out of the credibility of whoever championed it.
We build the production system first and let it be small. A narrow thing that is observable, tested, rate-limited and rollback-safe beats a broad thing that is none of those.
The Pipeline
Every change takes the same road, and one gate can stop it.
builds: 4,182·blocked by evals: 137
scroll the graph →
Commit
Trunk-based, short-lived branches, required review from a named owner per directory. CODEOWNERS is not optional.
Lint & types
Strict TypeScript, no `any` in shipped code, Ruff and mypy on Python services. Type errors are build failures, not warnings.
Unit & contract
Contract tests against every external boundary. If a vendor changes a schema, CI finds out before production does.
Eval gate
Can block a mergeThe one that doesn't exist at most shops. Golden-set evals, regression thresholds, hallucination and jailbreak suites. A model change that drops accuracy below threshold cannot merge.
Build
Reproducible, pinned, SBOM generated, image signed.
Canary
5% of traffic, automated comparison on latency, error rate, cost per request and eval score. Promotion is automatic; so is abort.
Production
OpenTelemetry traces end to end, drift detection on inputs and outputs, cost dashboards per feature, and a rollback that has actually been rehearsed.
The stack
Grouped by the decision, not by the logo.
Hover any of these for the version or the reason we reach for it. Knowing a tool is table stakes; knowing when not to use it is the part you are paying for.
Product surface
Server components where they pay, client where they don't. We don't ship a SPA to render a marketing page.
- Next.jsApp Router, server components by default
- React 19Actions and use() where they remove state
- TypeScript strictno implicit any, no unchecked index
- Tailwindtokens in config, not in a stylesheet
Services
Rust is not a personality. We reach for it when a hot path is measured, not when it's fashionable.
- Python · FastAPIasync, typed, OpenAPI for free
- Gofor services that are mostly I/O and must start fast
- Rustonly after a profile says so
Data
Most teams reach for a vector DB before they've outgrown pgvector. We check first.
- Postgresthe default until it measurably isn't
- pgvectorfine to several million vectors
- Pineconewhen recall at scale beats operational cost
- Weaviatehybrid search without a second store
- Databrickswhere the lakehouse already exists
- Snowflakesame — we build onto what you run
AI
Model choice is a per-task decision with a cost and eval column, not a vendor allegiance.
- Anthropiclong-context reasoning and tool use
- OpenAIwhere the eval set says it wins
- Llama · Mistralopen weights when data cannot leave
- LangGraphexplicit state machines over implicit chains
- LlamaIndexretrieval plumbing we don't need to own
Platform
Infrastructure as code, in your account, under your IAM. You can fire us on a Friday and still deploy on Monday.
- AWSmost of what we run
- Azurewhere the enterprise agreement already is
- GCPwhere the data gravity is
- Kuberneteswhen more than one team deploys
- Terraformstate in your account, reviewed in PRs
Observability
Vendor-neutral instrumentation. Your traces are yours, and they're portable.
- OpenTelemetrythe instrumentation layer, always
- Datadogwhere a team already lives there
- Grafanaself-hosted when egress costs matter
- Sentryexceptions routed to a named owner
AI-native engineering
Six capabilities, each one an answer to a specific failure.
Retrieval (RAG)
Naive chunking destroys meaning at exactly the boundaries that matter.
Hybrid retrieval with reranking, evaluated on a golden set per corpus. We publish retrieval precision alongside answer quality, because an answer that scores well on a document the system never found is not a win.
Agentic systems
An agent with unbounded tool access is an incident waiting for a calendar slot.
Scoped permissions, a step ceiling, a cost ceiling, and a full decision trace you can replay. If it did something expensive at 3am, you can read why.
Evals and LLMOps
Most eval suites live in a notebook that one person maintains out of goodwill.
Golden sets, regression thresholds, human review sampling, and a dashboard the product owner actually reads. Evals run in CI — see the gate in the pipeline above.
Fine-tuning
It is usually the wrong answer, and it is usually reached for first.
When it is the right one we say so with a cost comparison against prompting and retrieval, and we keep the base-model fallback wired. A tuned model you cannot roll off is a dependency, not an asset.
Red-teaming
Prompt injection arrives through retrieved content, which most threat models never look at.
Jailbreaks, injection through the corpus, data-exfiltration paths, tool-abuse chains. Findings go in a register with owners and dates, same as any other security finding.
Guardrails
A refusal with no escalation path is a dead end wearing a polite sentence.
Input and output filtering, grounded-claim checking, refusal policies, and a handoff to a human that is designed rather than bolted on.
Production discipline
The practices, and what each one means here.
| Practice | What it means here |
|---|---|
| Code review | Named owner per directory. No self-merge to main, including for us. |
| Test strategy | Unit, contract, integration, eval. Coverage thresholds enforced in CI. |
| Performance budgets | p95 latency and cost-per-request are build-gated, same as tests. |
| Error tracking | Every exception routed with an owner and an SLA. |
| Drift detection | Input distribution and output quality monitored; alerts route to on-call. |
| Rollback | Rehearsed quarterly. An untested rollback is a hope. |
| On-call | We carry the pager during the stabilisation window, alongside your team. |
| Runbooks | Written during the build, not after the first incident. |
The log stream
An incident, told straight.
- 02:14:07alertp95 latency 312ms → 2,940ms svc:retrieval
- 02:14:09pageon-call acknowledged
- 02:16:41notereranker container OOM after index rebuild
- 02:19:02actiontraffic shifted to previous index version
- 02:19:55alertp95 recovered to 340ms
- 02:21:00notecustomer impact: 6m 48s degraded, 0 failed requests
- 09:30:00actionpost-mortem scheduled, blameless
We sized the reranker for the old index and didn't re-run the memory profile after the rebuild. The rollback worked because we had rehearsed it. The fix was a memory ceiling in the deploy spec and a profile step added to the index pipeline.
Six minutes, forty-eight seconds.
That is the number, and we will give you the post-mortem.
“The rollback was boring. That is the entire point of rehearsing it — by the time you need it, it should be the least interesting thing that happens that night.”
Data, access and AI use
How we handle your data and code.
Your data is not used to train any third-party model. That is a contract term, not a policy page.
The full procurement packWe build in your cloud accounts and work inside your audited environments, under your controls. That is the short answer to most of a security questionnaire, and it is true before we sign anything.
For the rest, ask. We will send our current data handling and AI use practices in writing, name the specific tools and access rules, and tell you plainly which certifications we do and do not hold rather than implying either.
The checklist
Twenty-three things that must be true before an AI system goes live.
The checklist we run internally, published in full. Use it on us. Use it on your current vendor. Use it on your own team.
Download the checklist(PDF, 23 items)- A rollback that has been executed in the last 90 days, not just written down.01
- An eval suite that runs in CI and can fail the build.02
- A cost ceiling per request, enforced, with an alert below the ceiling.03
- Every exception routed to a named owner with an SLA.04
- Input and output drift monitored, with a threshold someone agreed to.05
- + 18 more
Ready to build it once?
Observability from day one, evals gating the build, and a rollback that has been rehearsed. It starts with a Discovery Sprint — fixed scope, fixed price, senior engineers only.
The Usool Method
- Principle
- Pattern
- Prototype
- Production
- Proof