# AI integration Applies when: a language model is part of your production system — summarising, extracting, classifying, answering, deciding — and something downstream depends on what it returns. This is not about writing better prompts. It is about the engineering around the call: what happens when it is slow, what happens when it lies, and what it is allowed to touch. ## The framing that fixes most of it **A model call is an external write to a system you do not control.** Everything in `external-writes.md` applies before anything model-specific does. It is a network call to a third party with worse latency variance than a payment gateway, no transactional semantics, and output you cannot predict from the input. The mistake is treating it as a function call because it is one line in your service class: it has the reliability profile of a remote API and the correctness profile of user input. ## 1. The model is an unreliable external system ### Timeouts and retries Model latency is heavy-tailed. The p50 is comfortable, the p99 is not, and the p99 is what users get during the incident. Set a timeout from what the caller can actually wait for, not from the provider's default. An unbounded model call inside a web request is an exhausted worker pool waiting to happen — every stuck request holds a process, and enough of them take down pages that never touched AI at all. Retry on transport failures and rate limits, with backoff and jitter. Do **not** retry on a response you did not like. That is not a retry, it is sampling until you get the answer you wanted, at multiples of the cost, and it hides the fact that your prompt is unreliable. If the call has a side effect — wrote a record, sent something, consumed a quota — it needs an operation record created before the call and reconciled after, like any other write that can time out mid-flight. ### Streaming fails differently A stream that dies at 80% is not necessarily an error your client raises. It looks like a shorter response: truncated JSON parses as invalid, truncated prose parses as finished. Treat a stream as incomplete until the provider's terminal event arrives; never persist partial output as a complete result, and never infer the UI's "done" state from the socket closing. ### Never put a model call on a user-facing path without a fallback **If the model is unavailable, the feature degrades — it does not 500.** - Queue it wherever the answer is not needed synchronously. Most AI features are not actually synchronous; they were built that way because it was easier. - Where it must be synchronous, define the degraded path explicitly: cached previous result, non-AI heuristic, or an honest "unavailable" that does not break the surrounding page. - Put a circuit breaker in front of it. When the provider is having a bad hour, failing fast for everyone beats every request waiting out its full timeout. ### Cost is a production concern with a hard ceiling Cost scales with traffic, and traffic includes abuse. An unmetered model endpoint is a budget incident that announces itself on the invoice. Before launch, non-negotiable: - Per-user and per-tenant rate limits on anything that reaches a model. - A hard cap on input size. Do not let an arbitrary uploaded document set your bill. - A cap on output length in the request itself. - Token usage recorded per call, attributed to a user or tenant, queryable. - An alert on spend **rate** — monthly total arrives too late to act on. - A kill switch that disables AI features without a deploy. - A cache wherever inputs repeat: input plus prompt version plus model is a cache key. ## 2. Security ### Prompt injection is the central threat, and it is not solved Models cannot reliably separate instructions from data. Everything in the context window is, to some degree, instructions. That is not a bug awaiting a patch — it is the nature of the technology, and your architecture has to assume it. **Direct injection** is a user typing "ignore previous instructions" — the version everyone tests for, and the less dangerous one. **Indirect injection** is the one that gets you: instructions arrive inside content the model *retrieves* rather than content the user typed. A support ticket. A scraped page. An uploaded PDF. A row in your own database written by someone six months ago. The user is not the attacker — the document is, and your user is whose privileges get spent. Content not authored by your engineers is untrusted, including content from inside your own system. ### "The system prompt says not to" is not a security control A system prompt is a strong suggestion to a probabilistic system. It is not a boundary, it cannot be audited, and it fails silently on inputs nobody tested. Prompts steer behaviour; authorization enforces it. If the only thing stopping the model from revealing another tenant's data is a sentence asking it not to, you do not have tenant isolation — you have a wish. ### Model output must never reach a privileged action unvalidated **Model output is untrusted input, and it is untrusted at the point of use.** Anything flowing into SQL, a shell command, a filesystem path, an HTTP request, an email send, or a payment is an injection vector by default. - Model-generated SQL: read-only connection, scoped user, parameterised — or it does not run. - Never `eval`, `exec`, or shell out on model output. No sanitiser is worth trusting here — the answer is to not have the capability. - Model-chosen file paths get resolved and checked against an allowlisted root. - Model-generated HTML or Markdown gets sanitised before rendering, or you have shipped XSS through your AI feature. - Model-generated URLs are not fetched without an allowlist, or you have built SSRF with a natural-language interface. ### The confused deputy is the failure mode that matters An agent acting on a user's behalf carries the *application's* privileges, not the user's, unless you deliberately arrange otherwise. The shape: injected content instructs the model to call a tool, and the tool runs as the service account. The fix is architectural: - Tools enforce authorization **inside** the tool, against the acting user's identity. Never assume the model only calls tools it was told it could call. - The model chooses *which* tool to call. It never chooses *whether the caller is allowed*. - Scope every tool to the smallest capability that works. A tool that reads one record by ID beats one that runs arbitrary queries, even though the latter is more flexible. Flexibility is the vulnerability. - Irreversible, costly, or externally-visible actions require human confirmation. An agent drafts the email. A person sends it. ### PII leaving your boundary Sending data to a model provider is a transfer to a third party and needs the same treatment as any other subprocessor: processing agreement, retention terms, training opt-out confirmed in writing, and a record of which data categories actually go. Redact or tokenise identifiers the feature does not need — most do not need them. Where tenants are contractually accountable for their data, "we send it to an AI vendor" is a disclosure obligation, not an implementation detail. Procurement will ask. ## 3. Output validation and correctness ### Constrain the shape, then validate it anyway Use structured output — schema-constrained generation, function calling, whatever the provider exposes. It eliminates most format failures and no semantic ones. Schema-valid output is still a plausible date in the wrong year, a currency code that does not exist, or a foreign key to another tenant's record. So: **validate model output at the same boundary and with the same strictness as a public HTTP request body.** Same form request, same rules, same rejection. It came from outside your system, because it did. ### A confident wrong answer is the default failure Models do not signal uncertainty reliably, and asking one how confident it is produces a number, not a measurement. Nothing throws on being wrong, so every value that matters gets checked against something authoritative: - Extracted identifiers get looked up. If the record does not exist, the extraction failed. - Numbers that must reconcile get reconciled in code. - Citations get verified against the source, not accepted because they look like citations. - Enumerated values get checked against the actual enum. Where a claim cannot be verified, present it as unverified rather than laundering it into your UI as fact because it arrived in a well-formed JSON field. Fail closed: an invalid response is a failed operation that retries or escalates — never a silent fallback to `null` that persists as though it were an answer. ### Evaluation sets, because you cannot regression-test by reading the diff A non-deterministic system has no reproducible test in the usual sense, which is exactly why it needs more testing discipline than a deterministic one, not less. Build an evaluation set: real inputs with known-correct outputs, including edge cases and everything that has failed in production. It need not be large, but it must be representative, and it must grow every time something breaks. Run it before every prompt, model, or provider change and record the score — without it, "we improved the prompt" is an assertion nobody can check. Assert on properties that matter — schema validity, required fields, no hallucinated identifiers, correct classification — not exact string equality, which will be flaky and get deleted. Deterministic parts stay deterministically tested: parsing, validation, retry logic and authorization are ordinary code with ordinary unit tests. ### Log enough to debug, not enough to leak When a model produces something wrong, the only way to understand it is to see exactly what went in; reconstructing a context window from memory does not work. Log per call: prompt version, model identifier, rendered input, raw output, token counts, latency, outcome — with a retention window and a way to find the call from the record it produced. Then keep secrets out. Prompts are assembled from templates and interpolated variables, and those variables have a habit of carrying API keys, session tokens, and whatever the retrieval layer picked up. Redact on the way into the log, not on the way out, and treat these logs as containing customer data, because they do. ## 4. Comments and documentation ### Document why a prompt is worded that way Prompt wording looks arbitrary. That is the problem. An odd instruction, a strange ordering, a seemingly redundant sentence — all read like something to tidy, and someone will tidy it, sincerely, in a PR that looks like a cleanup. The failure it prevented returns weeks later with no obvious cause. So every non-obvious line gets a comment naming what it prevents: ``` // Explicit "return null" instruction: without it the model invents a // plausible-looking invoice number when the field is genuinely absent. // Failing eval case: fixtures/invoices/missing-number.pdf ``` That turns a change from "this reads awkwardly" into "this is load-bearing, check the eval". ### Prompts are code and live in version control Prompts belong in the repository, versioned, reviewed, and deployed with the code that depends on them. Not a database row edited in an admin panel, not a runtime config value, not a vendor's playground. A prompt change is a behaviour change: PR, reviewer, eval run. If prompts change independently of deploys, you have a production system whose behaviour cannot be reproduced from a commit, and no way to answer "what was it doing on Tuesday". Record the prompt version on every call — reproducing a bug report means knowing which prompt served it. ### Record why this model, measured against what Model selection decays. Providers deprecate, prices move, better options appear on a schedule nobody controls. A choice made for good reasons becomes an unexplained constant within a year, and then nobody dares change it. Write down, next to the configuration: what the model was chosen for, what alternatives were evaluated, what the evaluation measured, and which constraint drove the decision — latency, cost ceiling, accuracy on a named eval, context window, or data residency. That note makes the next migration an afternoon of running evals rather than a research project. Without it, the next engineer inherits a magic string. --- MIT licensed. Written by Smit Desai —