Building AI inside regulated companies

Almost every early win sits in the first two. Finding the three clauses in a two hundred page contract that bear on a question, drafting the first version of a procedure from the rules that govern it, summarising a case file into the structure an analyst needs: all of these compress work without moving the decision.

Moving to the third mode is a different project. It requires the output to be constrained to a closed set, the reasoning to be reconstructible, the error modes to be measured on a real population, and a defined path for the affected person to contest the result.

In a regulated setting an unsourced answer is unusable, however good it is, because nobody can verify it and nobody can be accountable for it. Retrieval matters less for accuracy than for attribution: it turns an assertion into a claim with a reference.

That changes what to optimise. A system that answers eight questions out of ten with an exact citation and declines the other two is more useful than one that answers all ten fluently without references, because the second one transfers the verification cost to the reader without telling them.

Design the refusal path deliberately. "I could not find this in the corpus" is a correct and valuable answer, and models will not produce it unless the system makes it easy and the evaluation rewards it.

Public benchmarks say nothing about how a model performs on your contracts, your alerts, your customers and your vocabulary. The only evaluation that supports a decision to deploy is one built on a set of your own cases with known correct answers, assembled by the people who do the work today.

Two hundred well chosen cases beat twenty thousand scraped ones. Include the hard cases and the ambiguous ones, because those are where the model will fail in a way that matters. Keep the set versioned and rerun it on every change to the prompt, the model, the retrieval corpus or the chunking, because all four move performance and only one of them is usually noticed.

For any output that influences a regulated process, keep the inputs, the retrieved context, the model and version, the parameters, the output, and the human disposition. Nothing about this is exotic observability. It is the ordinary evidence requirement that applies to any control, applied to a component that happens to be probabilistic.

Model versions change under you when you use a hosted service. Pin what you can, record what you cannot, and treat an unannounced change in behaviour as an incident rather than as weather.

Where the data may go, who processes it, on what legal basis, under which contractual terms, and whether it may be used to improve someone else model. These questions decide the architecture more often than latency or cost, and they are answerable early.

The related habit worth installing: minimise what leaves. Most document work does not need whole documents in a prompt. Retrieval that sends three relevant paragraphs instead of forty pages is better on cost, better on accuracy, and better on every data protection question that follows.

Accuracy is rarely what kills these projects. The tool sits beside the workflow instead of inside it, so using it becomes an extra step that a busy person skips. Close behind: nobody agreed who owns the output when it turns out to be wrong.

Both are solvable before a line of model code. Put the assistance where the work already happens, and write down who is accountable for the result. Then the technical questions become the easy part.