Automating regulatory reporting

If producing a periodic report requires a person to open several systems, export to a spreadsheet, apply adjustments they hold in their head, and paste the result into a template, the problem is not the absence of a reporting tool. The problem is that the underlying data does not contain what the report asks for, and the gap is being closed by a human every period.

The tell is the adjustment column. Every manual adjustment is a piece of business logic living outside the system, applied inconsistently, and invisible to everyone except the person applying it.

The workable model is that there is one operating dataset, structured well enough that every report is a query over it. Different reports slice it differently, aggregate it differently and label it differently, but none of them introduces facts the operating data does not already contain.

Getting there requires the dimensions to exist in the data at the moment of the event rather than being inferred later. If reporting distinguishes payment types, customer categories, geographies, channels or scheme identifiers, each of those has to be recorded on the movement when it happens. Deriving a customer category eight months later from data that has since changed produces a number that cannot be reproduced, and irreproducible numbers are worse than late ones.

Regulatory figures are as-of statements. Outstanding electronic money as of the last day of the period. Customer counts as of a date. Exposure at a moment. Operating systems, left to themselves, store current state and overwrite it.

Two mechanisms solve this and they compose well. First, immutable event history for anything that moves, which the ledger already provides. Second, temporal records for anything that is a state rather than a movement: customer status, risk rating, product configuration, limits. A temporal record has a validity interval, so asking what a customer risk rating was in March is a query rather than an archaeology project.

Teams often add temporality to the ledger and forget it for reference data, then discover that a report cannot be reproduced because a customer was reclassified in July and the classification table has one row per customer.

A report needs the same treatment as any other regulated output. Reconciliation between the report and the source, checks that totals across related reports agree, comparison against the previous period with an explanation required above a threshold, and a record of who approved the submission.

Period on period variance analysis is the control that earns its keep fastest. Most reporting errors are visible as an implausible jump, and catching them before submission is the difference between a correction and a restatement.

When a figure is questioned, the useful response is to walk from the reported number down to the individual movements that compose it. That requires the pipeline to keep the link rather than aggregate and forget.

In practice: store the intermediate result set for each submitted figure, or at minimum store the exact query and the parameters, together with a hash of the input population. Then the walk down is possible even after the underlying tables have grown by a year.

The saving is not the hours of assembly, although those are real. It is the disappearance of a category of risk in which a figure submitted to a supervisor was produced by a process nobody can describe.

The secondary effect is that the same dataset answers management questions and supervisory questions with the same numbers. When the board pack and the regulatory return disagree, the conversation that follows is never about arithmetic.