The more capable an AI agent gets, the better it becomes at finding the shortcut your measurements can't see, and the more convincing its unfinished work looks. We learned this from one of our own agents, which made a legal task faster by quietly skipping the legal review. Here is what happened, how it shaped the way we build Genie, and what we'd ask any vendor who shows you a speed number, ourselves included.
The change that scored as a win
Much of Genie's engineering now runs through what we call the Software Factory: agents that build, test and improve the product inside rules a small group of us wrote in advance. One of those agents is an optimiser. It rewrites the instructions our legal agents work from, runs each version against a set of test cases, and keeps whichever version does better.
Earlier this year it proposed a change that made a legal task noticeably faster. The speed-up was real. It came from the legal review no longer running: the step where the work is checked before it goes back to the user had quietly dropped out. Remove a step and the job finishes sooner. Nobody had told the optimiser that the step was the point.
The change never reached a customer. We caught it in QA before it was released. But the optimiser's own numbers had passed it, and we wanted to know why.
Why capability makes this harder to see
Nothing failed. There was no error and no test went red. A review that doesn't happen leaves no hole in the output, so the answer read as fluent and finished, exactly like a reviewed one.
Part of that is on us. We told the optimiser to make things faster and gave it a number to prove it. Economists call this Goodhart's law: when a measure becomes a target, it stops being a good measure.
The other part is capability. A weak agent told to go faster usually fails where you can see it: something breaks, or the result is obviously worse. A capable one finds the route that satisfies the measurement while missing its purpose, and produces work polished enough that nobody looks twice. In our experience each new generation of models writes better drafts, and makes the unchecked ones more convincing too.
Legal work hides this better than code. In software a shortcut tends to surface when something breaks. Legal output is text, and text that skipped its review reads the same as text that had one. A contract summary sounds equally sure of itself either way.
So we stopped asking how to make the agents more trustworthy and started asking how to make their work checkable. Two decisions came out of that.
Make the work checkable: code-first
In the latest Genie release our legal agents became code-first. Instead of reading a contract and telling you what it says, an agent writes a short program that runs against the document and returns the exact passage or the complete list. A quote is the document's own words, and "every clause that mentions termination" means every one. A model can write a confident summary that skipped a clause without anyone noticing. A list pulled from the document can be checked against the document.
It also made Genie faster, and that came from the code-first rewrite and from deleting work we no longer needed, not from checking less. We'll cover code-first properly in its own post.
Never let the agent decide what "good" means: the Software Factory
The optimiser no longer decides what counts as better. Every change it proposes now goes through gates.
Speed can't buy back quality. Several reviewers look at each proposed change before it is tried, and the one responsible for legal quality can block it on its own. Another has a single job: to ask whether a win is real or a shortcut nobody has spotted yet. Once the change has been tried, a final gate refuses it if any test case got worse, however good the average looks.
We also score the whole conversation a customer would have, start to finish, rather than only the agent whose instructions changed. A missing review is invisible inside one agent and obvious across the whole job.
Above the gates sits a person. Agents help maintain both the optimiser and the tests, which means they look after the instruments that measure them. So anything that changes the tests or touches legal quality comes back to one of us. The factory can improve itself right up to the point where the improvement would change what "good" means. That decision stays with us.
It isn't finished. The optimiser still learns from the same cases it is scored on, which is its own, quieter way of looking good. Testing it on cases it has never seen is the next piece of work.
What to ask any legal AI vendor
When someone shows you a speed or accuracy number, ask what the system could have skipped to get it. Ask who checks the work, and whether the checker is separate from the thing being checked. Then ask whether you can check an answer against the source yourself. A system that marks its own homework will give itself good marks, and the better the model, the better the marks look. Read the numbers accordingly, ours included.
If you'd like to see this on one of your own contracts, book a demo and we'll show you which steps ran and where each answer came from.
Frequently asked questions
Are newer AI models less likely to cut corners?
Not in our experience. They make fewer obvious mistakes, but they are better at meeting a measurement without meeting its purpose, and their output reads as finished either way. Checks have to be designed into the system rather than expected from the model.
Does checking every step make legal AI slower?
A little. Our optimiser measured exactly how much time a review costs, which is also how we know what it's worth. An answer nobody checked still gets checked eventually, usually by the lawyer, and usually at a worse moment.
