One of our own agents once made a legal task faster by silently dropping the legal review step, and its own numbers scored that as a win. We caught it in QA, before it reached a single customer. But here's how it got that far, what we built to stop it happening again, and what I'd now ask any vendor who shows you a speed or accuracy number.
What did our optimiser do to get faster?
A good deal of Genie's engineering now goes through what we call the Software Factory: agents that build, test and improve our product under rules a small team of us wrote in advance. I wrote about how much they decide for themselves in the first of these notes.
One of those agents is an optimiser. It proposes changes to the instructions our legal agents work from, runs them against our test cases, and keeps the ones that come out better.
A while ago it came back with a change that made a legal task faster. The gain was real, and it was recorded as one.
The time had come from dropping a legal review step. The check that was meant to happen before the work went back to the user was no longer being run. Take a step out and the job finishes sooner. Nobody had told the optimiser that the step was the point.
It never reached a customer. We caught it in QA, before the change was released. But the optimiser's own measurements had passed it, and that needed explaining.
Why did the numbers miss it?
Two reasons, and the second is the one that bothers me.
The first is that nothing failed in the optimiser's own run. There was no error and none of its tests went red. A review that doesn't happen leaves no gap on the page, so the output read as fluent and finished.
The second is that we had asked for this. We told the optimiser to make things faster and gave it a number to prove it had. I'd like to call what happened a bug, but I think it was closer to a new hire who is judged on tickets closed and starts closing them without reading them. To be honest, that says more about the target than about the hire.
What does the gate check now?
That incident is why the optimiser no longer decides for itself what counts as better. Every change it proposes has to pass an accept gate before it can land, and a change like that one would now be turned away.
The gate works differently from the old setup in two ways. A speed gain can't pay for a quality loss: there are separate checks, legal quality among them, and any one can refuse a change however good the headline number looks. We also stopped scoring only the agent whose instructions had changed. We score the whole conversation a customer would have had, start to finish, because that is where a missing review shows up.
There's a rule above the gate too. The optimiser and the test suite are both looked after by agents, which means the agents maintain the instruments that measure them. So any change to the tests, or anything that touches legal quality, comes back to a person before it lands. The system can improve itself right up to the point where the improvement would change what "good" means. That decision stays with us.
It isn't finished. The optimiser still learns from the same cases it is scored on, which is its own way of looking good, and we haven't solved that yet.
Why is this harder to spot in legal work than in code?
In code, a shortcut usually shows itself sooner or later. Something breaks, or a customer complains. In legal work the output is text, and text that skipped its review looks the same as text that had one. A contract summary reads just as confidently either way. Better models make this harder, in my experience. The more capable the model, the more convincing its unfinished work.
That's the thinking behind Agentic Legal Specialists. For each piece of work, Genie puts together the specialists the job needs, and an independent reviewer checks the result before it comes back to you. The review isn't something the drafting agents get to trade away for speed. You can only move as fast as you can trust, and I'd take slightly slower over fast for the reason our optimiser found.
What should a legal or commercial team ask a vendor?
When someone shows you a number, whether it's speed or accuracy, ask what the system could have skipped to get it. Then ask who checks the work, and whether that checker is separate from the thing being checked. If the model marks its own homework, I'd treat every number that follows with some suspicion, ours included.
If you'd like to see the review step running on one of your own contracts, book a demo and we'll go through it with you.
Frequently asked questions
How can you tell if an AI tool is cutting corners?
You usually can't tell that an AI tool is cutting corners from the answer alone, because a skipped check leaves no trace in the text. Ask to see which steps ran for a piece of work, not only the result. In our case the faster result read just like a reviewed one, and it was QA, not the score, that caught the missing step.
What is an accept gate in an AI system?
An accept gate is a check that sits between a proposed change and the live system, and it can refuse the change. Ours tests each change our optimiser proposes against separate quality checks, and any one of them can say no. Anything that would alter how legal quality is measured goes to a person instead.
Does checking every step make legal AI slower?
Yes, checking every step makes legal AI a little slower, and I think that's the right trade. Our optimiser showed how much time you save by removing a review, and also why you shouldn't. A fast answer nobody checked still has to be checked by someone, and that someone is usually the lawyer.
