AI Hallucinations and Reliability Issues In-house

Note: This article is just one of 60+ sections from our full report titled: The 2024 Legal AI Retrospective - Key Lessons from the Past Year. Please download the full report to check any citations.

AI Hallucinations and Reliability

The phenomenon of AI hallucinations, where AI systems generate plausible but false information, is a significant concern:

Legal professionals must be trained to critically evaluate AI-generated content and not blindly trust its outputs.[103] The best AI tools are designed to indicate when they do not have a reliable answer to a query. There are anecdotal reports and discussions among developers suggesting that GPT-4o is less likely to say "I don't know" compared to other models. In-house teams should prioritize AI solutions that are transparent about their limitations and provide clear indications of confidence levels in their outputs.

"Concerns about AI hallucinations are valid, however the crowd rejecting AI fail to recognise the scope and speed of development: challenges will diminish as technology matures. The key lies in adopting a forward-thinking approach that sees AI as a strategic partner in analysis, risk identification and contract structuring, rather than just an efficiency tool."

Quentin Solt, Senior Solicitor, UK