Note: This article is just one of 60+ sections from our full report titled: The 2024 Legal AI Retrospective - Key Lessons from the Past Year. Please download the full report to check any citations.
Transformer Architecture Dominance
77% of businesses using NLP expect to increase their investment in NLP.[64] The transformer architecture has continued to be the backbone of most state-of-the-art NLP models:
• Improved Attention Mechanisms: Researchers have developed more efficient attention mechanisms, allowing for processing of longer sequences and better handling of contextual information.[65]
• Sparse Transformers: These models have gained traction, offering comparable performance to dense transformers while requiring less computational resources.[66]
Alex Denne, Head of Growth (Open Source Law) at Genie AI, is a legal tech leader and serial founder with over a decade of experience driving innovation and making legal services more accessible. Since joining in 2021, he has scaled the platform from 200 to over 120,000 users, combining deep contract law expertise with a data-driven, open-source approach. He is passionate about democratizing legal knowledge through AI, backed by strong academic credentials and experience leading major product and innovation initiatives.
Alex Denne, Head of Growth (Open Source Law) at Genie AI, is a legal tech leader and serial founder with over a decade of experience driving innovation and making legal services more accessible. Since joining in 2021, he has scaled the platform from 200 to over 120,000 users, combining deep contract law expertise with a data-driven, open-source approach. He is passionate about democratizing legal knowledge through AI, backed by strong academic credentials and experience leading major product and innovation initiatives.
.png)
.png)
.png)
.png)
