Intelligent LLM selection, GB charging, and efficient management of LLMs drastically reduce the cost of AI

Innovative engineering reduces the cost of data analysis and review by a factor of ten

[Portland, OR] – Enterprise legal teams face an increasing burden of manual work that delays company decision-making, increases the cost of litigation, and impacts the outcomes of legal cases.
Generative AI technology can reduce this burden by quickly and efficiently identifying and assessing the unstructured data legal teams need from sources like Slack, Microsoft Teams, documents, and emails, but the use of the technology has been cost-prohibitive.

Intelligent LLM orchestration

Rather than using API calls to extremely large language models like Chat-GPT, the company has engineered Spotlight AI to work with smaller models hosted on secure private cloud instances for each customer. This approach ensures optimum performance by orchestrating optimal and cost-effective models to achieve the required results.

GB charging model

Most generative AI systems charge based on the number of ‘Tokens’ created during data processing. Tokens are numerical representations of letters, characters, phrases, or sentences that allow neural networks to process human language.

The pricing model based on Tokens can be particularly costly in the legal industry where data discovery use cases require vast numbers of documents to be processed. Using generic LLMs like ChatGPT 4, which haven’t been optimized for the task, can make AI cost-prohibitive in the legal domain.

Due to its unique approach based on optimal orchestration of smaller LLM, Hanzo can provide a price that is more understandable (charging by GB, not token) and significantly (10 to 20x times) lower cost, which makes adoption of this exciting technology feasible.

More efficient management of models

Spotlight AI is designed so that a legal team’s private instance of the Large Language Model only operates when that team needs to search for data and is fully powered down when they don’t. This efficient management of the technology better aligns cost to a company’s usage and the value derived from AI.

“Generative AI is going to be transformational for legal and compliance functions but the cost is a barrier today. Hanzo’s work to engineer the cost out of this technology opens the door to a number of new use cases for Gen-AI in legal departments, from data discovery to litigation. We also see potential for the technology in compliance, with the ability to proactively identify risks.” said AVP of Information Governance and Discovery at a F100 media company.

Julien Masanès, CEO, Hanzo said: “Pricing of AI tools has been opaque, and frankly that’s limited the adoption of the technology across the profession.
Thanks to innovative and problem-oriented engineering, Spotlight AI can process data at a cost of $99 per GB, which is ten times more cost-effective than current industry norms. Using Generative AI for eDiscovery helps legal teams reduce data discovery time by 90%.

[View source.]