Trellis Data client success consultants Kate Schearer and Tom Booth. Photo: Trellis Data.
While the environmental impact of AI has been an ongoing concern, organisations like Trellis Data have been looking for solutions – recently finding one in large language model (LLM) decoders that are over three times faster.
Trellis Data has created the world’s fastest decoder, named the Dynamic Depth Decoding (D3) technique. Using speculative decoding, it achieves an average speed increase of 44 per cent compared with the previous fastest decoder.
Used on the architecture behind AI systems – LLMs – the D3 technique allows it to generate text approximately 3.16 times faster than standard decoding without limiting accuracy.
Given the World Economic Forum (WEF) estimates the computational power required to sustain global growth in AI is doubling every 100 days, Trellis Data CEO Michael Gately said, “speed is a key area” to make gains.
“There are three components critical to the success of any AI system: trust, speed and performance,” he noted.
“D3 enables us to address one of the key bottlenecks of speed – the decoder – offering customers a reduction in the cost of running AI servers and a lower carbon footprint.”
According to Trellis Data research submitted for global peer review, the improvement in speed delivered by D3 translates to an estimated 68.4 per cent reduction in computational power requirements, which in turn lowers the costs and carbon emissions associated with running LLMs.
CEO Michael Gately was recently a finalist in the Artificial Intelligence category of the 2024 AmCham Alliance Awards, and also spoke at MilCIS2024 in Canberra.
“Trellis Data is at the forefront of the rapidly developing AI field and we’re excited to make D3 available on our platform,” said Mr Gately.
“D3 will provide customers using our speech management and knowledge management capabilities immediate benefits, including an enhanced user experience and greater ability to scale.”
Trellis Data, which is headquartered in Canberra and has offices in Sydney, Adelaide and Arlington (Virginia, USA), says D3 is broadly applicable to generative AI use cases, including chatbots, text summarisation, translation, transcription and content generation.
With a diverse range of customers in government, defence, law enforcement and other high-security sectors, the company is also planning to expand into the Middle East and South East Asia.
Additionally, it boasts its reduced costs and carbon emissions as a competitive edge over the competition while LLM’s size and usage grow.
D3 is Trellis Data’s latest development in their strategy to address the concerns regarding the adoption of new AI technologies.
Original Article published by James Day on PS News.
Loading ...
