Meta Platform has this week unveiled the second generation of its custom AI processor, amid a slew of AI chip announcements from other tech firms.

Meta in blog post announced version 2 of its Meta Training and Inference Accelerator (MTIA), after it had released the first version back in May 2023.

Meta said the next generation of it’s large-scale infrastructure is being built with AI in mind, including supporting new generative AI (GenAI) products and services, recommendation systems, and advanced AI research.

Meta Training and Inference Accelerator (MTIA), Version 2.
Image credit Meta Platforms

MTIA v2

MTIA v1 was its first-generation AI inference accelerator that was designed in-house with Meta’s AI workloads in mind, namely its deep learning ranking and recommendation models.

“MTIA is a long-term venture to provide the most efficient architecture for Meta’s unique workloads,” it said. “MTIA v1 was an important step in improving the compute efficiency of our infrastructure and better supporting our software developers as they build AI models that will facilitate new and better user experiences.”

But now MTIA v2 (codenamed Artemis) has been designed to work with its current technology infrastructure, but also support and future AI advancements.

According to Meta, the architecture of the new generation MTIA is “fundamentally focused on providing the right balance of compute, memory bandwidth, and memory capacity for serving ranking and recommendation models.”

Meta is obviously hoping MTIA v2 will help lessen its reliance on Nvida’s AI chips and reduce its energy costs overall.

Nvidia is estimated to have a 80 percent share of the AI chip market, it’s H100 and A100 chips serve as a generalised, all-purpose AI processor for many AI customers.

But Nvidia’s AI chips are not cheap. While Nvidia does not disclose H100 prices, each chip can reportedly sell from $16,000 to $100,000 depending on the volume purchased and other factors.

Meta apparently plans to bring its total stock to 350,000 H100s chips in 2024, demonstrating the hefty financial investment required to compete in this sector, and the value of organisation supplying these semiconductors.

Under the hood

Meta said its new accelerator consists of an 8×8 grid of processing elements (PEs).

“These PEs provide significantly increased dense compute performance (3.5x over MTIA v1) and sparse compute performance (7x improvement),” said the firm. “This comes partly from improvements in the architecture associated with pipelining of sparse compute.”

“To support the next-generation silicon we have developed a large, rack-based system that holds up to 72 accelerators,” said Meta. “This consists of three chassis, each containing 12 boards that house two accelerators each. We specifically designed the system so that we could clock the chip at 1.35GHz (up from 800 MHz) and run it at 90 watts compared to 25 watts for our first-generation design.”

Meta said this design ensures it can provide denser capabilities with higher compute, memory bandwidth, and memory capacity.

This density allows Meta to more easily accommodate a broad range of model complexities and sizes.

Tom Jowitt

Tom Jowitt is a leading British tech freelancer and long standing contributor to Silicon UK. He is also a bit of a Lord of the Rings nut...

Recent Posts

UK AI Safety Institute To Open Office In US

Seeking collaboration on AI regulation, UK's AI Safety Institute to cross Atlantic and will open…

42 mins ago

Silicon In Focus Podcast: Does Security Block Innovation?

Explore the dynamic intersection of technology and security with Silicon In Focus Podcast: Does Security…

1 hour ago

EU Widens Investigations Into Chinese Imports, Subsidies

After the United States imposes 100 percent tariffs on certain Chinese goods, Europe widens its…

3 days ago

Reddit Deal With OpenAI Gives ChatGPT Access To Content

OpenAI strikes deal with Reddit to train its AI tech on user posts and give…

3 days ago

Microsoft Invests 4 Billion Euros In France For AI, Cloud

Global spending spree from Microsoft continues, with huge investment for new data centre to drive…

3 days ago

Toshiba Axes 4,000 Staff In Post-Delisting Restructuring Operation

Workforce blow. Newly privatised Toshiba has embarked on a 'revitalisation plan' that will entail the…

4 days ago