Baidu Says Ernie AI Beats ChatGPT In Key Metrics

Chinese search giant Baidu has said the latest iteration of its AI foundation model Ernie has surpassed OpenAI’s ChatGPT in several key metrics, amidst intense international competition around the technology.

Baidu was the first major Chinese tech company to launch a ChatGPT-style chatbot with its Ernie Bot in March, based on its foundation model Ernie 3.0.

A new iteration of the foundation model, Ernie 3.5, surpassed ChatGPT’s GPT-3.5 model in comprehensive ability scores during beta testing, Baidu said, citing a study by state newspaper China Science Daily.

The paper’s tests, based on AGIEval and C-Eval, two benchmarks for evaluating the performance of AI models, found that Ernie 3.5 surpassed OpenAI’s latest GPT-4 in several Chinese-language capabilities.

Plugin architecture

Baidu said the latest Ernie 3.5 model improves on its predecessor in areas such as creative writing, Q&A, reasoning and code generation.

The model also makes improvements in training performance and inference performance, with a two-fold training throughput improvement and 17-fold inference throughput increase, areas that promise faster and less costly future iterations, while resulting in an improved user experience, Baidu said.

The new Ernie 3.5 model introduces a plugin architecture, allowing the addition of specialised capabilities such as accessing up-to-date information via Baidu Search or summarising long texts or Q&A documents through a plugin called ChatFile.

Baidu said it plans to open up the plugin architecture to third parties, allowing them to essentially build Ernie-powered applications. ChatGPT added plugin support in March.

International competition

Ernie Bot has been available since March to beta testers on an invitation-only basis, with the offering being initially powered by Ernie 3.0.

Baidu said the latest iteration of Ernie Bot, version 2.1.0, released on 21 June, adds the ChatFile plugin and improved mathematical computation and creative writing features powered by Ernie 3.5.

Chinese tech giants Alibaba and Tencent have also introduced generative AI models, while US tech firms including Google and Amazon are pushing similar technologies.

Since the launch of ChatGPT last November AI has emerged as a new flashpoint for competition between the US and China, with a forthcoming US executive order expected to place restrictions on how domestic companies can invest in Chinese AI development.

The US has already barred Nvidia from selling its most advanced AI accelerator chips to Chinese companies.