Baidu Ranks Top In Chinese ChatGPT-Style Chatbot Tests

Baidu’s Ernie Bot ranked highest in Xinhua Institute tests of chatbots that have been made available in recent months by Chinese tech firms seeking to compete with ChatGPT.

Xinhua Institute, a think tank affiliated with the Xinhua news agency, said its tests found Ernie Bot led Alibaba’s Tongyi Qianwen, SparkDesk from voice recognition firm iFlytek, and SenseChat from image recognition company SenseTime.

But the Baidu chatbot still trailed Microsoft-backed OpenAI’s latest GPT-4 and GPT-3.5, which powered the version of ChatGPT released to the public last November.

The researchers tested general capabilities such as basic language skills, logical reasoning and subject knowledge in fields such as mathematics, physics, finance and literature.

Baidu chief executive Robin Li speaks at the World Artificial Intelligence Conference in July 2021. Image credit: Baidu

Competition

They also tested the bots’ ability to improve productivity in fields such as journalism, painting, design, marketing, law and research.

ChatGPT captured the public’s imagination on its release last year, quickly gaining more than 100 million users, and competitors including Google and Amazon quickly announced their own versions of the technology.

In China, where ChatGPT is not officially available, Ernie Bot became the first OpenAI competitor last March.

But Baidu suffered an backlash as investors reacted to what they felt was a disappointing prerecorded demonstration of the chatbot’s capabilities.

‘Subjectivity’

Baidu’s shares have experienced a surge since the end of last month, when the company announced it would soon launch a new version of the large language model (LLM) powering Ernie Bot.

But they are still trading about 20 percent below their peak in early February, when investors were hotly awaiting the company’s initial chatbot announcement.

A different test of chatbots by Clue, a Chinese website that tracks AI research, found Smart Brain from cybersecurity firm Qihoo 360 was the best performer, followed by SparkDesk.

Xinhua Institute acknowledged in its report that tests were subject to time and conditional constraints that may result in “a certain degree of subjectivity”.

Matthew Broersma

Matt Broersma is a long standing tech freelance, who has worked for Ziff-Davis, ZDnet and other leading publications

Recent Posts

Google Must Face Trial In Ad Tech Monopoly Case

Google loses bid for summary judgement as judge says 'too many facts in dispute' as…

11 hours ago

Silicon In Focus Podcast: Feeding the Machine

Learn how your business can meet the challenges associated with managing data across multiple platforms…

11 hours ago

Apple, Meta Likely To Face EU Antitrust Charges

Apple, Facebook parent Meta reportedly likely to face EU antitrust charges before August under new…

11 hours ago

Adobe Shares Jump On AI Success

Adobe shares post biggest gains in more than four years after it reports user take-up…

12 hours ago

Winklevoss’ Gemini To Pay $50m In Crypto Fraud Settlement

Winklevoss twins' Gemini Trust to pay $50m to settle cypto fraud claims over failed Gemini…

12 hours ago

Meta Delays EU AI Launch After Privacy Complaints

Meta delays Europe launch of AI in Europe after user, privacy group complaints over plans…

13 hours ago