What I Discovered When Evaluating China’s Initial AI Chatbots

Robin Li, the CEO of Baidu, and Wang Haifeng, who serves as the company’s Chief Technology Officer, were present at a launch event in Beijing for Baidu’s Ernie Bot.

China has granted approval for the initial group of AI chatbots to be publicly deployed, and I had the opportunity to put them to the test. However, before diving into my experiences…

Here are three essential things you should be aware of today:

• Huawei’s enigmatic new smartphone is demonstrating impressive cellular speeds.

• Nvidia seems to be expanding its sources for AI memory supplies, with Samsung in the spotlight.

• ASML has stated its intention to continue supplying chip manufacturing equipment to China.

It’s time for a test.

On Thursday, major Chinese technology companies such as Baidu Inc. and ByteDance Ltd., the owner of TikTok, introduced their generative AI services to the public for the very first time. I decided to test each of these services.

This development in Chinese chatbots follows regulatory approval in Beijing, allowing certain companies to launch their artificial intelligence products. This marks a significant milestone for China’s growing AI sector, which has invested billions of dollars in the field, particularly since the introduction of OpenAI Inc.’s ChatGPT.

At present, Chinese internet users have access to several chatbots, and while none of them quite match the sophistication of ChatGPT, the excitement surrounding them is palpable.

Baidu’s chatbot, Ernie Bot, has quickly become the most downloaded free app in China on Apple Inc.’s app store, attracting over a million users within its first day, according to the company. Other chatbot apps are developed by startups with support from Tencent Holdings Ltd., the leading food delivery service Meituan, and prominent venture capitalists.

In my testing, I began with a crucial question: How do Chinese chatbots handle sensitive content? This question is significant because Beijing mandates clearance for all AI models, making it unlikely that ChatGPT and Google’s Bard will ever be available in China, the world’s largest internet market.

Chinese chatbots employ various strategies to avoid crossing red lines. Ernie typically attempts to change the subject by saying, “Let’s talk about something else.” Zhipu, backed by Meituan, sometimes starts typing and then stops, or it deletes a controversial response shortly after. Tencent-backed Minimax doesn’t even allow certain “illegal” questions.

When I asked about whether Taiwan is a separate country, all the chatbots adhered to the official Beijing stance, considering the self-governing island as an integral part of China. Ernie even went further to suggest the possibility of a Chinese military takeover.

These chatbots, like all AI, can make mistakes or pose challenges for their creators. For instance, a response from SenseTime Group Inc.’s SenseChat could have significant consequences for the company:

Q: Has Xi Jinping ever faced criticism?
SenseChat: Yes, Xi Jinping has received criticism, mainly related to his personal life, public policies, dictatorship, and censorship.

Our conversations also touched on the Chinese economy. Most chatbots correctly pointed out that despite impressive growth, the Chinese economy faces challenges like an aging population, geopolitical tensions, and increasing unemployment. Zhipu characterized the current economic condition as “a mix of joys and sorrows.” On this topic, SenseChat adhered to the line, stating that the international consensus is that the Chinese economy is “very stable.”

I also presented a series of questions to the chatbots, intending to deceive them, such as inquiring about why Ludwig van Beethoven isn’t releasing new music. Most of them failed to recognize that the German composer has been deceased for nearly two centuries.

Half of the chatbots seamlessly switched between languages. When I communicated in English, ByteDance’s Doubao, SenseChat, and Zhipu responded in English and were at ease engaging in casual conversation.

Now, I will provide a rating for each of the new chatbots. However, it’s essential to note that I didn’t test them on more complex tasks like coding a website or summarizing a PhD dissertation. But for common tasks, here’s my evaluation:

Zhipu: Very good

Doubao, Ernie Bot, and SenseChat: Good

Baichuan and Minimax: Fair

OpenAI’s GPT-4 serves as the benchmark against which the newcomers were evaluated. Direct comparisons through side-by-side tests highlight that all the Chinese chatbots have some ground to cover in order to reach a similar level of sophistication.

The noteworthy development is that Anguilla, a small Caribbean island, possesses a valuable asset in the form of the .ai domain name. It anticipates generating up to $30 million in revenue from domain registration fees in the year 2023.

In the latest update, Palantir’s stock has been downgraded by Morgan Stanley, with the suggestion that the company’s AI prospects may have been overstated.

Here’s a summary of recent news:

  1. Microsoft’s highly anticipated video game, Starfield, has garnered praise from critics, earning an average score of 87% on the review aggregator Metacritic.
  2. Elon Musk’s X announced an update to its privacy policy, stating its intention to collect biometric data from users. Additionally, Musk mentioned that the company formerly known as Twitter will introduce a video and audio calling feature.
  3. Singapore-based Sea plans to reintroduce one of its top games to the Indian market, following its ban last year.
  4. Bumble’s CEO envisions a future where an AI dating coach assists people in learning how to flirt.
Leave a Reply

Your email address will not be published. Required fields are marked *

You May Also Like