Chinese AI models tumble down rankings after a crucial change to testing - Business Insider
By Google News AI ·
Chinese AI models have fallen sharply behind American rivals in a closely watched ranking, raising fresh questions about whether the threat from cheaper, freely available AI technology has been exaggerated. Artificial Analysis, a prominent independent firm that measures AI performance, recently overhauled its testing system to better reflect how people actually use the technology. One crucial change: Many of the tests are now kept secret, making it harder for AI companies to train their models specifically to perform well on them. The results were striking. Artificial Analysis said the best so-called open-weight AI models had trailed the overall leader by just three points under the previous system. Under the new system, that gap widened to 12 points. The 12-point gap included Moonshot's Kimi K3.1 Pro. Among models with publicly released weights, the leaders scored just 32 points, compared with 58 for the then overall leader, Anthropic's Claude Opus 5.5. The findings could have major implications for the AI industry and the billions of dollars investors are pouring into it. Chinese companies have been releasing powerful AI models that other businesses can download and adapt, often at lower costs than American alternatives. This fueled concern that companies such as OpenAI and Anthropic could struggle to charge premium prices for their proprietary technology. The new rankings suggest that threat may have been overstated, potentially strengthening the investment case for Anthropic's planned IPO and a possible future OpenAI listing. Sridhar Ramaswamy, CEO of Snowflake and a former Google executive, highlighted the significance of the testing change. "The day benchmarks went private, the open-model gap went from 3 points to 12," he wrote on X. "Were open models catching…
Originally published by Google News AI