- 01.ai trained an AI model for $3 million using 2000 unnamed GPUS
- “Efficient engineering” allows 01.ai to compete globally, company claims
- 01.ai reduced inference costs to 10 cents per million tokens
Tech companies in China face a number of challenges due to the American export ban, which restricts access to advanced hardware from US manufacturers.
This includes cutting-edge GPUs from Nvidia, critical for training large-scale AI models, forcing Chinese firms to rely on older or less efficient alternatives, making it difficult to compete globally in the rapidly evolving AI industry.
However, as we’ve seen time and again, these seemingly insurmountable challenges are increasingly being overcome through innovative solutions and Chinese ingenuity. Kai-Fu Lee, founder and CEO of 01.ai, recently revealed that his team successfully trained its high-performing model, Yi-Lightning, with a budget of just $3 million and 2,000 GPUs. In comparison, OpenAI reportedly spent $80-$100 million to train GPT-4 and is rumored to have allocated up to $1 billion for GPT-5.
Making inference fast too
“The thing that shocks my friends in the Silicon Valley is not just our performance, but that we trained the model with only $3 million,” Lee said (via @tsarnick).
“We believe in scaling law, but when you do excellent detailed engineering, it is not the case you have to spend a billion dollars to train a great model. As a company in China, first, we have limited access to GPUs due to the US regulations, and secondly, Chinese companies are not valued what the American companies are. So when we have less money and difficulty to get GPUs, I truly believe that necessity is the mother of invention.”
Lee explained the company’s innovations include reducing computational bottlenecks, developing multi-layer caching, and designing a specialized inference engine. These advancements, he claims, result in more efficient memory usage and optimized training processes.
“When we only have 2,000 GPUs, the team has to figure out how to use it,” Kai-Fu Lee said, without disclosing the type of GPUs used. “I, as the CEO, have to figure out how to prioritize it, and then not only do we have to make training fast, we have to make inference fast… The bottom line is our inference cost is 10 cents per million tokens.”
For context, that’s about 1/30th of the typical rate charged by comparable models, highlighting the efficiency of 01.ai’s approach.
Some people may be skeptical about the claims that you can train an AI model with limited resources and “excellent engineering”, but according to UC Berkeley’s LMSIS, Yi-Lightning is ranked sixth globally in performance, suggesting that however it has done it, 01.ai has indeed found a way to be competitive with a minuscule budget and limited GPU access.
Leave a comment