Nvidia Taylor Swift is a technology company.
the AI customers shell out $40,000 for advanced chips And sometimes they wait months for the company's technology to arrive, remaining loyal even as competing alternatives emerge. That dedication comes from his Nvidia presence. largest AI chip manufacturer game in town. But there are also big technical reasons why users keep coming back.
For example, it is not easy to swap one chip for another. Companies build AI products to the specifications of those chips. Switching to another option means going back and reconfiguring the AI model, which is time-consuming and expensive. It is also not easy to mix and match different types of chips. And it's not just the hardware. His Nvidia software CUDA, which controls AI chips known as GPUs, works very well, said Ray Wang, his CEO at Silicon Valley-based Constellation Research. Wang said it will help strengthen NVIDIA's market power.
“It's not like there's a lock-in,” he said. “There's just no one taking the time to say, 'Let's make something better.'”
That may be changing. Over the past two weeks Tech companies are starting to come to Nvidia's lunchwith parents on Facebook meta, Google parent alphabet, and AMD Reveal all new or updated chips.Others, including: microsoft and Amazon We also recently announced our own chip products.
meanwhile Nvidia is unlikely to fall from grace any time soonthese and other efforts may threaten the Company's interests. Market share estimated at 80% They can either exploit some of the chipmakers' weaknesses, take advantage of the changing ecosystem, or both.
Different chips are suitable for different AI tasks, but switching between different options can be a headache for developers. That can also be difficult when there are different models of the same chip, Wang said. Building software that works well across different chips creates new opportunities for competitors, he says. Points to one API As a startup already working on such a product.
“People are going to understand that sometimes they need a CPU, sometimes they need a GPU, sometimes they need a TPU, and they're going to have a system that actually accounts for all three of those things.” said Wang, referring to the central processing unit and graphics. He has three types of AI chips: processing unit, tensor processing unit.
2011, Venture Capitalist Marc Andreessen famously declared that software is eating the world.. This will remain true in 2024 when it comes to AI chips, which are increasingly driven by software innovation. Jonathan Rosenfeld, who leads his FundamentalAI group at MIT FutureTech, said the AI chip market is undergoing a familiar shift with echoes in the telecommunications sector, as enterprise customers move from dependence on multiple hardware components to integrated software solutions. He said he was experiencing it.
“When you look at the actual progress in hardware, it's not due to Moore's Law or anything like that, even remotely,” said the co-founder and CTO of an AI healthcare startup. Mr. Rosenfeld says: somite eye.
This evolution points to a future where software will play a key role in optimizing across different hardware platforms, reducing dependence on a single provider. Nvidia's CUDA is a powerful tool at the single-chip level, but the move to a software-dependent environment required for very large models spanning many GPUs may not necessarily benefit the company. not.
“We're likely to see more consolidation,” Rosenfeld said. “There are a lot of entrants and a lot of money, so there can definitely be a lot of optimization happening.”
Rosenfeld doesn't see a future without Nvidia as a major force in training AI models like ChatGPT. Training is how an AI model learns how to perform a task, and inference is how it uses that knowledge to perform an action, such as responding to a question a user asks the chatbot. The compute needs for these two steps are clear, and while Nvidia is well suited for the training part of the equation, its GPUs are not well set up for inference.
Nevertheless, the reasoning was explained The company's data center revenue is estimated at 40% Over the past year, Nvidia has said: Latest earnings report.
“Frankly, they're good at training,” says Jonathan Ross, CEO and founder of AI chip startup Groq. “You can’t build something that’s better at both things.”
Training is where you spend your money, Ross said, and inference is where you make your money. But companies can be surprised when AI models go into production and consume more computing power than expected, squeezing profits.
Additionally, the GPU, the main chip that Nvidia makes, can't output answers to chatbots all that quickly. The developer said he doesn't notice any lag or lag during the month-long training, but people who use chatbots want them to respond as quickly as possible.
So Ross, who previously worked on chip development at Google, launched Groq to build chips called language processing units (LPUs) built specifically for inference.Third party testing by artificial analysis We found that ChatGPT can run more than 13 times faster using Groq's chips.
Ross doesn't see Nvidia as a competitor, but he jokes that customers often buy chips from Groq and then move to the front of the line to get Nvidia chips. He sees them as cosmic colleagues who are training while Groq is doing his reasoning. In fact, Ross said Groq could help him sell more chips for Nvidia.
“The more people that end up starting to make money with inference, the more people are going to have to spend on training,” he says.