AMD's Ryzen AI CPUs outperform Intel's Core Ultra chips in new AI benchmarks showing LLM and GenAI workloads.
AMD shares new AI (LLM) benchmark with Ryzen AI CPUs, outperforming Intel Core Ultra chips while running on much lower power
AMD first entered the AI PC space with its first-generation Ryzen AI CPUs, codenamed “Phoenix,” released last year. The company then launched a revamped Ryzen AI lineup known as Hawk Point that offers enhanced “XDNA” NPUs, resulting in a 60% increase in AI TOPS. AMD appears to be putting a lot of effort into optimizing its software for client-side and localized AI workloads, as evidenced by new benchmarks published by the company.
In the new tests, AMD is focusing on running LLM locally on the CPU, which is possible on various models such as LLama 2, Mistral AI_, code llama, and RAG. Running localized AI models on your PC provides more privacy than models on online cloud platforms, doesn't require an online connection, and saves on subscription fees. The company is moving further into this space with a recent guide on how to set up your own local AI chatbot to rival his NVIDIA Chat with RTX Chatbot.
In performance testing, AMD used a Ryzen 7 7840U APU at 15W and compared it to an Intel Core Ultra 7 155H at 28W. Both chips work with 16GB of LPDDR5-6400 memory and the latest driver packages.
First up is Mistral Instruct 7B LLM. The AMD Ryzen 7 7840U CPU completes AI processing in just 61% of the time compared to Intel products, but Llama 2 7B Chat is even faster as the Ryzen AI chip completes the task in 54 hours. % probability.
To simplify things, AMD Ryzen 7 7840U (15W) CPU delivers up to 14% faster performance on LLama v2 Chat 7B (Q4 KM) and 17% faster performance on Mistral Instruct 7b (Q4 KM) To do. Time to first token is 79% faster with LLama v2 Chat and 41% faster with Mistral Instruct 7B LLM.
There are two things to note here. AMD Ryzen AI CPUs are not only faster than higher wattage Intel Core Ultra CPUs, but they also have slightly slower NPUs at 10 TOPs compared to Intel's Core Ultra chips, which have an SKU of 11 TOPs. is. This is not compared to the 28W SKU or the Hawk Point chips with up to 16 TOP of AI compute. AMD is also touting the cost advantage of its AI PC platform, which comes at a lower price point starting at approximately $899 (SEP) compared to $999 (SEP) for the Intel Core Ultra 155H chip.
The AMD Ryzen AI PC platform has the advantage of having been on the market for some time, leading the Red Team to make rapid AI optimizations for existing and next-generation CPUs. These AI workloads are expected to be further tuned in the future in preparation for the Zen 5-powered Strix Point family scheduled to launch later this year. Intel's Core Ultra platform has also been released for a few months now, and the company is rolling out a bold AI strategy for this space.
