the 50 TOPS of your NPU

AMD has debuted at the Computex 2024 fair with two big launches. The first, that of a new twist on its “traditional” processors, which are renewed with some promising AMD Ryzen 9000.

The second, even more striking, that of the new AMD Ryzen AI 300, which represent AMD’s first big bet for this new era of PCs with artificial intelligence. And pay attention, because here the firm has stepped up and promises to knock down all its rivals.

The AMD Ryzen AI 300 consist of three major components:

1. CPU. With Zen 5 architecture, the main processor of these processors will consist of up to 12 cores (4 Zen 5, 8 Zen 5C efficiency) and 24 threads, up to 5.1 GHz clock frequency and up to 36 MB cache. It is true that in this section the power is not what the best models in the Ryzen 9000 series offer—more geared toward gamers—but there will still be more than enough room for maneuver for most users.
2. GPU. With RDNA 3.5 architecture, the integrated graphics can be either a Radeon 880M (12 CU) or a Radeon 890M (16 CU), an evolution of a component that will allow you to enjoy video games with more than decent performance even if we are not talking about gaming equipment as such.
3. NPU. The real protagonists of these chips will be the neural processing units that have XDNA 2 architecture. We are facing an enormous leap since the first generation of these components: if the XDNA reached 10 TOPS in performance, these chips reach 50 TOPS.

We are therefore faced with proposals that will compete directly with the Qualcomm Snapdragon

Prepared for the era of Copilot+ PCs

These promised performances for their NPUs make it clear that these chips are especially prepared for a new batch of Copilot+ PCs in which AMD will offer all the power of the new artificial intelligence options of Windows 11.

Thus, it will be possible to take full advantage of functions such as Recall, translation or generation of subtitles in real time, or the creative options of Cocreator. The leap from the first and second generations of AMD chips with NPUs is notable, and gives rise to the use of new AI experiences, and above all to those experiences running directly locally, without connection to the cloud.

AMD has emphasized that the power of its XDNA2 NPU allows the response in the field of AI applications to be much higher than that of previous chips. When running Llama 2 7B, for example, the time it takes to generate the first token is five times less on the new processors than on the AMD Ryzen 8940HS, which had a first-generation NPU.

It is also much superior according to the firm than what the Intel Core Ultra 9 185H, Meteor Lake processors can offer in other areas, but which are not focused on AI. When Intel presents its future Lunar Lake, yes, the comparison will be much more interesting.

At AMD they also highlighted that in AI applications precision is critical. In operations with 8-bit integers there is more performance but lower precision, while in 16-bit floating point operations the precision is much higher but the performance drops.

Precisely with these chips AMD presents the first FP16 “Block” NPU, in which a mix of both characteristics is achieved. Given that most AI applications make use of 16 bts operations, AMD proposes this technology that makes the quantization of the models unnecessary.

Quantization is a technique used to reduce the precision of numerical values ​​(weights and activations) in an AI model. The main goal of quantization is to reduce the memory and computational requirements of the model, allowing for faster inference and deployment on devices with somewhat more limited resources, such as our PCs, laptops or mobile phones.

AMD boasts of crushing its rivals in performance. We will see the efficiency

AMD’s internal tests highlight that its chips are superior to Qualcomm’s Snapdragon

They also crush the Apple M3: 9% more performance, 70% more multitasking and 98% (almost double) more performance in 3D rendering. History repeats itself when comparing them with the Intel Core Ultra 185H: AMD boasts of offering up to 36% more performance for gamers who choose these chips.

These internal tests are certainly promising, but at the moment there is no data on the consumption and energy efficiency of these chips, a section in which Both Apple and Qualcomm are likely to overtake AMD. We will have to wait until we have this equipment to confirm both these internal tests and to be able to talk about the consumption and efficiency of each.

We will begin to see more than 100 computers based on the Ryzen AI 300 starting in July 2024. Acer, Dell, HP or Lenovo are among those protagonists, but also some of them such as ASUS or MSI will present their news at Computex based on these promising chips.

They will thus join the numerous proposals based on Qualcomm’s Snapdragon

In Xataka | Qualcomm is just the first step of Windows 11 with Copilot+. Intel and AMD chips wait in the rearguard