AI processors: Computers with an AI kick: Well-invested money or unnecessary?

When taking a photo with a smartphone today, an optimized result is often automatically produced: faces appear clearer, colors more vibrant, and distracting objects disappear on request. Similar to live translations or voice assistants, much of this happens in the background and is often calculated by special AI components, so-called Neural Processing Units (NPUs).

This technology is now also increasingly being used in classical computers and laptops. The idea behind it: a local AI boost. Artificial intelligence is supposed to run directly on the device, react quickly, and consume as little energy as possible.

The NPU: The special processor for AI tasks

Technically, the NPU is not a standalone chip like a graphics card, but rather a part of the processor. "These are additional circuits in the processor that are firmly integrated," says Christian Hirsch from the IT specialist magazine "c't". While the CPU (Central Processing Unit) acts as a versatile main processor that handles almost all tasks, and the GPU (Graphics Processing Unit) acts as a graphics processor primarily responsible for image calculations, the NPU focuses on AI applications.

Into the Matrix: Efficiency through simplified calculation

It is designed to execute typical arithmetic operations of neural networks particularly efficiently. "What NPUs are super at is matrix multiplications and additions," explains Jörg Geiger from the computer magazine "Chip". "This is needed for neural networks, i.e., the technical background of AI." A crucial difference lies in the working method: NPUs often work with lower computing accuracy, such as 4 to 8 bits instead of the usual 32 or 64 bits.

This is sufficient for many AI applications and also saves a significant amount of energy, says Christian Hirsch. CPUs and GPUs could also handle such tasks, but they would consume significantly more power.

Why the NPU is becoming important now? - for local AI models

The trend towards NPUs is closely related to the current AI boom. Many applications, from image editing to language models and assistants, are based on neural networks. However, services like ChatGPT or Gemini currently run in large data centers "in the cloud". "With an NPU, local AI models also become possible. These run directly on the phone or PC. This means that data is no longer sent to servers, processed there, and the result sent back," explains Jörg Geiger.

AI directly on the device: saves time, provides data protection

Local processing not only saves time but can also provide advantages in terms of data protection. Sensitive data no longer needs to leave the user's computer or network. This can also be an argument for a company to run AI applications on employees' devices. In this case, computers with an NPU would be the right choice, according to Geiger. Another effect: the AI application becomes independent of an internet connection.

An example of current applications are Microsoft's Copilot+ PCs, which focus on local AI functions. According to Christian Hirsch, this includes the much-discussed Recall function, which permanently records the screen content to analyze and make it searchable.

Still little software - but a growing market

However, the practical benefit for average consumers is currently still limited. "There are currently only a handful of programs that really use NPUs," says Christian Hirsch. Many applications still rely on classic processors or cloud services for AI features. Another reason for this is the lack of standardization. Different manufacturers rely on their own solutions, and uniform interfaces are only gradually developing.

For developers, this means additional effort - and for users, many functions are only available if hardware and software work together optimally.

Can you notice it in everyday life? - It strongly depends on the application

Whether users currently benefit from computers with NPUs strongly depends on the application. "If you only surf and write emails, you won't notice a difference," says Jörg Geiger. The technology will only be noticeable for computationally intensive tasks - such as automatically improving images, cutting videos faster, or translating speech in real-time.

Features like background blur in video conferences or automatic text creation can also benefit from NPUs. For many typical everyday applications like web surfing, office work, or streaming, however, the NPU plays a minor role.

Nevertheless, the offer is growing. Operating systems like Windows are increasingly integrating AI functions, and programs for image, audio, and video editing are also relying more and more on corresponding acceleration. In the long term, the NPU could establish itself similarly to the GPU.

Additional costs immediately incurred - NPU cannot be retrofitted

Since the NPU is a fixed part of the processor and cannot be retrofitted, it is usually not listed separately. "You often have to look closely at the specifications of a laptop to determine if an NPU is installed," says Christian Hirsch. The technology is primarily found in newer and better-equipped devices. Entry-level notebooks, on the other hand, often still rely on older chips without an NPU.

"As a rough rule, you can say that affordable laptops under around 800 euros usually do not have an NPU," says Hirsch. Devices with corresponding equipment are usually above this price and often also come with more powerful hardware overall.

Buying an NPU computer - or leaving it alone?

Whether it's worth buying a computer with an NPU depends on how the device will be used and when it might be renewed. Because the market is currently in a transition phase, says Jörg Geiger. "If you want to be set up for the future, buy a notebook with an NPU."