NPU
An NPU is a neural processing unit: a low-power AI accelerator built into laptops, phones and industrial devices, so that inference runs on the device itself with no round trip to the cloud.
No jargon, no math
In plain words, what is it?
An NPU does the same matrix maths as a GPU, but designed around a power budget of watts, not kilowatts. The model lives in the device's own memory, compressed and trimmed (quantized) to fit. Your input, a voice command, a camera frame, a vibration reading, is converted to numbers and fed to the NPU, which returns an answer in milliseconds because nothing travels across a network. Two consequences follow: it works with the internet down, and sensitive data never leaves the machine. Where it breaks: the chip is frozen at purchase time. An NPU runs small models brilliantly and will never run a 70-billion-parameter model; when the task outgrows the chip, the work still has to go to a server or the cloud.
Terms that live nearby.
Edge AI
Edge AI is running AI models on the device where the data is born: a camera, a sensor, a phone, a factory gateway, instead of shipping everything to a cloud data centre and waiting for the answer.
ReadGPU
A GPU is a processor built to run thousands of small calculations at the same time. Designed for graphics, it turned out to be the perfect engine for AI, because running a neural network is mostly one enormous pile of matrix multiplications.
ReadAI Accelerator
An AI accelerator is a processor built for one job: the matrix multiplications that neural networks are made of. GPUs, TPUs and NPUs are all accelerators; they trade the flexibility of a CPU for far higher throughput per watt on AI workloads.
Read§ 03
Want this running in your stack?
We design and ship AI systems with permissions, logs and human approval. It starts with a 30-minute scoping call.