GPU
A GPU is a processor built to run thousands of small calculations at the same time. Designed for graphics, it turned out to be the perfect engine for AI, because running a neural network is mostly one enormous pile of matrix multiplications.
No jargon, no math
In plain words, what is it?
A CPU is a few very fast, very clever cores; a GPU is thousands of simple ones. A neural network is a long chain of matrix multiplications, and every element of the result can be computed independently, so the GPU farms the work out across all its cores at once. The second half of the story is memory: the model's weights live in the chip's own high-speed memory (VRAM) and must stream to the cores fast, or the cores sit idle waiting for numbers. Training runs the maths forward, measures the error, then runs it backward to adjust the weights, millions of times; inference runs the forward pass only. Where it breaks: when a model is bigger than your VRAM it simply does not fit, and the same job on a CPU can be 10 to 50 times slower.
Terms that live nearby.
AI Accelerator
An AI accelerator is a processor built for one job: the matrix multiplications that neural networks are made of. GPUs, TPUs and NPUs are all accelerators; they trade the flexibility of a CPU for far higher throughput per watt on AI workloads.
ReadVRAM
VRAM is the dedicated memory on a graphics card or AI accelerator that holds the model's weights and the intermediate numbers of each calculation, so the processors doing the AI work never wait on slower system RAM.
ReadInference
Inference is using a trained model: you send input in, the model applies what training encoded in its parameters, and you get an answer out, one token at a time.
Read§ 03
Want this running in your stack?
We design and ship AI systems with permissions, logs and human approval. It starts with a 30-minute scoping call.