Edge AI
Edge AI is running AI models on the device where the data is born: a camera, a sensor, a phone, a factory gateway, instead of shipping everything to a cloud data centre and waiting for the answer.
No jargon, no math
In plain words, what is it?
A small model (quantized, trimmed) is installed next to the data source. The sensor produces data; the model turns it into numbers and processes it locally, in milliseconds, because there is no network trip. Only the outcome matters upstream: a flag, a count, an alert, or nothing at all. This is a division of labour. The edge handles the 99% of data that is boring, everything normal, and the cloud gets the exceptions and the heavy training. The payoff is latency, offline operation and privacy: nothing sensitive leaves the site unless you send it. Where it breaks: devices have tiny memory and power budgets, so models must be small and pruned. And updating five hundred scattered devices is an operations problem all of its own, nothing like updating one server.
- 01
Sense. A camera, sensor or machine produces raw data where it stands.
- 02
Score locally. A compact model on the device or its gateway processes the data in milliseconds.
- 03
Decide. The model judges whether anything is wrong: problems stay, everything else is discarded on site.
- 04
Act on the spot. Problems trigger a local alert or action, with no network wait.
- 05
Send only exceptions. Flagged events and summaries travel to the central system, nothing else.
Terms that live nearby.
NPU
An NPU is a neural processing unit: a low-power AI accelerator built into laptops, phones and industrial devices, so that inference runs on the device itself with no round trip to the cloud.
ReadLatency
Latency is the delay between a request and its answer, measured in milliseconds. In AI systems it adds up across queueing, prompt processing, token-by-token generation and network hops, and it decides whether a tool feels instant or unusable.
ReadInference
Inference is using a trained model: you send input in, the model applies what training encoded in its parameters, and you get an answer out, one token at a time.
Read§ 04
Want this running in your stack?
We design and ship AI systems with permissions, logs and human approval. It starts with a 30-minute scoping call.