Four processors, four completely different design philosophies. The CPU is a generalist that runs your operating system. The GPU is a parallel workhorse that runs your models. The TPU is Google's specialized machine for matrix math at cloud scale. The NPU is a low-power inference engine sitting inside your phone. Modern AI systems use all four together — the interesting question isn't which one wins, but which slice of the workload belongs to each.
The instinct when learning about AI hardware is to look for a winner. Which processor is fastest? Which one is the future? The honest answer is that these four chips were designed for such different jobs that comparing them by raw speed misses the point. It's like asking whether a sports car is better than a delivery truck. Depends what you need to do.
Let's build the mental model properly.
The Central Processing Unit is what most people picture when they hear "processor." It sits at the heart of every computer, phone, and embedded device. It runs your operating system, your browser, your background services. It's built for versatility, not raw parallel throughput.
A modern desktop CPU has 8-16 cores, each capable of running complex, branchy code — the kind of logic that says "if this, then that, otherwise loop back and try again." CPU cores are optimized for sequential decision-making: chase pointers through memory, handle interrupts, coordinate the many programs running at once.
Peak CPU throughput for numerical work is around 1-5 TFLOPS (trillion floating-point operations per second). That's respectable, but it's roughly two orders of magnitude below what a modern GPU can do on parallel math. That gap is why the whole rest of this list exists.
The Graphics Processing Unit was originally built to draw triangles. Millions of them, in parallel, thirty times a second. That workload — "do the same math on huge batches of independent data" — turned out to be exactly what neural networks need.
A modern data-center GPU (like NVIDIA's H100 or B200) has thousands of small cores plus specialized Tensor Cores for matrix operations. It delivers 80-300 TFLOPS at FP16, and much more at lower precision. It's flexible enough to handle almost any parallel workload — training LLMs, running inference, rendering games, simulating fluid dynamics, mining crypto (RIP).
The GPU's strength is its programming ecosystem. CUDA (NVIDIA's programming platform) has 20 years of libraries, tools, and community behind it. Every major machine learning framework — PyTorch, TensorFlow, JAX — runs on GPUs first. This ecosystem lock-in is why GPUs dominate training, even though other chips are technically more efficient at pure matrix math.
The Tensor Processing Unit is Google's custom-designed ASIC (Application-Specific Integrated Circuit) for AI. Introduced in 2016, TPUs give up the GPU's flexibility in exchange for extreme efficiency at one specific pattern: matrix multiplication.
At their core, TPUs use a systolic array — a grid of arithmetic units that pass data to their neighbors in lockstep, without repeatedly reading and writing memory between operations. This design cuts out most of the overhead a GPU carries around, letting TPUs deliver 90-420 TOPS at exceptional power efficiency for large matrix workloads.
The catch: TPUs are optimized for large batch sizes (typically 128-1024 samples at once). They shine when you're serving thousands of parallel requests or running massive training jobs. They struggle when you need to answer one query with sub-10ms latency — the batching overhead cancels out their efficiency. And they live inside Google's cloud, so you can't buy one and stick it in your server.
The Neural Processing Unit is designed for a completely different problem: run trained AI models on battery-powered devices without draining the battery in an hour.
NPUs live inside your phone (Apple's Neural Engine, Qualcomm's Hexagon NPU), your laptop (Intel's AI Boost, AMD's XDNA, Apple's Neural Engine again), and increasingly your car and smart home hardware. They deliver 1-50 TOPS — orders of magnitude less than a GPU — but at a fraction of the power draw. A GPU might use 400 watts under load; an NPU might use 1-2 watts.
The tradeoff: NPUs handle inference only, not training. They lean on low-precision math (INT8, INT4, sometimes lower) that would be unacceptable for training but works well for a model that's already been trained elsewhere. If your phone's camera app recognizes faces or applies a background blur in real time, that's an NPU doing the work — not the CPU or GPU, both of which would burn too much battery.
| CPU | GPU | TPU | NPU | |
|---|---|---|---|---|
| Built for | General logic | Parallel math | Tensor ops | Edge inference |
| Cores | 8–128 | 1,000s–10,000s | Systolic array | Specialized blocks |
| Peak throughput | 1–5 TFLOPS | 80–300 TFLOPS | 90–420 TOPS | 1–50 TOPS |
| Power | 15–250W | 200–700W | Data-center scale | 1–15W |
| Best at | Orchestration | Training + inference | Large-batch inference | Real-time on-device |
| Where you find it | Everywhere | Cloud, workstations | Google Cloud only | Phones, laptops, IoT |
The biggest misconception is that these processors compete. In reality, a single AI system usually uses several at once, each handling the part it's best at.
Training a large language model on Google Cloud: a fleet of TPUs does the tensor math, coordinated by CPUs that handle the data pipeline, checkpointing, and orchestration. Sometimes GPUs handle preprocessing.
Running that same model in production as a chatbot: GPUs serve most requests. CPUs route traffic, apply filters, and format responses.
Using that chatbot on your phone: your query travels to the cloud (where GPUs answer it), but the app itself uses your phone's NPU for local features — voice recognition, on-device transcription, real-time translation. The CPU ties it all together.
Modern AI infrastructure is a stack, not a race. Understanding what each chip is optimized for is understanding where in the stack it belongs.