The short answer
A GPU, or graphics processing unit, is a chip that performs huge numbers of calculations in parallel, which is why it is used to train and run AI models. The same maths that draws game graphics powers neural networks. NVIDIA's newest data-centre platform is Vera Rubin, and on 21 October 2026 it announced a container of seven Vera Rubin racks with deliveries expected from early 2027. You can use AI without owning one, because cloud services run the GPUs for you.
| Plain meaning | A chip that does many calculations at once, ideal for AI maths |
|---|---|
| Used for | Training models and running them (inference) |
| Current NVIDIA platforms | Blackwell and Blackwell Ultra; Vera Rubin, the newest |
| Alternatives | Custom AI chips such as Amazon's Trainium |
| Do you need one? | No for cloud tools; yes, or a capable laptop, to run open models locally |
What a GPU is, in plain English
The main chip in a computer, the CPU, is built to do a few complicated things one after another, very quickly. A GPU was built to draw graphics, which means doing the same simple sum for millions of pixels at the same time. AI models turn out to need exactly that: vast numbers of multiplications and additions on large tables of numbers, all at once.
NVIDIA's own pages make the contrast directly, describing workloads "bottlenecked by the serial nature of CPU-only computing" and the use of "parallel GPU computing to remove bottlenecks". Modern NVIDIA GPUs also include Tensor Cores, which NVIDIA describes as specialised cores that speed up AI maths. Even the open-source tools developers use say it plainly: Hugging Face's Diffusers documentation tells you to move models to a GPU "to speed up inference".
How GPUs are used in AI
- Training. Building a large model means running enormous amounts of data through it on many linked chips in a data centre.
- Inference. Every answer, image or voice clip you generate runs on chips too. NVIDIA notes that reasoning models create more output tokens at inference time and need more compute resources, which is why "thinking" modes and higher effort settings use up plan limits faster.
- Local use. A good desktop or laptop GPU can run smaller open-weight models. Black Forest Labs says FLUX 3 Image Klein, released with open weights on 15 October 2026, runs on consumer GPUs down to an RTX 3090.
The chips in the news
NVIDIA's data-centre range currently spans the Hopper, Blackwell and Blackwell Ultra architectures and the newer Vera Rubin platform, sold in rack-scale systems such as GB300 NVL72 and Vera Rubin NVL72. On 21 October 2026 NVIDIA announced the DSX Pod AI Factory, which packs seven Vera Rubin racks into a container weighing about 28,000 kilograms that can be shipped by truck, with deliveries expected to start in early 2027 through partners such as Foxconn, Lenovo and HPE, as CNBC reported. NVIDIA said it cuts the time for a new AI data centre to start serving tokens from six months to 14 days. NVIDIA did not give a price; analysts quoted by CNBC estimate a single Vera Rubin rack can cost more than $7 million.
NVIDIA is not the only option. Cloud companies design their own AI accelerators; Amazon, for example, offers Trainium, and its third-quarter 2026 results say it made Trainium easier to adopt by letting developers run existing PyTorch code on it.
Why GPUs matter at work
You will probably never buy a data-centre GPU, but they shape the AI tools you pay for.
- They explain usage limits. Every prompt uses chip time. Google says Gemini's limits are based on compute, not a fixed number of prompts, and that images, video and Deep Research use them faster. Plan your heaviest tasks accordingly.
- They explain pricing changes. Amazon spent $77.3 billion on property and equipment in the third quarter of 2026, up from $35.1 billion a year earlier, expects about $220 billion of capital spending in 2026 and says its fall in free cash flow mainly reflects AI investment. Spending at that scale is one reason vendors keep reworking plans and limits. List which of your tools charge per seat and which per use, so you are ready to negotiate.
- They make private AI possible. If data cannot leave your company, you can run an open model on your own GPUs or on a dedicated cloud server. Read how to run AI locally on your computer and private AI for sensitive data.
- They affect speed. Faster chips and smarter software cut waiting time. Inception says its Mercury diffusion language models generate tokens in parallel to maximise GPU efficiency. See what a diffusion model is.
A check you can try in ten minutes
Find out whether your own computer could run an open model, and whether it is worth it.
- Find your GPU. Look up your computer's graphics chip and how much memory it has in your system information or settings. On a Mac, note the chip name and the memory instead.
- Pick a model you might use. For images, FLUX 3 Image Klein is one example; for text, look at the open models in our open-source ChatGPT alternatives guide.
- Read the model card. The model's page on Hugging Face or the maker's documentation lists the hardware it needs.
- Ask an assistant to compare. Paste your specs and the requirements into the prompt below.
- Decide honestly. If your machine falls short, a cloud tool such as ChatGPT, Claude or Gemini on a business plan is usually the simpler private option.
Related terms worth knowing
- Inference: running a model, which every prompt needs.
- Open-source model: models you can download and run on your own GPU.
- Reasoning model: models that use more GPU time per answer.
- Fine-tuning: further training, which also needs GPUs.
- Large language model: the kind of model most AI chips serve today.
- Token: the unit GPU output is often measured and priced in.
Common misconceptions about GPUs and AI
- "I need a powerful GPU to use AI." Not for cloud tools. The GPUs sit in the provider's data centre. You only need one to run models yourself.
- "GPUs are just for gaming." They began with graphics, but the same parallel maths now runs most AI training and inference.
- "Only NVIDIA makes AI chips." NVIDIA is the best-known name, but cloud providers build their own accelerators, such as Amazon's Trainium.
- "More GPUs automatically make a smarter model." Compute helps, but data, model design and training methods also matter. The same chips can train a mediocre model.
- "Running locally is always cheaper." A capable GPU, electricity and set-up time cost money. For occasional use, a paid plan is often cheaper and better.
Prompts to copy
Can my computer run this model?
My computer: [operating system, GPU model and its memory, or Mac chip and unified memory, total RAM]. I want to run [model name] for [task]. Its stated requirements are: [paste from the model card]. Tell me whether it will run, how fast it is likely to feel, what settings or smaller version I should use if it is borderline, and whether a cloud tool would be the better choice.
You get: A straight answer on local AI before you spend an afternoon installing
Prepare for AI price changes
Here are the AI tools my team pays for, with seats and plan: [list]. For each, tell me whether it charges per seat, per use or by credits, what happens when we hit a limit, and one question to ask the vendor at renewal. Present it as a table.
You get: A renewal-ready view of how your AI costs work