The short answer
To run AI locally, install a free app such as LM Studio or Ollama, download an open model sized for your computer and chat with it offline. In Ollama, the command ollama run gemma4:e2b downloads Google DeepMind's Gemma 4 E2B model and starts a chat; Ollama recommends 8 GB of available graphics memory, or unified memory on a Mac, for it. LM Studio's privacy policy says that when you run models locally, your messages, chat histories and documents are never sent from your system.
Why run AI locally, and what you give up
A local model runs on your own processor and memory. Nothing you type goes to an AI company, there are no usage limits or subscriptions, and it works on a plane. That makes it a good fit for drafting with confidential material, summarising internal documents and learning how models behave.
The trade-off is capability. The models that fit on a laptop are much smaller than the flagship models behind ChatGPT, Claude or Gemini. Even the open models from big labs can be far too large: DeepSeek-V4-Pro, released with open weights on Hugging Face under the MIT licence, has 1.6 trillion parameters, which needs data-centre hardware. Expect a local model to be good at summarising, rewriting and simple questions, and weaker at complex reasoning, current events and long documents.
What you need first
- A recent computer with enough memory. LM Studio supports Apple Silicon Macs (M1 to M4) on macOS 14.0 or newer and recommends 16 GB of RAM or more, though 8 GB Macs can run smaller models. On Windows it needs an x64 processor with AVX2 or an ARM Snapdragon X Elite chip, with 16 GB of RAM and at least 4 GB of dedicated graphics memory recommended. On Linux it ships as an AppImage for Ubuntu 20.04 or newer. Intel Macs are not supported by LM Studio.
- Free disk space. A small model is a 5 to 10 GB download; Gemma 4 E2B in Ollama is 4.6 to 7.5 GB depending on the version.
- Permission, at work. Installing software on a work laptop usually needs IT approval. LM Studio's terms (version of 23 August 2026) allow personal and internal business use.
- An internet connection for setup only. You need it to download the app and the model. After that, both tools can run offline.
Option 1: LM Studio, step by step (easiest, no command line)
- Download the installer for your system from lmstudio.ai/download and install it.
- Open LM Studio and go to the Discover tab (Cmd + 2 on a Mac, Ctrl + 2 on Windows and Linux).
- Search for a model by name, such as "gemma". Models come from Hugging Face, and each usually has several versions with names like Q4 or Q8.
- Pick a version your machine can handle. Q stands for quantisation, which compresses the model at some cost to quality. LM Studio's advice is to choose a 4-bit option or higher if your machine can run it; smaller versions use less memory.
- Download it, load it and start chatting in the chat window.
- To work with your own files, attach a .docx, .pdf or .txt document to a chat message. Short documents go into the conversation in full; long ones are searched for the relevant sections.
Option 2: Ollama, step by step (best for developers and other apps)
- Download Ollama for macOS, Windows or Linux from ollama.com/download. On Linux, the install command is curl -fsSL https://ollama.com/install.sh | sh.
- Open the app, or type ollama in a terminal and follow the setup prompts. Choose a local model; signing in is only needed for Ollama's cloud models.
- Run ollama run gemma4:e2b. It downloads the model if needed and opens a chat in the terminal.
- Type your prompt and press Enter. Type /bye to exit.
- To use the model from other software, point it at Ollama's local server at http://localhost:11434, which offers OpenAI-compatible endpoints and needs no API key.
Gemma 4 comes in several sizes in Ollama: e2b and e4b with a 128K context window, and 12b, 26b and 31b with 256K. The 26b and 31b versions are 16 to 20 GB downloads, so start small and move up only if your computer stays responsive.
A worked example: summarising a confidential report offline
Say you have a 20-page draft restructuring plan that must not leave your laptop. Install LM Studio, download a small Gemma 4 model while you are online, then switch off Wi-Fi to prove to yourself that it works offline. Attach the PDF to a new chat and ask for a one-page summary with the five biggest risks and the page each comes from. Because a long document is searched rather than read in full, ask follow-up questions using the words you expect to appear in the source, such as "redundancy costs in the finance section". Check every point against the page it cites. When you are done, the summary exists only on your machine.
Common mistakes and how to fix them
- Picking a model too big for your memory. The computer slows to a crawl. Fix: start with the smallest version (such as e2b), then step up.
- Accidentally using a cloud model. Ollama's library includes models tagged "cloud", such as gemma4:cloud, which run on Ollama's servers, not your computer. Fix: for private work, use only models you have downloaded, and do not sign in to cloud models.
- Turning on cloud features in LM Studio. Its privacy policy says prompts and files leave your device only if you use its paid Cloud Services, such as cloud models or web search. Fix: leave them off for confidential work.
- Expecting ChatGPT-level answers. Fix: give small models short, specific tasks and check facts; they have no web access and can be confidently wrong.
- Running out of context. Long chats and big documents use memory. Fix: start a new chat for each task and ask about one section at a time.
- Assuming a model's licence allows anything. Fix: check the licence on the model's Hugging Face page before using it in a product.
LM Studio or Ollama: which should you use?
- LM Studio if you want a normal app with a chat window, document attachments and a model browser. It is the easier start for non-technical users.
- Ollama if you are comfortable in a terminal or want other tools on your computer, such as coding agents or your own scripts, to use local models through the localhost API.
If local is not practical for your team, read the best private AI for sensitive data for cloud options with strong data terms, and open-source ChatGPT alternatives. For background, see what an open-source model is and why AI needs a GPU, or compare open models in Qwen vs DeepSeek.
Prompts to copy
Offline summary with page references
Summarise the attached document in under 300 words for [audience]. Then list the five most important [risks / decisions / numbers], each with the page or section it comes from. If something is unclear in the document, say so instead of guessing.
You get: A summary you can check against the source
Rewrite confidential text
Rewrite this text for [audience] in a [tone] tone, under [number] words. Keep every fact and number exactly as written and do not add anything new: [paste text]
You get: A rewrite of sensitive text that never left your computer
Small-model task breakdown
I want to [goal]. Break it into small steps that each need only a short answer. Give me the first step only, and wait for my reply before the next.
You get: A way of working that suits smaller local models
Check my hardware
My computer has [chip or processor], [amount] of RAM and [graphics card and its memory, or 'Apple unified memory']. Explain in plain English which size of local model I can run comfortably, what quantisation means for me, and what to try first.
You get: A realistic starting point for your machine (check it against the tool's own requirements)
Extract fields from a document
From the attached document, extract these fields into a table: [field 1], [field 2], [field 3]. Use 'not stated' when a field is missing. Do not infer values.
You get: Structured data pulled from private files without an online service