Key points
- Open-weight models publish their trained parameters so anyone can download and run them; “open source” in the strict sense also requires openness about data and code.
- Running locally means your prompts never leave your device, it works offline and there’s no per-message cost.
- Trade-offs: you need capable hardware, some setup, and the best local models are usually less capable than top hosted ones.
- Beginner tools such as Ollama and llama.cpp-based apps make local AI much easier than it used to be.
Closed, open-weight and open source
When you use ChatGPT, Claude or Gemini, the model runs on the company’s servers. You can’t download it or inspect it; you access it through an app or API. These are often called closed models.
Open-weight models are different: the company or research group publishes the model’s trained parameters (the “weights”), so you can run the model on your own hardware or a server you control. Licences vary — some allow almost any use, others restrict commercial use or large companies — so always read the licence.
Open source, in the strict sense, goes further. The Open Source Initiative’s Open Source AI Definition 1.0 describes an open-source AI system as one you can use, study, modify and share, which requires access not only to the weights but also to sufficient detail about the training data and the code used to train and run it. Many well-known “open” models publish weights but not their full training data, so they’re open-weight rather than open source under that definition.
Why run AI locally?
- Privacy. Prompts and documents stay on your machine. That’s the strongest privacy option for sensitive notes, client data (where your obligations permit) or personal journals. See our AI privacy guide.
- Offline use. Once downloaded, many models run with no internet connection.
- No subscription or per-token cost. You pay for hardware and electricity, not usage.
- Control. No sudden changes to the model, its behaviour or its pricing; you choose when to update.
- Learning and tinkering. You can experiment with settings, customise models and build your own tools.
The trade-offs
- Capability. Top hosted models are generally stronger at complex reasoning, long documents and coding than models you can run on a typical laptop.
- Hardware. Larger models need a lot of memory. Smaller models run on everyday laptops but are less capable.
- Speed. Without a strong graphics card or modern unified-memory chip, answers can be slow.
- Features. Web search, image generation and file handling may require extra tools.
- Responsibility. You manage updates, licences and safe use yourself.
What hardware do you need?
The key resource is memory (RAM or graphics card memory), because the model’s weights must be loaded to run. Models come in sizes described by their parameter count, and many are offered in “quantised” versions that compress the weights to use less memory with some loss of quality. As a rough guide:
- Small models can run on a recent laptop with 8–16 GB of memory — good for drafting, summarising and simple questions.
- Medium models are more comfortable with 32 GB or more, or a graphics card with plenty of video memory.
- Large models need workstation-class hardware or a server.
Model pages on hubs like Hugging Face usually list the memory each version needs. Start small and move up if your machine handles it.
Beginner-friendly ways to start
Ollama
Ollama describes itself as a way to run AI models locally and in the cloud. You install the app, choose a model from its library and chat with it. It also offers a local API that other apps can connect to.
llama.cpp and apps built on it
llama.cpp is an open-source project for running language models efficiently in C/C++ on ordinary hardware. Many desktop chat apps use it under the hood. It’s a good choice if you want more control or want to run models on less powerful machines.
Hugging Face
Hugging Face hosts a very large collection of open models with model cards that describe their purpose, licence, limitations and memory needs. It’s the place to research models before downloading.
A quick start
- Check your computer’s memory.
- Install a reputable local AI app from its official website.
- Download a small, popular, instruction-tuned model.
- Test it on a real, non-critical task — summarising a document or drafting an email.
- Compare the result with a hosted chatbot to see whether the quality is enough for your needs.
Glossary for beginners
- Weights / parameters: the numbers learned during training that make up the model.
- Quantisation: compressing those numbers so the model needs less memory, with some loss of quality.
- Instruction-tuned: a version trained to follow instructions and chat, usually the one beginners want.
- Model card: the description page that explains a model’s purpose, licence and limitations.
Remove this box by adding your content here, or hide all notes with
"showEditorNotes": false in site.config.json.Open-weight models in the cloud
You don’t have to run open-weight models yourself to benefit from them. Some companies that release open-weight models also sell hosted access. DeepSeek, for example, publishes API pricing for its hosted models, which on November 19, 2026 listed peak-hour prices of $0.30 per million input tokens (cache miss) and $1.20 per million output tokens for its Flash model. Compare costs with our API cost calculator. Note that using a hosted version means your prompts go to that provider, so the privacy benefit of local use doesn’t apply.
Safety tips
- Download models and tools only from official project pages or well-known hubs.
- Read the licence before using a model commercially.
- Read the model card for known limitations and biases.
- Local models still hallucinate — verify facts as you would with any AI. See why AI hallucinates.
- Keep software updated; local AI tools are actively developed and receive security fixes.
Is local AI right for you?
If privacy is your top concern, you work offline, or you enjoy tinkering, local open-weight models are well worth trying. If you need the strongest reasoning, image generation and web search in one place, a hosted chatbot is easier. Many people use both: a hosted chatbot for general work and a local model for private material. Our chatbot quiz can help you decide.
Frequently asked questions
Is an open-weight model the same as open source?
Not always. Open-weight means the trained parameters are downloadable. The Open Source Initiative’s Open Source AI Definition also expects sufficient information about training data and the code used to train and run the model, under licences that allow use, study, modification and sharing. Many popular models publish weights but not everything else.
Can I run AI without the internet?
Yes. Once a model is downloaded, local tools can run it completely offline.
Are local models as good as ChatGPT or Claude?
The largest closed models generally remain stronger on hard tasks, but smaller open models can be very good for drafting, summarising, coding help and private notes — especially for their size.
What computer do I need?
Small models can run on a recent laptop with 8–16 GB of memory. Larger models need much more memory, ideally a capable graphics card or a computer with lots of unified memory.
Are local models safe to download?
Download only from reputable sources and official project pages, keep your tools updated, and check the licence. Treat model files and tools like any other software.
Sources
- Open Source Initiative — The Open Source AI Definition 1.0 (opens in a new tab)
- Hugging Face — Models documentation (opens in a new tab)
- llama.cpp (GitHub) (opens in a new tab)
- Ollama (opens in a new tab)
- DeepSeek API pricing (hosted open-weight family) (opens in a new tab)
- Wikipedia — Large language model (opens in a new tab)
Facts, prices and features were checked against these sources on November 19, 2026. AI products change quickly; if you spot something out of date, please tell us.