Local AI: open models on your own computer

Why do it (and why not)

Good reasons:

• maximum confidentiality: client documents, health data, trade secrets you won’t or can’t send to third parties;
• predictable costs for very high-volume work (classifying millions of documents);
• offline operation and independence from a provider;
• customisation: you can specialise the model on your data.

Bad reasons:

• “It’s free”: hardware costs, electricity too, and the time of whoever manages it even more.
• “It’s the same as ChatGPT”: it isn’t. According to Artificial Analysis, in October 2026 the best open model (MiMo-V2.6-Pro) scores 46 against 58 for Claude Opus 5.5, and models that fit on a laptop are far behind.

Which models exist

The strongest open models of 2026 come mostly from China and a few Western labs. From the Artificial Analysis leaderboard:

Model	Parameters (total / active)	Index	Notes
MiMo-V2.6-Pro (Xiaomi)	1,020B / 42B	46	Most capable open model; needs server hardware
GLM-5.3 (Z AI)	about 750B / 40B	45	Strong in reasoning and code
Kimi K3 (Moonshot)	2,800B / 104B	44	Very long context
Qwen3.8 27B (Alibaba)	27B (dense)	n/a*	The smallest open model in the family
DeepSeek V4.1 Flash	552B / 8-16B	39	Fast and efficient
Mistral Small 4	119B / 6.5B	9-17	Still needs a workstation
gpt-oss (OpenAI)	117B / 5.1B and 21B / 3.6B	9-17	The 21B version runs on a good laptop
Gemma 4 (Google)	31B, 26B / 3.8B, 4.5B and 2.3B	9-17	The small versions run on a laptop too

*On the Artificial Analysis index the score of 40 belongs to the largest model in the Qwen3.8 family, not the 27B: I don’t report a comparable figure for the 27B, but it’s noticeably further behind.

“Active parameters” means how many are used for each word: Mixture of Experts models (see below) have a huge number but switch on only a part. In memory, though, all of them must fit: the first five in the table need from hundreds of GB to over a terabyte.

“Small” models score low on general benchmarks, but for targeted tasks (summarising, classifying, extracting data, translating) they’re often enough.

What hardware you need

A model is made of parameters, i.e. numbers. To run it they must fit in memory (ideally the graphics card’s memory, or the unified memory of Apple-chip Macs). With 4-bit quantisation, which reduces number precision with limited quality loss, the rule of thumb is:

Model size	Indicative memory	Where it runs
4-8 billion parameters	4-6 GB	Almost any recent computer
12-14 billion	8-10 GB	Laptop with 16 GB RAM
27-35 billion	18-24 GB	32 GB Mac, PC with a 24 GB graphics card
70-120 billion	45-80 GB	Workstation or 64-128 GB Mac
Over 300 billion	Hundreds of GB	Multi-GPU server

Many modern models are Mixture of Experts (names like “35B A3B”): they have 35 billion parameters but use 3 per token. They take the memory of a big model, because all weights must be in memory, but compute much less per word: so they’re often faster than a dense model of the same size, though speed depends heavily on memory bandwidth.

Long context takes extra memory: if you want to analyse hundreds of pages, plan headroom.

The real problem: memory, cost and intelligence

Put plainly: today, getting a truly capable model running locally is complicated and very expensive, and for most people it isn’t worth it. There are three reasons.

1. You need enormous amounts of memory. Open models that come close to the frontier have from 500 billion to almost 3,000 billion parameters. Even compressed to 4 bits they take from 300 GB to over 1.5 TB, all of which must sit in fast memory. A laptop has 16-32 GB: only small models fit.

2. Memory has never been this expensive. 2026 brought a real RAM crisis: manufacturers are shifting capacity to chips for AI data centres and memory for ordinary computers is scarce. According to TrendForce, conventional DRAM contract prices rose 90-95% in the first quarter of 2026 alone, and a 32 GB DDR5 kit that cost about $120 a year ago now costs around $400. Manufacturers expect shortages at least through 2027. In practice: a computer with 128 GB or more, the minimum for mid-sized models, costs many thousands of euros; a server for the large models costs tens of thousands.

3. Local models are much less intelligent. Even if you spend a lot, the best open model is well behind the best cloud models (46 vs 58 on the Artificial Analysis index), and those that run on an ordinary computer are further behind still. The concrete risk is spending a lot to use a model that isn’t very capable, when €20 a month gets you the best models in the world.

Start in ten minutes

With LM Studio (recommended to start)

1. Download LM Studio from the official site and install it.
2. In the search section, look for a small model (e.g. from the Gemma or Qwen families) in the 4-bit quantised version the app marks as compatible with your computer.
3. Download it and open the chat: it works like ChatGPT, but entirely local.
4. To use it from other programs, turn on the local server: it exposes an OpenAI-compatible API.

With Ollama (for automation)

Ollama runs from the terminal: one command downloads the model, another starts it. It’s perfect as the “engine” for private automations, for example with n8n installed on the same server: documents are read, classified and summarised without ever leaving the office.

A typical setup for a firm

1. An office computer or small server with a 20-35 billion parameter model.
2. Ollama as the engine, n8n for flows (see the automation guide).
3. An index of internal documents for questions (see the RAG and embeddings guide).
4. For hard, non-confidential tasks, a frontier model via API with a business contract.

This is the hybrid approach: local for what’s confidential and repetitive, cloud for what needs maximum intelligence.

In practice

• Try LM Studio with a small model: in ten minutes you’ll see what it can and can’t do on your computer, for free.
• Before buying hardware, do the maths: at today’s RAM prices a computer suited to mid-sized models costs many thousands of euros, for models still less capable than cloud ones.
• Invest in local AI only with a big budget and a need for maximum confidentiality (with a truly isolated setup) or fine-tuning; otherwise pick a business cloud plan.
• Calculate the real cost: hardware, energy, management time.
• Always check the model’s licence.

Sources

• Artificial Analysis — open-weights model leaderboard
• LM Studio
• Ollama
• Tom's Hardware — DRAM and NAND prices still climbing in 2026 (TrendForce data)
• J.P. Morgan — The AI-driven memory shortage

Informational content, current as of the date shown. Prices, models and rules change often: always check official sources. Written with the support of AI tools and reviewed by the author.