Running a model on your own machine

The option most people do not know exists

Everything so far has assumed your text travels to somebody else's computer. It does not have to. A category of models is published with open weights — the model file itself is downloadable — and modern laptops can run useful ones.

Nothing leaves your machine. No account, no terms of service, no retention period, no training question, no jurisdiction problem, no subscription. For confidential work, that is not a small convenience; it changes what is permissible.

What it takes

Less than people expect.

• Software. Ollama is the simplest: install it, run one command, and a model downloads and starts. LM Studio offers the same with a normal desktop interface for people who would rather not use a terminal. Both are free.
• Hardware. A model with around 7 to 12 billion parameters, compressed to about 4 bits per weight, occupies roughly 4 to 8 GB and runs at readable speed on a laptop with 16 GB of memory, and comfortably on an Apple Silicon Mac. On 8 GB you can run something smaller and slower. A discrete graphics card makes it fast; the absence of one makes it usable, not impossible.
• Time. About twenty minutes from decision to first answer, most of it downloading.

# install Ollama, then:
ollama run llama3.1:8b

That is the whole setup. The prompt appears and the machine answers with its network cable unplugged, which is a demonstration worth doing once in front of a sceptical colleague.

The honest quality gap

This is where most enthusiastic write-ups stop being useful, so let us be direct. A model you can run on a laptop is meaningfully weaker than the best hosted models. The gap is small on some tasks and large on others.

Close to parity: rewriting, tone changes, summarising a few pages, extracting fields from a document, translating between major languages, drafting routine correspondence, explaining a concept, classifying a batch of short texts.

Clearly worse: long multi-step reasoning, subtle analysis of a complex document, code of any difficulty, anything requiring broad world knowledge, and long inputs — local models usually hold far less text at once, and quality falls off as you fill the window.

Not different at all: invention. A local model makes things up exactly as readily. Privacy and accuracy are separate properties, and gaining one buys you nothing of the other.

Notice that the "close to parity" list is most of module two and much of module three. A great deal of ordinary office work does not need a frontier model, and a lot of people are sending confidential text to one for tasks a laptop would have done.

Where this is the right answer

• Regulated text that must not leave. Patient notes, client files, case papers, HR records, unfiled financials, anything under an NDA.
• Organisations with no budget. Schools, small clinics, NGOs, single-office firms. Nothing to buy, no per-seat cost, no procurement.
• Poor or intermittent connectivity. It works offline, entirely.
• Demonstrating to a nervous employer. A pilot on a laptop with no data leaving the building is far easier to get approved than a licence, and it produces evidence for the licence conversation later.

What it does not solve

It removes the provider from the picture. It does not remove your duties. Personal data on a laptop is still personal data: it needs encryption at rest, a screen lock, a backup policy and a plan for when the laptop is stolen. If the output ends up in a patient's record or a client's file, every professional obligation about accuracy applies exactly as before.

And there is a maintenance cost somebody must carry — updating the software, choosing when to move to a newer model, and explaining to colleagues why this one is worse at some things. Small, but it is not zero, and it usually lands on whoever set it up.

The point worth keeping

The interesting fact is not that this is possible. It is that the choice is no longer between using AI and protecting confidentiality. For a large share of everyday work, you can have both, for nothing, on hardware you already own — and anyone who tells you the only options are a subscription or abstinence has not looked.