By Gavin Stevens · Updated October 2026
Run AI locally or in the cloud? Which jobs belong on your computer and which belong with ChatGPT or Claude
Local LLM vs cloud AI: a model on your own machine is a fixed effort budget, right for drafts, code first passes and anything you'll review, wrong for security review and hidden edge cases. How to split the work between local and ChatGPT, Claude or the API, with the benchmark data behind the line.
As an Amazon Associate I earn from qualifying purchases. Product links on this page go to Amazon; they are affiliate links, and the price you pay is the same.
The picks
NVIDIA GeForce RTX 5060 Ti (16GB)
The cheapest way to own the local half of the loop. A 14B model on this card handles drafts, sketches, and first passes; send the verification to the cloud.
For local LLMs, holding the model matters more than speed. This is the lowest price of entry to 16GB with CUDA. Get the 16GB version; the 8GB one is useless for this.
NVIDIA GeForce RTX 3090 (24GB, renewed)
24GB runs the 30B class, which closes most of the gap on well-specified work. The step up that changes what you send to the cloud.
For LLM inference, VRAM capacity matters more than compute. Two of these give you 48GB for less than one 5090.
Mac Studio (M5 Max, 128GB unified memory)
70B and gpt-oss-120B at a usable speed. Local drafts get good enough that the cloud verification pass rarely changes much.
The smallest quiet box that runs 70B models at a speed you'd use every day. The 128GB Strix Halo boxes fit the same models at less than half the bandwidth.
You can run AI locally now: Ollama or LM Studio, a model on your own GPU or Mac, nothing leaving the machine. You can also use a cloud model, ChatGPT, Claude, Gemini, or the API behind them. The question is which jobs go where. The answer is both, and the split is not about which model is smarter. It’s about how much effort the job needs.
The short answer: local for drafts, cloud for verification
Local for anything you’ll look at before it matters: drafts, brainstorming, sketches, small edits, first passes at code, summaries you’ll read anyway. Cloud, with effort turned up, for anything that has to be right without you checking it: security review, bug fixes in a codebase you don’t fully know, anything with hidden edge cases, and anything you want run end to end while you’re away.
The loop that works: spec it, draft it locally, review the gist, then send verification to the cloud.
The data: what extra effort actually buys
In September 2026 Anthropic’s Claude Code team published a study of what their “effort” setting actually buys. Effort tells the model how much compute to spend. The author ran the same tasks at low, medium, high, and max effort and dug through Terminal-Bench 3.0, a community benchmark of 70 hard tasks.
The finding that matters for you: extra effort buys verification and edge-case testing. It does not buy a better approach.
The numbers, Fable 5.1 on 370 attempts:
| Failure type | Low effort | Max effort |
|---|---|---|
| Passed | 140 | 214 |
| Missed a case | 59 | 24 |
| Wrong call (misread the task, wrong domain rule) | 133 | 107 |
| Median tokens per attempt | 73k | 222k |
Tripling the compute cut “missed a case” failures by more than half. “Wrong call” failures barely moved. A model that misunderstood the job keeps misunderstanding it no matter how long it thinks.
The cleanest example: an HTML sanitizer that has to strip every way of smuggling JavaScript into a page. At low effort, the model wrote the filter in one pass, tested it on one page, and passed 1 of 5 attempts in about 2 minutes each. At high effort it reviewed its own draft adversarially, read the parser’s source, ran a standard XSS suite, wrote a fuzzer, and passed 5 of 5 in about 33 minutes each.
Why a local LLM is a capped effort budget
A local model is a capped effort budget. A 14B model on a 16GB card, or a 70B on a 128GB Mac, will do the task once, reasonably, and stop. It won’t spend 33 minutes writing a fuzzer, and if you asked it to, the result would be weaker than the frontier model’s. What you get from local hardware is the low-effort tier: fast, private, free per token, and good enough for everything that doesn’t hinge on an edge case.
So the question “local or cloud” is really “does this job need the extra verification, and is anyone going to check it?”
When to run AI locally
You’re in the loop. The author’s own preference, even with the cloud available, was low effort for brainstorming, sketching, and easy changes. Faster turnaround beat polish because he was going to iterate anyway. That’s exactly where a local model lives: the answer comes back in seconds and you fix what’s wrong.
The job is well specified. When the author gave models a detailed spec instead of a one-line prompt, the outputs at every effort level came out much more alike. The gap between low and max shrank to details. A tight spec does for a local model what effort does for a cloud one: it removes the decisions the model would otherwise get wrong. Write the spec, and the local model gets closer than you’d expect.
The failures are “wrong call” failures anyway. If the risk in a task is that the model misreads what you want, more cloud effort won’t save you (133 to 107). Your review will. Do it locally and read the output.
Privacy, volume, or cost. Anything you’d rather not send off-machine, anything you run a thousand times a day, anything where the per-token bill matters more than the last ten percent of quality.
When to use the cloud (ChatGPT, Claude, or the API)
Hidden edge cases. Sanitizers, parsers, anything that touches untrusted input, performance work, bug fixes in a brownfield codebase. The author’s rule of thumb puts these at high effort, and the benchmark shows why. A local model will write the fix. It won’t reproduce the crash first, write a randomized test, and check that the test fails on a half-finished fix.
Nobody is checking. In one benchmark task, the high-effort run tried two ways of preparing the data, noticed the answer changed, and worked out which was right. The low-effort run picked one and reported it. If you were there, you’d have caught it. If the job runs overnight, you need the model that catches it.
These domains. Pass rate at low effort versus top effort, Fable 5.1:
| Domain | Low | Top | Lift |
|---|---|---|---|
| Security | 64% | 87% | 23 pts |
| Hardware | 34% | 75% | 41 pts |
| ML | 54% | 73% | 19 pts |
| Science | 41% | 61% | 20 pts |
| Software | 43% | 56% | 13 pts |
| Media | 18% | 30% | 12 pts |
| Operations | 12% | 22% | 10 pts |
Security, hardware, ML, and science got the most from extra effort. Operations and media, the rulebook-style work, got the least. If your job looks like following a procedure, local is fine. If it looks like finding what the procedure missed, send it out.
The local-plus-cloud workflow
The author’s workflow for feature work, with the local machine slotted in:
- Write the spec. Ask the model to interview you about what’s missing. Local is fine for this; it’s a conversation.
- Implement locally. Fast, private, no bill.
- Review it yourself. You’re checking for “wrong call” failures, which no amount of effort fixes.
- Send verification and testing to the cloud at high effort. This is where “missed a case” failures get caught, and it’s the only step that needs the big model.
Local is the editor. Cloud is the auditor.
Which computer for the local half
The memory table covers what fits where. For the loop above, the floor is a 16GB card running a 14B model: enough for drafts and first-pass code. A 24GB card or a 32GB Mac runs the 30B class, which closes most of the gap on well-specified work. The 128GB boxes in the desktop comparison run 70B and mixture-of-experts models that make step 2 good enough that step 4 rarely changes much.
None of them replace step 4 for security or edge-case work. Buy the hardware for the ninety percent of the day that’s step 1 through 3.
Common questions
Is a local LLM as good as ChatGPT or Claude?
On a well-specified task you’ll review, close enough that the difference is details. On a task with hidden edge cases or one nobody checks, no: the cloud model at high effort catches what a local 14B or 30B model never looks for. The gap is verification, not intelligence.
Is running AI locally cheaper than the cloud?
Per token, local is free after the hardware. The hardware is the bill: a 16GB card to a 128GB desktop, depending on model size. It pays off when you run a model all day, or when the data can’t leave the machine. For a few prompts a week, a cloud subscription is cheaper than any GPU.
Is local AI more private?
Yes. Nothing leaves your computer, no provider logs the prompt, and no terms of service change under you. That alone decides the question for medical, legal, client, or unreleased-product work.
Can I use local and cloud models together?
That’s the whole point of this page. Draft and iterate locally, send the verification pass to the cloud. Tools like Claude Code, Cursor, and Open WebUI let you switch models per task, so the split costs nothing to set up.
What this data doesn’t say
The study measured effort within one model family. It never ran a local 30B next to a frontier model at low effort, so “local equals low effort” is an inference from how both behave, not a number from the benchmark. Treat the domain table as a guide to which jobs are effort-sensitive, not as a measurement of your model.
Sources
- Using Claude Code: Spending your effort (Thariq Shihipar, Sep 2026; all effort and benchmark figures above)
- Terminal-Bench 3.0 task list