Local Coding LLMs on a 32 GB Mac

TL;DR

1 · Which model fits on 32 GB?

Twelve tasks with hidden tests: bug hunts from a README spec, implementing from spec, refactoring, a CLI feature that has to follow existing patterns, debugging from a production stack trace, a performance rewrite, and two honesty tasks where a referenced function or config key does not exist. Half the tasks are JavaScript, half Python. Everything ran through OpenCode against LM Studio with 65k context.

• On 32 GB, memory beats benchmarks. An agent like OpenCode sends about 10k tokens before the first tool call and quickly reaches 30–60k. Models with classic attention grow their KV cache by several GB over that range and hit the GPU memory limit. Qwen3.6 uses hybrid attention with a tiny cache, so it fits despite 20 GB of weights.
• Single runs are noisy. The same model and settings scored anywhere from 6 to 9 of 12. Do not trust a single run that differs by one or two tasks.
• Splash is the free win. Same weights, same answers, 2.5× faster decode. Per task the speedup is smaller (about 1.3×) because tool calls, tests and prompt processing take fixed time.

2 · Tuning prompts, sampling and max tokens

• The short rules prompt fixed honesty. Without it, the model invented a missing helper function in 8 of 8 runs; with it, it stopped and reported the gap in 3 of 3. The rules: read the spec and all relevant files first, change only what is needed, never invent missing things (stop and report MISSING:), verify every requirement with tests at the end.
• 4k max tokens is a trap. The model spends the whole budget thinking and returns an empty answer. 16k is fine, 32k does no harm.
• OpenCode sends no temperature at all for OpenAI-compatible providers, so the server defaults apply. Setting the officially recommended sampling made no measurable difference.

3 · Same model, nine coding agents

Eight tasks built to expose differences between agents: a one-function edit in a 2,100-line file, preserving tabs and CRLF line endings, renaming a feature flag across 314 files (including JSON, YAML and a concatenated string), npm test hanging in watch mode, project rules that exist only in AGENTS.md, a registry built from 40 plugin files, a test loop against 26 visible tests, and cleaning a git-ignored build/ folder without touching data/ or .env. Plus four general tasks.

* quick run on the five most discriminating tasks. Every agent ran once with an isolated, empty config directory. Claude Code works locally because LM Studio also serves an Anthropic-compatible /v1/messages endpoint; Codex used LM Studio's Responses endpoint.

4 · Real bug fixes from open-source history

SWE-bench style: eight real, merged fixes and small features from yargs-parser, qs, humanize, python-tabulate and python-dotenv. The repo is reset to the commit before the fix, the task is a paraphrase of the real issue, and grading uses the tests the maintainers added with the fix. Five of the eight fixes are from 2026, so the model cannot know them from training. All agent runs and all foreign test code ran inside a macOS sandbox (writes only inside the test folder, network only to the local model server).

** In this run OpenCode's search tool was broken by the sandbox (it tried to download ripgrep at first start), so this is a lower bound. The brief + test runs were repeated after the fix. R1 negative decimals in yargs-parser · R2 unpaired UTF-16 surrogates in qs (cause in a different file) · R3 new WHATWG output format in qs · R4 time-zone handling in humanize · R5 infinite loop with wide CJK characters in tabulate · R6 CRLF handling in python-dotenv · R7 opt-out env variable in python-dotenv · R8 alias plus dot-notation in yargs-parser.

• The acceptance test is what makes the difference. A brief without a test did not help measurably. With a test, every success came on the first attempt.
• The test has to cover every edge case of the issue. Once the acceptance test passed but a maintainer test with an extra edge case still failed. What the test does not check, nobody checks.
• Context is tight at 65k. Reading a 2,700-line file plus three test files overflowed it. Name the line range in the brief.
• Avoid absolute paths outside the repo in prompts. The model went looking for the code there.

The workflow that works

1. A stronger model (or you) reads only the relevant code and writes a 3–5 sentence brief: file, function, root cause, approach.
2. It writes an acceptance test from the examples and edge cases in the issue and checks that it fails on the current code.
3. The local agent works in an isolated git worktree until the test passes. If the test is still red, it gets the test output and one more attempt.
4. Only the test result and the diff are reviewed before merging.

In our runs this cost the stronger model roughly 3,000 tokens per task, while all exploration, editing and test runs (15 to 90 model requests per task) stayed on the local machine. That is an estimate, not a precise measurement.

Example brief (qs, unpaired surrogates):
Cause in lib/utils.js, function encode, surrogate branch at the end of the loop:
it assumes c is a high surrogate and the next char a low surrogate, but never
checks. Check both; if it is not a valid pair, emit %EF%BF%BD and do not
consume the next character.

Setup that worked

• LM Studio with the Splash runtime enabled (Settings → Runtime) and the Splash build of Qwen3.6-35B-A3B. Context 65,536, parallel 1. If the in-app download stalls, finishing it with the Hugging Face CLI and re-running lms get registered the model.
• OpenCode with an OpenAI-compatible provider pointing at http://127.0.0.1:1234/v1, output limit 16k, and "permission": {"external_directory": "deny"} for headless runs.
• pi reads providers from ~/.pi/agent/models.json. It has no permission prompts, so run it in a sandbox. In scripts, give it an empty stdin or it waits for input.
• Sandbox on macOS: sandbox-exec with a profile that denies all file writes except the project, temp and cache folders, and denies network except to localhost, was enough to run agents and foreign test code safely.

(version 1)(allow default)(deny file-write*)
(allow file-write* (subpath "/path/to/project") (subpath "/private/tmp")
                   (subpath "/private/var/folders") (subpath "/dev"))
(deny network*)(allow network* (local unix))
(allow network* (remote ip "localhost:*"))

Limits

• One machine, one night of runs, mostly one run per configuration. Differences of one or two tasks are noise.
• Most tasks were written for this test; only the last series uses real open-source history, and it has just eight tasks.
• Splash is new (we used runtime 1.3.0) and officially recommends 36 GB of memory. It ran stably for many hours at 65k context on 32 GB.
• Agents ran headless with default settings. Interactive use, other modes or newer versions may behave differently.