local-llm/README.md

local-llm

Local LLM setup on the Mac mini: Ollama serving models, opencode using them.

./setup.sh

does everything, and is safe to re-run:

• installs ollama/com.ollama.server.plist as the LaunchAgent that runs ollama serve (reloaded only when it changed, since that restarts Ollama);
• builds each model in ollama/models/ (pulling its base model if missing);
• symlinks ~/.config/opencode/opencode.json to opencode/opencode.json, so edits made either way end up here.

Needs brew install ollama opencode.

Models

Model	From	Used for
qwen3-coder:30b-32k	qwen3-coder:30b	opencode's default
qwen3:32b-32k	qwen3:32b	opencode, as an option
inspector	qwen3:32b	16k-context variant

Each derived model mostly exists to set num_ctx. Ollama's OpenAI-compatible API, which opencode talks to, ignores any context size sent with a request, so without it a long agent prompt is silently cut to Ollama's small default context. A Modelfile's # name: line is the name it's created under.

qwen3:32b is a thinking model, and in opencode it often ends its turn after thinking, without making the tool call it planned. qwen3-coder was trained for agentic tool use and doesn't think first, so it doesn't have that problem, and it's much faster: a mixture of experts with ~3B active parameters.

Server settings

From the LaunchAgent: listens on all interfaces (0.0.0.0:11434), keeps models loaded indefinitely (OLLAMA_KEEP_ALIVE=-1), flash attention on. Logs go to /tmp/ollama.log.