Skip to main content

Local models

Phonton can run goals on a model on your own machine. No key, no account, and a local goal never falls back to a cloud model.

Set up​

phonton models catalog                  # models that fit this machine
phonton models setup qwen2.5-coder:3b   # runtime, download, calibrate, select

Setup downloads the official Ollama runtime for your platform (Windows x64, Linux x64/arm64, macOS), checks its published SHA-256 and every file it unpacks, and starts it on 127.0.0.1:11434 with cloud models disabled. It is about 1.5 GB on Windows and Linux and 160 MB on macOS. Phonton then confirms the process it started is the one answering on that port, so goals run on a verified runtime. CLI versions before 0.23 did this on Windows only.

phonton models catalog shows what fits the memory free right now. Phonton keeps 2 GB free for the system, so on a Mac without a separate GPU qwen2.5-coder:3b needs about 6 GB of free memory.

Calibration measures which edit formats the model produces correctly on your hardware. It does not measure general coding quality. If no format passes, the model cannot be selected; pick another from the catalog.

CLI 0.22.0 needs the steps one at a time: phonton models setup, then install, calibrate, and select with the model name.

Run a goal​

In the TUI, run phonton in your project and type a goal. Each local goal pauses for plan approval before the model starts.

Headless:

phonton goal "Add a count() method to TodoStore" --yes --allow-host-checks
phonton goal --local show RUN_ID            # candidates, checks, receipt
phonton goal --local apply RUN_ID --yes     # land the verified candidate
phonton goal --local rollback RUN_ID --yes  # restore the original files

Edits happen in a copy of your project and your checks run there. Nothing in your tree changes until you apply.

A runtime Phonton did not start​

If an Ollama you installed already answers on port 11434, setup uses it. Phonton cannot verify an Ollama it did not start, and an Ollama install can relay cloud models. The TUI plan review says so, and approving the plan is consent for that goal. Headless runs need --allow-unverified-runtime. The receipt records the runtime's origin as unknown.

What to expect​

On our smoke benchmark (small Node tasks, three runs each, scored by hidden tests the run never sees), qwen2.5-coder:3b on a laptop with a 6 GB GPU passed 9 of 12 runs on 0.22.0, each in 10 to 20 seconds. The three misses were one task where the model changed the wrong file and still passed the existing tests; 0.22.2 scopes that kind of goal to the file holding the value. Checks are evidence, not proof, so review the diff. Small models do best with one focused change at a time, and naming the file or function helps.

The same suite runs in CI on a CPU-only Linux runner (local-smoke.yml), on the managed runtime since 0.23. There, qwen2.5-coder:3b passes 12 of 12 small-file runs and 2 of 24 runs on a 339-line and a 28 KB module; Phonton refuses most of the rest rather than handing over an edit that failed checks, but about 4 in 36 runs are marked ready and still fail tests the run never saw. qwen2.5-coder:7b passed 3 of 8 runs on the larger modules. Use a 7B or larger model for multi-function changes, and review the diff.

Phonton will not start a local run when the machine is short on free memory; close other heavy apps and retry.