Docs

Local models

Ollama, llama.cpp (llama-server) and MLX (mlx_lm.server) as first-class providers. No account, key or internet, and nothing metered.

LEVEL11 supports a model server on your own machine as a first-class provider. No account, no API key, no internet. Requests go only to the configured server address, never to the LEVEL11 gateway, and nothing is metered or billed. All three backends serve an OpenAI-compatible POST /v1/chat/completions, so one local provider in the app covers them.

At a glance

A few notes:

• Ollama defaults to a 4096-token context. For agent work start it with OLLAMA_CONTEXT_LENGTH=16384 ollama serve and set the same value as Context length in LEVEL11.
• llama.cpp needs --jinja for OpenAI-style tool calls. Without it the server answers 400 and LEVEL11 falls back to the text protocol automatically. Match -c with the Context length setting.
• MLX downloads the model from Hugging Face on first use and then works offline. MLX and llama.cpp share port 8080 by default, so change one --port if both run.

Connect in LEVEL11

1. Open Settings (⌘,) → Local model.
2. Choose the backend. Leave Server URL empty to use the default address.
3. Press Test connection. It reads the server's model list into the Model name dropdown (typing works too); for Ollama it also shows tool support per model, and for llama.cpp the build number and n_ctx. If the server is down, the dialog shows the command that starts that backend.
4. Pick a model and press Use this model, or choose Local model in the model picker. The composer and status bar then show e.g. "Ollama · qwen3.5:4b".
5. Tool calling can usually stay on Auto. Set Context length to match the model you started on the server.

An agent preset can override its model with local so that only that preset uses the local model. A custom model ID of local:<name> uses a different model on the same server than the one in Settings.

How it works

• Streaming: text flows straight into the chat. reasoning_content deltas and raw <think>…</think> tags go to the Thinking panel, not the chat body.
• Tool calling (Auto): OpenAI-style tools are sent first. If the server refuses with 400/422, that server-and-model pair is remembered as text-only and the request is retried at once. In the text protocol the tool catalogue and a call format go into the system prompt, and ```tool_call blocks (or <tool_call> tags) are parsed out of the reply. Native and Text force one mode.
• Approvals and checkpoints: Builder-mode approvals, checkpoints and revert work exactly as with cloud models.
• Fitting the context: local windows are small, so tool results are clamped and the oldest exchanges are dropped (call and result together) to fit the Context length.
• Isolation: the local provider only ever contacts the configured URL. It is independent of account state, gateway and API keys, and produces no usage or billing data.

Recommended models and troubleshooting

With 8–16 GB of memory start with qwen3.5:4b or Qwen3-4B / Qwen2.5-Coder-7B 4-bit; with 24–32 GB, Qwen3-14B/32B 4-bit, Gemma 4 27B or DeepSeek-Coder-V2-Lite; with more, Qwen3-Coder, Qwen3.8-27B or Llama 3.3 70B 4-bit. Whether tool calling works depends on both the model and the server. Models of 4B and below occasionally break the text format, but the parser accepts the common variants.