Local LLM Setup

Robota works with any local inference server that speaks the OpenAI-compatible API. This guide covers Ollama, LM Studio and the llama.cpp server. In Robota these all use the same provider type, gemma, shown in setup as Ollama / LM Studio / llama.cpp.

An API key only if your server requires one. These servers accept requests without a key by default; if you enable server authentication, enter its token as described below. Inference runs on the server you point Robota at, so prompts and code go no further than that server for model calls. Enabled tools can still reach the network on their own: WebFetch and any MCP servers you connect, including a web search server, send their requests wherever they are directed.

Quick Start

1. Start your local model server (see the options below).

2. Configure the robota CLI to use it:

   robota --configure

   Copy

   Choose Ollama / LM Studio / llama.cpp (gemma), then answer the two prompts. Press Enter to accept a default:

   Prompt	Default	What to enter
   Base URL	http://localhost:1234/v1	Your server's URL, including /v1
   Model	supergemma4-26b-uncensored-v2	The model name exactly as your server lists it

On a first run with no provider configured, robota offers the same setup: answer "No — use a local model", and after a short LM Studio guide it asks these two prompts. App setup asks for the same endpoint and model, without a key prompt.

To configure without prompts (for example in a setup script):

robota --configure-provider local --type gemma \
  --base-url http://localhost:11434/v1 --model llama3.2 --set-current

The profile is saved in <user-state>/settings.json, so you configure it once. Run robota --configure again, or use /provider inside a session, to change the provider, URL or model.

Servers with authentication enabled

Use /provider edit <profile> inside a terminal session, or Settings → Providers & Models → Edit… in the App, to enter the optional server API key. Leaving a previously configured masked key blank keeps it unchanged; leaving a keyless profile's field blank keeps it keyless.

For a terminal setup script, set an environment variable containing your server token and reference it without putting the token on the command line:

robota --configure-provider local --type gemma \
  --base-url http://localhost:1234/v1 --model your-installed-model \
  --api-key-env LOCAL_MODEL_API_KEY --set-current

Both methods retain your endpoint and use the supplied token for requests and connection tests. Robota supplies the client library's placeholder internally only when no server credential is configured. This local setup does not perform Ollama cloud sign-in.

Option 1: Ollama

Ollama downloads and serves models and exposes an OpenAI-compatible API.

Install and start

# macOS / Linux
curl -fsSL https://ollama.com/install.sh | sh
 
# Pull a model (examples)
ollama pull llama3.2          # small and fast
ollama pull qwen2.5-coder     # strong at TypeScript/Python
 
# Verify the server is running
curl http://localhost:11434/v1/models

Ollama listens on http://localhost:11434.

Configure the CLI

Run robota --configure, choose Ollama / LM Studio / llama.cpp (gemma), and enter:

• Base URL: http://localhost:11434/v1
• Model: the model name exactly as ollama list shows it (e.g. llama3.2)

Models for coding

Larger models produce better results but need more memory and run slower.

Option 2: LM Studio

LM Studio is a desktop app for downloading and running models, with a built-in local API server.

Install and start

1. Download LM Studio from lmstudio.ai.
2. Search for and download a model (for example a Gemma, Llama or Qwen Coder model).
3. Start the local server from the Developer tab.

The server runs on http://localhost:1234 by default, which is also the gemma provider's default base URL.

Configure the CLI

Run robota --configure, choose Ollama / LM Studio / llama.cpp (gemma), and enter:

• Base URL: http://localhost:1234/v1 (the default)
• Model: the model name exactly as LM Studio shows it for the loaded model

Option 3: llama.cpp server

If you build and run llama.cpp yourself:

./llama-server -m models/your-model.gguf --port 8080

Then run robota --configure, choose Ollama / LM Studio / llama.cpp (gemma), and enter http://localhost:8080/v1 as the base URL, then enter your installed model name.

Using a local model from code

The same provider is available to your own code as GemmaProvider:

import { GemmaProvider } from '@robota-sdk/agent-provider/gemma';
 
const provider = new GemmaProvider({
  apiKey: 'ollama', // placeholder; use your server's token if it requires one
  baseURL: 'http://localhost:11434/v1',
  defaultModel: 'llama3.2',
});

See Providers for its options.

Troubleshooting

Authentication rejected (HTTP 401 or 403)

If your server requires authentication, run /provider edit <profile> or use the App's provider Edit action to enter or correct its optional API key. If the profile uses $ENV:LOCAL_MODEL_API_KEY, ensure that variable is set in the environment running Robota. A local-compatible endpoint can require authentication; Robota does not assume every such server ignores keys.

"Connection refused" or "Network error"

• Check the server is running: curl http://localhost:11434/v1/models (Ollama) or curl http://localhost:1234/v1/models (LM Studio).
• Check the port in your base URL matches the server.
• Some servers bind to 127.0.0.1 only; try http://127.0.0.1:<port>/v1.

Model not responding

• The model name in your profile must match the model loaded in your server exactly.
• For Ollama, ollama list shows the names; LM Studio shows the name of the loaded model.

Slow responses

• Local models are slower than hosted APIs, especially on CPU.
• Smaller quantized models (e.g. q4_k_m variants) run faster.
• Enable GPU acceleration in Ollama or LM Studio if available.

Tool calling not working

Robota's agent works through tool calls. Some local models do not support the OpenAI function-calling format, and tool calls then fail or never happen. Use a model whose documentation says it supports tool or function calling (for example qwen2.5-coder or llama3.2).

Tips

• Context windows are smaller. Many local models have far smaller context windows than hosted models. Use /compact when the context fills up.
• Quality varies with size. Small models handle straightforward tasks; use larger ones for complex refactoring.
• Offline. Once the model is downloaded, model calls need no internet connection; tools such as WebFetch and a web search MCP server still do.
• Web search. A local model has no search of its own. Add a search MCP server with robota mcp add exa or robota mcp add parallel (neither needs an API key; your queries go to that service), or answer the question on the first interactive start.

Related

• CLI Reference — Connect a provider
• Providers
• Context Management