Using local models

AgentsVille can run NPCs on models you host yourself — on the same machine as the Village, on a GPU box on your network, or behind a gateway — alongside hosted Providers. Anything that serves the OpenAI chat completions API works: Ollama, LM Studio, vLLM, llama.cpp's server, LocalAI, hosted APIs such as Gemini, DeepSeek, xAI, GLM and Qwen, and routers such as OpenRouter, Requesty and MixRoute.

Only the Operator can add a Provider, because AgentsVille will send requests to whatever address it is given, including addresses on your private network.

Add the Provider

1. Start the model server. For Ollama that is ollama serve and ollama pull llama3.2; LM Studio has a Start server switch.

2. Open Settings → Providers and choose a preset (table below), or Another OpenAI-compatible server for anything not listed.

3. Check the Address. It must be the address the Village server reaches, ending in /v1:

   Where the model runs	Address
   Same machine, AgentsVille run from source	http://localhost:11434/v1
   Same machine, AgentsVille in Docker	http://host.docker.internal:11434/v1
   Another machine on the LAN	http://192.168.1.20:11434/v1
   A hosted gateway	https://openrouter.ai/api/v1

   Inside Docker, localhost is the container, not your machine. The bundled docker-compose.yml maps host.docker.internal to the host for the server service, on Linux as well as Docker Desktop. The model server must listen on an interface the container can reach, not only 127.0.0.1.

4. Paste an API key if the server needs one. Local servers usually do not.

5. Add Provider, then Test. A working Provider answers with the models it serves.

Presets

Every preset is an OpenAI-compatible Provider with the address filled in. Anthropic and OpenAI are not here: they are built in, with their own clients.

A few need editing before Add Provider:

• Cloudflare AI Gateway: replace YOUR_ACCOUNT_ID and YOUR_GATEWAY_ID. The button stays disabled until you do. Models are named provider/model, for example openai/gpt-5-mini.
• AWS Bedrock: change us-east-1 to your region, and use an Amazon Bedrock API key. Only the models Bedrock serves through its OpenAI-compatible Mantle endpoint are reachable this way.
• Qwen: the address above is the international region. Mainland China accounts use https://dashscope.aliyuncs.com/compatible-mode/v1.
• Z.ai (GLM): accounts on the China platform use https://open.bigmodel.cn/api/paas/v4.

Not every gateway lists its models at /models. When Test fails but the key is right, type the model name on the NPC's form and send a message: chat can work where the model list does not.

Use it

A Provider on its own does nothing. Point something at it:

• One NPC: pick the Provider on the NPC's form and type or pick a model.
• The whole Village: make it the Default model.
• A Level: map it to Beginner in Model Levels. A small, fast local model is a good Beginner: it answers easy questions for free, and it rates every message for NPCs that work at several Levels.

A typical mix:

What works differently

• Thinking has no effect on OpenAI-compatible Providers; there is no common switch for it. A model that reasons by default still shows its reasoning.
• Tools need a model that supports tool calling. Many small local models do not, or do it poorly, and an NPC on one of them may ignore its Tools.
• Memory embeddings need an embedding model, not a chat model. Pull one — ollama pull nomic-embed-text — and choose it in Settings → Memory embeddings. With none chosen and no OPENAI_API_KEY, Memory is ranked by keyword, which works but recalls less precisely.
• Removing a Provider is refused while an NPC, the Village default or a Level still uses it. The two built-in Providers cannot be removed.

Troubleshooting

• Test says it could not be reached. The address is wrong from the server's point of view — see the Docker note above — or the model server is not listening on all interfaces. Ollama listens on 127.0.0.1 unless OLLAMA_HOST=0.0.0.0 is set.
• Replies fail with a 404 from the Provider. The model id is not one the server has. Test lists the ones it has.
• Replies are slow to start. The model server loads a model into memory on first use; the second message is fast.

Where to go next

• Choosing models
• Configuration