Logs and observability

AgentsVille emits wide events — one structured record per request, carrying what happened rather than a scatter of lines you have to reassemble. Tool calls, Sandbox executions and Memory retrievals go through the same path rather than a parallel mechanism.

Reading them on the host

With no extra configuration, events go to stdout, which Docker captures:

docker compose logs -f serverdocker compose logs -f webdocker compose logs -f caddydocker compose logs -f sandbox-broker

Compose caps container logs at 10 MB across 3 files per service, so a chatty service cannot fill the disk and take MySQL down with it. That cap is also why stdout alone is not a retention strategy: on a busy Village, yesterday is already gone.

In development, with LOG_DRAIN unset, events are written to the filesystem instead, which is usually easier to grep than a terminal.

Sending them somewhere

LOG_DRAIN is a comma-separated list of destination names. Set it in the root .env and both the API server and the web app ship to every destination named:

LOG_DRAIN=axiom,sentry

The accepted names, in full:

axiom, better-stack, clickhouse, datadog, fs, hyperdx, loki, memory, otlp, posthog, sentry.

Names are trimmed and lowercased, so Axiom , sentry is the same list. An empty value means no destination: stdout only in production, the filesystem drain in development.

Several at once is a normal configuration — search in one, alerting in another.

Credentials for each destination

Each destination reads its own vendor credentials directly from the environment. Axiom wants AXIOM_API_KEY and AXIOM_DATASET; the others follow the same pattern with their own names.

Those variables are deliberately absent from AgentsVille's own configuration schema. Only LOG_DRAIN is validated. If every vendor key had to be declared there, supporting a new destination would mean editing a schema and adding a dependency, and the plug-and-play property would be a fiction. The reasoning is in ADR 0004.

The practical consequence for you: a missing or misspelled vendor key is not caught at boot. The destination is configured, the events go nowhere useful, and nothing says so. After pointing a Village at a new destination, confirm something arrived there before you rely on it.

A wrong name stops the server

An unrecognised destination name throws at startup rather than falling back to no destination at all. A typo that silently ships logs nowhere would only be discovered mid-incident, which is the worst possible time.

The error names the offending value and lists every name that is accepted, so a server that will not start after an edit to LOG_DRAIN tells you exactly what is wrong.

How events are delivered

When a destination is configured, events go through a batching pipeline rather than one HTTP request per event:

• Batched at 50 events or every 5 seconds, whichever comes first.
• Retried up to 3 times with exponential backoff, starting at 1 second.
• Batches that still fail are dropped, with a line on stderr saying how many and why. That line goes to the container log, so it is visible in docker compose logs even when the destination is the thing that is broken.

Anything still buffered is flushed when the server shuts down.

Before you point this at a vendor

Anything shipped to a hosted destination leaves your host permanently. AgentsVille masks email addresses when it identifies the Player on an event, and nothing else is redacted. Fields set on an event and the text of error messages go out as they are.

Review what your Village actually emits before enabling a hosted destination, particularly if your Players put anything sensitive into Conversations. If you want the events without the third party, fs writes them locally and otlp sends them to a collector you run.

Health checks

The API server answers two paths, and only one of them is a real check:

• GET /health runs SELECT 1 against MySQL. It answers {"status":"ok","service":"agentsville"} or, when the database is unreachable, a 503 with the code DATABASE_UNREACHABLE. This is what the container health check uses, and what an external monitor should use.
• GET / answers OK without touching anything. It reports healthy on a server whose database has gone, which is why it is not the health check.

Point your uptime monitor at https://api.example.com/health.

Next

• Troubleshooting — the failures worth recognising by name.
• Configuration — where LOG_DRAIN sits in the file.