panel/README.md

panel

An open-source, modular game-server management panel — AMP-class features, distributed Controller/Agent architecture, CEL-driven automation, full instance lifecycle (install → run → update → backup → restore), live dashboard, and CLI.

Written in Go. Single static binary per component. Docker-backed runtime. 26 modules ship in-tree, including the awkward Windows-only games (Conan Exiles, Empyrion, V Rising, Sons of the Forest, DayZ, Soulmask, Project Zomboid, Windrose) under Wine + Xvfb. The agent auto-builds every module's Docker image from its in-tree Dockerfile the first time an instance needs it — there is no manual per-module docker build step.

60-second quickstart

One box, one line. Docker is installed for you if missing; Postgres runs as a managed container; controller + agent become systemd units (Linux) or Scheduled Tasks (Windows), and the local agent is auto-paired over mTLS.

# Linux (Debian/Ubuntu/Fedora/RHEL/Raspbian) — run as root:
curl -fsSL https://raw.githubusercontent.com/dbledeez/panel/main/install.sh | sudo bash

# Windows (Docker Desktop must already be running):
powershell -ExecutionPolicy Bypass -File install.ps1

Then open http://<host>:8080/ and log in with the ADMIN BOOTSTRAP credentials printed in the controller log (or pre-seed via PANEL_ADMIN_PASSWORD). Full options, docker-compose, and from-source paths: docs/INSTALL.md.

                    ┌─────────────────────────┐
                    │       Operator          │
                    │  (browser / panelctl)   │
                    └────────────┬────────────┘
                                 │ HTTPS / gRPC
                    ┌────────────▼────────────┐
                    │       Controller        │
                    │  • Web dashboard :8080  │
                    │  • Agent gRPC    :8443  │
                    │  • Postgres             │
                    │  • Scheduler (cron+CEL) │
                    │  • Event bus + SSE      │
                    │  • Session auth         │
                    └────────────┬────────────┘
                                 │ gRPC bidi (mTLS)
            ┌────────────────────┼────────────────────┐
            │                    │                    │
   ┌────────▼────────┐  ┌────────▼────────┐  ┌────────▼────────┐
   │    Agent A      │  │    Agent B      │  │    Agent C      │
   │  • Docker       │  │  • Docker       │  │  • Docker       │
   │  • Modules      │  │  • Modules      │  │  • Modules      │
   │  • RCON clients │  │  • RCON clients │  │  • RCON clients │
   │  • State tracker│  │  • State tracker│  │  • State tracker│
   └─────────────────┘  └─────────────────┘  └─────────────────┘

---

Documentation

Doc	Covers
docs/INSTALL.md	Installers, docker-compose, from-source, flags reference, systemd, reverse proxy + TLS, firewall
docs/NETWORKING.md	Per-game port table, router port-forwarding, what the panel auto-allocates
docs/PAIRING.md	mTLS agent pairing, multi-host setup
docs/ADMIN.md	First-admin bootstrap, password reset, Steam sign-in, Steam credentials for paid games
docs/BACKUP.md	Backups, pg_dump, disaster-recovery runbook
docs/architecture.md	The real stack, component by component

Table of contents

• Status
• Why panel
• Architecture
• Quick start (dev)
• Production-ish layout
• Modules
• Module manifest
• Wine/Xvfb modules
• panelctl CLI
• HTTP API
• Admin HTTP (out-of-band repair)
• Scheduler
• Mods & Workshop
• Backups
• File manager
• Repository layout
• Data directory layout
• Build
• Working with the embedded UI
• Troubleshooting
• Contributing / writing a new module
• License

---

Status

panel is a working private deployment driving real game servers — not a toy. Feature completeness varies by game; the headline state is:

• ✅ Manifest-driven module plugins (26 shipped — see Modules)
• ✅ mTLS agent pairing: one-time token minted in the UI → agent --pair-token fetches the CA + a signed cert and exits; the agent then connects with mTLS and rogue agents are rejected at the TLS handshake (see docs/PAIRING.md)
• ✅ Agent auto-builds panel-* module images from the in-tree Dockerfiles on first use — pull-through for third-party images, no manual builds
• ✅ Distributed Controller + Agent over gRPC (bidi streams, reconnect + rehydrate, sidecar metadata survives restarts)
• ✅ Docker runtime with bind-mount and named-volume support, host networking, per-instance port allocation
• ✅ Wine + Xvfb pattern for Windows-only dedicated servers (Empyrion, Conan Exiles, V Rising, Sons of the Forest, DayZ, Soulmask, Project Zomboid, Windrose, Dragonwilds)
• ✅ Full state tracking: RCON polling + log-line event matching → typed PlayerEvent + AppState bus
• ✅ Telnet and Source RCON adapters (Minecraft, CS-derived games, Rust, Valheim w/ plugins, 7DTD, Conan, etc.) with redial-on-EOF
• ✅ Scheduler: 6-field cron triggers and CEL event triggers with sustained-for windows
• ✅ Config-file templating with auto-generated secrets; per-game custom Config tabs over ServerSettings.ini / GameUserSettings.ini / serverconfig.xml / properties / JSON
• ✅ File manager: web UI browses container volumes via docker exec + docker cp, bind-mount fallback, path-traversal defended, multi-root per module
• ✅ Update providers: Direct HTTP, GitHub releases, SteamCMD sidecar (with platform overrides for Windows-on-Linux), Steam beta branches (e.g. Conan Legacy UE4, Conan Enhanced UE5)
• ✅ Backup / restore via tar-sidecar containers
• ✅ Postgres persistence (agents, instances, schedules, backups, users)
• ✅ Local auth: argon2id + session cookies, admin bootstrap, change password, optional Sign-in-with-Steam linking
• ✅ Live event bus with SSE stream (Last-Event-ID resume + snapshot) + panelctl watch
• ✅ Template Library: 236-entry game catalog (static/catalog.json, distilled from CubeCoders' MIT templates by tools/amp-distill) with an admin one-click module scaffolder (POST /api/scaffold-module)
• ✅ Update-check (check-only): agent compares installed Steam build id vs app_info (15-min cache) → Updates-tab panel + card badge
• ✅ Prometheus /metrics (admin-token / loopback gated)
• ✅ Mods tab: ARK CurseForge / DayZ Workshop / Conan Workshop / V Rising PlasmaCore / Empyrion scenarios — search, install, version-pin
• ✅ ARK SA cluster registry with cross-server transfer + admin rebind
• 🟡 Cross-platform: Linux works for everything except modules that require nested KVM. Primary dev host is Windows + Docker Desktop.
• ❌ OIDC / WebAuthn / Casbin ACL (local auth only; role = admin / user)
• ❌ SFTP (web FM covers the common case)
• ❌ Host-mode runtime (cgroups v2 / Windows Job Objects)

Why panel

Pterodactyl/Pelican is good but Docker-only and PHP, AMP is closed-source and per-machine, GameServerApp and friends are SaaS. panel is what falls out if you want:

• One static Go binary per component, no Composer / npm / Node runtime on the host.
• Modules as plain YAML — drop a module.yaml in modules/<id>/, restart the agent, the game appears in the dashboard.
• Real distributed multi-host: Controller is one box, Agents run wherever Docker runs.
• A scheduler that can react to the world — "stop the server if it's been empty for 30 minutes", "restart at 3am only on Tuesdays" — rather than just cron.
• First-class Wine/Xvfb support: Windows-only dedicated servers run on Linux Docker hosts without per-game artisanal scripting.
• Out-of-band repair via a token-authed /admin/v1/* HTTP surface — the dashboard breaks, you SSH in, and panelctl admin recreate / set-config / ark-rebind still work.

Designed for someone running ~10 game servers across 2-4 boxes for themselves and a couple of communities, not for a hosting company. No multi-tenancy, no billing.

Architecture

Two long-running components plus optional sidecars.

Controller — ./controller/cmd/controller

• HTTP :8080 (configurable) — dashboard, REST API, SSE event stream
• gRPC :8443 — bidi stream from each Agent
• Postgres for durable state (no SQLite default — explicitly not sharded)
• Embedded migrations, embedded static dashboard (default UI = static/new.html shell + static/rcc.js + static/rcc.css, served hash-stamped with immutable caching via //go:embed; the legacy single-file static/index.html is the frozen "classic" UI, opt-in via the panel_ui=classic cookie)
• Scheduler in-process (cron + CEL)
• Prometheus GET /metrics (admin-token or loopback gated)

Agent — ./agent/cmd/agent

• Dials Controller, stays connected with reconnect + heartbeat
• Loads modules from --modules-dir at boot (default ./modules)
• Drives the local Docker daemon (/var/run/docker.sock or npipe:////./pipe/docker_engine on Windows)
• Spawns short-lived sidecar containers for SteamCMD updates and tar backup/restore
• Per-instance state tracker: RCON polling, log-line tail, event pattern matching → typed events streamed back to Controller

Sidecars (per-game)

• steamcmd-sidecar — SteamCMD with +force_install_dir mounted at the instance's game-content volume; supports +@sSteamCmdForcePlatformType windows for cross-platform Windows depots, beta branch pinning, and Workshop downloads.
• tar-sidecar — exec'd against the volumes for backup/restore so the game container doesn't have to expose its filesystem.

RCON adapters (agent/internal/rcon/)

• source_rcon — Source-RCON wire protocol (Minecraft Java, Conan, Factorio, Palworld, Project Zomboid)
• telnet — Telnet-style password gate then line protocol (7DTD, Empyrion)
• websocket_rcon — Rust's WebRCON
• stdio — attach to the container's stdin (Terraria, Valheim, Vein, Enshrouded)
• be_rcon — BattlEye RCON over UDP (DayZ)
• docker_exec_rcon — exec a CLI inside the container (ARK SA)

All adapters auto-redial on EOF (Conan and Palworld lessons baked in).

Quick start (dev)

Prereqs: Go 1.26+, Docker Desktop (Windows) or Docker Engine (Linux), buf on PATH for proto stubs.

# 1) Postgres (single dev container)
docker compose -f docker-compose.dev.yml up -d

# 2) Generate protobuf stubs (uses remote plugins — no local protoc needed)
buf generate

# 3) Start the controller. First run prints an ADMIN BOOTSTRAP banner
#    with the generated admin password — copy it out of stderr.
go run ./controller/cmd/controller

# 4) Start an agent (any host with Docker + reachable to controller)
go run ./agent/cmd/agent --insecure \
  --modules-dir=./modules \
  --data-root=./data/instances \
  --meta-dir=./data/instance-meta \
  --backup-dir=./data/backups

# 5) Open http://localhost:8080/, log in as admin@panel.local with
#    the bootstrap password.

If you'd rather pre-seed the password instead of reading it from logs:

PANEL_ADMIN_PASSWORD=changeme go run ./controller/cmd/controller

Production-ish layout

This is the layout the maintainer runs. Not prescriptive — adapt freely.

• One Controller box with Postgres co-located. Reverse-proxied behind Nginx Proxy Manager / Caddy / Cloudflare. Public dashboard URL has a real cert; gRPC :8443 is on the LAN only.
• N Agent boxes, each with Docker. Agents don't need to be reachable from the internet — they dial out to the Controller's gRPC port.
• A data/ directory per machine (Controller + each Agent) under whatever path you want. Don't put it in the repo working tree — data/ is gitignored for a reason: it holds per-instance state, RCON passwords, mTLS material, and game saves.
• One systemd unit per binary on Linux, one Service on Windows (NSSM works fine).

The agent is intentionally stateless beyond data/instance-meta/*.json (rehydrated from the Controller on connect). Wipe the meta directory and the agent rebuilds it from the Controller's authoritative DB.

Modules

Game modules live under modules/<id>/ and are picked up by every agent at boot. panel-* images are auto-built by the agent from the module's in-tree Dockerfile the first time an instance needs them (they are never published to a registry); third-party images are pulled. Each ships with a module.yaml and (where needed) an entrypoint.sh and Dockerfile.

Module	Image / build	RCON	Notes
7dtd	panel-7dtd (auto-built)	telnet	Canonical-nim freeze, cluster playbook
ark-sa	acekorneya/asa_server:latest	docker_exec_rcon	Cluster registry, CurseForge mods, transfer flow
barotrauma	panel-barotrauma (auto-built)	log only	Workshop mods
conan-exiles	panel-conan-exiles (auto-built, Wine)	source_rcon	UE4 Legacy / UE5 Enhanced edition picker, Workshop
core-keeper	panel-core-keeper (auto-built)	log only
dayz	panel-dayz (auto-built, Wine)	be_rcon	Steam Workshop manager; requires_steam_login
demo	alpine:3.21	none	Lifecycle / event sandbox
dragonwilds	panel-dragonwilds (auto-built, Wine)	log only
empyrion	panel-empyrion (auto-built, Wine)	telnet	Custom scenario manager, EAH bridge, discovery streamer
empyrion-bridge	multi-stage dotnet (auto-built)	bridge	EWA gateway sidecar
enshrouded	panel-enshrouded (auto-built)	stdio
factorio	panel-factorio (auto-built)	source_rcon
minecraft-bedrock	itzg/minecraft-bedrock-server	log only
minecraft-java	itzg/minecraft-server:latest	source_rcon
palworld	panel-palworld (auto-built)	source_rcon	RCON redial-on-EOF
project-zomboid	panel-project-zomboid (auto-built, Wine)	source_rcon
rust	panel-rust (auto-built)	websocket_rcon	Rust+ port
satisfactory	panel-satisfactory (auto-built)	none
sons-of-the-forest	panel-sons-of-the-forest (auto-built, Wine)	log only
soulmask	panel-soulmask (auto-built, Wine)	log only
steamcmd-test	alpine:3.21 + SteamCMD	none	HLDS smoke test (~930 MB)
terraria	panel-terraria (auto-built)	stdio
v-rising	panel-v-rising (auto-built, Wine)	log only	PlasmaCore mods
valheim	panel-valheim (auto-built)	stdio
vein	panel-vein (auto-built)	stdio
windrose	panel-windrose (auto-built, Wine)	log only

Per-game gotchas live in memory/gotchas.md — read that before debugging any specific game.

Module manifest

Every module is one declarative YAML file. The schema is in pkg/module/manifest.go. A condensed example:

id: my-game
name: "My Game"
version: 0.1.0
supported_modes: [docker]

runtime:
  docker:
    image: itzg/my-game-server:latest        # or local-built image tag
    network_mode: host                       # optional
    browseable_root: /data                    # File-manager root
    browseable_roots:                         # Optional multi-root
      - { name: "Saves", path: /data, hint: "world data" }
      - { name: "Game",  path: /game, hint: "engine binaries" }
    env:                                      # Pre-declare every env key
      MAX_PLAYERS: "20"                       #   you want forwarded;
      ADMIN_PASSWORD: ""                      #   resolve.go filters by
    volumes:                                  #   this map.
      - { type: volume, name: "panel-$INSTANCE_ID-data", container: /data }
      - { target: "$DATA_PATH/logs", container: /log }   # bind mount

ports:
  - { name: game,  proto: tcp, default: 25565, required: true,  env: GAME_PORT }
  - { name: query, proto: udp, default: 25566, required: false, env: QUERY_PORT }
  - { name: rcon,  proto: tcp, default: 25575, internal: true,  env: RCON_PORT }

resources:
  min_ram_mb: 2048
  recommended_ram_mb: 4096

rcon:
  adapter: source_rcon         # or telnet, stdio, websocket_rcon, be_rcon, docker_exec_rcon
  host_port: rcon
  password_secret: RCON_PASSWORD
  password_literal: "fallback"
  commands:
    list:      "list"
    broadcast: "say {msg}"
    kick:      "kick {player}"
    save:      "save-all"
    shutdown:  "stop"

state_sources:
  - type: rcon
    command: list
    every: 60s
    parse:
      kind: regex
      pattern: '(?P<n>\d+)\s+of\s+(?P<m>\d+) players'
      fields: { players_online: 1, players_max: 2 }

events:
  join:  { pattern: "(?P<name>\\S+) joined", kind: join }
  leave: { pattern: "(?P<name>\\S+) left",   kind: leave }
  chat:  { pattern: "<(?P<name>[^>]+)> (?P<msg>.+)", kind: chat }

config_files:
  - path: server.properties
    format: properties
    template: templates/server.properties.tmpl

config_values:                  # Drive the Config tab in the dashboard
  - key: MAX_PLAYERS
    label: "Max players"
    description: "Player slot limit."
    default: "20"
  - key: ADMIN_PASSWORD
    label: "Admin password"
    description: "In-game admin auth."
    default: ""

secrets:
  - { name: admin_token, generated: true }

update_providers:
  - id: stable
    kind: steamcmd
    app_id: "294420"
    install_path: /data
    skip_validate: false        # Set true for app_ids whose validate hangs

Key rules the loader enforces:

• Only env keys pre-declared under runtime.docker.env get forwarded to the container. Adding a config_values entry without the matching env declaration silently drops the value — this trips people up.
• volumes: entries use $INSTANCE_ID and $DATA_PATH substitutions resolved by the agent at create time.
• internal: true ports are not exposed by the allocator but still get their env: substituted.
• update_providers are picked from a dropdown in the UI; multiple providers per module is fine (Conan ships enhanced and legacy).

Wine/Xvfb modules

The Windows-only dedicated servers (Empyrion, Conan, V Rising, Sons of the Forest, DayZ, Soulmask, Project Zomboid, Windrose, Dragonwilds) run inside a Linux container under xvfb-run + wine. The pattern:

panel-<game>:latest    # custom image: debian + wineHQ + xvfb + setpriv
   ↓ docker run
container starts as root
   ↓ stage 1: chown volumes, symlink /Logs → save volume, drop caps
exec setpriv → user `panel`
   ↓ stage 2 (panel user): stamp env into game INI, then
xvfb-run -a wine /game/.../Win64-Shipping.exe ARGS

Per-game entrypoints live under modules/<id>/entrypoint.sh. They all:

• Refuse to run wine as root (wine policy).
• Allocate a fresh X display via xvfb-run -a to avoid colliding with host Xvfb (a bug we hit when host networking is on — see memory/gotchas.md "network_mode: host Xvfb collision").
• Stamp non-empty env values into the game's INI/JSON config files idempotently — empty env means "don't touch this key", so operators who edit the INI directly via the Files tab won't get clobbered.
• Trap SIGTERM and call wineserver -k for clean shutdown.

The shared SteamCMD sidecar runs +@sSteamCmdForcePlatformType windows to fetch the Windows depot onto a Linux volume.

panelctl CLI

Two surfaces in one binary at ./controller/cmd/panelctl.

Operator gRPC — talks to the Controller's gRPC Panel service. Day-to-day server lifecycle.

panelctl agents
panelctl instances [agent-id]
panelctl create   <agent-id> <instance-id> <module-id>
panelctl start    <agent-id> <instance-id>
panelctl stop     <agent-id> <instance-id> [grace-seconds]
panelctl delete   <agent-id> <instance-id> [--purge]
panelctl update   <agent-id> <instance-id> [provider-id]
panelctl rcon     <agent-id> <instance-id> <command...>
panelctl watch    [instance-id]
panelctl backup   create  <agent-id> <instance-id> [description]
panelctl backup   restore <instance-id> <backup-id>
panelctl modules

Admin HTTP — out-of-band repair. Drives the recreate dance (env-config edits, ARK cluster rebinds, generic recreate with mount overrides) without needing a dashboard session. Auth is loopback-bypass when run on the Controller host, OR a long-lived token at <data-dir>/admin-token (auto-generated on first start, mode 0600).

panelctl admin token-show
panelctl admin status
panelctl admin instances
panelctl admin ark-rebind <instance-id>
panelctl admin set-config <instance-id> KEY=VALUE [KEY=VALUE ...]
panelctl admin del-config <instance-id> KEY [KEY ...]
panelctl admin recreate   <instance-id> [--mount CONTAINER_PATH=HOST_PATH ...]

Common flags:

--http URL              Default http://localhost:8080
--admin-token TOK       Or $PANEL_ADMIN_TOKEN, or read from <data-dir>/admin-token
--data-dir DIR          Default ./data

HTTP API

All /api/* require an authenticated session cookie. /api/login is the only unauthenticated endpoint.

Method	Path	What it does
POST	/api/login	Issues session cookie
POST	/api/logout	Revokes session
GET	/api/me	Current user + role
POST	/api/change-password	Rotate own password
GET	/api/agents	Live-connected agents
GET	/api/modules	Union of modules across agents
GET	/api/instances[?agent_id=]	Persisted instance rows
POST	/api/instances	Create
POST	/api/instances/{id}/start	Start
POST	/api/instances/{id}/stop[?grace=N]	Stop (container retained)
DELETE	/api/instances/{id}[?purge=true]	Delete instance + optionally volumes
POST	/api/instances/{id}/rcon	Ad-hoc RCON exec
POST	/api/instances/{id}/update	Trigger update provider
GET	/api/instances/{id}/files[?path=]	List
GET	/api/instances/{id}/files/read?path=	Read file
POST	/api/instances/{id}/files	Write file
DELETE	/api/instances/{id}/files?path=	Delete
GET	/api/instances/{id}/backups	List backups
POST	/api/instances/{id}/backups	Create backup
POST	/api/instances/{id}/backups/{bkpId}/restore	Restore
DELETE	/api/instances/{id}/backups/{bkpId}	Remove DB row
GET	/api/schedules[?instance_id=]	List schedules
POST	/api/schedules	Create (cron or event trigger)
DELETE	/api/schedules/{id}	Remove
POST	/api/schedules/{id}/{enable|disable}	Toggle
GET	/api/events[?instance=&agent=]	SSE stream (supports Last-Event-ID / ?last_event_id= resume)
GET	/api/template-catalog	236-entry game template catalog
POST	/api/scaffold-module	Admin: scaffold a new modules/<id>/ from a catalog entry
GET	/api/instances/{id}/update-check	Last cached update-check result
POST	/api/instances/{id}/update-check	Run a check (installed build vs Steam app_info)
GET	/api/update-checks	All instances' update-check state
GET	/metrics	Prometheus metrics (outside /api; admin token or loopback only)

Admin HTTP (out-of-band repair)

Sits outside the session middleware. Auth: loopback-bypass OR X-Panel-Admin-Token: <hex> header (constant-time compared against <data-dir>/admin-token, auto-generated mode 0600).

Method	Path	What it does
GET	/admin/v1/health	{ok, version, hostname, loopback, time}
GET	/admin/v1/instances	All instances + config_values + assigned_ports + ark_cluster_id
POST	/admin/v1/instances/{id}/ark-cluster-rebind	Re-runs the cluster mount recreate against the registry's current cluster id
POST	/admin/v1/instances/{id}/config	Body {"set":{...},"delete":[...]} — merges config_values, runs the env-config recreate dance
POST	/admin/v1/instances/{id}/recreate	Body {"mount_overrides":{...}} — generic stop → delete-preserve → create → start

The point of this surface: when the dashboard is broken, you can SSH in, hit /admin/v1/* with curl, and rescue an instance.

Scheduler

Two trigger kinds.

cron — robfig/cron v3, 6-field (seconds resolution):

{
  "instance_id": "mc-1",
  "trigger_kind": "cron",
  "cron_spec": "0 0 3 * * *",
  "action": {"type": "instance", "op": "restart", "grace_seconds": 30}
}

event — CEL expression over the live event bus, with optional sustain-for window. The classic example: stop the server if it's been empty for 30 minutes.

{
  "instance_id": "mc-1",
  "trigger_kind": "event",
  "event_spec": "event_type == \"app_state\" && players_online == 0",
  "event_sustained_seconds": 1800,
  "action": {"type": "instance", "op": "stop", "grace_seconds": 60}
}

CEL input fields available at evaluation time:

event_type      "join" | "leave" | "chat" | "app_state" | "log_match" | …
instance_id     string
agent_id        string
players_online  int
players_max     int
uptime_seconds  int
instance_status "running" | "stopped" | "starting" | "exited"
exit_code       int     (only on exit events)
player_kind     "join" | "leave" | "chat"
player_name     string
player_id       string
player_detail   string
log_line        string  (raw matched line)
log_stream      "stdout" | "stderr"

Action types:

{"type": "rcon",     "command": "saveworld"}
{"type": "instance", "op": "start" | "stop" | "restart", "grace_seconds": 30}

Mods & Workshop

Per-game module integrations live in controller/cmd/controller/<game>mods.go. The current set:

• ARK SA — CurseForge API, search + install + version pin. Cluster registry tracks shared ClusterDirOverride mounts; rebind via Admin HTTP rewrites all member instances atomically.
• DayZ — Steam Workshop item search, mod-priority load order, deploy via SteamCMD sidecar.
• Conan Exiles — Steam Workshop with requiredtags[]=Enhanced|Legacy filtering driven by the instance's EDITION config_value, so UE5 servers don't see UE4-only mods. ServerModList= stamped into ServerSettings.ini and ConanSandbox/Mods/modlist.txt.
• V Rising — PlasmaCore + community .pak drops, install path pinned per-instance.
• Empyrion — custom scenario manager (zip uploads + scenario selector), discovery streamer for in-game POI markers.
• Barotrauma — Steam Workshop subscriptions written into the client/server WorkshopMods/ tree.

Volumes are reused across instances where it's safe: e.g. all Conan instances on a given agent share panel-conan-workshop for downloaded Workshop content, mounted at the game's expected /game/steamapps/workshop path.

Backups

POST /api/instances/{id}/backups spawns a tar-sidecar container that mounts the same volumes as the game container (read-only on the data side) and writes a tarball to the agent's --backup-dir (default ./data/backups/<instance-id>/). Restore reverses the flow, with the game container stopped first.

Backups are timestamped and labeled with an optional description, and can be scheduled (scheduler action {"type":"backup"}). Retention is manual right now — list, pick, delete. See docs/BACKUP.md.

File manager

Three operations against an instance's browseable_root (or any browseable_roots[i].path):

• List (GET /files?path=) — docker exec ls -la --full-time parsed into typed entries.
• Read (GET /files/read?path=) — docker cp to a temp file, stream back. Caps at a sane size; bigger files prompt for download.
• Write (POST /files) — temp file, docker cp, chown.
• Delete (DELETE /files) — docker exec rm. Symlinks are resolved with realpath first; path traversal blocked at the manifest's declared roots.

Bind-mounted volumes get a fast-path that skips docker cp entirely.

Repository layout

panel/
├── agent/                       # Per-host agent
│   ├── cmd/agent/               # Binary entrypoint
│   └── internal/
│       ├── dispatch/            # gRPC reply handlers — one file per concern
│       ├── module/              # Module loader + resolver
│       ├── rcon/                # Source/Telnet/MC RCON adapters
│       ├── runtime/             # Docker driver
│       ├── state/               # Per-instance state tracker
│       └── updater/             # SteamCMD/HTTP/GitHub sidecars
├── controller/
│   ├── cmd/
│   │   ├── controller/          # Web + gRPC server
│   │   │   └── static/          # Embedded dashboard (new.html + rcc.js/rcc.css;
│   │   │                        #   index.html = frozen classic; catalog.json)
│   │   └── panelctl/            # CLI
│   └── internal/
│       ├── db/                  # SQL migrations + queries
│       ├── scheduler/           # cron + CEL
│       └── eventbus/            # SSE + agent fanout
├── pkg/module/                  # Shared module-manifest parsing
├── proto/panel/v1/              # gRPC contract (buf-managed)
├── modules/                     # 26 modules — drop-in YAML + scripts
│   └── <game>/
│       ├── module.yaml
│       ├── Dockerfile           # if a custom image is needed
│       └── entrypoint.sh
├── docs/                        # INSTALL, NETWORKING, PAIRING, ADMIN, BACKUP, architecture
├── docker-compose.dev.yml       # Postgres only
├── go.work                      # Multi-module workspace
└── memory/                      # Maintainer's working notes (not user docs)

Data directory layout

data/ is gitignored. On a fresh install you'll grow:

data/
├── ark-clusters.json            # Operator-curated cluster config (TRACKED)
├── arkcluster/                  # Per-cluster shared state
├── backups/                     # Tarballs from tar-sidecar
├── ca/                          # Internal CA private key + server cert (mTLS)
├── certs/                       # Issued agent certs
├── instance-meta/<id>.json      # Per-instance metadata (rcon password, paths)
└── instances/<id>/              # Bind-mount volumes per instance
    ├── logs/
    └── ...

admin-token lives at <data-dir>/admin-token (mode 0600, auto-generated on first controller start).

Build

Multi-module Go workspace. The root go.work ties together agent/, controller/, pkg/, proto/.

# Generate proto stubs (once after a clone, and after any .proto change)
buf generate

# Linux/Mac:
go build -o bin/controller ./controller/cmd/controller
go build -o bin/agent      ./agent/cmd/agent
go build -o bin/panelctl   ./controller/cmd/panelctl

# Windows (PowerShell):
go build -o controller.exe .\controller\cmd\controller
go build -o agent.exe      .\agent\cmd\agent
go build -o panelctl.exe   .\controller\cmd\panelctl

Cross-compile a Linux agent from a Windows host:

$env:GOOS="linux"; $env:GOARCH="amd64"
go build -o bin/agent-linux ./agent/cmd/agent
Remove-Item Env:GOOS, Env:GOARCH

Working with the embedded UI

The default dashboard is split across three embedded files in controller/cmd/controller/static/: new.html (shell), rcc.js (all feature JS), rcc.css (design system) — plain JS + CSS, no build step. /rcc.js and /rcc.css are served hash-stamped with immutable cache headers, so browsers pick up new builds without cache pain. The legacy single-file index.html (~24k lines) is the frozen classic UI — it carries a freeze notice, is only served on panel_ui=classic, and no new features land there.

• Everything is embedded via //go:embed, so editing any static file requires a controller rebuild + restart.
• Before any controller build that touched the UI, run the parse gate: node controller/.test/gate-parse.mjs — a single JS syntax error blanks the whole dashboard and go build cannot catch it.
• Per-game Config tabs are keyed by module id with a small schema (groups[].fields[] with {prop, label, type, options, hint}) that the form renderer turns into INI/JSON read+write against the game's config file via the file manager API.
• Module icon/art/accent + readiness regex are manifest-driven: appearance: and ready_pattern: in module.yaml, served via GET /api/modules (inline maps remain as fallback).
• The shared form for config_values (env-driven settings) is rendered on the dashboard's Config Values sub-tab; this is separate from the game-specific config-file editors above.

Troubleshooting

• "agent connected but no modules" — agent's --modules-dir is wrong, or the manifest failed to parse. Check the agent's stderr.
• "create succeeded but instance never starts" — for SteamCMD-driven modules, the install volume is empty. Click Update once first; the entrypoints exit EX_CONFIG (78) with a hint when they can't find the dedicated server binary.
• "environment variable I added to module.yaml never reaches the container" — you forgot to pre-declare the key under runtime.docker.env. resolve.go silently filters unknown keys.
• "dashboard broken, can't recover" — panelctl admin recreate <instance-id>. If the controller is also down, the data directory
  • Postgres dump is enough to spin up a replacement.
• Wine modules: container stays "running" but the game never binds ports — usually a top-level launcher stub problem. Invoke the deep Binaries/Win64/<Game>Server-Win64-Shipping.exe directly (see Conan's entrypoint for the canonical pattern).
• network_mode: host + Xvfb collision — when two host-net containers both run xvfb on the same display number you'll see "Server is already active for display" in stderr. Always use xvfb-run -a to auto-allocate.

The maintainer's running list of caught-on-fire-once notes lives in memory/gotchas.md.

Contributing / writing a new module

1. Create modules/<id>/module.yaml. Easiest is to copy the closest existing module — Source-RCON game? Start from Conan's. Wine? V Rising's. Plain Linux dedicated server? Factorio's.
2. If you need a custom image, add a Dockerfile next to it. Include setpriv, xvfb-run, and wine for Windows-on-Linux modules.
3. Pre-declare every env key that any config_values entry maps to inside runtime.docker.env. Empty defaults are fine.
4. Restart the agent — modules load on boot. The new game appears in the dashboard's create-instance dropdown.
5. Test the lifecycle end-to-end: create → update → start → connect a client → stop → delete (no purge) → recreate → verify world saves survived.
6. If the game has a Workshop / mods scene, add a <game>mods.go to controller/cmd/controller/ and surface it in the dashboard's per-game Mods tab.

PRs welcome. The maintainer's editorial preferences are recorded in memory/ux_rules.md — read it before redesigning any tab.

License

Apache-2.0 — see LICENSE.