300 lines
16 KiB
Markdown
300 lines
16 KiB
Markdown
# 7 Days to Die — Cluster Canonical `.nim` ID Freeze
|
||
|
||
> How the panel stops a 7DTD cluster's **shared player saves** from corrupting on
|
||
> cross‑server transfer (items "morphing" into other items, characters resetting
|
||
> to level 1 / naked). Read this before touching cluster `.nim` files, the 7dtd
|
||
> `entrypoint.sh` freeze guard, the canonical, or regenerating a clustered world.
|
||
>
|
||
> Author: deployed live 2026‑06‑13 on cluster `cl_37e3a7f72cdc` (Refuge).
|
||
> Companion code: `modules/7dtd/entrypoint.sh` (the guard),
|
||
> `modules/7dtd/nim-freeze/rebuild.py` (regenerate the canonical), and
|
||
> `modules/7dtd/nim-freeze/remap_7dt.py` + `REMAP_RUNBOOK.md` (new‑world deco remap).
|
||
|
||
---
|
||
|
||
## ⚠️ CORRECTION / SECOND HALF OF THE PROBLEM (2026‑06‑13, later same day)
|
||
|
||
The freeze below fixes **inventory** (`.ttp`) transfers — correct and live. But it is
|
||
**NOT sufficient for a freshly‑generated DIFFERENT world**, and that's why insane/pvp
|
||
came up "online but unjoinable" with a `DecoManager.TryAddToOccupiedMap` NullReference.
|
||
|
||
Root cause (proven): a world also bakes its **decorations** into `decoration.7dt`
|
||
(and structures into `multiblocks.7dt`) **by numeric block id**, using the engine's
|
||
**native runtime ids at generation time**. The cluster's current runtime (RefugeBot
|
||
v1.2.39) assigns the tree‑deco band at ids **21895+**, while the season10‑anchored
|
||
canonical assigns the same trees at **24141+** (season10's world was baked *before* a
|
||
RefugeBot rebuild shifted the band). Stamping the canonical over a new world's `.nim`
|
||
makes its native‑baked deco ids resolve to a **null Block** → NRE → world‑load
|
||
coroutine dies → never `GameStartDone`. season10/creative load clean only because their
|
||
deco was baked when runtime == canonical.
|
||
|
||
**Fix for a new world = a one‑time per‑world deco remap** (`remap_7dt.py`): translate
|
||
`decoration.7dt`/`multiblocks.7dt` block ids `native → name → canonical`, preserving the
|
||
high/rotation bits, then stamp the canonical. The engine **preserves** loaded deco ids on
|
||
save (only writes native at *generation*), so the remap is **stable across reboots**.
|
||
`.7dt` format: `u8 ver=6, u32 count, count×17B recs; id = u16@+12 & 0x7FFF`. The cluster
|
||
runtime native table is identical across all instances (one capture is authoritative).
|
||
**Full step‑by‑step: `nim-freeze/REMAP_RUNBOOK.md`.** insane (Yixacove) + pvp (Tuxeno)
|
||
were fixed this way and verified: `GameStartDone`, `NRE=0`, deco band 24141, on canon.
|
||
|
||
Permanent removal of the per‑world remap step: pin block ids explicitly in a top‑priority
|
||
modlet (so runtime native == canonical) or freeze the RefugeBot build season10 was baked
|
||
under. Until then, every NEW clustered world needs this one remap.
|
||
|
||
---
|
||
|
||
## TL;DR
|
||
|
||
* 7DTD stores a player's **inventory by numeric ID, not by name**. Each *world*
|
||
bakes its own `name <-> id` lookup tables: `itemmappings.nim` and
|
||
`blockmappings.nim` in that world's save dir.
|
||
* Our cluster **shares one player save folder** across servers that each run a
|
||
**different world**. The shared `.ttp` carries one world's numbers; every other
|
||
world decodes them through a **different** table → glass block reads as car
|
||
hood, etc. Severe mismatches fail to load → fresh level‑1 character, written
|
||
back over the shared save = **permanent wipe**.
|
||
* **Fix:** build ONE *complete, season10‑anchored* canonical id table and **freeze
|
||
it onto every world in the cluster**. Because it preserves season10's exact ids
|
||
and contains every block/item the mods can produce, every world decodes the
|
||
shared `.ttp` identically and the engine never appends → it stays frozen.
|
||
* A small **entrypoint guard** re‑stamps the canonical on every boot, so it's
|
||
self‑healing and survives recreates/regens.
|
||
|
||
---
|
||
|
||
## The problem (root cause, proven by forensics)
|
||
|
||
1. A player's `.ttp` inventory region is a stream of **2‑byte numeric ids** — no
|
||
item/block name strings (names only appear later, for name‑keyed progression
|
||
like perks/recipes, which is why **level/skills survive** a transfer but
|
||
**inventory/placed‑blocks do not**).
|
||
2. Those numbers mean something only relative to the **per‑world** `.nim` tables.
|
||
The engine logs `INF Block IDs with mapping` / `INF ItemIDs from Mapping` on
|
||
every world load — it resolves stored ids → names **through that world's
|
||
`.nim`**.
|
||
3. The `.nim` files are **NOT shared** — only the `Player/` folder is symlinked to
|
||
`/cluster/Player`. Each world baked its table independently at genesis, so the
|
||
same id maps to a different name per world. Live proof on the cluster:
|
||
* block id **9243** = `steelShapes:cube` on season10 but `awningShapes:cube`
|
||
on the others.
|
||
* item id **66/67** = the inverse swap `meleeHandPlayer <-> meleeHandZombie01`.
|
||
4. **Why identical mods don't save you:** ids for un‑pinned/modded content are
|
||
**auto‑assigned at world genesis in encounter order and frozen into that
|
||
world's `.nim`**. Two worlds generated at different times/states freeze
|
||
*different* assignments even from byte‑identical XML. So mod‑syncing, mod
|
||
symlinks, identical load order — all irrelevant. The divergence lives in the
|
||
**saves** volume, not the mods.
|
||
5. The "naked / level 1" variant: the destination's table is missing ids the
|
||
`.ttp` references (e.g. season10 had 1888 item ids; a fresh RWG world's table
|
||
had ~133). Unresolvable ids are dropped; enough drops and the character loads
|
||
empty/level 1. Because the save is shared and **written back**, the wipe is
|
||
permanent (`.ttp` *and* `.ttp.bak` both overwritten).
|
||
|
||
This is the unsupported corner of 7DTD: **The Fun Pimps disable cross‑world
|
||
characters for exactly this reason.** Sharing one inventory across servers running
|
||
*different worlds* only works if every world uses an identical, frozen id table.
|
||
|
||
---
|
||
|
||
## The fix: a complete, frozen canonical table on every world
|
||
|
||
### What "canonical" means here
|
||
|
||
A single pair of `.nim` files that is:
|
||
|
||
* **Anchored to season10** — every id season10 currently uses is preserved
|
||
**byte‑for‑byte**. season10 authored the 87 shared inventories and has the
|
||
richest/oldest table, so the existing saves already match it. This makes the
|
||
freeze **non‑destructive to season10's world and all existing inventories**.
|
||
* **Complete** — contains every block and item the mods can ever produce:
|
||
* all named items + **all 237 `item_modifier` names** (these serialize into
|
||
`itemmappings.nim` when installed — the missing‑21 of these was the live
|
||
char‑wipe vector),
|
||
* all named blocks + the **entire `base:shape` cross‑product** (10 shape‑helper
|
||
bases × every shape in `shapes.xml`) + the engine's authoritative block dump.
|
||
|
||
Because it's complete, the engine never meets a name not already in the table →
|
||
the **append branch never fires** → the file is never rewritten → **frozen**.
|
||
Verified live on creative/season10: the `.nim` sha256 is byte‑identical before and
|
||
after boot + world‑load.
|
||
|
||
### Final numbers (cluster `cl_37e3a7f72cdc`)
|
||
|
||
| file | count | sha256 (prefix) |
|
||
|---|---|---|
|
||
| `blockmappings.nim` | 38,626 blocks (ids 0..63004) | `0a7db915…` |
|
||
| `itemmappings.nim` | 2,719 items (ids 1..3513) | `fa4af1df…` |
|
||
|
||
u16 ceiling is 65,535 — headroom remains.
|
||
|
||
---
|
||
|
||
## Components & locations
|
||
|
||
| What | Where |
|
||
|---|---|
|
||
| **Deployed canonical** (the source of truth the guard reads) | `figaro:/home/refuge/panel/data/7dtdcluster/<cluster_id>/canonical/{blockmappings.nim,itemmappings.nim}` — this is the host side of the shared `/cluster` bind mount, so it appears as `/cluster/canonical/` inside every member container. |
|
||
| **The freeze guard** | `modules/7dtd/entrypoint.sh`, the `--- cluster canonical .nim freeze guard ---` block. Baked into `panel-7dtd:latest`. |
|
||
| **Per‑world `.nim`** (what gets stamped) | `<saves‑vol>/.local/share/7DaysToDie/Saves/<World>/<GameName>/{block,item}mappings.nim` |
|
||
| **Rebuild tool** | `modules/7dtd/nim-freeze/rebuild.py` (re‑anchor) |
|
||
| **Build artifacts / name registries** (for a from‑scratch rebuild) | `figaro:/home/refuge/nimcanon/` — `tool/`, `registry/` (`blocks_shapes_complete.txt`, `item_modifiers_all.txt`, `blocks_named_currentconfig.txt`, …), `canonical_final/`, `src/` (anchor copies). Persisted out of `/tmp` so a figaro reboot doesn't lose them. |
|
||
|
||
---
|
||
|
||
## The entrypoint guard (the automation)
|
||
|
||
On every boot, after the cluster Player‑symlink setup, the entrypoint runs:
|
||
|
||
```sh
|
||
if [ -f /cluster/canonical/blockmappings.nim ] && [ -f /cluster/canonical/itemmappings.nim ] \
|
||
&& [ -d "$SAVE_BASE" ]; then
|
||
cp -f /cluster/canonical/itemmappings.nim "$SAVE_BASE/itemmappings.nim"
|
||
cp -f /cluster/canonical/blockmappings.nim "$SAVE_BASE/blockmappings.nim"
|
||
fi
|
||
```
|
||
|
||
* `SAVE_BASE` = `Saves/$CLUSTER_PLAYER_SAVE` (set by the Cluster‑tab "Shared player
|
||
world" picker), or the literal `Saves/$GameWorld/$GameName` fallback.
|
||
* Only fires for cluster members that have a `/cluster/canonical/` present →
|
||
non‑cluster servers and other clusters are untouched.
|
||
* **Idempotent + self‑healing**: a complete canonical never triggers an append, so
|
||
re‑stamping the same bytes every boot is a no‑op; if a stray append ever
|
||
happened, the next boot resets it.
|
||
|
||
You'll see this line in the boot log when it works:
|
||
|
||
```
|
||
[panel-7dtd] cluster <id>: stamped canonical id maps over .../<World>/<GameName> (frozen, divergence-proof)
|
||
```
|
||
|
||
---
|
||
|
||
## Is it automated? (what's hands‑off vs. operator action)
|
||
|
||
* **Hands‑off:** every existing frozen world stays frozen forever; every
|
||
boot/recreate/`/rebuild` re‑applies the canonical automatically.
|
||
* **One operator step for a NEW world:** when a clustered server **regenerates an
|
||
RWG world** (new seed), the new world's folder name is **seed‑derived and
|
||
unknown until after gen**, so the guard can't target it until you set the
|
||
Cluster‑tab **Shared player world** (`CLUSTER_PLAYER_SAVE`). Set that (and the
|
||
Region Medic `world`) to the new world and the guard freezes it on the next
|
||
boot. This is the same one‑click step a normal cluster *join* already needs.
|
||
* **Rebuild needed only if mods change** (see below).
|
||
|
||
> Future hardening (not yet done): make the guard/symlink **auto‑detect** the
|
||
> active world dir (newest `Saves/*/<GameName>/` with a `main.ttw`) so even RWG
|
||
> regens need zero manual settings. Deliberately left as an explicit Cluster‑tab
|
||
> setting for now.
|
||
|
||
---
|
||
|
||
## Operating
|
||
|
||
### Add a clustered server / regenerate a clustered world (RWG)
|
||
|
||
Goal: fresh world, frozen on the canonical, sharing the player folder. (Fresh
|
||
worlds are clean — worldgen places vanilla blocks whose ids already match the
|
||
canonical, so **no morph**.)
|
||
|
||
1. **New map:** change the seed — `POST /api/instances/<id>/env-config` body
|
||
`{"updates":{"world_seed":"<new>"}}`. The engine gens a new seed‑named world.
|
||
2. Find the new world dir (newest `Saves/*/<GameName>/main.ttw`).
|
||
3. **Point the cluster settings at it:**
|
||
* `POST .../env-config` `{"updates":{"CLUSTER_PLAYER_SAVE":"<NewWorld>/<GameName>"}}`
|
||
→ recreate → guard stamps it + symlinks `Player -> /cluster/Player`.
|
||
* Write `region-medic.json` `"world": "<NewWorld>/<GameName>"` into the saves
|
||
volume (keep `enabled`/`keep`/`discord_channel`).
|
||
4. Confirm the boot log shows the "stamped canonical" line and the world `.nim`
|
||
sha256 == the cluster canonical.
|
||
|
||
### After a mod change (new blocks/items added) — REBUILD the canonical
|
||
|
||
The canonical is a static snapshot of the merged config's name universe. New mods
|
||
add names it doesn't have, so those would append per‑world again.
|
||
|
||
1. Re‑harvest the complete name set (the two ultracode workflows did this; the
|
||
essential outputs are in `~/nimcanon/registry/`):
|
||
* blocks: `<base>:<shape>` cross‑product from `shapes.xml` × the 10 shape‑helper
|
||
bases, ∪ the engine's authoritative block dump (`exportcurrentconfigs` /
|
||
runtime `Block` list), ∪ named blocks.
|
||
* items: every `<item name=>` ∪ every `<item_modifier name=>`
|
||
(`grep -rhoE '<item_modifier name="[^"]+"' Data/Config/item_modifiers.xml Mods/*/Config/item_modifiers.xml`).
|
||
2. STOP season10, copy its current `.nim` as the **anchor**, run `rebuild.py`
|
||
(see its header) to produce a re‑anchored, extended, complete canonical.
|
||
3. Verify `GATE PASS: True` (0 anchor ids altered, no dup id/name, complete,
|
||
max id < 65536).
|
||
4. Copy into `…/<cluster_id>/canonical/` (chmod 644) and `/rebuild` each member.
|
||
|
||
### Verify a server is frozen
|
||
|
||
```sh
|
||
# .nim on disk == the cluster canonical:
|
||
docker run --rm -v panel-<id>-saves:/sv:ro debian:12-slim \
|
||
sha256sum "/sv/.local/share/7DaysToDie/Saves/<World>/<GameName>/blockmappings.nim"
|
||
# vs:
|
||
sha256sum /home/refuge/panel/data/7dtdcluster/<cluster_id>/canonical/blockmappings.nim
|
||
# and the boot log:
|
||
docker logs panel-<id> 2>&1 | grep "stamped canonical"
|
||
```
|
||
|
||
The **real** test is in‑game: transfer a player between two members and confirm
|
||
glass stays glass, inventory intact, no level reset.
|
||
|
||
### Troubleshooting / gotchas
|
||
|
||
* **`/rebuild` left the server STOPPED.** If an instance was stopped *before*
|
||
`/rebuild`, the recreate suppresses autostart and leaves it `Created`/stopped.
|
||
Just **Start** it — the guard runs on boot.
|
||
* **Guard didn't stamp / wrong dir.** `CLUSTER_PLAYER_SAVE` is empty or stale →
|
||
for RWG the fallback `Saves/RWG/<GameName>` doesn't exist, so the guard skips.
|
||
Set the Cluster‑tab Shared‑player‑world to the real (seed‑derived) world.
|
||
* **season10 grew between snapshot and stamp.** season10's `.nim` appends as
|
||
players place new shapes (we saw 12259 → 12263 in a few hours). **Always
|
||
re‑anchor (`rebuild.py`) against season10's CURRENT, STOPPED `.nim` immediately
|
||
before stamping season10.** Once season10 is frozen on the complete canonical it
|
||
stops growing.
|
||
* **Stamp only while the target is STOPPED**, as the runtime uid (`1000:1000`),
|
||
mode `0644`. An unreadable `.nim` makes the engine silently gen a fresh table =
|
||
full remap.
|
||
* Old orphaned worlds (e.g. `Tefasizi County`, `Kopaneke Territory`) are left in
|
||
the saves volumes after an RWG regen — safe to delete to reclaim space.
|
||
|
||
---
|
||
|
||
## If we need to pivot (alternatives, ranked)
|
||
|
||
If the freeze ever proves unworkable (e.g. a mod that registers ids at **runtime**
|
||
— none currently do; all 7DTD content here is static XML):
|
||
|
||
1. **One shared world** across the whole cluster (same GameWorld+seed+GameName).
|
||
One world → one `.nim` → impossible to diverge. Cost: every server is the SAME
|
||
map (defeats distinct‑experience servers). Simplest and bulletproof.
|
||
2. **Don't carry numeric‑id state across worlds.** Share only name‑keyed
|
||
progression (level/skills/perks/recipes — these already transfer cleanly) and
|
||
**strip bag/belt/equipment** on join to a world that didn't author the
|
||
inventory (an admin/RefugeBot hook). Players keep their character, not carried
|
||
items. Preserves distinct worlds.
|
||
3. **Re‑encode on transfer** (true but fiddly): decode the `.ttp` inventory
|
||
through the SOURCE world's `.nim` and re‑encode through the DESTINATION's,
|
||
dropping ids the destination lacks. Must run while the player is offline.
|
||
4. The current approach (**freeze a complete canonical**) is strictly better than
|
||
"sync the `.nim` across different worlds" — never try that bare, because the
|
||
`.nim` is also the codebook for each world's OWN placed blocks: swapping it on
|
||
a world with existing modded builds **morphs that world's terrain** (that's why
|
||
insane/pvp got fresh worlds instead of an in‑place stamp).
|
||
|
||
---
|
||
|
||
## Key facts to remember
|
||
|
||
* Inventory/placed blocks = **numeric id**; level/skills/perks/recipes =
|
||
**name‑keyed** (the latter survive any transfer).
|
||
* `.nim` is written **only on table growth**, not per save — a complete table is
|
||
therefore frozen.
|
||
* The canonical must be **anchored to season10** (the authoring world) or existing
|
||
inventories morph.
|
||
* New RWG world folder names are **seed‑derived**; you must set
|
||
`CLUSTER_PLAYER_SAVE` + medic `world` to the real name after gen.
|
||
* The deployed canonical lives in the **shared cluster dir** (`/cluster/canonical`)
|
||
so one copy serves every member and the guard is purely local file ops.
|