Module-scoped persistent storage (ctx.storage), first consumer: docker update state #37

New issue

Closed

opened 2026-08-15 23:32:55 +02:00 by julian · 0 comments

julian commented

2026-08-15 23:32:55 +02:00

Owner

Problem

Modules have no way to persist state across restarts. The SDK offers no storage capability — no KV, no module data directory. Any module holding derived-but-expensive state loses it when the container is recreated.

Concrete case: the docker module's update verdicts (state.updates in packages/modules/docker/backend/mod.ts) live in module memory. The schedule side is already persisted correctly — the interval-updates schedule row in DuckDB survives restarts and nextRunMs = max(lastRunMs + everyMs, now) deliberately avoids re-sweeping registries on every boot. But the sweep result dies with the process, so after recreating the OpsDeck container the UI shows no available updates until the next scheduled sweep, up to CHECK_INTERVAL_MS (default 6 h) later.

Firing the sweep at startup instead is the wrong fix: it resurrects the boot-anchored behaviour the persisted schedule window exists to prevent (a restart-heavy or crash-looping host would hammer rate-limited registries with a manifest request per image on every boot).

Proposal

Core: module-scoped KV storage, exposed on the module context:
- ctx.storage.get(key) / ctx.storage.set(key, value) / ctx.storage.delete(key), JSON values.
- Backed by a DuckDB table in the existing metrics database (/data), keyed (module, key), going through the single serialized DB queue like every other DuckDB access.
- Scoped by module name in the host (server/src/modules/host.ts), same pattern as metrics/scheduler/events — a module can never read another module's keys.
Docker module: persist update state.
- After each sweep (refreshUpdates), store { updates, updatesCheckedAt }.
- On register(), rehydrate state.updates from storage. Correctness is already handled by existing machinery: refreshStacks re-derives updateAvailable from local image digests vs the stored remoteDigest on every poll, and pruneUpdates drops images that no longer run. Per-image checkedAt survives too, which keeps the staleness-ordered rotation in checkUpdates fair across restarts.

Result

Update badges/counters survive a container recreate; the registry sweep cadence stays exactly as scheduled. The storage capability is reusable by any module caching expensive remote facts (e.g. module_teamspeak's badge catalog and icon cache).

## Problem Modules have no way to persist state across restarts. The SDK offers no storage capability — no KV, no module data directory. Any module holding derived-but-expensive state loses it when the container is recreated. Concrete case: the docker module's update verdicts (`state.updates` in `packages/modules/docker/backend/mod.ts`) live in module memory. The schedule side is already persisted correctly — the `interval-updates` schedule row in DuckDB survives restarts and `nextRunMs = max(lastRunMs + everyMs, now)` deliberately avoids re-sweeping registries on every boot. But the sweep *result* dies with the process, so after recreating the OpsDeck container the UI shows no available updates until the next scheduled sweep, up to `CHECK_INTERVAL_MS` (default 6 h) later. Firing the sweep at startup instead is the wrong fix: it resurrects the boot-anchored behaviour the persisted schedule window exists to prevent (a restart-heavy or crash-looping host would hammer rate-limited registries with a manifest request per image on every boot). ## Proposal 1. **Core: module-scoped KV storage**, exposed on the module context: - `ctx.storage.get(key)` / `ctx.storage.set(key, value)` / `ctx.storage.delete(key)`, JSON values. - Backed by a DuckDB table in the existing metrics database (`/data`), keyed `(module, key)`, going through the single serialized DB queue like every other DuckDB access. - Scoped by module name in the host (`server/src/modules/host.ts`), same pattern as metrics/scheduler/events — a module can never read another module's keys. 2. **Docker module: persist update state.** - After each sweep (`refreshUpdates`), store `{ updates, updatesCheckedAt }`. - On `register()`, rehydrate `state.updates` from storage. Correctness is already handled by existing machinery: `refreshStacks` re-derives `updateAvailable` from local image digests vs the stored `remoteDigest` on every poll, and `pruneUpdates` drops images that no longer run. Per-image `checkedAt` survives too, which keeps the staleness-ordered rotation in `checkUpdates` fair across restarts. ## Result Update badges/counters survive a container recreate; the registry sweep cadence stays exactly as scheduled. The storage capability is reusable by any module caching expensive remote facts (e.g. module_teamspeak's badge catalog and icon cache).

julian referenced this issue from OpsDeck/module_teamspeak

2026-08-15 23:33:03 +02:00

Persist badge catalog + icon cache via ctx.storage #1

julian referenced this issue from a commit

2026-08-15 23:43:36 +02:00

feat(storage): module-scoped persistent KV, and the docker verdicts survive a restart

julian referenced this issue from a pull request that will close it,

2026-08-15 23:43:47 +02:00

Module-scoped persistent storage (ctx.storage) + docker update state survives restarts #38