Squashed 'cms/core/' content from commit 59b9e20

git-subtree-dir: cms/core
git-subtree-split: 59b9e2075ac48cf017db38009d54dbe56a98fffc

HANDOVER.md

@@ -0,0 +1,187 @@
+# openbureau-core — handover
+
+## For a fresh instance — START HERE
+- **Repo (local):** cloned at `/home/karim/openbureau-core`. Remote
+  `git.openbureau.ch/karim/openbureau-core` is **private**; auth via an HTTPS token
+  in `~/.git-credentials` (the SSH key `id_ed25519_openbureau` is currently rejected
+  by gitea/proxmox — server seems rebuilt; fix the key or keep using the token).
+- **Where the work is:** branch **`stufe-2-5-scaffold`** (pushed, NOT merged). `main`
+  is still the pre-stage-2 foundation. Open a PR or merge when happy.
+- **Done:** stages 1–4 complete, stage 5 *scaffolded*. See "Migration stages" below
+  (markers `[done]`/`[scaffold]`). New code: `api/src/collections.js`,
+  `api/src/plugin-manager.js`, `api/src/plugins/dialog/index.js` + their tests.
+- **Run tests:** `cd api && npm install && node --test` → **44 green**. `node_modules`
+  is git-ignored; the fresh clone needs `npm install` once.
+- **Status:** stages 1–8 (+5.5) DONE; stage 6 partial (schema/system/System-panel done, editor-from-schema deferred). openbureau runs on core on dev CT 134, verified. **Next = stage 9** (onboard kgva: DB-less, auth:local, no plugins) — and finish stage 6's editor-from-schema there. Original stage-5 note kept below for history.
+- **Immediate next action (historic):** stage 5 + 5.5 **DONE** on the branch (commits `c4230a4`
+  cutover, `5e06337` DDL, `a93d11a` migration runner). Plugin-manager singleton
+  (`api/src/plugins.js`) wired into `index.js`/`stats.js`/`publish.js`, hard-coded
+  dialog removed, source moved under `api/src/plugins/dialog/`; DDL captured into
+  `plugins/dialog/migrations/001_dialog.sql` (idempotent); `api/src/migrate.js`
+  applies migrations once via `schema_migrations` at boot. **53 tests green** + a real
+  boot. **Live-verified on the dev stack** (CT 134): a 2nd cms instance from
+  `cms-cms:latest` (my `src` mounted over `/app/src`, real Supabase via `kong:8000`,
+  empty SITE_DIR → no writes) returned **byte-identical** `/api/forums` & `/api/recent`
+  vs the running container, `/api/content` 401 on both; the runner's tracking SQL was
+  proven on a throwaway DB. **NOT deployed** (the running stack still runs old code).
+  NEXT (stage 8): deploy core to openbureau — needs `DATABASE_URL` set for the cms
+  service (e.g. `postgres://postgres:$POSTGRES_PASSWORD@db:5432/postgres`) so the
+  runner can apply migrations, then full login/edit/publish verify before cutting the
+  live container over. Then stages 6 (admin `/api/schema`), 7 (local auth), 9 (kgva).
+  Dev stack: Proxmox CT 134 `openbureau-dev`, repo `/opt/openbureau`, compose
+  `cms/docker-compose.yml`, containers openbureau-{cms,auth,kong,rest,db}, dev URL
+  dev.openbureau.ch. **Do NOT push a blind refactor to prod.**
+- **Env gotcha:** editing a file that's open in Karim's VSCode makes the Edit/Write
+  tool hang/"interrupt" — write such files via Bash (`cat > f <<'EOF' … EOF`) instead.
+
+## Goal
+Extract a generic, **schema-driven** Hugo CMS engine ("openbureau-core") from the
+openbureau site's bundled CMS, so that **both** openbureau and
+karimgabrielevarano.xyz (kgva) consume it as a dependency. Decision: own repo
+(this one), and migrate openbureau to depend on it.
+
+## State (done this session)
+- Repo `karim/openbureau-core` created on Gitea, foundation pushed.
+  - `api/` = engine copied verbatim from `karim/OPENBUREAU` → `cms/api` (Hono/Node).
+  - `admin/` = React/Vite SPA copied verbatim from `OPENBUREAU/cms/admin`.
+  - `api/src/config.js` = NEW loader: reads `CMS_CONFIG` → a site config module.
+  - `examples/openbureau.config.js`, `examples/kgva.config.js` = target schemas.
+  - `README.md` = config API + plugin model + the migration checklist.
+- Nothing in the live openbureau CMS was changed yet.
+
+## Engine is ~80% generic. openbureau-specific seams to make config-driven:
+1. [DONE] `api/src/files.js` `classify(rel)` + the `order` map — now derived from
+   `config.collections` via `api/src/collections.js` (classify/buildPath/compareEntries);
+   `buildPath` exposed there too.
+2. [DONE] `api/src/routes/stats.js` — now per-collection (`statKey`/`draftStatKey`),
+   dialog counts gated by `hasPlugin('dialog')`.
+3. `api/src/index.js` — always mounts dialog routes + runs `syncLibrary` on boot.
+   → only when `config.plugins` includes `dialog`.
+4. `api/src/routes/publish.js` — calls `syncLibrary`. → gate by plugin.
+5. `admin/src/App.jsx` (714 lines) — hard-codes `SECTIONS`, `KIND_LABEL`, the field
+   set, type dropdown, `buildPath`. → render sidebar groups + editor fields from the
+   schema (add `GET /api/schema` returning `config.collections`).
+6. dialog subsystem (`dialog-store.js`, `routes/dialog.js`, `routes/comments.js`
+   + Supabase tables forums/threads/comments + library↔thread sync) = the `dialog`
+   **plugin** (openbureau on, kgva off). First gate by config, later move to
+   `api/src/plugins/dialog/`.
+
+## CRITICAL
+api and admin share a data contract (stats keys, field shapes). Generalising the
+**backend alone breaks the live openbureau admin** — they must change together and
+be tested on the live Supabase/Hugo stack. Do NOT push a blind half-refactor to
+production openbureau.
+
+## Plugin system (decided architecture)
+Everything beyond the generic engine is a **plugin**; the engine gets a small
+**plugin manager**. A plugin = `api/src/plugins/<name>/index.js` exporting a
+manifest `{ name, routes:[{path,app,public?,admin?}], onBoot, onPublish, onPreview,
+migrations:[…], stats, admin:{…} }`. The manager reads `config.plugins`, imports
+each, mounts routes in the right auth tier (public before `requireAuth`, rest
+after, `admin:true` behind `requireAdmin`), registers boot/publish/preview/stats
+hooks, runs migrations; the admin SPA loads UI from `admin/src/plugins/<name>`.
+openbureau's extras become plugins (first: `dialog`); kgva enables none.
+(Full spec in README → "Plugins".)
+
+## Migration stages (also README checklist)
+1. [done] repo + engine + config loader + example configs.
+2. [done] files.js classify/order/buildPath ← config.collections
+   (new `api/src/collections.js` = pure classify/buildPath/compareEntries/contentStats;
+   files.js uses it, loads config lazily in listEntries; `api/test/collections.test.js`
+   proves 1:1 vs openbureau.config.js). All api tests green.
+3. [done] stats.js ← collections: `content` via `statKey` + `draftStatKey`
+   (openbureau beitrag got `draftStatKey: 'entwuerfe'`), dialog counts gated by
+   `hasPlugin('dialog')` (pluginless site does zero DB reads here).
+4. [built, not wired] **plugin manager** — `api/src/plugin-manager.js`
+   (`loadPlugins(names)` imports `plugins/<name>/index.js`; `createManager(manifests)`
+   = pure wiring: `mountPublic`/`mountPrivate(+requireAdmin)`, `runBoot/runPublish/
+   runPreview`, `collectStats` (merged under plugin name), `migrations()` (resolves
+   file URLs; execution deferred to the live DB)). Unit-tested in
+   `api/test/plugin-manager.test.js` (tiers/hooks/stats/migrations). TODO: wire into
+   index.js (stage 5), and admin loads plugin UI from `admin/src/plugins/<name>`.
+5. [DONE — commits c4230a4 (cutover) + 5e06337 (DDL), live e2e verified] extract openbureau extras
+   into the **dialog** plugin. DONE: `api/src/plugins/dialog/index.js` = manifest
+   wrapping the existing modules unchanged (public reads + widget login(rate-limited)
+   + authed writes + self-guarding mod/adminForums sub-apps; onBoot/onPublish =
+   syncLibrary; stats = forum/thread/comment counts). CUTOVER DONE: shared manager
+   singleton `api/src/plugins.js` wired into index.js (mountPublic before requireAuth,
+   mountPrivate(+requireAdmin) after, manager.runBoot at serve()), publish.js
+   (manager.runPublish) and stats.js (manager.collectStats, `{content,users,dialog}`
+   contract preserved); hard-coded dialog REMOVED from all three; dialog source
+   physically moved to `plugins/dialog/{dialog-store,dialog,comments}.js` (imports
+   fixed). Tested: `api/test/{dialog-plugin,cutover}.test.js`, 48 green + real boot.
+   DDL CAPTURED (commit 5e06337): `migrations/001_dialog.sql` from the live dev DB,
+   idempotent, validated on a throwaway DB; manifest `migrations: ['001_dialog.sql']`.
+   LIVE E2E VERIFIED (dev CT 134): 2nd cms instance from cms-cms:latest with my src
+   mounted, real Supabase via kong:8000 — /api/forums & /api/recent byte-identical to
+   the running container, /api/content 401 both, no writes. Migration runner = stage
+   5.5 (commit a93d11a, see new item below). Deploy is stage 8.
+   NB: dialog-store.js still filters `e.kind === 'beitrag'` — openbureau-specific,
+   already moved with the plugin.
+5.5 [done, commit a93d11a] migration runner — `api/src/migrate.js` `runMigrations`
+   tracks applied migrations in `public.schema_migrations` and applies each declared
+   file once, in its own transaction, at boot (wired into index.js before runBoot).
+   Lean: no declared migrations → no DB connection, no pg import (kgva stays DB-less).
+   Executor is injectable (unit-tested without a DB); default uses a lazily-imported
+   `pg` against `DATABASE_URL`, warning-and-skipping if unset. Needs `DATABASE_URL`
+   wired into the cms service env for openbureau (stage 8). Tested: migrate.test.js +
+   tracking SQL proven on the live dev Postgres (throwaway DB).
+6. [PARTLY DONE, commit f709b5d] `GET /api/schema` (site/auth/plugins/collections)
+   + `GET /api/system` (admin: core version, plugins, content model, hugo, auth) +
+   an admin **System panel** — all additive, the editor is untouched (stays 1:1).
+   DEFERRED by design: making the editor itself render its fields from the schema —
+   that needs a 2nd content model (kgva) + visual verification, not a blind rewrite
+   of the live editor. Do it when onboarding kgva (stage 9).
+7. [DONE, commit 86f9f57] **auth provider** (`config.auth` = `supabase` | `local`).
+   `api/src/auth-local.js` = file-based users (bcryptjs) + self-signed HS256 JWTs of
+   the SAME claim shape; `routes/auth.js` POST /api/auth/login (mounted only when
+   auth:local); supabase.js made null-safe (fail-fast moved to index.js); auth/stats/
+   users.js branch local↔GoTrue; admin supabase.js gains a local-auth SHIM (same
+   interface) when VITE_SUPABASE_URL is absent → App/api untouched, supabase path
+   byte-identical. examples/kgva.config.js → auth:'local'. Verified: 59 tests +
+   DB-less boot smoke (no Supabase: local login → /api/me admin → stats-from-file)
+   + openbureau redeployed and confirmed unregressed (auth=supabase).
+8. [DONE, OB main 3d902a0] openbureau consumes core. Mechanism: **git subtree** at
+   `OPENBUREAU/cms/core` (fits the pull-based deploy; update via
+   `git subtree pull --prefix=cms/core <core> main --squash`). docker-compose builds
+   the cms image from `./core`; `cms/openbureau.config.js` (CMS_CONFIG) carries the
+   content model; old vendored cms/api+cms/admin removed. DATABASE_URL left unset (the
+   stack's migrate service owns db/schema.sql incl. dialog). Deployed to dev CT 134
+   and FULL write-verified: login/list/edit/preview/publish/dialog all identical, no
+   git divergence (GIT_PUBLISH=false). Rollback image `cms-cms:rollback` kept.
+9. onboard kgva (`CMS_CONFIG=…/kgva.config.js`, `auth: 'local'`, no plugins) as the
+   2nd consumer — DB-less.
+
+## How to run / test
+openbureau CMS stack: `OPENBUREAU/cms/docker-compose.yml` (Node api + Hugo +
+Supabase: postgres/kong/gotrue). Env in `cms/.env` (see `.env.example`):
+SUPABASE_URL/SERVICE_KEY, JWT_SECRET, ANON_KEY, SERVICE_ROLE_KEY (derive via
+`scripts/generate-keys.mjs`), ADMIN_EMAILS, SITE_URL, GIT_*. For core add
+`CMS_CONFIG` (path to the site config) and `SITE_DIR` (site repo mount, default
+`/site`). Admin at `/admin`, preview `/_preview`, publish builds `public/`.
+
+## Infra access (NO secrets in this file)
+- Proxmox node `192.168.1.2`, user `root`. Password: **ask the user** (provided ad
+  hoc, not stored). No SSH key installed — use SSH_ASKPASS, or install a key first.
+- Gitea = container 120 (`pct exec 120 …`), ROOT_URL `git.openbureau.ch`, sqlite at
+  `/var/lib/gitea/data/gitea.db`. Make a token:
+  `pct exec 120 -- su gitea -s /bin/bash -c "/usr/local/bin/gitea admin user
+  generate-access-token --username karim --scopes all --token-name X --raw
+  --config /etc/gitea/app.ini"`. Keep token `kgva-deploy`. Delete tokens via sqlite
+  (`DELETE FROM access_token WHERE name LIKE '…'`) since the API needs the account
+  password.
+- Push without local git auth: tar → scp to node → `pct push 120` into the gitea
+  container → `git push http://karim:<token>@127.0.0.1:3000/karim/<repo>.git`.
+- kgva site deploy: container 130 (kgva-website). nginx serves
+  `/var/www/karimgabrielevarano.xyz` (= ZFS `tank/kgva-website`, owned uid 100000).
+  systemd timer runs `/opt/kgva-deploy.sh` every 60s: pull `karim/kgva` (token
+  `kgva-deploy`) → `hugo` build → copy into webroot. So a push to `karim/kgva` is
+  live in ~1 min. Self-hosted video at `/var/www/media` (nginx `/media/` alias,
+  outside the build). Public TLS for the domain terminates upstream → 192.168.1.130:80.
+
+## Watch out
+- Gitea does NOT send CORS on its OAuth token endpoint → a browser git-CMS
+  (Decap-style) can't auth without a same-origin proxy. openbureau-core uses its own
+  Supabase auth, so this doesn't affect it; it's why kgva edits go via git push.
+- Hugo versions: openbureau CMS bundles 0.161.1; the kgva site needs 0.163.3
+  features. Mind the version a core instance builds with.