OPENBUREAU/cms/core/HANDOVER.md

openbureau-core — handover

For a fresh instance — START HERE

• Repo (local): cloned at /home/karim/openbureau-core. Remote git.openbureau.ch/karim/openbureau-core is private; auth via an HTTPS token in ~/.git-credentials (the SSH key id_ed25519_openbureau is currently rejected by gitea/proxmox — server seems rebuilt; fix the key or keep using the token).
• Where the work is: branch stufe-2-5-scaffold (pushed, NOT merged). main is still the pre-stage-2 foundation. Open a PR or merge when happy.
• Done: stages 1–4 complete, stage 5 scaffolded. See "Migration stages" below (markers [done]/[scaffold]). New code: api/src/collections.js, api/src/plugin-manager.js, api/src/plugins/dialog/index.js + their tests.
• Run tests: cd api && npm install && node --test → 44 green. node_modules is git-ignored; the fresh clone needs npm install once.
• Status: stages 1–9 DONE (6 = full schema-driven editor; 9 = kgva on core, DB-less, staged-verified, repo pushed). openbureau runs on core (dev CT 134); kgva LIVE site still static (unchanged) with the CMS dormant in-repo. Remaining: deliberate cutovers — (a) redeploy openbureau with the schema editor after Karim eyeballs it; (b) provision the kgva-core CMS container + Caddy switch (his call, repoints a live domain). Original stage-5 note kept below for history.
• Immediate next action (historic): stage 5 + 5.5 DONE on the branch (commits c4230a4 cutover, 5e06337 DDL, a93d11a migration runner). Plugin-manager singleton (api/src/plugins.js) wired into index.js/stats.js/publish.js, hard-coded dialog removed, source moved under api/src/plugins/dialog/; DDL captured into plugins/dialog/migrations/001_dialog.sql (idempotent); api/src/migrate.js applies migrations once via schema_migrations at boot. 53 tests green + a real boot. Live-verified on the dev stack (CT 134): a 2nd cms instance from cms-cms:latest (my src mounted over /app/src, real Supabase via kong:8000, empty SITE_DIR → no writes) returned byte-identical /api/forums & /api/recent vs the running container, /api/content 401 on both; the runner's tracking SQL was proven on a throwaway DB. NOT deployed (the running stack still runs old code). NEXT (stage 8): deploy core to openbureau — needs DATABASE_URL set for the cms service (e.g. postgres://postgres:$POSTGRES_PASSWORD@db:5432/postgres) so the runner can apply migrations, then full login/edit/publish verify before cutting the live container over. Then stages 6 (admin /api/schema), 7 (local auth), 9 (kgva). Dev stack: Proxmox CT 134 openbureau-dev, repo /opt/openbureau, compose cms/docker-compose.yml, containers openbureau-{cms,auth,kong,rest,db}, dev URL dev.openbureau.ch. Do NOT push a blind refactor to prod.
• Env gotcha: editing a file that's open in Karim's VSCode makes the Edit/Write tool hang/"interrupt" — write such files via Bash (cat > f <<'EOF' … EOF) instead.

Goal

Extract a generic, schema-driven Hugo CMS engine ("openbureau-core") from the openbureau site's bundled CMS, so that both openbureau and karimgabrielevarano.xyz (kgva) consume it as a dependency. Decision: own repo (this one), and migrate openbureau to depend on it.

State (done this session)

• Repo karim/openbureau-core created on Gitea, foundation pushed.
  • api/ = engine copied verbatim from karim/OPENBUREAU → cms/api (Hono/Node).
  • admin/ = React/Vite SPA copied verbatim from OPENBUREAU/cms/admin.
  • api/src/config.js = NEW loader: reads CMS_CONFIG → a site config module.
  • examples/openbureau.config.js, examples/kgva.config.js = target schemas.
  • README.md = config API + plugin model + the migration checklist.
• Nothing in the live openbureau CMS was changed yet.

Engine is ~80% generic. openbureau-specific seams to make config-driven:

1. [DONE] api/src/files.js classify(rel) + the order map — now derived from config.collections via api/src/collections.js (classify/buildPath/compareEntries); buildPath exposed there too.
2. [DONE] api/src/routes/stats.js — now per-collection (statKey/draftStatKey), dialog counts gated by hasPlugin('dialog').
3. api/src/index.js — always mounts dialog routes + runs syncLibrary on boot. → only when config.plugins includes dialog.
4. api/src/routes/publish.js — calls syncLibrary. → gate by plugin.
5. admin/src/App.jsx (714 lines) — hard-codes SECTIONS, KIND_LABEL, the field set, type dropdown, buildPath. → render sidebar groups + editor fields from the schema (add GET /api/schema returning config.collections).
6. dialog subsystem (dialog-store.js, routes/dialog.js, routes/comments.js
   • Supabase tables forums/threads/comments + library↔thread sync) = the dialog plugin (openbureau on, kgva off). First gate by config, later move to api/src/plugins/dialog/.

CRITICAL

api and admin share a data contract (stats keys, field shapes). Generalising the backend alone breaks the live openbureau admin — they must change together and be tested on the live Supabase/Hugo stack. Do NOT push a blind half-refactor to production openbureau.

Plugin system (decided architecture)

Everything beyond the generic engine is a plugin; the engine gets a small plugin manager. A plugin = api/src/plugins/<name>/index.js exporting a manifest { name, routes:[{path,app,public?,admin?}], onBoot, onPublish, onPreview, migrations:[…], stats, admin:{…} }. The manager reads config.plugins, imports each, mounts routes in the right auth tier (public before requireAuth, rest after, admin:true behind requireAdmin), registers boot/publish/preview/stats hooks, runs migrations; the admin SPA loads UI from admin/src/plugins/<name>. openbureau's extras become plugins (first: dialog); kgva enables none. (Full spec in README → "Plugins".)

Migration stages (also README checklist)

1. [done] repo + engine + config loader + example configs.
2. [done] files.js classify/order/buildPath ← config.collections (new api/src/collections.js = pure classify/buildPath/compareEntries/contentStats; files.js uses it, loads config lazily in listEntries; api/test/collections.test.js proves 1:1 vs openbureau.config.js). All api tests green.
3. [done] stats.js ← collections: content via statKey + draftStatKey (openbureau beitrag got draftStatKey: 'entwuerfe'), dialog counts gated by hasPlugin('dialog') (pluginless site does zero DB reads here).
4. [built, not wired] plugin manager — api/src/plugin-manager.js (loadPlugins(names) imports plugins/<name>/index.js; createManager(manifests) = pure wiring: mountPublic/mountPrivate(+requireAdmin), runBoot/runPublish/ runPreview, collectStats (merged under plugin name), migrations() (resolves file URLs; execution deferred to the live DB)). Unit-tested in api/test/plugin-manager.test.js (tiers/hooks/stats/migrations). TODO: wire into index.js (stage 5), and admin loads plugin UI from admin/src/plugins/<name>.
5. [DONE — commits c4230a4 (cutover) + 5e06337 (DDL), live e2e verified] extract openbureau extras into the dialog plugin. DONE: api/src/plugins/dialog/index.js = manifest wrapping the existing modules unchanged (public reads + widget login(rate-limited)
   • authed writes + self-guarding mod/adminForums sub-apps; onBoot/onPublish = syncLibrary; stats = forum/thread/comment counts). CUTOVER DONE: shared manager singleton api/src/plugins.js wired into index.js (mountPublic before requireAuth, mountPrivate(+requireAdmin) after, manager.runBoot at serve()), publish.js (manager.runPublish) and stats.js (manager.collectStats, {content,users,dialog} contract preserved); hard-coded dialog REMOVED from all three; dialog source physically moved to plugins/dialog/{dialog-store,dialog,comments}.js (imports fixed). Tested: api/test/{dialog-plugin,cutover}.test.js, 48 green + real boot. DDL CAPTURED (commit 5e06337): migrations/001_dialog.sql from the live dev DB, idempotent, validated on a throwaway DB; manifest migrations: ['001_dialog.sql']. LIVE E2E VERIFIED (dev CT 134): 2nd cms instance from cms-cms:latest with my src mounted, real Supabase via kong:8000 — /api/forums & /api/recent byte-identical to the running container, /api/content 401 both, no writes. Migration runner = stage 5.5 (commit a93d11a, see new item below). Deploy is stage 8. NB: dialog-store.js still filters e.kind === 'beitrag' — openbureau-specific, already moved with the plugin. 5.5 [done, commit a93d11a] migration runner — api/src/migrate.js runMigrations tracks applied migrations in public.schema_migrations and applies each declared file once, in its own transaction, at boot (wired into index.js before runBoot). Lean: no declared migrations → no DB connection, no pg import (kgva stays DB-less). Executor is injectable (unit-tested without a DB); default uses a lazily-imported pg against DATABASE_URL, warning-and-skipping if unset. Needs DATABASE_URL wired into the cms service env for openbureau (stage 8). Tested: migrate.test.js + tracking SQL proven on the live dev Postgres (throwaway DB).
6. [DONE, commits f709b5d + f8f4131 + 59b9e20] schema-driven admin. GET /api/schema
   • GET /api/system + a System panel; AND the editor now renders sidebar groups + fields from the site's collections (admin/src/fields.jsx: per-type controls string/text/slug/number/date/bool/select/list/image/color/markdown; form↔frontmatter
   • path building). One admin serves any site. Unknown frontmatter is PRESERVED on save (__raw) — no data loss. Verified working via kgva (stage 9). NOTE: openbureau is NOT yet redeployed with the schema editor (its running image keeps the bespoke one); when it is, its config field types (color→swatch, layout opts) are already tuned. Visual review of the openbureau editor by Karim still recommended before redeploying it.
7. [DONE, commit 86f9f57] auth provider (config.auth = supabase | local). api/src/auth-local.js = file-based users (bcryptjs) + self-signed HS256 JWTs of the SAME claim shape; routes/auth.js POST /api/auth/login (mounted only when auth:local); supabase.js made null-safe (fail-fast moved to index.js); auth/stats/ users.js branch local↔GoTrue; admin supabase.js gains a local-auth SHIM (same interface) when VITE_SUPABASE_URL is absent → App/api untouched, supabase path byte-identical. examples/kgva.config.js → auth:'local'. Verified: 59 tests + DB-less boot smoke (no Supabase: local login → /api/me admin → stats-from-file)
   • openbureau redeployed and confirmed unregressed (auth=supabase).
8. [DONE, OB main 3d902a0] openbureau consumes core. Mechanism: git subtree at OPENBUREAU/cms/core (fits the pull-based deploy; update via git subtree pull --prefix=cms/core <core> main --squash). docker-compose builds the cms image from ./core; cms/openbureau.config.js (CMS_CONFIG) carries the content model; old vendored cms/api+cms/admin removed. DATABASE_URL left unset (the stack's migrate service owns db/schema.sql incl. dialog). Deployed to dev CT 134 and FULL write-verified: login/list/edit/preview/publish/dialog all identical, no git divergence (GIT_PUBLISH=false). Rollback image cms-cms:rollback kept.
9. [DONE (repo + staged verify); live cutover PENDING] kgva consumes core. In karim/kgva (git.kgva.ch, main 51a94b8): cms/core = core subtree, cms/kgva.config.js (project/page/section, auth:'local', no plugins), docker-compose.yml builds the core image (Hugo 0.163.3, no VITE_SUPABASE_URL → local-auth admin), cms/seed-admin.mjs, updated deploy/provision-lxc.sh + ADMIN.md. VERIFIED DB-less end-to-end on real content (staged on CT134): local login → schema editor → 42 entries (de/en) → stats → create/preview/cleanup; image galleries preserved. The LIVE site is UNCHANGED — CT130 still deploys statically (timer → hugo → nginx; hugo ignores cms/); the CMS is dormant. PENDING (deliberate, his call — repoints a live domain): provision a kgva-core container (deploy/provision-lxc.sh, free CTID + temp IP) and decide topology — CMS serves the site (Caddy → it) OR CMS edits + GIT_PUBLISH=true commits to git and the CT130 static timer keeps serving. Then Caddy cutover per deploy/README. openbureau also not yet redeployed with the schema editor (stage 6 note).

How to run / test

openbureau CMS stack: OPENBUREAU/cms/docker-compose.yml (Node api + Hugo + Supabase: postgres/kong/gotrue). Env in cms/.env (see .env.example): SUPABASE_URL/SERVICE_KEY, JWT_SECRET, ANON_KEY, SERVICE_ROLE_KEY (derive via scripts/generate-keys.mjs), ADMIN_EMAILS, SITE_URL, GIT_*. For core add CMS_CONFIG (path to the site config) and SITE_DIR (site repo mount, default /site). Admin at /admin, preview /_preview, publish builds public/.

Infra access (NO secrets in this file)

• Proxmox node 192.168.1.2, user root. Password: ask the user (provided ad hoc, not stored). No SSH key installed — use SSH_ASKPASS, or install a key first.
• Gitea = container 120 (pct exec 120 …), ROOT_URL git.openbureau.ch, sqlite at /var/lib/gitea/data/gitea.db. Make a token: pct exec 120 -- su gitea -s /bin/bash -c "/usr/local/bin/gitea admin user generate-access-token --username karim --scopes all --token-name X --raw --config /etc/gitea/app.ini". Keep token kgva-deploy. Delete tokens via sqlite (DELETE FROM access_token WHERE name LIKE '…') since the API needs the account password.
• Push without local git auth: tar → scp to node → pct push 120 into the gitea container → git push http://karim:<token>@127.0.0.1:3000/karim/<repo>.git.
• kgva site deploy: container 130 (kgva-website). nginx serves /var/www/karimgabrielevarano.xyz (= ZFS tank/kgva-website, owned uid 100000). systemd timer runs /opt/kgva-deploy.sh every 60s: pull karim/kgva (token kgva-deploy) → hugo build → copy into webroot. So a push to karim/kgva is live in ~1 min. Self-hosted video at /var/www/media (nginx /media/ alias, outside the build). Public TLS for the domain terminates upstream → 192.168.1.130:80.

Watch out

• Gitea does NOT send CORS on its OAuth token endpoint → a browser git-CMS (Decap-style) can't auth without a same-origin proxy. openbureau-core uses its own Supabase auth, so this doesn't affect it; it's why kgva edits go via git push.
• Hugo versions: openbureau CMS bundles 0.161.1; the kgva site needs 0.163.3 features. Mind the version a core instance builds with.