Skip to main content

Implementation Plan — Transition to Self-Hosted (S3-compatible + Nextcloud)

Goal: remove the developer-hosted backend and its fen-provisioner control plane, and ship two owner-self-hosted BYO backends — S3-compatible and Nextcloud — selected once in setup/settings. Normative design: backend-architecture.md.

Accepted trades (do not "fix"): single shared credential per group; short-TTL client-enforced invites (no atomic single-use); weak shared-credential member removal on both backends (forward secrecy via key-epoch rotation only; no server-enforced revocation). Every phase that touches these must keep the docs' §5/§6 language intact.

Sequencing principle​

The app currently depends on the provisioner for group create / invite / join. So the in-app control plane (GroupProvisioner) must land and be wired before the provisioner is deleted. Order: add the new seam and the Nextcloud backend → move provisioning in-app → switch the app over → only then delete the server and infra. Keep main building and the app shippable after every phase; each phase is a reviewable slice.

Process (from AGENTS.md)​

Each phase = one branch + worktree off origin/main, one draft PR, gates green (flutter analyze, flutter test, dart test for touched pure-Dart packages, go test ./... -race only while the provisioner still exists), and cross-vendor review before gh pr ready. Security-sensitive phases (2, 4, 6 — anything touching fen_crypto, descriptors, invite/removal, VerifyingEventSink, roster authority) must not be reviewed solely by hermes.

Phase 0 — Decisions & scaffolding​

  • Confirm decisions in writing: two backends (s3, nextcloud); single shared credential; weak removal; runtime capability detection for Nextcloud (no hardcoded version). These are already captured in backend-architecture.md §0–§6.
  • Create a tracking issue linking this plan; enumerate the deleted guarantees (atomic single-use invites, liveness-independent revocation) so reviewers don't file them as regressions.
  • Done when: plan + design doc merged; tracking issue open.

Phase 1 — Descriptor & storage-binding model (foundation)​

  • packages/fen_events: turn StorageDescriptor into a two-variant sealed type — keep S3Descriptor, add NextcloudDescriptor { baseUrl, shareToken, sharePassword }. Update payloads.dart (invite package, StorageMigrated/StorageKeyRotated) and storage_descriptor_test.dart serialization round-trips.
  • packages/fen_sync/backend_descriptor.dart: storageBackendFromDescriptor gains a kind == 'nextcloud' branch (returns null on malformed, same contract as today).
  • apps/fen Drift group_storage table: generalize from S3 columns to { kind, descriptor_json } (or nullable per-kind columns) + a schema migration. Update group_storage_binder reads/writes.
  • Done when: both descriptor kinds serialize/deserialize; migration test passes; no behaviour change yet (S3 still the only producer).

Phase 2 — NextcloudBackend (data plane) + conformance​

  • packages/fen_sync/src/nextcloud_backend.dart implementing StorageBackend:
    • list = PROPFIND (Depth 1) with XML parse → RemoteEntry (path, ETag, size, mtime).
    • get/put/delete = WebDAV GET/PUT/DELETE; MKCOL helper for folder create.
    • Auth against the public WebDAV endpoint (/public.php/webdav/) using share token as username + share password.
    • probe() = capability check via GET /ocs/v2.php/cloud/capabilities (version + public-link sharing + public upload enabled) plus a list/put/delete smoke test. Map ETag/If-Match support into BackendCaps.conditionalWrite; if Nextcloud's If-Match proves unreliable, rely on the single-writer-per-file invariant and relax the usable requirement here (document the choice).
  • storage_conformance/: add a WebDAV/Nextcloud suite mirroring the S3 one (run against a disposable Nextcloud in CI or a docker-compose fixture).
  • Done when: conformance suite green against a real Nextcloud; probe() fails cleanly on a sharing-disabled instance.

Phase 3 — GroupProvisioner seam + two implementations​

  • Introduce GroupProvisioner (in apps/fen or fen_sync): createGroup, createInvite, removeMember/closeGroup, deleteGroup, best-effort purge. All run with the owner's credentials on-device.
  • NextcloudProvisioner: createGroup = MKCOL + OCS create public link (shareType=3, publicUpload=true, perms read+create+update+delete, password); createInvite returns the existing shared descriptor with a client TTL; closeGroup/deleteGroup = OCS delete share (+ WebDAV DELETE folder).
  • S3Provisioner: createGroup = no-op (implicit prefix); createInvite returns the shared descriptor + TTL; deleteGroup = DeleteObject sweep of groups/<id>/. Credential rotation for generic S3 is out-of-app/manual — surface it as such.
  • Done when: both provisioners unit-tested behind the interface; not yet wired into flows.

Phase 4 — Rewire app flows off ProvisionerClient​

  • Repoint these to GroupProvisioner (backend chosen from the group's stored descriptor/owner config): create_group_service, create_invite_service, join_invite_service, member_admission_service, relayed_invite_ingest_service, group_storage_binder, core/data/app_providers.
  • Delete core/net/provisioner_client.dart and the Ed25519 control-plane signing/canonical-message code once no caller remains.
  • Done when: create/invite/join/remove run end-to-end on both backends with no reference to api.fenapp.net; flutter test green.

Phase 5 — Setup / Settings backend chooser (UX, net-new)​

  • New onboarding + settings screen: pick S3-compatible or Nextcloud, per-backend form, Test connection (calls probe() + capabilities), and secure keychain storage of owner credentials. Extend doc/settings-ui.md.
  • Since the old Managed tier needed zero config, this whole surface is new; block "create group" until a backend is configured and validated.
  • Done when: an owner can configure either backend from a clean install and create a syncable group without leaving the app.

Phase 6 — Invite & removal semantics (accepted-weak model)​

  • Invites: embed the shared credential with a client-enforced TTL (default 30 min); recipient app rejects expired. Remove the invite-status/single-use reconciliation that depended on the provisioner — resolve competing MemberJoined events client-side.
  • Removal: implement/confirm KeyEpochRotated forward secrecy as authoritative; storage-credential rotation is best-effort owner-driven (delete+recreate Nextcloud link / manual S3 key). UI copy must reflect "no server-enforced revocation."
  • Done when: removal excludes a member from new epochs immediately; docs §5/§6 match behaviour; no code claims atomic single-use or deadline revocation.

Phase 7 — Remove the developer backend​

  • Delete services/fen-provisioner/** and its CI gates; delete infra/managed-backend/** (Garage, provisioner, cloudflared) and the operator group-stats dashboard.
  • Remove server-side reapers (45-day deletion, 6-month inactivity sweep, orphan prune). Re-add only the optional, best-effort client purge settings (Phase 3 purge).
  • Drop s3.fenapp.net / api.fenapp.net assumptions from code, config, and .well-known.
  • Done when: repo builds with no Go service; no runtime references to developer-hosted hosts.

Phase 8 — Documentation sweep​

  • Reconcile the master spec doc/fen.md (storage model, sync protocol, invite flow §2.4, security §4.5/§4.8) with the new design; regenerate fen.html via doc/generate-fen-html.sh.
  • Confirm ARCHITECTURE.md, README.md, doc/appendices.md §9 (done in this branch), doc/settings-ui.md, doc/operations.md, and doc/security.md all state: no developer backend, two BYO backends, single shared credential, explicit weak shared-credential removal.
  • Done when: no doc references a "Managed/default Garage" tier or the provisioner except in historical/changelog notes.

Phase 9 — Test, conformance & gates​

  • S3 + Nextcloud conformance suites in CI; delete Garage-fake and provisioner Go tests; keep/generalize sigv4/s3_backend tests for BYO-S3.
  • Full gates green: flutter analyze, flutter test, dart test for touched packages.
  • Verification (high-stakes → subagent review): independent pass confirming (a) the removed guarantees are documented, not silently dropped in code; (b) probe()/capability detection fails safe; (c) no plaintext or owner-only credential leaks into an invite beyond the intended shared credential.
  • Done when: green pipeline; verification sign-off recorded on the PR.

Rough effort​

Dominated by Phases 3–6 (in-app control plane + settings UX + invite/removal). Phases 2 and 9 are moderate; Phase 7 is bulky but mostly deletion; Phase 1 is small-but-cross-cutting. Overall on the order of 3–5 weeks of focused single-developer work. The StorageBackend seam keeps the data-plane swap cheap; the cost is the control-plane relocation and the new configuration surface.