docs: add Memory Graph View repository analysis and design

Repo-wide analysis of frontend/backend/DB/memory/ChromaDB/auth/Docker/
testing conventions, plus the accepted design (architecture, API, UI,
performance, security, migration/rollback, testing strategy) for the
Memory Graph View feature.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
yakamoz221 2026-07-29 20:15:39 +03:00
parent 25c9e735ef
commit 72e7d28ea7
2 changed files with 358 additions and 0 deletions

View file

@ -0,0 +1,170 @@
# Memory Graph View — Repository Analysis
Status: research only. No application code, dependencies, or database state was modified to produce this document.
Scope: this document inventories the parts of the Odysseus codebase relevant to building an interactive, Obsidian-like Memory Graph View, and identifies the safest points to extend the system. A companion document, `docs/memory-graph-design.md`, proposes the actual design based on these findings.
## 1. Frontend architecture
- No bundler, no build step. `static/js/package.json` sets `{ "type": "module" }` so the browser loads native ES modules directly. There is no webpack/vite/esbuild config anywhere in the repo. The root `package.json` has no `scripts` block; its only `devDependency` is `@antithesishq/bombadil`, a browser-fuzzing spec library unrelated to bundling.
- Entry point: `static/index.html` loads `static/app.js` via `<script type="module">`. `app.js` statically imports ~30 feature modules, each a plain ES module with a default export object (e.g. `import memoryModule from './js/memory.js?v=20260722memoryloading1'`). Cache-busting is done manually via query-string suffixes on the import path.
- Cross-module reachability: some modules are also hung off `window` (`window.themeModule`, `window.sessionModule`, `window.uiModule`, `window.adminModule`, `window.cookbookModule`) so unrelated modules can call into them without a formal event bus.
- `static/js/` has ~100 flat files plus subfolders for cohesive subsystems: `editor/` (canvas-based image editor), `compare/`, `research/`, `calendar/`, `emailLibrary/`, `markdown/`, `model/`, `color/`, `util/`. `static/js/MODULE_SUMMARY.md` is the authoritative, actively maintained architecture index and should be updated alongside any new module.
- Backend communication is plain `fetch()` returning JSON, plus one Server-Sent Events (SSE) stream for chat (`chat.js` posts to `/api/chat_stream`, reads via `res.body.getReader()`, parses `data: {...}` lines by `type`). No WebSockets exist anywhere in the codebase.
- `app.js` patches `window.fetch` globally so any `401` response redirects to `/login` — any new module's fetches automatically inherit this behavior.
## 2. Routing
- There is no client-side SPA router (no history-based route table, no hash router library). Odysseus is a single persistent DOM (`index.html`) where "navigation" means opening/closing modals and full-screen panels, not swapping views by URL.
- A lightweight deep-link opener exists in `app.js`: a `_routeOpen` map keyed by `window.location.pathname` (entries for `/notes`, `/calendar`, `/cookbook`, `/email`, `/memory`, `/gallery`, `/tasks`, `/library`). The server presumably serves `index.html` for these paths (catch-all), and this map simulates the click that would normally open the corresponding modal. A new `/memory-graph` entry point would follow this exact pattern, or the graph could live as a new tab inside the existing memory modal instead of a new top-level route.
- Everything else (Memory, Calendar, Gallery, Documents, Tasks, Compare, Cookbook, Settings) is a `.modal` element in `index.html` toggled by `modalManager.js`. There is no hash-based view dispatch beyond an ad hoc entity-hash regex in `init.js` used only for composer-restore behavior.
## 3. UI component library / design system
- No component framework (no React/Vue/etc.), no CSS framework (no Tailwind/Bootstrap). `static/style.css` is a single hand-written stylesheet (tens of thousands of lines) using CSS custom properties for theming (`:root { --bg; --fg; --red; ... }`), a `:root.light` override, density variants (`.density-compact`, `.density-spacious`), and a UI-scale zoom mechanism.
- The "component" pattern is plain JS factory functions building DOM via `document.createElement` and manual event wiring — no virtual DOM, no templating engine. `static/js/memory.js` builds each memory-list row this way.
- Modals share a common infrastructure:
- `static/js/modalManager.js` — central open/minimize/close/dock manager. Public API: `Modals.register(id, { railBtnId, restoreFn, closeFn })`, `Modals.toggle(id)`. Owns a draggable "minimized dock" tray with FLIP animations and magnetic close-on-drag-to-trash behavior. Each modal type has a label+icon entry in `_LABELS`, including an existing `'memory-modal': { label: 'Brain', icon: ... }` entry. **A new Memory Graph View, if its own modal/window, should register here** to get consistent minimize/dock/restore/z-order behavior for free.
- `static/js/modalSnap.js` — edge-docking/snap-to-zone logic, imported by `modalManager.js`.
- `static/js/tileManager.js` — desktop window tiling/snap-to-edge, used by `memory.js` for `snapModalToZone`.
- `static/js/windowDrag.js` — generic `makeWindowDraggable(modal, opts)` helper, used to make the memory modal's header draggable.
- `static/js/toolWindowZOrder.js` — monotonically increasing z-index (`nextToolWindowZ()`) so the most-recently-focused tool window stacks on top.
- `static/modal-control-variants.html`, `static/wave-variants.html`, `static/whirlpool-variants.html` are standalone design-exploration/prototyping pages, not wired into the live app bundle.
- Third-party libraries that are needed client-side are vendored directly into `static/lib/` (e.g. `docx.umd.min.js`, `highlight.min.js`, `xlsx.full.min.js`) rather than installed via npm — this is the established pattern for adding any graph-rendering library, since there is no bundler to run `npm install` through.
## 4. Existing memory UI
- `static/js/memory.js` (~1550 lines) implements the entire "Brain" modal. No graph, timeline, or relationship view exists today.
- Modal structure (`index.html`): `#memory-modal` with tabs — Browse, Skills, Add, Settings.
- Browse tab is a flat list view (`#memory-list`, `.memory-item` rows), not a card grid. Each row shows text, category badge, pinned badge, source (`auto`/`manual`), use-count, relative timestamp, and a kebab menu (Pin/Select/Edit/Delete).
- Supported interactions: free-text client-side search, category filter chips, sort dropdown (newest/oldest/A-Z/most-used) with pinned items floated to top, bulk multi-select with bulk-delete, inline double-click-to-edit, an AI-driven "Tidy" (dedupe/audit) action, import from file with LLM-extracted suggestion review, and JSON export.
- Categories are a fixed client-side set: `MEMORY_CATEGORIES = ['fact','identity','preference','contact','project','goal','task']`.
- The backend already exposes `GET /api/memory/timeline`, sorted by timestamp with resolved session names, but **no frontend module calls it today** — it is the closest existing "structured" memory endpoint and a plausible seed for a graph-view layout, but it is currently dead code from the UI's perspective.
## 5. Existing graph or visualization components
- None. An explicit search across `static/`, `src/`, `routes/`, and the whole repo for `d3.`, `cytoscape`, `vis-network`, `vis.js`, `sigma.js`, `force-graph`, `d3-force`, `three.js`, and `networkx` returned zero matches.
- The only `<canvas>` usage in the codebase is the image editor (`static/js/editor/*`, `static/js/galleryEditor.js`) for pixel/layer compositing — unrelated to node-link rendering.
- Conclusion: a Memory Graph View is a greenfield UI addition. There is no library, canvas renderer, or "relations" concept in the data model to build on top of.
## 6. Backend architecture
- Framework: FastAPI (`app.py`), served by Uvicorn. Two entrypoints: `app.py` (standard dev/server) and `launcher.py` (Windows portable/frozen build with a tray icon, same Uvicorn server underneath).
- Route registration is manual, not auto-discovered: every `routes/*.py` module exposes a `setup_*_routes(...)` factory that builds and returns an `APIRouter` with dependencies passed as plain constructor args (not FastAPI `Depends()`). `app.py` explicitly imports and `include_router()`s over 40 of these factories.
- Startup uses the modern FastAPI `lifespan` context manager. `src/app_initializer.py::initialize_managers()` is a pure component factory invoked once at startup: it builds `MemoryManager`, `SkillsManager`, `SessionManager`, `UploadHandler`, `PersonalDocsManager`, `APIKeyManager`, `PresetManager`, `MemoryVectorStore` (Chroma-backed, degrades gracefully if unhealthy), wraps memory in a `MemoryProviderRegistry`, and builds `ChatProcessor`/`ChatHandler`/`ModelDiscovery`.
- Route handlers are `async def`, but database access uses classic synchronous SQLAlchemy (`SessionLocal()`), occasionally wrapped in `asyncio.to_thread`. The process is single-instance by convention (log rotation is explicitly not multi-process safe).
- Dependency injection is a manual "component bag" pattern, not `Depends()`: singletons are built once in `app.py`/`app_initializer.py` and closed over by each router factory. `Request.state` carries auth-derived values (`current_user`, `api_token`, etc.) set by middleware and read directly inside handlers.
## 7. API routes
- Auth is enforced by `AuthMiddleware` (inside `app.py`, conditional on `AUTH_ENABLED=true`) plus per-route calls into `src/auth_helpers.py`: `get_current_user(request)` (soft), `require_user(request)` (401 if unauthenticated), `require_privilege(request, "can_manage_memory")` (401/403 by privilege flag), `effective_user(request)` (resolves the real owner behind a Bearer API token).
- Ownership is enforced manually per route via helpers like `_assert_session_owner`/`_verify_memory_owner` in `routes/memory/memory_routes.py`, all raising 404 (not 403) on cross-owner access to avoid confirming another user's resource exists.
- Request/response validation is inconsistent by design across routes: some use Pydantic models from `src/request_models.py` (`MemoryAddRequest`, etc., with lenient field validators that clamp/default rather than hard-reject), others use raw `Form(...)` parameters. Pick per-route based on whether the caller posts JSON or a browser form.
- Error handling is two-layered: routes raise `HTTPException` directly and inline; a small set of domain exceptions (`SessionNotFoundError`, `InvalidFileUploadError`, `LLMServiceError`, `WebSearchError`) are registered globally in `core/exceptions.py` / `app.py` with fixed JSON shapes and status codes.
- Rate limiting (`src/rate_limiter.py`) is a simple in-memory sliding-window limiter, wired only into `routes/auth_routes.py` (login/setup) — there is no global rate-limit middleware.
- Pagination has no single shared convention. History uses offset/limit with a "default to most recent page" fallback; documents use `Query(0, ge=0)`/`Query(20, ge=1, le=50)`; **memory routes have no pagination at all**`GET /api/memory` and `/timeline` return the full owner-scoped list every time. A graph endpoint returning potentially hundreds of nodes/edges will need new pagination/limiting that doesn't exist in the memory API today.
## 8. Database schema
- Engine: SQLite by default (`DATABASE_URL=sqlite:///{DATA_DIR}/app.db`), synchronous SQLAlchemy declarative ORM. A custom `EncryptedText` `TypeDecorator` transparently Fernet-encrypts sensitive columns at the ORM layer.
- Migrations: no Alembic. Hand-rolled, idempotent `_migrate_add_*` functions run in a fixed sequence from `init_db()`, each checking `PRAGMA table_info(table)` before `ALTER TABLE ... ADD COLUMN`, safe to re-run every startup. `init_db()` also calls `Base.metadata.create_all(bind=engine)` for any brand-new tables.
- Key tables (all in `core/database.py`): `sessions`, `chat_messages`, `memories` (see discrepancy below), `documents`/`document_versions`, `gallery_albums`/`gallery_images`, `notes`, `calendars`/`calendar_events`/`caldav_deleted_events`, `email_accounts`, `scheduled_tasks`/`task_runs`, plus supporting tables (`model_endpoints`, `api_tokens`, `mcp_servers`, `comparisons`, `signatures`, `webhooks`, `user_tools`, `crew_members`, `editor_drafts`, `integrations`).
- Users/auth are **not** in the SQL database — `core/auth.py::AuthManager` reads/writes `data/auth.json`. Ownership on DB rows is a plain `owner` string column (username), not a foreign key to a `users` table; `NULL` conventionally means "legacy/shared, visible to everyone."
- **Important discrepancy**: a `memories` SQL table exists with proper schema and a `session_id` FK, but the runtime memory store actually used everywhere (routes, chat context injection, agent tool) is `src/memory.py::MemoryManager`, which persists to a flat JSON file `data/memory.json`, not this table. The SQL `memories` table appears to be unused/legacy. Any Memory Graph View must treat `memory.json` (+ its Chroma vector index) as the real source of truth, not the SQL table.
## 9. Memory system
- Canonical storage: `src/memory.py::MemoryManager` — a JSON file (`data/memory.json`) holding flat entries: `id`, `text`, `category` (default `"fact"`), `source`, `owner`, `timestamp`, optional `session_id`, `pinned`, `uses`. **There is no relationship/edge field** — no `related_ids`, `links`, or similar. Any graph view must derive edges rather than read stored ones.
- `services/memory/memory.py` and `services/memory/memory_vector.py` are thin backward-compatibility shims re-exporting the canonical `src/memory.py` / `src/memory_vector.py` implementations.
- Backend routes (`routes/memory/memory_routes.py`, prefix `/api/memory`; `routes/memory_routes.py` is a compat shim re-exporting the same router object): `POST /add`, `GET ""` (list), `POST /search`, `GET /timeline`, `GET /by-session/{session_id}`, `POST /extract` (LLM suggestion extraction from a chat session), `POST /audit` (dedupe), `POST /import` (file-based extraction), `POST /{id}/pin`, `GET /{id}`, `PUT /{id}`, `DELETE /{id}`.
- Memory-to-session relationship is informational provenance only (`session_id` records which chat session a memory was extracted from) and is not a structural graph edge.
- Two independent paths connect memory to the agent/chat pipeline:
1. **Automatic read path**`src/chat_processor.py::ChatProcessor.build_context_preface(...)` runs on every chat/agent turn when `use_memory=True` (default): loads the owner's memories, selects relevant pinned + hybrid-retrieved (BM25 + vector) memories up to a context limit, injects them as untrusted-context messages before the LLM ever sees the turn, and bumps each injected memory's `uses` counter.
2. **Explicit write path** — the model can emit a `manage_memory` tool call (`src/ai_interaction.py::do_manage_memory`), supporting `list|add|edit|delete|search`, mutating `MemoryManager` + `MemoryVectorStore` directly and firing a `memory_added` event.
- `src/memory_provider.py` defines a `MemoryProvider` ABC / `MemoryProviderRegistry` intended to let alternate memory backends plug in behind the same interface; only `NativeMemoryProvider` is currently registered. This is the natural extension seam if a graph-capable provider is ever needed, though the design in this analysis targets the native provider directly since that's what all current UI and agent paths use.
## 10. ChromaDB integration / vector search
- `src/chroma_client.py` uses `chromadb.HttpClient(host, port)` — a real HTTP client against a **separate `chromadb` container**, not an embedded/local persistent client. Config: `CHROMADB_HOST` (default `localhost`), `CHROMADB_PORT` (default `8100`); docker-compose sets these to `chromadb`/`8000` for in-network container-to-container access. A TCP probe fails fast before constructing the client; `client.heartbeat()` verifies liveness before the singleton is cached.
- Collection names: `odysseus_memories` (`src/memory_vector.py::MemoryVectorStore.COLLECTION_NAME`) and `odysseus_rag` (`src/rag_vector.py`). Both are created lazily through `build_embedding_lanes()``chroma_client.get_or_create_collection(...)` (`src/embedding_lanes.py`), which supports multiple simultaneous embedding backends ("lanes": local FastEmbed ONNX, or a configured custom embedding endpoint), each with its own Chroma sub-collection, searched in parallel and de-duplicated.
- `MemoryVectorStore` exposes `add(memory_id, text)`, `remove(memory_id)`, `search(query, k)`, and a `healthy` flag. This is the natural, already-available source of pairwise similarity scores for auto-generating graph edges between memories, since the memory schema itself has no explicit relation data.
- Memories and personal-doc RAG chunks live in separate Chroma collections and are not cross-linked or queried together today.
## 11. Document storage
- Two distinct subsystems:
- "Living documents" (AI-editable canvas documents): `routes/document_routes.py`, backed entirely by the relational DB (`documents`/`document_versions` tables) — content lives in `Text` columns, not on disk. `Document.owner` is stamped independently of `session_id` so a document survives its parent session's deletion.
- "Personal Docs" (RAG document library): `src/personal_docs.py::PersonalDocsManager` walks a directory on disk, extracts text (PDF via `pypdf`, Office via `markitdown`), chunks it, and indexes into the `odysseus_rag` Chroma collection. Files stay on disk; there is no separate SQL metadata table — the Chroma collection's metadata is the catalog.
- Not directly part of the Memory Graph View's data model, but a candidate future edge type ("memory ⟷ document it was extracted from") if the design is extended later.
## 12. Conversation storage
- Sessions and messages are relational DB rows (`sessions`, `chat_messages`), managed exclusively through `core/session_manager.py::SessionManager`.
- `SessionManager` keeps an in-memory cache but only loads the 100 most-recently-accessed, non-archived, non-empty sessions' metadata at boot (not messages) to bound memory; message history is hydrated on demand.
- `routes/history/history_routes.py` supports offset/limit paging directly against `chat_messages`, independent of the in-memory cache, and lazily "hydrates" a session's in-RAM history from the DB if it's found to be behind.
- `sessions.owner` (nullable username string) is the ownership authority that memory, documents, and other per-session resources ultimately trace back to via their own `owner` column or `session_id` FK.
## 13. Agent architecture
- `src/agent_loop.py::stream_agent_loop(...)` is the central async-generator agent loop: streams SSE events (`delta`, `tool_start`, `tool_output`, `agent_step`, `metrics`, `[DONE]`), assembles a dynamic system prompt, and handles plan-mode, tool policies, and per-model quirks.
- `src/tool_execution.py` dispatches model-emitted tool calls either to MCP servers (`src/mcp_manager.py`) or native Python implementations under `src/agent_tools/`. Sensitive filesystem tools are admin-only and path-confined via deny/allow lists.
- `src/agent_runs.py` lets an SSE stream survive a browser disconnect by draining the generator server-side into a replay buffer per session id (does not survive a server restart) — this is the closest existing pattern to a "live push channel," and is the template to follow if the graph view needs live updates (see §14).
- Memory interacts with the agent loop via the two paths described in §9 (automatic context injection, explicit `manage_memory` tool), not via any structural graph traversal today.
## 14. Real-time/event infrastructure
- `src/event_bus.py::fire_event(event_name, owner)` is a **task-automation trigger bus**, not a pub/sub-to-frontend mechanism. It matches `ScheduledTask` rows with `trigger_type == "event"` against the fired event name and runs the matching automation once a threshold count is hit. It has no subscriber API that a browser tab could listen to.
- Publishers of `memory_added`: `routes/memory/memory_routes.py` (on add), `services/memory/memory_extractor.py`, `src/ai_interaction.py` (memory captured during chat). The only built-in consumer is the "Memory Tidy" automation (fires a `consolidate_memory` task every 5 adds). There is currently no `memory_updated` or `memory_deleted` event fired anywhere.
- Genuine per-session pub/sub does exist, but scoped to a single request's own output stream: `routes/chat_routes.py` + `src/agent_runs.py` implement `AgentRun.subscribers` as a `set` of `asyncio.Queue` (one per connected client), fed into a `StreamingResponse(media_type="text/event-stream")`. Similar SSE streaming exists in `routes/shell_routes.py`, `routes/model_routes.py`, `routes/research/research_routes.py`.
- Implication: there is no ready-made mechanism today to push "a memory changed elsewhere" to an open Memory Graph View. The design doc proposes either polling or a new lightweight per-owner SSE channel modeled directly on the `agent_runs.py` queue-per-subscriber pattern.
## 15. Authentication
- `core/auth.py::AuthManager` is JSON-file-backed (`data/auth.json` for users, `data/sessions.json` for session tokens), not a DB table.
- Two callable-facing auth mechanisms, both resolved in `AuthMiddleware` (in `app.py`, gated by `AUTH_ENABLED=true`):
- Cookie session auth: bcrypt-password-gated, 7-day TTL, sets `request.state.current_user` to the resolved username.
- Bearer API-token auth (`Authorization: Bearer ody_...`): matched against bcrypt-hashed `ApiToken` rows with scopes (`routes/api_token_routes.py` already defines `memory:read`/`memory:write` scopes); sets `request.state.current_user = "api"` plus `api_token_owner`/`api_token_scopes`.
- An internal-tool loopback path lets the agent's own HTTP tool calls reach admin-gated routes via `X-Odysseus-Internal-Token` / `X-Odysseus-Owner`.
- Multi-user support is real: `AuthManager.users` is a dict keyed by lowercase username, with admin-gated create/delete/rename, and reserved usernames that can never be created (`internal-tool`, `api`, `demo`, `system`).
- Privilege model: `DEFAULT_PRIVILEGES` includes `can_manage_memory` among others (`can_use_agent`, `can_use_documents`, `can_use_bash`, etc.). Admins get all privileges unconditionally. **Read-only memory routes (`GET /api/memory`, `/timeline`) do not require `can_manage_memory`** — only mutation (add/import) does. Per `THREAT_MODEL.md`, memory management is explicitly listed as available to both admins and non-admins, unlike shell/email/MCP/tokens/settings which are admin-only.
- Ownership pattern to reuse for a graph endpoint: resolve `owner = get_current_user(request)` (or `effective_user(request)` for Bearer-token callers), filter the memory load by that owner, and re-verify ownership on any caller-supplied id via the existing `_verify_memory_owner` helper rather than trusting the id in isolation.
## 16. Docker configuration
- Two-stage `Dockerfile` on `python:3.14-slim`, final image installs `build-essential, cmake, curl, git, nodejs, npm, chromium, tmux, openssh-client, gosu, libgl1, ...` plus the static Docker CLI (no daemon; host Docker socket is bind-mounted separately if enabled). Exposes port 7000; default CMD is `uvicorn app:app --host 0.0.0.0 --port 7000`, wrapped by `docker/entrypoint.sh`.
- `docker-compose.yml` composes four services:
- `odysseus` — the app container; volumes for `data`, `logs`, `.ssh`, `.cache/huggingface`, `.local`; huge environment block (LLM endpoints, embeddings, auth flags, upload limits, PUID/PGID); `depends_on: searxng (healthy), chromadb (started)`.
- `chromadb`**separate container**, image `chromadb/chroma:latest`, bound `127.0.0.1:8100:8000` on the host, named volume `chromadb-data`, `ANONYMIZED_TELEMETRY=FALSE`. No healthcheck (app relies on its own TCP probe + `heartbeat()` instead).
- `searxng` — pinned version, custom entrypoint templating `settings.yml`, healthcheck via Python urlopen, minimal Linux capabilities (`cap_drop: ALL` + a small `cap_add` set).
- `ntfy` — push-notification relay, bound `127.0.0.1:8091:80`.
- `docker/entrypoint.sh` implements the standard PUID/PGID drop-privilege pattern (create/reuse group+user matching host `PUID`/`PGID`, chown a bounded set of data directories, then `exec gosu $ODY_USER "$@"` so signals reach uvicorn directly).
- GPU overlays (`docker/gpu.nvidia.yml`, `docker/gpu.amd.yml`) and an opt-in `docker/host-docker.yml` (mounts the Docker socket) are pure compose overlays, not relevant to the Memory Graph View itself but relevant if it ever needs a background job (e.g. embedding recompute) that benefits from GPU access.
- Nothing about the graph feature requires new Docker services: no new database, no new container. It rides on the existing `odysseus` app container and the existing `chromadb` container.
## 17. Testing framework
- Backend: pytest, configured in `pyproject.toml` (`testpaths=["tests"]`, `asyncio_mode="auto"`, a fixed set of `area_*` taxonomy markers registered in `tests/_taxonomy.py`).
- `tests/conftest.py` is intentionally thin (repo convention: "prefer explicit local setup over hidden global fixtures", per `tests/README.md`). It forces `DATABASE_URL=sqlite:///:memory:` before any `core.database` import, pre-imports modules so later per-file mocking doesn't poison the real ORM for other tests, and stubs a fixed list of optional third-party deps if not installed. There is no shared `TestClient`/seeded-DB fixture.
- Route-testing convention (seen in `tests/test_memory_owner_isolation.py`, `tests/test_memory_routes_session_owner.py`): tests do not spin up a FastAPI `TestClient`/ASGI app. Instead they call the `setup_*_routes(...)` factory directly with real or `MagicMock()` dependencies, look up the target endpoint function off `router.routes` by path+method, and call it directly with a hand-built `Request` stand-in (`SimpleNamespace(state=SimpleNamespace(current_user=...))`). Auth is bypassed by monkeypatching `get_current_user`/`require_user`/`require_privilege` directly on the route module. Ownership tests assert `HTTPException(404)` on cross-owner access and that returned payloads only contain the caller's own data.
- `tests/helpers/` provides `sqlite_db.make_temp_sqlite` and `db_stubs.make_core_db_stub` for tests needing a real file-backed SQLite DB. `tests/run_focus.py` supports marker-based selective runs (e.g. `-m area_routes`).
- Frontend: no Jest/Vitest/Mocha/Playwright/Cypress. Pure-logic JS files are unit-tested via Node's built-in `node:test` runner, invoked as a subprocess from a thin pytest wrapper (e.g. `tests/test_streaming_segmenter_js.py` shells out to `node --test tests/streaming/*.test.mjs`), or via a `vm.createContext()` sandbox that string-shims `import`/`export` out of a production file before running it headlessly (`tests/markdown_codefence_placeholder_regression.mjs`). `tests/bombadil-spec.ts` is a separate fuzzing/property spec (Antithesis-style) for full-app exploration, not part of the normal pytest/CI gate. CI (`.github/workflows/ci.yml`) runs `node --check` (syntax only) over all frontend JS, plus `pytest -q` for everything else.
- Implication for the Memory Graph View: any pure-JS graph-layout/edge-derivation logic should get a `.test.mjs` file run via `node --test`, wrapped by a thin pytest shim, following the existing pattern. DOM/rendering behavior remains manually verified against the running app — there is no automated DOM test harness in this repo today.
## 18. Security posture (relevant to a new data-exposing endpoint)
- `THREAT_MODEL.md` frames Odysseus as a privileged local-access console for trusted users on a private network, not a public multi-tenant SaaS — but multi-user ownership isolation is still a real, tested boundary (see `tests/test_memory_owner_isolation.py`).
- `core/middleware.py`'s `SecurityHeadersMiddleware` already sets `X-Content-Type-Options`, `Referrer-Policy`, `Permissions-Policy`, HSTS (when HTTPS), and a nonce-based CSP (`script-src 'self' 'nonce-{nonce}' https://cdn.jsdelivr.net`) on all normal routes. A new graph view page doesn't need a special CSP branch unless it renders inline HTML/iframe content the way the visual-report/tool-render pages do.
- CORS is configured via `CORSMiddleware` with `allow_origins` from `ALLOWED_ORIGINS` (default `http://localhost,http://127.0.0.1`) and `allow_credentials=True`.
- Known gaps documented in `THREAT_MODEL.md` (no shell/filesystem sandbox for agent tools, an SSRF gap in chat `base_url`, coarse API token scopes) are not directly implicated by a read-only graph endpoint, but the "coarse token scopes" gap matters if the graph is ever exposed over a Bearer API token — it should reuse the existing `memory:read` scope rather than inventing a new one.
## 19. Safest extension points, summarized
Ranked from safest/most isolated to most invasive:
1. **New read-only backend route** `GET /api/memory/graph` inside the existing `routes/memory/memory_routes.py` module (or a new `routes/memory/memory_graph_routes.py` included the same way), reusing `_owner(request)` scoping and the existing `MemoryManager`/`MemoryVectorStore` singletons already wired up in `app_initializer.py`. No new tables, no new services, no new containers.
2. **New frontend module** `static/js/memoryGraph.js`, registered with `Modals.register(...)` the same way `memory-modal` already is, either as a new tab inside the existing Brain modal or a sibling modal reachable from it. No changes to `app.js`'s routing map are strictly required if it's a tab; one new `_routeOpen` entry if it's a standalone deep-linkable view.
3. **A vendored graph-rendering library** dropped into `static/lib/`, following the existing vendoring convention (no npm/bundler involvement).
4. **Optional: a new SSE channel** for live updates, modeled directly on `src/agent_runs.py`'s per-subscriber `asyncio.Queue` pattern, with new `memory_updated`/`memory_deleted` `fire_event()` calls added at the existing pin/update/delete call sites. This is additive and does not touch the existing task-automation consumer of `memory_added`.
5. **Not recommended as a first step**: touching the dormant SQL `memories` table, since it is not the runtime source of truth today and reconciling it would be a separate, larger migration unrelated to shipping a graph view.
No area inspected requires a new database engine, a schema migration to an existing hot-path table, or a new Docker service.

188
docs/memory-graph-design.md Normal file
View file

@ -0,0 +1,188 @@
# Memory Graph View — Design
Status: design only, not yet implemented. This proposal builds on the findings in `docs/memory-graph-analysis.md`. No application code, dependencies, or database state has been changed. Implementation should not begin until this design is reviewed and approved.
## Goals / non-goals
**Goals**: an interactive, pannable/zoomable node-link visualization of a user's own memories (Obsidian-graph-like), where nodes are memories and edges represent derived relationships (semantic similarity, shared session, shared category), reachable from the existing "Brain" memory modal, scoped strictly to the owning user.
**Non-goals for v1**: cross-user graphs, graphs spanning memories + documents + sessions in one view (a plausible v2 extension, not required now), real-time multi-tab collaborative editing of the graph, migrating the dormant SQL `memories` table into active use.
## 1. Architecture
The feature is additive and follows the existing "component bag" / manual-wiring conventions already used throughout the codebase — no new architectural pattern is introduced.
```
Browser (static/js/memoryGraph.js, new tab in #memory-modal)
│ fetch('/api/memory/graph?...')
FastAPI route: routes/memory/memory_graph_routes.py
setup_memory_graph_routes(memory_manager, memory_vector, session_manager)
Service: src/memory_graph.py (new, pure-ish module)
build_graph(owner, filters) -> {nodes, edges, meta}
│ │
▼ ▼
MemoryManager.load(owner=...) MemoryVectorStore.search(...) per-node kNN via Chroma
(data/memory.json, existing) (odysseus_memories collection, existing)
```
- **New backend module**: `routes/memory/memory_graph_routes.py`, mounted in `app.py` next to the existing `setup_memory_routes(...)` call, receiving the same already-constructed `memory_manager` and `memory_vector` singletons from `app_initializer.initialize_managers()`. No new singleton, no new startup step.
- **New service module**: `src/memory_graph.py` holds the graph-building logic (node assembly + edge derivation), kept separate from the route file so the edge-derivation logic is unit-testable without FastAPI in the loop (see §10).
- **New frontend module**: `static/js/memoryGraph.js`, imported from `static/app.js` alongside the existing `memory.js` import, following the same default-export-object + optional `window.*` exposure convention.
- **New tab**, not a new modal: added to the existing `.memory-tabs` strip in `index.html` (`Browse | Skills | Add | Settings | Graph`) and registered as a fifth tab panel inside `#memory-modal`. This reuses the modal's existing `Modals.register('memory-modal', ...)` entry, its drag/dock/minimize/z-order behavior, and its existing auth/ownership context — no new modal-manager registration needed. A standalone modal was considered and rejected for v1 (more registration/dock plumbing for no functional benefit; a tab is also the closer analogue to Obsidian's own "Graph view" living alongside the file browser).
- **Graph-rendering library**: recommend vendoring **Cytoscape.js** (single UMD file) into `static/lib/cytoscape.min.js`, following the existing vendoring convention used for `docx.umd.min.js`, `xlsx.full.min.js`, etc. Cytoscape is purpose-built for node-link graphs, ships a canvas renderer (needed for performance past a few hundred nodes — see §5), and bundles pan/zoom/drag interaction out of the box, so no additional libraries (separate zoom/drag/force packages) need to be vendored. D3 (force + zoom + drag composed manually) was considered as an alternative; Cytoscape was chosen because it is a single dependency rather than several composed low-level pieces, and its declarative style/selector API maps naturally onto category-based node styling. **This library choice is a recommendation, not a decision already made — flagging it explicitly for approval before implementation** (see Open Questions).
## 2. Database changes
**None required for v1.** Edges are computed on request from existing data (`data/memory.json` + the `odysseus_memories` Chroma collection); nothing new is persisted. This deliberately avoids two riskier paths identified in the analysis: touching the dormant SQL `memories` table (unused by the runtime today, so "fixing" it is an unrelated, larger migration) and adding a hand-rolled SQLite migration for a brand-new table.
If a later phase adds **explicit user-drawn links** (an Obsidian-style manual connection between two memories, as opposed to a derived one), the proposed change is still additive and still avoids SQL:
- Add an optional `links: []` array field to entries in `data/memory.json`, defaulting to `[]` when absent (`entry.get("links", [])`) so every existing entry remains valid without a migration pass.
- `MemoryManager` already owns read/write of this file; the change is a new optional key, not a schema/format break. Old app versions reading a file with this key simply ignore it (JSON is forward-tolerant), and rollback requires no data cleanup (see §9).
No changes to `core/database.py`, no new Alembic-equivalent `_migrate_add_*` function, no new SQLAlchemy model.
## 3. API design
New endpoints, added to the existing `/api/memory` prefix, following the file's existing conventions (owner-scoped via `get_current_user`/`effective_user`, 404-not-403 on cross-owner id access, `HTTPException` for errors):
### `GET /api/memory/graph`
Returns the full owner-scoped graph (nodes + derived edges), subject to server-side limits (see §5).
Query parameters (all optional):
| Param | Type | Default | Meaning |
|---|---|---|---|
| `category` | string, repeatable | none | Filter nodes to one or more of the existing `MEMORY_CATEGORIES` |
| `min_similarity` | float 01 | 0.75 | Minimum cosine similarity to draw a semantic edge |
| `max_edges_per_node` | int | 5 | Cap on nearest-neighbor edges per node (bounds edge count) |
| `include_session_edges` | bool | true | Whether to draw "extracted from the same session" edges |
| `include_category_edges` | bool | false | Whether to draw "same category" edges (off by default — high fan-out) |
| `limit` | int | 1000 | Max nodes returned (see §5 for behavior beyond this) |
| `since` | unix ts | none | Only include memories created/updated after this time (for incremental/lazy loading) |
Response shape:
```json
{
"nodes": [
{"id": "mem_123", "text": "...", "category": "preference", "pinned": false,
"uses": 3, "timestamp": 1769..., "session_id": "sess_abc"}
],
"edges": [
{"source": "mem_123", "target": "mem_456", "type": "similarity", "weight": 0.86},
{"source": "mem_123", "target": "mem_789", "type": "session", "weight": 1.0}
],
"meta": {"node_count": 214, "truncated": false, "generated_at": 1769...}
}
```
Auth/ownership: identical pattern to `GET /api/memory``require_user(request)` (no `can_manage_memory` needed, matching that read-only memory listing today requires no special privilege), owner resolved via `effective_user(request)` so Bearer-token callers are scoped correctly, and the response is built exclusively from `memory_manager.load(owner=user)` — no id-based lookup path exists on this endpoint, so there's no cross-owner leakage vector to guard beyond the load-time filter.
### `GET /api/memory/graph/{id}/neighbors`
Lazy-expansion endpoint for a single node's immediate neighbors, for progressive loading on large graphs (see §5). Reuses the existing `_verify_memory_owner(id, owner)` 404-on-mismatch helper before returning anything.
### `POST /api/memory/{id}/links` / `DELETE /api/memory/{id}/links/{target_id}` (phase 2, optional)
Manual link create/remove, gated by `require_privilege(request, "can_manage_memory")` exactly like other memory mutations (`PUT`/`DELETE /api/memory/{id}`). Not required for v1; documented here so the API surface is planned coherently rather than bolted on later.
### API token scope
The graph read endpoint is exposed under the **existing** `memory:read` scope already defined in `routes/api_token_routes.py` — no new scope is introduced, consistent with the analysis's note about avoiding further scope proliferation.
## 4. UI design
**Entry point**: new "Graph" tab in the existing `.memory-tabs` strip (`index.html`), alongside Browse/Skills/Add/Settings. Selecting it lazily creates the Cytoscape instance on first open (not on modal load) to avoid any cost for users who never click it.
**Canvas**: fills the tab panel body, resizing with the modal (a `ResizeObserver` on the panel calls `cy.resize()` + `cy.fit()`, since `windowDrag.js`/`tileManager.js` already support the user resizing/tiling the modal itself).
**Node styling** (mirrors the existing Browse-tab visual language rather than inventing a new one):
- Fill color by category, reusing the same category color mapping already defined for the Browse tab's category chips/badges in `style.css`.
- Size scaled by `uses` (more-referenced memories render larger, echoing their real weight in chat context injection).
- Pinned memories get a distinct border/ring, matching the pinned badge already used in the list view.
- Node label: truncated memory text (first ~40 chars), full text on hover tooltip.
**Edge styling**:
- Semantic-similarity edges: solid line, opacity/thickness scaled by `weight` (cosine similarity).
- Same-session edges: dashed line, fixed low opacity, to visually separate "structurally related" from "semantically related" without cluttering the primary read.
- A small legend (reusing the existing modal's compact panel styling) explains the two edge types and lets the user toggle each on/off — mapped directly to the `include_session_edges`/`include_category_edges` query params.
**Interactions**:
- Pan/zoom: native Cytoscape gestures.
- Click node: opens a side panel (or reuses the existing inline-edit affordance from the Browse tab) showing full text, category, pin state, use count, and Pin/Edit/Delete actions — deliberately reusing `memory.js`'s existing edit/pin/delete logic rather than re-implementing it, so behavior (and any future changes to it) stays in one place.
- Double-click node: same inline-edit flow already used in the Browse tab.
- Drag node: repositions it client-side only (not persisted), matching how Obsidian's own graph behaves — dragged position is not sent back to the server.
- Search box (reusing the existing `#memory-search` pattern): highlights/centers matching nodes instead of filtering a list.
- Category filter chips (reusing the existing Browse-tab chip component): toggle node visibility by category.
- Similarity threshold slider: re-queries `min_similarity` and redraws edges (debounced, see §5).
- Click-to-isolate: clicking a node dims everything outside its connected component, a common Obsidian-graph affordance, implemented purely client-side against the already-loaded graph (no extra request).
**Empty/loading/error states**: follow existing modal conventions — spinner (`spinner.js`) while the initial `GET /api/memory/graph` is in flight, an empty-state message reusing the Browse tab's "no memories yet" copy/style when the user has zero memories, and a plain inline error message (no full-modal takeover) on request failure, consistent with how other tabs degrade.
**Accessibility note**: a pure canvas graph is not screen-reader navigable; the existing Browse tab remains the accessible list view of the same data, so the Graph tab is explicitly an additional visualization, not a replacement — nothing about memory access requires the graph to work.
## 5. Performance considerations
- **Server-side edge computation must avoid O(n²) pairwise similarity.** Use Chroma's own ANN query per node (`MemoryVectorStore.search(text, k=max_edges_per_node)`) rather than brute-force pairwise cosine over all memories. This is a set of N approximate-nearest-neighbor queries (already fast, since Chroma is built for this), not N² raw comparisons.
- **Cap response size.** Default `limit=1000` nodes; if an owner has more memories than that, the endpoint returns the most-recent/most-used `limit` nodes plus `meta.truncated=true`, and the frontend shows a "showing N of M — refine filters to see more" notice rather than silently dropping data. The `/graph/{id}/neighbors` endpoint exists precisely so a user can drill into the truncated remainder on demand instead of the server ever needing to return everything at once.
- **Canvas over SVG.** Cytoscape's canvas renderer is required, not optional, once node count exceeds roughly 200300 — an SVG-per-node approach (as some hand-rolled D3 examples use) degrades badly at that scale by DOM node count alone. This is why Cytoscape (canvas-first) was preferred over a naive D3+SVG approach in §1.
- **Debounce filter changes.** The similarity-threshold slider and category toggles should debounce their re-fetch (e.g. 250ms) rather than firing a request per slider tick.
- **Server-side short-lived cache.** Cache the computed `{nodes, edges}` graph per owner (in-process, e.g. a small dict keyed by `owner` with a TTL of a minute or two, or invalidated eagerly on `memory_added`/`memory_updated`/`memory_deleted` if the optional SSE work in §1 of the analysis is picked up later). This avoids recomputing kNN edges on every tab-open within a short window, at negligible memory cost given the existing single-process assumption already baked into the app.
- **Target scale.** This is a personal/local-assistant memory store, not a general knowledge base — the design explicitly targets smooth interaction for roughly 2002,000 memories per user, with graceful truncation (not failure) beyond that, rather than engineering for unbounded scale from day one.
## 6. Scalability
- **Multi-user isolation carries no new risk**: the graph endpoint reads through the same `memory_manager.load(owner=user)` owner filter every other memory route already uses, so per-user data volume and per-user isolation both scale exactly as well (or as poorly) as the existing memory list/search endpoints do today.
- **No new infrastructure dependency.** No new database engine, no new container (confirmed in the analysis: ChromaDB is already a separate container the app already depends on). Scaling the graph feature scales exactly with scaling the app's existing single-process/single-SQLite/single-Chroma-container deployment model — this proposal does not change that model or its known limits.
- **Phase-2 scaling lever (only if needed)**: if a user's memory count grows large enough that on-request kNN computation becomes noticeably slow even with caching, the next lever is an incrementally-maintained nearest-neighbor cache (refreshed only for the new/changed memory on each `memory_added`/`memory_updated`, not recomputed from scratch), rather than anything to do with the graph *rendering* — the render side is already the cheap part once node count is capped per §5.
## 7. Security
- **No new privilege boundary.** `GET /api/memory/graph` requires only `require_user`, matching the existing `GET /api/memory` and `/timeline` endpoints — consistent with `THREAT_MODEL.md` explicitly listing memory management as available to non-admins. Mutating endpoints (optional manual links, phase 2) require `can_manage_memory`, matching existing memory-mutation routes.
- **No new cross-owner leakage vector.** The list endpoint has no id-based lookup path (built entirely from an owner-filtered load), and the neighbors/link endpoints reuse the existing `_verify_memory_owner` 404-on-mismatch helper rather than introducing a new ownership check pattern.
- **XSS hygiene in rendering.** Memory text is user-authored/LLM-extracted content rendered into node labels/tooltips — it must go through the same escaping helper already used elsewhere in the frontend (`uiModule.esc()` per the existing markdown-rendering convention) before being placed in any DOM tooltip; Cytoscape's own canvas node labels are not DOM-injected and are safe by construction, but any HTML side-panel showing full memory text on click must still escape it.
- **API token scope reuse, not expansion.** Exposing the read endpoint under the existing `memory:read` scope avoids adding to the token-scope surface, per the analysis's note on the "coarse scopes" gap already tracked in `THREAT_MODEL.md`.
- **Light rate limiting.** Even in a trusted-local-network threat model, add a light per-owner limit (reusing `src/rate_limiter.py`, the same mechanism already used for login) on `GET /api/memory/graph` so a misbehaving client/tab can't hammer Chroma with repeated kNN queries — cheap insurance, not a response to a specific known threat.
- **No prompt-injection surface introduced.** This is a display endpoint; it does not feed memory text back into an LLM prompt, so the existing `untrusted_context_message()` wrapping requirement (which already governs memory text reaching the agent loop) is unaffected and does not need to be duplicated here.
## 8. Migration strategy
No data migration is required for v1 (§2). The rollout is staged purely at the feature-flag / code level:
- **Phase 1 — backend only.** Ship `routes/memory/memory_graph_routes.py` + `src/memory_graph.py`, mounted in `app.py`, with no frontend entry point yet. Fully testable via the existing route-factory test convention (§10) and manual `curl`, with zero user-visible surface — the lowest-risk possible increment.
- **Phase 2 — frontend behind an opt-in setting.** Ship the "Graph" tab gated by a Settings-tab toggle (e.g. "Enable Memory Graph (beta)"), defaulting **off**, so existing users see no change until they opt in. This mirrors how a genuinely new, unproven interaction pattern should be introduced into an app whose UI conventions (flat list, modal-based navigation) this feature is a first departure from.
- **Phase 3 — default-on.** After a soak period with no material bug reports, flip the default to on and drop the "(beta)" label. The toggle itself can remain as a permanent "hide Graph tab" preference for users who simply don't want it, or be removed — a call to make at that time, not now.
- **Phase 4 — optional extensions**, proposed separately and not part of this design's approval scope: manual linking (§2/§3 phase-2 bits), live SSE-based updates (per the analysis's §14 finding that no such channel exists today), and cross-linking memories to the documents/sessions they were extracted from.
Each phase is independently shippable, independently revertable, and does not block on the next phase being designed yet.
## 9. Rollback strategy
- **Backend rollback**: the new route module is additive and isolated — removing its `include_router(...)` call (or the whole file) from `app.py` has zero blast radius on any other route, since it shares only already-existing, already-stable singletons (`memory_manager`, `memory_vector`) and defines no new schema. Rollback is a plain code revert.
- **Frontend rollback**: with the Phase-2 opt-in flag, disabling the flag hides the tab immediately with no deploy needed. A full revert (removing the module, the tab markup, and the flag) is likewise a plain code revert — no other module should come to depend on `memoryGraph.js` internals, which should be verified (grep for imports of it) before merging any phase past Phase 1.
- **No data to roll back.** Because v1 stores nothing new (§2), there is no "undo a migration" step, no backfill to reverse, and no risk of orphaned rows or half-migrated JSON — rollback is strictly a code-level operation at every phase of v1.
- **If the optional `links` JSON field (§2, phase-2 extension) ships later**: rolling back the code that reads/writes it leaves harmless, inert data behind (`links: []` or a populated array), which older `MemoryManager` code already tolerates by construction (unknown/extra JSON keys are never rejected) — no cleanup pass is required unless the user explicitly wants the field purged, which would be a separate, deliberate data-cleanup action, not an automatic part of rollback.
## 10. Testing strategy
- **Backend route tests** (`tests/test_memory_graph_routes.py`, new): follow the repo's established convention exactly — build the router via `setup_memory_graph_routes(...)` directly (no `TestClient`/ASGI app), look up the endpoint function off `router.routes` by path, call it directly with a hand-built `Request` stand-in, and bypass auth via `monkeypatch.setattr(memory_graph_routes, "get_current_user", ...)` / `require_user`. Required cases:
- Owner isolation: a graph built for owner A contains none of owner B's memories (mirrors `tests/test_memory_owner_isolation.py`).
- `min_similarity`/`max_edges_per_node`/`category` filters change the returned edge/node sets as expected.
- `limit`/`meta.truncated` behavior when an owner has more memories than the limit.
- `/graph/{id}/neighbors` 404s on a foreign-owned id (reusing the existing `_verify_memory_owner` test pattern).
- **Edge-derivation unit tests** (`tests/test_memory_graph_edges.py`, new): the similarity/session/category edge-building logic in `src/memory_graph.py` should be a pure function taking pre-fetched memory + neighbor data and returning edge lists, so it's table-driven-testable without Chroma or FastAPI in the loop — pass in synthetic embeddings/neighbor lists and assert threshold and top-k truncation behavior deterministically.
- **Frontend pure-logic tests**: any pure-JS piece of `memoryGraph.js` that isn't DOM-dependent (mapping an API response into Cytoscape `elements`, category→color lookup, client-side connected-component isolation on click) gets a `.test.mjs` file run via `node --test`, wrapped by a thin pytest shim — following the exact precedent of `tests/test_streaming_segmenter_js.py` / `tests/streaming/invariant.test.mjs`.
- **Manual/DOM verification**: since the repo has no DOM/browser test harness (§17 of the analysis), pan/zoom/drag/click interactions and visual correctness (category colors, edge styling, legend, resize behavior) are verified manually against the running dev server before merge, and this should be stated explicitly in the PR description rather than implied — consistent with how the repo already documents this limitation for other rendering-heavy modules.
- **Performance smoke test**: seed a temporary `MemoryManager` with a few hundred and a few thousand synthetic entries and assert `GET /api/memory/graph` completes within a defined latency budget with a warm cache — a regression guard against an accidental reintroduction of O(n²) pairwise comparison.
- **Marker tagging**: new tests get the existing `area_routes`/`area_services` markers per `tests/_taxonomy.py` so they're picked up by the same `-m area_routes` selective-run convention the rest of the suite already uses.
## Open questions requiring a decision before implementation
1. **Graph library**: confirm Cytoscape.js (recommended, §1) vs. a hand-composed D3 stack vs. a from-scratch canvas renderer. This is the single highest-leverage decision in the whole design.
2. **Tab-in-modal vs. standalone modal**: confirmed recommendation is a tab inside the existing Brain modal (§1); flag if a standalone full-screen view is actually wanted instead.
3. **Manual linking (phase 2)**: is user-drawn explicit linking between memories in scope at all, or should the feature stay purely derived-edges-only indefinitely? This affects whether the `links` JSON field (§2) is ever needed.
4. **Opt-in beta flag vs. shipping straight to default-on**: confirmed recommendation is opt-in first (§8); flag if the team prefers to skip that staging given how isolated the feature already is.
No further action will be taken until this design (and the open questions above) are reviewed.