New routes/memory/memory_graph_routes.py, mounted in app.py:
- GET /api/memory/graph — owner-scoped node/edge graph (require_user,
no new privilege beyond existing GET /api/memory).
- GET /api/memory/graph/{id}/neighbors — lazy single-node expansion for
graphs beyond the response limit.
- POST /api/memory/{id}/links, DELETE /api/memory/{id}/links/{target_id}
— manual relationship editing, gated by can_manage_memory like other
memory mutations, reusing the existing 404-on-owner-mismatch pattern.
Must be included before memory_router in app.py: memory_routes.py's
GET/PUT/DELETE /api/memory/{memory_id} wildcard would otherwise swallow
GET /api/memory/graph, since Starlette matches routes in registration
order, not by specificity. A dedicated TestClient-based regression test
(test_memory_graph_route_ordering.py) locks this in — the repo's usual
"call the endpoint function directly" test style can't catch this class
of bug since it looks up routes by exact path string, not by simulating
real request matching.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Slice 2j of the route-domain reorganization (#4082/#4071). Moves
search_routes.py into routes/search/, leaving a backward-compat
sys.modules shim. Pure file reorganization, no behavior change.
Slice 2i of the route-domain reorganization (#4082/#4071). Moves
compare_routes.py into routes/compare/, leaving a backward-compat
sys.modules shim at the old path. Pure file reorganization, no behavior
change.
The shim uses sys.modules replacement so the `import ... as cr` +
`monkeypatch.setattr(cr, "SessionLocal", ...)` / `"_owned_endpoint_by_url"`
/ `"_owned_endpoint_by_id"` pattern in test_endpoint_owner_scope_followup.py
reaches the canonical module.
Canonical module imports only from core/, src/, and routes.session_routes
(zero dependency on the legacy shim). One source-introspection test site
repointed: test_endpoint_owner_scope_followup.py (shared with other domains;
only the compare entry repointed here).
Adds tests/test_compare_routes_shim.py to pin the sys.modules shim
contract. Verified: compileall clean; targeted tests pass.
Slice 2h of the route-domain reorganization (#4082/#4071). Moves
admin_wipe_routes.py into routes/admin_wipe/, leaving a backward-compat
sys.modules shim at the old path. Pure file reorganization, no behavior
change.
The shim uses sys.modules replacement so the `import ... as
admin_wipe_routes` + `monkeypatch.setattr(admin_wipe_routes, "SessionLocal",
...)` / `"require_admin"` pattern in test_admin_wipe_gallery.py reaches
the canonical module.
Canonical module imports only from core/, src/, and stdlib (zero internal
routes/ coupling). Zero source-introspection landmines.
Adds tests/test_admin_wipe_routes_shim.py to pin the sys.modules shim
contract. Verified: compileall clean; targeted tests pass.
Slice 2g of the route-domain reorganization (#4082/#4071). Moves
cleanup_routes.py into routes/cleanup/, leaving a backward-compat
sys.modules shim at the old path. Pure file reorganization, no behavior
change.
The shim uses sys.modules replacement so string-targeted
monkeypatch.setattr("routes.cleanup_routes.*", ...) in
test_cleanup_owner_scope.py reaches the canonical module.
Canonical module imports only from src/ and stdlib (zero internal
routes/ coupling). Zero source-introspection landmines.
Adds tests/test_cleanup_routes_shim.py to pin the sys.modules shim
contract. Verified: compileall clean; targeted tests pass.
Slice 2f of the route-domain reorganization (#4082/#4071, per
specs/architecture-runtime-inventory.md §6.3). Moves note_routes.py into
routes/note/, leaving a backward-compat sys.modules shim at the old path.
Pure file reorganization, no behavior change.
The shim uses sys.modules replacement (same pattern as the merged gallery
#4903, research #4975, memory #5007, history #5090, and contacts #5227
slices) so that `import routes.note_routes`, `from routes.note_routes import
X`, `importlib.import_module(...)`, and the `import ... as note_routes` +
`monkeypatch.setattr(note_routes, "SessionLocal", ...)` pattern used by
test_note_reminder_fire_scope.py / test_notes_fail_closed_auth.py all
operate on the same module object the application uses.
The canonical module does NOT depend on the shim — routes/note/note_routes.py
imports only from core/, src/, and stdlib. The outbound email cross-domain
imports (routes.email_routes._get_email_config, routes.email_helpers.
_send_smtp_message) are function-local lazy imports that keep resolving
through the email module's own path (email is not yet migrated).
One source-introspection test site repointed to the new canonical path:
- test_model_helper_owner_scope.py (shared with history; history entry
already repointed in #5090, note entry repointed here)
Adds tests/test_note_routes_shim.py to pin the sys.modules shim contract
(legacy and canonical paths resolve to the same module object; monkeypatch
via legacy alias reaches the canonical module).
Verified: compileall clean; full suite 4487 passed, 3 skipped.
Slice 2e of the route-domain reorganization (#4082/#4071, per
specs/architecture-runtime-inventory.md §6.3). Moves contacts_routes.py into
routes/contacts/, leaving a backward-compat sys.modules shim at the old path.
Pure file reorganization, no behavior change.
The shim uses sys.modules replacement (same pattern as the merged gallery
#4903, research #4975, memory #5007, and history #5090 slices) so that
`import routes.contacts_routes`, `from routes.contacts_routes import X`,
`importlib.import_module(...)`, the string-targeted
`monkeypatch.setattr("routes.contacts_routes.SETTINGS_FILE", ...)` used by
test_carddav_password_encryption.py, and the `import ... as cr` +
`setattr(cr, ...)` pattern in test_contacts_add_null_name.py all operate on
the same module object the application uses. This also keeps the mutable
module state `_contact_cache` identical across import paths.
The canonical module does NOT depend on the shim — routes/contacts/
contacts_routes.py imports only from core/, src/, and stdlib (zero internal
routes/ coupling). The inbound edge from routes/email_helpers.py (imports
_fetch_contacts) keeps working through the shim.
Zero source-introspection landmines — no test reads this file by path.
Adds tests/test_contacts_routes_shim.py to pin the sys.modules shim contract
(same-object + string-targeted monkeypatch reach-through).
Verified: compileall clean; full suite 4485 passed, 3 skipped.
Slice 2d of the route-domain reorganization (#4082/#4071, per
specs/architecture-runtime-inventory.md §6.3). Moves history_routes.py into
routes/history/, leaving a backward-compat sys.modules shim at the old path.
Pure file reorganization, no behavior change.
The shim uses sys.modules replacement (same pattern as the merged gallery
#4903, research #4975, and memory #5007 slices) so that `import
routes.history_routes`, `from routes.history_routes import X`,
`importlib.import_module(...)`, and the `import ... as history_routes` +
`monkeypatch.setattr(history_routes, ...)` pattern used by
test_history_compact_tool_calls.py / test_fork_session_metadata.py all
operate on the same module object the application uses.
The canonical module does NOT depend on the shim — routes/history/
history_routes.py imports only from core/, src/, and routes.session_routes
(a sibling route module whose old import path stays valid via its own shim
when session is migrated later).
Three source-introspection test sites repointed to the new canonical path:
- test_history_db_fallback_hidden.py
- test_history_order_by_timestamp_regression.py
- test_model_helper_owner_scope.py
Adds tests/test_history_routes_shim.py to pin the sys.modules shim contract
(legacy and canonical paths resolve to the same module object; monkeypatch
via legacy alias reaches the canonical module).
Verified: compileall clean; full suite 4351 passed, 3 skipped.
Slice 2c of the route-domain reorganization (#4082/#4071, per
specs/architecture-runtime-inventory.md §6.3). Moves memory_routes.py into
routes/memory/, leaving a backward-compat sys.modules shim at the old path.
Pure file reorganization, no behavior change.
The shim uses sys.modules replacement (same pattern as the merged gallery
#4903 and research #4975 slices) so that `import routes.memory_routes`,
`from routes.memory_routes import X`, `importlib.import_module(...)`, and
the `import ... as mr` + `monkeypatch.setattr(mr, ...)` pattern used by
test_memory_routes_session_owner.py / test_memory_owner_isolation.py all
operate on the same module object the application uses.
The canonical module does NOT depend on the shim — routes/memory/
memory_routes.py imports only from services/, core/, src/, and stdlib (zero
internal routes/ coupling).
Four source-introspection test sites repointed to the new canonical path:
- test_direct_upload_limits.py
- test_upload_limits_centralized.py (two dict keys)
- test_vision_owner_scope.py
Adds tests/test_memory_routes_shim.py to pin the sys.modules shim contract
(legacy and canonical paths resolve to the same module object; monkeypatch
via legacy alias reaches the canonical module).
Verified: compileall clean; full suite 4219 passed, 3 skipped.
Move the research route domain into the canonical routes/research/ subpackage while preserving the legacy routes.research_routes import path through a sys.modules compatibility shim.
The moved canonical module is behavior-preserving, app wiring now imports the canonical route setup function, source-introspection tests point at the new canonical path, and shim regression coverage pins legacy/canonical same-object behavior plus string-targeted monkeypatch reach-through.
Refs #4082.
Refs #4071.
Follow-up to #4637. serve_index — the handler for / and the SPA deep-link
routes (/notes, /calendar, /cookbook, /email, /memory, /gallery, /tasks,
/library) — pre-checked os.path.exists and raised its own
HTTPException(404, "index.html not found") when the bundle was missing. So a
missing core template returned 404 before serve_html_with_nonce's 500 could
fire, the one inconsistency left after #4637.
index.html is a fixed, app-bundled template; a missing one is a broken
deployment (server fault), not a client "not found", so it should surface as a
logged 500 in 5xx alerting rather than a 404. Keep the static->root fallback,
drop the redundant existence guard and the dead-end 404, and let the shared
helper handle the missing case.
Verified against the running app: / and /notes return 200 with the bundle
present and a logged 500 when index.html is absent.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(routes): serve 404 instead of 500 when an HTML page file is missing
_serve_html_with_nonce opened the HTML file with no error handling, and
callers such as /backgrounds and /login pass their paths in with no
existence check, so a missing or unreadable file raised an unhandled
OSError that surfaced as a 500. Wrap the read and raise HTTPException(404)
instead; the normal render path (CSP-nonce substitution) is unchanged.
Fixes#4594
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(routes): distinguish missing page (404) from read failure (500)
The previous fix caught a broad OSError and returned 404 for every
failure, which masks real server-side problems (permission errors, I/O
failures) as "not found" and lets them slip past error alerting. Split
FileNotFoundError (genuine 404) from other OSError, which now logs the
exception and returns a generic 500 — without leaking the OS error
string or file path into the response body.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(routes): treat unreadable bundled HTML page as logged 500, not 404
Per PR #4637 review: every caller of the page-render helper serves a fixed,
server-owned template (index/login/backgrounds), never a client-supplied
path. So a missing or unreadable file is a server fault (broken deployment),
not a client "not found" — a 404 there mislabels a server error and hides a
missing core template from 5xx alerting, contradicting the OSError->500
rationale this PR is built on. Collapse both branches into a single logged,
leak-free 500.
Move the helper to src.app_helpers.serve_html_with_nonce so the behavior can
be unit-tested without importing the whole app (app.py is the slim
orchestrator; the test harness stubs src.database, so importing app in tests
is not viable). Add tests pinning missing/unreadable -> 500 (not 404) and
nonce injection on the happy path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The in-process tool loopback stamps current_user = "internal-tool" and
require_admin grants admin to that sentinel; it is also a reserved username.
That security-sensitive string was hand-typed in ~7 places (stamp, admin gate,
RESERVED_USERNAMES, and standalone admin-equivalent checks in note/research/
shell/task routes), where a typo silently breaks an auth gate.
Add INTERNAL_TOOL_USER in core/middleware.py next to INTERNAL_TOOL_TOKEN/
INTERNAL_TOOL_HEADER and use it at every such site. A typo is now an
ImportError, not a silent mismatch. auth.py importing middleware is acyclic
(middleware imports no app modules). Behaviour is unchanged.
The multi-sentinel sets bundling internal-tool with api/demo/system
(assistant_routes, task_scheduler, research_routes) are a separate reserved-set
dedup, left for a follow-up.
Closes#4332
* log(app): add warnings to silent except Exception blocks
- Internal tool auth header failure now logs a warning instead of
silently passing, making auth bypass easier to spot in logs.
- Token last_used_at update failure now logs at DEBUG (fire-and-forget,
non-critical, but useful when debugging token tracking issues).
- Image ownership verification failure now logs a warning so unexpected
access-check errors surface instead of silently allowing the request.
* log(chat_routes): add warnings to silent except Exception blocks
- clear_orphaned_session_endpoint: log before rollback so failures
appear in traces when users see stale/deleted model options.
- _endpoint_has_model (JSON parse): log malformed cached_models instead
of silently treating endpoint as valid.
- _has_any_visible_model (JSON parse): log malformed cached_models
instead of silently returning empty list.
- timezone header parse: log failure so time-zone-related tool bugs
(wrong scheduled times, calendar events) are traceable.
- attachments JSON parse: log failure so silently-dropped attachments
are visible in server logs.
* log(email_routes): add warnings to silent except Exception blocks
- Email alias resolution failure now logs a warning instead of silently
returning an empty list, making broken account configs diagnosable.
* log(document_routes): add warnings to silent except Exception blocks
- Export ZIP request body parse failure now logs a warning so empty
exports caused by malformed requests are diagnosable.
- clear_active_document failure on detach now logs a warning to help
trace doc re-injection bugs like #1160.
* log(agent_loop): add warnings to silent except Exception blocks
- builtin tool overrides load failure now logs a warning so misconfigured
settings don't silently fall back to defaults without a trace.
- Timezone context injection failure now logs a warning to help debug
incorrect scheduled times in agent-created tasks.
- PDF form-backed document detection failure now logs a warning so
broken form-doc UI is traceable to the root cause.
* log(llm_core): add warnings to silent except Exception blocks
- Malformed URL in _is_ollama_native_url now logs a warning so bad
endpoint configs are traceable instead of silently returning False.
- Model list fetch failure now logs a warning with the endpoint URL so
endpoints that silently vanish from the model picker are diagnosable.
* log: pass exception via exc_info instead of string interpolation
* fix(logging): avoid logging raw URLs in llm_core error paths
Drop the raw url/base_chat_url from the Ollama-detection and
model-list-fetch warning logs added by this sweep, since these values
can contain private hostnames, internal IPs, credentials, or other
deployment details.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(agent): workspace confinement via context-local binding + get_workspace tool
Bind the per-turn workspace once in execute_tool_block; the shared path
resolvers (_resolve_tool_path / _resolve_search_root) and the subprocess cwd
helper (agent_cwd) read it, so file tools + bash/python are confined centrally
and a new tool that uses the shared helpers cannot accidentally bypass it.
Adds the admin-gated /api/workspace/browse picker, a workspace pill + directory
modal (reusing existing modal/button CSS), the /workspace slash command, and a
get_workspace tool (replaces a system-prompt block). Confinement is OS-agnostic
(realpath/normcase/commonpath) and docker-safe (container paths, no host
assumptions). Reopens#2023.
* ux(workspace): clarify workspace is not a sandbox
Picker modal note + pill tooltip + get_workspace tool/output wording now state
plainly: read_file/write_file/edit_file/grep/glob/ls are confined to the folder,
but bash/python only start there (cwd) and are not sandboxed. Modal note reuses
the existing .muted class.
* fix(agent): treat an active workspace as file-work intent
A vague low-signal message (e.g. "look at the local project") matches no
domain keywords, so tool retrieval is skipped and only always-available tools
are offered — leaving the agent with no file access even though a workspace is
set. When a workspace is active, include the file/code tools (incl.
get_workspace) on low-signal turns so the agent can act on the folder.
Also requires the tool index (ChromaDB) to be reachable for normal retrieval;
that is an environment dependency, not part of this change.
* ux(workspace): hide pill + overflow entry in chat mode
Workspace only scopes the agent's file/shell tools, so the pill and the
overflow 'Workspace' entry are agent-only now — hidden in chat mode like the
bash toggle. Mode read from the DOM in syncWorkspaceIndicator; applyMode() is
called from the agent/chat setMode handler.
* prompt(tools): steer bash/python to defer to the dedicated file tools
bash/python schema descriptions (what native-tool-calling models read) were
bare and gave no steer, so models would do file ops via the shell (e.g. writing
SVG/HTML, which then dumps raw markup into the tool preview). Tell bash/python
in the schema + tool-index + prompt section to prefer read_file/write_file/
edit_file/grep/glob/ls and only be used for what those do not cover.
* prompt(tools): keep bash/python deferral generic (no hardcoded tool names)
Reference 'a dedicated tool' rather than listing read_file/write_file/grep/etc.
by name, so the guidance does not go stale if those tools are renamed.
* style(workspace): drop em-dashes from added code comments/strings
* ux(workspace): terser non-sandbox note in picker (no tool-name list)
* ux(workspace): mirror terse non-sandbox wording in pill tooltip
* chore: untrack local venv symlink (run-only, not part of the feature)
* prompt(workspace): keep get_workspace text generic (no hardcoded tool names)
* fix(agent): low-signal + workspace surfaces only read-only file tools
Intersect the files tool group with PLAN_MODE_READONLY_TOOLS so a vague message
in a workspace exposes read_file/grep/glob/ls/get_workspace for exploration, but
not write_file/edit_file/bash/python -- those wait for a request that actually
calls for them (RAG retrieval still adds them on a real ask).
* feat(workspace): cap browse listing at 500 dirs with a truncated hint
Mirror the filesystem_tools._CODENAV_MAX_HITS pattern with a module-local
_MAX_BROWSE_DIRS so a directory with thousands of children does not dump every
row into the picker; the response carries a truncated flag and the modal tells
the user to type a path to jump in.
* chore: untrack local venv symlink (run-only artifact)
* fix(workspace): vet the workspace root against the sensitive-path deny list at bind time
The in-workspace resolver deny-lists sensitive paths inside the workspace,
but the empty-path search root is the workspace itself, so a workspace of
~/.ssh could be listed via ls with no path. vet_workspace() (public, in
tool_execution next to the resolvers) rejects non-directories and sensitive
roots before the path is ever bound; chat_routes uses it instead of its
inline isdir check.
* fix(workspace): reject filesystem roots and stop showing rejected workspaces as active
Review findings from #3665:
P2: vet_workspace accepted / (and would accept drive/UNC roots), which makes
every absolute path 'inside' the workspace and collapses confinement into
host-wide file access. A root is its own dirname, so reject when
dirname(resolved) == resolved; the browse response now carries a selectable
flag and the picker disables 'Use this folder' on unselectable dirs.
P3: /workspace set stored any string client-side and the chat route silently
dropped rejected values, so the pill could claim a confinement that was not
in effect. New admin-gated /api/workspace/vet validates manual paths before
they persist (canonical path returned), and when a posted workspace is
rejected at send time the stream emits workspace_rejected so the client
clears the stored value and toasts instead of continuing silently.
* fix(workspace): check caller privilege before vetting the posted workspace
Review finding: /api/chat_stream called vet_workspace() on the posted value
for every caller and emitted workspace_rejected on failure, so a non-admin
who can chat but cannot use file/shell tools could distinguish existing
directories from missing/file/sensitive/root paths by whether the event
appeared. The resolution now lives in _resolve_request_workspace, which
drops the submitted value uniformly for non-admin callers, with no vetting
and no event, before the path ever touches the filesystem. Admin and
single-user behavior is unchanged. Test pins that valid and invalid paths
are indistinguishable for a non-admin and that vet_workspace is never
invoked for them.
_warmup_endpoints called model_discovery.get_endpoints(), which does not exist
on ModelDiscovery. It raised AttributeError on every startup and on every 60s
keepalive tick, was swallowed by the outer except, and pinged nothing, so the
cold-start prevention the loop exists for never ran.
Add ModelDiscovery.warmup_ping_urls(), which resolves the /models probe URLs
from the real discover_models() output, and call it from the warmup loop via
asyncio.to_thread (discovery does a blocking port scan, so keep it off the event
loop).
Adds tests/test_warmup_ping_urls.py: resolves /models URLs from discovered
items, honors the limit, degrades to [] on discovery failure, and documents that
get_endpoints never existed.
The frontend's text assets shipped uncompressed on every cold load. Add
Starlette's GZipMiddleware. Measured on the current assets:
- style.css 1,127 KB -> 238 KB (-79%)
- index.html 202 KB -> 35 KB (-83%)
- chat.js 238 KB -> 60 KB (-75%)
minimum_size=1024 skips tiny bodies; Starlette excludes `text/event-stream` by
default, so the SSE streams (chat, shell, research, model-probe — all served with
media_type="text/event-stream") are never compressed or buffered. Composes
cleanly with the existing security-header middleware. No behavioural change.
Built by OdWar -- with Claude thinking alongside.
* Add consolidated service health endpoint for degraded-state reporting
ROADMAP (High Priority) asks for "Better degraded-state reporting for
ChromaDB, SearXNG, email, ntfy, and provider probes." Until now there was no
single readout of which subsystems are actually working: /api/health is only a
liveness ping and each subsystem's signal lives in a different module, so a
misconfigured self-host install gives no consolidated picture.
This adds an admin-only GET /api/diagnostics/services endpoint backed by a new
src/service_health.py aggregator. Each subsystem reports a uniform
{name, status, detail, meta} where status is ok | degraded | down | disabled,
and the response rolls up an overall verdict (worst non-disabled status).
Probes are deliberately non-intrusive and safe to poll:
- ChromaDB: reads the .healthy flags on the RAG and memory vector stores.
- SearXNG: GET /healthz (2xx), falling back to the instance root (<500). No
search query is run.
- ntfy: GET the server's built-in /v1/health. No test notification is sent.
- email: short IMAP connect+logout per configured account (no credentials in
meta).
- providers: probe each enabled ModelEndpoint's model list (no api_key in meta).
Probe functions take their inputs as parameters and isolate the network call to
injectable callables, so they unit-test without touching the network (same
pattern as the merged provider-endpoint tests). Network probes run concurrently
off the event loop via asyncio.to_thread with bounded per-probe timeouts.
memory_vector is now passed into setup_diagnostics_routes (new optional param,
backward-compatible) so ChromaDB's vector-memory store can be reported too.
Tests: tests/test_service_health.py — 29 tests covering every status mapping
per subsystem, the overall rollup, and that no secrets leak into meta.
Verification:
python -m pytest tests/test_service_health.py -q # 29 passed
python -m py_compile src/service_health.py routes/diagnostics_routes.py app.py
python -m pytest tests/test_endpoint_resolver.py tests/test_provider_endpoints.py -q
Backend + tests only; an Admin/Settings UI badge that renders this endpoint is
a natural follow-up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(diagnostics): bound service-health wall-clock and redact secrets
Addresses review on #964.
Blocker 1 — genuinely bounded wall-clock:
- providers_health and email_health now fan out per-item probes across a
bounded thread pool (_bounded_map) with a hard total budget (_FANOUT_BUDGET),
instead of probing endpoints/accounts sequentially. Stragglers are reported
as a controlled `timeout` and never block; the pool is shut down with
wait=False so the response returns on time regardless of endpoint/account
count.
- The IMAP connect path now honors the service-health budget: _imap_connect
gained a pass-through `timeout` param and the probe calls it with
_PROBE_TIMEOUT instead of the default 15s.
- collect_service_health runs the four network subsystems concurrently, each
under a per-subsystem deadline (_SUBSYSTEM_DEADLINE), with an overall
wait_for ceiling (_AGGREGATE_DEADLINE) as a backstop.
Blocker 2 — no secret/raw-error leakage in the response:
- _safe_url strips userinfo, query, and fragment from every URL surfaced in
meta (searxng instance, ntfy base, provider name fallback), keeping only
scheme/host/port/path.
- _classify_error maps every probe failure to a controlled category token
(timeout, connection_refused, dns_error, tls_error, network_error,
http_error, auth_or_protocol_error, …) — raw str(exception), which can embed
credentialed URLs or server text, is never returned.
Tests (tests/test_service_health.py, +tests/test_diagnostics_service_route.py):
- URL userinfo/query redaction for searxng/ntfy/providers.
- secret-bearing exception strings map to categories and don't leak.
- multiple slow providers/accounts stay bounded (single + 25-endpoint cases).
- subsystems run concurrently; aggregate deadline yields a controlled result.
- route-level unauthenticated (401) / non-admin (403) / admin (200) coverage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(diagnostics): isolate route tests so they don't leak module globals
The new route tests replaced src.service_health.collect_service_health and
routes.diagnostics_routes.require_admin via direct assignment, which persisted
for the rest of the pytest session. In CI's full alphabetical run that fake
collector (returning services=[]) leaked into the later collect_service_health
tests and failed them. Switch to monkeypatch.setattr so both are restored after
each test. No production code change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
* docs: add implementation plan for fixing chat context drifting (#135)
* fix: make Session.history immutable + fix {}.history crash
- Session.history now exposes a COPY of the internal _history list
- add_message() replaces history with a fresh copy each time
- get_context_messages() derives from _history directly
- replace_messages() updates both _history and history
- truncate_messages() updates both _history and history
- _persist_message() line 207: fixed {}.history fallback crash
- Added 11 tests for session isolation and edge cases
Addresses #135 root cause #1: shared mutable references
* fix: task scheduler uses SessionManager methods instead of overwriting sessions
- Added ensure_task_session() to SessionManager (checks cache first)
- Task scheduler now uses ensure_task_session() instead of direct dict assignment
- Task scheduler now uses SessionManager.add_message() for message persistence
- Removed direct sess_obj.history.append() that was silently losing data
Addresses #135 root causes #2 and #3
* fix: add age guard to cleanup_empty_sessions — don't delete sessions <1h old
Prevents the cleanup task from deleting sessions that were just created
and haven't received any messages yet (message_count == 0).
Addresses #135 root cause #5
* test: comprehensive session isolation tests (10/10 passing)
* refactor: consolidate _session_manager into singleton pattern
- Added set_session_manager_instance / get_session_manager_instance to core/models
- kept backward-compat aliases (set_session_manager, get_session_manager)
- session_manager.py re-exports the singleton functions
- ai_interaction.set_session_manager now syncs with the core singleton
- context_compactor uses get_session_manager_instance() instead of getattr hack
- app.py initializes the singleton once
Addresses #135 root cause #4: fragile global wiring
* test: add concurrent session isolation integration tests
Verifies:
- Concurrent add_message to different sessions doesn't cross-contaminate
- Rapid parallel writes maintain isolation
- Read-write concurrent access is safe
All 3 async tests pass, proving the immutable history fix works under concurrency
* fix: pre-import core.models in conftest to prevent test pollution
test_agent_loop.py stubs sys.modules['core.models'] = MagicMock() at
module level during collection. Any test collected after it imports
Session as a MagicMock. Pre-importing core.models in conftest.py
before test_agent_loop.py's module-level code runs prevents this.
* fix: make .history authoritative mutable list, address PR review
Per review feedback: keep .history as the authoritative mutable list so
existing code doing .history.pop(), .history = [...], etc. still works.
Fix the cross-contamination bug by ensuring __post_init__() gives each
Session its OWN unique history list (never shared).
Changes:
- core/models.py: .history IS the authoritative list. _history aliases it.
Each Session gets its own list in __post_init__.
- core/session_manager.py: add_message() delegates to Session.add_message()
instead of appending directly — no double-append, single source of truth.
- tests/test_session_manager.py: updated test to reflect that .history
references see new messages (same list, not a snapshot).
- docs/plans/2026-06-01-fix-chat-context-drifting.md: removed (not for
shipping — useful design context but too much process/doc to ship).
All 272 tests pass (3 pre-existing failures unrelated).
* Fix session manager message persistence
* Fix session history alias regressions
* Fix session history aliasing and task delivery
The DB owner-rename loop in rename_user patched every SQL column named
owner, but three non-SQL stores were left behind:
1. session_manager.sessions -- in-memory Session objects carry s.owner
set at server-boot time. get_sessions_for_user() does an exact
s.owner == username check, so the renamed user chat sidebar goes empty
until a server restart.
2. data/deep_research/*.json -- each completed research report is a
standalone JSON file with an owner field. research_routes filters
by d.get(owner) == user, making every report invisible to the
renamed user.
3. data/memory.json -- a flat JSON array; each entry carries an owner
field. memory_manager.load(owner=user) filters on it, so all memories
vanish from the memory panel.
Fix: after the SQL loop, patch all three:
- iterate sm.sessions and update owner in-place (exposed via app.state)
- walk data/deep_research/*.json and rewrite owner with atomic_write_json
- update matching entries in memory.json with atomic_write_json
All three use the same case-insensitive lower() comparison the SQL loop
already uses. Each step is independently wrapped so a single failure
does not abort the others or the rename itself.
Fixes#3362
Surface a lot of accumulated cookbook + UI work as a single non-agent
commit so the agent rework lands cleanly.
Highlights:
- Ollama as a first-class backend in the Cookbook:
* Download input accepts ollama-style names (name:tag) → backend=ollama
* /api/cookbook/ollama/library (cached scrape of ollama.com + curated
fallback so classic models like qwen2.5 stay reachable)
* "Browse Ollama library" toggle below Download with size chips
* Engine=Ollama in hwfit toolbar merges the Ollama library into the
main scan list as per-tag rows with the same Fit/Param/Quant/VRAM
columns; click → fills Download input
- API Tokens form added to Integrations panel (matching wired
loadTokens()/initTokenForm() that had no HTML)
- Serve panel polish: Advanced fold tightening (-8px nudges on vLLM
checks, Extra args, Spec row), n_cpu_moe + Split Mode controls
pulled up 8px to align with the row's checkboxes, GGUF File dropdown
exposed for Ollama backend, GPU re-render on Edit serve restore,
_forceBackend flag so saved serveState wins over backend detection,
cookbook:servers-changed CustomEvent so panels don't need refresh
- Models page redesign: Add Models row (URL + hidden API key reveal +
Type select + Scan/Ollama/Key/Test/Add icon buttons), Probe All +
Clear-offline buttons in Added Models toolbar, offline-pill removed
(opacity already conveys state), Engine dropdown gains Ollama option
- _ping_endpoint probes /v1/models then base, accepts 4xx as
reachable (vLLM returns 404 on bare /v1, fully working endpoints
were showing offline)
- Diagnosis card: × dismiss + Copy bundle buttons restored on the
serve error feedback card
- Orphan tmux sweep re-enabled behind a 60s rate-limit + background
Thread (off the main event loop) so dead serves get discovered
- cookbook_routes auto-register watchdog: drops the endpoint if the
serve session exits non-zero within the first ~3min
- ollama-rocm sidecar awareness in download wrapper (`docker exec
ollama-rocm ollama pull` when host ollama isn't installed)
- Skill extractor sets initial_status="published" when
auto_approve_skills pref is on (audit demotes later)
- Skill list / model list / cookbook scan misc polish
* feat: Add ChatGPT Subscription support and related features
- Introduced a new provider option for ChatGPT Subscription in the endpoint selection UI.
- Implemented OAuth flow for ChatGPT Subscription sign-in, including polling for authorization status.
- Updated admin interface to handle ChatGPT Subscription, including disabling API key input and providing user guidance.
- Enhanced cost tracking logic to differentiate between subscription and non-subscription endpoints.
- Added new slash commands for managing skills, including listing, searching, and invoking skills.
- Implemented caching for skill catalog to optimize performance.
- Updated tests to cover new ChatGPT Subscription functionality and ensure proper endpoint probing.
- Refactored existing code to accommodate new features and improve maintainability.
* refactor: share provider device-flow setup
- reuse one device-flow backend for Copilot and ChatGPT Subscription
- add one frontend device-flow helper for Settings and /setup
- put GitHub Copilot back into Add Models, now as a dropdown option
- make provider selection just select; clicking Add starts sign-in
- stop ChatGPT Subscription setup from opening auth tabs automatically
- make /setup copilot and /setup chatgpt-subscription work from chat
- show ChatGPT Subscription in the /setup suggestions
- show the real error message when setup fails
- add focused tests for the shared flow and setup UI
* feat(chatgpt-subscription): harden credential lifecycle and streamline auth UX
Backend:
- Resolve runtime bearer for provider-auth endpoints at probe time via a
shared _resolve_probe_key() that delegates to resolve_endpoint_runtime,
applied across all probe/refresh call sites.
- Skip live completion probes and health pings for discovery-only providers
(centralized behind _is_discovery_only_provider) — the Codex/Responses API
has no such endpoints, so status is derived from cached models.
- Never persist the short lived ChatGPT bearer to the plaintext sessions
table; proactively clear any stale bearer left by an earlier code path.
- Revoke orphaned ProviderAuthSession credentials when the last endpoint
backing them is deleted (_delete_orphaned_provider_auth), surfaced via
cleared_provider_auth in the delete response.
Frontend (admin.js):
- Auto-start the device-auth flow on provider selection so the authorization
panel (code + Authorize) shows immediately instead of behind a "Sign in" click.
- Remove the redundant top button for device auth providers, move retry
into the panel via an inline "Try again".
- Drop the self-evident hint text and add an execCommand clipboard fallback so
Copy works in non-secure (HTTP/LAN) contexts.
* fix: harden chatgpt subscription provider
* chore: remove PR media from branch
* Fix chatgpt subscription recovery and token handling
---------
Co-authored-by: 5p00kyy <admin@5p00ky.dev>
* refactor(constants): single source of truth for data dir + merge core/src constants
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(contributing): use named src.constants for data paths, drop core/constants references
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
AuthMiddleware is the outermost middleware, so a credential-less CORS preflight
(OPTIONS + Access-Control-Request-Method) was rejected with 401 before
CORSMiddleware could answer it. That blocks every cross-origin browser/WebView
client: the preflight fails, so the real request is never sent.
Let a genuine preflight through at the top of AuthMiddleware.dispatch via a pure,
unit-tested predicate (core.middleware.is_cors_preflight). Precise -- only
OPTIONS carrying Access-Control-Request-Method; a credentialed request is never
matched -- and no data access.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Bring main's maintainer-curated work (cookbook scheduler, calendar rendering/sync, settings polish, agent debug loop) into dev so dev is a superset of main (resolves the dev/main drift, #2543).
- Schedule cookbook serves through the existing ScheduledTask system: the
serve preset gets a ^ button next to Launch that opens a daily/hourly/
weekly form mirroring the admin-switch style; the schedule action runs
action_cookbook_serve, which delegates to /api/model/serve and stamps
the resulting task with _scheduledStopAtMs. A background
cookbook_serve_lifecycle loop ticks every 60s and kills any serve
whose window has ended, also dropping the auto-registered endpoint
so the model picker doesn't keep pointing at a dead server.
- Stop and remove on a Running serve now awaits the SSH/tmux kill,
re-checks tmux has-session, and surfaces an error toast (leaving the
row) when the kill failed. Previously fire-and-forget, so a failed
SSH/tmux call silently left the live serve running while the row
vanished from the UI.
- Cookbook tasks/status orphan-adoption sweep no longer requires the
serve-/cookbook- session-id prefix; any tmux session whose pane is
running a known model-server process gets auto-pulled into Running.
Without this loosening, a cookbook-launched serve whose tmux id
fell back to a bare number was invisible — you couldn't see it,
let alone stop it.
- Ollama serve always launches a fresh process under cookbook's tmux
(no more monitor-mode reattach to a systemd/Docker ollama Stop can't
reach). The handler pre-picks a free port by probing the target
host over SSH and mutates req.cmd's OLLAMA_HOST so the runner script
AND the auto-registered endpoint agree on the same bind port.
- Auto-register uses host.docker.internal (when running inside Docker)
instead of localhost, matching the URL /setup adds for Ollama by
hand. Local cookbook serves now produce a chat-reachable endpoint
on first launch.
- Cascade-delete: removing a scheduled cookbook task also deletes any
linked calendar event (cookbook_task_id marker in the description).
- Tasks list groups cookbook_serve under a "Cookbook" category that
sorts above the rest, so scheduler-launched serves are easy to find.
* feat: Add workspace: confine agent tools to a folder
Pick a server folder as the agent's workspace so its file/shell tools work
there and don't touch files outside it. File tools are hard-confined; bash/
python run with cwd set to the folder.
Includes a slash command: `/workspace` (alias `/ws`) — show / `set <path>` /
`clear` / `pick` (open the directory browser).
- routes/workspace_routes.py: GET /api/workspace/browse (admin-only).
- src/tool_execution.py: hard path confinement for read_file/write_file;
bash/python cwd. Threaded route → stream_agent_loop → execute_tool_block.
- src/agent_loop.py: workspace note prepended to the system prompt.
- static/: overflow menu item, input-bar pill, directory-browser modal, and
the /workspace slash command.
- tests/test_workspace_confine.py.
* Wire workspace confinement into tools that landed after this PR
edit_file (#1239) and grep/glob/ls (#1670) merged after workspace-confine was
written, so they bypassed the workspace boundary. Thread the workspace through:
- edit_file: _do_edit_file resolves via _resolve_tool_path_in_workspace
- grep/glob/ls: _resolve_search_root confines to the workspace (root + paths)
- bash/python/bg cwd: workspace or _AGENT_WORKDIR (keep the #2586 data-dir
default when no workspace is set)
Tests cover edit_file + grep/ls confinement (inside ok, outside rejected).
* Workspace picker: editable path bar + modal style cohesion + cross-platform hardening
- Make the current-folder strip an editable address bar: type/paste a full
path and press Enter to navigate (also reaches other Windows drives and
hidden dirs the up-only browser cannot).
- Reuse shared modal CSS: drop bespoke .workspace-modal-content/.workspace-btn*
in favour of base .modal-content/.modal-body and the .confirm-btn button
family; separators/hover use var(--border). Net -31 CSS lines.
- Fix the path field overflowing the modal right edge (flex stretch + margin
vs an overflow:auto scrollbar-feedback loop): full-bleed, no h-margin.
- Cross-platform confinement: normcase the workspace commonpath check so
containment holds on case-insensitive filesystems (Windows/macOS).
- Make tests OS-portable: sibling temp dirs instead of /etc, python os.getcwd()
instead of pwd. 5 pass.
