Slice 2m of the route-domain reorganization (#4082/#4071, per
specs/architecture-runtime-inventory.md §6.3). Moves document_routes.py
(1810 lines) and document_helpers.py (243 lines) into routes/document/,
leaving backward-compat sys.modules shims at the old paths. Pure file
reorganization, no behavior change.
Both shims use sys.modules replacement so the `import ... as droutes` +
`droutes.SessionLocal = ...` / `monkeypatch.setattr(droutes, ...)` pattern
in multiple tests, and the `sys.modules.pop("routes.document_helpers")` +
re-import pattern in test_security_regressions.py, all reach the canonical
modules.
The canonical document_routes.py imports helpers from the canonical path
(routes.document.document_helpers), not the legacy shim.
Three source-introspection test sites repointed to the new canonical path:
- test_imap_mailbox_quoting.py
- test_model_helper_owner_scope.py
- test_vision_owner_scope.py (shared with other domains; document entry repointed)
Adds tests/test_document_routes_shim.py to pin the sys.modules shim contract
for both modules.
Verified: compileall clean; full suite 4789 passed, 3 skipped.
Slice 2l of the route-domain reorganization (#4082/#4071). Moves
webhook_routes.py into routes/webhook/, leaving a backward-compat
sys.modules shim. Pure file reorganization, no behavior change.
One source-introspection test repointed (test_api_chat_security.py).
Slice 2k of the route-domain reorganization (#4082/#4071). Moves
vault_routes.py into routes/vault/, leaving a backward-compat
sys.modules shim. Pure file reorganization, no behavior change.
The version pattern in _anthropic_rejects_temperature() required a minor
component, so major-only ids like `claude-opus-5` never matched and the
guard reported that the model accepts `temperature`. Anthropic rejects the
field outright on Opus 4.7+, so every such call returned HTTP 400 and the
stream aborted with zero tokens ("the model returned an empty response").
Make the minor optional and read a missing minor as `.0`. The major is also
capped at 1-2 digits with a no-trailing-digit lookahead, mirroring the
minor: once the minor is optional, a greedy major would swallow the date in
`claude-3-opus-20240229` and read it as version 20240229, dropping
temperature from a model that accepts it.
Fixes#5753
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
The placeholder-restore pass in mdToHtml put code, math, mermaid and
allowed-HTML blocks back with a string replacement, so String.replace read
`$&`, `` $` ``, `$'` and `$$` in the *replacement* as substitution patterns.
A fenced block containing them rendered corrupted: `$&` re-inserted the
placeholder (`perl -pe 's/world/$& again/'` became
`s/world/___CODE_BLOCK_0___amp; again/`), `` $` `` and `$'` spliced in the
surrounding document, and `$$` collapsed to a single `$`.
Pass a function replacer at all four sites, matching the inline-code site
below them, which was already fixed this way. A function's return value is
inserted verbatim with no `$` interpretation.
The inline-code comment claimed `echo $1` would be read as a back-reference;
with a string search value there are no capture groups, so `$1` is already
literal. Reworded to name the four sequences that do corrupt.
Fixes#5663
* fix(skills): replace deprecated utcnow in skill timestamp helper
_now_iso() builds the 'created' value in skill frontmatter. datetime.utcnow()
returns a naive datetime and has been deprecated since Python 3.12, scheduled
for removal. Switch to the timezone-aware datetime.now(timezone.utc), keeping
the serialized YYYY-MM-DDTHH:MM:SSZ shape unchanged so existing skill files
keep parsing.
timezone.utc is used rather than the datetime.UTC alias, which is 3.11+ only.
Adds regression tests covering the deprecation, the serialized shape, and
UTC correctness under a non-UTC local timezone -- the last guards against a
bare datetime.now(), which yields the same shape but local wall time.
Fixes#5697
* test(skills): skip timezone mutation where unsupported
---------
Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
Skill tests are background automation tasks (like auto-naming and
memory audit) and should use the configured utility model. Previously
they resolved via resolve_endpoint("default") which returned the
chat model, bypassing the utility model entirely.
This completes the sweep started in PR #4027 which fixed auto-naming
and memory audit but missed skill tests.
The issue-close lifecycle change is narrowly scoped and correct. Closed issues remove the stale \`ready for review\` label and return before normal validation can restore it. Focused regressions cover closure and subsequent edits to a closed issue.
The branch was updated onto current \`dev\`. The focused test, merged-result validation, diff checks, and GitHub CI passed. No blocking review threads remain.
Slice 2j of the route-domain reorganization (#4082/#4071). Moves
search_routes.py into routes/search/, leaving a backward-compat
sys.modules shim. Pure file reorganization, no behavior change.
* fix(rag): skip hidden and junk directories when indexing (#5559)
index_personal_documents walked the whole tree with no pruning, so
pointing RAG at a real-world folder silently swept in .obsidian/ plugin
JS, .git/ internals, node_modules/, and __pycache__/ — multiplying
indexing time and polluting retrieval with junk chunks.
Prune hidden directories and well-known junk directories from the walk,
and skip hidden files. The explicitly passed root is exempt, so a user
who deliberately indexes a hidden directory still gets its contents.
* fix(rag): prune hidden/junk dirs in the keyword index too, via a shared helper
The #5559 fix pruned only VectorRAG.index_personal_documents (the vector index).
The parallel keyword index built by PersonalDocsManager.refresh_index ->
load_personal_index walked the same tree unpruned, so .obsidian/, .git/,
node_modules/ etc. still swept into keyword retrieval and the file listing —
the 'end-to-end' guarantee was only half true.
Single-source the pruning policy in src/index_walk (prune_index_dirs +
is_indexable_file) and use it from both walkers so they cannot drift again.
The junk-dir match is now case-insensitive, so a Node_Modules on a
case-insensitive filesystem is pruned too.
Tests: keyword-path regressions covering hidden/junk dirs, hidden files, junk
at depth (not just top level), case-insensitive junk, and the explicit-hidden-
root exemption. The existing vector tests still pass against the shared helper.
* fix(email): use XOAUTH2 in test-connection for Google OAuth accounts
The test-connection endpoint was password-only and had no awareness of
OAuth accounts. For Google-connected accounts this caused two failures:
- IMAP: "Need IMAP host, username, and password" because imap_pass is
empty (no password is stored for OAuth accounts)
- SMTP: 535 BadCredentials because smtp.login() was called with an
empty password instead of an XOAUTH2 token
Fix: include oauth_provider and token fields in saved_body when hydrating
from the DB, then use conn.authenticate("XOAUTH2") / smtp.auth("XOAUTH2")
for Google accounts in both the IMAP and SMTP test paths, mirroring what
_send_smtp_message already does for real sends.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(email): add OAuth2 tests for test-connection endpoint
Covers the XOAUTH2 changes made to routes/email_routes.py:
- Google OAuth accounts must not be rejected with 'Need IMAP host,
username, and password' (no stored password for OAuth accounts)
- IMAP and SMTP test paths must use conn.authenticate('XOAUTH2')
for Google accounts
- Password accounts must still use conn.login() / smtp.login()
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(email): bind OAuth account tests to Google transport
---------
Co-authored-by: TNTBA <trynottobreakanything@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(config): forward Google OAuth env vars into Docker container and document setup
GOOGLE_OAUTH_CLIENT_ID, GOOGLE_OAUTH_CLIENT_SECRET, and GOOGLE_OAUTH_REDIRECT_URI
were read by the app but never forwarded through docker-compose.yml's explicit
environment allowlist, causing the "not set" error even when the vars existed in .env.
Also adds a documented section to .env.example with step-by-step GCP setup instructions
so users know where to get the credentials.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(config): cover OAuth in standalone compose files
* test(config): parse OAuth compose service env
* test(config): keep checkout skip wording neutral
---------
Co-authored-by: TNTBA <trynottobreakanything@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Slice 2i of the route-domain reorganization (#4082/#4071). Moves
compare_routes.py into routes/compare/, leaving a backward-compat
sys.modules shim at the old path. Pure file reorganization, no behavior
change.
The shim uses sys.modules replacement so the `import ... as cr` +
`monkeypatch.setattr(cr, "SessionLocal", ...)` / `"_owned_endpoint_by_url"`
/ `"_owned_endpoint_by_id"` pattern in test_endpoint_owner_scope_followup.py
reaches the canonical module.
Canonical module imports only from core/, src/, and routes.session_routes
(zero dependency on the legacy shim). One source-introspection test site
repointed: test_endpoint_owner_scope_followup.py (shared with other domains;
only the compare entry repointed here).
Adds tests/test_compare_routes_shim.py to pin the sys.modules shim
contract. Verified: compileall clean; targeted tests pass.
Slice 2h of the route-domain reorganization (#4082/#4071). Moves
admin_wipe_routes.py into routes/admin_wipe/, leaving a backward-compat
sys.modules shim at the old path. Pure file reorganization, no behavior
change.
The shim uses sys.modules replacement so the `import ... as
admin_wipe_routes` + `monkeypatch.setattr(admin_wipe_routes, "SessionLocal",
...)` / `"require_admin"` pattern in test_admin_wipe_gallery.py reaches
the canonical module.
Canonical module imports only from core/, src/, and stdlib (zero internal
routes/ coupling). Zero source-introspection landmines.
Adds tests/test_admin_wipe_routes_shim.py to pin the sys.modules shim
contract. Verified: compileall clean; targeted tests pass.
Slice 2g of the route-domain reorganization (#4082/#4071). Moves
cleanup_routes.py into routes/cleanup/, leaving a backward-compat
sys.modules shim at the old path. Pure file reorganization, no behavior
change.
The shim uses sys.modules replacement so string-targeted
monkeypatch.setattr("routes.cleanup_routes.*", ...) in
test_cleanup_owner_scope.py reaches the canonical module.
Canonical module imports only from src/ and stdlib (zero internal
routes/ coupling). Zero source-introspection landmines.
Adds tests/test_cleanup_routes_shim.py to pin the sys.modules shim
contract. Verified: compileall clean; targeted tests pass.
Slice 2f of the route-domain reorganization (#4082/#4071, per
specs/architecture-runtime-inventory.md §6.3). Moves note_routes.py into
routes/note/, leaving a backward-compat sys.modules shim at the old path.
Pure file reorganization, no behavior change.
The shim uses sys.modules replacement (same pattern as the merged gallery
#4903, research #4975, memory #5007, history #5090, and contacts #5227
slices) so that `import routes.note_routes`, `from routes.note_routes import
X`, `importlib.import_module(...)`, and the `import ... as note_routes` +
`monkeypatch.setattr(note_routes, "SessionLocal", ...)` pattern used by
test_note_reminder_fire_scope.py / test_notes_fail_closed_auth.py all
operate on the same module object the application uses.
The canonical module does NOT depend on the shim — routes/note/note_routes.py
imports only from core/, src/, and stdlib. The outbound email cross-domain
imports (routes.email_routes._get_email_config, routes.email_helpers.
_send_smtp_message) are function-local lazy imports that keep resolving
through the email module's own path (email is not yet migrated).
One source-introspection test site repointed to the new canonical path:
- test_model_helper_owner_scope.py (shared with history; history entry
already repointed in #5090, note entry repointed here)
Adds tests/test_note_routes_shim.py to pin the sys.modules shim contract
(legacy and canonical paths resolve to the same module object; monkeypatch
via legacy alias reaches the canonical module).
Verified: compileall clean; full suite 4487 passed, 3 skipped.
* security(url-safety): reject RFC 6598 shared address space in strict mode
Strict mode (block_private=True) is a full SSRF lockdown, but it only
rejected is_private and is_loopback targets. CPython does not classify RFC
6598 shared/CGNAT space (100.64.0.0/10) as is_private (it is "shared", not
"private"), so a public redirect into 100.64.0.1 passed the per-hop guard
and still issued the request to a potentially internal CGNAT service.
not is_global would also exclude it, but only on CPython 3.11.10+/3.12.4+/
3.13+; the CI matrix runs 3.11/3.12, so reject the range explicitly to stay
correct across patch levels and the 3.14 runtime image. Default local-first
mode is unchanged. Adds strict-mode coverage for shared, non-global, and
public targets.
* docs(url-safety): correct CGNAT is_global rationale in strict-mode comment
The prior comment claimed `not is_global` catches 100.64.0.0/10 only on
CPython 3.11.10+/3.12.4+/3.13+. That is inaccurate for CGNAT: is_global
is False for 100.64.0.1 on every supported version (verified 3.10-3.14).
The version-fragility applies to other ranges gh-113171 touched, not CGNAT.
The explicit range reject is still the right choice; restate the reason as
is_private not covering shared space, and not coupling strict mode to
is_global's broader, cross-version definition. No behavior change.
* feat(models): define capability schema and readers
* fix(models): harden Google catalog probing
Restrict native catalog probing to the Gemini host, keep provider keys out of request URLs, filter non-chat model resources, and preserve the manual refresh default in the built-in Google add flow.
Add Arch-specific package installation and NVIDIA runtime configuration
to the Docker setup guide. Cover passthrough verification, the NVIDIA
Compose overlay, and the distinction between GPU passthrough and
CUDA-backed model serving
Refs #831
* Harden skill importer against SSRF: block private targets + revalidate redirects per hop
The skill importer validated only the initial URL with the lenient SSRF guard
(block_private=False) and then fetched with follow_redirects=True, so a 3xx to
an internal/metadata address (169.254.169.254, 127.0.0.1, RFC-1918) was still
connected to — inconsistent with the hardened services/search/content.py
:_get_public_url path.
Add a _get_checked() helper that follows redirects manually and re-runs the
SSRF guard with block_private=True on every hop, and route all three fetch
sites (skills.sh unwrap, _fetch_bytes, _list_github_dir) through it. GitHub's
own redirects and the final-host _assert_github_url checks are preserved.
Adds hermetic regression tests (IP-literal hosts, faked HTTP layer) and updates
the existing mock signature for the new block_private kwarg.
Defense-in-depth: the endpoint is admin-gated (require_admin) and admins are
trusted per THREAT_MODEL.md, so this is not a cross-boundary vulnerability.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test: enforce follow_redirects=False invariant in mock client
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
