The version pattern in _anthropic_rejects_temperature() required a minor
component, so major-only ids like `claude-opus-5` never matched and the
guard reported that the model accepts `temperature`. Anthropic rejects the
field outright on Opus 4.7+, so every such call returned HTTP 400 and the
stream aborted with zero tokens ("the model returned an empty response").
Make the minor optional and read a missing minor as `.0`. The major is also
capped at 1-2 digits with a no-trailing-digit lookahead, mirroring the
minor: once the minor is optional, a greedy major would swallow the date in
`claude-3-opus-20240229` and read it as version 20240229, dropping
temperature from a model that accepts it.
Fixes#5753
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
The placeholder-restore pass in mdToHtml put code, math, mermaid and
allowed-HTML blocks back with a string replacement, so String.replace read
`$&`, `` $` ``, `$'` and `$$` in the *replacement* as substitution patterns.
A fenced block containing them rendered corrupted: `$&` re-inserted the
placeholder (`perl -pe 's/world/$& again/'` became
`s/world/___CODE_BLOCK_0___amp; again/`), `` $` `` and `$'` spliced in the
surrounding document, and `$$` collapsed to a single `$`.
Pass a function replacer at all four sites, matching the inline-code site
below them, which was already fixed this way. A function's return value is
inserted verbatim with no `$` interpretation.
The inline-code comment claimed `echo $1` would be read as a back-reference;
with a string search value there are no capture groups, so `$1` is already
literal. Reworded to name the four sequences that do corrupt.
Fixes#5663
* fix(skills): replace deprecated utcnow in skill timestamp helper
_now_iso() builds the 'created' value in skill frontmatter. datetime.utcnow()
returns a naive datetime and has been deprecated since Python 3.12, scheduled
for removal. Switch to the timezone-aware datetime.now(timezone.utc), keeping
the serialized YYYY-MM-DDTHH:MM:SSZ shape unchanged so existing skill files
keep parsing.
timezone.utc is used rather than the datetime.UTC alias, which is 3.11+ only.
Adds regression tests covering the deprecation, the serialized shape, and
UTC correctness under a non-UTC local timezone -- the last guards against a
bare datetime.now(), which yields the same shape but local wall time.
Fixes#5697
* test(skills): skip timezone mutation where unsupported
---------
Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
Skill tests are background automation tasks (like auto-naming and
memory audit) and should use the configured utility model. Previously
they resolved via resolve_endpoint("default") which returned the
chat model, bypassing the utility model entirely.
This completes the sweep started in PR #4027 which fixed auto-naming
and memory audit but missed skill tests.
The issue-close lifecycle change is narrowly scoped and correct. Closed issues remove the stale \`ready for review\` label and return before normal validation can restore it. Focused regressions cover closure and subsequent edits to a closed issue.
The branch was updated onto current \`dev\`. The focused test, merged-result validation, diff checks, and GitHub CI passed. No blocking review threads remain.
Slice 2j of the route-domain reorganization (#4082/#4071). Moves
search_routes.py into routes/search/, leaving a backward-compat
sys.modules shim. Pure file reorganization, no behavior change.
Follow-up hardening beyond the explicit review findings:
- Propagate session revocation across uvicorn workers: token
validation now syncs issuance AND revocation from sessions.json
(mtime-gated), _save_sessions merges on-disk state under an
inter-process flock so concurrent workers can't lose each other's
sessions, and revocation tombstones prevent a just-revoked token
from being re-merged.
- Restrict sessions.json and auth.json to 0600 (bearer tokens and
password hashes; same policy as data/app.db, #4420), applied
atomically at write time and retroactively at load.
- Password-login session cookie: SECURE_COOKIES=false can no longer
downgrade the cookie when the request arrived over HTTPS (spoofable
X-Forwarded-Proto still requires TRUST_PROXY_HEADERS opt-in).
- Document why OIDC state tokens are deliberately not single-use and
which mechanisms bound the replay window.
- Warn once per process (not twice per login) when
OIDC_ALLOW_INSECURE_COOKIES is enabled; pass the variable through
the Compose files so the documented dev override actually reaches
containers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GRiLb12nnLnBnYsg14oSWd
Addresses RaresKeY's review (5 findings) and follow-up Basic RP
validation comment on PR #3508:
- Add PKCE (RFC 7636, S256): code_challenge in the authorization
request, verifier carried in the Fernet-encrypted state, and
code_verifier sent to the token endpoint.
- Require the iat claim in id_tokens (OIDC Core §2); tokens without
iat are now rejected.
- Prefer client_secret_basic at the token endpoint per discovery
(OIDC default), falling back to client_secret_post only when the
provider excludes basic.
- Require HTTPS for the issuer and authorization endpoint, not just
the back-channel endpoints.
- Preserve OIDC subs exactly (no strip) so distinct whitespace-bearing
subjects can never collapse into one local account; same for the
UserInfo sub-binding comparison.
- Sync admin state only on a well-formed groups claim; UserInfo
availability alone (or a malformed groups value) no longer demotes
an existing admin.
- OIDC session/CSRF cookies are Secure by default regardless of
SECURE_COOKIES; explicit OIDC_ALLOW_INSECURE_COOKIES=true is the
only (documented, dev-only) opt-out.
- Make sessions issued by one uvicorn worker validate on others via
an mtime-gated read-through reload of sessions.json.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GRiLb12nnLnBnYsg14oSWd
All auth.json mutation methods now acquire _interprocess_auth_lock
(flock-based) + _config_lock with reload-before-save, matching the
pattern already used by create_user / create_user_oidc /
set_oidc_user_admin. This prevents stale-write clobbers when an OIDC
callback races a concurrent set_privileges, delete_user, rename_user,
set_admin, change_password, or TOTP mutation from another uvicorn
worker.
Also:
- Normalize setup() username (strip + lower) to match every other
user-creation path, preventing "Alice" vs "alice" lockout.
- Normalize in _create_user_locked() as defense-in-depth.
- Remove dead _setup_lock — its serialisation was subsumed by
_interprocess_auth_lock + _config_lock.
- Move pre-lock validation inside the critical section for
change_password, totp_generate_secret, and totp_confirm_enable.
- Re-read backup codes from disk inside the lock in totp_verify()
to prevent dual-consumption across workers.
Co-Authored-By: antigravity <agy@antigravity>
The reserved-username guard was only in create_user(), but setup()
now calls _create_user_locked() directly (to avoid nested flock
deadlock). Move the check into the shared helper so it applies
regardless of which path creates the user.
Fixes CI failure in test_setup_rejects_reserved_admin_username.
create_user() (password-user creation) now takes _interprocess_auth_lock
with reload-before-save, preventing a concurrent set_oidc_user_admin()
from losing a newly-created password user.
Introduces _create_user_locked() internal helper so setup() (which
already holds the inter-process lock) can create users without a
nested fcntl.flock deadlock.
Also changed _auth_intraprocess_lock to threading.RLock to support
safe nesting patterns across mutation methods.
Regression: test_create_user_survives_concurrent_set_oidc_user_admin
(109 total, 0 failures).
1. Serialize set_oidc_user_admin() across workers via
_interprocess_auth_lock() with reload-recheck-save, preventing
stale in-memory snapshots from overwriting users concurrently
created by another worker.
2. Accept single-element aud arrays without azp (OIDC Core §2 only
requires azp for multi-audience tokens). Normalize aud to a list
and only enforce azp when len > 1.
3. Add threading.Lock around _get_fernet() key creation to prevent
two threads in the same process from racing on the shared temp-file
path and destroying each other's work.
4. Guard import fcntl with try/except so AuthManager imports on native
Windows. _interprocess_auth_lock degrades to intra-process-only
when fcntl is unavailable.
Tests: +4 regression tests (108 total, 0 failures).
fcntl.flock serialises across processes (uvicorn workers) but does NOT
block threads within the same process — two threads calling
flock(LOCK_EX) on the same file both succeed immediately. This caused a
rare race in concurrent first-admin bootstrap where both
create_user_oidc() and setup() could enter the critical section
simultaneously, resulting in a lost write and zero admins.
Add a module-level threading.Lock acquired before the file lock so the
critical section is serialised across both threads and workers.
Fixes the flaky test:
TestInterprocessFirstAdminSerialisation::test_two_managers_single_first_admin
("Expected exactly 1 admin after concurrent bootstrap; found 0")
- Add fcntl.flock inter-process file lock shared by setup() and
create_user_oidc() so multi-worker first-admin bootstrap is
serialised across processes, not just threads within one worker.
Both methods reload auth.json inside the lock so the loser sees
the winner's write.
- _fetch_userinfo() now returns None (not {}) when discovery has
no userinfo_endpoint, and exchange_code() only sets
_userinfo_available=True when a live endpoint was reached.
Prevents the callback from treating 'no endpoint' as
authoritative group non-membership evidence.
- Rewrite _load_or_create_key() to write the Fernet key to a temp
file, fsync, then atomically os.link() into place. No reader
ever sees the final path before the complete key bytes are
available — a racing worker either sees no file or a complete
one, never an empty/partial file.
105 tests pass (97 existing + 8 new regressions covering the
three fixes).
Co-Authored-By: Kevin <holden093@users.noreply.github.com>
Three fixes from second review pass:
1. (security) Serialize the first-OIDC-user admin bootstrap inside
_config_lock. Previously the check and username
collision resolution happened outside the lock, so two concurrent
first-login callbacks could both observe an empty user map and
both persist as admin. Now idempotent lookup, bootstrap decision,
and collision resolution are all inside one critical section.
2. (auth) Make data/.app_key creation atomic via O_EXCL open so two
racing workers on a fresh deployment cannot generate different
keys. The loser reads the winner's key, guaranteeing every worker
shares the same Fernet key for OIDC state encryption.
3. (auth) Track whether UserInfo was successfully fetched
(_userinfo_available flag in claims). The callback now skips
admin group sync for existing users when UserInfo is unavailable
AND the id_token lacks a groups claim — a transient provider
failure no longer silently demotes existing OIDC admins. When
UserInfo succeeds or the id_token carries groups, admin status
syncs as before.
Regression tests: concurrent admin bootstrap, key-creation race,
UserInfo-unavailable preserves admin, UserInfo-available demotes,
id_token groups authoritative without UserInfo.
Two fixes from review feedback:
1. (security/availability) Record _last_jwks_refresh timestamp BEFORE
calling _refresh_jwks() so a failed JWKS fetch is also throttled
by the 60-second cooldown. Previously the timestamp was only set
after success, so an outage or key rotation would retry the IdP
on every login attempt.
2. (auth) Use the shared persistent app key from secret_storage
(_get_fernet) for OIDC state encryption instead of reading
data/.app_key directly and falling back to a per-process key.
This guarantees all uvicorn workers share the same Fernet key,
even on a fresh data directory — worker A's state is always
decryptable by worker B on the callback.
Add regression tests for both fixes.
1. JWKS fetch/parse errors now wrapped as OidcError instead of escaping
as raw exceptions. Transient network failures, HTTP errors, and bad
JSON responses from the JWKS endpoint all produce a controlled
OidcError, which the callback route already redirects to
/login?error=oidc_failed instead of a 500.
2. exchange_code() now extracts the stored redirect_uri from the
Fernet-encrypted state token and rejects any callback-derived
redirect_uri that differs. The token exchange POSTs the stored
value — the one the IdP saw in the authorization request — removing
the last callback dependence on request-derived URI behaviour.
Tests: 5 new (3 JWKS error wrapping + 2 redirect_uri binding)
Total: 130 passing (81 OIDC + 49 regression), 0 failures.
1. Discovery issuer mismatch now raises OidcError instead of logging
a warning (OIDC Discovery §1.1 requires mismatch abort).
2. Multi-audience ID tokens without azp are now rejected (OIDC Core
§2 requires azp when aud has multiple values).
3. /change-password, /2fa/setup, /2fa/confirm, and /2fa/disable now
reject OIDC users with a clear message. The frontend already hides
these cards, but the backend must also enforce the policy.
113 passing (76 OIDC + 37 regression), 0 failures.
Both vars were documented in .env.example and read by the application
code, but never forwarded to the container by any docker-compose file.
OIDC_REDIRECT_URI is essential for deployments behind a reverse proxy
to avoid deriving the wrong scheme from the inbound request.
1. Suppress first-user bootstrap admin when OIDC_ADMIN_GROUPS is
configured — a non-admin IdP user must not get admin just by being
the first to log in. Group membership is the only path when groups
are set.
2. Reject mismatched UserInfo sub: the UserInfo endpoint MUST NOT
overwrite the verified id_token subject. Also guard all verified
identity claims (sub, iss, aud, exp, iat, nonce, azp) from being
overwritten by UserInfo.
1. Fix bootstrap admin demotion: skip set_oidc_user_admin when
OIDC_ADMIN_GROUPS is unset so bootstrap/manual admin survives
subsequent logins.
2. Login CSRF: bind state to an HttpOnly cookie set at /login and
verified with constant-time compare at /callback.
3. JWKS cooldown: throttle refresh to once per 60s to prevent
attacker-triggered unbounded IdP fetches via random kid values.
4. Secure cookies: derive secure flag from request scheme when
SECURE_COOKIES is not explicitly set; OIDC session defaults
to secure on HTTPS connections.
5. Remove dead validation-shaped code: the jwt.decode block with
None key that swallowed all exceptions and discarded output.
1. aud validation: handle JSON array audiences per OIDC spec (check
membership, validate azp for multi-audience tokens).
2. JWKS caching: cache keys after first fetch; refresh only on unknown
kid — avoids live IdP round-trip on every login.
3. Algorithm pinning: restrict to RS256/ES256/etc from discovery doc,
never allow HS256 or 'none'.
4. OIDC_REDIRECT_URI: support a fixed redirect URI (proxy-safe,
doesn't trust Host header).
5. Stateless state: replace in-memory dict with Fernet-encrypted state
tokens — works across multiple uvicorn workers/processes.
6. OIDC_FIRST_USER_IS_ADMIN: bootstrap first OIDC user as admin when
no users exist and OIDC_ADMIN_GROUPS is unset — prevents zero-admin
lockout.
The base docker-compose.yml gained 6 OIDC_* env vars; the standalone
GPU files must mirror them or test_gpu_compose_standalone fails.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds OIDC authentication as an alternative to password login, enabling
sign-in via any standard provider (Authentik, Keycloak, Authelia, etc.).
New features:
- Generic OIDC provider support via .well-known discovery (authlib)
- Coexists with existing password auth — users choose at login
- Auto-creates local users on first OIDC login
- Admin group mapping: OIDC_ADMIN_GROUPS grants admin based on IdP groups
- Admin status syncs on every login (follows IdP membership)
- OIDC users cannot use password login or set up 2FA
- UI hides change-password and 2FA cards for OIDC users
New env vars:
- OIDC_ENABLED, OIDC_ISSUER, OIDC_CLIENT_ID, OIDC_CLIENT_SECRET
- OIDC_SCOPES, OIDC_ADMIN_GROUPS
New files:
- core/oidc.py — OidcManager (discovery, auth URL, code exchange,
id_token verification with JWT/JWKS)
- routes/oidc_routes.py — /api/auth/oidc/{login,callback,config}
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(rag): skip hidden and junk directories when indexing (#5559)
index_personal_documents walked the whole tree with no pruning, so
pointing RAG at a real-world folder silently swept in .obsidian/ plugin
JS, .git/ internals, node_modules/, and __pycache__/ — multiplying
indexing time and polluting retrieval with junk chunks.
Prune hidden directories and well-known junk directories from the walk,
and skip hidden files. The explicitly passed root is exempt, so a user
who deliberately indexes a hidden directory still gets its contents.
* fix(rag): prune hidden/junk dirs in the keyword index too, via a shared helper
The #5559 fix pruned only VectorRAG.index_personal_documents (the vector index).
The parallel keyword index built by PersonalDocsManager.refresh_index ->
load_personal_index walked the same tree unpruned, so .obsidian/, .git/,
node_modules/ etc. still swept into keyword retrieval and the file listing —
the 'end-to-end' guarantee was only half true.
Single-source the pruning policy in src/index_walk (prune_index_dirs +
is_indexable_file) and use it from both walkers so they cannot drift again.
The junk-dir match is now case-insensitive, so a Node_Modules on a
case-insensitive filesystem is pruned too.
Tests: keyword-path regressions covering hidden/junk dirs, hidden files, junk
at depth (not just top level), case-insensitive junk, and the explicit-hidden-
root exemption. The existing vector tests still pass against the shared helper.
* fix(email): use XOAUTH2 in test-connection for Google OAuth accounts
The test-connection endpoint was password-only and had no awareness of
OAuth accounts. For Google-connected accounts this caused two failures:
- IMAP: "Need IMAP host, username, and password" because imap_pass is
empty (no password is stored for OAuth accounts)
- SMTP: 535 BadCredentials because smtp.login() was called with an
empty password instead of an XOAUTH2 token
Fix: include oauth_provider and token fields in saved_body when hydrating
from the DB, then use conn.authenticate("XOAUTH2") / smtp.auth("XOAUTH2")
for Google accounts in both the IMAP and SMTP test paths, mirroring what
_send_smtp_message already does for real sends.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(email): add OAuth2 tests for test-connection endpoint
Covers the XOAUTH2 changes made to routes/email_routes.py:
- Google OAuth accounts must not be rejected with 'Need IMAP host,
username, and password' (no stored password for OAuth accounts)
- IMAP and SMTP test paths must use conn.authenticate('XOAUTH2')
for Google accounts
- Password accounts must still use conn.login() / smtp.login()
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(email): bind OAuth account tests to Google transport
---------
Co-authored-by: TNTBA <trynottobreakanything@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(config): forward Google OAuth env vars into Docker container and document setup
GOOGLE_OAUTH_CLIENT_ID, GOOGLE_OAUTH_CLIENT_SECRET, and GOOGLE_OAUTH_REDIRECT_URI
were read by the app but never forwarded through docker-compose.yml's explicit
environment allowlist, causing the "not set" error even when the vars existed in .env.
Also adds a documented section to .env.example with step-by-step GCP setup instructions
so users know where to get the credentials.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(config): cover OAuth in standalone compose files
* test(config): parse OAuth compose service env
* test(config): keep checkout skip wording neutral
---------
Co-authored-by: TNTBA <trynottobreakanything@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
