odysseus/src
Kenny Van de Maele ec04768ed0 fix(llm): stop sending llama.cpp slot-affinity fields to cloud providers (#3945)
* fix(llm): stop sending llama.cpp slot-affinity fields to cloud providers

_apply_local_cache_affinity adds session_id + cache_prompt for llama.cpp
KV-cache slot affinity (#2927), gated on _is_self_hosted_openai_compatible,
which treated any unknown OpenAI-compatible host as self-hosted. Strict
cloud providers added as custom endpoints (Mistral at api.mistral.ai)
reject unknown body fields, so every request failed with 422
extra_forbidden. Self-hosted now also requires the endpoint to resolve as
local via model_context.is_local_endpoint: loopback/private/tailscale
host, or endpoint kind explicitly configured as "local" (the escape hatch
for tunneled self-hosted servers). is_local_endpoint is promoted to a
public name since llm_core now shares it.

Fixes #3793

* test(llm): sweep cloud OpenAI-compatible hosts in affinity gating

Parametrized cases adapted from #3839 (credit: Shabablinchikow): deepseek,
x.ai, together, fireworks, and the Gemini OpenAI-compat endpoint must all
stay free of the llama.cpp extras, not just the Mistral host from #3793.

* fix(llm): narrow the Tailscale range to 100.64.0.0/10 in is_local_endpoint

Review finding on #3945: _PRIVATE_PREFIXES carried a bare "100." prefix,
treating all of 100.0.0.0/8 as local while Tailscale only uses the CGNAT
block 100.64.0.0/10. Public 100.x hosts (e.g. AWS ranges outside the
block) were classified local and still received the llama.cpp extras
this PR exists to keep away from strict providers. Match the narrowed
classification routes/model_routes.py already uses, with boundary tests
just below, inside, and just above the range.
2026-06-11 17:51:03 +02:00
..
agent_tools refactor(tools): extract document tools to handle registry (#3666) 2026-06-10 10:41:52 +02:00
search refactor(search): make src analytics a service shim (#2264) 2026-06-04 18:57:24 +02:00
action_intents.py fix(calendar): route read requests to agent (#2452) 2026-06-05 09:24:04 +01:00
agent_loop.py fix(chat): stabilize system prompt, sequence memory extraction, and send stable session id to preserve KV cache (#3360) 2026-06-09 22:46:54 +01:00
agent_runs.py Handle incomplete detached agent streams 2026-06-01 16:54:11 +09:00
ai_interaction.py fix: session context drifting — messages leaking between chats (#135) (#267) 2026-06-09 14:12:52 +01:00
api_key_manager.py Stop API key save() from writing other providers' keys as plaintext (#1944) 2026-06-04 04:47:13 +01:00
app_helpers.py fix: inside_base_dir raises TypeError on a non-string path instead of failing closed (#1619) 2026-06-03 09:00:04 +09:00
app_initializer.py feat(memory): add provider interface (#72) 2026-06-04 16:26:11 +01:00
assistant_log.py Odysseus v1.0 2026-05-31 23:58:26 +09:00
auth_helpers.py fix(auth): gate api tokens from user routes (#2992) 2026-06-07 12:55:01 +02:00
bg_jobs.py refactor(constants): single source of truth for data dir (#3368) 2026-06-08 09:58:52 +02:00
bg_monitor.py Ignore non-string background stream deltas (#1549) 2026-06-03 14:11:45 +09:00
builtin_actions.py fix(email): scope learned sender signatures by owner (#3724) 2026-06-11 13:26:59 +02:00
builtin_mcp.py fix: fall back for npx cache subprocess check (#3560) 2026-06-09 20:41:23 +01:00
caldav_sync.py fix(caldav): skip the prune when any object fails to parse (#3454) 2026-06-08 18:59:14 +02:00
caldav_writeback.py fix(caldav): disable redirects on the sync/write-back DAVClient (SSRF) (#2663) 2026-06-07 05:05:24 +01:00
chat_handler.py Scope vision model resolution by owner (#3009) 2026-06-07 12:39:02 +02:00
chat_helpers.py fix(chat): keep balanced trailing ')' when extracting URLs (#3406) 2026-06-08 21:33:29 +02:00
chat_processor.py fix(chat): stabilize system prompt, sequence memory extraction, and send stable session id to preserve KV cache (#3360) 2026-06-09 22:46:54 +01:00
chatgpt_subscription.py Fix model endpoint route test regressions 2026-06-09 10:16:38 +09:00
chroma_client.py fix: ChromaDB unreachable blocks app startup for 30-60s (#326) (#476) 2026-06-01 22:22:41 +09:00
cleanup_service.py Replace cleanup service datetime.utcnow calls (#1494) 2026-06-03 14:14:27 +09:00
config.py refactor(constants): single source of truth for data dir (#3368) 2026-06-08 09:58:52 +02:00
constants.py refactor(constants): single source of truth for data dir (#3368) 2026-06-08 09:58:52 +02:00
context_budget.py feat: adapt agent_input_token_budget to the model context window (#1170) (#1230) 2026-06-03 00:13:53 +09:00
context_compactor.py fix: session context drifting — messages leaking between chats (#135) (#267) 2026-06-09 14:12:52 +01:00
cookbook_serve_lifecycle.py refactor(constants): single source of truth for data dir (#3368) 2026-06-08 09:58:52 +02:00
copilot.py feat(provider): add GitHub Copilot provider with device-flow auth (#1480) 2026-06-04 21:13:14 +02:00
database.py Odysseus v1.0 2026-05-31 23:58:26 +09:00
deep_research.py fix(research): track analyzed URLs separately (#3125) 2026-06-10 12:08:22 +01:00
document_actions.py fix: document tidy crashes on a duplicate with NULL timestamps (#1772) 2026-06-03 13:23:01 +09:00
document_processor.py fix(documents): restore PDF library metadata and preview (#2483) 2026-06-07 23:23:27 +02:00
email_thread_parser.py Ignore non-string email thread bodies (#1654) 2026-06-03 14:06:31 +09:00
embedding_lanes.py fix(embeddings): survive numpy embeddings when restoring a reset lane (#3410) 2026-06-09 10:40:17 +02:00
embeddings.py refactor(constants): single source of truth for data dir (#3368) 2026-06-08 09:58:52 +02:00
endpoint_resolver.py fix(models): query v1 models for llama-server endpoints (#3380) 2026-06-09 01:09:02 +02:00
event_bus.py refactor(constants): single source of truth for data dir (#3368) 2026-06-08 09:58:52 +02:00
exceptions.py Odysseus v1.0 2026-05-31 23:58:26 +09:00
generated_images.py refactor(constants): single source of truth for data dir (#3368) 2026-06-08 09:58:52 +02:00
goal_based_extractor.py fix(deep-research): wrap fetched webpage content in untrusted-context sandbox 2026-06-06 03:37:10 -06:00
integrations.py fix(integrations): truncate api_call JSON lists with sentinel instead of mid-string cut (#3540) 2026-06-09 22:34:08 +01:00
llm_core.py fix(llm): stop sending llama.cpp slot-affinity fields to cloud providers (#3945) 2026-06-11 17:51:03 +02:00
markitdown_runtime.py fix: is_markitdown_format crashes on a non-string path (#1618) 2026-06-03 09:00:10 +09:00
mcp_manager.py feat: Add plan mode to the chat agent (#638) 2026-06-05 16:32:25 +02:00
mcp_oauth.py feat(mcp): add Streamable HTTP transport with OAuth 2.0 (#1033) 2026-06-05 02:40:52 +02:00
memory.py refactor(memory): canonicalize memory imports (#50) 2026-06-04 05:31:15 +01:00
memory_provider.py feat(memory): add provider interface (#72) 2026-06-04 16:26:11 +01:00
memory_vector.py fix: split Chroma embedding lanes (#3046) 2026-06-06 03:17:19 -06:00
model_context.py fix(llm): stop sending llama.cpp slot-affinity fields to cloud providers (#3945) 2026-06-11 17:51:03 +02:00
model_discovery.py fix(startup): ping real endpoints in warmup/keepalive (#3641) 2026-06-10 19:21:45 +02:00
pdf_form_doc.py refactor(tools): extract document tools to handle registry (#3666) 2026-06-10 10:41:52 +02:00
pdf_forms.py Odysseus v1.0 2026-05-31 23:58:26 +09:00
pdf_runtime.py Show a clear message when PyMuPDF is missing 2026-06-01 18:27:17 +09:00
personal_docs.py Skip malformed personal keyword index rows 2026-06-03 13:42:05 +09:00
preset_manager.py fix(presets): persist presets atomically to avoid corruption on crash (#2169) 2026-06-08 19:16:37 +02:00
prompt_security.py fix(security): harden untrusted_context_message against delimiter spoofing (#3086) 2026-06-07 22:15:50 +01:00
rag_manager.py refactor(constants): single source of truth for data dir (#3368) 2026-06-08 09:58:52 +02:00
rag_singleton.py refactor(constants): single source of truth for data dir (#3368) 2026-06-08 09:58:52 +02:00
rag_vector.py refactor(constants): single source of truth for data dir (#3368) 2026-06-08 09:58:52 +02:00
rate_limiter.py Odysseus v1.0 2026-05-31 23:58:26 +09:00
readiness.py feat: add /api/ready readiness probe (DB, data dir, local-first) (#1200) 2026-06-02 23:33:22 +09:00
request_models.py Odysseus v1.0 2026-05-31 23:58:26 +09:00
research_handler.py fix(research): migrate active task owners on rename (#3618) 2026-06-11 01:17:02 +02:00
research_utils.py Treat non-string research summaries as low quality 2026-06-03 13:42:24 +09:00
secret_storage.py refactor(constants): single source of truth for data dir (#3368) 2026-06-08 09:58:52 +02:00
service_health.py feat(diagnostics): add consolidated service health endpoint for degraded-state reporting (#964) 2026-06-09 16:00:24 +01:00
session_actions.py fix(sessions): keep fresh chats during auto tidy (#1871) 2026-06-09 01:06:20 +01:00
session_search.py fix(search): batch FTS hit lookups into one query (N+1) (#3909) 2026-06-11 16:31:54 +02:00
settings.py feat(reminders): add generic webhook as a fourth reminder channel (#2952) 2026-06-05 22:47:57 +02:00
settings_scrub.py fix(settings): scrub camelCase secret keys (#3707) 2026-06-11 12:53:33 +02:00
task_endpoint.py Harden session endpoint owner scope (#1308) 2026-06-03 02:40:22 +09:00
task_scheduler.py fix: session context drifting — messages leaking between chats (#135) (#267) 2026-06-09 14:12:52 +01:00
teacher_escalation.py fix(presets): scope expand-prompt model resolution to owner (#3477) 2026-06-08 21:12:02 +02:00
text_helpers.py fix: normalize Gemma 4 thought-channel output (#2224) 2026-06-04 19:26:58 +02:00
tls_overrides.py Support extra CA bundle for private-CA LLM providers (#769) 2026-06-04 13:18:50 +01:00
tool_execution.py refactor(tools): extract document tools to handle registry (#3666) 2026-06-10 10:41:52 +02:00
tool_implementations.py fix: handle batch events format in manage_calendar tool (#3503) 2026-06-10 19:13:08 +02:00
tool_index.py Fix remaining CI regressions 2026-06-09 10:21:56 +09:00
tool_parsing.py fix(agent): stop treating illustrative Markdown fences as tool calls for native function-calling models (#3356) 2026-06-08 22:25:28 +02:00
tool_policy.py fix(agent): enforce guide-only tool policy (#3088) 2026-06-06 18:48:24 -06:00
tool_schemas.py Prepare tested main sync cleanup 2026-06-09 09:34:42 +09:00
tool_security.py fix(security): don't grant tool access in the pre-setup window (#3506) 2026-06-10 14:37:26 +02:00
tool_utils.py refactor(tools): consolidate duplicated _truncate and get_mcp_manager into src/tool_utils (#3478) 2026-06-09 01:05:30 +02:00
topic_analyzer.py Topics: hydrate session history before analysis 2026-06-02 20:44:27 +09:00
upload_handler.py fix(uploads): migrate upload ownership on rename (#3617) 2026-06-11 16:01:04 +02:00
upload_limits.py refactor(uploads): centralize upload byte-limits in upload_limits.py (#3364) (#3518) 2026-06-09 01:24:30 +02:00
url_safety.py fix: check_outbound_url crashes on a truthy non-string URL (#1623) 2026-06-03 08:59:49 +09:00
url_security.py Reapply "Merge branch 'main' of github.com:pewdiepie-archdaemon/odysseus" 2026-06-03 22:47:00 +09:00
user_time.py fix(chat): stabilize system prompt, sequence memory extraction, and send stable session id to preserve KV cache (#3360) 2026-06-09 22:46:54 +01:00
visual_report.py security: sanitize rendered research-report HTML (#364) 2026-06-04 13:42:49 +01:00
webhook_manager.py fix(webhooks): keep references to in-flight delivery tasks (#3859) 2026-06-11 15:53:52 +02:00
youtube_handler.py fix: youtube transcript formatter crashes on a non-dict segment (#1745) 2026-06-03 13:29:08 +09:00