odysseus/specs/context-building.md

# Context Building

Last updated: dev@df2fad2 | 2026-07-12

## Scope

This spec covers model-context construction in:

- `src/chat_processor.py`;
- `src/chat_handler.py` and `src/youtube_handler.py`;
- `routes/chat_helpers.py` and context injection in `routes/chat_routes.py`;
- `src/agent_loop.py`;
- `src/tool_execution.py`;
- `src/attachment_refs.py` and uploaded-file manifest construction in
  `routes/chat_helpers.py`;
- `src/tool_policy.py`;
- `src/prompt_security.py`;
- URL fetchers in `src/search/content.py` and `services/search/content.py`;
- search orchestration in `services/search/core.py` and the compatibility wrapper in `src/search/core.py`;
- RAG and personal docs in `src/rag_singleton.py`, `src/rag_vector.py`, `src/rag_manager.py`, and `src/personal_docs.py`;
- research flows in `src/deep_research.py`, `src/research_handler.py`, and `services/research/research_handler.py`;
- memory and skills in `src/memory.py` and `services/memory/*`;
- related policy in `THREAT_MODEL.md`.

## Contract

Context-building tools gather evidence. They do not own user-intent routing.

Runtime rules:

- if external context is available, add it as compact untrusted source data;
- if an attempted source is unavailable and relevant, represent the unavailable state explicitly with source and reason when known;
- preserve the user's original message for the model;
- do not use regex preprocessing to force literal-vs-fetch intent;
- do not disable tools or force a reply style solely because preprocessing found a URL.

## Untrusted Data

`src.prompt_security` owns the untrusted wrapper:

- `UNTRUSTED_CONTEXT_POLICY` states global model policy;
- `untrusted_context_message(label, content)` wraps source content as user-role data with `metadata.trusted = False`.

Current untrusted context sources include:

- fetched URLs and web search results;
- webpage content passed into deep-research extraction;
- YouTube transcripts/comments;
- RAG/personal document chunks;
- memories and skills;
- notes and active editor documents;
- emails and attachments;
- tool output from external/user-controlled data.

Live multimodal provider blocks can contain data URLs, but persisted and
tool-facing context uses stable attachment references. Tool manifests carry an
`odysseus://attachment/<id>` URI and owner-checked read policy; local paths are
compatibility data added only after owner and root-confinement checks. Persisted
chat context keeps readable text/reference lines rather than reinserting raw
media bytes into later turns or search state.

## URL, Search, And Tool-Derived Context

Chat URL prefetch and agent `web_fetch` are different paths. Chat prefetch happens before the model call; `web_fetch` is a tool the model may choose later. Both should converge on the same intent: enrich context when content is available, represent unavailable content when it is not, and let the model interpret the user request.

Search results and fetched pages are evidence. `web_search` should not force a page fetch unless its explicit contract says it does. Failed fetches should not crash chat or silently imply content was read. Canonical search content fetchers can extract readable text from HTML, `text/*`, Markdown, `.txt`, `.json`, and `.jsonl` responses and should return shaped error results for HTTP status failures. URL fetches validate every redirect hop and pin the outbound connection to a public IP resolved during validation, so context-building callers do not need a second DNS-rebinding guard.

Current behavior is not yet unified:

- successful chat URL prefetch is wrapped as untrusted context, but failed chat URL prefetch can be dropped;
- agent `web_fetch` returns explicit URL-specific tool errors for timeout, unsupported scheme, fetch failure, or no readable text;
- comprehensive search reports provider-chain failures, but individual page-fetch failures can be logged and omitted;
- YouTube fetching is owned by `ChatHandler`/`youtube_handler`, while `routes.chat_helpers` only wraps the resulting transcript/comment strings.

`services/search/core.py` owns `comprehensive_web_search()` orchestration. `src/search/core.py` is a compatibility wrapper. `src/search/content.py` now aliases the canonical `services.search.content` module so old imports do not create a second fetch/extract implementation.

## Tool Result Envelope

`src.tool_execution` executes and formats tools. Tool output caps live in `src.constants` and are re-exported through older facades; shared native-tool truncation lives in `src.tool_utils`. `src.agent_loop._append_tool_results()` owns model re-entry: native tool calls return as provider-style `role: "tool"` messages, while fenced-tool results can become a bracketed user message. These results are untrusted, but they do not all currently use `untrusted_context_message()` or `metadata.trusted = False`.

Context budgeting uses known model context windows when available. `src.context_budget` treats the default 6000-token value as an automatic sentinel, scales to a capped fraction of known context length for non-explicit budgets, and leaves unknown windows on conservative defaults.

Side-effect enforcement lives outside context building. Chat route disabled-tool policy, `src.tool_security`, `src.tool_execution`, and `do_app_api()` block unsafe tool execution; prompt wording alone is not the authority.

Guide-only/no-tools policy can suppress context acquisition before the model call. `src.tool_policy` feeds chat route preprocessing and agent-loop assembly so tool-backed search/research/memory/RAG/skills/local-context paths are skipped when the latest user turn explicitly forbids tools.

## Degraded And Optional Dependencies

- ChromaDB, HTTP embeddings, and FastEmbed are installed/expected in normal setups but must degrade cleanly when a service, package, or embedding backend is unavailable.
- `src.rag_singleton.get_rag_manager()` owns RAG startup retry throttling; `src.rag_vector.VectorRAG` is the live owner-filtered path; `src.rag_manager.RAGManager` is compatibility/backward-compat behavior.
- Memory-vector and tool-index retrieval can fall back to keyword/text behavior when vector stores or embeddings fail.
- Docker compose and native installs use different Chroma host defaults; model endpoint loopback rewriting is owned by model/runtime specs.

## Current Call Sites Include

- `ChatProcessor.build_context_preface()` for memory, RAG, web search, URL content, and skills index;
- `ChatHandler.preprocess_message()` and the canonical `services.youtube.youtube_handler` import path for YouTube fetch/format, then `routes/chat_helpers.py` for wrapping prefetched search/Youtube context;
- `routes/chat_routes.py` research context injection;
- `src.agent_loop` for active editor document, skill context, and tool-result reinsertion;
- uploaded-file manifest/reference context for agent tools and later chat turns;
- `src.tool_execution` for `web_search`, `web_fetch`, file, shell, MCP, and other tool outputs;
- `src.deep_research` and research handlers for search/fetch/extract flows used by research jobs, with fetched webpage text wrapped before extraction and analyzed URLs tracked separately from source snippets.

## Current Gaps

- URL/search context result shape is not unified across chat prefetch, agent tools, and research.
- Some failed fetch states are still easier for code to drop than to represent explicitly.
- Tool/context wording is spread across schema, prompt, and retrieval surfaces.
- Agent tool-result reinjection lacks a unified untrusted wrapper/metadata envelope across native, fenced, MCP, and app API outputs.
- Source-specific wrapping and unavailable-state behavior need focused tests for chat URL prefetch, literal URL intent, search context, deep-research extraction, RAG/memory/skills, YouTube, and tool results.
- Compare pre-search context is computed but may not be submitted through the current compare stream form.