erabo/odysseus
1
0
Fork 0
mirror of https://github.com/pewdiepie-archdaemon/odysseus.git synced 2026-10-31 02:08:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
odysseus/src/constants.py
Léo 2a540f2acc fix(runtime): enforce workspace confinement in one place 
"Is this path inside that root" is asked in twenty places in this tree and
answered twenty times by a locally written realpath/commonpath pair. Nine test
files exist because nine call sites each needed their own proof. Each one is
defensible alone; together they are the defect, because the boundary has no
single definition and a site that gets a detail wrong is wrong by itself.

src/path_confinement.py is that definition, and it settles the details the
copies disagreed on. Both sides get canonicalized: comparing a realpath-ed
candidate against a root that was only abspath-ed is the macOS /tmp ->
/private/tmp mismatch that has already produced a false failure here, and
canonicalizing one side is worse than canonicalizing neither. commonpath rather
than startswith, because /a/bc begins with /a/b and is not inside it. A relative
candidate joins the root rather than os.getcwd(), which is whatever directory
the server happens to be running in. NUL and newline are refused with a reason
instead of caught by a bare `except Exception` and reported as an ordinary
escape. Eighteen call sites go through it now. It deliberately does not decide
whether a path is sensitive -- that deny list answers "allowed" rather than
"inside", and it stays with src/tool_execution, which owns it. The one
commonpath left in the tree, in src/workspace_paths.py, stays: that function
translates a host path into a container path, so canonicalizing either side
would change the relative path it computes and break the mapping. It is not a
confinement check.

Two of those sites were weaker than the rest and are fixed rather than moved.
The email attachment check used abspath, which folds `..` but does not resolve
symlinks, so a symlink written into the extraction directory passed it and was
then read through. The skill-reference guard compared a realpath-ed target
against a raw dirname, so on a host where the skills tree is reached through a
symlink the two sides never matched and the guard could not fire.

The execution boundary had two separate holes.

The workspace namespace bound /home and /mnt read-write. On the one platform
where that namespace engages at all, a command inside it reaches outside the
workspace and writes to the user's home directory -- measured by running this
argv on a Linux host with working bubblewrap, not inferred from the source.
Binding the user's whole home directory into a workspace-confinement namespace
gives back most of what the namespace was for. Both are read-only now. The
workspace is also bound writable at its real host path, not only at /workspace:
BashTool's own /tmp redirect rewrites `/tmp/` to `<agent_cwd()>/.tmp/` before
the namespace is built, so the command bwrap receives already names the real
path, and those writes previously landed only because the workspace happened to
sit under the writable /home.

`namespaced or _replace_workspace_alias(...)` chose between a mount namespace
and a regex with nothing in the result saying which one ran. The fallback
rewrites the literal token /workspace in the command string, so a command that
never mentions /workspace is untouched by it and runs on the host unrestricted
-- which is every agent shell command on macOS. Both tools now ask
containment.probe() instead of each deciding for itself, and every bash and
python result carries a containment block naming the mechanism and stating
whether the filesystem dimension actually held. Under enforcing mode the
command is not run and the result says so.

That block reports the filesystem dimension only, and says so in a
reported_dimensions field. The probe knows this host could also give a process
group and a real wall clock, but these two tools still assemble their own
create_subprocess_* call and pass neither, so listing those dimensions would be
exactly the false claim src/containment.py calls worse than an honest absence.

probe() is new on src/containment.py: the same mechanism table and the same
arithmetic as acquire(), stopping before the side effects. acquire() is the
wrong shape for a decision -- it writes a durable grant record, and a record
whose pid is never filled in and whose release() never runs is an entry a
restart reaper keeps finding.

CONTAINMENT_MODE stays report_only. Flipping it refuses every agent shell
command on macOS and on any Linux host without bubblewrap, which is a product
decision rather than a code one.

Smaller things in the same area: the /tmp redirect's makedirs was unguarded, so
a read-only workspace turned a command that merely mentioned `/tmp/` into an
OSError traceback instead of a tool error; it degrades now. WORKSPACE_MOUNT
moved to src/constants.py so the namespace and the path resolvers read one
definition of the contract rather than two. The ".tmp" dirname got a constant,
since it appeared in both tool paths.

One generated artifact moved with it: website/configuration-reference.md pins
the source line where each ODYSSEUS_* variable is read, and three of those
shifted. Regenerated with scripts/generate_env_reference.py; the diff is line
numbers only.

Three existing tests changed. test_workspace_artifact_tool_floor asserted that
an unsafe interpreter prefix produces no `--ro-bind <prefix> <prefix>`, which
now fires on /home because /home is legitimately a read-only base mount.
Asserting the absence of a literal flag string cannot distinguish "the prefix
was rejected" from "the argv mounted that root itself", so it compares the argv
against the no-prefix baseline instead: an unsafe prefix must add nothing.

The Windows bash test asserted dict equality on the
whole result, which makes adding a field to every bash result impossible without
touching a test about tmux; it asserts the shape now. The personal-dir symlink
test grepped the resolver's source for the literal "os.path.realpath", which is
gone because the resolution moved into the shared boundary -- it keeps the
negative assertion that the closure must not grow its own abspath check again,
and the behavioural half now runs against the boundary, where it covers every
call site instead of one closure.

Not verified: the bubblewrap argv is asserted, not executed. There is no bwrap
on macOS, and in Docker it needs --privileged to work at all -- default and
seccomp=unconfined both fail with "Creating new namespace failed", and
--cap-add=SYS_ADMIN fails at pivot_root. The Python tool's
needs_virtual_namespace gate means ordinary Python code gets no namespace even
on a Linux host that could provide one; that is reported now but deliberately
not changed, because it alters the Linux Python path on every call and cannot be
checked from here.
2026-10-01 19:45:59 +02:00

193 lines
8.6 KiB
Python
Raw Blame History

# src/constants.py
"""Application-wide constants and configuration values."""
import os
import subprocess
from src.runtime_paths import get_app_root, get_default_data_dir
APP_VERSION = "1.0.3"
BUILTIN_SKILLS_DIR = os.path.join(get_app_root(), "resources", "skills")
# Identifies the private maintainer-preview build without changing the public
# application semver used by release and readiness checks. Keep the API/UI
# value tied to HARNESS_VERSION so a version bump cannot leave the running
# service claiming an older harness build.
def _load_build_version() -> str:
override = os.getenv("ODYSSEUS_BUILD_VERSION", "").strip()
if override:
return override
try:
with open(os.path.join(get_app_root(), "HARNESS_VERSION"), encoding="utf-8") as fh:
value = fh.read().strip()
if value:
return value
except OSError:
pass
return "unknown"
APP_BUILD_VERSION = _load_build_version()
def _load_source_commit() -> str:
"""Identify the source tree loaded by this process for runtime provenance."""
override = os.getenv("ODYSSEUS_SOURCE_COMMIT", "").strip()
if override:
return override
try:
result = subprocess.run(
["git", "rev-parse", "HEAD"],
cwd=get_app_root(),
check=False,
capture_output=True,
text=True,
timeout=2,
)
except (OSError, subprocess.SubprocessError):
return "unknown"
commit = result.stdout.strip()
return commit if result.returncode == 0 and commit else "unknown"
APP_SOURCE_COMMIT = _load_source_commit()
# Base paths
BASE_DIR = os.path.join(get_app_root(), "")
STATIC_DIR = os.path.join(BASE_DIR, "static")
DATA_DIR = os.getenv("ODYSSEUS_DATA_DIR", get_default_data_dir())
# Data file paths
# Single source of truth: every persisted file/dir lives under DATA_DIR, which
# is the ONLY place ODYSSEUS_DATA_DIR is read. Import these constants instead of
# re-deriving paths from __file__ or a relative "data" literal.
SESSIONS_FILE = os.path.join(DATA_DIR, "sessions.json")
MEMORY_FILE = os.path.join(DATA_DIR, "memory.json")
PERSONAL_DIR = os.path.join(DATA_DIR, "personal_docs")
RUNBOOK_DIR = os.path.join(PERSONAL_DIR, "runbook")
UPLOAD_DIR = os.path.join(DATA_DIR, "uploads")
FEATURES_FILE = os.path.join(DATA_DIR, "features.json")
SETTINGS_FILE = os.path.join(DATA_DIR, "settings.json")
AUTH_FILE = os.path.join(DATA_DIR, "auth.json")
USER_PREFS_FILE = os.path.join(DATA_DIR, "user_prefs.json")
PRESETS_FILE = os.path.join(DATA_DIR, "presets.json")
INTEGRATIONS_FILE = os.path.join(DATA_DIR, "integrations.json")
CONTACTS_FILE = os.path.join(DATA_DIR, "contacts.json")
APP_KEY_FILE = os.path.join(DATA_DIR, ".app_key")
EMBEDDING_ENDPOINT_FILE = os.path.join(DATA_DIR, "embedding_endpoint.json")
COOKBOOK_STATE_FILE = os.path.join(DATA_DIR, "cookbook_state.json")
BG_JOBS_FILE = os.path.join(DATA_DIR, "bg_jobs.json")
CONTAINMENT_STATE_FILE = os.path.join(DATA_DIR, "containment_grants.json")
VAULT_FILE = os.path.join(DATA_DIR, "vault.json")
TIDY_CALENDAR_STATE_FILE = os.path.join(DATA_DIR, "tidy_calendar_state.json")
SKILLS_FILE = os.path.join(DATA_DIR, "skills.json")
APP_DB = os.path.join(DATA_DIR, "app.db")
SCHEDULED_EMAILS_DB = os.path.join(DATA_DIR, "scheduled_emails.db")
EMAIL_CACHE_DB = os.path.join(DATA_DIR, "email_cache.db")
# Data subdirectories
PERSONAL_UPLOADS_DIR = os.path.join(DATA_DIR, "personal_uploads")
EMOJI_CACHE_DIR = os.path.join(DATA_DIR, "emoji_cache")
RAG_DIR = os.path.join(DATA_DIR, "rag")
CHROMA_DIR = os.path.join(DATA_DIR, "chroma")
BG_JOBS_DIR = os.path.join(DATA_DIR, "bg_jobs")
DEEP_RESEARCH_DIR = os.path.join(DATA_DIR, "deep_research")
MCP_OAUTH_DIR = os.path.join(DATA_DIR, "mcp_oauth")
GENERATED_IMAGES_DIR = os.path.join(DATA_DIR, "generated_images")
TTS_CACHE_DIR = os.path.join(DATA_DIR, "tts_cache")
EMAIL_URGENCY_CACHE_DIR = os.path.join(DATA_DIR, "email_urgency_cache")
SKILLS_DIR = os.path.join(DATA_DIR, "skills")
GALLERY_DIR = os.path.join(DATA_DIR, "gallery")
GALLERY_UPLOADS_DIR = os.path.join(DATA_DIR, "gallery_uploads")
MEMORY_VECTORS_DIR = os.path.join(DATA_DIR, "memory_vectors")
# Paths with an intentional dedicated env override, defaulting under DATA_DIR.
MAIL_ATTACHMENTS_DIR = os.getenv("ODYSSEUS_MAIL_ATTACHMENTS_DIR", os.path.join(DATA_DIR, "mail-attachments"))
# `or` (not os.getenv's default arg) so a PRESENT-but-EMPTY value falls back to
# the default. docker-compose.yml injects `FASTEMBED_CACHE_PATH=${FASTEMBED_CACHE_PATH:-}`,
# which sets the var to "" when the host hasn't defined it. os.getenv(name, default)
# only returns the default when the var is ABSENT, so the empty string would win →
# os.makedirs("") raises [Errno 2] No such file or directory: '' → FastEmbed fails to
# init and all vector features (RAG, semantic memory, tool index) silently degrade.
FASTEMBED_CACHE_DIR = os.getenv("FASTEMBED_CACHE_PATH") or os.path.join(DATA_DIR, "fastembed_cache")
# Agent tool output limits (single source of truth — imported by tool_execution.py,
# tool_implementations.py, agent_tools.py, and any other module that needs them)
MAX_OUTPUT_CHARS = 10_000 # cap for bash/python/web_search/web_fetch output
MAX_READ_CHARS = 20_000 # cap for read_file / document preview
MAX_DIFF_LINES = 400 # cap for edit_file unified-diff display
# web_fetch response-size policy (#3812). MAX_OUTPUT_CHARS above only trims
# what the agent SEES; these caps bound what the server downloads, parses,
# and writes to the content cache. The soft cap is the default download
# budget; the agent can raise it per call (full/max_bytes) but never past
# the hard cap, so a model can't decide to pull a multi-GB file.
WEB_FETCH_SOFT_MAX_BYTES = 2_000_000 # default download budget (2 MB)
WEB_FETCH_HARD_MAX_BYTES = 20_000_000 # absolute ceiling, even with override (20 MB)
# API Configuration
MAX_CONTEXT_MESSAGES = 90
REQUEST_TIMEOUT = 20
OPENAI_COMPAT_PATH = "/v1/chat/completions"
# Outbound UA for web_fetch / web_search scraping; common desktop UA so pages serve normal HTML.
WEB_FETCH_USER_AGENT = os.environ.get(
"WEB_FETCH_USER_AGENT",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36",
)
# Environment variables with defaults
DEFAULT_HOST = os.getenv("LLM_HOST", "localhost")
LLM_HOSTS = [h.strip() for h in os.getenv("LLM_HOSTS", "").split(",") if h.strip()]
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
SEARXNG_INSTANCE = os.getenv("SEARXNG_INSTANCE", "http://localhost:8080")
# Scholarly title resolution. These are the only third-party metadata APIs the
# search path calls directly, so they get named constants rather than literals
# repeated at each call site. The budget bounds the whole SearXNG -> OpenAlex ->
# arXiv chain: each hop used to get its own full timeout, so one scholarly query
# could stall a user-facing search for the sum of all three.
ARXIV_API_URL = "https://export.arxiv.org/api/query"
OPENALEX_API_URL = "https://api.openalex.org/works"
SCHOLARLY_LOOKUP_TIMEOUT = 12.0
SCHOLARLY_LOOKUP_TOTAL_BUDGET = 20.0
# Cleanup configuration
CLEANUP_ENABLED = os.getenv("CLEANUP_ENABLED", "True").lower() == "true"
CLEANUP_INTERVAL_HOURS = int(os.getenv("CLEANUP_INTERVAL_HOURS", "24"))
# Agent workspace
# The stable virtual root the tool contract promises an agent, independent of
# where the workspace physically lives. Both the mount namespace and the
# path resolvers map it to the active workspace, so it is the one absolute path
# a contained command may assume.
WORKSPACE_MOUNT = "/workspace"
# Scratch directory inside the workspace that agent shell commands get in place
# of the host /tmp. A dirname rather than a path: the workspace is dynamic, so
# the full path is only knowable per turn.
AGENT_ISOLATED_TMP_DIRNAME = ".tmp"
# Auth policy
PASSWORD_MIN_LENGTH = 8
# Default parameters
DEFAULT_TEMPERATURE = 1.0
DEFAULT_MAX_TOKENS = 32768
def internal_api_base() -> str:
"""Base URL for in-process loopback calls to Odysseus's own API.
Agent tools and background jobs reach admin-gated routes by calling the
running server over HTTP. Resolution order:
1. ODYSSEUS_INTERNAL_BASE - explicit override (e.g. behind a TLS proxy).
2. APP_PORT - http://127.0.0.1:$APP_PORT (docker-compose).
3. Fallback http://127.0.0.1:7011 - matches app.py's bind default.
127.0.0.1 (not "localhost") avoids IPv6/DNS ambiguity for a strictly-local
call. Without this, loopback tools fail with "All connection attempts
failed" whenever the server is not on port 7000.
"""
override = os.environ.get("ODYSSEUS_INTERNAL_BASE")
if override:
return override.rstrip("/")
return f"http://127.0.0.1:{os.environ.get('APP_PORT', '7011')}"
Reference in a new issue View git blame Copy permalink