mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-08-05 02:45:28 +00:00
fix(embeddings): make the HTTP lane's read timeout configurable
The 10s read budget holds for a warm endpoint — embedding a short string does return in well under a second, as the comment says. But the FIRST call after a cold start also pays for loading the embedding model into memory, and on a modest or busy machine that is comfortably past 10s. The failure is silent, which is the part worth fixing. The timeout fires, the lane produces no vectors, its collection stays empty, and retrieval degrades to whatever other lane happens to be populated. Nothing marks the endpoint as the cause, so the symptom shows up much later as poor retrieval quality rather than as an embedding error — in our case French queries ranking badly because only the English-only fallback lane had any content. Adds EMBEDDING_TIMEOUT alongside the existing EMBEDDING_BATCH_SIZE and EMBEDDING_MAX_CHARS, floored at 1s. The default is unchanged at 10s, so nothing moves for existing installs; a slow first load can now be given room without patching. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
d8a2059df8
commit
b414aa27c7
1 changed files with 9 additions and 1 deletions
|
|
@ -54,7 +54,15 @@ class EmbeddingClient:
|
||||||
# running on :11434) fast-fails to the local FastEmbed fallback instead
|
# running on :11434) fast-fails to the local FastEmbed fallback instead
|
||||||
# of stalling startup ~30s per probe. Read stays generous for a real
|
# of stalling startup ~30s per probe. Read stays generous for a real
|
||||||
# endpoint (embedding a short string returns in well under a second).
|
# endpoint (embedding a short string returns in well under a second).
|
||||||
self._client = httpx.Client(timeout=httpx.Timeout(connect=3.0, read=10.0, write=5.0, pool=3.0))
|
# The read budget holds for a warm endpoint, but the FIRST call after a
|
||||||
|
# cold start also pays for loading the embedding model into memory — on
|
||||||
|
# a modest or busy machine that is well past 10s. The timeout then fires
|
||||||
|
# silently: the lane produces no vectors, its collection stays empty,
|
||||||
|
# and retrieval quietly degrades to whatever other lane is present, with
|
||||||
|
# nothing marking the endpoint as the cause. Overridable, default
|
||||||
|
# unchanged.
|
||||||
|
_read_timeout = max(1.0, float(os.getenv("EMBEDDING_TIMEOUT", "10")))
|
||||||
|
self._client = httpx.Client(timeout=httpx.Timeout(connect=3.0, read=_read_timeout, write=5.0, pool=3.0))
|
||||||
self._batch_size = max(1, int(os.getenv("EMBEDDING_BATCH_SIZE", "8")))
|
self._batch_size = max(1, int(os.getenv("EMBEDDING_BATCH_SIZE", "8")))
|
||||||
self._max_chars = max(200, int(os.getenv("EMBEDDING_MAX_CHARS", "900")))
|
self._max_chars = max(200, int(os.getenv("EMBEDDING_MAX_CHARS", "900")))
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue