fix(embeddings): make the HTTP lane's read timeout configurable

The 10s read budget holds for a warm endpoint — embedding a short string
does return in well under a second, as the comment says. But the FIRST
call after a cold start also pays for loading the embedding model into
memory, and on a modest or busy machine that is comfortably past 10s.

The failure is silent, which is the part worth fixing. The timeout fires,
the lane produces no vectors, its collection stays empty, and retrieval
degrades to whatever other lane happens to be populated. Nothing marks
the endpoint as the cause, so the symptom shows up much later as poor
retrieval quality rather than as an embedding error — in our case French
queries ranking badly because only the English-only fallback lane had any
content.

Adds EMBEDDING_TIMEOUT alongside the existing EMBEDDING_BATCH_SIZE and
EMBEDDING_MAX_CHARS, floored at 1s. The default is unchanged at 10s, so
nothing moves for existing installs; a slow first load can now be given
room without patching.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Matthieu 2026-07-26 21:25:27 +02:00
parent d8a2059df8
commit b414aa27c7

View file

@ -54,7 +54,15 @@ class EmbeddingClient:
# running on :11434) fast-fails to the local FastEmbed fallback instead # running on :11434) fast-fails to the local FastEmbed fallback instead
# of stalling startup ~30s per probe. Read stays generous for a real # of stalling startup ~30s per probe. Read stays generous for a real
# endpoint (embedding a short string returns in well under a second). # endpoint (embedding a short string returns in well under a second).
self._client = httpx.Client(timeout=httpx.Timeout(connect=3.0, read=10.0, write=5.0, pool=3.0)) # The read budget holds for a warm endpoint, but the FIRST call after a
# cold start also pays for loading the embedding model into memory — on
# a modest or busy machine that is well past 10s. The timeout then fires
# silently: the lane produces no vectors, its collection stays empty,
# and retrieval quietly degrades to whatever other lane is present, with
# nothing marking the endpoint as the cause. Overridable, default
# unchanged.
_read_timeout = max(1.0, float(os.getenv("EMBEDDING_TIMEOUT", "10")))
self._client = httpx.Client(timeout=httpx.Timeout(connect=3.0, read=_read_timeout, write=5.0, pool=3.0))
self._batch_size = max(1, int(os.getenv("EMBEDDING_BATCH_SIZE", "8"))) self._batch_size = max(1, int(os.getenv("EMBEDDING_BATCH_SIZE", "8")))
self._max_chars = max(200, int(os.getenv("EMBEDDING_MAX_CHARS", "900"))) self._max_chars = max(200, int(os.getenv("EMBEDDING_MAX_CHARS", "900")))