Open WebUI embeddings + reranking — operator reference

Target: operators wiring Open WebUI's RAG pipeline to HuggingFace Text Embeddings Inference (TEI) via LiteLLM. Three hops, each with its own wire-shape quirks. Most failure modes silently degrade to "answer quality dropped" rather than visible errors — this skill is a triage for catching them at config-time.

The architecture in 30 seconds

Open WebUI → LiteLLM proxy → TEI (GPU)
            └ embed: openai-driver → /v1/embeddings
            └ rerank: huggingface-driver → /rerank (Cohere↔TEI translation)

Why proxy through LiteLLM rather than point Open WebUI at TEI directly?

• Embed: TEI exposes /v1/embeddings natively (OpenAI-compat) — direct path works. LiteLLM adds: virtual-key auth, per-model rate limits, request logging, optional caching.
• Rerank: TEI's native /rerank is {query, texts} → [{index, score}]. Open WebUI's ExternalReranker sends Cohere shape {query, documents, top_n} → {results: [{index, relevance_score}]}. Direct path fails with HTTP 422 — wire shapes do not match. LiteLLM's HuggingFace rerank handler translates between the two.

Skipping LiteLLM is therefore feasible only for embed; rerank requires either LiteLLM (or another Cohere↔TEI shim) unless Open WebUI itself is patched.

Wire shapes (exact)

Embed — Open WebUI code path

backend/open_webui/retrieval/utils.py:677 (generate_openai_batch_embeddings):

POST {RAG_OPENAI_API_BASE_URL}/embeddings        ← URL is auto-appended
Authorization: Bearer {RAG_OPENAI_API_KEY}
Content-Type: application/json

{"input": ["text1", "text2", ...], "model": "{RAG_EMBEDDING_MODEL}"}

Response parsed as data["data"][i]["embedding"] (OpenAI shape).

Async fan-out (utils.py:905, get_embedding_function → asyncio.gather at utils.py:963): chunks bundled into batches of RAG_EMBEDDING_BATCH_SIZE (default 1); all batches dispatched concurrently via asyncio.gather with optional semaphore from RAG_EMBEDDING_CONCURRENT_REQUESTS (default 0 = unlimited). A 100-chunk file at default config fires 100 concurrent single-chunk requests.

Rerank — Open WebUI code path

backend/open_webui/retrieval/models/external.py:14 (ExternalReranker, predict at line 27):

POST {RAG_EXTERNAL_RERANKER_URL}                 ← URL is exact, NOT appended
Authorization: Bearer {RAG_EXTERNAL_RERANKER_API_KEY}
Content-Type: application/json

{"model": "{RAG_RERANKING_MODEL}", "query": "...",
 "documents": ["doc1", "doc2", ...], "top_n": N}

Response parsed: data["results"] sorted by index, extracts relevance_score. Cohere shape, strict.

Failure handling: requests.post() exception or non-2xx → predict() returns None → retrieval silently downgrades to un-reranked hybrid order. No user-visible error in Open WebUI. Always alert on rerank-side 4xx in TEI/LiteLLM logs.

Open WebUI environment variables

Triage table

Reference index

• `references/gotchas.md` — nine gotchas with HTTP error strings, root causes, and fixes. Load when triage table points here.
• `references/end-to-end-config.md` — full working LiteLLM + Open WebUI + TEI config (BGE-M3 + BGE-Reranker-v2-m3 worked example). Load when bootstrapping a new deployment.
• `references/performance.md` — quality verification (cross-engine numerical-identity check) + throughput baseline. Load for sizing or post-deployment health checks.
• `references/sources.md` — authoritative source files and PR/issue URLs underlying every claim. Load to verify a specific claim or run freshen mode.