KamoCRM

Size every TEI thread pool to the 1-CPU limit, and let probes outlast a batch

FixKlusterServices
Verschifft
25. September 2026 um 10:25 UTC
Autor
Kamo
Ausschuss
5edb887

TEI 1.5's ONNX backend starts one inference thread per physical host core (24 on k1m1) and reads no thread setting, so under the pod's 1-CPU quota the CFS throttle froze the pool mid-batch: a batch of 8 went from 0.3 s to 5-11 s, /health (which waits for the batch in flight) outlasted kubelet's 1-s timeout, and liveness killed the model twice in one reindex. RAGService acks an event whose embed failed, so those KB and note updates were lost, and 106 Help Center articles never got vectors. - image cpu-1.5 -> cpu-1.9.4: from 1.6 the ONNX pool is sized from the cgroup CPU limit. Same model files; the vectors are bit-identical to 1.5's, so Qdrant's points stay valid. SP01's reranker runs the same tag. - --tokenization-workers 1, and OMP/MKL/RAYON_NUM_THREADS=1 (the image sets RAYON to 8), so every pool matches the limit even if TEI falls back to its candle backend. - liveness and readiness timeoutSeconds 1 -> 10. **************** (python3 -m unittest discover -s embedding-model/tests) pins the manifest, and with TEI_ROOTFS/TEI_MODEL_DIR runs the manifest's image, args and env under its CPU quota with five RAGService-shaped clients: cpu-1.5 ran 35 threads, 2.7 chunks/s and failed 102 of 121 probes; cpu-1.9.4 runs 6 threads, 8.1 chunks/s and fails none. KamoAI SP00 final review I-1.

Alle Änderungen

Wie, was Sie sehen Versand?

Alles kommt in Ihrem Arbeitsbereich für sich. Starten Sie mit dem kostenlosen Plan und lesen Sie diese Seite in einem Monat wieder.

Free Forever startenPreisgestaltung anzeigen