The AI assistant inside KamoCRM reads a tenant's own documents before it answers. Making that work meant running the whole chain ourselves: a model server, an embedding model, a vector database, speech recognition, speech synthesis and machine translation, all inside our cluster, all on data that never leaves it.
| Measure | Value | Source | Measured |
|---|---|---|---|
| Self-hosted inference and speech services | 6 (model server, embedding model, vector database, speech recognition, speech synthesis, machine translation) | kubectl get deployments,statefulsets --all-namespaces | |
| Dedicated retrieval and vector services in the platform | 3 (RAG service, vector service, embedding model) | kubectl get deployments -n kamo | |
| Languages served by the self-hosted translation pipeline | 22 | messages/*.json in the marketing site repository | |
| Optical character recognition error rate | 0.180 CER | ocrservice evaluation run |
What building it taught us
Calling a hosted model API is an afternoon's work and we do it too, where it is the right answer. Running inference yourself is a different discipline, and it is the one clients ask about the moment their data is sensitive — healthcare, finance, legal, anything under a data-residency obligation. At that point 'we send it to a third party' ends the conversation, and only a team that has actually operated the alternative can continue it.
Retrieval is where these systems are won or lost, and it is mostly unglamorous. Chunking documents so that a retrieved passage is self-contained. Deciding what a query even means when the user typed four words. Making the retrieval respect permissions, so that an assistant cannot answer from a document the person asking is not allowed to read — which is easy to get wrong, and the kind of wrong that ends up in a regulator's letter rather than a bug tracker. Our knowledge base has visibility tiers that the retrieval layer has to honour exactly as the human-facing reads do, in two separate places that must agree.
We also run speech recognition and synthesis, and machine translation across twenty-two languages. The translation pipeline taught us the most, mostly by failing silently: a bulk update running without a transaction succeeded on paper and wrote nothing for twenty-one of the languages, and because nothing threw, nobody noticed for a long time. Silent failure is the defining hazard of batch AI work, and we now instrument for it by default.
Built with
- Ollama
- Qdrant
- Whisper
- Coqui TTS
- LibreTranslate
- Python
- Java
- Spring Boot
What this qualifies us for
Tell us what you are trying to do
A short conversation is usually enough to tell whether we are the right people. If we are not, we will say so and point you somewhere better.