Architecture

I3K RAG Enterprise Community is one Rust binary that runs entirely inside your perimeter. It is not a stack you assemble: it starts the vector database and the inference engine itself, and computes embeddings in its own process.

For installation, see the quickstart. For operating it, the deployment guide.

Components

Component	Role
API and server	Rust, axum — REST surface, JWT user management, query orchestration
Frontend	React + Vite, compiled and served from the binary's directory
Embeddings	`BAAI/bge-m3` through Candle, in-process, GPU or CPU
Vector database	Qdrant, 1024 dimensions, cosine distance — started by the engine
Language model	eullm as a separate process, started and kept up to date by the engine
Application database	SQLite through sqlx — users, documents, conversations
OCR	pdfium for rasterising, Tesseract for recognition — both loaded at runtime from a bundled path, not linked at compile time

Everything listens on one port (8000 by default). There is no Docker, no Compose and no Java anywhere in this picture.

Why the embedding model loads first

The embedding model is loaded before the language model on purpose. eullm sizes its own GPU offload from the free VRAM it observes at startup, so it has to see the memory the embedding model has already taken. Reversing the order makes eullm over-commit.

The pipeline

Ingest. Documents arrive through the web UI or the REST API: PDF, DOCX, XLSX, HTML, TXT, Markdown and CSV. Scanned pages are detected automatically — when a page yields too little text it is rasterised with pdfium and passed through Tesseract in Italian and English.
Embed and store. Text is split into overlapping chunks and embedded with BAAI/bge-m3 — 1024 dimensions, multilingual across 100+ languages — then stored in Qdrant with the metadata used for role filtering.
Retrieve. The question is embedded the same way and answered from the closest passages. Relevance threshold and top-K are configurable, and role filtering is applied here, so a query never reaches the model carrying passages the caller may not see.
Generate. The passages go to the local language model through eullm, which streams the answer token by token with each source shown. Conversations are kept and history is carried into follow-up questions.

Users and roles

JWT authentication with three roles:

User — ask questions and read answers.
Super user — everything a user can do, plus uploading and deleting documents.
Admin — full system management, including accounts and configuration.

Enforcement happens at the retrieval layer, translated into metadata filters on the vector store, rather than being applied to the answer after the fact.

Data sovereignty

The design rule is that nothing leaves the perimeter.

The language model runs on your hardware through eullm, a process on the same machine.
Embeddings are computed inside the engine process, not called out to a service.
Qdrant stores vectors on local disk; SQLite holds users, documents and conversations on local disk.
After the first run no network access is needed at all.
There is no telemetry. Nothing is collected or sent, ever.

Security

Transport — terminate TLS at a reverse proxy in front of the engine.
Authentication — JWT tokens signed with AUTH__JWT_SECRET, the one setting with no default.
Component integrity — every downloaded component is verified against a sha256 pinned in the manifest; a mismatch aborts the run.
Network egress — none after the first run. The install can be operated fully air-gapped.

Backups

A scheduled daily archive holds the SQLite database and a Qdrant snapshot taken together, so the pair is consistent, plus an admin endpoint to restore one over the running installation. Backups are local files under BACKUP__DIR; nothing is uploaded anywhere.

Measuring it

The binary can benchmark itself on your own hardware and your own documents:

./i3k-rag-engine --bench /path/to/document.pdf

It writes a Markdown report timing each stage — extraction, chunking, embedding, upsert, prefill, decode — with charts showing where the time goes. --bench-live instead records every real ingestion and query of a session and reports on shutdown.

This matters for sizing: it turns "how much hardware do we need" from an estimate into a measurement on your actual corpus.

Licence

The Community engine is published under AGPL-3.0-only. Releases up to and including v0.1.39 were published under Apache-2.0 and remain available under those terms.

Running it privately — inside your own company, on your own documents — carries no source-disclosure obligation. AGPL section 13 applies when you offer a modified version to users over a network. For organisations that cannot accept those terms, a separate commercial licence is available.

Source: github.com/I3K-IT/RAG-Enterprise.