Deployment

The engine is one binary on one host. That removes most of what usually counts as deployment work, and leaves a short list of decisions that genuinely matter: where its data lives, whether it may fall back to CPU, how it is exposed, and how it is restored when something goes wrong.

Install steps are in the quickstart.

Configuration

Settings come from the environment or from a .env file next to the binary, using __ to separate levels. Exactly one has no default:

AUTH__JWT_SECRET=…            # required — signs the session tokens

Everything else is optional:

SERVER__PORT=8000
AUTH__ADMIN_DEFAULT_PASSWORD=…   # otherwise a random one is generated and logged
QDRANT__COLLECTION=rag_documents
EMBEDDINGS__REQUIRE_GPU=false    # true = refuse to start without CUDA rather than
                                 # silently falling back to a much slower CPU
DATA__DIR=/path/to/data          # defaults to the binary's own directory
BACKUP__DIR=…                    # defaults to {DATA__DIR}/backups
RUST_LOG=info

Set EMBEDDINGS__REQUIRE_GPU in production

Without it, a broken CUDA setup degrades quietly to CPU and ingestion goes from seconds to minutes — easy not to notice until someone complains. The fallback is always logged at error level and exposed on GET /api/info, but on a production host you usually want a hard failure instead.

Where the data lives

DATA__DIR defaults to the directory holding the executable, and it holds everything: the downloaded components, the Qdrant storage, the SQLite database and the uploaded documents.

Point it at a dedicated volume before the first run. Doing it afterwards means the components are fetched again at the new path, since nothing there exists yet.

Sizing

Measure rather than estimate. The binary benchmarks itself on your own hardware and your own documents:

./i3k-rag-engine --bench /path/to/document.pdf

The Markdown report it writes breaks the time down by stage — extraction, chunking, embedding, upsert, prefill, decode — which tells you whether you are short of GPU, of CPU, or of nothing at all. --bench-live records a real session instead and reports on shutdown.

As a starting point:

Resource	Guidance
GPU	An NVIDIA GPU is detected and used automatically, for generation and — where asked — for embedding. Without one, everything runs on CPU
RAM	16 GB minimum, 32 GB comfortable
Storage	30 GB free before the first run, then growing with the corpus
OS	Linux x86_64, Linux ARM64, or 64-bit Windows
Network	For the first run only

Exposing it

The engine speaks plain HTTP on SERVER__PORT. Put a reverse proxy in front of it and terminate TLS there — nginx, Caddy or whatever your organisation already runs. Do not expose the engine's port directly to an untrusted network.

Set AUTH__ADMIN_DEFAULT_PASSWORD before the first run, or capture the generated one from the log immediately: it is printed once and not shown again.

Backups and restore

A scheduled daily archive holds the SQLite database and a Qdrant snapshot taken together, so the two are consistent with each other. An admin endpoint restores an archive over the running installation.

Archives are local files under BACKUP__DIR. Nothing is uploaded anywhere, which means offsite copies are your responsibility: point your existing backup tooling at that directory, the same way you would at any other data volume.

Test the restore, not the backup

A backup that has never been restored is a hypothesis. Restore one onto a spare host before you need to do it under pressure — the archive is designed so that the database and the vector snapshot come back as a matching pair, and that is exactly the property worth verifying once.

Air-gapped operation

After the first run the engine needs no network at all: every component it uses has already been fetched and verified.

For a host that will never have connectivity, do the first run on a staging machine with the same platform, then move the whole DATA__DIR across. Because components are only re-fetched when missing or failing their pinned checksum, the engine finds everything in place and starts without reaching out.

Upgrading

Extract the new release over the existing install — same directory, overwriting the binary, .env.example and frontend/dist/. Everything already downloaded is reused; only components whose pinned version actually changed are fetched again. Your .env is never touched, because the tarball only ships .env.example.

Read the CHANGELOG before upgrading. The 0.1.x line is a new codebase whose storage layout and HTTP surface are not settled yet.

Building from source

Most people do not need to: the release tarballs are self-contained. If you want to modify the engine or target a platform we do not publish, BUILD.md covers the native dependencies and feature flags.

cargo build --release --features cuda

CUDA is the only build-time system dependency, and only for that feature.