The RAG generator.

Point it at a folder. Watch the engine compile it. Then search the result and export it. A local, single-user workbench that turns "we have a shared drive full of documents" into a retrievable corpus you can interrogate before you trust it.

What it is

Everything the generator does is available through the engine API: submit a batch to /v1/jobs, poll it, read /v1/jobs/{id}/documents/{doc}/chunks, query /v1/search. What the generator adds is the loop around them, so the whole thing is something you can watch and judge rather than a sequence of curl commands whose output you have to hold in your head.

It is deliberately small and deliberately local: one browser tab, one user, no accounts, no database of its own beyond a JSON file recording which documents came from which folder.

Figure: the workbench on the Search tab. Every hit carries the file, page and section it came from.

How the pieces fit

The generator is a client. It ships no parser, no models and no index; it calls an engine you already run. That means three things have to be alive, and the second one catches people out:

• api accepts uploads and answers status. In the standard topology it runs NAUTRIS_WORKERS: "0", so it does no parsing itself.
• worker does the actual extraction. An engine with no worker looks perfectly healthy and accepts your files, then every job sits at queued forever.
• a vector index, enabled by setting NAUTRIS_RAG_DB on both. Without it documents still compile, but /v1/search answers 409 and nothing is retrievable.

Diagram: the generator only ever talks to your engine, and the engine only ever talks to itself.

Run it with Compose

One file brings up the engine and the generator together. The generator image is private to your organization; the engine image is public, and the license is the gate.

# docker-compose.yml - engine + RAG generator, one command
services:
  postgres:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER: nautris
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?required}
      POSTGRES_DB: nautris
    volumes: ["pgdata:/var/lib/postgresql/data"]
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U nautris -d nautris"]
      interval: 10s
      timeout: 5s
      retries: 5

  api:
    image: registry.nautris.com/nautris:latest
    restart: unless-stopped
    environment:
      NAUTRIS_LICENSE: ${NAUTRIS_LICENSE:?required}
      NAUTRIS_DATABASE_URL: postgresql://nautris:${POSTGRES_PASSWORD}@postgres:5432/nautris
      NAUTRIS_RAG_DB: /data/lancedb   # without this, search returns 409
      NAUTRIS_WORKERS: "0"            # the api enqueues...
      NAUTRIS_BACKGROUND: "1"         # ...the worker processes
      NAUTRIS_API_KEYS: "rag=${ENGINE_KEY:?required}"
    volumes: ["engine-data:/data"]
    depends_on:
      postgres: { condition: service_healthy }

  # Scale the parsing tier:  docker compose up -d --scale worker=4
  worker:
    image: registry.nautris.com/nautris:latest
    restart: unless-stopped
    command: ["python", "-m", "nautris.worker"]
    environment:
      NAUTRIS_LICENSE: ${NAUTRIS_LICENSE:?required}
      NAUTRIS_DATABASE_URL: postgresql://nautris:${POSTGRES_PASSWORD}@postgres:5432/nautris
      NAUTRIS_RAG_DB: /data/lancedb   # must match the api
    volumes: ["engine-data:/data"]
    depends_on:
      postgres: { condition: service_healthy }

  rag:
    image: ghcr.io/mhchlagou/nautris-rag-generator:latest
    restart: unless-stopped
    ports: ["3100:3100"]
    environment:
      NAUTRIS_URL: http://api:8000   # resolvable INSIDE the network
      RAG_DATA_DIR: /data
    volumes:
      - rag-data:/data
      # The folder you want to compile. Read-only is enough: the generator
      # never writes to your documents.
      - /srv/documents:/docs:ro
    depends_on:
      api: { condition: service_started }
      # Without a worker every job stays queued forever, so depend on it.
      worker: { condition: service_started }

volumes:
  pgdata:
  engine-data:
  rag-data:

docker compose up -d
# then open http://localhost:3100 and scan /docs

Which folders it can see

This is the single most common source of confusion, and it has a one-sentence rule: the generator scans its own filesystem. It does not upload from your browser and it cannot reach your engine's disk.

So in a container it sees exactly what you mount. Mount read-only; it never writes to your documents:

    volumes:
      - /srv/documents:/docs:ro     # then scan  /docs
      - /home/you/reports:/reports:ro  # then scan  /reports

If you would rather type any path on your machine with no mounting at all, run the generator directly instead of in a container, and point it at an engine that publishes a port:

npm install && npm run start     # http://localhost:3000
# engine URL: http://localhost:8000  (add  ports: ["8000:8000"]  to the api service)

The four steps

1. Connect

Enter the engine URL and an API key. The generator reports back the plan, days remaining, page quota position, queue depth, whether a vector index exists, and whether folder linking is included in your license. Everything that changes what you can do next is stated here rather than discovered as a failure three steps later.

The key is used server-side and kept in session storage only, so it disappears when you close the tab.

2. Folder

The scan walks the folder and types every file by content, using the same magic-byte routing the engine applies at intake. A PDF named .txt is reported as a PDF, because that is what the engine will make of it.

Nothing is uploaded yet, so this is the cheap moment to discover that half the folder is archives. Files are set aside with a reason:

Hidden folders, caches and dependency directories are skipped, and symlinks are never followed.

3. Compile

Files go up in batches of 200, the engine's own limit, and each document is reported as it settles: pages, blocks, chunks, how long it took, any flags raised for review, and the engine's exact reason for a failure. Stopping mid-run keeps everything already compiled, and re-running is safe because documents are keyed by content rather than by filename.

4. The RAG

Four tabs. Search opens first, because "is this any good?" is the question you actually arrive with. Chunks lists every indexed chunk with its provenance and a redaction toggle. Overview shows the shape of the corpus, review flags, and the documents that failed. Export takes it elsewhere.

Validating retrieval

The point of the Search tab is to let you disbelieve the corpus productively. Ask something you already know the answer to. If the right passage comes back with the right page, the corpus is sound. If it does not, two things usually explain why.

The mode matters. Semantic search finds meaning and tolerates different wording; keyword (BM25) finds exact terms, names, codes and identifiers that an embedding will happily miss; hybrid fuses both and is the best default. A term that returns nothing under one mode is often the top hit under another.

Provenance tells you where a hit came from. Every result carries its source file, page, section breadcrumb, the block ids it was built from and, for formats that have geometry, the bounding boxes. When retrieval surfaces something odd, that trail usually shows whether the problem is the query, the chunking, or the document itself.

Linked folders

By default the generator uploads. If your license includes the sources feature, it can instead register the folder with the engine and let the engine pull it, so a re-run reprocesses only what changed:

new: 1   changed: 1   removed: 1   unchanged: 11

Two constraints, both surfaced in the interface rather than left to fail:

• The feature is part of trial, pro and enterprise. On personal and starter the engine answers 402 and the option is hidden. Uploading works on every plan.
• The path is resolved by the engine, not the generator. If the engine runs in a container, the folder must be mounted into it at the same path.

Exporting the RAG

A RAG you cannot move is a RAG you do not own. Everything is regenerated from the engine's stored trees at the moment you ask, using the same deterministic chunker that filled the index, so the same project always exports the same bytes.

Troubleshooting