A fully local RAG system that uses Mistral‑7B‑Instruct (GGUF) via llama.cpp, bundled with Docker and a Streamlit frontend.

Features

  • One‑click scripts for Windows, macOS, and Linux (run.bat, run.command, run.sh).
  • First run builds the Docker image, downloads the ~4–5 GB model, and launches the app at http://localhost:8501.
  • Bring your own documents; everything works offline after the initial model download.

Quick start

# Windows
run.bat

# macOS
./run.command

# Linux
./run.sh

# App opens at:
# http://localhost:8501

Repository