PrivateGPT
Open-source API layer from Zylon for building RAG and assistant applications on top of self-hosted language models.
What it does
PrivateGPT is an open-source API layer for AI applications built on local or self-hosted language models. It does not run models itself but connects to an OpenAI-compatible inference server such as Ollama, llama.cpp or vLLM. Its interface follows the Claude API: document ingestion, answers with citations, tool use, MCP connectors, web search, code execution, queries against databases and CSV files, and embeddings. A simple web interface is included for testing.
Who it suits
Development teams and IT departments that want to build their own assistants or document search without handing data to cloud services. Installation and running models require technical know-how.
Data protection for businesses
You run PrivateGPT on your own hardware or in your own cloud, so you decide where the data lives. With a local model, inputs never leave your network and Zylon does not receive them for training. If you connect an external model server, its terms apply. There is no data processing agreement for the open-source version, since Zylon processes no data in that setup. For businesses, Zylon offers a paid platform with Kubernetes deployment, LDAP/AD, role-based access, audit logs and support; the vendor shares prices only on request.
Limits
Quality and speed depend on the chosen model and your hardware. Prompt caching, OAuth and organization management are missing, and skills are only basic so far. The web interface is not a finished chat tool for staff.