Kotaemon and Zilliz Cloud Integration
Kotaemon and Zilliz Cloud integrate to build AI-powered document QA applications, combining Kotaemon's open-source customizable RAG UI with Zilliz Cloud's scalable vector database for efficient hybrid retrieval, multi-modal question-answering, and advanced citations with document previews.
Use this integration for FreeWhat is Kotaemon
Kotaemon is an open-source, clean and customizable RAG UI for chatting with your documents. Built with both end users and developers in mind, it provides a customizable, multi-user document QA web-UI supporting local and API-based LLMs. It offers a hybrid RAG pipeline with full-text and vector retrieval, multi-modal QA for documents with figures and tables, and advanced citations with document previews. It supports complex reasoning methods like ReAct and ReWOO, and provides configurable settings for retrieval and generation.
By integrating with Zilliz Cloud (fully managed Milvus), Kotaemon gains access to a fully managed, scalable vector database that enhances its retrieval capabilities, enabling efficient similarity search across large document collections for enterprise document search, research assistance, and collaborative document analysis scenarios.
Benefits of the Kotaemon + Zilliz Cloud Integration
- Multi-user document QA with scalable storage: Kotaemon's multi-user web interface paired with Zilliz Cloud's scalable vector storage enables teams to collaboratively chat with and analyze documents at scale.
- Hybrid retrieval for better accuracy: The integration supports hybrid RAG combining full-text and vector search powered by Zilliz Cloud, improving retrieval accuracy compared to single-method approaches.
- Multi-modal QA support: Kotaemon handles documents with figures and tables through multi-modal question-answering, with Zilliz Cloud efficiently storing and retrieving the corresponding embeddings.
- Advanced citations with document previews: The system provides detailed citations with document previews and confidence scores, backed by Zilliz Cloud's fast and precise vector retrieval.
- Flexible model configuration: Users can configure and switch between multiple LLMs and embedding models while Zilliz Cloud handles the vector storage layer seamlessly.
How the Integration Works
Kotaemon serves as the application and UI layer, providing a web-based interface for document upload, chat-based QA, conversation management, and result display with citations. It handles document processing, embedding generation, and orchestrates the RAG pipeline with support for complex reasoning methods.
Zilliz Cloud serves as the vector database layer, storing and indexing document embeddings generated by Kotaemon for fast similarity search. It provides high-performance retrieval with low latency, enabling efficient document retrieval across large knowledge bases.
Together, Kotaemon and Zilliz Cloud create a complete document QA solution: users upload documents through Kotaemon's web interface, which processes and embeds them into Zilliz Cloud. When users ask questions, Kotaemon's hybrid retrieval pipeline leverages Zilliz Cloud's vector search alongside full-text search to find the most relevant context, then generates answers with advanced citations and confidence scores.
Step-by-Step Guide
1. Install Kotaemon
We recommend installing Kotaemon in this way:
# optional (setup env) conda create -n kotaemon python=3.10 conda activate kotaemon git clone https://github.com/Cinnamon/kotaemon cd kotaemon pip install -e "libs/kotaemon[all]" pip install -e "libs/ktem"Besides this way, there are some other ways to install Kotaemon. You can refer to the official documentation for more details.
2. Set Milvus as the Default Vector Storage
To change the default vector storage to Milvus, you have to modify the
flowsettings.pyfile by switchingKH_VECTORSTOREto:"__type__": "kotaemon.storages.MilvusVectorStore"3. Set Environment Variables
You can configure the models via the
.envfile with the information needed to connect to the LLMs and embedding models. e.g. OpenAI, Azure, Ollama, etc.4. Run Kotaemon
After setting up the environment variables and changing the vector storage, you can run Kotaemon by running the following command:
python app.pyDefault username / password are:
admin/admin5. Add Your AI Models
In the
Resourcestab, you can add and set your LLMs and embedding models. You can add multiple models and set them as active or inactive. You only need to provide at least one. You can also provide multiple models to allow switching between them.6. Upload Your Documents
In order to do QA on your documents, you need to upload them to the application first. Navigate to the
File Indextab, and you can upload and manage your custom documents.By default, all application data are stored in
./ktem_app_datafolder. The Milvus database data is stored in./ktem_app_data/user_data/vectorstore. You can back up or copy this folder to move your installation to a new machine.7. Chat with Your Documents
Now navigate back to the
Chattab. The Chat tab consists of 3 regions: the Conversation Settings Panel, where you manage conversations and file references; the Chat Panel for interacting with the chatbot; and the Information Panel, which displays supporting evidence, confidence scores, and relevance ratings for answers.You can select your documents in the Conversation Settings Panel. Then just start RAG with your documents by typing a message in the input box and send it to the chatbot.
Learn More
- Kotaemon RAG with Milvus — Official Milvus tutorial for customizing Kotaemon with Milvus
- Kotaemon GitHub Repository — Kotaemon source code and community resources
- Kotaemon Official Documentation — Official Kotaemon usage guide
- Build AI Apps with Milvus: Tutorials & Notebooks — Zilliz collection of Milvus tutorials and notebooks
- Build Advanced RAG and Multi-Modal Queries with Milvus and Friendli AI — Zilliz blog on advanced RAG and multi-modal queries