FastGPT and Zilliz Cloud Integration
FastGPT and Zilliz Cloud integrate to build powerful knowledge-based question-answering systems, combining FastGPT's LLM-powered platform with visual workflow orchestration and Zilliz Cloud's scalable vector database for fast and accurate RAG retrieval.
Use this integration for FreeWhat is FastGPT
FastGPT is a knowledge-based question and answer system built on large language models (LLMs), offering ready-to-use capabilities for data processing and model invocation. It enables workflow orchestration through Flow visualization, facilitating complex question and answer scenarios. With its out-of-the-box features, developers can quickly create and deploy sophisticated QA systems across various knowledge domains.
By integrating with Zilliz Cloud (fully managed Milvus), FastGPT gains access to a fully managed, scalable vector database for storing and retrieving knowledge embeddings, enabling faster retrieval of relevant information, improved accuracy in answering queries, and the ability to handle large-scale knowledge bases effectively.
Benefits of the FastGPT + Zilliz Cloud Integration
- Ready-to-use QA system with scalable storage: FastGPT provides out-of-the-box data processing and model invocation capabilities, while Zilliz Cloud ensures efficient vector storage and retrieval as your knowledge base grows.
- Visual workflow orchestration: FastGPT's Flow visualization lets you design complex question-answering workflows visually, backed by Zilliz Cloud's high-performance similarity search for the retrieval step.
- Docker-based rapid deployment: The integration supports both Milvus and Zilliz Cloud via Docker Compose, allowing teams to quickly deploy a complete QA system with minimal configuration.
- Improved retrieval accuracy: Zilliz Cloud's vector similarity search ensures that the most relevant knowledge is retrieved to augment LLM responses, reducing hallucinations and improving answer quality.
How the Integration Works
FastGPT serves as the application layer, providing the LLM-based question-answering platform with visual workflow orchestration, data processing, and model invocation capabilities. It handles user queries, orchestrates the RAG pipeline through its Flow interface, and generates responses using the retrieved context.
Zilliz Cloud serves as the vector database layer, storing and indexing knowledge base embeddings for fast similarity search. When a query comes in, Zilliz Cloud retrieves the most relevant documents to provide context for FastGPT's language model to generate accurate answers.
Together, FastGPT and Zilliz Cloud create a complete knowledge-based QA solution: documents are processed and embedded into Zilliz Cloud, and when users ask questions, FastGPT orchestrates the retrieval of relevant context through Zilliz Cloud's vector search, then generates informed responses using the LLM — all deployable via Docker Compose with minimal setup.
Step-by-Step Guide
1. Download docker-compose.yml
Ensure that you have already installed Docker Compose. Execute the commands below to download the configuration files.
$ mkdir fastgpt $ cd fastgpt $ curl -O https://raw.githubusercontent.com/labring/FastGPT/main/projects/app/data/config.json # milvus version $ curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/files/docker/docker-compose-milvus.yml # zilliz version # curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/files/docker/docker-compose-zilliz.ymlIf you're using the Zilliz version, adjust the
MILVUS_ADDRESSandMILVUS_TOKENlink parameters in the docker-compose.yml file, which corresponds to the Public Endpoint and API Key in Zilliz Cloud.2. Launch the Container
Execute in the same directory as docker-compose.yml. Ensure that the docker-compose version is ideally above 2.17, as some automation commands may not function otherwise.
# Launch the container $ docker compose up -d # Wait for 10s, OneAPI typically needs to restart a few times to initially connect to Mysql $ sleep 10 # Restart oneapi $ docker restart oneapi3. Access OneAPI to Add Models
OneAPI can be accessed at
ip:3001. The default username is root, and the password is 123456. You can alter the password after logging in. Using OpenAI's model as an example, click on the "Channel" tab, and select your chat model and embedding model under "Models". Input your OpenAI API Key in the "Secrets" section.4. Setting Tokens
Click on the "Tokens" tab. By default, there is a token
Initial Root Token. You can also create a new token and set a quota on your own. Click "Copy" on your token, ensuring that the value of this token matches theCHAT_API_KEYvalue set in the docker-compose.yml file.5. Accessing FastGPT
FastGPT can be directly accessed at
ip:3000(please mind the firewall). The login username is root, with the password set toDEFAULT_ROOT_PSWwithin the docker-compose.yml environment variable. Should you require domain name access, you would need to install and configure Nginx on your own.6. Stop the Container
Run the following command to stop the container.
$ docker compose downLearn More
- Deploying FastGPT with Milvus — Official Milvus tutorial for deploying FastGPT
- FastGPT GitHub Repository — FastGPT source code and community resources
- Build AI Apps with Milvus: Tutorials & Notebooks — Zilliz collection of Milvus tutorials and notebooks
- Enhancing ChatGPT with Milvus: Powering AI with Long-Term Memory — Zilliz tutorial on enhancing LLMs with vector storage
- FastGPT Documentation — Official FastGPT documentation