ChatGPT retrieval plugin with Zilliz and Milvus
Introduction
OpenAI, the artificial intelligence research laboratory, recently announced that they had open-sourced the code for a knowledge base retrieval plugin. This project, which I was involved in, allows ChatGPT to augment its information by retrieving knowledge-based data from relevant document snippets from various data sources. In this blog post, I will discuss the implications of this project and the benefits it offers for enterprises.
The Knowledge Base Retrieval Plugin
The knowledge base retrieval plugin is a tool that allows ChatGPT to retrieve information from relevant document snippets that an outside party hosts. ChatGPT can retrieve pertinent data from various sources that you have collected and stored, such as files, notes, or emails, by asking questions or expressing needs in natural language. This feature offers an easy and efficient way for ChatGPT to retrieve information, making it more accurate and reliable.
Benefits for Enterprises
Enterprises can benefit greatly from this plugin by making their internal documents available to their employees through ChatGPT. This offers a streamlined way for employees to retrieve information from internal documents without searching multiple sources. It also ensures that the information retrieved is accurate and up-to-date. Developers can also use this plugin to host their own knowledge bases, making retrieving data from their own sources easier.
How it works
The plugin uses OpenAI's text-embedding-ada-002 embeddings model to generate embeddings of document chunks and then stores and queries the embedding into a vector database. This is where I come in. I worked with the OpenAI team to make Milvus and Zilliz one of the preferred vector databases to store these embeddings.
The simple API exposes the plugin's endpoints for upserting, querying, and deleting documents. You can refine their search results using metadata filters by source, date, author, etc. To keep Milvus or Zilliz updated with the latest documents, the plugin can continuously process and store documents from various data sources, using incoming webhooks to the upsert and delete endpoints.
Here's a quick example using the Zilliz Plugin to find the DRM policy within Apple's terms of Service (TOS). ChatGPT begins by searching the prompt within Zilliz Cloud and then summarizes the results. ChatGPT then accesses the Zilliz via the retrieval plugin to search for the most relevant data.
How to setup Zilliz Cloud
With Zilliz Cloud you simply need to create an account and set up your instance with your embeddings. Here is the quick start guide and the environmental variables in the retrieval plugin you need to configure:
Environmental variables
| Name | Required | Description |
|---|---|---|
DATASTORE | Yes | Datastore name, set to zilliz |
BEARER_TOKEN | Yes | Your secret token |
OPENAI_API_KEY | Yes | Your OpenAI API key |
ZILLIZ_COLLECTION | Optional | Zilliz collection name. Defaults to a random UUID |
ZILLIZ_URI | Optional | URI for the Zilliz instance |
ZILLIZ_USER | Optional | Zilliz username |
MILVUS_USER | Optional | Zilliz password |
Once you have this configured, run the following test to confirm that the integration is working with this command.
pytest ./tests/datastore/providers/zilliz/test_zilliz_datastore.py
How to set up Milvus
Of course, if you prefer open source Milvus, you can deploy and manage Milvus using Docker Compose, Helm, K8's Operator, or Ansible. Follow the instructions to get started.
Environmental variables
| Name | Required | Description |
|---|---|---|
DATASTORE | Yes | Datastore name, set to milvus |
BEARER_TOKEN | Yes | Your bearer token |
OPENAI_API_KEY | Yes | Your OpenAI API key |
MILVUS_COLLECTION | Optional | Milvus collection name, defaults to a random UUID |
MILVUS_HOST | Optional | Milvus host IP, defaults to localhost |
MILVUS_PORT | Optional | Milvus port, defaults to 19530 |
MILVUS_USER | Optional | Milvus username if RBAC is enabled, defaults to None |
MILVUS_PASSWORD | Optional | Milvus password if required, defaults to None |
Once you have this configured, run the following test to confirm that the integration is working with this command.
pytest ./tests/datastore/providers/milvus/test_milvus_datastore.py
Deploy the plugin
With the plugin tested we can move to deploying it. ChatGPT accesses plugins by sending requests to wherever your plugin server is running, whether it be Amazon ECS, Fly.io, Heroku, etc. For a detailed guide on how to configure this, take a look at the Deployment section in the GitHub readme.
The Memory Feature
One of the cool features with this new plugin is a feature that they call memory. This gives ChatGPT the ability to remember information from conversations and store it in a vector database for later use.
Acknowledgments
This was a fun project and I’d like to thank the OpenAI team of Isabella Fulford and Ziliz team of Frank Liu and Robert Guo. If you have any feedback or want to try it out for yourself, check out the GitHub repo. I personally am looking forward to contributing to one of the many great ideas listed on the ChatGPT Retrieval Plugin readme!
Filip HaltmayerFilip Haltmayer is a Software Engineer at Zilliz working in both software and community development. His contributions mainly revolve around the Milvus and Towhee projects, helping develop both applications and helping grow their respective user bases through client interaction, integrations, and technical talks.
- Introduction
- The Knowledge Base Retrieval Plugin
- Benefits for Enterprises
- How it works
- How to setup Zilliz Cloud
- How to set up Milvus
- Deploy the plugin
- The Memory Feature
Content
Start Free, Scale Easily
Try the fully-managed vector database built for your GenAI applications.
Try Zilliz Cloud for Free