ChatGPT retrieval plugin with Zilliz and Milvus

Introduction

OpenAI, the artificial intelligence research laboratory, recently announced that they had open-sourced the code for a knowledge base retrieval plugin. This project, which I was involved in, allows ChatGPT to augment its information by retrieving knowledge-based data from relevant document snippets from various data sources. In this blog post, I will discuss the implications of this project and the benefits it offers for enterprises.

The Knowledge Base Retrieval Plugin

The knowledge base retrieval plugin is a tool that allows ChatGPT to retrieve information from relevant document snippets that an outside party hosts. ChatGPT can retrieve pertinent data from various sources that you have collected and stored, such as files, notes, or emails, by asking questions or expressing needs in natural language. This feature offers an easy and efficient way for ChatGPT to retrieve information, making it more accurate and reliable.

Benefits for Enterprises

Enterprises can benefit greatly from this plugin by making their internal documents available to their employees through ChatGPT. This offers a streamlined way for employees to retrieve information from internal documents without searching multiple sources. It also ensures that the information retrieved is accurate and up-to-date. Developers can also use this plugin to host their own knowledge bases, making retrieving data from their own sources easier.

How it works

The plugin uses OpenAI's text-embedding-ada-002 embeddings model to generate embeddings of document chunks and then stores and queries the embedding into a vector database. This is where I come in. I worked with the OpenAI team to make Milvus and Zilliz one of the preferred vector databases to store these embeddings.

The simple API exposes the plugin's endpoints for upserting, querying, and deleting documents. You can refine their search results using metadata filters by source, date, author, etc. To keep Milvus or Zilliz updated with the latest documents, the plugin can continuously process and store documents from various data sources, using incoming webhooks to the upsert and delete endpoints.

Here's a quick example using the Zilliz Plugin to find the DRM policy within Apple's terms of Service (TOS). ChatGPT begins by searching the prompt within Zilliz Cloud and then summarizes the results. ChatGPT then accesses the Zilliz via the retrieval plugin to search for the most relevant data.

How to setup Zilliz Cloud

With Zilliz Cloud you simply need to create an account and set up your instance with your embeddings. Here is the quick start guide and the environmental variables in the retrieval plugin you need to configure:

Environmental variables

Once you have this configured, run the following test to confirm that the integration is working with this command. pytest ./tests/datastore/providers/zilliz/test_zilliz_datastore.py

How to set up Milvus

Of course, if you prefer open source Milvus, you can deploy and manage Milvus using Docker Compose, Helm, K8's Operator, or Ansible. Follow the instructions to get started.

Environmental variables

Once you have this configured, run the following test to confirm that the integration is working with this command. pytest ./tests/datastore/providers/milvus/test_milvus_datastore.py

Deploy the plugin

With the plugin tested we can move to deploying it. ChatGPT accesses plugins by sending requests to wherever your plugin server is running, whether it be Amazon ECS, Fly.io, Heroku, etc. For a detailed guide on how to configure this, take a look at the Deployment section in the GitHub readme.

The Memory Feature

One of the cool features with this new plugin is a feature that they call memory. This gives ChatGPT the ability to remember information from conversations and store it in a vector database for later use.

Acknowledgments

This was a fun project and I’d like to thank the OpenAI team of Isabella Fulford and Ziliz team of Frank Liu and Robert Guo. If you have any feedback or want to try it out for yourself, check out the GitHub repo. I personally am looking forward to contributing to one of the many great ideas listed on the ChatGPT Retrieval Plugin readme!