Documentation QA using LlamaIndex and Zilliz Cloud

With ChatGPT taking the headlines, many companies are wondering how they can take advantage of it for their current products. One big use case that stands out is improving the tedious and limited search functionality of product documentation. Currently, if a user wants to figure out how to use a product, they must comb through all the document pages hoping to come up with an answer. What if we could replace this tedious process with ChatGPT? What if ChatGPT could summarize all the info that is needed and answer any questions that a user might have? This is where LlamaIndex and Zilliz Cloud come in.

LlamaIndex and Zilliz Cloud are worth together to ingest and retrieve relevant info. LlamaIndex begins by taking in all the different documents you may have and embedding them using OpenAI. Once we have the embeddings we can push them into Zilliz Cloud along with any relevant text and metadata. When a user wants to ask a question, LlamaIndex will search through Zilliz Cloud for the closest answers and use ChatGPT to summarize those answers.

For this example, the documentation that we are going to be searching through is the documentation found on the Milvus website.

This example was run on a Zilliz Cloud instance using 1CU.

Let's get started.

Installing requirements

For this example, we are going to be using pymilvus to connect to use Zilliz Cloud and llama-index to handle the data manipulation and pipelining. This example will also require having an OpenAI API key for embedding generation.

pip install pymilvus llama-index

Grabbing the data

We are going to use git to pull the Milvus website data. A majority of the documents come in the form of markdown files.

git clone https://github.com/milvus-io/milvus-docs

Global parameters

Here we can find the main parameters that need to be modified for running with your own accounts. Beside each is a description of what it is.

from os import environ

HOST = "<instance-id>.<cloud-region-id>.vectordb.zillizcloud.com" # Host in Zilliz Cloud endpoint
PORT = 443  # Port in Zilliz Cloud endpoint

USER = "db_admin" # Username for the cluster
PASSWORD = "***"  # Password that goes with the user

environ["OPENAI_API_KEY"] = "sk-******" # OpenAI API Key

Consuming the knowledge

Once we have our data on the system we can proceed to consume it using LlamaIndex and upload it to Zilliz Cloud. This comes in the form of two steps. We begin by Loading in a markdown reader from Llama Hub and converting all our markdowns into documents.

from llama_index import download_loader
from glob import glob

# Load the markdown reader from the hub
MarkdownReader = download_loader("MarkdownReader")
markdownreader = MarkdownReader()

# Grab all markdown files and convert them using the reader
docs = []
for file in glob("./milvus-docs/site/en/**/*.md", recursive=True):
    docs.extend(markdownreader.load_data(file=file))
print(len(docs))

Once we have our documents formed, we can proceed to push them through into Zilliz Cloud. This step requires the configs for both Zilliz Cloud and OpenAI.

from llama_index import Document, GPTVectorStoreIndex, SimpleDirectoryReader
from llama_index.storage.storage_context import StorageContext
from llama_index.vector_stores import MilvusVectorStore

vector_store = MilvusVectorStore(overwrite=True, host=HOST, port=PORT, use_secure=True, user=USER, password=PASSWORD)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = GPTVectorStoreIndex.from_documents(docs, storage_context=storage_context)
query_engine = index.as_query_engine()

Asking a Question

With our documents loaded into Zilliz Cloud, we can begin asking questions. The questions will be searched against the knowledge base and any relevant documents will be used to generate an answer.

res = query_engine.query("What is a collection?")
print(res)

# Output:
# A collection in Milvus is a logical container used to store and manage entities, similar to a table in a relational database management system (RDBMS). The collection.load() method is used to load entities into a collection.

We are also able to save our connection information and reload it using save_to_dict() and load_from_dict().

del index, vector_store, storage_context, query_engine

vector_store = MilvusVectorStore(overwrite=False, host=HOST, port=PORT, use_secure=True, user=USER, password=PASSWORD)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index =GPTVectorStoreIndex(nodes=[], storage_context=storage_context)
query_engine = index.as_query_engine()

res = query_engine.query("What communication protocol is used in Pymilvus for communicating with Milvus?")
print(res)

# Output
# The communication protocol used in Pymilvus for communicating with Milvus is gRPC.

On this page