Documentation QA using LlamaIndex and Zilliz Cloud
With ChatGPT taking the headlines, many companies are wondering how they can take advantage of it for their current products. One big use case that stands out is improving the tedious and limited search functionality of product documentation. Currently, if a user wants to figure out how to use a product, they must comb through all the document pages hoping to come up with an answer. What if we could replace this tedious process with ChatGPT? What if ChatGPT could summarize all the info that is needed and answer any questions that a user might have? This is where LlamaIndex and Zilliz Cloud come in.
LlamaIndex and Zilliz Cloud are worth together to ingest and retrieve relevant info. LlamaIndex begins by taking in all the different documents you may have and embedding them using OpenAI. Once we have the embeddings we can push them into Zilliz Cloud along with any relevant text and metadata. When a user wants to ask a question, LlamaIndex will search through Zilliz Cloud for the closest answers and use ChatGPT to summarize those answers.
For this example, the documentation that we are going to be searching through is the documentation found on the Milvus website.
This example was run on a Zilliz Cloud instance using 1CU.
Let's get started.
For this example, we are going to be using
pymilvus to connect to use Zilliz Cloud and
llama-index to handle the data manipulation and pipelining. This example will also require having an OpenAI API key for embedding generation.
pip install pymilvus llama-index
Grabbing the data
We are going to use
git to pull the Milvus website data. A majority of the documents come in the form of markdown files.
git clone https://github.com/milvus-io/milvus-docs
Here we can find the main parameters that need to be modified for running with your own accounts. Beside each is a description of what it is.
from os import environ HOST = "<instance-id>.<cloud-region-id>.vectordb.zillizcloud.com" # Host in Zilliz Cloud endpoint PORT = 443 # Port in Zilliz Cloud endpoint USER = "db_admin" # Username for the cluster PASSWORD = "***" # Password that goes with the user environ["OPENAI_API_KEY"] = "sk-******" # OpenAI API Key
Consuming the knowledge
Once we have our data on the system we can proceed to consume it using LlamaIndex and upload it to Zilliz Cloud. This comes in the form of two steps. We begin by Loading in a markdown reader from Llama Hub and converting all our markdowns into documents.
from llama_index import download_loader from glob import glob # Load the markdown reader from the hub MarkdownReader = download_loader("MarkdownReader") markdownreader = MarkdownReader() # Grab all markdown files and convert them using the reader docs =  for file in glob("./milvus-docs/site/en/**/*.md", recursive=True): docs.extend(markdownreader.load_data(file=file)) print(len(docs))
Once we have our documents formed, we can proceed to push them through into Zilliz Cloud. This step requires the configs for both Zilliz Cloud and OpenAI.
from llama_index import Document, GPTVectorStoreIndex, SimpleDirectoryReader from llama_index.storage.storage_context import StorageContext from llama_index.vector_stores import MilvusVectorStore vector_store = MilvusVectorStore(overwrite=True, host=HOST, port=PORT, use_secure=True, user=USER, password=PASSWORD) storage_context = StorageContext.from_defaults(vector_store=vector_store) index = GPTVectorStoreIndex.from_documents(docs, storage_context=storage_context) query_engine = index.as_query_engine()
Asking a Question
With our documents loaded into Zilliz Cloud, we can begin asking questions. The questions will be searched against the knowledge base and any relevant documents will be used to generate an answer.
res = query_engine.query("What is a collection?") print(res) # Output: # A collection in Milvus is a logical container used to store and manage entities, similar to a table in a relational database management system (RDBMS). The collection.load() method is used to load entities into a collection.
We are also able to save our connection information and reload it using
del index, vector_store, storage_context, query_engine vector_store = MilvusVectorStore(overwrite=False, host=HOST, port=PORT, use_secure=True, user=USER, password=PASSWORD) storage_context = StorageContext.from_defaults(vector_store=vector_store) index =GPTVectorStoreIndex(nodes=, storage_context=storage_context) query_engine = index.as_query_engine() res = query_engine.query("What communication protocol is used in Pymilvus for communicating with Milvus?") print(res) # Output # The communication protocol used in Pymilvus for communicating with Milvus is gRPC.