Question Answering over Documents with Zilliz Cloud and LangChain

This guide demonstrates how to build an LLM-driven question-answering application with Zilliz Cloud and LangChain.

Before you begin

Code snippets on this page require pymilvus and langchain installed. OpenAI's embedding API has also been used to embed docs into the vector store, and therefore openai and tiktoken are also required. If they are not present on your system, run the following commands to install them.

! python -m pip install --upgrade pymilvus langchain openai tiktoken

Global parameters

In this section, you need to set up all parameters to be used in the following code snippets.

from os import environ

ZILLIZ_ENDPOINT = "https://<instance-id>.<cloud-region-id>.vectordb.zillizcloud.com:19535" # example: "in01-17f69c292d4a50a.aws-us-west-2.vectordb.zillizcloud.com"
ZILLIZ_USER = "db_admin" # database user name
ZILLIZ_PASS = "******" # password of the above database user
OPENAI_API_KEY = "sk-******" # OpenAI API key, example: "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

## Set up environment variables
environ["OPENAI_API_KEY"] = OPENAI_API_KEY

Prepare data

Before you dive in, you should finish the following steps:

Prepare the documents you want the LLM to peak at when it thinks.
Set up an embedding model to convert documents into vector embeddings.
Set up a vector store used to save the vector embeddings.

from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import Zilliz Cloud
from langchain.document_loaders import WebBaseLoader
from langchain.text_splitter import CharacterTextSplitter

# Use the WebBaseLoader to load specified web pages into documents
loader = WebBaseLoader([
    "https://milvus.io/docs/overview.md",
])

docs = loader.load()

# Split the documents into smaller chunks
text_splitter = CharacterTextSplitter(chunk_size=1024, chunk_overlap=0)
docs = text_splitter.split_documents(docs)

The output of the text splitter would be similar to the following:

Created a chunk of size 1745, which is longer than the specified 1024
Created a chunk of size 1278, which is longer than the specified 1024

Once the documents are ready, we need to convert them into vector embeddings and save them into the vector store.

# Set up an embedding model to covert documents into vector embeddings.
embeddings = OpenAIEmbeddings(model="ada")

# Set up a vector store used to save the vector embeddings. Here we use Milvus as the vector store.
vector_store = Zilliz.from_documents(
    docs,
    embedding=embeddings,
    connection_args={"uri": ZILLIZ_ENDPOINT, "user": ZILLIZ_USER, "password": ZILLIZ_PASS, "secure": True}
)

You can try text-to-text similarity searches using the following code snippet. The results returned will be the most relevant text in the document to the queries.

query = "What is milvus?"
docs = vector_store.similarity_search(query)

print(docs)

The output should be similar to the following:

[Document(page_content='Milvus workflow.', metadata={'source': 'https://milvus.io/docs/overview.md', 'title': 'Introduction Milvus documentation', 'description': 'Milvus is an open-source vector database designed specifically for AI application development, embeddings similarity search, and MLOps v2.2.x.', 'language': 'en'}), Document(page_content="Installat...rved.", metadata={'source': 'https://milvus.io/docs/overview.md', 'title': 'Introduction Milvus documentation', 'description': 'Milvus is an open-source vector database designed specifically for AI application development, embeddings similarity search, and MLOps v2.2.x.', 'language': 'en'}), Document(page_content='Introduction ... Milvus is able to analyze the correlation between two vectors by calculating their similarity distance. If the two embedding vectors are very similar, it means that the original data sources are similar as well.', metadata={'source': 'https://milvus.io/docs/overview.md', 'title': 'Introduction Milvus documentation', 'description': 'Milvus is an open-source vector database designed specifically for AI application development, embeddings similarity search, and MLOps v2.2.x.', 'language': 'en'}), Document(page_content="Key concepts\n...search algorithms are used to accelerate the searching process. If the two embedding vectors are very similar, it means that the original data sources are similar as well.\nWhy Milvus?", metadata={'source': 'https://milvus.io/docs/overview.md', 'title': 'Introduction Milvus documentation', 'description': 'Milvus is an open-source vector database designed specifically for AI application development, embeddings similarity search, and MLOps v2.2.x.', 'language': 'en'})]

Ask your question

Once the documents are ready to serve, you can set up a chain to include them in a prompt so that LLM will use the docs as a reference when preparing answers.

Note that LangChain offers four chain types for question-answering with sources, namely stuff, map_reduce, refine, and map-rerank. In simple terms, a stuff chain will include the document as a whole, which is only suitable for small documents. As most LLMs impose restrictions on the maximum number of tokens a prompt can contain, it is recommended to use the other three types of chains. These chains split the input document into smaller pieces and feed them to the LLM in different ways. For details, refer to Index-related chains in LangChain documents.

The following code snippet sets up a chain using OpenAI as the LLM and map-reduce the chain type.

from langchain.chains.qa_with_sources import load_qa_with_sources_chain
from langchain.llms import OpenAI

chain = load_qa_with_sources_chain(OpenAI(temperature=0), chain_type="map_reduce", return_intermediate_steps=True)
query = "When was milvus created?"
chain({"input_documents": docs, "question": query}, return_only_outputs=True)

The returned results include both the intermediate_steps and output_text. The former indicates what documents it refers to during the search, and the latter is the final answer to the question.

{'intermediate_steps': [' No relevant text.',
  ' Milvus was created in 2019 with a singular goal: store, index, and manage massive embedding vectors generated by deep neural networks and other machine learning (ML) models.',
  ' Milvus. 2023 All rights reserved.',
  ' None'],
 'output_text': ' Milvus was created in 2019.\nSOURCES: https://milvus.io/docs/overview.md'}

On this page