Hugging Face / all-MiniLM-L12-v2

Introduction to the all-MiniLM-L12-v2 embedding model

• all-MiniLM-L12-v2 is a sentence and short paragraph encoder that outputs a vector capturing the semantic information of the input text.
• The model is based on the pre-trained Microsoft/MiniLM-L12-H384-uncased model and was fine-tuned using a contrastive learning objective on a large dataset of over 1 billion sentence pairs.
• It maps sentences and paragraphs to a 384-dimensional dense vector space, which can be used for tasks like information retrieval, clustering, and semantic search.

How to create vector embeddings with the all-MiniLM-L12-v2 model

There are two primary ways to create vector embeddings:

1. PyMilvus: the Python SDK for Milvus that seamlessly integrates the sentence-transformer model.
2. SentenceTransformer library: the Python library of sentence-transformer.

Once the vector embeddings are generated, they can be stored in Zilliz Cloud (a fully managed vector database service powered by Milvus) and used for semantic similarity search. Here are four key steps:

1. Sign up for a Zilliz Cloud account for free.
2. Set up a serverless cluster and obtain the Public Endpoint and API Key.
3. Create a vector collection and insert your vector embeddings.
4. Run a semantic search on the stored embeddings.

Generate vector embeddings via PyMilvus and insert them into Zilliz Cloud for semantic search

from pymilvus.model.dense import SentenceTransformerEmbeddingFunction

ef = SentenceTransformerEmbeddingFunction("sentence-transformers/all-MiniLM-L12-v2")

docs = [
   "Artificial intelligence was founded as an academic discipline in 1956.",
   "Alan Turing was the first person to conduct substantial research in AI.",
   "Born in Maida Vale, London, Turing was raised in southern England."
]
# Generate embeddings for documents
docs_embeddings = ef(docs)

queries = ["When was artificial intelligence founded",
          "Where was Alan Turing born?"]
# Generate embeddings for queries
query_embeddings = ef(queries)

# Connect to Zilliz Cloud with Public Endpoint and API Key
client = MilvusClient(
    uri=ZILLIZ_PUBLIC_ENDPOINT,
    token=ZILLIZ_API_KEY)

COLLECTION = "documents"
if client.has_collection(collection_name=COLLECTION):
    client.drop_collection(collection_name=COLLECTION)
client.create_collection(
    collection_name=COLLECTION,
    dimension=ef.dim,
    auto_id=True)

for doc, embedding in zip(docs, docs_embeddings):
    client.insert(COLLECTION, {"text": doc, "vector": embedding})
    
results = client.search(
    collection_name=COLLECTION,
    data=query_embeddings,
    output_fields=["text"])

For more information, refer to our PyMilvus Embedding Model documentation.

Generate vector embeddings via SentenceTransformer and insert them into Zilliz Cloud for semantic search

from sentence_transformers import SentenceTransformer
from pymilvus import MilvusClient

model = SentenceTransformer("sentence-transformers/all-MiniLM-L12-v2")

docs = [
   "Artificial intelligence was founded as an academic discipline in 1956.",
   "Alan Turing was the first person to conduct substantial research in AI.",
   "Born in Maida Vale, London, Turing was raised in southern England."
]
# Generate embeddings for documents
docs_embeddings = model.encode(docs, normalize_embeddings=True)

queries = ["query: When was artificial intelligence founded",
           "query: Wo wurde Alan Turing geboren?" ] 
# Generate embeddings for queries
query_embeddings = model.encode(queries, normalize_embeddings=True)

# Connect to Zilliz Cloud with Public Endpoint and API Key
client = MilvusClient(
    uri=ZILLIZ_PUBLIC_ENDPOINT,
    token=ZILLIZ_API_KEY)

COLLECTION = "documents"
if client.has_collection(collection_name=COLLECTION):
    client.drop_collection(collection_name=COLLECTION)
client.create_collection(
    collection_name=COLLECTION,
    dimension=384,
    auto_id=True)

for doc, embedding in zip(docs, docs_embeddings):
    client.insert(COLLECTION, {"text": doc, "vector": embedding})
    
results = client.search(
    collection_name=COLLECTION,
    data=query_embeddings,
    consistency_level="Strong", 
    output_fields=["text"])

For more information, refer to SentenceTransformer documentation.