all-mpnet-base-v2 is typically slower than MiniLM embedding models because it is a larger encoder with more parameters and heavier per-token computation. In most practical deployments, you should expect MiniLM to produce embeddings with lower latency and higher throughput on the same hardware, especially on CPU. The exact speed ratio varies with batch size, sequence length, and runtime (PyTorch vs ONNX), but the direction is consistent: mpnet-base models trade speed for stronger representation quality, while MiniLM models are designed to be lightweight and fast.
From a systems standpoint, the speed gap shows up in two places: offline corpus embedding and online query embedding. Offline, if you embed millions of chunks, mpnet-base can add substantial wall-clock time unless you batch efficiently or use GPUs. Online, if you embed every user query, higher per-request latency can become your p95 bottleneck unless you optimize. The best way to make mpnet-base “fast enough” is to control inputs: keep texts short (it’s intended for sentences/short passages), batch requests, and consider exporting to an optimized runtime. Also remember that embedding generation is only part of the request path; vector search can be very fast, so the model can dominate end-to-end latency if you don’t optimize it.
If you want to make this decision with data, measure it in your actual pipeline: same text lengths, same batching, same host class, and include tokenization time. Then compare “cost per embedded token” and “p95 latency per query.” Many teams end up using MiniLM for high-QPS endpoints where cost/latency dominate, and mpnet-base when they need better retrieval quality on a smaller candidate set or for premium queries. Regardless of which model you pick, storing embeddings in a vector database such as Milvus or Zilliz Cloud helps you keep the retrieval portion fast and lets you A/B test models by swapping which collection you query without changing application logic.
For more information, click here: https://zilliz.com/ai-models/all-mpnet-base-v2