Fivetran
Zilliz Cloud and Fivetran simplify building AI search solutions with efficient data ingestion.
Use this integration for FreeAbout Fivetran
Fivetran is a data integration platform that automates the process of extracting, loading, and transforming data from various sources into a centralized destination. With over 500 pre-built connectors, Fivetran enables businesses to move data reliably and securely from major business applications, databases, and other sources. The platform supports built-in schema migration and offers fully managed connectors for seamless data replication.
Why Zilliz Cloud and Fivetran
Combining Zilliz Cloud and Fivetran creates a powerful solution for building AI-powered search applications. Zilliz Cloud, a fully managed version of the Milvus vector database, provides high-performance vector search capabilities essential for semantic search and retrieval-augmented generation (RAG). Fivetran's Milvus destination simplifies the process of ingesting data from various sources into Zilliz Cloud, eliminating the need for complex data pipelines.
This integration allows businesses to easily connect their diverse data sources to Zilliz Cloud, enabling efficient AI-powered search across structured and unstructured data. By streamlining data ingestion and leveraging Zilliz Cloud's advanced vector search capabilities, developers can quickly build AI applications that fully utilize their organization's data resources.
How Zilliz Cloud and Fivetran works
The integration between Zilliz Cloud and Fivetran works by using Fivetran's Milvus destination connector. This connector allows data to be easily moved from over 500 sources into Zilliz Cloud. The process begins with Fivetran extracting data from various sources such as cloud storage, business applications, and relational databases. This data is then loaded into a centralized repository like a data warehouse or data lakehouse. Next, the data can be transformed as needed, including denormalization, concatenation, and chunking. The Fivetran Milvus destination then connects to this processed data and automatically handles the vectorization of unstructured data using OpenAI embedding models. These vectors, along with any associated metadata, are efficiently stored in Zilliz Cloud. This setup enables fast and scalable AI-powered search solutions, allowing applications to perform semantic searches and use metadata filtering for more precise results.
Learn
The best way to start is with a hands-on tutorial. This tutorial will walk you through how to build a data ingestion pipeline for various data sources with Fivetran & Zilliz Cloud.
And here are a few more resources:
- Fivetran's Milvus Destination documentation
- Zilliz Cloud quickstart guide
- Webinar: Building a real-time search with Fivetran and Zilliz Cloud (September 26, 2024)
- Blog post: Unlock AI-powered search with Fivetran and Milvus