Integrations
Snowflake and Zilliz Cloud Integration

Snowflake and Zilliz Cloud Integration

Snowflake and Zilliz Cloud integrate through Snowpark Container Services to deploy Milvus within the Snowflake ecosystem, enabling organizations to run vector search workloads directly inside Snowflake without data egress, combining Snowflake's data platform with Milvus's high-performance vector database capabilities.

Use this integration for Free

What is Snowflake
Snowflake is a cloud data platform that provides data warehousing, data lake, and data sharing capabilities. Snowpark Container Services is a fully managed container offering within the Snowflake ecosystem, designed to facilitate the deployment, management, and scaling of containerized applications. This service enables users to run containerized workloads directly within Snowflake, ensuring that data doesn't need to be moved out of the Snowflake environment for processing.

By integrating with Zilliz Cloud (fully managed Milvus), Snowflake users can deploy Milvus as a containerized service within Snowpark Container Services, enabling vector search and AI workloads to run directly inside the Snowflake ecosystem — processing data where it resides to optimize performance, minimize latency, and eliminate the need for data egress.
Benefits of the Snowflake + Zilliz Cloud Integration
- No data egress: Running Milvus within Snowpark Container Services means data stays inside the Snowflake ecosystem, eliminating the need to move data to external vector databases and reducing security and compliance risks.
- Fully managed container deployment: Snowpark Container Services handles the deployment, management, and scaling of the Milvus container, reducing operational overhead for running vector search workloads.
- Data proximity: Processing vector search workloads where the data resides optimizes performance and minimizes latency, enabling faster RAG and semantic search applications.
- Seamless Snowflake integration: Organizations already using Snowflake can add vector search capabilities without introducing new external infrastructure, leveraging existing roles, warehouses, and security configurations.
How the Integration Works
Snowflake provides the cloud data platform and Snowpark Container Services infrastructure for hosting containerized applications. It handles compute pool management, service orchestration, networking, and authentication, enabling Milvus to run as a managed service within the Snowflake ecosystem.

Zilliz Cloud provides Milvus, the high-performance vector database that runs as a containerized service within Snowpark. It handles vector storage, indexing, and similarity search for AI workloads including RAG, semantic search, and recommendation systems.

Together, Snowflake and Milvus create a unified data and AI platform: data stored in Snowflake is processed and embedded into vectors, which are stored and indexed by Milvus running within Snowpark Container Services. Users can then perform vector similarity search directly inside the Snowflake ecosystem, enabling AI applications like RAG without moving data outside the platform.

Step-by-Step Guide

1. Obtain Account Information and Log In

Download the SPCS client SnowSQL, then log in to your account:

snowsql -a ${instance_name} -u ${user_name}

The rule of ${instance_name} is ${org_name}-${acct_name}. The relevant information can be obtained by logging in to app.snowflake.com and checking the personal account information.

2. Configure Role and Privileges

Configure OAUTH integration and create a role for the service:

USE ROLE ACCOUNTADMIN;
CREATE SECURITY INTEGRATION SNOWSERVICES_INGRESS_OAUTH
  TYPE=oauth
  OAUTH_CLIENT=snowservices_ingress
  ENABLED=true;

USE ROLE ACCOUNTADMIN;
GRANT BIND SERVICE ENDPOINT ON ACCOUNT TO ROLE SYSADMIN;

USE ROLE SECURITYADMIN;
CREATE ROLE MILVUS_ROLE;

USE ROLE USERADMIN;
CREATE USER milvus_user
  PASSWORD='milvususerok'
  DEFAULT_ROLE = MILVUS_ROLE
  DEFAULT_SECONDARY_ROLES = ('ALL')
  MUST_CHANGE_PASSWORD = FALSE;

USE ROLE SECURITYADMIN;
GRANT ROLE MILVUS_ROLE TO USER milvus_user;

3. Create Data Storage Configuration

Create warehouse, database, image repository, and stages:

USE ROLE SYSADMIN;
CREATE OR REPLACE WAREHOUSE MILVUS_WAREHOUSE WITH
WAREHOUSE_SIZE='X-SMALL'
AUTO_SUSPEND = 180
AUTO_RESUME = true
INITIALLY_SUSPENDED=false;

USE ROLE SYSADMIN;
CREATE DATABASE IF NOT EXISTS MILVUS_DEMO;
USE DATABASE MILVUS_DEMO;
CREATE IMAGE REPOSITORY MILVUS_DEMO.PUBLIC.MILVUS_REPO;
CREATE OR REPLACE STAGE YAML_STAGE;
CREATE OR REPLACE STAGE DATA ENCRYPTION = (TYPE = 'SNOWFLAKE_SSE');
CREATE OR REPLACE STAGE FILES ENCRYPTION = (TYPE = 'SNOWFLAKE_SSE');

Grant role privileges and configure ACL:

USE ROLE SECURITYADMIN;
GRANT ALL PRIVILEGES ON DATABASE MILVUS_DEMO TO MILVUS_ROLE;
GRANT ALL PRIVILEGES ON SCHEMA MILVUS_DEMO.PUBLIC TO MILVUS_ROLE;
GRANT ALL PRIVILEGES ON WAREHOUSE MILVUS_WAREHOUSE TO MILVUS_ROLE;
GRANT ALL PRIVILEGES ON STAGE MILVUS_DEMO.PUBLIC.FILES TO MILVUS_ROLE;

USE ROLE ACCOUNTADMIN;
USE DATABASE MILVUS_DEMO;
USE SCHEMA PUBLIC;
CREATE NETWORK RULE allow_all_rule
TYPE = 'HOST_PORT'
MODE= 'EGRESS'
VALUE_LIST = ('0.0.0.0:443','0.0.0.0:80');

CREATE EXTERNAL ACCESS INTEGRATION allow_all_eai
ALLOWED_NETWORK_RULES=(allow_all_rule)
ENABLED=TRUE;

GRANT USAGE ON INTEGRATION allow_all_eai TO ROLE SYSADMIN;

4. Build and Push Docker Images

Build the Milvus and Jupyter images locally using the configuration from this repo:

cd ${repo_git_root_path}
docker build --rm --no-cache --platform linux/amd64 -t milvus ./images/milvus
docker build --rm --no-cache --platform linux/amd64 -t jupyter ./images/jupyter

Tag and push images to SPCS:

docker login ${instance_name}.registry.snowflakecomputing.com -u ${user_name}

docker tag milvus ${instance_name}.registry.snowflakecomputing.com/milvus_demo/public/milvus_repo/milvus
docker tag jupyter ${instance_name}.registry.snowflakecomputing.com/milvus_demo/public/milvus_repo/jupyter

docker push ${instance_name}.registry.snowflakecomputing.com/milvus_demo/public/milvus_repo/milvus
docker push ${instance_name}.registry.snowflakecomputing.com/milvus_demo/public/milvus_repo/jupyter

5. Create Compute Pools and Start Services

Create compute pools and upload spec files:

USE ROLE SYSADMIN;
CREATE COMPUTE POOL IF NOT EXISTS MILVUS_COMPUTE_POOL
  MIN_NODES = 1
  MAX_NODES = 1
  INSTANCE_FAMILY = CPU_X64_S
  AUTO_RESUME = true;
CREATE COMPUTE POOL IF NOT EXISTS JUPYTER_COMPUTE_POOL
  MIN_NODES = 1
  MAX_NODES = 1
  INSTANCE_FAMILY = CPU_X64_S
  AUTO_RESUME = true;

Upload spec files and create services:

PUT file://${path/to/jupyter.yaml} @yaml_stage overwrite=true auto_compress=false;
PUT file://${path/to/milvus.yaml} @yaml_stage overwrite=true auto_compress=false;

USE ROLE SYSADMIN;
USE DATABASE MILVUS_DEMO;
USE SCHEMA PUBLIC;

CREATE SERVICE MILVUS
  IN COMPUTE POOL MILVUS_COMPUTE_POOL
  FROM @YAML_STAGE
  SPEC='milvus.yaml'
  MIN_INSTANCES=1
  MAX_INSTANCES=1;

CREATE SERVICE JUPYTER
  IN COMPUTE POOL JUPYTER_COMPUTE_POOL
  FROM @YAML_STAGE
  SPEC='jupyter.yaml'
  MIN_INSTANCES=1
  MAX_INSTANCES=1;

6. Use the Notebook

Grant permissions and access the Jupyter notebook:

USE ROLE SECURITYADMIN;
GRANT USAGE ON SERVICE MILVUS_DEMO.PUBLIC.JUPYTER TO ROLE MILVUS_ROLE;

USE ROLE SYSADMIN;
SHOW ENDPOINTS IN SERVICE MILVUS_DEMO.PUBLIC.JUPYTER;

Record the ingress_url, open it in a browser, and log in with the milvus_user account. Open TestMilvus.ipynb to try out Milvus with vector search operations.

7. Clean Up

After verification, clean up the services, roles, and data resources:

USE ROLE ACCOUNTADMIN;
DROP USER milvus_user;

USE ROLE SYSADMIN;
DROP SERVICE MILVUS;
DROP SERVICE JUPYTER;
DROP COMPUTE POOL MILVUS_COMPUTE_POOL;
DROP COMPUTE POOL JUPYTER_COMPUTE_POOL;
DROP IMAGE REPOSITORY MILVUS_DEMO.PUBLIC.MILVUS_REPO;
DROP DATABASE MILVUS_DEMO;
DROP WAREHOUSE MILVUS_WAREHOUSE;

USE ROLE ACCOUNTADMIN;
DROP ROLE MILVUS_ROLE;
DROP SECURITY INTEGRATION SNOWSERVICES_INGRESS_OAUTH;

Learn More
- Milvus on Snowpark Container Services — Official Milvus tutorial for deploying on Snowpark
- Building RAG with Self-Deployed Milvus and Snowpark Container Services — Zilliz blog on building RAG with Snowpark
- Snowpark Container Services Overview — Official Snowflake documentation on SPCS
- Milvus on SPCS GitHub Repository — Docker configuration and spec files for the deployment
- Milvus Quickstart — Milvus quickstart documentation

Snowflake and Zilliz Cloud Integration

What is Snowflake

Benefits of the Snowflake + Zilliz Cloud Integration

How the Integration Works

Step-by-Step Guide

Learn More

Related Resources

Building RAG with Self-Deployed Milvus and Snowpark Container Services

Milvus Quickstart: Install Milvus Vector Database in 5 Minutes

Up to 50x Cost Cut for Vector Search using Zilliz Cloud Serverless

AI Assistant